Nov 29 03:01:46 user nova-compute[75234]: DEBUG os_vif [-] Loaded VIF plugin class '<class 'vif_plug_linux_bridge.linux_bridge.LinuxBridgePlugin'>' with name 'linux_bridge' {{(pid=75234) initialize /opt/stack/data/venv/lib/python3.10/site-packages/os_vif/__init__.py:44}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG os_vif [-] Loaded VIF plugin class '<class 'vif_plug_noop.noop.NoOpPlugin'>' with name 'noop' {{(pid=75234) initialize /opt/stack/data/venv/lib/python3.10/site-packages/os_vif/__init__.py:44}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG os_vif [-] Loaded VIF plugin class '<class 'vif_plug_ovs.ovs.OvsPlugin'>' with name 'ovs' {{(pid=75234) initialize /opt/stack/data/venv/lib/python3.10/site-packages/os_vif/__init__.py:44}}
Nov 29 03:01:46 user nova-compute[75234]: INFO os_vif [-] Loaded VIF plugins: linux_bridge, noop, ovs
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): grep -F node.session.scan /sbin/iscsiadm {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "grep -F node.session.scan /sbin/iscsiadm" returned: 0 in 0.019s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:01:46 user nova-compute[75234]: INFO nova.virt.driver [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] Loading compute driver 'libvirt.LibvirtDriver'
Nov 29 03:01:46 user nova-compute[75234]: INFO nova.compute.provider_config [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] No provider configs found in /etc/nova/provider_config/. If files are present, ensure the Nova process has access.
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] Acquiring lock "singleton_lock" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] Acquired lock "singleton_lock" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] Releasing lock "singleton_lock" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] Full set of CONF: {{(pid=75234) _wait_for_exit_or_signal /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/service.py:362}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] ******************************************************************************** {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2589}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] Configuration options gathered from: {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2590}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] command line args: ['--config-file', '/etc/nova/nova-cpu.conf'] {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2591}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] config files: ['/etc/nova/nova-cpu.conf'] {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2592}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] ================================================================================ {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2594}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] allow_resize_to_same_host      = True {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] arq_binding_timeout            = 300 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] backdoor_port                  = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] backdoor_socket                = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] block_device_allocate_retries  = 300 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] block_device_allocate_retries_interval = 5 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] cert                           = self.pem {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] compute_driver                 = libvirt.LibvirtDriver {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] compute_monitors               = [] {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] config_dir                     = [] {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] config_drive_format            = iso9660 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] config_file                    = ['/etc/nova/nova-cpu.conf'] {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] config_source                  = [] {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] console_host                   = user {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] control_exchange               = nova {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] cpu_allocation_ratio           = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] daemon                         = False {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] debug                          = True {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] default_access_ip_network_name = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] default_availability_zone      = nova {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] default_ephemeral_format       = ext4 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] default_green_pool_size        = 1000 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] default_log_levels             = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'glanceclient=WARN', 'oslo.privsep.daemon=INFO'] {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] default_schedule_zone          = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] disk_allocation_ratio          = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] enable_new_services            = True {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] enabled_apis                   = ['osapi_compute'] {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] enabled_ssl_apis               = [] {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] flat_injected                  = False {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] force_config_drive             = False {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] force_raw_images               = True {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] graceful_shutdown_timeout      = 5 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] heal_instance_info_cache_interval = 60 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] host                           = user {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] initial_cpu_allocation_ratio   = 4.0 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] initial_disk_allocation_ratio  = 1.0 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] initial_ram_allocation_ratio   = 1.0 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] injected_network_template      = /opt/stack/nova/nova/virt/interfaces.template {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] instance_build_timeout         = 0 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] instance_delete_interval       = 300 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] instance_format                = [instance: %(uuid)s]  {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] instance_name_template         = instance-%08x {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] instance_usage_audit           = False {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] instance_usage_audit_period    = month {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] instance_uuid_format           = [instance: %(uuid)s]  {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] instances_path                 = /opt/stack/data/nova/instances {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] internal_service_availability_zone = internal {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] key                            = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] live_migration_retry_count     = 30 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] log_config_append              = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] log_date_format                = %Y-%m-%d %H:%M:%S {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] log_dir                        = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] log_file                       = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] log_options                    = True {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] log_rotate_interval            = 1 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] log_rotate_interval_type       = days {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] log_rotation_type              = none {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] logging_context_format_string  = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] logging_debug_format_suffix    = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] logging_default_format_string  = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] logging_exception_prefix       = ERROR %(name)s %(instance)s {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] logging_user_identity_format   = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] long_rpc_timeout               = 1800 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] max_concurrent_builds          = 10 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] max_concurrent_live_migrations = 1 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] max_concurrent_snapshots       = 5 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] max_local_block_devices        = 3 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] max_logfile_count              = 30 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] max_logfile_size_mb            = 200 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] maximum_instance_delete_attempts = 5 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] metadata_listen                = 0.0.0.0 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] metadata_listen_port           = 8775 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] metadata_workers               = 3 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] migrate_max_retries            = -1 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] mkisofs_cmd                    = genisoimage {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] my_block_storage_ip            = 10.0.0.210 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] my_ip                          = 10.0.0.210 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] network_allocate_retries       = 0 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] non_inheritable_image_properties = ['cache_in_nova', 'bittorrent'] {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] osapi_compute_listen           = 0.0.0.0 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] osapi_compute_listen_port      = 8774 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] osapi_compute_unique_server_name_scope =  {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] osapi_compute_workers          = 3 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] password_length                = 12 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] periodic_enable                = True {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] periodic_fuzzy_delay           = 60 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] pointer_model                  = ps2mouse {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] preallocate_images             = none {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] publish_errors                 = False {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] pybasedir                      = /opt/stack/nova {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] ram_allocation_ratio           = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] rate_limit_burst               = 0 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] rate_limit_except_level        = CRITICAL {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] rate_limit_interval            = 0 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] reboot_timeout                 = 0 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] reclaim_instance_interval      = 0 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] record                         = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] reimage_timeout_per_gb         = 60 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] report_interval                = 120 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] rescue_timeout                 = 0 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] reserved_host_cpus             = 0 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] reserved_host_disk_mb          = 0 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] reserved_host_memory_mb        = 512 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] reserved_huge_pages            = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] resize_confirm_window          = 0 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] resize_fs_using_block_device   = False {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] resume_guests_state_on_host_boot = False {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] rootwrap_config                = /etc/nova/rootwrap.conf {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] rpc_response_timeout           = 60 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] run_external_periodic_tasks    = True {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] running_deleted_instance_action = reap {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] running_deleted_instance_poll_interval = 1800 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] running_deleted_instance_timeout = 0 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] scheduler_instance_sync_interval = 120 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] service_down_time              = 720 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] servicegroup_driver            = db {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] shelved_offload_time           = 0 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] shelved_poll_interval          = 3600 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] shutdown_timeout               = 0 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] source_is_ipv6                 = False {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] ssl_only                       = False {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] state_path                     = /opt/stack/data/nova {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] sync_power_state_interval      = 600 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] sync_power_state_pool_size     = 1000 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] syslog_log_facility            = LOG_USER {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] tempdir                        = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] timeout_nbd                    = 10 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] transport_url                  = **** {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] update_resources_interval      = 0 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] use_cow_images                 = True {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] use_eventlog                   = False {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] use_journal                    = False {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] use_json                       = False {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] use_rootwrap_daemon            = False {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] use_stderr                     = False {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] use_syslog                     = False {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] vcpu_pin_set                   = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] vif_plugging_is_fatal          = False {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] vif_plugging_timeout           = 0 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] virt_mkfs                      = [] {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] volume_usage_poll_interval     = 0 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] watch_log_file                 = False {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] web                            = /usr/share/spice-html5 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] oslo_concurrency.disable_process_locking = False {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] oslo_concurrency.lock_path     = /opt/stack/data/nova {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] oslo_messaging_metrics.metrics_enabled = False {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] oslo_messaging_metrics.metrics_process_name =  {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] api.auth_strategy              = keystone {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] api.compute_link_prefix        = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] api.config_drive_skip_versions = 1.0 2007-01-19 2007-03-01 2007-08-29 2007-10-10 2007-12-15 2008-02-01 2008-09-01 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] api.dhcp_domain                = novalocal {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] api.enable_instance_password   = True {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] api.glance_link_prefix         = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] api.instance_list_cells_batch_fixed_size = 100 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] api.instance_list_cells_batch_strategy = distributed {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] api.instance_list_per_project_cells = False {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] api.list_records_by_skipping_down_cells = True {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] api.local_metadata_per_cell    = False {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] api.max_limit                  = 1000 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] api.metadata_cache_expiration  = 15 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] api.neutron_default_tenant_id  = default {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] api.use_forwarded_for          = False {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] api.use_neutron_default_nets   = False {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] api.vendordata_dynamic_connect_timeout = 5 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] api.vendordata_dynamic_failure_fatal = False {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] api.vendordata_dynamic_read_timeout = 5 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] api.vendordata_dynamic_ssl_certfile =  {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] api.vendordata_dynamic_targets = [] {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] api.vendordata_jsonfile_path   = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] api.vendordata_providers       = ['StaticJSON'] {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] cache.backend                  = dogpile.cache.memcached {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] cache.backend_argument         = **** {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] cache.config_prefix            = cache.oslo {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] cache.dead_timeout             = 60.0 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] cache.debug_cache_backend      = False {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] cache.enable_retry_client      = False {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] cache.enable_socket_keepalive  = False {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] cache.enabled                  = True {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] cache.expiration_time          = 600 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] cache.hashclient_retry_attempts = 2 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] cache.hashclient_retry_delay   = 1.0 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] cache.memcache_dead_retry      = 300 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] cache.memcache_password        =  {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] cache.memcache_pool_connection_get_timeout = 10 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] cache.memcache_pool_flush_on_reconnect = False {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] cache.memcache_pool_maxsize    = 10 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] cache.memcache_pool_unused_timeout = 60 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] cache.memcache_sasl_enabled    = False {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] cache.memcache_servers         = ['localhost:11211'] {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] cache.memcache_socket_timeout  = 1.0 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] cache.memcache_username        =  {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] cache.proxies                  = [] {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] cache.retry_attempts           = 2 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] cache.retry_delay              = 0.0 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] cache.socket_keepalive_count   = 1 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] cache.socket_keepalive_idle    = 1 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] cache.socket_keepalive_interval = 1 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] cache.tls_allowed_ciphers      = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] cache.tls_cafile               = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] cache.tls_certfile             = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] cache.tls_enabled              = False {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] cache.tls_keyfile              = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] cinder.auth_section            = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] cinder.auth_type               = password {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] cinder.cafile                  = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] cinder.catalog_info            = volumev3::publicURL {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] cinder.certfile                = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] cinder.collect_timing          = False {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] cinder.cross_az_attach         = True {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] cinder.debug                   = False {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] cinder.endpoint_template       = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] cinder.http_retries            = 3 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] cinder.insecure                = False {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] cinder.keyfile                 = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] cinder.os_region_name          = RegionOne {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] cinder.split_loggers           = False {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] cinder.timeout                 = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] compute.consecutive_build_service_disable_threshold = 10 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] compute.cpu_dedicated_set      = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] compute.cpu_shared_set         = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] compute.image_type_exclude_list = [] {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] compute.live_migration_wait_for_vif_plug = True {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] compute.max_concurrent_disk_ops = 0 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] compute.max_disk_devices_to_attach = -1 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] compute.packing_host_numa_cells_allocation_strategy = False {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] compute.provider_config_location = /etc/nova/provider_config/ {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] compute.resource_provider_association_refresh = 300 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] compute.shutdown_retry_interval = 10 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] compute.vmdk_allowed_types     = ['streamOptimized', 'monolithicSparse'] {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] conductor.workers              = 3 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] console.allowed_origins        = [] {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] console.ssl_ciphers            = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] console.ssl_minimum_version    = default {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] consoleauth.token_ttl          = 600 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] cyborg.cafile                  = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] cyborg.certfile                = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] cyborg.collect_timing          = False {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] cyborg.connect_retries         = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] cyborg.connect_retry_delay     = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] cyborg.endpoint_override       = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] cyborg.insecure                = False {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] cyborg.keyfile                 = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] cyborg.max_version             = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] cyborg.min_version             = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] cyborg.region_name             = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] cyborg.service_name            = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] cyborg.service_type            = accelerator {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] cyborg.split_loggers           = False {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] cyborg.status_code_retries     = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] cyborg.status_code_retry_delay = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] cyborg.timeout                 = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] cyborg.valid_interfaces        = ['internal', 'public'] {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] cyborg.version                 = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] database.backend               = sqlalchemy {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] database.connection            = **** {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] database.connection_debug      = 0 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] database.connection_parameters =  {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] database.connection_recycle_time = 3600 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] database.connection_trace      = False {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] database.db_inc_retry_interval = True {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] database.db_max_retries        = 20 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] database.db_max_retry_interval = 10 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] database.db_retry_interval     = 1 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] database.max_overflow          = 50 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] database.max_pool_size         = 5 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] database.max_retries           = 10 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] database.mysql_sql_mode        = TRADITIONAL {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] database.mysql_wsrep_sync_wait = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] database.pool_timeout          = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] database.retry_interval        = 10 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] database.slave_connection      = **** {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] database.sqlite_synchronous    = True {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] api_database.backend           = sqlalchemy {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] api_database.connection        = **** {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] api_database.connection_debug  = 0 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] api_database.connection_parameters =  {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] api_database.connection_recycle_time = 3600 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] api_database.connection_trace  = False {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] api_database.db_inc_retry_interval = True {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] api_database.db_max_retries    = 20 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] api_database.db_max_retry_interval = 10 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] api_database.db_retry_interval = 1 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] api_database.max_overflow      = 50 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] api_database.max_pool_size     = 5 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] api_database.max_retries       = 10 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] api_database.mysql_sql_mode    = TRADITIONAL {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] api_database.mysql_wsrep_sync_wait = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] api_database.pool_timeout      = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] api_database.retry_interval    = 10 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] api_database.slave_connection  = **** {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] api_database.sqlite_synchronous = True {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] devices.enabled_mdev_types     = [] {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] ephemeral_storage_encryption.cipher = aes-xts-plain64 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] ephemeral_storage_encryption.enabled = False {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] ephemeral_storage_encryption.key_size = 512 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] glance.api_servers             = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] glance.cafile                  = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] glance.certfile                = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] glance.collect_timing          = False {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] glance.connect_retries         = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] glance.connect_retry_delay     = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] glance.debug                   = False {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] glance.default_trusted_certificate_ids = [] {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] glance.enable_certificate_validation = False {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] glance.enable_rbd_download     = False {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] glance.endpoint_override       = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] glance.insecure                = False {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] glance.keyfile                 = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] glance.max_version             = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] glance.min_version             = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] glance.num_retries             = 3 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] glance.rbd_ceph_conf           =  {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] glance.rbd_connect_timeout     = 5 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] glance.rbd_pool                =  {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] glance.rbd_user                =  {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] glance.region_name             = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] glance.service_name            = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] glance.service_type            = image {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] glance.split_loggers           = False {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] glance.status_code_retries     = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] glance.status_code_retry_delay = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] glance.timeout                 = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] glance.valid_interfaces        = ['internal', 'public'] {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] glance.verify_glance_signatures = False {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] glance.version                 = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] guestfs.debug                  = False {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] hyperv.config_drive_cdrom      = False {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] hyperv.config_drive_inject_password = False {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] hyperv.dynamic_memory_ratio    = 1.0 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] hyperv.enable_instance_metrics_collection = False {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] hyperv.enable_remotefx         = False {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] hyperv.instances_path_share    =  {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] hyperv.iscsi_initiator_list    = [] {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] hyperv.limit_cpu_features      = False {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] hyperv.mounted_disk_query_retry_count = 10 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] hyperv.mounted_disk_query_retry_interval = 5 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] hyperv.power_state_check_timeframe = 60 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] hyperv.power_state_event_polling_interval = 2 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] hyperv.qemu_img_cmd            = qemu-img.exe {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] hyperv.use_multipath_io        = False {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] hyperv.volume_attach_retry_count = 10 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] hyperv.volume_attach_retry_interval = 5 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] hyperv.vswitch_name            = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] hyperv.wait_soft_reboot_seconds = 60 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] mks.enabled                    = False {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] mks.mksproxy_base_url          = http://127.0.0.1:6090/ {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] image_cache.manager_interval   = 2400 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] image_cache.precache_concurrency = 1 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] image_cache.remove_unused_base_images = True {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] image_cache.remove_unused_original_minimum_age_seconds = 86400 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] image_cache.remove_unused_resized_minimum_age_seconds = 3600 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] image_cache.subdirectory_name  = _base {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] ironic.api_max_retries         = 60 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] ironic.api_retry_interval      = 2 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] ironic.auth_section            = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] ironic.auth_type               = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] ironic.cafile                  = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] ironic.certfile                = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] ironic.collect_timing          = False {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] ironic.conductor_group         = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] ironic.connect_retries         = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] ironic.connect_retry_delay     = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] ironic.endpoint_override       = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] ironic.insecure                = False {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] ironic.keyfile                 = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] ironic.max_version             = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] ironic.min_version             = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] ironic.peer_list               = [] {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] ironic.region_name             = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] ironic.serial_console_state_timeout = 10 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] ironic.service_name            = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] ironic.service_type            = baremetal {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] ironic.split_loggers           = False {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] ironic.status_code_retries     = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] ironic.status_code_retry_delay = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] ironic.timeout                 = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] ironic.valid_interfaces        = ['internal', 'public'] {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] ironic.version                 = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] key_manager.backend            = nova.keymgr.conf_key_mgr.ConfKeyManager {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] key_manager.fixed_key          = **** {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] barbican.auth_endpoint         = http://localhost/identity/v3 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] barbican.barbican_api_version  = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] barbican.barbican_endpoint     = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] barbican.barbican_endpoint_type = public {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] barbican.barbican_region_name  = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] barbican.cafile                = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] barbican.certfile              = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] barbican.collect_timing        = False {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] barbican.insecure              = False {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] barbican.keyfile               = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] barbican.number_of_retries     = 60 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] barbican.retry_delay           = 1 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] barbican.send_service_user_token = False {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] barbican.split_loggers         = False {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] barbican.timeout               = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] barbican.verify_ssl            = True {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] barbican.verify_ssl_path       = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] barbican_service_user.auth_section = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] barbican_service_user.auth_type = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] barbican_service_user.cafile   = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] barbican_service_user.certfile = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] barbican_service_user.collect_timing = False {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] barbican_service_user.insecure = False {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] barbican_service_user.keyfile  = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] barbican_service_user.split_loggers = False {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] barbican_service_user.timeout  = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] vault.approle_role_id          = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] vault.approle_secret_id        = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] vault.cafile                   = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] vault.certfile                 = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] vault.collect_timing           = False {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] vault.insecure                 = False {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] vault.keyfile                  = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] vault.kv_mountpoint            = secret {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] vault.kv_path                  = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] vault.kv_version               = 2 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] vault.namespace                = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] vault.root_token_id            = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] vault.split_loggers            = False {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] vault.ssl_ca_crt_file          = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] vault.timeout                  = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] vault.use_ssl                  = False {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] vault.vault_url                = http://127.0.0.1:8200 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] keystone.auth_section          = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] keystone.auth_type             = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] keystone.cafile                = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] keystone.certfile              = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] keystone.collect_timing        = False {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] keystone.connect_retries       = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] keystone.connect_retry_delay   = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] keystone.endpoint_override     = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] keystone.insecure              = False {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] keystone.keyfile               = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] keystone.max_version           = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] keystone.min_version           = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] keystone.region_name           = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] keystone.service_name          = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] keystone.service_type          = identity {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] keystone.split_loggers         = False {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] keystone.status_code_retries   = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] keystone.status_code_retry_delay = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] keystone.timeout               = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] keystone.valid_interfaces      = ['internal', 'public'] {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] keystone.version               = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] libvirt.connection_uri         =  {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] libvirt.cpu_mode               = custom {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] libvirt.cpu_model_extra_flags  = [] {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: WARNING oslo_config.cfg [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] Deprecated: Option "cpu_model" from group "libvirt" is deprecated. Use option "cpu_models" from group "libvirt".
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] libvirt.cpu_models             = ['Nehalem'] {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] libvirt.cpu_power_governor_high = performance {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] libvirt.cpu_power_governor_low = powersave {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] libvirt.cpu_power_management   = False {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] libvirt.cpu_power_management_strategy = cpu_state {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] libvirt.device_detach_attempts = 8 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] libvirt.device_detach_timeout  = 20 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] libvirt.disk_cachemodes        = [] {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] libvirt.disk_prefix            = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] libvirt.enabled_perf_events    = [] {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] libvirt.file_backed_memory     = 0 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] libvirt.gid_maps               = [] {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] libvirt.hw_disk_discard        = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] libvirt.hw_machine_type        = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] libvirt.images_rbd_ceph_conf   =  {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] libvirt.images_rbd_glance_copy_poll_interval = 15 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] libvirt.images_rbd_glance_copy_timeout = 600 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] libvirt.images_rbd_glance_store_name =  {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] libvirt.images_rbd_pool        = rbd {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] libvirt.images_type            = default {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] libvirt.images_volume_group    = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] libvirt.inject_key             = False {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] libvirt.inject_partition       = -2 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] libvirt.inject_password        = False {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] libvirt.iscsi_iface            = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] libvirt.iser_use_multipath     = False {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] libvirt.live_migration_bandwidth = 0 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] libvirt.live_migration_completion_timeout = 800 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] libvirt.live_migration_downtime = 500 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] libvirt.live_migration_downtime_delay = 75 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] libvirt.live_migration_downtime_steps = 10 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] libvirt.live_migration_inbound_addr = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] libvirt.live_migration_permit_auto_converge = False {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] libvirt.live_migration_permit_post_copy = False {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] libvirt.live_migration_scheme  = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] libvirt.live_migration_timeout_action = abort {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] libvirt.live_migration_tunnelled = False {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: WARNING oslo_config.cfg [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] Deprecated: Option "live_migration_uri" from group "libvirt" is deprecated for removal (
Nov 29 03:01:46 user nova-compute[75234]: live_migration_uri is deprecated for removal in favor of two other options that
Nov 29 03:01:46 user nova-compute[75234]: allow to change live migration scheme and target URI: ``live_migration_scheme``
Nov 29 03:01:46 user nova-compute[75234]: and ``live_migration_inbound_addr`` respectively.
Nov 29 03:01:46 user nova-compute[75234]: ).  Its value may be silently ignored in the future.
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] libvirt.live_migration_uri     = qemu+ssh://stack@%s/system {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] libvirt.live_migration_with_native_tls = False {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] libvirt.max_queues             = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] libvirt.mem_stats_period_seconds = 10 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] libvirt.nfs_mount_options      = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] libvirt.nfs_mount_point_base   = /opt/stack/data/nova/mnt {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] libvirt.num_aoe_discover_tries = 3 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] libvirt.num_iser_scan_tries    = 5 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] libvirt.num_memory_encrypted_guests = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] libvirt.num_nvme_discover_tries = 5 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] libvirt.num_pcie_ports         = 0 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] libvirt.num_volume_scan_tries  = 5 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] libvirt.pmem_namespaces        = [] {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] libvirt.quobyte_client_cfg     = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] libvirt.quobyte_mount_point_base = /opt/stack/data/nova/mnt {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] libvirt.rbd_connect_timeout    = 5 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] libvirt.rbd_destroy_volume_retries = 12 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] libvirt.rbd_destroy_volume_retry_interval = 5 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] libvirt.rbd_secret_uuid        = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] libvirt.rbd_user               = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] libvirt.realtime_scheduler_priority = 1 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] libvirt.remote_filesystem_transport = ssh {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] libvirt.rescue_image_id        = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] libvirt.rescue_kernel_id       = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] libvirt.rescue_ramdisk_id      = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] libvirt.rng_dev_path           = /dev/urandom {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] libvirt.rx_queue_size          = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] libvirt.smbfs_mount_options    =  {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] libvirt.smbfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] libvirt.snapshot_compression   = False {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] libvirt.snapshot_image_format  = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] libvirt.snapshots_directory    = /opt/stack/data/nova/instances/snapshots {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] libvirt.sparse_logical_volumes = False {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] libvirt.swtpm_enabled          = False {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] libvirt.swtpm_group            = tss {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] libvirt.swtpm_user             = tss {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] libvirt.sysinfo_serial         = unique {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] libvirt.tb_cache_size          = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] libvirt.tx_queue_size          = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] libvirt.uid_maps               = [] {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] libvirt.use_virtio_for_bridges = True {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] libvirt.virt_type              = kvm {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] libvirt.volume_clear           = zero {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] libvirt.volume_clear_size      = 0 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] libvirt.volume_use_multipath   = False {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] libvirt.vzstorage_cache_path   = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] libvirt.vzstorage_log_path     = /var/log/vstorage/%(cluster_name)s/nova.log.gz {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] libvirt.vzstorage_mount_group  = qemu {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] libvirt.vzstorage_mount_opts   = [] {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] libvirt.vzstorage_mount_perms  = 0770 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] libvirt.vzstorage_mount_point_base = /opt/stack/data/nova/mnt {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] libvirt.vzstorage_mount_user   = stack {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] libvirt.wait_soft_reboot_seconds = 120 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] neutron.auth_section           = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] neutron.auth_type              = password {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] neutron.cafile                 = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] neutron.certfile               = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] neutron.collect_timing         = False {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] neutron.connect_retries        = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] neutron.connect_retry_delay    = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] neutron.default_floating_pool  = public {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] neutron.endpoint_override      = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] neutron.extension_sync_interval = 600 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] neutron.http_retries           = 3 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] neutron.insecure               = False {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] neutron.keyfile                = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] neutron.max_version            = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] neutron.metadata_proxy_shared_secret = **** {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] neutron.min_version            = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] neutron.ovs_bridge             = br-int {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] neutron.physnets               = [] {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] neutron.region_name            = RegionOne {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] neutron.service_metadata_proxy = True {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] neutron.service_name           = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] neutron.service_type           = network {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] neutron.split_loggers          = False {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] neutron.status_code_retries    = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] neutron.status_code_retry_delay = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] neutron.timeout                = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] neutron.valid_interfaces       = ['internal', 'public'] {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] neutron.version                = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] notifications.bdms_in_notifications = False {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] notifications.default_level    = INFO {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] notifications.notification_format = unversioned {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] notifications.notify_on_state_change = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] notifications.versioned_notifications_topics = ['versioned_notifications'] {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] pci.alias                      = [] {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] pci.device_spec                = [] {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] pci.report_in_placement        = False {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] placement.auth_section         = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] placement.auth_type            = password {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] placement.auth_url             = http://10.0.0.210/identity {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] placement.cafile               = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] placement.certfile             = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] placement.collect_timing       = False {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] placement.connect_retries      = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] placement.connect_retry_delay  = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] placement.default_domain_id    = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] placement.default_domain_name  = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] placement.domain_id            = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] placement.domain_name          = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] placement.endpoint_override    = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] placement.insecure             = False {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] placement.keyfile              = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] placement.max_version          = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] placement.min_version          = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] placement.password             = **** {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] placement.project_domain_id    = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] placement.project_domain_name  = Default {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] placement.project_id           = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] placement.project_name         = service {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] placement.region_name          = RegionOne {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] placement.service_name         = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] placement.service_type         = placement {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] placement.split_loggers        = False {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] placement.status_code_retries  = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] placement.status_code_retry_delay = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] placement.system_scope         = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] placement.timeout              = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] placement.trust_id             = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] placement.user_domain_id       = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] placement.user_domain_name     = Default {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] placement.user_id              = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] placement.username             = placement {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] placement.valid_interfaces     = ['internal', 'public'] {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] placement.version              = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] quota.cores                    = 20 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] quota.count_usage_from_placement = False {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] quota.driver                   = nova.quota.DbQuotaDriver {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] quota.injected_file_content_bytes = 10240 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] quota.injected_file_path_length = 255 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] quota.injected_files           = 5 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] quota.instances                = 10 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] quota.key_pairs                = 100 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] quota.metadata_items           = 128 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] quota.ram                      = 51200 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] quota.recheck_quota            = True {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] quota.server_group_members     = 10 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] quota.server_groups            = 10 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] rdp.enabled                    = False {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] rdp.html5_proxy_base_url       = http://127.0.0.1:6083/ {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] scheduler.discover_hosts_in_cells_interval = -1 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] scheduler.enable_isolated_aggregate_filtering = False {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] scheduler.image_metadata_prefilter = False {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] scheduler.limit_tenants_to_placement_aggregate = False {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] scheduler.max_attempts         = 3 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] scheduler.max_placement_results = 1000 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] scheduler.placement_aggregate_required_for_tenants = False {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] scheduler.query_placement_for_image_type_support = False {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] scheduler.query_placement_for_routed_network_aggregates = False {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] scheduler.workers              = 3 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] filter_scheduler.aggregate_image_properties_isolation_namespace = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] filter_scheduler.aggregate_image_properties_isolation_separator = . {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] filter_scheduler.available_filters = ['nova.scheduler.filters.all_filters'] {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] filter_scheduler.build_failure_weight_multiplier = 1000000.0 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] filter_scheduler.cpu_weight_multiplier = 1.0 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] filter_scheduler.cross_cell_move_weight_multiplier = 1000000.0 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] filter_scheduler.disk_weight_multiplier = 1.0 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] filter_scheduler.enabled_filters = ['ComputeFilter', 'ComputeCapabilitiesFilter', 'ImagePropertiesFilter', 'ServerGroupAntiAffinityFilter', 'ServerGroupAffinityFilter', 'SameHostFilter', 'DifferentHostFilter'] {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] filter_scheduler.host_subset_size = 1 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] filter_scheduler.hypervisor_version_weight_multiplier = 1.0 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] filter_scheduler.image_properties_default_architecture = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] filter_scheduler.io_ops_weight_multiplier = -1.0 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] filter_scheduler.isolated_hosts = [] {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] filter_scheduler.isolated_images = [] {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] filter_scheduler.max_instances_per_host = 50 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] filter_scheduler.max_io_ops_per_host = 8 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] filter_scheduler.num_instances_weight_multiplier = 0.0 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] filter_scheduler.pci_in_placement = False {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] filter_scheduler.pci_weight_multiplier = 1.0 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] filter_scheduler.ram_weight_multiplier = 1.0 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] filter_scheduler.restrict_isolated_hosts_to_isolated_images = True {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] filter_scheduler.shuffle_best_same_weighed_hosts = False {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] filter_scheduler.soft_affinity_weight_multiplier = 1.0 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] filter_scheduler.soft_anti_affinity_weight_multiplier = 1.0 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] filter_scheduler.track_instance_changes = False {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] filter_scheduler.weight_classes = ['nova.scheduler.weights.all_weighers'] {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] metrics.required               = True {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] metrics.weight_multiplier      = 1.0 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] metrics.weight_of_unavailable  = -10000.0 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] metrics.weight_setting         = [] {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] serial_console.base_url        = ws://127.0.0.1:6083/ {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] serial_console.enabled         = False {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] serial_console.port_range      = 10000:20000 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] serial_console.proxyclient_address = 127.0.0.1 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] serial_console.serialproxy_host = 0.0.0.0 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] serial_console.serialproxy_port = 6083 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] service_user.auth_section      = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] service_user.auth_type         = password {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] service_user.cafile            = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] service_user.certfile          = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] service_user.collect_timing    = False {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] service_user.insecure          = False {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] service_user.keyfile           = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] service_user.send_service_user_token = True {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] service_user.split_loggers     = False {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] service_user.timeout           = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] spice.agent_enabled            = True {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] spice.enabled                  = False {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] spice.html5proxy_base_url      = http://10.0.0.210:6081/spice_auto.html {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] spice.html5proxy_host          = 0.0.0.0 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] spice.html5proxy_port          = 6082 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] spice.image_compression        = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] spice.jpeg_compression         = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] spice.playback_compression     = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] spice.server_listen            = 127.0.0.1 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] spice.server_proxyclient_address = 127.0.0.1 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] spice.streaming_mode           = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] spice.zlib_compression         = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] upgrade_levels.baseapi         = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] upgrade_levels.compute         = auto {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] upgrade_levels.conductor       = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] upgrade_levels.scheduler       = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] vendordata_dynamic_auth.auth_section = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] vendordata_dynamic_auth.auth_type = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] vendordata_dynamic_auth.cafile = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] vendordata_dynamic_auth.certfile = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] vendordata_dynamic_auth.collect_timing = False {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] vendordata_dynamic_auth.insecure = False {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] vendordata_dynamic_auth.keyfile = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] vendordata_dynamic_auth.split_loggers = False {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] vendordata_dynamic_auth.timeout = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] vmware.api_retry_count         = 10 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] vmware.ca_file                 = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] vmware.cache_prefix            = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] vmware.cluster_name            = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] vmware.connection_pool_size    = 10 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] vmware.console_delay_seconds   = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] vmware.datastore_regex         = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] vmware.host_ip                 = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] vmware.host_password           = **** {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] vmware.host_port               = 443 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] vmware.host_username           = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] vmware.insecure                = False {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] vmware.integration_bridge      = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] vmware.maximum_objects         = 100 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] vmware.pbm_default_policy      = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] vmware.pbm_enabled             = False {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] vmware.pbm_wsdl_location       = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] vmware.serial_log_dir          = /opt/vmware/vspc {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] vmware.serial_port_proxy_uri   = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] vmware.serial_port_service_uri = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] vmware.task_poll_interval      = 0.5 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] vmware.use_linked_clone        = True {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] vmware.vnc_keymap              = en-us {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] vmware.vnc_port                = 5900 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] vmware.vnc_port_total          = 10000 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] vnc.auth_schemes               = ['none'] {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] vnc.enabled                    = True {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] vnc.novncproxy_base_url        = http://10.0.0.210:6080/vnc_lite.html {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:46 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] vnc.novncproxy_host            = 0.0.0.0 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:47 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] vnc.novncproxy_port            = 6080 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:47 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] vnc.server_listen              = 0.0.0.0 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:47 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] vnc.server_proxyclient_address = 10.0.0.210 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:47 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] vnc.vencrypt_ca_certs          = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:47 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] vnc.vencrypt_client_cert       = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:47 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] vnc.vencrypt_client_key        = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:47 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] workarounds.disable_compute_service_check_for_ffu = False {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:47 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] workarounds.disable_fallback_pcpu_query = False {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:47 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] workarounds.disable_group_policy_check_upcall = True {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:47 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] workarounds.disable_libvirt_livesnapshot = False {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:47 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] workarounds.disable_rootwrap   = False {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:47 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] workarounds.enable_numa_live_migration = False {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:47 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] workarounds.enable_qemu_monitor_announce_self = False {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:47 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] workarounds.ensure_libvirt_rbd_instance_dir_cleanup = False {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:47 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] workarounds.handle_virt_lifecycle_events = True {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:47 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] workarounds.libvirt_disable_apic = False {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:47 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] workarounds.never_download_image_if_on_rbd = False {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:47 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] workarounds.qemu_monitor_announce_self_count = 3 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:47 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] workarounds.qemu_monitor_announce_self_interval = 1 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:47 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] workarounds.reserve_disk_resource_for_image_cache = False {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:47 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] workarounds.skip_cpu_compare_at_startup = False {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:47 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] workarounds.skip_cpu_compare_on_dest = False {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:47 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] workarounds.skip_hypervisor_version_check_on_lm = False {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:47 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] workarounds.skip_reserve_in_use_ironic_nodes = False {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:47 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] workarounds.unified_limits_count_pcpu_as_vcpu = False {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:47 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] workarounds.wait_for_vif_plugged_event_during_hard_reboot = [] {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:47 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] wsgi.api_paste_config          = /etc/nova/api-paste.ini {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:47 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] wsgi.client_socket_timeout     = 900 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:47 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] wsgi.default_pool_size         = 1000 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:47 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] wsgi.keep_alive                = True {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:47 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] wsgi.max_header_line           = 16384 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:47 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] wsgi.secure_proxy_ssl_header   = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:47 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] wsgi.ssl_ca_file               = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:47 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] wsgi.ssl_cert_file             = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:47 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] wsgi.ssl_key_file              = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:47 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] wsgi.tcp_keepidle              = 600 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:47 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] wsgi.wsgi_log_format           = %(client_ip)s "%(request_line)s" status: %(status_code)s len: %(body_length)s time: %(wall_seconds).7f {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:47 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] zvm.ca_file                    = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:47 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] zvm.cloud_connector_url        = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:47 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] zvm.image_tmp_path             = /opt/stack/data/nova/images {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:47 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] zvm.reachable_timeout          = 300 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:47 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] oslo_policy.enforce_new_defaults = True {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:47 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] oslo_policy.enforce_scope      = True {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:47 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] oslo_policy.policy_default_rule = default {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:47 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] oslo_policy.policy_dirs        = ['policy.d'] {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:47 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] oslo_policy.policy_file        = policy.yaml {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:47 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:47 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:47 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] oslo_policy.remote_ssl_client_crt_file = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:47 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] oslo_policy.remote_ssl_client_key_file = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:47 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:47 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:47 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:47 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] profiler.connection_string     = messaging:// {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:47 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] profiler.enabled               = False {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:47 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] profiler.es_doc_type           = notification {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:47 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] profiler.es_scroll_size        = 10000 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:47 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] profiler.es_scroll_time        = 2m {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:47 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] profiler.filter_error_trace    = False {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:47 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] profiler.hmac_keys             = **** {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:47 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] profiler.sentinel_service_name = mymaster {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:47 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] profiler.socket_timeout        = 0.1 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:47 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] profiler.trace_requests        = False {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:47 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] profiler.trace_sqlalchemy      = False {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:47 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] profiler_jaeger.process_tags   = {} {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:47 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] profiler_jaeger.service_name_prefix = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:47 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] profiler_otlp.service_name_prefix = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:47 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] remote_debug.host              = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:47 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] remote_debug.port              = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:47 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:47 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:47 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:47 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:47 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:47 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:47 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] oslo_messaging_rabbit.heartbeat_in_pthread = False {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:47 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] oslo_messaging_rabbit.heartbeat_rate = 3 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:47 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] oslo_messaging_rabbit.heartbeat_timeout_threshold = 60 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:47 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] oslo_messaging_rabbit.kombu_compression = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:47 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:47 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:47 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:47 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:47 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:47 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:47 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:47 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] oslo_messaging_rabbit.rabbit_quorum_delivery_limit = 0 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:47 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] oslo_messaging_rabbit.rabbit_quorum_max_memory_bytes = 0 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:47 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] oslo_messaging_rabbit.rabbit_quorum_max_memory_length = 0 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:47 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] oslo_messaging_rabbit.rabbit_quorum_queue = False {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:47 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:47 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:47 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:47 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:47 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] oslo_messaging_rabbit.ssl      = False {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:47 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] oslo_messaging_rabbit.ssl_ca_file =  {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:47 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] oslo_messaging_rabbit.ssl_cert_file =  {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:47 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] oslo_messaging_rabbit.ssl_enforce_fips_mode = False {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:47 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] oslo_messaging_rabbit.ssl_key_file =  {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:47 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] oslo_messaging_rabbit.ssl_version =  {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:47 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] oslo_messaging_notifications.driver = ['messagingv2'] {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:47 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] oslo_messaging_notifications.retry = -1 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:47 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] oslo_messaging_notifications.topics = ['notifications'] {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:47 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] oslo_messaging_notifications.transport_url = **** {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:47 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] oslo_limit.auth_section        = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:47 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] oslo_limit.auth_type           = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:47 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] oslo_limit.cafile              = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:47 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] oslo_limit.certfile            = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:47 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] oslo_limit.collect_timing      = False {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:47 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] oslo_limit.connect_retries     = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:47 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] oslo_limit.connect_retry_delay = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:47 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] oslo_limit.endpoint_id         = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:47 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] oslo_limit.endpoint_override   = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:47 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] oslo_limit.insecure            = False {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:47 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] oslo_limit.keyfile             = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:47 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] oslo_limit.max_version         = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:47 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] oslo_limit.min_version         = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:47 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] oslo_limit.region_name         = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:47 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] oslo_limit.service_name        = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:47 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] oslo_limit.service_type        = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:47 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] oslo_limit.split_loggers       = False {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:47 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] oslo_limit.status_code_retries = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:47 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] oslo_limit.status_code_retry_delay = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:47 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] oslo_limit.timeout             = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:47 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] oslo_limit.valid_interfaces    = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:47 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] oslo_limit.version             = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:47 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] oslo_reports.file_event_handler = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:47 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] oslo_reports.file_event_handler_interval = 1 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:47 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] oslo_reports.log_dir           = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:47 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] vif_plug_linux_bridge_privileged.capabilities = [12] {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:47 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] vif_plug_linux_bridge_privileged.group = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:47 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] vif_plug_linux_bridge_privileged.helper_command = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:47 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] vif_plug_linux_bridge_privileged.logger_name = oslo_privsep.daemon {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:47 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] vif_plug_linux_bridge_privileged.thread_pool_size = 12 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:47 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] vif_plug_linux_bridge_privileged.user = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:47 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] vif_plug_ovs_privileged.capabilities = [12, 1] {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:47 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] vif_plug_ovs_privileged.group  = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:47 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] vif_plug_ovs_privileged.helper_command = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:47 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] vif_plug_ovs_privileged.logger_name = oslo_privsep.daemon {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:47 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] vif_plug_ovs_privileged.thread_pool_size = 12 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:47 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] vif_plug_ovs_privileged.user   = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:47 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] os_vif_linux_bridge.flat_interface = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:47 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] os_vif_linux_bridge.forward_bridge_interface = ['all'] {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:47 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] os_vif_linux_bridge.iptables_bottom_regex =  {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:47 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] os_vif_linux_bridge.iptables_drop_action = DROP {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:47 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] os_vif_linux_bridge.iptables_top_regex =  {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:47 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] os_vif_linux_bridge.network_device_mtu = 1500 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:47 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] os_vif_linux_bridge.use_ipv6   = False {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:47 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] os_vif_linux_bridge.vlan_interface = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:47 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] os_vif_ovs.default_qos_type    = linux-noop {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:47 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] os_vif_ovs.isolate_vif         = False {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:47 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] os_vif_ovs.network_device_mtu  = 1500 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:47 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] os_vif_ovs.ovs_vsctl_timeout   = 120 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:47 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] os_vif_ovs.ovsdb_connection    = tcp:127.0.0.1:6640 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:47 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] os_vif_ovs.ovsdb_interface     = native {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:47 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] os_vif_ovs.per_port_bridge     = False {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:47 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] os_brick.lock_path             = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:47 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] privsep_osbrick.capabilities   = [21] {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:47 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] privsep_osbrick.group          = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:47 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] privsep_osbrick.helper_command = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:47 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] privsep_osbrick.logger_name    = os_brick.privileged {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:47 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] privsep_osbrick.thread_pool_size = 12 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:47 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] privsep_osbrick.user           = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:47 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] nova_sys_admin.capabilities    = [0, 1, 2, 3, 12, 21] {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:47 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] nova_sys_admin.group           = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:47 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] nova_sys_admin.helper_command  = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:47 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] nova_sys_admin.logger_name     = oslo_privsep.daemon {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:47 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] nova_sys_admin.thread_pool_size = 12 {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:47 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] nova_sys_admin.user            = None {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2609}}
Nov 29 03:01:47 user nova-compute[75234]: DEBUG oslo_service.service [None req-94a708b3-bc26-4241-aee8-43e501b23a5b None None] ******************************************************************************** {{(pid=75234) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}}
Nov 29 03:01:47 user nova-compute[75234]: INFO nova.service [-] Starting compute node (version 0.0.0)
Nov 29 03:01:47 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-ef65ec89-6c59-4f39-9adc-df4888ffd41b None None] Connecting to libvirt: qemu:///system {{(pid=75234) _get_new_connection /opt/stack/nova/nova/virt/libvirt/host.py:508}}
Nov 29 03:01:47 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-ef65ec89-6c59-4f39-9adc-df4888ffd41b None None] Registering for lifecycle events <nova.virt.libvirt.host.Host object at 0x7f8a68455c90> {{(pid=75234) _get_new_connection /opt/stack/nova/nova/virt/libvirt/host.py:514}}
Nov 29 03:01:47 user nova-compute[75234]: libvirt:  error : internal error: could not initialize domain event timer
Nov 29 03:01:47 user nova-compute[75234]: WARNING nova.virt.libvirt.host [None req-ef65ec89-6c59-4f39-9adc-df4888ffd41b None None] URI qemu:///system does not support events: internal error: could not initialize domain event timer: libvirt.libvirtError: internal error: could not initialize domain event timer
Nov 29 03:01:47 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-ef65ec89-6c59-4f39-9adc-df4888ffd41b None None] Registering for connection events: <nova.virt.libvirt.host.Host object at 0x7f8a68455c90> {{(pid=75234) _get_new_connection /opt/stack/nova/nova/virt/libvirt/host.py:535}}
Nov 29 03:01:47 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-ef65ec89-6c59-4f39-9adc-df4888ffd41b None None] Starting native event thread {{(pid=75234) _init_events /opt/stack/nova/nova/virt/libvirt/host.py:494}}
Nov 29 03:01:47 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-ef65ec89-6c59-4f39-9adc-df4888ffd41b None None] Starting green dispatch thread {{(pid=75234) _init_events /opt/stack/nova/nova/virt/libvirt/host.py:500}}
Nov 29 03:01:47 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-ef65ec89-6c59-4f39-9adc-df4888ffd41b None None] Starting connection event dispatch thread {{(pid=75234) _init_events /opt/stack/nova/nova/virt/libvirt/host.py:503}}
Nov 29 03:01:47 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-ef65ec89-6c59-4f39-9adc-df4888ffd41b None None] Connection event '1' reason 'None'
Nov 29 03:01:47 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-ef65ec89-6c59-4f39-9adc-df4888ffd41b None None] Cannot update service status on host "user" since it is not registered.: nova.exception_Remote.ComputeHostNotFound_Remote: Compute host user could not be found.
Nov 29 03:01:47 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.mount [None req-ef65ec89-6c59-4f39-9adc-df4888ffd41b None None] Initialising _HostMountState generation 0 {{(pid=75234) host_up /opt/stack/nova/nova/virt/libvirt/volume/mount.py:130}}
Nov 29 03:01:54 user nova-compute[75234]: INFO nova.virt.libvirt.host [None req-ef65ec89-6c59-4f39-9adc-df4888ffd41b None None] Libvirt host capabilities <capabilities>
Nov 29 03:01:54 user nova-compute[75234]:   <host>
Nov 29 03:01:54 user nova-compute[75234]:     <uuid>e20c3142-5af9-7467-ecd8-70b2e4a210d6</uuid>
Nov 29 03:01:54 user nova-compute[75234]:     <cpu>
Nov 29 03:01:54 user nova-compute[75234]:       <arch>x86_64</arch>
Nov 29 03:01:54 user nova-compute[75234]:       <model>IvyBridge-IBRS</model>
Nov 29 03:01:54 user nova-compute[75234]:       <vendor>Intel</vendor>
Nov 29 03:01:54 user nova-compute[75234]:       <microcode version='1070'/>
Nov 29 03:01:54 user nova-compute[75234]:       <counter name='tsc' frequency='2094952000' scaling='no'/>
Nov 29 03:01:54 user nova-compute[75234]:       <topology sockets='6' dies='1' cores='1' threads='1'/>
Nov 29 03:01:54 user nova-compute[75234]:       <feature name='ss'/>
Nov 29 03:01:54 user nova-compute[75234]:       <feature name='vmx'/>
Nov 29 03:01:54 user nova-compute[75234]:       <feature name='pcid'/>
Nov 29 03:01:54 user nova-compute[75234]:       <feature name='osxsave'/>
Nov 29 03:01:54 user nova-compute[75234]:       <feature name='hypervisor'/>
Nov 29 03:01:54 user nova-compute[75234]:       <feature name='arat'/>
Nov 29 03:01:54 user nova-compute[75234]:       <feature name='tsc_adjust'/>
Nov 29 03:01:54 user nova-compute[75234]:       <feature name='md-clear'/>
Nov 29 03:01:54 user nova-compute[75234]:       <feature name='stibp'/>
Nov 29 03:01:54 user nova-compute[75234]:       <feature name='arch-capabilities'/>
Nov 29 03:01:54 user nova-compute[75234]:       <feature name='ssbd'/>
Nov 29 03:01:54 user nova-compute[75234]:       <feature name='invtsc'/>
Nov 29 03:01:54 user nova-compute[75234]:       <feature name='rsba'/>
Nov 29 03:01:54 user nova-compute[75234]:       <feature name='skip-l1dfl-vmentry'/>
Nov 29 03:01:54 user nova-compute[75234]:       <pages unit='KiB' size='4'/>
Nov 29 03:01:54 user nova-compute[75234]:       <pages unit='KiB' size='2048'/>
Nov 29 03:01:54 user nova-compute[75234]:     </cpu>
Nov 29 03:01:54 user nova-compute[75234]:     <power_management>
Nov 29 03:01:54 user nova-compute[75234]:       <suspend_mem/>
Nov 29 03:01:54 user nova-compute[75234]:       <suspend_disk/>
Nov 29 03:01:54 user nova-compute[75234]:       <suspend_hybrid/>
Nov 29 03:01:54 user nova-compute[75234]:     </power_management>
Nov 29 03:01:54 user nova-compute[75234]:     <iommu support='no'/>
Nov 29 03:01:54 user nova-compute[75234]:     <migration_features>
Nov 29 03:01:54 user nova-compute[75234]:       <live/>
Nov 29 03:01:54 user nova-compute[75234]:       <uri_transports>
Nov 29 03:01:54 user nova-compute[75234]:         <uri_transport>tcp</uri_transport>
Nov 29 03:01:54 user nova-compute[75234]:         <uri_transport>rdma</uri_transport>
Nov 29 03:01:54 user nova-compute[75234]:       </uri_transports>
Nov 29 03:01:54 user nova-compute[75234]:     </migration_features>
Nov 29 03:01:54 user nova-compute[75234]:     <topology>
Nov 29 03:01:54 user nova-compute[75234]:       <cells num='2'>
Nov 29 03:01:54 user nova-compute[75234]:         <cell id='0'>
Nov 29 03:01:54 user nova-compute[75234]:           <memory unit='KiB'>8140616</memory>
Nov 29 03:01:54 user nova-compute[75234]:           <pages unit='KiB' size='4'>2035154</pages>
Nov 29 03:01:54 user nova-compute[75234]:           <pages unit='KiB' size='2048'>0</pages>
Nov 29 03:01:54 user nova-compute[75234]:           <distances>
Nov 29 03:01:54 user nova-compute[75234]:             <sibling id='0' value='10'/>
Nov 29 03:01:54 user nova-compute[75234]:             <sibling id='1' value='20'/>
Nov 29 03:01:54 user nova-compute[75234]:           </distances>
Nov 29 03:01:54 user nova-compute[75234]:           <cpus num='6'>
Nov 29 03:01:54 user nova-compute[75234]:             <cpu id='0' socket_id='0' die_id='0' core_id='0' siblings='0'/>
Nov 29 03:01:54 user nova-compute[75234]:             <cpu id='1' socket_id='2' die_id='0' core_id='0' siblings='1'/>
Nov 29 03:01:54 user nova-compute[75234]:             <cpu id='2' socket_id='4' die_id='0' core_id='0' siblings='2'/>
Nov 29 03:01:54 user nova-compute[75234]:             <cpu id='3' socket_id='6' die_id='0' core_id='0' siblings='3'/>
Nov 29 03:01:54 user nova-compute[75234]:             <cpu id='4' socket_id='8' die_id='0' core_id='0' siblings='4'/>
Nov 29 03:01:54 user nova-compute[75234]:             <cpu id='5' socket_id='10' die_id='0' core_id='0' siblings='5'/>
Nov 29 03:01:54 user nova-compute[75234]:           </cpus>
Nov 29 03:01:54 user nova-compute[75234]:         </cell>
Nov 29 03:01:54 user nova-compute[75234]:         <cell id='1'>
Nov 29 03:01:54 user nova-compute[75234]:           <memory unit='KiB'>8255068</memory>
Nov 29 03:01:54 user nova-compute[75234]:           <pages unit='KiB' size='4'>2063767</pages>
Nov 29 03:01:54 user nova-compute[75234]:           <pages unit='KiB' size='2048'>0</pages>
Nov 29 03:01:54 user nova-compute[75234]:           <distances>
Nov 29 03:01:54 user nova-compute[75234]:             <sibling id='0' value='20'/>
Nov 29 03:01:54 user nova-compute[75234]:             <sibling id='1' value='10'/>
Nov 29 03:01:54 user nova-compute[75234]:           </distances>
Nov 29 03:01:54 user nova-compute[75234]:           <cpus num='6'>
Nov 29 03:01:54 user nova-compute[75234]:             <cpu id='6' socket_id='12' die_id='0' core_id='0' siblings='6'/>
Nov 29 03:01:54 user nova-compute[75234]:             <cpu id='7' socket_id='14' die_id='0' core_id='0' siblings='7'/>
Nov 29 03:01:54 user nova-compute[75234]:             <cpu id='8' socket_id='16' die_id='0' core_id='0' siblings='8'/>
Nov 29 03:01:54 user nova-compute[75234]:             <cpu id='9' socket_id='18' die_id='0' core_id='0' siblings='9'/>
Nov 29 03:01:54 user nova-compute[75234]:             <cpu id='10' socket_id='20' die_id='0' core_id='0' siblings='10'/>
Nov 29 03:01:54 user nova-compute[75234]:             <cpu id='11' socket_id='22' die_id='0' core_id='0' siblings='11'/>
Nov 29 03:01:54 user nova-compute[75234]:           </cpus>
Nov 29 03:01:54 user nova-compute[75234]:         </cell>
Nov 29 03:01:54 user nova-compute[75234]:       </cells>
Nov 29 03:01:54 user nova-compute[75234]:     </topology>
Nov 29 03:01:54 user nova-compute[75234]:     <cache>
Nov 29 03:01:54 user nova-compute[75234]:       <bank id='0' level='3' type='both' size='15' unit='MiB' cpus='0'/>
Nov 29 03:01:54 user nova-compute[75234]:       <bank id='2' level='3' type='both' size='15' unit='MiB' cpus='1'/>
Nov 29 03:01:54 user nova-compute[75234]:       <bank id='4' level='3' type='both' size='15' unit='MiB' cpus='2'/>
Nov 29 03:01:54 user nova-compute[75234]:       <bank id='6' level='3' type='both' size='15' unit='MiB' cpus='3'/>
Nov 29 03:01:54 user nova-compute[75234]:       <bank id='8' level='3' type='both' size='15' unit='MiB' cpus='4'/>
Nov 29 03:01:54 user nova-compute[75234]:       <bank id='10' level='3' type='both' size='15' unit='MiB' cpus='5'/>
Nov 29 03:01:54 user nova-compute[75234]:       <bank id='12' level='3' type='both' size='15' unit='MiB' cpus='6'/>
Nov 29 03:01:54 user nova-compute[75234]:       <bank id='14' level='3' type='both' size='15' unit='MiB' cpus='7'/>
Nov 29 03:01:54 user nova-compute[75234]:       <bank id='16' level='3' type='both' size='15' unit='MiB' cpus='8'/>
Nov 29 03:01:54 user nova-compute[75234]:       <bank id='18' level='3' type='both' size='15' unit='MiB' cpus='9'/>
Nov 29 03:01:54 user nova-compute[75234]:       <bank id='20' level='3' type='both' size='15' unit='MiB' cpus='10'/>
Nov 29 03:01:54 user nova-compute[75234]:       <bank id='22' level='3' type='both' size='15' unit='MiB' cpus='11'/>
Nov 29 03:01:54 user nova-compute[75234]:     </cache>
Nov 29 03:01:54 user nova-compute[75234]:     <secmodel>
Nov 29 03:01:54 user nova-compute[75234]:       <model>apparmor</model>
Nov 29 03:01:54 user nova-compute[75234]:       <doi>0</doi>
Nov 29 03:01:54 user nova-compute[75234]:     </secmodel>
Nov 29 03:01:54 user nova-compute[75234]:     <secmodel>
Nov 29 03:01:54 user nova-compute[75234]:       <model>dac</model>
Nov 29 03:01:54 user nova-compute[75234]:       <doi>0</doi>
Nov 29 03:01:54 user nova-compute[75234]:       <baselabel type='kvm'>+64055:+108</baselabel>
Nov 29 03:01:54 user nova-compute[75234]:       <baselabel type='qemu'>+64055:+108</baselabel>
Nov 29 03:01:54 user nova-compute[75234]:     </secmodel>
Nov 29 03:01:54 user nova-compute[75234]:   </host>
Nov 29 03:01:54 user nova-compute[75234]:   <guest>
Nov 29 03:01:54 user nova-compute[75234]:     <os_type>hvm</os_type>
Nov 29 03:01:54 user nova-compute[75234]:     <arch name='alpha'>
Nov 29 03:01:54 user nova-compute[75234]:       <wordsize>64</wordsize>
Nov 29 03:01:54 user nova-compute[75234]:       <emulator>/usr/bin/qemu-system-alpha</emulator>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='4'>clipper</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <domain type='qemu'/>
Nov 29 03:01:54 user nova-compute[75234]:     </arch>
Nov 29 03:01:54 user nova-compute[75234]:     <features>
Nov 29 03:01:54 user nova-compute[75234]:       <cpuselection/>
Nov 29 03:01:54 user nova-compute[75234]:       <deviceboot/>
Nov 29 03:01:54 user nova-compute[75234]:       <disksnapshot default='on' toggle='no'/>
Nov 29 03:01:54 user nova-compute[75234]:     </features>
Nov 29 03:01:54 user nova-compute[75234]:   </guest>
Nov 29 03:01:54 user nova-compute[75234]:   <guest>
Nov 29 03:01:54 user nova-compute[75234]:     <os_type>hvm</os_type>
Nov 29 03:01:54 user nova-compute[75234]:     <arch name='armv6l'>
Nov 29 03:01:54 user nova-compute[75234]:       <wordsize>32</wordsize>
Nov 29 03:01:54 user nova-compute[75234]:       <emulator>/usr/bin/qemu-system-arm</emulator>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>integratorcp</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='2'>ast2600-evb</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>borzoi</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>spitz</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='255'>virt-2.7</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='2'>nuri</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='2'>mcimx7d-sabre</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>romulus-bmc</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='512'>virt-3.0</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='512'>virt-5.0</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='2'>npcm750-evb</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='255'>virt-2.10</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='2'>rainier-bmc</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>mps3-an547</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='2'>musca-b1</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='4'>realview-pbx-a9</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>versatileab</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>kzm</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='255'>virt-2.8</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='2'>musca-a</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='512'>virt-3.1</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>mcimx6ul-evk</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='512'>virt-5.1</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='2'>smdkc210</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>sx1</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='255'>virt-2.11</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>imx25-pdk</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>stm32vldiscovery</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='255'>virt-2.9</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='4'>orangepi-pc</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>quanta-q71l-bmc</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>z2</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='512'>virt-5.2</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>xilinx-zynq-a9</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>tosa</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>mps2-an500</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='255'>virt-2.12</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='2'>mps2-an521</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='4'>sabrelite</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>mps2-an511</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>canon-a1100</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>realview-eb</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='2'>quanta-gbs-bmc</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>emcraft-sf2</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>realview-pb-a8</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='512'>virt-4.0</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>raspi1ap</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>palmetto-bmc</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>sx1-v1</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>n810</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>g220a-bmc</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>n800</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='2'>tacoma-bmc</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='512'>virt-4.1</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='2'>quanta-gsj</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>versatilepb</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>terrier</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>mainstone</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='4'>realview-eb-mpcore</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>supermicrox11-bmc</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='512'>virt-4.2</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>witherspoon-bmc</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='2'>mps3-an524</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1' deprecated='yes'>swift-bmc</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='2'>kudo-bmc</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='4'>vexpress-a9</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='4'>midway</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>musicpal</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>lm3s811evb</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>lm3s6965evb</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>microbit</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>mps2-an505</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>mps2-an385</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='512'>virt-6.0</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>cubieboard</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>verdex</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>netduino2</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>mps2-an386</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='512'>virt-6.1</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='4'>raspi2b</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='4'>vexpress-a15</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='2'>fuji-bmc</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='512'>virt-6.2</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine canonical='virt-6.2' maxCpus='512'>virt</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>sonorapass-bmc</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>cheetah</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='255'>virt-2.6</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>ast2500-evb</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='4'>highbank</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>akita</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>connex</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>netduinoplus2</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>collie</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>raspi0</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>fp5280g2-bmc</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <domain type='qemu'/>
Nov 29 03:01:54 user nova-compute[75234]:     </arch>
Nov 29 03:01:54 user nova-compute[75234]:     <features>
Nov 29 03:01:54 user nova-compute[75234]:       <cpuselection/>
Nov 29 03:01:54 user nova-compute[75234]:       <deviceboot/>
Nov 29 03:01:54 user nova-compute[75234]:       <disksnapshot default='on' toggle='no'/>
Nov 29 03:01:54 user nova-compute[75234]:     </features>
Nov 29 03:01:54 user nova-compute[75234]:   </guest>
Nov 29 03:01:54 user nova-compute[75234]:   <guest>
Nov 29 03:01:54 user nova-compute[75234]:     <os_type>hvm</os_type>
Nov 29 03:01:54 user nova-compute[75234]:     <arch name='armv7l'>
Nov 29 03:01:54 user nova-compute[75234]:       <wordsize>32</wordsize>
Nov 29 03:01:54 user nova-compute[75234]:       <emulator>/usr/bin/qemu-system-arm</emulator>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>integratorcp</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='2'>ast2600-evb</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>borzoi</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>spitz</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='255'>virt-2.7</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='2'>nuri</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='2'>mcimx7d-sabre</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>romulus-bmc</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='512'>virt-3.0</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='512'>virt-5.0</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='2'>npcm750-evb</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='255'>virt-2.10</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='2'>rainier-bmc</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>mps3-an547</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='2'>musca-b1</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='4'>realview-pbx-a9</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>versatileab</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>kzm</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='255'>virt-2.8</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='2'>musca-a</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='512'>virt-3.1</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>mcimx6ul-evk</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='512'>virt-5.1</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='2'>smdkc210</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>sx1</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='255'>virt-2.11</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>imx25-pdk</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>stm32vldiscovery</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='255'>virt-2.9</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='4'>orangepi-pc</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>quanta-q71l-bmc</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>z2</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='512'>virt-5.2</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>xilinx-zynq-a9</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>tosa</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>mps2-an500</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='255'>virt-2.12</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='2'>mps2-an521</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='4'>sabrelite</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>mps2-an511</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>canon-a1100</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>realview-eb</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='2'>quanta-gbs-bmc</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>emcraft-sf2</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>realview-pb-a8</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='512'>virt-4.0</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>raspi1ap</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>palmetto-bmc</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>sx1-v1</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>n810</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>g220a-bmc</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>n800</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='2'>tacoma-bmc</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='512'>virt-4.1</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='2'>quanta-gsj</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>versatilepb</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>terrier</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>mainstone</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='4'>realview-eb-mpcore</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>supermicrox11-bmc</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='512'>virt-4.2</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>witherspoon-bmc</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='2'>mps3-an524</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1' deprecated='yes'>swift-bmc</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='2'>kudo-bmc</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='4'>vexpress-a9</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='4'>midway</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>musicpal</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>lm3s811evb</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>lm3s6965evb</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>microbit</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>mps2-an505</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>mps2-an385</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='512'>virt-6.0</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>cubieboard</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>verdex</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>netduino2</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>mps2-an386</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='512'>virt-6.1</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='4'>raspi2b</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='4'>vexpress-a15</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='2'>fuji-bmc</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='512'>virt-6.2</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine canonical='virt-6.2' maxCpus='512'>virt</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>sonorapass-bmc</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>cheetah</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='255'>virt-2.6</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>ast2500-evb</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='4'>highbank</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>akita</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>connex</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>netduinoplus2</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>collie</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>raspi0</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>fp5280g2-bmc</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <domain type='qemu'/>
Nov 29 03:01:54 user nova-compute[75234]:     </arch>
Nov 29 03:01:54 user nova-compute[75234]:     <features>
Nov 29 03:01:54 user nova-compute[75234]:       <cpuselection/>
Nov 29 03:01:54 user nova-compute[75234]:       <deviceboot/>
Nov 29 03:01:54 user nova-compute[75234]:       <disksnapshot default='on' toggle='no'/>
Nov 29 03:01:54 user nova-compute[75234]:     </features>
Nov 29 03:01:54 user nova-compute[75234]:   </guest>
Nov 29 03:01:54 user nova-compute[75234]:   <guest>
Nov 29 03:01:54 user nova-compute[75234]:     <os_type>hvm</os_type>
Nov 29 03:01:54 user nova-compute[75234]:     <arch name='aarch64'>
Nov 29 03:01:54 user nova-compute[75234]:       <wordsize>64</wordsize>
Nov 29 03:01:54 user nova-compute[75234]:       <emulator>/usr/bin/qemu-system-aarch64</emulator>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>integratorcp</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='2'>ast2600-evb</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>borzoi</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>spitz</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='255'>virt-2.7</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='2'>nuri</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='2'>mcimx7d-sabre</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>romulus-bmc</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='512'>virt-3.0</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='512'>virt-5.0</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='2'>npcm750-evb</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='255'>virt-2.10</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='2'>rainier-bmc</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>mps3-an547</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='255'>virt-2.8</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='2'>musca-b1</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='4'>realview-pbx-a9</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>versatileab</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>kzm</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='2'>musca-a</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='512'>virt-3.1</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>mcimx6ul-evk</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='512'>virt-5.1</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='2'>smdkc210</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>sx1</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='255'>virt-2.11</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>imx25-pdk</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>stm32vldiscovery</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='255'>virt-2.9</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='4'>orangepi-pc</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>quanta-q71l-bmc</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>z2</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='512'>virt-5.2</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>xilinx-zynq-a9</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='6'>xlnx-zcu102</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>tosa</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>mps2-an500</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='255'>virt-2.12</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='2'>mps2-an521</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='4'>sabrelite</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>mps2-an511</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>canon-a1100</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>realview-eb</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='2'>quanta-gbs-bmc</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>emcraft-sf2</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>realview-pb-a8</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='512'>sbsa-ref</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='512'>virt-4.0</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>raspi1ap</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>palmetto-bmc</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>sx1-v1</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>n810</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>g220a-bmc</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>n800</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='2'>tacoma-bmc</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='512'>virt-4.1</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='2'>quanta-gsj</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>versatilepb</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>terrier</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>mainstone</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='4'>realview-eb-mpcore</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>supermicrox11-bmc</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='512'>virt-4.2</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>witherspoon-bmc</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='2'>mps3-an524</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1' deprecated='yes'>swift-bmc</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='2'>kudo-bmc</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='4'>vexpress-a9</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='4'>midway</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>musicpal</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>lm3s811evb</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>lm3s6965evb</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>microbit</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>mps2-an505</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>mps2-an385</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='512'>virt-6.0</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='4'>raspi3ap</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>cubieboard</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>verdex</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>netduino2</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='2'>xlnx-versal-virt</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>mps2-an386</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='512'>virt-6.1</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='4'>raspi3b</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='4'>raspi2b</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='4'>vexpress-a15</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='2'>fuji-bmc</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='512'>virt-6.2</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine canonical='virt-6.2' maxCpus='512'>virt</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>sonorapass-bmc</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>cheetah</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='255'>virt-2.6</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>ast2500-evb</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='4'>highbank</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>akita</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>connex</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>netduinoplus2</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>collie</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>raspi0</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>fp5280g2-bmc</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <domain type='qemu'/>
Nov 29 03:01:54 user nova-compute[75234]:     </arch>
Nov 29 03:01:54 user nova-compute[75234]:     <features>
Nov 29 03:01:54 user nova-compute[75234]:       <acpi default='on' toggle='yes'/>
Nov 29 03:01:54 user nova-compute[75234]:       <cpuselection/>
Nov 29 03:01:54 user nova-compute[75234]:       <deviceboot/>
Nov 29 03:01:54 user nova-compute[75234]:       <disksnapshot default='on' toggle='no'/>
Nov 29 03:01:54 user nova-compute[75234]:     </features>
Nov 29 03:01:54 user nova-compute[75234]:   </guest>
Nov 29 03:01:54 user nova-compute[75234]:   <guest>
Nov 29 03:01:54 user nova-compute[75234]:     <os_type>hvm</os_type>
Nov 29 03:01:54 user nova-compute[75234]:     <arch name='cris'>
Nov 29 03:01:54 user nova-compute[75234]:       <wordsize>32</wordsize>
Nov 29 03:01:54 user nova-compute[75234]:       <emulator>/usr/bin/qemu-system-cris</emulator>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>axis-dev88</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <domain type='qemu'/>
Nov 29 03:01:54 user nova-compute[75234]:     </arch>
Nov 29 03:01:54 user nova-compute[75234]:     <features>
Nov 29 03:01:54 user nova-compute[75234]:       <cpuselection/>
Nov 29 03:01:54 user nova-compute[75234]:       <deviceboot/>
Nov 29 03:01:54 user nova-compute[75234]:       <disksnapshot default='on' toggle='no'/>
Nov 29 03:01:54 user nova-compute[75234]:     </features>
Nov 29 03:01:54 user nova-compute[75234]:   </guest>
Nov 29 03:01:54 user nova-compute[75234]:   <guest>
Nov 29 03:01:54 user nova-compute[75234]:     <os_type>hvm</os_type>
Nov 29 03:01:54 user nova-compute[75234]:     <arch name='i686'>
Nov 29 03:01:54 user nova-compute[75234]:       <wordsize>32</wordsize>
Nov 29 03:01:54 user nova-compute[75234]:       <emulator>/usr/bin/qemu-system-i386</emulator>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='255'>pc-i440fx-jammy</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine canonical='pc-i440fx-jammy' maxCpus='255'>ubuntu</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='255'>pc-i440fx-impish-hpb</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='288'>pc-q35-5.2</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='255'>pc-i440fx-2.12</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='255'>pc-i440fx-2.0</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='255'>pc-i440fx-xenial</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='255'>pc-i440fx-6.2</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine canonical='pc-i440fx-6.2' maxCpus='255'>pc</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='288'>pc-q35-4.2</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='255'>pc-i440fx-2.5</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='255'>pc-i440fx-4.2</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='255'>pc-i440fx-focal</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='255'>pc-i440fx-hirsute</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='255'>pc-q35-xenial</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='255'>pc-i440fx-jammy-hpb</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='255'>pc-i440fx-5.2</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='255'>pc-i440fx-1.5</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='255'>pc-q35-2.7</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='288'>pc-q35-eoan-hpb</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='255'>pc-i440fx-zesty</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='255'>pc-i440fx-disco-hpb</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='288'>pc-q35-groovy</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='255'>pc-i440fx-groovy</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='288'>pc-q35-artful</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='255'>pc-i440fx-2.2</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='255'>pc-i440fx-trusty</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='255'>pc-i440fx-eoan-hpb</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='288'>pc-q35-focal-hpb</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='288'>pc-q35-bionic-hpb</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='255'>pc-i440fx-artful</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='255'>pc-i440fx-2.7</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='288'>pc-q35-6.1</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='255'>pc-i440fx-yakkety</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='255'>pc-q35-2.4</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='288'>pc-q35-cosmic-hpb</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='288'>pc-q35-2.10</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>x-remote</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='288'>pc-q35-5.1</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='255'>pc-i440fx-1.7</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='288'>pc-q35-2.9</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='255'>pc-i440fx-2.11</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='288'>pc-q35-3.1</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='255'>pc-i440fx-6.1</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='288'>pc-q35-4.1</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='288'>pc-q35-jammy</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine canonical='pc-q35-jammy' maxCpus='288'>ubuntu-q35</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='255'>pc-i440fx-2.4</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='255'>pc-i440fx-4.1</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='288'>pc-q35-eoan</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='288'>pc-q35-jammy-hpb</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='255'>pc-i440fx-5.1</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='255'>pc-i440fx-2.9</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='255'>pc-i440fx-bionic-hpb</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>isapc</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='255'>pc-i440fx-1.4</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='288'>pc-q35-cosmic</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='255'>pc-q35-2.6</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='255'>pc-i440fx-3.1</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='288'>pc-q35-bionic</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='288'>pc-q35-disco-hpb</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='255'>pc-i440fx-cosmic</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='288'>pc-q35-2.12</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='255'>pc-i440fx-bionic</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='288'>pc-q35-groovy-hpb</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='288'>pc-q35-disco</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='255'>pc-i440fx-cosmic-hpb</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='255'>pc-i440fx-2.1</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='255'>pc-i440fx-wily</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='288'>pc-q35-impish</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='288'>pc-q35-6.0</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='255'>pc-i440fx-impish</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='255'>pc-i440fx-2.6</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='288'>pc-q35-impish-hpb</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='288'>pc-q35-hirsute</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='288'>pc-q35-4.0.1</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='288'>pc-q35-hirsute-hpb</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='255'>pc-i440fx-1.6</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='288'>pc-q35-5.0</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='288'>pc-q35-2.8</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='255'>pc-i440fx-2.10</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='288'>pc-q35-3.0</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='255'>pc-i440fx-6.0</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='288'>pc-q35-zesty</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='288'>pc-q35-4.0</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='288'>pc-q35-focal</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='288'>microvm</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='255'>pc-i440fx-2.3</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='255'>pc-i440fx-focal-hpb</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='255'>pc-i440fx-disco</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='255'>pc-i440fx-4.0</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='255'>pc-i440fx-groovy-hpb</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='255'>pc-i440fx-hirsute-hpb</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='255'>pc-i440fx-5.0</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='288'>pc-q35-6.2</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine canonical='pc-q35-6.2' maxCpus='288'>q35</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='255'>pc-i440fx-2.8</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='255'>pc-i440fx-eoan</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='255'>pc-q35-2.5</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='255'>pc-i440fx-3.0</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='255'>pc-q35-yakkety</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='288'>pc-q35-2.11</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <domain type='qemu'/>
Nov 29 03:01:54 user nova-compute[75234]:       <domain type='kvm'/>
Nov 29 03:01:54 user nova-compute[75234]:     </arch>
Nov 29 03:01:54 user nova-compute[75234]:     <features>
Nov 29 03:01:54 user nova-compute[75234]:       <pae/>
Nov 29 03:01:54 user nova-compute[75234]:       <nonpae/>
Nov 29 03:01:54 user nova-compute[75234]:       <acpi default='on' toggle='yes'/>
Nov 29 03:01:54 user nova-compute[75234]:       <apic default='on' toggle='no'/>
Nov 29 03:01:54 user nova-compute[75234]:       <cpuselection/>
Nov 29 03:01:54 user nova-compute[75234]:       <deviceboot/>
Nov 29 03:01:54 user nova-compute[75234]:       <disksnapshot default='on' toggle='no'/>
Nov 29 03:01:54 user nova-compute[75234]:     </features>
Nov 29 03:01:54 user nova-compute[75234]:   </guest>
Nov 29 03:01:54 user nova-compute[75234]:   <guest>
Nov 29 03:01:54 user nova-compute[75234]:     <os_type>hvm</os_type>
Nov 29 03:01:54 user nova-compute[75234]:     <arch name='m68k'>
Nov 29 03:01:54 user nova-compute[75234]:       <wordsize>32</wordsize>
Nov 29 03:01:54 user nova-compute[75234]:       <emulator>/usr/bin/qemu-system-m68k</emulator>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>mcf5208evb</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>an5206</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>virt-6.0</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>q800</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>virt-6.2</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine canonical='virt-6.2' maxCpus='1'>virt</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>next-cube</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>virt-6.1</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <domain type='qemu'/>
Nov 29 03:01:54 user nova-compute[75234]:     </arch>
Nov 29 03:01:54 user nova-compute[75234]:     <features>
Nov 29 03:01:54 user nova-compute[75234]:       <cpuselection/>
Nov 29 03:01:54 user nova-compute[75234]:       <deviceboot/>
Nov 29 03:01:54 user nova-compute[75234]:       <disksnapshot default='on' toggle='no'/>
Nov 29 03:01:54 user nova-compute[75234]:     </features>
Nov 29 03:01:54 user nova-compute[75234]:   </guest>
Nov 29 03:01:54 user nova-compute[75234]:   <guest>
Nov 29 03:01:54 user nova-compute[75234]:     <os_type>hvm</os_type>
Nov 29 03:01:54 user nova-compute[75234]:     <arch name='microblaze'>
Nov 29 03:01:54 user nova-compute[75234]:       <wordsize>32</wordsize>
Nov 29 03:01:54 user nova-compute[75234]:       <emulator>/usr/bin/qemu-system-microblaze</emulator>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>petalogix-s3adsp1800</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>petalogix-ml605</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>xlnx-zynqmp-pmu</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <domain type='qemu'/>
Nov 29 03:01:54 user nova-compute[75234]:     </arch>
Nov 29 03:01:54 user nova-compute[75234]:     <features>
Nov 29 03:01:54 user nova-compute[75234]:       <cpuselection/>
Nov 29 03:01:54 user nova-compute[75234]:       <deviceboot/>
Nov 29 03:01:54 user nova-compute[75234]:       <disksnapshot default='on' toggle='no'/>
Nov 29 03:01:54 user nova-compute[75234]:     </features>
Nov 29 03:01:54 user nova-compute[75234]:   </guest>
Nov 29 03:01:54 user nova-compute[75234]:   <guest>
Nov 29 03:01:54 user nova-compute[75234]:     <os_type>hvm</os_type>
Nov 29 03:01:54 user nova-compute[75234]:     <arch name='microblazeel'>
Nov 29 03:01:54 user nova-compute[75234]:       <wordsize>32</wordsize>
Nov 29 03:01:54 user nova-compute[75234]:       <emulator>/usr/bin/qemu-system-microblazeel</emulator>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>petalogix-s3adsp1800</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>petalogix-ml605</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>xlnx-zynqmp-pmu</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <domain type='qemu'/>
Nov 29 03:01:54 user nova-compute[75234]:     </arch>
Nov 29 03:01:54 user nova-compute[75234]:     <features>
Nov 29 03:01:54 user nova-compute[75234]:       <cpuselection/>
Nov 29 03:01:54 user nova-compute[75234]:       <deviceboot/>
Nov 29 03:01:54 user nova-compute[75234]:       <disksnapshot default='on' toggle='no'/>
Nov 29 03:01:54 user nova-compute[75234]:     </features>
Nov 29 03:01:54 user nova-compute[75234]:   </guest>
Nov 29 03:01:54 user nova-compute[75234]:   <guest>
Nov 29 03:01:54 user nova-compute[75234]:     <os_type>hvm</os_type>
Nov 29 03:01:54 user nova-compute[75234]:     <arch name='mips'>
Nov 29 03:01:54 user nova-compute[75234]:       <wordsize>32</wordsize>
Nov 29 03:01:54 user nova-compute[75234]:       <emulator>/usr/bin/qemu-system-mips</emulator>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='16'>malta</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>mipssim</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <domain type='qemu'/>
Nov 29 03:01:54 user nova-compute[75234]:     </arch>
Nov 29 03:01:54 user nova-compute[75234]:     <features>
Nov 29 03:01:54 user nova-compute[75234]:       <cpuselection/>
Nov 29 03:01:54 user nova-compute[75234]:       <deviceboot/>
Nov 29 03:01:54 user nova-compute[75234]:       <disksnapshot default='on' toggle='no'/>
Nov 29 03:01:54 user nova-compute[75234]:     </features>
Nov 29 03:01:54 user nova-compute[75234]:   </guest>
Nov 29 03:01:54 user nova-compute[75234]:   <guest>
Nov 29 03:01:54 user nova-compute[75234]:     <os_type>hvm</os_type>
Nov 29 03:01:54 user nova-compute[75234]:     <arch name='mipsel'>
Nov 29 03:01:54 user nova-compute[75234]:       <wordsize>32</wordsize>
Nov 29 03:01:54 user nova-compute[75234]:       <emulator>/usr/bin/qemu-system-mipsel</emulator>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='16'>malta</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>mipssim</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <domain type='qemu'/>
Nov 29 03:01:54 user nova-compute[75234]:     </arch>
Nov 29 03:01:54 user nova-compute[75234]:     <features>
Nov 29 03:01:54 user nova-compute[75234]:       <cpuselection/>
Nov 29 03:01:54 user nova-compute[75234]:       <deviceboot/>
Nov 29 03:01:54 user nova-compute[75234]:       <disksnapshot default='on' toggle='no'/>
Nov 29 03:01:54 user nova-compute[75234]:     </features>
Nov 29 03:01:54 user nova-compute[75234]:   </guest>
Nov 29 03:01:54 user nova-compute[75234]:   <guest>
Nov 29 03:01:54 user nova-compute[75234]:     <os_type>hvm</os_type>
Nov 29 03:01:54 user nova-compute[75234]:     <arch name='mips64'>
Nov 29 03:01:54 user nova-compute[75234]:       <wordsize>64</wordsize>
Nov 29 03:01:54 user nova-compute[75234]:       <emulator>/usr/bin/qemu-system-mips64</emulator>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='16'>malta</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>mipssim</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>pica61</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>magnum</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <domain type='qemu'/>
Nov 29 03:01:54 user nova-compute[75234]:     </arch>
Nov 29 03:01:54 user nova-compute[75234]:     <features>
Nov 29 03:01:54 user nova-compute[75234]:       <cpuselection/>
Nov 29 03:01:54 user nova-compute[75234]:       <deviceboot/>
Nov 29 03:01:54 user nova-compute[75234]:       <disksnapshot default='on' toggle='no'/>
Nov 29 03:01:54 user nova-compute[75234]:     </features>
Nov 29 03:01:54 user nova-compute[75234]:   </guest>
Nov 29 03:01:54 user nova-compute[75234]:   <guest>
Nov 29 03:01:54 user nova-compute[75234]:     <os_type>hvm</os_type>
Nov 29 03:01:54 user nova-compute[75234]:     <arch name='mips64el'>
Nov 29 03:01:54 user nova-compute[75234]:       <wordsize>64</wordsize>
Nov 29 03:01:54 user nova-compute[75234]:       <emulator>/usr/bin/qemu-system-mips64el</emulator>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='16'>malta</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='16'>loongson3-virt</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>mipssim</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>pica61</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>magnum</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='16'>boston</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>fuloong2e</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <domain type='qemu'/>
Nov 29 03:01:54 user nova-compute[75234]:     </arch>
Nov 29 03:01:54 user nova-compute[75234]:     <features>
Nov 29 03:01:54 user nova-compute[75234]:       <cpuselection/>
Nov 29 03:01:54 user nova-compute[75234]:       <deviceboot/>
Nov 29 03:01:54 user nova-compute[75234]:       <disksnapshot default='on' toggle='no'/>
Nov 29 03:01:54 user nova-compute[75234]:     </features>
Nov 29 03:01:54 user nova-compute[75234]:   </guest>
Nov 29 03:01:54 user nova-compute[75234]:   <guest>
Nov 29 03:01:54 user nova-compute[75234]:     <os_type>hvm</os_type>
Nov 29 03:01:54 user nova-compute[75234]:     <arch name='ppc'>
Nov 29 03:01:54 user nova-compute[75234]:       <wordsize>32</wordsize>
Nov 29 03:01:54 user nova-compute[75234]:       <emulator>/usr/bin/qemu-system-ppc</emulator>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>g3beige</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>virtex-ml507</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>mac99</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='32'>ppce500</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>pegasos2</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>sam460ex</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>bamboo</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>40p</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>ref405ep</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='15'>mpc8544ds</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>taihu</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <domain type='qemu'/>
Nov 29 03:01:54 user nova-compute[75234]:     </arch>
Nov 29 03:01:54 user nova-compute[75234]:     <features>
Nov 29 03:01:54 user nova-compute[75234]:       <cpuselection/>
Nov 29 03:01:54 user nova-compute[75234]:       <deviceboot/>
Nov 29 03:01:54 user nova-compute[75234]:       <disksnapshot default='on' toggle='no'/>
Nov 29 03:01:54 user nova-compute[75234]:     </features>
Nov 29 03:01:54 user nova-compute[75234]:   </guest>
Nov 29 03:01:54 user nova-compute[75234]:   <guest>
Nov 29 03:01:54 user nova-compute[75234]:     <os_type>hvm</os_type>
Nov 29 03:01:54 user nova-compute[75234]:     <arch name='ppc64'>
Nov 29 03:01:54 user nova-compute[75234]:       <wordsize>64</wordsize>
Nov 29 03:01:54 user nova-compute[75234]:       <emulator>/usr/bin/qemu-system-ppc64</emulator>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='2147483647'>pseries-jammy</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine canonical='pseries-jammy' maxCpus='2147483647'>pseries</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='2048'>powernv9</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine canonical='powernv9' maxCpus='2048'>powernv</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>taihu</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='2147483647'>pseries-4.1</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='15'>mpc8544ds</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='2147483647'>pseries-6.1</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='2147483647'>pseries-2.5</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='2048'>powernv10</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='2147483647'>pseries-xenial</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='2147483647'>pseries-4.2</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='2147483647'>pseries-6.2</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='2147483647'>pseries-yakkety</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='2147483647'>pseries-2.6</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='32'>ppce500</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='2147483647'>pseries-bionic-sxxm</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='2147483647'>pseries-2.7</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='2147483647'>pseries-3.0</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='2147483647'>pseries-5.0</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>40p</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='2147483647'>pseries-2.8</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>pegasos2</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='2147483647'>pseries-hirsute</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='2147483647'>pseries-3.1</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='2147483647'>pseries-5.1</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='2147483647'>pseries-eoan</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='2147483647'>pseries-2.9</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='2147483647'>pseries-zesty</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>bamboo</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='2147483647'>pseries-groovy</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='2147483647'>pseries-focal</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>g3beige</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='2147483647'>pseries-5.2</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='2147483647'>pseries-disco</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='2147483647'>pseries-2.12-sxxm</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='2147483647'>pseries-2.10</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>virtex-ml507</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='2147483647'>pseries-2.11</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='2147483647'>pseries-2.1</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='2147483647'>pseries-cosmic</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='2147483647'>pseries-bionic</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='2147483647'>pseries-2.12</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='2147483647'>pseries-2.2</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>mac99</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='2147483647'>pseries-impish</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='2147483647'>pseries-artful</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>sam460ex</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>ref405ep</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='2147483647'>pseries-2.3</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='2048'>powernv8</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='2147483647'>pseries-4.0</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='2147483647'>pseries-6.0</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='2147483647'>pseries-2.4</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <domain type='qemu'/>
Nov 29 03:01:54 user nova-compute[75234]:     </arch>
Nov 29 03:01:54 user nova-compute[75234]:     <features>
Nov 29 03:01:54 user nova-compute[75234]:       <cpuselection/>
Nov 29 03:01:54 user nova-compute[75234]:       <deviceboot/>
Nov 29 03:01:54 user nova-compute[75234]:       <disksnapshot default='on' toggle='no'/>
Nov 29 03:01:54 user nova-compute[75234]:     </features>
Nov 29 03:01:54 user nova-compute[75234]:   </guest>
Nov 29 03:01:54 user nova-compute[75234]:   <guest>
Nov 29 03:01:54 user nova-compute[75234]:     <os_type>hvm</os_type>
Nov 29 03:01:54 user nova-compute[75234]:     <arch name='ppc64le'>
Nov 29 03:01:54 user nova-compute[75234]:       <wordsize>64</wordsize>
Nov 29 03:01:54 user nova-compute[75234]:       <emulator>/usr/bin/qemu-system-ppc64le</emulator>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='2147483647'>pseries-jammy</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine canonical='pseries-jammy' maxCpus='2147483647'>pseries</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='2048'>powernv9</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine canonical='powernv9' maxCpus='2048'>powernv</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>taihu</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='2147483647'>pseries-4.1</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='15'>mpc8544ds</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='2147483647'>pseries-6.1</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='2147483647'>pseries-2.5</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='2048'>powernv10</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='2147483647'>pseries-xenial</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='2147483647'>pseries-4.2</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='2147483647'>pseries-6.2</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='2147483647'>pseries-yakkety</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='2147483647'>pseries-2.6</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='32'>ppce500</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='2147483647'>pseries-bionic-sxxm</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='2147483647'>pseries-2.7</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='2147483647'>pseries-3.0</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='2147483647'>pseries-5.0</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>40p</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='2147483647'>pseries-2.8</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>pegasos2</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='2147483647'>pseries-hirsute</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='2147483647'>pseries-3.1</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='2147483647'>pseries-5.1</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='2147483647'>pseries-eoan</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='2147483647'>pseries-2.9</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='2147483647'>pseries-zesty</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>bamboo</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='2147483647'>pseries-groovy</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='2147483647'>pseries-focal</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>g3beige</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='2147483647'>pseries-5.2</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='2147483647'>pseries-disco</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='2147483647'>pseries-2.12-sxxm</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='2147483647'>pseries-2.10</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>virtex-ml507</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='2147483647'>pseries-2.11</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='2147483647'>pseries-2.1</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='2147483647'>pseries-cosmic</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='2147483647'>pseries-bionic</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='2147483647'>pseries-2.12</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='2147483647'>pseries-2.2</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>mac99</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='2147483647'>pseries-impish</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='2147483647'>pseries-artful</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>sam460ex</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>ref405ep</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='2147483647'>pseries-2.3</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='2048'>powernv8</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='2147483647'>pseries-4.0</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='2147483647'>pseries-6.0</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='2147483647'>pseries-2.4</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <domain type='qemu'/>
Nov 29 03:01:54 user nova-compute[75234]:     </arch>
Nov 29 03:01:54 user nova-compute[75234]:     <features>
Nov 29 03:01:54 user nova-compute[75234]:       <cpuselection/>
Nov 29 03:01:54 user nova-compute[75234]:       <deviceboot/>
Nov 29 03:01:54 user nova-compute[75234]:       <disksnapshot default='on' toggle='no'/>
Nov 29 03:01:54 user nova-compute[75234]:     </features>
Nov 29 03:01:54 user nova-compute[75234]:   </guest>
Nov 29 03:01:54 user nova-compute[75234]:   <guest>
Nov 29 03:01:54 user nova-compute[75234]:     <os_type>hvm</os_type>
Nov 29 03:01:54 user nova-compute[75234]:     <arch name='riscv32'>
Nov 29 03:01:54 user nova-compute[75234]:       <wordsize>32</wordsize>
Nov 29 03:01:54 user nova-compute[75234]:       <emulator>/usr/bin/qemu-system-riscv32</emulator>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='8'>spike</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>opentitan</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='5'>sifive_u</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>sifive_e</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='8'>virt</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <domain type='qemu'/>
Nov 29 03:01:54 user nova-compute[75234]:     </arch>
Nov 29 03:01:54 user nova-compute[75234]:     <features>
Nov 29 03:01:54 user nova-compute[75234]:       <cpuselection/>
Nov 29 03:01:54 user nova-compute[75234]:       <deviceboot/>
Nov 29 03:01:54 user nova-compute[75234]:       <disksnapshot default='on' toggle='no'/>
Nov 29 03:01:54 user nova-compute[75234]:     </features>
Nov 29 03:01:54 user nova-compute[75234]:   </guest>
Nov 29 03:01:54 user nova-compute[75234]:   <guest>
Nov 29 03:01:54 user nova-compute[75234]:     <os_type>hvm</os_type>
Nov 29 03:01:54 user nova-compute[75234]:     <arch name='riscv64'>
Nov 29 03:01:54 user nova-compute[75234]:       <wordsize>64</wordsize>
Nov 29 03:01:54 user nova-compute[75234]:       <emulator>/usr/bin/qemu-system-riscv64</emulator>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='8'>spike</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='5'>microchip-icicle-kit</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='5'>sifive_u</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>shakti_c</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>sifive_e</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='8'>virt</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <domain type='qemu'/>
Nov 29 03:01:54 user nova-compute[75234]:     </arch>
Nov 29 03:01:54 user nova-compute[75234]:     <features>
Nov 29 03:01:54 user nova-compute[75234]:       <cpuselection/>
Nov 29 03:01:54 user nova-compute[75234]:       <deviceboot/>
Nov 29 03:01:54 user nova-compute[75234]:       <disksnapshot default='on' toggle='no'/>
Nov 29 03:01:54 user nova-compute[75234]:     </features>
Nov 29 03:01:54 user nova-compute[75234]:   </guest>
Nov 29 03:01:54 user nova-compute[75234]:   <guest>
Nov 29 03:01:54 user nova-compute[75234]:     <os_type>hvm</os_type>
Nov 29 03:01:54 user nova-compute[75234]:     <arch name='s390x'>
Nov 29 03:01:54 user nova-compute[75234]:       <wordsize>64</wordsize>
Nov 29 03:01:54 user nova-compute[75234]:       <emulator>/usr/bin/qemu-system-s390x</emulator>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='248'>s390-ccw-virtio-jammy</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine canonical='s390-ccw-virtio-jammy' maxCpus='248'>s390-ccw-virtio</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='248'>s390-ccw-virtio-4.0</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='248'>s390-ccw-virtio-5.2</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='248'>s390-ccw-virtio-artful</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='248'>s390-ccw-virtio-3.1</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='248'>s390-ccw-virtio-groovy</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='248'>s390-ccw-virtio-hirsute</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='248'>s390-ccw-virtio-disco</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='248'>s390-ccw-virtio-2.12</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='248'>s390-ccw-virtio-2.6</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='248'>s390-ccw-virtio-yakkety</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='248'>s390-ccw-virtio-eoan</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='248'>s390-ccw-virtio-2.9</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='248'>s390-ccw-virtio-6.0</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='248'>s390-ccw-virtio-5.1</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='248'>s390-ccw-virtio-3.0</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='248'>s390-ccw-virtio-4.2</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='248'>s390-ccw-virtio-2.5</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='248'>s390-ccw-virtio-2.11</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='248'>s390-ccw-virtio-xenial</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='248'>s390-ccw-virtio-focal</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='248'>s390-ccw-virtio-2.8</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='248'>s390-ccw-virtio-impish</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='248'>s390-ccw-virtio-bionic</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='248'>s390-ccw-virtio-5.0</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='248'>s390-ccw-virtio-6.2</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='248'>s390-ccw-virtio-zesty</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='248'>s390-ccw-virtio-4.1</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='248'>s390-ccw-virtio-cosmic</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='248'>s390-ccw-virtio-2.4</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='248'>s390-ccw-virtio-2.10</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='248'>s390-ccw-virtio-2.7</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='248'>s390-ccw-virtio-6.1</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <domain type='qemu'/>
Nov 29 03:01:54 user nova-compute[75234]:     </arch>
Nov 29 03:01:54 user nova-compute[75234]:     <features>
Nov 29 03:01:54 user nova-compute[75234]:       <cpuselection/>
Nov 29 03:01:54 user nova-compute[75234]:       <deviceboot/>
Nov 29 03:01:54 user nova-compute[75234]:       <disksnapshot default='on' toggle='no'/>
Nov 29 03:01:54 user nova-compute[75234]:     </features>
Nov 29 03:01:54 user nova-compute[75234]:   </guest>
Nov 29 03:01:54 user nova-compute[75234]:   <guest>
Nov 29 03:01:54 user nova-compute[75234]:     <os_type>hvm</os_type>
Nov 29 03:01:54 user nova-compute[75234]:     <arch name='sh4'>
Nov 29 03:01:54 user nova-compute[75234]:       <wordsize>32</wordsize>
Nov 29 03:01:54 user nova-compute[75234]:       <emulator>/usr/bin/qemu-system-sh4</emulator>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>shix</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>r2d</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <domain type='qemu'/>
Nov 29 03:01:54 user nova-compute[75234]:     </arch>
Nov 29 03:01:54 user nova-compute[75234]:     <features>
Nov 29 03:01:54 user nova-compute[75234]:       <cpuselection/>
Nov 29 03:01:54 user nova-compute[75234]:       <deviceboot/>
Nov 29 03:01:54 user nova-compute[75234]:       <disksnapshot default='on' toggle='no'/>
Nov 29 03:01:54 user nova-compute[75234]:     </features>
Nov 29 03:01:54 user nova-compute[75234]:   </guest>
Nov 29 03:01:54 user nova-compute[75234]:   <guest>
Nov 29 03:01:54 user nova-compute[75234]:     <os_type>hvm</os_type>
Nov 29 03:01:54 user nova-compute[75234]:     <arch name='sh4eb'>
Nov 29 03:01:54 user nova-compute[75234]:       <wordsize>64</wordsize>
Nov 29 03:01:54 user nova-compute[75234]:       <emulator>/usr/bin/qemu-system-sh4eb</emulator>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>shix</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>r2d</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <domain type='qemu'/>
Nov 29 03:01:54 user nova-compute[75234]:     </arch>
Nov 29 03:01:54 user nova-compute[75234]:     <features>
Nov 29 03:01:54 user nova-compute[75234]:       <cpuselection/>
Nov 29 03:01:54 user nova-compute[75234]:       <deviceboot/>
Nov 29 03:01:54 user nova-compute[75234]:       <disksnapshot default='on' toggle='no'/>
Nov 29 03:01:54 user nova-compute[75234]:     </features>
Nov 29 03:01:54 user nova-compute[75234]:   </guest>
Nov 29 03:01:54 user nova-compute[75234]:   <guest>
Nov 29 03:01:54 user nova-compute[75234]:     <os_type>hvm</os_type>
Nov 29 03:01:54 user nova-compute[75234]:     <arch name='sparc'>
Nov 29 03:01:54 user nova-compute[75234]:       <wordsize>32</wordsize>
Nov 29 03:01:54 user nova-compute[75234]:       <emulator>/usr/bin/qemu-system-sparc</emulator>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>SS-5</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='4'>SS-20</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>LX</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>SPARCClassic</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>leon3_generic</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>SPARCbook</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>SS-4</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='4'>SS-600MP</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='4'>SS-10</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>Voyager</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <domain type='qemu'/>
Nov 29 03:01:54 user nova-compute[75234]:     </arch>
Nov 29 03:01:54 user nova-compute[75234]:     <features>
Nov 29 03:01:54 user nova-compute[75234]:       <cpuselection/>
Nov 29 03:01:54 user nova-compute[75234]:       <deviceboot/>
Nov 29 03:01:54 user nova-compute[75234]:       <disksnapshot default='on' toggle='no'/>
Nov 29 03:01:54 user nova-compute[75234]:     </features>
Nov 29 03:01:54 user nova-compute[75234]:   </guest>
Nov 29 03:01:54 user nova-compute[75234]:   <guest>
Nov 29 03:01:54 user nova-compute[75234]:     <os_type>hvm</os_type>
Nov 29 03:01:54 user nova-compute[75234]:     <arch name='sparc64'>
Nov 29 03:01:54 user nova-compute[75234]:       <wordsize>64</wordsize>
Nov 29 03:01:54 user nova-compute[75234]:       <emulator>/usr/bin/qemu-system-sparc64</emulator>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>sun4u</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>niagara</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>sun4v</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <domain type='qemu'/>
Nov 29 03:01:54 user nova-compute[75234]:     </arch>
Nov 29 03:01:54 user nova-compute[75234]:     <features>
Nov 29 03:01:54 user nova-compute[75234]:       <cpuselection/>
Nov 29 03:01:54 user nova-compute[75234]:       <deviceboot/>
Nov 29 03:01:54 user nova-compute[75234]:       <disksnapshot default='on' toggle='no'/>
Nov 29 03:01:54 user nova-compute[75234]:     </features>
Nov 29 03:01:54 user nova-compute[75234]:   </guest>
Nov 29 03:01:54 user nova-compute[75234]:   <guest>
Nov 29 03:01:54 user nova-compute[75234]:     <os_type>hvm</os_type>
Nov 29 03:01:54 user nova-compute[75234]:     <arch name='x86_64'>
Nov 29 03:01:54 user nova-compute[75234]:       <wordsize>64</wordsize>
Nov 29 03:01:54 user nova-compute[75234]:       <emulator>/usr/bin/qemu-system-x86_64</emulator>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='255'>pc-i440fx-jammy</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine canonical='pc-i440fx-jammy' maxCpus='255'>ubuntu</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='255'>pc-i440fx-impish-hpb</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='288'>pc-q35-5.2</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='255'>pc-i440fx-2.12</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='255'>pc-i440fx-2.0</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='255'>pc-i440fx-xenial</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='255'>pc-i440fx-6.2</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine canonical='pc-i440fx-6.2' maxCpus='255'>pc</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='288'>pc-q35-4.2</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='255'>pc-i440fx-2.5</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='255'>pc-i440fx-4.2</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='255'>pc-i440fx-hirsute</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='255'>pc-i440fx-focal</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='255'>pc-q35-xenial</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='255'>pc-i440fx-jammy-hpb</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='255'>pc-i440fx-5.2</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='255'>pc-i440fx-1.5</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='255'>pc-q35-2.7</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='288'>pc-q35-eoan-hpb</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='255'>pc-i440fx-zesty</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='255'>pc-i440fx-disco-hpb</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='288'>pc-q35-groovy</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='255'>pc-i440fx-groovy</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='288'>pc-q35-artful</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='255'>pc-i440fx-trusty</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='255'>pc-i440fx-2.2</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='288'>pc-q35-focal-hpb</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='255'>pc-i440fx-eoan-hpb</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='288'>pc-q35-bionic-hpb</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='255'>pc-i440fx-artful</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='255'>pc-i440fx-2.7</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='288'>pc-q35-6.1</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='255'>pc-i440fx-yakkety</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='255'>pc-q35-2.4</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='288'>pc-q35-cosmic-hpb</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='288'>pc-q35-2.10</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>x-remote</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='288'>pc-q35-5.1</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='255'>pc-i440fx-1.7</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='288'>pc-q35-2.9</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='255'>pc-i440fx-2.11</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='288'>pc-q35-3.1</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='255'>pc-i440fx-6.1</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='288'>pc-q35-4.1</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='288'>pc-q35-jammy</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine canonical='pc-q35-jammy' maxCpus='288'>ubuntu-q35</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='255'>pc-i440fx-2.4</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='255'>pc-i440fx-4.1</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='288'>pc-q35-eoan</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='288'>pc-q35-jammy-hpb</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='255'>pc-i440fx-5.1</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='255'>pc-i440fx-2.9</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='255'>pc-i440fx-bionic-hpb</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='1'>isapc</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='255'>pc-i440fx-1.4</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='288'>pc-q35-cosmic</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='255'>pc-q35-2.6</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='255'>pc-i440fx-3.1</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='288'>pc-q35-bionic</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='288'>pc-q35-disco-hpb</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='255'>pc-i440fx-cosmic</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='288'>pc-q35-2.12</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='255'>pc-i440fx-bionic</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='288'>pc-q35-groovy-hpb</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='288'>pc-q35-disco</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='255'>pc-i440fx-cosmic-hpb</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='255'>pc-i440fx-2.1</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='255'>pc-i440fx-wily</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='288'>pc-q35-impish</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='255'>pc-i440fx-2.6</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='288'>pc-q35-6.0</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='255'>pc-i440fx-impish</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='288'>pc-q35-impish-hpb</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='288'>pc-q35-hirsute</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='288'>pc-q35-4.0.1</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='288'>pc-q35-hirsute-hpb</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='255'>pc-i440fx-1.6</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='288'>pc-q35-5.0</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='288'>pc-q35-2.8</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='255'>pc-i440fx-2.10</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='288'>pc-q35-3.0</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='288'>pc-q35-zesty</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='288'>pc-q35-4.0</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='288'>pc-q35-focal</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='288'>microvm</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='255'>pc-i440fx-6.0</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='255'>pc-i440fx-2.3</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='255'>pc-i440fx-disco</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='255'>pc-i440fx-focal-hpb</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='255'>pc-i440fx-4.0</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='255'>pc-i440fx-groovy-hpb</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='255'>pc-i440fx-hirsute-hpb</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='255'>pc-i440fx-5.0</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='255'>pc-i440fx-2.8</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='288'>pc-q35-6.2</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine canonical='pc-q35-6.2' maxCpus='288'>q35</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='255'>pc-i440fx-eoan</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='255'>pc-q35-2.5</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='255'>pc-i440fx-3.0</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='255'>pc-q35-yakkety</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='288'>pc-q35-2.11</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <domain type='qemu'/>
Nov 29 03:01:54 user nova-compute[75234]:       <domain type='kvm'/>
Nov 29 03:01:54 user nova-compute[75234]:     </arch>
Nov 29 03:01:54 user nova-compute[75234]:     <features>
Nov 29 03:01:54 user nova-compute[75234]:       <acpi default='on' toggle='yes'/>
Nov 29 03:01:54 user nova-compute[75234]:       <apic default='on' toggle='no'/>
Nov 29 03:01:54 user nova-compute[75234]:       <cpuselection/>
Nov 29 03:01:54 user nova-compute[75234]:       <deviceboot/>
Nov 29 03:01:54 user nova-compute[75234]:       <disksnapshot default='on' toggle='no'/>
Nov 29 03:01:54 user nova-compute[75234]:     </features>
Nov 29 03:01:54 user nova-compute[75234]:   </guest>
Nov 29 03:01:54 user nova-compute[75234]:   <guest>
Nov 29 03:01:54 user nova-compute[75234]:     <os_type>hvm</os_type>
Nov 29 03:01:54 user nova-compute[75234]:     <arch name='xtensa'>
Nov 29 03:01:54 user nova-compute[75234]:       <wordsize>32</wordsize>
Nov 29 03:01:54 user nova-compute[75234]:       <emulator>/usr/bin/qemu-system-xtensa</emulator>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='4'>sim</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='32'>kc705</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='32'>ml605</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='32'>ml605-nommu</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='32'>virt</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='32'>lx60-nommu</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='32'>lx200</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='32'>lx200-nommu</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='32'>lx60</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='32'>kc705-nommu</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <domain type='qemu'/>
Nov 29 03:01:54 user nova-compute[75234]:     </arch>
Nov 29 03:01:54 user nova-compute[75234]:     <features>
Nov 29 03:01:54 user nova-compute[75234]:       <cpuselection/>
Nov 29 03:01:54 user nova-compute[75234]:       <deviceboot/>
Nov 29 03:01:54 user nova-compute[75234]:       <disksnapshot default='on' toggle='no'/>
Nov 29 03:01:54 user nova-compute[75234]:     </features>
Nov 29 03:01:54 user nova-compute[75234]:   </guest>
Nov 29 03:01:54 user nova-compute[75234]:   <guest>
Nov 29 03:01:54 user nova-compute[75234]:     <os_type>hvm</os_type>
Nov 29 03:01:54 user nova-compute[75234]:     <arch name='xtensaeb'>
Nov 29 03:01:54 user nova-compute[75234]:       <wordsize>32</wordsize>
Nov 29 03:01:54 user nova-compute[75234]:       <emulator>/usr/bin/qemu-system-xtensaeb</emulator>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='4'>sim</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='32'>kc705</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='32'>ml605</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='32'>ml605-nommu</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='32'>virt</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='32'>lx60-nommu</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='32'>lx200</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='32'>lx200-nommu</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='32'>lx60</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <machine maxCpus='32'>kc705-nommu</machine>
Nov 29 03:01:54 user nova-compute[75234]:       <domain type='qemu'/>
Nov 29 03:01:54 user nova-compute[75234]:     </arch>
Nov 29 03:01:54 user nova-compute[75234]:     <features>
Nov 29 03:01:54 user nova-compute[75234]:       <cpuselection/>
Nov 29 03:01:54 user nova-compute[75234]:       <deviceboot/>
Nov 29 03:01:54 user nova-compute[75234]:       <disksnapshot default='on' toggle='no'/>
Nov 29 03:01:54 user nova-compute[75234]:     </features>
Nov 29 03:01:54 user nova-compute[75234]:   </guest>
Nov 29 03:01:54 user nova-compute[75234]: </capabilities>
Nov 29 03:01:54 user nova-compute[75234]: 
Nov 29 03:01:54 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-ef65ec89-6c59-4f39-9adc-df4888ffd41b None None] Getting domain capabilities for alpha via machine types: {None} {{(pid=75234) _get_machine_types /opt/stack/nova/nova/virt/libvirt/host.py:954}}
Nov 29 03:01:54 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-ef65ec89-6c59-4f39-9adc-df4888ffd41b None None] Error from libvirt when retrieving domain capabilities for arch alpha / virt_type kvm / machine_type None: [Error Code 8]: invalid argument: KVM is not supported by '/usr/bin/qemu-system-alpha' on this host {{(pid=75234) _add_to_domain_capabilities /opt/stack/nova/nova/virt/libvirt/host.py:993}}
Nov 29 03:01:54 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-ef65ec89-6c59-4f39-9adc-df4888ffd41b None None] Getting domain capabilities for armv6l via machine types: {'virt', None} {{(pid=75234) _get_machine_types /opt/stack/nova/nova/virt/libvirt/host.py:954}}
Nov 29 03:01:54 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-ef65ec89-6c59-4f39-9adc-df4888ffd41b None None] Error from libvirt when retrieving domain capabilities for arch armv6l / virt_type kvm / machine_type virt: [Error Code 8]: invalid argument: KVM is not supported by '/usr/bin/qemu-system-arm' on this host {{(pid=75234) _add_to_domain_capabilities /opt/stack/nova/nova/virt/libvirt/host.py:993}}
Nov 29 03:01:54 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-ef65ec89-6c59-4f39-9adc-df4888ffd41b None None] Error from libvirt when retrieving domain capabilities for arch armv6l / virt_type kvm / machine_type None: [Error Code 8]: invalid argument: KVM is not supported by '/usr/bin/qemu-system-arm' on this host {{(pid=75234) _add_to_domain_capabilities /opt/stack/nova/nova/virt/libvirt/host.py:993}}
Nov 29 03:01:54 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-ef65ec89-6c59-4f39-9adc-df4888ffd41b None None] Getting domain capabilities for armv7l via machine types: {'virt'} {{(pid=75234) _get_machine_types /opt/stack/nova/nova/virt/libvirt/host.py:954}}
Nov 29 03:01:54 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-ef65ec89-6c59-4f39-9adc-df4888ffd41b None None] Error from libvirt when retrieving domain capabilities for arch armv7l / virt_type kvm / machine_type virt: [Error Code 8]: invalid argument: KVM is not supported by '/usr/bin/qemu-system-arm' on this host {{(pid=75234) _add_to_domain_capabilities /opt/stack/nova/nova/virt/libvirt/host.py:993}}
Nov 29 03:01:54 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-ef65ec89-6c59-4f39-9adc-df4888ffd41b None None] Getting domain capabilities for aarch64 via machine types: {'virt'} {{(pid=75234) _get_machine_types /opt/stack/nova/nova/virt/libvirt/host.py:954}}
Nov 29 03:01:54 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-ef65ec89-6c59-4f39-9adc-df4888ffd41b None None] Error from libvirt when retrieving domain capabilities for arch aarch64 / virt_type kvm / machine_type virt: [Error Code 8]: invalid argument: KVM is not supported by '/usr/bin/qemu-system-aarch64' on this host {{(pid=75234) _add_to_domain_capabilities /opt/stack/nova/nova/virt/libvirt/host.py:993}}
Nov 29 03:01:54 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-ef65ec89-6c59-4f39-9adc-df4888ffd41b None None] Getting domain capabilities for cris via machine types: {None} {{(pid=75234) _get_machine_types /opt/stack/nova/nova/virt/libvirt/host.py:954}}
Nov 29 03:01:54 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-ef65ec89-6c59-4f39-9adc-df4888ffd41b None None] Error from libvirt when retrieving domain capabilities for arch cris / virt_type kvm / machine_type None: [Error Code 8]: invalid argument: KVM is not supported by '/usr/bin/qemu-system-cris' on this host {{(pid=75234) _add_to_domain_capabilities /opt/stack/nova/nova/virt/libvirt/host.py:993}}
Nov 29 03:01:54 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-ef65ec89-6c59-4f39-9adc-df4888ffd41b None None] Getting domain capabilities for i686 via machine types: {'ubuntu-q35', 'ubuntu', 'q35', 'pc'} {{(pid=75234) _get_machine_types /opt/stack/nova/nova/virt/libvirt/host.py:954}}
Nov 29 03:01:54 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-ef65ec89-6c59-4f39-9adc-df4888ffd41b None None] Libvirt host hypervisor capabilities for arch=i686 and machine_type=ubuntu-q35:
Nov 29 03:01:54 user nova-compute[75234]: <domainCapabilities>
Nov 29 03:01:54 user nova-compute[75234]:   <path>/usr/bin/qemu-system-i386</path>
Nov 29 03:01:54 user nova-compute[75234]:   <domain>kvm</domain>
Nov 29 03:01:54 user nova-compute[75234]:   <machine>pc-q35-jammy</machine>
Nov 29 03:01:54 user nova-compute[75234]:   <arch>i686</arch>
Nov 29 03:01:54 user nova-compute[75234]:   <vcpu max='288'/>
Nov 29 03:01:54 user nova-compute[75234]:   <iothreads supported='yes'/>
Nov 29 03:01:54 user nova-compute[75234]:   <os supported='yes'>
Nov 29 03:01:54 user nova-compute[75234]:     <enum name='firmware'/>
Nov 29 03:01:54 user nova-compute[75234]:     <loader supported='yes'>
Nov 29 03:01:54 user nova-compute[75234]:       <value>/usr/share/OVMF/OVMF_CODE.fd</value>
Nov 29 03:01:54 user nova-compute[75234]:       <value>/usr/share/OVMF/OVMF_CODE.secboot.fd</value>
Nov 29 03:01:54 user nova-compute[75234]:       <value>/usr/share/AAVMF/AAVMF_CODE.fd</value>
Nov 29 03:01:54 user nova-compute[75234]:       <value>/usr/share/AAVMF/AAVMF32_CODE.fd</value>
Nov 29 03:01:54 user nova-compute[75234]:       <value>/usr/share/OVMF/OVMF_CODE.ms.fd</value>
Nov 29 03:01:54 user nova-compute[75234]:       <enum name='type'>
Nov 29 03:01:54 user nova-compute[75234]:         <value>rom</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>pflash</value>
Nov 29 03:01:54 user nova-compute[75234]:       </enum>
Nov 29 03:01:54 user nova-compute[75234]:       <enum name='readonly'>
Nov 29 03:01:54 user nova-compute[75234]:         <value>yes</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>no</value>
Nov 29 03:01:54 user nova-compute[75234]:       </enum>
Nov 29 03:01:54 user nova-compute[75234]:       <enum name='secure'>
Nov 29 03:01:54 user nova-compute[75234]:         <value>no</value>
Nov 29 03:01:54 user nova-compute[75234]:       </enum>
Nov 29 03:01:54 user nova-compute[75234]:     </loader>
Nov 29 03:01:54 user nova-compute[75234]:   </os>
Nov 29 03:01:54 user nova-compute[75234]:   <cpu>
Nov 29 03:01:54 user nova-compute[75234]:     <mode name='host-passthrough' supported='yes'>
Nov 29 03:01:54 user nova-compute[75234]:       <enum name='hostPassthroughMigratable'>
Nov 29 03:01:54 user nova-compute[75234]:         <value>off</value>
Nov 29 03:01:54 user nova-compute[75234]:       </enum>
Nov 29 03:01:54 user nova-compute[75234]:     </mode>
Nov 29 03:01:54 user nova-compute[75234]:     <mode name='maximum' supported='yes'>
Nov 29 03:01:54 user nova-compute[75234]:       <enum name='maximumMigratable'>
Nov 29 03:01:54 user nova-compute[75234]:         <value>on</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>off</value>
Nov 29 03:01:54 user nova-compute[75234]:       </enum>
Nov 29 03:01:54 user nova-compute[75234]:     </mode>
Nov 29 03:01:54 user nova-compute[75234]:     <mode name='host-model' supported='yes'>
Nov 29 03:01:54 user nova-compute[75234]:       <model fallback='forbid'>IvyBridge-IBRS</model>
Nov 29 03:01:54 user nova-compute[75234]:       <vendor>Intel</vendor>
Nov 29 03:01:54 user nova-compute[75234]:       <feature policy='require' name='ss'/>
Nov 29 03:01:54 user nova-compute[75234]:       <feature policy='require' name='vmx'/>
Nov 29 03:01:54 user nova-compute[75234]:       <feature policy='require' name='pcid'/>
Nov 29 03:01:54 user nova-compute[75234]:       <feature policy='require' name='hypervisor'/>
Nov 29 03:01:54 user nova-compute[75234]:       <feature policy='require' name='arat'/>
Nov 29 03:01:54 user nova-compute[75234]:       <feature policy='require' name='tsc_adjust'/>
Nov 29 03:01:54 user nova-compute[75234]:       <feature policy='require' name='umip'/>
Nov 29 03:01:54 user nova-compute[75234]:       <feature policy='require' name='md-clear'/>
Nov 29 03:01:54 user nova-compute[75234]:       <feature policy='require' name='stibp'/>
Nov 29 03:01:54 user nova-compute[75234]:       <feature policy='require' name='arch-capabilities'/>
Nov 29 03:01:54 user nova-compute[75234]:       <feature policy='require' name='ssbd'/>
Nov 29 03:01:54 user nova-compute[75234]:       <feature policy='require' name='ibpb'/>
Nov 29 03:01:54 user nova-compute[75234]:       <feature policy='require' name='ibrs'/>
Nov 29 03:01:54 user nova-compute[75234]:       <feature policy='require' name='amd-stibp'/>
Nov 29 03:01:54 user nova-compute[75234]:       <feature policy='require' name='amd-ssbd'/>
Nov 29 03:01:54 user nova-compute[75234]:       <feature policy='disable' name='erms'/>
Nov 29 03:01:54 user nova-compute[75234]:       <feature policy='disable' name='xsaveopt'/>
Nov 29 03:01:54 user nova-compute[75234]:       <feature policy='disable' name='lm'/>
Nov 29 03:01:54 user nova-compute[75234]:     </mode>
Nov 29 03:01:54 user nova-compute[75234]:     <mode name='custom' supported='yes'>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>qemu64</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='yes'>qemu32</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>phenom</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='yes'>pentium3</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='yes'>pentium2</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='yes'>pentium</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>n270</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>kvm64</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='yes'>kvm32</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='yes'>coreduo</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>core2duo</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>athlon</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Westmere-IBRS</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Westmere</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Snowridge</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Skylake-Server-noTSX-IBRS</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Skylake-Server-IBRS</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Skylake-Server</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Skylake-Client-noTSX-IBRS</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Skylake-Client-IBRS</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Skylake-Client</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>SandyBridge-IBRS</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>SandyBridge</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Penryn</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Opteron_G5</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Opteron_G4</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Opteron_G3</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Opteron_G2</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Opteron_G1</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Nehalem-IBRS</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Nehalem</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>IvyBridge-IBRS</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>IvyBridge</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Icelake-Server-noTSX</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Icelake-Server</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no' deprecated='yes'>Icelake-Client-noTSX</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no' deprecated='yes'>Icelake-Client</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Haswell-noTSX-IBRS</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Haswell-noTSX</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Haswell-IBRS</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Haswell</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>EPYC-Rome</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>EPYC-Milan</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>EPYC-IBPB</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>EPYC</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Dhyana</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Cooperlake</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Conroe</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Cascadelake-Server-noTSX</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Cascadelake-Server</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Broadwell-noTSX-IBRS</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Broadwell-noTSX</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Broadwell-IBRS</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Broadwell</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='yes'>486</model>
Nov 29 03:01:54 user nova-compute[75234]:     </mode>
Nov 29 03:01:54 user nova-compute[75234]:   </cpu>
Nov 29 03:01:54 user nova-compute[75234]:   <memoryBacking supported='yes'>
Nov 29 03:01:54 user nova-compute[75234]:     <enum name='sourceType'>
Nov 29 03:01:54 user nova-compute[75234]:       <value>file</value>
Nov 29 03:01:54 user nova-compute[75234]:       <value>anonymous</value>
Nov 29 03:01:54 user nova-compute[75234]:       <value>memfd</value>
Nov 29 03:01:54 user nova-compute[75234]:     </enum>
Nov 29 03:01:54 user nova-compute[75234]:   </memoryBacking>
Nov 29 03:01:54 user nova-compute[75234]:   <devices>
Nov 29 03:01:54 user nova-compute[75234]:     <disk supported='yes'>
Nov 29 03:01:54 user nova-compute[75234]:       <enum name='diskDevice'>
Nov 29 03:01:54 user nova-compute[75234]:         <value>disk</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>cdrom</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>floppy</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>lun</value>
Nov 29 03:01:54 user nova-compute[75234]:       </enum>
Nov 29 03:01:54 user nova-compute[75234]:       <enum name='bus'>
Nov 29 03:01:54 user nova-compute[75234]:         <value>fdc</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>scsi</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>virtio</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>usb</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>sata</value>
Nov 29 03:01:54 user nova-compute[75234]:       </enum>
Nov 29 03:01:54 user nova-compute[75234]:       <enum name='model'>
Nov 29 03:01:54 user nova-compute[75234]:         <value>virtio</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>virtio-transitional</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>virtio-non-transitional</value>
Nov 29 03:01:54 user nova-compute[75234]:       </enum>
Nov 29 03:01:54 user nova-compute[75234]:     </disk>
Nov 29 03:01:54 user nova-compute[75234]:     <graphics supported='yes'>
Nov 29 03:01:54 user nova-compute[75234]:       <enum name='type'>
Nov 29 03:01:54 user nova-compute[75234]:         <value>sdl</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>vnc</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>spice</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>egl-headless</value>
Nov 29 03:01:54 user nova-compute[75234]:       </enum>
Nov 29 03:01:54 user nova-compute[75234]:     </graphics>
Nov 29 03:01:54 user nova-compute[75234]:     <video supported='yes'>
Nov 29 03:01:54 user nova-compute[75234]:       <enum name='modelType'>
Nov 29 03:01:54 user nova-compute[75234]:         <value>vga</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>cirrus</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>vmvga</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>qxl</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>virtio</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>none</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>bochs</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>ramfb</value>
Nov 29 03:01:54 user nova-compute[75234]:       </enum>
Nov 29 03:01:54 user nova-compute[75234]:     </video>
Nov 29 03:01:54 user nova-compute[75234]:     <hostdev supported='yes'>
Nov 29 03:01:54 user nova-compute[75234]:       <enum name='mode'>
Nov 29 03:01:54 user nova-compute[75234]:         <value>subsystem</value>
Nov 29 03:01:54 user nova-compute[75234]:       </enum>
Nov 29 03:01:54 user nova-compute[75234]:       <enum name='startupPolicy'>
Nov 29 03:01:54 user nova-compute[75234]:         <value>default</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>mandatory</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>requisite</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>optional</value>
Nov 29 03:01:54 user nova-compute[75234]:       </enum>
Nov 29 03:01:54 user nova-compute[75234]:       <enum name='subsysType'>
Nov 29 03:01:54 user nova-compute[75234]:         <value>usb</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>pci</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>scsi</value>
Nov 29 03:01:54 user nova-compute[75234]:       </enum>
Nov 29 03:01:54 user nova-compute[75234]:       <enum name='capsType'/>
Nov 29 03:01:54 user nova-compute[75234]:       <enum name='pciBackend'/>
Nov 29 03:01:54 user nova-compute[75234]:     </hostdev>
Nov 29 03:01:54 user nova-compute[75234]:     <rng supported='yes'>
Nov 29 03:01:54 user nova-compute[75234]:       <enum name='model'>
Nov 29 03:01:54 user nova-compute[75234]:         <value>virtio</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>virtio-transitional</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>virtio-non-transitional</value>
Nov 29 03:01:54 user nova-compute[75234]:       </enum>
Nov 29 03:01:54 user nova-compute[75234]:       <enum name='backendModel'>
Nov 29 03:01:54 user nova-compute[75234]:         <value>random</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>egd</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>builtin</value>
Nov 29 03:01:54 user nova-compute[75234]:       </enum>
Nov 29 03:01:54 user nova-compute[75234]:     </rng>
Nov 29 03:01:54 user nova-compute[75234]:     <filesystem supported='yes'>
Nov 29 03:01:54 user nova-compute[75234]:       <enum name='driverType'>
Nov 29 03:01:54 user nova-compute[75234]:         <value>path</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>handle</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>virtiofs</value>
Nov 29 03:01:54 user nova-compute[75234]:       </enum>
Nov 29 03:01:54 user nova-compute[75234]:     </filesystem>
Nov 29 03:01:54 user nova-compute[75234]:     <tpm supported='yes'>
Nov 29 03:01:54 user nova-compute[75234]:       <enum name='model'>
Nov 29 03:01:54 user nova-compute[75234]:         <value>tpm-tis</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>tpm-crb</value>
Nov 29 03:01:54 user nova-compute[75234]:       </enum>
Nov 29 03:01:54 user nova-compute[75234]:       <enum name='backendModel'>
Nov 29 03:01:54 user nova-compute[75234]:         <value>passthrough</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>emulator</value>
Nov 29 03:01:54 user nova-compute[75234]:       </enum>
Nov 29 03:01:54 user nova-compute[75234]:     </tpm>
Nov 29 03:01:54 user nova-compute[75234]:   </devices>
Nov 29 03:01:54 user nova-compute[75234]:   <features>
Nov 29 03:01:54 user nova-compute[75234]:     <gic supported='no'/>
Nov 29 03:01:54 user nova-compute[75234]:     <vmcoreinfo supported='yes'/>
Nov 29 03:01:54 user nova-compute[75234]:     <genid supported='yes'/>
Nov 29 03:01:54 user nova-compute[75234]:     <backingStoreInput supported='yes'/>
Nov 29 03:01:54 user nova-compute[75234]:     <backup supported='yes'/>
Nov 29 03:01:54 user nova-compute[75234]:     <sev supported='no'/>
Nov 29 03:01:54 user nova-compute[75234]:   </features>
Nov 29 03:01:54 user nova-compute[75234]: </domainCapabilities>
Nov 29 03:01:54 user nova-compute[75234]:  {{(pid=75234) _get_domain_capabilities /opt/stack/nova/nova/virt/libvirt/host.py:1039}}
Nov 29 03:01:54 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-ef65ec89-6c59-4f39-9adc-df4888ffd41b None None] Libvirt host hypervisor capabilities for arch=i686 and machine_type=ubuntu:
Nov 29 03:01:54 user nova-compute[75234]: <domainCapabilities>
Nov 29 03:01:54 user nova-compute[75234]:   <path>/usr/bin/qemu-system-i386</path>
Nov 29 03:01:54 user nova-compute[75234]:   <domain>kvm</domain>
Nov 29 03:01:54 user nova-compute[75234]:   <machine>pc-i440fx-jammy</machine>
Nov 29 03:01:54 user nova-compute[75234]:   <arch>i686</arch>
Nov 29 03:01:54 user nova-compute[75234]:   <vcpu max='255'/>
Nov 29 03:01:54 user nova-compute[75234]:   <iothreads supported='yes'/>
Nov 29 03:01:54 user nova-compute[75234]:   <os supported='yes'>
Nov 29 03:01:54 user nova-compute[75234]:     <enum name='firmware'/>
Nov 29 03:01:54 user nova-compute[75234]:     <loader supported='yes'>
Nov 29 03:01:54 user nova-compute[75234]:       <value>/usr/share/OVMF/OVMF_CODE.fd</value>
Nov 29 03:01:54 user nova-compute[75234]:       <value>/usr/share/OVMF/OVMF_CODE.secboot.fd</value>
Nov 29 03:01:54 user nova-compute[75234]:       <value>/usr/share/AAVMF/AAVMF_CODE.fd</value>
Nov 29 03:01:54 user nova-compute[75234]:       <value>/usr/share/AAVMF/AAVMF32_CODE.fd</value>
Nov 29 03:01:54 user nova-compute[75234]:       <value>/usr/share/OVMF/OVMF_CODE.ms.fd</value>
Nov 29 03:01:54 user nova-compute[75234]:       <enum name='type'>
Nov 29 03:01:54 user nova-compute[75234]:         <value>rom</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>pflash</value>
Nov 29 03:01:54 user nova-compute[75234]:       </enum>
Nov 29 03:01:54 user nova-compute[75234]:       <enum name='readonly'>
Nov 29 03:01:54 user nova-compute[75234]:         <value>yes</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>no</value>
Nov 29 03:01:54 user nova-compute[75234]:       </enum>
Nov 29 03:01:54 user nova-compute[75234]:       <enum name='secure'>
Nov 29 03:01:54 user nova-compute[75234]:         <value>no</value>
Nov 29 03:01:54 user nova-compute[75234]:       </enum>
Nov 29 03:01:54 user nova-compute[75234]:     </loader>
Nov 29 03:01:54 user nova-compute[75234]:   </os>
Nov 29 03:01:54 user nova-compute[75234]:   <cpu>
Nov 29 03:01:54 user nova-compute[75234]:     <mode name='host-passthrough' supported='yes'>
Nov 29 03:01:54 user nova-compute[75234]:       <enum name='hostPassthroughMigratable'>
Nov 29 03:01:54 user nova-compute[75234]:         <value>off</value>
Nov 29 03:01:54 user nova-compute[75234]:       </enum>
Nov 29 03:01:54 user nova-compute[75234]:     </mode>
Nov 29 03:01:54 user nova-compute[75234]:     <mode name='maximum' supported='yes'>
Nov 29 03:01:54 user nova-compute[75234]:       <enum name='maximumMigratable'>
Nov 29 03:01:54 user nova-compute[75234]:         <value>on</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>off</value>
Nov 29 03:01:54 user nova-compute[75234]:       </enum>
Nov 29 03:01:54 user nova-compute[75234]:     </mode>
Nov 29 03:01:54 user nova-compute[75234]:     <mode name='host-model' supported='yes'>
Nov 29 03:01:54 user nova-compute[75234]:       <model fallback='forbid'>IvyBridge-IBRS</model>
Nov 29 03:01:54 user nova-compute[75234]:       <vendor>Intel</vendor>
Nov 29 03:01:54 user nova-compute[75234]:       <feature policy='require' name='ss'/>
Nov 29 03:01:54 user nova-compute[75234]:       <feature policy='require' name='vmx'/>
Nov 29 03:01:54 user nova-compute[75234]:       <feature policy='require' name='pcid'/>
Nov 29 03:01:54 user nova-compute[75234]:       <feature policy='require' name='hypervisor'/>
Nov 29 03:01:54 user nova-compute[75234]:       <feature policy='require' name='arat'/>
Nov 29 03:01:54 user nova-compute[75234]:       <feature policy='require' name='tsc_adjust'/>
Nov 29 03:01:54 user nova-compute[75234]:       <feature policy='require' name='umip'/>
Nov 29 03:01:54 user nova-compute[75234]:       <feature policy='require' name='md-clear'/>
Nov 29 03:01:54 user nova-compute[75234]:       <feature policy='require' name='stibp'/>
Nov 29 03:01:54 user nova-compute[75234]:       <feature policy='require' name='arch-capabilities'/>
Nov 29 03:01:54 user nova-compute[75234]:       <feature policy='require' name='ssbd'/>
Nov 29 03:01:54 user nova-compute[75234]:       <feature policy='require' name='ibpb'/>
Nov 29 03:01:54 user nova-compute[75234]:       <feature policy='require' name='ibrs'/>
Nov 29 03:01:54 user nova-compute[75234]:       <feature policy='require' name='amd-stibp'/>
Nov 29 03:01:54 user nova-compute[75234]:       <feature policy='require' name='amd-ssbd'/>
Nov 29 03:01:54 user nova-compute[75234]:       <feature policy='disable' name='erms'/>
Nov 29 03:01:54 user nova-compute[75234]:       <feature policy='disable' name='xsaveopt'/>
Nov 29 03:01:54 user nova-compute[75234]:       <feature policy='disable' name='lm'/>
Nov 29 03:01:54 user nova-compute[75234]:     </mode>
Nov 29 03:01:54 user nova-compute[75234]:     <mode name='custom' supported='yes'>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>qemu64</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='yes'>qemu32</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>phenom</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='yes'>pentium3</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='yes'>pentium2</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='yes'>pentium</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>n270</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>kvm64</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='yes'>kvm32</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='yes'>coreduo</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>core2duo</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>athlon</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Westmere-IBRS</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Westmere</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Snowridge</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Skylake-Server-noTSX-IBRS</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Skylake-Server-IBRS</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Skylake-Server</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Skylake-Client-noTSX-IBRS</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Skylake-Client-IBRS</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Skylake-Client</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>SandyBridge-IBRS</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>SandyBridge</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Penryn</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Opteron_G5</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Opteron_G4</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Opteron_G3</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Opteron_G2</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Opteron_G1</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Nehalem-IBRS</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Nehalem</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>IvyBridge-IBRS</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>IvyBridge</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Icelake-Server-noTSX</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Icelake-Server</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no' deprecated='yes'>Icelake-Client-noTSX</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no' deprecated='yes'>Icelake-Client</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Haswell-noTSX-IBRS</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Haswell-noTSX</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Haswell-IBRS</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Haswell</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>EPYC-Rome</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>EPYC-Milan</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>EPYC-IBPB</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>EPYC</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Dhyana</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Cooperlake</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Conroe</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Cascadelake-Server-noTSX</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Cascadelake-Server</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Broadwell-noTSX-IBRS</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Broadwell-noTSX</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Broadwell-IBRS</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Broadwell</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='yes'>486</model>
Nov 29 03:01:54 user nova-compute[75234]:     </mode>
Nov 29 03:01:54 user nova-compute[75234]:   </cpu>
Nov 29 03:01:54 user nova-compute[75234]:   <memoryBacking supported='yes'>
Nov 29 03:01:54 user nova-compute[75234]:     <enum name='sourceType'>
Nov 29 03:01:54 user nova-compute[75234]:       <value>file</value>
Nov 29 03:01:54 user nova-compute[75234]:       <value>anonymous</value>
Nov 29 03:01:54 user nova-compute[75234]:       <value>memfd</value>
Nov 29 03:01:54 user nova-compute[75234]:     </enum>
Nov 29 03:01:54 user nova-compute[75234]:   </memoryBacking>
Nov 29 03:01:54 user nova-compute[75234]:   <devices>
Nov 29 03:01:54 user nova-compute[75234]:     <disk supported='yes'>
Nov 29 03:01:54 user nova-compute[75234]:       <enum name='diskDevice'>
Nov 29 03:01:54 user nova-compute[75234]:         <value>disk</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>cdrom</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>floppy</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>lun</value>
Nov 29 03:01:54 user nova-compute[75234]:       </enum>
Nov 29 03:01:54 user nova-compute[75234]:       <enum name='bus'>
Nov 29 03:01:54 user nova-compute[75234]:         <value>ide</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>fdc</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>scsi</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>virtio</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>usb</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>sata</value>
Nov 29 03:01:54 user nova-compute[75234]:       </enum>
Nov 29 03:01:54 user nova-compute[75234]:       <enum name='model'>
Nov 29 03:01:54 user nova-compute[75234]:         <value>virtio</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>virtio-transitional</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>virtio-non-transitional</value>
Nov 29 03:01:54 user nova-compute[75234]:       </enum>
Nov 29 03:01:54 user nova-compute[75234]:     </disk>
Nov 29 03:01:54 user nova-compute[75234]:     <graphics supported='yes'>
Nov 29 03:01:54 user nova-compute[75234]:       <enum name='type'>
Nov 29 03:01:54 user nova-compute[75234]:         <value>sdl</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>vnc</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>spice</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>egl-headless</value>
Nov 29 03:01:54 user nova-compute[75234]:       </enum>
Nov 29 03:01:54 user nova-compute[75234]:     </graphics>
Nov 29 03:01:54 user nova-compute[75234]:     <video supported='yes'>
Nov 29 03:01:54 user nova-compute[75234]:       <enum name='modelType'>
Nov 29 03:01:54 user nova-compute[75234]:         <value>vga</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>cirrus</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>vmvga</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>qxl</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>virtio</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>none</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>bochs</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>ramfb</value>
Nov 29 03:01:54 user nova-compute[75234]:       </enum>
Nov 29 03:01:54 user nova-compute[75234]:     </video>
Nov 29 03:01:54 user nova-compute[75234]:     <hostdev supported='yes'>
Nov 29 03:01:54 user nova-compute[75234]:       <enum name='mode'>
Nov 29 03:01:54 user nova-compute[75234]:         <value>subsystem</value>
Nov 29 03:01:54 user nova-compute[75234]:       </enum>
Nov 29 03:01:54 user nova-compute[75234]:       <enum name='startupPolicy'>
Nov 29 03:01:54 user nova-compute[75234]:         <value>default</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>mandatory</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>requisite</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>optional</value>
Nov 29 03:01:54 user nova-compute[75234]:       </enum>
Nov 29 03:01:54 user nova-compute[75234]:       <enum name='subsysType'>
Nov 29 03:01:54 user nova-compute[75234]:         <value>usb</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>pci</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>scsi</value>
Nov 29 03:01:54 user nova-compute[75234]:       </enum>
Nov 29 03:01:54 user nova-compute[75234]:       <enum name='capsType'/>
Nov 29 03:01:54 user nova-compute[75234]:       <enum name='pciBackend'/>
Nov 29 03:01:54 user nova-compute[75234]:     </hostdev>
Nov 29 03:01:54 user nova-compute[75234]:     <rng supported='yes'>
Nov 29 03:01:54 user nova-compute[75234]:       <enum name='model'>
Nov 29 03:01:54 user nova-compute[75234]:         <value>virtio</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>virtio-transitional</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>virtio-non-transitional</value>
Nov 29 03:01:54 user nova-compute[75234]:       </enum>
Nov 29 03:01:54 user nova-compute[75234]:       <enum name='backendModel'>
Nov 29 03:01:54 user nova-compute[75234]:         <value>random</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>egd</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>builtin</value>
Nov 29 03:01:54 user nova-compute[75234]:       </enum>
Nov 29 03:01:54 user nova-compute[75234]:     </rng>
Nov 29 03:01:54 user nova-compute[75234]:     <filesystem supported='yes'>
Nov 29 03:01:54 user nova-compute[75234]:       <enum name='driverType'>
Nov 29 03:01:54 user nova-compute[75234]:         <value>path</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>handle</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>virtiofs</value>
Nov 29 03:01:54 user nova-compute[75234]:       </enum>
Nov 29 03:01:54 user nova-compute[75234]:     </filesystem>
Nov 29 03:01:54 user nova-compute[75234]:     <tpm supported='yes'>
Nov 29 03:01:54 user nova-compute[75234]:       <enum name='model'>
Nov 29 03:01:54 user nova-compute[75234]:         <value>tpm-tis</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>tpm-crb</value>
Nov 29 03:01:54 user nova-compute[75234]:       </enum>
Nov 29 03:01:54 user nova-compute[75234]:       <enum name='backendModel'>
Nov 29 03:01:54 user nova-compute[75234]:         <value>passthrough</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>emulator</value>
Nov 29 03:01:54 user nova-compute[75234]:       </enum>
Nov 29 03:01:54 user nova-compute[75234]:     </tpm>
Nov 29 03:01:54 user nova-compute[75234]:   </devices>
Nov 29 03:01:54 user nova-compute[75234]:   <features>
Nov 29 03:01:54 user nova-compute[75234]:     <gic supported='no'/>
Nov 29 03:01:54 user nova-compute[75234]:     <vmcoreinfo supported='yes'/>
Nov 29 03:01:54 user nova-compute[75234]:     <genid supported='yes'/>
Nov 29 03:01:54 user nova-compute[75234]:     <backingStoreInput supported='yes'/>
Nov 29 03:01:54 user nova-compute[75234]:     <backup supported='yes'/>
Nov 29 03:01:54 user nova-compute[75234]:     <sev supported='no'/>
Nov 29 03:01:54 user nova-compute[75234]:   </features>
Nov 29 03:01:54 user nova-compute[75234]: </domainCapabilities>
Nov 29 03:01:54 user nova-compute[75234]:  {{(pid=75234) _get_domain_capabilities /opt/stack/nova/nova/virt/libvirt/host.py:1039}}
Nov 29 03:01:54 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-ef65ec89-6c59-4f39-9adc-df4888ffd41b None None] Libvirt host hypervisor capabilities for arch=i686 and machine_type=q35:
Nov 29 03:01:54 user nova-compute[75234]: <domainCapabilities>
Nov 29 03:01:54 user nova-compute[75234]:   <path>/usr/bin/qemu-system-i386</path>
Nov 29 03:01:54 user nova-compute[75234]:   <domain>kvm</domain>
Nov 29 03:01:54 user nova-compute[75234]:   <machine>pc-q35-6.2</machine>
Nov 29 03:01:54 user nova-compute[75234]:   <arch>i686</arch>
Nov 29 03:01:54 user nova-compute[75234]:   <vcpu max='288'/>
Nov 29 03:01:54 user nova-compute[75234]:   <iothreads supported='yes'/>
Nov 29 03:01:54 user nova-compute[75234]:   <os supported='yes'>
Nov 29 03:01:54 user nova-compute[75234]:     <enum name='firmware'/>
Nov 29 03:01:54 user nova-compute[75234]:     <loader supported='yes'>
Nov 29 03:01:54 user nova-compute[75234]:       <value>/usr/share/OVMF/OVMF_CODE.fd</value>
Nov 29 03:01:54 user nova-compute[75234]:       <value>/usr/share/OVMF/OVMF_CODE.secboot.fd</value>
Nov 29 03:01:54 user nova-compute[75234]:       <value>/usr/share/AAVMF/AAVMF_CODE.fd</value>
Nov 29 03:01:54 user nova-compute[75234]:       <value>/usr/share/AAVMF/AAVMF32_CODE.fd</value>
Nov 29 03:01:54 user nova-compute[75234]:       <value>/usr/share/OVMF/OVMF_CODE.ms.fd</value>
Nov 29 03:01:54 user nova-compute[75234]:       <enum name='type'>
Nov 29 03:01:54 user nova-compute[75234]:         <value>rom</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>pflash</value>
Nov 29 03:01:54 user nova-compute[75234]:       </enum>
Nov 29 03:01:54 user nova-compute[75234]:       <enum name='readonly'>
Nov 29 03:01:54 user nova-compute[75234]:         <value>yes</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>no</value>
Nov 29 03:01:54 user nova-compute[75234]:       </enum>
Nov 29 03:01:54 user nova-compute[75234]:       <enum name='secure'>
Nov 29 03:01:54 user nova-compute[75234]:         <value>no</value>
Nov 29 03:01:54 user nova-compute[75234]:       </enum>
Nov 29 03:01:54 user nova-compute[75234]:     </loader>
Nov 29 03:01:54 user nova-compute[75234]:   </os>
Nov 29 03:01:54 user nova-compute[75234]:   <cpu>
Nov 29 03:01:54 user nova-compute[75234]:     <mode name='host-passthrough' supported='yes'>
Nov 29 03:01:54 user nova-compute[75234]:       <enum name='hostPassthroughMigratable'>
Nov 29 03:01:54 user nova-compute[75234]:         <value>off</value>
Nov 29 03:01:54 user nova-compute[75234]:       </enum>
Nov 29 03:01:54 user nova-compute[75234]:     </mode>
Nov 29 03:01:54 user nova-compute[75234]:     <mode name='maximum' supported='yes'>
Nov 29 03:01:54 user nova-compute[75234]:       <enum name='maximumMigratable'>
Nov 29 03:01:54 user nova-compute[75234]:         <value>on</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>off</value>
Nov 29 03:01:54 user nova-compute[75234]:       </enum>
Nov 29 03:01:54 user nova-compute[75234]:     </mode>
Nov 29 03:01:54 user nova-compute[75234]:     <mode name='host-model' supported='yes'>
Nov 29 03:01:54 user nova-compute[75234]:       <model fallback='forbid'>IvyBridge-IBRS</model>
Nov 29 03:01:54 user nova-compute[75234]:       <vendor>Intel</vendor>
Nov 29 03:01:54 user nova-compute[75234]:       <feature policy='require' name='ss'/>
Nov 29 03:01:54 user nova-compute[75234]:       <feature policy='require' name='vmx'/>
Nov 29 03:01:54 user nova-compute[75234]:       <feature policy='require' name='pcid'/>
Nov 29 03:01:54 user nova-compute[75234]:       <feature policy='require' name='hypervisor'/>
Nov 29 03:01:54 user nova-compute[75234]:       <feature policy='require' name='arat'/>
Nov 29 03:01:54 user nova-compute[75234]:       <feature policy='require' name='tsc_adjust'/>
Nov 29 03:01:54 user nova-compute[75234]:       <feature policy='require' name='umip'/>
Nov 29 03:01:54 user nova-compute[75234]:       <feature policy='require' name='md-clear'/>
Nov 29 03:01:54 user nova-compute[75234]:       <feature policy='require' name='stibp'/>
Nov 29 03:01:54 user nova-compute[75234]:       <feature policy='require' name='arch-capabilities'/>
Nov 29 03:01:54 user nova-compute[75234]:       <feature policy='require' name='ssbd'/>
Nov 29 03:01:54 user nova-compute[75234]:       <feature policy='require' name='ibpb'/>
Nov 29 03:01:54 user nova-compute[75234]:       <feature policy='require' name='ibrs'/>
Nov 29 03:01:54 user nova-compute[75234]:       <feature policy='require' name='amd-stibp'/>
Nov 29 03:01:54 user nova-compute[75234]:       <feature policy='require' name='amd-ssbd'/>
Nov 29 03:01:54 user nova-compute[75234]:       <feature policy='disable' name='erms'/>
Nov 29 03:01:54 user nova-compute[75234]:       <feature policy='disable' name='xsaveopt'/>
Nov 29 03:01:54 user nova-compute[75234]:       <feature policy='disable' name='lm'/>
Nov 29 03:01:54 user nova-compute[75234]:     </mode>
Nov 29 03:01:54 user nova-compute[75234]:     <mode name='custom' supported='yes'>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>qemu64</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='yes'>qemu32</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>phenom</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='yes'>pentium3</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='yes'>pentium2</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='yes'>pentium</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>n270</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>kvm64</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='yes'>kvm32</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='yes'>coreduo</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>core2duo</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>athlon</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Westmere-IBRS</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Westmere</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Snowridge</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Skylake-Server-noTSX-IBRS</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Skylake-Server-IBRS</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Skylake-Server</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Skylake-Client-noTSX-IBRS</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Skylake-Client-IBRS</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Skylake-Client</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>SandyBridge-IBRS</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>SandyBridge</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Penryn</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Opteron_G5</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Opteron_G4</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Opteron_G3</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Opteron_G2</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Opteron_G1</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Nehalem-IBRS</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Nehalem</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>IvyBridge-IBRS</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>IvyBridge</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Icelake-Server-noTSX</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Icelake-Server</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no' deprecated='yes'>Icelake-Client-noTSX</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no' deprecated='yes'>Icelake-Client</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Haswell-noTSX-IBRS</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Haswell-noTSX</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Haswell-IBRS</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Haswell</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>EPYC-Rome</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>EPYC-Milan</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>EPYC-IBPB</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>EPYC</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Dhyana</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Cooperlake</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Conroe</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Cascadelake-Server-noTSX</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Cascadelake-Server</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Broadwell-noTSX-IBRS</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Broadwell-noTSX</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Broadwell-IBRS</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Broadwell</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='yes'>486</model>
Nov 29 03:01:54 user nova-compute[75234]:     </mode>
Nov 29 03:01:54 user nova-compute[75234]:   </cpu>
Nov 29 03:01:54 user nova-compute[75234]:   <memoryBacking supported='yes'>
Nov 29 03:01:54 user nova-compute[75234]:     <enum name='sourceType'>
Nov 29 03:01:54 user nova-compute[75234]:       <value>file</value>
Nov 29 03:01:54 user nova-compute[75234]:       <value>anonymous</value>
Nov 29 03:01:54 user nova-compute[75234]:       <value>memfd</value>
Nov 29 03:01:54 user nova-compute[75234]:     </enum>
Nov 29 03:01:54 user nova-compute[75234]:   </memoryBacking>
Nov 29 03:01:54 user nova-compute[75234]:   <devices>
Nov 29 03:01:54 user nova-compute[75234]:     <disk supported='yes'>
Nov 29 03:01:54 user nova-compute[75234]:       <enum name='diskDevice'>
Nov 29 03:01:54 user nova-compute[75234]:         <value>disk</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>cdrom</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>floppy</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>lun</value>
Nov 29 03:01:54 user nova-compute[75234]:       </enum>
Nov 29 03:01:54 user nova-compute[75234]:       <enum name='bus'>
Nov 29 03:01:54 user nova-compute[75234]:         <value>fdc</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>scsi</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>virtio</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>usb</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>sata</value>
Nov 29 03:01:54 user nova-compute[75234]:       </enum>
Nov 29 03:01:54 user nova-compute[75234]:       <enum name='model'>
Nov 29 03:01:54 user nova-compute[75234]:         <value>virtio</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>virtio-transitional</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>virtio-non-transitional</value>
Nov 29 03:01:54 user nova-compute[75234]:       </enum>
Nov 29 03:01:54 user nova-compute[75234]:     </disk>
Nov 29 03:01:54 user nova-compute[75234]:     <graphics supported='yes'>
Nov 29 03:01:54 user nova-compute[75234]:       <enum name='type'>
Nov 29 03:01:54 user nova-compute[75234]:         <value>sdl</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>vnc</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>spice</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>egl-headless</value>
Nov 29 03:01:54 user nova-compute[75234]:       </enum>
Nov 29 03:01:54 user nova-compute[75234]:     </graphics>
Nov 29 03:01:54 user nova-compute[75234]:     <video supported='yes'>
Nov 29 03:01:54 user nova-compute[75234]:       <enum name='modelType'>
Nov 29 03:01:54 user nova-compute[75234]:         <value>vga</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>cirrus</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>vmvga</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>qxl</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>virtio</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>none</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>bochs</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>ramfb</value>
Nov 29 03:01:54 user nova-compute[75234]:       </enum>
Nov 29 03:01:54 user nova-compute[75234]:     </video>
Nov 29 03:01:54 user nova-compute[75234]:     <hostdev supported='yes'>
Nov 29 03:01:54 user nova-compute[75234]:       <enum name='mode'>
Nov 29 03:01:54 user nova-compute[75234]:         <value>subsystem</value>
Nov 29 03:01:54 user nova-compute[75234]:       </enum>
Nov 29 03:01:54 user nova-compute[75234]:       <enum name='startupPolicy'>
Nov 29 03:01:54 user nova-compute[75234]:         <value>default</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>mandatory</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>requisite</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>optional</value>
Nov 29 03:01:54 user nova-compute[75234]:       </enum>
Nov 29 03:01:54 user nova-compute[75234]:       <enum name='subsysType'>
Nov 29 03:01:54 user nova-compute[75234]:         <value>usb</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>pci</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>scsi</value>
Nov 29 03:01:54 user nova-compute[75234]:       </enum>
Nov 29 03:01:54 user nova-compute[75234]:       <enum name='capsType'/>
Nov 29 03:01:54 user nova-compute[75234]:       <enum name='pciBackend'/>
Nov 29 03:01:54 user nova-compute[75234]:     </hostdev>
Nov 29 03:01:54 user nova-compute[75234]:     <rng supported='yes'>
Nov 29 03:01:54 user nova-compute[75234]:       <enum name='model'>
Nov 29 03:01:54 user nova-compute[75234]:         <value>virtio</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>virtio-transitional</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>virtio-non-transitional</value>
Nov 29 03:01:54 user nova-compute[75234]:       </enum>
Nov 29 03:01:54 user nova-compute[75234]:       <enum name='backendModel'>
Nov 29 03:01:54 user nova-compute[75234]:         <value>random</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>egd</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>builtin</value>
Nov 29 03:01:54 user nova-compute[75234]:       </enum>
Nov 29 03:01:54 user nova-compute[75234]:     </rng>
Nov 29 03:01:54 user nova-compute[75234]:     <filesystem supported='yes'>
Nov 29 03:01:54 user nova-compute[75234]:       <enum name='driverType'>
Nov 29 03:01:54 user nova-compute[75234]:         <value>path</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>handle</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>virtiofs</value>
Nov 29 03:01:54 user nova-compute[75234]:       </enum>
Nov 29 03:01:54 user nova-compute[75234]:     </filesystem>
Nov 29 03:01:54 user nova-compute[75234]:     <tpm supported='yes'>
Nov 29 03:01:54 user nova-compute[75234]:       <enum name='model'>
Nov 29 03:01:54 user nova-compute[75234]:         <value>tpm-tis</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>tpm-crb</value>
Nov 29 03:01:54 user nova-compute[75234]:       </enum>
Nov 29 03:01:54 user nova-compute[75234]:       <enum name='backendModel'>
Nov 29 03:01:54 user nova-compute[75234]:         <value>passthrough</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>emulator</value>
Nov 29 03:01:54 user nova-compute[75234]:       </enum>
Nov 29 03:01:54 user nova-compute[75234]:     </tpm>
Nov 29 03:01:54 user nova-compute[75234]:   </devices>
Nov 29 03:01:54 user nova-compute[75234]:   <features>
Nov 29 03:01:54 user nova-compute[75234]:     <gic supported='no'/>
Nov 29 03:01:54 user nova-compute[75234]:     <vmcoreinfo supported='yes'/>
Nov 29 03:01:54 user nova-compute[75234]:     <genid supported='yes'/>
Nov 29 03:01:54 user nova-compute[75234]:     <backingStoreInput supported='yes'/>
Nov 29 03:01:54 user nova-compute[75234]:     <backup supported='yes'/>
Nov 29 03:01:54 user nova-compute[75234]:     <sev supported='no'/>
Nov 29 03:01:54 user nova-compute[75234]:   </features>
Nov 29 03:01:54 user nova-compute[75234]: </domainCapabilities>
Nov 29 03:01:54 user nova-compute[75234]:  {{(pid=75234) _get_domain_capabilities /opt/stack/nova/nova/virt/libvirt/host.py:1039}}
Nov 29 03:01:54 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-ef65ec89-6c59-4f39-9adc-df4888ffd41b None None] Libvirt host hypervisor capabilities for arch=i686 and machine_type=pc:
Nov 29 03:01:54 user nova-compute[75234]: <domainCapabilities>
Nov 29 03:01:54 user nova-compute[75234]:   <path>/usr/bin/qemu-system-i386</path>
Nov 29 03:01:54 user nova-compute[75234]:   <domain>kvm</domain>
Nov 29 03:01:54 user nova-compute[75234]:   <machine>pc-i440fx-6.2</machine>
Nov 29 03:01:54 user nova-compute[75234]:   <arch>i686</arch>
Nov 29 03:01:54 user nova-compute[75234]:   <vcpu max='255'/>
Nov 29 03:01:54 user nova-compute[75234]:   <iothreads supported='yes'/>
Nov 29 03:01:54 user nova-compute[75234]:   <os supported='yes'>
Nov 29 03:01:54 user nova-compute[75234]:     <enum name='firmware'/>
Nov 29 03:01:54 user nova-compute[75234]:     <loader supported='yes'>
Nov 29 03:01:54 user nova-compute[75234]:       <value>/usr/share/OVMF/OVMF_CODE.fd</value>
Nov 29 03:01:54 user nova-compute[75234]:       <value>/usr/share/OVMF/OVMF_CODE.secboot.fd</value>
Nov 29 03:01:54 user nova-compute[75234]:       <value>/usr/share/AAVMF/AAVMF_CODE.fd</value>
Nov 29 03:01:54 user nova-compute[75234]:       <value>/usr/share/AAVMF/AAVMF32_CODE.fd</value>
Nov 29 03:01:54 user nova-compute[75234]:       <value>/usr/share/OVMF/OVMF_CODE.ms.fd</value>
Nov 29 03:01:54 user nova-compute[75234]:       <enum name='type'>
Nov 29 03:01:54 user nova-compute[75234]:         <value>rom</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>pflash</value>
Nov 29 03:01:54 user nova-compute[75234]:       </enum>
Nov 29 03:01:54 user nova-compute[75234]:       <enum name='readonly'>
Nov 29 03:01:54 user nova-compute[75234]:         <value>yes</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>no</value>
Nov 29 03:01:54 user nova-compute[75234]:       </enum>
Nov 29 03:01:54 user nova-compute[75234]:       <enum name='secure'>
Nov 29 03:01:54 user nova-compute[75234]:         <value>no</value>
Nov 29 03:01:54 user nova-compute[75234]:       </enum>
Nov 29 03:01:54 user nova-compute[75234]:     </loader>
Nov 29 03:01:54 user nova-compute[75234]:   </os>
Nov 29 03:01:54 user nova-compute[75234]:   <cpu>
Nov 29 03:01:54 user nova-compute[75234]:     <mode name='host-passthrough' supported='yes'>
Nov 29 03:01:54 user nova-compute[75234]:       <enum name='hostPassthroughMigratable'>
Nov 29 03:01:54 user nova-compute[75234]:         <value>off</value>
Nov 29 03:01:54 user nova-compute[75234]:       </enum>
Nov 29 03:01:54 user nova-compute[75234]:     </mode>
Nov 29 03:01:54 user nova-compute[75234]:     <mode name='maximum' supported='yes'>
Nov 29 03:01:54 user nova-compute[75234]:       <enum name='maximumMigratable'>
Nov 29 03:01:54 user nova-compute[75234]:         <value>on</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>off</value>
Nov 29 03:01:54 user nova-compute[75234]:       </enum>
Nov 29 03:01:54 user nova-compute[75234]:     </mode>
Nov 29 03:01:54 user nova-compute[75234]:     <mode name='host-model' supported='yes'>
Nov 29 03:01:54 user nova-compute[75234]:       <model fallback='forbid'>IvyBridge-IBRS</model>
Nov 29 03:01:54 user nova-compute[75234]:       <vendor>Intel</vendor>
Nov 29 03:01:54 user nova-compute[75234]:       <feature policy='require' name='ss'/>
Nov 29 03:01:54 user nova-compute[75234]:       <feature policy='require' name='vmx'/>
Nov 29 03:01:54 user nova-compute[75234]:       <feature policy='require' name='pcid'/>
Nov 29 03:01:54 user nova-compute[75234]:       <feature policy='require' name='hypervisor'/>
Nov 29 03:01:54 user nova-compute[75234]:       <feature policy='require' name='arat'/>
Nov 29 03:01:54 user nova-compute[75234]:       <feature policy='require' name='tsc_adjust'/>
Nov 29 03:01:54 user nova-compute[75234]:       <feature policy='require' name='umip'/>
Nov 29 03:01:54 user nova-compute[75234]:       <feature policy='require' name='md-clear'/>
Nov 29 03:01:54 user nova-compute[75234]:       <feature policy='require' name='stibp'/>
Nov 29 03:01:54 user nova-compute[75234]:       <feature policy='require' name='arch-capabilities'/>
Nov 29 03:01:54 user nova-compute[75234]:       <feature policy='require' name='ssbd'/>
Nov 29 03:01:54 user nova-compute[75234]:       <feature policy='require' name='ibpb'/>
Nov 29 03:01:54 user nova-compute[75234]:       <feature policy='require' name='ibrs'/>
Nov 29 03:01:54 user nova-compute[75234]:       <feature policy='require' name='amd-stibp'/>
Nov 29 03:01:54 user nova-compute[75234]:       <feature policy='require' name='amd-ssbd'/>
Nov 29 03:01:54 user nova-compute[75234]:       <feature policy='disable' name='erms'/>
Nov 29 03:01:54 user nova-compute[75234]:       <feature policy='disable' name='xsaveopt'/>
Nov 29 03:01:54 user nova-compute[75234]:       <feature policy='disable' name='lm'/>
Nov 29 03:01:54 user nova-compute[75234]:     </mode>
Nov 29 03:01:54 user nova-compute[75234]:     <mode name='custom' supported='yes'>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>qemu64</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='yes'>qemu32</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>phenom</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='yes'>pentium3</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='yes'>pentium2</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='yes'>pentium</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>n270</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>kvm64</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='yes'>kvm32</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='yes'>coreduo</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>core2duo</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>athlon</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Westmere-IBRS</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Westmere</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Snowridge</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Skylake-Server-noTSX-IBRS</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Skylake-Server-IBRS</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Skylake-Server</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Skylake-Client-noTSX-IBRS</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Skylake-Client-IBRS</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Skylake-Client</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>SandyBridge-IBRS</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>SandyBridge</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Penryn</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Opteron_G5</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Opteron_G4</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Opteron_G3</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Opteron_G2</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Opteron_G1</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Nehalem-IBRS</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Nehalem</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>IvyBridge-IBRS</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>IvyBridge</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Icelake-Server-noTSX</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Icelake-Server</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no' deprecated='yes'>Icelake-Client-noTSX</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no' deprecated='yes'>Icelake-Client</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Haswell-noTSX-IBRS</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Haswell-noTSX</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Haswell-IBRS</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Haswell</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>EPYC-Rome</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>EPYC-Milan</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>EPYC-IBPB</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>EPYC</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Dhyana</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Cooperlake</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Conroe</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Cascadelake-Server-noTSX</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Cascadelake-Server</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Broadwell-noTSX-IBRS</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Broadwell-noTSX</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Broadwell-IBRS</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Broadwell</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='yes'>486</model>
Nov 29 03:01:54 user nova-compute[75234]:     </mode>
Nov 29 03:01:54 user nova-compute[75234]:   </cpu>
Nov 29 03:01:54 user nova-compute[75234]:   <memoryBacking supported='yes'>
Nov 29 03:01:54 user nova-compute[75234]:     <enum name='sourceType'>
Nov 29 03:01:54 user nova-compute[75234]:       <value>file</value>
Nov 29 03:01:54 user nova-compute[75234]:       <value>anonymous</value>
Nov 29 03:01:54 user nova-compute[75234]:       <value>memfd</value>
Nov 29 03:01:54 user nova-compute[75234]:     </enum>
Nov 29 03:01:54 user nova-compute[75234]:   </memoryBacking>
Nov 29 03:01:54 user nova-compute[75234]:   <devices>
Nov 29 03:01:54 user nova-compute[75234]:     <disk supported='yes'>
Nov 29 03:01:54 user nova-compute[75234]:       <enum name='diskDevice'>
Nov 29 03:01:54 user nova-compute[75234]:         <value>disk</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>cdrom</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>floppy</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>lun</value>
Nov 29 03:01:54 user nova-compute[75234]:       </enum>
Nov 29 03:01:54 user nova-compute[75234]:       <enum name='bus'>
Nov 29 03:01:54 user nova-compute[75234]:         <value>ide</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>fdc</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>scsi</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>virtio</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>usb</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>sata</value>
Nov 29 03:01:54 user nova-compute[75234]:       </enum>
Nov 29 03:01:54 user nova-compute[75234]:       <enum name='model'>
Nov 29 03:01:54 user nova-compute[75234]:         <value>virtio</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>virtio-transitional</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>virtio-non-transitional</value>
Nov 29 03:01:54 user nova-compute[75234]:       </enum>
Nov 29 03:01:54 user nova-compute[75234]:     </disk>
Nov 29 03:01:54 user nova-compute[75234]:     <graphics supported='yes'>
Nov 29 03:01:54 user nova-compute[75234]:       <enum name='type'>
Nov 29 03:01:54 user nova-compute[75234]:         <value>sdl</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>vnc</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>spice</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>egl-headless</value>
Nov 29 03:01:54 user nova-compute[75234]:       </enum>
Nov 29 03:01:54 user nova-compute[75234]:     </graphics>
Nov 29 03:01:54 user nova-compute[75234]:     <video supported='yes'>
Nov 29 03:01:54 user nova-compute[75234]:       <enum name='modelType'>
Nov 29 03:01:54 user nova-compute[75234]:         <value>vga</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>cirrus</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>vmvga</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>qxl</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>virtio</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>none</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>bochs</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>ramfb</value>
Nov 29 03:01:54 user nova-compute[75234]:       </enum>
Nov 29 03:01:54 user nova-compute[75234]:     </video>
Nov 29 03:01:54 user nova-compute[75234]:     <hostdev supported='yes'>
Nov 29 03:01:54 user nova-compute[75234]:       <enum name='mode'>
Nov 29 03:01:54 user nova-compute[75234]:         <value>subsystem</value>
Nov 29 03:01:54 user nova-compute[75234]:       </enum>
Nov 29 03:01:54 user nova-compute[75234]:       <enum name='startupPolicy'>
Nov 29 03:01:54 user nova-compute[75234]:         <value>default</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>mandatory</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>requisite</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>optional</value>
Nov 29 03:01:54 user nova-compute[75234]:       </enum>
Nov 29 03:01:54 user nova-compute[75234]:       <enum name='subsysType'>
Nov 29 03:01:54 user nova-compute[75234]:         <value>usb</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>pci</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>scsi</value>
Nov 29 03:01:54 user nova-compute[75234]:       </enum>
Nov 29 03:01:54 user nova-compute[75234]:       <enum name='capsType'/>
Nov 29 03:01:54 user nova-compute[75234]:       <enum name='pciBackend'/>
Nov 29 03:01:54 user nova-compute[75234]:     </hostdev>
Nov 29 03:01:54 user nova-compute[75234]:     <rng supported='yes'>
Nov 29 03:01:54 user nova-compute[75234]:       <enum name='model'>
Nov 29 03:01:54 user nova-compute[75234]:         <value>virtio</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>virtio-transitional</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>virtio-non-transitional</value>
Nov 29 03:01:54 user nova-compute[75234]:       </enum>
Nov 29 03:01:54 user nova-compute[75234]:       <enum name='backendModel'>
Nov 29 03:01:54 user nova-compute[75234]:         <value>random</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>egd</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>builtin</value>
Nov 29 03:01:54 user nova-compute[75234]:       </enum>
Nov 29 03:01:54 user nova-compute[75234]:     </rng>
Nov 29 03:01:54 user nova-compute[75234]:     <filesystem supported='yes'>
Nov 29 03:01:54 user nova-compute[75234]:       <enum name='driverType'>
Nov 29 03:01:54 user nova-compute[75234]:         <value>path</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>handle</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>virtiofs</value>
Nov 29 03:01:54 user nova-compute[75234]:       </enum>
Nov 29 03:01:54 user nova-compute[75234]:     </filesystem>
Nov 29 03:01:54 user nova-compute[75234]:     <tpm supported='yes'>
Nov 29 03:01:54 user nova-compute[75234]:       <enum name='model'>
Nov 29 03:01:54 user nova-compute[75234]:         <value>tpm-tis</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>tpm-crb</value>
Nov 29 03:01:54 user nova-compute[75234]:       </enum>
Nov 29 03:01:54 user nova-compute[75234]:       <enum name='backendModel'>
Nov 29 03:01:54 user nova-compute[75234]:         <value>passthrough</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>emulator</value>
Nov 29 03:01:54 user nova-compute[75234]:       </enum>
Nov 29 03:01:54 user nova-compute[75234]:     </tpm>
Nov 29 03:01:54 user nova-compute[75234]:   </devices>
Nov 29 03:01:54 user nova-compute[75234]:   <features>
Nov 29 03:01:54 user nova-compute[75234]:     <gic supported='no'/>
Nov 29 03:01:54 user nova-compute[75234]:     <vmcoreinfo supported='yes'/>
Nov 29 03:01:54 user nova-compute[75234]:     <genid supported='yes'/>
Nov 29 03:01:54 user nova-compute[75234]:     <backingStoreInput supported='yes'/>
Nov 29 03:01:54 user nova-compute[75234]:     <backup supported='yes'/>
Nov 29 03:01:54 user nova-compute[75234]:     <sev supported='no'/>
Nov 29 03:01:54 user nova-compute[75234]:   </features>
Nov 29 03:01:54 user nova-compute[75234]: </domainCapabilities>
Nov 29 03:01:54 user nova-compute[75234]:  {{(pid=75234) _get_domain_capabilities /opt/stack/nova/nova/virt/libvirt/host.py:1039}}
Nov 29 03:01:54 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-ef65ec89-6c59-4f39-9adc-df4888ffd41b None None] Getting domain capabilities for m68k via machine types: {'virt', None} {{(pid=75234) _get_machine_types /opt/stack/nova/nova/virt/libvirt/host.py:954}}
Nov 29 03:01:54 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-ef65ec89-6c59-4f39-9adc-df4888ffd41b None None] Error from libvirt when retrieving domain capabilities for arch m68k / virt_type kvm / machine_type virt: [Error Code 8]: invalid argument: KVM is not supported by '/usr/bin/qemu-system-m68k' on this host {{(pid=75234) _add_to_domain_capabilities /opt/stack/nova/nova/virt/libvirt/host.py:993}}
Nov 29 03:01:54 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-ef65ec89-6c59-4f39-9adc-df4888ffd41b None None] Error from libvirt when retrieving domain capabilities for arch m68k / virt_type kvm / machine_type None: [Error Code 8]: invalid argument: KVM is not supported by '/usr/bin/qemu-system-m68k' on this host {{(pid=75234) _add_to_domain_capabilities /opt/stack/nova/nova/virt/libvirt/host.py:993}}
Nov 29 03:01:54 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-ef65ec89-6c59-4f39-9adc-df4888ffd41b None None] Getting domain capabilities for microblaze via machine types: {None} {{(pid=75234) _get_machine_types /opt/stack/nova/nova/virt/libvirt/host.py:954}}
Nov 29 03:01:54 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-ef65ec89-6c59-4f39-9adc-df4888ffd41b None None] Error from libvirt when retrieving domain capabilities for arch microblaze / virt_type kvm / machine_type None: [Error Code 8]: invalid argument: KVM is not supported by '/usr/bin/qemu-system-microblaze' on this host {{(pid=75234) _add_to_domain_capabilities /opt/stack/nova/nova/virt/libvirt/host.py:993}}
Nov 29 03:01:54 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-ef65ec89-6c59-4f39-9adc-df4888ffd41b None None] Getting domain capabilities for microblazeel via machine types: {None} {{(pid=75234) _get_machine_types /opt/stack/nova/nova/virt/libvirt/host.py:954}}
Nov 29 03:01:54 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-ef65ec89-6c59-4f39-9adc-df4888ffd41b None None] Error from libvirt when retrieving domain capabilities for arch microblazeel / virt_type kvm / machine_type None: [Error Code 8]: invalid argument: KVM is not supported by '/usr/bin/qemu-system-microblazeel' on this host {{(pid=75234) _add_to_domain_capabilities /opt/stack/nova/nova/virt/libvirt/host.py:993}}
Nov 29 03:01:54 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-ef65ec89-6c59-4f39-9adc-df4888ffd41b None None] Getting domain capabilities for mips via machine types: {None} {{(pid=75234) _get_machine_types /opt/stack/nova/nova/virt/libvirt/host.py:954}}
Nov 29 03:01:54 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-ef65ec89-6c59-4f39-9adc-df4888ffd41b None None] Error from libvirt when retrieving domain capabilities for arch mips / virt_type kvm / machine_type None: [Error Code 8]: invalid argument: KVM is not supported by '/usr/bin/qemu-system-mips' on this host {{(pid=75234) _add_to_domain_capabilities /opt/stack/nova/nova/virt/libvirt/host.py:993}}
Nov 29 03:01:54 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-ef65ec89-6c59-4f39-9adc-df4888ffd41b None None] Getting domain capabilities for mipsel via machine types: {None} {{(pid=75234) _get_machine_types /opt/stack/nova/nova/virt/libvirt/host.py:954}}
Nov 29 03:01:54 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-ef65ec89-6c59-4f39-9adc-df4888ffd41b None None] Error from libvirt when retrieving domain capabilities for arch mipsel / virt_type kvm / machine_type None: [Error Code 8]: invalid argument: KVM is not supported by '/usr/bin/qemu-system-mipsel' on this host {{(pid=75234) _add_to_domain_capabilities /opt/stack/nova/nova/virt/libvirt/host.py:993}}
Nov 29 03:01:54 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-ef65ec89-6c59-4f39-9adc-df4888ffd41b None None] Getting domain capabilities for mips64 via machine types: {None} {{(pid=75234) _get_machine_types /opt/stack/nova/nova/virt/libvirt/host.py:954}}
Nov 29 03:01:54 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-ef65ec89-6c59-4f39-9adc-df4888ffd41b None None] Error from libvirt when retrieving domain capabilities for arch mips64 / virt_type kvm / machine_type None: [Error Code 8]: invalid argument: KVM is not supported by '/usr/bin/qemu-system-mips64' on this host {{(pid=75234) _add_to_domain_capabilities /opt/stack/nova/nova/virt/libvirt/host.py:993}}
Nov 29 03:01:54 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-ef65ec89-6c59-4f39-9adc-df4888ffd41b None None] Getting domain capabilities for mips64el via machine types: {None} {{(pid=75234) _get_machine_types /opt/stack/nova/nova/virt/libvirt/host.py:954}}
Nov 29 03:01:54 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-ef65ec89-6c59-4f39-9adc-df4888ffd41b None None] Error from libvirt when retrieving domain capabilities for arch mips64el / virt_type kvm / machine_type None: [Error Code 8]: invalid argument: KVM is not supported by '/usr/bin/qemu-system-mips64el' on this host {{(pid=75234) _add_to_domain_capabilities /opt/stack/nova/nova/virt/libvirt/host.py:993}}
Nov 29 03:01:54 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-ef65ec89-6c59-4f39-9adc-df4888ffd41b None None] Getting domain capabilities for ppc via machine types: {None} {{(pid=75234) _get_machine_types /opt/stack/nova/nova/virt/libvirt/host.py:954}}
Nov 29 03:01:54 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-ef65ec89-6c59-4f39-9adc-df4888ffd41b None None] Error from libvirt when retrieving domain capabilities for arch ppc / virt_type kvm / machine_type None: [Error Code 8]: invalid argument: KVM is not supported by '/usr/bin/qemu-system-ppc' on this host {{(pid=75234) _add_to_domain_capabilities /opt/stack/nova/nova/virt/libvirt/host.py:993}}
Nov 29 03:01:54 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-ef65ec89-6c59-4f39-9adc-df4888ffd41b None None] Getting domain capabilities for ppc64 via machine types: {'powernv', 'pseries', None} {{(pid=75234) _get_machine_types /opt/stack/nova/nova/virt/libvirt/host.py:954}}
Nov 29 03:01:54 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-ef65ec89-6c59-4f39-9adc-df4888ffd41b None None] Error from libvirt when retrieving domain capabilities for arch ppc64 / virt_type kvm / machine_type powernv: [Error Code 8]: invalid argument: KVM is not supported by '/usr/bin/qemu-system-ppc64' on this host {{(pid=75234) _add_to_domain_capabilities /opt/stack/nova/nova/virt/libvirt/host.py:993}}
Nov 29 03:01:54 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-ef65ec89-6c59-4f39-9adc-df4888ffd41b None None] Error from libvirt when retrieving domain capabilities for arch ppc64 / virt_type kvm / machine_type pseries: [Error Code 8]: invalid argument: KVM is not supported by '/usr/bin/qemu-system-ppc64' on this host {{(pid=75234) _add_to_domain_capabilities /opt/stack/nova/nova/virt/libvirt/host.py:993}}
Nov 29 03:01:54 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-ef65ec89-6c59-4f39-9adc-df4888ffd41b None None] Error from libvirt when retrieving domain capabilities for arch ppc64 / virt_type kvm / machine_type None: [Error Code 8]: invalid argument: KVM is not supported by '/usr/bin/qemu-system-ppc64' on this host {{(pid=75234) _add_to_domain_capabilities /opt/stack/nova/nova/virt/libvirt/host.py:993}}
Nov 29 03:01:54 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-ef65ec89-6c59-4f39-9adc-df4888ffd41b None None] Getting domain capabilities for ppc64le via machine types: {'pseries', 'powernv'} {{(pid=75234) _get_machine_types /opt/stack/nova/nova/virt/libvirt/host.py:954}}
Nov 29 03:01:54 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-ef65ec89-6c59-4f39-9adc-df4888ffd41b None None] Error from libvirt when retrieving domain capabilities for arch ppc64le / virt_type kvm / machine_type pseries: [Error Code 8]: invalid argument: KVM is not supported by '/usr/bin/qemu-system-ppc64le' on this host {{(pid=75234) _add_to_domain_capabilities /opt/stack/nova/nova/virt/libvirt/host.py:993}}
Nov 29 03:01:54 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-ef65ec89-6c59-4f39-9adc-df4888ffd41b None None] Error from libvirt when retrieving domain capabilities for arch ppc64le / virt_type kvm / machine_type powernv: [Error Code 8]: invalid argument: KVM is not supported by '/usr/bin/qemu-system-ppc64le' on this host {{(pid=75234) _add_to_domain_capabilities /opt/stack/nova/nova/virt/libvirt/host.py:993}}
Nov 29 03:01:54 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-ef65ec89-6c59-4f39-9adc-df4888ffd41b None None] Getting domain capabilities for riscv32 via machine types: {None} {{(pid=75234) _get_machine_types /opt/stack/nova/nova/virt/libvirt/host.py:954}}
Nov 29 03:01:54 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-ef65ec89-6c59-4f39-9adc-df4888ffd41b None None] Error from libvirt when retrieving domain capabilities for arch riscv32 / virt_type kvm / machine_type None: [Error Code 8]: invalid argument: KVM is not supported by '/usr/bin/qemu-system-riscv32' on this host {{(pid=75234) _add_to_domain_capabilities /opt/stack/nova/nova/virt/libvirt/host.py:993}}
Nov 29 03:01:54 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-ef65ec89-6c59-4f39-9adc-df4888ffd41b None None] Getting domain capabilities for riscv64 via machine types: {None} {{(pid=75234) _get_machine_types /opt/stack/nova/nova/virt/libvirt/host.py:954}}
Nov 29 03:01:54 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-ef65ec89-6c59-4f39-9adc-df4888ffd41b None None] Error from libvirt when retrieving domain capabilities for arch riscv64 / virt_type kvm / machine_type None: [Error Code 8]: invalid argument: KVM is not supported by '/usr/bin/qemu-system-riscv64' on this host {{(pid=75234) _add_to_domain_capabilities /opt/stack/nova/nova/virt/libvirt/host.py:993}}
Nov 29 03:01:54 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-ef65ec89-6c59-4f39-9adc-df4888ffd41b None None] Getting domain capabilities for s390x via machine types: {'s390-ccw-virtio'} {{(pid=75234) _get_machine_types /opt/stack/nova/nova/virt/libvirt/host.py:954}}
Nov 29 03:01:54 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-ef65ec89-6c59-4f39-9adc-df4888ffd41b None None] Error from libvirt when retrieving domain capabilities for arch s390x / virt_type kvm / machine_type s390-ccw-virtio: [Error Code 8]: invalid argument: KVM is not supported by '/usr/bin/qemu-system-s390x' on this host {{(pid=75234) _add_to_domain_capabilities /opt/stack/nova/nova/virt/libvirt/host.py:993}}
Nov 29 03:01:54 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-ef65ec89-6c59-4f39-9adc-df4888ffd41b None None] Getting domain capabilities for sh4 via machine types: {None} {{(pid=75234) _get_machine_types /opt/stack/nova/nova/virt/libvirt/host.py:954}}
Nov 29 03:01:54 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-ef65ec89-6c59-4f39-9adc-df4888ffd41b None None] Error from libvirt when retrieving domain capabilities for arch sh4 / virt_type kvm / machine_type None: [Error Code 8]: invalid argument: KVM is not supported by '/usr/bin/qemu-system-sh4' on this host {{(pid=75234) _add_to_domain_capabilities /opt/stack/nova/nova/virt/libvirt/host.py:993}}
Nov 29 03:01:54 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-ef65ec89-6c59-4f39-9adc-df4888ffd41b None None] Getting domain capabilities for sh4eb via machine types: {None} {{(pid=75234) _get_machine_types /opt/stack/nova/nova/virt/libvirt/host.py:954}}
Nov 29 03:01:54 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-ef65ec89-6c59-4f39-9adc-df4888ffd41b None None] Error from libvirt when retrieving domain capabilities for arch sh4eb / virt_type kvm / machine_type None: [Error Code 8]: invalid argument: KVM is not supported by '/usr/bin/qemu-system-sh4eb' on this host {{(pid=75234) _add_to_domain_capabilities /opt/stack/nova/nova/virt/libvirt/host.py:993}}
Nov 29 03:01:54 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-ef65ec89-6c59-4f39-9adc-df4888ffd41b None None] Getting domain capabilities for sparc via machine types: {None} {{(pid=75234) _get_machine_types /opt/stack/nova/nova/virt/libvirt/host.py:954}}
Nov 29 03:01:54 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-ef65ec89-6c59-4f39-9adc-df4888ffd41b None None] Error from libvirt when retrieving domain capabilities for arch sparc / virt_type kvm / machine_type None: [Error Code 8]: invalid argument: KVM is not supported by '/usr/bin/qemu-system-sparc' on this host {{(pid=75234) _add_to_domain_capabilities /opt/stack/nova/nova/virt/libvirt/host.py:993}}
Nov 29 03:01:54 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-ef65ec89-6c59-4f39-9adc-df4888ffd41b None None] Getting domain capabilities for sparc64 via machine types: {None} {{(pid=75234) _get_machine_types /opt/stack/nova/nova/virt/libvirt/host.py:954}}
Nov 29 03:01:54 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-ef65ec89-6c59-4f39-9adc-df4888ffd41b None None] Error from libvirt when retrieving domain capabilities for arch sparc64 / virt_type kvm / machine_type None: [Error Code 8]: invalid argument: KVM is not supported by '/usr/bin/qemu-system-sparc64' on this host {{(pid=75234) _add_to_domain_capabilities /opt/stack/nova/nova/virt/libvirt/host.py:993}}
Nov 29 03:01:54 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-ef65ec89-6c59-4f39-9adc-df4888ffd41b None None] Getting domain capabilities for x86_64 via machine types: {'ubuntu-q35', 'ubuntu', 'q35', 'pc'} {{(pid=75234) _get_machine_types /opt/stack/nova/nova/virt/libvirt/host.py:954}}
Nov 29 03:01:54 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-ef65ec89-6c59-4f39-9adc-df4888ffd41b None None] Libvirt host hypervisor capabilities for arch=x86_64 and machine_type=ubuntu-q35:
Nov 29 03:01:54 user nova-compute[75234]: <domainCapabilities>
Nov 29 03:01:54 user nova-compute[75234]:   <path>/usr/bin/qemu-system-x86_64</path>
Nov 29 03:01:54 user nova-compute[75234]:   <domain>kvm</domain>
Nov 29 03:01:54 user nova-compute[75234]:   <machine>pc-q35-jammy</machine>
Nov 29 03:01:54 user nova-compute[75234]:   <arch>x86_64</arch>
Nov 29 03:01:54 user nova-compute[75234]:   <vcpu max='288'/>
Nov 29 03:01:54 user nova-compute[75234]:   <iothreads supported='yes'/>
Nov 29 03:01:54 user nova-compute[75234]:   <os supported='yes'>
Nov 29 03:01:54 user nova-compute[75234]:     <enum name='firmware'>
Nov 29 03:01:54 user nova-compute[75234]:       <value>efi</value>
Nov 29 03:01:54 user nova-compute[75234]:     </enum>
Nov 29 03:01:54 user nova-compute[75234]:     <loader supported='yes'>
Nov 29 03:01:54 user nova-compute[75234]:       <value>/usr/share/OVMF/OVMF_CODE_4M.ms.fd</value>
Nov 29 03:01:54 user nova-compute[75234]:       <value>/usr/share/OVMF/OVMF_CODE_4M.secboot.fd</value>
Nov 29 03:01:54 user nova-compute[75234]:       <value>/usr/share/OVMF/OVMF_CODE_4M.fd</value>
Nov 29 03:01:54 user nova-compute[75234]:       <enum name='type'>
Nov 29 03:01:54 user nova-compute[75234]:         <value>rom</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>pflash</value>
Nov 29 03:01:54 user nova-compute[75234]:       </enum>
Nov 29 03:01:54 user nova-compute[75234]:       <enum name='readonly'>
Nov 29 03:01:54 user nova-compute[75234]:         <value>yes</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>no</value>
Nov 29 03:01:54 user nova-compute[75234]:       </enum>
Nov 29 03:01:54 user nova-compute[75234]:       <enum name='secure'>
Nov 29 03:01:54 user nova-compute[75234]:         <value>yes</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>no</value>
Nov 29 03:01:54 user nova-compute[75234]:       </enum>
Nov 29 03:01:54 user nova-compute[75234]:     </loader>
Nov 29 03:01:54 user nova-compute[75234]:   </os>
Nov 29 03:01:54 user nova-compute[75234]:   <cpu>
Nov 29 03:01:54 user nova-compute[75234]:     <mode name='host-passthrough' supported='yes'>
Nov 29 03:01:54 user nova-compute[75234]:       <enum name='hostPassthroughMigratable'>
Nov 29 03:01:54 user nova-compute[75234]:         <value>on</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>off</value>
Nov 29 03:01:54 user nova-compute[75234]:       </enum>
Nov 29 03:01:54 user nova-compute[75234]:     </mode>
Nov 29 03:01:54 user nova-compute[75234]:     <mode name='maximum' supported='yes'>
Nov 29 03:01:54 user nova-compute[75234]:       <enum name='maximumMigratable'>
Nov 29 03:01:54 user nova-compute[75234]:         <value>on</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>off</value>
Nov 29 03:01:54 user nova-compute[75234]:       </enum>
Nov 29 03:01:54 user nova-compute[75234]:     </mode>
Nov 29 03:01:54 user nova-compute[75234]:     <mode name='host-model' supported='yes'>
Nov 29 03:01:54 user nova-compute[75234]:       <model fallback='forbid'>IvyBridge-IBRS</model>
Nov 29 03:01:54 user nova-compute[75234]:       <vendor>Intel</vendor>
Nov 29 03:01:54 user nova-compute[75234]:       <feature policy='require' name='ss'/>
Nov 29 03:01:54 user nova-compute[75234]:       <feature policy='require' name='vmx'/>
Nov 29 03:01:54 user nova-compute[75234]:       <feature policy='require' name='pcid'/>
Nov 29 03:01:54 user nova-compute[75234]:       <feature policy='require' name='hypervisor'/>
Nov 29 03:01:54 user nova-compute[75234]:       <feature policy='require' name='arat'/>
Nov 29 03:01:54 user nova-compute[75234]:       <feature policy='require' name='tsc_adjust'/>
Nov 29 03:01:54 user nova-compute[75234]:       <feature policy='require' name='umip'/>
Nov 29 03:01:54 user nova-compute[75234]:       <feature policy='require' name='md-clear'/>
Nov 29 03:01:54 user nova-compute[75234]:       <feature policy='require' name='stibp'/>
Nov 29 03:01:54 user nova-compute[75234]:       <feature policy='require' name='arch-capabilities'/>
Nov 29 03:01:54 user nova-compute[75234]:       <feature policy='require' name='ssbd'/>
Nov 29 03:01:54 user nova-compute[75234]:       <feature policy='require' name='invtsc'/>
Nov 29 03:01:54 user nova-compute[75234]:       <feature policy='require' name='ibpb'/>
Nov 29 03:01:54 user nova-compute[75234]:       <feature policy='require' name='ibrs'/>
Nov 29 03:01:54 user nova-compute[75234]:       <feature policy='require' name='amd-stibp'/>
Nov 29 03:01:54 user nova-compute[75234]:       <feature policy='require' name='amd-ssbd'/>
Nov 29 03:01:54 user nova-compute[75234]:       <feature policy='require' name='rsba'/>
Nov 29 03:01:54 user nova-compute[75234]:       <feature policy='require' name='skip-l1dfl-vmentry'/>
Nov 29 03:01:54 user nova-compute[75234]:       <feature policy='require' name='pschange-mc-no'/>
Nov 29 03:01:54 user nova-compute[75234]:       <feature policy='disable' name='erms'/>
Nov 29 03:01:54 user nova-compute[75234]:       <feature policy='disable' name='xsaveopt'/>
Nov 29 03:01:54 user nova-compute[75234]:     </mode>
Nov 29 03:01:54 user nova-compute[75234]:     <mode name='custom' supported='yes'>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>qemu64</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='yes'>qemu32</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>phenom</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='yes'>pentium3</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='yes'>pentium2</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='yes'>pentium</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>n270</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='yes'>kvm64</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='yes'>kvm32</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='yes'>coreduo</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='yes'>core2duo</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>athlon</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='yes'>Westmere-IBRS</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='yes'>Westmere</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Snowridge</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Skylake-Server-noTSX-IBRS</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Skylake-Server-IBRS</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Skylake-Server</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Skylake-Client-noTSX-IBRS</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Skylake-Client-IBRS</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Skylake-Client</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>SandyBridge-IBRS</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>SandyBridge</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='yes'>Penryn</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Opteron_G5</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Opteron_G4</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Opteron_G3</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Opteron_G2</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='yes'>Opteron_G1</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='yes'>Nehalem-IBRS</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='yes'>Nehalem</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>IvyBridge-IBRS</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>IvyBridge</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Icelake-Server-noTSX</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Icelake-Server</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no' deprecated='yes'>Icelake-Client-noTSX</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no' deprecated='yes'>Icelake-Client</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Haswell-noTSX-IBRS</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Haswell-noTSX</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Haswell-IBRS</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Haswell</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>EPYC-Rome</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>EPYC-Milan</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>EPYC-IBPB</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>EPYC</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Dhyana</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Cooperlake</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='yes'>Conroe</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Cascadelake-Server-noTSX</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Cascadelake-Server</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Broadwell-noTSX-IBRS</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Broadwell-noTSX</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Broadwell-IBRS</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Broadwell</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='yes'>486</model>
Nov 29 03:01:54 user nova-compute[75234]:     </mode>
Nov 29 03:01:54 user nova-compute[75234]:   </cpu>
Nov 29 03:01:54 user nova-compute[75234]:   <memoryBacking supported='yes'>
Nov 29 03:01:54 user nova-compute[75234]:     <enum name='sourceType'>
Nov 29 03:01:54 user nova-compute[75234]:       <value>file</value>
Nov 29 03:01:54 user nova-compute[75234]:       <value>anonymous</value>
Nov 29 03:01:54 user nova-compute[75234]:       <value>memfd</value>
Nov 29 03:01:54 user nova-compute[75234]:     </enum>
Nov 29 03:01:54 user nova-compute[75234]:   </memoryBacking>
Nov 29 03:01:54 user nova-compute[75234]:   <devices>
Nov 29 03:01:54 user nova-compute[75234]:     <disk supported='yes'>
Nov 29 03:01:54 user nova-compute[75234]:       <enum name='diskDevice'>
Nov 29 03:01:54 user nova-compute[75234]:         <value>disk</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>cdrom</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>floppy</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>lun</value>
Nov 29 03:01:54 user nova-compute[75234]:       </enum>
Nov 29 03:01:54 user nova-compute[75234]:       <enum name='bus'>
Nov 29 03:01:54 user nova-compute[75234]:         <value>fdc</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>scsi</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>virtio</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>usb</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>sata</value>
Nov 29 03:01:54 user nova-compute[75234]:       </enum>
Nov 29 03:01:54 user nova-compute[75234]:       <enum name='model'>
Nov 29 03:01:54 user nova-compute[75234]:         <value>virtio</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>virtio-transitional</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>virtio-non-transitional</value>
Nov 29 03:01:54 user nova-compute[75234]:       </enum>
Nov 29 03:01:54 user nova-compute[75234]:     </disk>
Nov 29 03:01:54 user nova-compute[75234]:     <graphics supported='yes'>
Nov 29 03:01:54 user nova-compute[75234]:       <enum name='type'>
Nov 29 03:01:54 user nova-compute[75234]:         <value>sdl</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>vnc</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>spice</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>egl-headless</value>
Nov 29 03:01:54 user nova-compute[75234]:       </enum>
Nov 29 03:01:54 user nova-compute[75234]:     </graphics>
Nov 29 03:01:54 user nova-compute[75234]:     <video supported='yes'>
Nov 29 03:01:54 user nova-compute[75234]:       <enum name='modelType'>
Nov 29 03:01:54 user nova-compute[75234]:         <value>vga</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>cirrus</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>vmvga</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>qxl</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>virtio</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>none</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>bochs</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>ramfb</value>
Nov 29 03:01:54 user nova-compute[75234]:       </enum>
Nov 29 03:01:54 user nova-compute[75234]:     </video>
Nov 29 03:01:54 user nova-compute[75234]:     <hostdev supported='yes'>
Nov 29 03:01:54 user nova-compute[75234]:       <enum name='mode'>
Nov 29 03:01:54 user nova-compute[75234]:         <value>subsystem</value>
Nov 29 03:01:54 user nova-compute[75234]:       </enum>
Nov 29 03:01:54 user nova-compute[75234]:       <enum name='startupPolicy'>
Nov 29 03:01:54 user nova-compute[75234]:         <value>default</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>mandatory</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>requisite</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>optional</value>
Nov 29 03:01:54 user nova-compute[75234]:       </enum>
Nov 29 03:01:54 user nova-compute[75234]:       <enum name='subsysType'>
Nov 29 03:01:54 user nova-compute[75234]:         <value>usb</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>pci</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>scsi</value>
Nov 29 03:01:54 user nova-compute[75234]:       </enum>
Nov 29 03:01:54 user nova-compute[75234]:       <enum name='capsType'/>
Nov 29 03:01:54 user nova-compute[75234]:       <enum name='pciBackend'/>
Nov 29 03:01:54 user nova-compute[75234]:     </hostdev>
Nov 29 03:01:54 user nova-compute[75234]:     <rng supported='yes'>
Nov 29 03:01:54 user nova-compute[75234]:       <enum name='model'>
Nov 29 03:01:54 user nova-compute[75234]:         <value>virtio</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>virtio-transitional</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>virtio-non-transitional</value>
Nov 29 03:01:54 user nova-compute[75234]:       </enum>
Nov 29 03:01:54 user nova-compute[75234]:       <enum name='backendModel'>
Nov 29 03:01:54 user nova-compute[75234]:         <value>random</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>egd</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>builtin</value>
Nov 29 03:01:54 user nova-compute[75234]:       </enum>
Nov 29 03:01:54 user nova-compute[75234]:     </rng>
Nov 29 03:01:54 user nova-compute[75234]:     <filesystem supported='yes'>
Nov 29 03:01:54 user nova-compute[75234]:       <enum name='driverType'>
Nov 29 03:01:54 user nova-compute[75234]:         <value>path</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>handle</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>virtiofs</value>
Nov 29 03:01:54 user nova-compute[75234]:       </enum>
Nov 29 03:01:54 user nova-compute[75234]:     </filesystem>
Nov 29 03:01:54 user nova-compute[75234]:     <tpm supported='yes'>
Nov 29 03:01:54 user nova-compute[75234]:       <enum name='model'>
Nov 29 03:01:54 user nova-compute[75234]:         <value>tpm-tis</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>tpm-crb</value>
Nov 29 03:01:54 user nova-compute[75234]:       </enum>
Nov 29 03:01:54 user nova-compute[75234]:       <enum name='backendModel'>
Nov 29 03:01:54 user nova-compute[75234]:         <value>passthrough</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>emulator</value>
Nov 29 03:01:54 user nova-compute[75234]:       </enum>
Nov 29 03:01:54 user nova-compute[75234]:     </tpm>
Nov 29 03:01:54 user nova-compute[75234]:   </devices>
Nov 29 03:01:54 user nova-compute[75234]:   <features>
Nov 29 03:01:54 user nova-compute[75234]:     <gic supported='no'/>
Nov 29 03:01:54 user nova-compute[75234]:     <vmcoreinfo supported='yes'/>
Nov 29 03:01:54 user nova-compute[75234]:     <genid supported='yes'/>
Nov 29 03:01:54 user nova-compute[75234]:     <backingStoreInput supported='yes'/>
Nov 29 03:01:54 user nova-compute[75234]:     <backup supported='yes'/>
Nov 29 03:01:54 user nova-compute[75234]:     <sev supported='no'/>
Nov 29 03:01:54 user nova-compute[75234]:   </features>
Nov 29 03:01:54 user nova-compute[75234]: </domainCapabilities>
Nov 29 03:01:54 user nova-compute[75234]:  {{(pid=75234) _get_domain_capabilities /opt/stack/nova/nova/virt/libvirt/host.py:1039}}
Nov 29 03:01:54 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-ef65ec89-6c59-4f39-9adc-df4888ffd41b None None] Libvirt host hypervisor capabilities for arch=x86_64 and machine_type=ubuntu:
Nov 29 03:01:54 user nova-compute[75234]: <domainCapabilities>
Nov 29 03:01:54 user nova-compute[75234]:   <path>/usr/bin/qemu-system-x86_64</path>
Nov 29 03:01:54 user nova-compute[75234]:   <domain>kvm</domain>
Nov 29 03:01:54 user nova-compute[75234]:   <machine>pc-i440fx-jammy</machine>
Nov 29 03:01:54 user nova-compute[75234]:   <arch>x86_64</arch>
Nov 29 03:01:54 user nova-compute[75234]:   <vcpu max='255'/>
Nov 29 03:01:54 user nova-compute[75234]:   <iothreads supported='yes'/>
Nov 29 03:01:54 user nova-compute[75234]:   <os supported='yes'>
Nov 29 03:01:54 user nova-compute[75234]:     <enum name='firmware'>
Nov 29 03:01:54 user nova-compute[75234]:       <value>efi</value>
Nov 29 03:01:54 user nova-compute[75234]:     </enum>
Nov 29 03:01:54 user nova-compute[75234]:     <loader supported='yes'>
Nov 29 03:01:54 user nova-compute[75234]:       <value>/usr/share/OVMF/OVMF_CODE_4M.fd</value>
Nov 29 03:01:54 user nova-compute[75234]:       <enum name='type'>
Nov 29 03:01:54 user nova-compute[75234]:         <value>rom</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>pflash</value>
Nov 29 03:01:54 user nova-compute[75234]:       </enum>
Nov 29 03:01:54 user nova-compute[75234]:       <enum name='readonly'>
Nov 29 03:01:54 user nova-compute[75234]:         <value>yes</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>no</value>
Nov 29 03:01:54 user nova-compute[75234]:       </enum>
Nov 29 03:01:54 user nova-compute[75234]:       <enum name='secure'>
Nov 29 03:01:54 user nova-compute[75234]:         <value>no</value>
Nov 29 03:01:54 user nova-compute[75234]:       </enum>
Nov 29 03:01:54 user nova-compute[75234]:     </loader>
Nov 29 03:01:54 user nova-compute[75234]:   </os>
Nov 29 03:01:54 user nova-compute[75234]:   <cpu>
Nov 29 03:01:54 user nova-compute[75234]:     <mode name='host-passthrough' supported='yes'>
Nov 29 03:01:54 user nova-compute[75234]:       <enum name='hostPassthroughMigratable'>
Nov 29 03:01:54 user nova-compute[75234]:         <value>on</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>off</value>
Nov 29 03:01:54 user nova-compute[75234]:       </enum>
Nov 29 03:01:54 user nova-compute[75234]:     </mode>
Nov 29 03:01:54 user nova-compute[75234]:     <mode name='maximum' supported='yes'>
Nov 29 03:01:54 user nova-compute[75234]:       <enum name='maximumMigratable'>
Nov 29 03:01:54 user nova-compute[75234]:         <value>on</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>off</value>
Nov 29 03:01:54 user nova-compute[75234]:       </enum>
Nov 29 03:01:54 user nova-compute[75234]:     </mode>
Nov 29 03:01:54 user nova-compute[75234]:     <mode name='host-model' supported='yes'>
Nov 29 03:01:54 user nova-compute[75234]:       <model fallback='forbid'>IvyBridge-IBRS</model>
Nov 29 03:01:54 user nova-compute[75234]:       <vendor>Intel</vendor>
Nov 29 03:01:54 user nova-compute[75234]:       <feature policy='require' name='ss'/>
Nov 29 03:01:54 user nova-compute[75234]:       <feature policy='require' name='vmx'/>
Nov 29 03:01:54 user nova-compute[75234]:       <feature policy='require' name='pcid'/>
Nov 29 03:01:54 user nova-compute[75234]:       <feature policy='require' name='hypervisor'/>
Nov 29 03:01:54 user nova-compute[75234]:       <feature policy='require' name='arat'/>
Nov 29 03:01:54 user nova-compute[75234]:       <feature policy='require' name='tsc_adjust'/>
Nov 29 03:01:54 user nova-compute[75234]:       <feature policy='require' name='umip'/>
Nov 29 03:01:54 user nova-compute[75234]:       <feature policy='require' name='md-clear'/>
Nov 29 03:01:54 user nova-compute[75234]:       <feature policy='require' name='stibp'/>
Nov 29 03:01:54 user nova-compute[75234]:       <feature policy='require' name='arch-capabilities'/>
Nov 29 03:01:54 user nova-compute[75234]:       <feature policy='require' name='ssbd'/>
Nov 29 03:01:54 user nova-compute[75234]:       <feature policy='require' name='invtsc'/>
Nov 29 03:01:54 user nova-compute[75234]:       <feature policy='require' name='ibpb'/>
Nov 29 03:01:54 user nova-compute[75234]:       <feature policy='require' name='ibrs'/>
Nov 29 03:01:54 user nova-compute[75234]:       <feature policy='require' name='amd-stibp'/>
Nov 29 03:01:54 user nova-compute[75234]:       <feature policy='require' name='amd-ssbd'/>
Nov 29 03:01:54 user nova-compute[75234]:       <feature policy='require' name='rsba'/>
Nov 29 03:01:54 user nova-compute[75234]:       <feature policy='require' name='skip-l1dfl-vmentry'/>
Nov 29 03:01:54 user nova-compute[75234]:       <feature policy='require' name='pschange-mc-no'/>
Nov 29 03:01:54 user nova-compute[75234]:       <feature policy='disable' name='erms'/>
Nov 29 03:01:54 user nova-compute[75234]:       <feature policy='disable' name='xsaveopt'/>
Nov 29 03:01:54 user nova-compute[75234]:     </mode>
Nov 29 03:01:54 user nova-compute[75234]:     <mode name='custom' supported='yes'>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>qemu64</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='yes'>qemu32</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>phenom</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='yes'>pentium3</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='yes'>pentium2</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='yes'>pentium</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>n270</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='yes'>kvm64</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='yes'>kvm32</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='yes'>coreduo</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='yes'>core2duo</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>athlon</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='yes'>Westmere-IBRS</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='yes'>Westmere</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Snowridge</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Skylake-Server-noTSX-IBRS</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Skylake-Server-IBRS</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Skylake-Server</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Skylake-Client-noTSX-IBRS</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Skylake-Client-IBRS</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Skylake-Client</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>SandyBridge-IBRS</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>SandyBridge</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='yes'>Penryn</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Opteron_G5</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Opteron_G4</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Opteron_G3</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Opteron_G2</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='yes'>Opteron_G1</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='yes'>Nehalem-IBRS</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='yes'>Nehalem</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>IvyBridge-IBRS</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>IvyBridge</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Icelake-Server-noTSX</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Icelake-Server</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no' deprecated='yes'>Icelake-Client-noTSX</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no' deprecated='yes'>Icelake-Client</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Haswell-noTSX-IBRS</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Haswell-noTSX</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Haswell-IBRS</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Haswell</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>EPYC-Rome</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>EPYC-Milan</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>EPYC-IBPB</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>EPYC</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Dhyana</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Cooperlake</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='yes'>Conroe</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Cascadelake-Server-noTSX</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Cascadelake-Server</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Broadwell-noTSX-IBRS</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Broadwell-noTSX</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Broadwell-IBRS</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Broadwell</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='yes'>486</model>
Nov 29 03:01:54 user nova-compute[75234]:     </mode>
Nov 29 03:01:54 user nova-compute[75234]:   </cpu>
Nov 29 03:01:54 user nova-compute[75234]:   <memoryBacking supported='yes'>
Nov 29 03:01:54 user nova-compute[75234]:     <enum name='sourceType'>
Nov 29 03:01:54 user nova-compute[75234]:       <value>file</value>
Nov 29 03:01:54 user nova-compute[75234]:       <value>anonymous</value>
Nov 29 03:01:54 user nova-compute[75234]:       <value>memfd</value>
Nov 29 03:01:54 user nova-compute[75234]:     </enum>
Nov 29 03:01:54 user nova-compute[75234]:   </memoryBacking>
Nov 29 03:01:54 user nova-compute[75234]:   <devices>
Nov 29 03:01:54 user nova-compute[75234]:     <disk supported='yes'>
Nov 29 03:01:54 user nova-compute[75234]:       <enum name='diskDevice'>
Nov 29 03:01:54 user nova-compute[75234]:         <value>disk</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>cdrom</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>floppy</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>lun</value>
Nov 29 03:01:54 user nova-compute[75234]:       </enum>
Nov 29 03:01:54 user nova-compute[75234]:       <enum name='bus'>
Nov 29 03:01:54 user nova-compute[75234]:         <value>ide</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>fdc</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>scsi</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>virtio</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>usb</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>sata</value>
Nov 29 03:01:54 user nova-compute[75234]:       </enum>
Nov 29 03:01:54 user nova-compute[75234]:       <enum name='model'>
Nov 29 03:01:54 user nova-compute[75234]:         <value>virtio</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>virtio-transitional</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>virtio-non-transitional</value>
Nov 29 03:01:54 user nova-compute[75234]:       </enum>
Nov 29 03:01:54 user nova-compute[75234]:     </disk>
Nov 29 03:01:54 user nova-compute[75234]:     <graphics supported='yes'>
Nov 29 03:01:54 user nova-compute[75234]:       <enum name='type'>
Nov 29 03:01:54 user nova-compute[75234]:         <value>sdl</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>vnc</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>spice</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>egl-headless</value>
Nov 29 03:01:54 user nova-compute[75234]:       </enum>
Nov 29 03:01:54 user nova-compute[75234]:     </graphics>
Nov 29 03:01:54 user nova-compute[75234]:     <video supported='yes'>
Nov 29 03:01:54 user nova-compute[75234]:       <enum name='modelType'>
Nov 29 03:01:54 user nova-compute[75234]:         <value>vga</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>cirrus</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>vmvga</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>qxl</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>virtio</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>none</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>bochs</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>ramfb</value>
Nov 29 03:01:54 user nova-compute[75234]:       </enum>
Nov 29 03:01:54 user nova-compute[75234]:     </video>
Nov 29 03:01:54 user nova-compute[75234]:     <hostdev supported='yes'>
Nov 29 03:01:54 user nova-compute[75234]:       <enum name='mode'>
Nov 29 03:01:54 user nova-compute[75234]:         <value>subsystem</value>
Nov 29 03:01:54 user nova-compute[75234]:       </enum>
Nov 29 03:01:54 user nova-compute[75234]:       <enum name='startupPolicy'>
Nov 29 03:01:54 user nova-compute[75234]:         <value>default</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>mandatory</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>requisite</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>optional</value>
Nov 29 03:01:54 user nova-compute[75234]:       </enum>
Nov 29 03:01:54 user nova-compute[75234]:       <enum name='subsysType'>
Nov 29 03:01:54 user nova-compute[75234]:         <value>usb</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>pci</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>scsi</value>
Nov 29 03:01:54 user nova-compute[75234]:       </enum>
Nov 29 03:01:54 user nova-compute[75234]:       <enum name='capsType'/>
Nov 29 03:01:54 user nova-compute[75234]:       <enum name='pciBackend'/>
Nov 29 03:01:54 user nova-compute[75234]:     </hostdev>
Nov 29 03:01:54 user nova-compute[75234]:     <rng supported='yes'>
Nov 29 03:01:54 user nova-compute[75234]:       <enum name='model'>
Nov 29 03:01:54 user nova-compute[75234]:         <value>virtio</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>virtio-transitional</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>virtio-non-transitional</value>
Nov 29 03:01:54 user nova-compute[75234]:       </enum>
Nov 29 03:01:54 user nova-compute[75234]:       <enum name='backendModel'>
Nov 29 03:01:54 user nova-compute[75234]:         <value>random</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>egd</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>builtin</value>
Nov 29 03:01:54 user nova-compute[75234]:       </enum>
Nov 29 03:01:54 user nova-compute[75234]:     </rng>
Nov 29 03:01:54 user nova-compute[75234]:     <filesystem supported='yes'>
Nov 29 03:01:54 user nova-compute[75234]:       <enum name='driverType'>
Nov 29 03:01:54 user nova-compute[75234]:         <value>path</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>handle</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>virtiofs</value>
Nov 29 03:01:54 user nova-compute[75234]:       </enum>
Nov 29 03:01:54 user nova-compute[75234]:     </filesystem>
Nov 29 03:01:54 user nova-compute[75234]:     <tpm supported='yes'>
Nov 29 03:01:54 user nova-compute[75234]:       <enum name='model'>
Nov 29 03:01:54 user nova-compute[75234]:         <value>tpm-tis</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>tpm-crb</value>
Nov 29 03:01:54 user nova-compute[75234]:       </enum>
Nov 29 03:01:54 user nova-compute[75234]:       <enum name='backendModel'>
Nov 29 03:01:54 user nova-compute[75234]:         <value>passthrough</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>emulator</value>
Nov 29 03:01:54 user nova-compute[75234]:       </enum>
Nov 29 03:01:54 user nova-compute[75234]:     </tpm>
Nov 29 03:01:54 user nova-compute[75234]:   </devices>
Nov 29 03:01:54 user nova-compute[75234]:   <features>
Nov 29 03:01:54 user nova-compute[75234]:     <gic supported='no'/>
Nov 29 03:01:54 user nova-compute[75234]:     <vmcoreinfo supported='yes'/>
Nov 29 03:01:54 user nova-compute[75234]:     <genid supported='yes'/>
Nov 29 03:01:54 user nova-compute[75234]:     <backingStoreInput supported='yes'/>
Nov 29 03:01:54 user nova-compute[75234]:     <backup supported='yes'/>
Nov 29 03:01:54 user nova-compute[75234]:     <sev supported='no'/>
Nov 29 03:01:54 user nova-compute[75234]:   </features>
Nov 29 03:01:54 user nova-compute[75234]: </domainCapabilities>
Nov 29 03:01:54 user nova-compute[75234]:  {{(pid=75234) _get_domain_capabilities /opt/stack/nova/nova/virt/libvirt/host.py:1039}}
Nov 29 03:01:54 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-ef65ec89-6c59-4f39-9adc-df4888ffd41b None None] Libvirt host hypervisor capabilities for arch=x86_64 and machine_type=q35:
Nov 29 03:01:54 user nova-compute[75234]: <domainCapabilities>
Nov 29 03:01:54 user nova-compute[75234]:   <path>/usr/bin/qemu-system-x86_64</path>
Nov 29 03:01:54 user nova-compute[75234]:   <domain>kvm</domain>
Nov 29 03:01:54 user nova-compute[75234]:   <machine>pc-q35-6.2</machine>
Nov 29 03:01:54 user nova-compute[75234]:   <arch>x86_64</arch>
Nov 29 03:01:54 user nova-compute[75234]:   <vcpu max='288'/>
Nov 29 03:01:54 user nova-compute[75234]:   <iothreads supported='yes'/>
Nov 29 03:01:54 user nova-compute[75234]:   <os supported='yes'>
Nov 29 03:01:54 user nova-compute[75234]:     <enum name='firmware'>
Nov 29 03:01:54 user nova-compute[75234]:       <value>efi</value>
Nov 29 03:01:54 user nova-compute[75234]:     </enum>
Nov 29 03:01:54 user nova-compute[75234]:     <loader supported='yes'>
Nov 29 03:01:54 user nova-compute[75234]:       <value>/usr/share/OVMF/OVMF_CODE_4M.ms.fd</value>
Nov 29 03:01:54 user nova-compute[75234]:       <value>/usr/share/OVMF/OVMF_CODE_4M.secboot.fd</value>
Nov 29 03:01:54 user nova-compute[75234]:       <value>/usr/share/OVMF/OVMF_CODE_4M.fd</value>
Nov 29 03:01:54 user nova-compute[75234]:       <enum name='type'>
Nov 29 03:01:54 user nova-compute[75234]:         <value>rom</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>pflash</value>
Nov 29 03:01:54 user nova-compute[75234]:       </enum>
Nov 29 03:01:54 user nova-compute[75234]:       <enum name='readonly'>
Nov 29 03:01:54 user nova-compute[75234]:         <value>yes</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>no</value>
Nov 29 03:01:54 user nova-compute[75234]:       </enum>
Nov 29 03:01:54 user nova-compute[75234]:       <enum name='secure'>
Nov 29 03:01:54 user nova-compute[75234]:         <value>yes</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>no</value>
Nov 29 03:01:54 user nova-compute[75234]:       </enum>
Nov 29 03:01:54 user nova-compute[75234]:     </loader>
Nov 29 03:01:54 user nova-compute[75234]:   </os>
Nov 29 03:01:54 user nova-compute[75234]:   <cpu>
Nov 29 03:01:54 user nova-compute[75234]:     <mode name='host-passthrough' supported='yes'>
Nov 29 03:01:54 user nova-compute[75234]:       <enum name='hostPassthroughMigratable'>
Nov 29 03:01:54 user nova-compute[75234]:         <value>on</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>off</value>
Nov 29 03:01:54 user nova-compute[75234]:       </enum>
Nov 29 03:01:54 user nova-compute[75234]:     </mode>
Nov 29 03:01:54 user nova-compute[75234]:     <mode name='maximum' supported='yes'>
Nov 29 03:01:54 user nova-compute[75234]:       <enum name='maximumMigratable'>
Nov 29 03:01:54 user nova-compute[75234]:         <value>on</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>off</value>
Nov 29 03:01:54 user nova-compute[75234]:       </enum>
Nov 29 03:01:54 user nova-compute[75234]:     </mode>
Nov 29 03:01:54 user nova-compute[75234]:     <mode name='host-model' supported='yes'>
Nov 29 03:01:54 user nova-compute[75234]:       <model fallback='forbid'>IvyBridge-IBRS</model>
Nov 29 03:01:54 user nova-compute[75234]:       <vendor>Intel</vendor>
Nov 29 03:01:54 user nova-compute[75234]:       <feature policy='require' name='ss'/>
Nov 29 03:01:54 user nova-compute[75234]:       <feature policy='require' name='vmx'/>
Nov 29 03:01:54 user nova-compute[75234]:       <feature policy='require' name='pcid'/>
Nov 29 03:01:54 user nova-compute[75234]:       <feature policy='require' name='hypervisor'/>
Nov 29 03:01:54 user nova-compute[75234]:       <feature policy='require' name='arat'/>
Nov 29 03:01:54 user nova-compute[75234]:       <feature policy='require' name='tsc_adjust'/>
Nov 29 03:01:54 user nova-compute[75234]:       <feature policy='require' name='umip'/>
Nov 29 03:01:54 user nova-compute[75234]:       <feature policy='require' name='md-clear'/>
Nov 29 03:01:54 user nova-compute[75234]:       <feature policy='require' name='stibp'/>
Nov 29 03:01:54 user nova-compute[75234]:       <feature policy='require' name='arch-capabilities'/>
Nov 29 03:01:54 user nova-compute[75234]:       <feature policy='require' name='ssbd'/>
Nov 29 03:01:54 user nova-compute[75234]:       <feature policy='require' name='invtsc'/>
Nov 29 03:01:54 user nova-compute[75234]:       <feature policy='require' name='ibpb'/>
Nov 29 03:01:54 user nova-compute[75234]:       <feature policy='require' name='ibrs'/>
Nov 29 03:01:54 user nova-compute[75234]:       <feature policy='require' name='amd-stibp'/>
Nov 29 03:01:54 user nova-compute[75234]:       <feature policy='require' name='amd-ssbd'/>
Nov 29 03:01:54 user nova-compute[75234]:       <feature policy='require' name='rsba'/>
Nov 29 03:01:54 user nova-compute[75234]:       <feature policy='require' name='skip-l1dfl-vmentry'/>
Nov 29 03:01:54 user nova-compute[75234]:       <feature policy='require' name='pschange-mc-no'/>
Nov 29 03:01:54 user nova-compute[75234]:       <feature policy='disable' name='erms'/>
Nov 29 03:01:54 user nova-compute[75234]:       <feature policy='disable' name='xsaveopt'/>
Nov 29 03:01:54 user nova-compute[75234]:     </mode>
Nov 29 03:01:54 user nova-compute[75234]:     <mode name='custom' supported='yes'>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>qemu64</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='yes'>qemu32</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>phenom</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='yes'>pentium3</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='yes'>pentium2</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='yes'>pentium</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>n270</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='yes'>kvm64</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='yes'>kvm32</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='yes'>coreduo</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='yes'>core2duo</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>athlon</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='yes'>Westmere-IBRS</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='yes'>Westmere</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Snowridge</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Skylake-Server-noTSX-IBRS</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Skylake-Server-IBRS</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Skylake-Server</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Skylake-Client-noTSX-IBRS</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Skylake-Client-IBRS</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Skylake-Client</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>SandyBridge-IBRS</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>SandyBridge</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='yes'>Penryn</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Opteron_G5</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Opteron_G4</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Opteron_G3</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Opteron_G2</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='yes'>Opteron_G1</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='yes'>Nehalem-IBRS</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='yes'>Nehalem</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>IvyBridge-IBRS</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>IvyBridge</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Icelake-Server-noTSX</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Icelake-Server</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no' deprecated='yes'>Icelake-Client-noTSX</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no' deprecated='yes'>Icelake-Client</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Haswell-noTSX-IBRS</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Haswell-noTSX</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Haswell-IBRS</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Haswell</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>EPYC-Rome</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>EPYC-Milan</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>EPYC-IBPB</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>EPYC</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Dhyana</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Cooperlake</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='yes'>Conroe</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Cascadelake-Server-noTSX</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Cascadelake-Server</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Broadwell-noTSX-IBRS</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Broadwell-noTSX</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Broadwell-IBRS</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Broadwell</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='yes'>486</model>
Nov 29 03:01:54 user nova-compute[75234]:     </mode>
Nov 29 03:01:54 user nova-compute[75234]:   </cpu>
Nov 29 03:01:54 user nova-compute[75234]:   <memoryBacking supported='yes'>
Nov 29 03:01:54 user nova-compute[75234]:     <enum name='sourceType'>
Nov 29 03:01:54 user nova-compute[75234]:       <value>file</value>
Nov 29 03:01:54 user nova-compute[75234]:       <value>anonymous</value>
Nov 29 03:01:54 user nova-compute[75234]:       <value>memfd</value>
Nov 29 03:01:54 user nova-compute[75234]:     </enum>
Nov 29 03:01:54 user nova-compute[75234]:   </memoryBacking>
Nov 29 03:01:54 user nova-compute[75234]:   <devices>
Nov 29 03:01:54 user nova-compute[75234]:     <disk supported='yes'>
Nov 29 03:01:54 user nova-compute[75234]:       <enum name='diskDevice'>
Nov 29 03:01:54 user nova-compute[75234]:         <value>disk</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>cdrom</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>floppy</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>lun</value>
Nov 29 03:01:54 user nova-compute[75234]:       </enum>
Nov 29 03:01:54 user nova-compute[75234]:       <enum name='bus'>
Nov 29 03:01:54 user nova-compute[75234]:         <value>fdc</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>scsi</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>virtio</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>usb</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>sata</value>
Nov 29 03:01:54 user nova-compute[75234]:       </enum>
Nov 29 03:01:54 user nova-compute[75234]:       <enum name='model'>
Nov 29 03:01:54 user nova-compute[75234]:         <value>virtio</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>virtio-transitional</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>virtio-non-transitional</value>
Nov 29 03:01:54 user nova-compute[75234]:       </enum>
Nov 29 03:01:54 user nova-compute[75234]:     </disk>
Nov 29 03:01:54 user nova-compute[75234]:     <graphics supported='yes'>
Nov 29 03:01:54 user nova-compute[75234]:       <enum name='type'>
Nov 29 03:01:54 user nova-compute[75234]:         <value>sdl</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>vnc</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>spice</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>egl-headless</value>
Nov 29 03:01:54 user nova-compute[75234]:       </enum>
Nov 29 03:01:54 user nova-compute[75234]:     </graphics>
Nov 29 03:01:54 user nova-compute[75234]:     <video supported='yes'>
Nov 29 03:01:54 user nova-compute[75234]:       <enum name='modelType'>
Nov 29 03:01:54 user nova-compute[75234]:         <value>vga</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>cirrus</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>vmvga</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>qxl</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>virtio</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>none</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>bochs</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>ramfb</value>
Nov 29 03:01:54 user nova-compute[75234]:       </enum>
Nov 29 03:01:54 user nova-compute[75234]:     </video>
Nov 29 03:01:54 user nova-compute[75234]:     <hostdev supported='yes'>
Nov 29 03:01:54 user nova-compute[75234]:       <enum name='mode'>
Nov 29 03:01:54 user nova-compute[75234]:         <value>subsystem</value>
Nov 29 03:01:54 user nova-compute[75234]:       </enum>
Nov 29 03:01:54 user nova-compute[75234]:       <enum name='startupPolicy'>
Nov 29 03:01:54 user nova-compute[75234]:         <value>default</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>mandatory</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>requisite</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>optional</value>
Nov 29 03:01:54 user nova-compute[75234]:       </enum>
Nov 29 03:01:54 user nova-compute[75234]:       <enum name='subsysType'>
Nov 29 03:01:54 user nova-compute[75234]:         <value>usb</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>pci</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>scsi</value>
Nov 29 03:01:54 user nova-compute[75234]:       </enum>
Nov 29 03:01:54 user nova-compute[75234]:       <enum name='capsType'/>
Nov 29 03:01:54 user nova-compute[75234]:       <enum name='pciBackend'/>
Nov 29 03:01:54 user nova-compute[75234]:     </hostdev>
Nov 29 03:01:54 user nova-compute[75234]:     <rng supported='yes'>
Nov 29 03:01:54 user nova-compute[75234]:       <enum name='model'>
Nov 29 03:01:54 user nova-compute[75234]:         <value>virtio</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>virtio-transitional</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>virtio-non-transitional</value>
Nov 29 03:01:54 user nova-compute[75234]:       </enum>
Nov 29 03:01:54 user nova-compute[75234]:       <enum name='backendModel'>
Nov 29 03:01:54 user nova-compute[75234]:         <value>random</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>egd</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>builtin</value>
Nov 29 03:01:54 user nova-compute[75234]:       </enum>
Nov 29 03:01:54 user nova-compute[75234]:     </rng>
Nov 29 03:01:54 user nova-compute[75234]:     <filesystem supported='yes'>
Nov 29 03:01:54 user nova-compute[75234]:       <enum name='driverType'>
Nov 29 03:01:54 user nova-compute[75234]:         <value>path</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>handle</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>virtiofs</value>
Nov 29 03:01:54 user nova-compute[75234]:       </enum>
Nov 29 03:01:54 user nova-compute[75234]:     </filesystem>
Nov 29 03:01:54 user nova-compute[75234]:     <tpm supported='yes'>
Nov 29 03:01:54 user nova-compute[75234]:       <enum name='model'>
Nov 29 03:01:54 user nova-compute[75234]:         <value>tpm-tis</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>tpm-crb</value>
Nov 29 03:01:54 user nova-compute[75234]:       </enum>
Nov 29 03:01:54 user nova-compute[75234]:       <enum name='backendModel'>
Nov 29 03:01:54 user nova-compute[75234]:         <value>passthrough</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>emulator</value>
Nov 29 03:01:54 user nova-compute[75234]:       </enum>
Nov 29 03:01:54 user nova-compute[75234]:     </tpm>
Nov 29 03:01:54 user nova-compute[75234]:   </devices>
Nov 29 03:01:54 user nova-compute[75234]:   <features>
Nov 29 03:01:54 user nova-compute[75234]:     <gic supported='no'/>
Nov 29 03:01:54 user nova-compute[75234]:     <vmcoreinfo supported='yes'/>
Nov 29 03:01:54 user nova-compute[75234]:     <genid supported='yes'/>
Nov 29 03:01:54 user nova-compute[75234]:     <backingStoreInput supported='yes'/>
Nov 29 03:01:54 user nova-compute[75234]:     <backup supported='yes'/>
Nov 29 03:01:54 user nova-compute[75234]:     <sev supported='no'/>
Nov 29 03:01:54 user nova-compute[75234]:   </features>
Nov 29 03:01:54 user nova-compute[75234]: </domainCapabilities>
Nov 29 03:01:54 user nova-compute[75234]:  {{(pid=75234) _get_domain_capabilities /opt/stack/nova/nova/virt/libvirt/host.py:1039}}
Nov 29 03:01:54 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-ef65ec89-6c59-4f39-9adc-df4888ffd41b None None] Libvirt host hypervisor capabilities for arch=x86_64 and machine_type=pc:
Nov 29 03:01:54 user nova-compute[75234]: <domainCapabilities>
Nov 29 03:01:54 user nova-compute[75234]:   <path>/usr/bin/qemu-system-x86_64</path>
Nov 29 03:01:54 user nova-compute[75234]:   <domain>kvm</domain>
Nov 29 03:01:54 user nova-compute[75234]:   <machine>pc-i440fx-6.2</machine>
Nov 29 03:01:54 user nova-compute[75234]:   <arch>x86_64</arch>
Nov 29 03:01:54 user nova-compute[75234]:   <vcpu max='255'/>
Nov 29 03:01:54 user nova-compute[75234]:   <iothreads supported='yes'/>
Nov 29 03:01:54 user nova-compute[75234]:   <os supported='yes'>
Nov 29 03:01:54 user nova-compute[75234]:     <enum name='firmware'>
Nov 29 03:01:54 user nova-compute[75234]:       <value>efi</value>
Nov 29 03:01:54 user nova-compute[75234]:     </enum>
Nov 29 03:01:54 user nova-compute[75234]:     <loader supported='yes'>
Nov 29 03:01:54 user nova-compute[75234]:       <value>/usr/share/OVMF/OVMF_CODE_4M.fd</value>
Nov 29 03:01:54 user nova-compute[75234]:       <enum name='type'>
Nov 29 03:01:54 user nova-compute[75234]:         <value>rom</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>pflash</value>
Nov 29 03:01:54 user nova-compute[75234]:       </enum>
Nov 29 03:01:54 user nova-compute[75234]:       <enum name='readonly'>
Nov 29 03:01:54 user nova-compute[75234]:         <value>yes</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>no</value>
Nov 29 03:01:54 user nova-compute[75234]:       </enum>
Nov 29 03:01:54 user nova-compute[75234]:       <enum name='secure'>
Nov 29 03:01:54 user nova-compute[75234]:         <value>no</value>
Nov 29 03:01:54 user nova-compute[75234]:       </enum>
Nov 29 03:01:54 user nova-compute[75234]:     </loader>
Nov 29 03:01:54 user nova-compute[75234]:   </os>
Nov 29 03:01:54 user nova-compute[75234]:   <cpu>
Nov 29 03:01:54 user nova-compute[75234]:     <mode name='host-passthrough' supported='yes'>
Nov 29 03:01:54 user nova-compute[75234]:       <enum name='hostPassthroughMigratable'>
Nov 29 03:01:54 user nova-compute[75234]:         <value>on</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>off</value>
Nov 29 03:01:54 user nova-compute[75234]:       </enum>
Nov 29 03:01:54 user nova-compute[75234]:     </mode>
Nov 29 03:01:54 user nova-compute[75234]:     <mode name='maximum' supported='yes'>
Nov 29 03:01:54 user nova-compute[75234]:       <enum name='maximumMigratable'>
Nov 29 03:01:54 user nova-compute[75234]:         <value>on</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>off</value>
Nov 29 03:01:54 user nova-compute[75234]:       </enum>
Nov 29 03:01:54 user nova-compute[75234]:     </mode>
Nov 29 03:01:54 user nova-compute[75234]:     <mode name='host-model' supported='yes'>
Nov 29 03:01:54 user nova-compute[75234]:       <model fallback='forbid'>IvyBridge-IBRS</model>
Nov 29 03:01:54 user nova-compute[75234]:       <vendor>Intel</vendor>
Nov 29 03:01:54 user nova-compute[75234]:       <feature policy='require' name='ss'/>
Nov 29 03:01:54 user nova-compute[75234]:       <feature policy='require' name='vmx'/>
Nov 29 03:01:54 user nova-compute[75234]:       <feature policy='require' name='pcid'/>
Nov 29 03:01:54 user nova-compute[75234]:       <feature policy='require' name='hypervisor'/>
Nov 29 03:01:54 user nova-compute[75234]:       <feature policy='require' name='arat'/>
Nov 29 03:01:54 user nova-compute[75234]:       <feature policy='require' name='tsc_adjust'/>
Nov 29 03:01:54 user nova-compute[75234]:       <feature policy='require' name='umip'/>
Nov 29 03:01:54 user nova-compute[75234]:       <feature policy='require' name='md-clear'/>
Nov 29 03:01:54 user nova-compute[75234]:       <feature policy='require' name='stibp'/>
Nov 29 03:01:54 user nova-compute[75234]:       <feature policy='require' name='arch-capabilities'/>
Nov 29 03:01:54 user nova-compute[75234]:       <feature policy='require' name='ssbd'/>
Nov 29 03:01:54 user nova-compute[75234]:       <feature policy='require' name='invtsc'/>
Nov 29 03:01:54 user nova-compute[75234]:       <feature policy='require' name='ibpb'/>
Nov 29 03:01:54 user nova-compute[75234]:       <feature policy='require' name='ibrs'/>
Nov 29 03:01:54 user nova-compute[75234]:       <feature policy='require' name='amd-stibp'/>
Nov 29 03:01:54 user nova-compute[75234]:       <feature policy='require' name='amd-ssbd'/>
Nov 29 03:01:54 user nova-compute[75234]:       <feature policy='require' name='rsba'/>
Nov 29 03:01:54 user nova-compute[75234]:       <feature policy='require' name='skip-l1dfl-vmentry'/>
Nov 29 03:01:54 user nova-compute[75234]:       <feature policy='require' name='pschange-mc-no'/>
Nov 29 03:01:54 user nova-compute[75234]:       <feature policy='disable' name='erms'/>
Nov 29 03:01:54 user nova-compute[75234]:       <feature policy='disable' name='xsaveopt'/>
Nov 29 03:01:54 user nova-compute[75234]:     </mode>
Nov 29 03:01:54 user nova-compute[75234]:     <mode name='custom' supported='yes'>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>qemu64</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='yes'>qemu32</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>phenom</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='yes'>pentium3</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='yes'>pentium2</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='yes'>pentium</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>n270</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='yes'>kvm64</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='yes'>kvm32</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='yes'>coreduo</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='yes'>core2duo</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>athlon</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='yes'>Westmere-IBRS</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='yes'>Westmere</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Snowridge</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Skylake-Server-noTSX-IBRS</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Skylake-Server-IBRS</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Skylake-Server</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Skylake-Client-noTSX-IBRS</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Skylake-Client-IBRS</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Skylake-Client</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>SandyBridge-IBRS</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>SandyBridge</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='yes'>Penryn</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Opteron_G5</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Opteron_G4</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Opteron_G3</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Opteron_G2</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='yes'>Opteron_G1</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='yes'>Nehalem-IBRS</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='yes'>Nehalem</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>IvyBridge-IBRS</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>IvyBridge</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Icelake-Server-noTSX</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Icelake-Server</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no' deprecated='yes'>Icelake-Client-noTSX</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no' deprecated='yes'>Icelake-Client</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Haswell-noTSX-IBRS</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Haswell-noTSX</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Haswell-IBRS</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Haswell</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>EPYC-Rome</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>EPYC-Milan</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>EPYC-IBPB</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>EPYC</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Dhyana</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Cooperlake</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='yes'>Conroe</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Cascadelake-Server-noTSX</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Cascadelake-Server</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Broadwell-noTSX-IBRS</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Broadwell-noTSX</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Broadwell-IBRS</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='no'>Broadwell</model>
Nov 29 03:01:54 user nova-compute[75234]:       <model usable='yes'>486</model>
Nov 29 03:01:54 user nova-compute[75234]:     </mode>
Nov 29 03:01:54 user nova-compute[75234]:   </cpu>
Nov 29 03:01:54 user nova-compute[75234]:   <memoryBacking supported='yes'>
Nov 29 03:01:54 user nova-compute[75234]:     <enum name='sourceType'>
Nov 29 03:01:54 user nova-compute[75234]:       <value>file</value>
Nov 29 03:01:54 user nova-compute[75234]:       <value>anonymous</value>
Nov 29 03:01:54 user nova-compute[75234]:       <value>memfd</value>
Nov 29 03:01:54 user nova-compute[75234]:     </enum>
Nov 29 03:01:54 user nova-compute[75234]:   </memoryBacking>
Nov 29 03:01:54 user nova-compute[75234]:   <devices>
Nov 29 03:01:54 user nova-compute[75234]:     <disk supported='yes'>
Nov 29 03:01:54 user nova-compute[75234]:       <enum name='diskDevice'>
Nov 29 03:01:54 user nova-compute[75234]:         <value>disk</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>cdrom</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>floppy</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>lun</value>
Nov 29 03:01:54 user nova-compute[75234]:       </enum>
Nov 29 03:01:54 user nova-compute[75234]:       <enum name='bus'>
Nov 29 03:01:54 user nova-compute[75234]:         <value>ide</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>fdc</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>scsi</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>virtio</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>usb</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>sata</value>
Nov 29 03:01:54 user nova-compute[75234]:       </enum>
Nov 29 03:01:54 user nova-compute[75234]:       <enum name='model'>
Nov 29 03:01:54 user nova-compute[75234]:         <value>virtio</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>virtio-transitional</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>virtio-non-transitional</value>
Nov 29 03:01:54 user nova-compute[75234]:       </enum>
Nov 29 03:01:54 user nova-compute[75234]:     </disk>
Nov 29 03:01:54 user nova-compute[75234]:     <graphics supported='yes'>
Nov 29 03:01:54 user nova-compute[75234]:       <enum name='type'>
Nov 29 03:01:54 user nova-compute[75234]:         <value>sdl</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>vnc</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>spice</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>egl-headless</value>
Nov 29 03:01:54 user nova-compute[75234]:       </enum>
Nov 29 03:01:54 user nova-compute[75234]:     </graphics>
Nov 29 03:01:54 user nova-compute[75234]:     <video supported='yes'>
Nov 29 03:01:54 user nova-compute[75234]:       <enum name='modelType'>
Nov 29 03:01:54 user nova-compute[75234]:         <value>vga</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>cirrus</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>vmvga</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>qxl</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>virtio</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>none</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>bochs</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>ramfb</value>
Nov 29 03:01:54 user nova-compute[75234]:       </enum>
Nov 29 03:01:54 user nova-compute[75234]:     </video>
Nov 29 03:01:54 user nova-compute[75234]:     <hostdev supported='yes'>
Nov 29 03:01:54 user nova-compute[75234]:       <enum name='mode'>
Nov 29 03:01:54 user nova-compute[75234]:         <value>subsystem</value>
Nov 29 03:01:54 user nova-compute[75234]:       </enum>
Nov 29 03:01:54 user nova-compute[75234]:       <enum name='startupPolicy'>
Nov 29 03:01:54 user nova-compute[75234]:         <value>default</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>mandatory</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>requisite</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>optional</value>
Nov 29 03:01:54 user nova-compute[75234]:       </enum>
Nov 29 03:01:54 user nova-compute[75234]:       <enum name='subsysType'>
Nov 29 03:01:54 user nova-compute[75234]:         <value>usb</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>pci</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>scsi</value>
Nov 29 03:01:54 user nova-compute[75234]:       </enum>
Nov 29 03:01:54 user nova-compute[75234]:       <enum name='capsType'/>
Nov 29 03:01:54 user nova-compute[75234]:       <enum name='pciBackend'/>
Nov 29 03:01:54 user nova-compute[75234]:     </hostdev>
Nov 29 03:01:54 user nova-compute[75234]:     <rng supported='yes'>
Nov 29 03:01:54 user nova-compute[75234]:       <enum name='model'>
Nov 29 03:01:54 user nova-compute[75234]:         <value>virtio</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>virtio-transitional</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>virtio-non-transitional</value>
Nov 29 03:01:54 user nova-compute[75234]:       </enum>
Nov 29 03:01:54 user nova-compute[75234]:       <enum name='backendModel'>
Nov 29 03:01:54 user nova-compute[75234]:         <value>random</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>egd</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>builtin</value>
Nov 29 03:01:54 user nova-compute[75234]:       </enum>
Nov 29 03:01:54 user nova-compute[75234]:     </rng>
Nov 29 03:01:54 user nova-compute[75234]:     <filesystem supported='yes'>
Nov 29 03:01:54 user nova-compute[75234]:       <enum name='driverType'>
Nov 29 03:01:54 user nova-compute[75234]:         <value>path</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>handle</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>virtiofs</value>
Nov 29 03:01:54 user nova-compute[75234]:       </enum>
Nov 29 03:01:54 user nova-compute[75234]:     </filesystem>
Nov 29 03:01:54 user nova-compute[75234]:     <tpm supported='yes'>
Nov 29 03:01:54 user nova-compute[75234]:       <enum name='model'>
Nov 29 03:01:54 user nova-compute[75234]:         <value>tpm-tis</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>tpm-crb</value>
Nov 29 03:01:54 user nova-compute[75234]:       </enum>
Nov 29 03:01:54 user nova-compute[75234]:       <enum name='backendModel'>
Nov 29 03:01:54 user nova-compute[75234]:         <value>passthrough</value>
Nov 29 03:01:54 user nova-compute[75234]:         <value>emulator</value>
Nov 29 03:01:54 user nova-compute[75234]:       </enum>
Nov 29 03:01:54 user nova-compute[75234]:     </tpm>
Nov 29 03:01:54 user nova-compute[75234]:   </devices>
Nov 29 03:01:54 user nova-compute[75234]:   <features>
Nov 29 03:01:54 user nova-compute[75234]:     <gic supported='no'/>
Nov 29 03:01:54 user nova-compute[75234]:     <vmcoreinfo supported='yes'/>
Nov 29 03:01:54 user nova-compute[75234]:     <genid supported='yes'/>
Nov 29 03:01:54 user nova-compute[75234]:     <backingStoreInput supported='yes'/>
Nov 29 03:01:54 user nova-compute[75234]:     <backup supported='yes'/>
Nov 29 03:01:54 user nova-compute[75234]:     <sev supported='no'/>
Nov 29 03:01:54 user nova-compute[75234]:   </features>
Nov 29 03:01:54 user nova-compute[75234]: </domainCapabilities>
Nov 29 03:01:54 user nova-compute[75234]:  {{(pid=75234) _get_domain_capabilities /opt/stack/nova/nova/virt/libvirt/host.py:1039}}
Nov 29 03:01:54 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-ef65ec89-6c59-4f39-9adc-df4888ffd41b None None] Getting domain capabilities for xtensa via machine types: {None} {{(pid=75234) _get_machine_types /opt/stack/nova/nova/virt/libvirt/host.py:954}}
Nov 29 03:01:54 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-ef65ec89-6c59-4f39-9adc-df4888ffd41b None None] Error from libvirt when retrieving domain capabilities for arch xtensa / virt_type kvm / machine_type None: [Error Code 8]: invalid argument: KVM is not supported by '/usr/bin/qemu-system-xtensa' on this host {{(pid=75234) _add_to_domain_capabilities /opt/stack/nova/nova/virt/libvirt/host.py:993}}
Nov 29 03:01:54 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-ef65ec89-6c59-4f39-9adc-df4888ffd41b None None] Getting domain capabilities for xtensaeb via machine types: {None} {{(pid=75234) _get_machine_types /opt/stack/nova/nova/virt/libvirt/host.py:954}}
Nov 29 03:01:54 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-ef65ec89-6c59-4f39-9adc-df4888ffd41b None None] Error from libvirt when retrieving domain capabilities for arch xtensaeb / virt_type kvm / machine_type None: [Error Code 8]: invalid argument: KVM is not supported by '/usr/bin/qemu-system-xtensaeb' on this host {{(pid=75234) _add_to_domain_capabilities /opt/stack/nova/nova/virt/libvirt/host.py:993}}
Nov 29 03:01:54 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-ef65ec89-6c59-4f39-9adc-df4888ffd41b None None] Checking secure boot support for host arch (x86_64) {{(pid=75234) supports_secure_boot /opt/stack/nova/nova/virt/libvirt/host.py:1781}}
Nov 29 03:01:54 user nova-compute[75234]: INFO nova.virt.libvirt.host [None req-ef65ec89-6c59-4f39-9adc-df4888ffd41b None None] Secure Boot support detected
Nov 29 03:01:54 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-ef65ec89-6c59-4f39-9adc-df4888ffd41b None None] cpu compare xml: <cpu match="exact">
Nov 29 03:01:54 user nova-compute[75234]:   <model>Nehalem</model>
Nov 29 03:01:54 user nova-compute[75234]: </cpu>
Nov 29 03:01:54 user nova-compute[75234]:  {{(pid=75234) _compare_cpu /opt/stack/nova/nova/virt/libvirt/driver.py:10020}}
Nov 29 03:01:54 user nova-compute[75234]: INFO nova.virt.node [None req-ef65ec89-6c59-4f39-9adc-df4888ffd41b None None] Generated node identity 4d291220-7c37-4839-a0e3-b23c66502b74
Nov 29 03:01:54 user nova-compute[75234]: INFO nova.virt.node [None req-ef65ec89-6c59-4f39-9adc-df4888ffd41b None None] Wrote node identity 4d291220-7c37-4839-a0e3-b23c66502b74 to /opt/stack/data/nova/compute_id
Nov 29 03:01:54 user nova-compute[75234]: WARNING nova.compute.manager [None req-ef65ec89-6c59-4f39-9adc-df4888ffd41b None None] Compute nodes ['4d291220-7c37-4839-a0e3-b23c66502b74'] for host user were not found in the database. If this is the first time this service is starting on this host, then you can ignore this warning.
Nov 29 03:01:54 user nova-compute[75234]: INFO nova.compute.manager [None req-ef65ec89-6c59-4f39-9adc-df4888ffd41b None None] Looking for unclaimed instances stuck in BUILDING status for nodes managed by this host
Nov 29 03:01:55 user nova-compute[75234]: WARNING nova.compute.manager [None req-ef65ec89-6c59-4f39-9adc-df4888ffd41b None None] No compute node record found for host user. If this is the first time this service is starting on this host, then you can ignore this warning.: nova.exception_Remote.ComputeHostNotFound_Remote: Compute host user could not be found.
Nov 29 03:01:55 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ef65ec89-6c59-4f39-9adc-df4888ffd41b None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:01:55 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ef65ec89-6c59-4f39-9adc-df4888ffd41b None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:01:55 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ef65ec89-6c59-4f39-9adc-df4888ffd41b None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:01:55 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ef65ec89-6c59-4f39-9adc-df4888ffd41b None None] Auditing locally available compute resources for user (node: user) {{(pid=75234) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:907}}
Nov 29 03:01:55 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-ef65ec89-6c59-4f39-9adc-df4888ffd41b None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:01:55 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-ef65ec89-6c59-4f39-9adc-df4888ffd41b None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:01:55 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ef65ec89-6c59-4f39-9adc-df4888ffd41b None None] Hypervisor/Node resource view: name=user free_ram=10957MB free_disk=26.259780883789062GB free_vcpus=12 pci_devices=[{"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}] {{(pid=75234) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1106}}
Nov 29 03:01:55 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ef65ec89-6c59-4f39-9adc-df4888ffd41b None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:01:55 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ef65ec89-6c59-4f39-9adc-df4888ffd41b None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:01:55 user nova-compute[75234]: WARNING nova.compute.resource_tracker [None req-ef65ec89-6c59-4f39-9adc-df4888ffd41b None None] No compute node record for user:4d291220-7c37-4839-a0e3-b23c66502b74: nova.exception_Remote.ComputeHostNotFound_Remote: Compute host 4d291220-7c37-4839-a0e3-b23c66502b74 could not be found.
Nov 29 03:01:55 user nova-compute[75234]: INFO nova.compute.resource_tracker [None req-ef65ec89-6c59-4f39-9adc-df4888ffd41b None None] Compute node record created for user:user with uuid: 4d291220-7c37-4839-a0e3-b23c66502b74
Nov 29 03:01:55 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ef65ec89-6c59-4f39-9adc-df4888ffd41b None None] Total usable vcpus: 12, total allocated vcpus: 0 {{(pid=75234) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1129}}
Nov 29 03:01:55 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ef65ec89-6c59-4f39-9adc-df4888ffd41b None None] Final resource view: name=user phys_ram=16011MB used_ram=512MB phys_disk=40GB used_disk=0GB total_vcpus=12 used_vcpus=0 pci_stats=[] {{(pid=75234) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1138}}
Nov 29 03:01:55 user nova-compute[75234]: INFO nova.scheduler.client.report [None req-ef65ec89-6c59-4f39-9adc-df4888ffd41b None None] [req-b8f5fe3f-9c5d-4b9b-91f9-7254132f8465] Created resource provider record via placement API for resource provider with UUID 4d291220-7c37-4839-a0e3-b23c66502b74 and name user.
Nov 29 03:01:55 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-ef65ec89-6c59-4f39-9adc-df4888ffd41b None None] /sys/module/kvm_amd/parameters/sev does not exist {{(pid=75234) _kernel_supports_amd_sev /opt/stack/nova/nova/virt/libvirt/host.py:1797}}
Nov 29 03:01:55 user nova-compute[75234]: INFO nova.virt.libvirt.host [None req-ef65ec89-6c59-4f39-9adc-df4888ffd41b None None] kernel doesn't support AMD SEV
Nov 29 03:01:55 user nova-compute[75234]: DEBUG nova.compute.provider_tree [None req-ef65ec89-6c59-4f39-9adc-df4888ffd41b None None] Updating inventory in ProviderTree for provider 4d291220-7c37-4839-a0e3-b23c66502b74 with inventory: {'MEMORY_MB': {'total': 16011, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0, 'reserved': 512}, 'VCPU': {'total': 12, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0, 'reserved': 0}, 'DISK_GB': {'total': 40, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0, 'reserved': 0}} {{(pid=75234) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:176}}
Nov 29 03:01:55 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-ef65ec89-6c59-4f39-9adc-df4888ffd41b None None] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=75234) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5400}}
Nov 29 03:01:55 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-ef65ec89-6c59-4f39-9adc-df4888ffd41b None None] Libvirt baseline CPU <cpu>
Nov 29 03:01:55 user nova-compute[75234]:   <arch>x86_64</arch>
Nov 29 03:01:55 user nova-compute[75234]:   <model>Nehalem</model>
Nov 29 03:01:55 user nova-compute[75234]:   <vendor>Intel</vendor>
Nov 29 03:01:55 user nova-compute[75234]:   <topology sockets="6" cores="1" threads="1"/>
Nov 29 03:01:55 user nova-compute[75234]: </cpu>
Nov 29 03:01:55 user nova-compute[75234]:  {{(pid=75234) _get_guest_baseline_cpu_features /opt/stack/nova/nova/virt/libvirt/driver.py:12510}}
Nov 29 03:01:55 user nova-compute[75234]: DEBUG nova.scheduler.client.report [None req-ef65ec89-6c59-4f39-9adc-df4888ffd41b None None] Updated inventory for provider 4d291220-7c37-4839-a0e3-b23c66502b74 with generation 0 in Placement from set_inventory_for_provider using data: {'MEMORY_MB': {'total': 16011, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0, 'reserved': 512}, 'VCPU': {'total': 12, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0, 'reserved': 0}, 'DISK_GB': {'total': 40, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0, 'reserved': 0}} {{(pid=75234) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:957}}
Nov 29 03:01:55 user nova-compute[75234]: DEBUG nova.compute.provider_tree [None req-ef65ec89-6c59-4f39-9adc-df4888ffd41b None None] Updating resource provider 4d291220-7c37-4839-a0e3-b23c66502b74 generation from 0 to 1 during operation: update_inventory {{(pid=75234) _update_generation /opt/stack/nova/nova/compute/provider_tree.py:164}}
Nov 29 03:01:55 user nova-compute[75234]: DEBUG nova.compute.provider_tree [None req-ef65ec89-6c59-4f39-9adc-df4888ffd41b None None] Updating inventory in ProviderTree for provider 4d291220-7c37-4839-a0e3-b23c66502b74 with inventory: {'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=75234) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:176}}
Nov 29 03:01:55 user nova-compute[75234]: DEBUG nova.compute.provider_tree [None req-ef65ec89-6c59-4f39-9adc-df4888ffd41b None None] Updating resource provider 4d291220-7c37-4839-a0e3-b23c66502b74 generation from 1 to 2 during operation: update_traits {{(pid=75234) _update_generation /opt/stack/nova/nova/compute/provider_tree.py:164}}
Nov 29 03:01:55 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ef65ec89-6c59-4f39-9adc-df4888ffd41b None None] Compute_service record updated for user:user {{(pid=75234) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:1067}}
Nov 29 03:01:55 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ef65ec89-6c59-4f39-9adc-df4888ffd41b None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.589s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:01:55 user nova-compute[75234]: DEBUG nova.service [None req-ef65ec89-6c59-4f39-9adc-df4888ffd41b None None] Creating RPC server for service compute {{(pid=75234) start /opt/stack/nova/nova/service.py:182}}
Nov 29 03:01:55 user nova-compute[75234]: DEBUG nova.service [None req-ef65ec89-6c59-4f39-9adc-df4888ffd41b None None] Join ServiceGroup membership for this service compute {{(pid=75234) start /opt/stack/nova/nova/service.py:199}}
Nov 29 03:01:55 user nova-compute[75234]: DEBUG nova.servicegroup.drivers.db [None req-ef65ec89-6c59-4f39-9adc-df4888ffd41b None None] DB_Driver: join new ServiceGroup member user to the compute group, service = <Service: host=user, binary=nova-compute, manager_class_name=nova.compute.manager.ComputeManager> {{(pid=75234) join /opt/stack/nova/nova/servicegroup/drivers/db.py:44}}
Nov 29 03:02:04 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._sync_power_states {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:02:04 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._cleanup_running_deleted_instances {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:02:46 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:02:46 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:02:46 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Starting heal instance info cache {{(pid=75234) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9922}}
Nov 29 03:02:46 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Rebuilding the list of instances to heal {{(pid=75234) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9926}}
Nov 29 03:02:46 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Didn't find any instances for network info cache update. {{(pid=75234) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:10008}}
Nov 29 03:02:46 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:02:46 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:02:46 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:02:46 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:02:46 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:02:46 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:02:46 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=75234) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10541}}
Nov 29 03:02:46 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager.update_available_resource {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:02:46 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:02:46 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:02:46 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:02:46 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Auditing locally available compute resources for user (node: user) {{(pid=75234) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:907}}
Nov 29 03:02:46 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:02:46 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:02:46 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Hypervisor/Node resource view: name=user free_ram=10431MB free_disk=26.163562774658203GB free_vcpus=12 pci_devices=[{"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}] {{(pid=75234) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1106}}
Nov 29 03:02:46 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:02:46 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:02:46 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Total usable vcpus: 12, total allocated vcpus: 0 {{(pid=75234) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1129}}
Nov 29 03:02:46 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Final resource view: name=user phys_ram=16011MB used_ram=512MB phys_disk=40GB used_disk=0GB total_vcpus=12 used_vcpus=0 pci_stats=[] {{(pid=75234) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1138}}
Nov 29 03:02:46 user nova-compute[75234]: DEBUG nova.compute.provider_tree [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Inventory has not changed in ProviderTree for provider: 4d291220-7c37-4839-a0e3-b23c66502b74 {{(pid=75234) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Nov 29 03:02:46 user nova-compute[75234]: DEBUG nova.scheduler.client.report [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Inventory has not changed for provider 4d291220-7c37-4839-a0e3-b23c66502b74 based on inventory data: {'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=75234) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Nov 29 03:02:46 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Compute_service record updated for user:user {{(pid=75234) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:1067}}
Nov 29 03:02:46 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.137s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:03:46 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:03:46 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._sync_scheduler_instance_info {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:03:46 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:03:46 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:03:46 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=75234) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10541}}
Nov 29 03:03:47 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:03:47 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager.update_available_resource {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:03:47 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:03:47 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:03:47 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:03:47 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Auditing locally available compute resources for user (node: user) {{(pid=75234) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:907}}
Nov 29 03:03:47 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:03:47 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:03:47 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Hypervisor/Node resource view: name=user free_ram=10324MB free_disk=26.22852325439453GB free_vcpus=12 pci_devices=[{"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}] {{(pid=75234) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1106}}
Nov 29 03:03:47 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:03:47 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:03:47 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Total usable vcpus: 12, total allocated vcpus: 0 {{(pid=75234) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1129}}
Nov 29 03:03:47 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Final resource view: name=user phys_ram=16011MB used_ram=512MB phys_disk=40GB used_disk=0GB total_vcpus=12 used_vcpus=0 pci_stats=[] {{(pid=75234) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1138}}
Nov 29 03:03:47 user nova-compute[75234]: DEBUG nova.compute.provider_tree [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Inventory has not changed in ProviderTree for provider: 4d291220-7c37-4839-a0e3-b23c66502b74 {{(pid=75234) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Nov 29 03:03:47 user nova-compute[75234]: DEBUG nova.scheduler.client.report [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Inventory has not changed for provider 4d291220-7c37-4839-a0e3-b23c66502b74 based on inventory data: {'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=75234) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Nov 29 03:03:47 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Compute_service record updated for user:user {{(pid=75234) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:1067}}
Nov 29 03:03:47 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.110s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:03:48 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:03:48 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Starting heal instance info cache {{(pid=75234) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9922}}
Nov 29 03:03:48 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Rebuilding the list of instances to heal {{(pid=75234) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9926}}
Nov 29 03:03:48 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Didn't find any instances for network info cache update. {{(pid=75234) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:10008}}
Nov 29 03:03:48 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:03:48 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:03:48 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:04:46 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:04:46 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:04:46 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=75234) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10541}}
Nov 29 03:04:47 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:04:47 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:04:48 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:04:48 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Starting heal instance info cache {{(pid=75234) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9922}}
Nov 29 03:04:48 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Rebuilding the list of instances to heal {{(pid=75234) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9926}}
Nov 29 03:04:48 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Didn't find any instances for network info cache update. {{(pid=75234) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:10008}}
Nov 29 03:04:48 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:04:49 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager.update_available_resource {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:04:49 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:04:49 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:04:49 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:04:49 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Auditing locally available compute resources for user (node: user) {{(pid=75234) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:907}}
Nov 29 03:04:49 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:04:49 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:04:49 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Hypervisor/Node resource view: name=user free_ram=10340MB free_disk=25.98023223876953GB free_vcpus=12 pci_devices=[{"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}] {{(pid=75234) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1106}}
Nov 29 03:04:49 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:04:49 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:04:49 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Total usable vcpus: 12, total allocated vcpus: 0 {{(pid=75234) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1129}}
Nov 29 03:04:49 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Final resource view: name=user phys_ram=16011MB used_ram=512MB phys_disk=40GB used_disk=0GB total_vcpus=12 used_vcpus=0 pci_stats=[] {{(pid=75234) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1138}}
Nov 29 03:04:49 user nova-compute[75234]: DEBUG nova.compute.provider_tree [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Inventory has not changed in ProviderTree for provider: 4d291220-7c37-4839-a0e3-b23c66502b74 {{(pid=75234) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Nov 29 03:04:49 user nova-compute[75234]: DEBUG nova.scheduler.client.report [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Inventory has not changed for provider 4d291220-7c37-4839-a0e3-b23c66502b74 based on inventory data: {'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=75234) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Nov 29 03:04:49 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Compute_service record updated for user:user {{(pid=75234) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:1067}}
Nov 29 03:04:49 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.177s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:04:50 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:04:50 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:05:46 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:05:46 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:05:46 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=75234) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10541}}
Nov 29 03:05:47 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._sync_scheduler_instance_info {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:05:48 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:05:49 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:05:49 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:05:49 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Starting heal instance info cache {{(pid=75234) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9922}}
Nov 29 03:05:49 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Rebuilding the list of instances to heal {{(pid=75234) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9926}}
Nov 29 03:05:49 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Didn't find any instances for network info cache update. {{(pid=75234) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:10008}}
Nov 29 03:05:50 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:05:50 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:05:50 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager.update_available_resource {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:05:50 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:05:50 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:05:50 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:05:50 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Auditing locally available compute resources for user (node: user) {{(pid=75234) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:907}}
Nov 29 03:05:50 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:05:50 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:05:50 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Hypervisor/Node resource view: name=user free_ram=9641MB free_disk=26.00088119506836GB free_vcpus=12 pci_devices=[{"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}] {{(pid=75234) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1106}}
Nov 29 03:05:50 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:05:50 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:05:50 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Total usable vcpus: 12, total allocated vcpus: 0 {{(pid=75234) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1129}}
Nov 29 03:05:50 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Final resource view: name=user phys_ram=16011MB used_ram=512MB phys_disk=40GB used_disk=0GB total_vcpus=12 used_vcpus=0 pci_stats=[] {{(pid=75234) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1138}}
Nov 29 03:05:51 user nova-compute[75234]: DEBUG nova.compute.provider_tree [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Inventory has not changed in ProviderTree for provider: 4d291220-7c37-4839-a0e3-b23c66502b74 {{(pid=75234) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Nov 29 03:05:51 user nova-compute[75234]: DEBUG nova.scheduler.client.report [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Inventory has not changed for provider 4d291220-7c37-4839-a0e3-b23c66502b74 based on inventory data: {'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=75234) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Nov 29 03:05:51 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Compute_service record updated for user:user {{(pid=75234) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:1067}}
Nov 29 03:05:51 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.190s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:05:53 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:05:55 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-61f7b6c4-f50c-48e8-be6f-f4711383b8cc tempest-DeleteServersTestJSON-1668990756 tempest-DeleteServersTestJSON-1668990756-project-member] Acquiring lock "9f7876ad-0487-426d-8b19-b718dccdc4fe" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:05:55 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-61f7b6c4-f50c-48e8-be6f-f4711383b8cc tempest-DeleteServersTestJSON-1668990756 tempest-DeleteServersTestJSON-1668990756-project-member] Lock "9f7876ad-0487-426d-8b19-b718dccdc4fe" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:05:55 user nova-compute[75234]: DEBUG nova.compute.manager [None req-61f7b6c4-f50c-48e8-be6f-f4711383b8cc tempest-DeleteServersTestJSON-1668990756 tempest-DeleteServersTestJSON-1668990756-project-member] [instance: 9f7876ad-0487-426d-8b19-b718dccdc4fe] Starting instance... {{(pid=75234) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2422}}
Nov 29 03:05:55 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-61f7b6c4-f50c-48e8-be6f-f4711383b8cc tempest-DeleteServersTestJSON-1668990756 tempest-DeleteServersTestJSON-1668990756-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:05:55 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-61f7b6c4-f50c-48e8-be6f-f4711383b8cc tempest-DeleteServersTestJSON-1668990756 tempest-DeleteServersTestJSON-1668990756-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:05:55 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-61f7b6c4-f50c-48e8-be6f-f4711383b8cc tempest-DeleteServersTestJSON-1668990756 tempest-DeleteServersTestJSON-1668990756-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=75234) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2366}}
Nov 29 03:05:55 user nova-compute[75234]: INFO nova.compute.claims [None req-61f7b6c4-f50c-48e8-be6f-f4711383b8cc tempest-DeleteServersTestJSON-1668990756 tempest-DeleteServersTestJSON-1668990756-project-member] [instance: 9f7876ad-0487-426d-8b19-b718dccdc4fe] Claim successful on node user
Nov 29 03:05:55 user nova-compute[75234]: DEBUG nova.compute.provider_tree [None req-61f7b6c4-f50c-48e8-be6f-f4711383b8cc tempest-DeleteServersTestJSON-1668990756 tempest-DeleteServersTestJSON-1668990756-project-member] Inventory has not changed in ProviderTree for provider: 4d291220-7c37-4839-a0e3-b23c66502b74 {{(pid=75234) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Nov 29 03:05:55 user nova-compute[75234]: DEBUG nova.scheduler.client.report [None req-61f7b6c4-f50c-48e8-be6f-f4711383b8cc tempest-DeleteServersTestJSON-1668990756 tempest-DeleteServersTestJSON-1668990756-project-member] Inventory has not changed for provider 4d291220-7c37-4839-a0e3-b23c66502b74 based on inventory data: {'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=75234) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Nov 29 03:05:55 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-61f7b6c4-f50c-48e8-be6f-f4711383b8cc tempest-DeleteServersTestJSON-1668990756 tempest-DeleteServersTestJSON-1668990756-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.328s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:05:55 user nova-compute[75234]: DEBUG nova.compute.manager [None req-61f7b6c4-f50c-48e8-be6f-f4711383b8cc tempest-DeleteServersTestJSON-1668990756 tempest-DeleteServersTestJSON-1668990756-project-member] [instance: 9f7876ad-0487-426d-8b19-b718dccdc4fe] Start building networks asynchronously for instance. {{(pid=75234) _build_resources /opt/stack/nova/nova/compute/manager.py:2819}}
Nov 29 03:05:55 user nova-compute[75234]: DEBUG nova.compute.manager [None req-61f7b6c4-f50c-48e8-be6f-f4711383b8cc tempest-DeleteServersTestJSON-1668990756 tempest-DeleteServersTestJSON-1668990756-project-member] [instance: 9f7876ad-0487-426d-8b19-b718dccdc4fe] Allocating IP information in the background. {{(pid=75234) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1972}}
Nov 29 03:05:55 user nova-compute[75234]: DEBUG nova.network.neutron [None req-61f7b6c4-f50c-48e8-be6f-f4711383b8cc tempest-DeleteServersTestJSON-1668990756 tempest-DeleteServersTestJSON-1668990756-project-member] [instance: 9f7876ad-0487-426d-8b19-b718dccdc4fe] allocate_for_instance() {{(pid=75234) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1156}}
Nov 29 03:05:55 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-61f7b6c4-f50c-48e8-be6f-f4711383b8cc tempest-DeleteServersTestJSON-1668990756 tempest-DeleteServersTestJSON-1668990756-project-member] [instance: 9f7876ad-0487-426d-8b19-b718dccdc4fe] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Nov 29 03:05:56 user nova-compute[75234]: DEBUG nova.compute.manager [None req-61f7b6c4-f50c-48e8-be6f-f4711383b8cc tempest-DeleteServersTestJSON-1668990756 tempest-DeleteServersTestJSON-1668990756-project-member] [instance: 9f7876ad-0487-426d-8b19-b718dccdc4fe] Start building block device mappings for instance. {{(pid=75234) _build_resources /opt/stack/nova/nova/compute/manager.py:2854}}
Nov 29 03:05:56 user nova-compute[75234]: DEBUG nova.compute.manager [None req-61f7b6c4-f50c-48e8-be6f-f4711383b8cc tempest-DeleteServersTestJSON-1668990756 tempest-DeleteServersTestJSON-1668990756-project-member] [instance: 9f7876ad-0487-426d-8b19-b718dccdc4fe] Start spawning the instance on the hypervisor. {{(pid=75234) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2628}}
Nov 29 03:05:56 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-61f7b6c4-f50c-48e8-be6f-f4711383b8cc tempest-DeleteServersTestJSON-1668990756 tempest-DeleteServersTestJSON-1668990756-project-member] [instance: 9f7876ad-0487-426d-8b19-b718dccdc4fe] Creating instance directory {{(pid=75234) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4727}}
Nov 29 03:05:56 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-61f7b6c4-f50c-48e8-be6f-f4711383b8cc tempest-DeleteServersTestJSON-1668990756 tempest-DeleteServersTestJSON-1668990756-project-member] [instance: 9f7876ad-0487-426d-8b19-b718dccdc4fe] Creating image(s)
Nov 29 03:05:56 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-61f7b6c4-f50c-48e8-be6f-f4711383b8cc tempest-DeleteServersTestJSON-1668990756 tempest-DeleteServersTestJSON-1668990756-project-member] Acquiring lock "/opt/stack/data/nova/instances/9f7876ad-0487-426d-8b19-b718dccdc4fe/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:05:56 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-61f7b6c4-f50c-48e8-be6f-f4711383b8cc tempest-DeleteServersTestJSON-1668990756 tempest-DeleteServersTestJSON-1668990756-project-member] Lock "/opt/stack/data/nova/instances/9f7876ad-0487-426d-8b19-b718dccdc4fe/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:05:56 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-61f7b6c4-f50c-48e8-be6f-f4711383b8cc tempest-DeleteServersTestJSON-1668990756 tempest-DeleteServersTestJSON-1668990756-project-member] Lock "/opt/stack/data/nova/instances/9f7876ad-0487-426d-8b19-b718dccdc4fe/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:05:56 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-61f7b6c4-f50c-48e8-be6f-f4711383b8cc tempest-DeleteServersTestJSON-1668990756 tempest-DeleteServersTestJSON-1668990756-project-member] Acquiring lock "80686e29dc9db6600d03e85cb62af298e12d7918" by "nova.virt.libvirt.imagebackend.Image.cache.<locals>.fetch_func_sync" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:05:56 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-61f7b6c4-f50c-48e8-be6f-f4711383b8cc tempest-DeleteServersTestJSON-1668990756 tempest-DeleteServersTestJSON-1668990756-project-member] Lock "80686e29dc9db6600d03e85cb62af298e12d7918" acquired by "nova.virt.libvirt.imagebackend.Image.cache.<locals>.fetch_func_sync" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:05:56 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-61f7b6c4-f50c-48e8-be6f-f4711383b8cc tempest-DeleteServersTestJSON-1668990756 tempest-DeleteServersTestJSON-1668990756-project-member] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918.part --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:05:57 user nova-compute[75234]: DEBUG nova.policy [None req-61f7b6c4-f50c-48e8-be6f-f4711383b8cc tempest-DeleteServersTestJSON-1668990756 tempest-DeleteServersTestJSON-1668990756-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '7a95c076ff434b43ac1264b5098b94fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a21183b0b4f947e39218bb8e7c857e50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=75234) authorize /opt/stack/nova/nova/policy.py:203}}
Nov 29 03:05:57 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-61f7b6c4-f50c-48e8-be6f-f4711383b8cc tempest-DeleteServersTestJSON-1668990756 tempest-DeleteServersTestJSON-1668990756-project-member] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918.part --force-share --output=json" returned: 0 in 0.206s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:05:57 user nova-compute[75234]: DEBUG nova.virt.images [None req-61f7b6c4-f50c-48e8-be6f-f4711383b8cc tempest-DeleteServersTestJSON-1668990756 tempest-DeleteServersTestJSON-1668990756-project-member] 96c30ba7-9b73-4a30-8ba4-7138879573b7 was qcow2, converting to raw {{(pid=75234) fetch_to_raw /opt/stack/nova/nova/virt/images.py:165}}
Nov 29 03:05:57 user nova-compute[75234]: DEBUG nova.privsep.utils [None req-61f7b6c4-f50c-48e8-be6f-f4711383b8cc tempest-DeleteServersTestJSON-1668990756 tempest-DeleteServersTestJSON-1668990756-project-member] Path '/opt/stack/data/nova/instances' supports direct I/O {{(pid=75234) supports_direct_io /opt/stack/nova/nova/privsep/utils.py:63}}
Nov 29 03:05:57 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-61f7b6c4-f50c-48e8-be6f-f4711383b8cc tempest-DeleteServersTestJSON-1668990756 tempest-DeleteServersTestJSON-1668990756-project-member] Running cmd (subprocess): qemu-img convert -t none -O raw -f qcow2 /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918.part /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918.converted {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:05:57 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-61f7b6c4-f50c-48e8-be6f-f4711383b8cc tempest-DeleteServersTestJSON-1668990756 tempest-DeleteServersTestJSON-1668990756-project-member] CMD "qemu-img convert -t none -O raw -f qcow2 /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918.part /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918.converted" returned: 0 in 0.433s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:05:57 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-61f7b6c4-f50c-48e8-be6f-f4711383b8cc tempest-DeleteServersTestJSON-1668990756 tempest-DeleteServersTestJSON-1668990756-project-member] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918.converted --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:05:57 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-61f7b6c4-f50c-48e8-be6f-f4711383b8cc tempest-DeleteServersTestJSON-1668990756 tempest-DeleteServersTestJSON-1668990756-project-member] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918.converted --force-share --output=json" returned: 0 in 0.203s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:05:57 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-61f7b6c4-f50c-48e8-be6f-f4711383b8cc tempest-DeleteServersTestJSON-1668990756 tempest-DeleteServersTestJSON-1668990756-project-member] Lock "80686e29dc9db6600d03e85cb62af298e12d7918" "released" by "nova.virt.libvirt.imagebackend.Image.cache.<locals>.fetch_func_sync" :: held 1.638s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:05:57 user nova-compute[75234]: INFO oslo.privsep.daemon [None req-61f7b6c4-f50c-48e8-be6f-f4711383b8cc tempest-DeleteServersTestJSON-1668990756 tempest-DeleteServersTestJSON-1668990756-project-member] Running privsep helper: ['sudo', 'nova-rootwrap', '/etc/nova/rootwrap.conf', 'privsep-helper', '--config-file', '/etc/nova/nova-cpu.conf', '--privsep_context', 'nova.privsep.sys_admin_pctxt', '--privsep_sock_path', '/tmp/tmpqifdo7fl/privsep.sock']
Nov 29 03:05:57 user sudo[84473]:    stack : PWD=/ ; USER=root ; COMMAND=/opt/stack/data/venv/bin/nova-rootwrap /etc/nova/rootwrap.conf privsep-helper --config-file /etc/nova/nova-cpu.conf --privsep_context nova.privsep.sys_admin_pctxt --privsep_sock_path /tmp/tmpqifdo7fl/privsep.sock
Nov 29 03:05:57 user sudo[84473]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=1001)
Nov 29 03:05:59 user sudo[84473]: pam_unix(sudo:session): session closed for user root
Nov 29 03:05:59 user nova-compute[75234]: INFO oslo.privsep.daemon [None req-61f7b6c4-f50c-48e8-be6f-f4711383b8cc tempest-DeleteServersTestJSON-1668990756 tempest-DeleteServersTestJSON-1668990756-project-member] Spawned new privsep daemon via rootwrap
Nov 29 03:05:59 user nova-compute[75234]: INFO oslo.privsep.daemon [-] privsep daemon starting
Nov 29 03:05:59 user nova-compute[75234]: INFO oslo.privsep.daemon [-] privsep process running with uid/gid: 0/0
Nov 29 03:05:59 user nova-compute[75234]: INFO oslo.privsep.daemon [-] privsep process running with capabilities (eff/prm/inh): CAP_CHOWN|CAP_DAC_OVERRIDE|CAP_DAC_READ_SEARCH|CAP_FOWNER|CAP_NET_ADMIN|CAP_SYS_ADMIN/CAP_CHOWN|CAP_DAC_OVERRIDE|CAP_DAC_READ_SEARCH|CAP_FOWNER|CAP_NET_ADMIN|CAP_SYS_ADMIN/none
Nov 29 03:05:59 user nova-compute[75234]: INFO oslo.privsep.daemon [-] privsep daemon running as pid 84478
Nov 29 03:05:59 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-61f7b6c4-f50c-48e8-be6f-f4711383b8cc tempest-DeleteServersTestJSON-1668990756 tempest-DeleteServersTestJSON-1668990756-project-member] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:05:59 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-61f7b6c4-f50c-48e8-be6f-f4711383b8cc tempest-DeleteServersTestJSON-1668990756 tempest-DeleteServersTestJSON-1668990756-project-member] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json" returned: 0 in 0.146s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:05:59 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-61f7b6c4-f50c-48e8-be6f-f4711383b8cc tempest-DeleteServersTestJSON-1668990756 tempest-DeleteServersTestJSON-1668990756-project-member] Acquiring lock "80686e29dc9db6600d03e85cb62af298e12d7918" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:05:59 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-61f7b6c4-f50c-48e8-be6f-f4711383b8cc tempest-DeleteServersTestJSON-1668990756 tempest-DeleteServersTestJSON-1668990756-project-member] Lock "80686e29dc9db6600d03e85cb62af298e12d7918" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.002s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:05:59 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-61f7b6c4-f50c-48e8-be6f-f4711383b8cc tempest-DeleteServersTestJSON-1668990756 tempest-DeleteServersTestJSON-1668990756-project-member] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:05:59 user nova-compute[75234]: DEBUG nova.network.neutron [None req-61f7b6c4-f50c-48e8-be6f-f4711383b8cc tempest-DeleteServersTestJSON-1668990756 tempest-DeleteServersTestJSON-1668990756-project-member] [instance: 9f7876ad-0487-426d-8b19-b718dccdc4fe] Successfully created port: 7dd048ce-2c9b-46ac-8e9d-8c6a2ab201f5 {{(pid=75234) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:548}}
Nov 29 03:05:59 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-61f7b6c4-f50c-48e8-be6f-f4711383b8cc tempest-DeleteServersTestJSON-1668990756 tempest-DeleteServersTestJSON-1668990756-project-member] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json" returned: 0 in 0.178s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:05:59 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-61f7b6c4-f50c-48e8-be6f-f4711383b8cc tempest-DeleteServersTestJSON-1668990756 tempest-DeleteServersTestJSON-1668990756-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918,backing_fmt=raw /opt/stack/data/nova/instances/9f7876ad-0487-426d-8b19-b718dccdc4fe/disk 1073741824 {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:05:59 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-61f7b6c4-f50c-48e8-be6f-f4711383b8cc tempest-DeleteServersTestJSON-1668990756 tempest-DeleteServersTestJSON-1668990756-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918,backing_fmt=raw /opt/stack/data/nova/instances/9f7876ad-0487-426d-8b19-b718dccdc4fe/disk 1073741824" returned: 0 in 0.049s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:05:59 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-61f7b6c4-f50c-48e8-be6f-f4711383b8cc tempest-DeleteServersTestJSON-1668990756 tempest-DeleteServersTestJSON-1668990756-project-member] Lock "80686e29dc9db6600d03e85cb62af298e12d7918" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.233s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:05:59 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-61f7b6c4-f50c-48e8-be6f-f4711383b8cc tempest-DeleteServersTestJSON-1668990756 tempest-DeleteServersTestJSON-1668990756-project-member] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:06:00 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-61f7b6c4-f50c-48e8-be6f-f4711383b8cc tempest-DeleteServersTestJSON-1668990756 tempest-DeleteServersTestJSON-1668990756-project-member] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json" returned: 0 in 0.196s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:06:00 user nova-compute[75234]: DEBUG nova.virt.disk.api [None req-61f7b6c4-f50c-48e8-be6f-f4711383b8cc tempest-DeleteServersTestJSON-1668990756 tempest-DeleteServersTestJSON-1668990756-project-member] Checking if we can resize image /opt/stack/data/nova/instances/9f7876ad-0487-426d-8b19-b718dccdc4fe/disk. size=1073741824 {{(pid=75234) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:166}}
Nov 29 03:06:00 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-61f7b6c4-f50c-48e8-be6f-f4711383b8cc tempest-DeleteServersTestJSON-1668990756 tempest-DeleteServersTestJSON-1668990756-project-member] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/9f7876ad-0487-426d-8b19-b718dccdc4fe/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:06:00 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-61f7b6c4-f50c-48e8-be6f-f4711383b8cc tempest-DeleteServersTestJSON-1668990756 tempest-DeleteServersTestJSON-1668990756-project-member] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/9f7876ad-0487-426d-8b19-b718dccdc4fe/disk --force-share --output=json" returned: 0 in 0.206s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:06:00 user nova-compute[75234]: DEBUG nova.virt.disk.api [None req-61f7b6c4-f50c-48e8-be6f-f4711383b8cc tempest-DeleteServersTestJSON-1668990756 tempest-DeleteServersTestJSON-1668990756-project-member] Cannot resize image /opt/stack/data/nova/instances/9f7876ad-0487-426d-8b19-b718dccdc4fe/disk to a smaller size. {{(pid=75234) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:172}}
Nov 29 03:06:00 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-61f7b6c4-f50c-48e8-be6f-f4711383b8cc tempest-DeleteServersTestJSON-1668990756 tempest-DeleteServersTestJSON-1668990756-project-member] [instance: 9f7876ad-0487-426d-8b19-b718dccdc4fe] Created local disks {{(pid=75234) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4861}}
Nov 29 03:06:00 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-61f7b6c4-f50c-48e8-be6f-f4711383b8cc tempest-DeleteServersTestJSON-1668990756 tempest-DeleteServersTestJSON-1668990756-project-member] [instance: 9f7876ad-0487-426d-8b19-b718dccdc4fe] Ensure instance console log exists: /opt/stack/data/nova/instances/9f7876ad-0487-426d-8b19-b718dccdc4fe/console.log {{(pid=75234) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4613}}
Nov 29 03:06:00 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-61f7b6c4-f50c-48e8-be6f-f4711383b8cc tempest-DeleteServersTestJSON-1668990756 tempest-DeleteServersTestJSON-1668990756-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:06:00 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-61f7b6c4-f50c-48e8-be6f-f4711383b8cc tempest-DeleteServersTestJSON-1668990756 tempest-DeleteServersTestJSON-1668990756-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:06:00 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-61f7b6c4-f50c-48e8-be6f-f4711383b8cc tempest-DeleteServersTestJSON-1668990756 tempest-DeleteServersTestJSON-1668990756-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:06:00 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-997f592d-b1eb-43b5-b6d7-0f22555038e1 tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] Acquiring lock "2bf9fe41-63c2-4882-bcdd-8e314e734ffe" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:06:00 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-997f592d-b1eb-43b5-b6d7-0f22555038e1 tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] Lock "2bf9fe41-63c2-4882-bcdd-8e314e734ffe" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:06:00 user nova-compute[75234]: DEBUG nova.compute.manager [None req-997f592d-b1eb-43b5-b6d7-0f22555038e1 tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] [instance: 2bf9fe41-63c2-4882-bcdd-8e314e734ffe] Starting instance... {{(pid=75234) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2422}}
Nov 29 03:06:00 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-997f592d-b1eb-43b5-b6d7-0f22555038e1 tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:06:00 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-997f592d-b1eb-43b5-b6d7-0f22555038e1 tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:06:00 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-997f592d-b1eb-43b5-b6d7-0f22555038e1 tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=75234) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2366}}
Nov 29 03:06:00 user nova-compute[75234]: INFO nova.compute.claims [None req-997f592d-b1eb-43b5-b6d7-0f22555038e1 tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] [instance: 2bf9fe41-63c2-4882-bcdd-8e314e734ffe] Claim successful on node user
Nov 29 03:06:00 user nova-compute[75234]: DEBUG nova.compute.provider_tree [None req-997f592d-b1eb-43b5-b6d7-0f22555038e1 tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] Inventory has not changed in ProviderTree for provider: 4d291220-7c37-4839-a0e3-b23c66502b74 {{(pid=75234) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Nov 29 03:06:00 user nova-compute[75234]: DEBUG nova.scheduler.client.report [None req-997f592d-b1eb-43b5-b6d7-0f22555038e1 tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] Inventory has not changed for provider 4d291220-7c37-4839-a0e3-b23c66502b74 based on inventory data: {'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=75234) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Nov 29 03:06:00 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-997f592d-b1eb-43b5-b6d7-0f22555038e1 tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.282s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:06:00 user nova-compute[75234]: DEBUG nova.compute.manager [None req-997f592d-b1eb-43b5-b6d7-0f22555038e1 tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] [instance: 2bf9fe41-63c2-4882-bcdd-8e314e734ffe] Start building networks asynchronously for instance. {{(pid=75234) _build_resources /opt/stack/nova/nova/compute/manager.py:2819}}
Nov 29 03:06:01 user nova-compute[75234]: DEBUG nova.compute.manager [None req-997f592d-b1eb-43b5-b6d7-0f22555038e1 tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] [instance: 2bf9fe41-63c2-4882-bcdd-8e314e734ffe] Allocating IP information in the background. {{(pid=75234) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1972}}
Nov 29 03:06:01 user nova-compute[75234]: DEBUG nova.network.neutron [None req-997f592d-b1eb-43b5-b6d7-0f22555038e1 tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] [instance: 2bf9fe41-63c2-4882-bcdd-8e314e734ffe] allocate_for_instance() {{(pid=75234) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1156}}
Nov 29 03:06:01 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-997f592d-b1eb-43b5-b6d7-0f22555038e1 tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] [instance: 2bf9fe41-63c2-4882-bcdd-8e314e734ffe] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Nov 29 03:06:01 user nova-compute[75234]: DEBUG nova.compute.manager [None req-997f592d-b1eb-43b5-b6d7-0f22555038e1 tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] [instance: 2bf9fe41-63c2-4882-bcdd-8e314e734ffe] Start building block device mappings for instance. {{(pid=75234) _build_resources /opt/stack/nova/nova/compute/manager.py:2854}}
Nov 29 03:06:01 user nova-compute[75234]: DEBUG nova.compute.manager [None req-997f592d-b1eb-43b5-b6d7-0f22555038e1 tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] [instance: 2bf9fe41-63c2-4882-bcdd-8e314e734ffe] Start spawning the instance on the hypervisor. {{(pid=75234) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2628}}
Nov 29 03:06:01 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-997f592d-b1eb-43b5-b6d7-0f22555038e1 tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] [instance: 2bf9fe41-63c2-4882-bcdd-8e314e734ffe] Creating instance directory {{(pid=75234) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4727}}
Nov 29 03:06:01 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-997f592d-b1eb-43b5-b6d7-0f22555038e1 tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] [instance: 2bf9fe41-63c2-4882-bcdd-8e314e734ffe] Creating image(s)
Nov 29 03:06:01 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-997f592d-b1eb-43b5-b6d7-0f22555038e1 tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] Acquiring lock "/opt/stack/data/nova/instances/2bf9fe41-63c2-4882-bcdd-8e314e734ffe/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:06:01 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-997f592d-b1eb-43b5-b6d7-0f22555038e1 tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] Lock "/opt/stack/data/nova/instances/2bf9fe41-63c2-4882-bcdd-8e314e734ffe/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:06:01 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-997f592d-b1eb-43b5-b6d7-0f22555038e1 tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] Lock "/opt/stack/data/nova/instances/2bf9fe41-63c2-4882-bcdd-8e314e734ffe/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.002s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:06:01 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-997f592d-b1eb-43b5-b6d7-0f22555038e1 tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:06:01 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-997f592d-b1eb-43b5-b6d7-0f22555038e1 tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json" returned: 0 in 0.169s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:06:01 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-997f592d-b1eb-43b5-b6d7-0f22555038e1 tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] Acquiring lock "80686e29dc9db6600d03e85cb62af298e12d7918" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:06:01 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-997f592d-b1eb-43b5-b6d7-0f22555038e1 tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] Lock "80686e29dc9db6600d03e85cb62af298e12d7918" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:06:01 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-997f592d-b1eb-43b5-b6d7-0f22555038e1 tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:06:01 user nova-compute[75234]: DEBUG nova.policy [None req-997f592d-b1eb-43b5-b6d7-0f22555038e1 tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '178bfe0ea37e45d8abd2d6b45bd5660c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b877668e02eb441c85dc2a7003a94c7a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=75234) authorize /opt/stack/nova/nova/policy.py:203}}
Nov 29 03:06:01 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-997f592d-b1eb-43b5-b6d7-0f22555038e1 tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json" returned: 0 in 0.176s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:06:01 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-997f592d-b1eb-43b5-b6d7-0f22555038e1 tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918,backing_fmt=raw /opt/stack/data/nova/instances/2bf9fe41-63c2-4882-bcdd-8e314e734ffe/disk 1073741824 {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:06:01 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-997f592d-b1eb-43b5-b6d7-0f22555038e1 tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918,backing_fmt=raw /opt/stack/data/nova/instances/2bf9fe41-63c2-4882-bcdd-8e314e734ffe/disk 1073741824" returned: 0 in 0.211s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:06:01 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-997f592d-b1eb-43b5-b6d7-0f22555038e1 tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] Lock "80686e29dc9db6600d03e85cb62af298e12d7918" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.389s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:06:01 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-997f592d-b1eb-43b5-b6d7-0f22555038e1 tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:06:02 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-997f592d-b1eb-43b5-b6d7-0f22555038e1 tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json" returned: 0 in 0.179s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:06:02 user nova-compute[75234]: DEBUG nova.virt.disk.api [None req-997f592d-b1eb-43b5-b6d7-0f22555038e1 tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] Checking if we can resize image /opt/stack/data/nova/instances/2bf9fe41-63c2-4882-bcdd-8e314e734ffe/disk. size=1073741824 {{(pid=75234) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:166}}
Nov 29 03:06:02 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-997f592d-b1eb-43b5-b6d7-0f22555038e1 tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2bf9fe41-63c2-4882-bcdd-8e314e734ffe/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:06:02 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-997f592d-b1eb-43b5-b6d7-0f22555038e1 tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2bf9fe41-63c2-4882-bcdd-8e314e734ffe/disk --force-share --output=json" returned: 0 in 0.317s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:06:02 user nova-compute[75234]: DEBUG nova.virt.disk.api [None req-997f592d-b1eb-43b5-b6d7-0f22555038e1 tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] Cannot resize image /opt/stack/data/nova/instances/2bf9fe41-63c2-4882-bcdd-8e314e734ffe/disk to a smaller size. {{(pid=75234) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:172}}
Nov 29 03:06:02 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-997f592d-b1eb-43b5-b6d7-0f22555038e1 tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] [instance: 2bf9fe41-63c2-4882-bcdd-8e314e734ffe] Created local disks {{(pid=75234) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4861}}
Nov 29 03:06:02 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-997f592d-b1eb-43b5-b6d7-0f22555038e1 tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] [instance: 2bf9fe41-63c2-4882-bcdd-8e314e734ffe] Ensure instance console log exists: /opt/stack/data/nova/instances/2bf9fe41-63c2-4882-bcdd-8e314e734ffe/console.log {{(pid=75234) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4613}}
Nov 29 03:06:02 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-997f592d-b1eb-43b5-b6d7-0f22555038e1 tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:06:02 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-997f592d-b1eb-43b5-b6d7-0f22555038e1 tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:06:02 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-997f592d-b1eb-43b5-b6d7-0f22555038e1 tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:06:02 user nova-compute[75234]: DEBUG nova.network.neutron [None req-61f7b6c4-f50c-48e8-be6f-f4711383b8cc tempest-DeleteServersTestJSON-1668990756 tempest-DeleteServersTestJSON-1668990756-project-member] [instance: 9f7876ad-0487-426d-8b19-b718dccdc4fe] Successfully updated port: 7dd048ce-2c9b-46ac-8e9d-8c6a2ab201f5 {{(pid=75234) _update_port /opt/stack/nova/nova/network/neutron.py:586}}
Nov 29 03:06:02 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-61f7b6c4-f50c-48e8-be6f-f4711383b8cc tempest-DeleteServersTestJSON-1668990756 tempest-DeleteServersTestJSON-1668990756-project-member] Acquiring lock "refresh_cache-9f7876ad-0487-426d-8b19-b718dccdc4fe" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:06:02 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-61f7b6c4-f50c-48e8-be6f-f4711383b8cc tempest-DeleteServersTestJSON-1668990756 tempest-DeleteServersTestJSON-1668990756-project-member] Acquired lock "refresh_cache-9f7876ad-0487-426d-8b19-b718dccdc4fe" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:06:02 user nova-compute[75234]: DEBUG nova.network.neutron [None req-61f7b6c4-f50c-48e8-be6f-f4711383b8cc tempest-DeleteServersTestJSON-1668990756 tempest-DeleteServersTestJSON-1668990756-project-member] [instance: 9f7876ad-0487-426d-8b19-b718dccdc4fe] Building network info cache for instance {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2009}}
Nov 29 03:06:02 user nova-compute[75234]: DEBUG nova.network.neutron [None req-61f7b6c4-f50c-48e8-be6f-f4711383b8cc tempest-DeleteServersTestJSON-1668990756 tempest-DeleteServersTestJSON-1668990756-project-member] [instance: 9f7876ad-0487-426d-8b19-b718dccdc4fe] Instance cache missing network info. {{(pid=75234) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3322}}
Nov 29 03:06:02 user nova-compute[75234]: DEBUG nova.compute.manager [req-a995e396-36ff-4fa1-9130-a7ce05652708 req-7db4e245-1415-4138-b920-299ae83bdf2c service nova] [instance: 9f7876ad-0487-426d-8b19-b718dccdc4fe] Received event network-changed-7dd048ce-2c9b-46ac-8e9d-8c6a2ab201f5 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:06:02 user nova-compute[75234]: DEBUG nova.compute.manager [req-a995e396-36ff-4fa1-9130-a7ce05652708 req-7db4e245-1415-4138-b920-299ae83bdf2c service nova] [instance: 9f7876ad-0487-426d-8b19-b718dccdc4fe] Refreshing instance network info cache due to event network-changed-7dd048ce-2c9b-46ac-8e9d-8c6a2ab201f5. {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11117}}
Nov 29 03:06:02 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-a995e396-36ff-4fa1-9130-a7ce05652708 req-7db4e245-1415-4138-b920-299ae83bdf2c service nova] Acquiring lock "refresh_cache-9f7876ad-0487-426d-8b19-b718dccdc4fe" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:06:03 user nova-compute[75234]: DEBUG nova.network.neutron [None req-61f7b6c4-f50c-48e8-be6f-f4711383b8cc tempest-DeleteServersTestJSON-1668990756 tempest-DeleteServersTestJSON-1668990756-project-member] [instance: 9f7876ad-0487-426d-8b19-b718dccdc4fe] Updating instance_info_cache with network_info: [{"id": "7dd048ce-2c9b-46ac-8e9d-8c6a2ab201f5", "address": "fa:16:3e:fd:bb:d1", "network": {"id": "c810861d-949e-423d-a3b5-a37f66f29342", "bridge": "br-int", "label": "tempest-DeleteServersTestJSON-2024350588-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a21183b0b4f947e39218bb8e7c857e50", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap7dd048ce-2c", "ovs_interfaceid": "7dd048ce-2c9b-46ac-8e9d-8c6a2ab201f5", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:06:03 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-61f7b6c4-f50c-48e8-be6f-f4711383b8cc tempest-DeleteServersTestJSON-1668990756 tempest-DeleteServersTestJSON-1668990756-project-member] Releasing lock "refresh_cache-9f7876ad-0487-426d-8b19-b718dccdc4fe" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:06:03 user nova-compute[75234]: DEBUG nova.compute.manager [None req-61f7b6c4-f50c-48e8-be6f-f4711383b8cc tempest-DeleteServersTestJSON-1668990756 tempest-DeleteServersTestJSON-1668990756-project-member] [instance: 9f7876ad-0487-426d-8b19-b718dccdc4fe] Instance network_info: |[{"id": "7dd048ce-2c9b-46ac-8e9d-8c6a2ab201f5", "address": "fa:16:3e:fd:bb:d1", "network": {"id": "c810861d-949e-423d-a3b5-a37f66f29342", "bridge": "br-int", "label": "tempest-DeleteServersTestJSON-2024350588-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a21183b0b4f947e39218bb8e7c857e50", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap7dd048ce-2c", "ovs_interfaceid": "7dd048ce-2c9b-46ac-8e9d-8c6a2ab201f5", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=75234) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1987}}
Nov 29 03:06:03 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-a995e396-36ff-4fa1-9130-a7ce05652708 req-7db4e245-1415-4138-b920-299ae83bdf2c service nova] Acquired lock "refresh_cache-9f7876ad-0487-426d-8b19-b718dccdc4fe" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:06:03 user nova-compute[75234]: DEBUG nova.network.neutron [req-a995e396-36ff-4fa1-9130-a7ce05652708 req-7db4e245-1415-4138-b920-299ae83bdf2c service nova] [instance: 9f7876ad-0487-426d-8b19-b718dccdc4fe] Refreshing network info cache for port 7dd048ce-2c9b-46ac-8e9d-8c6a2ab201f5 {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2006}}
Nov 29 03:06:03 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-61f7b6c4-f50c-48e8-be6f-f4711383b8cc tempest-DeleteServersTestJSON-1668990756 tempest-DeleteServersTestJSON-1668990756-project-member] [instance: 9f7876ad-0487-426d-8b19-b718dccdc4fe] Start _get_guest_xml network_info=[{"id": "7dd048ce-2c9b-46ac-8e9d-8c6a2ab201f5", "address": "fa:16:3e:fd:bb:d1", "network": {"id": "c810861d-949e-423d-a3b5-a37f66f29342", "bridge": "br-int", "label": "tempest-DeleteServersTestJSON-2024350588-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a21183b0b4f947e39218bb8e7c857e50", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap7dd048ce-2c", "ovs_interfaceid": "7dd048ce-2c9b-46ac-8e9d-8c6a2ab201f5", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum='c8fc807773e5354afe61636071771906',container_format='bare',created_at=2023-11-29T03:02:15Z,direct_url=<?>,disk_format='qcow2',id=96c30ba7-9b73-4a30-8ba4-7138879573b7,min_disk=0,min_ram=0,name='cirros-0.6.2-x86_64-disk',owner='b18e380921c748b78517b6f1211cd0f1',properties=ImageMetaProps,protected=<?>,size=21430272,status='active',tags=<?>,updated_at=2023-11-29T03:02:17Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [{'device_name': '/dev/vda', 'encryption_format': None, 'encrypted': False, 'device_type': 'disk', 'size': 0, 'disk_bus': 'virtio', 'boot_index': 0, 'encryption_options': None, 'encryption_secret_uuid': None, 'guest_format': None, 'image_id': '96c30ba7-9b73-4a30-8ba4-7138879573b7'}], 'ephemerals': [], 'block_device_mapping': [], 'swap': None} {{(pid=75234) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7562}}
Nov 29 03:06:03 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-61f7b6c4-f50c-48e8-be6f-f4711383b8cc tempest-DeleteServersTestJSON-1668990756 tempest-DeleteServersTestJSON-1668990756-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:06:03 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-61f7b6c4-f50c-48e8-be6f-f4711383b8cc tempest-DeleteServersTestJSON-1668990756 tempest-DeleteServersTestJSON-1668990756-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:06:03 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-61f7b6c4-f50c-48e8-be6f-f4711383b8cc tempest-DeleteServersTestJSON-1668990756 tempest-DeleteServersTestJSON-1668990756-project-member] Searching host: 'user' for CPU controller through CGroups V1... {{(pid=75234) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1652}}
Nov 29 03:06:03 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-61f7b6c4-f50c-48e8-be6f-f4711383b8cc tempest-DeleteServersTestJSON-1668990756 tempest-DeleteServersTestJSON-1668990756-project-member] CPU controller missing on host. {{(pid=75234) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1662}}
Nov 29 03:06:03 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-61f7b6c4-f50c-48e8-be6f-f4711383b8cc tempest-DeleteServersTestJSON-1668990756 tempest-DeleteServersTestJSON-1668990756-project-member] Searching host: 'user' for CPU controller through CGroups V2... {{(pid=75234) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1671}}
Nov 29 03:06:03 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-61f7b6c4-f50c-48e8-be6f-f4711383b8cc tempest-DeleteServersTestJSON-1668990756 tempest-DeleteServersTestJSON-1668990756-project-member] CPU controller found on host. {{(pid=75234) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1678}}
Nov 29 03:06:03 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-61f7b6c4-f50c-48e8-be6f-f4711383b8cc tempest-DeleteServersTestJSON-1668990756 tempest-DeleteServersTestJSON-1668990756-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=75234) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5400}}
Nov 29 03:06:03 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-61f7b6c4-f50c-48e8-be6f-f4711383b8cc tempest-DeleteServersTestJSON-1668990756 tempest-DeleteServersTestJSON-1668990756-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-11-29T03:02:04Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='1',id=6,is_public=True,memory_mb=512,name='m1.tiny',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='c8fc807773e5354afe61636071771906',container_format='bare',created_at=2023-11-29T03:02:15Z,direct_url=<?>,disk_format='qcow2',id=96c30ba7-9b73-4a30-8ba4-7138879573b7,min_disk=0,min_ram=0,name='cirros-0.6.2-x86_64-disk',owner='b18e380921c748b78517b6f1211cd0f1',properties=ImageMetaProps,protected=<?>,size=21430272,status='active',tags=<?>,updated_at=2023-11-29T03:02:17Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:563}}
Nov 29 03:06:03 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-61f7b6c4-f50c-48e8-be6f-f4711383b8cc tempest-DeleteServersTestJSON-1668990756 tempest-DeleteServersTestJSON-1668990756-project-member] Flavor limits 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:348}}
Nov 29 03:06:03 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-61f7b6c4-f50c-48e8-be6f-f4711383b8cc tempest-DeleteServersTestJSON-1668990756 tempest-DeleteServersTestJSON-1668990756-project-member] Image limits 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:352}}
Nov 29 03:06:03 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-61f7b6c4-f50c-48e8-be6f-f4711383b8cc tempest-DeleteServersTestJSON-1668990756 tempest-DeleteServersTestJSON-1668990756-project-member] Flavor pref 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:388}}
Nov 29 03:06:03 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-61f7b6c4-f50c-48e8-be6f-f4711383b8cc tempest-DeleteServersTestJSON-1668990756 tempest-DeleteServersTestJSON-1668990756-project-member] Image pref 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:392}}
Nov 29 03:06:03 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-61f7b6c4-f50c-48e8-be6f-f4711383b8cc tempest-DeleteServersTestJSON-1668990756 tempest-DeleteServersTestJSON-1668990756-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:430}}
Nov 29 03:06:03 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-61f7b6c4-f50c-48e8-be6f-f4711383b8cc tempest-DeleteServersTestJSON-1668990756 tempest-DeleteServersTestJSON-1668990756-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:569}}
Nov 29 03:06:03 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-61f7b6c4-f50c-48e8-be6f-f4711383b8cc tempest-DeleteServersTestJSON-1668990756 tempest-DeleteServersTestJSON-1668990756-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=75234) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:471}}
Nov 29 03:06:03 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-61f7b6c4-f50c-48e8-be6f-f4711383b8cc tempest-DeleteServersTestJSON-1668990756 tempest-DeleteServersTestJSON-1668990756-project-member] Got 1 possible topologies {{(pid=75234) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:501}}
Nov 29 03:06:03 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-61f7b6c4-f50c-48e8-be6f-f4711383b8cc tempest-DeleteServersTestJSON-1668990756 tempest-DeleteServersTestJSON-1668990756-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:575}}
Nov 29 03:06:03 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-61f7b6c4-f50c-48e8-be6f-f4711383b8cc tempest-DeleteServersTestJSON-1668990756 tempest-DeleteServersTestJSON-1668990756-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:577}}
Nov 29 03:06:03 user nova-compute[75234]: DEBUG nova.privsep.utils [None req-61f7b6c4-f50c-48e8-be6f-f4711383b8cc tempest-DeleteServersTestJSON-1668990756 tempest-DeleteServersTestJSON-1668990756-project-member] Path '/opt/stack/data/nova/instances' supports direct I/O {{(pid=75234) supports_direct_io /opt/stack/nova/nova/privsep/utils.py:63}}
Nov 29 03:06:03 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-61f7b6c4-f50c-48e8-be6f-f4711383b8cc tempest-DeleteServersTestJSON-1668990756 tempest-DeleteServersTestJSON-1668990756-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-11-29T03:05:53Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-DeleteServersTestJSON-server-1027639586',display_name='tempest-DeleteServersTestJSON-server-1027639586',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-deleteserverstestjson-server-1027639586',id=1,image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data=None,key_name=None,keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='a21183b0b4f947e39218bb8e7c857e50',ramdisk_id='',reservation_id='r-0a0qd15t',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-DeleteServersTestJSON-1668990756',owner_user_name='tempest-DeleteServersTestJSON-1668990756-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-11-29T03:05:56Z,user_data=None,user_id='7a95c076ff434b43ac1264b5098b94fc',uuid=9f7876ad-0487-426d-8b19-b718dccdc4fe,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "7dd048ce-2c9b-46ac-8e9d-8c6a2ab201f5", "address": "fa:16:3e:fd:bb:d1", "network": {"id": "c810861d-949e-423d-a3b5-a37f66f29342", "bridge": "br-int", "label": "tempest-DeleteServersTestJSON-2024350588-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a21183b0b4f947e39218bb8e7c857e50", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap7dd048ce-2c", "ovs_interfaceid": "7dd048ce-2c9b-46ac-8e9d-8c6a2ab201f5", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=75234) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Nov 29 03:06:03 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-61f7b6c4-f50c-48e8-be6f-f4711383b8cc tempest-DeleteServersTestJSON-1668990756 tempest-DeleteServersTestJSON-1668990756-project-member] Converting VIF {"id": "7dd048ce-2c9b-46ac-8e9d-8c6a2ab201f5", "address": "fa:16:3e:fd:bb:d1", "network": {"id": "c810861d-949e-423d-a3b5-a37f66f29342", "bridge": "br-int", "label": "tempest-DeleteServersTestJSON-2024350588-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a21183b0b4f947e39218bb8e7c857e50", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap7dd048ce-2c", "ovs_interfaceid": "7dd048ce-2c9b-46ac-8e9d-8c6a2ab201f5", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:06:03 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-61f7b6c4-f50c-48e8-be6f-f4711383b8cc tempest-DeleteServersTestJSON-1668990756 tempest-DeleteServersTestJSON-1668990756-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:fd:bb:d1,bridge_name='br-int',has_traffic_filtering=True,id=7dd048ce-2c9b-46ac-8e9d-8c6a2ab201f5,network=Network(c810861d-949e-423d-a3b5-a37f66f29342),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap7dd048ce-2c') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:06:03 user nova-compute[75234]: DEBUG nova.objects.instance [None req-61f7b6c4-f50c-48e8-be6f-f4711383b8cc tempest-DeleteServersTestJSON-1668990756 tempest-DeleteServersTestJSON-1668990756-project-member] Lazy-loading 'pci_devices' on Instance uuid 9f7876ad-0487-426d-8b19-b718dccdc4fe {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:06:03 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-61f7b6c4-f50c-48e8-be6f-f4711383b8cc tempest-DeleteServersTestJSON-1668990756 tempest-DeleteServersTestJSON-1668990756-project-member] [instance: 9f7876ad-0487-426d-8b19-b718dccdc4fe] End _get_guest_xml xml=<domain type="kvm">
Nov 29 03:06:03 user nova-compute[75234]:   <uuid>9f7876ad-0487-426d-8b19-b718dccdc4fe</uuid>
Nov 29 03:06:03 user nova-compute[75234]:   <name>instance-00000001</name>
Nov 29 03:06:03 user nova-compute[75234]:   <memory>524288</memory>
Nov 29 03:06:03 user nova-compute[75234]:   <vcpu>1</vcpu>
Nov 29 03:06:03 user nova-compute[75234]:   <metadata>
Nov 29 03:06:03 user nova-compute[75234]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Nov 29 03:06:03 user nova-compute[75234]:       <nova:package version="0.0.0"/>
Nov 29 03:06:03 user nova-compute[75234]:       <nova:name>tempest-DeleteServersTestJSON-server-1027639586</nova:name>
Nov 29 03:06:03 user nova-compute[75234]:       <nova:creationTime>2023-11-29 03:06:03</nova:creationTime>
Nov 29 03:06:03 user nova-compute[75234]:       <nova:flavor name="m1.tiny">
Nov 29 03:06:03 user nova-compute[75234]:         <nova:memory>512</nova:memory>
Nov 29 03:06:03 user nova-compute[75234]:         <nova:disk>1</nova:disk>
Nov 29 03:06:03 user nova-compute[75234]:         <nova:swap>0</nova:swap>
Nov 29 03:06:03 user nova-compute[75234]:         <nova:ephemeral>0</nova:ephemeral>
Nov 29 03:06:03 user nova-compute[75234]:         <nova:vcpus>1</nova:vcpus>
Nov 29 03:06:03 user nova-compute[75234]:       </nova:flavor>
Nov 29 03:06:03 user nova-compute[75234]:       <nova:owner>
Nov 29 03:06:03 user nova-compute[75234]:         <nova:user uuid="7a95c076ff434b43ac1264b5098b94fc">tempest-DeleteServersTestJSON-1668990756-project-member</nova:user>
Nov 29 03:06:03 user nova-compute[75234]:         <nova:project uuid="a21183b0b4f947e39218bb8e7c857e50">tempest-DeleteServersTestJSON-1668990756</nova:project>
Nov 29 03:06:03 user nova-compute[75234]:       </nova:owner>
Nov 29 03:06:03 user nova-compute[75234]:       <nova:root type="image" uuid="96c30ba7-9b73-4a30-8ba4-7138879573b7"/>
Nov 29 03:06:03 user nova-compute[75234]:       <nova:ports>
Nov 29 03:06:03 user nova-compute[75234]:         <nova:port uuid="7dd048ce-2c9b-46ac-8e9d-8c6a2ab201f5">
Nov 29 03:06:03 user nova-compute[75234]:           <nova:ip type="fixed" address="10.0.0.13" ipVersion="4"/>
Nov 29 03:06:03 user nova-compute[75234]:         </nova:port>
Nov 29 03:06:03 user nova-compute[75234]:       </nova:ports>
Nov 29 03:06:03 user nova-compute[75234]:     </nova:instance>
Nov 29 03:06:03 user nova-compute[75234]:   </metadata>
Nov 29 03:06:03 user nova-compute[75234]:   <sysinfo type="smbios">
Nov 29 03:06:03 user nova-compute[75234]:     <system>
Nov 29 03:06:03 user nova-compute[75234]:       <entry name="manufacturer">OpenStack Foundation</entry>
Nov 29 03:06:03 user nova-compute[75234]:       <entry name="product">OpenStack Nova</entry>
Nov 29 03:06:03 user nova-compute[75234]:       <entry name="version">0.0.0</entry>
Nov 29 03:06:03 user nova-compute[75234]:       <entry name="serial">9f7876ad-0487-426d-8b19-b718dccdc4fe</entry>
Nov 29 03:06:03 user nova-compute[75234]:       <entry name="uuid">9f7876ad-0487-426d-8b19-b718dccdc4fe</entry>
Nov 29 03:06:03 user nova-compute[75234]:       <entry name="family">Virtual Machine</entry>
Nov 29 03:06:03 user nova-compute[75234]:     </system>
Nov 29 03:06:03 user nova-compute[75234]:   </sysinfo>
Nov 29 03:06:03 user nova-compute[75234]:   <os>
Nov 29 03:06:03 user nova-compute[75234]:     <type arch="x86_64" machine="pc">hvm</type>
Nov 29 03:06:03 user nova-compute[75234]:     <boot dev="hd"/>
Nov 29 03:06:03 user nova-compute[75234]:     <smbios mode="sysinfo"/>
Nov 29 03:06:03 user nova-compute[75234]:   </os>
Nov 29 03:06:03 user nova-compute[75234]:   <features>
Nov 29 03:06:03 user nova-compute[75234]:     <acpi/>
Nov 29 03:06:03 user nova-compute[75234]:     <apic/>
Nov 29 03:06:03 user nova-compute[75234]:     <vmcoreinfo/>
Nov 29 03:06:03 user nova-compute[75234]:   </features>
Nov 29 03:06:03 user nova-compute[75234]:   <clock offset="utc">
Nov 29 03:06:03 user nova-compute[75234]:     <timer name="pit" tickpolicy="delay"/>
Nov 29 03:06:03 user nova-compute[75234]:     <timer name="rtc" tickpolicy="catchup"/>
Nov 29 03:06:03 user nova-compute[75234]:     <timer name="hpet" present="no"/>
Nov 29 03:06:03 user nova-compute[75234]:   </clock>
Nov 29 03:06:03 user nova-compute[75234]:   <cpu mode="custom" match="exact">
Nov 29 03:06:03 user nova-compute[75234]:     <model>Nehalem</model>
Nov 29 03:06:03 user nova-compute[75234]:     <topology sockets="1" cores="1" threads="1"/>
Nov 29 03:06:03 user nova-compute[75234]:   </cpu>
Nov 29 03:06:03 user nova-compute[75234]:   <devices>
Nov 29 03:06:03 user nova-compute[75234]:     <disk type="file" device="disk">
Nov 29 03:06:03 user nova-compute[75234]:       <driver name="qemu" type="qcow2" cache="none"/>
Nov 29 03:06:03 user nova-compute[75234]:       <source file="/opt/stack/data/nova/instances/9f7876ad-0487-426d-8b19-b718dccdc4fe/disk"/>
Nov 29 03:06:03 user nova-compute[75234]:       <target dev="vda" bus="virtio"/>
Nov 29 03:06:03 user nova-compute[75234]:     </disk>
Nov 29 03:06:03 user nova-compute[75234]:     <interface type="ethernet">
Nov 29 03:06:03 user nova-compute[75234]:       <mac address="fa:16:3e:fd:bb:d1"/>
Nov 29 03:06:03 user nova-compute[75234]:       <model type="virtio"/>
Nov 29 03:06:03 user nova-compute[75234]:       <mtu size="1442"/>
Nov 29 03:06:03 user nova-compute[75234]:       <target dev="tap7dd048ce-2c"/>
Nov 29 03:06:03 user nova-compute[75234]:     </interface>
Nov 29 03:06:03 user nova-compute[75234]:     <serial type="pty">
Nov 29 03:06:03 user nova-compute[75234]:       <log file="/opt/stack/data/nova/instances/9f7876ad-0487-426d-8b19-b718dccdc4fe/console.log" append="off"/>
Nov 29 03:06:03 user nova-compute[75234]:     </serial>
Nov 29 03:06:03 user nova-compute[75234]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Nov 29 03:06:03 user nova-compute[75234]:     <video>
Nov 29 03:06:03 user nova-compute[75234]:       <model type="virtio"/>
Nov 29 03:06:03 user nova-compute[75234]:     </video>
Nov 29 03:06:03 user nova-compute[75234]:     <rng model="virtio">
Nov 29 03:06:03 user nova-compute[75234]:       <backend model="random">/dev/urandom</backend>
Nov 29 03:06:03 user nova-compute[75234]:     </rng>
Nov 29 03:06:03 user nova-compute[75234]:     <controller type="usb" index="0" model="none"/>
Nov 29 03:06:03 user nova-compute[75234]:     <memballoon model="virtio">
Nov 29 03:06:03 user nova-compute[75234]:       <stats period="10"/>
Nov 29 03:06:03 user nova-compute[75234]:     </memballoon>
Nov 29 03:06:03 user nova-compute[75234]:   </devices>
Nov 29 03:06:03 user nova-compute[75234]: </domain>
Nov 29 03:06:03 user nova-compute[75234]:  {{(pid=75234) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7568}}
Nov 29 03:06:03 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-61f7b6c4-f50c-48e8-be6f-f4711383b8cc tempest-DeleteServersTestJSON-1668990756 tempest-DeleteServersTestJSON-1668990756-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-11-29T03:05:53Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-DeleteServersTestJSON-server-1027639586',display_name='tempest-DeleteServersTestJSON-server-1027639586',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-deleteserverstestjson-server-1027639586',id=1,image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data=None,key_name=None,keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='a21183b0b4f947e39218bb8e7c857e50',ramdisk_id='',reservation_id='r-0a0qd15t',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-DeleteServersTestJSON-1668990756',owner_user_name='tempest-DeleteServersTestJSON-1668990756-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-11-29T03:05:56Z,user_data=None,user_id='7a95c076ff434b43ac1264b5098b94fc',uuid=9f7876ad-0487-426d-8b19-b718dccdc4fe,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "7dd048ce-2c9b-46ac-8e9d-8c6a2ab201f5", "address": "fa:16:3e:fd:bb:d1", "network": {"id": "c810861d-949e-423d-a3b5-a37f66f29342", "bridge": "br-int", "label": "tempest-DeleteServersTestJSON-2024350588-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a21183b0b4f947e39218bb8e7c857e50", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap7dd048ce-2c", "ovs_interfaceid": "7dd048ce-2c9b-46ac-8e9d-8c6a2ab201f5", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Nov 29 03:06:03 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-61f7b6c4-f50c-48e8-be6f-f4711383b8cc tempest-DeleteServersTestJSON-1668990756 tempest-DeleteServersTestJSON-1668990756-project-member] Converting VIF {"id": "7dd048ce-2c9b-46ac-8e9d-8c6a2ab201f5", "address": "fa:16:3e:fd:bb:d1", "network": {"id": "c810861d-949e-423d-a3b5-a37f66f29342", "bridge": "br-int", "label": "tempest-DeleteServersTestJSON-2024350588-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a21183b0b4f947e39218bb8e7c857e50", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap7dd048ce-2c", "ovs_interfaceid": "7dd048ce-2c9b-46ac-8e9d-8c6a2ab201f5", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:06:03 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-61f7b6c4-f50c-48e8-be6f-f4711383b8cc tempest-DeleteServersTestJSON-1668990756 tempest-DeleteServersTestJSON-1668990756-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:fd:bb:d1,bridge_name='br-int',has_traffic_filtering=True,id=7dd048ce-2c9b-46ac-8e9d-8c6a2ab201f5,network=Network(c810861d-949e-423d-a3b5-a37f66f29342),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap7dd048ce-2c') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:06:03 user nova-compute[75234]: DEBUG os_vif [None req-61f7b6c4-f50c-48e8-be6f-f4711383b8cc tempest-DeleteServersTestJSON-1668990756 tempest-DeleteServersTestJSON-1668990756-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:fd:bb:d1,bridge_name='br-int',has_traffic_filtering=True,id=7dd048ce-2c9b-46ac-8e9d-8c6a2ab201f5,network=Network(c810861d-949e-423d-a3b5-a37f66f29342),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap7dd048ce-2c') {{(pid=75234) plug /opt/stack/data/venv/lib/python3.10/site-packages/os_vif/__init__.py:76}}
Nov 29 03:06:03 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl [None req-61f7b6c4-f50c-48e8-be6f-f4711383b8cc tempest-DeleteServersTestJSON-1668990756 tempest-DeleteServersTestJSON-1668990756-project-member] Created schema index Interface.name {{(pid=75234) autocreate_indices /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:106}}
Nov 29 03:06:03 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl [None req-61f7b6c4-f50c-48e8-be6f-f4711383b8cc tempest-DeleteServersTestJSON-1668990756 tempest-DeleteServersTestJSON-1668990756-project-member] Created schema index Port.name {{(pid=75234) autocreate_indices /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:106}}
Nov 29 03:06:03 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl [None req-61f7b6c4-f50c-48e8-be6f-f4711383b8cc tempest-DeleteServersTestJSON-1668990756 tempest-DeleteServersTestJSON-1668990756-project-member] Created schema index Bridge.name {{(pid=75234) autocreate_indices /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/__init__.py:106}}
Nov 29 03:06:03 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [None req-61f7b6c4-f50c-48e8-be6f-f4711383b8cc tempest-DeleteServersTestJSON-1668990756 tempest-DeleteServersTestJSON-1668990756-project-member] tcp:127.0.0.1:6640: entering CONNECTING {{(pid=75234) _transition /opt/stack/data/venv/lib/python3.10/site-packages/ovs/reconnect.py:519}}
Nov 29 03:06:03 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [None req-61f7b6c4-f50c-48e8-be6f-f4711383b8cc tempest-DeleteServersTestJSON-1668990756 tempest-DeleteServersTestJSON-1668990756-project-member] [POLLOUT] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:06:03 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [None req-61f7b6c4-f50c-48e8-be6f-f4711383b8cc tempest-DeleteServersTestJSON-1668990756 tempest-DeleteServersTestJSON-1668990756-project-member] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=75234) _transition /opt/stack/data/venv/lib/python3.10/site-packages/ovs/reconnect.py:519}}
Nov 29 03:06:03 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [None req-61f7b6c4-f50c-48e8-be6f-f4711383b8cc tempest-DeleteServersTestJSON-1668990756 tempest-DeleteServersTestJSON-1668990756-project-member] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:06:03 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [None req-61f7b6c4-f50c-48e8-be6f-f4711383b8cc tempest-DeleteServersTestJSON-1668990756 tempest-DeleteServersTestJSON-1668990756-project-member] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:06:03 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [None req-61f7b6c4-f50c-48e8-be6f-f4711383b8cc tempest-DeleteServersTestJSON-1668990756 tempest-DeleteServersTestJSON-1668990756-project-member] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:06:03 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:06:03 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:06:03 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Nov 29 03:06:03 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:06:03 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbCreateCommand(_result=None, table=QoS, columns={'type': 'linux-noop', 'external_ids': {'id': '21783219-6d5a-581d-82fd-0ef249c653da', '_type': 'linux-noop'}}, row=False) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:06:03 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:06:03 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:06:03 user nova-compute[75234]: INFO oslo.privsep.daemon [None req-61f7b6c4-f50c-48e8-be6f-f4711383b8cc tempest-DeleteServersTestJSON-1668990756 tempest-DeleteServersTestJSON-1668990756-project-member] Running privsep helper: ['sudo', 'nova-rootwrap', '/etc/nova/rootwrap.conf', 'privsep-helper', '--config-file', '/etc/nova/nova-cpu.conf', '--privsep_context', 'vif_plug_ovs.privsep.vif_plug', '--privsep_sock_path', '/tmp/tmpsdyjw7dw/privsep.sock']
Nov 29 03:06:03 user sudo[84515]:    stack : PWD=/ ; USER=root ; COMMAND=/opt/stack/data/venv/bin/nova-rootwrap /etc/nova/rootwrap.conf privsep-helper --config-file /etc/nova/nova-cpu.conf --privsep_context vif_plug_ovs.privsep.vif_plug --privsep_sock_path /tmp/tmpsdyjw7dw/privsep.sock
Nov 29 03:06:03 user sudo[84515]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=1001)
Nov 29 03:06:03 user nova-compute[75234]: DEBUG nova.network.neutron [None req-997f592d-b1eb-43b5-b6d7-0f22555038e1 tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] [instance: 2bf9fe41-63c2-4882-bcdd-8e314e734ffe] Successfully created port: c8bb20fd-5c85-4e89-8c61-13a4f17dd4cc {{(pid=75234) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:548}}
Nov 29 03:06:04 user nova-compute[75234]: DEBUG nova.network.neutron [req-a995e396-36ff-4fa1-9130-a7ce05652708 req-7db4e245-1415-4138-b920-299ae83bdf2c service nova] [instance: 9f7876ad-0487-426d-8b19-b718dccdc4fe] Updated VIF entry in instance network info cache for port 7dd048ce-2c9b-46ac-8e9d-8c6a2ab201f5. {{(pid=75234) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3481}}
Nov 29 03:06:04 user nova-compute[75234]: DEBUG nova.network.neutron [req-a995e396-36ff-4fa1-9130-a7ce05652708 req-7db4e245-1415-4138-b920-299ae83bdf2c service nova] [instance: 9f7876ad-0487-426d-8b19-b718dccdc4fe] Updating instance_info_cache with network_info: [{"id": "7dd048ce-2c9b-46ac-8e9d-8c6a2ab201f5", "address": "fa:16:3e:fd:bb:d1", "network": {"id": "c810861d-949e-423d-a3b5-a37f66f29342", "bridge": "br-int", "label": "tempest-DeleteServersTestJSON-2024350588-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a21183b0b4f947e39218bb8e7c857e50", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap7dd048ce-2c", "ovs_interfaceid": "7dd048ce-2c9b-46ac-8e9d-8c6a2ab201f5", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:06:04 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-a995e396-36ff-4fa1-9130-a7ce05652708 req-7db4e245-1415-4138-b920-299ae83bdf2c service nova] Releasing lock "refresh_cache-9f7876ad-0487-426d-8b19-b718dccdc4fe" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:06:04 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:06:04 user sudo[84515]: pam_unix(sudo:session): session closed for user root
Nov 29 03:06:04 user nova-compute[75234]: INFO oslo.privsep.daemon [None req-61f7b6c4-f50c-48e8-be6f-f4711383b8cc tempest-DeleteServersTestJSON-1668990756 tempest-DeleteServersTestJSON-1668990756-project-member] Spawned new privsep daemon via rootwrap
Nov 29 03:06:04 user nova-compute[75234]: INFO oslo.privsep.daemon [-] privsep daemon starting
Nov 29 03:06:04 user nova-compute[75234]: INFO oslo.privsep.daemon [-] privsep process running with uid/gid: 0/0
Nov 29 03:06:04 user nova-compute[75234]: INFO oslo.privsep.daemon [-] privsep process running with capabilities (eff/prm/inh): CAP_DAC_OVERRIDE|CAP_NET_ADMIN/CAP_DAC_OVERRIDE|CAP_NET_ADMIN/none
Nov 29 03:06:04 user nova-compute[75234]: INFO oslo.privsep.daemon [-] privsep daemon running as pid 84519
Nov 29 03:06:05 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:06:05 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tap7dd048ce-2c, may_exist=True, interface_attrs={}) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:06:05 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Port, record=tap7dd048ce-2c, col_values=(('qos', UUID('ca3ebb37-3ef3-4d9d-8787-1e1608b3fbac')),), if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:06:05 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DbSetCommand(_result=None, table=Interface, record=tap7dd048ce-2c, col_values=(('external_ids', {'iface-id': '7dd048ce-2c9b-46ac-8e9d-8c6a2ab201f5', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:fd:bb:d1', 'vm-uuid': '9f7876ad-0487-426d-8b19-b718dccdc4fe'}),), if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:06:05 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:06:05 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:06:05 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:06:05 user nova-compute[75234]: INFO os_vif [None req-61f7b6c4-f50c-48e8-be6f-f4711383b8cc tempest-DeleteServersTestJSON-1668990756 tempest-DeleteServersTestJSON-1668990756-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:fd:bb:d1,bridge_name='br-int',has_traffic_filtering=True,id=7dd048ce-2c9b-46ac-8e9d-8c6a2ab201f5,network=Network(c810861d-949e-423d-a3b5-a37f66f29342),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap7dd048ce-2c')
Nov 29 03:06:05 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-61f7b6c4-f50c-48e8-be6f-f4711383b8cc tempest-DeleteServersTestJSON-1668990756 tempest-DeleteServersTestJSON-1668990756-project-member] No BDM found with device name vda, not building metadata. {{(pid=75234) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12089}}
Nov 29 03:06:05 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-61f7b6c4-f50c-48e8-be6f-f4711383b8cc tempest-DeleteServersTestJSON-1668990756 tempest-DeleteServersTestJSON-1668990756-project-member] No VIF found with MAC fa:16:3e:fd:bb:d1, not building metadata {{(pid=75234) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Nov 29 03:06:06 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-fcb1dbf7-0559-48fa-8e1a-0e39441efbec tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Acquiring lock "21e9c6cd-c39f-41c0-8768-5dd64589c3f3" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:06:06 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-fcb1dbf7-0559-48fa-8e1a-0e39441efbec tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Lock "21e9c6cd-c39f-41c0-8768-5dd64589c3f3" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:06:06 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:06:06 user nova-compute[75234]: DEBUG nova.compute.manager [None req-fcb1dbf7-0559-48fa-8e1a-0e39441efbec tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] [instance: 21e9c6cd-c39f-41c0-8768-5dd64589c3f3] Starting instance... {{(pid=75234) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2422}}
Nov 29 03:06:06 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-fcb1dbf7-0559-48fa-8e1a-0e39441efbec tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:06:06 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-fcb1dbf7-0559-48fa-8e1a-0e39441efbec tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:06:06 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-fcb1dbf7-0559-48fa-8e1a-0e39441efbec tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=75234) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2366}}
Nov 29 03:06:06 user nova-compute[75234]: INFO nova.compute.claims [None req-fcb1dbf7-0559-48fa-8e1a-0e39441efbec tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] [instance: 21e9c6cd-c39f-41c0-8768-5dd64589c3f3] Claim successful on node user
Nov 29 03:06:06 user nova-compute[75234]: DEBUG nova.compute.provider_tree [None req-fcb1dbf7-0559-48fa-8e1a-0e39441efbec tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Inventory has not changed in ProviderTree for provider: 4d291220-7c37-4839-a0e3-b23c66502b74 {{(pid=75234) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Nov 29 03:06:06 user nova-compute[75234]: DEBUG nova.scheduler.client.report [None req-fcb1dbf7-0559-48fa-8e1a-0e39441efbec tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Inventory has not changed for provider 4d291220-7c37-4839-a0e3-b23c66502b74 based on inventory data: {'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=75234) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Nov 29 03:06:06 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-fcb1dbf7-0559-48fa-8e1a-0e39441efbec tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.490s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:06:06 user nova-compute[75234]: DEBUG nova.compute.manager [None req-fcb1dbf7-0559-48fa-8e1a-0e39441efbec tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] [instance: 21e9c6cd-c39f-41c0-8768-5dd64589c3f3] Start building networks asynchronously for instance. {{(pid=75234) _build_resources /opt/stack/nova/nova/compute/manager.py:2819}}
Nov 29 03:06:06 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:06:06 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:06:06 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:06:06 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:06:06 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:06:06 user nova-compute[75234]: DEBUG nova.compute.manager [None req-fcb1dbf7-0559-48fa-8e1a-0e39441efbec tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] [instance: 21e9c6cd-c39f-41c0-8768-5dd64589c3f3] Allocating IP information in the background. {{(pid=75234) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1972}}
Nov 29 03:06:06 user nova-compute[75234]: DEBUG nova.network.neutron [None req-fcb1dbf7-0559-48fa-8e1a-0e39441efbec tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] [instance: 21e9c6cd-c39f-41c0-8768-5dd64589c3f3] allocate_for_instance() {{(pid=75234) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1156}}
Nov 29 03:06:06 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-fcb1dbf7-0559-48fa-8e1a-0e39441efbec tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] [instance: 21e9c6cd-c39f-41c0-8768-5dd64589c3f3] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Nov 29 03:06:06 user nova-compute[75234]: DEBUG nova.network.neutron [None req-997f592d-b1eb-43b5-b6d7-0f22555038e1 tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] [instance: 2bf9fe41-63c2-4882-bcdd-8e314e734ffe] Successfully updated port: c8bb20fd-5c85-4e89-8c61-13a4f17dd4cc {{(pid=75234) _update_port /opt/stack/nova/nova/network/neutron.py:586}}
Nov 29 03:06:06 user nova-compute[75234]: DEBUG nova.compute.manager [None req-fcb1dbf7-0559-48fa-8e1a-0e39441efbec tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] [instance: 21e9c6cd-c39f-41c0-8768-5dd64589c3f3] Start building block device mappings for instance. {{(pid=75234) _build_resources /opt/stack/nova/nova/compute/manager.py:2854}}
Nov 29 03:06:06 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-997f592d-b1eb-43b5-b6d7-0f22555038e1 tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] Acquiring lock "refresh_cache-2bf9fe41-63c2-4882-bcdd-8e314e734ffe" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:06:06 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-997f592d-b1eb-43b5-b6d7-0f22555038e1 tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] Acquired lock "refresh_cache-2bf9fe41-63c2-4882-bcdd-8e314e734ffe" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:06:06 user nova-compute[75234]: DEBUG nova.network.neutron [None req-997f592d-b1eb-43b5-b6d7-0f22555038e1 tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] [instance: 2bf9fe41-63c2-4882-bcdd-8e314e734ffe] Building network info cache for instance {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2009}}
Nov 29 03:06:07 user nova-compute[75234]: DEBUG nova.policy [None req-fcb1dbf7-0559-48fa-8e1a-0e39441efbec tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': 'd611826f3be947c0947b99c763c26d44', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c1cb3d1f12f242a785a688d59dbe3cd6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=75234) authorize /opt/stack/nova/nova/policy.py:203}}
Nov 29 03:06:07 user nova-compute[75234]: DEBUG nova.network.neutron [None req-997f592d-b1eb-43b5-b6d7-0f22555038e1 tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] [instance: 2bf9fe41-63c2-4882-bcdd-8e314e734ffe] Instance cache missing network info. {{(pid=75234) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3322}}
Nov 29 03:06:07 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ceb7826f-126f-40b1-b7f1-9afa48110fc2 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Acquiring lock "f609a7b7-1be2-44b4-9177-7e72a4d4c060" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:06:07 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ceb7826f-126f-40b1-b7f1-9afa48110fc2 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Lock "f609a7b7-1be2-44b4-9177-7e72a4d4c060" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:06:07 user nova-compute[75234]: DEBUG nova.compute.manager [None req-fcb1dbf7-0559-48fa-8e1a-0e39441efbec tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] [instance: 21e9c6cd-c39f-41c0-8768-5dd64589c3f3] Start spawning the instance on the hypervisor. {{(pid=75234) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2628}}
Nov 29 03:06:07 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-fcb1dbf7-0559-48fa-8e1a-0e39441efbec tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] [instance: 21e9c6cd-c39f-41c0-8768-5dd64589c3f3] Creating instance directory {{(pid=75234) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4727}}
Nov 29 03:06:07 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-fcb1dbf7-0559-48fa-8e1a-0e39441efbec tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] [instance: 21e9c6cd-c39f-41c0-8768-5dd64589c3f3] Creating image(s)
Nov 29 03:06:07 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-fcb1dbf7-0559-48fa-8e1a-0e39441efbec tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Acquiring lock "/opt/stack/data/nova/instances/21e9c6cd-c39f-41c0-8768-5dd64589c3f3/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:06:07 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-fcb1dbf7-0559-48fa-8e1a-0e39441efbec tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Lock "/opt/stack/data/nova/instances/21e9c6cd-c39f-41c0-8768-5dd64589c3f3/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:06:07 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-fcb1dbf7-0559-48fa-8e1a-0e39441efbec tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Lock "/opt/stack/data/nova/instances/21e9c6cd-c39f-41c0-8768-5dd64589c3f3/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.002s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:06:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-fcb1dbf7-0559-48fa-8e1a-0e39441efbec tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:06:07 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ceb7826f-126f-40b1-b7f1-9afa48110fc2 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] [instance: f609a7b7-1be2-44b4-9177-7e72a4d4c060] Starting instance... {{(pid=75234) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2422}}
Nov 29 03:06:07 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-c696a156-5f79-4939-99e6-fd7a518710c4 tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Acquiring lock "a9350a18-14e6-42c6-a150-0a1487f20ad9" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:06:07 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-c696a156-5f79-4939-99e6-fd7a518710c4 tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Lock "a9350a18-14e6-42c6-a150-0a1487f20ad9" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:06:07 user nova-compute[75234]: DEBUG nova.compute.manager [None req-c696a156-5f79-4939-99e6-fd7a518710c4 tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] [instance: a9350a18-14e6-42c6-a150-0a1487f20ad9] Starting instance... {{(pid=75234) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2422}}
Nov 29 03:06:07 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ceb7826f-126f-40b1-b7f1-9afa48110fc2 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:06:07 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ceb7826f-126f-40b1-b7f1-9afa48110fc2 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:06:07 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-ceb7826f-126f-40b1-b7f1-9afa48110fc2 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=75234) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2366}}
Nov 29 03:06:07 user nova-compute[75234]: INFO nova.compute.claims [None req-ceb7826f-126f-40b1-b7f1-9afa48110fc2 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] [instance: f609a7b7-1be2-44b4-9177-7e72a4d4c060] Claim successful on node user
Nov 29 03:06:07 user nova-compute[75234]: DEBUG nova.compute.manager [req-51c33efb-7cdd-45dd-aa86-1c3fbd7ae2c3 req-28d56f46-5d0c-4f24-ba58-2a14b05abc57 service nova] [instance: 2bf9fe41-63c2-4882-bcdd-8e314e734ffe] Received event network-changed-c8bb20fd-5c85-4e89-8c61-13a4f17dd4cc {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:06:07 user nova-compute[75234]: DEBUG nova.compute.manager [req-51c33efb-7cdd-45dd-aa86-1c3fbd7ae2c3 req-28d56f46-5d0c-4f24-ba58-2a14b05abc57 service nova] [instance: 2bf9fe41-63c2-4882-bcdd-8e314e734ffe] Refreshing instance network info cache due to event network-changed-c8bb20fd-5c85-4e89-8c61-13a4f17dd4cc. {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11117}}
Nov 29 03:06:07 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-51c33efb-7cdd-45dd-aa86-1c3fbd7ae2c3 req-28d56f46-5d0c-4f24-ba58-2a14b05abc57 service nova] Acquiring lock "refresh_cache-2bf9fe41-63c2-4882-bcdd-8e314e734ffe" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:06:07 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-c696a156-5f79-4939-99e6-fd7a518710c4 tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:06:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-fcb1dbf7-0559-48fa-8e1a-0e39441efbec tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json" returned: 0 in 0.289s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:06:07 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-fcb1dbf7-0559-48fa-8e1a-0e39441efbec tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Acquiring lock "80686e29dc9db6600d03e85cb62af298e12d7918" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:06:07 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-fcb1dbf7-0559-48fa-8e1a-0e39441efbec tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Lock "80686e29dc9db6600d03e85cb62af298e12d7918" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:06:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-fcb1dbf7-0559-48fa-8e1a-0e39441efbec tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:06:07 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:06:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-fcb1dbf7-0559-48fa-8e1a-0e39441efbec tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json" returned: 0 in 0.176s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:06:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-fcb1dbf7-0559-48fa-8e1a-0e39441efbec tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918,backing_fmt=raw /opt/stack/data/nova/instances/21e9c6cd-c39f-41c0-8768-5dd64589c3f3/disk 1073741824 {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:06:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-fcb1dbf7-0559-48fa-8e1a-0e39441efbec tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918,backing_fmt=raw /opt/stack/data/nova/instances/21e9c6cd-c39f-41c0-8768-5dd64589c3f3/disk 1073741824" returned: 0 in 0.052s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:06:07 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-fcb1dbf7-0559-48fa-8e1a-0e39441efbec tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Lock "80686e29dc9db6600d03e85cb62af298e12d7918" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.233s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:06:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-fcb1dbf7-0559-48fa-8e1a-0e39441efbec tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:06:07 user nova-compute[75234]: DEBUG nova.compute.provider_tree [None req-ceb7826f-126f-40b1-b7f1-9afa48110fc2 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Inventory has not changed in ProviderTree for provider: 4d291220-7c37-4839-a0e3-b23c66502b74 {{(pid=75234) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Nov 29 03:06:07 user nova-compute[75234]: DEBUG nova.scheduler.client.report [None req-ceb7826f-126f-40b1-b7f1-9afa48110fc2 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Inventory has not changed for provider 4d291220-7c37-4839-a0e3-b23c66502b74 based on inventory data: {'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=75234) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Nov 29 03:06:07 user nova-compute[75234]: DEBUG nova.network.neutron [None req-997f592d-b1eb-43b5-b6d7-0f22555038e1 tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] [instance: 2bf9fe41-63c2-4882-bcdd-8e314e734ffe] Updating instance_info_cache with network_info: [{"id": "c8bb20fd-5c85-4e89-8c61-13a4f17dd4cc", "address": "fa:16:3e:2c:c9:1d", "network": {"id": "b27ac31e-3601-4055-be79-34b0946b0913", "bridge": "br-int", "label": "tempest-ServersNegativeTestJSON-1491620118-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "b877668e02eb441c85dc2a7003a94c7a", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc8bb20fd-5c", "ovs_interfaceid": "c8bb20fd-5c85-4e89-8c61-13a4f17dd4cc", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:06:07 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ceb7826f-126f-40b1-b7f1-9afa48110fc2 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.507s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:06:07 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ceb7826f-126f-40b1-b7f1-9afa48110fc2 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] [instance: f609a7b7-1be2-44b4-9177-7e72a4d4c060] Start building networks asynchronously for instance. {{(pid=75234) _build_resources /opt/stack/nova/nova/compute/manager.py:2819}}
Nov 29 03:06:07 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-c696a156-5f79-4939-99e6-fd7a518710c4 tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.470s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:06:07 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-997f592d-b1eb-43b5-b6d7-0f22555038e1 tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] Releasing lock "refresh_cache-2bf9fe41-63c2-4882-bcdd-8e314e734ffe" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:06:07 user nova-compute[75234]: DEBUG nova.compute.manager [None req-997f592d-b1eb-43b5-b6d7-0f22555038e1 tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] [instance: 2bf9fe41-63c2-4882-bcdd-8e314e734ffe] Instance network_info: |[{"id": "c8bb20fd-5c85-4e89-8c61-13a4f17dd4cc", "address": "fa:16:3e:2c:c9:1d", "network": {"id": "b27ac31e-3601-4055-be79-34b0946b0913", "bridge": "br-int", "label": "tempest-ServersNegativeTestJSON-1491620118-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "b877668e02eb441c85dc2a7003a94c7a", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc8bb20fd-5c", "ovs_interfaceid": "c8bb20fd-5c85-4e89-8c61-13a4f17dd4cc", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=75234) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1987}}
Nov 29 03:06:07 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-51c33efb-7cdd-45dd-aa86-1c3fbd7ae2c3 req-28d56f46-5d0c-4f24-ba58-2a14b05abc57 service nova] Acquired lock "refresh_cache-2bf9fe41-63c2-4882-bcdd-8e314e734ffe" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:06:07 user nova-compute[75234]: DEBUG nova.network.neutron [req-51c33efb-7cdd-45dd-aa86-1c3fbd7ae2c3 req-28d56f46-5d0c-4f24-ba58-2a14b05abc57 service nova] [instance: 2bf9fe41-63c2-4882-bcdd-8e314e734ffe] Refreshing network info cache for port c8bb20fd-5c85-4e89-8c61-13a4f17dd4cc {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2006}}
Nov 29 03:06:07 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-997f592d-b1eb-43b5-b6d7-0f22555038e1 tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] [instance: 2bf9fe41-63c2-4882-bcdd-8e314e734ffe] Start _get_guest_xml network_info=[{"id": "c8bb20fd-5c85-4e89-8c61-13a4f17dd4cc", "address": "fa:16:3e:2c:c9:1d", "network": {"id": "b27ac31e-3601-4055-be79-34b0946b0913", "bridge": "br-int", "label": "tempest-ServersNegativeTestJSON-1491620118-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "b877668e02eb441c85dc2a7003a94c7a", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc8bb20fd-5c", "ovs_interfaceid": "c8bb20fd-5c85-4e89-8c61-13a4f17dd4cc", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum='c8fc807773e5354afe61636071771906',container_format='bare',created_at=2023-11-29T03:02:15Z,direct_url=<?>,disk_format='qcow2',id=96c30ba7-9b73-4a30-8ba4-7138879573b7,min_disk=0,min_ram=0,name='cirros-0.6.2-x86_64-disk',owner='b18e380921c748b78517b6f1211cd0f1',properties=ImageMetaProps,protected=<?>,size=21430272,status='active',tags=<?>,updated_at=2023-11-29T03:02:17Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [{'device_name': '/dev/vda', 'encryption_format': None, 'encrypted': False, 'device_type': 'disk', 'size': 0, 'disk_bus': 'virtio', 'boot_index': 0, 'encryption_options': None, 'encryption_secret_uuid': None, 'guest_format': None, 'image_id': '96c30ba7-9b73-4a30-8ba4-7138879573b7'}], 'ephemerals': [], 'block_device_mapping': [], 'swap': None} {{(pid=75234) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7562}}
Nov 29 03:06:07 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-c696a156-5f79-4939-99e6-fd7a518710c4 tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=75234) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2366}}
Nov 29 03:06:07 user nova-compute[75234]: INFO nova.compute.claims [None req-c696a156-5f79-4939-99e6-fd7a518710c4 tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] [instance: a9350a18-14e6-42c6-a150-0a1487f20ad9] Claim successful on node user
Nov 29 03:06:07 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-997f592d-b1eb-43b5-b6d7-0f22555038e1 tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:06:07 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-997f592d-b1eb-43b5-b6d7-0f22555038e1 tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:06:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-fcb1dbf7-0559-48fa-8e1a-0e39441efbec tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json" returned: 0 in 0.195s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:06:07 user nova-compute[75234]: DEBUG nova.virt.disk.api [None req-fcb1dbf7-0559-48fa-8e1a-0e39441efbec tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Checking if we can resize image /opt/stack/data/nova/instances/21e9c6cd-c39f-41c0-8768-5dd64589c3f3/disk. size=1073741824 {{(pid=75234) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:166}}
Nov 29 03:06:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-fcb1dbf7-0559-48fa-8e1a-0e39441efbec tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/21e9c6cd-c39f-41c0-8768-5dd64589c3f3/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:06:07 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ceb7826f-126f-40b1-b7f1-9afa48110fc2 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] [instance: f609a7b7-1be2-44b4-9177-7e72a4d4c060] Allocating IP information in the background. {{(pid=75234) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1972}}
Nov 29 03:06:07 user nova-compute[75234]: DEBUG nova.network.neutron [None req-ceb7826f-126f-40b1-b7f1-9afa48110fc2 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] [instance: f609a7b7-1be2-44b4-9177-7e72a4d4c060] allocate_for_instance() {{(pid=75234) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1156}}
Nov 29 03:06:07 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-997f592d-b1eb-43b5-b6d7-0f22555038e1 tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] Searching host: 'user' for CPU controller through CGroups V1... {{(pid=75234) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1652}}
Nov 29 03:06:07 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-997f592d-b1eb-43b5-b6d7-0f22555038e1 tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] CPU controller missing on host. {{(pid=75234) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1662}}
Nov 29 03:06:07 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-997f592d-b1eb-43b5-b6d7-0f22555038e1 tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] Searching host: 'user' for CPU controller through CGroups V2... {{(pid=75234) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1671}}
Nov 29 03:06:07 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-997f592d-b1eb-43b5-b6d7-0f22555038e1 tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] CPU controller found on host. {{(pid=75234) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1678}}
Nov 29 03:06:07 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-997f592d-b1eb-43b5-b6d7-0f22555038e1 tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=75234) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5400}}
Nov 29 03:06:07 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-997f592d-b1eb-43b5-b6d7-0f22555038e1 tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-11-29T03:02:04Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='1',id=6,is_public=True,memory_mb=512,name='m1.tiny',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='c8fc807773e5354afe61636071771906',container_format='bare',created_at=2023-11-29T03:02:15Z,direct_url=<?>,disk_format='qcow2',id=96c30ba7-9b73-4a30-8ba4-7138879573b7,min_disk=0,min_ram=0,name='cirros-0.6.2-x86_64-disk',owner='b18e380921c748b78517b6f1211cd0f1',properties=ImageMetaProps,protected=<?>,size=21430272,status='active',tags=<?>,updated_at=2023-11-29T03:02:17Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:563}}
Nov 29 03:06:07 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-997f592d-b1eb-43b5-b6d7-0f22555038e1 tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] Flavor limits 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:348}}
Nov 29 03:06:07 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-997f592d-b1eb-43b5-b6d7-0f22555038e1 tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] Image limits 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:352}}
Nov 29 03:06:07 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-997f592d-b1eb-43b5-b6d7-0f22555038e1 tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] Flavor pref 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:388}}
Nov 29 03:06:07 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-997f592d-b1eb-43b5-b6d7-0f22555038e1 tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] Image pref 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:392}}
Nov 29 03:06:07 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-997f592d-b1eb-43b5-b6d7-0f22555038e1 tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:430}}
Nov 29 03:06:07 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-997f592d-b1eb-43b5-b6d7-0f22555038e1 tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:569}}
Nov 29 03:06:07 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-997f592d-b1eb-43b5-b6d7-0f22555038e1 tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=75234) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:471}}
Nov 29 03:06:07 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-997f592d-b1eb-43b5-b6d7-0f22555038e1 tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] Got 1 possible topologies {{(pid=75234) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:501}}
Nov 29 03:06:07 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-997f592d-b1eb-43b5-b6d7-0f22555038e1 tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:575}}
Nov 29 03:06:07 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-997f592d-b1eb-43b5-b6d7-0f22555038e1 tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:577}}
Nov 29 03:06:08 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-997f592d-b1eb-43b5-b6d7-0f22555038e1 tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-11-29T03:05:59Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-ServersNegativeTestJSON-server-618399672',display_name='tempest-ServersNegativeTestJSON-server-618399672',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-serversnegativetestjson-server-618399672',id=2,image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data=None,key_name=None,keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='b877668e02eb441c85dc2a7003a94c7a',ramdisk_id='',reservation_id='r-poo1d0eh',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-ServersNegativeTestJSON-20765276',owner_user_name='tempest-ServersNegativeTestJSON-20765276-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-11-29T03:06:01Z,user_data=None,user_id='178bfe0ea37e45d8abd2d6b45bd5660c',uuid=2bf9fe41-63c2-4882-bcdd-8e314e734ffe,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "c8bb20fd-5c85-4e89-8c61-13a4f17dd4cc", "address": "fa:16:3e:2c:c9:1d", "network": {"id": "b27ac31e-3601-4055-be79-34b0946b0913", "bridge": "br-int", "label": "tempest-ServersNegativeTestJSON-1491620118-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "b877668e02eb441c85dc2a7003a94c7a", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc8bb20fd-5c", "ovs_interfaceid": "c8bb20fd-5c85-4e89-8c61-13a4f17dd4cc", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=75234) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Nov 29 03:06:08 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-997f592d-b1eb-43b5-b6d7-0f22555038e1 tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] Converting VIF {"id": "c8bb20fd-5c85-4e89-8c61-13a4f17dd4cc", "address": "fa:16:3e:2c:c9:1d", "network": {"id": "b27ac31e-3601-4055-be79-34b0946b0913", "bridge": "br-int", "label": "tempest-ServersNegativeTestJSON-1491620118-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "b877668e02eb441c85dc2a7003a94c7a", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc8bb20fd-5c", "ovs_interfaceid": "c8bb20fd-5c85-4e89-8c61-13a4f17dd4cc", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:06:08 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-997f592d-b1eb-43b5-b6d7-0f22555038e1 tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:2c:c9:1d,bridge_name='br-int',has_traffic_filtering=True,id=c8bb20fd-5c85-4e89-8c61-13a4f17dd4cc,network=Network(b27ac31e-3601-4055-be79-34b0946b0913),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapc8bb20fd-5c') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:06:08 user nova-compute[75234]: DEBUG nova.objects.instance [None req-997f592d-b1eb-43b5-b6d7-0f22555038e1 tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] Lazy-loading 'pci_devices' on Instance uuid 2bf9fe41-63c2-4882-bcdd-8e314e734ffe {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:06:08 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-ceb7826f-126f-40b1-b7f1-9afa48110fc2 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] [instance: f609a7b7-1be2-44b4-9177-7e72a4d4c060] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Nov 29 03:06:08 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-997f592d-b1eb-43b5-b6d7-0f22555038e1 tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] [instance: 2bf9fe41-63c2-4882-bcdd-8e314e734ffe] End _get_guest_xml xml=<domain type="kvm">
Nov 29 03:06:08 user nova-compute[75234]:   <uuid>2bf9fe41-63c2-4882-bcdd-8e314e734ffe</uuid>
Nov 29 03:06:08 user nova-compute[75234]:   <name>instance-00000002</name>
Nov 29 03:06:08 user nova-compute[75234]:   <memory>524288</memory>
Nov 29 03:06:08 user nova-compute[75234]:   <vcpu>1</vcpu>
Nov 29 03:06:08 user nova-compute[75234]:   <metadata>
Nov 29 03:06:08 user nova-compute[75234]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Nov 29 03:06:08 user nova-compute[75234]:       <nova:package version="0.0.0"/>
Nov 29 03:06:08 user nova-compute[75234]:       <nova:name>tempest-ServersNegativeTestJSON-server-618399672</nova:name>
Nov 29 03:06:08 user nova-compute[75234]:       <nova:creationTime>2023-11-29 03:06:07</nova:creationTime>
Nov 29 03:06:08 user nova-compute[75234]:       <nova:flavor name="m1.tiny">
Nov 29 03:06:08 user nova-compute[75234]:         <nova:memory>512</nova:memory>
Nov 29 03:06:08 user nova-compute[75234]:         <nova:disk>1</nova:disk>
Nov 29 03:06:08 user nova-compute[75234]:         <nova:swap>0</nova:swap>
Nov 29 03:06:08 user nova-compute[75234]:         <nova:ephemeral>0</nova:ephemeral>
Nov 29 03:06:08 user nova-compute[75234]:         <nova:vcpus>1</nova:vcpus>
Nov 29 03:06:08 user nova-compute[75234]:       </nova:flavor>
Nov 29 03:06:08 user nova-compute[75234]:       <nova:owner>
Nov 29 03:06:08 user nova-compute[75234]:         <nova:user uuid="178bfe0ea37e45d8abd2d6b45bd5660c">tempest-ServersNegativeTestJSON-20765276-project-member</nova:user>
Nov 29 03:06:08 user nova-compute[75234]:         <nova:project uuid="b877668e02eb441c85dc2a7003a94c7a">tempest-ServersNegativeTestJSON-20765276</nova:project>
Nov 29 03:06:08 user nova-compute[75234]:       </nova:owner>
Nov 29 03:06:08 user nova-compute[75234]:       <nova:root type="image" uuid="96c30ba7-9b73-4a30-8ba4-7138879573b7"/>
Nov 29 03:06:08 user nova-compute[75234]:       <nova:ports>
Nov 29 03:06:08 user nova-compute[75234]:         <nova:port uuid="c8bb20fd-5c85-4e89-8c61-13a4f17dd4cc">
Nov 29 03:06:08 user nova-compute[75234]:           <nova:ip type="fixed" address="10.0.0.10" ipVersion="4"/>
Nov 29 03:06:08 user nova-compute[75234]:         </nova:port>
Nov 29 03:06:08 user nova-compute[75234]:       </nova:ports>
Nov 29 03:06:08 user nova-compute[75234]:     </nova:instance>
Nov 29 03:06:08 user nova-compute[75234]:   </metadata>
Nov 29 03:06:08 user nova-compute[75234]:   <sysinfo type="smbios">
Nov 29 03:06:08 user nova-compute[75234]:     <system>
Nov 29 03:06:08 user nova-compute[75234]:       <entry name="manufacturer">OpenStack Foundation</entry>
Nov 29 03:06:08 user nova-compute[75234]:       <entry name="product">OpenStack Nova</entry>
Nov 29 03:06:08 user nova-compute[75234]:       <entry name="version">0.0.0</entry>
Nov 29 03:06:08 user nova-compute[75234]:       <entry name="serial">2bf9fe41-63c2-4882-bcdd-8e314e734ffe</entry>
Nov 29 03:06:08 user nova-compute[75234]:       <entry name="uuid">2bf9fe41-63c2-4882-bcdd-8e314e734ffe</entry>
Nov 29 03:06:08 user nova-compute[75234]:       <entry name="family">Virtual Machine</entry>
Nov 29 03:06:08 user nova-compute[75234]:     </system>
Nov 29 03:06:08 user nova-compute[75234]:   </sysinfo>
Nov 29 03:06:08 user nova-compute[75234]:   <os>
Nov 29 03:06:08 user nova-compute[75234]:     <type arch="x86_64" machine="pc">hvm</type>
Nov 29 03:06:08 user nova-compute[75234]:     <boot dev="hd"/>
Nov 29 03:06:08 user nova-compute[75234]:     <smbios mode="sysinfo"/>
Nov 29 03:06:08 user nova-compute[75234]:   </os>
Nov 29 03:06:08 user nova-compute[75234]:   <features>
Nov 29 03:06:08 user nova-compute[75234]:     <acpi/>
Nov 29 03:06:08 user nova-compute[75234]:     <apic/>
Nov 29 03:06:08 user nova-compute[75234]:     <vmcoreinfo/>
Nov 29 03:06:08 user nova-compute[75234]:   </features>
Nov 29 03:06:08 user nova-compute[75234]:   <clock offset="utc">
Nov 29 03:06:08 user nova-compute[75234]:     <timer name="pit" tickpolicy="delay"/>
Nov 29 03:06:08 user nova-compute[75234]:     <timer name="rtc" tickpolicy="catchup"/>
Nov 29 03:06:08 user nova-compute[75234]:     <timer name="hpet" present="no"/>
Nov 29 03:06:08 user nova-compute[75234]:   </clock>
Nov 29 03:06:08 user nova-compute[75234]:   <cpu mode="custom" match="exact">
Nov 29 03:06:08 user nova-compute[75234]:     <model>Nehalem</model>
Nov 29 03:06:08 user nova-compute[75234]:     <topology sockets="1" cores="1" threads="1"/>
Nov 29 03:06:08 user nova-compute[75234]:   </cpu>
Nov 29 03:06:08 user nova-compute[75234]:   <devices>
Nov 29 03:06:08 user nova-compute[75234]:     <disk type="file" device="disk">
Nov 29 03:06:08 user nova-compute[75234]:       <driver name="qemu" type="qcow2" cache="none"/>
Nov 29 03:06:08 user nova-compute[75234]:       <source file="/opt/stack/data/nova/instances/2bf9fe41-63c2-4882-bcdd-8e314e734ffe/disk"/>
Nov 29 03:06:08 user nova-compute[75234]:       <target dev="vda" bus="virtio"/>
Nov 29 03:06:08 user nova-compute[75234]:     </disk>
Nov 29 03:06:08 user nova-compute[75234]:     <interface type="ethernet">
Nov 29 03:06:08 user nova-compute[75234]:       <mac address="fa:16:3e:2c:c9:1d"/>
Nov 29 03:06:08 user nova-compute[75234]:       <model type="virtio"/>
Nov 29 03:06:08 user nova-compute[75234]:       <mtu size="1442"/>
Nov 29 03:06:08 user nova-compute[75234]:       <target dev="tapc8bb20fd-5c"/>
Nov 29 03:06:08 user nova-compute[75234]:     </interface>
Nov 29 03:06:08 user nova-compute[75234]:     <serial type="pty">
Nov 29 03:06:08 user nova-compute[75234]:       <log file="/opt/stack/data/nova/instances/2bf9fe41-63c2-4882-bcdd-8e314e734ffe/console.log" append="off"/>
Nov 29 03:06:08 user nova-compute[75234]:     </serial>
Nov 29 03:06:08 user nova-compute[75234]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Nov 29 03:06:08 user nova-compute[75234]:     <video>
Nov 29 03:06:08 user nova-compute[75234]:       <model type="virtio"/>
Nov 29 03:06:08 user nova-compute[75234]:     </video>
Nov 29 03:06:08 user nova-compute[75234]:     <rng model="virtio">
Nov 29 03:06:08 user nova-compute[75234]:       <backend model="random">/dev/urandom</backend>
Nov 29 03:06:08 user nova-compute[75234]:     </rng>
Nov 29 03:06:08 user nova-compute[75234]:     <controller type="usb" index="0" model="none"/>
Nov 29 03:06:08 user nova-compute[75234]:     <memballoon model="virtio">
Nov 29 03:06:08 user nova-compute[75234]:       <stats period="10"/>
Nov 29 03:06:08 user nova-compute[75234]:     </memballoon>
Nov 29 03:06:08 user nova-compute[75234]:   </devices>
Nov 29 03:06:08 user nova-compute[75234]: </domain>
Nov 29 03:06:08 user nova-compute[75234]:  {{(pid=75234) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7568}}
Nov 29 03:06:08 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-997f592d-b1eb-43b5-b6d7-0f22555038e1 tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-11-29T03:05:59Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-ServersNegativeTestJSON-server-618399672',display_name='tempest-ServersNegativeTestJSON-server-618399672',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-serversnegativetestjson-server-618399672',id=2,image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data=None,key_name=None,keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='b877668e02eb441c85dc2a7003a94c7a',ramdisk_id='',reservation_id='r-poo1d0eh',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-ServersNegativeTestJSON-20765276',owner_user_name='tempest-ServersNegativeTestJSON-20765276-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-11-29T03:06:01Z,user_data=None,user_id='178bfe0ea37e45d8abd2d6b45bd5660c',uuid=2bf9fe41-63c2-4882-bcdd-8e314e734ffe,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "c8bb20fd-5c85-4e89-8c61-13a4f17dd4cc", "address": "fa:16:3e:2c:c9:1d", "network": {"id": "b27ac31e-3601-4055-be79-34b0946b0913", "bridge": "br-int", "label": "tempest-ServersNegativeTestJSON-1491620118-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "b877668e02eb441c85dc2a7003a94c7a", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc8bb20fd-5c", "ovs_interfaceid": "c8bb20fd-5c85-4e89-8c61-13a4f17dd4cc", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Nov 29 03:06:08 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-997f592d-b1eb-43b5-b6d7-0f22555038e1 tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] Converting VIF {"id": "c8bb20fd-5c85-4e89-8c61-13a4f17dd4cc", "address": "fa:16:3e:2c:c9:1d", "network": {"id": "b27ac31e-3601-4055-be79-34b0946b0913", "bridge": "br-int", "label": "tempest-ServersNegativeTestJSON-1491620118-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "b877668e02eb441c85dc2a7003a94c7a", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc8bb20fd-5c", "ovs_interfaceid": "c8bb20fd-5c85-4e89-8c61-13a4f17dd4cc", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:06:08 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-997f592d-b1eb-43b5-b6d7-0f22555038e1 tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:2c:c9:1d,bridge_name='br-int',has_traffic_filtering=True,id=c8bb20fd-5c85-4e89-8c61-13a4f17dd4cc,network=Network(b27ac31e-3601-4055-be79-34b0946b0913),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapc8bb20fd-5c') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:06:08 user nova-compute[75234]: DEBUG os_vif [None req-997f592d-b1eb-43b5-b6d7-0f22555038e1 tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:2c:c9:1d,bridge_name='br-int',has_traffic_filtering=True,id=c8bb20fd-5c85-4e89-8c61-13a4f17dd4cc,network=Network(b27ac31e-3601-4055-be79-34b0946b0913),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapc8bb20fd-5c') {{(pid=75234) plug /opt/stack/data/venv/lib/python3.10/site-packages/os_vif/__init__.py:76}}
Nov 29 03:06:08 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ceb7826f-126f-40b1-b7f1-9afa48110fc2 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] [instance: f609a7b7-1be2-44b4-9177-7e72a4d4c060] Start building block device mappings for instance. {{(pid=75234) _build_resources /opt/stack/nova/nova/compute/manager.py:2854}}
Nov 29 03:06:08 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:06:08 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:06:08 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Nov 29 03:06:08 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:06:08 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbCreateCommand(_result=None, table=QoS, columns={'type': 'linux-noop', 'external_ids': {'id': 'bfda8a38-967e-59d3-a415-7c9acb4106d7', '_type': 'linux-noop'}}, row=False) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:06:08 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:06:08 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:06:08 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:06:08 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tapc8bb20fd-5c, may_exist=True, interface_attrs={}) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:06:08 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Port, record=tapc8bb20fd-5c, col_values=(('qos', UUID('1cf52a9a-1591-4592-b347-b5ae0527d249')),), if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:06:08 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DbSetCommand(_result=None, table=Interface, record=tapc8bb20fd-5c, col_values=(('external_ids', {'iface-id': 'c8bb20fd-5c85-4e89-8c61-13a4f17dd4cc', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:2c:c9:1d', 'vm-uuid': '2bf9fe41-63c2-4882-bcdd-8e314e734ffe'}),), if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:06:08 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:06:08 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-fcb1dbf7-0559-48fa-8e1a-0e39441efbec tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/21e9c6cd-c39f-41c0-8768-5dd64589c3f3/disk --force-share --output=json" returned: 0 in 0.261s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:06:08 user nova-compute[75234]: DEBUG nova.virt.disk.api [None req-fcb1dbf7-0559-48fa-8e1a-0e39441efbec tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Cannot resize image /opt/stack/data/nova/instances/21e9c6cd-c39f-41c0-8768-5dd64589c3f3/disk to a smaller size. {{(pid=75234) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:172}}
Nov 29 03:06:08 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-fcb1dbf7-0559-48fa-8e1a-0e39441efbec tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] [instance: 21e9c6cd-c39f-41c0-8768-5dd64589c3f3] Created local disks {{(pid=75234) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4861}}
Nov 29 03:06:08 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-fcb1dbf7-0559-48fa-8e1a-0e39441efbec tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] [instance: 21e9c6cd-c39f-41c0-8768-5dd64589c3f3] Ensure instance console log exists: /opt/stack/data/nova/instances/21e9c6cd-c39f-41c0-8768-5dd64589c3f3/console.log {{(pid=75234) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4613}}
Nov 29 03:06:08 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-fcb1dbf7-0559-48fa-8e1a-0e39441efbec tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:06:08 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-fcb1dbf7-0559-48fa-8e1a-0e39441efbec tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:06:08 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-fcb1dbf7-0559-48fa-8e1a-0e39441efbec tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:06:08 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:06:08 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:06:08 user nova-compute[75234]: INFO os_vif [None req-997f592d-b1eb-43b5-b6d7-0f22555038e1 tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:2c:c9:1d,bridge_name='br-int',has_traffic_filtering=True,id=c8bb20fd-5c85-4e89-8c61-13a4f17dd4cc,network=Network(b27ac31e-3601-4055-be79-34b0946b0913),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapc8bb20fd-5c')
Nov 29 03:06:08 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-997f592d-b1eb-43b5-b6d7-0f22555038e1 tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] No BDM found with device name vda, not building metadata. {{(pid=75234) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12089}}
Nov 29 03:06:08 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-997f592d-b1eb-43b5-b6d7-0f22555038e1 tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] No VIF found with MAC fa:16:3e:2c:c9:1d, not building metadata {{(pid=75234) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Nov 29 03:06:08 user nova-compute[75234]: DEBUG nova.policy [None req-ceb7826f-126f-40b1-b7f1-9afa48110fc2 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': 'bb1126c998ee430baac2e2494f30c0aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e393d23e783b4a40986674d782606c69', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=75234) authorize /opt/stack/nova/nova/policy.py:203}}
Nov 29 03:06:08 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ceb7826f-126f-40b1-b7f1-9afa48110fc2 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] [instance: f609a7b7-1be2-44b4-9177-7e72a4d4c060] Start spawning the instance on the hypervisor. {{(pid=75234) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2628}}
Nov 29 03:06:08 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-ceb7826f-126f-40b1-b7f1-9afa48110fc2 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] [instance: f609a7b7-1be2-44b4-9177-7e72a4d4c060] Creating instance directory {{(pid=75234) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4727}}
Nov 29 03:06:08 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-ceb7826f-126f-40b1-b7f1-9afa48110fc2 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] [instance: f609a7b7-1be2-44b4-9177-7e72a4d4c060] Creating image(s)
Nov 29 03:06:08 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ceb7826f-126f-40b1-b7f1-9afa48110fc2 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Acquiring lock "/opt/stack/data/nova/instances/f609a7b7-1be2-44b4-9177-7e72a4d4c060/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:06:08 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ceb7826f-126f-40b1-b7f1-9afa48110fc2 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Lock "/opt/stack/data/nova/instances/f609a7b7-1be2-44b4-9177-7e72a4d4c060/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:06:08 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ceb7826f-126f-40b1-b7f1-9afa48110fc2 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Lock "/opt/stack/data/nova/instances/f609a7b7-1be2-44b4-9177-7e72a4d4c060/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:06:08 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ceb7826f-126f-40b1-b7f1-9afa48110fc2 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:06:08 user nova-compute[75234]: DEBUG nova.compute.provider_tree [None req-c696a156-5f79-4939-99e6-fd7a518710c4 tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Inventory has not changed in ProviderTree for provider: 4d291220-7c37-4839-a0e3-b23c66502b74 {{(pid=75234) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Nov 29 03:06:08 user nova-compute[75234]: DEBUG nova.scheduler.client.report [None req-c696a156-5f79-4939-99e6-fd7a518710c4 tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Inventory has not changed for provider 4d291220-7c37-4839-a0e3-b23c66502b74 based on inventory data: {'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=75234) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Nov 29 03:06:08 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-c696a156-5f79-4939-99e6-fd7a518710c4 tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.636s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:06:08 user nova-compute[75234]: DEBUG nova.compute.manager [None req-c696a156-5f79-4939-99e6-fd7a518710c4 tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] [instance: a9350a18-14e6-42c6-a150-0a1487f20ad9] Start building networks asynchronously for instance. {{(pid=75234) _build_resources /opt/stack/nova/nova/compute/manager.py:2819}}
Nov 29 03:06:08 user nova-compute[75234]: DEBUG nova.compute.manager [None req-c696a156-5f79-4939-99e6-fd7a518710c4 tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] [instance: a9350a18-14e6-42c6-a150-0a1487f20ad9] Allocating IP information in the background. {{(pid=75234) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1972}}
Nov 29 03:06:08 user nova-compute[75234]: DEBUG nova.network.neutron [None req-c696a156-5f79-4939-99e6-fd7a518710c4 tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] [instance: a9350a18-14e6-42c6-a150-0a1487f20ad9] allocate_for_instance() {{(pid=75234) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1156}}
Nov 29 03:06:08 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ceb7826f-126f-40b1-b7f1-9afa48110fc2 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json" returned: 0 in 0.273s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:06:08 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ceb7826f-126f-40b1-b7f1-9afa48110fc2 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Acquiring lock "80686e29dc9db6600d03e85cb62af298e12d7918" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:06:08 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ceb7826f-126f-40b1-b7f1-9afa48110fc2 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Lock "80686e29dc9db6600d03e85cb62af298e12d7918" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:06:08 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ceb7826f-126f-40b1-b7f1-9afa48110fc2 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:06:08 user nova-compute[75234]: DEBUG nova.compute.manager [None req-61f7b6c4-f50c-48e8-be6f-f4711383b8cc tempest-DeleteServersTestJSON-1668990756 tempest-DeleteServersTestJSON-1668990756-project-member] [instance: 9f7876ad-0487-426d-8b19-b718dccdc4fe] Instance event wait completed in 0 seconds for  {{(pid=75234) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Nov 29 03:06:08 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-61f7b6c4-f50c-48e8-be6f-f4711383b8cc tempest-DeleteServersTestJSON-1668990756 tempest-DeleteServersTestJSON-1668990756-project-member] [instance: 9f7876ad-0487-426d-8b19-b718dccdc4fe] Guest created on hypervisor {{(pid=75234) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4421}}
Nov 29 03:06:08 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-c696a156-5f79-4939-99e6-fd7a518710c4 tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] [instance: a9350a18-14e6-42c6-a150-0a1487f20ad9] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Nov 29 03:06:08 user nova-compute[75234]: DEBUG nova.compute.manager [None req-c696a156-5f79-4939-99e6-fd7a518710c4 tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] [instance: a9350a18-14e6-42c6-a150-0a1487f20ad9] Start building block device mappings for instance. {{(pid=75234) _build_resources /opt/stack/nova/nova/compute/manager.py:2854}}
Nov 29 03:06:08 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:06:08 user nova-compute[75234]: INFO nova.virt.libvirt.driver [-] [instance: 9f7876ad-0487-426d-8b19-b718dccdc4fe] Instance spawned successfully.
Nov 29 03:06:08 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-61f7b6c4-f50c-48e8-be6f-f4711383b8cc tempest-DeleteServersTestJSON-1668990756 tempest-DeleteServersTestJSON-1668990756-project-member] [instance: 9f7876ad-0487-426d-8b19-b718dccdc4fe] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:904}}
Nov 29 03:06:08 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-61f7b6c4-f50c-48e8-be6f-f4711383b8cc tempest-DeleteServersTestJSON-1668990756 tempest-DeleteServersTestJSON-1668990756-project-member] [instance: 9f7876ad-0487-426d-8b19-b718dccdc4fe] Found default for hw_cdrom_bus of ide {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:06:08 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-61f7b6c4-f50c-48e8-be6f-f4711383b8cc tempest-DeleteServersTestJSON-1668990756 tempest-DeleteServersTestJSON-1668990756-project-member] [instance: 9f7876ad-0487-426d-8b19-b718dccdc4fe] Found default for hw_disk_bus of virtio {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:06:08 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-61f7b6c4-f50c-48e8-be6f-f4711383b8cc tempest-DeleteServersTestJSON-1668990756 tempest-DeleteServersTestJSON-1668990756-project-member] [instance: 9f7876ad-0487-426d-8b19-b718dccdc4fe] Found default for hw_input_bus of None {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:06:08 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-61f7b6c4-f50c-48e8-be6f-f4711383b8cc tempest-DeleteServersTestJSON-1668990756 tempest-DeleteServersTestJSON-1668990756-project-member] [instance: 9f7876ad-0487-426d-8b19-b718dccdc4fe] Found default for hw_pointer_model of None {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:06:08 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-61f7b6c4-f50c-48e8-be6f-f4711383b8cc tempest-DeleteServersTestJSON-1668990756 tempest-DeleteServersTestJSON-1668990756-project-member] [instance: 9f7876ad-0487-426d-8b19-b718dccdc4fe] Found default for hw_video_model of virtio {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:06:08 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-61f7b6c4-f50c-48e8-be6f-f4711383b8cc tempest-DeleteServersTestJSON-1668990756 tempest-DeleteServersTestJSON-1668990756-project-member] [instance: 9f7876ad-0487-426d-8b19-b718dccdc4fe] Found default for hw_vif_model of virtio {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:06:08 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-71e7f72f-cfc6-42f1-9655-a38fb0588077 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] Acquiring lock "8297c264-a213-44ee-be46-5c08eabda9f7" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:06:08 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-71e7f72f-cfc6-42f1-9655-a38fb0588077 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] Lock "8297c264-a213-44ee-be46-5c08eabda9f7" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:06:08 user nova-compute[75234]: DEBUG nova.compute.manager [None req-71e7f72f-cfc6-42f1-9655-a38fb0588077 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] [instance: 8297c264-a213-44ee-be46-5c08eabda9f7] Starting instance... {{(pid=75234) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2422}}
Nov 29 03:06:08 user nova-compute[75234]: DEBUG nova.policy [None req-c696a156-5f79-4939-99e6-fd7a518710c4 tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': 'bdddba0ef9c44b6cb43c32a9ab0e29f6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ad5ea22c63f04f91a3f2fa665e4fc8a1', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=75234) authorize /opt/stack/nova/nova/policy.py:203}}
Nov 29 03:06:08 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ceb7826f-126f-40b1-b7f1-9afa48110fc2 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json" returned: 0 in 0.258s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:06:08 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ceb7826f-126f-40b1-b7f1-9afa48110fc2 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918,backing_fmt=raw /opt/stack/data/nova/instances/f609a7b7-1be2-44b4-9177-7e72a4d4c060/disk 1073741824 {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:06:09 user nova-compute[75234]: DEBUG nova.compute.manager [None req-c696a156-5f79-4939-99e6-fd7a518710c4 tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] [instance: a9350a18-14e6-42c6-a150-0a1487f20ad9] Start spawning the instance on the hypervisor. {{(pid=75234) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2628}}
Nov 29 03:06:09 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-c696a156-5f79-4939-99e6-fd7a518710c4 tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] [instance: a9350a18-14e6-42c6-a150-0a1487f20ad9] Creating instance directory {{(pid=75234) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4727}}
Nov 29 03:06:09 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-c696a156-5f79-4939-99e6-fd7a518710c4 tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] [instance: a9350a18-14e6-42c6-a150-0a1487f20ad9] Creating image(s)
Nov 29 03:06:09 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-c696a156-5f79-4939-99e6-fd7a518710c4 tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Acquiring lock "/opt/stack/data/nova/instances/a9350a18-14e6-42c6-a150-0a1487f20ad9/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:06:09 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-c696a156-5f79-4939-99e6-fd7a518710c4 tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Lock "/opt/stack/data/nova/instances/a9350a18-14e6-42c6-a150-0a1487f20ad9/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:06:09 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-c696a156-5f79-4939-99e6-fd7a518710c4 tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Lock "/opt/stack/data/nova/instances/a9350a18-14e6-42c6-a150-0a1487f20ad9/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:06:09 user nova-compute[75234]: INFO nova.compute.manager [None req-61f7b6c4-f50c-48e8-be6f-f4711383b8cc tempest-DeleteServersTestJSON-1668990756 tempest-DeleteServersTestJSON-1668990756-project-member] [instance: 9f7876ad-0487-426d-8b19-b718dccdc4fe] Took 12.91 seconds to spawn the instance on the hypervisor.
Nov 29 03:06:09 user nova-compute[75234]: DEBUG nova.compute.manager [None req-61f7b6c4-f50c-48e8-be6f-f4711383b8cc tempest-DeleteServersTestJSON-1668990756 tempest-DeleteServersTestJSON-1668990756-project-member] [instance: 9f7876ad-0487-426d-8b19-b718dccdc4fe] Checking state {{(pid=75234) _get_power_state /opt/stack/nova/nova/compute/manager.py:1782}}
Nov 29 03:06:09 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-c696a156-5f79-4939-99e6-fd7a518710c4 tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:06:09 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ceb7826f-126f-40b1-b7f1-9afa48110fc2 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918,backing_fmt=raw /opt/stack/data/nova/instances/f609a7b7-1be2-44b4-9177-7e72a4d4c060/disk 1073741824" returned: 0 in 0.197s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:06:09 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ceb7826f-126f-40b1-b7f1-9afa48110fc2 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Lock "80686e29dc9db6600d03e85cb62af298e12d7918" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.495s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:06:09 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ceb7826f-126f-40b1-b7f1-9afa48110fc2 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:06:09 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-71e7f72f-cfc6-42f1-9655-a38fb0588077 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:06:09 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-71e7f72f-cfc6-42f1-9655-a38fb0588077 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.002s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:06:09 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-71e7f72f-cfc6-42f1-9655-a38fb0588077 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=75234) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2366}}
Nov 29 03:06:09 user nova-compute[75234]: INFO nova.compute.claims [None req-71e7f72f-cfc6-42f1-9655-a38fb0588077 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] [instance: 8297c264-a213-44ee-be46-5c08eabda9f7] Claim successful on node user
Nov 29 03:06:09 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-c696a156-5f79-4939-99e6-fd7a518710c4 tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json" returned: 0 in 0.238s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:06:09 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-c696a156-5f79-4939-99e6-fd7a518710c4 tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Acquiring lock "80686e29dc9db6600d03e85cb62af298e12d7918" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:06:09 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-c696a156-5f79-4939-99e6-fd7a518710c4 tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Lock "80686e29dc9db6600d03e85cb62af298e12d7918" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:06:09 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-c696a156-5f79-4939-99e6-fd7a518710c4 tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:06:09 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ceb7826f-126f-40b1-b7f1-9afa48110fc2 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json" returned: 0 in 0.248s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:06:09 user nova-compute[75234]: DEBUG nova.virt.disk.api [None req-ceb7826f-126f-40b1-b7f1-9afa48110fc2 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Checking if we can resize image /opt/stack/data/nova/instances/f609a7b7-1be2-44b4-9177-7e72a4d4c060/disk. size=1073741824 {{(pid=75234) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:166}}
Nov 29 03:06:09 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ceb7826f-126f-40b1-b7f1-9afa48110fc2 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f609a7b7-1be2-44b4-9177-7e72a4d4c060/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:06:09 user nova-compute[75234]: DEBUG nova.network.neutron [req-51c33efb-7cdd-45dd-aa86-1c3fbd7ae2c3 req-28d56f46-5d0c-4f24-ba58-2a14b05abc57 service nova] [instance: 2bf9fe41-63c2-4882-bcdd-8e314e734ffe] Updated VIF entry in instance network info cache for port c8bb20fd-5c85-4e89-8c61-13a4f17dd4cc. {{(pid=75234) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3481}}
Nov 29 03:06:09 user nova-compute[75234]: DEBUG nova.network.neutron [req-51c33efb-7cdd-45dd-aa86-1c3fbd7ae2c3 req-28d56f46-5d0c-4f24-ba58-2a14b05abc57 service nova] [instance: 2bf9fe41-63c2-4882-bcdd-8e314e734ffe] Updating instance_info_cache with network_info: [{"id": "c8bb20fd-5c85-4e89-8c61-13a4f17dd4cc", "address": "fa:16:3e:2c:c9:1d", "network": {"id": "b27ac31e-3601-4055-be79-34b0946b0913", "bridge": "br-int", "label": "tempest-ServersNegativeTestJSON-1491620118-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "b877668e02eb441c85dc2a7003a94c7a", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc8bb20fd-5c", "ovs_interfaceid": "c8bb20fd-5c85-4e89-8c61-13a4f17dd4cc", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:06:09 user nova-compute[75234]: INFO nova.compute.manager [None req-61f7b6c4-f50c-48e8-be6f-f4711383b8cc tempest-DeleteServersTestJSON-1668990756 tempest-DeleteServersTestJSON-1668990756-project-member] [instance: 9f7876ad-0487-426d-8b19-b718dccdc4fe] Took 14.20 seconds to build instance.
Nov 29 03:06:09 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-51c33efb-7cdd-45dd-aa86-1c3fbd7ae2c3 req-28d56f46-5d0c-4f24-ba58-2a14b05abc57 service nova] Releasing lock "refresh_cache-2bf9fe41-63c2-4882-bcdd-8e314e734ffe" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:06:09 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-61f7b6c4-f50c-48e8-be6f-f4711383b8cc tempest-DeleteServersTestJSON-1668990756 tempest-DeleteServersTestJSON-1668990756-project-member] Lock "9f7876ad-0487-426d-8b19-b718dccdc4fe" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 14.455s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:06:09 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-c696a156-5f79-4939-99e6-fd7a518710c4 tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json" returned: 0 in 0.262s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:06:09 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-c696a156-5f79-4939-99e6-fd7a518710c4 tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918,backing_fmt=raw /opt/stack/data/nova/instances/a9350a18-14e6-42c6-a150-0a1487f20ad9/disk 1073741824 {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:06:09 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:06:09 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ceb7826f-126f-40b1-b7f1-9afa48110fc2 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f609a7b7-1be2-44b4-9177-7e72a4d4c060/disk --force-share --output=json" returned: 0 in 0.245s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:06:09 user nova-compute[75234]: DEBUG nova.virt.disk.api [None req-ceb7826f-126f-40b1-b7f1-9afa48110fc2 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Cannot resize image /opt/stack/data/nova/instances/f609a7b7-1be2-44b4-9177-7e72a4d4c060/disk to a smaller size. {{(pid=75234) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:172}}
Nov 29 03:06:09 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-ceb7826f-126f-40b1-b7f1-9afa48110fc2 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] [instance: f609a7b7-1be2-44b4-9177-7e72a4d4c060] Created local disks {{(pid=75234) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4861}}
Nov 29 03:06:09 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-ceb7826f-126f-40b1-b7f1-9afa48110fc2 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] [instance: f609a7b7-1be2-44b4-9177-7e72a4d4c060] Ensure instance console log exists: /opt/stack/data/nova/instances/f609a7b7-1be2-44b4-9177-7e72a4d4c060/console.log {{(pid=75234) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4613}}
Nov 29 03:06:09 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ceb7826f-126f-40b1-b7f1-9afa48110fc2 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:06:09 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ceb7826f-126f-40b1-b7f1-9afa48110fc2 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:06:09 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ceb7826f-126f-40b1-b7f1-9afa48110fc2 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:06:09 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-c696a156-5f79-4939-99e6-fd7a518710c4 tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918,backing_fmt=raw /opt/stack/data/nova/instances/a9350a18-14e6-42c6-a150-0a1487f20ad9/disk 1073741824" returned: 0 in 0.094s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:06:09 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-c696a156-5f79-4939-99e6-fd7a518710c4 tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Lock "80686e29dc9db6600d03e85cb62af298e12d7918" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.360s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:06:09 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-c696a156-5f79-4939-99e6-fd7a518710c4 tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:06:09 user nova-compute[75234]: DEBUG nova.network.neutron [None req-fcb1dbf7-0559-48fa-8e1a-0e39441efbec tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] [instance: 21e9c6cd-c39f-41c0-8768-5dd64589c3f3] Successfully created port: 87f1e736-6665-4945-bd19-0cf93bff905e {{(pid=75234) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:548}}
Nov 29 03:06:09 user nova-compute[75234]: DEBUG nova.compute.provider_tree [None req-71e7f72f-cfc6-42f1-9655-a38fb0588077 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] Inventory has not changed in ProviderTree for provider: 4d291220-7c37-4839-a0e3-b23c66502b74 {{(pid=75234) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Nov 29 03:06:09 user nova-compute[75234]: DEBUG nova.scheduler.client.report [None req-71e7f72f-cfc6-42f1-9655-a38fb0588077 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] Inventory has not changed for provider 4d291220-7c37-4839-a0e3-b23c66502b74 based on inventory data: {'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=75234) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Nov 29 03:06:09 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-71e7f72f-cfc6-42f1-9655-a38fb0588077 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.641s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:06:09 user nova-compute[75234]: DEBUG nova.compute.manager [None req-71e7f72f-cfc6-42f1-9655-a38fb0588077 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] [instance: 8297c264-a213-44ee-be46-5c08eabda9f7] Start building networks asynchronously for instance. {{(pid=75234) _build_resources /opt/stack/nova/nova/compute/manager.py:2819}}
Nov 29 03:06:09 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:06:10 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-c696a156-5f79-4939-99e6-fd7a518710c4 tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json" returned: 0 in 0.340s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:06:10 user nova-compute[75234]: DEBUG nova.virt.disk.api [None req-c696a156-5f79-4939-99e6-fd7a518710c4 tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Checking if we can resize image /opt/stack/data/nova/instances/a9350a18-14e6-42c6-a150-0a1487f20ad9/disk. size=1073741824 {{(pid=75234) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:166}}
Nov 29 03:06:10 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-c696a156-5f79-4939-99e6-fd7a518710c4 tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a9350a18-14e6-42c6-a150-0a1487f20ad9/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:06:10 user nova-compute[75234]: DEBUG nova.compute.manager [None req-71e7f72f-cfc6-42f1-9655-a38fb0588077 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] [instance: 8297c264-a213-44ee-be46-5c08eabda9f7] Allocating IP information in the background. {{(pid=75234) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1972}}
Nov 29 03:06:10 user nova-compute[75234]: DEBUG nova.network.neutron [None req-71e7f72f-cfc6-42f1-9655-a38fb0588077 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] [instance: 8297c264-a213-44ee-be46-5c08eabda9f7] allocate_for_instance() {{(pid=75234) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1156}}
Nov 29 03:06:10 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-71e7f72f-cfc6-42f1-9655-a38fb0588077 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] [instance: 8297c264-a213-44ee-be46-5c08eabda9f7] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Nov 29 03:06:10 user nova-compute[75234]: DEBUG nova.compute.manager [None req-71e7f72f-cfc6-42f1-9655-a38fb0588077 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] [instance: 8297c264-a213-44ee-be46-5c08eabda9f7] Start building block device mappings for instance. {{(pid=75234) _build_resources /opt/stack/nova/nova/compute/manager.py:2854}}
Nov 29 03:06:10 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-c696a156-5f79-4939-99e6-fd7a518710c4 tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a9350a18-14e6-42c6-a150-0a1487f20ad9/disk --force-share --output=json" returned: 0 in 0.212s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:06:10 user nova-compute[75234]: DEBUG nova.virt.disk.api [None req-c696a156-5f79-4939-99e6-fd7a518710c4 tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Cannot resize image /opt/stack/data/nova/instances/a9350a18-14e6-42c6-a150-0a1487f20ad9/disk to a smaller size. {{(pid=75234) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:172}}
Nov 29 03:06:10 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-c696a156-5f79-4939-99e6-fd7a518710c4 tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] [instance: a9350a18-14e6-42c6-a150-0a1487f20ad9] Created local disks {{(pid=75234) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4861}}
Nov 29 03:06:10 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-c696a156-5f79-4939-99e6-fd7a518710c4 tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] [instance: a9350a18-14e6-42c6-a150-0a1487f20ad9] Ensure instance console log exists: /opt/stack/data/nova/instances/a9350a18-14e6-42c6-a150-0a1487f20ad9/console.log {{(pid=75234) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4613}}
Nov 29 03:06:10 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-c696a156-5f79-4939-99e6-fd7a518710c4 tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:06:10 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-c696a156-5f79-4939-99e6-fd7a518710c4 tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:06:10 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-c696a156-5f79-4939-99e6-fd7a518710c4 tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:06:10 user nova-compute[75234]: DEBUG nova.compute.manager [None req-71e7f72f-cfc6-42f1-9655-a38fb0588077 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] [instance: 8297c264-a213-44ee-be46-5c08eabda9f7] Start spawning the instance on the hypervisor. {{(pid=75234) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2628}}
Nov 29 03:06:10 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-71e7f72f-cfc6-42f1-9655-a38fb0588077 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] [instance: 8297c264-a213-44ee-be46-5c08eabda9f7] Creating instance directory {{(pid=75234) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4727}}
Nov 29 03:06:10 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-71e7f72f-cfc6-42f1-9655-a38fb0588077 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] [instance: 8297c264-a213-44ee-be46-5c08eabda9f7] Creating image(s)
Nov 29 03:06:10 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-71e7f72f-cfc6-42f1-9655-a38fb0588077 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] Acquiring lock "/opt/stack/data/nova/instances/8297c264-a213-44ee-be46-5c08eabda9f7/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:06:10 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-71e7f72f-cfc6-42f1-9655-a38fb0588077 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] Lock "/opt/stack/data/nova/instances/8297c264-a213-44ee-be46-5c08eabda9f7/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:06:10 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-71e7f72f-cfc6-42f1-9655-a38fb0588077 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] Lock "/opt/stack/data/nova/instances/8297c264-a213-44ee-be46-5c08eabda9f7/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.002s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:06:10 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-71e7f72f-cfc6-42f1-9655-a38fb0588077 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:06:10 user nova-compute[75234]: DEBUG nova.policy [None req-71e7f72f-cfc6-42f1-9655-a38fb0588077 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '566e3dc84bbb40d9badcacbcaec9b987', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '745bdb3c72794155829dc0a69f35b321', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=75234) authorize /opt/stack/nova/nova/policy.py:203}}
Nov 29 03:06:10 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-71e7f72f-cfc6-42f1-9655-a38fb0588077 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json" returned: 0 in 0.211s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:06:10 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-71e7f72f-cfc6-42f1-9655-a38fb0588077 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] Acquiring lock "80686e29dc9db6600d03e85cb62af298e12d7918" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:06:10 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-71e7f72f-cfc6-42f1-9655-a38fb0588077 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] Lock "80686e29dc9db6600d03e85cb62af298e12d7918" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.003s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:06:10 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-71e7f72f-cfc6-42f1-9655-a38fb0588077 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:06:10 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-71e7f72f-cfc6-42f1-9655-a38fb0588077 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json" returned: 0 in 0.204s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:06:10 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-71e7f72f-cfc6-42f1-9655-a38fb0588077 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918,backing_fmt=raw /opt/stack/data/nova/instances/8297c264-a213-44ee-be46-5c08eabda9f7/disk 1073741824 {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:06:10 user nova-compute[75234]: DEBUG nova.network.neutron [None req-ceb7826f-126f-40b1-b7f1-9afa48110fc2 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] [instance: f609a7b7-1be2-44b4-9177-7e72a4d4c060] Successfully created port: 5543f379-2b3a-4dc2-bfe9-52f66a942257 {{(pid=75234) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:548}}
Nov 29 03:06:10 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-71e7f72f-cfc6-42f1-9655-a38fb0588077 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918,backing_fmt=raw /opt/stack/data/nova/instances/8297c264-a213-44ee-be46-5c08eabda9f7/disk 1073741824" returned: 0 in 0.072s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:06:10 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-71e7f72f-cfc6-42f1-9655-a38fb0588077 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] Lock "80686e29dc9db6600d03e85cb62af298e12d7918" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.314s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:06:10 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-71e7f72f-cfc6-42f1-9655-a38fb0588077 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:06:11 user nova-compute[75234]: DEBUG nova.network.neutron [None req-c696a156-5f79-4939-99e6-fd7a518710c4 tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] [instance: a9350a18-14e6-42c6-a150-0a1487f20ad9] Successfully created port: 52024efc-8cde-4610-b12c-198f19d75ccf {{(pid=75234) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:548}}
Nov 29 03:06:11 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:06:11 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-71e7f72f-cfc6-42f1-9655-a38fb0588077 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json" returned: 0 in 0.252s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:06:11 user nova-compute[75234]: DEBUG nova.virt.disk.api [None req-71e7f72f-cfc6-42f1-9655-a38fb0588077 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] Checking if we can resize image /opt/stack/data/nova/instances/8297c264-a213-44ee-be46-5c08eabda9f7/disk. size=1073741824 {{(pid=75234) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:166}}
Nov 29 03:06:11 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-71e7f72f-cfc6-42f1-9655-a38fb0588077 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/8297c264-a213-44ee-be46-5c08eabda9f7/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:06:11 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-71e7f72f-cfc6-42f1-9655-a38fb0588077 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/8297c264-a213-44ee-be46-5c08eabda9f7/disk --force-share --output=json" returned: 0 in 0.265s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:06:11 user nova-compute[75234]: DEBUG nova.virt.disk.api [None req-71e7f72f-cfc6-42f1-9655-a38fb0588077 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] Cannot resize image /opt/stack/data/nova/instances/8297c264-a213-44ee-be46-5c08eabda9f7/disk to a smaller size. {{(pid=75234) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:172}}
Nov 29 03:06:11 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-71e7f72f-cfc6-42f1-9655-a38fb0588077 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] [instance: 8297c264-a213-44ee-be46-5c08eabda9f7] Created local disks {{(pid=75234) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4861}}
Nov 29 03:06:11 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-71e7f72f-cfc6-42f1-9655-a38fb0588077 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] [instance: 8297c264-a213-44ee-be46-5c08eabda9f7] Ensure instance console log exists: /opt/stack/data/nova/instances/8297c264-a213-44ee-be46-5c08eabda9f7/console.log {{(pid=75234) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4613}}
Nov 29 03:06:11 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-71e7f72f-cfc6-42f1-9655-a38fb0588077 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:06:11 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-71e7f72f-cfc6-42f1-9655-a38fb0588077 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:06:11 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-71e7f72f-cfc6-42f1-9655-a38fb0588077 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:06:11 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:06:11 user nova-compute[75234]: DEBUG nova.network.neutron [None req-fcb1dbf7-0559-48fa-8e1a-0e39441efbec tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] [instance: 21e9c6cd-c39f-41c0-8768-5dd64589c3f3] Successfully updated port: 87f1e736-6665-4945-bd19-0cf93bff905e {{(pid=75234) _update_port /opt/stack/nova/nova/network/neutron.py:586}}
Nov 29 03:06:11 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-fcb1dbf7-0559-48fa-8e1a-0e39441efbec tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Acquiring lock "refresh_cache-21e9c6cd-c39f-41c0-8768-5dd64589c3f3" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:06:11 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-fcb1dbf7-0559-48fa-8e1a-0e39441efbec tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Acquired lock "refresh_cache-21e9c6cd-c39f-41c0-8768-5dd64589c3f3" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:06:11 user nova-compute[75234]: DEBUG nova.network.neutron [None req-fcb1dbf7-0559-48fa-8e1a-0e39441efbec tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] [instance: 21e9c6cd-c39f-41c0-8768-5dd64589c3f3] Building network info cache for instance {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2009}}
Nov 29 03:06:11 user nova-compute[75234]: DEBUG nova.compute.manager [req-0c017171-5ad2-402d-9a29-0f1c120ebb78 req-2aeee900-797c-4c78-9a55-3ab11037d3c6 service nova] [instance: 21e9c6cd-c39f-41c0-8768-5dd64589c3f3] Received event network-changed-87f1e736-6665-4945-bd19-0cf93bff905e {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:06:11 user nova-compute[75234]: DEBUG nova.compute.manager [req-0c017171-5ad2-402d-9a29-0f1c120ebb78 req-2aeee900-797c-4c78-9a55-3ab11037d3c6 service nova] [instance: 21e9c6cd-c39f-41c0-8768-5dd64589c3f3] Refreshing instance network info cache due to event network-changed-87f1e736-6665-4945-bd19-0cf93bff905e. {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11117}}
Nov 29 03:06:11 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-0c017171-5ad2-402d-9a29-0f1c120ebb78 req-2aeee900-797c-4c78-9a55-3ab11037d3c6 service nova] Acquiring lock "refresh_cache-21e9c6cd-c39f-41c0-8768-5dd64589c3f3" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:06:12 user nova-compute[75234]: DEBUG nova.network.neutron [None req-fcb1dbf7-0559-48fa-8e1a-0e39441efbec tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] [instance: 21e9c6cd-c39f-41c0-8768-5dd64589c3f3] Instance cache missing network info. {{(pid=75234) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3322}}
Nov 29 03:06:12 user nova-compute[75234]: DEBUG nova.compute.manager [None req-997f592d-b1eb-43b5-b6d7-0f22555038e1 tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] [instance: 2bf9fe41-63c2-4882-bcdd-8e314e734ffe] Instance event wait completed in 0 seconds for  {{(pid=75234) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Nov 29 03:06:12 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-997f592d-b1eb-43b5-b6d7-0f22555038e1 tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] [instance: 2bf9fe41-63c2-4882-bcdd-8e314e734ffe] Guest created on hypervisor {{(pid=75234) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4421}}
Nov 29 03:06:12 user nova-compute[75234]: INFO nova.virt.libvirt.driver [-] [instance: 2bf9fe41-63c2-4882-bcdd-8e314e734ffe] Instance spawned successfully.
Nov 29 03:06:12 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-997f592d-b1eb-43b5-b6d7-0f22555038e1 tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] [instance: 2bf9fe41-63c2-4882-bcdd-8e314e734ffe] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:904}}
Nov 29 03:06:12 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-997f592d-b1eb-43b5-b6d7-0f22555038e1 tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] [instance: 2bf9fe41-63c2-4882-bcdd-8e314e734ffe] Found default for hw_cdrom_bus of ide {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:06:12 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-997f592d-b1eb-43b5-b6d7-0f22555038e1 tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] [instance: 2bf9fe41-63c2-4882-bcdd-8e314e734ffe] Found default for hw_disk_bus of virtio {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:06:12 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-997f592d-b1eb-43b5-b6d7-0f22555038e1 tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] [instance: 2bf9fe41-63c2-4882-bcdd-8e314e734ffe] Found default for hw_input_bus of None {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:06:12 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-997f592d-b1eb-43b5-b6d7-0f22555038e1 tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] [instance: 2bf9fe41-63c2-4882-bcdd-8e314e734ffe] Found default for hw_pointer_model of None {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:06:12 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-997f592d-b1eb-43b5-b6d7-0f22555038e1 tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] [instance: 2bf9fe41-63c2-4882-bcdd-8e314e734ffe] Found default for hw_video_model of virtio {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:06:12 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-997f592d-b1eb-43b5-b6d7-0f22555038e1 tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] [instance: 2bf9fe41-63c2-4882-bcdd-8e314e734ffe] Found default for hw_vif_model of virtio {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:06:12 user nova-compute[75234]: INFO nova.compute.manager [None req-997f592d-b1eb-43b5-b6d7-0f22555038e1 tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] [instance: 2bf9fe41-63c2-4882-bcdd-8e314e734ffe] Took 11.19 seconds to spawn the instance on the hypervisor.
Nov 29 03:06:12 user nova-compute[75234]: DEBUG nova.compute.manager [None req-997f592d-b1eb-43b5-b6d7-0f22555038e1 tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] [instance: 2bf9fe41-63c2-4882-bcdd-8e314e734ffe] Checking state {{(pid=75234) _get_power_state /opt/stack/nova/nova/compute/manager.py:1782}}
Nov 29 03:06:12 user nova-compute[75234]: INFO nova.compute.manager [None req-997f592d-b1eb-43b5-b6d7-0f22555038e1 tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] [instance: 2bf9fe41-63c2-4882-bcdd-8e314e734ffe] Took 11.99 seconds to build instance.
Nov 29 03:06:12 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-997f592d-b1eb-43b5-b6d7-0f22555038e1 tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] Lock "2bf9fe41-63c2-4882-bcdd-8e314e734ffe" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 12.245s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:06:13 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:06:13 user nova-compute[75234]: DEBUG nova.network.neutron [None req-fcb1dbf7-0559-48fa-8e1a-0e39441efbec tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] [instance: 21e9c6cd-c39f-41c0-8768-5dd64589c3f3] Updating instance_info_cache with network_info: [{"id": "87f1e736-6665-4945-bd19-0cf93bff905e", "address": "fa:16:3e:40:14:df", "network": {"id": "1f946e69-dbec-429c-83f8-a186bad4f2cc", "bridge": "br-int", "label": "tempest-ServerActionsTestOtherA-486369423-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "c1cb3d1f12f242a785a688d59dbe3cd6", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap87f1e736-66", "ovs_interfaceid": "87f1e736-6665-4945-bd19-0cf93bff905e", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:06:13 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-fcb1dbf7-0559-48fa-8e1a-0e39441efbec tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Releasing lock "refresh_cache-21e9c6cd-c39f-41c0-8768-5dd64589c3f3" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:06:13 user nova-compute[75234]: DEBUG nova.compute.manager [None req-fcb1dbf7-0559-48fa-8e1a-0e39441efbec tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] [instance: 21e9c6cd-c39f-41c0-8768-5dd64589c3f3] Instance network_info: |[{"id": "87f1e736-6665-4945-bd19-0cf93bff905e", "address": "fa:16:3e:40:14:df", "network": {"id": "1f946e69-dbec-429c-83f8-a186bad4f2cc", "bridge": "br-int", "label": "tempest-ServerActionsTestOtherA-486369423-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "c1cb3d1f12f242a785a688d59dbe3cd6", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap87f1e736-66", "ovs_interfaceid": "87f1e736-6665-4945-bd19-0cf93bff905e", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=75234) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1987}}
Nov 29 03:06:13 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-0c017171-5ad2-402d-9a29-0f1c120ebb78 req-2aeee900-797c-4c78-9a55-3ab11037d3c6 service nova] Acquired lock "refresh_cache-21e9c6cd-c39f-41c0-8768-5dd64589c3f3" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:06:13 user nova-compute[75234]: DEBUG nova.network.neutron [req-0c017171-5ad2-402d-9a29-0f1c120ebb78 req-2aeee900-797c-4c78-9a55-3ab11037d3c6 service nova] [instance: 21e9c6cd-c39f-41c0-8768-5dd64589c3f3] Refreshing network info cache for port 87f1e736-6665-4945-bd19-0cf93bff905e {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2006}}
Nov 29 03:06:13 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-fcb1dbf7-0559-48fa-8e1a-0e39441efbec tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] [instance: 21e9c6cd-c39f-41c0-8768-5dd64589c3f3] Start _get_guest_xml network_info=[{"id": "87f1e736-6665-4945-bd19-0cf93bff905e", "address": "fa:16:3e:40:14:df", "network": {"id": "1f946e69-dbec-429c-83f8-a186bad4f2cc", "bridge": "br-int", "label": "tempest-ServerActionsTestOtherA-486369423-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "c1cb3d1f12f242a785a688d59dbe3cd6", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap87f1e736-66", "ovs_interfaceid": "87f1e736-6665-4945-bd19-0cf93bff905e", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum='c8fc807773e5354afe61636071771906',container_format='bare',created_at=2023-11-29T03:02:15Z,direct_url=<?>,disk_format='qcow2',id=96c30ba7-9b73-4a30-8ba4-7138879573b7,min_disk=0,min_ram=0,name='cirros-0.6.2-x86_64-disk',owner='b18e380921c748b78517b6f1211cd0f1',properties=ImageMetaProps,protected=<?>,size=21430272,status='active',tags=<?>,updated_at=2023-11-29T03:02:17Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [{'device_name': '/dev/vda', 'encryption_format': None, 'encrypted': False, 'device_type': 'disk', 'size': 0, 'disk_bus': 'virtio', 'boot_index': 0, 'encryption_options': None, 'encryption_secret_uuid': None, 'guest_format': None, 'image_id': '96c30ba7-9b73-4a30-8ba4-7138879573b7'}], 'ephemerals': [], 'block_device_mapping': [], 'swap': None} {{(pid=75234) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7562}}
Nov 29 03:06:13 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-fcb1dbf7-0559-48fa-8e1a-0e39441efbec tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:06:13 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-fcb1dbf7-0559-48fa-8e1a-0e39441efbec tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:06:13 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-fcb1dbf7-0559-48fa-8e1a-0e39441efbec tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Searching host: 'user' for CPU controller through CGroups V1... {{(pid=75234) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1652}}
Nov 29 03:06:13 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-fcb1dbf7-0559-48fa-8e1a-0e39441efbec tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] CPU controller missing on host. {{(pid=75234) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1662}}
Nov 29 03:06:13 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-fcb1dbf7-0559-48fa-8e1a-0e39441efbec tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Searching host: 'user' for CPU controller through CGroups V2... {{(pid=75234) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1671}}
Nov 29 03:06:13 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-fcb1dbf7-0559-48fa-8e1a-0e39441efbec tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] CPU controller found on host. {{(pid=75234) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1678}}
Nov 29 03:06:13 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-fcb1dbf7-0559-48fa-8e1a-0e39441efbec tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=75234) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5400}}
Nov 29 03:06:13 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-fcb1dbf7-0559-48fa-8e1a-0e39441efbec tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-11-29T03:02:04Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='1',id=6,is_public=True,memory_mb=512,name='m1.tiny',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='c8fc807773e5354afe61636071771906',container_format='bare',created_at=2023-11-29T03:02:15Z,direct_url=<?>,disk_format='qcow2',id=96c30ba7-9b73-4a30-8ba4-7138879573b7,min_disk=0,min_ram=0,name='cirros-0.6.2-x86_64-disk',owner='b18e380921c748b78517b6f1211cd0f1',properties=ImageMetaProps,protected=<?>,size=21430272,status='active',tags=<?>,updated_at=2023-11-29T03:02:17Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:563}}
Nov 29 03:06:13 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-fcb1dbf7-0559-48fa-8e1a-0e39441efbec tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Flavor limits 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:348}}
Nov 29 03:06:13 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-fcb1dbf7-0559-48fa-8e1a-0e39441efbec tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Image limits 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:352}}
Nov 29 03:06:13 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-fcb1dbf7-0559-48fa-8e1a-0e39441efbec tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Flavor pref 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:388}}
Nov 29 03:06:13 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-fcb1dbf7-0559-48fa-8e1a-0e39441efbec tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Image pref 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:392}}
Nov 29 03:06:13 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-fcb1dbf7-0559-48fa-8e1a-0e39441efbec tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:430}}
Nov 29 03:06:13 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-fcb1dbf7-0559-48fa-8e1a-0e39441efbec tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:569}}
Nov 29 03:06:13 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-fcb1dbf7-0559-48fa-8e1a-0e39441efbec tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=75234) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:471}}
Nov 29 03:06:13 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-fcb1dbf7-0559-48fa-8e1a-0e39441efbec tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Got 1 possible topologies {{(pid=75234) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:501}}
Nov 29 03:06:13 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-fcb1dbf7-0559-48fa-8e1a-0e39441efbec tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:575}}
Nov 29 03:06:13 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-fcb1dbf7-0559-48fa-8e1a-0e39441efbec tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:577}}
Nov 29 03:06:13 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-fcb1dbf7-0559-48fa-8e1a-0e39441efbec tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-11-29T03:06:05Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-ServerActionsTestOtherA-server-1105149564',display_name='tempest-ServerActionsTestOtherA-server-1105149564',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-serveractionstestothera-server-1105149564',id=3,image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBGysIZGz8py0LojrFdRCkkIN/DpphiAjtTxjJQY7OVo0ZaKHLKAtSriy5/opnav7HzQyT3MeG6wW+2nQfBNyK0gW9HDU5His6qvZYEbhyEyMiaUzdXksC9FbE2bauudLhQ==',key_name='tempest-keypair-1767034584',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='c1cb3d1f12f242a785a688d59dbe3cd6',ramdisk_id='',reservation_id='r-nnmj1n3l',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-ServerActionsTestOtherA-1495067208',owner_user_name='tempest-ServerActionsTestOtherA-1495067208-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-11-29T03:06:07Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='d611826f3be947c0947b99c763c26d44',uuid=21e9c6cd-c39f-41c0-8768-5dd64589c3f3,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "87f1e736-6665-4945-bd19-0cf93bff905e", "address": "fa:16:3e:40:14:df", "network": {"id": "1f946e69-dbec-429c-83f8-a186bad4f2cc", "bridge": "br-int", "label": "tempest-ServerActionsTestOtherA-486369423-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "c1cb3d1f12f242a785a688d59dbe3cd6", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap87f1e736-66", "ovs_interfaceid": "87f1e736-6665-4945-bd19-0cf93bff905e", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=75234) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Nov 29 03:06:13 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-fcb1dbf7-0559-48fa-8e1a-0e39441efbec tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Converting VIF {"id": "87f1e736-6665-4945-bd19-0cf93bff905e", "address": "fa:16:3e:40:14:df", "network": {"id": "1f946e69-dbec-429c-83f8-a186bad4f2cc", "bridge": "br-int", "label": "tempest-ServerActionsTestOtherA-486369423-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "c1cb3d1f12f242a785a688d59dbe3cd6", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap87f1e736-66", "ovs_interfaceid": "87f1e736-6665-4945-bd19-0cf93bff905e", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:06:13 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-fcb1dbf7-0559-48fa-8e1a-0e39441efbec tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:40:14:df,bridge_name='br-int',has_traffic_filtering=True,id=87f1e736-6665-4945-bd19-0cf93bff905e,network=Network(1f946e69-dbec-429c-83f8-a186bad4f2cc),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap87f1e736-66') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:06:13 user nova-compute[75234]: DEBUG nova.objects.instance [None req-fcb1dbf7-0559-48fa-8e1a-0e39441efbec tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Lazy-loading 'pci_devices' on Instance uuid 21e9c6cd-c39f-41c0-8768-5dd64589c3f3 {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:06:13 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-fcb1dbf7-0559-48fa-8e1a-0e39441efbec tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] [instance: 21e9c6cd-c39f-41c0-8768-5dd64589c3f3] End _get_guest_xml xml=<domain type="kvm">
Nov 29 03:06:13 user nova-compute[75234]:   <uuid>21e9c6cd-c39f-41c0-8768-5dd64589c3f3</uuid>
Nov 29 03:06:13 user nova-compute[75234]:   <name>instance-00000003</name>
Nov 29 03:06:13 user nova-compute[75234]:   <memory>524288</memory>
Nov 29 03:06:13 user nova-compute[75234]:   <vcpu>1</vcpu>
Nov 29 03:06:13 user nova-compute[75234]:   <metadata>
Nov 29 03:06:13 user nova-compute[75234]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Nov 29 03:06:13 user nova-compute[75234]:       <nova:package version="0.0.0"/>
Nov 29 03:06:13 user nova-compute[75234]:       <nova:name>tempest-ServerActionsTestOtherA-server-1105149564</nova:name>
Nov 29 03:06:13 user nova-compute[75234]:       <nova:creationTime>2023-11-29 03:06:13</nova:creationTime>
Nov 29 03:06:13 user nova-compute[75234]:       <nova:flavor name="m1.tiny">
Nov 29 03:06:13 user nova-compute[75234]:         <nova:memory>512</nova:memory>
Nov 29 03:06:13 user nova-compute[75234]:         <nova:disk>1</nova:disk>
Nov 29 03:06:13 user nova-compute[75234]:         <nova:swap>0</nova:swap>
Nov 29 03:06:13 user nova-compute[75234]:         <nova:ephemeral>0</nova:ephemeral>
Nov 29 03:06:13 user nova-compute[75234]:         <nova:vcpus>1</nova:vcpus>
Nov 29 03:06:13 user nova-compute[75234]:       </nova:flavor>
Nov 29 03:06:13 user nova-compute[75234]:       <nova:owner>
Nov 29 03:06:13 user nova-compute[75234]:         <nova:user uuid="d611826f3be947c0947b99c763c26d44">tempest-ServerActionsTestOtherA-1495067208-project-member</nova:user>
Nov 29 03:06:13 user nova-compute[75234]:         <nova:project uuid="c1cb3d1f12f242a785a688d59dbe3cd6">tempest-ServerActionsTestOtherA-1495067208</nova:project>
Nov 29 03:06:13 user nova-compute[75234]:       </nova:owner>
Nov 29 03:06:13 user nova-compute[75234]:       <nova:root type="image" uuid="96c30ba7-9b73-4a30-8ba4-7138879573b7"/>
Nov 29 03:06:13 user nova-compute[75234]:       <nova:ports>
Nov 29 03:06:13 user nova-compute[75234]:         <nova:port uuid="87f1e736-6665-4945-bd19-0cf93bff905e">
Nov 29 03:06:13 user nova-compute[75234]:           <nova:ip type="fixed" address="10.0.0.5" ipVersion="4"/>
Nov 29 03:06:13 user nova-compute[75234]:         </nova:port>
Nov 29 03:06:13 user nova-compute[75234]:       </nova:ports>
Nov 29 03:06:13 user nova-compute[75234]:     </nova:instance>
Nov 29 03:06:13 user nova-compute[75234]:   </metadata>
Nov 29 03:06:13 user nova-compute[75234]:   <sysinfo type="smbios">
Nov 29 03:06:13 user nova-compute[75234]:     <system>
Nov 29 03:06:13 user nova-compute[75234]:       <entry name="manufacturer">OpenStack Foundation</entry>
Nov 29 03:06:13 user nova-compute[75234]:       <entry name="product">OpenStack Nova</entry>
Nov 29 03:06:13 user nova-compute[75234]:       <entry name="version">0.0.0</entry>
Nov 29 03:06:13 user nova-compute[75234]:       <entry name="serial">21e9c6cd-c39f-41c0-8768-5dd64589c3f3</entry>
Nov 29 03:06:13 user nova-compute[75234]:       <entry name="uuid">21e9c6cd-c39f-41c0-8768-5dd64589c3f3</entry>
Nov 29 03:06:13 user nova-compute[75234]:       <entry name="family">Virtual Machine</entry>
Nov 29 03:06:13 user nova-compute[75234]:     </system>
Nov 29 03:06:13 user nova-compute[75234]:   </sysinfo>
Nov 29 03:06:13 user nova-compute[75234]:   <os>
Nov 29 03:06:13 user nova-compute[75234]:     <type arch="x86_64" machine="pc">hvm</type>
Nov 29 03:06:13 user nova-compute[75234]:     <boot dev="hd"/>
Nov 29 03:06:13 user nova-compute[75234]:     <smbios mode="sysinfo"/>
Nov 29 03:06:13 user nova-compute[75234]:   </os>
Nov 29 03:06:13 user nova-compute[75234]:   <features>
Nov 29 03:06:13 user nova-compute[75234]:     <acpi/>
Nov 29 03:06:13 user nova-compute[75234]:     <apic/>
Nov 29 03:06:13 user nova-compute[75234]:     <vmcoreinfo/>
Nov 29 03:06:13 user nova-compute[75234]:   </features>
Nov 29 03:06:13 user nova-compute[75234]:   <clock offset="utc">
Nov 29 03:06:13 user nova-compute[75234]:     <timer name="pit" tickpolicy="delay"/>
Nov 29 03:06:13 user nova-compute[75234]:     <timer name="rtc" tickpolicy="catchup"/>
Nov 29 03:06:13 user nova-compute[75234]:     <timer name="hpet" present="no"/>
Nov 29 03:06:13 user nova-compute[75234]:   </clock>
Nov 29 03:06:13 user nova-compute[75234]:   <cpu mode="custom" match="exact">
Nov 29 03:06:13 user nova-compute[75234]:     <model>Nehalem</model>
Nov 29 03:06:13 user nova-compute[75234]:     <topology sockets="1" cores="1" threads="1"/>
Nov 29 03:06:13 user nova-compute[75234]:   </cpu>
Nov 29 03:06:13 user nova-compute[75234]:   <devices>
Nov 29 03:06:13 user nova-compute[75234]:     <disk type="file" device="disk">
Nov 29 03:06:13 user nova-compute[75234]:       <driver name="qemu" type="qcow2" cache="none"/>
Nov 29 03:06:13 user nova-compute[75234]:       <source file="/opt/stack/data/nova/instances/21e9c6cd-c39f-41c0-8768-5dd64589c3f3/disk"/>
Nov 29 03:06:13 user nova-compute[75234]:       <target dev="vda" bus="virtio"/>
Nov 29 03:06:13 user nova-compute[75234]:     </disk>
Nov 29 03:06:13 user nova-compute[75234]:     <interface type="ethernet">
Nov 29 03:06:13 user nova-compute[75234]:       <mac address="fa:16:3e:40:14:df"/>
Nov 29 03:06:13 user nova-compute[75234]:       <model type="virtio"/>
Nov 29 03:06:13 user nova-compute[75234]:       <mtu size="1442"/>
Nov 29 03:06:13 user nova-compute[75234]:       <target dev="tap87f1e736-66"/>
Nov 29 03:06:13 user nova-compute[75234]:     </interface>
Nov 29 03:06:13 user nova-compute[75234]:     <serial type="pty">
Nov 29 03:06:13 user nova-compute[75234]:       <log file="/opt/stack/data/nova/instances/21e9c6cd-c39f-41c0-8768-5dd64589c3f3/console.log" append="off"/>
Nov 29 03:06:13 user nova-compute[75234]:     </serial>
Nov 29 03:06:13 user nova-compute[75234]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Nov 29 03:06:13 user nova-compute[75234]:     <video>
Nov 29 03:06:13 user nova-compute[75234]:       <model type="virtio"/>
Nov 29 03:06:13 user nova-compute[75234]:     </video>
Nov 29 03:06:13 user nova-compute[75234]:     <rng model="virtio">
Nov 29 03:06:13 user nova-compute[75234]:       <backend model="random">/dev/urandom</backend>
Nov 29 03:06:13 user nova-compute[75234]:     </rng>
Nov 29 03:06:13 user nova-compute[75234]:     <controller type="usb" index="0" model="none"/>
Nov 29 03:06:13 user nova-compute[75234]:     <memballoon model="virtio">
Nov 29 03:06:13 user nova-compute[75234]:       <stats period="10"/>
Nov 29 03:06:13 user nova-compute[75234]:     </memballoon>
Nov 29 03:06:13 user nova-compute[75234]:   </devices>
Nov 29 03:06:13 user nova-compute[75234]: </domain>
Nov 29 03:06:13 user nova-compute[75234]:  {{(pid=75234) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7568}}
Nov 29 03:06:13 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-fcb1dbf7-0559-48fa-8e1a-0e39441efbec tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-11-29T03:06:05Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-ServerActionsTestOtherA-server-1105149564',display_name='tempest-ServerActionsTestOtherA-server-1105149564',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-serveractionstestothera-server-1105149564',id=3,image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBGysIZGz8py0LojrFdRCkkIN/DpphiAjtTxjJQY7OVo0ZaKHLKAtSriy5/opnav7HzQyT3MeG6wW+2nQfBNyK0gW9HDU5His6qvZYEbhyEyMiaUzdXksC9FbE2bauudLhQ==',key_name='tempest-keypair-1767034584',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='c1cb3d1f12f242a785a688d59dbe3cd6',ramdisk_id='',reservation_id='r-nnmj1n3l',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-ServerActionsTestOtherA-1495067208',owner_user_name='tempest-ServerActionsTestOtherA-1495067208-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-11-29T03:06:07Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='d611826f3be947c0947b99c763c26d44',uuid=21e9c6cd-c39f-41c0-8768-5dd64589c3f3,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "87f1e736-6665-4945-bd19-0cf93bff905e", "address": "fa:16:3e:40:14:df", "network": {"id": "1f946e69-dbec-429c-83f8-a186bad4f2cc", "bridge": "br-int", "label": "tempest-ServerActionsTestOtherA-486369423-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "c1cb3d1f12f242a785a688d59dbe3cd6", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap87f1e736-66", "ovs_interfaceid": "87f1e736-6665-4945-bd19-0cf93bff905e", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Nov 29 03:06:13 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-fcb1dbf7-0559-48fa-8e1a-0e39441efbec tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Converting VIF {"id": "87f1e736-6665-4945-bd19-0cf93bff905e", "address": "fa:16:3e:40:14:df", "network": {"id": "1f946e69-dbec-429c-83f8-a186bad4f2cc", "bridge": "br-int", "label": "tempest-ServerActionsTestOtherA-486369423-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "c1cb3d1f12f242a785a688d59dbe3cd6", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap87f1e736-66", "ovs_interfaceid": "87f1e736-6665-4945-bd19-0cf93bff905e", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:06:13 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-fcb1dbf7-0559-48fa-8e1a-0e39441efbec tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:40:14:df,bridge_name='br-int',has_traffic_filtering=True,id=87f1e736-6665-4945-bd19-0cf93bff905e,network=Network(1f946e69-dbec-429c-83f8-a186bad4f2cc),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap87f1e736-66') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:06:13 user nova-compute[75234]: DEBUG os_vif [None req-fcb1dbf7-0559-48fa-8e1a-0e39441efbec tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:40:14:df,bridge_name='br-int',has_traffic_filtering=True,id=87f1e736-6665-4945-bd19-0cf93bff905e,network=Network(1f946e69-dbec-429c-83f8-a186bad4f2cc),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap87f1e736-66') {{(pid=75234) plug /opt/stack/data/venv/lib/python3.10/site-packages/os_vif/__init__.py:76}}
Nov 29 03:06:13 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:06:13 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:06:13 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Nov 29 03:06:13 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:06:13 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbCreateCommand(_result=None, table=QoS, columns={'type': 'linux-noop', 'external_ids': {'id': '9d8a0be7-7847-502c-80b9-cdc12f2878c0', '_type': 'linux-noop'}}, row=False) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:06:13 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:06:13 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:06:13 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:06:13 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tap87f1e736-66, may_exist=True, interface_attrs={}) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:06:13 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Port, record=tap87f1e736-66, col_values=(('qos', UUID('70224693-48a0-479d-be80-bc6803e99fb2')),), if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:06:13 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DbSetCommand(_result=None, table=Interface, record=tap87f1e736-66, col_values=(('external_ids', {'iface-id': '87f1e736-6665-4945-bd19-0cf93bff905e', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:40:14:df', 'vm-uuid': '21e9c6cd-c39f-41c0-8768-5dd64589c3f3'}),), if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:06:13 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:06:13 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:06:13 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:06:13 user nova-compute[75234]: INFO os_vif [None req-fcb1dbf7-0559-48fa-8e1a-0e39441efbec tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:40:14:df,bridge_name='br-int',has_traffic_filtering=True,id=87f1e736-6665-4945-bd19-0cf93bff905e,network=Network(1f946e69-dbec-429c-83f8-a186bad4f2cc),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap87f1e736-66')
Nov 29 03:06:13 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-fcb1dbf7-0559-48fa-8e1a-0e39441efbec tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] No BDM found with device name vda, not building metadata. {{(pid=75234) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12089}}
Nov 29 03:06:13 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-fcb1dbf7-0559-48fa-8e1a-0e39441efbec tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] No VIF found with MAC fa:16:3e:40:14:df, not building metadata {{(pid=75234) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Nov 29 03:06:13 user nova-compute[75234]: DEBUG nova.network.neutron [None req-71e7f72f-cfc6-42f1-9655-a38fb0588077 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] [instance: 8297c264-a213-44ee-be46-5c08eabda9f7] Successfully created port: d26334d2-8040-4022-b658-69be1cf6a025 {{(pid=75234) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:548}}
Nov 29 03:06:13 user nova-compute[75234]: DEBUG nova.network.neutron [None req-c696a156-5f79-4939-99e6-fd7a518710c4 tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] [instance: a9350a18-14e6-42c6-a150-0a1487f20ad9] Successfully updated port: 52024efc-8cde-4610-b12c-198f19d75ccf {{(pid=75234) _update_port /opt/stack/nova/nova/network/neutron.py:586}}
Nov 29 03:06:13 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:06:13 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-c696a156-5f79-4939-99e6-fd7a518710c4 tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Acquiring lock "refresh_cache-a9350a18-14e6-42c6-a150-0a1487f20ad9" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:06:13 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-c696a156-5f79-4939-99e6-fd7a518710c4 tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Acquired lock "refresh_cache-a9350a18-14e6-42c6-a150-0a1487f20ad9" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:06:13 user nova-compute[75234]: DEBUG nova.network.neutron [None req-c696a156-5f79-4939-99e6-fd7a518710c4 tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] [instance: a9350a18-14e6-42c6-a150-0a1487f20ad9] Building network info cache for instance {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2009}}
Nov 29 03:06:13 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:06:13 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:06:13 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:06:14 user nova-compute[75234]: DEBUG nova.network.neutron [None req-c696a156-5f79-4939-99e6-fd7a518710c4 tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] [instance: a9350a18-14e6-42c6-a150-0a1487f20ad9] Instance cache missing network info. {{(pid=75234) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3322}}
Nov 29 03:06:14 user nova-compute[75234]: DEBUG nova.compute.manager [req-0490d72a-ebac-4951-8707-204cd2ab47f9 req-fa56be0e-a566-4909-993b-4c029ebcdcea service nova] [instance: a9350a18-14e6-42c6-a150-0a1487f20ad9] Received event network-changed-52024efc-8cde-4610-b12c-198f19d75ccf {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:06:14 user nova-compute[75234]: DEBUG nova.compute.manager [req-0490d72a-ebac-4951-8707-204cd2ab47f9 req-fa56be0e-a566-4909-993b-4c029ebcdcea service nova] [instance: a9350a18-14e6-42c6-a150-0a1487f20ad9] Refreshing instance network info cache due to event network-changed-52024efc-8cde-4610-b12c-198f19d75ccf. {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11117}}
Nov 29 03:06:14 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-0490d72a-ebac-4951-8707-204cd2ab47f9 req-fa56be0e-a566-4909-993b-4c029ebcdcea service nova] Acquiring lock "refresh_cache-a9350a18-14e6-42c6-a150-0a1487f20ad9" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:06:14 user nova-compute[75234]: DEBUG nova.network.neutron [None req-ceb7826f-126f-40b1-b7f1-9afa48110fc2 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] [instance: f609a7b7-1be2-44b4-9177-7e72a4d4c060] Successfully updated port: 5543f379-2b3a-4dc2-bfe9-52f66a942257 {{(pid=75234) _update_port /opt/stack/nova/nova/network/neutron.py:586}}
Nov 29 03:06:14 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ceb7826f-126f-40b1-b7f1-9afa48110fc2 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Acquiring lock "refresh_cache-f609a7b7-1be2-44b4-9177-7e72a4d4c060" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:06:14 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ceb7826f-126f-40b1-b7f1-9afa48110fc2 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Acquired lock "refresh_cache-f609a7b7-1be2-44b4-9177-7e72a4d4c060" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:06:14 user nova-compute[75234]: DEBUG nova.network.neutron [None req-ceb7826f-126f-40b1-b7f1-9afa48110fc2 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] [instance: f609a7b7-1be2-44b4-9177-7e72a4d4c060] Building network info cache for instance {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2009}}
Nov 29 03:06:14 user nova-compute[75234]: DEBUG nova.compute.manager [req-2a161956-e709-4afa-9279-09cb95f44581 req-2e65d174-b979-4c95-b78c-205663a9972c service nova] [instance: f609a7b7-1be2-44b4-9177-7e72a4d4c060] Received event network-changed-5543f379-2b3a-4dc2-bfe9-52f66a942257 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:06:14 user nova-compute[75234]: DEBUG nova.compute.manager [req-2a161956-e709-4afa-9279-09cb95f44581 req-2e65d174-b979-4c95-b78c-205663a9972c service nova] [instance: f609a7b7-1be2-44b4-9177-7e72a4d4c060] Refreshing instance network info cache due to event network-changed-5543f379-2b3a-4dc2-bfe9-52f66a942257. {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11117}}
Nov 29 03:06:14 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-2a161956-e709-4afa-9279-09cb95f44581 req-2e65d174-b979-4c95-b78c-205663a9972c service nova] Acquiring lock "refresh_cache-f609a7b7-1be2-44b4-9177-7e72a4d4c060" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:06:14 user nova-compute[75234]: DEBUG nova.network.neutron [None req-ceb7826f-126f-40b1-b7f1-9afa48110fc2 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] [instance: f609a7b7-1be2-44b4-9177-7e72a4d4c060] Instance cache missing network info. {{(pid=75234) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3322}}
Nov 29 03:06:14 user nova-compute[75234]: DEBUG nova.network.neutron [req-0c017171-5ad2-402d-9a29-0f1c120ebb78 req-2aeee900-797c-4c78-9a55-3ab11037d3c6 service nova] [instance: 21e9c6cd-c39f-41c0-8768-5dd64589c3f3] Updated VIF entry in instance network info cache for port 87f1e736-6665-4945-bd19-0cf93bff905e. {{(pid=75234) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3481}}
Nov 29 03:06:14 user nova-compute[75234]: DEBUG nova.network.neutron [req-0c017171-5ad2-402d-9a29-0f1c120ebb78 req-2aeee900-797c-4c78-9a55-3ab11037d3c6 service nova] [instance: 21e9c6cd-c39f-41c0-8768-5dd64589c3f3] Updating instance_info_cache with network_info: [{"id": "87f1e736-6665-4945-bd19-0cf93bff905e", "address": "fa:16:3e:40:14:df", "network": {"id": "1f946e69-dbec-429c-83f8-a186bad4f2cc", "bridge": "br-int", "label": "tempest-ServerActionsTestOtherA-486369423-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "c1cb3d1f12f242a785a688d59dbe3cd6", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap87f1e736-66", "ovs_interfaceid": "87f1e736-6665-4945-bd19-0cf93bff905e", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:06:14 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-0c017171-5ad2-402d-9a29-0f1c120ebb78 req-2aeee900-797c-4c78-9a55-3ab11037d3c6 service nova] Releasing lock "refresh_cache-21e9c6cd-c39f-41c0-8768-5dd64589c3f3" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:06:14 user nova-compute[75234]: DEBUG nova.network.neutron [None req-c696a156-5f79-4939-99e6-fd7a518710c4 tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] [instance: a9350a18-14e6-42c6-a150-0a1487f20ad9] Updating instance_info_cache with network_info: [{"id": "52024efc-8cde-4610-b12c-198f19d75ccf", "address": "fa:16:3e:09:b8:70", "network": {"id": "31c4ca3b-3a2f-47df-87fa-4ad095ee5cf9", "bridge": "br-int", "label": "tempest-ServerActionsTestOtherB-1680745008-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "ad5ea22c63f04f91a3f2fa665e4fc8a1", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap52024efc-8c", "ovs_interfaceid": "52024efc-8cde-4610-b12c-198f19d75ccf", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:06:14 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-c696a156-5f79-4939-99e6-fd7a518710c4 tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Releasing lock "refresh_cache-a9350a18-14e6-42c6-a150-0a1487f20ad9" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:06:14 user nova-compute[75234]: DEBUG nova.compute.manager [None req-c696a156-5f79-4939-99e6-fd7a518710c4 tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] [instance: a9350a18-14e6-42c6-a150-0a1487f20ad9] Instance network_info: |[{"id": "52024efc-8cde-4610-b12c-198f19d75ccf", "address": "fa:16:3e:09:b8:70", "network": {"id": "31c4ca3b-3a2f-47df-87fa-4ad095ee5cf9", "bridge": "br-int", "label": "tempest-ServerActionsTestOtherB-1680745008-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "ad5ea22c63f04f91a3f2fa665e4fc8a1", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap52024efc-8c", "ovs_interfaceid": "52024efc-8cde-4610-b12c-198f19d75ccf", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=75234) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1987}}
Nov 29 03:06:14 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-0490d72a-ebac-4951-8707-204cd2ab47f9 req-fa56be0e-a566-4909-993b-4c029ebcdcea service nova] Acquired lock "refresh_cache-a9350a18-14e6-42c6-a150-0a1487f20ad9" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:06:14 user nova-compute[75234]: DEBUG nova.network.neutron [req-0490d72a-ebac-4951-8707-204cd2ab47f9 req-fa56be0e-a566-4909-993b-4c029ebcdcea service nova] [instance: a9350a18-14e6-42c6-a150-0a1487f20ad9] Refreshing network info cache for port 52024efc-8cde-4610-b12c-198f19d75ccf {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2006}}
Nov 29 03:06:14 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-c696a156-5f79-4939-99e6-fd7a518710c4 tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] [instance: a9350a18-14e6-42c6-a150-0a1487f20ad9] Start _get_guest_xml network_info=[{"id": "52024efc-8cde-4610-b12c-198f19d75ccf", "address": "fa:16:3e:09:b8:70", "network": {"id": "31c4ca3b-3a2f-47df-87fa-4ad095ee5cf9", "bridge": "br-int", "label": "tempest-ServerActionsTestOtherB-1680745008-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "ad5ea22c63f04f91a3f2fa665e4fc8a1", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap52024efc-8c", "ovs_interfaceid": "52024efc-8cde-4610-b12c-198f19d75ccf", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum='c8fc807773e5354afe61636071771906',container_format='bare',created_at=2023-11-29T03:02:15Z,direct_url=<?>,disk_format='qcow2',id=96c30ba7-9b73-4a30-8ba4-7138879573b7,min_disk=0,min_ram=0,name='cirros-0.6.2-x86_64-disk',owner='b18e380921c748b78517b6f1211cd0f1',properties=ImageMetaProps,protected=<?>,size=21430272,status='active',tags=<?>,updated_at=2023-11-29T03:02:17Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [{'device_name': '/dev/vda', 'encryption_format': None, 'encrypted': False, 'device_type': 'disk', 'size': 0, 'disk_bus': 'virtio', 'boot_index': 0, 'encryption_options': None, 'encryption_secret_uuid': None, 'guest_format': None, 'image_id': '96c30ba7-9b73-4a30-8ba4-7138879573b7'}], 'ephemerals': [], 'block_device_mapping': [], 'swap': None} {{(pid=75234) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7562}}
Nov 29 03:06:14 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-c696a156-5f79-4939-99e6-fd7a518710c4 tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:06:14 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-c696a156-5f79-4939-99e6-fd7a518710c4 tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:06:14 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-c696a156-5f79-4939-99e6-fd7a518710c4 tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Searching host: 'user' for CPU controller through CGroups V1... {{(pid=75234) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1652}}
Nov 29 03:06:14 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-c696a156-5f79-4939-99e6-fd7a518710c4 tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] CPU controller missing on host. {{(pid=75234) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1662}}
Nov 29 03:06:14 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-c696a156-5f79-4939-99e6-fd7a518710c4 tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Searching host: 'user' for CPU controller through CGroups V2... {{(pid=75234) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1671}}
Nov 29 03:06:14 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-c696a156-5f79-4939-99e6-fd7a518710c4 tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] CPU controller found on host. {{(pid=75234) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1678}}
Nov 29 03:06:14 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-c696a156-5f79-4939-99e6-fd7a518710c4 tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=75234) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5400}}
Nov 29 03:06:14 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-c696a156-5f79-4939-99e6-fd7a518710c4 tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-11-29T03:02:04Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='1',id=6,is_public=True,memory_mb=512,name='m1.tiny',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='c8fc807773e5354afe61636071771906',container_format='bare',created_at=2023-11-29T03:02:15Z,direct_url=<?>,disk_format='qcow2',id=96c30ba7-9b73-4a30-8ba4-7138879573b7,min_disk=0,min_ram=0,name='cirros-0.6.2-x86_64-disk',owner='b18e380921c748b78517b6f1211cd0f1',properties=ImageMetaProps,protected=<?>,size=21430272,status='active',tags=<?>,updated_at=2023-11-29T03:02:17Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:563}}
Nov 29 03:06:14 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-c696a156-5f79-4939-99e6-fd7a518710c4 tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Flavor limits 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:348}}
Nov 29 03:06:14 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-c696a156-5f79-4939-99e6-fd7a518710c4 tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Image limits 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:352}}
Nov 29 03:06:14 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-c696a156-5f79-4939-99e6-fd7a518710c4 tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Flavor pref 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:388}}
Nov 29 03:06:14 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-c696a156-5f79-4939-99e6-fd7a518710c4 tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Image pref 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:392}}
Nov 29 03:06:14 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-c696a156-5f79-4939-99e6-fd7a518710c4 tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:430}}
Nov 29 03:06:14 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-c696a156-5f79-4939-99e6-fd7a518710c4 tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:569}}
Nov 29 03:06:14 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-c696a156-5f79-4939-99e6-fd7a518710c4 tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=75234) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:471}}
Nov 29 03:06:14 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-c696a156-5f79-4939-99e6-fd7a518710c4 tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Got 1 possible topologies {{(pid=75234) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:501}}
Nov 29 03:06:14 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-c696a156-5f79-4939-99e6-fd7a518710c4 tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:575}}
Nov 29 03:06:14 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-c696a156-5f79-4939-99e6-fd7a518710c4 tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:577}}
Nov 29 03:06:14 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-c696a156-5f79-4939-99e6-fd7a518710c4 tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-11-29T03:06:06Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-ServerActionsTestOtherB-server-372321920',display_name='tempest-ServerActionsTestOtherB-server-372321920',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-serveractionstestotherb-server-372321920',id=5,image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBFOggqgO1QPxMZPb2P6SX24mx6LOsfQfrzx4GSpNnJiixHqqOn61ObLwI9/TG/r6plC8w1huVCtu+OqAyk9QXUlEr0+VsPNunPSN9WPXBAM4+U+3iCOgHXExdr++34pXkw==',key_name='tempest-keypair-1692341699',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='ad5ea22c63f04f91a3f2fa665e4fc8a1',ramdisk_id='',reservation_id='r-f5lkid4e',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-ServerActionsTestOtherB-70688183',owner_user_name='tempest-ServerActionsTestOtherB-70688183-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-11-29T03:06:09Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='bdddba0ef9c44b6cb43c32a9ab0e29f6',uuid=a9350a18-14e6-42c6-a150-0a1487f20ad9,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "52024efc-8cde-4610-b12c-198f19d75ccf", "address": "fa:16:3e:09:b8:70", "network": {"id": "31c4ca3b-3a2f-47df-87fa-4ad095ee5cf9", "bridge": "br-int", "label": "tempest-ServerActionsTestOtherB-1680745008-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "ad5ea22c63f04f91a3f2fa665e4fc8a1", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap52024efc-8c", "ovs_interfaceid": "52024efc-8cde-4610-b12c-198f19d75ccf", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=75234) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Nov 29 03:06:14 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-c696a156-5f79-4939-99e6-fd7a518710c4 tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Converting VIF {"id": "52024efc-8cde-4610-b12c-198f19d75ccf", "address": "fa:16:3e:09:b8:70", "network": {"id": "31c4ca3b-3a2f-47df-87fa-4ad095ee5cf9", "bridge": "br-int", "label": "tempest-ServerActionsTestOtherB-1680745008-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "ad5ea22c63f04f91a3f2fa665e4fc8a1", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap52024efc-8c", "ovs_interfaceid": "52024efc-8cde-4610-b12c-198f19d75ccf", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:06:14 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-c696a156-5f79-4939-99e6-fd7a518710c4 tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:09:b8:70,bridge_name='br-int',has_traffic_filtering=True,id=52024efc-8cde-4610-b12c-198f19d75ccf,network=Network(31c4ca3b-3a2f-47df-87fa-4ad095ee5cf9),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap52024efc-8c') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:06:14 user nova-compute[75234]: DEBUG nova.objects.instance [None req-c696a156-5f79-4939-99e6-fd7a518710c4 tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Lazy-loading 'pci_devices' on Instance uuid a9350a18-14e6-42c6-a150-0a1487f20ad9 {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:06:14 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-c696a156-5f79-4939-99e6-fd7a518710c4 tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] [instance: a9350a18-14e6-42c6-a150-0a1487f20ad9] End _get_guest_xml xml=<domain type="kvm">
Nov 29 03:06:14 user nova-compute[75234]:   <uuid>a9350a18-14e6-42c6-a150-0a1487f20ad9</uuid>
Nov 29 03:06:14 user nova-compute[75234]:   <name>instance-00000005</name>
Nov 29 03:06:14 user nova-compute[75234]:   <memory>524288</memory>
Nov 29 03:06:14 user nova-compute[75234]:   <vcpu>1</vcpu>
Nov 29 03:06:14 user nova-compute[75234]:   <metadata>
Nov 29 03:06:14 user nova-compute[75234]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Nov 29 03:06:14 user nova-compute[75234]:       <nova:package version="0.0.0"/>
Nov 29 03:06:14 user nova-compute[75234]:       <nova:name>tempest-ServerActionsTestOtherB-server-372321920</nova:name>
Nov 29 03:06:14 user nova-compute[75234]:       <nova:creationTime>2023-11-29 03:06:14</nova:creationTime>
Nov 29 03:06:14 user nova-compute[75234]:       <nova:flavor name="m1.tiny">
Nov 29 03:06:14 user nova-compute[75234]:         <nova:memory>512</nova:memory>
Nov 29 03:06:14 user nova-compute[75234]:         <nova:disk>1</nova:disk>
Nov 29 03:06:14 user nova-compute[75234]:         <nova:swap>0</nova:swap>
Nov 29 03:06:14 user nova-compute[75234]:         <nova:ephemeral>0</nova:ephemeral>
Nov 29 03:06:14 user nova-compute[75234]:         <nova:vcpus>1</nova:vcpus>
Nov 29 03:06:14 user nova-compute[75234]:       </nova:flavor>
Nov 29 03:06:14 user nova-compute[75234]:       <nova:owner>
Nov 29 03:06:14 user nova-compute[75234]:         <nova:user uuid="bdddba0ef9c44b6cb43c32a9ab0e29f6">tempest-ServerActionsTestOtherB-70688183-project-member</nova:user>
Nov 29 03:06:14 user nova-compute[75234]:         <nova:project uuid="ad5ea22c63f04f91a3f2fa665e4fc8a1">tempest-ServerActionsTestOtherB-70688183</nova:project>
Nov 29 03:06:14 user nova-compute[75234]:       </nova:owner>
Nov 29 03:06:14 user nova-compute[75234]:       <nova:root type="image" uuid="96c30ba7-9b73-4a30-8ba4-7138879573b7"/>
Nov 29 03:06:14 user nova-compute[75234]:       <nova:ports>
Nov 29 03:06:14 user nova-compute[75234]:         <nova:port uuid="52024efc-8cde-4610-b12c-198f19d75ccf">
Nov 29 03:06:14 user nova-compute[75234]:           <nova:ip type="fixed" address="10.0.0.5" ipVersion="4"/>
Nov 29 03:06:14 user nova-compute[75234]:         </nova:port>
Nov 29 03:06:14 user nova-compute[75234]:       </nova:ports>
Nov 29 03:06:14 user nova-compute[75234]:     </nova:instance>
Nov 29 03:06:14 user nova-compute[75234]:   </metadata>
Nov 29 03:06:14 user nova-compute[75234]:   <sysinfo type="smbios">
Nov 29 03:06:14 user nova-compute[75234]:     <system>
Nov 29 03:06:14 user nova-compute[75234]:       <entry name="manufacturer">OpenStack Foundation</entry>
Nov 29 03:06:14 user nova-compute[75234]:       <entry name="product">OpenStack Nova</entry>
Nov 29 03:06:14 user nova-compute[75234]:       <entry name="version">0.0.0</entry>
Nov 29 03:06:14 user nova-compute[75234]:       <entry name="serial">a9350a18-14e6-42c6-a150-0a1487f20ad9</entry>
Nov 29 03:06:14 user nova-compute[75234]:       <entry name="uuid">a9350a18-14e6-42c6-a150-0a1487f20ad9</entry>
Nov 29 03:06:14 user nova-compute[75234]:       <entry name="family">Virtual Machine</entry>
Nov 29 03:06:14 user nova-compute[75234]:     </system>
Nov 29 03:06:14 user nova-compute[75234]:   </sysinfo>
Nov 29 03:06:14 user nova-compute[75234]:   <os>
Nov 29 03:06:14 user nova-compute[75234]:     <type arch="x86_64" machine="pc">hvm</type>
Nov 29 03:06:14 user nova-compute[75234]:     <boot dev="hd"/>
Nov 29 03:06:14 user nova-compute[75234]:     <smbios mode="sysinfo"/>
Nov 29 03:06:14 user nova-compute[75234]:   </os>
Nov 29 03:06:14 user nova-compute[75234]:   <features>
Nov 29 03:06:14 user nova-compute[75234]:     <acpi/>
Nov 29 03:06:14 user nova-compute[75234]:     <apic/>
Nov 29 03:06:14 user nova-compute[75234]:     <vmcoreinfo/>
Nov 29 03:06:14 user nova-compute[75234]:   </features>
Nov 29 03:06:14 user nova-compute[75234]:   <clock offset="utc">
Nov 29 03:06:14 user nova-compute[75234]:     <timer name="pit" tickpolicy="delay"/>
Nov 29 03:06:14 user nova-compute[75234]:     <timer name="rtc" tickpolicy="catchup"/>
Nov 29 03:06:14 user nova-compute[75234]:     <timer name="hpet" present="no"/>
Nov 29 03:06:14 user nova-compute[75234]:   </clock>
Nov 29 03:06:14 user nova-compute[75234]:   <cpu mode="custom" match="exact">
Nov 29 03:06:14 user nova-compute[75234]:     <model>Nehalem</model>
Nov 29 03:06:14 user nova-compute[75234]:     <topology sockets="1" cores="1" threads="1"/>
Nov 29 03:06:14 user nova-compute[75234]:   </cpu>
Nov 29 03:06:14 user nova-compute[75234]:   <devices>
Nov 29 03:06:14 user nova-compute[75234]:     <disk type="file" device="disk">
Nov 29 03:06:14 user nova-compute[75234]:       <driver name="qemu" type="qcow2" cache="none"/>
Nov 29 03:06:14 user nova-compute[75234]:       <source file="/opt/stack/data/nova/instances/a9350a18-14e6-42c6-a150-0a1487f20ad9/disk"/>
Nov 29 03:06:14 user nova-compute[75234]:       <target dev="vda" bus="virtio"/>
Nov 29 03:06:14 user nova-compute[75234]:     </disk>
Nov 29 03:06:14 user nova-compute[75234]:     <interface type="ethernet">
Nov 29 03:06:14 user nova-compute[75234]:       <mac address="fa:16:3e:09:b8:70"/>
Nov 29 03:06:14 user nova-compute[75234]:       <model type="virtio"/>
Nov 29 03:06:14 user nova-compute[75234]:       <mtu size="1442"/>
Nov 29 03:06:14 user nova-compute[75234]:       <target dev="tap52024efc-8c"/>
Nov 29 03:06:14 user nova-compute[75234]:     </interface>
Nov 29 03:06:14 user nova-compute[75234]:     <serial type="pty">
Nov 29 03:06:14 user nova-compute[75234]:       <log file="/opt/stack/data/nova/instances/a9350a18-14e6-42c6-a150-0a1487f20ad9/console.log" append="off"/>
Nov 29 03:06:14 user nova-compute[75234]:     </serial>
Nov 29 03:06:14 user nova-compute[75234]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Nov 29 03:06:14 user nova-compute[75234]:     <video>
Nov 29 03:06:14 user nova-compute[75234]:       <model type="virtio"/>
Nov 29 03:06:14 user nova-compute[75234]:     </video>
Nov 29 03:06:14 user nova-compute[75234]:     <rng model="virtio">
Nov 29 03:06:14 user nova-compute[75234]:       <backend model="random">/dev/urandom</backend>
Nov 29 03:06:14 user nova-compute[75234]:     </rng>
Nov 29 03:06:14 user nova-compute[75234]:     <controller type="usb" index="0" model="none"/>
Nov 29 03:06:14 user nova-compute[75234]:     <memballoon model="virtio">
Nov 29 03:06:14 user nova-compute[75234]:       <stats period="10"/>
Nov 29 03:06:14 user nova-compute[75234]:     </memballoon>
Nov 29 03:06:14 user nova-compute[75234]:   </devices>
Nov 29 03:06:14 user nova-compute[75234]: </domain>
Nov 29 03:06:14 user nova-compute[75234]:  {{(pid=75234) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7568}}
Nov 29 03:06:14 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-c696a156-5f79-4939-99e6-fd7a518710c4 tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-11-29T03:06:06Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-ServerActionsTestOtherB-server-372321920',display_name='tempest-ServerActionsTestOtherB-server-372321920',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-serveractionstestotherb-server-372321920',id=5,image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBFOggqgO1QPxMZPb2P6SX24mx6LOsfQfrzx4GSpNnJiixHqqOn61ObLwI9/TG/r6plC8w1huVCtu+OqAyk9QXUlEr0+VsPNunPSN9WPXBAM4+U+3iCOgHXExdr++34pXkw==',key_name='tempest-keypair-1692341699',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='ad5ea22c63f04f91a3f2fa665e4fc8a1',ramdisk_id='',reservation_id='r-f5lkid4e',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-ServerActionsTestOtherB-70688183',owner_user_name='tempest-ServerActionsTestOtherB-70688183-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-11-29T03:06:09Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='bdddba0ef9c44b6cb43c32a9ab0e29f6',uuid=a9350a18-14e6-42c6-a150-0a1487f20ad9,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "52024efc-8cde-4610-b12c-198f19d75ccf", "address": "fa:16:3e:09:b8:70", "network": {"id": "31c4ca3b-3a2f-47df-87fa-4ad095ee5cf9", "bridge": "br-int", "label": "tempest-ServerActionsTestOtherB-1680745008-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "ad5ea22c63f04f91a3f2fa665e4fc8a1", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap52024efc-8c", "ovs_interfaceid": "52024efc-8cde-4610-b12c-198f19d75ccf", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Nov 29 03:06:14 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-c696a156-5f79-4939-99e6-fd7a518710c4 tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Converting VIF {"id": "52024efc-8cde-4610-b12c-198f19d75ccf", "address": "fa:16:3e:09:b8:70", "network": {"id": "31c4ca3b-3a2f-47df-87fa-4ad095ee5cf9", "bridge": "br-int", "label": "tempest-ServerActionsTestOtherB-1680745008-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "ad5ea22c63f04f91a3f2fa665e4fc8a1", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap52024efc-8c", "ovs_interfaceid": "52024efc-8cde-4610-b12c-198f19d75ccf", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:06:14 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-c696a156-5f79-4939-99e6-fd7a518710c4 tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:09:b8:70,bridge_name='br-int',has_traffic_filtering=True,id=52024efc-8cde-4610-b12c-198f19d75ccf,network=Network(31c4ca3b-3a2f-47df-87fa-4ad095ee5cf9),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap52024efc-8c') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:06:14 user nova-compute[75234]: DEBUG os_vif [None req-c696a156-5f79-4939-99e6-fd7a518710c4 tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:09:b8:70,bridge_name='br-int',has_traffic_filtering=True,id=52024efc-8cde-4610-b12c-198f19d75ccf,network=Network(31c4ca3b-3a2f-47df-87fa-4ad095ee5cf9),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap52024efc-8c') {{(pid=75234) plug /opt/stack/data/venv/lib/python3.10/site-packages/os_vif/__init__.py:76}}
Nov 29 03:06:14 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:06:14 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:06:14 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Nov 29 03:06:14 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:06:14 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbCreateCommand(_result=None, table=QoS, columns={'type': 'linux-noop', 'external_ids': {'id': '2ef8c24f-0c97-504b-852e-38bf52f37f86', '_type': 'linux-noop'}}, row=False) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:06:14 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:06:14 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:06:14 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:06:14 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:06:14 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tap52024efc-8c, may_exist=True, interface_attrs={}) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:06:14 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Port, record=tap52024efc-8c, col_values=(('qos', UUID('1df22596-2c4c-4f7a-952f-593c7370f497')),), if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:06:14 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DbSetCommand(_result=None, table=Interface, record=tap52024efc-8c, col_values=(('external_ids', {'iface-id': '52024efc-8cde-4610-b12c-198f19d75ccf', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:09:b8:70', 'vm-uuid': 'a9350a18-14e6-42c6-a150-0a1487f20ad9'}),), if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:06:14 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:06:14 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:06:14 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:06:14 user nova-compute[75234]: INFO os_vif [None req-c696a156-5f79-4939-99e6-fd7a518710c4 tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:09:b8:70,bridge_name='br-int',has_traffic_filtering=True,id=52024efc-8cde-4610-b12c-198f19d75ccf,network=Network(31c4ca3b-3a2f-47df-87fa-4ad095ee5cf9),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap52024efc-8c')
Nov 29 03:06:15 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:06:15 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-c696a156-5f79-4939-99e6-fd7a518710c4 tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] No BDM found with device name vda, not building metadata. {{(pid=75234) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12089}}
Nov 29 03:06:15 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-c696a156-5f79-4939-99e6-fd7a518710c4 tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] No VIF found with MAC fa:16:3e:09:b8:70, not building metadata {{(pid=75234) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Nov 29 03:06:15 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:06:15 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:06:15 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:06:15 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:06:15 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:06:15 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:06:15 user nova-compute[75234]: DEBUG nova.network.neutron [None req-ceb7826f-126f-40b1-b7f1-9afa48110fc2 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] [instance: f609a7b7-1be2-44b4-9177-7e72a4d4c060] Updating instance_info_cache with network_info: [{"id": "5543f379-2b3a-4dc2-bfe9-52f66a942257", "address": "fa:16:3e:04:ae:43", "network": {"id": "294c3aac-8c9b-45f0-8c0a-344e2d7d8b7c", "bridge": "br-int", "label": "tempest-AttachVolumeTestJSON-700098935-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "e393d23e783b4a40986674d782606c69", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap5543f379-2b", "ovs_interfaceid": "5543f379-2b3a-4dc2-bfe9-52f66a942257", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:06:15 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:06:15 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ceb7826f-126f-40b1-b7f1-9afa48110fc2 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Releasing lock "refresh_cache-f609a7b7-1be2-44b4-9177-7e72a4d4c060" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:06:15 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ceb7826f-126f-40b1-b7f1-9afa48110fc2 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] [instance: f609a7b7-1be2-44b4-9177-7e72a4d4c060] Instance network_info: |[{"id": "5543f379-2b3a-4dc2-bfe9-52f66a942257", "address": "fa:16:3e:04:ae:43", "network": {"id": "294c3aac-8c9b-45f0-8c0a-344e2d7d8b7c", "bridge": "br-int", "label": "tempest-AttachVolumeTestJSON-700098935-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "e393d23e783b4a40986674d782606c69", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap5543f379-2b", "ovs_interfaceid": "5543f379-2b3a-4dc2-bfe9-52f66a942257", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=75234) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1987}}
Nov 29 03:06:15 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-2a161956-e709-4afa-9279-09cb95f44581 req-2e65d174-b979-4c95-b78c-205663a9972c service nova] Acquired lock "refresh_cache-f609a7b7-1be2-44b4-9177-7e72a4d4c060" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:06:15 user nova-compute[75234]: DEBUG nova.network.neutron [req-2a161956-e709-4afa-9279-09cb95f44581 req-2e65d174-b979-4c95-b78c-205663a9972c service nova] [instance: f609a7b7-1be2-44b4-9177-7e72a4d4c060] Refreshing network info cache for port 5543f379-2b3a-4dc2-bfe9-52f66a942257 {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2006}}
Nov 29 03:06:15 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-ceb7826f-126f-40b1-b7f1-9afa48110fc2 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] [instance: f609a7b7-1be2-44b4-9177-7e72a4d4c060] Start _get_guest_xml network_info=[{"id": "5543f379-2b3a-4dc2-bfe9-52f66a942257", "address": "fa:16:3e:04:ae:43", "network": {"id": "294c3aac-8c9b-45f0-8c0a-344e2d7d8b7c", "bridge": "br-int", "label": "tempest-AttachVolumeTestJSON-700098935-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "e393d23e783b4a40986674d782606c69", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap5543f379-2b", "ovs_interfaceid": "5543f379-2b3a-4dc2-bfe9-52f66a942257", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum='c8fc807773e5354afe61636071771906',container_format='bare',created_at=2023-11-29T03:02:15Z,direct_url=<?>,disk_format='qcow2',id=96c30ba7-9b73-4a30-8ba4-7138879573b7,min_disk=0,min_ram=0,name='cirros-0.6.2-x86_64-disk',owner='b18e380921c748b78517b6f1211cd0f1',properties=ImageMetaProps,protected=<?>,size=21430272,status='active',tags=<?>,updated_at=2023-11-29T03:02:17Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [{'device_name': '/dev/vda', 'encryption_format': None, 'encrypted': False, 'device_type': 'disk', 'size': 0, 'disk_bus': 'virtio', 'boot_index': 0, 'encryption_options': None, 'encryption_secret_uuid': None, 'guest_format': None, 'image_id': '96c30ba7-9b73-4a30-8ba4-7138879573b7'}], 'ephemerals': [], 'block_device_mapping': [], 'swap': None} {{(pid=75234) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7562}}
Nov 29 03:06:15 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-ceb7826f-126f-40b1-b7f1-9afa48110fc2 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:06:15 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-ceb7826f-126f-40b1-b7f1-9afa48110fc2 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:06:15 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-ceb7826f-126f-40b1-b7f1-9afa48110fc2 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Searching host: 'user' for CPU controller through CGroups V1... {{(pid=75234) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1652}}
Nov 29 03:06:15 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-ceb7826f-126f-40b1-b7f1-9afa48110fc2 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] CPU controller missing on host. {{(pid=75234) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1662}}
Nov 29 03:06:15 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-ceb7826f-126f-40b1-b7f1-9afa48110fc2 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Searching host: 'user' for CPU controller through CGroups V2... {{(pid=75234) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1671}}
Nov 29 03:06:15 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-ceb7826f-126f-40b1-b7f1-9afa48110fc2 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] CPU controller found on host. {{(pid=75234) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1678}}
Nov 29 03:06:15 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-ceb7826f-126f-40b1-b7f1-9afa48110fc2 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=75234) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5400}}
Nov 29 03:06:15 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-ceb7826f-126f-40b1-b7f1-9afa48110fc2 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-11-29T03:02:04Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='1',id=6,is_public=True,memory_mb=512,name='m1.tiny',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='c8fc807773e5354afe61636071771906',container_format='bare',created_at=2023-11-29T03:02:15Z,direct_url=<?>,disk_format='qcow2',id=96c30ba7-9b73-4a30-8ba4-7138879573b7,min_disk=0,min_ram=0,name='cirros-0.6.2-x86_64-disk',owner='b18e380921c748b78517b6f1211cd0f1',properties=ImageMetaProps,protected=<?>,size=21430272,status='active',tags=<?>,updated_at=2023-11-29T03:02:17Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:563}}
Nov 29 03:06:15 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-ceb7826f-126f-40b1-b7f1-9afa48110fc2 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Flavor limits 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:348}}
Nov 29 03:06:15 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-ceb7826f-126f-40b1-b7f1-9afa48110fc2 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Image limits 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:352}}
Nov 29 03:06:15 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-ceb7826f-126f-40b1-b7f1-9afa48110fc2 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Flavor pref 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:388}}
Nov 29 03:06:15 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-ceb7826f-126f-40b1-b7f1-9afa48110fc2 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Image pref 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:392}}
Nov 29 03:06:15 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-ceb7826f-126f-40b1-b7f1-9afa48110fc2 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:430}}
Nov 29 03:06:15 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-ceb7826f-126f-40b1-b7f1-9afa48110fc2 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:569}}
Nov 29 03:06:15 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-ceb7826f-126f-40b1-b7f1-9afa48110fc2 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=75234) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:471}}
Nov 29 03:06:15 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-ceb7826f-126f-40b1-b7f1-9afa48110fc2 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Got 1 possible topologies {{(pid=75234) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:501}}
Nov 29 03:06:15 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-ceb7826f-126f-40b1-b7f1-9afa48110fc2 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:575}}
Nov 29 03:06:15 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-ceb7826f-126f-40b1-b7f1-9afa48110fc2 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:577}}
Nov 29 03:06:15 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-ceb7826f-126f-40b1-b7f1-9afa48110fc2 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-11-29T03:06:06Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-AttachVolumeTestJSON-server-394712256',display_name='tempest-AttachVolumeTestJSON-server-394712256',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-attachvolumetestjson-server-394712256',id=4,image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBGphX+MWMhY0YZ/wlOvEQFKFHTIX4g9KMhxkTe+ErHeVHCk7d+KpOlTDVK9edUWFFxjgJKboch3pp1xrAYGa/rIEWyUV4YOfCGPFoAeneAuweb6ekmwg4qgLxfML0aXASQ==',key_name='tempest-keypair-213174705',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='e393d23e783b4a40986674d782606c69',ramdisk_id='',reservation_id='r-9o6ggnbe',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-AttachVolumeTestJSON-1911423306',owner_user_name='tempest-AttachVolumeTestJSON-1911423306-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-11-29T03:06:08Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='bb1126c998ee430baac2e2494f30c0aa',uuid=f609a7b7-1be2-44b4-9177-7e72a4d4c060,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "5543f379-2b3a-4dc2-bfe9-52f66a942257", "address": "fa:16:3e:04:ae:43", "network": {"id": "294c3aac-8c9b-45f0-8c0a-344e2d7d8b7c", "bridge": "br-int", "label": "tempest-AttachVolumeTestJSON-700098935-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "e393d23e783b4a40986674d782606c69", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap5543f379-2b", "ovs_interfaceid": "5543f379-2b3a-4dc2-bfe9-52f66a942257", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=75234) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Nov 29 03:06:15 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-ceb7826f-126f-40b1-b7f1-9afa48110fc2 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Converting VIF {"id": "5543f379-2b3a-4dc2-bfe9-52f66a942257", "address": "fa:16:3e:04:ae:43", "network": {"id": "294c3aac-8c9b-45f0-8c0a-344e2d7d8b7c", "bridge": "br-int", "label": "tempest-AttachVolumeTestJSON-700098935-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "e393d23e783b4a40986674d782606c69", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap5543f379-2b", "ovs_interfaceid": "5543f379-2b3a-4dc2-bfe9-52f66a942257", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:06:15 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-ceb7826f-126f-40b1-b7f1-9afa48110fc2 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:04:ae:43,bridge_name='br-int',has_traffic_filtering=True,id=5543f379-2b3a-4dc2-bfe9-52f66a942257,network=Network(294c3aac-8c9b-45f0-8c0a-344e2d7d8b7c),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap5543f379-2b') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:06:15 user nova-compute[75234]: DEBUG nova.objects.instance [None req-ceb7826f-126f-40b1-b7f1-9afa48110fc2 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Lazy-loading 'pci_devices' on Instance uuid f609a7b7-1be2-44b4-9177-7e72a4d4c060 {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:06:15 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-ceb7826f-126f-40b1-b7f1-9afa48110fc2 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] [instance: f609a7b7-1be2-44b4-9177-7e72a4d4c060] End _get_guest_xml xml=<domain type="kvm">
Nov 29 03:06:15 user nova-compute[75234]:   <uuid>f609a7b7-1be2-44b4-9177-7e72a4d4c060</uuid>
Nov 29 03:06:15 user nova-compute[75234]:   <name>instance-00000004</name>
Nov 29 03:06:15 user nova-compute[75234]:   <memory>524288</memory>
Nov 29 03:06:15 user nova-compute[75234]:   <vcpu>1</vcpu>
Nov 29 03:06:15 user nova-compute[75234]:   <metadata>
Nov 29 03:06:15 user nova-compute[75234]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Nov 29 03:06:15 user nova-compute[75234]:       <nova:package version="0.0.0"/>
Nov 29 03:06:15 user nova-compute[75234]:       <nova:name>tempest-AttachVolumeTestJSON-server-394712256</nova:name>
Nov 29 03:06:15 user nova-compute[75234]:       <nova:creationTime>2023-11-29 03:06:15</nova:creationTime>
Nov 29 03:06:15 user nova-compute[75234]:       <nova:flavor name="m1.tiny">
Nov 29 03:06:15 user nova-compute[75234]:         <nova:memory>512</nova:memory>
Nov 29 03:06:15 user nova-compute[75234]:         <nova:disk>1</nova:disk>
Nov 29 03:06:15 user nova-compute[75234]:         <nova:swap>0</nova:swap>
Nov 29 03:06:15 user nova-compute[75234]:         <nova:ephemeral>0</nova:ephemeral>
Nov 29 03:06:15 user nova-compute[75234]:         <nova:vcpus>1</nova:vcpus>
Nov 29 03:06:15 user nova-compute[75234]:       </nova:flavor>
Nov 29 03:06:15 user nova-compute[75234]:       <nova:owner>
Nov 29 03:06:15 user nova-compute[75234]:         <nova:user uuid="bb1126c998ee430baac2e2494f30c0aa">tempest-AttachVolumeTestJSON-1911423306-project-member</nova:user>
Nov 29 03:06:15 user nova-compute[75234]:         <nova:project uuid="e393d23e783b4a40986674d782606c69">tempest-AttachVolumeTestJSON-1911423306</nova:project>
Nov 29 03:06:15 user nova-compute[75234]:       </nova:owner>
Nov 29 03:06:15 user nova-compute[75234]:       <nova:root type="image" uuid="96c30ba7-9b73-4a30-8ba4-7138879573b7"/>
Nov 29 03:06:15 user nova-compute[75234]:       <nova:ports>
Nov 29 03:06:15 user nova-compute[75234]:         <nova:port uuid="5543f379-2b3a-4dc2-bfe9-52f66a942257">
Nov 29 03:06:15 user nova-compute[75234]:           <nova:ip type="fixed" address="10.0.0.3" ipVersion="4"/>
Nov 29 03:06:15 user nova-compute[75234]:         </nova:port>
Nov 29 03:06:15 user nova-compute[75234]:       </nova:ports>
Nov 29 03:06:15 user nova-compute[75234]:     </nova:instance>
Nov 29 03:06:15 user nova-compute[75234]:   </metadata>
Nov 29 03:06:15 user nova-compute[75234]:   <sysinfo type="smbios">
Nov 29 03:06:15 user nova-compute[75234]:     <system>
Nov 29 03:06:15 user nova-compute[75234]:       <entry name="manufacturer">OpenStack Foundation</entry>
Nov 29 03:06:15 user nova-compute[75234]:       <entry name="product">OpenStack Nova</entry>
Nov 29 03:06:15 user nova-compute[75234]:       <entry name="version">0.0.0</entry>
Nov 29 03:06:15 user nova-compute[75234]:       <entry name="serial">f609a7b7-1be2-44b4-9177-7e72a4d4c060</entry>
Nov 29 03:06:15 user nova-compute[75234]:       <entry name="uuid">f609a7b7-1be2-44b4-9177-7e72a4d4c060</entry>
Nov 29 03:06:15 user nova-compute[75234]:       <entry name="family">Virtual Machine</entry>
Nov 29 03:06:15 user nova-compute[75234]:     </system>
Nov 29 03:06:15 user nova-compute[75234]:   </sysinfo>
Nov 29 03:06:15 user nova-compute[75234]:   <os>
Nov 29 03:06:15 user nova-compute[75234]:     <type arch="x86_64" machine="pc">hvm</type>
Nov 29 03:06:15 user nova-compute[75234]:     <boot dev="hd"/>
Nov 29 03:06:15 user nova-compute[75234]:     <smbios mode="sysinfo"/>
Nov 29 03:06:15 user nova-compute[75234]:   </os>
Nov 29 03:06:15 user nova-compute[75234]:   <features>
Nov 29 03:06:15 user nova-compute[75234]:     <acpi/>
Nov 29 03:06:15 user nova-compute[75234]:     <apic/>
Nov 29 03:06:15 user nova-compute[75234]:     <vmcoreinfo/>
Nov 29 03:06:15 user nova-compute[75234]:   </features>
Nov 29 03:06:15 user nova-compute[75234]:   <clock offset="utc">
Nov 29 03:06:15 user nova-compute[75234]:     <timer name="pit" tickpolicy="delay"/>
Nov 29 03:06:15 user nova-compute[75234]:     <timer name="rtc" tickpolicy="catchup"/>
Nov 29 03:06:15 user nova-compute[75234]:     <timer name="hpet" present="no"/>
Nov 29 03:06:15 user nova-compute[75234]:   </clock>
Nov 29 03:06:15 user nova-compute[75234]:   <cpu mode="custom" match="exact">
Nov 29 03:06:15 user nova-compute[75234]:     <model>Nehalem</model>
Nov 29 03:06:15 user nova-compute[75234]:     <topology sockets="1" cores="1" threads="1"/>
Nov 29 03:06:15 user nova-compute[75234]:   </cpu>
Nov 29 03:06:15 user nova-compute[75234]:   <devices>
Nov 29 03:06:15 user nova-compute[75234]:     <disk type="file" device="disk">
Nov 29 03:06:15 user nova-compute[75234]:       <driver name="qemu" type="qcow2" cache="none"/>
Nov 29 03:06:15 user nova-compute[75234]:       <source file="/opt/stack/data/nova/instances/f609a7b7-1be2-44b4-9177-7e72a4d4c060/disk"/>
Nov 29 03:06:15 user nova-compute[75234]:       <target dev="vda" bus="virtio"/>
Nov 29 03:06:15 user nova-compute[75234]:     </disk>
Nov 29 03:06:15 user nova-compute[75234]:     <interface type="ethernet">
Nov 29 03:06:15 user nova-compute[75234]:       <mac address="fa:16:3e:04:ae:43"/>
Nov 29 03:06:15 user nova-compute[75234]:       <model type="virtio"/>
Nov 29 03:06:15 user nova-compute[75234]:       <mtu size="1442"/>
Nov 29 03:06:15 user nova-compute[75234]:       <target dev="tap5543f379-2b"/>
Nov 29 03:06:15 user nova-compute[75234]:     </interface>
Nov 29 03:06:15 user nova-compute[75234]:     <serial type="pty">
Nov 29 03:06:15 user nova-compute[75234]:       <log file="/opt/stack/data/nova/instances/f609a7b7-1be2-44b4-9177-7e72a4d4c060/console.log" append="off"/>
Nov 29 03:06:15 user nova-compute[75234]:     </serial>
Nov 29 03:06:15 user nova-compute[75234]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Nov 29 03:06:15 user nova-compute[75234]:     <video>
Nov 29 03:06:15 user nova-compute[75234]:       <model type="virtio"/>
Nov 29 03:06:15 user nova-compute[75234]:     </video>
Nov 29 03:06:15 user nova-compute[75234]:     <rng model="virtio">
Nov 29 03:06:15 user nova-compute[75234]:       <backend model="random">/dev/urandom</backend>
Nov 29 03:06:15 user nova-compute[75234]:     </rng>
Nov 29 03:06:15 user nova-compute[75234]:     <controller type="usb" index="0" model="none"/>
Nov 29 03:06:15 user nova-compute[75234]:     <memballoon model="virtio">
Nov 29 03:06:15 user nova-compute[75234]:       <stats period="10"/>
Nov 29 03:06:15 user nova-compute[75234]:     </memballoon>
Nov 29 03:06:15 user nova-compute[75234]:   </devices>
Nov 29 03:06:15 user nova-compute[75234]: </domain>
Nov 29 03:06:15 user nova-compute[75234]:  {{(pid=75234) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7568}}
Nov 29 03:06:15 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-ceb7826f-126f-40b1-b7f1-9afa48110fc2 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-11-29T03:06:06Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-AttachVolumeTestJSON-server-394712256',display_name='tempest-AttachVolumeTestJSON-server-394712256',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-attachvolumetestjson-server-394712256',id=4,image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBGphX+MWMhY0YZ/wlOvEQFKFHTIX4g9KMhxkTe+ErHeVHCk7d+KpOlTDVK9edUWFFxjgJKboch3pp1xrAYGa/rIEWyUV4YOfCGPFoAeneAuweb6ekmwg4qgLxfML0aXASQ==',key_name='tempest-keypair-213174705',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='e393d23e783b4a40986674d782606c69',ramdisk_id='',reservation_id='r-9o6ggnbe',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-AttachVolumeTestJSON-1911423306',owner_user_name='tempest-AttachVolumeTestJSON-1911423306-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-11-29T03:06:08Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='bb1126c998ee430baac2e2494f30c0aa',uuid=f609a7b7-1be2-44b4-9177-7e72a4d4c060,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "5543f379-2b3a-4dc2-bfe9-52f66a942257", "address": "fa:16:3e:04:ae:43", "network": {"id": "294c3aac-8c9b-45f0-8c0a-344e2d7d8b7c", "bridge": "br-int", "label": "tempest-AttachVolumeTestJSON-700098935-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "e393d23e783b4a40986674d782606c69", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap5543f379-2b", "ovs_interfaceid": "5543f379-2b3a-4dc2-bfe9-52f66a942257", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Nov 29 03:06:15 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-ceb7826f-126f-40b1-b7f1-9afa48110fc2 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Converting VIF {"id": "5543f379-2b3a-4dc2-bfe9-52f66a942257", "address": "fa:16:3e:04:ae:43", "network": {"id": "294c3aac-8c9b-45f0-8c0a-344e2d7d8b7c", "bridge": "br-int", "label": "tempest-AttachVolumeTestJSON-700098935-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "e393d23e783b4a40986674d782606c69", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap5543f379-2b", "ovs_interfaceid": "5543f379-2b3a-4dc2-bfe9-52f66a942257", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:06:15 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-ceb7826f-126f-40b1-b7f1-9afa48110fc2 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:04:ae:43,bridge_name='br-int',has_traffic_filtering=True,id=5543f379-2b3a-4dc2-bfe9-52f66a942257,network=Network(294c3aac-8c9b-45f0-8c0a-344e2d7d8b7c),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap5543f379-2b') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:06:15 user nova-compute[75234]: DEBUG os_vif [None req-ceb7826f-126f-40b1-b7f1-9afa48110fc2 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:04:ae:43,bridge_name='br-int',has_traffic_filtering=True,id=5543f379-2b3a-4dc2-bfe9-52f66a942257,network=Network(294c3aac-8c9b-45f0-8c0a-344e2d7d8b7c),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap5543f379-2b') {{(pid=75234) plug /opt/stack/data/venv/lib/python3.10/site-packages/os_vif/__init__.py:76}}
Nov 29 03:06:15 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:06:15 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:06:15 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Nov 29 03:06:15 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:06:15 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbCreateCommand(_result=None, table=QoS, columns={'type': 'linux-noop', 'external_ids': {'id': 'abd56afa-ec17-518b-948a-b5862316fd9e', '_type': 'linux-noop'}}, row=False) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:06:15 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:06:15 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:06:15 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:06:15 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:06:15 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tap5543f379-2b, may_exist=True, interface_attrs={}) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:06:15 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Port, record=tap5543f379-2b, col_values=(('qos', UUID('8f761051-aad1-489a-9d20-44261a252c60')),), if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:06:15 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DbSetCommand(_result=None, table=Interface, record=tap5543f379-2b, col_values=(('external_ids', {'iface-id': '5543f379-2b3a-4dc2-bfe9-52f66a942257', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:04:ae:43', 'vm-uuid': 'f609a7b7-1be2-44b4-9177-7e72a4d4c060'}),), if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:06:15 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:06:15 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:06:15 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:06:15 user nova-compute[75234]: INFO os_vif [None req-ceb7826f-126f-40b1-b7f1-9afa48110fc2 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:04:ae:43,bridge_name='br-int',has_traffic_filtering=True,id=5543f379-2b3a-4dc2-bfe9-52f66a942257,network=Network(294c3aac-8c9b-45f0-8c0a-344e2d7d8b7c),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap5543f379-2b')
Nov 29 03:06:15 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-ceb7826f-126f-40b1-b7f1-9afa48110fc2 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] No BDM found with device name vda, not building metadata. {{(pid=75234) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12089}}
Nov 29 03:06:15 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-ceb7826f-126f-40b1-b7f1-9afa48110fc2 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] No VIF found with MAC fa:16:3e:04:ae:43, not building metadata {{(pid=75234) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Nov 29 03:06:16 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:06:16 user nova-compute[75234]: DEBUG nova.network.neutron [req-0490d72a-ebac-4951-8707-204cd2ab47f9 req-fa56be0e-a566-4909-993b-4c029ebcdcea service nova] [instance: a9350a18-14e6-42c6-a150-0a1487f20ad9] Updated VIF entry in instance network info cache for port 52024efc-8cde-4610-b12c-198f19d75ccf. {{(pid=75234) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3481}}
Nov 29 03:06:16 user nova-compute[75234]: DEBUG nova.network.neutron [req-0490d72a-ebac-4951-8707-204cd2ab47f9 req-fa56be0e-a566-4909-993b-4c029ebcdcea service nova] [instance: a9350a18-14e6-42c6-a150-0a1487f20ad9] Updating instance_info_cache with network_info: [{"id": "52024efc-8cde-4610-b12c-198f19d75ccf", "address": "fa:16:3e:09:b8:70", "network": {"id": "31c4ca3b-3a2f-47df-87fa-4ad095ee5cf9", "bridge": "br-int", "label": "tempest-ServerActionsTestOtherB-1680745008-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "ad5ea22c63f04f91a3f2fa665e4fc8a1", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap52024efc-8c", "ovs_interfaceid": "52024efc-8cde-4610-b12c-198f19d75ccf", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:06:16 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-0490d72a-ebac-4951-8707-204cd2ab47f9 req-fa56be0e-a566-4909-993b-4c029ebcdcea service nova] Releasing lock "refresh_cache-a9350a18-14e6-42c6-a150-0a1487f20ad9" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:06:16 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:06:16 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:06:16 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:06:16 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:06:16 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:06:16 user nova-compute[75234]: DEBUG nova.network.neutron [None req-71e7f72f-cfc6-42f1-9655-a38fb0588077 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] [instance: 8297c264-a213-44ee-be46-5c08eabda9f7] Successfully updated port: d26334d2-8040-4022-b658-69be1cf6a025 {{(pid=75234) _update_port /opt/stack/nova/nova/network/neutron.py:586}}
Nov 29 03:06:16 user nova-compute[75234]: DEBUG nova.compute.manager [req-5a71089c-dcaa-4052-b465-171f19b92644 req-993676e8-2888-49c3-a233-fdd3e379a0e7 service nova] [instance: 8297c264-a213-44ee-be46-5c08eabda9f7] Received event network-changed-d26334d2-8040-4022-b658-69be1cf6a025 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:06:16 user nova-compute[75234]: DEBUG nova.compute.manager [req-5a71089c-dcaa-4052-b465-171f19b92644 req-993676e8-2888-49c3-a233-fdd3e379a0e7 service nova] [instance: 8297c264-a213-44ee-be46-5c08eabda9f7] Refreshing instance network info cache due to event network-changed-d26334d2-8040-4022-b658-69be1cf6a025. {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11117}}
Nov 29 03:06:16 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-5a71089c-dcaa-4052-b465-171f19b92644 req-993676e8-2888-49c3-a233-fdd3e379a0e7 service nova] Acquiring lock "refresh_cache-8297c264-a213-44ee-be46-5c08eabda9f7" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:06:16 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-5a71089c-dcaa-4052-b465-171f19b92644 req-993676e8-2888-49c3-a233-fdd3e379a0e7 service nova] Acquired lock "refresh_cache-8297c264-a213-44ee-be46-5c08eabda9f7" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:06:16 user nova-compute[75234]: DEBUG nova.network.neutron [req-5a71089c-dcaa-4052-b465-171f19b92644 req-993676e8-2888-49c3-a233-fdd3e379a0e7 service nova] [instance: 8297c264-a213-44ee-be46-5c08eabda9f7] Refreshing network info cache for port d26334d2-8040-4022-b658-69be1cf6a025 {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2006}}
Nov 29 03:06:16 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-71e7f72f-cfc6-42f1-9655-a38fb0588077 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] Acquiring lock "refresh_cache-8297c264-a213-44ee-be46-5c08eabda9f7" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:06:16 user nova-compute[75234]: DEBUG nova.network.neutron [req-5a71089c-dcaa-4052-b465-171f19b92644 req-993676e8-2888-49c3-a233-fdd3e379a0e7 service nova] [instance: 8297c264-a213-44ee-be46-5c08eabda9f7] Instance cache missing network info. {{(pid=75234) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3322}}
Nov 29 03:06:16 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:06:16 user nova-compute[75234]: DEBUG nova.network.neutron [req-2a161956-e709-4afa-9279-09cb95f44581 req-2e65d174-b979-4c95-b78c-205663a9972c service nova] [instance: f609a7b7-1be2-44b4-9177-7e72a4d4c060] Updated VIF entry in instance network info cache for port 5543f379-2b3a-4dc2-bfe9-52f66a942257. {{(pid=75234) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3481}}
Nov 29 03:06:16 user nova-compute[75234]: DEBUG nova.network.neutron [req-2a161956-e709-4afa-9279-09cb95f44581 req-2e65d174-b979-4c95-b78c-205663a9972c service nova] [instance: f609a7b7-1be2-44b4-9177-7e72a4d4c060] Updating instance_info_cache with network_info: [{"id": "5543f379-2b3a-4dc2-bfe9-52f66a942257", "address": "fa:16:3e:04:ae:43", "network": {"id": "294c3aac-8c9b-45f0-8c0a-344e2d7d8b7c", "bridge": "br-int", "label": "tempest-AttachVolumeTestJSON-700098935-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "e393d23e783b4a40986674d782606c69", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap5543f379-2b", "ovs_interfaceid": "5543f379-2b3a-4dc2-bfe9-52f66a942257", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:06:17 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-2a161956-e709-4afa-9279-09cb95f44581 req-2e65d174-b979-4c95-b78c-205663a9972c service nova] Releasing lock "refresh_cache-f609a7b7-1be2-44b4-9177-7e72a4d4c060" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:06:17 user nova-compute[75234]: DEBUG nova.network.neutron [req-5a71089c-dcaa-4052-b465-171f19b92644 req-993676e8-2888-49c3-a233-fdd3e379a0e7 service nova] [instance: 8297c264-a213-44ee-be46-5c08eabda9f7] Updating instance_info_cache with network_info: [] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:06:17 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-5a71089c-dcaa-4052-b465-171f19b92644 req-993676e8-2888-49c3-a233-fdd3e379a0e7 service nova] Releasing lock "refresh_cache-8297c264-a213-44ee-be46-5c08eabda9f7" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:06:17 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-71e7f72f-cfc6-42f1-9655-a38fb0588077 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] Acquired lock "refresh_cache-8297c264-a213-44ee-be46-5c08eabda9f7" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:06:17 user nova-compute[75234]: DEBUG nova.network.neutron [None req-71e7f72f-cfc6-42f1-9655-a38fb0588077 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] [instance: 8297c264-a213-44ee-be46-5c08eabda9f7] Building network info cache for instance {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2009}}
Nov 29 03:06:17 user nova-compute[75234]: DEBUG nova.network.neutron [None req-71e7f72f-cfc6-42f1-9655-a38fb0588077 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] [instance: 8297c264-a213-44ee-be46-5c08eabda9f7] Instance cache missing network info. {{(pid=75234) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3322}}
Nov 29 03:06:17 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:06:17 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:06:18 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:06:18 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:06:18 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:06:18 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:06:18 user nova-compute[75234]: DEBUG nova.network.neutron [None req-71e7f72f-cfc6-42f1-9655-a38fb0588077 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] [instance: 8297c264-a213-44ee-be46-5c08eabda9f7] Updating instance_info_cache with network_info: [{"id": "d26334d2-8040-4022-b658-69be1cf6a025", "address": "fa:16:3e:5f:9d:5a", "network": {"id": "e0366490-9fe2-4429-88c8-6625128e645d", "bridge": "br-int", "label": "tempest-ServerStableDeviceRescueTest-804259918-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "745bdb3c72794155829dc0a69f35b321", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapd26334d2-80", "ovs_interfaceid": "d26334d2-8040-4022-b658-69be1cf6a025", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:06:18 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-71e7f72f-cfc6-42f1-9655-a38fb0588077 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] Releasing lock "refresh_cache-8297c264-a213-44ee-be46-5c08eabda9f7" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:06:18 user nova-compute[75234]: DEBUG nova.compute.manager [None req-71e7f72f-cfc6-42f1-9655-a38fb0588077 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] [instance: 8297c264-a213-44ee-be46-5c08eabda9f7] Instance network_info: |[{"id": "d26334d2-8040-4022-b658-69be1cf6a025", "address": "fa:16:3e:5f:9d:5a", "network": {"id": "e0366490-9fe2-4429-88c8-6625128e645d", "bridge": "br-int", "label": "tempest-ServerStableDeviceRescueTest-804259918-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "745bdb3c72794155829dc0a69f35b321", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapd26334d2-80", "ovs_interfaceid": "d26334d2-8040-4022-b658-69be1cf6a025", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=75234) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1987}}
Nov 29 03:06:18 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-71e7f72f-cfc6-42f1-9655-a38fb0588077 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] [instance: 8297c264-a213-44ee-be46-5c08eabda9f7] Start _get_guest_xml network_info=[{"id": "d26334d2-8040-4022-b658-69be1cf6a025", "address": "fa:16:3e:5f:9d:5a", "network": {"id": "e0366490-9fe2-4429-88c8-6625128e645d", "bridge": "br-int", "label": "tempest-ServerStableDeviceRescueTest-804259918-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "745bdb3c72794155829dc0a69f35b321", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapd26334d2-80", "ovs_interfaceid": "d26334d2-8040-4022-b658-69be1cf6a025", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum='c8fc807773e5354afe61636071771906',container_format='bare',created_at=2023-11-29T03:02:15Z,direct_url=<?>,disk_format='qcow2',id=96c30ba7-9b73-4a30-8ba4-7138879573b7,min_disk=0,min_ram=0,name='cirros-0.6.2-x86_64-disk',owner='b18e380921c748b78517b6f1211cd0f1',properties=ImageMetaProps,protected=<?>,size=21430272,status='active',tags=<?>,updated_at=2023-11-29T03:02:17Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [{'device_name': '/dev/vda', 'encryption_format': None, 'encrypted': False, 'device_type': 'disk', 'size': 0, 'disk_bus': 'virtio', 'boot_index': 0, 'encryption_options': None, 'encryption_secret_uuid': None, 'guest_format': None, 'image_id': '96c30ba7-9b73-4a30-8ba4-7138879573b7'}], 'ephemerals': [], 'block_device_mapping': [], 'swap': None} {{(pid=75234) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7562}}
Nov 29 03:06:18 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-71e7f72f-cfc6-42f1-9655-a38fb0588077 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:06:18 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-71e7f72f-cfc6-42f1-9655-a38fb0588077 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:06:18 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-71e7f72f-cfc6-42f1-9655-a38fb0588077 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] Searching host: 'user' for CPU controller through CGroups V1... {{(pid=75234) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1652}}
Nov 29 03:06:18 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-71e7f72f-cfc6-42f1-9655-a38fb0588077 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] CPU controller missing on host. {{(pid=75234) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1662}}
Nov 29 03:06:18 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-71e7f72f-cfc6-42f1-9655-a38fb0588077 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] Searching host: 'user' for CPU controller through CGroups V2... {{(pid=75234) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1671}}
Nov 29 03:06:18 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-71e7f72f-cfc6-42f1-9655-a38fb0588077 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] CPU controller found on host. {{(pid=75234) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1678}}
Nov 29 03:06:18 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-71e7f72f-cfc6-42f1-9655-a38fb0588077 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=75234) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5400}}
Nov 29 03:06:18 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-71e7f72f-cfc6-42f1-9655-a38fb0588077 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-11-29T03:02:04Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='1',id=6,is_public=True,memory_mb=512,name='m1.tiny',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='c8fc807773e5354afe61636071771906',container_format='bare',created_at=2023-11-29T03:02:15Z,direct_url=<?>,disk_format='qcow2',id=96c30ba7-9b73-4a30-8ba4-7138879573b7,min_disk=0,min_ram=0,name='cirros-0.6.2-x86_64-disk',owner='b18e380921c748b78517b6f1211cd0f1',properties=ImageMetaProps,protected=<?>,size=21430272,status='active',tags=<?>,updated_at=2023-11-29T03:02:17Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:563}}
Nov 29 03:06:18 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-71e7f72f-cfc6-42f1-9655-a38fb0588077 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] Flavor limits 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:348}}
Nov 29 03:06:18 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-71e7f72f-cfc6-42f1-9655-a38fb0588077 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] Image limits 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:352}}
Nov 29 03:06:18 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-71e7f72f-cfc6-42f1-9655-a38fb0588077 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] Flavor pref 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:388}}
Nov 29 03:06:18 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-71e7f72f-cfc6-42f1-9655-a38fb0588077 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] Image pref 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:392}}
Nov 29 03:06:18 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-71e7f72f-cfc6-42f1-9655-a38fb0588077 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:430}}
Nov 29 03:06:18 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-71e7f72f-cfc6-42f1-9655-a38fb0588077 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:569}}
Nov 29 03:06:18 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-71e7f72f-cfc6-42f1-9655-a38fb0588077 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=75234) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:471}}
Nov 29 03:06:18 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-71e7f72f-cfc6-42f1-9655-a38fb0588077 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] Got 1 possible topologies {{(pid=75234) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:501}}
Nov 29 03:06:18 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-71e7f72f-cfc6-42f1-9655-a38fb0588077 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:575}}
Nov 29 03:06:18 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-71e7f72f-cfc6-42f1-9655-a38fb0588077 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:577}}
Nov 29 03:06:18 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-71e7f72f-cfc6-42f1-9655-a38fb0588077 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-11-29T03:06:07Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-ServerStableDeviceRescueTest-server-428917867',display_name='tempest-ServerStableDeviceRescueTest-server-428917867',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-serverstabledevicerescuetest-server-428917867',id=6,image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBPAGeJQ8OrWhDv/r5lktd6O/obaXRK7/7VEiU/V0MksOr2sSmn5M84mkyhNPvWnFo3k1J68udxxAQOPj9YvQEkVpNgH7z7grcEsUVqlgMVin9R9cWVCI3LAQJjKBdTQkeA==',key_name='tempest-keypair-1167119775',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='745bdb3c72794155829dc0a69f35b321',ramdisk_id='',reservation_id='r-tt096l76',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-ServerStableDeviceRescueTest-1509102897',owner_user_name='tempest-ServerStableDeviceRescueTest-1509102897-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-11-29T03:06:10Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='566e3dc84bbb40d9badcacbcaec9b987',uuid=8297c264-a213-44ee-be46-5c08eabda9f7,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "d26334d2-8040-4022-b658-69be1cf6a025", "address": "fa:16:3e:5f:9d:5a", "network": {"id": "e0366490-9fe2-4429-88c8-6625128e645d", "bridge": "br-int", "label": "tempest-ServerStableDeviceRescueTest-804259918-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "745bdb3c72794155829dc0a69f35b321", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapd26334d2-80", "ovs_interfaceid": "d26334d2-8040-4022-b658-69be1cf6a025", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=75234) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Nov 29 03:06:18 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-71e7f72f-cfc6-42f1-9655-a38fb0588077 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] Converting VIF {"id": "d26334d2-8040-4022-b658-69be1cf6a025", "address": "fa:16:3e:5f:9d:5a", "network": {"id": "e0366490-9fe2-4429-88c8-6625128e645d", "bridge": "br-int", "label": "tempest-ServerStableDeviceRescueTest-804259918-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "745bdb3c72794155829dc0a69f35b321", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapd26334d2-80", "ovs_interfaceid": "d26334d2-8040-4022-b658-69be1cf6a025", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:06:18 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-71e7f72f-cfc6-42f1-9655-a38fb0588077 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:5f:9d:5a,bridge_name='br-int',has_traffic_filtering=True,id=d26334d2-8040-4022-b658-69be1cf6a025,network=Network(e0366490-9fe2-4429-88c8-6625128e645d),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapd26334d2-80') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:06:18 user nova-compute[75234]: DEBUG nova.objects.instance [None req-71e7f72f-cfc6-42f1-9655-a38fb0588077 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] Lazy-loading 'pci_devices' on Instance uuid 8297c264-a213-44ee-be46-5c08eabda9f7 {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:06:18 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:06:18 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-71e7f72f-cfc6-42f1-9655-a38fb0588077 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] [instance: 8297c264-a213-44ee-be46-5c08eabda9f7] End _get_guest_xml xml=<domain type="kvm">
Nov 29 03:06:18 user nova-compute[75234]:   <uuid>8297c264-a213-44ee-be46-5c08eabda9f7</uuid>
Nov 29 03:06:18 user nova-compute[75234]:   <name>instance-00000006</name>
Nov 29 03:06:18 user nova-compute[75234]:   <memory>524288</memory>
Nov 29 03:06:18 user nova-compute[75234]:   <vcpu>1</vcpu>
Nov 29 03:06:18 user nova-compute[75234]:   <metadata>
Nov 29 03:06:18 user nova-compute[75234]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Nov 29 03:06:18 user nova-compute[75234]:       <nova:package version="0.0.0"/>
Nov 29 03:06:18 user nova-compute[75234]:       <nova:name>tempest-ServerStableDeviceRescueTest-server-428917867</nova:name>
Nov 29 03:06:18 user nova-compute[75234]:       <nova:creationTime>2023-11-29 03:06:18</nova:creationTime>
Nov 29 03:06:18 user nova-compute[75234]:       <nova:flavor name="m1.tiny">
Nov 29 03:06:18 user nova-compute[75234]:         <nova:memory>512</nova:memory>
Nov 29 03:06:18 user nova-compute[75234]:         <nova:disk>1</nova:disk>
Nov 29 03:06:18 user nova-compute[75234]:         <nova:swap>0</nova:swap>
Nov 29 03:06:18 user nova-compute[75234]:         <nova:ephemeral>0</nova:ephemeral>
Nov 29 03:06:18 user nova-compute[75234]:         <nova:vcpus>1</nova:vcpus>
Nov 29 03:06:18 user nova-compute[75234]:       </nova:flavor>
Nov 29 03:06:18 user nova-compute[75234]:       <nova:owner>
Nov 29 03:06:18 user nova-compute[75234]:         <nova:user uuid="566e3dc84bbb40d9badcacbcaec9b987">tempest-ServerStableDeviceRescueTest-1509102897-project-member</nova:user>
Nov 29 03:06:18 user nova-compute[75234]:         <nova:project uuid="745bdb3c72794155829dc0a69f35b321">tempest-ServerStableDeviceRescueTest-1509102897</nova:project>
Nov 29 03:06:18 user nova-compute[75234]:       </nova:owner>
Nov 29 03:06:18 user nova-compute[75234]:       <nova:root type="image" uuid="96c30ba7-9b73-4a30-8ba4-7138879573b7"/>
Nov 29 03:06:18 user nova-compute[75234]:       <nova:ports>
Nov 29 03:06:18 user nova-compute[75234]:         <nova:port uuid="d26334d2-8040-4022-b658-69be1cf6a025">
Nov 29 03:06:18 user nova-compute[75234]:           <nova:ip type="fixed" address="10.0.0.12" ipVersion="4"/>
Nov 29 03:06:18 user nova-compute[75234]:         </nova:port>
Nov 29 03:06:18 user nova-compute[75234]:       </nova:ports>
Nov 29 03:06:18 user nova-compute[75234]:     </nova:instance>
Nov 29 03:06:18 user nova-compute[75234]:   </metadata>
Nov 29 03:06:18 user nova-compute[75234]:   <sysinfo type="smbios">
Nov 29 03:06:18 user nova-compute[75234]:     <system>
Nov 29 03:06:18 user nova-compute[75234]:       <entry name="manufacturer">OpenStack Foundation</entry>
Nov 29 03:06:18 user nova-compute[75234]:       <entry name="product">OpenStack Nova</entry>
Nov 29 03:06:18 user nova-compute[75234]:       <entry name="version">0.0.0</entry>
Nov 29 03:06:18 user nova-compute[75234]:       <entry name="serial">8297c264-a213-44ee-be46-5c08eabda9f7</entry>
Nov 29 03:06:18 user nova-compute[75234]:       <entry name="uuid">8297c264-a213-44ee-be46-5c08eabda9f7</entry>
Nov 29 03:06:18 user nova-compute[75234]:       <entry name="family">Virtual Machine</entry>
Nov 29 03:06:18 user nova-compute[75234]:     </system>
Nov 29 03:06:18 user nova-compute[75234]:   </sysinfo>
Nov 29 03:06:18 user nova-compute[75234]:   <os>
Nov 29 03:06:18 user nova-compute[75234]:     <type arch="x86_64" machine="pc">hvm</type>
Nov 29 03:06:18 user nova-compute[75234]:     <boot dev="hd"/>
Nov 29 03:06:18 user nova-compute[75234]:     <smbios mode="sysinfo"/>
Nov 29 03:06:18 user nova-compute[75234]:   </os>
Nov 29 03:06:18 user nova-compute[75234]:   <features>
Nov 29 03:06:18 user nova-compute[75234]:     <acpi/>
Nov 29 03:06:18 user nova-compute[75234]:     <apic/>
Nov 29 03:06:18 user nova-compute[75234]:     <vmcoreinfo/>
Nov 29 03:06:18 user nova-compute[75234]:   </features>
Nov 29 03:06:18 user nova-compute[75234]:   <clock offset="utc">
Nov 29 03:06:18 user nova-compute[75234]:     <timer name="pit" tickpolicy="delay"/>
Nov 29 03:06:18 user nova-compute[75234]:     <timer name="rtc" tickpolicy="catchup"/>
Nov 29 03:06:18 user nova-compute[75234]:     <timer name="hpet" present="no"/>
Nov 29 03:06:18 user nova-compute[75234]:   </clock>
Nov 29 03:06:18 user nova-compute[75234]:   <cpu mode="custom" match="exact">
Nov 29 03:06:18 user nova-compute[75234]:     <model>Nehalem</model>
Nov 29 03:06:18 user nova-compute[75234]:     <topology sockets="1" cores="1" threads="1"/>
Nov 29 03:06:18 user nova-compute[75234]:   </cpu>
Nov 29 03:06:18 user nova-compute[75234]:   <devices>
Nov 29 03:06:18 user nova-compute[75234]:     <disk type="file" device="disk">
Nov 29 03:06:18 user nova-compute[75234]:       <driver name="qemu" type="qcow2" cache="none"/>
Nov 29 03:06:18 user nova-compute[75234]:       <source file="/opt/stack/data/nova/instances/8297c264-a213-44ee-be46-5c08eabda9f7/disk"/>
Nov 29 03:06:18 user nova-compute[75234]:       <target dev="vda" bus="virtio"/>
Nov 29 03:06:18 user nova-compute[75234]:     </disk>
Nov 29 03:06:18 user nova-compute[75234]:     <interface type="ethernet">
Nov 29 03:06:18 user nova-compute[75234]:       <mac address="fa:16:3e:5f:9d:5a"/>
Nov 29 03:06:18 user nova-compute[75234]:       <model type="virtio"/>
Nov 29 03:06:18 user nova-compute[75234]:       <mtu size="1442"/>
Nov 29 03:06:18 user nova-compute[75234]:       <target dev="tapd26334d2-80"/>
Nov 29 03:06:18 user nova-compute[75234]:     </interface>
Nov 29 03:06:18 user nova-compute[75234]:     <serial type="pty">
Nov 29 03:06:18 user nova-compute[75234]:       <log file="/opt/stack/data/nova/instances/8297c264-a213-44ee-be46-5c08eabda9f7/console.log" append="off"/>
Nov 29 03:06:18 user nova-compute[75234]:     </serial>
Nov 29 03:06:18 user nova-compute[75234]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Nov 29 03:06:18 user nova-compute[75234]:     <video>
Nov 29 03:06:18 user nova-compute[75234]:       <model type="virtio"/>
Nov 29 03:06:18 user nova-compute[75234]:     </video>
Nov 29 03:06:18 user nova-compute[75234]:     <rng model="virtio">
Nov 29 03:06:18 user nova-compute[75234]:       <backend model="random">/dev/urandom</backend>
Nov 29 03:06:18 user nova-compute[75234]:     </rng>
Nov 29 03:06:18 user nova-compute[75234]:     <controller type="usb" index="0" model="none"/>
Nov 29 03:06:18 user nova-compute[75234]:     <memballoon model="virtio">
Nov 29 03:06:18 user nova-compute[75234]:       <stats period="10"/>
Nov 29 03:06:18 user nova-compute[75234]:     </memballoon>
Nov 29 03:06:18 user nova-compute[75234]:   </devices>
Nov 29 03:06:18 user nova-compute[75234]: </domain>
Nov 29 03:06:18 user nova-compute[75234]:  {{(pid=75234) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7568}}
Nov 29 03:06:18 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-71e7f72f-cfc6-42f1-9655-a38fb0588077 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-11-29T03:06:07Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-ServerStableDeviceRescueTest-server-428917867',display_name='tempest-ServerStableDeviceRescueTest-server-428917867',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-serverstabledevicerescuetest-server-428917867',id=6,image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBPAGeJQ8OrWhDv/r5lktd6O/obaXRK7/7VEiU/V0MksOr2sSmn5M84mkyhNPvWnFo3k1J68udxxAQOPj9YvQEkVpNgH7z7grcEsUVqlgMVin9R9cWVCI3LAQJjKBdTQkeA==',key_name='tempest-keypair-1167119775',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='745bdb3c72794155829dc0a69f35b321',ramdisk_id='',reservation_id='r-tt096l76',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-ServerStableDeviceRescueTest-1509102897',owner_user_name='tempest-ServerStableDeviceRescueTest-1509102897-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-11-29T03:06:10Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='566e3dc84bbb40d9badcacbcaec9b987',uuid=8297c264-a213-44ee-be46-5c08eabda9f7,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "d26334d2-8040-4022-b658-69be1cf6a025", "address": "fa:16:3e:5f:9d:5a", "network": {"id": "e0366490-9fe2-4429-88c8-6625128e645d", "bridge": "br-int", "label": "tempest-ServerStableDeviceRescueTest-804259918-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "745bdb3c72794155829dc0a69f35b321", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapd26334d2-80", "ovs_interfaceid": "d26334d2-8040-4022-b658-69be1cf6a025", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Nov 29 03:06:18 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-71e7f72f-cfc6-42f1-9655-a38fb0588077 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] Converting VIF {"id": "d26334d2-8040-4022-b658-69be1cf6a025", "address": "fa:16:3e:5f:9d:5a", "network": {"id": "e0366490-9fe2-4429-88c8-6625128e645d", "bridge": "br-int", "label": "tempest-ServerStableDeviceRescueTest-804259918-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "745bdb3c72794155829dc0a69f35b321", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapd26334d2-80", "ovs_interfaceid": "d26334d2-8040-4022-b658-69be1cf6a025", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:06:18 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-71e7f72f-cfc6-42f1-9655-a38fb0588077 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:5f:9d:5a,bridge_name='br-int',has_traffic_filtering=True,id=d26334d2-8040-4022-b658-69be1cf6a025,network=Network(e0366490-9fe2-4429-88c8-6625128e645d),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapd26334d2-80') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:06:18 user nova-compute[75234]: DEBUG os_vif [None req-71e7f72f-cfc6-42f1-9655-a38fb0588077 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:5f:9d:5a,bridge_name='br-int',has_traffic_filtering=True,id=d26334d2-8040-4022-b658-69be1cf6a025,network=Network(e0366490-9fe2-4429-88c8-6625128e645d),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapd26334d2-80') {{(pid=75234) plug /opt/stack/data/venv/lib/python3.10/site-packages/os_vif/__init__.py:76}}
Nov 29 03:06:18 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:06:18 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:06:18 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Nov 29 03:06:18 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:06:18 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbCreateCommand(_result=None, table=QoS, columns={'type': 'linux-noop', 'external_ids': {'id': 'b4c23e6c-873e-507b-8be0-f09d343b78e1', '_type': 'linux-noop'}}, row=False) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:06:18 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:06:18 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:06:18 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:06:18 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tapd26334d2-80, may_exist=True, interface_attrs={}) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:06:18 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Port, record=tapd26334d2-80, col_values=(('qos', UUID('b05cf5f1-a00b-4a61-bb4b-b25b1545a2a2')),), if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:06:18 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DbSetCommand(_result=None, table=Interface, record=tapd26334d2-80, col_values=(('external_ids', {'iface-id': 'd26334d2-8040-4022-b658-69be1cf6a025', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:5f:9d:5a', 'vm-uuid': '8297c264-a213-44ee-be46-5c08eabda9f7'}),), if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:06:18 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:06:18 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:06:18 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:06:18 user nova-compute[75234]: INFO os_vif [None req-71e7f72f-cfc6-42f1-9655-a38fb0588077 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:5f:9d:5a,bridge_name='br-int',has_traffic_filtering=True,id=d26334d2-8040-4022-b658-69be1cf6a025,network=Network(e0366490-9fe2-4429-88c8-6625128e645d),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapd26334d2-80')
Nov 29 03:06:18 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-71e7f72f-cfc6-42f1-9655-a38fb0588077 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] No BDM found with device name vda, not building metadata. {{(pid=75234) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12089}}
Nov 29 03:06:18 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-71e7f72f-cfc6-42f1-9655-a38fb0588077 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] No VIF found with MAC fa:16:3e:5f:9d:5a, not building metadata {{(pid=75234) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Nov 29 03:06:18 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:06:19 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:06:19 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:06:19 user nova-compute[75234]: DEBUG nova.compute.manager [req-12fe6346-75b7-4b65-9add-1e347cebf619 req-df696be2-d500-4759-a179-1a1c8b7bba2d service nova] [instance: a9350a18-14e6-42c6-a150-0a1487f20ad9] Received event network-vif-plugged-52024efc-8cde-4610-b12c-198f19d75ccf {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:06:19 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-12fe6346-75b7-4b65-9add-1e347cebf619 req-df696be2-d500-4759-a179-1a1c8b7bba2d service nova] Acquiring lock "a9350a18-14e6-42c6-a150-0a1487f20ad9-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:06:19 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-12fe6346-75b7-4b65-9add-1e347cebf619 req-df696be2-d500-4759-a179-1a1c8b7bba2d service nova] Lock "a9350a18-14e6-42c6-a150-0a1487f20ad9-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:06:19 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-12fe6346-75b7-4b65-9add-1e347cebf619 req-df696be2-d500-4759-a179-1a1c8b7bba2d service nova] Lock "a9350a18-14e6-42c6-a150-0a1487f20ad9-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:06:19 user nova-compute[75234]: DEBUG nova.compute.manager [req-12fe6346-75b7-4b65-9add-1e347cebf619 req-df696be2-d500-4759-a179-1a1c8b7bba2d service nova] [instance: a9350a18-14e6-42c6-a150-0a1487f20ad9] No waiting events found dispatching network-vif-plugged-52024efc-8cde-4610-b12c-198f19d75ccf {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:06:19 user nova-compute[75234]: WARNING nova.compute.manager [req-12fe6346-75b7-4b65-9add-1e347cebf619 req-df696be2-d500-4759-a179-1a1c8b7bba2d service nova] [instance: a9350a18-14e6-42c6-a150-0a1487f20ad9] Received unexpected event network-vif-plugged-52024efc-8cde-4610-b12c-198f19d75ccf for instance with vm_state building and task_state spawning.
Nov 29 03:06:19 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:06:19 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:06:20 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:06:20 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:06:20 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:06:20 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:06:20 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:06:20 user nova-compute[75234]: DEBUG nova.compute.manager [None req-fcb1dbf7-0559-48fa-8e1a-0e39441efbec tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] [instance: 21e9c6cd-c39f-41c0-8768-5dd64589c3f3] Instance event wait completed in 0 seconds for  {{(pid=75234) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Nov 29 03:06:20 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-fcb1dbf7-0559-48fa-8e1a-0e39441efbec tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] [instance: 21e9c6cd-c39f-41c0-8768-5dd64589c3f3] Guest created on hypervisor {{(pid=75234) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4421}}
Nov 29 03:06:20 user nova-compute[75234]: INFO nova.virt.libvirt.driver [-] [instance: 21e9c6cd-c39f-41c0-8768-5dd64589c3f3] Instance spawned successfully.
Nov 29 03:06:20 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-fcb1dbf7-0559-48fa-8e1a-0e39441efbec tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] [instance: 21e9c6cd-c39f-41c0-8768-5dd64589c3f3] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:904}}
Nov 29 03:06:20 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-fcb1dbf7-0559-48fa-8e1a-0e39441efbec tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] [instance: 21e9c6cd-c39f-41c0-8768-5dd64589c3f3] Found default for hw_cdrom_bus of ide {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:06:20 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-fcb1dbf7-0559-48fa-8e1a-0e39441efbec tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] [instance: 21e9c6cd-c39f-41c0-8768-5dd64589c3f3] Found default for hw_disk_bus of virtio {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:06:20 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-fcb1dbf7-0559-48fa-8e1a-0e39441efbec tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] [instance: 21e9c6cd-c39f-41c0-8768-5dd64589c3f3] Found default for hw_input_bus of None {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:06:20 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-fcb1dbf7-0559-48fa-8e1a-0e39441efbec tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] [instance: 21e9c6cd-c39f-41c0-8768-5dd64589c3f3] Found default for hw_pointer_model of None {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:06:20 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-fcb1dbf7-0559-48fa-8e1a-0e39441efbec tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] [instance: 21e9c6cd-c39f-41c0-8768-5dd64589c3f3] Found default for hw_video_model of virtio {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:06:20 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-fcb1dbf7-0559-48fa-8e1a-0e39441efbec tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] [instance: 21e9c6cd-c39f-41c0-8768-5dd64589c3f3] Found default for hw_vif_model of virtio {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:06:20 user nova-compute[75234]: INFO nova.compute.manager [None req-fcb1dbf7-0559-48fa-8e1a-0e39441efbec tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] [instance: 21e9c6cd-c39f-41c0-8768-5dd64589c3f3] Took 13.31 seconds to spawn the instance on the hypervisor.
Nov 29 03:06:20 user nova-compute[75234]: DEBUG nova.compute.manager [None req-fcb1dbf7-0559-48fa-8e1a-0e39441efbec tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] [instance: 21e9c6cd-c39f-41c0-8768-5dd64589c3f3] Checking state {{(pid=75234) _get_power_state /opt/stack/nova/nova/compute/manager.py:1782}}
Nov 29 03:06:20 user nova-compute[75234]: INFO nova.compute.manager [None req-fcb1dbf7-0559-48fa-8e1a-0e39441efbec tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] [instance: 21e9c6cd-c39f-41c0-8768-5dd64589c3f3] Took 14.43 seconds to build instance.
Nov 29 03:06:20 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-fcb1dbf7-0559-48fa-8e1a-0e39441efbec tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Lock "21e9c6cd-c39f-41c0-8768-5dd64589c3f3" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 14.779s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:06:20 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:06:20 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:06:20 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:06:21 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:06:21 user nova-compute[75234]: DEBUG nova.compute.manager [req-340921cf-9a29-4aa9-bf65-92dec58e5fc5 req-97763a9d-67e0-4dd1-b59e-897f2b72551d service nova] [instance: a9350a18-14e6-42c6-a150-0a1487f20ad9] Received event network-vif-plugged-52024efc-8cde-4610-b12c-198f19d75ccf {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:06:21 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-340921cf-9a29-4aa9-bf65-92dec58e5fc5 req-97763a9d-67e0-4dd1-b59e-897f2b72551d service nova] Acquiring lock "a9350a18-14e6-42c6-a150-0a1487f20ad9-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:06:21 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-340921cf-9a29-4aa9-bf65-92dec58e5fc5 req-97763a9d-67e0-4dd1-b59e-897f2b72551d service nova] Lock "a9350a18-14e6-42c6-a150-0a1487f20ad9-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:06:21 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-340921cf-9a29-4aa9-bf65-92dec58e5fc5 req-97763a9d-67e0-4dd1-b59e-897f2b72551d service nova] Lock "a9350a18-14e6-42c6-a150-0a1487f20ad9-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:06:21 user nova-compute[75234]: DEBUG nova.compute.manager [req-340921cf-9a29-4aa9-bf65-92dec58e5fc5 req-97763a9d-67e0-4dd1-b59e-897f2b72551d service nova] [instance: a9350a18-14e6-42c6-a150-0a1487f20ad9] No waiting events found dispatching network-vif-plugged-52024efc-8cde-4610-b12c-198f19d75ccf {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:06:21 user nova-compute[75234]: WARNING nova.compute.manager [req-340921cf-9a29-4aa9-bf65-92dec58e5fc5 req-97763a9d-67e0-4dd1-b59e-897f2b72551d service nova] [instance: a9350a18-14e6-42c6-a150-0a1487f20ad9] Received unexpected event network-vif-plugged-52024efc-8cde-4610-b12c-198f19d75ccf for instance with vm_state building and task_state spawning.
Nov 29 03:06:21 user nova-compute[75234]: DEBUG nova.compute.manager [req-340921cf-9a29-4aa9-bf65-92dec58e5fc5 req-97763a9d-67e0-4dd1-b59e-897f2b72551d service nova] [instance: f609a7b7-1be2-44b4-9177-7e72a4d4c060] Received event network-vif-plugged-5543f379-2b3a-4dc2-bfe9-52f66a942257 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:06:21 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-340921cf-9a29-4aa9-bf65-92dec58e5fc5 req-97763a9d-67e0-4dd1-b59e-897f2b72551d service nova] Acquiring lock "f609a7b7-1be2-44b4-9177-7e72a4d4c060-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:06:21 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-340921cf-9a29-4aa9-bf65-92dec58e5fc5 req-97763a9d-67e0-4dd1-b59e-897f2b72551d service nova] Lock "f609a7b7-1be2-44b4-9177-7e72a4d4c060-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:06:21 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-340921cf-9a29-4aa9-bf65-92dec58e5fc5 req-97763a9d-67e0-4dd1-b59e-897f2b72551d service nova] Lock "f609a7b7-1be2-44b4-9177-7e72a4d4c060-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:06:21 user nova-compute[75234]: DEBUG nova.compute.manager [req-340921cf-9a29-4aa9-bf65-92dec58e5fc5 req-97763a9d-67e0-4dd1-b59e-897f2b72551d service nova] [instance: f609a7b7-1be2-44b4-9177-7e72a4d4c060] No waiting events found dispatching network-vif-plugged-5543f379-2b3a-4dc2-bfe9-52f66a942257 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:06:21 user nova-compute[75234]: WARNING nova.compute.manager [req-340921cf-9a29-4aa9-bf65-92dec58e5fc5 req-97763a9d-67e0-4dd1-b59e-897f2b72551d service nova] [instance: f609a7b7-1be2-44b4-9177-7e72a4d4c060] Received unexpected event network-vif-plugged-5543f379-2b3a-4dc2-bfe9-52f66a942257 for instance with vm_state building and task_state spawning.
Nov 29 03:06:21 user nova-compute[75234]: DEBUG nova.compute.manager [req-340921cf-9a29-4aa9-bf65-92dec58e5fc5 req-97763a9d-67e0-4dd1-b59e-897f2b72551d service nova] [instance: f609a7b7-1be2-44b4-9177-7e72a4d4c060] Received event network-vif-plugged-5543f379-2b3a-4dc2-bfe9-52f66a942257 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:06:21 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-340921cf-9a29-4aa9-bf65-92dec58e5fc5 req-97763a9d-67e0-4dd1-b59e-897f2b72551d service nova] Acquiring lock "f609a7b7-1be2-44b4-9177-7e72a4d4c060-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:06:21 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-340921cf-9a29-4aa9-bf65-92dec58e5fc5 req-97763a9d-67e0-4dd1-b59e-897f2b72551d service nova] Lock "f609a7b7-1be2-44b4-9177-7e72a4d4c060-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:06:21 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-340921cf-9a29-4aa9-bf65-92dec58e5fc5 req-97763a9d-67e0-4dd1-b59e-897f2b72551d service nova] Lock "f609a7b7-1be2-44b4-9177-7e72a4d4c060-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:06:21 user nova-compute[75234]: DEBUG nova.compute.manager [req-340921cf-9a29-4aa9-bf65-92dec58e5fc5 req-97763a9d-67e0-4dd1-b59e-897f2b72551d service nova] [instance: f609a7b7-1be2-44b4-9177-7e72a4d4c060] No waiting events found dispatching network-vif-plugged-5543f379-2b3a-4dc2-bfe9-52f66a942257 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:06:21 user nova-compute[75234]: WARNING nova.compute.manager [req-340921cf-9a29-4aa9-bf65-92dec58e5fc5 req-97763a9d-67e0-4dd1-b59e-897f2b72551d service nova] [instance: f609a7b7-1be2-44b4-9177-7e72a4d4c060] Received unexpected event network-vif-plugged-5543f379-2b3a-4dc2-bfe9-52f66a942257 for instance with vm_state building and task_state spawning.
Nov 29 03:06:21 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:06:21 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:06:21 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:06:22 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:06:23 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ceb7826f-126f-40b1-b7f1-9afa48110fc2 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] [instance: f609a7b7-1be2-44b4-9177-7e72a4d4c060] Instance event wait completed in 0 seconds for  {{(pid=75234) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Nov 29 03:06:23 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-ceb7826f-126f-40b1-b7f1-9afa48110fc2 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] [instance: f609a7b7-1be2-44b4-9177-7e72a4d4c060] Guest created on hypervisor {{(pid=75234) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4421}}
Nov 29 03:06:23 user nova-compute[75234]: INFO nova.virt.libvirt.driver [-] [instance: f609a7b7-1be2-44b4-9177-7e72a4d4c060] Instance spawned successfully.
Nov 29 03:06:23 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-ceb7826f-126f-40b1-b7f1-9afa48110fc2 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] [instance: f609a7b7-1be2-44b4-9177-7e72a4d4c060] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:904}}
Nov 29 03:06:23 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-ceb7826f-126f-40b1-b7f1-9afa48110fc2 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] [instance: f609a7b7-1be2-44b4-9177-7e72a4d4c060] Found default for hw_cdrom_bus of ide {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:06:23 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-ceb7826f-126f-40b1-b7f1-9afa48110fc2 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] [instance: f609a7b7-1be2-44b4-9177-7e72a4d4c060] Found default for hw_disk_bus of virtio {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:06:23 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-ceb7826f-126f-40b1-b7f1-9afa48110fc2 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] [instance: f609a7b7-1be2-44b4-9177-7e72a4d4c060] Found default for hw_input_bus of None {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:06:23 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-ceb7826f-126f-40b1-b7f1-9afa48110fc2 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] [instance: f609a7b7-1be2-44b4-9177-7e72a4d4c060] Found default for hw_pointer_model of None {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:06:23 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-ceb7826f-126f-40b1-b7f1-9afa48110fc2 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] [instance: f609a7b7-1be2-44b4-9177-7e72a4d4c060] Found default for hw_video_model of virtio {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:06:23 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-ceb7826f-126f-40b1-b7f1-9afa48110fc2 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] [instance: f609a7b7-1be2-44b4-9177-7e72a4d4c060] Found default for hw_vif_model of virtio {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:06:23 user nova-compute[75234]: DEBUG nova.compute.manager [None req-c696a156-5f79-4939-99e6-fd7a518710c4 tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] [instance: a9350a18-14e6-42c6-a150-0a1487f20ad9] Instance event wait completed in 0 seconds for  {{(pid=75234) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Nov 29 03:06:23 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-c696a156-5f79-4939-99e6-fd7a518710c4 tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] [instance: a9350a18-14e6-42c6-a150-0a1487f20ad9] Guest created on hypervisor {{(pid=75234) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4421}}
Nov 29 03:06:23 user nova-compute[75234]: DEBUG nova.compute.manager [req-64701125-f1c3-48b2-acf9-38501c509417 req-bbb68404-7f60-4719-9bda-276cff6476a5 service nova] [instance: 8297c264-a213-44ee-be46-5c08eabda9f7] Received event network-vif-plugged-d26334d2-8040-4022-b658-69be1cf6a025 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:06:23 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-64701125-f1c3-48b2-acf9-38501c509417 req-bbb68404-7f60-4719-9bda-276cff6476a5 service nova] Acquiring lock "8297c264-a213-44ee-be46-5c08eabda9f7-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:06:23 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-64701125-f1c3-48b2-acf9-38501c509417 req-bbb68404-7f60-4719-9bda-276cff6476a5 service nova] Lock "8297c264-a213-44ee-be46-5c08eabda9f7-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:06:23 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-64701125-f1c3-48b2-acf9-38501c509417 req-bbb68404-7f60-4719-9bda-276cff6476a5 service nova] Lock "8297c264-a213-44ee-be46-5c08eabda9f7-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:06:23 user nova-compute[75234]: DEBUG nova.compute.manager [req-64701125-f1c3-48b2-acf9-38501c509417 req-bbb68404-7f60-4719-9bda-276cff6476a5 service nova] [instance: 8297c264-a213-44ee-be46-5c08eabda9f7] No waiting events found dispatching network-vif-plugged-d26334d2-8040-4022-b658-69be1cf6a025 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:06:23 user nova-compute[75234]: WARNING nova.compute.manager [req-64701125-f1c3-48b2-acf9-38501c509417 req-bbb68404-7f60-4719-9bda-276cff6476a5 service nova] [instance: 8297c264-a213-44ee-be46-5c08eabda9f7] Received unexpected event network-vif-plugged-d26334d2-8040-4022-b658-69be1cf6a025 for instance with vm_state building and task_state spawning.
Nov 29 03:06:23 user nova-compute[75234]: DEBUG nova.compute.manager [req-64701125-f1c3-48b2-acf9-38501c509417 req-bbb68404-7f60-4719-9bda-276cff6476a5 service nova] [instance: 8297c264-a213-44ee-be46-5c08eabda9f7] Received event network-vif-plugged-d26334d2-8040-4022-b658-69be1cf6a025 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:06:23 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-64701125-f1c3-48b2-acf9-38501c509417 req-bbb68404-7f60-4719-9bda-276cff6476a5 service nova] Acquiring lock "8297c264-a213-44ee-be46-5c08eabda9f7-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:06:23 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-64701125-f1c3-48b2-acf9-38501c509417 req-bbb68404-7f60-4719-9bda-276cff6476a5 service nova] Lock "8297c264-a213-44ee-be46-5c08eabda9f7-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:06:23 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-64701125-f1c3-48b2-acf9-38501c509417 req-bbb68404-7f60-4719-9bda-276cff6476a5 service nova] Lock "8297c264-a213-44ee-be46-5c08eabda9f7-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:06:23 user nova-compute[75234]: DEBUG nova.compute.manager [req-64701125-f1c3-48b2-acf9-38501c509417 req-bbb68404-7f60-4719-9bda-276cff6476a5 service nova] [instance: 8297c264-a213-44ee-be46-5c08eabda9f7] No waiting events found dispatching network-vif-plugged-d26334d2-8040-4022-b658-69be1cf6a025 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:06:23 user nova-compute[75234]: WARNING nova.compute.manager [req-64701125-f1c3-48b2-acf9-38501c509417 req-bbb68404-7f60-4719-9bda-276cff6476a5 service nova] [instance: 8297c264-a213-44ee-be46-5c08eabda9f7] Received unexpected event network-vif-plugged-d26334d2-8040-4022-b658-69be1cf6a025 for instance with vm_state building and task_state spawning.
Nov 29 03:06:23 user nova-compute[75234]: INFO nova.virt.libvirt.driver [-] [instance: a9350a18-14e6-42c6-a150-0a1487f20ad9] Instance spawned successfully.
Nov 29 03:06:23 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-c696a156-5f79-4939-99e6-fd7a518710c4 tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] [instance: a9350a18-14e6-42c6-a150-0a1487f20ad9] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:904}}
Nov 29 03:06:23 user nova-compute[75234]: INFO nova.compute.manager [None req-ceb7826f-126f-40b1-b7f1-9afa48110fc2 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] [instance: f609a7b7-1be2-44b4-9177-7e72a4d4c060] Took 14.98 seconds to spawn the instance on the hypervisor.
Nov 29 03:06:23 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ceb7826f-126f-40b1-b7f1-9afa48110fc2 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] [instance: f609a7b7-1be2-44b4-9177-7e72a4d4c060] Checking state {{(pid=75234) _get_power_state /opt/stack/nova/nova/compute/manager.py:1782}}
Nov 29 03:06:23 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-c696a156-5f79-4939-99e6-fd7a518710c4 tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] [instance: a9350a18-14e6-42c6-a150-0a1487f20ad9] Found default for hw_cdrom_bus of ide {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:06:23 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-c696a156-5f79-4939-99e6-fd7a518710c4 tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] [instance: a9350a18-14e6-42c6-a150-0a1487f20ad9] Found default for hw_disk_bus of virtio {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:06:23 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-c696a156-5f79-4939-99e6-fd7a518710c4 tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] [instance: a9350a18-14e6-42c6-a150-0a1487f20ad9] Found default for hw_input_bus of None {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:06:23 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-c696a156-5f79-4939-99e6-fd7a518710c4 tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] [instance: a9350a18-14e6-42c6-a150-0a1487f20ad9] Found default for hw_pointer_model of None {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:06:23 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-c696a156-5f79-4939-99e6-fd7a518710c4 tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] [instance: a9350a18-14e6-42c6-a150-0a1487f20ad9] Found default for hw_video_model of virtio {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:06:23 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-c696a156-5f79-4939-99e6-fd7a518710c4 tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] [instance: a9350a18-14e6-42c6-a150-0a1487f20ad9] Found default for hw_vif_model of virtio {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:06:23 user nova-compute[75234]: INFO nova.compute.manager [None req-ceb7826f-126f-40b1-b7f1-9afa48110fc2 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] [instance: f609a7b7-1be2-44b4-9177-7e72a4d4c060] Took 16.23 seconds to build instance.
Nov 29 03:06:23 user nova-compute[75234]: INFO nova.compute.manager [None req-c696a156-5f79-4939-99e6-fd7a518710c4 tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] [instance: a9350a18-14e6-42c6-a150-0a1487f20ad9] Took 14.53 seconds to spawn the instance on the hypervisor.
Nov 29 03:06:23 user nova-compute[75234]: DEBUG nova.compute.manager [None req-c696a156-5f79-4939-99e6-fd7a518710c4 tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] [instance: a9350a18-14e6-42c6-a150-0a1487f20ad9] Checking state {{(pid=75234) _get_power_state /opt/stack/nova/nova/compute/manager.py:1782}}
Nov 29 03:06:23 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:06:23 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ceb7826f-126f-40b1-b7f1-9afa48110fc2 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Lock "f609a7b7-1be2-44b4-9177-7e72a4d4c060" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 16.488s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:06:23 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:06:23 user nova-compute[75234]: DEBUG nova.compute.manager [None req-71e7f72f-cfc6-42f1-9655-a38fb0588077 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] [instance: 8297c264-a213-44ee-be46-5c08eabda9f7] Instance event wait completed in 0 seconds for  {{(pid=75234) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Nov 29 03:06:23 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-71e7f72f-cfc6-42f1-9655-a38fb0588077 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] [instance: 8297c264-a213-44ee-be46-5c08eabda9f7] Guest created on hypervisor {{(pid=75234) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4421}}
Nov 29 03:06:23 user nova-compute[75234]: INFO nova.virt.libvirt.driver [-] [instance: 8297c264-a213-44ee-be46-5c08eabda9f7] Instance spawned successfully.
Nov 29 03:06:23 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-71e7f72f-cfc6-42f1-9655-a38fb0588077 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] [instance: 8297c264-a213-44ee-be46-5c08eabda9f7] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:904}}
Nov 29 03:06:23 user nova-compute[75234]: INFO nova.compute.manager [None req-c696a156-5f79-4939-99e6-fd7a518710c4 tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] [instance: a9350a18-14e6-42c6-a150-0a1487f20ad9] Took 16.40 seconds to build instance.
Nov 29 03:06:23 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-71e7f72f-cfc6-42f1-9655-a38fb0588077 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] [instance: 8297c264-a213-44ee-be46-5c08eabda9f7] Found default for hw_cdrom_bus of ide {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:06:23 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-71e7f72f-cfc6-42f1-9655-a38fb0588077 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] [instance: 8297c264-a213-44ee-be46-5c08eabda9f7] Found default for hw_disk_bus of virtio {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:06:23 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-71e7f72f-cfc6-42f1-9655-a38fb0588077 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] [instance: 8297c264-a213-44ee-be46-5c08eabda9f7] Found default for hw_input_bus of None {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:06:23 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-71e7f72f-cfc6-42f1-9655-a38fb0588077 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] [instance: 8297c264-a213-44ee-be46-5c08eabda9f7] Found default for hw_pointer_model of None {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:06:23 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-71e7f72f-cfc6-42f1-9655-a38fb0588077 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] [instance: 8297c264-a213-44ee-be46-5c08eabda9f7] Found default for hw_video_model of virtio {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:06:23 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-71e7f72f-cfc6-42f1-9655-a38fb0588077 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] [instance: 8297c264-a213-44ee-be46-5c08eabda9f7] Found default for hw_vif_model of virtio {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:06:23 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-c696a156-5f79-4939-99e6-fd7a518710c4 tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Lock "a9350a18-14e6-42c6-a150-0a1487f20ad9" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 16.541s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:06:23 user nova-compute[75234]: INFO nova.compute.manager [None req-71e7f72f-cfc6-42f1-9655-a38fb0588077 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] [instance: 8297c264-a213-44ee-be46-5c08eabda9f7] Took 13.56 seconds to spawn the instance on the hypervisor.
Nov 29 03:06:23 user nova-compute[75234]: DEBUG nova.compute.manager [None req-71e7f72f-cfc6-42f1-9655-a38fb0588077 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] [instance: 8297c264-a213-44ee-be46-5c08eabda9f7] Checking state {{(pid=75234) _get_power_state /opt/stack/nova/nova/compute/manager.py:1782}}
Nov 29 03:06:24 user nova-compute[75234]: INFO nova.compute.manager [None req-71e7f72f-cfc6-42f1-9655-a38fb0588077 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] [instance: 8297c264-a213-44ee-be46-5c08eabda9f7] Took 15.04 seconds to build instance.
Nov 29 03:06:24 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-71e7f72f-cfc6-42f1-9655-a38fb0588077 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] Lock "8297c264-a213-44ee-be46-5c08eabda9f7" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 15.333s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:06:26 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:06:26 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:06:27 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-a745e7d4-15ea-45ba-a537-b658a23b24ed tempest-AttachSCSIVolumeTestJSON-719577128 tempest-AttachSCSIVolumeTestJSON-719577128-project-member] Acquiring lock "da625f9c-b7ed-4afc-875c-6436bf58e9cc" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:06:27 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-a745e7d4-15ea-45ba-a537-b658a23b24ed tempest-AttachSCSIVolumeTestJSON-719577128 tempest-AttachSCSIVolumeTestJSON-719577128-project-member] Lock "da625f9c-b7ed-4afc-875c-6436bf58e9cc" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:06:28 user nova-compute[75234]: DEBUG nova.compute.manager [None req-a745e7d4-15ea-45ba-a537-b658a23b24ed tempest-AttachSCSIVolumeTestJSON-719577128 tempest-AttachSCSIVolumeTestJSON-719577128-project-member] [instance: da625f9c-b7ed-4afc-875c-6436bf58e9cc] Starting instance... {{(pid=75234) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2422}}
Nov 29 03:06:28 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-a745e7d4-15ea-45ba-a537-b658a23b24ed tempest-AttachSCSIVolumeTestJSON-719577128 tempest-AttachSCSIVolumeTestJSON-719577128-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:06:28 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-a745e7d4-15ea-45ba-a537-b658a23b24ed tempest-AttachSCSIVolumeTestJSON-719577128 tempest-AttachSCSIVolumeTestJSON-719577128-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:06:28 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-a745e7d4-15ea-45ba-a537-b658a23b24ed tempest-AttachSCSIVolumeTestJSON-719577128 tempest-AttachSCSIVolumeTestJSON-719577128-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=75234) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2366}}
Nov 29 03:06:28 user nova-compute[75234]: INFO nova.compute.claims [None req-a745e7d4-15ea-45ba-a537-b658a23b24ed tempest-AttachSCSIVolumeTestJSON-719577128 tempest-AttachSCSIVolumeTestJSON-719577128-project-member] [instance: da625f9c-b7ed-4afc-875c-6436bf58e9cc] Claim successful on node user
Nov 29 03:06:28 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:06:28 user nova-compute[75234]: DEBUG nova.compute.provider_tree [None req-a745e7d4-15ea-45ba-a537-b658a23b24ed tempest-AttachSCSIVolumeTestJSON-719577128 tempest-AttachSCSIVolumeTestJSON-719577128-project-member] Inventory has not changed in ProviderTree for provider: 4d291220-7c37-4839-a0e3-b23c66502b74 {{(pid=75234) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Nov 29 03:06:28 user nova-compute[75234]: DEBUG nova.scheduler.client.report [None req-a745e7d4-15ea-45ba-a537-b658a23b24ed tempest-AttachSCSIVolumeTestJSON-719577128 tempest-AttachSCSIVolumeTestJSON-719577128-project-member] Inventory has not changed for provider 4d291220-7c37-4839-a0e3-b23c66502b74 based on inventory data: {'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=75234) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Nov 29 03:06:28 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-a745e7d4-15ea-45ba-a537-b658a23b24ed tempest-AttachSCSIVolumeTestJSON-719577128 tempest-AttachSCSIVolumeTestJSON-719577128-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.580s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:06:28 user nova-compute[75234]: DEBUG nova.compute.manager [None req-a745e7d4-15ea-45ba-a537-b658a23b24ed tempest-AttachSCSIVolumeTestJSON-719577128 tempest-AttachSCSIVolumeTestJSON-719577128-project-member] [instance: da625f9c-b7ed-4afc-875c-6436bf58e9cc] Start building networks asynchronously for instance. {{(pid=75234) _build_resources /opt/stack/nova/nova/compute/manager.py:2819}}
Nov 29 03:06:28 user nova-compute[75234]: DEBUG nova.compute.manager [None req-a745e7d4-15ea-45ba-a537-b658a23b24ed tempest-AttachSCSIVolumeTestJSON-719577128 tempest-AttachSCSIVolumeTestJSON-719577128-project-member] [instance: da625f9c-b7ed-4afc-875c-6436bf58e9cc] Allocating IP information in the background. {{(pid=75234) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1972}}
Nov 29 03:06:28 user nova-compute[75234]: DEBUG nova.network.neutron [None req-a745e7d4-15ea-45ba-a537-b658a23b24ed tempest-AttachSCSIVolumeTestJSON-719577128 tempest-AttachSCSIVolumeTestJSON-719577128-project-member] [instance: da625f9c-b7ed-4afc-875c-6436bf58e9cc] allocate_for_instance() {{(pid=75234) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1156}}
Nov 29 03:06:29 user nova-compute[75234]: DEBUG nova.policy [None req-a745e7d4-15ea-45ba-a537-b658a23b24ed tempest-AttachSCSIVolumeTestJSON-719577128 tempest-AttachSCSIVolumeTestJSON-719577128-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': 'ef10980c66b44933bc463320af40dfca', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'df2e94062c154555ab1ee0c3bf8e0f84', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=75234) authorize /opt/stack/nova/nova/policy.py:203}}
Nov 29 03:06:29 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-a745e7d4-15ea-45ba-a537-b658a23b24ed tempest-AttachSCSIVolumeTestJSON-719577128 tempest-AttachSCSIVolumeTestJSON-719577128-project-member] [instance: da625f9c-b7ed-4afc-875c-6436bf58e9cc] Ignoring supplied device name: /dev/sda. Libvirt can't honour user-supplied dev names
Nov 29 03:06:29 user nova-compute[75234]: DEBUG nova.compute.manager [None req-a745e7d4-15ea-45ba-a537-b658a23b24ed tempest-AttachSCSIVolumeTestJSON-719577128 tempest-AttachSCSIVolumeTestJSON-719577128-project-member] [instance: da625f9c-b7ed-4afc-875c-6436bf58e9cc] Start building block device mappings for instance. {{(pid=75234) _build_resources /opt/stack/nova/nova/compute/manager.py:2854}}
Nov 29 03:06:29 user nova-compute[75234]: DEBUG nova.compute.manager [None req-a745e7d4-15ea-45ba-a537-b658a23b24ed tempest-AttachSCSIVolumeTestJSON-719577128 tempest-AttachSCSIVolumeTestJSON-719577128-project-member] [instance: da625f9c-b7ed-4afc-875c-6436bf58e9cc] Start spawning the instance on the hypervisor. {{(pid=75234) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2628}}
Nov 29 03:06:29 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-a745e7d4-15ea-45ba-a537-b658a23b24ed tempest-AttachSCSIVolumeTestJSON-719577128 tempest-AttachSCSIVolumeTestJSON-719577128-project-member] [instance: da625f9c-b7ed-4afc-875c-6436bf58e9cc] Creating instance directory {{(pid=75234) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4727}}
Nov 29 03:06:29 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-a745e7d4-15ea-45ba-a537-b658a23b24ed tempest-AttachSCSIVolumeTestJSON-719577128 tempest-AttachSCSIVolumeTestJSON-719577128-project-member] [instance: da625f9c-b7ed-4afc-875c-6436bf58e9cc] Creating image(s)
Nov 29 03:06:29 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-a745e7d4-15ea-45ba-a537-b658a23b24ed tempest-AttachSCSIVolumeTestJSON-719577128 tempest-AttachSCSIVolumeTestJSON-719577128-project-member] Acquiring lock "/opt/stack/data/nova/instances/da625f9c-b7ed-4afc-875c-6436bf58e9cc/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:06:29 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-a745e7d4-15ea-45ba-a537-b658a23b24ed tempest-AttachSCSIVolumeTestJSON-719577128 tempest-AttachSCSIVolumeTestJSON-719577128-project-member] Lock "/opt/stack/data/nova/instances/da625f9c-b7ed-4afc-875c-6436bf58e9cc/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:06:29 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-a745e7d4-15ea-45ba-a537-b658a23b24ed tempest-AttachSCSIVolumeTestJSON-719577128 tempest-AttachSCSIVolumeTestJSON-719577128-project-member] Lock "/opt/stack/data/nova/instances/da625f9c-b7ed-4afc-875c-6436bf58e9cc/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.002s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:06:29 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-a745e7d4-15ea-45ba-a537-b658a23b24ed tempest-AttachSCSIVolumeTestJSON-719577128 tempest-AttachSCSIVolumeTestJSON-719577128-project-member] Acquiring lock "7f7598d3f4a27ebdcdcae511dd4e04ef3c67d957" by "nova.virt.libvirt.imagebackend.Image.cache.<locals>.fetch_func_sync" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:06:29 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-a745e7d4-15ea-45ba-a537-b658a23b24ed tempest-AttachSCSIVolumeTestJSON-719577128 tempest-AttachSCSIVolumeTestJSON-719577128-project-member] Lock "7f7598d3f4a27ebdcdcae511dd4e04ef3c67d957" acquired by "nova.virt.libvirt.imagebackend.Image.cache.<locals>.fetch_func_sync" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:06:29 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-a745e7d4-15ea-45ba-a537-b658a23b24ed tempest-AttachSCSIVolumeTestJSON-719577128 tempest-AttachSCSIVolumeTestJSON-719577128-project-member] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/7f7598d3f4a27ebdcdcae511dd4e04ef3c67d957.part --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:06:29 user nova-compute[75234]: DEBUG nova.network.neutron [None req-a745e7d4-15ea-45ba-a537-b658a23b24ed tempest-AttachSCSIVolumeTestJSON-719577128 tempest-AttachSCSIVolumeTestJSON-719577128-project-member] [instance: da625f9c-b7ed-4afc-875c-6436bf58e9cc] Successfully created port: a23b8769-d871-4d44-8cea-5d899f514579 {{(pid=75234) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:548}}
Nov 29 03:06:30 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-684945de-57fe-475b-b990-ae7897d5ffb4 tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] Acquiring lock "aa3c7432-bf15-408e-9fbb-993135734161" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:06:30 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-684945de-57fe-475b-b990-ae7897d5ffb4 tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] Lock "aa3c7432-bf15-408e-9fbb-993135734161" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:06:30 user nova-compute[75234]: DEBUG nova.compute.manager [None req-684945de-57fe-475b-b990-ae7897d5ffb4 tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] [instance: aa3c7432-bf15-408e-9fbb-993135734161] Starting instance... {{(pid=75234) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2422}}
Nov 29 03:06:30 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-a745e7d4-15ea-45ba-a537-b658a23b24ed tempest-AttachSCSIVolumeTestJSON-719577128 tempest-AttachSCSIVolumeTestJSON-719577128-project-member] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/7f7598d3f4a27ebdcdcae511dd4e04ef3c67d957.part --force-share --output=json" returned: 0 in 0.273s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:06:30 user nova-compute[75234]: DEBUG nova.virt.images [None req-a745e7d4-15ea-45ba-a537-b658a23b24ed tempest-AttachSCSIVolumeTestJSON-719577128 tempest-AttachSCSIVolumeTestJSON-719577128-project-member] b25373f9-ce4e-4a59-b733-b4caaf9c34c6 was qcow2, converting to raw {{(pid=75234) fetch_to_raw /opt/stack/nova/nova/virt/images.py:165}}
Nov 29 03:06:30 user nova-compute[75234]: DEBUG nova.privsep.utils [None req-a745e7d4-15ea-45ba-a537-b658a23b24ed tempest-AttachSCSIVolumeTestJSON-719577128 tempest-AttachSCSIVolumeTestJSON-719577128-project-member] Path '/opt/stack/data/nova/instances' supports direct I/O {{(pid=75234) supports_direct_io /opt/stack/nova/nova/privsep/utils.py:63}}
Nov 29 03:06:30 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-a745e7d4-15ea-45ba-a537-b658a23b24ed tempest-AttachSCSIVolumeTestJSON-719577128 tempest-AttachSCSIVolumeTestJSON-719577128-project-member] Running cmd (subprocess): qemu-img convert -t none -O raw -f qcow2 /opt/stack/data/nova/instances/_base/7f7598d3f4a27ebdcdcae511dd4e04ef3c67d957.part /opt/stack/data/nova/instances/_base/7f7598d3f4a27ebdcdcae511dd4e04ef3c67d957.converted {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:06:30 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-684945de-57fe-475b-b990-ae7897d5ffb4 tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:06:30 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-684945de-57fe-475b-b990-ae7897d5ffb4 tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.002s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:06:30 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-684945de-57fe-475b-b990-ae7897d5ffb4 tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=75234) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2366}}
Nov 29 03:06:30 user nova-compute[75234]: INFO nova.compute.claims [None req-684945de-57fe-475b-b990-ae7897d5ffb4 tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] [instance: aa3c7432-bf15-408e-9fbb-993135734161] Claim successful on node user
Nov 29 03:06:30 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-a745e7d4-15ea-45ba-a537-b658a23b24ed tempest-AttachSCSIVolumeTestJSON-719577128 tempest-AttachSCSIVolumeTestJSON-719577128-project-member] CMD "qemu-img convert -t none -O raw -f qcow2 /opt/stack/data/nova/instances/_base/7f7598d3f4a27ebdcdcae511dd4e04ef3c67d957.part /opt/stack/data/nova/instances/_base/7f7598d3f4a27ebdcdcae511dd4e04ef3c67d957.converted" returned: 0 in 0.262s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:06:30 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-a745e7d4-15ea-45ba-a537-b658a23b24ed tempest-AttachSCSIVolumeTestJSON-719577128 tempest-AttachSCSIVolumeTestJSON-719577128-project-member] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/7f7598d3f4a27ebdcdcae511dd4e04ef3c67d957.converted --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:06:30 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-a745e7d4-15ea-45ba-a537-b658a23b24ed tempest-AttachSCSIVolumeTestJSON-719577128 tempest-AttachSCSIVolumeTestJSON-719577128-project-member] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/7f7598d3f4a27ebdcdcae511dd4e04ef3c67d957.converted --force-share --output=json" returned: 0 in 0.231s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:06:30 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-a745e7d4-15ea-45ba-a537-b658a23b24ed tempest-AttachSCSIVolumeTestJSON-719577128 tempest-AttachSCSIVolumeTestJSON-719577128-project-member] Lock "7f7598d3f4a27ebdcdcae511dd4e04ef3c67d957" "released" by "nova.virt.libvirt.imagebackend.Image.cache.<locals>.fetch_func_sync" :: held 1.405s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:06:30 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-a745e7d4-15ea-45ba-a537-b658a23b24ed tempest-AttachSCSIVolumeTestJSON-719577128 tempest-AttachSCSIVolumeTestJSON-719577128-project-member] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/7f7598d3f4a27ebdcdcae511dd4e04ef3c67d957 --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:06:30 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-a745e7d4-15ea-45ba-a537-b658a23b24ed tempest-AttachSCSIVolumeTestJSON-719577128 tempest-AttachSCSIVolumeTestJSON-719577128-project-member] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/7f7598d3f4a27ebdcdcae511dd4e04ef3c67d957 --force-share --output=json" returned: 0 in 0.204s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:06:30 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-a745e7d4-15ea-45ba-a537-b658a23b24ed tempest-AttachSCSIVolumeTestJSON-719577128 tempest-AttachSCSIVolumeTestJSON-719577128-project-member] Acquiring lock "7f7598d3f4a27ebdcdcae511dd4e04ef3c67d957" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:06:30 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-a745e7d4-15ea-45ba-a537-b658a23b24ed tempest-AttachSCSIVolumeTestJSON-719577128 tempest-AttachSCSIVolumeTestJSON-719577128-project-member] Lock "7f7598d3f4a27ebdcdcae511dd4e04ef3c67d957" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.002s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:06:30 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-a745e7d4-15ea-45ba-a537-b658a23b24ed tempest-AttachSCSIVolumeTestJSON-719577128 tempest-AttachSCSIVolumeTestJSON-719577128-project-member] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/7f7598d3f4a27ebdcdcae511dd4e04ef3c67d957 --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:06:31 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:06:31 user nova-compute[75234]: DEBUG nova.compute.provider_tree [None req-684945de-57fe-475b-b990-ae7897d5ffb4 tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] Inventory has not changed in ProviderTree for provider: 4d291220-7c37-4839-a0e3-b23c66502b74 {{(pid=75234) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Nov 29 03:06:31 user nova-compute[75234]: DEBUG nova.scheduler.client.report [None req-684945de-57fe-475b-b990-ae7897d5ffb4 tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] Inventory has not changed for provider 4d291220-7c37-4839-a0e3-b23c66502b74 based on inventory data: {'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=75234) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Nov 29 03:06:31 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-a745e7d4-15ea-45ba-a537-b658a23b24ed tempest-AttachSCSIVolumeTestJSON-719577128 tempest-AttachSCSIVolumeTestJSON-719577128-project-member] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/7f7598d3f4a27ebdcdcae511dd4e04ef3c67d957 --force-share --output=json" returned: 0 in 0.246s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:06:31 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-a745e7d4-15ea-45ba-a537-b658a23b24ed tempest-AttachSCSIVolumeTestJSON-719577128 tempest-AttachSCSIVolumeTestJSON-719577128-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/7f7598d3f4a27ebdcdcae511dd4e04ef3c67d957,backing_fmt=raw /opt/stack/data/nova/instances/da625f9c-b7ed-4afc-875c-6436bf58e9cc/disk 1073741824 {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:06:31 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-684945de-57fe-475b-b990-ae7897d5ffb4 tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.957s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:06:31 user nova-compute[75234]: DEBUG nova.compute.manager [None req-684945de-57fe-475b-b990-ae7897d5ffb4 tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] [instance: aa3c7432-bf15-408e-9fbb-993135734161] Start building networks asynchronously for instance. {{(pid=75234) _build_resources /opt/stack/nova/nova/compute/manager.py:2819}}
Nov 29 03:06:31 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-a745e7d4-15ea-45ba-a537-b658a23b24ed tempest-AttachSCSIVolumeTestJSON-719577128 tempest-AttachSCSIVolumeTestJSON-719577128-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/7f7598d3f4a27ebdcdcae511dd4e04ef3c67d957,backing_fmt=raw /opt/stack/data/nova/instances/da625f9c-b7ed-4afc-875c-6436bf58e9cc/disk 1073741824" returned: 0 in 0.077s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:06:31 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-a745e7d4-15ea-45ba-a537-b658a23b24ed tempest-AttachSCSIVolumeTestJSON-719577128 tempest-AttachSCSIVolumeTestJSON-719577128-project-member] Lock "7f7598d3f4a27ebdcdcae511dd4e04ef3c67d957" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.328s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:06:31 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-a745e7d4-15ea-45ba-a537-b658a23b24ed tempest-AttachSCSIVolumeTestJSON-719577128 tempest-AttachSCSIVolumeTestJSON-719577128-project-member] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/7f7598d3f4a27ebdcdcae511dd4e04ef3c67d957 --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:06:31 user nova-compute[75234]: DEBUG nova.network.neutron [None req-a745e7d4-15ea-45ba-a537-b658a23b24ed tempest-AttachSCSIVolumeTestJSON-719577128 tempest-AttachSCSIVolumeTestJSON-719577128-project-member] [instance: da625f9c-b7ed-4afc-875c-6436bf58e9cc] Successfully updated port: a23b8769-d871-4d44-8cea-5d899f514579 {{(pid=75234) _update_port /opt/stack/nova/nova/network/neutron.py:586}}
Nov 29 03:06:31 user nova-compute[75234]: DEBUG nova.compute.manager [req-87713994-cd6d-496e-982f-4e70accd4589 req-88ed79cf-bb5b-4c35-a1af-5142bdb8e9bf service nova] [instance: da625f9c-b7ed-4afc-875c-6436bf58e9cc] Received event network-changed-a23b8769-d871-4d44-8cea-5d899f514579 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:06:31 user nova-compute[75234]: DEBUG nova.compute.manager [req-87713994-cd6d-496e-982f-4e70accd4589 req-88ed79cf-bb5b-4c35-a1af-5142bdb8e9bf service nova] [instance: da625f9c-b7ed-4afc-875c-6436bf58e9cc] Refreshing instance network info cache due to event network-changed-a23b8769-d871-4d44-8cea-5d899f514579. {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11117}}
Nov 29 03:06:31 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-87713994-cd6d-496e-982f-4e70accd4589 req-88ed79cf-bb5b-4c35-a1af-5142bdb8e9bf service nova] Acquiring lock "refresh_cache-da625f9c-b7ed-4afc-875c-6436bf58e9cc" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:06:31 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-87713994-cd6d-496e-982f-4e70accd4589 req-88ed79cf-bb5b-4c35-a1af-5142bdb8e9bf service nova] Acquired lock "refresh_cache-da625f9c-b7ed-4afc-875c-6436bf58e9cc" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:06:31 user nova-compute[75234]: DEBUG nova.network.neutron [req-87713994-cd6d-496e-982f-4e70accd4589 req-88ed79cf-bb5b-4c35-a1af-5142bdb8e9bf service nova] [instance: da625f9c-b7ed-4afc-875c-6436bf58e9cc] Refreshing network info cache for port a23b8769-d871-4d44-8cea-5d899f514579 {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2006}}
Nov 29 03:06:31 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-a745e7d4-15ea-45ba-a537-b658a23b24ed tempest-AttachSCSIVolumeTestJSON-719577128 tempest-AttachSCSIVolumeTestJSON-719577128-project-member] Acquiring lock "refresh_cache-da625f9c-b7ed-4afc-875c-6436bf58e9cc" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:06:31 user nova-compute[75234]: DEBUG nova.compute.manager [None req-684945de-57fe-475b-b990-ae7897d5ffb4 tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] [instance: aa3c7432-bf15-408e-9fbb-993135734161] Allocating IP information in the background. {{(pid=75234) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1972}}
Nov 29 03:06:31 user nova-compute[75234]: DEBUG nova.network.neutron [None req-684945de-57fe-475b-b990-ae7897d5ffb4 tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] [instance: aa3c7432-bf15-408e-9fbb-993135734161] allocate_for_instance() {{(pid=75234) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1156}}
Nov 29 03:06:31 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-684945de-57fe-475b-b990-ae7897d5ffb4 tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] [instance: aa3c7432-bf15-408e-9fbb-993135734161] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Nov 29 03:06:31 user nova-compute[75234]: DEBUG nova.compute.manager [None req-684945de-57fe-475b-b990-ae7897d5ffb4 tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] [instance: aa3c7432-bf15-408e-9fbb-993135734161] Start building block device mappings for instance. {{(pid=75234) _build_resources /opt/stack/nova/nova/compute/manager.py:2854}}
Nov 29 03:06:31 user nova-compute[75234]: DEBUG nova.network.neutron [req-87713994-cd6d-496e-982f-4e70accd4589 req-88ed79cf-bb5b-4c35-a1af-5142bdb8e9bf service nova] [instance: da625f9c-b7ed-4afc-875c-6436bf58e9cc] Instance cache missing network info. {{(pid=75234) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3322}}
Nov 29 03:06:31 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-a745e7d4-15ea-45ba-a537-b658a23b24ed tempest-AttachSCSIVolumeTestJSON-719577128 tempest-AttachSCSIVolumeTestJSON-719577128-project-member] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/7f7598d3f4a27ebdcdcae511dd4e04ef3c67d957 --force-share --output=json" returned: 0 in 0.322s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:06:31 user nova-compute[75234]: DEBUG nova.virt.disk.api [None req-a745e7d4-15ea-45ba-a537-b658a23b24ed tempest-AttachSCSIVolumeTestJSON-719577128 tempest-AttachSCSIVolumeTestJSON-719577128-project-member] Checking if we can resize image /opt/stack/data/nova/instances/da625f9c-b7ed-4afc-875c-6436bf58e9cc/disk. size=1073741824 {{(pid=75234) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:166}}
Nov 29 03:06:31 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-a745e7d4-15ea-45ba-a537-b658a23b24ed tempest-AttachSCSIVolumeTestJSON-719577128 tempest-AttachSCSIVolumeTestJSON-719577128-project-member] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/da625f9c-b7ed-4afc-875c-6436bf58e9cc/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:06:31 user nova-compute[75234]: DEBUG nova.policy [None req-684945de-57fe-475b-b990-ae7897d5ffb4 tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '940a684aa3b6419798bd87562bbc5d23', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '46577f787c244fb2a4db052a0c42eceb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=75234) authorize /opt/stack/nova/nova/policy.py:203}}
Nov 29 03:06:31 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-a745e7d4-15ea-45ba-a537-b658a23b24ed tempest-AttachSCSIVolumeTestJSON-719577128 tempest-AttachSCSIVolumeTestJSON-719577128-project-member] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/da625f9c-b7ed-4afc-875c-6436bf58e9cc/disk --force-share --output=json" returned: 0 in 0.232s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:06:31 user nova-compute[75234]: DEBUG nova.virt.disk.api [None req-a745e7d4-15ea-45ba-a537-b658a23b24ed tempest-AttachSCSIVolumeTestJSON-719577128 tempest-AttachSCSIVolumeTestJSON-719577128-project-member] Cannot resize image /opt/stack/data/nova/instances/da625f9c-b7ed-4afc-875c-6436bf58e9cc/disk to a smaller size. {{(pid=75234) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:172}}
Nov 29 03:06:31 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-a745e7d4-15ea-45ba-a537-b658a23b24ed tempest-AttachSCSIVolumeTestJSON-719577128 tempest-AttachSCSIVolumeTestJSON-719577128-project-member] [instance: da625f9c-b7ed-4afc-875c-6436bf58e9cc] Created local disks {{(pid=75234) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4861}}
Nov 29 03:06:31 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-a745e7d4-15ea-45ba-a537-b658a23b24ed tempest-AttachSCSIVolumeTestJSON-719577128 tempest-AttachSCSIVolumeTestJSON-719577128-project-member] [instance: da625f9c-b7ed-4afc-875c-6436bf58e9cc] Ensure instance console log exists: /opt/stack/data/nova/instances/da625f9c-b7ed-4afc-875c-6436bf58e9cc/console.log {{(pid=75234) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4613}}
Nov 29 03:06:31 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-a745e7d4-15ea-45ba-a537-b658a23b24ed tempest-AttachSCSIVolumeTestJSON-719577128 tempest-AttachSCSIVolumeTestJSON-719577128-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:06:31 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-a745e7d4-15ea-45ba-a537-b658a23b24ed tempest-AttachSCSIVolumeTestJSON-719577128 tempest-AttachSCSIVolumeTestJSON-719577128-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:06:31 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-a745e7d4-15ea-45ba-a537-b658a23b24ed tempest-AttachSCSIVolumeTestJSON-719577128 tempest-AttachSCSIVolumeTestJSON-719577128-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:06:31 user nova-compute[75234]: DEBUG nova.compute.manager [None req-684945de-57fe-475b-b990-ae7897d5ffb4 tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] [instance: aa3c7432-bf15-408e-9fbb-993135734161] Start spawning the instance on the hypervisor. {{(pid=75234) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2628}}
Nov 29 03:06:31 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-684945de-57fe-475b-b990-ae7897d5ffb4 tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] [instance: aa3c7432-bf15-408e-9fbb-993135734161] Creating instance directory {{(pid=75234) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4727}}
Nov 29 03:06:31 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-684945de-57fe-475b-b990-ae7897d5ffb4 tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] [instance: aa3c7432-bf15-408e-9fbb-993135734161] Creating image(s)
Nov 29 03:06:31 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-684945de-57fe-475b-b990-ae7897d5ffb4 tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] Acquiring lock "/opt/stack/data/nova/instances/aa3c7432-bf15-408e-9fbb-993135734161/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:06:31 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-684945de-57fe-475b-b990-ae7897d5ffb4 tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] Lock "/opt/stack/data/nova/instances/aa3c7432-bf15-408e-9fbb-993135734161/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:06:31 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-684945de-57fe-475b-b990-ae7897d5ffb4 tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] Lock "/opt/stack/data/nova/instances/aa3c7432-bf15-408e-9fbb-993135734161/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.003s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:06:31 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-684945de-57fe-475b-b990-ae7897d5ffb4 tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:06:31 user nova-compute[75234]: DEBUG nova.network.neutron [req-87713994-cd6d-496e-982f-4e70accd4589 req-88ed79cf-bb5b-4c35-a1af-5142bdb8e9bf service nova] [instance: da625f9c-b7ed-4afc-875c-6436bf58e9cc] Updating instance_info_cache with network_info: [] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:06:32 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-87713994-cd6d-496e-982f-4e70accd4589 req-88ed79cf-bb5b-4c35-a1af-5142bdb8e9bf service nova] Releasing lock "refresh_cache-da625f9c-b7ed-4afc-875c-6436bf58e9cc" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:06:32 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-a745e7d4-15ea-45ba-a537-b658a23b24ed tempest-AttachSCSIVolumeTestJSON-719577128 tempest-AttachSCSIVolumeTestJSON-719577128-project-member] Acquired lock "refresh_cache-da625f9c-b7ed-4afc-875c-6436bf58e9cc" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:06:32 user nova-compute[75234]: DEBUG nova.network.neutron [None req-a745e7d4-15ea-45ba-a537-b658a23b24ed tempest-AttachSCSIVolumeTestJSON-719577128 tempest-AttachSCSIVolumeTestJSON-719577128-project-member] [instance: da625f9c-b7ed-4afc-875c-6436bf58e9cc] Building network info cache for instance {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2009}}
Nov 29 03:06:32 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-684945de-57fe-475b-b990-ae7897d5ffb4 tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json" returned: 0 in 0.523s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:06:32 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-684945de-57fe-475b-b990-ae7897d5ffb4 tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] Acquiring lock "80686e29dc9db6600d03e85cb62af298e12d7918" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:06:32 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-684945de-57fe-475b-b990-ae7897d5ffb4 tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] Lock "80686e29dc9db6600d03e85cb62af298e12d7918" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:06:32 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-684945de-57fe-475b-b990-ae7897d5ffb4 tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:06:32 user nova-compute[75234]: DEBUG nova.network.neutron [None req-a745e7d4-15ea-45ba-a537-b658a23b24ed tempest-AttachSCSIVolumeTestJSON-719577128 tempest-AttachSCSIVolumeTestJSON-719577128-project-member] [instance: da625f9c-b7ed-4afc-875c-6436bf58e9cc] Instance cache missing network info. {{(pid=75234) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3322}}
Nov 29 03:06:32 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-684945de-57fe-475b-b990-ae7897d5ffb4 tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json" returned: 0 in 0.253s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:06:32 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-684945de-57fe-475b-b990-ae7897d5ffb4 tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918,backing_fmt=raw /opt/stack/data/nova/instances/aa3c7432-bf15-408e-9fbb-993135734161/disk 1073741824 {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:06:32 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-684945de-57fe-475b-b990-ae7897d5ffb4 tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918,backing_fmt=raw /opt/stack/data/nova/instances/aa3c7432-bf15-408e-9fbb-993135734161/disk 1073741824" returned: 0 in 0.077s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:06:32 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-684945de-57fe-475b-b990-ae7897d5ffb4 tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] Lock "80686e29dc9db6600d03e85cb62af298e12d7918" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.336s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:06:32 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-684945de-57fe-475b-b990-ae7897d5ffb4 tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:06:33 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-684945de-57fe-475b-b990-ae7897d5ffb4 tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json" returned: 0 in 0.242s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:06:33 user nova-compute[75234]: DEBUG nova.virt.disk.api [None req-684945de-57fe-475b-b990-ae7897d5ffb4 tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] Checking if we can resize image /opt/stack/data/nova/instances/aa3c7432-bf15-408e-9fbb-993135734161/disk. size=1073741824 {{(pid=75234) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:166}}
Nov 29 03:06:33 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-684945de-57fe-475b-b990-ae7897d5ffb4 tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/aa3c7432-bf15-408e-9fbb-993135734161/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:06:33 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-684945de-57fe-475b-b990-ae7897d5ffb4 tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/aa3c7432-bf15-408e-9fbb-993135734161/disk --force-share --output=json" returned: 0 in 0.211s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:06:33 user nova-compute[75234]: DEBUG nova.virt.disk.api [None req-684945de-57fe-475b-b990-ae7897d5ffb4 tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] Cannot resize image /opt/stack/data/nova/instances/aa3c7432-bf15-408e-9fbb-993135734161/disk to a smaller size. {{(pid=75234) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:172}}
Nov 29 03:06:33 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-684945de-57fe-475b-b990-ae7897d5ffb4 tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] [instance: aa3c7432-bf15-408e-9fbb-993135734161] Created local disks {{(pid=75234) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4861}}
Nov 29 03:06:33 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-684945de-57fe-475b-b990-ae7897d5ffb4 tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] [instance: aa3c7432-bf15-408e-9fbb-993135734161] Ensure instance console log exists: /opt/stack/data/nova/instances/aa3c7432-bf15-408e-9fbb-993135734161/console.log {{(pid=75234) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4613}}
Nov 29 03:06:33 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-684945de-57fe-475b-b990-ae7897d5ffb4 tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:06:33 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-684945de-57fe-475b-b990-ae7897d5ffb4 tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.002s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:06:33 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-684945de-57fe-475b-b990-ae7897d5ffb4 tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:06:33 user nova-compute[75234]: DEBUG nova.network.neutron [None req-a745e7d4-15ea-45ba-a537-b658a23b24ed tempest-AttachSCSIVolumeTestJSON-719577128 tempest-AttachSCSIVolumeTestJSON-719577128-project-member] [instance: da625f9c-b7ed-4afc-875c-6436bf58e9cc] Updating instance_info_cache with network_info: [{"id": "a23b8769-d871-4d44-8cea-5d899f514579", "address": "fa:16:3e:7f:17:4d", "network": {"id": "5bbe9946-1c42-4c9e-8cf7-5ee0a54ebc8d", "bridge": "br-int", "label": "tempest-AttachSCSIVolumeTestJSON-411635514-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "df2e94062c154555ab1ee0c3bf8e0f84", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapa23b8769-d8", "ovs_interfaceid": "a23b8769-d871-4d44-8cea-5d899f514579", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:06:33 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-a745e7d4-15ea-45ba-a537-b658a23b24ed tempest-AttachSCSIVolumeTestJSON-719577128 tempest-AttachSCSIVolumeTestJSON-719577128-project-member] Releasing lock "refresh_cache-da625f9c-b7ed-4afc-875c-6436bf58e9cc" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:06:33 user nova-compute[75234]: DEBUG nova.compute.manager [None req-a745e7d4-15ea-45ba-a537-b658a23b24ed tempest-AttachSCSIVolumeTestJSON-719577128 tempest-AttachSCSIVolumeTestJSON-719577128-project-member] [instance: da625f9c-b7ed-4afc-875c-6436bf58e9cc] Instance network_info: |[{"id": "a23b8769-d871-4d44-8cea-5d899f514579", "address": "fa:16:3e:7f:17:4d", "network": {"id": "5bbe9946-1c42-4c9e-8cf7-5ee0a54ebc8d", "bridge": "br-int", "label": "tempest-AttachSCSIVolumeTestJSON-411635514-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "df2e94062c154555ab1ee0c3bf8e0f84", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapa23b8769-d8", "ovs_interfaceid": "a23b8769-d871-4d44-8cea-5d899f514579", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=75234) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1987}}
Nov 29 03:06:33 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-a745e7d4-15ea-45ba-a537-b658a23b24ed tempest-AttachSCSIVolumeTestJSON-719577128 tempest-AttachSCSIVolumeTestJSON-719577128-project-member] [instance: da625f9c-b7ed-4afc-875c-6436bf58e9cc] Start _get_guest_xml network_info=[{"id": "a23b8769-d871-4d44-8cea-5d899f514579", "address": "fa:16:3e:7f:17:4d", "network": {"id": "5bbe9946-1c42-4c9e-8cf7-5ee0a54ebc8d", "bridge": "br-int", "label": "tempest-AttachSCSIVolumeTestJSON-411635514-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "df2e94062c154555ab1ee0c3bf8e0f84", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapa23b8769-d8", "ovs_interfaceid": "a23b8769-d871-4d44-8cea-5d899f514579", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'scsi', 'cdrom_bus': 'scsi', 'mapping': {'root': {'bus': 'scsi', 'dev': 'sda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'scsi', 'dev': 'sda', 'type': 'disk', 'boot_index': '1'}, 'disk.config': {'bus': 'scsi', 'dev': 'sdb', 'type': 'cdrom'}}} image_meta=ImageMeta(checksum='c8fc807773e5354afe61636071771906',container_format='bare',created_at=2023-11-29T03:06:19Z,direct_url=<?>,disk_format='qcow2',id=b25373f9-ce4e-4a59-b733-b4caaf9c34c6,min_disk=0,min_ram=0,name='',owner='3a60e149d9ef475880052660428c463a',properties=ImageMetaProps,protected=<?>,size=21430272,status='active',tags=<?>,updated_at=2023-11-29T03:06:22Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/sda', 'image': [{'device_name': '/dev/sda', 'encryption_format': None, 'encrypted': False, 'device_type': 'disk', 'size': 0, 'disk_bus': 'scsi', 'boot_index': 0, 'encryption_options': None, 'encryption_secret_uuid': None, 'guest_format': None, 'image_id': 'b25373f9-ce4e-4a59-b733-b4caaf9c34c6'}], 'ephemerals': [], 'block_device_mapping': [], 'swap': None} {{(pid=75234) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7562}}
Nov 29 03:06:33 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-a745e7d4-15ea-45ba-a537-b658a23b24ed tempest-AttachSCSIVolumeTestJSON-719577128 tempest-AttachSCSIVolumeTestJSON-719577128-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:06:33 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-a745e7d4-15ea-45ba-a537-b658a23b24ed tempest-AttachSCSIVolumeTestJSON-719577128 tempest-AttachSCSIVolumeTestJSON-719577128-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:06:33 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-a745e7d4-15ea-45ba-a537-b658a23b24ed tempest-AttachSCSIVolumeTestJSON-719577128 tempest-AttachSCSIVolumeTestJSON-719577128-project-member] Searching host: 'user' for CPU controller through CGroups V1... {{(pid=75234) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1652}}
Nov 29 03:06:33 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-a745e7d4-15ea-45ba-a537-b658a23b24ed tempest-AttachSCSIVolumeTestJSON-719577128 tempest-AttachSCSIVolumeTestJSON-719577128-project-member] CPU controller missing on host. {{(pid=75234) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1662}}
Nov 29 03:06:33 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-a745e7d4-15ea-45ba-a537-b658a23b24ed tempest-AttachSCSIVolumeTestJSON-719577128 tempest-AttachSCSIVolumeTestJSON-719577128-project-member] Searching host: 'user' for CPU controller through CGroups V2... {{(pid=75234) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1671}}
Nov 29 03:06:33 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-a745e7d4-15ea-45ba-a537-b658a23b24ed tempest-AttachSCSIVolumeTestJSON-719577128 tempest-AttachSCSIVolumeTestJSON-719577128-project-member] CPU controller found on host. {{(pid=75234) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1678}}
Nov 29 03:06:33 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-a745e7d4-15ea-45ba-a537-b658a23b24ed tempest-AttachSCSIVolumeTestJSON-719577128 tempest-AttachSCSIVolumeTestJSON-719577128-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=75234) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5400}}
Nov 29 03:06:33 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-a745e7d4-15ea-45ba-a537-b658a23b24ed tempest-AttachSCSIVolumeTestJSON-719577128 tempest-AttachSCSIVolumeTestJSON-719577128-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-11-29T03:02:04Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='1',id=6,is_public=True,memory_mb=512,name='m1.tiny',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='c8fc807773e5354afe61636071771906',container_format='bare',created_at=2023-11-29T03:06:19Z,direct_url=<?>,disk_format='qcow2',id=b25373f9-ce4e-4a59-b733-b4caaf9c34c6,min_disk=0,min_ram=0,name='',owner='3a60e149d9ef475880052660428c463a',properties=ImageMetaProps,protected=<?>,size=21430272,status='active',tags=<?>,updated_at=2023-11-29T03:06:22Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:563}}
Nov 29 03:06:33 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-a745e7d4-15ea-45ba-a537-b658a23b24ed tempest-AttachSCSIVolumeTestJSON-719577128 tempest-AttachSCSIVolumeTestJSON-719577128-project-member] Flavor limits 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:348}}
Nov 29 03:06:33 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-a745e7d4-15ea-45ba-a537-b658a23b24ed tempest-AttachSCSIVolumeTestJSON-719577128 tempest-AttachSCSIVolumeTestJSON-719577128-project-member] Image limits 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:352}}
Nov 29 03:06:33 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-a745e7d4-15ea-45ba-a537-b658a23b24ed tempest-AttachSCSIVolumeTestJSON-719577128 tempest-AttachSCSIVolumeTestJSON-719577128-project-member] Flavor pref 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:388}}
Nov 29 03:06:33 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-a745e7d4-15ea-45ba-a537-b658a23b24ed tempest-AttachSCSIVolumeTestJSON-719577128 tempest-AttachSCSIVolumeTestJSON-719577128-project-member] Image pref 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:392}}
Nov 29 03:06:33 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-a745e7d4-15ea-45ba-a537-b658a23b24ed tempest-AttachSCSIVolumeTestJSON-719577128 tempest-AttachSCSIVolumeTestJSON-719577128-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:430}}
Nov 29 03:06:33 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-a745e7d4-15ea-45ba-a537-b658a23b24ed tempest-AttachSCSIVolumeTestJSON-719577128 tempest-AttachSCSIVolumeTestJSON-719577128-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:569}}
Nov 29 03:06:33 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-a745e7d4-15ea-45ba-a537-b658a23b24ed tempest-AttachSCSIVolumeTestJSON-719577128 tempest-AttachSCSIVolumeTestJSON-719577128-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=75234) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:471}}
Nov 29 03:06:33 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-a745e7d4-15ea-45ba-a537-b658a23b24ed tempest-AttachSCSIVolumeTestJSON-719577128 tempest-AttachSCSIVolumeTestJSON-719577128-project-member] Got 1 possible topologies {{(pid=75234) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:501}}
Nov 29 03:06:33 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-a745e7d4-15ea-45ba-a537-b658a23b24ed tempest-AttachSCSIVolumeTestJSON-719577128 tempest-AttachSCSIVolumeTestJSON-719577128-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:575}}
Nov 29 03:06:33 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-a745e7d4-15ea-45ba-a537-b658a23b24ed tempest-AttachSCSIVolumeTestJSON-719577128 tempest-AttachSCSIVolumeTestJSON-719577128-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:577}}
Nov 29 03:06:33 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-a745e7d4-15ea-45ba-a537-b658a23b24ed tempest-AttachSCSIVolumeTestJSON-719577128 tempest-AttachSCSIVolumeTestJSON-719577128-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='True',created_at=2023-11-29T03:06:27Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-AttachSCSIVolumeTestJSON-server-171828016',display_name='tempest-AttachSCSIVolumeTestJSON-server-171828016',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-attachscsivolumetestjson-server-171828016',id=7,image_ref='b25373f9-ce4e-4a59-b733-b4caaf9c34c6',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBC4uploT/5M0XHJIlQuZn+27+ncVPyCTxw0QjUV1DHWW3WslgbLdQdc14yzb4DwiiAP8Omme0YQgb6GOuaNR43ED2DaxENLJ5i6sS1MBovu0WHlXf/5HMiLederK2qneAQ==',key_name='tempest-keypair-473777431',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='df2e94062c154555ab1ee0c3bf8e0f84',ramdisk_id='',reservation_id='r-801mj20k',resources=None,root_device_name='/dev/sda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='b25373f9-ce4e-4a59-b733-b4caaf9c34c6',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='scsi',image_hw_disk_bus='scsi',image_hw_machine_type='pc',image_hw_scsi_model='virtio-scsi',image_min_disk='1',image_min_ram='0',network_allocated='True',owner_project_name='tempest-AttachSCSIVolumeTestJSON-719577128',owner_user_name='tempest-AttachSCSIVolumeTestJSON-719577128-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-11-29T03:06:29Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='ef10980c66b44933bc463320af40dfca',uuid=da625f9c-b7ed-4afc-875c-6436bf58e9cc,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "a23b8769-d871-4d44-8cea-5d899f514579", "address": "fa:16:3e:7f:17:4d", "network": {"id": "5bbe9946-1c42-4c9e-8cf7-5ee0a54ebc8d", "bridge": "br-int", "label": "tempest-AttachSCSIVolumeTestJSON-411635514-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "df2e94062c154555ab1ee0c3bf8e0f84", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapa23b8769-d8", "ovs_interfaceid": "a23b8769-d871-4d44-8cea-5d899f514579", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=75234) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Nov 29 03:06:33 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-a745e7d4-15ea-45ba-a537-b658a23b24ed tempest-AttachSCSIVolumeTestJSON-719577128 tempest-AttachSCSIVolumeTestJSON-719577128-project-member] Converting VIF {"id": "a23b8769-d871-4d44-8cea-5d899f514579", "address": "fa:16:3e:7f:17:4d", "network": {"id": "5bbe9946-1c42-4c9e-8cf7-5ee0a54ebc8d", "bridge": "br-int", "label": "tempest-AttachSCSIVolumeTestJSON-411635514-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "df2e94062c154555ab1ee0c3bf8e0f84", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapa23b8769-d8", "ovs_interfaceid": "a23b8769-d871-4d44-8cea-5d899f514579", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:06:33 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-a745e7d4-15ea-45ba-a537-b658a23b24ed tempest-AttachSCSIVolumeTestJSON-719577128 tempest-AttachSCSIVolumeTestJSON-719577128-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:7f:17:4d,bridge_name='br-int',has_traffic_filtering=True,id=a23b8769-d871-4d44-8cea-5d899f514579,network=Network(5bbe9946-1c42-4c9e-8cf7-5ee0a54ebc8d),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapa23b8769-d8') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:06:33 user nova-compute[75234]: DEBUG nova.objects.instance [None req-a745e7d4-15ea-45ba-a537-b658a23b24ed tempest-AttachSCSIVolumeTestJSON-719577128 tempest-AttachSCSIVolumeTestJSON-719577128-project-member] Lazy-loading 'pci_devices' on Instance uuid da625f9c-b7ed-4afc-875c-6436bf58e9cc {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:06:33 user nova-compute[75234]: DEBUG nova.network.neutron [None req-684945de-57fe-475b-b990-ae7897d5ffb4 tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] [instance: aa3c7432-bf15-408e-9fbb-993135734161] Successfully created port: aaac2504-1067-43c5-b3bf-316996e0f56c {{(pid=75234) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:548}}
Nov 29 03:06:33 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:06:33 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-a745e7d4-15ea-45ba-a537-b658a23b24ed tempest-AttachSCSIVolumeTestJSON-719577128 tempest-AttachSCSIVolumeTestJSON-719577128-project-member] [instance: da625f9c-b7ed-4afc-875c-6436bf58e9cc] End _get_guest_xml xml=<domain type="kvm">
Nov 29 03:06:33 user nova-compute[75234]:   <uuid>da625f9c-b7ed-4afc-875c-6436bf58e9cc</uuid>
Nov 29 03:06:33 user nova-compute[75234]:   <name>instance-00000007</name>
Nov 29 03:06:33 user nova-compute[75234]:   <memory>524288</memory>
Nov 29 03:06:33 user nova-compute[75234]:   <vcpu>1</vcpu>
Nov 29 03:06:33 user nova-compute[75234]:   <metadata>
Nov 29 03:06:33 user nova-compute[75234]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Nov 29 03:06:33 user nova-compute[75234]:       <nova:package version="0.0.0"/>
Nov 29 03:06:33 user nova-compute[75234]:       <nova:name>tempest-AttachSCSIVolumeTestJSON-server-171828016</nova:name>
Nov 29 03:06:33 user nova-compute[75234]:       <nova:creationTime>2023-11-29 03:06:33</nova:creationTime>
Nov 29 03:06:33 user nova-compute[75234]:       <nova:flavor name="m1.tiny">
Nov 29 03:06:33 user nova-compute[75234]:         <nova:memory>512</nova:memory>
Nov 29 03:06:33 user nova-compute[75234]:         <nova:disk>1</nova:disk>
Nov 29 03:06:33 user nova-compute[75234]:         <nova:swap>0</nova:swap>
Nov 29 03:06:33 user nova-compute[75234]:         <nova:ephemeral>0</nova:ephemeral>
Nov 29 03:06:33 user nova-compute[75234]:         <nova:vcpus>1</nova:vcpus>
Nov 29 03:06:33 user nova-compute[75234]:       </nova:flavor>
Nov 29 03:06:33 user nova-compute[75234]:       <nova:owner>
Nov 29 03:06:33 user nova-compute[75234]:         <nova:user uuid="ef10980c66b44933bc463320af40dfca">tempest-AttachSCSIVolumeTestJSON-719577128-project-member</nova:user>
Nov 29 03:06:33 user nova-compute[75234]:         <nova:project uuid="df2e94062c154555ab1ee0c3bf8e0f84">tempest-AttachSCSIVolumeTestJSON-719577128</nova:project>
Nov 29 03:06:33 user nova-compute[75234]:       </nova:owner>
Nov 29 03:06:33 user nova-compute[75234]:       <nova:root type="image" uuid="b25373f9-ce4e-4a59-b733-b4caaf9c34c6"/>
Nov 29 03:06:33 user nova-compute[75234]:       <nova:ports>
Nov 29 03:06:33 user nova-compute[75234]:         <nova:port uuid="a23b8769-d871-4d44-8cea-5d899f514579">
Nov 29 03:06:33 user nova-compute[75234]:           <nova:ip type="fixed" address="10.0.0.12" ipVersion="4"/>
Nov 29 03:06:33 user nova-compute[75234]:         </nova:port>
Nov 29 03:06:33 user nova-compute[75234]:       </nova:ports>
Nov 29 03:06:33 user nova-compute[75234]:     </nova:instance>
Nov 29 03:06:33 user nova-compute[75234]:   </metadata>
Nov 29 03:06:33 user nova-compute[75234]:   <sysinfo type="smbios">
Nov 29 03:06:33 user nova-compute[75234]:     <system>
Nov 29 03:06:33 user nova-compute[75234]:       <entry name="manufacturer">OpenStack Foundation</entry>
Nov 29 03:06:33 user nova-compute[75234]:       <entry name="product">OpenStack Nova</entry>
Nov 29 03:06:33 user nova-compute[75234]:       <entry name="version">0.0.0</entry>
Nov 29 03:06:33 user nova-compute[75234]:       <entry name="serial">da625f9c-b7ed-4afc-875c-6436bf58e9cc</entry>
Nov 29 03:06:33 user nova-compute[75234]:       <entry name="uuid">da625f9c-b7ed-4afc-875c-6436bf58e9cc</entry>
Nov 29 03:06:33 user nova-compute[75234]:       <entry name="family">Virtual Machine</entry>
Nov 29 03:06:33 user nova-compute[75234]:     </system>
Nov 29 03:06:33 user nova-compute[75234]:   </sysinfo>
Nov 29 03:06:33 user nova-compute[75234]:   <os>
Nov 29 03:06:33 user nova-compute[75234]:     <type arch="x86_64" machine="pc">hvm</type>
Nov 29 03:06:33 user nova-compute[75234]:     <boot dev="hd"/>
Nov 29 03:06:33 user nova-compute[75234]:     <smbios mode="sysinfo"/>
Nov 29 03:06:33 user nova-compute[75234]:   </os>
Nov 29 03:06:33 user nova-compute[75234]:   <features>
Nov 29 03:06:33 user nova-compute[75234]:     <acpi/>
Nov 29 03:06:33 user nova-compute[75234]:     <apic/>
Nov 29 03:06:33 user nova-compute[75234]:     <vmcoreinfo/>
Nov 29 03:06:33 user nova-compute[75234]:   </features>
Nov 29 03:06:33 user nova-compute[75234]:   <clock offset="utc">
Nov 29 03:06:33 user nova-compute[75234]:     <timer name="pit" tickpolicy="delay"/>
Nov 29 03:06:33 user nova-compute[75234]:     <timer name="rtc" tickpolicy="catchup"/>
Nov 29 03:06:33 user nova-compute[75234]:     <timer name="hpet" present="no"/>
Nov 29 03:06:33 user nova-compute[75234]:   </clock>
Nov 29 03:06:33 user nova-compute[75234]:   <cpu mode="custom" match="exact">
Nov 29 03:06:33 user nova-compute[75234]:     <model>Nehalem</model>
Nov 29 03:06:33 user nova-compute[75234]:     <topology sockets="1" cores="1" threads="1"/>
Nov 29 03:06:33 user nova-compute[75234]:   </cpu>
Nov 29 03:06:33 user nova-compute[75234]:   <devices>
Nov 29 03:06:33 user nova-compute[75234]:     <disk type="file" device="disk">
Nov 29 03:06:33 user nova-compute[75234]:       <driver name="qemu" type="qcow2" cache="none"/>
Nov 29 03:06:33 user nova-compute[75234]:       <source file="/opt/stack/data/nova/instances/da625f9c-b7ed-4afc-875c-6436bf58e9cc/disk"/>
Nov 29 03:06:33 user nova-compute[75234]:       <target dev="sda" bus="scsi"/>
Nov 29 03:06:33 user nova-compute[75234]:       <address type="drive" controller="0" unit="0"/>
Nov 29 03:06:33 user nova-compute[75234]:     </disk>
Nov 29 03:06:33 user nova-compute[75234]:     <disk type="file" device="cdrom">
Nov 29 03:06:33 user nova-compute[75234]:       <driver name="qemu" type="raw" cache="none"/>
Nov 29 03:06:33 user nova-compute[75234]:       <source file="/opt/stack/data/nova/instances/da625f9c-b7ed-4afc-875c-6436bf58e9cc/disk.config"/>
Nov 29 03:06:33 user nova-compute[75234]:       <target dev="sdb" bus="scsi"/>
Nov 29 03:06:33 user nova-compute[75234]:       <address type="drive" controller="0" unit="1"/>
Nov 29 03:06:33 user nova-compute[75234]:     </disk>
Nov 29 03:06:33 user nova-compute[75234]:     <controller type="scsi" index="0" model="virtio-scsi"/>
Nov 29 03:06:33 user nova-compute[75234]:     <interface type="ethernet">
Nov 29 03:06:33 user nova-compute[75234]:       <mac address="fa:16:3e:7f:17:4d"/>
Nov 29 03:06:33 user nova-compute[75234]:       <model type="virtio"/>
Nov 29 03:06:33 user nova-compute[75234]:       <mtu size="1442"/>
Nov 29 03:06:33 user nova-compute[75234]:       <target dev="tapa23b8769-d8"/>
Nov 29 03:06:33 user nova-compute[75234]:     </interface>
Nov 29 03:06:33 user nova-compute[75234]:     <serial type="pty">
Nov 29 03:06:33 user nova-compute[75234]:       <log file="/opt/stack/data/nova/instances/da625f9c-b7ed-4afc-875c-6436bf58e9cc/console.log" append="off"/>
Nov 29 03:06:33 user nova-compute[75234]:     </serial>
Nov 29 03:06:33 user nova-compute[75234]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Nov 29 03:06:33 user nova-compute[75234]:     <video>
Nov 29 03:06:33 user nova-compute[75234]:       <model type="virtio"/>
Nov 29 03:06:33 user nova-compute[75234]:     </video>
Nov 29 03:06:33 user nova-compute[75234]:     <rng model="virtio">
Nov 29 03:06:33 user nova-compute[75234]:       <backend model="random">/dev/urandom</backend>
Nov 29 03:06:33 user nova-compute[75234]:     </rng>
Nov 29 03:06:33 user nova-compute[75234]:     <controller type="usb" index="0" model="none"/>
Nov 29 03:06:33 user nova-compute[75234]:     <memballoon model="virtio">
Nov 29 03:06:33 user nova-compute[75234]:       <stats period="10"/>
Nov 29 03:06:33 user nova-compute[75234]:     </memballoon>
Nov 29 03:06:33 user nova-compute[75234]:   </devices>
Nov 29 03:06:33 user nova-compute[75234]: </domain>
Nov 29 03:06:33 user nova-compute[75234]:  {{(pid=75234) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7568}}
Nov 29 03:06:33 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-a745e7d4-15ea-45ba-a537-b658a23b24ed tempest-AttachSCSIVolumeTestJSON-719577128 tempest-AttachSCSIVolumeTestJSON-719577128-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='True',created_at=2023-11-29T03:06:27Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-AttachSCSIVolumeTestJSON-server-171828016',display_name='tempest-AttachSCSIVolumeTestJSON-server-171828016',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-attachscsivolumetestjson-server-171828016',id=7,image_ref='b25373f9-ce4e-4a59-b733-b4caaf9c34c6',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBC4uploT/5M0XHJIlQuZn+27+ncVPyCTxw0QjUV1DHWW3WslgbLdQdc14yzb4DwiiAP8Omme0YQgb6GOuaNR43ED2DaxENLJ5i6sS1MBovu0WHlXf/5HMiLederK2qneAQ==',key_name='tempest-keypair-473777431',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='df2e94062c154555ab1ee0c3bf8e0f84',ramdisk_id='',reservation_id='r-801mj20k',resources=None,root_device_name='/dev/sda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='b25373f9-ce4e-4a59-b733-b4caaf9c34c6',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='scsi',image_hw_disk_bus='scsi',image_hw_machine_type='pc',image_hw_scsi_model='virtio-scsi',image_min_disk='1',image_min_ram='0',network_allocated='True',owner_project_name='tempest-AttachSCSIVolumeTestJSON-719577128',owner_user_name='tempest-AttachSCSIVolumeTestJSON-719577128-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-11-29T03:06:29Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='ef10980c66b44933bc463320af40dfca',uuid=da625f9c-b7ed-4afc-875c-6436bf58e9cc,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "a23b8769-d871-4d44-8cea-5d899f514579", "address": "fa:16:3e:7f:17:4d", "network": {"id": "5bbe9946-1c42-4c9e-8cf7-5ee0a54ebc8d", "bridge": "br-int", "label": "tempest-AttachSCSIVolumeTestJSON-411635514-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "df2e94062c154555ab1ee0c3bf8e0f84", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapa23b8769-d8", "ovs_interfaceid": "a23b8769-d871-4d44-8cea-5d899f514579", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Nov 29 03:06:33 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-a745e7d4-15ea-45ba-a537-b658a23b24ed tempest-AttachSCSIVolumeTestJSON-719577128 tempest-AttachSCSIVolumeTestJSON-719577128-project-member] Converting VIF {"id": "a23b8769-d871-4d44-8cea-5d899f514579", "address": "fa:16:3e:7f:17:4d", "network": {"id": "5bbe9946-1c42-4c9e-8cf7-5ee0a54ebc8d", "bridge": "br-int", "label": "tempest-AttachSCSIVolumeTestJSON-411635514-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "df2e94062c154555ab1ee0c3bf8e0f84", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapa23b8769-d8", "ovs_interfaceid": "a23b8769-d871-4d44-8cea-5d899f514579", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:06:33 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-a745e7d4-15ea-45ba-a537-b658a23b24ed tempest-AttachSCSIVolumeTestJSON-719577128 tempest-AttachSCSIVolumeTestJSON-719577128-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:7f:17:4d,bridge_name='br-int',has_traffic_filtering=True,id=a23b8769-d871-4d44-8cea-5d899f514579,network=Network(5bbe9946-1c42-4c9e-8cf7-5ee0a54ebc8d),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapa23b8769-d8') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:06:33 user nova-compute[75234]: DEBUG os_vif [None req-a745e7d4-15ea-45ba-a537-b658a23b24ed tempest-AttachSCSIVolumeTestJSON-719577128 tempest-AttachSCSIVolumeTestJSON-719577128-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:7f:17:4d,bridge_name='br-int',has_traffic_filtering=True,id=a23b8769-d871-4d44-8cea-5d899f514579,network=Network(5bbe9946-1c42-4c9e-8cf7-5ee0a54ebc8d),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapa23b8769-d8') {{(pid=75234) plug /opt/stack/data/venv/lib/python3.10/site-packages/os_vif/__init__.py:76}}
Nov 29 03:06:33 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:06:33 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:06:33 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Nov 29 03:06:33 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:06:33 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbCreateCommand(_result=None, table=QoS, columns={'type': 'linux-noop', 'external_ids': {'id': 'eb0d96a3-55ae-5c93-85c8-86cd9eeb8fcf', '_type': 'linux-noop'}}, row=False) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:06:33 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:06:33 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:06:33 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:06:33 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tapa23b8769-d8, may_exist=True, interface_attrs={}) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:06:33 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Port, record=tapa23b8769-d8, col_values=(('qos', UUID('df5d30c3-fd7c-45e1-91d0-c284d514245e')),), if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:06:33 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DbSetCommand(_result=None, table=Interface, record=tapa23b8769-d8, col_values=(('external_ids', {'iface-id': 'a23b8769-d871-4d44-8cea-5d899f514579', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:7f:17:4d', 'vm-uuid': 'da625f9c-b7ed-4afc-875c-6436bf58e9cc'}),), if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:06:33 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:06:33 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:06:33 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:06:33 user nova-compute[75234]: INFO os_vif [None req-a745e7d4-15ea-45ba-a537-b658a23b24ed tempest-AttachSCSIVolumeTestJSON-719577128 tempest-AttachSCSIVolumeTestJSON-719577128-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:7f:17:4d,bridge_name='br-int',has_traffic_filtering=True,id=a23b8769-d871-4d44-8cea-5d899f514579,network=Network(5bbe9946-1c42-4c9e-8cf7-5ee0a54ebc8d),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapa23b8769-d8')
Nov 29 03:06:33 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-a745e7d4-15ea-45ba-a537-b658a23b24ed tempest-AttachSCSIVolumeTestJSON-719577128 tempest-AttachSCSIVolumeTestJSON-719577128-project-member] No BDM found with device name sda, not building metadata. {{(pid=75234) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12089}}
Nov 29 03:06:33 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-a745e7d4-15ea-45ba-a537-b658a23b24ed tempest-AttachSCSIVolumeTestJSON-719577128 tempest-AttachSCSIVolumeTestJSON-719577128-project-member] No BDM found with device name sdb, not building metadata. {{(pid=75234) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12089}}
Nov 29 03:06:33 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-a745e7d4-15ea-45ba-a537-b658a23b24ed tempest-AttachSCSIVolumeTestJSON-719577128 tempest-AttachSCSIVolumeTestJSON-719577128-project-member] No VIF found with MAC fa:16:3e:7f:17:4d, not building metadata {{(pid=75234) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Nov 29 03:06:33 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-a745e7d4-15ea-45ba-a537-b658a23b24ed tempest-AttachSCSIVolumeTestJSON-719577128 tempest-AttachSCSIVolumeTestJSON-719577128-project-member] [instance: da625f9c-b7ed-4afc-875c-6436bf58e9cc] Using config drive
Nov 29 03:06:34 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-a745e7d4-15ea-45ba-a537-b658a23b24ed tempest-AttachSCSIVolumeTestJSON-719577128 tempest-AttachSCSIVolumeTestJSON-719577128-project-member] [instance: da625f9c-b7ed-4afc-875c-6436bf58e9cc] Creating config drive at /opt/stack/data/nova/instances/da625f9c-b7ed-4afc-875c-6436bf58e9cc/disk.config
Nov 29 03:06:34 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-a745e7d4-15ea-45ba-a537-b658a23b24ed tempest-AttachSCSIVolumeTestJSON-719577128 tempest-AttachSCSIVolumeTestJSON-719577128-project-member] Running cmd (subprocess): genisoimage -o /opt/stack/data/nova/instances/da625f9c-b7ed-4afc-875c-6436bf58e9cc/disk.config -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 0.0.0 -quiet -J -r -V config-2 /tmp/tmp7u3t1fxb {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:06:35 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-a745e7d4-15ea-45ba-a537-b658a23b24ed tempest-AttachSCSIVolumeTestJSON-719577128 tempest-AttachSCSIVolumeTestJSON-719577128-project-member] CMD "genisoimage -o /opt/stack/data/nova/instances/da625f9c-b7ed-4afc-875c-6436bf58e9cc/disk.config -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 0.0.0 -quiet -J -r -V config-2 /tmp/tmp7u3t1fxb" returned: 0 in 0.071s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:06:35 user nova-compute[75234]: DEBUG nova.network.neutron [None req-684945de-57fe-475b-b990-ae7897d5ffb4 tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] [instance: aa3c7432-bf15-408e-9fbb-993135734161] Successfully updated port: aaac2504-1067-43c5-b3bf-316996e0f56c {{(pid=75234) _update_port /opt/stack/nova/nova/network/neutron.py:586}}
Nov 29 03:06:35 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-684945de-57fe-475b-b990-ae7897d5ffb4 tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] Acquiring lock "refresh_cache-aa3c7432-bf15-408e-9fbb-993135734161" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:06:35 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-684945de-57fe-475b-b990-ae7897d5ffb4 tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] Acquired lock "refresh_cache-aa3c7432-bf15-408e-9fbb-993135734161" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:06:35 user nova-compute[75234]: DEBUG nova.network.neutron [None req-684945de-57fe-475b-b990-ae7897d5ffb4 tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] [instance: aa3c7432-bf15-408e-9fbb-993135734161] Building network info cache for instance {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2009}}
Nov 29 03:06:35 user nova-compute[75234]: DEBUG nova.compute.manager [req-08fc98d2-a314-41e2-aa34-d71928fff2bf req-29fd0052-e5cb-42e0-a90f-47c80031e585 service nova] [instance: aa3c7432-bf15-408e-9fbb-993135734161] Received event network-changed-aaac2504-1067-43c5-b3bf-316996e0f56c {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:06:35 user nova-compute[75234]: DEBUG nova.compute.manager [req-08fc98d2-a314-41e2-aa34-d71928fff2bf req-29fd0052-e5cb-42e0-a90f-47c80031e585 service nova] [instance: aa3c7432-bf15-408e-9fbb-993135734161] Refreshing instance network info cache due to event network-changed-aaac2504-1067-43c5-b3bf-316996e0f56c. {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11117}}
Nov 29 03:06:35 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-08fc98d2-a314-41e2-aa34-d71928fff2bf req-29fd0052-e5cb-42e0-a90f-47c80031e585 service nova] Acquiring lock "refresh_cache-aa3c7432-bf15-408e-9fbb-993135734161" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:06:35 user nova-compute[75234]: DEBUG nova.network.neutron [None req-684945de-57fe-475b-b990-ae7897d5ffb4 tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] [instance: aa3c7432-bf15-408e-9fbb-993135734161] Instance cache missing network info. {{(pid=75234) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3322}}
Nov 29 03:06:35 user nova-compute[75234]: DEBUG nova.network.neutron [None req-684945de-57fe-475b-b990-ae7897d5ffb4 tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] [instance: aa3c7432-bf15-408e-9fbb-993135734161] Updating instance_info_cache with network_info: [{"id": "aaac2504-1067-43c5-b3bf-316996e0f56c", "address": "fa:16:3e:2a:34:9b", "network": {"id": "68a1f282-91d0-47c8-82f3-5ebf1c0f572c", "bridge": "br-int", "label": "tempest-VolumesActionsTest-570720740-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "46577f787c244fb2a4db052a0c42eceb", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapaaac2504-10", "ovs_interfaceid": "aaac2504-1067-43c5-b3bf-316996e0f56c", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:06:35 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-684945de-57fe-475b-b990-ae7897d5ffb4 tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] Releasing lock "refresh_cache-aa3c7432-bf15-408e-9fbb-993135734161" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:06:35 user nova-compute[75234]: DEBUG nova.compute.manager [None req-684945de-57fe-475b-b990-ae7897d5ffb4 tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] [instance: aa3c7432-bf15-408e-9fbb-993135734161] Instance network_info: |[{"id": "aaac2504-1067-43c5-b3bf-316996e0f56c", "address": "fa:16:3e:2a:34:9b", "network": {"id": "68a1f282-91d0-47c8-82f3-5ebf1c0f572c", "bridge": "br-int", "label": "tempest-VolumesActionsTest-570720740-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "46577f787c244fb2a4db052a0c42eceb", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapaaac2504-10", "ovs_interfaceid": "aaac2504-1067-43c5-b3bf-316996e0f56c", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=75234) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1987}}
Nov 29 03:06:35 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-08fc98d2-a314-41e2-aa34-d71928fff2bf req-29fd0052-e5cb-42e0-a90f-47c80031e585 service nova] Acquired lock "refresh_cache-aa3c7432-bf15-408e-9fbb-993135734161" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:06:35 user nova-compute[75234]: DEBUG nova.network.neutron [req-08fc98d2-a314-41e2-aa34-d71928fff2bf req-29fd0052-e5cb-42e0-a90f-47c80031e585 service nova] [instance: aa3c7432-bf15-408e-9fbb-993135734161] Refreshing network info cache for port aaac2504-1067-43c5-b3bf-316996e0f56c {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2006}}
Nov 29 03:06:35 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-684945de-57fe-475b-b990-ae7897d5ffb4 tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] [instance: aa3c7432-bf15-408e-9fbb-993135734161] Start _get_guest_xml network_info=[{"id": "aaac2504-1067-43c5-b3bf-316996e0f56c", "address": "fa:16:3e:2a:34:9b", "network": {"id": "68a1f282-91d0-47c8-82f3-5ebf1c0f572c", "bridge": "br-int", "label": "tempest-VolumesActionsTest-570720740-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "46577f787c244fb2a4db052a0c42eceb", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapaaac2504-10", "ovs_interfaceid": "aaac2504-1067-43c5-b3bf-316996e0f56c", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum='c8fc807773e5354afe61636071771906',container_format='bare',created_at=2023-11-29T03:02:15Z,direct_url=<?>,disk_format='qcow2',id=96c30ba7-9b73-4a30-8ba4-7138879573b7,min_disk=0,min_ram=0,name='cirros-0.6.2-x86_64-disk',owner='b18e380921c748b78517b6f1211cd0f1',properties=ImageMetaProps,protected=<?>,size=21430272,status='active',tags=<?>,updated_at=2023-11-29T03:02:17Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [{'device_name': '/dev/vda', 'encryption_format': None, 'encrypted': False, 'device_type': 'disk', 'size': 0, 'disk_bus': 'virtio', 'boot_index': 0, 'encryption_options': None, 'encryption_secret_uuid': None, 'guest_format': None, 'image_id': '96c30ba7-9b73-4a30-8ba4-7138879573b7'}], 'ephemerals': [], 'block_device_mapping': [], 'swap': None} {{(pid=75234) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7562}}
Nov 29 03:06:35 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-684945de-57fe-475b-b990-ae7897d5ffb4 tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:06:35 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-684945de-57fe-475b-b990-ae7897d5ffb4 tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:06:35 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-684945de-57fe-475b-b990-ae7897d5ffb4 tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] Searching host: 'user' for CPU controller through CGroups V1... {{(pid=75234) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1652}}
Nov 29 03:06:35 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-684945de-57fe-475b-b990-ae7897d5ffb4 tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] CPU controller missing on host. {{(pid=75234) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1662}}
Nov 29 03:06:35 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-684945de-57fe-475b-b990-ae7897d5ffb4 tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] Searching host: 'user' for CPU controller through CGroups V2... {{(pid=75234) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1671}}
Nov 29 03:06:35 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-684945de-57fe-475b-b990-ae7897d5ffb4 tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] CPU controller found on host. {{(pid=75234) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1678}}
Nov 29 03:06:35 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-684945de-57fe-475b-b990-ae7897d5ffb4 tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=75234) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5400}}
Nov 29 03:06:35 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-684945de-57fe-475b-b990-ae7897d5ffb4 tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-11-29T03:02:04Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='1',id=6,is_public=True,memory_mb=512,name='m1.tiny',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='c8fc807773e5354afe61636071771906',container_format='bare',created_at=2023-11-29T03:02:15Z,direct_url=<?>,disk_format='qcow2',id=96c30ba7-9b73-4a30-8ba4-7138879573b7,min_disk=0,min_ram=0,name='cirros-0.6.2-x86_64-disk',owner='b18e380921c748b78517b6f1211cd0f1',properties=ImageMetaProps,protected=<?>,size=21430272,status='active',tags=<?>,updated_at=2023-11-29T03:02:17Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:563}}
Nov 29 03:06:35 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-684945de-57fe-475b-b990-ae7897d5ffb4 tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] Flavor limits 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:348}}
Nov 29 03:06:35 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-684945de-57fe-475b-b990-ae7897d5ffb4 tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] Image limits 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:352}}
Nov 29 03:06:35 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-684945de-57fe-475b-b990-ae7897d5ffb4 tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] Flavor pref 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:388}}
Nov 29 03:06:35 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-684945de-57fe-475b-b990-ae7897d5ffb4 tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] Image pref 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:392}}
Nov 29 03:06:35 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-684945de-57fe-475b-b990-ae7897d5ffb4 tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:430}}
Nov 29 03:06:35 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-684945de-57fe-475b-b990-ae7897d5ffb4 tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:569}}
Nov 29 03:06:35 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-684945de-57fe-475b-b990-ae7897d5ffb4 tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=75234) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:471}}
Nov 29 03:06:35 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-684945de-57fe-475b-b990-ae7897d5ffb4 tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] Got 1 possible topologies {{(pid=75234) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:501}}
Nov 29 03:06:35 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-684945de-57fe-475b-b990-ae7897d5ffb4 tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:575}}
Nov 29 03:06:35 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-684945de-57fe-475b-b990-ae7897d5ffb4 tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:577}}
Nov 29 03:06:35 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-684945de-57fe-475b-b990-ae7897d5ffb4 tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-11-29T03:06:28Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-VolumesActionsTest-instance-958290084',display_name='tempest-VolumesActionsTest-instance-958290084',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-volumesactionstest-instance-958290084',id=8,image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data=None,key_name=None,keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='46577f787c244fb2a4db052a0c42eceb',ramdisk_id='',reservation_id='r-xwyt8hlf',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-VolumesActionsTest-196322865',owner_user_name='tempest-VolumesActionsTest-196322865-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-11-29T03:06:32Z,user_data=None,user_id='940a684aa3b6419798bd87562bbc5d23',uuid=aa3c7432-bf15-408e-9fbb-993135734161,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "aaac2504-1067-43c5-b3bf-316996e0f56c", "address": "fa:16:3e:2a:34:9b", "network": {"id": "68a1f282-91d0-47c8-82f3-5ebf1c0f572c", "bridge": "br-int", "label": "tempest-VolumesActionsTest-570720740-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "46577f787c244fb2a4db052a0c42eceb", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapaaac2504-10", "ovs_interfaceid": "aaac2504-1067-43c5-b3bf-316996e0f56c", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=75234) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Nov 29 03:06:35 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-684945de-57fe-475b-b990-ae7897d5ffb4 tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] Converting VIF {"id": "aaac2504-1067-43c5-b3bf-316996e0f56c", "address": "fa:16:3e:2a:34:9b", "network": {"id": "68a1f282-91d0-47c8-82f3-5ebf1c0f572c", "bridge": "br-int", "label": "tempest-VolumesActionsTest-570720740-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "46577f787c244fb2a4db052a0c42eceb", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapaaac2504-10", "ovs_interfaceid": "aaac2504-1067-43c5-b3bf-316996e0f56c", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:06:35 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-684945de-57fe-475b-b990-ae7897d5ffb4 tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:2a:34:9b,bridge_name='br-int',has_traffic_filtering=True,id=aaac2504-1067-43c5-b3bf-316996e0f56c,network=Network(68a1f282-91d0-47c8-82f3-5ebf1c0f572c),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapaaac2504-10') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:06:35 user nova-compute[75234]: DEBUG nova.objects.instance [None req-684945de-57fe-475b-b990-ae7897d5ffb4 tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] Lazy-loading 'pci_devices' on Instance uuid aa3c7432-bf15-408e-9fbb-993135734161 {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:06:35 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-40dafa9d-38c5-48f3-acf9-e33e8af5728b tempest-VolumesAssistedSnapshotsTest-1323460801 tempest-VolumesAssistedSnapshotsTest-1323460801-project-member] Acquiring lock "c9663ab3-3374-400e-ab0a-769731b78cca" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:06:35 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-40dafa9d-38c5-48f3-acf9-e33e8af5728b tempest-VolumesAssistedSnapshotsTest-1323460801 tempest-VolumesAssistedSnapshotsTest-1323460801-project-member] Lock "c9663ab3-3374-400e-ab0a-769731b78cca" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:06:35 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-684945de-57fe-475b-b990-ae7897d5ffb4 tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] [instance: aa3c7432-bf15-408e-9fbb-993135734161] End _get_guest_xml xml=<domain type="kvm">
Nov 29 03:06:35 user nova-compute[75234]:   <uuid>aa3c7432-bf15-408e-9fbb-993135734161</uuid>
Nov 29 03:06:35 user nova-compute[75234]:   <name>instance-00000008</name>
Nov 29 03:06:35 user nova-compute[75234]:   <memory>524288</memory>
Nov 29 03:06:35 user nova-compute[75234]:   <vcpu>1</vcpu>
Nov 29 03:06:35 user nova-compute[75234]:   <metadata>
Nov 29 03:06:35 user nova-compute[75234]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Nov 29 03:06:35 user nova-compute[75234]:       <nova:package version="0.0.0"/>
Nov 29 03:06:35 user nova-compute[75234]:       <nova:name>tempest-VolumesActionsTest-instance-958290084</nova:name>
Nov 29 03:06:35 user nova-compute[75234]:       <nova:creationTime>2023-11-29 03:06:35</nova:creationTime>
Nov 29 03:06:35 user nova-compute[75234]:       <nova:flavor name="m1.tiny">
Nov 29 03:06:35 user nova-compute[75234]:         <nova:memory>512</nova:memory>
Nov 29 03:06:35 user nova-compute[75234]:         <nova:disk>1</nova:disk>
Nov 29 03:06:35 user nova-compute[75234]:         <nova:swap>0</nova:swap>
Nov 29 03:06:35 user nova-compute[75234]:         <nova:ephemeral>0</nova:ephemeral>
Nov 29 03:06:35 user nova-compute[75234]:         <nova:vcpus>1</nova:vcpus>
Nov 29 03:06:35 user nova-compute[75234]:       </nova:flavor>
Nov 29 03:06:35 user nova-compute[75234]:       <nova:owner>
Nov 29 03:06:35 user nova-compute[75234]:         <nova:user uuid="940a684aa3b6419798bd87562bbc5d23">tempest-VolumesActionsTest-196322865-project-member</nova:user>
Nov 29 03:06:35 user nova-compute[75234]:         <nova:project uuid="46577f787c244fb2a4db052a0c42eceb">tempest-VolumesActionsTest-196322865</nova:project>
Nov 29 03:06:35 user nova-compute[75234]:       </nova:owner>
Nov 29 03:06:35 user nova-compute[75234]:       <nova:root type="image" uuid="96c30ba7-9b73-4a30-8ba4-7138879573b7"/>
Nov 29 03:06:35 user nova-compute[75234]:       <nova:ports>
Nov 29 03:06:35 user nova-compute[75234]:         <nova:port uuid="aaac2504-1067-43c5-b3bf-316996e0f56c">
Nov 29 03:06:35 user nova-compute[75234]:           <nova:ip type="fixed" address="10.0.0.4" ipVersion="4"/>
Nov 29 03:06:35 user nova-compute[75234]:         </nova:port>
Nov 29 03:06:35 user nova-compute[75234]:       </nova:ports>
Nov 29 03:06:35 user nova-compute[75234]:     </nova:instance>
Nov 29 03:06:35 user nova-compute[75234]:   </metadata>
Nov 29 03:06:35 user nova-compute[75234]:   <sysinfo type="smbios">
Nov 29 03:06:35 user nova-compute[75234]:     <system>
Nov 29 03:06:35 user nova-compute[75234]:       <entry name="manufacturer">OpenStack Foundation</entry>
Nov 29 03:06:35 user nova-compute[75234]:       <entry name="product">OpenStack Nova</entry>
Nov 29 03:06:35 user nova-compute[75234]:       <entry name="version">0.0.0</entry>
Nov 29 03:06:35 user nova-compute[75234]:       <entry name="serial">aa3c7432-bf15-408e-9fbb-993135734161</entry>
Nov 29 03:06:35 user nova-compute[75234]:       <entry name="uuid">aa3c7432-bf15-408e-9fbb-993135734161</entry>
Nov 29 03:06:35 user nova-compute[75234]:       <entry name="family">Virtual Machine</entry>
Nov 29 03:06:35 user nova-compute[75234]:     </system>
Nov 29 03:06:35 user nova-compute[75234]:   </sysinfo>
Nov 29 03:06:35 user nova-compute[75234]:   <os>
Nov 29 03:06:35 user nova-compute[75234]:     <type arch="x86_64" machine="pc">hvm</type>
Nov 29 03:06:35 user nova-compute[75234]:     <boot dev="hd"/>
Nov 29 03:06:35 user nova-compute[75234]:     <smbios mode="sysinfo"/>
Nov 29 03:06:35 user nova-compute[75234]:   </os>
Nov 29 03:06:35 user nova-compute[75234]:   <features>
Nov 29 03:06:35 user nova-compute[75234]:     <acpi/>
Nov 29 03:06:35 user nova-compute[75234]:     <apic/>
Nov 29 03:06:35 user nova-compute[75234]:     <vmcoreinfo/>
Nov 29 03:06:35 user nova-compute[75234]:   </features>
Nov 29 03:06:35 user nova-compute[75234]:   <clock offset="utc">
Nov 29 03:06:35 user nova-compute[75234]:     <timer name="pit" tickpolicy="delay"/>
Nov 29 03:06:35 user nova-compute[75234]:     <timer name="rtc" tickpolicy="catchup"/>
Nov 29 03:06:35 user nova-compute[75234]:     <timer name="hpet" present="no"/>
Nov 29 03:06:35 user nova-compute[75234]:   </clock>
Nov 29 03:06:35 user nova-compute[75234]:   <cpu mode="custom" match="exact">
Nov 29 03:06:35 user nova-compute[75234]:     <model>Nehalem</model>
Nov 29 03:06:35 user nova-compute[75234]:     <topology sockets="1" cores="1" threads="1"/>
Nov 29 03:06:35 user nova-compute[75234]:   </cpu>
Nov 29 03:06:35 user nova-compute[75234]:   <devices>
Nov 29 03:06:35 user nova-compute[75234]:     <disk type="file" device="disk">
Nov 29 03:06:35 user nova-compute[75234]:       <driver name="qemu" type="qcow2" cache="none"/>
Nov 29 03:06:35 user nova-compute[75234]:       <source file="/opt/stack/data/nova/instances/aa3c7432-bf15-408e-9fbb-993135734161/disk"/>
Nov 29 03:06:35 user nova-compute[75234]:       <target dev="vda" bus="virtio"/>
Nov 29 03:06:35 user nova-compute[75234]:     </disk>
Nov 29 03:06:35 user nova-compute[75234]:     <interface type="ethernet">
Nov 29 03:06:35 user nova-compute[75234]:       <mac address="fa:16:3e:2a:34:9b"/>
Nov 29 03:06:35 user nova-compute[75234]:       <model type="virtio"/>
Nov 29 03:06:35 user nova-compute[75234]:       <mtu size="1442"/>
Nov 29 03:06:35 user nova-compute[75234]:       <target dev="tapaaac2504-10"/>
Nov 29 03:06:35 user nova-compute[75234]:     </interface>
Nov 29 03:06:35 user nova-compute[75234]:     <serial type="pty">
Nov 29 03:06:35 user nova-compute[75234]:       <log file="/opt/stack/data/nova/instances/aa3c7432-bf15-408e-9fbb-993135734161/console.log" append="off"/>
Nov 29 03:06:35 user nova-compute[75234]:     </serial>
Nov 29 03:06:35 user nova-compute[75234]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Nov 29 03:06:35 user nova-compute[75234]:     <video>
Nov 29 03:06:35 user nova-compute[75234]:       <model type="virtio"/>
Nov 29 03:06:35 user nova-compute[75234]:     </video>
Nov 29 03:06:35 user nova-compute[75234]:     <rng model="virtio">
Nov 29 03:06:35 user nova-compute[75234]:       <backend model="random">/dev/urandom</backend>
Nov 29 03:06:35 user nova-compute[75234]:     </rng>
Nov 29 03:06:35 user nova-compute[75234]:     <controller type="usb" index="0" model="none"/>
Nov 29 03:06:35 user nova-compute[75234]:     <memballoon model="virtio">
Nov 29 03:06:35 user nova-compute[75234]:       <stats period="10"/>
Nov 29 03:06:35 user nova-compute[75234]:     </memballoon>
Nov 29 03:06:35 user nova-compute[75234]:   </devices>
Nov 29 03:06:35 user nova-compute[75234]: </domain>
Nov 29 03:06:35 user nova-compute[75234]:  {{(pid=75234) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7568}}
Nov 29 03:06:35 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-684945de-57fe-475b-b990-ae7897d5ffb4 tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-11-29T03:06:28Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-VolumesActionsTest-instance-958290084',display_name='tempest-VolumesActionsTest-instance-958290084',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-volumesactionstest-instance-958290084',id=8,image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data=None,key_name=None,keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='46577f787c244fb2a4db052a0c42eceb',ramdisk_id='',reservation_id='r-xwyt8hlf',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-VolumesActionsTest-196322865',owner_user_name='tempest-VolumesActionsTest-196322865-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-11-29T03:06:32Z,user_data=None,user_id='940a684aa3b6419798bd87562bbc5d23',uuid=aa3c7432-bf15-408e-9fbb-993135734161,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "aaac2504-1067-43c5-b3bf-316996e0f56c", "address": "fa:16:3e:2a:34:9b", "network": {"id": "68a1f282-91d0-47c8-82f3-5ebf1c0f572c", "bridge": "br-int", "label": "tempest-VolumesActionsTest-570720740-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "46577f787c244fb2a4db052a0c42eceb", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapaaac2504-10", "ovs_interfaceid": "aaac2504-1067-43c5-b3bf-316996e0f56c", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Nov 29 03:06:35 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-684945de-57fe-475b-b990-ae7897d5ffb4 tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] Converting VIF {"id": "aaac2504-1067-43c5-b3bf-316996e0f56c", "address": "fa:16:3e:2a:34:9b", "network": {"id": "68a1f282-91d0-47c8-82f3-5ebf1c0f572c", "bridge": "br-int", "label": "tempest-VolumesActionsTest-570720740-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "46577f787c244fb2a4db052a0c42eceb", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapaaac2504-10", "ovs_interfaceid": "aaac2504-1067-43c5-b3bf-316996e0f56c", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:06:35 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-684945de-57fe-475b-b990-ae7897d5ffb4 tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:2a:34:9b,bridge_name='br-int',has_traffic_filtering=True,id=aaac2504-1067-43c5-b3bf-316996e0f56c,network=Network(68a1f282-91d0-47c8-82f3-5ebf1c0f572c),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapaaac2504-10') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:06:35 user nova-compute[75234]: DEBUG os_vif [None req-684945de-57fe-475b-b990-ae7897d5ffb4 tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:2a:34:9b,bridge_name='br-int',has_traffic_filtering=True,id=aaac2504-1067-43c5-b3bf-316996e0f56c,network=Network(68a1f282-91d0-47c8-82f3-5ebf1c0f572c),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapaaac2504-10') {{(pid=75234) plug /opt/stack/data/venv/lib/python3.10/site-packages/os_vif/__init__.py:76}}
Nov 29 03:06:35 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:06:35 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:06:35 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Nov 29 03:06:36 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:06:36 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbCreateCommand(_result=None, table=QoS, columns={'type': 'linux-noop', 'external_ids': {'id': 'ffded63a-22a0-53e9-ade6-9cf2fbdcb997', '_type': 'linux-noop'}}, row=False) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:06:36 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:06:36 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:06:36 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:06:36 user nova-compute[75234]: DEBUG nova.compute.manager [None req-40dafa9d-38c5-48f3-acf9-e33e8af5728b tempest-VolumesAssistedSnapshotsTest-1323460801 tempest-VolumesAssistedSnapshotsTest-1323460801-project-member] [instance: c9663ab3-3374-400e-ab0a-769731b78cca] Starting instance... {{(pid=75234) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2422}}
Nov 29 03:06:36 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:06:36 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tapaaac2504-10, may_exist=True, interface_attrs={}) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:06:36 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Port, record=tapaaac2504-10, col_values=(('qos', UUID('4f364bff-bc4b-43d9-b56e-5597796afeb0')),), if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:06:36 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DbSetCommand(_result=None, table=Interface, record=tapaaac2504-10, col_values=(('external_ids', {'iface-id': 'aaac2504-1067-43c5-b3bf-316996e0f56c', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:2a:34:9b', 'vm-uuid': 'aa3c7432-bf15-408e-9fbb-993135734161'}),), if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:06:36 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:06:36 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:06:36 user nova-compute[75234]: INFO os_vif [None req-684945de-57fe-475b-b990-ae7897d5ffb4 tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:2a:34:9b,bridge_name='br-int',has_traffic_filtering=True,id=aaac2504-1067-43c5-b3bf-316996e0f56c,network=Network(68a1f282-91d0-47c8-82f3-5ebf1c0f572c),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapaaac2504-10')
Nov 29 03:06:36 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-684945de-57fe-475b-b990-ae7897d5ffb4 tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] No BDM found with device name vda, not building metadata. {{(pid=75234) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12089}}
Nov 29 03:06:36 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-684945de-57fe-475b-b990-ae7897d5ffb4 tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] No VIF found with MAC fa:16:3e:2a:34:9b, not building metadata {{(pid=75234) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Nov 29 03:06:36 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-40dafa9d-38c5-48f3-acf9-e33e8af5728b tempest-VolumesAssistedSnapshotsTest-1323460801 tempest-VolumesAssistedSnapshotsTest-1323460801-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:06:36 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-40dafa9d-38c5-48f3-acf9-e33e8af5728b tempest-VolumesAssistedSnapshotsTest-1323460801 tempest-VolumesAssistedSnapshotsTest-1323460801-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:06:36 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-40dafa9d-38c5-48f3-acf9-e33e8af5728b tempest-VolumesAssistedSnapshotsTest-1323460801 tempest-VolumesAssistedSnapshotsTest-1323460801-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=75234) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2366}}
Nov 29 03:06:36 user nova-compute[75234]: INFO nova.compute.claims [None req-40dafa9d-38c5-48f3-acf9-e33e8af5728b tempest-VolumesAssistedSnapshotsTest-1323460801 tempest-VolumesAssistedSnapshotsTest-1323460801-project-member] [instance: c9663ab3-3374-400e-ab0a-769731b78cca] Claim successful on node user
Nov 29 03:06:36 user nova-compute[75234]: DEBUG nova.network.neutron [req-08fc98d2-a314-41e2-aa34-d71928fff2bf req-29fd0052-e5cb-42e0-a90f-47c80031e585 service nova] [instance: aa3c7432-bf15-408e-9fbb-993135734161] Updated VIF entry in instance network info cache for port aaac2504-1067-43c5-b3bf-316996e0f56c. {{(pid=75234) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3481}}
Nov 29 03:06:36 user nova-compute[75234]: DEBUG nova.network.neutron [req-08fc98d2-a314-41e2-aa34-d71928fff2bf req-29fd0052-e5cb-42e0-a90f-47c80031e585 service nova] [instance: aa3c7432-bf15-408e-9fbb-993135734161] Updating instance_info_cache with network_info: [{"id": "aaac2504-1067-43c5-b3bf-316996e0f56c", "address": "fa:16:3e:2a:34:9b", "network": {"id": "68a1f282-91d0-47c8-82f3-5ebf1c0f572c", "bridge": "br-int", "label": "tempest-VolumesActionsTest-570720740-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "46577f787c244fb2a4db052a0c42eceb", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapaaac2504-10", "ovs_interfaceid": "aaac2504-1067-43c5-b3bf-316996e0f56c", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:06:36 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-08fc98d2-a314-41e2-aa34-d71928fff2bf req-29fd0052-e5cb-42e0-a90f-47c80031e585 service nova] Releasing lock "refresh_cache-aa3c7432-bf15-408e-9fbb-993135734161" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:06:36 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:06:36 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:06:36 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:06:36 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:06:36 user nova-compute[75234]: DEBUG nova.compute.provider_tree [None req-40dafa9d-38c5-48f3-acf9-e33e8af5728b tempest-VolumesAssistedSnapshotsTest-1323460801 tempest-VolumesAssistedSnapshotsTest-1323460801-project-member] Inventory has not changed in ProviderTree for provider: 4d291220-7c37-4839-a0e3-b23c66502b74 {{(pid=75234) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Nov 29 03:06:36 user nova-compute[75234]: DEBUG nova.scheduler.client.report [None req-40dafa9d-38c5-48f3-acf9-e33e8af5728b tempest-VolumesAssistedSnapshotsTest-1323460801 tempest-VolumesAssistedSnapshotsTest-1323460801-project-member] Inventory has not changed for provider 4d291220-7c37-4839-a0e3-b23c66502b74 based on inventory data: {'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=75234) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Nov 29 03:06:37 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-40dafa9d-38c5-48f3-acf9-e33e8af5728b tempest-VolumesAssistedSnapshotsTest-1323460801 tempest-VolumesAssistedSnapshotsTest-1323460801-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.763s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:06:37 user nova-compute[75234]: DEBUG nova.compute.manager [None req-40dafa9d-38c5-48f3-acf9-e33e8af5728b tempest-VolumesAssistedSnapshotsTest-1323460801 tempest-VolumesAssistedSnapshotsTest-1323460801-project-member] [instance: c9663ab3-3374-400e-ab0a-769731b78cca] Start building networks asynchronously for instance. {{(pid=75234) _build_resources /opt/stack/nova/nova/compute/manager.py:2819}}
Nov 29 03:06:37 user nova-compute[75234]: DEBUG nova.compute.manager [None req-40dafa9d-38c5-48f3-acf9-e33e8af5728b tempest-VolumesAssistedSnapshotsTest-1323460801 tempest-VolumesAssistedSnapshotsTest-1323460801-project-member] [instance: c9663ab3-3374-400e-ab0a-769731b78cca] Allocating IP information in the background. {{(pid=75234) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1972}}
Nov 29 03:06:37 user nova-compute[75234]: DEBUG nova.network.neutron [None req-40dafa9d-38c5-48f3-acf9-e33e8af5728b tempest-VolumesAssistedSnapshotsTest-1323460801 tempest-VolumesAssistedSnapshotsTest-1323460801-project-member] [instance: c9663ab3-3374-400e-ab0a-769731b78cca] allocate_for_instance() {{(pid=75234) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1156}}
Nov 29 03:06:37 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-40dafa9d-38c5-48f3-acf9-e33e8af5728b tempest-VolumesAssistedSnapshotsTest-1323460801 tempest-VolumesAssistedSnapshotsTest-1323460801-project-member] [instance: c9663ab3-3374-400e-ab0a-769731b78cca] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Nov 29 03:06:37 user nova-compute[75234]: DEBUG nova.compute.manager [None req-40dafa9d-38c5-48f3-acf9-e33e8af5728b tempest-VolumesAssistedSnapshotsTest-1323460801 tempest-VolumesAssistedSnapshotsTest-1323460801-project-member] [instance: c9663ab3-3374-400e-ab0a-769731b78cca] Start building block device mappings for instance. {{(pid=75234) _build_resources /opt/stack/nova/nova/compute/manager.py:2854}}
Nov 29 03:06:37 user nova-compute[75234]: DEBUG nova.policy [None req-40dafa9d-38c5-48f3-acf9-e33e8af5728b tempest-VolumesAssistedSnapshotsTest-1323460801 tempest-VolumesAssistedSnapshotsTest-1323460801-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': 'f90b22e80de849c3b1ca951f7650fe84', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd59c913723894ba1938fb8177c97e627', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=75234) authorize /opt/stack/nova/nova/policy.py:203}}
Nov 29 03:06:37 user nova-compute[75234]: DEBUG nova.compute.manager [req-d2241f6d-bd47-4bc9-8269-9b0c2e27a0ef req-f1d12653-5d75-474c-8eec-d94c58715ec0 service nova] [instance: 9f7876ad-0487-426d-8b19-b718dccdc4fe] Received event network-vif-plugged-7dd048ce-2c9b-46ac-8e9d-8c6a2ab201f5 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:06:37 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-d2241f6d-bd47-4bc9-8269-9b0c2e27a0ef req-f1d12653-5d75-474c-8eec-d94c58715ec0 service nova] Acquiring lock "9f7876ad-0487-426d-8b19-b718dccdc4fe-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:06:37 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-d2241f6d-bd47-4bc9-8269-9b0c2e27a0ef req-f1d12653-5d75-474c-8eec-d94c58715ec0 service nova] Lock "9f7876ad-0487-426d-8b19-b718dccdc4fe-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:06:37 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-d2241f6d-bd47-4bc9-8269-9b0c2e27a0ef req-f1d12653-5d75-474c-8eec-d94c58715ec0 service nova] Lock "9f7876ad-0487-426d-8b19-b718dccdc4fe-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:06:37 user nova-compute[75234]: DEBUG nova.compute.manager [req-d2241f6d-bd47-4bc9-8269-9b0c2e27a0ef req-f1d12653-5d75-474c-8eec-d94c58715ec0 service nova] [instance: 9f7876ad-0487-426d-8b19-b718dccdc4fe] No waiting events found dispatching network-vif-plugged-7dd048ce-2c9b-46ac-8e9d-8c6a2ab201f5 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:06:37 user nova-compute[75234]: WARNING nova.compute.manager [req-d2241f6d-bd47-4bc9-8269-9b0c2e27a0ef req-f1d12653-5d75-474c-8eec-d94c58715ec0 service nova] [instance: 9f7876ad-0487-426d-8b19-b718dccdc4fe] Received unexpected event network-vif-plugged-7dd048ce-2c9b-46ac-8e9d-8c6a2ab201f5 for instance with vm_state active and task_state None.
Nov 29 03:06:37 user nova-compute[75234]: DEBUG nova.compute.manager [req-d2241f6d-bd47-4bc9-8269-9b0c2e27a0ef req-f1d12653-5d75-474c-8eec-d94c58715ec0 service nova] [instance: 9f7876ad-0487-426d-8b19-b718dccdc4fe] Received event network-vif-plugged-7dd048ce-2c9b-46ac-8e9d-8c6a2ab201f5 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:06:37 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-d2241f6d-bd47-4bc9-8269-9b0c2e27a0ef req-f1d12653-5d75-474c-8eec-d94c58715ec0 service nova] Acquiring lock "9f7876ad-0487-426d-8b19-b718dccdc4fe-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:06:37 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-d2241f6d-bd47-4bc9-8269-9b0c2e27a0ef req-f1d12653-5d75-474c-8eec-d94c58715ec0 service nova] Lock "9f7876ad-0487-426d-8b19-b718dccdc4fe-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:06:37 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-d2241f6d-bd47-4bc9-8269-9b0c2e27a0ef req-f1d12653-5d75-474c-8eec-d94c58715ec0 service nova] Lock "9f7876ad-0487-426d-8b19-b718dccdc4fe-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:06:37 user nova-compute[75234]: DEBUG nova.compute.manager [req-d2241f6d-bd47-4bc9-8269-9b0c2e27a0ef req-f1d12653-5d75-474c-8eec-d94c58715ec0 service nova] [instance: 9f7876ad-0487-426d-8b19-b718dccdc4fe] No waiting events found dispatching network-vif-plugged-7dd048ce-2c9b-46ac-8e9d-8c6a2ab201f5 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:06:37 user nova-compute[75234]: WARNING nova.compute.manager [req-d2241f6d-bd47-4bc9-8269-9b0c2e27a0ef req-f1d12653-5d75-474c-8eec-d94c58715ec0 service nova] [instance: 9f7876ad-0487-426d-8b19-b718dccdc4fe] Received unexpected event network-vif-plugged-7dd048ce-2c9b-46ac-8e9d-8c6a2ab201f5 for instance with vm_state active and task_state None.
Nov 29 03:06:37 user nova-compute[75234]: DEBUG nova.compute.manager [None req-40dafa9d-38c5-48f3-acf9-e33e8af5728b tempest-VolumesAssistedSnapshotsTest-1323460801 tempest-VolumesAssistedSnapshotsTest-1323460801-project-member] [instance: c9663ab3-3374-400e-ab0a-769731b78cca] Start spawning the instance on the hypervisor. {{(pid=75234) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2628}}
Nov 29 03:06:37 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-40dafa9d-38c5-48f3-acf9-e33e8af5728b tempest-VolumesAssistedSnapshotsTest-1323460801 tempest-VolumesAssistedSnapshotsTest-1323460801-project-member] [instance: c9663ab3-3374-400e-ab0a-769731b78cca] Creating instance directory {{(pid=75234) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4727}}
Nov 29 03:06:37 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-40dafa9d-38c5-48f3-acf9-e33e8af5728b tempest-VolumesAssistedSnapshotsTest-1323460801 tempest-VolumesAssistedSnapshotsTest-1323460801-project-member] [instance: c9663ab3-3374-400e-ab0a-769731b78cca] Creating image(s)
Nov 29 03:06:37 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-40dafa9d-38c5-48f3-acf9-e33e8af5728b tempest-VolumesAssistedSnapshotsTest-1323460801 tempest-VolumesAssistedSnapshotsTest-1323460801-project-member] Acquiring lock "/opt/stack/data/nova/instances/c9663ab3-3374-400e-ab0a-769731b78cca/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:06:37 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-40dafa9d-38c5-48f3-acf9-e33e8af5728b tempest-VolumesAssistedSnapshotsTest-1323460801 tempest-VolumesAssistedSnapshotsTest-1323460801-project-member] Lock "/opt/stack/data/nova/instances/c9663ab3-3374-400e-ab0a-769731b78cca/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:06:37 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-40dafa9d-38c5-48f3-acf9-e33e8af5728b tempest-VolumesAssistedSnapshotsTest-1323460801 tempest-VolumesAssistedSnapshotsTest-1323460801-project-member] Lock "/opt/stack/data/nova/instances/c9663ab3-3374-400e-ab0a-769731b78cca/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.003s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:06:37 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-40dafa9d-38c5-48f3-acf9-e33e8af5728b tempest-VolumesAssistedSnapshotsTest-1323460801 tempest-VolumesAssistedSnapshotsTest-1323460801-project-member] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:06:37 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:06:37 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:06:37 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:06:37 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:06:37 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:06:37 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-40dafa9d-38c5-48f3-acf9-e33e8af5728b tempest-VolumesAssistedSnapshotsTest-1323460801 tempest-VolumesAssistedSnapshotsTest-1323460801-project-member] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json" returned: 0 in 0.219s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:06:37 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-40dafa9d-38c5-48f3-acf9-e33e8af5728b tempest-VolumesAssistedSnapshotsTest-1323460801 tempest-VolumesAssistedSnapshotsTest-1323460801-project-member] Acquiring lock "80686e29dc9db6600d03e85cb62af298e12d7918" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:06:37 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-40dafa9d-38c5-48f3-acf9-e33e8af5728b tempest-VolumesAssistedSnapshotsTest-1323460801 tempest-VolumesAssistedSnapshotsTest-1323460801-project-member] Lock "80686e29dc9db6600d03e85cb62af298e12d7918" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.002s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:06:37 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-40dafa9d-38c5-48f3-acf9-e33e8af5728b tempest-VolumesAssistedSnapshotsTest-1323460801 tempest-VolumesAssistedSnapshotsTest-1323460801-project-member] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:06:38 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-40dafa9d-38c5-48f3-acf9-e33e8af5728b tempest-VolumesAssistedSnapshotsTest-1323460801 tempest-VolumesAssistedSnapshotsTest-1323460801-project-member] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json" returned: 0 in 0.238s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:06:38 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-40dafa9d-38c5-48f3-acf9-e33e8af5728b tempest-VolumesAssistedSnapshotsTest-1323460801 tempest-VolumesAssistedSnapshotsTest-1323460801-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918,backing_fmt=raw /opt/stack/data/nova/instances/c9663ab3-3374-400e-ab0a-769731b78cca/disk 1073741824 {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:06:38 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-40dafa9d-38c5-48f3-acf9-e33e8af5728b tempest-VolumesAssistedSnapshotsTest-1323460801 tempest-VolumesAssistedSnapshotsTest-1323460801-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918,backing_fmt=raw /opt/stack/data/nova/instances/c9663ab3-3374-400e-ab0a-769731b78cca/disk 1073741824" returned: 0 in 0.063s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:06:38 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-40dafa9d-38c5-48f3-acf9-e33e8af5728b tempest-VolumesAssistedSnapshotsTest-1323460801 tempest-VolumesAssistedSnapshotsTest-1323460801-project-member] Lock "80686e29dc9db6600d03e85cb62af298e12d7918" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.311s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:06:38 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-40dafa9d-38c5-48f3-acf9-e33e8af5728b tempest-VolumesAssistedSnapshotsTest-1323460801 tempest-VolumesAssistedSnapshotsTest-1323460801-project-member] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:06:38 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-40dafa9d-38c5-48f3-acf9-e33e8af5728b tempest-VolumesAssistedSnapshotsTest-1323460801 tempest-VolumesAssistedSnapshotsTest-1323460801-project-member] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json" returned: 0 in 0.210s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:06:38 user nova-compute[75234]: DEBUG nova.virt.disk.api [None req-40dafa9d-38c5-48f3-acf9-e33e8af5728b tempest-VolumesAssistedSnapshotsTest-1323460801 tempest-VolumesAssistedSnapshotsTest-1323460801-project-member] Checking if we can resize image /opt/stack/data/nova/instances/c9663ab3-3374-400e-ab0a-769731b78cca/disk. size=1073741824 {{(pid=75234) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:166}}
Nov 29 03:06:38 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-40dafa9d-38c5-48f3-acf9-e33e8af5728b tempest-VolumesAssistedSnapshotsTest-1323460801 tempest-VolumesAssistedSnapshotsTest-1323460801-project-member] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/c9663ab3-3374-400e-ab0a-769731b78cca/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:06:38 user nova-compute[75234]: DEBUG nova.network.neutron [None req-40dafa9d-38c5-48f3-acf9-e33e8af5728b tempest-VolumesAssistedSnapshotsTest-1323460801 tempest-VolumesAssistedSnapshotsTest-1323460801-project-member] [instance: c9663ab3-3374-400e-ab0a-769731b78cca] Successfully created port: b55d6eaa-b0d4-4a67-8407-3a8b69558e43 {{(pid=75234) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:548}}
Nov 29 03:06:38 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:06:38 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:06:38 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:06:38 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:06:38 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-40dafa9d-38c5-48f3-acf9-e33e8af5728b tempest-VolumesAssistedSnapshotsTest-1323460801 tempest-VolumesAssistedSnapshotsTest-1323460801-project-member] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/c9663ab3-3374-400e-ab0a-769731b78cca/disk --force-share --output=json" returned: 0 in 0.256s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:06:38 user nova-compute[75234]: DEBUG nova.virt.disk.api [None req-40dafa9d-38c5-48f3-acf9-e33e8af5728b tempest-VolumesAssistedSnapshotsTest-1323460801 tempest-VolumesAssistedSnapshotsTest-1323460801-project-member] Cannot resize image /opt/stack/data/nova/instances/c9663ab3-3374-400e-ab0a-769731b78cca/disk to a smaller size. {{(pid=75234) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:172}}
Nov 29 03:06:38 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-40dafa9d-38c5-48f3-acf9-e33e8af5728b tempest-VolumesAssistedSnapshotsTest-1323460801 tempest-VolumesAssistedSnapshotsTest-1323460801-project-member] [instance: c9663ab3-3374-400e-ab0a-769731b78cca] Created local disks {{(pid=75234) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4861}}
Nov 29 03:06:38 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-40dafa9d-38c5-48f3-acf9-e33e8af5728b tempest-VolumesAssistedSnapshotsTest-1323460801 tempest-VolumesAssistedSnapshotsTest-1323460801-project-member] [instance: c9663ab3-3374-400e-ab0a-769731b78cca] Ensure instance console log exists: /opt/stack/data/nova/instances/c9663ab3-3374-400e-ab0a-769731b78cca/console.log {{(pid=75234) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4613}}
Nov 29 03:06:38 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-40dafa9d-38c5-48f3-acf9-e33e8af5728b tempest-VolumesAssistedSnapshotsTest-1323460801 tempest-VolumesAssistedSnapshotsTest-1323460801-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:06:38 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-40dafa9d-38c5-48f3-acf9-e33e8af5728b tempest-VolumesAssistedSnapshotsTest-1323460801 tempest-VolumesAssistedSnapshotsTest-1323460801-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:06:38 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-40dafa9d-38c5-48f3-acf9-e33e8af5728b tempest-VolumesAssistedSnapshotsTest-1323460801 tempest-VolumesAssistedSnapshotsTest-1323460801-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:06:38 user nova-compute[75234]: DEBUG nova.compute.manager [req-80773d63-9d99-449d-bfa8-a8a2d93f63d5 req-4e80cd1f-6a75-4ca0-af61-76a143176ca5 service nova] [instance: 21e9c6cd-c39f-41c0-8768-5dd64589c3f3] Received event network-vif-plugged-87f1e736-6665-4945-bd19-0cf93bff905e {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:06:38 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-80773d63-9d99-449d-bfa8-a8a2d93f63d5 req-4e80cd1f-6a75-4ca0-af61-76a143176ca5 service nova] Acquiring lock "21e9c6cd-c39f-41c0-8768-5dd64589c3f3-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:06:38 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-80773d63-9d99-449d-bfa8-a8a2d93f63d5 req-4e80cd1f-6a75-4ca0-af61-76a143176ca5 service nova] Lock "21e9c6cd-c39f-41c0-8768-5dd64589c3f3-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:06:38 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-80773d63-9d99-449d-bfa8-a8a2d93f63d5 req-4e80cd1f-6a75-4ca0-af61-76a143176ca5 service nova] Lock "21e9c6cd-c39f-41c0-8768-5dd64589c3f3-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:06:38 user nova-compute[75234]: DEBUG nova.compute.manager [req-80773d63-9d99-449d-bfa8-a8a2d93f63d5 req-4e80cd1f-6a75-4ca0-af61-76a143176ca5 service nova] [instance: 21e9c6cd-c39f-41c0-8768-5dd64589c3f3] No waiting events found dispatching network-vif-plugged-87f1e736-6665-4945-bd19-0cf93bff905e {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:06:38 user nova-compute[75234]: WARNING nova.compute.manager [req-80773d63-9d99-449d-bfa8-a8a2d93f63d5 req-4e80cd1f-6a75-4ca0-af61-76a143176ca5 service nova] [instance: 21e9c6cd-c39f-41c0-8768-5dd64589c3f3] Received unexpected event network-vif-plugged-87f1e736-6665-4945-bd19-0cf93bff905e for instance with vm_state active and task_state None.
Nov 29 03:06:39 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:06:39 user nova-compute[75234]: DEBUG nova.network.neutron [None req-40dafa9d-38c5-48f3-acf9-e33e8af5728b tempest-VolumesAssistedSnapshotsTest-1323460801 tempest-VolumesAssistedSnapshotsTest-1323460801-project-member] [instance: c9663ab3-3374-400e-ab0a-769731b78cca] Successfully updated port: b55d6eaa-b0d4-4a67-8407-3a8b69558e43 {{(pid=75234) _update_port /opt/stack/nova/nova/network/neutron.py:586}}
Nov 29 03:06:39 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-40dafa9d-38c5-48f3-acf9-e33e8af5728b tempest-VolumesAssistedSnapshotsTest-1323460801 tempest-VolumesAssistedSnapshotsTest-1323460801-project-member] Acquiring lock "refresh_cache-c9663ab3-3374-400e-ab0a-769731b78cca" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:06:39 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-40dafa9d-38c5-48f3-acf9-e33e8af5728b tempest-VolumesAssistedSnapshotsTest-1323460801 tempest-VolumesAssistedSnapshotsTest-1323460801-project-member] Acquired lock "refresh_cache-c9663ab3-3374-400e-ab0a-769731b78cca" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:06:39 user nova-compute[75234]: DEBUG nova.network.neutron [None req-40dafa9d-38c5-48f3-acf9-e33e8af5728b tempest-VolumesAssistedSnapshotsTest-1323460801 tempest-VolumesAssistedSnapshotsTest-1323460801-project-member] [instance: c9663ab3-3374-400e-ab0a-769731b78cca] Building network info cache for instance {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2009}}
Nov 29 03:06:39 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:06:39 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:06:39 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:06:39 user nova-compute[75234]: DEBUG nova.network.neutron [None req-40dafa9d-38c5-48f3-acf9-e33e8af5728b tempest-VolumesAssistedSnapshotsTest-1323460801 tempest-VolumesAssistedSnapshotsTest-1323460801-project-member] [instance: c9663ab3-3374-400e-ab0a-769731b78cca] Instance cache missing network info. {{(pid=75234) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3322}}
Nov 29 03:06:39 user nova-compute[75234]: DEBUG nova.compute.manager [req-1f349cac-06e8-441f-8b32-e796697c18b5 req-5b99f850-ad56-4a5a-bb7b-617f9a878b6c service nova] [instance: c9663ab3-3374-400e-ab0a-769731b78cca] Received event network-changed-b55d6eaa-b0d4-4a67-8407-3a8b69558e43 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:06:39 user nova-compute[75234]: DEBUG nova.compute.manager [req-1f349cac-06e8-441f-8b32-e796697c18b5 req-5b99f850-ad56-4a5a-bb7b-617f9a878b6c service nova] [instance: c9663ab3-3374-400e-ab0a-769731b78cca] Refreshing instance network info cache due to event network-changed-b55d6eaa-b0d4-4a67-8407-3a8b69558e43. {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11117}}
Nov 29 03:06:39 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-1f349cac-06e8-441f-8b32-e796697c18b5 req-5b99f850-ad56-4a5a-bb7b-617f9a878b6c service nova] Acquiring lock "refresh_cache-c9663ab3-3374-400e-ab0a-769731b78cca" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:06:39 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:06:39 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:06:39 user nova-compute[75234]: DEBUG nova.network.neutron [None req-40dafa9d-38c5-48f3-acf9-e33e8af5728b tempest-VolumesAssistedSnapshotsTest-1323460801 tempest-VolumesAssistedSnapshotsTest-1323460801-project-member] [instance: c9663ab3-3374-400e-ab0a-769731b78cca] Updating instance_info_cache with network_info: [{"id": "b55d6eaa-b0d4-4a67-8407-3a8b69558e43", "address": "fa:16:3e:3e:b6:0b", "network": {"id": "8d7bb68b-4e8e-42c9-8d1a-3d0e301a7b73", "bridge": "br-int", "label": "tempest-VolumesAssistedSnapshotsTest-1523028903-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "d59c913723894ba1938fb8177c97e627", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapb55d6eaa-b0", "ovs_interfaceid": "b55d6eaa-b0d4-4a67-8407-3a8b69558e43", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:06:39 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-40dafa9d-38c5-48f3-acf9-e33e8af5728b tempest-VolumesAssistedSnapshotsTest-1323460801 tempest-VolumesAssistedSnapshotsTest-1323460801-project-member] Releasing lock "refresh_cache-c9663ab3-3374-400e-ab0a-769731b78cca" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:06:40 user nova-compute[75234]: DEBUG nova.compute.manager [None req-40dafa9d-38c5-48f3-acf9-e33e8af5728b tempest-VolumesAssistedSnapshotsTest-1323460801 tempest-VolumesAssistedSnapshotsTest-1323460801-project-member] [instance: c9663ab3-3374-400e-ab0a-769731b78cca] Instance network_info: |[{"id": "b55d6eaa-b0d4-4a67-8407-3a8b69558e43", "address": "fa:16:3e:3e:b6:0b", "network": {"id": "8d7bb68b-4e8e-42c9-8d1a-3d0e301a7b73", "bridge": "br-int", "label": "tempest-VolumesAssistedSnapshotsTest-1523028903-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "d59c913723894ba1938fb8177c97e627", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapb55d6eaa-b0", "ovs_interfaceid": "b55d6eaa-b0d4-4a67-8407-3a8b69558e43", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=75234) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1987}}
Nov 29 03:06:40 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-1f349cac-06e8-441f-8b32-e796697c18b5 req-5b99f850-ad56-4a5a-bb7b-617f9a878b6c service nova] Acquired lock "refresh_cache-c9663ab3-3374-400e-ab0a-769731b78cca" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:06:40 user nova-compute[75234]: DEBUG nova.network.neutron [req-1f349cac-06e8-441f-8b32-e796697c18b5 req-5b99f850-ad56-4a5a-bb7b-617f9a878b6c service nova] [instance: c9663ab3-3374-400e-ab0a-769731b78cca] Refreshing network info cache for port b55d6eaa-b0d4-4a67-8407-3a8b69558e43 {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2006}}
Nov 29 03:06:40 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-40dafa9d-38c5-48f3-acf9-e33e8af5728b tempest-VolumesAssistedSnapshotsTest-1323460801 tempest-VolumesAssistedSnapshotsTest-1323460801-project-member] [instance: c9663ab3-3374-400e-ab0a-769731b78cca] Start _get_guest_xml network_info=[{"id": "b55d6eaa-b0d4-4a67-8407-3a8b69558e43", "address": "fa:16:3e:3e:b6:0b", "network": {"id": "8d7bb68b-4e8e-42c9-8d1a-3d0e301a7b73", "bridge": "br-int", "label": "tempest-VolumesAssistedSnapshotsTest-1523028903-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "d59c913723894ba1938fb8177c97e627", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapb55d6eaa-b0", "ovs_interfaceid": "b55d6eaa-b0d4-4a67-8407-3a8b69558e43", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum='c8fc807773e5354afe61636071771906',container_format='bare',created_at=2023-11-29T03:02:15Z,direct_url=<?>,disk_format='qcow2',id=96c30ba7-9b73-4a30-8ba4-7138879573b7,min_disk=0,min_ram=0,name='cirros-0.6.2-x86_64-disk',owner='b18e380921c748b78517b6f1211cd0f1',properties=ImageMetaProps,protected=<?>,size=21430272,status='active',tags=<?>,updated_at=2023-11-29T03:02:17Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [{'device_name': '/dev/vda', 'encryption_format': None, 'encrypted': False, 'device_type': 'disk', 'size': 0, 'disk_bus': 'virtio', 'boot_index': 0, 'encryption_options': None, 'encryption_secret_uuid': None, 'guest_format': None, 'image_id': '96c30ba7-9b73-4a30-8ba4-7138879573b7'}], 'ephemerals': [], 'block_device_mapping': [], 'swap': None} {{(pid=75234) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7562}}
Nov 29 03:06:40 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-40dafa9d-38c5-48f3-acf9-e33e8af5728b tempest-VolumesAssistedSnapshotsTest-1323460801 tempest-VolumesAssistedSnapshotsTest-1323460801-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:06:40 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-40dafa9d-38c5-48f3-acf9-e33e8af5728b tempest-VolumesAssistedSnapshotsTest-1323460801 tempest-VolumesAssistedSnapshotsTest-1323460801-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:06:40 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-40dafa9d-38c5-48f3-acf9-e33e8af5728b tempest-VolumesAssistedSnapshotsTest-1323460801 tempest-VolumesAssistedSnapshotsTest-1323460801-project-member] Searching host: 'user' for CPU controller through CGroups V1... {{(pid=75234) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1652}}
Nov 29 03:06:40 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-40dafa9d-38c5-48f3-acf9-e33e8af5728b tempest-VolumesAssistedSnapshotsTest-1323460801 tempest-VolumesAssistedSnapshotsTest-1323460801-project-member] CPU controller missing on host. {{(pid=75234) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1662}}
Nov 29 03:06:40 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-40dafa9d-38c5-48f3-acf9-e33e8af5728b tempest-VolumesAssistedSnapshotsTest-1323460801 tempest-VolumesAssistedSnapshotsTest-1323460801-project-member] Searching host: 'user' for CPU controller through CGroups V2... {{(pid=75234) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1671}}
Nov 29 03:06:40 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-40dafa9d-38c5-48f3-acf9-e33e8af5728b tempest-VolumesAssistedSnapshotsTest-1323460801 tempest-VolumesAssistedSnapshotsTest-1323460801-project-member] CPU controller found on host. {{(pid=75234) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1678}}
Nov 29 03:06:40 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-40dafa9d-38c5-48f3-acf9-e33e8af5728b tempest-VolumesAssistedSnapshotsTest-1323460801 tempest-VolumesAssistedSnapshotsTest-1323460801-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=75234) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5400}}
Nov 29 03:06:40 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-40dafa9d-38c5-48f3-acf9-e33e8af5728b tempest-VolumesAssistedSnapshotsTest-1323460801 tempest-VolumesAssistedSnapshotsTest-1323460801-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-11-29T03:02:04Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='1',id=6,is_public=True,memory_mb=512,name='m1.tiny',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='c8fc807773e5354afe61636071771906',container_format='bare',created_at=2023-11-29T03:02:15Z,direct_url=<?>,disk_format='qcow2',id=96c30ba7-9b73-4a30-8ba4-7138879573b7,min_disk=0,min_ram=0,name='cirros-0.6.2-x86_64-disk',owner='b18e380921c748b78517b6f1211cd0f1',properties=ImageMetaProps,protected=<?>,size=21430272,status='active',tags=<?>,updated_at=2023-11-29T03:02:17Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:563}}
Nov 29 03:06:40 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-40dafa9d-38c5-48f3-acf9-e33e8af5728b tempest-VolumesAssistedSnapshotsTest-1323460801 tempest-VolumesAssistedSnapshotsTest-1323460801-project-member] Flavor limits 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:348}}
Nov 29 03:06:40 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-40dafa9d-38c5-48f3-acf9-e33e8af5728b tempest-VolumesAssistedSnapshotsTest-1323460801 tempest-VolumesAssistedSnapshotsTest-1323460801-project-member] Image limits 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:352}}
Nov 29 03:06:40 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-40dafa9d-38c5-48f3-acf9-e33e8af5728b tempest-VolumesAssistedSnapshotsTest-1323460801 tempest-VolumesAssistedSnapshotsTest-1323460801-project-member] Flavor pref 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:388}}
Nov 29 03:06:40 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-40dafa9d-38c5-48f3-acf9-e33e8af5728b tempest-VolumesAssistedSnapshotsTest-1323460801 tempest-VolumesAssistedSnapshotsTest-1323460801-project-member] Image pref 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:392}}
Nov 29 03:06:40 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-40dafa9d-38c5-48f3-acf9-e33e8af5728b tempest-VolumesAssistedSnapshotsTest-1323460801 tempest-VolumesAssistedSnapshotsTest-1323460801-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:430}}
Nov 29 03:06:40 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-40dafa9d-38c5-48f3-acf9-e33e8af5728b tempest-VolumesAssistedSnapshotsTest-1323460801 tempest-VolumesAssistedSnapshotsTest-1323460801-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:569}}
Nov 29 03:06:40 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-40dafa9d-38c5-48f3-acf9-e33e8af5728b tempest-VolumesAssistedSnapshotsTest-1323460801 tempest-VolumesAssistedSnapshotsTest-1323460801-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=75234) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:471}}
Nov 29 03:06:40 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-40dafa9d-38c5-48f3-acf9-e33e8af5728b tempest-VolumesAssistedSnapshotsTest-1323460801 tempest-VolumesAssistedSnapshotsTest-1323460801-project-member] Got 1 possible topologies {{(pid=75234) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:501}}
Nov 29 03:06:40 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-40dafa9d-38c5-48f3-acf9-e33e8af5728b tempest-VolumesAssistedSnapshotsTest-1323460801 tempest-VolumesAssistedSnapshotsTest-1323460801-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:575}}
Nov 29 03:06:40 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-40dafa9d-38c5-48f3-acf9-e33e8af5728b tempest-VolumesAssistedSnapshotsTest-1323460801 tempest-VolumesAssistedSnapshotsTest-1323460801-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:577}}
Nov 29 03:06:40 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-40dafa9d-38c5-48f3-acf9-e33e8af5728b tempest-VolumesAssistedSnapshotsTest-1323460801 tempest-VolumesAssistedSnapshotsTest-1323460801-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-11-29T03:06:35Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-VolumesAssistedSnapshotsTest-server-15749010',display_name='tempest-VolumesAssistedSnapshotsTest-server-15749010',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-volumesassistedsnapshotstest-server-15749010',id=9,image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBOpV1r9kDBNAejJiIxQHnv5nT3IVZ/wlovVaZh1jY/ciYwn121v+8yMvPVcNbJUyt62m/+e6PHsXJeMqoN77/bu6BfTHWz76/IxLZ6WUZ95en78z4N4sjKvEYdHAkzx67g==',key_name='tempest-keypair-973383639',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='d59c913723894ba1938fb8177c97e627',ramdisk_id='',reservation_id='r-3beeznuu',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-VolumesAssistedSnapshotsTest-1323460801',owner_user_name='tempest-VolumesAssistedSnapshotsTest-1323460801-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-11-29T03:06:37Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='f90b22e80de849c3b1ca951f7650fe84',uuid=c9663ab3-3374-400e-ab0a-769731b78cca,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "b55d6eaa-b0d4-4a67-8407-3a8b69558e43", "address": "fa:16:3e:3e:b6:0b", "network": {"id": "8d7bb68b-4e8e-42c9-8d1a-3d0e301a7b73", "bridge": "br-int", "label": "tempest-VolumesAssistedSnapshotsTest-1523028903-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "d59c913723894ba1938fb8177c97e627", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapb55d6eaa-b0", "ovs_interfaceid": "b55d6eaa-b0d4-4a67-8407-3a8b69558e43", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=75234) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Nov 29 03:06:40 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-40dafa9d-38c5-48f3-acf9-e33e8af5728b tempest-VolumesAssistedSnapshotsTest-1323460801 tempest-VolumesAssistedSnapshotsTest-1323460801-project-member] Converting VIF {"id": "b55d6eaa-b0d4-4a67-8407-3a8b69558e43", "address": "fa:16:3e:3e:b6:0b", "network": {"id": "8d7bb68b-4e8e-42c9-8d1a-3d0e301a7b73", "bridge": "br-int", "label": "tempest-VolumesAssistedSnapshotsTest-1523028903-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "d59c913723894ba1938fb8177c97e627", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapb55d6eaa-b0", "ovs_interfaceid": "b55d6eaa-b0d4-4a67-8407-3a8b69558e43", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:06:40 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-40dafa9d-38c5-48f3-acf9-e33e8af5728b tempest-VolumesAssistedSnapshotsTest-1323460801 tempest-VolumesAssistedSnapshotsTest-1323460801-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:3e:b6:0b,bridge_name='br-int',has_traffic_filtering=True,id=b55d6eaa-b0d4-4a67-8407-3a8b69558e43,network=Network(8d7bb68b-4e8e-42c9-8d1a-3d0e301a7b73),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapb55d6eaa-b0') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:06:40 user nova-compute[75234]: DEBUG nova.objects.instance [None req-40dafa9d-38c5-48f3-acf9-e33e8af5728b tempest-VolumesAssistedSnapshotsTest-1323460801 tempest-VolumesAssistedSnapshotsTest-1323460801-project-member] Lazy-loading 'pci_devices' on Instance uuid c9663ab3-3374-400e-ab0a-769731b78cca {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:06:40 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-40dafa9d-38c5-48f3-acf9-e33e8af5728b tempest-VolumesAssistedSnapshotsTest-1323460801 tempest-VolumesAssistedSnapshotsTest-1323460801-project-member] [instance: c9663ab3-3374-400e-ab0a-769731b78cca] End _get_guest_xml xml=<domain type="kvm">
Nov 29 03:06:40 user nova-compute[75234]:   <uuid>c9663ab3-3374-400e-ab0a-769731b78cca</uuid>
Nov 29 03:06:40 user nova-compute[75234]:   <name>instance-00000009</name>
Nov 29 03:06:40 user nova-compute[75234]:   <memory>524288</memory>
Nov 29 03:06:40 user nova-compute[75234]:   <vcpu>1</vcpu>
Nov 29 03:06:40 user nova-compute[75234]:   <metadata>
Nov 29 03:06:40 user nova-compute[75234]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Nov 29 03:06:40 user nova-compute[75234]:       <nova:package version="0.0.0"/>
Nov 29 03:06:40 user nova-compute[75234]:       <nova:name>tempest-VolumesAssistedSnapshotsTest-server-15749010</nova:name>
Nov 29 03:06:40 user nova-compute[75234]:       <nova:creationTime>2023-11-29 03:06:40</nova:creationTime>
Nov 29 03:06:40 user nova-compute[75234]:       <nova:flavor name="m1.tiny">
Nov 29 03:06:40 user nova-compute[75234]:         <nova:memory>512</nova:memory>
Nov 29 03:06:40 user nova-compute[75234]:         <nova:disk>1</nova:disk>
Nov 29 03:06:40 user nova-compute[75234]:         <nova:swap>0</nova:swap>
Nov 29 03:06:40 user nova-compute[75234]:         <nova:ephemeral>0</nova:ephemeral>
Nov 29 03:06:40 user nova-compute[75234]:         <nova:vcpus>1</nova:vcpus>
Nov 29 03:06:40 user nova-compute[75234]:       </nova:flavor>
Nov 29 03:06:40 user nova-compute[75234]:       <nova:owner>
Nov 29 03:06:40 user nova-compute[75234]:         <nova:user uuid="f90b22e80de849c3b1ca951f7650fe84">tempest-VolumesAssistedSnapshotsTest-1323460801-project-member</nova:user>
Nov 29 03:06:40 user nova-compute[75234]:         <nova:project uuid="d59c913723894ba1938fb8177c97e627">tempest-VolumesAssistedSnapshotsTest-1323460801</nova:project>
Nov 29 03:06:40 user nova-compute[75234]:       </nova:owner>
Nov 29 03:06:40 user nova-compute[75234]:       <nova:root type="image" uuid="96c30ba7-9b73-4a30-8ba4-7138879573b7"/>
Nov 29 03:06:40 user nova-compute[75234]:       <nova:ports>
Nov 29 03:06:40 user nova-compute[75234]:         <nova:port uuid="b55d6eaa-b0d4-4a67-8407-3a8b69558e43">
Nov 29 03:06:40 user nova-compute[75234]:           <nova:ip type="fixed" address="10.0.0.7" ipVersion="4"/>
Nov 29 03:06:40 user nova-compute[75234]:         </nova:port>
Nov 29 03:06:40 user nova-compute[75234]:       </nova:ports>
Nov 29 03:06:40 user nova-compute[75234]:     </nova:instance>
Nov 29 03:06:40 user nova-compute[75234]:   </metadata>
Nov 29 03:06:40 user nova-compute[75234]:   <sysinfo type="smbios">
Nov 29 03:06:40 user nova-compute[75234]:     <system>
Nov 29 03:06:40 user nova-compute[75234]:       <entry name="manufacturer">OpenStack Foundation</entry>
Nov 29 03:06:40 user nova-compute[75234]:       <entry name="product">OpenStack Nova</entry>
Nov 29 03:06:40 user nova-compute[75234]:       <entry name="version">0.0.0</entry>
Nov 29 03:06:40 user nova-compute[75234]:       <entry name="serial">c9663ab3-3374-400e-ab0a-769731b78cca</entry>
Nov 29 03:06:40 user nova-compute[75234]:       <entry name="uuid">c9663ab3-3374-400e-ab0a-769731b78cca</entry>
Nov 29 03:06:40 user nova-compute[75234]:       <entry name="family">Virtual Machine</entry>
Nov 29 03:06:40 user nova-compute[75234]:     </system>
Nov 29 03:06:40 user nova-compute[75234]:   </sysinfo>
Nov 29 03:06:40 user nova-compute[75234]:   <os>
Nov 29 03:06:40 user nova-compute[75234]:     <type arch="x86_64" machine="pc">hvm</type>
Nov 29 03:06:40 user nova-compute[75234]:     <boot dev="hd"/>
Nov 29 03:06:40 user nova-compute[75234]:     <smbios mode="sysinfo"/>
Nov 29 03:06:40 user nova-compute[75234]:   </os>
Nov 29 03:06:40 user nova-compute[75234]:   <features>
Nov 29 03:06:40 user nova-compute[75234]:     <acpi/>
Nov 29 03:06:40 user nova-compute[75234]:     <apic/>
Nov 29 03:06:40 user nova-compute[75234]:     <vmcoreinfo/>
Nov 29 03:06:40 user nova-compute[75234]:   </features>
Nov 29 03:06:40 user nova-compute[75234]:   <clock offset="utc">
Nov 29 03:06:40 user nova-compute[75234]:     <timer name="pit" tickpolicy="delay"/>
Nov 29 03:06:40 user nova-compute[75234]:     <timer name="rtc" tickpolicy="catchup"/>
Nov 29 03:06:40 user nova-compute[75234]:     <timer name="hpet" present="no"/>
Nov 29 03:06:40 user nova-compute[75234]:   </clock>
Nov 29 03:06:40 user nova-compute[75234]:   <cpu mode="custom" match="exact">
Nov 29 03:06:40 user nova-compute[75234]:     <model>Nehalem</model>
Nov 29 03:06:40 user nova-compute[75234]:     <topology sockets="1" cores="1" threads="1"/>
Nov 29 03:06:40 user nova-compute[75234]:   </cpu>
Nov 29 03:06:40 user nova-compute[75234]:   <devices>
Nov 29 03:06:40 user nova-compute[75234]:     <disk type="file" device="disk">
Nov 29 03:06:40 user nova-compute[75234]:       <driver name="qemu" type="qcow2" cache="none"/>
Nov 29 03:06:40 user nova-compute[75234]:       <source file="/opt/stack/data/nova/instances/c9663ab3-3374-400e-ab0a-769731b78cca/disk"/>
Nov 29 03:06:40 user nova-compute[75234]:       <target dev="vda" bus="virtio"/>
Nov 29 03:06:40 user nova-compute[75234]:     </disk>
Nov 29 03:06:40 user nova-compute[75234]:     <interface type="ethernet">
Nov 29 03:06:40 user nova-compute[75234]:       <mac address="fa:16:3e:3e:b6:0b"/>
Nov 29 03:06:40 user nova-compute[75234]:       <model type="virtio"/>
Nov 29 03:06:40 user nova-compute[75234]:       <mtu size="1442"/>
Nov 29 03:06:40 user nova-compute[75234]:       <target dev="tapb55d6eaa-b0"/>
Nov 29 03:06:40 user nova-compute[75234]:     </interface>
Nov 29 03:06:40 user nova-compute[75234]:     <serial type="pty">
Nov 29 03:06:40 user nova-compute[75234]:       <log file="/opt/stack/data/nova/instances/c9663ab3-3374-400e-ab0a-769731b78cca/console.log" append="off"/>
Nov 29 03:06:40 user nova-compute[75234]:     </serial>
Nov 29 03:06:40 user nova-compute[75234]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Nov 29 03:06:40 user nova-compute[75234]:     <video>
Nov 29 03:06:40 user nova-compute[75234]:       <model type="virtio"/>
Nov 29 03:06:40 user nova-compute[75234]:     </video>
Nov 29 03:06:40 user nova-compute[75234]:     <rng model="virtio">
Nov 29 03:06:40 user nova-compute[75234]:       <backend model="random">/dev/urandom</backend>
Nov 29 03:06:40 user nova-compute[75234]:     </rng>
Nov 29 03:06:40 user nova-compute[75234]:     <controller type="usb" index="0" model="none"/>
Nov 29 03:06:40 user nova-compute[75234]:     <memballoon model="virtio">
Nov 29 03:06:40 user nova-compute[75234]:       <stats period="10"/>
Nov 29 03:06:40 user nova-compute[75234]:     </memballoon>
Nov 29 03:06:40 user nova-compute[75234]:   </devices>
Nov 29 03:06:40 user nova-compute[75234]: </domain>
Nov 29 03:06:40 user nova-compute[75234]:  {{(pid=75234) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7568}}
Nov 29 03:06:40 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-40dafa9d-38c5-48f3-acf9-e33e8af5728b tempest-VolumesAssistedSnapshotsTest-1323460801 tempest-VolumesAssistedSnapshotsTest-1323460801-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-11-29T03:06:35Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-VolumesAssistedSnapshotsTest-server-15749010',display_name='tempest-VolumesAssistedSnapshotsTest-server-15749010',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-volumesassistedsnapshotstest-server-15749010',id=9,image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBOpV1r9kDBNAejJiIxQHnv5nT3IVZ/wlovVaZh1jY/ciYwn121v+8yMvPVcNbJUyt62m/+e6PHsXJeMqoN77/bu6BfTHWz76/IxLZ6WUZ95en78z4N4sjKvEYdHAkzx67g==',key_name='tempest-keypair-973383639',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='d59c913723894ba1938fb8177c97e627',ramdisk_id='',reservation_id='r-3beeznuu',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-VolumesAssistedSnapshotsTest-1323460801',owner_user_name='tempest-VolumesAssistedSnapshotsTest-1323460801-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-11-29T03:06:37Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='f90b22e80de849c3b1ca951f7650fe84',uuid=c9663ab3-3374-400e-ab0a-769731b78cca,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "b55d6eaa-b0d4-4a67-8407-3a8b69558e43", "address": "fa:16:3e:3e:b6:0b", "network": {"id": "8d7bb68b-4e8e-42c9-8d1a-3d0e301a7b73", "bridge": "br-int", "label": "tempest-VolumesAssistedSnapshotsTest-1523028903-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "d59c913723894ba1938fb8177c97e627", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapb55d6eaa-b0", "ovs_interfaceid": "b55d6eaa-b0d4-4a67-8407-3a8b69558e43", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Nov 29 03:06:40 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-40dafa9d-38c5-48f3-acf9-e33e8af5728b tempest-VolumesAssistedSnapshotsTest-1323460801 tempest-VolumesAssistedSnapshotsTest-1323460801-project-member] Converting VIF {"id": "b55d6eaa-b0d4-4a67-8407-3a8b69558e43", "address": "fa:16:3e:3e:b6:0b", "network": {"id": "8d7bb68b-4e8e-42c9-8d1a-3d0e301a7b73", "bridge": "br-int", "label": "tempest-VolumesAssistedSnapshotsTest-1523028903-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "d59c913723894ba1938fb8177c97e627", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapb55d6eaa-b0", "ovs_interfaceid": "b55d6eaa-b0d4-4a67-8407-3a8b69558e43", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:06:40 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-40dafa9d-38c5-48f3-acf9-e33e8af5728b tempest-VolumesAssistedSnapshotsTest-1323460801 tempest-VolumesAssistedSnapshotsTest-1323460801-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:3e:b6:0b,bridge_name='br-int',has_traffic_filtering=True,id=b55d6eaa-b0d4-4a67-8407-3a8b69558e43,network=Network(8d7bb68b-4e8e-42c9-8d1a-3d0e301a7b73),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapb55d6eaa-b0') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:06:40 user nova-compute[75234]: DEBUG os_vif [None req-40dafa9d-38c5-48f3-acf9-e33e8af5728b tempest-VolumesAssistedSnapshotsTest-1323460801 tempest-VolumesAssistedSnapshotsTest-1323460801-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:3e:b6:0b,bridge_name='br-int',has_traffic_filtering=True,id=b55d6eaa-b0d4-4a67-8407-3a8b69558e43,network=Network(8d7bb68b-4e8e-42c9-8d1a-3d0e301a7b73),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapb55d6eaa-b0') {{(pid=75234) plug /opt/stack/data/venv/lib/python3.10/site-packages/os_vif/__init__.py:76}}
Nov 29 03:06:40 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:06:40 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:06:40 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Nov 29 03:06:40 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:06:40 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbCreateCommand(_result=None, table=QoS, columns={'type': 'linux-noop', 'external_ids': {'id': '2ee7aaf4-8c51-5cc7-b51c-093e61bc3356', '_type': 'linux-noop'}}, row=False) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:06:40 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:06:40 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:06:40 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:06:40 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tapb55d6eaa-b0, may_exist=True, interface_attrs={}) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:06:40 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Port, record=tapb55d6eaa-b0, col_values=(('qos', UUID('c27d0110-659d-48ee-917e-82dd53a5bc62')),), if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:06:40 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DbSetCommand(_result=None, table=Interface, record=tapb55d6eaa-b0, col_values=(('external_ids', {'iface-id': 'b55d6eaa-b0d4-4a67-8407-3a8b69558e43', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:3e:b6:0b', 'vm-uuid': 'c9663ab3-3374-400e-ab0a-769731b78cca'}),), if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:06:40 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:06:40 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:06:40 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:06:40 user nova-compute[75234]: INFO os_vif [None req-40dafa9d-38c5-48f3-acf9-e33e8af5728b tempest-VolumesAssistedSnapshotsTest-1323460801 tempest-VolumesAssistedSnapshotsTest-1323460801-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:3e:b6:0b,bridge_name='br-int',has_traffic_filtering=True,id=b55d6eaa-b0d4-4a67-8407-3a8b69558e43,network=Network(8d7bb68b-4e8e-42c9-8d1a-3d0e301a7b73),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapb55d6eaa-b0')
Nov 29 03:06:40 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-40dafa9d-38c5-48f3-acf9-e33e8af5728b tempest-VolumesAssistedSnapshotsTest-1323460801 tempest-VolumesAssistedSnapshotsTest-1323460801-project-member] No BDM found with device name vda, not building metadata. {{(pid=75234) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12089}}
Nov 29 03:06:40 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-40dafa9d-38c5-48f3-acf9-e33e8af5728b tempest-VolumesAssistedSnapshotsTest-1323460801 tempest-VolumesAssistedSnapshotsTest-1323460801-project-member] No VIF found with MAC fa:16:3e:3e:b6:0b, not building metadata {{(pid=75234) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Nov 29 03:06:40 user nova-compute[75234]: DEBUG nova.network.neutron [req-1f349cac-06e8-441f-8b32-e796697c18b5 req-5b99f850-ad56-4a5a-bb7b-617f9a878b6c service nova] [instance: c9663ab3-3374-400e-ab0a-769731b78cca] Updated VIF entry in instance network info cache for port b55d6eaa-b0d4-4a67-8407-3a8b69558e43. {{(pid=75234) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3481}}
Nov 29 03:06:40 user nova-compute[75234]: DEBUG nova.network.neutron [req-1f349cac-06e8-441f-8b32-e796697c18b5 req-5b99f850-ad56-4a5a-bb7b-617f9a878b6c service nova] [instance: c9663ab3-3374-400e-ab0a-769731b78cca] Updating instance_info_cache with network_info: [{"id": "b55d6eaa-b0d4-4a67-8407-3a8b69558e43", "address": "fa:16:3e:3e:b6:0b", "network": {"id": "8d7bb68b-4e8e-42c9-8d1a-3d0e301a7b73", "bridge": "br-int", "label": "tempest-VolumesAssistedSnapshotsTest-1523028903-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "d59c913723894ba1938fb8177c97e627", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapb55d6eaa-b0", "ovs_interfaceid": "b55d6eaa-b0d4-4a67-8407-3a8b69558e43", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:06:40 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-1f349cac-06e8-441f-8b32-e796697c18b5 req-5b99f850-ad56-4a5a-bb7b-617f9a878b6c service nova] Releasing lock "refresh_cache-c9663ab3-3374-400e-ab0a-769731b78cca" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:06:40 user nova-compute[75234]: DEBUG nova.compute.manager [None req-a745e7d4-15ea-45ba-a537-b658a23b24ed tempest-AttachSCSIVolumeTestJSON-719577128 tempest-AttachSCSIVolumeTestJSON-719577128-project-member] [instance: da625f9c-b7ed-4afc-875c-6436bf58e9cc] Instance event wait completed in 0 seconds for  {{(pid=75234) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Nov 29 03:06:40 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-a745e7d4-15ea-45ba-a537-b658a23b24ed tempest-AttachSCSIVolumeTestJSON-719577128 tempest-AttachSCSIVolumeTestJSON-719577128-project-member] [instance: da625f9c-b7ed-4afc-875c-6436bf58e9cc] Guest created on hypervisor {{(pid=75234) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4421}}
Nov 29 03:06:40 user nova-compute[75234]: INFO nova.virt.libvirt.driver [-] [instance: da625f9c-b7ed-4afc-875c-6436bf58e9cc] Instance spawned successfully.
Nov 29 03:06:40 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-a745e7d4-15ea-45ba-a537-b658a23b24ed tempest-AttachSCSIVolumeTestJSON-719577128 tempest-AttachSCSIVolumeTestJSON-719577128-project-member] [instance: da625f9c-b7ed-4afc-875c-6436bf58e9cc] Attempting to register defaults for the following image properties: ['hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:904}}
Nov 29 03:06:40 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-a745e7d4-15ea-45ba-a537-b658a23b24ed tempest-AttachSCSIVolumeTestJSON-719577128 tempest-AttachSCSIVolumeTestJSON-719577128-project-member] [instance: da625f9c-b7ed-4afc-875c-6436bf58e9cc] Found default for hw_input_bus of None {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:06:40 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-a745e7d4-15ea-45ba-a537-b658a23b24ed tempest-AttachSCSIVolumeTestJSON-719577128 tempest-AttachSCSIVolumeTestJSON-719577128-project-member] [instance: da625f9c-b7ed-4afc-875c-6436bf58e9cc] Found default for hw_pointer_model of None {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:06:40 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-a745e7d4-15ea-45ba-a537-b658a23b24ed tempest-AttachSCSIVolumeTestJSON-719577128 tempest-AttachSCSIVolumeTestJSON-719577128-project-member] [instance: da625f9c-b7ed-4afc-875c-6436bf58e9cc] Found default for hw_video_model of virtio {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:06:40 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-a745e7d4-15ea-45ba-a537-b658a23b24ed tempest-AttachSCSIVolumeTestJSON-719577128 tempest-AttachSCSIVolumeTestJSON-719577128-project-member] [instance: da625f9c-b7ed-4afc-875c-6436bf58e9cc] Found default for hw_vif_model of virtio {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:06:40 user nova-compute[75234]: INFO nova.compute.manager [None req-a745e7d4-15ea-45ba-a537-b658a23b24ed tempest-AttachSCSIVolumeTestJSON-719577128 tempest-AttachSCSIVolumeTestJSON-719577128-project-member] [instance: da625f9c-b7ed-4afc-875c-6436bf58e9cc] Took 11.60 seconds to spawn the instance on the hypervisor.
Nov 29 03:06:40 user nova-compute[75234]: DEBUG nova.compute.manager [None req-a745e7d4-15ea-45ba-a537-b658a23b24ed tempest-AttachSCSIVolumeTestJSON-719577128 tempest-AttachSCSIVolumeTestJSON-719577128-project-member] [instance: da625f9c-b7ed-4afc-875c-6436bf58e9cc] Checking state {{(pid=75234) _get_power_state /opt/stack/nova/nova/compute/manager.py:1782}}
Nov 29 03:06:41 user nova-compute[75234]: INFO nova.compute.manager [None req-a745e7d4-15ea-45ba-a537-b658a23b24ed tempest-AttachSCSIVolumeTestJSON-719577128 tempest-AttachSCSIVolumeTestJSON-719577128-project-member] [instance: da625f9c-b7ed-4afc-875c-6436bf58e9cc] Took 12.81 seconds to build instance.
Nov 29 03:06:41 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-a745e7d4-15ea-45ba-a537-b658a23b24ed tempest-AttachSCSIVolumeTestJSON-719577128 tempest-AttachSCSIVolumeTestJSON-719577128-project-member] Lock "da625f9c-b7ed-4afc-875c-6436bf58e9cc" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 13.056s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:06:41 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:06:41 user nova-compute[75234]: DEBUG nova.compute.manager [req-a8a3ff45-ebbd-4d5b-a313-b22ca0f6d78c req-9d1d75dc-55e5-4a74-9836-e83da67542b3 service nova] [instance: 21e9c6cd-c39f-41c0-8768-5dd64589c3f3] Received event network-vif-plugged-87f1e736-6665-4945-bd19-0cf93bff905e {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:06:41 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-a8a3ff45-ebbd-4d5b-a313-b22ca0f6d78c req-9d1d75dc-55e5-4a74-9836-e83da67542b3 service nova] Acquiring lock "21e9c6cd-c39f-41c0-8768-5dd64589c3f3-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:06:41 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-a8a3ff45-ebbd-4d5b-a313-b22ca0f6d78c req-9d1d75dc-55e5-4a74-9836-e83da67542b3 service nova] Lock "21e9c6cd-c39f-41c0-8768-5dd64589c3f3-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:06:41 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-a8a3ff45-ebbd-4d5b-a313-b22ca0f6d78c req-9d1d75dc-55e5-4a74-9836-e83da67542b3 service nova] Lock "21e9c6cd-c39f-41c0-8768-5dd64589c3f3-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:06:41 user nova-compute[75234]: DEBUG nova.compute.manager [req-a8a3ff45-ebbd-4d5b-a313-b22ca0f6d78c req-9d1d75dc-55e5-4a74-9836-e83da67542b3 service nova] [instance: 21e9c6cd-c39f-41c0-8768-5dd64589c3f3] No waiting events found dispatching network-vif-plugged-87f1e736-6665-4945-bd19-0cf93bff905e {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:06:41 user nova-compute[75234]: WARNING nova.compute.manager [req-a8a3ff45-ebbd-4d5b-a313-b22ca0f6d78c req-9d1d75dc-55e5-4a74-9836-e83da67542b3 service nova] [instance: 21e9c6cd-c39f-41c0-8768-5dd64589c3f3] Received unexpected event network-vif-plugged-87f1e736-6665-4945-bd19-0cf93bff905e for instance with vm_state active and task_state None.
Nov 29 03:06:41 user nova-compute[75234]: DEBUG nova.compute.manager [None req-684945de-57fe-475b-b990-ae7897d5ffb4 tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] [instance: aa3c7432-bf15-408e-9fbb-993135734161] Instance event wait completed in 0 seconds for  {{(pid=75234) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Nov 29 03:06:41 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-684945de-57fe-475b-b990-ae7897d5ffb4 tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] [instance: aa3c7432-bf15-408e-9fbb-993135734161] Guest created on hypervisor {{(pid=75234) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4421}}
Nov 29 03:06:41 user nova-compute[75234]: INFO nova.virt.libvirt.driver [-] [instance: aa3c7432-bf15-408e-9fbb-993135734161] Instance spawned successfully.
Nov 29 03:06:41 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-684945de-57fe-475b-b990-ae7897d5ffb4 tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] [instance: aa3c7432-bf15-408e-9fbb-993135734161] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:904}}
Nov 29 03:06:41 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-684945de-57fe-475b-b990-ae7897d5ffb4 tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] [instance: aa3c7432-bf15-408e-9fbb-993135734161] Found default for hw_cdrom_bus of ide {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:06:41 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-684945de-57fe-475b-b990-ae7897d5ffb4 tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] [instance: aa3c7432-bf15-408e-9fbb-993135734161] Found default for hw_disk_bus of virtio {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:06:41 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-684945de-57fe-475b-b990-ae7897d5ffb4 tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] [instance: aa3c7432-bf15-408e-9fbb-993135734161] Found default for hw_input_bus of None {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:06:41 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-684945de-57fe-475b-b990-ae7897d5ffb4 tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] [instance: aa3c7432-bf15-408e-9fbb-993135734161] Found default for hw_pointer_model of None {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:06:41 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-684945de-57fe-475b-b990-ae7897d5ffb4 tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] [instance: aa3c7432-bf15-408e-9fbb-993135734161] Found default for hw_video_model of virtio {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:06:41 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-684945de-57fe-475b-b990-ae7897d5ffb4 tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] [instance: aa3c7432-bf15-408e-9fbb-993135734161] Found default for hw_vif_model of virtio {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:06:41 user nova-compute[75234]: INFO nova.compute.manager [None req-684945de-57fe-475b-b990-ae7897d5ffb4 tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] [instance: aa3c7432-bf15-408e-9fbb-993135734161] Took 9.52 seconds to spawn the instance on the hypervisor.
Nov 29 03:06:41 user nova-compute[75234]: DEBUG nova.compute.manager [None req-684945de-57fe-475b-b990-ae7897d5ffb4 tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] [instance: aa3c7432-bf15-408e-9fbb-993135734161] Checking state {{(pid=75234) _get_power_state /opt/stack/nova/nova/compute/manager.py:1782}}
Nov 29 03:06:41 user nova-compute[75234]: INFO nova.compute.manager [None req-684945de-57fe-475b-b990-ae7897d5ffb4 tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] [instance: aa3c7432-bf15-408e-9fbb-993135734161] Took 11.33 seconds to build instance.
Nov 29 03:06:41 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-684945de-57fe-475b-b990-ae7897d5ffb4 tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] Lock "aa3c7432-bf15-408e-9fbb-993135734161" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 11.538s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:06:41 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:06:41 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:06:41 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:06:42 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:06:42 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:06:42 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:06:42 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:06:42 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:06:43 user nova-compute[75234]: DEBUG nova.compute.manager [req-18861a5c-b910-41e7-ab31-24b37b36ec10 req-57fb883c-59a5-47ab-9cb0-5e2b32548e2e service nova] [instance: 2bf9fe41-63c2-4882-bcdd-8e314e734ffe] Received event network-vif-plugged-c8bb20fd-5c85-4e89-8c61-13a4f17dd4cc {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:06:43 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-18861a5c-b910-41e7-ab31-24b37b36ec10 req-57fb883c-59a5-47ab-9cb0-5e2b32548e2e service nova] Acquiring lock "2bf9fe41-63c2-4882-bcdd-8e314e734ffe-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:06:43 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-18861a5c-b910-41e7-ab31-24b37b36ec10 req-57fb883c-59a5-47ab-9cb0-5e2b32548e2e service nova] Lock "2bf9fe41-63c2-4882-bcdd-8e314e734ffe-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:06:43 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-18861a5c-b910-41e7-ab31-24b37b36ec10 req-57fb883c-59a5-47ab-9cb0-5e2b32548e2e service nova] Lock "2bf9fe41-63c2-4882-bcdd-8e314e734ffe-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:06:43 user nova-compute[75234]: DEBUG nova.compute.manager [req-18861a5c-b910-41e7-ab31-24b37b36ec10 req-57fb883c-59a5-47ab-9cb0-5e2b32548e2e service nova] [instance: 2bf9fe41-63c2-4882-bcdd-8e314e734ffe] No waiting events found dispatching network-vif-plugged-c8bb20fd-5c85-4e89-8c61-13a4f17dd4cc {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:06:43 user nova-compute[75234]: WARNING nova.compute.manager [req-18861a5c-b910-41e7-ab31-24b37b36ec10 req-57fb883c-59a5-47ab-9cb0-5e2b32548e2e service nova] [instance: 2bf9fe41-63c2-4882-bcdd-8e314e734ffe] Received unexpected event network-vif-plugged-c8bb20fd-5c85-4e89-8c61-13a4f17dd4cc for instance with vm_state active and task_state None.
Nov 29 03:06:44 user nova-compute[75234]: DEBUG nova.compute.manager [None req-40dafa9d-38c5-48f3-acf9-e33e8af5728b tempest-VolumesAssistedSnapshotsTest-1323460801 tempest-VolumesAssistedSnapshotsTest-1323460801-project-member] [instance: c9663ab3-3374-400e-ab0a-769731b78cca] Instance event wait completed in 0 seconds for  {{(pid=75234) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Nov 29 03:06:44 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-40dafa9d-38c5-48f3-acf9-e33e8af5728b tempest-VolumesAssistedSnapshotsTest-1323460801 tempest-VolumesAssistedSnapshotsTest-1323460801-project-member] [instance: c9663ab3-3374-400e-ab0a-769731b78cca] Guest created on hypervisor {{(pid=75234) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4421}}
Nov 29 03:06:44 user nova-compute[75234]: INFO nova.virt.libvirt.driver [-] [instance: c9663ab3-3374-400e-ab0a-769731b78cca] Instance spawned successfully.
Nov 29 03:06:44 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-40dafa9d-38c5-48f3-acf9-e33e8af5728b tempest-VolumesAssistedSnapshotsTest-1323460801 tempest-VolumesAssistedSnapshotsTest-1323460801-project-member] [instance: c9663ab3-3374-400e-ab0a-769731b78cca] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:904}}
Nov 29 03:06:44 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-40dafa9d-38c5-48f3-acf9-e33e8af5728b tempest-VolumesAssistedSnapshotsTest-1323460801 tempest-VolumesAssistedSnapshotsTest-1323460801-project-member] [instance: c9663ab3-3374-400e-ab0a-769731b78cca] Found default for hw_cdrom_bus of ide {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:06:44 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-40dafa9d-38c5-48f3-acf9-e33e8af5728b tempest-VolumesAssistedSnapshotsTest-1323460801 tempest-VolumesAssistedSnapshotsTest-1323460801-project-member] [instance: c9663ab3-3374-400e-ab0a-769731b78cca] Found default for hw_disk_bus of virtio {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:06:44 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-40dafa9d-38c5-48f3-acf9-e33e8af5728b tempest-VolumesAssistedSnapshotsTest-1323460801 tempest-VolumesAssistedSnapshotsTest-1323460801-project-member] [instance: c9663ab3-3374-400e-ab0a-769731b78cca] Found default for hw_input_bus of None {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:06:44 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-40dafa9d-38c5-48f3-acf9-e33e8af5728b tempest-VolumesAssistedSnapshotsTest-1323460801 tempest-VolumesAssistedSnapshotsTest-1323460801-project-member] [instance: c9663ab3-3374-400e-ab0a-769731b78cca] Found default for hw_pointer_model of None {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:06:44 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-40dafa9d-38c5-48f3-acf9-e33e8af5728b tempest-VolumesAssistedSnapshotsTest-1323460801 tempest-VolumesAssistedSnapshotsTest-1323460801-project-member] [instance: c9663ab3-3374-400e-ab0a-769731b78cca] Found default for hw_video_model of virtio {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:06:44 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-40dafa9d-38c5-48f3-acf9-e33e8af5728b tempest-VolumesAssistedSnapshotsTest-1323460801 tempest-VolumesAssistedSnapshotsTest-1323460801-project-member] [instance: c9663ab3-3374-400e-ab0a-769731b78cca] Found default for hw_vif_model of virtio {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:06:44 user nova-compute[75234]: INFO nova.compute.manager [None req-40dafa9d-38c5-48f3-acf9-e33e8af5728b tempest-VolumesAssistedSnapshotsTest-1323460801 tempest-VolumesAssistedSnapshotsTest-1323460801-project-member] [instance: c9663ab3-3374-400e-ab0a-769731b78cca] Took 6.93 seconds to spawn the instance on the hypervisor.
Nov 29 03:06:44 user nova-compute[75234]: DEBUG nova.compute.manager [None req-40dafa9d-38c5-48f3-acf9-e33e8af5728b tempest-VolumesAssistedSnapshotsTest-1323460801 tempest-VolumesAssistedSnapshotsTest-1323460801-project-member] [instance: c9663ab3-3374-400e-ab0a-769731b78cca] Checking state {{(pid=75234) _get_power_state /opt/stack/nova/nova/compute/manager.py:1782}}
Nov 29 03:06:44 user nova-compute[75234]: INFO nova.compute.manager [None req-40dafa9d-38c5-48f3-acf9-e33e8af5728b tempest-VolumesAssistedSnapshotsTest-1323460801 tempest-VolumesAssistedSnapshotsTest-1323460801-project-member] [instance: c9663ab3-3374-400e-ab0a-769731b78cca] Took 8.45 seconds to build instance.
Nov 29 03:06:44 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-40dafa9d-38c5-48f3-acf9-e33e8af5728b tempest-VolumesAssistedSnapshotsTest-1323460801 tempest-VolumesAssistedSnapshotsTest-1323460801-project-member] Lock "c9663ab3-3374-400e-ab0a-769731b78cca" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 8.744s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:06:45 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:06:45 user nova-compute[75234]: DEBUG nova.compute.manager [req-582a1d7a-e413-4a65-ab10-b89487ff68ef req-ee5b7e84-7cbe-46f3-b860-72d0b182c2eb service nova] [instance: da625f9c-b7ed-4afc-875c-6436bf58e9cc] Received event network-vif-plugged-a23b8769-d871-4d44-8cea-5d899f514579 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:06:45 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-582a1d7a-e413-4a65-ab10-b89487ff68ef req-ee5b7e84-7cbe-46f3-b860-72d0b182c2eb service nova] Acquiring lock "da625f9c-b7ed-4afc-875c-6436bf58e9cc-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:06:45 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-582a1d7a-e413-4a65-ab10-b89487ff68ef req-ee5b7e84-7cbe-46f3-b860-72d0b182c2eb service nova] Lock "da625f9c-b7ed-4afc-875c-6436bf58e9cc-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:06:45 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-582a1d7a-e413-4a65-ab10-b89487ff68ef req-ee5b7e84-7cbe-46f3-b860-72d0b182c2eb service nova] Lock "da625f9c-b7ed-4afc-875c-6436bf58e9cc-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:06:45 user nova-compute[75234]: DEBUG nova.compute.manager [req-582a1d7a-e413-4a65-ab10-b89487ff68ef req-ee5b7e84-7cbe-46f3-b860-72d0b182c2eb service nova] [instance: da625f9c-b7ed-4afc-875c-6436bf58e9cc] No waiting events found dispatching network-vif-plugged-a23b8769-d871-4d44-8cea-5d899f514579 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:06:45 user nova-compute[75234]: WARNING nova.compute.manager [req-582a1d7a-e413-4a65-ab10-b89487ff68ef req-ee5b7e84-7cbe-46f3-b860-72d0b182c2eb service nova] [instance: da625f9c-b7ed-4afc-875c-6436bf58e9cc] Received unexpected event network-vif-plugged-a23b8769-d871-4d44-8cea-5d899f514579 for instance with vm_state active and task_state None.
Nov 29 03:06:45 user nova-compute[75234]: DEBUG nova.compute.manager [req-5a62d5e9-eecd-43a2-9848-f7b289431d34 req-b2823afd-b8b2-43b5-a5df-7754cf8b32c3 service nova] [instance: 2bf9fe41-63c2-4882-bcdd-8e314e734ffe] Received event network-vif-plugged-c8bb20fd-5c85-4e89-8c61-13a4f17dd4cc {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:06:45 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-5a62d5e9-eecd-43a2-9848-f7b289431d34 req-b2823afd-b8b2-43b5-a5df-7754cf8b32c3 service nova] Acquiring lock "2bf9fe41-63c2-4882-bcdd-8e314e734ffe-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:06:45 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-5a62d5e9-eecd-43a2-9848-f7b289431d34 req-b2823afd-b8b2-43b5-a5df-7754cf8b32c3 service nova] Lock "2bf9fe41-63c2-4882-bcdd-8e314e734ffe-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.004s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:06:45 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-5a62d5e9-eecd-43a2-9848-f7b289431d34 req-b2823afd-b8b2-43b5-a5df-7754cf8b32c3 service nova] Lock "2bf9fe41-63c2-4882-bcdd-8e314e734ffe-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:06:45 user nova-compute[75234]: DEBUG nova.compute.manager [req-5a62d5e9-eecd-43a2-9848-f7b289431d34 req-b2823afd-b8b2-43b5-a5df-7754cf8b32c3 service nova] [instance: 2bf9fe41-63c2-4882-bcdd-8e314e734ffe] No waiting events found dispatching network-vif-plugged-c8bb20fd-5c85-4e89-8c61-13a4f17dd4cc {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:06:45 user nova-compute[75234]: WARNING nova.compute.manager [req-5a62d5e9-eecd-43a2-9848-f7b289431d34 req-b2823afd-b8b2-43b5-a5df-7754cf8b32c3 service nova] [instance: 2bf9fe41-63c2-4882-bcdd-8e314e734ffe] Received unexpected event network-vif-plugged-c8bb20fd-5c85-4e89-8c61-13a4f17dd4cc for instance with vm_state active and task_state None.
Nov 29 03:06:46 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:06:46 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._run_pending_deletes {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:06:46 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Cleaning up deleted instances {{(pid=75234) _run_pending_deletes /opt/stack/nova/nova/compute/manager.py:11209}}
Nov 29 03:06:46 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] There are 0 instances to clean {{(pid=75234) _run_pending_deletes /opt/stack/nova/nova/compute/manager.py:11218}}
Nov 29 03:06:46 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._cleanup_incomplete_migrations {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:06:46 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Cleaning up deleted instances with incomplete migration  {{(pid=75234) _cleanup_incomplete_migrations /opt/stack/nova/nova/compute/manager.py:11247}}
Nov 29 03:06:46 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._cleanup_expired_console_auth_tokens {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:06:47 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:06:47 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:06:47 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=75234) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10541}}
Nov 29 03:06:47 user nova-compute[75234]: DEBUG nova.compute.manager [req-c7a54586-1b6b-4423-8c5d-e21af75f8602 req-9f8d7a43-d3af-4a25-9cd1-888726d6f62c service nova] [instance: da625f9c-b7ed-4afc-875c-6436bf58e9cc] Received event network-vif-plugged-a23b8769-d871-4d44-8cea-5d899f514579 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:06:47 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-c7a54586-1b6b-4423-8c5d-e21af75f8602 req-9f8d7a43-d3af-4a25-9cd1-888726d6f62c service nova] Acquiring lock "da625f9c-b7ed-4afc-875c-6436bf58e9cc-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:06:47 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-c7a54586-1b6b-4423-8c5d-e21af75f8602 req-9f8d7a43-d3af-4a25-9cd1-888726d6f62c service nova] Lock "da625f9c-b7ed-4afc-875c-6436bf58e9cc-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:06:47 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-c7a54586-1b6b-4423-8c5d-e21af75f8602 req-9f8d7a43-d3af-4a25-9cd1-888726d6f62c service nova] Lock "da625f9c-b7ed-4afc-875c-6436bf58e9cc-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:06:47 user nova-compute[75234]: DEBUG nova.compute.manager [req-c7a54586-1b6b-4423-8c5d-e21af75f8602 req-9f8d7a43-d3af-4a25-9cd1-888726d6f62c service nova] [instance: da625f9c-b7ed-4afc-875c-6436bf58e9cc] No waiting events found dispatching network-vif-plugged-a23b8769-d871-4d44-8cea-5d899f514579 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:06:47 user nova-compute[75234]: WARNING nova.compute.manager [req-c7a54586-1b6b-4423-8c5d-e21af75f8602 req-9f8d7a43-d3af-4a25-9cd1-888726d6f62c service nova] [instance: da625f9c-b7ed-4afc-875c-6436bf58e9cc] Received unexpected event network-vif-plugged-a23b8769-d871-4d44-8cea-5d899f514579 for instance with vm_state active and task_state None.
Nov 29 03:06:47 user nova-compute[75234]: DEBUG nova.compute.manager [req-c7a54586-1b6b-4423-8c5d-e21af75f8602 req-9f8d7a43-d3af-4a25-9cd1-888726d6f62c service nova] [instance: c9663ab3-3374-400e-ab0a-769731b78cca] Received event network-vif-plugged-b55d6eaa-b0d4-4a67-8407-3a8b69558e43 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:06:47 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-c7a54586-1b6b-4423-8c5d-e21af75f8602 req-9f8d7a43-d3af-4a25-9cd1-888726d6f62c service nova] Acquiring lock "c9663ab3-3374-400e-ab0a-769731b78cca-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:06:47 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-c7a54586-1b6b-4423-8c5d-e21af75f8602 req-9f8d7a43-d3af-4a25-9cd1-888726d6f62c service nova] Lock "c9663ab3-3374-400e-ab0a-769731b78cca-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:06:47 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-c7a54586-1b6b-4423-8c5d-e21af75f8602 req-9f8d7a43-d3af-4a25-9cd1-888726d6f62c service nova] Lock "c9663ab3-3374-400e-ab0a-769731b78cca-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:06:47 user nova-compute[75234]: DEBUG nova.compute.manager [req-c7a54586-1b6b-4423-8c5d-e21af75f8602 req-9f8d7a43-d3af-4a25-9cd1-888726d6f62c service nova] [instance: c9663ab3-3374-400e-ab0a-769731b78cca] No waiting events found dispatching network-vif-plugged-b55d6eaa-b0d4-4a67-8407-3a8b69558e43 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:06:47 user nova-compute[75234]: WARNING nova.compute.manager [req-c7a54586-1b6b-4423-8c5d-e21af75f8602 req-9f8d7a43-d3af-4a25-9cd1-888726d6f62c service nova] [instance: c9663ab3-3374-400e-ab0a-769731b78cca] Received unexpected event network-vif-plugged-b55d6eaa-b0d4-4a67-8407-3a8b69558e43 for instance with vm_state active and task_state None.
Nov 29 03:06:47 user nova-compute[75234]: DEBUG nova.compute.manager [req-c7a54586-1b6b-4423-8c5d-e21af75f8602 req-9f8d7a43-d3af-4a25-9cd1-888726d6f62c service nova] [instance: c9663ab3-3374-400e-ab0a-769731b78cca] Received event network-vif-plugged-b55d6eaa-b0d4-4a67-8407-3a8b69558e43 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:06:47 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-c7a54586-1b6b-4423-8c5d-e21af75f8602 req-9f8d7a43-d3af-4a25-9cd1-888726d6f62c service nova] Acquiring lock "c9663ab3-3374-400e-ab0a-769731b78cca-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:06:47 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-c7a54586-1b6b-4423-8c5d-e21af75f8602 req-9f8d7a43-d3af-4a25-9cd1-888726d6f62c service nova] Lock "c9663ab3-3374-400e-ab0a-769731b78cca-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:06:47 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-c7a54586-1b6b-4423-8c5d-e21af75f8602 req-9f8d7a43-d3af-4a25-9cd1-888726d6f62c service nova] Lock "c9663ab3-3374-400e-ab0a-769731b78cca-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:06:47 user nova-compute[75234]: DEBUG nova.compute.manager [req-c7a54586-1b6b-4423-8c5d-e21af75f8602 req-9f8d7a43-d3af-4a25-9cd1-888726d6f62c service nova] [instance: c9663ab3-3374-400e-ab0a-769731b78cca] No waiting events found dispatching network-vif-plugged-b55d6eaa-b0d4-4a67-8407-3a8b69558e43 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:06:47 user nova-compute[75234]: WARNING nova.compute.manager [req-c7a54586-1b6b-4423-8c5d-e21af75f8602 req-9f8d7a43-d3af-4a25-9cd1-888726d6f62c service nova] [instance: c9663ab3-3374-400e-ab0a-769731b78cca] Received unexpected event network-vif-plugged-b55d6eaa-b0d4-4a67-8407-3a8b69558e43 for instance with vm_state active and task_state None.
Nov 29 03:06:49 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:06:49 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:06:49 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Starting heal instance info cache {{(pid=75234) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9922}}
Nov 29 03:06:49 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Rebuilding the list of instances to heal {{(pid=75234) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9926}}
Nov 29 03:06:49 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Acquiring lock "refresh_cache-9f7876ad-0487-426d-8b19-b718dccdc4fe" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:06:49 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Acquired lock "refresh_cache-9f7876ad-0487-426d-8b19-b718dccdc4fe" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:06:49 user nova-compute[75234]: DEBUG nova.network.neutron [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] [instance: 9f7876ad-0487-426d-8b19-b718dccdc4fe] Forcefully refreshing network info cache for instance {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2003}}
Nov 29 03:06:49 user nova-compute[75234]: DEBUG nova.objects.instance [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lazy-loading 'info_cache' on Instance uuid 9f7876ad-0487-426d-8b19-b718dccdc4fe {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:06:50 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:06:50 user nova-compute[75234]: DEBUG nova.network.neutron [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] [instance: 9f7876ad-0487-426d-8b19-b718dccdc4fe] Updating instance_info_cache with network_info: [{"id": "7dd048ce-2c9b-46ac-8e9d-8c6a2ab201f5", "address": "fa:16:3e:fd:bb:d1", "network": {"id": "c810861d-949e-423d-a3b5-a37f66f29342", "bridge": "br-int", "label": "tempest-DeleteServersTestJSON-2024350588-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a21183b0b4f947e39218bb8e7c857e50", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap7dd048ce-2c", "ovs_interfaceid": "7dd048ce-2c9b-46ac-8e9d-8c6a2ab201f5", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:06:50 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Releasing lock "refresh_cache-9f7876ad-0487-426d-8b19-b718dccdc4fe" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:06:50 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] [instance: 9f7876ad-0487-426d-8b19-b718dccdc4fe] Updated the network info_cache for instance {{(pid=75234) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9993}}
Nov 29 03:06:50 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:06:51 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:06:51 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:06:51 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager.update_available_resource {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:06:51 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:06:51 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:06:51 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:06:51 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Auditing locally available compute resources for user (node: user) {{(pid=75234) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:907}}
Nov 29 03:06:51 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a9350a18-14e6-42c6-a150-0a1487f20ad9/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:06:51 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a9350a18-14e6-42c6-a150-0a1487f20ad9/disk --force-share --output=json" returned: 0 in 0.195s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:06:51 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a9350a18-14e6-42c6-a150-0a1487f20ad9/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:06:52 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a9350a18-14e6-42c6-a150-0a1487f20ad9/disk --force-share --output=json" returned: 0 in 0.183s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:06:52 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/8297c264-a213-44ee-be46-5c08eabda9f7/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:06:52 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/8297c264-a213-44ee-be46-5c08eabda9f7/disk --force-share --output=json" returned: 0 in 0.180s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:06:52 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/8297c264-a213-44ee-be46-5c08eabda9f7/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:06:52 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/8297c264-a213-44ee-be46-5c08eabda9f7/disk --force-share --output=json" returned: 0 in 0.239s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:06:52 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/da625f9c-b7ed-4afc-875c-6436bf58e9cc/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:06:52 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/da625f9c-b7ed-4afc-875c-6436bf58e9cc/disk --force-share --output=json" returned: 0 in 0.174s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:06:52 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/da625f9c-b7ed-4afc-875c-6436bf58e9cc/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:06:52 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/da625f9c-b7ed-4afc-875c-6436bf58e9cc/disk --force-share --output=json" returned: 0 in 0.202s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:06:52 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2bf9fe41-63c2-4882-bcdd-8e314e734ffe/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:06:53 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2bf9fe41-63c2-4882-bcdd-8e314e734ffe/disk --force-share --output=json" returned: 0 in 0.185s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:06:53 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2bf9fe41-63c2-4882-bcdd-8e314e734ffe/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:06:53 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2bf9fe41-63c2-4882-bcdd-8e314e734ffe/disk --force-share --output=json" returned: 0 in 0.209s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:06:53 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/9f7876ad-0487-426d-8b19-b718dccdc4fe/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:06:53 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/9f7876ad-0487-426d-8b19-b718dccdc4fe/disk --force-share --output=json" returned: 0 in 0.194s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:06:53 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/9f7876ad-0487-426d-8b19-b718dccdc4fe/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:06:53 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/9f7876ad-0487-426d-8b19-b718dccdc4fe/disk --force-share --output=json" returned: 0 in 0.185s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:06:53 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/21e9c6cd-c39f-41c0-8768-5dd64589c3f3/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:06:53 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/21e9c6cd-c39f-41c0-8768-5dd64589c3f3/disk --force-share --output=json" returned: 0 in 0.220s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:06:53 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/21e9c6cd-c39f-41c0-8768-5dd64589c3f3/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:06:54 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/21e9c6cd-c39f-41c0-8768-5dd64589c3f3/disk --force-share --output=json" returned: 0 in 0.180s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:06:54 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f609a7b7-1be2-44b4-9177-7e72a4d4c060/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:06:54 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f609a7b7-1be2-44b4-9177-7e72a4d4c060/disk --force-share --output=json" returned: 0 in 0.176s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:06:54 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f609a7b7-1be2-44b4-9177-7e72a4d4c060/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:06:54 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f609a7b7-1be2-44b4-9177-7e72a4d4c060/disk --force-share --output=json" returned: 0 in 0.212s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:06:54 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/c9663ab3-3374-400e-ab0a-769731b78cca/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:06:54 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/c9663ab3-3374-400e-ab0a-769731b78cca/disk --force-share --output=json" returned: 0 in 0.166s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:06:54 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/c9663ab3-3374-400e-ab0a-769731b78cca/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:06:55 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/c9663ab3-3374-400e-ab0a-769731b78cca/disk --force-share --output=json" returned: 0 in 0.253s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:06:55 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/aa3c7432-bf15-408e-9fbb-993135734161/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:06:55 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/aa3c7432-bf15-408e-9fbb-993135734161/disk --force-share --output=json" returned: 0 in 0.174s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:06:55 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/aa3c7432-bf15-408e-9fbb-993135734161/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:06:55 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:06:55 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/aa3c7432-bf15-408e-9fbb-993135734161/disk --force-share --output=json" returned: 0 in 0.181s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:06:55 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:06:55 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:06:55 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Hypervisor/Node resource view: name=user free_ram=7442MB free_disk=25.69886016845703GB free_vcpus=3 pci_devices=[{"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}] {{(pid=75234) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1106}}
Nov 29 03:06:55 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:06:55 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:06:56 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:06:56 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance 9f7876ad-0487-426d-8b19-b718dccdc4fe actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:06:56 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance 2bf9fe41-63c2-4882-bcdd-8e314e734ffe actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:06:56 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance 21e9c6cd-c39f-41c0-8768-5dd64589c3f3 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:06:56 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance a9350a18-14e6-42c6-a150-0a1487f20ad9 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:06:56 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance f609a7b7-1be2-44b4-9177-7e72a4d4c060 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:06:56 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance 8297c264-a213-44ee-be46-5c08eabda9f7 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:06:56 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance da625f9c-b7ed-4afc-875c-6436bf58e9cc actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:06:56 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance aa3c7432-bf15-408e-9fbb-993135734161 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:06:56 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance c9663ab3-3374-400e-ab0a-769731b78cca actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:06:56 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Total usable vcpus: 12, total allocated vcpus: 9 {{(pid=75234) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1129}}
Nov 29 03:06:56 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Final resource view: name=user phys_ram=16011MB used_ram=5120MB phys_disk=40GB used_disk=9GB total_vcpus=12 used_vcpus=9 pci_stats=[] {{(pid=75234) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1138}}
Nov 29 03:06:56 user nova-compute[75234]: DEBUG nova.scheduler.client.report [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Refreshing inventories for resource provider 4d291220-7c37-4839-a0e3-b23c66502b74 {{(pid=75234) _refresh_associations /opt/stack/nova/nova/scheduler/client/report.py:804}}
Nov 29 03:06:56 user nova-compute[75234]: DEBUG nova.scheduler.client.report [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Updating ProviderTree inventory for provider 4d291220-7c37-4839-a0e3-b23c66502b74 from _refresh_and_get_inventory using data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=75234) _refresh_and_get_inventory /opt/stack/nova/nova/scheduler/client/report.py:768}}
Nov 29 03:06:56 user nova-compute[75234]: DEBUG nova.compute.provider_tree [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Updating inventory in ProviderTree for provider 4d291220-7c37-4839-a0e3-b23c66502b74 with inventory: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=75234) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:176}}
Nov 29 03:06:56 user nova-compute[75234]: DEBUG nova.scheduler.client.report [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Refreshing aggregate associations for resource provider 4d291220-7c37-4839-a0e3-b23c66502b74, aggregates: None {{(pid=75234) _refresh_associations /opt/stack/nova/nova/scheduler/client/report.py:813}}
Nov 29 03:06:56 user nova-compute[75234]: DEBUG nova.compute.manager [req-4c3ade05-50df-43c7-be93-139786fd1a88 req-8a2e1271-5338-4eb6-a716-7b1e464b0087 service nova] [instance: aa3c7432-bf15-408e-9fbb-993135734161] Received event network-vif-plugged-aaac2504-1067-43c5-b3bf-316996e0f56c {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:06:56 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-4c3ade05-50df-43c7-be93-139786fd1a88 req-8a2e1271-5338-4eb6-a716-7b1e464b0087 service nova] Acquiring lock "aa3c7432-bf15-408e-9fbb-993135734161-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:06:56 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-4c3ade05-50df-43c7-be93-139786fd1a88 req-8a2e1271-5338-4eb6-a716-7b1e464b0087 service nova] Lock "aa3c7432-bf15-408e-9fbb-993135734161-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:06:56 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-4c3ade05-50df-43c7-be93-139786fd1a88 req-8a2e1271-5338-4eb6-a716-7b1e464b0087 service nova] Lock "aa3c7432-bf15-408e-9fbb-993135734161-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:06:56 user nova-compute[75234]: DEBUG nova.compute.manager [req-4c3ade05-50df-43c7-be93-139786fd1a88 req-8a2e1271-5338-4eb6-a716-7b1e464b0087 service nova] [instance: aa3c7432-bf15-408e-9fbb-993135734161] No waiting events found dispatching network-vif-plugged-aaac2504-1067-43c5-b3bf-316996e0f56c {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:06:56 user nova-compute[75234]: WARNING nova.compute.manager [req-4c3ade05-50df-43c7-be93-139786fd1a88 req-8a2e1271-5338-4eb6-a716-7b1e464b0087 service nova] [instance: aa3c7432-bf15-408e-9fbb-993135734161] Received unexpected event network-vif-plugged-aaac2504-1067-43c5-b3bf-316996e0f56c for instance with vm_state active and task_state None.
Nov 29 03:06:56 user nova-compute[75234]: DEBUG nova.scheduler.client.report [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Refreshing trait associations for resource provider 4d291220-7c37-4839-a0e3-b23c66502b74, traits: COMPUTE_IMAGE_TYPE_AMI,COMPUTE_GRAPHICS_MODEL_CIRRUS,COMPUTE_NET_VIF_MODEL_VMXNET3,COMPUTE_STORAGE_BUS_VIRTIO,COMPUTE_GRAPHICS_MODEL_NONE,COMPUTE_IMAGE_TYPE_ISO,COMPUTE_NET_VIF_MODEL_RTL8139,HW_CPU_X86_SSE,COMPUTE_IMAGE_TYPE_QCOW2,COMPUTE_SOCKET_PCI_NUMA_AFFINITY,COMPUTE_GRAPHICS_MODEL_BOCHS,COMPUTE_VOLUME_ATTACH_WITH_TAG,COMPUTE_ACCELERATORS,HW_CPU_X86_SSE2,COMPUTE_STORAGE_BUS_SCSI,COMPUTE_GRAPHICS_MODEL_VGA,COMPUTE_NET_ATTACH_INTERFACE_WITH_TAG,COMPUTE_SECURITY_UEFI_SECURE_BOOT,HW_CPU_X86_SSE41,COMPUTE_NET_VIF_MODEL_E1000E,COMPUTE_TRUSTED_CERTS,COMPUTE_STORAGE_BUS_FDC,COMPUTE_VIOMMU_MODEL_INTEL,COMPUTE_GRAPHICS_MODEL_VIRTIO,COMPUTE_VIOMMU_MODEL_AUTO,COMPUTE_IMAGE_TYPE_AKI,COMPUTE_NET_VIF_MODEL_PCNET,COMPUTE_GRAPHICS_MODEL_QXL,COMPUTE_NET_VIF_MODEL_VIRTIO,COMPUTE_GRAPHICS_MODEL_VMVGA,COMPUTE_VOLUME_MULTI_ATTACH,COMPUTE_STORAGE_BUS_SATA,HW_CPU_X86_MMX,COMPUTE_NET_VIF_MODEL_E1000,COMPUTE_IMAGE_TYPE_ARI,COMPUTE_STORAGE_BUS_IDE,COMPUTE_NODE,HW_CPU_X86_SSSE3,COMPUTE_RESCUE_BFV,COMPUTE_NET_ATTACH_INTERFACE,COMPUTE_STORAGE_BUS_USB,COMPUTE_NET_VIF_MODEL_SPAPR_VLAN,COMPUTE_IMAGE_TYPE_RAW,COMPUTE_NET_VIF_MODEL_NE2K_PCI,COMPUTE_VOLUME_EXTEND,COMPUTE_DEVICE_TAGGING,HW_CPU_X86_SSE42 {{(pid=75234) _refresh_associations /opt/stack/nova/nova/scheduler/client/report.py:825}}
Nov 29 03:06:57 user nova-compute[75234]: DEBUG nova.compute.provider_tree [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Inventory has not changed in ProviderTree for provider: 4d291220-7c37-4839-a0e3-b23c66502b74 {{(pid=75234) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Nov 29 03:06:57 user nova-compute[75234]: DEBUG nova.scheduler.client.report [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Inventory has not changed for provider 4d291220-7c37-4839-a0e3-b23c66502b74 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=75234) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Nov 29 03:06:57 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Compute_service record updated for user:user {{(pid=75234) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:1067}}
Nov 29 03:06:57 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 1.303s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:06:58 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:06:58 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:06:58 user nova-compute[75234]: DEBUG nova.compute.manager [req-e3272281-4e6d-4c6b-9bfd-a4c50eb9f7aa req-cf7c70d1-b63e-4e6a-bf43-ccc5e292513f service nova] [instance: aa3c7432-bf15-408e-9fbb-993135734161] Received event network-vif-plugged-aaac2504-1067-43c5-b3bf-316996e0f56c {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:06:58 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-e3272281-4e6d-4c6b-9bfd-a4c50eb9f7aa req-cf7c70d1-b63e-4e6a-bf43-ccc5e292513f service nova] Acquiring lock "aa3c7432-bf15-408e-9fbb-993135734161-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:06:58 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-e3272281-4e6d-4c6b-9bfd-a4c50eb9f7aa req-cf7c70d1-b63e-4e6a-bf43-ccc5e292513f service nova] Lock "aa3c7432-bf15-408e-9fbb-993135734161-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:06:58 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-e3272281-4e6d-4c6b-9bfd-a4c50eb9f7aa req-cf7c70d1-b63e-4e6a-bf43-ccc5e292513f service nova] Lock "aa3c7432-bf15-408e-9fbb-993135734161-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:06:58 user nova-compute[75234]: DEBUG nova.compute.manager [req-e3272281-4e6d-4c6b-9bfd-a4c50eb9f7aa req-cf7c70d1-b63e-4e6a-bf43-ccc5e292513f service nova] [instance: aa3c7432-bf15-408e-9fbb-993135734161] No waiting events found dispatching network-vif-plugged-aaac2504-1067-43c5-b3bf-316996e0f56c {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:06:58 user nova-compute[75234]: WARNING nova.compute.manager [req-e3272281-4e6d-4c6b-9bfd-a4c50eb9f7aa req-cf7c70d1-b63e-4e6a-bf43-ccc5e292513f service nova] [instance: aa3c7432-bf15-408e-9fbb-993135734161] Received unexpected event network-vif-plugged-aaac2504-1067-43c5-b3bf-316996e0f56c for instance with vm_state active and task_state None.
Nov 29 03:06:59 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] Acquiring lock "cb569e8d-e4a5-4076-bb8b-deeefad033e2" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:06:59 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] Lock "cb569e8d-e4a5-4076-bb8b-deeefad033e2" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:06:59 user nova-compute[75234]: DEBUG nova.compute.manager [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] [instance: cb569e8d-e4a5-4076-bb8b-deeefad033e2] Starting instance... {{(pid=75234) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2422}}
Nov 29 03:06:59 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:06:59 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:06:59 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=75234) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2366}}
Nov 29 03:06:59 user nova-compute[75234]: INFO nova.compute.claims [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] [instance: cb569e8d-e4a5-4076-bb8b-deeefad033e2] Claim successful on node user
Nov 29 03:06:59 user nova-compute[75234]: DEBUG nova.compute.provider_tree [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] Inventory has not changed in ProviderTree for provider: 4d291220-7c37-4839-a0e3-b23c66502b74 {{(pid=75234) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Nov 29 03:06:59 user nova-compute[75234]: DEBUG nova.scheduler.client.report [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] Inventory has not changed for provider 4d291220-7c37-4839-a0e3-b23c66502b74 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=75234) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Nov 29 03:07:00 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.553s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:07:00 user nova-compute[75234]: DEBUG nova.compute.manager [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] [instance: cb569e8d-e4a5-4076-bb8b-deeefad033e2] Start building networks asynchronously for instance. {{(pid=75234) _build_resources /opt/stack/nova/nova/compute/manager.py:2819}}
Nov 29 03:07:00 user nova-compute[75234]: DEBUG nova.compute.manager [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] [instance: cb569e8d-e4a5-4076-bb8b-deeefad033e2] Allocating IP information in the background. {{(pid=75234) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1972}}
Nov 29 03:07:00 user nova-compute[75234]: DEBUG nova.network.neutron [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] [instance: cb569e8d-e4a5-4076-bb8b-deeefad033e2] allocate_for_instance() {{(pid=75234) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1156}}
Nov 29 03:07:00 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] [instance: cb569e8d-e4a5-4076-bb8b-deeefad033e2] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Nov 29 03:07:00 user nova-compute[75234]: DEBUG nova.compute.manager [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] [instance: cb569e8d-e4a5-4076-bb8b-deeefad033e2] Start building block device mappings for instance. {{(pid=75234) _build_resources /opt/stack/nova/nova/compute/manager.py:2854}}
Nov 29 03:07:00 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:00 user nova-compute[75234]: INFO nova.virt.block_device [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] [instance: cb569e8d-e4a5-4076-bb8b-deeefad033e2] Booting with volume bb98586e-4f75-46be-a626-0b73ebda11eb at /dev/vda
Nov 29 03:07:00 user nova-compute[75234]: DEBUG os_brick.utils [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:176}}
Nov 29 03:07:00 user nova-compute[75234]: INFO oslo.privsep.daemon [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] Running privsep helper: ['sudo', 'nova-rootwrap', '/etc/nova/rootwrap.conf', 'privsep-helper', '--config-file', '/etc/nova/nova-cpu.conf', '--privsep_context', 'os_brick.privileged.default', '--privsep_sock_path', '/tmp/tmp7baec_g3/privsep.sock']
Nov 29 03:07:00 user nova-compute[75234]: DEBUG nova.policy [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '36c915d13a4943549549177c756da93b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'ee1117bd3511400f9f45cc8f9bb9247d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=75234) authorize /opt/stack/nova/nova/policy.py:203}}
Nov 29 03:07:00 user sudo[86321]:    stack : PWD=/ ; USER=root ; COMMAND=/opt/stack/data/venv/bin/nova-rootwrap /etc/nova/rootwrap.conf privsep-helper --config-file /etc/nova/nova-cpu.conf --privsep_context os_brick.privileged.default --privsep_sock_path /tmp/tmp7baec_g3/privsep.sock
Nov 29 03:07:00 user sudo[86321]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=1001)
Nov 29 03:07:01 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:01 user nova-compute[75234]: DEBUG nova.network.neutron [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] [instance: cb569e8d-e4a5-4076-bb8b-deeefad033e2] Successfully created port: 2b12ec96-26fa-4974-8a7a-22e85ec20544 {{(pid=75234) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:548}}
Nov 29 03:07:01 user nova-compute[75234]: DEBUG nova.network.neutron [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] [instance: cb569e8d-e4a5-4076-bb8b-deeefad033e2] Successfully created port: 026f606a-534c-4cdd-992a-9f33268b374f {{(pid=75234) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:548}}
Nov 29 03:07:02 user sudo[86321]: pam_unix(sudo:session): session closed for user root
Nov 29 03:07:02 user nova-compute[75234]: INFO oslo.privsep.daemon [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] Spawned new privsep daemon via rootwrap
Nov 29 03:07:02 user nova-compute[75234]: INFO oslo.privsep.daemon [-] privsep daemon starting
Nov 29 03:07:02 user nova-compute[75234]: INFO oslo.privsep.daemon [-] privsep process running with uid/gid: 0/0
Nov 29 03:07:02 user nova-compute[75234]: INFO oslo.privsep.daemon [-] privsep process running with capabilities (eff/prm/inh): CAP_SYS_ADMIN/CAP_SYS_ADMIN/none
Nov 29 03:07:02 user nova-compute[75234]: INFO oslo.privsep.daemon [-] privsep daemon running as pid 86324
Nov 29 03:07:02 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[49648219-b976-4605-82e3-7797cd14314c]: (2,) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:07:02 user nova-compute[75234]: DEBUG nova.network.neutron [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] [instance: cb569e8d-e4a5-4076-bb8b-deeefad033e2] Successfully created port: 67446920-3741-4542-84c6-493388395f26 {{(pid=75234) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:548}}
Nov 29 03:07:02 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:07:02 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.012s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:07:02 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[180e30e3-8ecc-4bb3-898d-37ef1b235b1e]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:dd1368c5ba2c\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:07:02 user nova-compute[75234]: WARNING os_brick.initiator.connectors.nvmeof [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] Could not find nvme_core/parameters/multipath: FileNotFoundError: [Errno 2] No such file or directory: '/sys/module/nvme_core/parameters/multipath'
Nov 29 03:07:02 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt -v / -n -o SOURCE {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:07:02 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt -v / -n -o SOURCE" returned: 0 in 0.016s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:07:02 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[8ae56860-8980-4887-bc80-8cddf0a5cea2]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:07:02 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:07:02 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.015s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:07:02 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[9d10cadf-f8b9-4f33-a5e2-b28b39e76e6d]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:07:02 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[edf471af-5233-40af-9fb4-76ace8081c2d]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:07:02 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] Running cmd (subprocess): nvme version {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:07:02 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] 'nvme version' failed. Not Retrying. {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:473}}
Nov 29 03:07:02 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.nvmeof [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] nvme not present on system {{(pid=75234) nvme_present /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/nvmeof.py:756}}
Nov 29 03:07:02 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:07:02 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:473}}
Nov 29 03:07:02 user nova-compute[75234]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Nov 29 03:07:02 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[b1cf48db-8127-4820-a289-524abee08ab2]: (4, '') {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:07:02 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.lightos [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=75234) find_dsc /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/lightos.py:98}}
Nov 29 03:07:02 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.lightos [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=75234) get_connector_properties /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/lightos.py:76}}
Nov 29 03:07:02 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.lightos [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] LIGHTOS: no hostnqn found. {{(pid=75234) get_connector_properties /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/lightos.py:84}}
Nov 29 03:07:02 user nova-compute[75234]: DEBUG os_brick.utils [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] <== get_connector_properties: return (2066ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:dd1368c5ba2c', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:203}}
Nov 29 03:07:02 user nova-compute[75234]: DEBUG nova.virt.block_device [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] [instance: cb569e8d-e4a5-4076-bb8b-deeefad033e2] Updating existing volume attachment record: 62d5b7f3-07b7-4276-8480-7caf07b59a10 {{(pid=75234) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Nov 29 03:07:02 user nova-compute[75234]: DEBUG nova.network.neutron [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] [instance: cb569e8d-e4a5-4076-bb8b-deeefad033e2] Successfully created port: d7e92753-492b-4fc0-8048-f4792cd3b4b6 {{(pid=75234) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:548}}
Nov 29 03:07:03 user nova-compute[75234]: DEBUG nova.network.neutron [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] [instance: cb569e8d-e4a5-4076-bb8b-deeefad033e2] Successfully created port: 06db3012-0042-457a-a3c8-14e3a8d2e89e {{(pid=75234) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:548}}
Nov 29 03:07:04 user nova-compute[75234]: DEBUG nova.network.neutron [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] [instance: cb569e8d-e4a5-4076-bb8b-deeefad033e2] Successfully updated port: 2b12ec96-26fa-4974-8a7a-22e85ec20544 {{(pid=75234) _update_port /opt/stack/nova/nova/network/neutron.py:586}}
Nov 29 03:07:04 user nova-compute[75234]: DEBUG nova.compute.manager [req-0b92a65b-cdc3-41fd-8bfa-7416f224c350 req-afc72f4f-50f8-48d6-ab6f-3a9459cd62c1 service nova] [instance: cb569e8d-e4a5-4076-bb8b-deeefad033e2] Received event network-changed-2b12ec96-26fa-4974-8a7a-22e85ec20544 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:07:04 user nova-compute[75234]: DEBUG nova.compute.manager [req-0b92a65b-cdc3-41fd-8bfa-7416f224c350 req-afc72f4f-50f8-48d6-ab6f-3a9459cd62c1 service nova] [instance: cb569e8d-e4a5-4076-bb8b-deeefad033e2] Refreshing instance network info cache due to event network-changed-2b12ec96-26fa-4974-8a7a-22e85ec20544. {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11117}}
Nov 29 03:07:04 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-0b92a65b-cdc3-41fd-8bfa-7416f224c350 req-afc72f4f-50f8-48d6-ab6f-3a9459cd62c1 service nova] Acquiring lock "refresh_cache-cb569e8d-e4a5-4076-bb8b-deeefad033e2" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:07:04 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-0b92a65b-cdc3-41fd-8bfa-7416f224c350 req-afc72f4f-50f8-48d6-ab6f-3a9459cd62c1 service nova] Acquired lock "refresh_cache-cb569e8d-e4a5-4076-bb8b-deeefad033e2" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:07:04 user nova-compute[75234]: DEBUG nova.network.neutron [req-0b92a65b-cdc3-41fd-8bfa-7416f224c350 req-afc72f4f-50f8-48d6-ab6f-3a9459cd62c1 service nova] [instance: cb569e8d-e4a5-4076-bb8b-deeefad033e2] Refreshing network info cache for port 2b12ec96-26fa-4974-8a7a-22e85ec20544 {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2006}}
Nov 29 03:07:04 user nova-compute[75234]: DEBUG nova.network.neutron [req-0b92a65b-cdc3-41fd-8bfa-7416f224c350 req-afc72f4f-50f8-48d6-ab6f-3a9459cd62c1 service nova] [instance: cb569e8d-e4a5-4076-bb8b-deeefad033e2] Instance cache missing network info. {{(pid=75234) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3322}}
Nov 29 03:07:05 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:05 user nova-compute[75234]: DEBUG nova.network.neutron [req-0b92a65b-cdc3-41fd-8bfa-7416f224c350 req-afc72f4f-50f8-48d6-ab6f-3a9459cd62c1 service nova] [instance: cb569e8d-e4a5-4076-bb8b-deeefad033e2] Updating instance_info_cache with network_info: [] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:07:05 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-0b92a65b-cdc3-41fd-8bfa-7416f224c350 req-afc72f4f-50f8-48d6-ab6f-3a9459cd62c1 service nova] Releasing lock "refresh_cache-cb569e8d-e4a5-4076-bb8b-deeefad033e2" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:07:06 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:06 user nova-compute[75234]: DEBUG nova.network.neutron [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] [instance: cb569e8d-e4a5-4076-bb8b-deeefad033e2] Successfully updated port: 0bd5d735-f566-4a6d-b49f-b24fd403a87a {{(pid=75234) _update_port /opt/stack/nova/nova/network/neutron.py:586}}
Nov 29 03:07:06 user nova-compute[75234]: DEBUG nova.compute.manager [req-51fa6ab6-72f2-4ab7-a69b-add70a42bbec req-fadd66c7-3270-4e19-a1d5-311f85e719c4 service nova] [instance: cb569e8d-e4a5-4076-bb8b-deeefad033e2] Received event network-changed-0bd5d735-f566-4a6d-b49f-b24fd403a87a {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:07:06 user nova-compute[75234]: DEBUG nova.compute.manager [req-51fa6ab6-72f2-4ab7-a69b-add70a42bbec req-fadd66c7-3270-4e19-a1d5-311f85e719c4 service nova] [instance: cb569e8d-e4a5-4076-bb8b-deeefad033e2] Refreshing instance network info cache due to event network-changed-0bd5d735-f566-4a6d-b49f-b24fd403a87a. {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11117}}
Nov 29 03:07:06 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-51fa6ab6-72f2-4ab7-a69b-add70a42bbec req-fadd66c7-3270-4e19-a1d5-311f85e719c4 service nova] Acquiring lock "refresh_cache-cb569e8d-e4a5-4076-bb8b-deeefad033e2" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:07:06 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-51fa6ab6-72f2-4ab7-a69b-add70a42bbec req-fadd66c7-3270-4e19-a1d5-311f85e719c4 service nova] Acquired lock "refresh_cache-cb569e8d-e4a5-4076-bb8b-deeefad033e2" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:07:06 user nova-compute[75234]: DEBUG nova.network.neutron [req-51fa6ab6-72f2-4ab7-a69b-add70a42bbec req-fadd66c7-3270-4e19-a1d5-311f85e719c4 service nova] [instance: cb569e8d-e4a5-4076-bb8b-deeefad033e2] Refreshing network info cache for port 0bd5d735-f566-4a6d-b49f-b24fd403a87a {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2006}}
Nov 29 03:07:07 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:07 user nova-compute[75234]: DEBUG nova.network.neutron [req-51fa6ab6-72f2-4ab7-a69b-add70a42bbec req-fadd66c7-3270-4e19-a1d5-311f85e719c4 service nova] [instance: cb569e8d-e4a5-4076-bb8b-deeefad033e2] Instance cache missing network info. {{(pid=75234) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3322}}
Nov 29 03:07:07 user nova-compute[75234]: DEBUG nova.network.neutron [req-51fa6ab6-72f2-4ab7-a69b-add70a42bbec req-fadd66c7-3270-4e19-a1d5-311f85e719c4 service nova] [instance: cb569e8d-e4a5-4076-bb8b-deeefad033e2] Updating instance_info_cache with network_info: [] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:07:07 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-51fa6ab6-72f2-4ab7-a69b-add70a42bbec req-fadd66c7-3270-4e19-a1d5-311f85e719c4 service nova] Releasing lock "refresh_cache-cb569e8d-e4a5-4076-bb8b-deeefad033e2" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:07:08 user nova-compute[75234]: DEBUG nova.network.neutron [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] [instance: cb569e8d-e4a5-4076-bb8b-deeefad033e2] Successfully updated port: 27e72461-f85b-42c9-8824-0d760b82a725 {{(pid=75234) _update_port /opt/stack/nova/nova/network/neutron.py:586}}
Nov 29 03:07:08 user nova-compute[75234]: DEBUG nova.compute.manager [req-5683e1ef-0602-40cc-9be0-b980c10c7830 req-abab86ae-0c58-4e25-9c87-b50fcf1ff10b service nova] [instance: cb569e8d-e4a5-4076-bb8b-deeefad033e2] Received event network-changed-27e72461-f85b-42c9-8824-0d760b82a725 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:07:08 user nova-compute[75234]: DEBUG nova.compute.manager [req-5683e1ef-0602-40cc-9be0-b980c10c7830 req-abab86ae-0c58-4e25-9c87-b50fcf1ff10b service nova] [instance: cb569e8d-e4a5-4076-bb8b-deeefad033e2] Refreshing instance network info cache due to event network-changed-27e72461-f85b-42c9-8824-0d760b82a725. {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11117}}
Nov 29 03:07:08 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-5683e1ef-0602-40cc-9be0-b980c10c7830 req-abab86ae-0c58-4e25-9c87-b50fcf1ff10b service nova] Acquiring lock "refresh_cache-cb569e8d-e4a5-4076-bb8b-deeefad033e2" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:07:08 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-5683e1ef-0602-40cc-9be0-b980c10c7830 req-abab86ae-0c58-4e25-9c87-b50fcf1ff10b service nova] Acquired lock "refresh_cache-cb569e8d-e4a5-4076-bb8b-deeefad033e2" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:07:08 user nova-compute[75234]: DEBUG nova.network.neutron [req-5683e1ef-0602-40cc-9be0-b980c10c7830 req-abab86ae-0c58-4e25-9c87-b50fcf1ff10b service nova] [instance: cb569e8d-e4a5-4076-bb8b-deeefad033e2] Refreshing network info cache for port 27e72461-f85b-42c9-8824-0d760b82a725 {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2006}}
Nov 29 03:07:09 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:09 user nova-compute[75234]: DEBUG nova.network.neutron [req-5683e1ef-0602-40cc-9be0-b980c10c7830 req-abab86ae-0c58-4e25-9c87-b50fcf1ff10b service nova] [instance: cb569e8d-e4a5-4076-bb8b-deeefad033e2] Instance cache missing network info. {{(pid=75234) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3322}}
Nov 29 03:07:09 user nova-compute[75234]: DEBUG nova.network.neutron [req-5683e1ef-0602-40cc-9be0-b980c10c7830 req-abab86ae-0c58-4e25-9c87-b50fcf1ff10b service nova] [instance: cb569e8d-e4a5-4076-bb8b-deeefad033e2] Updating instance_info_cache with network_info: [] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:07:09 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-5683e1ef-0602-40cc-9be0-b980c10c7830 req-abab86ae-0c58-4e25-9c87-b50fcf1ff10b service nova] Releasing lock "refresh_cache-cb569e8d-e4a5-4076-bb8b-deeefad033e2" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:07:10 user nova-compute[75234]: INFO nova.virt.block_device [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] [instance: cb569e8d-e4a5-4076-bb8b-deeefad033e2] Booting with volume ab278679-6cfd-4c7c-a930-c078843640d3 at /dev/vdb
Nov 29 03:07:10 user nova-compute[75234]: DEBUG os_brick.utils [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:176}}
Nov 29 03:07:10 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:07:10 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.013s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:07:10 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[db404883-f26e-43e6-ad49-8a8425d020c2]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:dd1368c5ba2c\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:07:10 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt -v / -n -o SOURCE {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:07:10 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt -v / -n -o SOURCE" returned: 0 in 0.019s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:07:10 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[d90343ec-3398-4519-87d6-d8ca79e2ef2e]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:07:10 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:07:10 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.018s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:07:10 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[b394cb1a-24f3-40b8-a24a-3ce5356a58f1]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:07:10 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[e7e03e06-71d6-4fe7-a440-3d4c757e8ead]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:07:10 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] Running cmd (subprocess): nvme version {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:07:10 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] 'nvme version' failed. Not Retrying. {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:473}}
Nov 29 03:07:10 user nova-compute[75234]: DEBUG nova.network.neutron [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] [instance: cb569e8d-e4a5-4076-bb8b-deeefad033e2] Successfully updated port: 026f606a-534c-4cdd-992a-9f33268b374f {{(pid=75234) _update_port /opt/stack/nova/nova/network/neutron.py:586}}
Nov 29 03:07:10 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:10 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.nvmeof [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] nvme not present on system {{(pid=75234) nvme_present /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/nvmeof.py:756}}
Nov 29 03:07:10 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:07:10 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:473}}
Nov 29 03:07:10 user nova-compute[75234]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Nov 29 03:07:10 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[3abfaf8d-1995-4366-88eb-071fb7edae5d]: (4, '') {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:07:10 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.lightos [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=75234) find_dsc /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/lightos.py:98}}
Nov 29 03:07:10 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.lightos [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=75234) get_connector_properties /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/lightos.py:76}}
Nov 29 03:07:10 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.lightos [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] LIGHTOS: no hostnqn found. {{(pid=75234) get_connector_properties /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/lightos.py:84}}
Nov 29 03:07:10 user nova-compute[75234]: DEBUG os_brick.utils [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] <== get_connector_properties: return (157ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:dd1368c5ba2c', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:203}}
Nov 29 03:07:10 user nova-compute[75234]: DEBUG nova.virt.block_device [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] [instance: cb569e8d-e4a5-4076-bb8b-deeefad033e2] Updating existing volume attachment record: 03b0ad93-0fd2-469c-9d43-4f1c9e5ddcf5 {{(pid=75234) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Nov 29 03:07:10 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] Acquiring lock "763a3e14-4563-4dc2-99ae-c340edbb93df" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:07:10 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] Lock "763a3e14-4563-4dc2-99ae-c340edbb93df" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:07:10 user nova-compute[75234]: DEBUG nova.compute.manager [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] [instance: 763a3e14-4563-4dc2-99ae-c340edbb93df] Starting instance... {{(pid=75234) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2422}}
Nov 29 03:07:10 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:07:10 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:07:10 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=75234) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2366}}
Nov 29 03:07:10 user nova-compute[75234]: INFO nova.compute.claims [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] [instance: 763a3e14-4563-4dc2-99ae-c340edbb93df] Claim successful on node user
Nov 29 03:07:10 user nova-compute[75234]: DEBUG nova.compute.provider_tree [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] Inventory has not changed in ProviderTree for provider: 4d291220-7c37-4839-a0e3-b23c66502b74 {{(pid=75234) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Nov 29 03:07:10 user nova-compute[75234]: DEBUG nova.scheduler.client.report [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] Inventory has not changed for provider 4d291220-7c37-4839-a0e3-b23c66502b74 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=75234) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Nov 29 03:07:10 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.467s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:07:10 user nova-compute[75234]: DEBUG nova.compute.manager [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] [instance: 763a3e14-4563-4dc2-99ae-c340edbb93df] Start building networks asynchronously for instance. {{(pid=75234) _build_resources /opt/stack/nova/nova/compute/manager.py:2819}}
Nov 29 03:07:11 user nova-compute[75234]: DEBUG nova.compute.manager [req-e67efddf-b1dc-4d12-9aad-bcf25a235614 req-da49e459-a653-4b6f-bffa-31b13e242035 service nova] [instance: cb569e8d-e4a5-4076-bb8b-deeefad033e2] Received event network-changed-026f606a-534c-4cdd-992a-9f33268b374f {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:07:11 user nova-compute[75234]: DEBUG nova.compute.manager [req-e67efddf-b1dc-4d12-9aad-bcf25a235614 req-da49e459-a653-4b6f-bffa-31b13e242035 service nova] [instance: cb569e8d-e4a5-4076-bb8b-deeefad033e2] Refreshing instance network info cache due to event network-changed-026f606a-534c-4cdd-992a-9f33268b374f. {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11117}}
Nov 29 03:07:11 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-e67efddf-b1dc-4d12-9aad-bcf25a235614 req-da49e459-a653-4b6f-bffa-31b13e242035 service nova] Acquiring lock "refresh_cache-cb569e8d-e4a5-4076-bb8b-deeefad033e2" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:07:11 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-e67efddf-b1dc-4d12-9aad-bcf25a235614 req-da49e459-a653-4b6f-bffa-31b13e242035 service nova] Acquired lock "refresh_cache-cb569e8d-e4a5-4076-bb8b-deeefad033e2" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:07:11 user nova-compute[75234]: DEBUG nova.network.neutron [req-e67efddf-b1dc-4d12-9aad-bcf25a235614 req-da49e459-a653-4b6f-bffa-31b13e242035 service nova] [instance: cb569e8d-e4a5-4076-bb8b-deeefad033e2] Refreshing network info cache for port 026f606a-534c-4cdd-992a-9f33268b374f {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2006}}
Nov 29 03:07:11 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:11 user nova-compute[75234]: DEBUG nova.compute.manager [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] [instance: 763a3e14-4563-4dc2-99ae-c340edbb93df] Allocating IP information in the background. {{(pid=75234) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1972}}
Nov 29 03:07:11 user nova-compute[75234]: DEBUG nova.network.neutron [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] [instance: 763a3e14-4563-4dc2-99ae-c340edbb93df] allocate_for_instance() {{(pid=75234) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1156}}
Nov 29 03:07:11 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] [instance: 763a3e14-4563-4dc2-99ae-c340edbb93df] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Nov 29 03:07:11 user nova-compute[75234]: DEBUG nova.compute.manager [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] [instance: 763a3e14-4563-4dc2-99ae-c340edbb93df] Start building block device mappings for instance. {{(pid=75234) _build_resources /opt/stack/nova/nova/compute/manager.py:2854}}
Nov 29 03:07:11 user nova-compute[75234]: DEBUG nova.network.neutron [req-e67efddf-b1dc-4d12-9aad-bcf25a235614 req-da49e459-a653-4b6f-bffa-31b13e242035 service nova] [instance: cb569e8d-e4a5-4076-bb8b-deeefad033e2] Instance cache missing network info. {{(pid=75234) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3322}}
Nov 29 03:07:11 user nova-compute[75234]: INFO nova.virt.block_device [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] [instance: 763a3e14-4563-4dc2-99ae-c340edbb93df] Booting with volume b1068fd1-6636-43cd-9725-4a37f708b2bc at /dev/vda
Nov 29 03:07:11 user nova-compute[75234]: DEBUG os_brick.utils [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:176}}
Nov 29 03:07:11 user nova-compute[75234]: DEBUG nova.policy [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': 'f2e1fe315fb14410b5c0448525d8b9f7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '325013d2bedc498e91c2145c2b398803', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=75234) authorize /opt/stack/nova/nova/policy.py:203}}
Nov 29 03:07:11 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:07:11 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.018s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:07:11 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[af9673b0-912c-4ce7-b894-10b2abc1cdc9]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:dd1368c5ba2c\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:07:11 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt -v / -n -o SOURCE {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:07:11 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt -v / -n -o SOURCE" returned: 0 in 0.015s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:07:11 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[121902aa-e270-4f12-9f75-6c476d4200c4]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:07:11 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:07:11 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.015s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:07:11 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[4925b9b4-f79f-49cd-bfa8-9bf10b9e0f4b]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:07:11 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[46ceb01f-594c-4eaf-8f9b-94ee5a8c27a3]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:07:11 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] Running cmd (subprocess): nvme version {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:07:11 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] 'nvme version' failed. Not Retrying. {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:473}}
Nov 29 03:07:11 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.nvmeof [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] nvme not present on system {{(pid=75234) nvme_present /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/nvmeof.py:756}}
Nov 29 03:07:11 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:07:11 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:473}}
Nov 29 03:07:11 user nova-compute[75234]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Nov 29 03:07:11 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[38ef950e-937e-4892-9feb-ee913a63a647]: (4, '') {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:07:11 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.lightos [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=75234) find_dsc /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/lightos.py:98}}
Nov 29 03:07:11 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.lightos [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=75234) get_connector_properties /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/lightos.py:76}}
Nov 29 03:07:11 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.lightos [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] LIGHTOS: no hostnqn found. {{(pid=75234) get_connector_properties /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/lightos.py:84}}
Nov 29 03:07:11 user nova-compute[75234]: DEBUG os_brick.utils [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] <== get_connector_properties: return (121ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:dd1368c5ba2c', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:203}}
Nov 29 03:07:11 user nova-compute[75234]: DEBUG nova.virt.block_device [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] [instance: 763a3e14-4563-4dc2-99ae-c340edbb93df] Updating existing volume attachment record: 294a0160-6e5e-4430-8d76-f861ff5dbb23 {{(pid=75234) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Nov 29 03:07:11 user nova-compute[75234]: DEBUG nova.network.neutron [req-e67efddf-b1dc-4d12-9aad-bcf25a235614 req-da49e459-a653-4b6f-bffa-31b13e242035 service nova] [instance: cb569e8d-e4a5-4076-bb8b-deeefad033e2] Updating instance_info_cache with network_info: [] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:07:11 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-e67efddf-b1dc-4d12-9aad-bcf25a235614 req-da49e459-a653-4b6f-bffa-31b13e242035 service nova] Releasing lock "refresh_cache-cb569e8d-e4a5-4076-bb8b-deeefad033e2" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:07:11 user nova-compute[75234]: DEBUG nova.network.neutron [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] [instance: cb569e8d-e4a5-4076-bb8b-deeefad033e2] Successfully updated port: 67446920-3741-4542-84c6-493388395f26 {{(pid=75234) _update_port /opt/stack/nova/nova/network/neutron.py:586}}
Nov 29 03:07:11 user nova-compute[75234]: DEBUG nova.network.neutron [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] [instance: 763a3e14-4563-4dc2-99ae-c340edbb93df] Successfully created port: dc96090f-e855-4d3e-854c-50ea700dc8ea {{(pid=75234) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:548}}
Nov 29 03:07:12 user nova-compute[75234]: DEBUG nova.network.neutron [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] [instance: 763a3e14-4563-4dc2-99ae-c340edbb93df] Successfully created port: d520a88d-ec02-45b8-adbc-484d38b9c18c {{(pid=75234) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:548}}
Nov 29 03:07:12 user nova-compute[75234]: DEBUG nova.network.neutron [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] [instance: cb569e8d-e4a5-4076-bb8b-deeefad033e2] Successfully updated port: d7e92753-492b-4fc0-8048-f4792cd3b4b6 {{(pid=75234) _update_port /opt/stack/nova/nova/network/neutron.py:586}}
Nov 29 03:07:13 user nova-compute[75234]: DEBUG nova.network.neutron [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] [instance: 763a3e14-4563-4dc2-99ae-c340edbb93df] Successfully created port: c50f64cd-b7fa-4580-bb78-bff756a0773a {{(pid=75234) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:548}}
Nov 29 03:07:13 user nova-compute[75234]: DEBUG nova.compute.manager [req-cc37ca25-7492-47a0-8520-cbbfa1dad17d req-9d651768-8ee5-4878-acac-59a58c30ebaa service nova] [instance: cb569e8d-e4a5-4076-bb8b-deeefad033e2] Received event network-changed-67446920-3741-4542-84c6-493388395f26 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:07:13 user nova-compute[75234]: DEBUG nova.compute.manager [req-cc37ca25-7492-47a0-8520-cbbfa1dad17d req-9d651768-8ee5-4878-acac-59a58c30ebaa service nova] [instance: cb569e8d-e4a5-4076-bb8b-deeefad033e2] Refreshing instance network info cache due to event network-changed-67446920-3741-4542-84c6-493388395f26. {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11117}}
Nov 29 03:07:13 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-cc37ca25-7492-47a0-8520-cbbfa1dad17d req-9d651768-8ee5-4878-acac-59a58c30ebaa service nova] Acquiring lock "refresh_cache-cb569e8d-e4a5-4076-bb8b-deeefad033e2" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:07:13 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-cc37ca25-7492-47a0-8520-cbbfa1dad17d req-9d651768-8ee5-4878-acac-59a58c30ebaa service nova] Acquired lock "refresh_cache-cb569e8d-e4a5-4076-bb8b-deeefad033e2" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:07:13 user nova-compute[75234]: DEBUG nova.network.neutron [req-cc37ca25-7492-47a0-8520-cbbfa1dad17d req-9d651768-8ee5-4878-acac-59a58c30ebaa service nova] [instance: cb569e8d-e4a5-4076-bb8b-deeefad033e2] Refreshing network info cache for port 67446920-3741-4542-84c6-493388395f26 {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2006}}
Nov 29 03:07:13 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:13 user nova-compute[75234]: DEBUG nova.network.neutron [req-cc37ca25-7492-47a0-8520-cbbfa1dad17d req-9d651768-8ee5-4878-acac-59a58c30ebaa service nova] [instance: cb569e8d-e4a5-4076-bb8b-deeefad033e2] Instance cache missing network info. {{(pid=75234) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3322}}
Nov 29 03:07:13 user nova-compute[75234]: DEBUG nova.network.neutron [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] [instance: cb569e8d-e4a5-4076-bb8b-deeefad033e2] Successfully updated port: 06db3012-0042-457a-a3c8-14e3a8d2e89e {{(pid=75234) _update_port /opt/stack/nova/nova/network/neutron.py:586}}
Nov 29 03:07:13 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] Acquiring lock "refresh_cache-cb569e8d-e4a5-4076-bb8b-deeefad033e2" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:07:13 user nova-compute[75234]: DEBUG nova.network.neutron [req-cc37ca25-7492-47a0-8520-cbbfa1dad17d req-9d651768-8ee5-4878-acac-59a58c30ebaa service nova] [instance: cb569e8d-e4a5-4076-bb8b-deeefad033e2] Updating instance_info_cache with network_info: [] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:07:13 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-cc37ca25-7492-47a0-8520-cbbfa1dad17d req-9d651768-8ee5-4878-acac-59a58c30ebaa service nova] Releasing lock "refresh_cache-cb569e8d-e4a5-4076-bb8b-deeefad033e2" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:07:13 user nova-compute[75234]: DEBUG nova.compute.manager [req-cc37ca25-7492-47a0-8520-cbbfa1dad17d req-9d651768-8ee5-4878-acac-59a58c30ebaa service nova] [instance: cb569e8d-e4a5-4076-bb8b-deeefad033e2] Received event network-changed-d7e92753-492b-4fc0-8048-f4792cd3b4b6 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:07:13 user nova-compute[75234]: DEBUG nova.compute.manager [req-cc37ca25-7492-47a0-8520-cbbfa1dad17d req-9d651768-8ee5-4878-acac-59a58c30ebaa service nova] [instance: cb569e8d-e4a5-4076-bb8b-deeefad033e2] Refreshing instance network info cache due to event network-changed-d7e92753-492b-4fc0-8048-f4792cd3b4b6. {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11117}}
Nov 29 03:07:13 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-cc37ca25-7492-47a0-8520-cbbfa1dad17d req-9d651768-8ee5-4878-acac-59a58c30ebaa service nova] Acquiring lock "refresh_cache-cb569e8d-e4a5-4076-bb8b-deeefad033e2" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:07:13 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-cc37ca25-7492-47a0-8520-cbbfa1dad17d req-9d651768-8ee5-4878-acac-59a58c30ebaa service nova] Acquired lock "refresh_cache-cb569e8d-e4a5-4076-bb8b-deeefad033e2" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:07:13 user nova-compute[75234]: DEBUG nova.network.neutron [req-cc37ca25-7492-47a0-8520-cbbfa1dad17d req-9d651768-8ee5-4878-acac-59a58c30ebaa service nova] [instance: cb569e8d-e4a5-4076-bb8b-deeefad033e2] Refreshing network info cache for port d7e92753-492b-4fc0-8048-f4792cd3b4b6 {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2006}}
Nov 29 03:07:13 user nova-compute[75234]: DEBUG nova.network.neutron [req-cc37ca25-7492-47a0-8520-cbbfa1dad17d req-9d651768-8ee5-4878-acac-59a58c30ebaa service nova] [instance: cb569e8d-e4a5-4076-bb8b-deeefad033e2] Instance cache missing network info. {{(pid=75234) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3322}}
Nov 29 03:07:13 user nova-compute[75234]: DEBUG nova.network.neutron [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] [instance: 763a3e14-4563-4dc2-99ae-c340edbb93df] Successfully created port: ef8ba969-6186-4b9a-b30f-ec9a70036bab {{(pid=75234) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:548}}
Nov 29 03:07:13 user nova-compute[75234]: DEBUG nova.network.neutron [req-cc37ca25-7492-47a0-8520-cbbfa1dad17d req-9d651768-8ee5-4878-acac-59a58c30ebaa service nova] [instance: cb569e8d-e4a5-4076-bb8b-deeefad033e2] Updating instance_info_cache with network_info: [] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:07:13 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-cc37ca25-7492-47a0-8520-cbbfa1dad17d req-9d651768-8ee5-4878-acac-59a58c30ebaa service nova] Releasing lock "refresh_cache-cb569e8d-e4a5-4076-bb8b-deeefad033e2" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:07:13 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] Acquired lock "refresh_cache-cb569e8d-e4a5-4076-bb8b-deeefad033e2" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:07:13 user nova-compute[75234]: DEBUG nova.network.neutron [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] [instance: cb569e8d-e4a5-4076-bb8b-deeefad033e2] Building network info cache for instance {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2009}}
Nov 29 03:07:14 user nova-compute[75234]: DEBUG nova.network.neutron [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] [instance: cb569e8d-e4a5-4076-bb8b-deeefad033e2] Instance cache missing network info. {{(pid=75234) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3322}}
Nov 29 03:07:14 user nova-compute[75234]: DEBUG nova.network.neutron [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] [instance: 763a3e14-4563-4dc2-99ae-c340edbb93df] Successfully created port: 92917a0d-59ee-4167-8693-262a21f6e156 {{(pid=75234) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:548}}
Nov 29 03:07:15 user nova-compute[75234]: DEBUG nova.compute.manager [req-8d23b404-7b15-4735-8ceb-bd4cc1e96d6f req-848cd5c2-8ca3-4dd2-a47b-61a4b54016a0 service nova] [instance: cb569e8d-e4a5-4076-bb8b-deeefad033e2] Received event network-changed-06db3012-0042-457a-a3c8-14e3a8d2e89e {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:07:15 user nova-compute[75234]: DEBUG nova.compute.manager [req-8d23b404-7b15-4735-8ceb-bd4cc1e96d6f req-848cd5c2-8ca3-4dd2-a47b-61a4b54016a0 service nova] [instance: cb569e8d-e4a5-4076-bb8b-deeefad033e2] Refreshing instance network info cache due to event network-changed-06db3012-0042-457a-a3c8-14e3a8d2e89e. {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11117}}
Nov 29 03:07:15 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-8d23b404-7b15-4735-8ceb-bd4cc1e96d6f req-848cd5c2-8ca3-4dd2-a47b-61a4b54016a0 service nova] Acquiring lock "refresh_cache-cb569e8d-e4a5-4076-bb8b-deeefad033e2" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:07:15 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:15 user nova-compute[75234]: DEBUG nova.network.neutron [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] [instance: 763a3e14-4563-4dc2-99ae-c340edbb93df] Successfully updated port: dc96090f-e855-4d3e-854c-50ea700dc8ea {{(pid=75234) _update_port /opt/stack/nova/nova/network/neutron.py:586}}
Nov 29 03:07:15 user nova-compute[75234]: DEBUG nova.compute.manager [req-b00695ef-8f03-4abe-b9d1-11d926d3642c req-ac3191a4-8819-4c43-814e-dedfb62df6b7 service nova] [instance: 763a3e14-4563-4dc2-99ae-c340edbb93df] Received event network-changed-dc96090f-e855-4d3e-854c-50ea700dc8ea {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:07:15 user nova-compute[75234]: DEBUG nova.compute.manager [req-b00695ef-8f03-4abe-b9d1-11d926d3642c req-ac3191a4-8819-4c43-814e-dedfb62df6b7 service nova] [instance: 763a3e14-4563-4dc2-99ae-c340edbb93df] Refreshing instance network info cache due to event network-changed-dc96090f-e855-4d3e-854c-50ea700dc8ea. {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11117}}
Nov 29 03:07:15 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-b00695ef-8f03-4abe-b9d1-11d926d3642c req-ac3191a4-8819-4c43-814e-dedfb62df6b7 service nova] Acquiring lock "refresh_cache-763a3e14-4563-4dc2-99ae-c340edbb93df" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:07:15 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-b00695ef-8f03-4abe-b9d1-11d926d3642c req-ac3191a4-8819-4c43-814e-dedfb62df6b7 service nova] Acquired lock "refresh_cache-763a3e14-4563-4dc2-99ae-c340edbb93df" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:07:15 user nova-compute[75234]: DEBUG nova.network.neutron [req-b00695ef-8f03-4abe-b9d1-11d926d3642c req-ac3191a4-8819-4c43-814e-dedfb62df6b7 service nova] [instance: 763a3e14-4563-4dc2-99ae-c340edbb93df] Refreshing network info cache for port dc96090f-e855-4d3e-854c-50ea700dc8ea {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2006}}
Nov 29 03:07:16 user nova-compute[75234]: DEBUG nova.network.neutron [req-b00695ef-8f03-4abe-b9d1-11d926d3642c req-ac3191a4-8819-4c43-814e-dedfb62df6b7 service nova] [instance: 763a3e14-4563-4dc2-99ae-c340edbb93df] Instance cache missing network info. {{(pid=75234) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3322}}
Nov 29 03:07:16 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:16 user nova-compute[75234]: DEBUG nova.network.neutron [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] [instance: cb569e8d-e4a5-4076-bb8b-deeefad033e2] Updating instance_info_cache with network_info: [{"id": "2b12ec96-26fa-4974-8a7a-22e85ec20544", "address": "fa:16:3e:fd:00:a6", "network": {"id": "1869abec-0b45-446a-9dc8-5871de9dbfc5", "bridge": "br-int", "label": "tempest-TaggedBootDevicesTest-1812676341-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "ee1117bd3511400f9f45cc8f9bb9247d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2b12ec96-26", "ovs_interfaceid": "2b12ec96-26fa-4974-8a7a-22e85ec20544", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "0bd5d735-f566-4a6d-b49f-b24fd403a87a", "address": "fa:16:3e:aa:c2:ae", "network": {"id": "005139f3-3686-493a-aa8f-0cdc455befa0", "bridge": "br-int", "label": "tempest-device-tagging-net1-1510949192", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.85", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "ee1117bd3511400f9f45cc8f9bb9247d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap0bd5d735-f5", "ovs_interfaceid": "0bd5d735-f566-4a6d-b49f-b24fd403a87a", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}}, {"id": "27e72461-f85b-42c9-8824-0d760b82a725", "address": "fa:16:3e:8e:29:fd", "network": {"id": "005139f3-3686-493a-aa8f-0cdc455befa0", "bridge": "br-int", "label": "tempest-device-tagging-net1-1510949192", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.127", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "ee1117bd3511400f9f45cc8f9bb9247d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap27e72461-f8", "ovs_interfaceid": "27e72461-f85b-42c9-8824-0d760b82a725", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}}, {"id": "026f606a-534c-4cdd-992a-9f33268b374f", "address": "fa:16:3e:3e:3b:29", "network": {"id": "005139f3-3686-493a-aa8f-0cdc455befa0", "bridge": "br-int", "label": "tempest-device-tagging-net1-1510949192", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.41", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "ee1117bd3511400f9f45cc8f9bb9247d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap026f606a-53", "ovs_interfaceid": "026f606a-534c-4cdd-992a-9f33268b374f", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "67446920-3741-4542-84c6-493388395f26", "address": "fa:16:3e:f6:7a:76", "network": {"id": "005139f3-3686-493a-aa8f-0cdc455befa0", "bridge": "br-int", "label": "tempest-device-tagging-net1-1510949192", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.230", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "ee1117bd3511400f9f45cc8f9bb9247d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap67446920-37", "ovs_interfaceid": "67446920-3741-4542-84c6-493388395f26", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "d7e92753-492b-4fc0-8048-f4792cd3b4b6", "address": "fa:16:3e:68:b2:33", "network": {"id": "cc6c7cdf-8c81-4906-9fd8-de815fea3374", "bridge": "br-int", "label": "tempest-device-tagging-net2-1248280764", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.100", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "ee1117bd3511400f9f45cc8f9bb9247d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapd7e92753-49", "ovs_interfaceid": "d7e92753-492b-4fc0-8048-f4792cd3b4b6", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "06db3012-0042-457a-a3c8-14e3a8d2e89e", "address": "fa:16:3e:27:54:30", "network": {"id": "cc6c7cdf-8c81-4906-9fd8-de815fea3374", "bridge": "br-int", "label": "tempest-device-tagging-net2-1248280764", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.200", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "ee1117bd3511400f9f45cc8f9bb9247d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap06db3012-00", "ovs_interfaceid": "06db3012-0042-457a-a3c8-14e3a8d2e89e", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:07:16 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] Releasing lock "refresh_cache-cb569e8d-e4a5-4076-bb8b-deeefad033e2" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:07:16 user nova-compute[75234]: DEBUG nova.compute.manager [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] [instance: cb569e8d-e4a5-4076-bb8b-deeefad033e2] Instance network_info: |[{"id": "2b12ec96-26fa-4974-8a7a-22e85ec20544", "address": "fa:16:3e:fd:00:a6", "network": {"id": "1869abec-0b45-446a-9dc8-5871de9dbfc5", "bridge": "br-int", "label": "tempest-TaggedBootDevicesTest-1812676341-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "ee1117bd3511400f9f45cc8f9bb9247d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2b12ec96-26", "ovs_interfaceid": "2b12ec96-26fa-4974-8a7a-22e85ec20544", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "0bd5d735-f566-4a6d-b49f-b24fd403a87a", "address": "fa:16:3e:aa:c2:ae", "network": {"id": "005139f3-3686-493a-aa8f-0cdc455befa0", "bridge": "br-int", "label": "tempest-device-tagging-net1-1510949192", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.85", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "ee1117bd3511400f9f45cc8f9bb9247d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap0bd5d735-f5", "ovs_interfaceid": "0bd5d735-f566-4a6d-b49f-b24fd403a87a", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}}, {"id": "27e72461-f85b-42c9-8824-0d760b82a725", "address": "fa:16:3e:8e:29:fd", "network": {"id": "005139f3-3686-493a-aa8f-0cdc455befa0", "bridge": "br-int", "label": "tempest-device-tagging-net1-1510949192", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.127", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "ee1117bd3511400f9f45cc8f9bb9247d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap27e72461-f8", "ovs_interfaceid": "27e72461-f85b-42c9-8824-0d760b82a725", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}}, {"id": "026f606a-534c-4cdd-992a-9f33268b374f", "address": "fa:16:3e:3e:3b:29", "network": {"id": "005139f3-3686-493a-aa8f-0cdc455befa0", "bridge": "br-int", "label": "tempest-device-tagging-net1-1510949192", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.41", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "ee1117bd3511400f9f45cc8f9bb9247d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap026f606a-53", "ovs_interfaceid": "026f606a-534c-4cdd-992a-9f33268b374f", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "67446920-3741-4542-84c6-493388395f26", "address": "fa:16:3e:f6:7a:76", "network": {"id": "005139f3-3686-493a-aa8f-0cdc455befa0", "bridge": "br-int", "label": "tempest-device-tagging-net1-1510949192", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.230", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "ee1117bd3511400f9f45cc8f9bb9247d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap67446920-37", "ovs_interfaceid": "67446920-3741-4542-84c6-493388395f26", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "d7e92753-492b-4fc0-8048-f4792cd3b4b6", "address": "fa:16:3e:68:b2:33", "network": {"id": "cc6c7cdf-8c81-4906-9fd8-de815fea3374", "bridge": "br-int", "label": "tempest-device-tagging-net2-1248280764", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.100", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "ee1117bd3511400f9f45cc8f9bb9247d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapd7e92753-49", "ovs_interfaceid": "d7e92753-492b-4fc0-8048-f4792cd3b4b6", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "06db3012-0042-457a-a3c8-14e3a8d2e89e", "address": "fa:16:3e:27:54:30", "network": {"id": "cc6c7cdf-8c81-4906-9fd8-de815fea3374", "bridge": "br-int", "label": "tempest-device-tagging-net2-1248280764", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.200", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "ee1117bd3511400f9f45cc8f9bb9247d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap06db3012-00", "ovs_interfaceid": "06db3012-0042-457a-a3c8-14e3a8d2e89e", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=75234) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1987}}
Nov 29 03:07:16 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-8d23b404-7b15-4735-8ceb-bd4cc1e96d6f req-848cd5c2-8ca3-4dd2-a47b-61a4b54016a0 service nova] Acquired lock "refresh_cache-cb569e8d-e4a5-4076-bb8b-deeefad033e2" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:07:16 user nova-compute[75234]: DEBUG nova.network.neutron [req-8d23b404-7b15-4735-8ceb-bd4cc1e96d6f req-848cd5c2-8ca3-4dd2-a47b-61a4b54016a0 service nova] [instance: cb569e8d-e4a5-4076-bb8b-deeefad033e2] Refreshing network info cache for port 06db3012-0042-457a-a3c8-14e3a8d2e89e {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2006}}
Nov 29 03:07:16 user nova-compute[75234]: DEBUG nova.network.neutron [req-b00695ef-8f03-4abe-b9d1-11d926d3642c req-ac3191a4-8819-4c43-814e-dedfb62df6b7 service nova] [instance: 763a3e14-4563-4dc2-99ae-c340edbb93df] Updating instance_info_cache with network_info: [] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:07:16 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-b00695ef-8f03-4abe-b9d1-11d926d3642c req-ac3191a4-8819-4c43-814e-dedfb62df6b7 service nova] Releasing lock "refresh_cache-763a3e14-4563-4dc2-99ae-c340edbb93df" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:07:16 user nova-compute[75234]: DEBUG nova.network.neutron [req-8d23b404-7b15-4735-8ceb-bd4cc1e96d6f req-848cd5c2-8ca3-4dd2-a47b-61a4b54016a0 service nova] [instance: cb569e8d-e4a5-4076-bb8b-deeefad033e2] Updated VIF entry in instance network info cache for port 06db3012-0042-457a-a3c8-14e3a8d2e89e. {{(pid=75234) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3481}}
Nov 29 03:07:16 user nova-compute[75234]: DEBUG nova.network.neutron [req-8d23b404-7b15-4735-8ceb-bd4cc1e96d6f req-848cd5c2-8ca3-4dd2-a47b-61a4b54016a0 service nova] [instance: cb569e8d-e4a5-4076-bb8b-deeefad033e2] Updating instance_info_cache with network_info: [{"id": "2b12ec96-26fa-4974-8a7a-22e85ec20544", "address": "fa:16:3e:fd:00:a6", "network": {"id": "1869abec-0b45-446a-9dc8-5871de9dbfc5", "bridge": "br-int", "label": "tempest-TaggedBootDevicesTest-1812676341-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "ee1117bd3511400f9f45cc8f9bb9247d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2b12ec96-26", "ovs_interfaceid": "2b12ec96-26fa-4974-8a7a-22e85ec20544", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "0bd5d735-f566-4a6d-b49f-b24fd403a87a", "address": "fa:16:3e:aa:c2:ae", "network": {"id": "005139f3-3686-493a-aa8f-0cdc455befa0", "bridge": "br-int", "label": "tempest-device-tagging-net1-1510949192", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.85", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "ee1117bd3511400f9f45cc8f9bb9247d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap0bd5d735-f5", "ovs_interfaceid": "0bd5d735-f566-4a6d-b49f-b24fd403a87a", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}}, {"id": "27e72461-f85b-42c9-8824-0d760b82a725", "address": "fa:16:3e:8e:29:fd", "network": {"id": "005139f3-3686-493a-aa8f-0cdc455befa0", "bridge": "br-int", "label": "tempest-device-tagging-net1-1510949192", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.127", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "ee1117bd3511400f9f45cc8f9bb9247d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap27e72461-f8", "ovs_interfaceid": "27e72461-f85b-42c9-8824-0d760b82a725", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}}, {"id": "026f606a-534c-4cdd-992a-9f33268b374f", "address": "fa:16:3e:3e:3b:29", "network": {"id": "005139f3-3686-493a-aa8f-0cdc455befa0", "bridge": "br-int", "label": "tempest-device-tagging-net1-1510949192", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.41", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "ee1117bd3511400f9f45cc8f9bb9247d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap026f606a-53", "ovs_interfaceid": "026f606a-534c-4cdd-992a-9f33268b374f", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "67446920-3741-4542-84c6-493388395f26", "address": "fa:16:3e:f6:7a:76", "network": {"id": "005139f3-3686-493a-aa8f-0cdc455befa0", "bridge": "br-int", "label": "tempest-device-tagging-net1-1510949192", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.230", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "ee1117bd3511400f9f45cc8f9bb9247d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap67446920-37", "ovs_interfaceid": "67446920-3741-4542-84c6-493388395f26", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "d7e92753-492b-4fc0-8048-f4792cd3b4b6", "address": "fa:16:3e:68:b2:33", "network": {"id": "cc6c7cdf-8c81-4906-9fd8-de815fea3374", "bridge": "br-int", "label": "tempest-device-tagging-net2-1248280764", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.100", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "ee1117bd3511400f9f45cc8f9bb9247d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapd7e92753-49", "ovs_interfaceid": "d7e92753-492b-4fc0-8048-f4792cd3b4b6", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "06db3012-0042-457a-a3c8-14e3a8d2e89e", "address": "fa:16:3e:27:54:30", "network": {"id": "cc6c7cdf-8c81-4906-9fd8-de815fea3374", "bridge": "br-int", "label": "tempest-device-tagging-net2-1248280764", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.200", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "ee1117bd3511400f9f45cc8f9bb9247d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap06db3012-00", "ovs_interfaceid": "06db3012-0042-457a-a3c8-14e3a8d2e89e", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:07:16 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-8d23b404-7b15-4735-8ceb-bd4cc1e96d6f req-848cd5c2-8ca3-4dd2-a47b-61a4b54016a0 service nova] Releasing lock "refresh_cache-cb569e8d-e4a5-4076-bb8b-deeefad033e2" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:07:16 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:17 user nova-compute[75234]: DEBUG nova.network.neutron [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] [instance: 763a3e14-4563-4dc2-99ae-c340edbb93df] Successfully updated port: 2027f7a1-36e4-4434-a063-61e817758917 {{(pid=75234) _update_port /opt/stack/nova/nova/network/neutron.py:586}}
Nov 29 03:07:17 user nova-compute[75234]: DEBUG nova.network.neutron [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] [instance: 763a3e14-4563-4dc2-99ae-c340edbb93df] Successfully updated port: de4ad363-fae0-4d1b-9820-5a91a27d630d {{(pid=75234) _update_port /opt/stack/nova/nova/network/neutron.py:586}}
Nov 29 03:07:18 user nova-compute[75234]: DEBUG nova.compute.manager [req-1febeb4e-ce89-4d39-a259-eeb0490d02cb req-6b18f2c1-7772-489c-8e6e-10dcc9998c6d service nova] [instance: 763a3e14-4563-4dc2-99ae-c340edbb93df] Received event network-changed-2027f7a1-36e4-4434-a063-61e817758917 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:07:18 user nova-compute[75234]: DEBUG nova.compute.manager [req-1febeb4e-ce89-4d39-a259-eeb0490d02cb req-6b18f2c1-7772-489c-8e6e-10dcc9998c6d service nova] [instance: 763a3e14-4563-4dc2-99ae-c340edbb93df] Refreshing instance network info cache due to event network-changed-2027f7a1-36e4-4434-a063-61e817758917. {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11117}}
Nov 29 03:07:18 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-1febeb4e-ce89-4d39-a259-eeb0490d02cb req-6b18f2c1-7772-489c-8e6e-10dcc9998c6d service nova] Acquiring lock "refresh_cache-763a3e14-4563-4dc2-99ae-c340edbb93df" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:07:18 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-1febeb4e-ce89-4d39-a259-eeb0490d02cb req-6b18f2c1-7772-489c-8e6e-10dcc9998c6d service nova] Acquired lock "refresh_cache-763a3e14-4563-4dc2-99ae-c340edbb93df" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:07:18 user nova-compute[75234]: DEBUG nova.network.neutron [req-1febeb4e-ce89-4d39-a259-eeb0490d02cb req-6b18f2c1-7772-489c-8e6e-10dcc9998c6d service nova] [instance: 763a3e14-4563-4dc2-99ae-c340edbb93df] Refreshing network info cache for port 2027f7a1-36e4-4434-a063-61e817758917 {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2006}}
Nov 29 03:07:18 user nova-compute[75234]: DEBUG nova.network.neutron [req-1febeb4e-ce89-4d39-a259-eeb0490d02cb req-6b18f2c1-7772-489c-8e6e-10dcc9998c6d service nova] [instance: 763a3e14-4563-4dc2-99ae-c340edbb93df] Instance cache missing network info. {{(pid=75234) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3322}}
Nov 29 03:07:18 user nova-compute[75234]: DEBUG nova.network.neutron [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] [instance: 763a3e14-4563-4dc2-99ae-c340edbb93df] Successfully updated port: d520a88d-ec02-45b8-adbc-484d38b9c18c {{(pid=75234) _update_port /opt/stack/nova/nova/network/neutron.py:586}}
Nov 29 03:07:18 user nova-compute[75234]: DEBUG nova.network.neutron [req-1febeb4e-ce89-4d39-a259-eeb0490d02cb req-6b18f2c1-7772-489c-8e6e-10dcc9998c6d service nova] [instance: 763a3e14-4563-4dc2-99ae-c340edbb93df] Updating instance_info_cache with network_info: [] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:07:18 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-1febeb4e-ce89-4d39-a259-eeb0490d02cb req-6b18f2c1-7772-489c-8e6e-10dcc9998c6d service nova] Releasing lock "refresh_cache-763a3e14-4563-4dc2-99ae-c340edbb93df" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:07:18 user nova-compute[75234]: DEBUG nova.compute.manager [req-1febeb4e-ce89-4d39-a259-eeb0490d02cb req-6b18f2c1-7772-489c-8e6e-10dcc9998c6d service nova] [instance: 763a3e14-4563-4dc2-99ae-c340edbb93df] Received event network-changed-de4ad363-fae0-4d1b-9820-5a91a27d630d {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:07:18 user nova-compute[75234]: DEBUG nova.compute.manager [req-1febeb4e-ce89-4d39-a259-eeb0490d02cb req-6b18f2c1-7772-489c-8e6e-10dcc9998c6d service nova] [instance: 763a3e14-4563-4dc2-99ae-c340edbb93df] Refreshing instance network info cache due to event network-changed-de4ad363-fae0-4d1b-9820-5a91a27d630d. {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11117}}
Nov 29 03:07:18 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-1febeb4e-ce89-4d39-a259-eeb0490d02cb req-6b18f2c1-7772-489c-8e6e-10dcc9998c6d service nova] Acquiring lock "refresh_cache-763a3e14-4563-4dc2-99ae-c340edbb93df" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:07:18 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-1febeb4e-ce89-4d39-a259-eeb0490d02cb req-6b18f2c1-7772-489c-8e6e-10dcc9998c6d service nova] Acquired lock "refresh_cache-763a3e14-4563-4dc2-99ae-c340edbb93df" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:07:18 user nova-compute[75234]: DEBUG nova.network.neutron [req-1febeb4e-ce89-4d39-a259-eeb0490d02cb req-6b18f2c1-7772-489c-8e6e-10dcc9998c6d service nova] [instance: 763a3e14-4563-4dc2-99ae-c340edbb93df] Refreshing network info cache for port de4ad363-fae0-4d1b-9820-5a91a27d630d {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2006}}
Nov 29 03:07:18 user nova-compute[75234]: DEBUG nova.network.neutron [req-1febeb4e-ce89-4d39-a259-eeb0490d02cb req-6b18f2c1-7772-489c-8e6e-10dcc9998c6d service nova] [instance: 763a3e14-4563-4dc2-99ae-c340edbb93df] Instance cache missing network info. {{(pid=75234) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3322}}
Nov 29 03:07:18 user nova-compute[75234]: DEBUG nova.network.neutron [req-1febeb4e-ce89-4d39-a259-eeb0490d02cb req-6b18f2c1-7772-489c-8e6e-10dcc9998c6d service nova] [instance: 763a3e14-4563-4dc2-99ae-c340edbb93df] Updating instance_info_cache with network_info: [] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:07:18 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-1febeb4e-ce89-4d39-a259-eeb0490d02cb req-6b18f2c1-7772-489c-8e6e-10dcc9998c6d service nova] Releasing lock "refresh_cache-763a3e14-4563-4dc2-99ae-c340edbb93df" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:07:18 user nova-compute[75234]: INFO nova.virt.block_device [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] [instance: 763a3e14-4563-4dc2-99ae-c340edbb93df] Booting with volume 0d2c0c22-2b3a-43db-b9af-e60a2514e48e at /dev/vdb
Nov 29 03:07:18 user nova-compute[75234]: DEBUG nova.network.neutron [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] [instance: 763a3e14-4563-4dc2-99ae-c340edbb93df] Successfully updated port: c50f64cd-b7fa-4580-bb78-bff756a0773a {{(pid=75234) _update_port /opt/stack/nova/nova/network/neutron.py:586}}
Nov 29 03:07:19 user nova-compute[75234]: DEBUG os_brick.utils [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:176}}
Nov 29 03:07:19 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:07:19 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.016s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:07:19 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[328de2a1-44c8-4400-9962-de7baaabdfd5]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:dd1368c5ba2c\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:07:19 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt -v / -n -o SOURCE {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:07:19 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt -v / -n -o SOURCE" returned: 0 in 0.016s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:07:19 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[d38bac41-4f5d-404d-8ae6-ad33fcf9b6fc]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:07:19 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:07:19 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.016s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:07:19 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[3ce664a5-288c-4fca-be3b-c17ac8f5d584]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:07:19 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[04becfb9-cd06-41fe-8505-142a6c80cf7a]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:07:19 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] Running cmd (subprocess): nvme version {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:07:19 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] 'nvme version' failed. Not Retrying. {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:473}}
Nov 29 03:07:19 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.nvmeof [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] nvme not present on system {{(pid=75234) nvme_present /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/nvmeof.py:756}}
Nov 29 03:07:19 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:07:19 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:473}}
Nov 29 03:07:19 user nova-compute[75234]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Nov 29 03:07:19 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[f83bcf6c-c8f2-49a8-8ae8-12905755add8]: (4, '') {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:07:19 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.lightos [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=75234) find_dsc /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/lightos.py:98}}
Nov 29 03:07:19 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.lightos [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=75234) get_connector_properties /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/lightos.py:76}}
Nov 29 03:07:19 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.lightos [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] LIGHTOS: no hostnqn found. {{(pid=75234) get_connector_properties /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/lightos.py:84}}
Nov 29 03:07:19 user nova-compute[75234]: DEBUG os_brick.utils [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] <== get_connector_properties: return (134ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:dd1368c5ba2c', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:203}}
Nov 29 03:07:19 user nova-compute[75234]: DEBUG nova.virt.block_device [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] [instance: 763a3e14-4563-4dc2-99ae-c340edbb93df] Updating existing volume attachment record: 729ab20a-0394-426d-9bc7-b8f7bdc02d1d {{(pid=75234) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Nov 29 03:07:19 user nova-compute[75234]: INFO nova.virt.block_device [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] [instance: cb569e8d-e4a5-4076-bb8b-deeefad033e2] Booting with volume 920d6cad-25ab-4f3d-941b-8ec841d43184 at /dev/vdc
Nov 29 03:07:19 user nova-compute[75234]: DEBUG os_brick.utils [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:176}}
Nov 29 03:07:19 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:07:19 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.013s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:07:19 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[3abc3958-fde1-4f1f-8fc1-b3083c0778cb]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:dd1368c5ba2c\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:07:19 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt -v / -n -o SOURCE {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:07:19 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt -v / -n -o SOURCE" returned: 0 in 0.014s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:07:19 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[0840f4a3-0fe2-4109-a281-fc993784cd0f]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:07:19 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:07:19 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.014s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:07:19 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[e0028103-5ee6-442b-87f6-9d4f8f8e81ba]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:07:19 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[e1f78cc2-192d-4002-9130-fc421f7a9629]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:07:19 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] Running cmd (subprocess): nvme version {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:07:19 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] 'nvme version' failed. Not Retrying. {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:473}}
Nov 29 03:07:19 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.nvmeof [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] nvme not present on system {{(pid=75234) nvme_present /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/nvmeof.py:756}}
Nov 29 03:07:19 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:07:19 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:473}}
Nov 29 03:07:19 user nova-compute[75234]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Nov 29 03:07:19 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[df1f6ac6-dae7-40f3-84ba-82e6aea59f58]: (4, '') {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:07:19 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.lightos [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=75234) find_dsc /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/lightos.py:98}}
Nov 29 03:07:19 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.lightos [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=75234) get_connector_properties /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/lightos.py:76}}
Nov 29 03:07:19 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.lightos [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] LIGHTOS: no hostnqn found. {{(pid=75234) get_connector_properties /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/lightos.py:84}}
Nov 29 03:07:19 user nova-compute[75234]: DEBUG os_brick.utils [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] <== get_connector_properties: return (108ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:dd1368c5ba2c', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:203}}
Nov 29 03:07:19 user nova-compute[75234]: DEBUG nova.virt.block_device [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] [instance: cb569e8d-e4a5-4076-bb8b-deeefad033e2] Updating existing volume attachment record: 853e8d4a-de7a-4184-a592-0855eb8020d7 {{(pid=75234) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Nov 29 03:07:19 user nova-compute[75234]: DEBUG nova.network.neutron [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] [instance: 763a3e14-4563-4dc2-99ae-c340edbb93df] Successfully updated port: ef8ba969-6186-4b9a-b30f-ec9a70036bab {{(pid=75234) _update_port /opt/stack/nova/nova/network/neutron.py:586}}
Nov 29 03:07:20 user nova-compute[75234]: DEBUG nova.compute.manager [req-a4c25f06-f403-4d6a-8693-3d4311ea74a3 req-cf16079c-ad19-432a-8160-8f104df582f1 service nova] [instance: 763a3e14-4563-4dc2-99ae-c340edbb93df] Received event network-changed-d520a88d-ec02-45b8-adbc-484d38b9c18c {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:07:20 user nova-compute[75234]: DEBUG nova.compute.manager [req-a4c25f06-f403-4d6a-8693-3d4311ea74a3 req-cf16079c-ad19-432a-8160-8f104df582f1 service nova] [instance: 763a3e14-4563-4dc2-99ae-c340edbb93df] Refreshing instance network info cache due to event network-changed-d520a88d-ec02-45b8-adbc-484d38b9c18c. {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11117}}
Nov 29 03:07:20 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-a4c25f06-f403-4d6a-8693-3d4311ea74a3 req-cf16079c-ad19-432a-8160-8f104df582f1 service nova] Acquiring lock "refresh_cache-763a3e14-4563-4dc2-99ae-c340edbb93df" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:07:20 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-a4c25f06-f403-4d6a-8693-3d4311ea74a3 req-cf16079c-ad19-432a-8160-8f104df582f1 service nova] Acquired lock "refresh_cache-763a3e14-4563-4dc2-99ae-c340edbb93df" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:07:20 user nova-compute[75234]: DEBUG nova.network.neutron [req-a4c25f06-f403-4d6a-8693-3d4311ea74a3 req-cf16079c-ad19-432a-8160-8f104df582f1 service nova] [instance: 763a3e14-4563-4dc2-99ae-c340edbb93df] Refreshing network info cache for port d520a88d-ec02-45b8-adbc-484d38b9c18c {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2006}}
Nov 29 03:07:20 user nova-compute[75234]: DEBUG nova.network.neutron [req-a4c25f06-f403-4d6a-8693-3d4311ea74a3 req-cf16079c-ad19-432a-8160-8f104df582f1 service nova] [instance: 763a3e14-4563-4dc2-99ae-c340edbb93df] Instance cache missing network info. {{(pid=75234) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3322}}
Nov 29 03:07:20 user nova-compute[75234]: DEBUG nova.network.neutron [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] [instance: 763a3e14-4563-4dc2-99ae-c340edbb93df] Successfully updated port: 92917a0d-59ee-4167-8693-262a21f6e156 {{(pid=75234) _update_port /opt/stack/nova/nova/network/neutron.py:586}}
Nov 29 03:07:20 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:20 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] Acquiring lock "refresh_cache-763a3e14-4563-4dc2-99ae-c340edbb93df" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:07:20 user nova-compute[75234]: DEBUG nova.network.neutron [req-a4c25f06-f403-4d6a-8693-3d4311ea74a3 req-cf16079c-ad19-432a-8160-8f104df582f1 service nova] [instance: 763a3e14-4563-4dc2-99ae-c340edbb93df] Updating instance_info_cache with network_info: [] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:07:20 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-a4c25f06-f403-4d6a-8693-3d4311ea74a3 req-cf16079c-ad19-432a-8160-8f104df582f1 service nova] Releasing lock "refresh_cache-763a3e14-4563-4dc2-99ae-c340edbb93df" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:07:20 user nova-compute[75234]: DEBUG nova.compute.manager [req-a4c25f06-f403-4d6a-8693-3d4311ea74a3 req-cf16079c-ad19-432a-8160-8f104df582f1 service nova] [instance: 763a3e14-4563-4dc2-99ae-c340edbb93df] Received event network-changed-c50f64cd-b7fa-4580-bb78-bff756a0773a {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:07:20 user nova-compute[75234]: DEBUG nova.compute.manager [req-a4c25f06-f403-4d6a-8693-3d4311ea74a3 req-cf16079c-ad19-432a-8160-8f104df582f1 service nova] [instance: 763a3e14-4563-4dc2-99ae-c340edbb93df] Refreshing instance network info cache due to event network-changed-c50f64cd-b7fa-4580-bb78-bff756a0773a. {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11117}}
Nov 29 03:07:20 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-a4c25f06-f403-4d6a-8693-3d4311ea74a3 req-cf16079c-ad19-432a-8160-8f104df582f1 service nova] Acquiring lock "refresh_cache-763a3e14-4563-4dc2-99ae-c340edbb93df" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:07:20 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-a4c25f06-f403-4d6a-8693-3d4311ea74a3 req-cf16079c-ad19-432a-8160-8f104df582f1 service nova] Acquired lock "refresh_cache-763a3e14-4563-4dc2-99ae-c340edbb93df" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:07:20 user nova-compute[75234]: DEBUG nova.network.neutron [req-a4c25f06-f403-4d6a-8693-3d4311ea74a3 req-cf16079c-ad19-432a-8160-8f104df582f1 service nova] [instance: 763a3e14-4563-4dc2-99ae-c340edbb93df] Refreshing network info cache for port c50f64cd-b7fa-4580-bb78-bff756a0773a {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2006}}
Nov 29 03:07:20 user nova-compute[75234]: DEBUG nova.network.neutron [req-a4c25f06-f403-4d6a-8693-3d4311ea74a3 req-cf16079c-ad19-432a-8160-8f104df582f1 service nova] [instance: 763a3e14-4563-4dc2-99ae-c340edbb93df] Instance cache missing network info. {{(pid=75234) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3322}}
Nov 29 03:07:20 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-78ea8aa1-48fd-4c8d-b0d1-dcdfe7cfafaf tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] Acquiring lock "f7d0bdeb-6a05-47ac-87f2-c8380a9a4286" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:07:20 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-78ea8aa1-48fd-4c8d-b0d1-dcdfe7cfafaf tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] Lock "f7d0bdeb-6a05-47ac-87f2-c8380a9a4286" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:07:20 user nova-compute[75234]: DEBUG nova.compute.manager [None req-78ea8aa1-48fd-4c8d-b0d1-dcdfe7cfafaf tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] [instance: f7d0bdeb-6a05-47ac-87f2-c8380a9a4286] Starting instance... {{(pid=75234) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2422}}
Nov 29 03:07:20 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-78ea8aa1-48fd-4c8d-b0d1-dcdfe7cfafaf tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:07:20 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-78ea8aa1-48fd-4c8d-b0d1-dcdfe7cfafaf tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:07:20 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-78ea8aa1-48fd-4c8d-b0d1-dcdfe7cfafaf tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=75234) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2366}}
Nov 29 03:07:20 user nova-compute[75234]: INFO nova.compute.claims [None req-78ea8aa1-48fd-4c8d-b0d1-dcdfe7cfafaf tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] [instance: f7d0bdeb-6a05-47ac-87f2-c8380a9a4286] Claim successful on node user
Nov 29 03:07:20 user nova-compute[75234]: DEBUG nova.network.neutron [req-a4c25f06-f403-4d6a-8693-3d4311ea74a3 req-cf16079c-ad19-432a-8160-8f104df582f1 service nova] [instance: 763a3e14-4563-4dc2-99ae-c340edbb93df] Updating instance_info_cache with network_info: [] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:07:20 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-a4c25f06-f403-4d6a-8693-3d4311ea74a3 req-cf16079c-ad19-432a-8160-8f104df582f1 service nova] Releasing lock "refresh_cache-763a3e14-4563-4dc2-99ae-c340edbb93df" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:07:20 user nova-compute[75234]: DEBUG nova.compute.manager [req-a4c25f06-f403-4d6a-8693-3d4311ea74a3 req-cf16079c-ad19-432a-8160-8f104df582f1 service nova] [instance: 763a3e14-4563-4dc2-99ae-c340edbb93df] Received event network-changed-ef8ba969-6186-4b9a-b30f-ec9a70036bab {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:07:20 user nova-compute[75234]: DEBUG nova.compute.manager [req-a4c25f06-f403-4d6a-8693-3d4311ea74a3 req-cf16079c-ad19-432a-8160-8f104df582f1 service nova] [instance: 763a3e14-4563-4dc2-99ae-c340edbb93df] Refreshing instance network info cache due to event network-changed-ef8ba969-6186-4b9a-b30f-ec9a70036bab. {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11117}}
Nov 29 03:07:20 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-a4c25f06-f403-4d6a-8693-3d4311ea74a3 req-cf16079c-ad19-432a-8160-8f104df582f1 service nova] Acquiring lock "refresh_cache-763a3e14-4563-4dc2-99ae-c340edbb93df" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:07:20 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-a4c25f06-f403-4d6a-8693-3d4311ea74a3 req-cf16079c-ad19-432a-8160-8f104df582f1 service nova] Acquired lock "refresh_cache-763a3e14-4563-4dc2-99ae-c340edbb93df" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:07:20 user nova-compute[75234]: DEBUG nova.network.neutron [req-a4c25f06-f403-4d6a-8693-3d4311ea74a3 req-cf16079c-ad19-432a-8160-8f104df582f1 service nova] [instance: 763a3e14-4563-4dc2-99ae-c340edbb93df] Refreshing network info cache for port ef8ba969-6186-4b9a-b30f-ec9a70036bab {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2006}}
Nov 29 03:07:20 user nova-compute[75234]: DEBUG nova.network.neutron [req-a4c25f06-f403-4d6a-8693-3d4311ea74a3 req-cf16079c-ad19-432a-8160-8f104df582f1 service nova] [instance: 763a3e14-4563-4dc2-99ae-c340edbb93df] Instance cache missing network info. {{(pid=75234) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3322}}
Nov 29 03:07:20 user nova-compute[75234]: DEBUG nova.network.neutron [req-a4c25f06-f403-4d6a-8693-3d4311ea74a3 req-cf16079c-ad19-432a-8160-8f104df582f1 service nova] [instance: 763a3e14-4563-4dc2-99ae-c340edbb93df] Updating instance_info_cache with network_info: [] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:07:20 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-a4c25f06-f403-4d6a-8693-3d4311ea74a3 req-cf16079c-ad19-432a-8160-8f104df582f1 service nova] Releasing lock "refresh_cache-763a3e14-4563-4dc2-99ae-c340edbb93df" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:07:20 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] Acquired lock "refresh_cache-763a3e14-4563-4dc2-99ae-c340edbb93df" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:07:20 user nova-compute[75234]: DEBUG nova.network.neutron [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] [instance: 763a3e14-4563-4dc2-99ae-c340edbb93df] Building network info cache for instance {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2009}}
Nov 29 03:07:21 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:21 user nova-compute[75234]: DEBUG nova.compute.provider_tree [None req-78ea8aa1-48fd-4c8d-b0d1-dcdfe7cfafaf tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] Inventory has not changed in ProviderTree for provider: 4d291220-7c37-4839-a0e3-b23c66502b74 {{(pid=75234) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Nov 29 03:07:21 user nova-compute[75234]: DEBUG nova.scheduler.client.report [None req-78ea8aa1-48fd-4c8d-b0d1-dcdfe7cfafaf tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] Inventory has not changed for provider 4d291220-7c37-4839-a0e3-b23c66502b74 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=75234) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Nov 29 03:07:21 user nova-compute[75234]: DEBUG nova.network.neutron [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] [instance: 763a3e14-4563-4dc2-99ae-c340edbb93df] Instance cache missing network info. {{(pid=75234) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3322}}
Nov 29 03:07:21 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-78ea8aa1-48fd-4c8d-b0d1-dcdfe7cfafaf tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.447s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:07:21 user nova-compute[75234]: DEBUG nova.compute.manager [None req-78ea8aa1-48fd-4c8d-b0d1-dcdfe7cfafaf tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] [instance: f7d0bdeb-6a05-47ac-87f2-c8380a9a4286] Start building networks asynchronously for instance. {{(pid=75234) _build_resources /opt/stack/nova/nova/compute/manager.py:2819}}
Nov 29 03:07:21 user nova-compute[75234]: DEBUG nova.compute.manager [None req-78ea8aa1-48fd-4c8d-b0d1-dcdfe7cfafaf tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] [instance: f7d0bdeb-6a05-47ac-87f2-c8380a9a4286] Allocating IP information in the background. {{(pid=75234) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1972}}
Nov 29 03:07:21 user nova-compute[75234]: DEBUG nova.network.neutron [None req-78ea8aa1-48fd-4c8d-b0d1-dcdfe7cfafaf tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] [instance: f7d0bdeb-6a05-47ac-87f2-c8380a9a4286] allocate_for_instance() {{(pid=75234) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1156}}
Nov 29 03:07:21 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-78ea8aa1-48fd-4c8d-b0d1-dcdfe7cfafaf tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] [instance: f7d0bdeb-6a05-47ac-87f2-c8380a9a4286] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Nov 29 03:07:21 user nova-compute[75234]: DEBUG nova.compute.manager [None req-78ea8aa1-48fd-4c8d-b0d1-dcdfe7cfafaf tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] [instance: f7d0bdeb-6a05-47ac-87f2-c8380a9a4286] Start building block device mappings for instance. {{(pid=75234) _build_resources /opt/stack/nova/nova/compute/manager.py:2854}}
Nov 29 03:07:21 user nova-compute[75234]: DEBUG nova.policy [None req-78ea8aa1-48fd-4c8d-b0d1-dcdfe7cfafaf tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '36692f8f1e9649c79183c447666364e2', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b793707804a24961968ac432a37d9581', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=75234) authorize /opt/stack/nova/nova/policy.py:203}}
Nov 29 03:07:21 user nova-compute[75234]: DEBUG nova.compute.manager [None req-78ea8aa1-48fd-4c8d-b0d1-dcdfe7cfafaf tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] [instance: f7d0bdeb-6a05-47ac-87f2-c8380a9a4286] Start spawning the instance on the hypervisor. {{(pid=75234) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2628}}
Nov 29 03:07:21 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-78ea8aa1-48fd-4c8d-b0d1-dcdfe7cfafaf tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] [instance: f7d0bdeb-6a05-47ac-87f2-c8380a9a4286] Creating instance directory {{(pid=75234) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4727}}
Nov 29 03:07:21 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-78ea8aa1-48fd-4c8d-b0d1-dcdfe7cfafaf tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] [instance: f7d0bdeb-6a05-47ac-87f2-c8380a9a4286] Creating image(s)
Nov 29 03:07:21 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-78ea8aa1-48fd-4c8d-b0d1-dcdfe7cfafaf tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] Acquiring lock "/opt/stack/data/nova/instances/f7d0bdeb-6a05-47ac-87f2-c8380a9a4286/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:07:21 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-78ea8aa1-48fd-4c8d-b0d1-dcdfe7cfafaf tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] Lock "/opt/stack/data/nova/instances/f7d0bdeb-6a05-47ac-87f2-c8380a9a4286/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:07:21 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-78ea8aa1-48fd-4c8d-b0d1-dcdfe7cfafaf tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] Lock "/opt/stack/data/nova/instances/f7d0bdeb-6a05-47ac-87f2-c8380a9a4286/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.007s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:07:21 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-78ea8aa1-48fd-4c8d-b0d1-dcdfe7cfafaf tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:07:21 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-78ea8aa1-48fd-4c8d-b0d1-dcdfe7cfafaf tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json" returned: 0 in 0.242s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:07:21 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-78ea8aa1-48fd-4c8d-b0d1-dcdfe7cfafaf tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] Acquiring lock "80686e29dc9db6600d03e85cb62af298e12d7918" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:07:21 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-78ea8aa1-48fd-4c8d-b0d1-dcdfe7cfafaf tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] Lock "80686e29dc9db6600d03e85cb62af298e12d7918" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:07:21 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-78ea8aa1-48fd-4c8d-b0d1-dcdfe7cfafaf tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:07:21 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-78ea8aa1-48fd-4c8d-b0d1-dcdfe7cfafaf tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json" returned: 0 in 0.194s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:07:21 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-78ea8aa1-48fd-4c8d-b0d1-dcdfe7cfafaf tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918,backing_fmt=raw /opt/stack/data/nova/instances/f7d0bdeb-6a05-47ac-87f2-c8380a9a4286/disk 1073741824 {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:07:21 user nova-compute[75234]: DEBUG nova.network.neutron [None req-78ea8aa1-48fd-4c8d-b0d1-dcdfe7cfafaf tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] [instance: f7d0bdeb-6a05-47ac-87f2-c8380a9a4286] Successfully created port: b5192c04-0323-48b8-a326-ddf6b21f9019 {{(pid=75234) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:548}}
Nov 29 03:07:21 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-78ea8aa1-48fd-4c8d-b0d1-dcdfe7cfafaf tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918,backing_fmt=raw /opt/stack/data/nova/instances/f7d0bdeb-6a05-47ac-87f2-c8380a9a4286/disk 1073741824" returned: 0 in 0.054s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:07:21 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-78ea8aa1-48fd-4c8d-b0d1-dcdfe7cfafaf tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] Lock "80686e29dc9db6600d03e85cb62af298e12d7918" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.252s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:07:21 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-78ea8aa1-48fd-4c8d-b0d1-dcdfe7cfafaf tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:07:22 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-78ea8aa1-48fd-4c8d-b0d1-dcdfe7cfafaf tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json" returned: 0 in 0.175s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:07:22 user nova-compute[75234]: DEBUG nova.virt.disk.api [None req-78ea8aa1-48fd-4c8d-b0d1-dcdfe7cfafaf tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] Checking if we can resize image /opt/stack/data/nova/instances/f7d0bdeb-6a05-47ac-87f2-c8380a9a4286/disk. size=1073741824 {{(pid=75234) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:166}}
Nov 29 03:07:22 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-78ea8aa1-48fd-4c8d-b0d1-dcdfe7cfafaf tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f7d0bdeb-6a05-47ac-87f2-c8380a9a4286/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:07:22 user nova-compute[75234]: DEBUG nova.compute.manager [req-1723ca6d-b764-4095-9908-763c710d897f req-f26ad7a9-3a9a-44db-a977-3aafc50355ab service nova] [instance: 763a3e14-4563-4dc2-99ae-c340edbb93df] Received event network-changed-92917a0d-59ee-4167-8693-262a21f6e156 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:07:22 user nova-compute[75234]: DEBUG nova.compute.manager [req-1723ca6d-b764-4095-9908-763c710d897f req-f26ad7a9-3a9a-44db-a977-3aafc50355ab service nova] [instance: 763a3e14-4563-4dc2-99ae-c340edbb93df] Refreshing instance network info cache due to event network-changed-92917a0d-59ee-4167-8693-262a21f6e156. {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11117}}
Nov 29 03:07:22 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-1723ca6d-b764-4095-9908-763c710d897f req-f26ad7a9-3a9a-44db-a977-3aafc50355ab service nova] Acquiring lock "refresh_cache-763a3e14-4563-4dc2-99ae-c340edbb93df" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:07:22 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-78ea8aa1-48fd-4c8d-b0d1-dcdfe7cfafaf tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f7d0bdeb-6a05-47ac-87f2-c8380a9a4286/disk --force-share --output=json" returned: 0 in 0.169s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:07:22 user nova-compute[75234]: DEBUG nova.virt.disk.api [None req-78ea8aa1-48fd-4c8d-b0d1-dcdfe7cfafaf tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] Cannot resize image /opt/stack/data/nova/instances/f7d0bdeb-6a05-47ac-87f2-c8380a9a4286/disk to a smaller size. {{(pid=75234) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:172}}
Nov 29 03:07:22 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-78ea8aa1-48fd-4c8d-b0d1-dcdfe7cfafaf tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] [instance: f7d0bdeb-6a05-47ac-87f2-c8380a9a4286] Created local disks {{(pid=75234) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4861}}
Nov 29 03:07:22 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-78ea8aa1-48fd-4c8d-b0d1-dcdfe7cfafaf tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] [instance: f7d0bdeb-6a05-47ac-87f2-c8380a9a4286] Ensure instance console log exists: /opt/stack/data/nova/instances/f7d0bdeb-6a05-47ac-87f2-c8380a9a4286/console.log {{(pid=75234) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4613}}
Nov 29 03:07:22 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-78ea8aa1-48fd-4c8d-b0d1-dcdfe7cfafaf tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:07:22 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-78ea8aa1-48fd-4c8d-b0d1-dcdfe7cfafaf tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:07:22 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-78ea8aa1-48fd-4c8d-b0d1-dcdfe7cfafaf tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:07:22 user nova-compute[75234]: DEBUG nova.network.neutron [None req-78ea8aa1-48fd-4c8d-b0d1-dcdfe7cfafaf tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] [instance: f7d0bdeb-6a05-47ac-87f2-c8380a9a4286] Successfully updated port: b5192c04-0323-48b8-a326-ddf6b21f9019 {{(pid=75234) _update_port /opt/stack/nova/nova/network/neutron.py:586}}
Nov 29 03:07:22 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-78ea8aa1-48fd-4c8d-b0d1-dcdfe7cfafaf tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] Acquiring lock "refresh_cache-f7d0bdeb-6a05-47ac-87f2-c8380a9a4286" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:07:22 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-78ea8aa1-48fd-4c8d-b0d1-dcdfe7cfafaf tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] Acquired lock "refresh_cache-f7d0bdeb-6a05-47ac-87f2-c8380a9a4286" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:07:22 user nova-compute[75234]: DEBUG nova.network.neutron [None req-78ea8aa1-48fd-4c8d-b0d1-dcdfe7cfafaf tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] [instance: f7d0bdeb-6a05-47ac-87f2-c8380a9a4286] Building network info cache for instance {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2009}}
Nov 29 03:07:22 user nova-compute[75234]: DEBUG nova.network.neutron [None req-78ea8aa1-48fd-4c8d-b0d1-dcdfe7cfafaf tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] [instance: f7d0bdeb-6a05-47ac-87f2-c8380a9a4286] Instance cache missing network info. {{(pid=75234) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3322}}
Nov 29 03:07:22 user nova-compute[75234]: DEBUG nova.network.neutron [None req-78ea8aa1-48fd-4c8d-b0d1-dcdfe7cfafaf tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] [instance: f7d0bdeb-6a05-47ac-87f2-c8380a9a4286] Updating instance_info_cache with network_info: [{"id": "b5192c04-0323-48b8-a326-ddf6b21f9019", "address": "fa:16:3e:59:dd:b5", "network": {"id": "22b1262f-108b-45c1-91cb-a0f77f9bfccf", "bridge": "br-int", "label": "tempest-TestServerVolumeAttachmentScenario-1154356954-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "b793707804a24961968ac432a37d9581", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapb5192c04-03", "ovs_interfaceid": "b5192c04-0323-48b8-a326-ddf6b21f9019", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:07:22 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-78ea8aa1-48fd-4c8d-b0d1-dcdfe7cfafaf tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] Releasing lock "refresh_cache-f7d0bdeb-6a05-47ac-87f2-c8380a9a4286" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:07:22 user nova-compute[75234]: DEBUG nova.compute.manager [None req-78ea8aa1-48fd-4c8d-b0d1-dcdfe7cfafaf tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] [instance: f7d0bdeb-6a05-47ac-87f2-c8380a9a4286] Instance network_info: |[{"id": "b5192c04-0323-48b8-a326-ddf6b21f9019", "address": "fa:16:3e:59:dd:b5", "network": {"id": "22b1262f-108b-45c1-91cb-a0f77f9bfccf", "bridge": "br-int", "label": "tempest-TestServerVolumeAttachmentScenario-1154356954-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "b793707804a24961968ac432a37d9581", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapb5192c04-03", "ovs_interfaceid": "b5192c04-0323-48b8-a326-ddf6b21f9019", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=75234) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1987}}
Nov 29 03:07:22 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-78ea8aa1-48fd-4c8d-b0d1-dcdfe7cfafaf tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] [instance: f7d0bdeb-6a05-47ac-87f2-c8380a9a4286] Start _get_guest_xml network_info=[{"id": "b5192c04-0323-48b8-a326-ddf6b21f9019", "address": "fa:16:3e:59:dd:b5", "network": {"id": "22b1262f-108b-45c1-91cb-a0f77f9bfccf", "bridge": "br-int", "label": "tempest-TestServerVolumeAttachmentScenario-1154356954-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "b793707804a24961968ac432a37d9581", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapb5192c04-03", "ovs_interfaceid": "b5192c04-0323-48b8-a326-ddf6b21f9019", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum='c8fc807773e5354afe61636071771906',container_format='bare',created_at=2023-11-29T03:02:15Z,direct_url=<?>,disk_format='qcow2',id=96c30ba7-9b73-4a30-8ba4-7138879573b7,min_disk=0,min_ram=0,name='cirros-0.6.2-x86_64-disk',owner='b18e380921c748b78517b6f1211cd0f1',properties=ImageMetaProps,protected=<?>,size=21430272,status='active',tags=<?>,updated_at=2023-11-29T03:02:17Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [{'device_name': '/dev/vda', 'encryption_format': None, 'encrypted': False, 'device_type': 'disk', 'size': 0, 'disk_bus': 'virtio', 'boot_index': 0, 'encryption_options': None, 'encryption_secret_uuid': None, 'guest_format': None, 'image_id': '96c30ba7-9b73-4a30-8ba4-7138879573b7'}], 'ephemerals': [], 'block_device_mapping': [], 'swap': None} {{(pid=75234) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7562}}
Nov 29 03:07:22 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-78ea8aa1-48fd-4c8d-b0d1-dcdfe7cfafaf tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:07:22 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-78ea8aa1-48fd-4c8d-b0d1-dcdfe7cfafaf tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:07:22 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-78ea8aa1-48fd-4c8d-b0d1-dcdfe7cfafaf tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] Searching host: 'user' for CPU controller through CGroups V1... {{(pid=75234) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1652}}
Nov 29 03:07:22 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-78ea8aa1-48fd-4c8d-b0d1-dcdfe7cfafaf tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] CPU controller missing on host. {{(pid=75234) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1662}}
Nov 29 03:07:22 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-78ea8aa1-48fd-4c8d-b0d1-dcdfe7cfafaf tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] Searching host: 'user' for CPU controller through CGroups V2... {{(pid=75234) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1671}}
Nov 29 03:07:22 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-78ea8aa1-48fd-4c8d-b0d1-dcdfe7cfafaf tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] CPU controller found on host. {{(pid=75234) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1678}}
Nov 29 03:07:22 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-78ea8aa1-48fd-4c8d-b0d1-dcdfe7cfafaf tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=75234) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5400}}
Nov 29 03:07:22 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-78ea8aa1-48fd-4c8d-b0d1-dcdfe7cfafaf tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-11-29T03:02:04Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='1',id=6,is_public=True,memory_mb=512,name='m1.tiny',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='c8fc807773e5354afe61636071771906',container_format='bare',created_at=2023-11-29T03:02:15Z,direct_url=<?>,disk_format='qcow2',id=96c30ba7-9b73-4a30-8ba4-7138879573b7,min_disk=0,min_ram=0,name='cirros-0.6.2-x86_64-disk',owner='b18e380921c748b78517b6f1211cd0f1',properties=ImageMetaProps,protected=<?>,size=21430272,status='active',tags=<?>,updated_at=2023-11-29T03:02:17Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:563}}
Nov 29 03:07:22 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-78ea8aa1-48fd-4c8d-b0d1-dcdfe7cfafaf tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] Flavor limits 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:348}}
Nov 29 03:07:22 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-78ea8aa1-48fd-4c8d-b0d1-dcdfe7cfafaf tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] Image limits 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:352}}
Nov 29 03:07:22 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-78ea8aa1-48fd-4c8d-b0d1-dcdfe7cfafaf tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] Flavor pref 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:388}}
Nov 29 03:07:22 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-78ea8aa1-48fd-4c8d-b0d1-dcdfe7cfafaf tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] Image pref 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:392}}
Nov 29 03:07:22 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-78ea8aa1-48fd-4c8d-b0d1-dcdfe7cfafaf tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:430}}
Nov 29 03:07:22 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-78ea8aa1-48fd-4c8d-b0d1-dcdfe7cfafaf tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:569}}
Nov 29 03:07:23 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-78ea8aa1-48fd-4c8d-b0d1-dcdfe7cfafaf tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=75234) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:471}}
Nov 29 03:07:23 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-78ea8aa1-48fd-4c8d-b0d1-dcdfe7cfafaf tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] Got 1 possible topologies {{(pid=75234) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:501}}
Nov 29 03:07:23 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-78ea8aa1-48fd-4c8d-b0d1-dcdfe7cfafaf tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:575}}
Nov 29 03:07:23 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-78ea8aa1-48fd-4c8d-b0d1-dcdfe7cfafaf tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:577}}
Nov 29 03:07:23 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-78ea8aa1-48fd-4c8d-b0d1-dcdfe7cfafaf tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-11-29T03:07:20Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-TestServerVolumeAttachmentScenario-server-2022761297',display_name='tempest-TestServerVolumeAttachmentScenario-server-2022761297',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-testservervolumeattachmentscenario-server-2022761297',id=12,image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBLSetanuoGY87fz1sA6LqyoFxZXw/IT2g+PnGJrAk3M0CxyfGSx2WYx6YjLRXYZ+sLnP2egb5hufwvhkiC775qvjKUtZ+ivGOwDpgIHPbDOr0aWZsSvtPkGBAHDATSfNHg==',key_name='tempest-keypair-645674634',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='b793707804a24961968ac432a37d9581',ramdisk_id='',reservation_id='r-7jpogwhz',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TestServerVolumeAttachmentScenario-1219797116',owner_user_name='tempest-TestServerVolumeAttachmentScenario-1219797116-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-11-29T03:07:21Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='36692f8f1e9649c79183c447666364e2',uuid=f7d0bdeb-6a05-47ac-87f2-c8380a9a4286,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "b5192c04-0323-48b8-a326-ddf6b21f9019", "address": "fa:16:3e:59:dd:b5", "network": {"id": "22b1262f-108b-45c1-91cb-a0f77f9bfccf", "bridge": "br-int", "label": "tempest-TestServerVolumeAttachmentScenario-1154356954-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "b793707804a24961968ac432a37d9581", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapb5192c04-03", "ovs_interfaceid": "b5192c04-0323-48b8-a326-ddf6b21f9019", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=75234) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Nov 29 03:07:23 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-78ea8aa1-48fd-4c8d-b0d1-dcdfe7cfafaf tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] Converting VIF {"id": "b5192c04-0323-48b8-a326-ddf6b21f9019", "address": "fa:16:3e:59:dd:b5", "network": {"id": "22b1262f-108b-45c1-91cb-a0f77f9bfccf", "bridge": "br-int", "label": "tempest-TestServerVolumeAttachmentScenario-1154356954-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "b793707804a24961968ac432a37d9581", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapb5192c04-03", "ovs_interfaceid": "b5192c04-0323-48b8-a326-ddf6b21f9019", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:07:23 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-78ea8aa1-48fd-4c8d-b0d1-dcdfe7cfafaf tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:59:dd:b5,bridge_name='br-int',has_traffic_filtering=True,id=b5192c04-0323-48b8-a326-ddf6b21f9019,network=Network(22b1262f-108b-45c1-91cb-a0f77f9bfccf),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapb5192c04-03') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:07:23 user nova-compute[75234]: DEBUG nova.objects.instance [None req-78ea8aa1-48fd-4c8d-b0d1-dcdfe7cfafaf tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] Lazy-loading 'pci_devices' on Instance uuid f7d0bdeb-6a05-47ac-87f2-c8380a9a4286 {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:07:23 user nova-compute[75234]: DEBUG nova.network.neutron [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] [instance: 763a3e14-4563-4dc2-99ae-c340edbb93df] Updating instance_info_cache with network_info: [{"id": "dc96090f-e855-4d3e-854c-50ea700dc8ea", "address": "fa:16:3e:9e:2b:8a", "network": {"id": "ea3baf6d-ba72-4615-bc0d-604e4fcd8e4e", "bridge": "br-int", "label": "tempest-TaggedBootDevicesTest_v242-1704264625-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "325013d2bedc498e91c2145c2b398803", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapdc96090f-e8", "ovs_interfaceid": "dc96090f-e855-4d3e-854c-50ea700dc8ea", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "2027f7a1-36e4-4434-a063-61e817758917", "address": "fa:16:3e:4d:66:94", "network": {"id": "60b448f6-90dc-4dac-b24e-9370262e6dac", "bridge": "br-int", "label": "tempest-device-tagging-net1-431494419", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.80", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "325013d2bedc498e91c2145c2b398803", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2027f7a1-36", "ovs_interfaceid": "2027f7a1-36e4-4434-a063-61e817758917", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}}, {"id": "de4ad363-fae0-4d1b-9820-5a91a27d630d", "address": "fa:16:3e:94:49:94", "network": {"id": "60b448f6-90dc-4dac-b24e-9370262e6dac", "bridge": "br-int", "label": "tempest-device-tagging-net1-431494419", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.218", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "325013d2bedc498e91c2145c2b398803", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapde4ad363-fa", "ovs_interfaceid": "de4ad363-fae0-4d1b-9820-5a91a27d630d", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}}, {"id": "d520a88d-ec02-45b8-adbc-484d38b9c18c", "address": "fa:16:3e:ac:cd:aa", "network": {"id": "60b448f6-90dc-4dac-b24e-9370262e6dac", "bridge": "br-int", "label": "tempest-device-tagging-net1-431494419", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.226", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "325013d2bedc498e91c2145c2b398803", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapd520a88d-ec", "ovs_interfaceid": "d520a88d-ec02-45b8-adbc-484d38b9c18c", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "c50f64cd-b7fa-4580-bb78-bff756a0773a", "address": "fa:16:3e:36:0b:d9", "network": {"id": "60b448f6-90dc-4dac-b24e-9370262e6dac", "bridge": "br-int", "label": "tempest-device-tagging-net1-431494419", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.18", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "325013d2bedc498e91c2145c2b398803", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc50f64cd-b7", "ovs_interfaceid": "c50f64cd-b7fa-4580-bb78-bff756a0773a", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "ef8ba969-6186-4b9a-b30f-ec9a70036bab", "address": "fa:16:3e:0f:ed:74", "network": {"id": "31e25520-c84d-451c-9132-ddc8c1e443a6", "bridge": "br-int", "label": "tempest-device-tagging-net2-653330954", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.100", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "325013d2bedc498e91c2145c2b398803", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapef8ba969-61", "ovs_interfaceid": "ef8ba969-6186-4b9a-b30f-ec9a70036bab", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "92917a0d-59ee-4167-8693-262a21f6e156", "address": "fa:16:3e:65:95:01", "network": {"id": "31e25520-c84d-451c-9132-ddc8c1e443a6", "bridge": "br-int", "label": "tempest-device-tagging-net2-653330954", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.200", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "325013d2bedc498e91c2145c2b398803", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap92917a0d-59", "ovs_interfaceid": "92917a0d-59ee-4167-8693-262a21f6e156", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:07:23 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-78ea8aa1-48fd-4c8d-b0d1-dcdfe7cfafaf tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] [instance: f7d0bdeb-6a05-47ac-87f2-c8380a9a4286] End _get_guest_xml xml=<domain type="kvm">
Nov 29 03:07:23 user nova-compute[75234]:   <uuid>f7d0bdeb-6a05-47ac-87f2-c8380a9a4286</uuid>
Nov 29 03:07:23 user nova-compute[75234]:   <name>instance-0000000c</name>
Nov 29 03:07:23 user nova-compute[75234]:   <memory>524288</memory>
Nov 29 03:07:23 user nova-compute[75234]:   <vcpu>1</vcpu>
Nov 29 03:07:23 user nova-compute[75234]:   <metadata>
Nov 29 03:07:23 user nova-compute[75234]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Nov 29 03:07:23 user nova-compute[75234]:       <nova:package version="0.0.0"/>
Nov 29 03:07:23 user nova-compute[75234]:       <nova:name>tempest-TestServerVolumeAttachmentScenario-server-2022761297</nova:name>
Nov 29 03:07:23 user nova-compute[75234]:       <nova:creationTime>2023-11-29 03:07:22</nova:creationTime>
Nov 29 03:07:23 user nova-compute[75234]:       <nova:flavor name="m1.tiny">
Nov 29 03:07:23 user nova-compute[75234]:         <nova:memory>512</nova:memory>
Nov 29 03:07:23 user nova-compute[75234]:         <nova:disk>1</nova:disk>
Nov 29 03:07:23 user nova-compute[75234]:         <nova:swap>0</nova:swap>
Nov 29 03:07:23 user nova-compute[75234]:         <nova:ephemeral>0</nova:ephemeral>
Nov 29 03:07:23 user nova-compute[75234]:         <nova:vcpus>1</nova:vcpus>
Nov 29 03:07:23 user nova-compute[75234]:       </nova:flavor>
Nov 29 03:07:23 user nova-compute[75234]:       <nova:owner>
Nov 29 03:07:23 user nova-compute[75234]:         <nova:user uuid="36692f8f1e9649c79183c447666364e2">tempest-TestServerVolumeAttachmentScenario-1219797116-project-member</nova:user>
Nov 29 03:07:23 user nova-compute[75234]:         <nova:project uuid="b793707804a24961968ac432a37d9581">tempest-TestServerVolumeAttachmentScenario-1219797116</nova:project>
Nov 29 03:07:23 user nova-compute[75234]:       </nova:owner>
Nov 29 03:07:23 user nova-compute[75234]:       <nova:root type="image" uuid="96c30ba7-9b73-4a30-8ba4-7138879573b7"/>
Nov 29 03:07:23 user nova-compute[75234]:       <nova:ports>
Nov 29 03:07:23 user nova-compute[75234]:         <nova:port uuid="b5192c04-0323-48b8-a326-ddf6b21f9019">
Nov 29 03:07:23 user nova-compute[75234]:           <nova:ip type="fixed" address="10.0.0.11" ipVersion="4"/>
Nov 29 03:07:23 user nova-compute[75234]:         </nova:port>
Nov 29 03:07:23 user nova-compute[75234]:       </nova:ports>
Nov 29 03:07:23 user nova-compute[75234]:     </nova:instance>
Nov 29 03:07:23 user nova-compute[75234]:   </metadata>
Nov 29 03:07:23 user nova-compute[75234]:   <sysinfo type="smbios">
Nov 29 03:07:23 user nova-compute[75234]:     <system>
Nov 29 03:07:23 user nova-compute[75234]:       <entry name="manufacturer">OpenStack Foundation</entry>
Nov 29 03:07:23 user nova-compute[75234]:       <entry name="product">OpenStack Nova</entry>
Nov 29 03:07:23 user nova-compute[75234]:       <entry name="version">0.0.0</entry>
Nov 29 03:07:23 user nova-compute[75234]:       <entry name="serial">f7d0bdeb-6a05-47ac-87f2-c8380a9a4286</entry>
Nov 29 03:07:23 user nova-compute[75234]:       <entry name="uuid">f7d0bdeb-6a05-47ac-87f2-c8380a9a4286</entry>
Nov 29 03:07:23 user nova-compute[75234]:       <entry name="family">Virtual Machine</entry>
Nov 29 03:07:23 user nova-compute[75234]:     </system>
Nov 29 03:07:23 user nova-compute[75234]:   </sysinfo>
Nov 29 03:07:23 user nova-compute[75234]:   <os>
Nov 29 03:07:23 user nova-compute[75234]:     <type arch="x86_64" machine="pc">hvm</type>
Nov 29 03:07:23 user nova-compute[75234]:     <boot dev="hd"/>
Nov 29 03:07:23 user nova-compute[75234]:     <smbios mode="sysinfo"/>
Nov 29 03:07:23 user nova-compute[75234]:   </os>
Nov 29 03:07:23 user nova-compute[75234]:   <features>
Nov 29 03:07:23 user nova-compute[75234]:     <acpi/>
Nov 29 03:07:23 user nova-compute[75234]:     <apic/>
Nov 29 03:07:23 user nova-compute[75234]:     <vmcoreinfo/>
Nov 29 03:07:23 user nova-compute[75234]:   </features>
Nov 29 03:07:23 user nova-compute[75234]:   <clock offset="utc">
Nov 29 03:07:23 user nova-compute[75234]:     <timer name="pit" tickpolicy="delay"/>
Nov 29 03:07:23 user nova-compute[75234]:     <timer name="rtc" tickpolicy="catchup"/>
Nov 29 03:07:23 user nova-compute[75234]:     <timer name="hpet" present="no"/>
Nov 29 03:07:23 user nova-compute[75234]:   </clock>
Nov 29 03:07:23 user nova-compute[75234]:   <cpu mode="custom" match="exact">
Nov 29 03:07:23 user nova-compute[75234]:     <model>Nehalem</model>
Nov 29 03:07:23 user nova-compute[75234]:     <topology sockets="1" cores="1" threads="1"/>
Nov 29 03:07:23 user nova-compute[75234]:   </cpu>
Nov 29 03:07:23 user nova-compute[75234]:   <devices>
Nov 29 03:07:23 user nova-compute[75234]:     <disk type="file" device="disk">
Nov 29 03:07:23 user nova-compute[75234]:       <driver name="qemu" type="qcow2" cache="none"/>
Nov 29 03:07:23 user nova-compute[75234]:       <source file="/opt/stack/data/nova/instances/f7d0bdeb-6a05-47ac-87f2-c8380a9a4286/disk"/>
Nov 29 03:07:23 user nova-compute[75234]:       <target dev="vda" bus="virtio"/>
Nov 29 03:07:23 user nova-compute[75234]:     </disk>
Nov 29 03:07:23 user nova-compute[75234]:     <interface type="ethernet">
Nov 29 03:07:23 user nova-compute[75234]:       <mac address="fa:16:3e:59:dd:b5"/>
Nov 29 03:07:23 user nova-compute[75234]:       <model type="virtio"/>
Nov 29 03:07:23 user nova-compute[75234]:       <mtu size="1442"/>
Nov 29 03:07:23 user nova-compute[75234]:       <target dev="tapb5192c04-03"/>
Nov 29 03:07:23 user nova-compute[75234]:     </interface>
Nov 29 03:07:23 user nova-compute[75234]:     <serial type="pty">
Nov 29 03:07:23 user nova-compute[75234]:       <log file="/opt/stack/data/nova/instances/f7d0bdeb-6a05-47ac-87f2-c8380a9a4286/console.log" append="off"/>
Nov 29 03:07:23 user nova-compute[75234]:     </serial>
Nov 29 03:07:23 user nova-compute[75234]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Nov 29 03:07:23 user nova-compute[75234]:     <video>
Nov 29 03:07:23 user nova-compute[75234]:       <model type="virtio"/>
Nov 29 03:07:23 user nova-compute[75234]:     </video>
Nov 29 03:07:23 user nova-compute[75234]:     <rng model="virtio">
Nov 29 03:07:23 user nova-compute[75234]:       <backend model="random">/dev/urandom</backend>
Nov 29 03:07:23 user nova-compute[75234]:     </rng>
Nov 29 03:07:23 user nova-compute[75234]:     <controller type="usb" index="0" model="none"/>
Nov 29 03:07:23 user nova-compute[75234]:     <memballoon model="virtio">
Nov 29 03:07:23 user nova-compute[75234]:       <stats period="10"/>
Nov 29 03:07:23 user nova-compute[75234]:     </memballoon>
Nov 29 03:07:23 user nova-compute[75234]:   </devices>
Nov 29 03:07:23 user nova-compute[75234]: </domain>
Nov 29 03:07:23 user nova-compute[75234]:  {{(pid=75234) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7568}}
Nov 29 03:07:23 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-78ea8aa1-48fd-4c8d-b0d1-dcdfe7cfafaf tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-11-29T03:07:20Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-TestServerVolumeAttachmentScenario-server-2022761297',display_name='tempest-TestServerVolumeAttachmentScenario-server-2022761297',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-testservervolumeattachmentscenario-server-2022761297',id=12,image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBLSetanuoGY87fz1sA6LqyoFxZXw/IT2g+PnGJrAk3M0CxyfGSx2WYx6YjLRXYZ+sLnP2egb5hufwvhkiC775qvjKUtZ+ivGOwDpgIHPbDOr0aWZsSvtPkGBAHDATSfNHg==',key_name='tempest-keypair-645674634',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='b793707804a24961968ac432a37d9581',ramdisk_id='',reservation_id='r-7jpogwhz',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TestServerVolumeAttachmentScenario-1219797116',owner_user_name='tempest-TestServerVolumeAttachmentScenario-1219797116-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-11-29T03:07:21Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='36692f8f1e9649c79183c447666364e2',uuid=f7d0bdeb-6a05-47ac-87f2-c8380a9a4286,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "b5192c04-0323-48b8-a326-ddf6b21f9019", "address": "fa:16:3e:59:dd:b5", "network": {"id": "22b1262f-108b-45c1-91cb-a0f77f9bfccf", "bridge": "br-int", "label": "tempest-TestServerVolumeAttachmentScenario-1154356954-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "b793707804a24961968ac432a37d9581", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapb5192c04-03", "ovs_interfaceid": "b5192c04-0323-48b8-a326-ddf6b21f9019", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Nov 29 03:07:23 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-78ea8aa1-48fd-4c8d-b0d1-dcdfe7cfafaf tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] Converting VIF {"id": "b5192c04-0323-48b8-a326-ddf6b21f9019", "address": "fa:16:3e:59:dd:b5", "network": {"id": "22b1262f-108b-45c1-91cb-a0f77f9bfccf", "bridge": "br-int", "label": "tempest-TestServerVolumeAttachmentScenario-1154356954-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "b793707804a24961968ac432a37d9581", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapb5192c04-03", "ovs_interfaceid": "b5192c04-0323-48b8-a326-ddf6b21f9019", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:07:23 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-78ea8aa1-48fd-4c8d-b0d1-dcdfe7cfafaf tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:59:dd:b5,bridge_name='br-int',has_traffic_filtering=True,id=b5192c04-0323-48b8-a326-ddf6b21f9019,network=Network(22b1262f-108b-45c1-91cb-a0f77f9bfccf),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapb5192c04-03') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:07:23 user nova-compute[75234]: DEBUG os_vif [None req-78ea8aa1-48fd-4c8d-b0d1-dcdfe7cfafaf tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:59:dd:b5,bridge_name='br-int',has_traffic_filtering=True,id=b5192c04-0323-48b8-a326-ddf6b21f9019,network=Network(22b1262f-108b-45c1-91cb-a0f77f9bfccf),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapb5192c04-03') {{(pid=75234) plug /opt/stack/data/venv/lib/python3.10/site-packages/os_vif/__init__.py:76}}
Nov 29 03:07:23 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:23 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:07:23 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Nov 29 03:07:23 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:23 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbCreateCommand(_result=None, table=QoS, columns={'type': 'linux-noop', 'external_ids': {'id': 'fd962d41-59f7-562b-b197-fa67434268ed', '_type': 'linux-noop'}}, row=False) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:07:23 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] Releasing lock "refresh_cache-763a3e14-4563-4dc2-99ae-c340edbb93df" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:07:23 user nova-compute[75234]: DEBUG nova.compute.manager [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] [instance: 763a3e14-4563-4dc2-99ae-c340edbb93df] Instance network_info: |[{"id": "dc96090f-e855-4d3e-854c-50ea700dc8ea", "address": "fa:16:3e:9e:2b:8a", "network": {"id": "ea3baf6d-ba72-4615-bc0d-604e4fcd8e4e", "bridge": "br-int", "label": "tempest-TaggedBootDevicesTest_v242-1704264625-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "325013d2bedc498e91c2145c2b398803", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapdc96090f-e8", "ovs_interfaceid": "dc96090f-e855-4d3e-854c-50ea700dc8ea", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "2027f7a1-36e4-4434-a063-61e817758917", "address": "fa:16:3e:4d:66:94", "network": {"id": "60b448f6-90dc-4dac-b24e-9370262e6dac", "bridge": "br-int", "label": "tempest-device-tagging-net1-431494419", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.80", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "325013d2bedc498e91c2145c2b398803", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2027f7a1-36", "ovs_interfaceid": "2027f7a1-36e4-4434-a063-61e817758917", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}}, {"id": "de4ad363-fae0-4d1b-9820-5a91a27d630d", "address": "fa:16:3e:94:49:94", "network": {"id": "60b448f6-90dc-4dac-b24e-9370262e6dac", "bridge": "br-int", "label": "tempest-device-tagging-net1-431494419", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.218", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "325013d2bedc498e91c2145c2b398803", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapde4ad363-fa", "ovs_interfaceid": "de4ad363-fae0-4d1b-9820-5a91a27d630d", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}}, {"id": "d520a88d-ec02-45b8-adbc-484d38b9c18c", "address": "fa:16:3e:ac:cd:aa", "network": {"id": "60b448f6-90dc-4dac-b24e-9370262e6dac", "bridge": "br-int", "label": "tempest-device-tagging-net1-431494419", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.226", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "325013d2bedc498e91c2145c2b398803", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapd520a88d-ec", "ovs_interfaceid": "d520a88d-ec02-45b8-adbc-484d38b9c18c", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "c50f64cd-b7fa-4580-bb78-bff756a0773a", "address": "fa:16:3e:36:0b:d9", "network": {"id": "60b448f6-90dc-4dac-b24e-9370262e6dac", "bridge": "br-int", "label": "tempest-device-tagging-net1-431494419", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.18", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "325013d2bedc498e91c2145c2b398803", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc50f64cd-b7", "ovs_interfaceid": "c50f64cd-b7fa-4580-bb78-bff756a0773a", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "ef8ba969-6186-4b9a-b30f-ec9a70036bab", "address": "fa:16:3e:0f:ed:74", "network": {"id": "31e25520-c84d-451c-9132-ddc8c1e443a6", "bridge": "br-int", "label": "tempest-device-tagging-net2-653330954", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.100", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "325013d2bedc498e91c2145c2b398803", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapef8ba969-61", "ovs_interfaceid": "ef8ba969-6186-4b9a-b30f-ec9a70036bab", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "92917a0d-59ee-4167-8693-262a21f6e156", "address": "fa:16:3e:65:95:01", "network": {"id": "31e25520-c84d-451c-9132-ddc8c1e443a6", "bridge": "br-int", "label": "tempest-device-tagging-net2-653330954", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.200", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "325013d2bedc498e91c2145c2b398803", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap92917a0d-59", "ovs_interfaceid": "92917a0d-59ee-4167-8693-262a21f6e156", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=75234) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1987}}
Nov 29 03:07:23 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:07:23 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-1723ca6d-b764-4095-9908-763c710d897f req-f26ad7a9-3a9a-44db-a977-3aafc50355ab service nova] Acquired lock "refresh_cache-763a3e14-4563-4dc2-99ae-c340edbb93df" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:07:23 user nova-compute[75234]: DEBUG nova.network.neutron [req-1723ca6d-b764-4095-9908-763c710d897f req-f26ad7a9-3a9a-44db-a977-3aafc50355ab service nova] [instance: 763a3e14-4563-4dc2-99ae-c340edbb93df] Refreshing network info cache for port 92917a0d-59ee-4167-8693-262a21f6e156 {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2006}}
Nov 29 03:07:23 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:23 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tapb5192c04-03, may_exist=True, interface_attrs={}) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:07:23 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Port, record=tapb5192c04-03, col_values=(('qos', UUID('53b88442-2bba-4191-8bad-c0ea61ad9462')),), if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:07:23 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DbSetCommand(_result=None, table=Interface, record=tapb5192c04-03, col_values=(('external_ids', {'iface-id': 'b5192c04-0323-48b8-a326-ddf6b21f9019', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:59:dd:b5', 'vm-uuid': 'f7d0bdeb-6a05-47ac-87f2-c8380a9a4286'}),), if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:07:23 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:07:23 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:23 user nova-compute[75234]: INFO os_vif [None req-78ea8aa1-48fd-4c8d-b0d1-dcdfe7cfafaf tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:59:dd:b5,bridge_name='br-int',has_traffic_filtering=True,id=b5192c04-0323-48b8-a326-ddf6b21f9019,network=Network(22b1262f-108b-45c1-91cb-a0f77f9bfccf),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapb5192c04-03')
Nov 29 03:07:23 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-78ea8aa1-48fd-4c8d-b0d1-dcdfe7cfafaf tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] No BDM found with device name vda, not building metadata. {{(pid=75234) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12089}}
Nov 29 03:07:23 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-78ea8aa1-48fd-4c8d-b0d1-dcdfe7cfafaf tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] No VIF found with MAC fa:16:3e:59:dd:b5, not building metadata {{(pid=75234) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Nov 29 03:07:23 user nova-compute[75234]: DEBUG nova.network.neutron [req-1723ca6d-b764-4095-9908-763c710d897f req-f26ad7a9-3a9a-44db-a977-3aafc50355ab service nova] [instance: 763a3e14-4563-4dc2-99ae-c340edbb93df] Updated VIF entry in instance network info cache for port 92917a0d-59ee-4167-8693-262a21f6e156. {{(pid=75234) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3481}}
Nov 29 03:07:23 user nova-compute[75234]: DEBUG nova.network.neutron [req-1723ca6d-b764-4095-9908-763c710d897f req-f26ad7a9-3a9a-44db-a977-3aafc50355ab service nova] [instance: 763a3e14-4563-4dc2-99ae-c340edbb93df] Updating instance_info_cache with network_info: [{"id": "dc96090f-e855-4d3e-854c-50ea700dc8ea", "address": "fa:16:3e:9e:2b:8a", "network": {"id": "ea3baf6d-ba72-4615-bc0d-604e4fcd8e4e", "bridge": "br-int", "label": "tempest-TaggedBootDevicesTest_v242-1704264625-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "325013d2bedc498e91c2145c2b398803", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapdc96090f-e8", "ovs_interfaceid": "dc96090f-e855-4d3e-854c-50ea700dc8ea", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "2027f7a1-36e4-4434-a063-61e817758917", "address": "fa:16:3e:4d:66:94", "network": {"id": "60b448f6-90dc-4dac-b24e-9370262e6dac", "bridge": "br-int", "label": "tempest-device-tagging-net1-431494419", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.80", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "325013d2bedc498e91c2145c2b398803", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2027f7a1-36", "ovs_interfaceid": "2027f7a1-36e4-4434-a063-61e817758917", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}}, {"id": "de4ad363-fae0-4d1b-9820-5a91a27d630d", "address": "fa:16:3e:94:49:94", "network": {"id": "60b448f6-90dc-4dac-b24e-9370262e6dac", "bridge": "br-int", "label": "tempest-device-tagging-net1-431494419", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.218", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "325013d2bedc498e91c2145c2b398803", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapde4ad363-fa", "ovs_interfaceid": "de4ad363-fae0-4d1b-9820-5a91a27d630d", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}}, {"id": "d520a88d-ec02-45b8-adbc-484d38b9c18c", "address": "fa:16:3e:ac:cd:aa", "network": {"id": "60b448f6-90dc-4dac-b24e-9370262e6dac", "bridge": "br-int", "label": "tempest-device-tagging-net1-431494419", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.226", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "325013d2bedc498e91c2145c2b398803", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapd520a88d-ec", "ovs_interfaceid": "d520a88d-ec02-45b8-adbc-484d38b9c18c", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "c50f64cd-b7fa-4580-bb78-bff756a0773a", "address": "fa:16:3e:36:0b:d9", "network": {"id": "60b448f6-90dc-4dac-b24e-9370262e6dac", "bridge": "br-int", "label": "tempest-device-tagging-net1-431494419", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.18", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "325013d2bedc498e91c2145c2b398803", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc50f64cd-b7", "ovs_interfaceid": "c50f64cd-b7fa-4580-bb78-bff756a0773a", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "ef8ba969-6186-4b9a-b30f-ec9a70036bab", "address": "fa:16:3e:0f:ed:74", "network": {"id": "31e25520-c84d-451c-9132-ddc8c1e443a6", "bridge": "br-int", "label": "tempest-device-tagging-net2-653330954", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.100", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "325013d2bedc498e91c2145c2b398803", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapef8ba969-61", "ovs_interfaceid": "ef8ba969-6186-4b9a-b30f-ec9a70036bab", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "92917a0d-59ee-4167-8693-262a21f6e156", "address": "fa:16:3e:65:95:01", "network": {"id": "31e25520-c84d-451c-9132-ddc8c1e443a6", "bridge": "br-int", "label": "tempest-device-tagging-net2-653330954", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.200", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "325013d2bedc498e91c2145c2b398803", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap92917a0d-59", "ovs_interfaceid": "92917a0d-59ee-4167-8693-262a21f6e156", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:07:23 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-1723ca6d-b764-4095-9908-763c710d897f req-f26ad7a9-3a9a-44db-a977-3aafc50355ab service nova] Releasing lock "refresh_cache-763a3e14-4563-4dc2-99ae-c340edbb93df" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:07:24 user nova-compute[75234]: DEBUG nova.compute.manager [req-d00e67d9-9a55-4bd9-bd40-ca233eea0874 req-5cfb196b-c343-4d4a-a2ac-905ba53f1329 service nova] [instance: f7d0bdeb-6a05-47ac-87f2-c8380a9a4286] Received event network-changed-b5192c04-0323-48b8-a326-ddf6b21f9019 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:07:24 user nova-compute[75234]: DEBUG nova.compute.manager [req-d00e67d9-9a55-4bd9-bd40-ca233eea0874 req-5cfb196b-c343-4d4a-a2ac-905ba53f1329 service nova] [instance: f7d0bdeb-6a05-47ac-87f2-c8380a9a4286] Refreshing instance network info cache due to event network-changed-b5192c04-0323-48b8-a326-ddf6b21f9019. {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11117}}
Nov 29 03:07:24 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-d00e67d9-9a55-4bd9-bd40-ca233eea0874 req-5cfb196b-c343-4d4a-a2ac-905ba53f1329 service nova] Acquiring lock "refresh_cache-f7d0bdeb-6a05-47ac-87f2-c8380a9a4286" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:07:24 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-d00e67d9-9a55-4bd9-bd40-ca233eea0874 req-5cfb196b-c343-4d4a-a2ac-905ba53f1329 service nova] Acquired lock "refresh_cache-f7d0bdeb-6a05-47ac-87f2-c8380a9a4286" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:07:24 user nova-compute[75234]: DEBUG nova.network.neutron [req-d00e67d9-9a55-4bd9-bd40-ca233eea0874 req-5cfb196b-c343-4d4a-a2ac-905ba53f1329 service nova] [instance: f7d0bdeb-6a05-47ac-87f2-c8380a9a4286] Refreshing network info cache for port b5192c04-0323-48b8-a326-ddf6b21f9019 {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2006}}
Nov 29 03:07:24 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:24 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:24 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:24 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:24 user nova-compute[75234]: DEBUG nova.network.neutron [req-d00e67d9-9a55-4bd9-bd40-ca233eea0874 req-5cfb196b-c343-4d4a-a2ac-905ba53f1329 service nova] [instance: f7d0bdeb-6a05-47ac-87f2-c8380a9a4286] Updated VIF entry in instance network info cache for port b5192c04-0323-48b8-a326-ddf6b21f9019. {{(pid=75234) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3481}}
Nov 29 03:07:24 user nova-compute[75234]: DEBUG nova.network.neutron [req-d00e67d9-9a55-4bd9-bd40-ca233eea0874 req-5cfb196b-c343-4d4a-a2ac-905ba53f1329 service nova] [instance: f7d0bdeb-6a05-47ac-87f2-c8380a9a4286] Updating instance_info_cache with network_info: [{"id": "b5192c04-0323-48b8-a326-ddf6b21f9019", "address": "fa:16:3e:59:dd:b5", "network": {"id": "22b1262f-108b-45c1-91cb-a0f77f9bfccf", "bridge": "br-int", "label": "tempest-TestServerVolumeAttachmentScenario-1154356954-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "b793707804a24961968ac432a37d9581", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapb5192c04-03", "ovs_interfaceid": "b5192c04-0323-48b8-a326-ddf6b21f9019", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:07:24 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-d00e67d9-9a55-4bd9-bd40-ca233eea0874 req-5cfb196b-c343-4d4a-a2ac-905ba53f1329 service nova] Releasing lock "refresh_cache-f7d0bdeb-6a05-47ac-87f2-c8380a9a4286" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:07:25 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:25 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:26 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:26 user nova-compute[75234]: DEBUG nova.compute.manager [req-e83e05f6-dbe7-46fc-9baf-2f380cc14897 req-e8ec221f-f9aa-4f45-912b-0541190810be service nova] [instance: f7d0bdeb-6a05-47ac-87f2-c8380a9a4286] Received event network-vif-plugged-b5192c04-0323-48b8-a326-ddf6b21f9019 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:07:26 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-e83e05f6-dbe7-46fc-9baf-2f380cc14897 req-e8ec221f-f9aa-4f45-912b-0541190810be service nova] Acquiring lock "f7d0bdeb-6a05-47ac-87f2-c8380a9a4286-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:07:26 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-e83e05f6-dbe7-46fc-9baf-2f380cc14897 req-e8ec221f-f9aa-4f45-912b-0541190810be service nova] Lock "f7d0bdeb-6a05-47ac-87f2-c8380a9a4286-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:07:26 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-e83e05f6-dbe7-46fc-9baf-2f380cc14897 req-e8ec221f-f9aa-4f45-912b-0541190810be service nova] Lock "f7d0bdeb-6a05-47ac-87f2-c8380a9a4286-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:07:26 user nova-compute[75234]: DEBUG nova.compute.manager [req-e83e05f6-dbe7-46fc-9baf-2f380cc14897 req-e8ec221f-f9aa-4f45-912b-0541190810be service nova] [instance: f7d0bdeb-6a05-47ac-87f2-c8380a9a4286] No waiting events found dispatching network-vif-plugged-b5192c04-0323-48b8-a326-ddf6b21f9019 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:07:26 user nova-compute[75234]: WARNING nova.compute.manager [req-e83e05f6-dbe7-46fc-9baf-2f380cc14897 req-e8ec221f-f9aa-4f45-912b-0541190810be service nova] [instance: f7d0bdeb-6a05-47ac-87f2-c8380a9a4286] Received unexpected event network-vif-plugged-b5192c04-0323-48b8-a326-ddf6b21f9019 for instance with vm_state building and task_state spawning.
Nov 29 03:07:26 user nova-compute[75234]: DEBUG nova.compute.manager [req-e83e05f6-dbe7-46fc-9baf-2f380cc14897 req-e8ec221f-f9aa-4f45-912b-0541190810be service nova] [instance: f7d0bdeb-6a05-47ac-87f2-c8380a9a4286] Received event network-vif-plugged-b5192c04-0323-48b8-a326-ddf6b21f9019 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:07:26 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-e83e05f6-dbe7-46fc-9baf-2f380cc14897 req-e8ec221f-f9aa-4f45-912b-0541190810be service nova] Acquiring lock "f7d0bdeb-6a05-47ac-87f2-c8380a9a4286-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:07:26 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-e83e05f6-dbe7-46fc-9baf-2f380cc14897 req-e8ec221f-f9aa-4f45-912b-0541190810be service nova] Lock "f7d0bdeb-6a05-47ac-87f2-c8380a9a4286-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:07:26 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-e83e05f6-dbe7-46fc-9baf-2f380cc14897 req-e8ec221f-f9aa-4f45-912b-0541190810be service nova] Lock "f7d0bdeb-6a05-47ac-87f2-c8380a9a4286-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:07:26 user nova-compute[75234]: DEBUG nova.compute.manager [req-e83e05f6-dbe7-46fc-9baf-2f380cc14897 req-e8ec221f-f9aa-4f45-912b-0541190810be service nova] [instance: f7d0bdeb-6a05-47ac-87f2-c8380a9a4286] No waiting events found dispatching network-vif-plugged-b5192c04-0323-48b8-a326-ddf6b21f9019 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:07:26 user nova-compute[75234]: WARNING nova.compute.manager [req-e83e05f6-dbe7-46fc-9baf-2f380cc14897 req-e8ec221f-f9aa-4f45-912b-0541190810be service nova] [instance: f7d0bdeb-6a05-47ac-87f2-c8380a9a4286] Received unexpected event network-vif-plugged-b5192c04-0323-48b8-a326-ddf6b21f9019 for instance with vm_state building and task_state spawning.
Nov 29 03:07:26 user nova-compute[75234]: DEBUG nova.compute.manager [None req-78ea8aa1-48fd-4c8d-b0d1-dcdfe7cfafaf tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] [instance: f7d0bdeb-6a05-47ac-87f2-c8380a9a4286] Instance event wait completed in 0 seconds for  {{(pid=75234) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Nov 29 03:07:26 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-78ea8aa1-48fd-4c8d-b0d1-dcdfe7cfafaf tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] [instance: f7d0bdeb-6a05-47ac-87f2-c8380a9a4286] Guest created on hypervisor {{(pid=75234) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4421}}
Nov 29 03:07:26 user nova-compute[75234]: INFO nova.virt.libvirt.driver [-] [instance: f7d0bdeb-6a05-47ac-87f2-c8380a9a4286] Instance spawned successfully.
Nov 29 03:07:26 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-78ea8aa1-48fd-4c8d-b0d1-dcdfe7cfafaf tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] [instance: f7d0bdeb-6a05-47ac-87f2-c8380a9a4286] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:904}}
Nov 29 03:07:26 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-78ea8aa1-48fd-4c8d-b0d1-dcdfe7cfafaf tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] [instance: f7d0bdeb-6a05-47ac-87f2-c8380a9a4286] Found default for hw_cdrom_bus of ide {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:07:26 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-78ea8aa1-48fd-4c8d-b0d1-dcdfe7cfafaf tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] [instance: f7d0bdeb-6a05-47ac-87f2-c8380a9a4286] Found default for hw_disk_bus of virtio {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:07:26 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-78ea8aa1-48fd-4c8d-b0d1-dcdfe7cfafaf tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] [instance: f7d0bdeb-6a05-47ac-87f2-c8380a9a4286] Found default for hw_input_bus of None {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:07:26 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-78ea8aa1-48fd-4c8d-b0d1-dcdfe7cfafaf tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] [instance: f7d0bdeb-6a05-47ac-87f2-c8380a9a4286] Found default for hw_pointer_model of None {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:07:26 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-78ea8aa1-48fd-4c8d-b0d1-dcdfe7cfafaf tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] [instance: f7d0bdeb-6a05-47ac-87f2-c8380a9a4286] Found default for hw_video_model of virtio {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:07:26 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-78ea8aa1-48fd-4c8d-b0d1-dcdfe7cfafaf tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] [instance: f7d0bdeb-6a05-47ac-87f2-c8380a9a4286] Found default for hw_vif_model of virtio {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:07:26 user nova-compute[75234]: INFO nova.compute.manager [None req-78ea8aa1-48fd-4c8d-b0d1-dcdfe7cfafaf tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] [instance: f7d0bdeb-6a05-47ac-87f2-c8380a9a4286] Took 5.49 seconds to spawn the instance on the hypervisor.
Nov 29 03:07:26 user nova-compute[75234]: DEBUG nova.compute.manager [None req-78ea8aa1-48fd-4c8d-b0d1-dcdfe7cfafaf tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] [instance: f7d0bdeb-6a05-47ac-87f2-c8380a9a4286] Checking state {{(pid=75234) _get_power_state /opt/stack/nova/nova/compute/manager.py:1782}}
Nov 29 03:07:26 user nova-compute[75234]: INFO nova.compute.manager [None req-78ea8aa1-48fd-4c8d-b0d1-dcdfe7cfafaf tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] [instance: f7d0bdeb-6a05-47ac-87f2-c8380a9a4286] Took 6.28 seconds to build instance.
Nov 29 03:07:26 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-78ea8aa1-48fd-4c8d-b0d1-dcdfe7cfafaf tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] Lock "f7d0bdeb-6a05-47ac-87f2-c8380a9a4286" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 6.389s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:07:27 user nova-compute[75234]: INFO nova.virt.block_device [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] [instance: 763a3e14-4563-4dc2-99ae-c340edbb93df] Booting with volume 5fcd66fc-a479-4613-b1f8-e6ef12ca470a at /dev/vdc
Nov 29 03:07:27 user nova-compute[75234]: DEBUG os_brick.utils [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:176}}
Nov 29 03:07:27 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:07:27 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.014s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:07:27 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[28efd0ed-5e29-40f5-98e6-cfc360ccf1d7]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:dd1368c5ba2c\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:07:27 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt -v / -n -o SOURCE {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:07:27 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt -v / -n -o SOURCE" returned: 0 in 0.015s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:07:27 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[9e69635e-cc1f-4155-b93a-4fd0be333ba8]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:07:27 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:07:27 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.019s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:07:27 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[385bc6cc-73e5-4855-b6d4-4b1100eea20a]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:07:27 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[750deab6-7aae-42b1-b8b0-4a517f17c97d]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:07:27 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] Running cmd (subprocess): nvme version {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:07:27 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] 'nvme version' failed. Not Retrying. {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:473}}
Nov 29 03:07:27 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.nvmeof [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] nvme not present on system {{(pid=75234) nvme_present /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/nvmeof.py:756}}
Nov 29 03:07:27 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:07:27 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:473}}
Nov 29 03:07:27 user nova-compute[75234]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Nov 29 03:07:27 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[c9dc6041-3652-4f3c-adbf-c67d2c732716]: (4, '') {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:07:27 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.lightos [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=75234) find_dsc /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/lightos.py:98}}
Nov 29 03:07:27 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.lightos [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=75234) get_connector_properties /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/lightos.py:76}}
Nov 29 03:07:27 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.lightos [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] LIGHTOS: no hostnqn found. {{(pid=75234) get_connector_properties /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/lightos.py:84}}
Nov 29 03:07:27 user nova-compute[75234]: DEBUG os_brick.utils [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] <== get_connector_properties: return (116ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:dd1368c5ba2c', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:203}}
Nov 29 03:07:27 user nova-compute[75234]: DEBUG nova.virt.block_device [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] [instance: 763a3e14-4563-4dc2-99ae-c340edbb93df] Updating existing volume attachment record: c95e0a76-151c-4590-b598-dbe54926b619 {{(pid=75234) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Nov 29 03:07:27 user nova-compute[75234]: DEBUG nova.compute.manager [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] [instance: cb569e8d-e4a5-4076-bb8b-deeefad033e2] Start spawning the instance on the hypervisor. {{(pid=75234) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2628}}
Nov 29 03:07:27 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] [instance: cb569e8d-e4a5-4076-bb8b-deeefad033e2] Creating instance directory {{(pid=75234) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4727}}
Nov 29 03:07:27 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] [instance: cb569e8d-e4a5-4076-bb8b-deeefad033e2] Creating image(s)
Nov 29 03:07:27 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] [instance: cb569e8d-e4a5-4076-bb8b-deeefad033e2] Did not create local disks {{(pid=75234) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4863}}
Nov 29 03:07:27 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] [instance: cb569e8d-e4a5-4076-bb8b-deeefad033e2] Ensure instance console log exists: /opt/stack/data/nova/instances/cb569e8d-e4a5-4076-bb8b-deeefad033e2/console.log {{(pid=75234) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4613}}
Nov 29 03:07:27 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:07:27 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:07:27 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:07:27 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] [instance: cb569e8d-e4a5-4076-bb8b-deeefad033e2] Start _get_guest_xml network_info=[{"id": "2b12ec96-26fa-4974-8a7a-22e85ec20544", "address": "fa:16:3e:fd:00:a6", "network": {"id": "1869abec-0b45-446a-9dc8-5871de9dbfc5", "bridge": "br-int", "label": "tempest-TaggedBootDevicesTest-1812676341-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "ee1117bd3511400f9f45cc8f9bb9247d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2b12ec96-26", "ovs_interfaceid": "2b12ec96-26fa-4974-8a7a-22e85ec20544", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "0bd5d735-f566-4a6d-b49f-b24fd403a87a", "address": "fa:16:3e:aa:c2:ae", "network": {"id": "005139f3-3686-493a-aa8f-0cdc455befa0", "bridge": "br-int", "label": "tempest-device-tagging-net1-1510949192", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.85", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "ee1117bd3511400f9f45cc8f9bb9247d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap0bd5d735-f5", "ovs_interfaceid": "0bd5d735-f566-4a6d-b49f-b24fd403a87a", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}}, {"id": "27e72461-f85b-42c9-8824-0d760b82a725", "address": "fa:16:3e:8e:29:fd", "network": {"id": "005139f3-3686-493a-aa8f-0cdc455befa0", "bridge": "br-int", "label": "tempest-device-tagging-net1-1510949192", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.127", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "ee1117bd3511400f9f45cc8f9bb9247d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap27e72461-f8", "ovs_interfaceid": "27e72461-f85b-42c9-8824-0d760b82a725", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}}, {"id": "026f606a-534c-4cdd-992a-9f33268b374f", "address": "fa:16:3e:3e:3b:29", "network": {"id": "005139f3-3686-493a-aa8f-0cdc455befa0", "bridge": "br-int", "label": "tempest-device-tagging-net1-1510949192", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.41", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "ee1117bd3511400f9f45cc8f9bb9247d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap026f606a-53", "ovs_interfaceid": "026f606a-534c-4cdd-992a-9f33268b374f", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "67446920-3741-4542-84c6-493388395f26", "address": "fa:16:3e:f6:7a:76", "network": {"id": "005139f3-3686-493a-aa8f-0cdc455befa0", "bridge": "br-int", "label": "tempest-device-tagging-net1-1510949192", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.230", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "ee1117bd3511400f9f45cc8f9bb9247d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap67446920-37", "ovs_interfaceid": "67446920-3741-4542-84c6-493388395f26", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "d7e92753-492b-4fc0-8048-f4792cd3b4b6", "address": "fa:16:3e:68:b2:33", "network": {"id": "cc6c7cdf-8c81-4906-9fd8-de815fea3374", "bridge": "br-int", "label": "tempest-device-tagging-net2-1248280764", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.100", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "ee1117bd3511400f9f45cc8f9bb9247d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapd7e92753-49", "ovs_interfaceid": "d7e92753-492b-4fc0-8048-f4792cd3b4b6", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "06db3012-0042-457a-a3c8-14e3a8d2e89e", "address": "fa:16:3e:27:54:30", "network": {"id": "cc6c7cdf-8c81-4906-9fd8-de815fea3374", "bridge": "br-int", "label": "tempest-device-tagging-net2-1248280764", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.200", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "ee1117bd3511400f9f45cc8f9bb9247d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap06db3012-00", "ovs_interfaceid": "06db3012-0042-457a-a3c8-14e3a8d2e89e", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, '/dev/vda': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, '/dev/vdb': {'bus': 'virtio', 'dev': 'vdb', 'type': 'disk', 'boot_index': '2'}, '/dev/vdc': {'bus': 'virtio', 'dev': 'vdc', 'type': 'disk', 'boot_index': '3'}, 'disk.config': {'bus': 'ide', 'dev': 'hda', 'type': 'cdrom'}}} image_meta=ImageMeta(checksum='c8fc807773e5354afe61636071771906',container_format='bare',created_at=2023-11-29T03:02:15Z,direct_url=<?>,disk_format='qcow2',id=96c30ba7-9b73-4a30-8ba4-7138879573b7,min_disk=0,min_ram=0,name='cirros-0.6.2-x86_64-disk',owner='b18e380921c748b78517b6f1211cd0f1',properties=ImageMetaProps,protected=<?>,size=21430272,status='active',tags=<?>,updated_at=2023-11-29T03:02:17Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [], 'ephemerals': [], 'block_device_mapping': [{'mount_device': '/dev/vda', 'attachment_id': '62d5b7f3-07b7-4276-8480-7caf07b59a10', 'disk_bus': 'virtio', 'device_type': 'disk', 'boot_index': 0, 'guest_format': None, 'delete_on_termination': False, 'connection_info': {'driver_volume_type': 'iscsi', 'data': {'target_iqn': 'iqn.2010-10.org.openstack:bb98586e-4f75-46be-a626-0b73ebda11eb', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'hngCANBF', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}, 'status': 'reserved', 'instance': 'cb569e8d-e4a5-4076-bb8b-deeefad033e2', 'attached_at': '', 'detached_at': '', 'volume_id': 'bb98586e-4f75-46be-a626-0b73ebda11eb', 'serial': 'bb98586e-4f75-46be-a626-0b73ebda11eb'}, 'volume_type': None}, {'mount_device': '/dev/vdb', 'attachment_id': '03b0ad93-0fd2-469c-9d43-4f1c9e5ddcf5', 'disk_bus': 'virtio', 'device_type': 'disk', 'boot_index': 1, 'guest_format': None, 'delete_on_termination': False, 'connection_info': {'driver_volume_type': 'iscsi', 'data': {'target_iqn': 'iqn.2010-10.org.openstack:ab278679-6cfd-4c7c-a930-c078843640d3', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'JhsSAlap', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}, 'status': 'reserved', 'instance': 'cb569e8d-e4a5-4076-bb8b-deeefad033e2', 'attached_at': '', 'detached_at': '', 'volume_id': 'ab278679-6cfd-4c7c-a930-c078843640d3', 'serial': 'ab278679-6cfd-4c7c-a930-c078843640d3'}, 'volume_type': None}, {'mount_device': '/dev/vdc', 'attachment_id': '853e8d4a-de7a-4184-a592-0855eb8020d7', 'disk_bus': 'virtio', 'device_type': 'disk', 'boot_index': 2, 'guest_format': None, 'delete_on_termination': False, 'connection_info': {'driver_volume_type': 'iscsi', 'data': {'target_iqn': 'iqn.2010-10.org.openstack:920d6cad-25ab-4f3d-941b-8ec841d43184', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'fynrLYlU', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}, 'status': 'reserved', 'instance': 'cb569e8d-e4a5-4076-bb8b-deeefad033e2', 'attached_at': '', 'detached_at': '', 'volume_id': '920d6cad-25ab-4f3d-941b-8ec841d43184', 'serial': '920d6cad-25ab-4f3d-941b-8ec841d43184'}, 'volume_type': None} {{(pid=75234) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7562}}
Nov 29 03:07:27 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:07:27 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:07:27 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] Searching host: 'user' for CPU controller through CGroups V1... {{(pid=75234) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1652}}
Nov 29 03:07:27 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] CPU controller missing on host. {{(pid=75234) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1662}}
Nov 29 03:07:27 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] Searching host: 'user' for CPU controller through CGroups V2... {{(pid=75234) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1671}}
Nov 29 03:07:27 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] CPU controller found on host. {{(pid=75234) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1678}}
Nov 29 03:07:27 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=75234) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5400}}
Nov 29 03:07:27 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-11-29T03:02:04Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='1',id=6,is_public=True,memory_mb=512,name='m1.tiny',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='c8fc807773e5354afe61636071771906',container_format='bare',created_at=2023-11-29T03:02:15Z,direct_url=<?>,disk_format='qcow2',id=96c30ba7-9b73-4a30-8ba4-7138879573b7,min_disk=0,min_ram=0,name='cirros-0.6.2-x86_64-disk',owner='b18e380921c748b78517b6f1211cd0f1',properties=ImageMetaProps,protected=<?>,size=21430272,status='active',tags=<?>,updated_at=2023-11-29T03:02:17Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:563}}
Nov 29 03:07:27 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] Flavor limits 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:348}}
Nov 29 03:07:27 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] Image limits 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:352}}
Nov 29 03:07:27 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] Flavor pref 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:388}}
Nov 29 03:07:27 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] Image pref 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:392}}
Nov 29 03:07:27 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:430}}
Nov 29 03:07:27 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:569}}
Nov 29 03:07:27 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=75234) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:471}}
Nov 29 03:07:27 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] Got 1 possible topologies {{(pid=75234) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:501}}
Nov 29 03:07:27 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:575}}
Nov 29 03:07:27 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:577}}
Nov 29 03:07:27 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] Acquiring lock "cache_volume_driver" by "nova.virt.libvirt.driver.LibvirtDriver._get_volume_driver.<locals>._cache_volume_driver" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:07:27 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] Lock "cache_volume_driver" acquired by "nova.virt.libvirt.driver.LibvirtDriver._get_volume_driver.<locals>._cache_volume_driver" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:07:27 user nova-compute[75234]: DEBUG os_brick.initiator.connector [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] Factory for ISCSI on None {{(pid=75234) factory /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connector.py:281}}
Nov 29 03:07:27 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] Lock "cache_volume_driver" "released" by "nova.virt.libvirt.driver.LibvirtDriver._get_volume_driver.<locals>._cache_volume_driver" :: held 0.006s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:07:27 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] [instance: cb569e8d-e4a5-4076-bb8b-deeefad033e2] Calling os-brick to attach iSCSI Volume {{(pid=75234) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Nov 29 03:07:27 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f8a5050c9a0>, 'connection_properties': {'target_iqn': 'iqn.2010-10.org.openstack:bb98586e-4f75-46be-a626-0b73ebda11eb', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'hngCANBF', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:176}}
Nov 29 03:07:27 user nova-compute[75234]: WARNING os_brick.initiator.connectors.base [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Nov 29 03:07:27 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:68}}
Nov 29 03:07:27 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.002s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:73}}
Nov 29 03:07:27 user nova-compute[75234]: WARNING os_brick.initiator.connectors.base [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Nov 29 03:07:27 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:bb98586e-4f75-46be-a626-0b73ebda11eb" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:68}}
Nov 29 03:07:27 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:bb98586e-4f75-46be-a626-0b73ebda11eb" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.002s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:73}}
Nov 29 03:07:27 user nova-compute[75234]: INFO os_brick.initiator.connectors.iscsi [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Nov 29 03:07:27 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:bb98586e-4f75-46be-a626-0b73ebda11eb -p 172.16.0.220:3260 {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:07:27 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:bb98586e-4f75-46be-a626-0b73ebda11eb -p 172.16.0.220:3260" returned: 21 in 0.021s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:07:27 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[ab74c661-02a2-4296-8287-e74264713d64]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:07:27 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Nov 29 03:07:27 user nova-compute[75234]:  {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:07:27 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:bb98586e-4f75-46be-a626-0b73ebda11eb -p 172.16.0.220:3260 --interface default --op new {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:07:27 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:bb98586e-4f75-46be-a626-0b73ebda11eb -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.018s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:07:27 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[8798a6a9-7361-4e3e-b8e1-312648ca5299]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:bb98586e-4f75-46be-a626-0b73ebda11eb] added\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:07:27 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:bb98586e-4f75-46be-a626-0b73ebda11eb] added
Nov 29 03:07:27 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:07:27 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:bb98586e-4f75-46be-a626-0b73ebda11eb -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:07:27 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:bb98586e-4f75-46be-a626-0b73ebda11eb -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.058s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:07:27 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[290e0486-7bc1-44d3-a84b-33cdee65dabf]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:07:27 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:07:27 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:bb98586e-4f75-46be-a626-0b73ebda11eb -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:07:27 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:bb98586e-4f75-46be-a626-0b73ebda11eb -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.019s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:07:27 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[0fc341ca-a620-4e35-96e2-fe52d9fe941f]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:07:27 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:07:27 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:bb98586e-4f75-46be-a626-0b73ebda11eb -p 172.16.0.220:3260 --op update -n node.session.auth.username -v hngCANBF {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:07:27 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:bb98586e-4f75-46be-a626-0b73ebda11eb -p 172.16.0.220:3260 --op update -n node.session.auth.username -v hngCANBF" returned: 0 in 0.016s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:07:27 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[fc82725c-3cb6-4594-9461-c13821b13f12]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:07:27 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'hngCANBF'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:07:27 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:bb98586e-4f75-46be-a626-0b73ebda11eb -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:07:27 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:bb98586e-4f75-46be-a626-0b73ebda11eb -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.022s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:07:27 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[c79b4ccb-14fa-47e1-a15a-2cabadf82d72]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:07:27 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:07:27 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:07:27 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 21 in 0.015s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:07:27 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[2b3dfa00-ade0-49c0-82fa-041e54a09222]: (4, ('', 'iscsiadm: No active sessions.\n')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:07:27 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] iscsiadm ('-m', 'session'): stdout= stderr=iscsiadm: No active sessions.
Nov 29 03:07:27 user nova-compute[75234]:  {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:07:27 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] iscsi session list stdout= stderr=iscsiadm: No active sessions.
Nov 29 03:07:27 user nova-compute[75234]:  {{(pid=75234) _run_iscsi_session /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Nov 29 03:07:27 user nova-compute[75234]: WARNING os_brick.initiator.connectors.iscsi [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] iscsiadm stderr output when getting sessions: iscsiadm: No active sessions.
Nov 29 03:07:27 user nova-compute[75234]: 
Nov 29 03:07:27 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:bb98586e-4f75-46be-a626-0b73ebda11eb -p 172.16.0.220:3260 --login {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:07:27 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:bb98586e-4f75-46be-a626-0b73ebda11eb -p 172.16.0.220:3260 --login" returned: 0 in 0.049s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:07:27 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[783dad0a-3a5a-49bb-9e89-50e50b9559d6]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:bb98586e-4f75-46be-a626-0b73ebda11eb, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:bb98586e-4f75-46be-a626-0b73ebda11eb, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:07:27 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:bb98586e-4f75-46be-a626-0b73ebda11eb, portal: 172.16.0.220,3260]
Nov 29 03:07:27 user nova-compute[75234]: Login to [iface: default, target: iqn.2010-10.org.openstack:bb98586e-4f75-46be-a626-0b73ebda11eb, portal: 172.16.0.220,3260] successful.
Nov 29 03:07:27 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:07:27 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:bb98586e-4f75-46be-a626-0b73ebda11eb -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:07:27 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:bb98586e-4f75-46be-a626-0b73ebda11eb -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.019s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:07:27 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[4ea86fb8-492e-4a0f-a56f-8f6cb53f47e3]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:07:27 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:07:27 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:07:27 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.019s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:07:27 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[169f4e21-2818-4864-afcc-b354718d58b6]: (4, ('tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bb98586e-4f75-46be-a626-0b73ebda11eb (non-flash)\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:07:27 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bb98586e-4f75-46be-a626-0b73ebda11eb (non-flash)
Nov 29 03:07:27 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:07:27 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] iscsi session list stdout=tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bb98586e-4f75-46be-a626-0b73ebda11eb (non-flash)
Nov 29 03:07:27 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsi_session /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Nov 29 03:07:27 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:bb98586e-4f75-46be-a626-0b73ebda11eb" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.303s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:87}}
Nov 29 03:07:27 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] Connected to 172.16.0.220:3260 {{(pid=75234) _connect_vol /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:644}}
Nov 29 03:07:27 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] HCTL ('33', '-', '-', 0) found on session 2 with lun 0 {{(pid=75234) get_hctl /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:770}}
Nov 29 03:07:27 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] Scanning host 33 c: -, t: -, l: 0) {{(pid=75234) scan_iscsi /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:795}}
Nov 29 03:07:27 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host33/scan {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:07:27 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host33/scan" returned: 0 in 0.027s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:07:27 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[77b16741-37db-4b90-bed8-32056a7eabbe]: (4, ('- - 0', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:07:27 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] Searching for a device in session 2 and hctl ['33', '*', '*', 0] yield: sdb {{(pid=75234) device_name_by_hctl /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:789}}
Nov 29 03:07:27 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] Connected to sdb using {'target_iqn': 'iqn.2010-10.org.openstack:bb98586e-4f75-46be-a626-0b73ebda11eb', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'hngCANBF', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False} {{(pid=75234) _connect_vol /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:672}}
Nov 29 03:07:28 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:28 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.353s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:87}}
Nov 29 03:07:28 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] <== connect_volume: return (1355ms) {'type': 'block', 'scsi_wwn': '26563353832343139', 'path': '/dev/sdb'} {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:203}}
Nov 29 03:07:28 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] [instance: cb569e8d-e4a5-4076-bb8b-deeefad033e2] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '26563353832343139', 'path': '/dev/sdb'} {{(pid=75234) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Nov 29 03:07:28 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] [instance: cb569e8d-e4a5-4076-bb8b-deeefad033e2] Calling os-brick to attach iSCSI Volume {{(pid=75234) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Nov 29 03:07:28 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f8a5050c9a0>, 'connection_properties': {'target_iqn': 'iqn.2010-10.org.openstack:ab278679-6cfd-4c7c-a930-c078843640d3', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'JhsSAlap', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:176}}
Nov 29 03:07:28 user nova-compute[75234]: WARNING os_brick.initiator.connectors.base [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Nov 29 03:07:28 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:68}}
Nov 29 03:07:28 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:73}}
Nov 29 03:07:28 user nova-compute[75234]: WARNING os_brick.initiator.connectors.base [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Nov 29 03:07:28 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:ab278679-6cfd-4c7c-a930-c078843640d3" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:68}}
Nov 29 03:07:28 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:ab278679-6cfd-4c7c-a930-c078843640d3" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:73}}
Nov 29 03:07:28 user nova-compute[75234]: INFO os_brick.initiator.connectors.iscsi [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Nov 29 03:07:29 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:ab278679-6cfd-4c7c-a930-c078843640d3 -p 172.16.0.220:3260 {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:07:29 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:ab278679-6cfd-4c7c-a930-c078843640d3 -p 172.16.0.220:3260" returned: 21 in 0.016s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:07:29 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[f8f555d5-891a-4fd7-a87d-77593930ae7d]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:07:29 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Nov 29 03:07:29 user nova-compute[75234]:  {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:07:29 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:ab278679-6cfd-4c7c-a930-c078843640d3 -p 172.16.0.220:3260 --interface default --op new {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:07:29 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:ab278679-6cfd-4c7c-a930-c078843640d3 -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.022s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:07:29 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[d38f8451-4f2b-4351-87b0-9d5fd0ec639e]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:ab278679-6cfd-4c7c-a930-c078843640d3] added\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:07:29 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:ab278679-6cfd-4c7c-a930-c078843640d3] added
Nov 29 03:07:29 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:07:29 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:ab278679-6cfd-4c7c-a930-c078843640d3 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:07:29 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:ab278679-6cfd-4c7c-a930-c078843640d3 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.022s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:07:29 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[ea169368-fbb9-4fd6-ac6d-07e96c54a334]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:07:29 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:07:29 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:ab278679-6cfd-4c7c-a930-c078843640d3 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:07:29 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:ab278679-6cfd-4c7c-a930-c078843640d3 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.019s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:07:29 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[323d7678-13a7-4fb4-b5e9-3b5c168f174b]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:07:29 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:07:29 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:ab278679-6cfd-4c7c-a930-c078843640d3 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v JhsSAlap {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:07:29 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:ab278679-6cfd-4c7c-a930-c078843640d3 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v JhsSAlap" returned: 0 in 0.018s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:07:29 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[941dbefd-9397-41bf-b431-769fcd41999c]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:07:29 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'JhsSAlap'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:07:29 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:ab278679-6cfd-4c7c-a930-c078843640d3 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:07:29 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:ab278679-6cfd-4c7c-a930-c078843640d3 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.017s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:07:29 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[33117ad2-21dc-4d60-a5c2-a602b2daf287]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:07:29 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:07:29 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:07:29 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.020s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:07:29 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[9c964987-6a2f-4928-b7f2-ce06e9e70286]: (4, ('tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bb98586e-4f75-46be-a626-0b73ebda11eb (non-flash)\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:07:29 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bb98586e-4f75-46be-a626-0b73ebda11eb (non-flash)
Nov 29 03:07:29 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:07:29 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] iscsi session list stdout=tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bb98586e-4f75-46be-a626-0b73ebda11eb (non-flash)
Nov 29 03:07:29 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsi_session /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Nov 29 03:07:29 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:ab278679-6cfd-4c7c-a930-c078843640d3 -p 172.16.0.220:3260 --login {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:07:29 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:ab278679-6cfd-4c7c-a930-c078843640d3 -p 172.16.0.220:3260 --login" returned: 0 in 0.055s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:07:29 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[ad5a4580-0eac-4684-b1cc-6e4aa69a2d4f]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:ab278679-6cfd-4c7c-a930-c078843640d3, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:ab278679-6cfd-4c7c-a930-c078843640d3, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:07:29 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:ab278679-6cfd-4c7c-a930-c078843640d3, portal: 172.16.0.220,3260]
Nov 29 03:07:29 user nova-compute[75234]: Login to [iface: default, target: iqn.2010-10.org.openstack:ab278679-6cfd-4c7c-a930-c078843640d3, portal: 172.16.0.220,3260] successful.
Nov 29 03:07:29 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:07:29 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:ab278679-6cfd-4c7c-a930-c078843640d3 -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:07:29 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:ab278679-6cfd-4c7c-a930-c078843640d3 -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.020s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:07:29 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[e340ef24-ed8e-4308-a549-3f3f9d6ea2be]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:07:29 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:07:29 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:07:29 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.029s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:07:29 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[83ea8479-9475-45a8-ae51-b6657f1b9616]: (4, ('tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bb98586e-4f75-46be-a626-0b73ebda11eb (non-flash)\ntcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ab278679-6cfd-4c7c-a930-c078843640d3 (non-flash)\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:07:29 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bb98586e-4f75-46be-a626-0b73ebda11eb (non-flash)
Nov 29 03:07:29 user nova-compute[75234]: tcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ab278679-6cfd-4c7c-a930-c078843640d3 (non-flash)
Nov 29 03:07:29 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:07:29 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] iscsi session list stdout=tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bb98586e-4f75-46be-a626-0b73ebda11eb (non-flash)
Nov 29 03:07:29 user nova-compute[75234]: tcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ab278679-6cfd-4c7c-a930-c078843640d3 (non-flash)
Nov 29 03:07:29 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsi_session /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Nov 29 03:07:29 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:ab278679-6cfd-4c7c-a930-c078843640d3" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.278s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:87}}
Nov 29 03:07:29 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] Connected to 172.16.0.220:3260 {{(pid=75234) _connect_vol /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:644}}
Nov 29 03:07:29 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] HCTL ('34', '-', '-', 0) found on session 3 with lun 0 {{(pid=75234) get_hctl /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:770}}
Nov 29 03:07:29 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] Scanning host 34 c: -, t: -, l: 0) {{(pid=75234) scan_iscsi /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:795}}
Nov 29 03:07:29 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host34/scan {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:07:29 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host34/scan" returned: 0 in 0.021s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:07:29 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[9867d1fe-a28b-48ee-8416-54c5e370ae82]: (4, ('- - 0', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:07:29 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] Searching for a device in session 3 and hctl ['34', '*', '*', 0] yield: sdc {{(pid=75234) device_name_by_hctl /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:789}}
Nov 29 03:07:29 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] Connected to sdc using {'target_iqn': 'iqn.2010-10.org.openstack:ab278679-6cfd-4c7c-a930-c078843640d3', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'JhsSAlap', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False} {{(pid=75234) _connect_vol /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:672}}
Nov 29 03:07:30 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.329s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:87}}
Nov 29 03:07:30 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] <== connect_volume: return (1331ms) {'type': 'block', 'scsi_wwn': '26666623136303566', 'path': '/dev/sdc'} {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:203}}
Nov 29 03:07:30 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] [instance: cb569e8d-e4a5-4076-bb8b-deeefad033e2] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '26666623136303566', 'path': '/dev/sdc'} {{(pid=75234) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Nov 29 03:07:30 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] [instance: cb569e8d-e4a5-4076-bb8b-deeefad033e2] Calling os-brick to attach iSCSI Volume {{(pid=75234) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Nov 29 03:07:30 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f8a5050c9a0>, 'connection_properties': {'target_iqn': 'iqn.2010-10.org.openstack:920d6cad-25ab-4f3d-941b-8ec841d43184', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'fynrLYlU', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:176}}
Nov 29 03:07:30 user nova-compute[75234]: WARNING os_brick.initiator.connectors.base [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Nov 29 03:07:30 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:68}}
Nov 29 03:07:30 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.002s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:73}}
Nov 29 03:07:30 user nova-compute[75234]: WARNING os_brick.initiator.connectors.base [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Nov 29 03:07:30 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:920d6cad-25ab-4f3d-941b-8ec841d43184" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:68}}
Nov 29 03:07:30 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:920d6cad-25ab-4f3d-941b-8ec841d43184" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.002s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:73}}
Nov 29 03:07:30 user nova-compute[75234]: INFO os_brick.initiator.connectors.iscsi [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Nov 29 03:07:30 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:920d6cad-25ab-4f3d-941b-8ec841d43184 -p 172.16.0.220:3260 {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:07:30 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:920d6cad-25ab-4f3d-941b-8ec841d43184 -p 172.16.0.220:3260" returned: 21 in 0.017s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:07:30 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[91d6bdeb-e73a-434f-8b27-f5fc5a3351d7]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:07:30 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Nov 29 03:07:30 user nova-compute[75234]:  {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:07:30 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:920d6cad-25ab-4f3d-941b-8ec841d43184 -p 172.16.0.220:3260 --interface default --op new {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:07:30 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:920d6cad-25ab-4f3d-941b-8ec841d43184 -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.017s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:07:30 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[29c3a60e-97c2-4c12-9743-bde936df0cba]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:920d6cad-25ab-4f3d-941b-8ec841d43184] added\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:07:30 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:920d6cad-25ab-4f3d-941b-8ec841d43184] added
Nov 29 03:07:30 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:07:30 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:920d6cad-25ab-4f3d-941b-8ec841d43184 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:07:30 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:920d6cad-25ab-4f3d-941b-8ec841d43184 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.018s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:07:30 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[019db42d-7043-4e3a-a124-3b18474208ed]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:07:30 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:07:30 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:920d6cad-25ab-4f3d-941b-8ec841d43184 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:07:30 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:920d6cad-25ab-4f3d-941b-8ec841d43184 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.017s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:07:30 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[805ef100-388a-4731-8510-0d9b9f89344b]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:07:30 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:07:30 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:920d6cad-25ab-4f3d-941b-8ec841d43184 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v fynrLYlU {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:07:30 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:920d6cad-25ab-4f3d-941b-8ec841d43184 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v fynrLYlU" returned: 0 in 0.018s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:07:30 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[1c3149bc-c8e1-4c02-8ec1-adfb59a26611]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:07:30 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'fynrLYlU'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:07:30 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:920d6cad-25ab-4f3d-941b-8ec841d43184 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:07:30 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:920d6cad-25ab-4f3d-941b-8ec841d43184 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.018s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:07:30 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[d18510bc-2945-4e16-b633-ea49360ac332]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:07:30 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:07:30 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:07:30 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.020s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:07:30 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[9b6f21e3-a3f9-4764-9e46-cbfd85481e09]: (4, ('tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bb98586e-4f75-46be-a626-0b73ebda11eb (non-flash)\ntcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ab278679-6cfd-4c7c-a930-c078843640d3 (non-flash)\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:07:30 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bb98586e-4f75-46be-a626-0b73ebda11eb (non-flash)
Nov 29 03:07:30 user nova-compute[75234]: tcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ab278679-6cfd-4c7c-a930-c078843640d3 (non-flash)
Nov 29 03:07:30 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:07:30 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] iscsi session list stdout=tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bb98586e-4f75-46be-a626-0b73ebda11eb (non-flash)
Nov 29 03:07:30 user nova-compute[75234]: tcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ab278679-6cfd-4c7c-a930-c078843640d3 (non-flash)
Nov 29 03:07:30 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsi_session /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Nov 29 03:07:30 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:920d6cad-25ab-4f3d-941b-8ec841d43184 -p 172.16.0.220:3260 --login {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:07:30 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:920d6cad-25ab-4f3d-941b-8ec841d43184 -p 172.16.0.220:3260 --login" returned: 0 in 0.060s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:07:30 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[8642ab89-351b-4032-aa1a-ccc3b9a1ab36]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:920d6cad-25ab-4f3d-941b-8ec841d43184, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:920d6cad-25ab-4f3d-941b-8ec841d43184, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:07:30 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:920d6cad-25ab-4f3d-941b-8ec841d43184, portal: 172.16.0.220,3260]
Nov 29 03:07:30 user nova-compute[75234]: Login to [iface: default, target: iqn.2010-10.org.openstack:920d6cad-25ab-4f3d-941b-8ec841d43184, portal: 172.16.0.220,3260] successful.
Nov 29 03:07:30 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:07:30 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:920d6cad-25ab-4f3d-941b-8ec841d43184 -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:07:30 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:920d6cad-25ab-4f3d-941b-8ec841d43184 -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.019s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:07:30 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[8b03fbb7-9aa0-49ea-b382-138a0842a8d5]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:07:30 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:07:30 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:07:30 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.021s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:07:30 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[93761a02-093f-44f4-8a43-3684102528a2]: (4, ('tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bb98586e-4f75-46be-a626-0b73ebda11eb (non-flash)\ntcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ab278679-6cfd-4c7c-a930-c078843640d3 (non-flash)\ntcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:920d6cad-25ab-4f3d-941b-8ec841d43184 (non-flash)\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:07:30 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bb98586e-4f75-46be-a626-0b73ebda11eb (non-flash)
Nov 29 03:07:30 user nova-compute[75234]: tcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ab278679-6cfd-4c7c-a930-c078843640d3 (non-flash)
Nov 29 03:07:30 user nova-compute[75234]: tcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:920d6cad-25ab-4f3d-941b-8ec841d43184 (non-flash)
Nov 29 03:07:30 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:07:30 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] iscsi session list stdout=tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bb98586e-4f75-46be-a626-0b73ebda11eb (non-flash)
Nov 29 03:07:30 user nova-compute[75234]: tcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ab278679-6cfd-4c7c-a930-c078843640d3 (non-flash)
Nov 29 03:07:30 user nova-compute[75234]: tcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:920d6cad-25ab-4f3d-941b-8ec841d43184 (non-flash)
Nov 29 03:07:30 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsi_session /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Nov 29 03:07:30 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:920d6cad-25ab-4f3d-941b-8ec841d43184" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.265s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:87}}
Nov 29 03:07:30 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] Connected to 172.16.0.220:3260 {{(pid=75234) _connect_vol /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:644}}
Nov 29 03:07:30 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] HCTL ('35', '-', '-', 0) found on session 4 with lun 0 {{(pid=75234) get_hctl /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:770}}
Nov 29 03:07:30 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] Scanning host 35 c: -, t: -, l: 0) {{(pid=75234) scan_iscsi /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:795}}
Nov 29 03:07:30 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host35/scan {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:07:30 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host35/scan" returned: 0 in 0.016s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:07:30 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[e4349d7c-51fa-4c3d-83bc-8c23189ef4dd]: (4, ('- - 0', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:07:30 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] Searching for a device in session 4 and hctl ['35', '*', '*', 0] yield: None {{(pid=75234) device_name_by_hctl /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:789}}
Nov 29 03:07:31 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:31 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] Searching for a device in session 4 and hctl ['35', '*', '*', 0] yield: sdd {{(pid=75234) device_name_by_hctl /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:789}}
Nov 29 03:07:31 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] Connected to sdd using {'target_iqn': 'iqn.2010-10.org.openstack:920d6cad-25ab-4f3d-941b-8ec841d43184', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'fynrLYlU', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False} {{(pid=75234) _connect_vol /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:672}}
Nov 29 03:07:31 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.294s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:87}}
Nov 29 03:07:31 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] <== connect_volume: return (1296ms) {'type': 'block', 'scsi_wwn': '23137333564356365', 'path': '/dev/sdd'} {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:203}}
Nov 29 03:07:31 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] [instance: cb569e8d-e4a5-4076-bb8b-deeefad033e2] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '23137333564356365', 'path': '/dev/sdd'} {{(pid=75234) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Nov 29 03:07:31 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='True',created_at=2023-11-29T03:06:58Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-1051091393',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-device-tagging-server-1051091393',id=10,image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBOxFHXB9/btSEa91bTsp1yfkF2UsLgRU7yInw9joeBa8Nj3E7aswz+RinNuyzQo33k8c3J3qTezyAtFU7X0/nxAqdQ5O2Sv0+97iAivTta7NtcdKSix5ZmkHgQqMMjKOGw==',key_name='tempest-keypair-1672591556',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='ee1117bd3511400f9f45cc8f9bb9247d',ramdisk_id='',reservation_id='r-936jf1h5',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TaggedBootDevicesTest-1858653514',owner_user_name='tempest-TaggedBootDevicesTest-1858653514-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-11-29T03:07:00Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='36c915d13a4943549549177c756da93b',uuid=cb569e8d-e4a5-4076-bb8b-deeefad033e2,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "2b12ec96-26fa-4974-8a7a-22e85ec20544", "address": "fa:16:3e:fd:00:a6", "network": {"id": "1869abec-0b45-446a-9dc8-5871de9dbfc5", "bridge": "br-int", "label": "tempest-TaggedBootDevicesTest-1812676341-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "ee1117bd3511400f9f45cc8f9bb9247d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2b12ec96-26", "ovs_interfaceid": "2b12ec96-26fa-4974-8a7a-22e85ec20544", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=75234) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Nov 29 03:07:31 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] Converting VIF {"id": "2b12ec96-26fa-4974-8a7a-22e85ec20544", "address": "fa:16:3e:fd:00:a6", "network": {"id": "1869abec-0b45-446a-9dc8-5871de9dbfc5", "bridge": "br-int", "label": "tempest-TaggedBootDevicesTest-1812676341-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "ee1117bd3511400f9f45cc8f9bb9247d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2b12ec96-26", "ovs_interfaceid": "2b12ec96-26fa-4974-8a7a-22e85ec20544", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:07:31 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:fd:00:a6,bridge_name='br-int',has_traffic_filtering=True,id=2b12ec96-26fa-4974-8a7a-22e85ec20544,network=Network(1869abec-0b45-446a-9dc8-5871de9dbfc5),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap2b12ec96-26') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:07:31 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='True',created_at=2023-11-29T03:06:58Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-1051091393',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-device-tagging-server-1051091393',id=10,image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBOxFHXB9/btSEa91bTsp1yfkF2UsLgRU7yInw9joeBa8Nj3E7aswz+RinNuyzQo33k8c3J3qTezyAtFU7X0/nxAqdQ5O2Sv0+97iAivTta7NtcdKSix5ZmkHgQqMMjKOGw==',key_name='tempest-keypair-1672591556',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='ee1117bd3511400f9f45cc8f9bb9247d',ramdisk_id='',reservation_id='r-936jf1h5',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TaggedBootDevicesTest-1858653514',owner_user_name='tempest-TaggedBootDevicesTest-1858653514-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-11-29T03:07:00Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='36c915d13a4943549549177c756da93b',uuid=cb569e8d-e4a5-4076-bb8b-deeefad033e2,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "0bd5d735-f566-4a6d-b49f-b24fd403a87a", "address": "fa:16:3e:aa:c2:ae", "network": {"id": "005139f3-3686-493a-aa8f-0cdc455befa0", "bridge": "br-int", "label": "tempest-device-tagging-net1-1510949192", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.85", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "ee1117bd3511400f9f45cc8f9bb9247d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap0bd5d735-f5", "ovs_interfaceid": "0bd5d735-f566-4a6d-b49f-b24fd403a87a", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=75234) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Nov 29 03:07:31 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] Converting VIF {"id": "0bd5d735-f566-4a6d-b49f-b24fd403a87a", "address": "fa:16:3e:aa:c2:ae", "network": {"id": "005139f3-3686-493a-aa8f-0cdc455befa0", "bridge": "br-int", "label": "tempest-device-tagging-net1-1510949192", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.85", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "ee1117bd3511400f9f45cc8f9bb9247d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap0bd5d735-f5", "ovs_interfaceid": "0bd5d735-f566-4a6d-b49f-b24fd403a87a", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:07:31 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:aa:c2:ae,bridge_name='br-int',has_traffic_filtering=True,id=0bd5d735-f566-4a6d-b49f-b24fd403a87a,network=Network(005139f3-3686-493a-aa8f-0cdc455befa0),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=True,vif_name='tap0bd5d735-f5') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:07:31 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='True',created_at=2023-11-29T03:06:58Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-1051091393',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-device-tagging-server-1051091393',id=10,image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBOxFHXB9/btSEa91bTsp1yfkF2UsLgRU7yInw9joeBa8Nj3E7aswz+RinNuyzQo33k8c3J3qTezyAtFU7X0/nxAqdQ5O2Sv0+97iAivTta7NtcdKSix5ZmkHgQqMMjKOGw==',key_name='tempest-keypair-1672591556',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='ee1117bd3511400f9f45cc8f9bb9247d',ramdisk_id='',reservation_id='r-936jf1h5',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TaggedBootDevicesTest-1858653514',owner_user_name='tempest-TaggedBootDevicesTest-1858653514-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-11-29T03:07:00Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='36c915d13a4943549549177c756da93b',uuid=cb569e8d-e4a5-4076-bb8b-deeefad033e2,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "27e72461-f85b-42c9-8824-0d760b82a725", "address": "fa:16:3e:8e:29:fd", "network": {"id": "005139f3-3686-493a-aa8f-0cdc455befa0", "bridge": "br-int", "label": "tempest-device-tagging-net1-1510949192", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.127", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "ee1117bd3511400f9f45cc8f9bb9247d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap27e72461-f8", "ovs_interfaceid": "27e72461-f85b-42c9-8824-0d760b82a725", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=75234) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Nov 29 03:07:31 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] Converting VIF {"id": "27e72461-f85b-42c9-8824-0d760b82a725", "address": "fa:16:3e:8e:29:fd", "network": {"id": "005139f3-3686-493a-aa8f-0cdc455befa0", "bridge": "br-int", "label": "tempest-device-tagging-net1-1510949192", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.127", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "ee1117bd3511400f9f45cc8f9bb9247d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap27e72461-f8", "ovs_interfaceid": "27e72461-f85b-42c9-8824-0d760b82a725", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:07:31 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:8e:29:fd,bridge_name='br-int',has_traffic_filtering=True,id=27e72461-f85b-42c9-8824-0d760b82a725,network=Network(005139f3-3686-493a-aa8f-0cdc455befa0),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=True,vif_name='tap27e72461-f8') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:07:31 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='True',created_at=2023-11-29T03:06:58Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-1051091393',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-device-tagging-server-1051091393',id=10,image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBOxFHXB9/btSEa91bTsp1yfkF2UsLgRU7yInw9joeBa8Nj3E7aswz+RinNuyzQo33k8c3J3qTezyAtFU7X0/nxAqdQ5O2Sv0+97iAivTta7NtcdKSix5ZmkHgQqMMjKOGw==',key_name='tempest-keypair-1672591556',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='ee1117bd3511400f9f45cc8f9bb9247d',ramdisk_id='',reservation_id='r-936jf1h5',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TaggedBootDevicesTest-1858653514',owner_user_name='tempest-TaggedBootDevicesTest-1858653514-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-11-29T03:07:00Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='36c915d13a4943549549177c756da93b',uuid=cb569e8d-e4a5-4076-bb8b-deeefad033e2,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "026f606a-534c-4cdd-992a-9f33268b374f", "address": "fa:16:3e:3e:3b:29", "network": {"id": "005139f3-3686-493a-aa8f-0cdc455befa0", "bridge": "br-int", "label": "tempest-device-tagging-net1-1510949192", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.41", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "ee1117bd3511400f9f45cc8f9bb9247d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap026f606a-53", "ovs_interfaceid": "026f606a-534c-4cdd-992a-9f33268b374f", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=75234) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Nov 29 03:07:31 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] Converting VIF {"id": "026f606a-534c-4cdd-992a-9f33268b374f", "address": "fa:16:3e:3e:3b:29", "network": {"id": "005139f3-3686-493a-aa8f-0cdc455befa0", "bridge": "br-int", "label": "tempest-device-tagging-net1-1510949192", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.41", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "ee1117bd3511400f9f45cc8f9bb9247d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap026f606a-53", "ovs_interfaceid": "026f606a-534c-4cdd-992a-9f33268b374f", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:07:31 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:3e:3b:29,bridge_name='br-int',has_traffic_filtering=True,id=026f606a-534c-4cdd-992a-9f33268b374f,network=Network(005139f3-3686-493a-aa8f-0cdc455befa0),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap026f606a-53') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:07:31 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='True',created_at=2023-11-29T03:06:58Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-1051091393',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-device-tagging-server-1051091393',id=10,image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBOxFHXB9/btSEa91bTsp1yfkF2UsLgRU7yInw9joeBa8Nj3E7aswz+RinNuyzQo33k8c3J3qTezyAtFU7X0/nxAqdQ5O2Sv0+97iAivTta7NtcdKSix5ZmkHgQqMMjKOGw==',key_name='tempest-keypair-1672591556',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='ee1117bd3511400f9f45cc8f9bb9247d',ramdisk_id='',reservation_id='r-936jf1h5',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TaggedBootDevicesTest-1858653514',owner_user_name='tempest-TaggedBootDevicesTest-1858653514-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-11-29T03:07:00Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='36c915d13a4943549549177c756da93b',uuid=cb569e8d-e4a5-4076-bb8b-deeefad033e2,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "67446920-3741-4542-84c6-493388395f26", "address": "fa:16:3e:f6:7a:76", "network": {"id": "005139f3-3686-493a-aa8f-0cdc455befa0", "bridge": "br-int", "label": "tempest-device-tagging-net1-1510949192", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.230", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "ee1117bd3511400f9f45cc8f9bb9247d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap67446920-37", "ovs_interfaceid": "67446920-3741-4542-84c6-493388395f26", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=75234) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Nov 29 03:07:31 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] Converting VIF {"id": "67446920-3741-4542-84c6-493388395f26", "address": "fa:16:3e:f6:7a:76", "network": {"id": "005139f3-3686-493a-aa8f-0cdc455befa0", "bridge": "br-int", "label": "tempest-device-tagging-net1-1510949192", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.230", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "ee1117bd3511400f9f45cc8f9bb9247d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap67446920-37", "ovs_interfaceid": "67446920-3741-4542-84c6-493388395f26", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:07:31 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:f6:7a:76,bridge_name='br-int',has_traffic_filtering=True,id=67446920-3741-4542-84c6-493388395f26,network=Network(005139f3-3686-493a-aa8f-0cdc455befa0),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap67446920-37') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:07:31 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='True',created_at=2023-11-29T03:06:58Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-1051091393',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-device-tagging-server-1051091393',id=10,image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBOxFHXB9/btSEa91bTsp1yfkF2UsLgRU7yInw9joeBa8Nj3E7aswz+RinNuyzQo33k8c3J3qTezyAtFU7X0/nxAqdQ5O2Sv0+97iAivTta7NtcdKSix5ZmkHgQqMMjKOGw==',key_name='tempest-keypair-1672591556',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='ee1117bd3511400f9f45cc8f9bb9247d',ramdisk_id='',reservation_id='r-936jf1h5',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TaggedBootDevicesTest-1858653514',owner_user_name='tempest-TaggedBootDevicesTest-1858653514-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-11-29T03:07:00Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='36c915d13a4943549549177c756da93b',uuid=cb569e8d-e4a5-4076-bb8b-deeefad033e2,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "d7e92753-492b-4fc0-8048-f4792cd3b4b6", "address": "fa:16:3e:68:b2:33", "network": {"id": "cc6c7cdf-8c81-4906-9fd8-de815fea3374", "bridge": "br-int", "label": "tempest-device-tagging-net2-1248280764", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.100", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "ee1117bd3511400f9f45cc8f9bb9247d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapd7e92753-49", "ovs_interfaceid": "d7e92753-492b-4fc0-8048-f4792cd3b4b6", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=75234) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Nov 29 03:07:31 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] Converting VIF {"id": "d7e92753-492b-4fc0-8048-f4792cd3b4b6", "address": "fa:16:3e:68:b2:33", "network": {"id": "cc6c7cdf-8c81-4906-9fd8-de815fea3374", "bridge": "br-int", "label": "tempest-device-tagging-net2-1248280764", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.100", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "ee1117bd3511400f9f45cc8f9bb9247d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapd7e92753-49", "ovs_interfaceid": "d7e92753-492b-4fc0-8048-f4792cd3b4b6", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:07:31 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:68:b2:33,bridge_name='br-int',has_traffic_filtering=True,id=d7e92753-492b-4fc0-8048-f4792cd3b4b6,network=Network(cc6c7cdf-8c81-4906-9fd8-de815fea3374),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapd7e92753-49') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:07:31 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='True',created_at=2023-11-29T03:06:58Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-1051091393',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-device-tagging-server-1051091393',id=10,image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBOxFHXB9/btSEa91bTsp1yfkF2UsLgRU7yInw9joeBa8Nj3E7aswz+RinNuyzQo33k8c3J3qTezyAtFU7X0/nxAqdQ5O2Sv0+97iAivTta7NtcdKSix5ZmkHgQqMMjKOGw==',key_name='tempest-keypair-1672591556',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='ee1117bd3511400f9f45cc8f9bb9247d',ramdisk_id='',reservation_id='r-936jf1h5',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TaggedBootDevicesTest-1858653514',owner_user_name='tempest-TaggedBootDevicesTest-1858653514-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-11-29T03:07:00Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='36c915d13a4943549549177c756da93b',uuid=cb569e8d-e4a5-4076-bb8b-deeefad033e2,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "06db3012-0042-457a-a3c8-14e3a8d2e89e", "address": "fa:16:3e:27:54:30", "network": {"id": "cc6c7cdf-8c81-4906-9fd8-de815fea3374", "bridge": "br-int", "label": "tempest-device-tagging-net2-1248280764", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.200", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "ee1117bd3511400f9f45cc8f9bb9247d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap06db3012-00", "ovs_interfaceid": "06db3012-0042-457a-a3c8-14e3a8d2e89e", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=75234) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Nov 29 03:07:31 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] Converting VIF {"id": "06db3012-0042-457a-a3c8-14e3a8d2e89e", "address": "fa:16:3e:27:54:30", "network": {"id": "cc6c7cdf-8c81-4906-9fd8-de815fea3374", "bridge": "br-int", "label": "tempest-device-tagging-net2-1248280764", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.200", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "ee1117bd3511400f9f45cc8f9bb9247d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap06db3012-00", "ovs_interfaceid": "06db3012-0042-457a-a3c8-14e3a8d2e89e", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:07:31 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:27:54:30,bridge_name='br-int',has_traffic_filtering=True,id=06db3012-0042-457a-a3c8-14e3a8d2e89e,network=Network(cc6c7cdf-8c81-4906-9fd8-de815fea3374),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap06db3012-00') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:07:31 user nova-compute[75234]: DEBUG nova.objects.instance [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] Lazy-loading 'pci_devices' on Instance uuid cb569e8d-e4a5-4076-bb8b-deeefad033e2 {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:07:31 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] [instance: cb569e8d-e4a5-4076-bb8b-deeefad033e2] End _get_guest_xml xml=<domain type="kvm">
Nov 29 03:07:31 user nova-compute[75234]:   <uuid>cb569e8d-e4a5-4076-bb8b-deeefad033e2</uuid>
Nov 29 03:07:31 user nova-compute[75234]:   <name>instance-0000000a</name>
Nov 29 03:07:31 user nova-compute[75234]:   <memory>524288</memory>
Nov 29 03:07:31 user nova-compute[75234]:   <vcpu>1</vcpu>
Nov 29 03:07:31 user nova-compute[75234]:   <metadata>
Nov 29 03:07:31 user nova-compute[75234]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Nov 29 03:07:31 user nova-compute[75234]:       <nova:package version="0.0.0"/>
Nov 29 03:07:31 user nova-compute[75234]:       <nova:name>tempest-device-tagging-server-1051091393</nova:name>
Nov 29 03:07:31 user nova-compute[75234]:       <nova:creationTime>2023-11-29 03:07:27</nova:creationTime>
Nov 29 03:07:31 user nova-compute[75234]:       <nova:flavor name="m1.tiny">
Nov 29 03:07:31 user nova-compute[75234]:         <nova:memory>512</nova:memory>
Nov 29 03:07:31 user nova-compute[75234]:         <nova:disk>1</nova:disk>
Nov 29 03:07:31 user nova-compute[75234]:         <nova:swap>0</nova:swap>
Nov 29 03:07:31 user nova-compute[75234]:         <nova:ephemeral>0</nova:ephemeral>
Nov 29 03:07:31 user nova-compute[75234]:         <nova:vcpus>1</nova:vcpus>
Nov 29 03:07:31 user nova-compute[75234]:       </nova:flavor>
Nov 29 03:07:31 user nova-compute[75234]:       <nova:owner>
Nov 29 03:07:31 user nova-compute[75234]:         <nova:user uuid="36c915d13a4943549549177c756da93b">tempest-TaggedBootDevicesTest-1858653514-project-member</nova:user>
Nov 29 03:07:31 user nova-compute[75234]:         <nova:project uuid="ee1117bd3511400f9f45cc8f9bb9247d">tempest-TaggedBootDevicesTest-1858653514</nova:project>
Nov 29 03:07:31 user nova-compute[75234]:       </nova:owner>
Nov 29 03:07:31 user nova-compute[75234]:       <nova:root type="image" uuid="96c30ba7-9b73-4a30-8ba4-7138879573b7"/>
Nov 29 03:07:31 user nova-compute[75234]:       <nova:ports>
Nov 29 03:07:31 user nova-compute[75234]:         <nova:port uuid="2b12ec96-26fa-4974-8a7a-22e85ec20544">
Nov 29 03:07:31 user nova-compute[75234]:           <nova:ip type="fixed" address="10.0.0.8" ipVersion="4"/>
Nov 29 03:07:31 user nova-compute[75234]:         </nova:port>
Nov 29 03:07:31 user nova-compute[75234]:         <nova:port uuid="0bd5d735-f566-4a6d-b49f-b24fd403a87a">
Nov 29 03:07:31 user nova-compute[75234]:           <nova:ip type="fixed" address="10.1.1.85" ipVersion="4"/>
Nov 29 03:07:31 user nova-compute[75234]:         </nova:port>
Nov 29 03:07:31 user nova-compute[75234]:         <nova:port uuid="27e72461-f85b-42c9-8824-0d760b82a725">
Nov 29 03:07:31 user nova-compute[75234]:           <nova:ip type="fixed" address="10.1.1.127" ipVersion="4"/>
Nov 29 03:07:31 user nova-compute[75234]:         </nova:port>
Nov 29 03:07:31 user nova-compute[75234]:         <nova:port uuid="026f606a-534c-4cdd-992a-9f33268b374f">
Nov 29 03:07:31 user nova-compute[75234]:           <nova:ip type="fixed" address="10.1.1.41" ipVersion="4"/>
Nov 29 03:07:31 user nova-compute[75234]:         </nova:port>
Nov 29 03:07:31 user nova-compute[75234]:         <nova:port uuid="67446920-3741-4542-84c6-493388395f26">
Nov 29 03:07:31 user nova-compute[75234]:           <nova:ip type="fixed" address="10.1.1.230" ipVersion="4"/>
Nov 29 03:07:31 user nova-compute[75234]:         </nova:port>
Nov 29 03:07:31 user nova-compute[75234]:         <nova:port uuid="d7e92753-492b-4fc0-8048-f4792cd3b4b6">
Nov 29 03:07:31 user nova-compute[75234]:           <nova:ip type="fixed" address="10.2.2.100" ipVersion="4"/>
Nov 29 03:07:31 user nova-compute[75234]:         </nova:port>
Nov 29 03:07:31 user nova-compute[75234]:         <nova:port uuid="06db3012-0042-457a-a3c8-14e3a8d2e89e">
Nov 29 03:07:31 user nova-compute[75234]:           <nova:ip type="fixed" address="10.2.2.200" ipVersion="4"/>
Nov 29 03:07:31 user nova-compute[75234]:         </nova:port>
Nov 29 03:07:31 user nova-compute[75234]:       </nova:ports>
Nov 29 03:07:31 user nova-compute[75234]:     </nova:instance>
Nov 29 03:07:31 user nova-compute[75234]:   </metadata>
Nov 29 03:07:31 user nova-compute[75234]:   <sysinfo type="smbios">
Nov 29 03:07:31 user nova-compute[75234]:     <system>
Nov 29 03:07:31 user nova-compute[75234]:       <entry name="manufacturer">OpenStack Foundation</entry>
Nov 29 03:07:31 user nova-compute[75234]:       <entry name="product">OpenStack Nova</entry>
Nov 29 03:07:31 user nova-compute[75234]:       <entry name="version">0.0.0</entry>
Nov 29 03:07:31 user nova-compute[75234]:       <entry name="serial">cb569e8d-e4a5-4076-bb8b-deeefad033e2</entry>
Nov 29 03:07:31 user nova-compute[75234]:       <entry name="uuid">cb569e8d-e4a5-4076-bb8b-deeefad033e2</entry>
Nov 29 03:07:31 user nova-compute[75234]:       <entry name="family">Virtual Machine</entry>
Nov 29 03:07:31 user nova-compute[75234]:     </system>
Nov 29 03:07:31 user nova-compute[75234]:   </sysinfo>
Nov 29 03:07:31 user nova-compute[75234]:   <os>
Nov 29 03:07:31 user nova-compute[75234]:     <type arch="x86_64" machine="pc">hvm</type>
Nov 29 03:07:31 user nova-compute[75234]:     <boot dev="hd"/>
Nov 29 03:07:31 user nova-compute[75234]:     <smbios mode="sysinfo"/>
Nov 29 03:07:31 user nova-compute[75234]:   </os>
Nov 29 03:07:31 user nova-compute[75234]:   <features>
Nov 29 03:07:31 user nova-compute[75234]:     <acpi/>
Nov 29 03:07:31 user nova-compute[75234]:     <apic/>
Nov 29 03:07:31 user nova-compute[75234]:     <vmcoreinfo/>
Nov 29 03:07:31 user nova-compute[75234]:   </features>
Nov 29 03:07:31 user nova-compute[75234]:   <clock offset="utc">
Nov 29 03:07:31 user nova-compute[75234]:     <timer name="pit" tickpolicy="delay"/>
Nov 29 03:07:31 user nova-compute[75234]:     <timer name="rtc" tickpolicy="catchup"/>
Nov 29 03:07:31 user nova-compute[75234]:     <timer name="hpet" present="no"/>
Nov 29 03:07:31 user nova-compute[75234]:   </clock>
Nov 29 03:07:31 user nova-compute[75234]:   <cpu mode="custom" match="exact">
Nov 29 03:07:31 user nova-compute[75234]:     <model>Nehalem</model>
Nov 29 03:07:31 user nova-compute[75234]:     <topology sockets="1" cores="1" threads="1"/>
Nov 29 03:07:31 user nova-compute[75234]:   </cpu>
Nov 29 03:07:31 user nova-compute[75234]:   <devices>
Nov 29 03:07:31 user nova-compute[75234]:     <disk type="file" device="cdrom">
Nov 29 03:07:31 user nova-compute[75234]:       <driver name="qemu" type="raw" cache="none"/>
Nov 29 03:07:31 user nova-compute[75234]:       <source file="/opt/stack/data/nova/instances/cb569e8d-e4a5-4076-bb8b-deeefad033e2/disk.config"/>
Nov 29 03:07:31 user nova-compute[75234]:       <target dev="hda" bus="ide"/>
Nov 29 03:07:31 user nova-compute[75234]:     </disk>
Nov 29 03:07:31 user nova-compute[75234]:     <disk type="block" device="disk">
Nov 29 03:07:31 user nova-compute[75234]:       <driver name="qemu" type="raw" cache="none" io="native"/>
Nov 29 03:07:31 user nova-compute[75234]:       <source dev="/dev/sdb"/>
Nov 29 03:07:31 user nova-compute[75234]:       <target dev="vda" bus="virtio"/>
Nov 29 03:07:31 user nova-compute[75234]:       <serial>bb98586e-4f75-46be-a626-0b73ebda11eb</serial>
Nov 29 03:07:31 user nova-compute[75234]:     </disk>
Nov 29 03:07:31 user nova-compute[75234]:     <disk type="block" device="disk">
Nov 29 03:07:31 user nova-compute[75234]:       <driver name="qemu" type="raw" cache="none" io="native"/>
Nov 29 03:07:31 user nova-compute[75234]:       <source dev="/dev/sdc"/>
Nov 29 03:07:31 user nova-compute[75234]:       <target dev="vdb" bus="virtio"/>
Nov 29 03:07:31 user nova-compute[75234]:       <serial>ab278679-6cfd-4c7c-a930-c078843640d3</serial>
Nov 29 03:07:31 user nova-compute[75234]:     </disk>
Nov 29 03:07:31 user nova-compute[75234]:     <disk type="block" device="disk">
Nov 29 03:07:31 user nova-compute[75234]:       <driver name="qemu" type="raw" cache="none" io="native"/>
Nov 29 03:07:31 user nova-compute[75234]:       <source dev="/dev/sdd"/>
Nov 29 03:07:31 user nova-compute[75234]:       <target dev="vdc" bus="virtio"/>
Nov 29 03:07:31 user nova-compute[75234]:       <serial>920d6cad-25ab-4f3d-941b-8ec841d43184</serial>
Nov 29 03:07:31 user nova-compute[75234]:     </disk>
Nov 29 03:07:31 user nova-compute[75234]:     <interface type="ethernet">
Nov 29 03:07:31 user nova-compute[75234]:       <mac address="fa:16:3e:fd:00:a6"/>
Nov 29 03:07:31 user nova-compute[75234]:       <model type="virtio"/>
Nov 29 03:07:31 user nova-compute[75234]:       <mtu size="1442"/>
Nov 29 03:07:31 user nova-compute[75234]:       <target dev="tap2b12ec96-26"/>
Nov 29 03:07:31 user nova-compute[75234]:     </interface>
Nov 29 03:07:31 user nova-compute[75234]:     <interface type="ethernet">
Nov 29 03:07:31 user nova-compute[75234]:       <mac address="fa:16:3e:aa:c2:ae"/>
Nov 29 03:07:31 user nova-compute[75234]:       <model type="virtio"/>
Nov 29 03:07:31 user nova-compute[75234]:       <mtu size="1442"/>
Nov 29 03:07:31 user nova-compute[75234]:       <target dev="tap0bd5d735-f5"/>
Nov 29 03:07:31 user nova-compute[75234]:     </interface>
Nov 29 03:07:31 user nova-compute[75234]:     <interface type="ethernet">
Nov 29 03:07:31 user nova-compute[75234]:       <mac address="fa:16:3e:8e:29:fd"/>
Nov 29 03:07:31 user nova-compute[75234]:       <model type="virtio"/>
Nov 29 03:07:31 user nova-compute[75234]:       <mtu size="1442"/>
Nov 29 03:07:31 user nova-compute[75234]:       <target dev="tap27e72461-f8"/>
Nov 29 03:07:31 user nova-compute[75234]:     </interface>
Nov 29 03:07:31 user nova-compute[75234]:     <interface type="ethernet">
Nov 29 03:07:31 user nova-compute[75234]:       <mac address="fa:16:3e:3e:3b:29"/>
Nov 29 03:07:31 user nova-compute[75234]:       <model type="virtio"/>
Nov 29 03:07:31 user nova-compute[75234]:       <mtu size="1442"/>
Nov 29 03:07:31 user nova-compute[75234]:       <target dev="tap026f606a-53"/>
Nov 29 03:07:31 user nova-compute[75234]:     </interface>
Nov 29 03:07:31 user nova-compute[75234]:     <interface type="ethernet">
Nov 29 03:07:31 user nova-compute[75234]:       <mac address="fa:16:3e:f6:7a:76"/>
Nov 29 03:07:31 user nova-compute[75234]:       <model type="virtio"/>
Nov 29 03:07:31 user nova-compute[75234]:       <mtu size="1442"/>
Nov 29 03:07:31 user nova-compute[75234]:       <target dev="tap67446920-37"/>
Nov 29 03:07:31 user nova-compute[75234]:     </interface>
Nov 29 03:07:31 user nova-compute[75234]:     <interface type="ethernet">
Nov 29 03:07:31 user nova-compute[75234]:       <mac address="fa:16:3e:68:b2:33"/>
Nov 29 03:07:31 user nova-compute[75234]:       <model type="virtio"/>
Nov 29 03:07:31 user nova-compute[75234]:       <mtu size="1442"/>
Nov 29 03:07:31 user nova-compute[75234]:       <target dev="tapd7e92753-49"/>
Nov 29 03:07:31 user nova-compute[75234]:     </interface>
Nov 29 03:07:31 user nova-compute[75234]:     <interface type="ethernet">
Nov 29 03:07:31 user nova-compute[75234]:       <mac address="fa:16:3e:27:54:30"/>
Nov 29 03:07:31 user nova-compute[75234]:       <model type="virtio"/>
Nov 29 03:07:31 user nova-compute[75234]:       <mtu size="1442"/>
Nov 29 03:07:31 user nova-compute[75234]:       <target dev="tap06db3012-00"/>
Nov 29 03:07:31 user nova-compute[75234]:     </interface>
Nov 29 03:07:31 user nova-compute[75234]:     <serial type="pty">
Nov 29 03:07:31 user nova-compute[75234]:       <log file="/opt/stack/data/nova/instances/cb569e8d-e4a5-4076-bb8b-deeefad033e2/console.log" append="off"/>
Nov 29 03:07:31 user nova-compute[75234]:     </serial>
Nov 29 03:07:31 user nova-compute[75234]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Nov 29 03:07:31 user nova-compute[75234]:     <video>
Nov 29 03:07:31 user nova-compute[75234]:       <model type="virtio"/>
Nov 29 03:07:31 user nova-compute[75234]:     </video>
Nov 29 03:07:31 user nova-compute[75234]:     <rng model="virtio">
Nov 29 03:07:31 user nova-compute[75234]:       <backend model="random">/dev/urandom</backend>
Nov 29 03:07:31 user nova-compute[75234]:     </rng>
Nov 29 03:07:31 user nova-compute[75234]:     <controller type="usb" index="0" model="none"/>
Nov 29 03:07:31 user nova-compute[75234]:     <memballoon model="virtio">
Nov 29 03:07:31 user nova-compute[75234]:       <stats period="10"/>
Nov 29 03:07:31 user nova-compute[75234]:     </memballoon>
Nov 29 03:07:31 user nova-compute[75234]:   </devices>
Nov 29 03:07:31 user nova-compute[75234]: </domain>
Nov 29 03:07:31 user nova-compute[75234]:  {{(pid=75234) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7568}}
Nov 29 03:07:31 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='True',created_at=2023-11-29T03:06:58Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-1051091393',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-device-tagging-server-1051091393',id=10,image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBOxFHXB9/btSEa91bTsp1yfkF2UsLgRU7yInw9joeBa8Nj3E7aswz+RinNuyzQo33k8c3J3qTezyAtFU7X0/nxAqdQ5O2Sv0+97iAivTta7NtcdKSix5ZmkHgQqMMjKOGw==',key_name='tempest-keypair-1672591556',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='ee1117bd3511400f9f45cc8f9bb9247d',ramdisk_id='',reservation_id='r-936jf1h5',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TaggedBootDevicesTest-1858653514',owner_user_name='tempest-TaggedBootDevicesTest-1858653514-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-11-29T03:07:00Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='36c915d13a4943549549177c756da93b',uuid=cb569e8d-e4a5-4076-bb8b-deeefad033e2,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "2b12ec96-26fa-4974-8a7a-22e85ec20544", "address": "fa:16:3e:fd:00:a6", "network": {"id": "1869abec-0b45-446a-9dc8-5871de9dbfc5", "bridge": "br-int", "label": "tempest-TaggedBootDevicesTest-1812676341-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "ee1117bd3511400f9f45cc8f9bb9247d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2b12ec96-26", "ovs_interfaceid": "2b12ec96-26fa-4974-8a7a-22e85ec20544", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Nov 29 03:07:31 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] Converting VIF {"id": "2b12ec96-26fa-4974-8a7a-22e85ec20544", "address": "fa:16:3e:fd:00:a6", "network": {"id": "1869abec-0b45-446a-9dc8-5871de9dbfc5", "bridge": "br-int", "label": "tempest-TaggedBootDevicesTest-1812676341-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "ee1117bd3511400f9f45cc8f9bb9247d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2b12ec96-26", "ovs_interfaceid": "2b12ec96-26fa-4974-8a7a-22e85ec20544", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:07:31 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:fd:00:a6,bridge_name='br-int',has_traffic_filtering=True,id=2b12ec96-26fa-4974-8a7a-22e85ec20544,network=Network(1869abec-0b45-446a-9dc8-5871de9dbfc5),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap2b12ec96-26') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:07:31 user nova-compute[75234]: DEBUG os_vif [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:fd:00:a6,bridge_name='br-int',has_traffic_filtering=True,id=2b12ec96-26fa-4974-8a7a-22e85ec20544,network=Network(1869abec-0b45-446a-9dc8-5871de9dbfc5),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap2b12ec96-26') {{(pid=75234) plug /opt/stack/data/venv/lib/python3.10/site-packages/os_vif/__init__.py:76}}
Nov 29 03:07:31 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:31 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:07:31 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Nov 29 03:07:31 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:31 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbCreateCommand(_result=None, table=QoS, columns={'type': 'linux-noop', 'external_ids': {'id': '23ab0089-337d-587f-8443-53086bba0b45', '_type': 'linux-noop'}}, row=False) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:07:31 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:31 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:07:31 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:31 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tap2b12ec96-26, may_exist=True, interface_attrs={}) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:07:31 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Port, record=tap2b12ec96-26, col_values=(('qos', UUID('969108ac-b9d9-4aad-8009-826f457ca50f')),), if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:07:31 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DbSetCommand(_result=None, table=Interface, record=tap2b12ec96-26, col_values=(('external_ids', {'iface-id': '2b12ec96-26fa-4974-8a7a-22e85ec20544', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:fd:00:a6', 'vm-uuid': 'cb569e8d-e4a5-4076-bb8b-deeefad033e2'}),), if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:07:31 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:31 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:07:31 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:31 user nova-compute[75234]: INFO os_vif [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:fd:00:a6,bridge_name='br-int',has_traffic_filtering=True,id=2b12ec96-26fa-4974-8a7a-22e85ec20544,network=Network(1869abec-0b45-446a-9dc8-5871de9dbfc5),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap2b12ec96-26')
Nov 29 03:07:31 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='True',created_at=2023-11-29T03:06:58Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-1051091393',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-device-tagging-server-1051091393',id=10,image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBOxFHXB9/btSEa91bTsp1yfkF2UsLgRU7yInw9joeBa8Nj3E7aswz+RinNuyzQo33k8c3J3qTezyAtFU7X0/nxAqdQ5O2Sv0+97iAivTta7NtcdKSix5ZmkHgQqMMjKOGw==',key_name='tempest-keypair-1672591556',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='ee1117bd3511400f9f45cc8f9bb9247d',ramdisk_id='',reservation_id='r-936jf1h5',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TaggedBootDevicesTest-1858653514',owner_user_name='tempest-TaggedBootDevicesTest-1858653514-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-11-29T03:07:00Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='36c915d13a4943549549177c756da93b',uuid=cb569e8d-e4a5-4076-bb8b-deeefad033e2,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "0bd5d735-f566-4a6d-b49f-b24fd403a87a", "address": "fa:16:3e:aa:c2:ae", "network": {"id": "005139f3-3686-493a-aa8f-0cdc455befa0", "bridge": "br-int", "label": "tempest-device-tagging-net1-1510949192", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.85", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "ee1117bd3511400f9f45cc8f9bb9247d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap0bd5d735-f5", "ovs_interfaceid": "0bd5d735-f566-4a6d-b49f-b24fd403a87a", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}} {{(pid=75234) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Nov 29 03:07:31 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] Converting VIF {"id": "0bd5d735-f566-4a6d-b49f-b24fd403a87a", "address": "fa:16:3e:aa:c2:ae", "network": {"id": "005139f3-3686-493a-aa8f-0cdc455befa0", "bridge": "br-int", "label": "tempest-device-tagging-net1-1510949192", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.85", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "ee1117bd3511400f9f45cc8f9bb9247d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap0bd5d735-f5", "ovs_interfaceid": "0bd5d735-f566-4a6d-b49f-b24fd403a87a", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:07:31 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:aa:c2:ae,bridge_name='br-int',has_traffic_filtering=True,id=0bd5d735-f566-4a6d-b49f-b24fd403a87a,network=Network(005139f3-3686-493a-aa8f-0cdc455befa0),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=True,vif_name='tap0bd5d735-f5') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:07:31 user nova-compute[75234]: DEBUG os_vif [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:aa:c2:ae,bridge_name='br-int',has_traffic_filtering=True,id=0bd5d735-f566-4a6d-b49f-b24fd403a87a,network=Network(005139f3-3686-493a-aa8f-0cdc455befa0),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=True,vif_name='tap0bd5d735-f5') {{(pid=75234) plug /opt/stack/data/venv/lib/python3.10/site-packages/os_vif/__init__.py:76}}
Nov 29 03:07:31 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:31 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:07:31 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Nov 29 03:07:31 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:31 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbCreateCommand(_result=None, table=QoS, columns={'type': 'linux-noop', 'external_ids': {'id': '0b76d2ab-5b52-51de-8165-9e144b5e2e7e', '_type': 'linux-noop'}}, row=False) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:07:31 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:31 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:07:31 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:31 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:31 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tap0bd5d735-f5, may_exist=True, interface_attrs={}) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:07:31 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Port, record=tap0bd5d735-f5, col_values=(('qos', UUID('d93826bd-c224-4a03-8c14-2a2bba2568ac')),), if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:07:31 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DbSetCommand(_result=None, table=Interface, record=tap0bd5d735-f5, col_values=(('external_ids', {'iface-id': '0bd5d735-f566-4a6d-b49f-b24fd403a87a', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:aa:c2:ae', 'vm-uuid': 'cb569e8d-e4a5-4076-bb8b-deeefad033e2'}),), if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:07:31 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:31 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:07:31 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:31 user nova-compute[75234]: INFO os_vif [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:aa:c2:ae,bridge_name='br-int',has_traffic_filtering=True,id=0bd5d735-f566-4a6d-b49f-b24fd403a87a,network=Network(005139f3-3686-493a-aa8f-0cdc455befa0),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=True,vif_name='tap0bd5d735-f5')
Nov 29 03:07:31 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='True',created_at=2023-11-29T03:06:58Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-1051091393',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-device-tagging-server-1051091393',id=10,image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBOxFHXB9/btSEa91bTsp1yfkF2UsLgRU7yInw9joeBa8Nj3E7aswz+RinNuyzQo33k8c3J3qTezyAtFU7X0/nxAqdQ5O2Sv0+97iAivTta7NtcdKSix5ZmkHgQqMMjKOGw==',key_name='tempest-keypair-1672591556',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='ee1117bd3511400f9f45cc8f9bb9247d',ramdisk_id='',reservation_id='r-936jf1h5',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TaggedBootDevicesTest-1858653514',owner_user_name='tempest-TaggedBootDevicesTest-1858653514-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-11-29T03:07:00Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='36c915d13a4943549549177c756da93b',uuid=cb569e8d-e4a5-4076-bb8b-deeefad033e2,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "27e72461-f85b-42c9-8824-0d760b82a725", "address": "fa:16:3e:8e:29:fd", "network": {"id": "005139f3-3686-493a-aa8f-0cdc455befa0", "bridge": "br-int", "label": "tempest-device-tagging-net1-1510949192", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.127", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "ee1117bd3511400f9f45cc8f9bb9247d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap27e72461-f8", "ovs_interfaceid": "27e72461-f85b-42c9-8824-0d760b82a725", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}} {{(pid=75234) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Nov 29 03:07:31 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] Converting VIF {"id": "27e72461-f85b-42c9-8824-0d760b82a725", "address": "fa:16:3e:8e:29:fd", "network": {"id": "005139f3-3686-493a-aa8f-0cdc455befa0", "bridge": "br-int", "label": "tempest-device-tagging-net1-1510949192", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.127", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "ee1117bd3511400f9f45cc8f9bb9247d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap27e72461-f8", "ovs_interfaceid": "27e72461-f85b-42c9-8824-0d760b82a725", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:07:31 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:8e:29:fd,bridge_name='br-int',has_traffic_filtering=True,id=27e72461-f85b-42c9-8824-0d760b82a725,network=Network(005139f3-3686-493a-aa8f-0cdc455befa0),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=True,vif_name='tap27e72461-f8') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:07:31 user nova-compute[75234]: DEBUG os_vif [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:8e:29:fd,bridge_name='br-int',has_traffic_filtering=True,id=27e72461-f85b-42c9-8824-0d760b82a725,network=Network(005139f3-3686-493a-aa8f-0cdc455befa0),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=True,vif_name='tap27e72461-f8') {{(pid=75234) plug /opt/stack/data/venv/lib/python3.10/site-packages/os_vif/__init__.py:76}}
Nov 29 03:07:31 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:31 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:07:31 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Nov 29 03:07:31 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:31 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbCreateCommand(_result=None, table=QoS, columns={'type': 'linux-noop', 'external_ids': {'id': '2329f90c-aacc-57bc-ad97-7036a90dae25', '_type': 'linux-noop'}}, row=False) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:07:31 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:31 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:07:31 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:31 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:31 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tap27e72461-f8, may_exist=True, interface_attrs={}) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:07:31 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Port, record=tap27e72461-f8, col_values=(('qos', UUID('5d91513d-f0d4-4351-a36a-2225af2b7c52')),), if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:07:31 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DbSetCommand(_result=None, table=Interface, record=tap27e72461-f8, col_values=(('external_ids', {'iface-id': '27e72461-f85b-42c9-8824-0d760b82a725', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:8e:29:fd', 'vm-uuid': 'cb569e8d-e4a5-4076-bb8b-deeefad033e2'}),), if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:07:31 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:31 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:07:31 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:31 user nova-compute[75234]: INFO os_vif [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:8e:29:fd,bridge_name='br-int',has_traffic_filtering=True,id=27e72461-f85b-42c9-8824-0d760b82a725,network=Network(005139f3-3686-493a-aa8f-0cdc455befa0),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=True,vif_name='tap27e72461-f8')
Nov 29 03:07:31 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='True',created_at=2023-11-29T03:06:58Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-1051091393',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-device-tagging-server-1051091393',id=10,image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBOxFHXB9/btSEa91bTsp1yfkF2UsLgRU7yInw9joeBa8Nj3E7aswz+RinNuyzQo33k8c3J3qTezyAtFU7X0/nxAqdQ5O2Sv0+97iAivTta7NtcdKSix5ZmkHgQqMMjKOGw==',key_name='tempest-keypair-1672591556',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='ee1117bd3511400f9f45cc8f9bb9247d',ramdisk_id='',reservation_id='r-936jf1h5',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TaggedBootDevicesTest-1858653514',owner_user_name='tempest-TaggedBootDevicesTest-1858653514-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-11-29T03:07:00Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='36c915d13a4943549549177c756da93b',uuid=cb569e8d-e4a5-4076-bb8b-deeefad033e2,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "026f606a-534c-4cdd-992a-9f33268b374f", "address": "fa:16:3e:3e:3b:29", "network": {"id": "005139f3-3686-493a-aa8f-0cdc455befa0", "bridge": "br-int", "label": "tempest-device-tagging-net1-1510949192", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.41", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "ee1117bd3511400f9f45cc8f9bb9247d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap026f606a-53", "ovs_interfaceid": "026f606a-534c-4cdd-992a-9f33268b374f", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Nov 29 03:07:31 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] Converting VIF {"id": "026f606a-534c-4cdd-992a-9f33268b374f", "address": "fa:16:3e:3e:3b:29", "network": {"id": "005139f3-3686-493a-aa8f-0cdc455befa0", "bridge": "br-int", "label": "tempest-device-tagging-net1-1510949192", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.41", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "ee1117bd3511400f9f45cc8f9bb9247d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap026f606a-53", "ovs_interfaceid": "026f606a-534c-4cdd-992a-9f33268b374f", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:07:31 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:3e:3b:29,bridge_name='br-int',has_traffic_filtering=True,id=026f606a-534c-4cdd-992a-9f33268b374f,network=Network(005139f3-3686-493a-aa8f-0cdc455befa0),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap026f606a-53') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:07:31 user nova-compute[75234]: DEBUG os_vif [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:3e:3b:29,bridge_name='br-int',has_traffic_filtering=True,id=026f606a-534c-4cdd-992a-9f33268b374f,network=Network(005139f3-3686-493a-aa8f-0cdc455befa0),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap026f606a-53') {{(pid=75234) plug /opt/stack/data/venv/lib/python3.10/site-packages/os_vif/__init__.py:76}}
Nov 29 03:07:31 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:31 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:07:31 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Nov 29 03:07:31 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:31 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbCreateCommand(_result=None, table=QoS, columns={'type': 'linux-noop', 'external_ids': {'id': 'f3158e29-feaa-5abd-9536-99073f82f542', '_type': 'linux-noop'}}, row=False) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:07:31 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:31 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:07:31 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:31 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:31 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tap026f606a-53, may_exist=True, interface_attrs={}) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:07:31 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Port, record=tap026f606a-53, col_values=(('qos', UUID('3a7aa241-830c-48d4-b27d-c679a692a4cb')),), if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:07:31 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DbSetCommand(_result=None, table=Interface, record=tap026f606a-53, col_values=(('external_ids', {'iface-id': '026f606a-534c-4cdd-992a-9f33268b374f', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:3e:3b:29', 'vm-uuid': 'cb569e8d-e4a5-4076-bb8b-deeefad033e2'}),), if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:07:31 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:31 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:07:31 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:31 user nova-compute[75234]: INFO os_vif [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:3e:3b:29,bridge_name='br-int',has_traffic_filtering=True,id=026f606a-534c-4cdd-992a-9f33268b374f,network=Network(005139f3-3686-493a-aa8f-0cdc455befa0),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap026f606a-53')
Nov 29 03:07:31 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='True',created_at=2023-11-29T03:06:58Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-1051091393',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-device-tagging-server-1051091393',id=10,image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBOxFHXB9/btSEa91bTsp1yfkF2UsLgRU7yInw9joeBa8Nj3E7aswz+RinNuyzQo33k8c3J3qTezyAtFU7X0/nxAqdQ5O2Sv0+97iAivTta7NtcdKSix5ZmkHgQqMMjKOGw==',key_name='tempest-keypair-1672591556',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='ee1117bd3511400f9f45cc8f9bb9247d',ramdisk_id='',reservation_id='r-936jf1h5',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TaggedBootDevicesTest-1858653514',owner_user_name='tempest-TaggedBootDevicesTest-1858653514-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-11-29T03:07:00Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='36c915d13a4943549549177c756da93b',uuid=cb569e8d-e4a5-4076-bb8b-deeefad033e2,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "67446920-3741-4542-84c6-493388395f26", "address": "fa:16:3e:f6:7a:76", "network": {"id": "005139f3-3686-493a-aa8f-0cdc455befa0", "bridge": "br-int", "label": "tempest-device-tagging-net1-1510949192", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.230", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "ee1117bd3511400f9f45cc8f9bb9247d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap67446920-37", "ovs_interfaceid": "67446920-3741-4542-84c6-493388395f26", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Nov 29 03:07:31 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] Converting VIF {"id": "67446920-3741-4542-84c6-493388395f26", "address": "fa:16:3e:f6:7a:76", "network": {"id": "005139f3-3686-493a-aa8f-0cdc455befa0", "bridge": "br-int", "label": "tempest-device-tagging-net1-1510949192", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.230", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "ee1117bd3511400f9f45cc8f9bb9247d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap67446920-37", "ovs_interfaceid": "67446920-3741-4542-84c6-493388395f26", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:07:31 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:f6:7a:76,bridge_name='br-int',has_traffic_filtering=True,id=67446920-3741-4542-84c6-493388395f26,network=Network(005139f3-3686-493a-aa8f-0cdc455befa0),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap67446920-37') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:07:31 user nova-compute[75234]: DEBUG os_vif [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:f6:7a:76,bridge_name='br-int',has_traffic_filtering=True,id=67446920-3741-4542-84c6-493388395f26,network=Network(005139f3-3686-493a-aa8f-0cdc455befa0),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap67446920-37') {{(pid=75234) plug /opt/stack/data/venv/lib/python3.10/site-packages/os_vif/__init__.py:76}}
Nov 29 03:07:31 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:31 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:07:32 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Nov 29 03:07:32 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:32 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbCreateCommand(_result=None, table=QoS, columns={'type': 'linux-noop', 'external_ids': {'id': '70b8d47d-8cad-5f1a-8970-64cad8f11c49', '_type': 'linux-noop'}}, row=False) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:07:32 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:32 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:07:32 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:32 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:32 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tap67446920-37, may_exist=True, interface_attrs={}) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:07:32 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Port, record=tap67446920-37, col_values=(('qos', UUID('29a4a37a-747b-40e3-beca-787c66f2ee25')),), if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:07:32 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DbSetCommand(_result=None, table=Interface, record=tap67446920-37, col_values=(('external_ids', {'iface-id': '67446920-3741-4542-84c6-493388395f26', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:f6:7a:76', 'vm-uuid': 'cb569e8d-e4a5-4076-bb8b-deeefad033e2'}),), if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:07:32 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:32 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:07:32 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:32 user nova-compute[75234]: INFO os_vif [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:f6:7a:76,bridge_name='br-int',has_traffic_filtering=True,id=67446920-3741-4542-84c6-493388395f26,network=Network(005139f3-3686-493a-aa8f-0cdc455befa0),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap67446920-37')
Nov 29 03:07:32 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='True',created_at=2023-11-29T03:06:58Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-1051091393',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-device-tagging-server-1051091393',id=10,image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBOxFHXB9/btSEa91bTsp1yfkF2UsLgRU7yInw9joeBa8Nj3E7aswz+RinNuyzQo33k8c3J3qTezyAtFU7X0/nxAqdQ5O2Sv0+97iAivTta7NtcdKSix5ZmkHgQqMMjKOGw==',key_name='tempest-keypair-1672591556',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='ee1117bd3511400f9f45cc8f9bb9247d',ramdisk_id='',reservation_id='r-936jf1h5',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TaggedBootDevicesTest-1858653514',owner_user_name='tempest-TaggedBootDevicesTest-1858653514-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-11-29T03:07:00Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='36c915d13a4943549549177c756da93b',uuid=cb569e8d-e4a5-4076-bb8b-deeefad033e2,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "d7e92753-492b-4fc0-8048-f4792cd3b4b6", "address": "fa:16:3e:68:b2:33", "network": {"id": "cc6c7cdf-8c81-4906-9fd8-de815fea3374", "bridge": "br-int", "label": "tempest-device-tagging-net2-1248280764", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.100", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "ee1117bd3511400f9f45cc8f9bb9247d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapd7e92753-49", "ovs_interfaceid": "d7e92753-492b-4fc0-8048-f4792cd3b4b6", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Nov 29 03:07:32 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] Converting VIF {"id": "d7e92753-492b-4fc0-8048-f4792cd3b4b6", "address": "fa:16:3e:68:b2:33", "network": {"id": "cc6c7cdf-8c81-4906-9fd8-de815fea3374", "bridge": "br-int", "label": "tempest-device-tagging-net2-1248280764", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.100", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "ee1117bd3511400f9f45cc8f9bb9247d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapd7e92753-49", "ovs_interfaceid": "d7e92753-492b-4fc0-8048-f4792cd3b4b6", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:07:32 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:68:b2:33,bridge_name='br-int',has_traffic_filtering=True,id=d7e92753-492b-4fc0-8048-f4792cd3b4b6,network=Network(cc6c7cdf-8c81-4906-9fd8-de815fea3374),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapd7e92753-49') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:07:32 user nova-compute[75234]: DEBUG os_vif [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:68:b2:33,bridge_name='br-int',has_traffic_filtering=True,id=d7e92753-492b-4fc0-8048-f4792cd3b4b6,network=Network(cc6c7cdf-8c81-4906-9fd8-de815fea3374),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapd7e92753-49') {{(pid=75234) plug /opt/stack/data/venv/lib/python3.10/site-packages/os_vif/__init__.py:76}}
Nov 29 03:07:32 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:32 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:07:32 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Nov 29 03:07:32 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:32 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbCreateCommand(_result=None, table=QoS, columns={'type': 'linux-noop', 'external_ids': {'id': 'f8f9e079-efdb-53f7-a7a2-3a5ccfceb9af', '_type': 'linux-noop'}}, row=False) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:07:32 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:32 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:07:32 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:32 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:32 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tapd7e92753-49, may_exist=True, interface_attrs={}) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:07:32 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Port, record=tapd7e92753-49, col_values=(('qos', UUID('7ec833be-07f5-430a-83d6-e92b4b25db40')),), if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:07:32 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DbSetCommand(_result=None, table=Interface, record=tapd7e92753-49, col_values=(('external_ids', {'iface-id': 'd7e92753-492b-4fc0-8048-f4792cd3b4b6', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:68:b2:33', 'vm-uuid': 'cb569e8d-e4a5-4076-bb8b-deeefad033e2'}),), if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:07:32 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:32 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:07:32 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:32 user nova-compute[75234]: INFO os_vif [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:68:b2:33,bridge_name='br-int',has_traffic_filtering=True,id=d7e92753-492b-4fc0-8048-f4792cd3b4b6,network=Network(cc6c7cdf-8c81-4906-9fd8-de815fea3374),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapd7e92753-49')
Nov 29 03:07:32 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='True',created_at=2023-11-29T03:06:58Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-1051091393',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-device-tagging-server-1051091393',id=10,image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBOxFHXB9/btSEa91bTsp1yfkF2UsLgRU7yInw9joeBa8Nj3E7aswz+RinNuyzQo33k8c3J3qTezyAtFU7X0/nxAqdQ5O2Sv0+97iAivTta7NtcdKSix5ZmkHgQqMMjKOGw==',key_name='tempest-keypair-1672591556',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='ee1117bd3511400f9f45cc8f9bb9247d',ramdisk_id='',reservation_id='r-936jf1h5',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TaggedBootDevicesTest-1858653514',owner_user_name='tempest-TaggedBootDevicesTest-1858653514-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-11-29T03:07:00Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='36c915d13a4943549549177c756da93b',uuid=cb569e8d-e4a5-4076-bb8b-deeefad033e2,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "06db3012-0042-457a-a3c8-14e3a8d2e89e", "address": "fa:16:3e:27:54:30", "network": {"id": "cc6c7cdf-8c81-4906-9fd8-de815fea3374", "bridge": "br-int", "label": "tempest-device-tagging-net2-1248280764", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.200", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "ee1117bd3511400f9f45cc8f9bb9247d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap06db3012-00", "ovs_interfaceid": "06db3012-0042-457a-a3c8-14e3a8d2e89e", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Nov 29 03:07:32 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] Converting VIF {"id": "06db3012-0042-457a-a3c8-14e3a8d2e89e", "address": "fa:16:3e:27:54:30", "network": {"id": "cc6c7cdf-8c81-4906-9fd8-de815fea3374", "bridge": "br-int", "label": "tempest-device-tagging-net2-1248280764", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.200", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "ee1117bd3511400f9f45cc8f9bb9247d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap06db3012-00", "ovs_interfaceid": "06db3012-0042-457a-a3c8-14e3a8d2e89e", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:07:32 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:27:54:30,bridge_name='br-int',has_traffic_filtering=True,id=06db3012-0042-457a-a3c8-14e3a8d2e89e,network=Network(cc6c7cdf-8c81-4906-9fd8-de815fea3374),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap06db3012-00') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:07:32 user nova-compute[75234]: DEBUG os_vif [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:27:54:30,bridge_name='br-int',has_traffic_filtering=True,id=06db3012-0042-457a-a3c8-14e3a8d2e89e,network=Network(cc6c7cdf-8c81-4906-9fd8-de815fea3374),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap06db3012-00') {{(pid=75234) plug /opt/stack/data/venv/lib/python3.10/site-packages/os_vif/__init__.py:76}}
Nov 29 03:07:32 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:32 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:07:32 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Nov 29 03:07:32 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:32 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbCreateCommand(_result=None, table=QoS, columns={'type': 'linux-noop', 'external_ids': {'id': '2f1fa78c-a0a8-5e63-a76c-59b482ed09e8', '_type': 'linux-noop'}}, row=False) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:07:32 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:32 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:07:32 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:32 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:32 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tap06db3012-00, may_exist=True, interface_attrs={}) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:07:32 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Port, record=tap06db3012-00, col_values=(('qos', UUID('320e66cf-bfaa-4eb7-a6a6-636644483966')),), if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:07:32 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DbSetCommand(_result=None, table=Interface, record=tap06db3012-00, col_values=(('external_ids', {'iface-id': '06db3012-0042-457a-a3c8-14e3a8d2e89e', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:27:54:30', 'vm-uuid': 'cb569e8d-e4a5-4076-bb8b-deeefad033e2'}),), if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:07:32 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:32 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:07:32 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:32 user nova-compute[75234]: INFO os_vif [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:27:54:30,bridge_name='br-int',has_traffic_filtering=True,id=06db3012-0042-457a-a3c8-14e3a8d2e89e,network=Network(cc6c7cdf-8c81-4906-9fd8-de815fea3374),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap06db3012-00')
Nov 29 03:07:32 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] No BDM found with device name hda, not building metadata. {{(pid=75234) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12089}}
Nov 29 03:07:32 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] No BDM found with device name vdc, not building metadata. {{(pid=75234) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12089}}
Nov 29 03:07:32 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] No VIF found with MAC fa:16:3e:fd:00:a6, not building metadata {{(pid=75234) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Nov 29 03:07:32 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] No VIF found with MAC fa:16:3e:f6:7a:76, not building metadata {{(pid=75234) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Nov 29 03:07:32 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] [instance: cb569e8d-e4a5-4076-bb8b-deeefad033e2] Using config drive
Nov 29 03:07:32 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] [instance: cb569e8d-e4a5-4076-bb8b-deeefad033e2] Creating config drive at /opt/stack/data/nova/instances/cb569e8d-e4a5-4076-bb8b-deeefad033e2/disk.config
Nov 29 03:07:32 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] Running cmd (subprocess): genisoimage -o /opt/stack/data/nova/instances/cb569e8d-e4a5-4076-bb8b-deeefad033e2/disk.config -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 0.0.0 -quiet -J -r -V config-2 /tmp/tmphunpfkml {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:07:32 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] CMD "genisoimage -o /opt/stack/data/nova/instances/cb569e8d-e4a5-4076-bb8b-deeefad033e2/disk.config -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 0.0.0 -quiet -J -r -V config-2 /tmp/tmphunpfkml" returned: 0 in 0.125s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:07:34 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:34 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:34 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:34 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:34 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:34 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:34 user nova-compute[75234]: DEBUG nova.compute.manager [req-bb356494-58d5-4d2b-88df-3764c6b7e21f req-4286d9dc-a70e-4844-b902-da722e222365 service nova] [instance: cb569e8d-e4a5-4076-bb8b-deeefad033e2] Received event network-vif-plugged-2b12ec96-26fa-4974-8a7a-22e85ec20544 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:07:34 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-bb356494-58d5-4d2b-88df-3764c6b7e21f req-4286d9dc-a70e-4844-b902-da722e222365 service nova] Acquiring lock "cb569e8d-e4a5-4076-bb8b-deeefad033e2-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:07:34 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-bb356494-58d5-4d2b-88df-3764c6b7e21f req-4286d9dc-a70e-4844-b902-da722e222365 service nova] Lock "cb569e8d-e4a5-4076-bb8b-deeefad033e2-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:07:34 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-bb356494-58d5-4d2b-88df-3764c6b7e21f req-4286d9dc-a70e-4844-b902-da722e222365 service nova] Lock "cb569e8d-e4a5-4076-bb8b-deeefad033e2-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:07:34 user nova-compute[75234]: DEBUG nova.compute.manager [req-bb356494-58d5-4d2b-88df-3764c6b7e21f req-4286d9dc-a70e-4844-b902-da722e222365 service nova] [instance: cb569e8d-e4a5-4076-bb8b-deeefad033e2] No waiting events found dispatching network-vif-plugged-2b12ec96-26fa-4974-8a7a-22e85ec20544 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:07:34 user nova-compute[75234]: WARNING nova.compute.manager [req-bb356494-58d5-4d2b-88df-3764c6b7e21f req-4286d9dc-a70e-4844-b902-da722e222365 service nova] [instance: cb569e8d-e4a5-4076-bb8b-deeefad033e2] Received unexpected event network-vif-plugged-2b12ec96-26fa-4974-8a7a-22e85ec20544 for instance with vm_state building and task_state spawning.
Nov 29 03:07:34 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:34 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:34 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:35 user nova-compute[75234]: DEBUG nova.compute.manager [req-7cdb2ece-eef7-4ece-b395-7696a80306d0 req-fcd3feb1-5a49-4d26-9cff-ed0c2c33fa98 service nova] [instance: cb569e8d-e4a5-4076-bb8b-deeefad033e2] Received event network-vif-plugged-0bd5d735-f566-4a6d-b49f-b24fd403a87a {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:07:35 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-7cdb2ece-eef7-4ece-b395-7696a80306d0 req-fcd3feb1-5a49-4d26-9cff-ed0c2c33fa98 service nova] Acquiring lock "cb569e8d-e4a5-4076-bb8b-deeefad033e2-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:07:35 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-7cdb2ece-eef7-4ece-b395-7696a80306d0 req-fcd3feb1-5a49-4d26-9cff-ed0c2c33fa98 service nova] Lock "cb569e8d-e4a5-4076-bb8b-deeefad033e2-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:07:35 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-7cdb2ece-eef7-4ece-b395-7696a80306d0 req-fcd3feb1-5a49-4d26-9cff-ed0c2c33fa98 service nova] Lock "cb569e8d-e4a5-4076-bb8b-deeefad033e2-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:07:35 user nova-compute[75234]: DEBUG nova.compute.manager [req-7cdb2ece-eef7-4ece-b395-7696a80306d0 req-fcd3feb1-5a49-4d26-9cff-ed0c2c33fa98 service nova] [instance: cb569e8d-e4a5-4076-bb8b-deeefad033e2] No waiting events found dispatching network-vif-plugged-0bd5d735-f566-4a6d-b49f-b24fd403a87a {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:07:35 user nova-compute[75234]: WARNING nova.compute.manager [req-7cdb2ece-eef7-4ece-b395-7696a80306d0 req-fcd3feb1-5a49-4d26-9cff-ed0c2c33fa98 service nova] [instance: cb569e8d-e4a5-4076-bb8b-deeefad033e2] Received unexpected event network-vif-plugged-0bd5d735-f566-4a6d-b49f-b24fd403a87a for instance with vm_state building and task_state spawning.
Nov 29 03:07:35 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:35 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:35 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:35 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:35 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:35 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:35 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:35 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:35 user nova-compute[75234]: DEBUG nova.compute.manager [req-db0aa511-e1dc-4343-bc2b-b46645f0f97f req-c1f5ee2f-67b2-41df-aa31-301ab69b4e33 service nova] [instance: cb569e8d-e4a5-4076-bb8b-deeefad033e2] Received event network-vif-plugged-026f606a-534c-4cdd-992a-9f33268b374f {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:07:35 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-db0aa511-e1dc-4343-bc2b-b46645f0f97f req-c1f5ee2f-67b2-41df-aa31-301ab69b4e33 service nova] Acquiring lock "cb569e8d-e4a5-4076-bb8b-deeefad033e2-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:07:35 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-db0aa511-e1dc-4343-bc2b-b46645f0f97f req-c1f5ee2f-67b2-41df-aa31-301ab69b4e33 service nova] Lock "cb569e8d-e4a5-4076-bb8b-deeefad033e2-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:07:35 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-db0aa511-e1dc-4343-bc2b-b46645f0f97f req-c1f5ee2f-67b2-41df-aa31-301ab69b4e33 service nova] Lock "cb569e8d-e4a5-4076-bb8b-deeefad033e2-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:07:35 user nova-compute[75234]: DEBUG nova.compute.manager [req-db0aa511-e1dc-4343-bc2b-b46645f0f97f req-c1f5ee2f-67b2-41df-aa31-301ab69b4e33 service nova] [instance: cb569e8d-e4a5-4076-bb8b-deeefad033e2] No waiting events found dispatching network-vif-plugged-026f606a-534c-4cdd-992a-9f33268b374f {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:07:35 user nova-compute[75234]: WARNING nova.compute.manager [req-db0aa511-e1dc-4343-bc2b-b46645f0f97f req-c1f5ee2f-67b2-41df-aa31-301ab69b4e33 service nova] [instance: cb569e8d-e4a5-4076-bb8b-deeefad033e2] Received unexpected event network-vif-plugged-026f606a-534c-4cdd-992a-9f33268b374f for instance with vm_state building and task_state spawning.
Nov 29 03:07:35 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:35 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:35 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:35 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:35 user nova-compute[75234]: DEBUG nova.compute.manager [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] [instance: 763a3e14-4563-4dc2-99ae-c340edbb93df] Start spawning the instance on the hypervisor. {{(pid=75234) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2628}}
Nov 29 03:07:35 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] [instance: 763a3e14-4563-4dc2-99ae-c340edbb93df] Creating instance directory {{(pid=75234) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4727}}
Nov 29 03:07:35 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] [instance: 763a3e14-4563-4dc2-99ae-c340edbb93df] Creating image(s)
Nov 29 03:07:35 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] [instance: 763a3e14-4563-4dc2-99ae-c340edbb93df] Did not create local disks {{(pid=75234) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4863}}
Nov 29 03:07:35 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] [instance: 763a3e14-4563-4dc2-99ae-c340edbb93df] Ensure instance console log exists: /opt/stack/data/nova/instances/763a3e14-4563-4dc2-99ae-c340edbb93df/console.log {{(pid=75234) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4613}}
Nov 29 03:07:35 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:07:35 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:07:35 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:07:35 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] [instance: 763a3e14-4563-4dc2-99ae-c340edbb93df] Start _get_guest_xml network_info=[{"id": "dc96090f-e855-4d3e-854c-50ea700dc8ea", "address": "fa:16:3e:9e:2b:8a", "network": {"id": "ea3baf6d-ba72-4615-bc0d-604e4fcd8e4e", "bridge": "br-int", "label": "tempest-TaggedBootDevicesTest_v242-1704264625-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "325013d2bedc498e91c2145c2b398803", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapdc96090f-e8", "ovs_interfaceid": "dc96090f-e855-4d3e-854c-50ea700dc8ea", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "2027f7a1-36e4-4434-a063-61e817758917", "address": "fa:16:3e:4d:66:94", "network": {"id": "60b448f6-90dc-4dac-b24e-9370262e6dac", "bridge": "br-int", "label": "tempest-device-tagging-net1-431494419", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.80", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "325013d2bedc498e91c2145c2b398803", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2027f7a1-36", "ovs_interfaceid": "2027f7a1-36e4-4434-a063-61e817758917", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}}, {"id": "de4ad363-fae0-4d1b-9820-5a91a27d630d", "address": "fa:16:3e:94:49:94", "network": {"id": "60b448f6-90dc-4dac-b24e-9370262e6dac", "bridge": "br-int", "label": "tempest-device-tagging-net1-431494419", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.218", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "325013d2bedc498e91c2145c2b398803", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapde4ad363-fa", "ovs_interfaceid": "de4ad363-fae0-4d1b-9820-5a91a27d630d", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}}, {"id": "d520a88d-ec02-45b8-adbc-484d38b9c18c", "address": "fa:16:3e:ac:cd:aa", "network": {"id": "60b448f6-90dc-4dac-b24e-9370262e6dac", "bridge": "br-int", "label": "tempest-device-tagging-net1-431494419", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.226", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "325013d2bedc498e91c2145c2b398803", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapd520a88d-ec", "ovs_interfaceid": "d520a88d-ec02-45b8-adbc-484d38b9c18c", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "c50f64cd-b7fa-4580-bb78-bff756a0773a", "address": "fa:16:3e:36:0b:d9", "network": {"id": "60b448f6-90dc-4dac-b24e-9370262e6dac", "bridge": "br-int", "label": "tempest-device-tagging-net1-431494419", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.18", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "325013d2bedc498e91c2145c2b398803", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc50f64cd-b7", "ovs_interfaceid": "c50f64cd-b7fa-4580-bb78-bff756a0773a", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "ef8ba969-6186-4b9a-b30f-ec9a70036bab", "address": "fa:16:3e:0f:ed:74", "network": {"id": "31e25520-c84d-451c-9132-ddc8c1e443a6", "bridge": "br-int", "label": "tempest-device-tagging-net2-653330954", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.100", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "325013d2bedc498e91c2145c2b398803", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapef8ba969-61", "ovs_interfaceid": "ef8ba969-6186-4b9a-b30f-ec9a70036bab", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "92917a0d-59ee-4167-8693-262a21f6e156", "address": "fa:16:3e:65:95:01", "network": {"id": "31e25520-c84d-451c-9132-ddc8c1e443a6", "bridge": "br-int", "label": "tempest-device-tagging-net2-653330954", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.200", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "325013d2bedc498e91c2145c2b398803", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap92917a0d-59", "ovs_interfaceid": "92917a0d-59ee-4167-8693-262a21f6e156", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, '/dev/vda': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, '/dev/vdb': {'bus': 'virtio', 'dev': 'vdb', 'type': 'disk', 'boot_index': '2'}, '/dev/vdc': {'bus': 'virtio', 'dev': 'vdc', 'type': 'disk', 'boot_index': '3'}, 'disk.config': {'bus': 'ide', 'dev': 'hda', 'type': 'cdrom'}}} image_meta=ImageMeta(checksum='c8fc807773e5354afe61636071771906',container_format='bare',created_at=2023-11-29T03:02:15Z,direct_url=<?>,disk_format='qcow2',id=96c30ba7-9b73-4a30-8ba4-7138879573b7,min_disk=0,min_ram=0,name='cirros-0.6.2-x86_64-disk',owner='b18e380921c748b78517b6f1211cd0f1',properties=ImageMetaProps,protected=<?>,size=21430272,status='active',tags=<?>,updated_at=2023-11-29T03:02:17Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [], 'ephemerals': [], 'block_device_mapping': [{'mount_device': '/dev/vda', 'attachment_id': '294a0160-6e5e-4430-8d76-f861ff5dbb23', 'disk_bus': 'virtio', 'device_type': 'disk', 'boot_index': 0, 'guest_format': None, 'delete_on_termination': False, 'connection_info': {'driver_volume_type': 'iscsi', 'data': {'target_iqn': 'iqn.2010-10.org.openstack:b1068fd1-6636-43cd-9725-4a37f708b2bc', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'LpdSswjf', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}, 'status': 'reserved', 'instance': '763a3e14-4563-4dc2-99ae-c340edbb93df', 'attached_at': '', 'detached_at': '', 'volume_id': 'b1068fd1-6636-43cd-9725-4a37f708b2bc', 'serial': 'b1068fd1-6636-43cd-9725-4a37f708b2bc'}, 'volume_type': None}, {'mount_device': '/dev/vdb', 'attachment_id': '729ab20a-0394-426d-9bc7-b8f7bdc02d1d', 'disk_bus': 'virtio', 'device_type': 'disk', 'boot_index': 1, 'guest_format': None, 'delete_on_termination': False, 'connection_info': {'driver_volume_type': 'iscsi', 'data': {'target_iqn': 'iqn.2010-10.org.openstack:0d2c0c22-2b3a-43db-b9af-e60a2514e48e', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'oOMLrjWB', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}, 'status': 'reserved', 'instance': '763a3e14-4563-4dc2-99ae-c340edbb93df', 'attached_at': '', 'detached_at': '', 'volume_id': '0d2c0c22-2b3a-43db-b9af-e60a2514e48e', 'serial': '0d2c0c22-2b3a-43db-b9af-e60a2514e48e'}, 'volume_type': None}, {'mount_device': '/dev/vdc', 'attachment_id': 'c95e0a76-151c-4590-b598-dbe54926b619', 'disk_bus': 'virtio', 'device_type': 'disk', 'boot_index': 2, 'guest_format': None, 'delete_on_termination': False, 'connection_info': {'driver_volume_type': 'iscsi', 'data': {'target_iqn': 'iqn.2010-10.org.openstack:5fcd66fc-a479-4613-b1f8-e6ef12ca470a', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'lbfrXqMs', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}, 'status': 'reserved', 'instance': '763a3e14-4563-4dc2-99ae-c340edbb93df', 'attached_at': '', 'detached_at': '', 'volume_id': '5fcd66fc-a479-4613-b1f8-e6ef12ca470a', 'serial': '5fcd66fc-a479-4613-b1f8-e6ef12ca470a'}, 'volume_type': None} {{(pid=75234) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7562}}
Nov 29 03:07:35 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:07:35 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:07:35 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] Searching host: 'user' for CPU controller through CGroups V1... {{(pid=75234) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1652}}
Nov 29 03:07:35 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] CPU controller missing on host. {{(pid=75234) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1662}}
Nov 29 03:07:35 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] Searching host: 'user' for CPU controller through CGroups V2... {{(pid=75234) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1671}}
Nov 29 03:07:35 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] CPU controller found on host. {{(pid=75234) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1678}}
Nov 29 03:07:35 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=75234) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5400}}
Nov 29 03:07:35 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-11-29T03:02:04Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='1',id=6,is_public=True,memory_mb=512,name='m1.tiny',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='c8fc807773e5354afe61636071771906',container_format='bare',created_at=2023-11-29T03:02:15Z,direct_url=<?>,disk_format='qcow2',id=96c30ba7-9b73-4a30-8ba4-7138879573b7,min_disk=0,min_ram=0,name='cirros-0.6.2-x86_64-disk',owner='b18e380921c748b78517b6f1211cd0f1',properties=ImageMetaProps,protected=<?>,size=21430272,status='active',tags=<?>,updated_at=2023-11-29T03:02:17Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:563}}
Nov 29 03:07:35 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] Flavor limits 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:348}}
Nov 29 03:07:35 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] Image limits 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:352}}
Nov 29 03:07:35 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] Flavor pref 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:388}}
Nov 29 03:07:35 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] Image pref 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:392}}
Nov 29 03:07:35 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:430}}
Nov 29 03:07:35 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:569}}
Nov 29 03:07:35 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=75234) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:471}}
Nov 29 03:07:35 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] Got 1 possible topologies {{(pid=75234) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:501}}
Nov 29 03:07:35 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:575}}
Nov 29 03:07:35 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:577}}
Nov 29 03:07:35 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] [instance: 763a3e14-4563-4dc2-99ae-c340edbb93df] Calling os-brick to attach iSCSI Volume {{(pid=75234) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Nov 29 03:07:35 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f8a5050c9a0>, 'connection_properties': {'target_iqn': 'iqn.2010-10.org.openstack:b1068fd1-6636-43cd-9725-4a37f708b2bc', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'LpdSswjf', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:176}}
Nov 29 03:07:35 user nova-compute[75234]: WARNING os_brick.initiator.connectors.base [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Nov 29 03:07:35 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:68}}
Nov 29 03:07:35 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:73}}
Nov 29 03:07:35 user nova-compute[75234]: WARNING os_brick.initiator.connectors.base [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Nov 29 03:07:35 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:b1068fd1-6636-43cd-9725-4a37f708b2bc" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:68}}
Nov 29 03:07:35 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:b1068fd1-6636-43cd-9725-4a37f708b2bc" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:73}}
Nov 29 03:07:35 user nova-compute[75234]: INFO os_brick.initiator.connectors.iscsi [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Nov 29 03:07:35 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:b1068fd1-6636-43cd-9725-4a37f708b2bc -p 172.16.0.220:3260 {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:07:35 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:b1068fd1-6636-43cd-9725-4a37f708b2bc -p 172.16.0.220:3260" returned: 21 in 0.021s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:07:35 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[8a5e0e68-6223-49b5-8733-b10635fbaac0]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:07:35 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Nov 29 03:07:35 user nova-compute[75234]:  {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:07:35 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:b1068fd1-6636-43cd-9725-4a37f708b2bc -p 172.16.0.220:3260 --interface default --op new {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:07:35 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:b1068fd1-6636-43cd-9725-4a37f708b2bc -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.020s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:07:35 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[63a75df7-674f-40a9-a6e7-1ba76e6bf8ac]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:b1068fd1-6636-43cd-9725-4a37f708b2bc] added\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:07:35 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:b1068fd1-6636-43cd-9725-4a37f708b2bc] added
Nov 29 03:07:35 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:07:35 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:b1068fd1-6636-43cd-9725-4a37f708b2bc -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:07:35 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:35 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:35 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:b1068fd1-6636-43cd-9725-4a37f708b2bc -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.029s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:07:35 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[eac0816f-8d02-44e1-a595-81cc344924f9]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:07:35 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:07:35 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:b1068fd1-6636-43cd-9725-4a37f708b2bc -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:07:36 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:b1068fd1-6636-43cd-9725-4a37f708b2bc -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.025s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:07:36 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[3c907af0-d980-4f89-a463-538a2d50cb3b]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:07:36 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:07:36 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:b1068fd1-6636-43cd-9725-4a37f708b2bc -p 172.16.0.220:3260 --op update -n node.session.auth.username -v LpdSswjf {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:07:36 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:b1068fd1-6636-43cd-9725-4a37f708b2bc -p 172.16.0.220:3260 --op update -n node.session.auth.username -v LpdSswjf" returned: 0 in 0.024s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:07:36 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[84425da8-92b7-4c98-befc-144064c66be1]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:07:36 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'LpdSswjf'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:07:36 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:b1068fd1-6636-43cd-9725-4a37f708b2bc -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:07:36 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:b1068fd1-6636-43cd-9725-4a37f708b2bc -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.025s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:07:36 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[15f0fe35-43d4-44c6-9d1a-e6d4099af5e6]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:07:36 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:07:36 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:07:36 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:36 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:36 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.024s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:07:36 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[5e7de298-6dc1-4e0b-9ed8-e15ef7c792b4]: (4, ('tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bb98586e-4f75-46be-a626-0b73ebda11eb (non-flash)\ntcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ab278679-6cfd-4c7c-a930-c078843640d3 (non-flash)\ntcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:920d6cad-25ab-4f3d-941b-8ec841d43184 (non-flash)\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:07:36 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bb98586e-4f75-46be-a626-0b73ebda11eb (non-flash)
Nov 29 03:07:36 user nova-compute[75234]: tcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ab278679-6cfd-4c7c-a930-c078843640d3 (non-flash)
Nov 29 03:07:36 user nova-compute[75234]: tcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:920d6cad-25ab-4f3d-941b-8ec841d43184 (non-flash)
Nov 29 03:07:36 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:07:36 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] iscsi session list stdout=tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bb98586e-4f75-46be-a626-0b73ebda11eb (non-flash)
Nov 29 03:07:36 user nova-compute[75234]: tcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ab278679-6cfd-4c7c-a930-c078843640d3 (non-flash)
Nov 29 03:07:36 user nova-compute[75234]: tcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:920d6cad-25ab-4f3d-941b-8ec841d43184 (non-flash)
Nov 29 03:07:36 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsi_session /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Nov 29 03:07:36 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:36 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:b1068fd1-6636-43cd-9725-4a37f708b2bc -p 172.16.0.220:3260 --login {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:07:36 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:36 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:36 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:36 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:b1068fd1-6636-43cd-9725-4a37f708b2bc -p 172.16.0.220:3260 --login" returned: 0 in 0.072s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:07:36 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[0e598501-5dd9-495c-8bfa-a464e69eee0a]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:b1068fd1-6636-43cd-9725-4a37f708b2bc, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:b1068fd1-6636-43cd-9725-4a37f708b2bc, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:07:36 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:b1068fd1-6636-43cd-9725-4a37f708b2bc, portal: 172.16.0.220,3260]
Nov 29 03:07:36 user nova-compute[75234]: Login to [iface: default, target: iqn.2010-10.org.openstack:b1068fd1-6636-43cd-9725-4a37f708b2bc, portal: 172.16.0.220,3260] successful.
Nov 29 03:07:36 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:07:36 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:b1068fd1-6636-43cd-9725-4a37f708b2bc -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:07:36 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:b1068fd1-6636-43cd-9725-4a37f708b2bc -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.018s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:07:36 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[1d293428-f30b-4792-9093-8e528a62dcc9]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:07:36 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:07:36 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:07:36 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.042s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:07:36 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[1dca374d-8b5c-43f0-b84b-38e18fc0eb09]: (4, ('tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bb98586e-4f75-46be-a626-0b73ebda11eb (non-flash)\ntcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ab278679-6cfd-4c7c-a930-c078843640d3 (non-flash)\ntcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:920d6cad-25ab-4f3d-941b-8ec841d43184 (non-flash)\ntcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b1068fd1-6636-43cd-9725-4a37f708b2bc (non-flash)\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:07:36 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bb98586e-4f75-46be-a626-0b73ebda11eb (non-flash)
Nov 29 03:07:36 user nova-compute[75234]: tcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ab278679-6cfd-4c7c-a930-c078843640d3 (non-flash)
Nov 29 03:07:36 user nova-compute[75234]: tcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:920d6cad-25ab-4f3d-941b-8ec841d43184 (non-flash)
Nov 29 03:07:36 user nova-compute[75234]: tcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b1068fd1-6636-43cd-9725-4a37f708b2bc (non-flash)
Nov 29 03:07:36 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:07:36 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] iscsi session list stdout=tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bb98586e-4f75-46be-a626-0b73ebda11eb (non-flash)
Nov 29 03:07:36 user nova-compute[75234]: tcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ab278679-6cfd-4c7c-a930-c078843640d3 (non-flash)
Nov 29 03:07:36 user nova-compute[75234]: tcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:920d6cad-25ab-4f3d-941b-8ec841d43184 (non-flash)
Nov 29 03:07:36 user nova-compute[75234]: tcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b1068fd1-6636-43cd-9725-4a37f708b2bc (non-flash)
Nov 29 03:07:36 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsi_session /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Nov 29 03:07:36 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:b1068fd1-6636-43cd-9725-4a37f708b2bc" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.374s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:87}}
Nov 29 03:07:36 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] Connected to 172.16.0.220:3260 {{(pid=75234) _connect_vol /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:644}}
Nov 29 03:07:36 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] HCTL ('36', '-', '-', 0) found on session 5 with lun 0 {{(pid=75234) get_hctl /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:770}}
Nov 29 03:07:36 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] Scanning host 36 c: -, t: -, l: 0) {{(pid=75234) scan_iscsi /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:795}}
Nov 29 03:07:36 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host36/scan {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:07:36 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host36/scan" returned: 0 in 0.025s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:07:36 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:36 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[bd958a18-92b6-40df-8a1f-1b6832031d80]: (4, ('- - 0', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:07:36 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] Searching for a device in session 5 and hctl ['36', '*', '*', 0] yield: sde {{(pid=75234) device_name_by_hctl /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:789}}
Nov 29 03:07:36 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] Connected to sde using {'target_iqn': 'iqn.2010-10.org.openstack:b1068fd1-6636-43cd-9725-4a37f708b2bc', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'LpdSswjf', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False} {{(pid=75234) _connect_vol /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:672}}
Nov 29 03:07:36 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:36 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:36 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:36 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:36 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:36 user nova-compute[75234]: DEBUG nova.compute.manager [req-33ac7710-7730-4781-bbd1-da56565ece99 req-090dca1e-8e28-4dcd-8a46-50f389594bea service nova] [instance: cb569e8d-e4a5-4076-bb8b-deeefad033e2] Received event network-vif-plugged-2b12ec96-26fa-4974-8a7a-22e85ec20544 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:07:36 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-33ac7710-7730-4781-bbd1-da56565ece99 req-090dca1e-8e28-4dcd-8a46-50f389594bea service nova] Acquiring lock "cb569e8d-e4a5-4076-bb8b-deeefad033e2-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:07:36 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-33ac7710-7730-4781-bbd1-da56565ece99 req-090dca1e-8e28-4dcd-8a46-50f389594bea service nova] Lock "cb569e8d-e4a5-4076-bb8b-deeefad033e2-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:07:36 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-33ac7710-7730-4781-bbd1-da56565ece99 req-090dca1e-8e28-4dcd-8a46-50f389594bea service nova] Lock "cb569e8d-e4a5-4076-bb8b-deeefad033e2-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:07:36 user nova-compute[75234]: DEBUG nova.compute.manager [req-33ac7710-7730-4781-bbd1-da56565ece99 req-090dca1e-8e28-4dcd-8a46-50f389594bea service nova] [instance: cb569e8d-e4a5-4076-bb8b-deeefad033e2] No waiting events found dispatching network-vif-plugged-2b12ec96-26fa-4974-8a7a-22e85ec20544 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:07:36 user nova-compute[75234]: WARNING nova.compute.manager [req-33ac7710-7730-4781-bbd1-da56565ece99 req-090dca1e-8e28-4dcd-8a46-50f389594bea service nova] [instance: cb569e8d-e4a5-4076-bb8b-deeefad033e2] Received unexpected event network-vif-plugged-2b12ec96-26fa-4974-8a7a-22e85ec20544 for instance with vm_state building and task_state spawning.
Nov 29 03:07:36 user nova-compute[75234]: DEBUG nova.compute.manager [req-33ac7710-7730-4781-bbd1-da56565ece99 req-090dca1e-8e28-4dcd-8a46-50f389594bea service nova] [instance: cb569e8d-e4a5-4076-bb8b-deeefad033e2] Received event network-vif-plugged-d7e92753-492b-4fc0-8048-f4792cd3b4b6 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:07:36 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-33ac7710-7730-4781-bbd1-da56565ece99 req-090dca1e-8e28-4dcd-8a46-50f389594bea service nova] Acquiring lock "cb569e8d-e4a5-4076-bb8b-deeefad033e2-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:07:36 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-33ac7710-7730-4781-bbd1-da56565ece99 req-090dca1e-8e28-4dcd-8a46-50f389594bea service nova] Lock "cb569e8d-e4a5-4076-bb8b-deeefad033e2-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:07:36 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-33ac7710-7730-4781-bbd1-da56565ece99 req-090dca1e-8e28-4dcd-8a46-50f389594bea service nova] Lock "cb569e8d-e4a5-4076-bb8b-deeefad033e2-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:07:36 user nova-compute[75234]: DEBUG nova.compute.manager [req-33ac7710-7730-4781-bbd1-da56565ece99 req-090dca1e-8e28-4dcd-8a46-50f389594bea service nova] [instance: cb569e8d-e4a5-4076-bb8b-deeefad033e2] No waiting events found dispatching network-vif-plugged-d7e92753-492b-4fc0-8048-f4792cd3b4b6 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:07:36 user nova-compute[75234]: WARNING nova.compute.manager [req-33ac7710-7730-4781-bbd1-da56565ece99 req-090dca1e-8e28-4dcd-8a46-50f389594bea service nova] [instance: cb569e8d-e4a5-4076-bb8b-deeefad033e2] Received unexpected event network-vif-plugged-d7e92753-492b-4fc0-8048-f4792cd3b4b6 for instance with vm_state building and task_state spawning.
Nov 29 03:07:36 user nova-compute[75234]: DEBUG nova.compute.manager [req-33ac7710-7730-4781-bbd1-da56565ece99 req-090dca1e-8e28-4dcd-8a46-50f389594bea service nova] [instance: cb569e8d-e4a5-4076-bb8b-deeefad033e2] Received event network-vif-plugged-d7e92753-492b-4fc0-8048-f4792cd3b4b6 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:07:36 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-33ac7710-7730-4781-bbd1-da56565ece99 req-090dca1e-8e28-4dcd-8a46-50f389594bea service nova] Acquiring lock "cb569e8d-e4a5-4076-bb8b-deeefad033e2-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:07:36 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-33ac7710-7730-4781-bbd1-da56565ece99 req-090dca1e-8e28-4dcd-8a46-50f389594bea service nova] Lock "cb569e8d-e4a5-4076-bb8b-deeefad033e2-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:07:36 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-33ac7710-7730-4781-bbd1-da56565ece99 req-090dca1e-8e28-4dcd-8a46-50f389594bea service nova] Lock "cb569e8d-e4a5-4076-bb8b-deeefad033e2-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:07:36 user nova-compute[75234]: DEBUG nova.compute.manager [req-33ac7710-7730-4781-bbd1-da56565ece99 req-090dca1e-8e28-4dcd-8a46-50f389594bea service nova] [instance: cb569e8d-e4a5-4076-bb8b-deeefad033e2] No waiting events found dispatching network-vif-plugged-d7e92753-492b-4fc0-8048-f4792cd3b4b6 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:07:36 user nova-compute[75234]: WARNING nova.compute.manager [req-33ac7710-7730-4781-bbd1-da56565ece99 req-090dca1e-8e28-4dcd-8a46-50f389594bea service nova] [instance: cb569e8d-e4a5-4076-bb8b-deeefad033e2] Received unexpected event network-vif-plugged-d7e92753-492b-4fc0-8048-f4792cd3b4b6 for instance with vm_state building and task_state spawning.
Nov 29 03:07:36 user nova-compute[75234]: DEBUG nova.compute.manager [req-33ac7710-7730-4781-bbd1-da56565ece99 req-090dca1e-8e28-4dcd-8a46-50f389594bea service nova] [instance: cb569e8d-e4a5-4076-bb8b-deeefad033e2] Received event network-vif-plugged-06db3012-0042-457a-a3c8-14e3a8d2e89e {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:07:36 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-33ac7710-7730-4781-bbd1-da56565ece99 req-090dca1e-8e28-4dcd-8a46-50f389594bea service nova] Acquiring lock "cb569e8d-e4a5-4076-bb8b-deeefad033e2-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:07:36 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-33ac7710-7730-4781-bbd1-da56565ece99 req-090dca1e-8e28-4dcd-8a46-50f389594bea service nova] Lock "cb569e8d-e4a5-4076-bb8b-deeefad033e2-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:07:36 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-33ac7710-7730-4781-bbd1-da56565ece99 req-090dca1e-8e28-4dcd-8a46-50f389594bea service nova] Lock "cb569e8d-e4a5-4076-bb8b-deeefad033e2-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:07:36 user nova-compute[75234]: DEBUG nova.compute.manager [req-33ac7710-7730-4781-bbd1-da56565ece99 req-090dca1e-8e28-4dcd-8a46-50f389594bea service nova] [instance: cb569e8d-e4a5-4076-bb8b-deeefad033e2] No waiting events found dispatching network-vif-plugged-06db3012-0042-457a-a3c8-14e3a8d2e89e {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:07:36 user nova-compute[75234]: WARNING nova.compute.manager [req-33ac7710-7730-4781-bbd1-da56565ece99 req-090dca1e-8e28-4dcd-8a46-50f389594bea service nova] [instance: cb569e8d-e4a5-4076-bb8b-deeefad033e2] Received unexpected event network-vif-plugged-06db3012-0042-457a-a3c8-14e3a8d2e89e for instance with vm_state building and task_state spawning.
Nov 29 03:07:36 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:36 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:37 user nova-compute[75234]: DEBUG nova.compute.manager [req-d14f0920-f073-496a-8556-9a09f9cf6ce8 req-a0294952-5a67-4c8d-9729-ffebb70c5146 service nova] [instance: cb569e8d-e4a5-4076-bb8b-deeefad033e2] Received event network-vif-plugged-0bd5d735-f566-4a6d-b49f-b24fd403a87a {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:07:37 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-d14f0920-f073-496a-8556-9a09f9cf6ce8 req-a0294952-5a67-4c8d-9729-ffebb70c5146 service nova] Acquiring lock "cb569e8d-e4a5-4076-bb8b-deeefad033e2-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:07:37 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-d14f0920-f073-496a-8556-9a09f9cf6ce8 req-a0294952-5a67-4c8d-9729-ffebb70c5146 service nova] Lock "cb569e8d-e4a5-4076-bb8b-deeefad033e2-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:07:37 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-d14f0920-f073-496a-8556-9a09f9cf6ce8 req-a0294952-5a67-4c8d-9729-ffebb70c5146 service nova] Lock "cb569e8d-e4a5-4076-bb8b-deeefad033e2-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:07:37 user nova-compute[75234]: DEBUG nova.compute.manager [req-d14f0920-f073-496a-8556-9a09f9cf6ce8 req-a0294952-5a67-4c8d-9729-ffebb70c5146 service nova] [instance: cb569e8d-e4a5-4076-bb8b-deeefad033e2] No waiting events found dispatching network-vif-plugged-0bd5d735-f566-4a6d-b49f-b24fd403a87a {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:07:37 user nova-compute[75234]: WARNING nova.compute.manager [req-d14f0920-f073-496a-8556-9a09f9cf6ce8 req-a0294952-5a67-4c8d-9729-ffebb70c5146 service nova] [instance: cb569e8d-e4a5-4076-bb8b-deeefad033e2] Received unexpected event network-vif-plugged-0bd5d735-f566-4a6d-b49f-b24fd403a87a for instance with vm_state building and task_state spawning.
Nov 29 03:07:37 user nova-compute[75234]: DEBUG nova.compute.manager [req-d14f0920-f073-496a-8556-9a09f9cf6ce8 req-a0294952-5a67-4c8d-9729-ffebb70c5146 service nova] [instance: cb569e8d-e4a5-4076-bb8b-deeefad033e2] Received event network-vif-plugged-27e72461-f85b-42c9-8824-0d760b82a725 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:07:37 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-d14f0920-f073-496a-8556-9a09f9cf6ce8 req-a0294952-5a67-4c8d-9729-ffebb70c5146 service nova] Acquiring lock "cb569e8d-e4a5-4076-bb8b-deeefad033e2-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:07:37 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-d14f0920-f073-496a-8556-9a09f9cf6ce8 req-a0294952-5a67-4c8d-9729-ffebb70c5146 service nova] Lock "cb569e8d-e4a5-4076-bb8b-deeefad033e2-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:07:37 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-d14f0920-f073-496a-8556-9a09f9cf6ce8 req-a0294952-5a67-4c8d-9729-ffebb70c5146 service nova] Lock "cb569e8d-e4a5-4076-bb8b-deeefad033e2-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:07:37 user nova-compute[75234]: DEBUG nova.compute.manager [req-d14f0920-f073-496a-8556-9a09f9cf6ce8 req-a0294952-5a67-4c8d-9729-ffebb70c5146 service nova] [instance: cb569e8d-e4a5-4076-bb8b-deeefad033e2] No waiting events found dispatching network-vif-plugged-27e72461-f85b-42c9-8824-0d760b82a725 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:07:37 user nova-compute[75234]: WARNING nova.compute.manager [req-d14f0920-f073-496a-8556-9a09f9cf6ce8 req-a0294952-5a67-4c8d-9729-ffebb70c5146 service nova] [instance: cb569e8d-e4a5-4076-bb8b-deeefad033e2] Received unexpected event network-vif-plugged-27e72461-f85b-42c9-8824-0d760b82a725 for instance with vm_state building and task_state spawning.
Nov 29 03:07:37 user nova-compute[75234]: DEBUG nova.compute.manager [req-d14f0920-f073-496a-8556-9a09f9cf6ce8 req-a0294952-5a67-4c8d-9729-ffebb70c5146 service nova] [instance: cb569e8d-e4a5-4076-bb8b-deeefad033e2] Received event network-vif-plugged-27e72461-f85b-42c9-8824-0d760b82a725 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:07:37 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-d14f0920-f073-496a-8556-9a09f9cf6ce8 req-a0294952-5a67-4c8d-9729-ffebb70c5146 service nova] Acquiring lock "cb569e8d-e4a5-4076-bb8b-deeefad033e2-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:07:37 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-d14f0920-f073-496a-8556-9a09f9cf6ce8 req-a0294952-5a67-4c8d-9729-ffebb70c5146 service nova] Lock "cb569e8d-e4a5-4076-bb8b-deeefad033e2-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:07:37 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-d14f0920-f073-496a-8556-9a09f9cf6ce8 req-a0294952-5a67-4c8d-9729-ffebb70c5146 service nova] Lock "cb569e8d-e4a5-4076-bb8b-deeefad033e2-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:07:37 user nova-compute[75234]: DEBUG nova.compute.manager [req-d14f0920-f073-496a-8556-9a09f9cf6ce8 req-a0294952-5a67-4c8d-9729-ffebb70c5146 service nova] [instance: cb569e8d-e4a5-4076-bb8b-deeefad033e2] No waiting events found dispatching network-vif-plugged-27e72461-f85b-42c9-8824-0d760b82a725 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:07:37 user nova-compute[75234]: WARNING nova.compute.manager [req-d14f0920-f073-496a-8556-9a09f9cf6ce8 req-a0294952-5a67-4c8d-9729-ffebb70c5146 service nova] [instance: cb569e8d-e4a5-4076-bb8b-deeefad033e2] Received unexpected event network-vif-plugged-27e72461-f85b-42c9-8824-0d760b82a725 for instance with vm_state building and task_state spawning.
Nov 29 03:07:37 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:37 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.452s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:87}}
Nov 29 03:07:37 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] <== connect_volume: return (1454ms) {'type': 'block', 'scsi_wwn': '23035626665396437', 'path': '/dev/sde'} {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:203}}
Nov 29 03:07:37 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] [instance: 763a3e14-4563-4dc2-99ae-c340edbb93df] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '23035626665396437', 'path': '/dev/sde'} {{(pid=75234) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Nov 29 03:07:37 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] [instance: 763a3e14-4563-4dc2-99ae-c340edbb93df] Calling os-brick to attach iSCSI Volume {{(pid=75234) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Nov 29 03:07:37 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f8a5050c9a0>, 'connection_properties': {'target_iqn': 'iqn.2010-10.org.openstack:0d2c0c22-2b3a-43db-b9af-e60a2514e48e', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'oOMLrjWB', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:176}}
Nov 29 03:07:37 user nova-compute[75234]: WARNING os_brick.initiator.connectors.base [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Nov 29 03:07:37 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:68}}
Nov 29 03:07:37 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:73}}
Nov 29 03:07:37 user nova-compute[75234]: WARNING os_brick.initiator.connectors.base [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Nov 29 03:07:37 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:0d2c0c22-2b3a-43db-b9af-e60a2514e48e" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:68}}
Nov 29 03:07:37 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:0d2c0c22-2b3a-43db-b9af-e60a2514e48e" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:73}}
Nov 29 03:07:37 user nova-compute[75234]: INFO os_brick.initiator.connectors.iscsi [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Nov 29 03:07:37 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:0d2c0c22-2b3a-43db-b9af-e60a2514e48e -p 172.16.0.220:3260 {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:07:37 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:0d2c0c22-2b3a-43db-b9af-e60a2514e48e -p 172.16.0.220:3260" returned: 21 in 0.016s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:07:37 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[d560e998-62ce-47f7-bab1-38e55f5da4d5]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:07:37 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Nov 29 03:07:37 user nova-compute[75234]:  {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:07:37 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:0d2c0c22-2b3a-43db-b9af-e60a2514e48e -p 172.16.0.220:3260 --interface default --op new {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:07:37 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:0d2c0c22-2b3a-43db-b9af-e60a2514e48e -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.021s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:07:37 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[fe1709e5-407f-4b4e-8763-3af1f0edff10]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:0d2c0c22-2b3a-43db-b9af-e60a2514e48e] added\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:07:37 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:0d2c0c22-2b3a-43db-b9af-e60a2514e48e] added
Nov 29 03:07:37 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:07:37 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:0d2c0c22-2b3a-43db-b9af-e60a2514e48e -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:07:37 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:0d2c0c22-2b3a-43db-b9af-e60a2514e48e -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.017s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:07:37 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[f871c9aa-ffcd-434c-9ff9-c1480b527a68]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:07:37 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:07:37 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:0d2c0c22-2b3a-43db-b9af-e60a2514e48e -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:07:37 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:0d2c0c22-2b3a-43db-b9af-e60a2514e48e -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.016s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:07:37 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[db590c14-2380-400b-949d-89d398c41d92]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:07:37 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:07:37 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:0d2c0c22-2b3a-43db-b9af-e60a2514e48e -p 172.16.0.220:3260 --op update -n node.session.auth.username -v oOMLrjWB {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:07:37 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:0d2c0c22-2b3a-43db-b9af-e60a2514e48e -p 172.16.0.220:3260 --op update -n node.session.auth.username -v oOMLrjWB" returned: 0 in 0.017s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:07:37 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[38791fa7-7a32-4f85-8d15-a463067b0c1a]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:07:37 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'oOMLrjWB'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:07:37 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:0d2c0c22-2b3a-43db-b9af-e60a2514e48e -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:07:37 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:0d2c0c22-2b3a-43db-b9af-e60a2514e48e -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.017s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:07:37 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[f28278b9-6274-4ae7-9ad7-2f8953344ad5]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:07:37 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:07:37 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:07:37 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.025s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:07:37 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[cb95aba8-1fc5-4be4-8e73-5233a72cd575]: (4, ('tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bb98586e-4f75-46be-a626-0b73ebda11eb (non-flash)\ntcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ab278679-6cfd-4c7c-a930-c078843640d3 (non-flash)\ntcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:920d6cad-25ab-4f3d-941b-8ec841d43184 (non-flash)\ntcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b1068fd1-6636-43cd-9725-4a37f708b2bc (non-flash)\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:07:37 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bb98586e-4f75-46be-a626-0b73ebda11eb (non-flash)
Nov 29 03:07:37 user nova-compute[75234]: tcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ab278679-6cfd-4c7c-a930-c078843640d3 (non-flash)
Nov 29 03:07:37 user nova-compute[75234]: tcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:920d6cad-25ab-4f3d-941b-8ec841d43184 (non-flash)
Nov 29 03:07:37 user nova-compute[75234]: tcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b1068fd1-6636-43cd-9725-4a37f708b2bc (non-flash)
Nov 29 03:07:37 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:07:37 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] iscsi session list stdout=tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bb98586e-4f75-46be-a626-0b73ebda11eb (non-flash)
Nov 29 03:07:37 user nova-compute[75234]: tcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ab278679-6cfd-4c7c-a930-c078843640d3 (non-flash)
Nov 29 03:07:37 user nova-compute[75234]: tcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:920d6cad-25ab-4f3d-941b-8ec841d43184 (non-flash)
Nov 29 03:07:37 user nova-compute[75234]: tcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b1068fd1-6636-43cd-9725-4a37f708b2bc (non-flash)
Nov 29 03:07:37 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsi_session /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Nov 29 03:07:37 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:0d2c0c22-2b3a-43db-b9af-e60a2514e48e -p 172.16.0.220:3260 --login {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:07:37 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:0d2c0c22-2b3a-43db-b9af-e60a2514e48e -p 172.16.0.220:3260 --login" returned: 0 in 0.072s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:07:37 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[d124a68f-af5a-46d9-9108-a6976e31c6cc]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:0d2c0c22-2b3a-43db-b9af-e60a2514e48e, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:0d2c0c22-2b3a-43db-b9af-e60a2514e48e, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:07:37 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:37 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:0d2c0c22-2b3a-43db-b9af-e60a2514e48e, portal: 172.16.0.220,3260]
Nov 29 03:07:37 user nova-compute[75234]: Login to [iface: default, target: iqn.2010-10.org.openstack:0d2c0c22-2b3a-43db-b9af-e60a2514e48e, portal: 172.16.0.220,3260] successful.
Nov 29 03:07:37 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:07:37 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:0d2c0c22-2b3a-43db-b9af-e60a2514e48e -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:07:37 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:37 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:0d2c0c22-2b3a-43db-b9af-e60a2514e48e -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.018s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:07:37 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[d1731abf-c4cc-45b3-a151-943b0df08264]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:07:37 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:07:37 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:07:37 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:37 user nova-compute[75234]: DEBUG nova.compute.manager [req-77a9645f-f732-490c-ad84-d7195c32966b req-7bed93c2-816b-4832-afcd-94ae7af611c4 service nova] [instance: cb569e8d-e4a5-4076-bb8b-deeefad033e2] Received event network-vif-plugged-026f606a-534c-4cdd-992a-9f33268b374f {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:07:37 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-77a9645f-f732-490c-ad84-d7195c32966b req-7bed93c2-816b-4832-afcd-94ae7af611c4 service nova] Acquiring lock "cb569e8d-e4a5-4076-bb8b-deeefad033e2-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:07:37 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-77a9645f-f732-490c-ad84-d7195c32966b req-7bed93c2-816b-4832-afcd-94ae7af611c4 service nova] Lock "cb569e8d-e4a5-4076-bb8b-deeefad033e2-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:07:37 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-77a9645f-f732-490c-ad84-d7195c32966b req-7bed93c2-816b-4832-afcd-94ae7af611c4 service nova] Lock "cb569e8d-e4a5-4076-bb8b-deeefad033e2-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:07:37 user nova-compute[75234]: DEBUG nova.compute.manager [req-77a9645f-f732-490c-ad84-d7195c32966b req-7bed93c2-816b-4832-afcd-94ae7af611c4 service nova] [instance: cb569e8d-e4a5-4076-bb8b-deeefad033e2] No waiting events found dispatching network-vif-plugged-026f606a-534c-4cdd-992a-9f33268b374f {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:07:37 user nova-compute[75234]: WARNING nova.compute.manager [req-77a9645f-f732-490c-ad84-d7195c32966b req-7bed93c2-816b-4832-afcd-94ae7af611c4 service nova] [instance: cb569e8d-e4a5-4076-bb8b-deeefad033e2] Received unexpected event network-vif-plugged-026f606a-534c-4cdd-992a-9f33268b374f for instance with vm_state building and task_state spawning.
Nov 29 03:07:37 user nova-compute[75234]: DEBUG nova.compute.manager [req-77a9645f-f732-490c-ad84-d7195c32966b req-7bed93c2-816b-4832-afcd-94ae7af611c4 service nova] [instance: cb569e8d-e4a5-4076-bb8b-deeefad033e2] Received event network-vif-plugged-67446920-3741-4542-84c6-493388395f26 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:07:37 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-77a9645f-f732-490c-ad84-d7195c32966b req-7bed93c2-816b-4832-afcd-94ae7af611c4 service nova] Acquiring lock "cb569e8d-e4a5-4076-bb8b-deeefad033e2-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:07:37 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-77a9645f-f732-490c-ad84-d7195c32966b req-7bed93c2-816b-4832-afcd-94ae7af611c4 service nova] Lock "cb569e8d-e4a5-4076-bb8b-deeefad033e2-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:07:37 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-77a9645f-f732-490c-ad84-d7195c32966b req-7bed93c2-816b-4832-afcd-94ae7af611c4 service nova] Lock "cb569e8d-e4a5-4076-bb8b-deeefad033e2-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:07:37 user nova-compute[75234]: DEBUG nova.compute.manager [req-77a9645f-f732-490c-ad84-d7195c32966b req-7bed93c2-816b-4832-afcd-94ae7af611c4 service nova] [instance: cb569e8d-e4a5-4076-bb8b-deeefad033e2] No waiting events found dispatching network-vif-plugged-67446920-3741-4542-84c6-493388395f26 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:07:37 user nova-compute[75234]: WARNING nova.compute.manager [req-77a9645f-f732-490c-ad84-d7195c32966b req-7bed93c2-816b-4832-afcd-94ae7af611c4 service nova] [instance: cb569e8d-e4a5-4076-bb8b-deeefad033e2] Received unexpected event network-vif-plugged-67446920-3741-4542-84c6-493388395f26 for instance with vm_state building and task_state spawning.
Nov 29 03:07:37 user nova-compute[75234]: DEBUG nova.compute.manager [req-77a9645f-f732-490c-ad84-d7195c32966b req-7bed93c2-816b-4832-afcd-94ae7af611c4 service nova] [instance: cb569e8d-e4a5-4076-bb8b-deeefad033e2] Received event network-vif-plugged-67446920-3741-4542-84c6-493388395f26 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:07:37 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-77a9645f-f732-490c-ad84-d7195c32966b req-7bed93c2-816b-4832-afcd-94ae7af611c4 service nova] Acquiring lock "cb569e8d-e4a5-4076-bb8b-deeefad033e2-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:07:37 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-77a9645f-f732-490c-ad84-d7195c32966b req-7bed93c2-816b-4832-afcd-94ae7af611c4 service nova] Lock "cb569e8d-e4a5-4076-bb8b-deeefad033e2-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:07:37 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-77a9645f-f732-490c-ad84-d7195c32966b req-7bed93c2-816b-4832-afcd-94ae7af611c4 service nova] Lock "cb569e8d-e4a5-4076-bb8b-deeefad033e2-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:07:37 user nova-compute[75234]: DEBUG nova.compute.manager [req-77a9645f-f732-490c-ad84-d7195c32966b req-7bed93c2-816b-4832-afcd-94ae7af611c4 service nova] [instance: cb569e8d-e4a5-4076-bb8b-deeefad033e2] No waiting events found dispatching network-vif-plugged-67446920-3741-4542-84c6-493388395f26 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:07:37 user nova-compute[75234]: WARNING nova.compute.manager [req-77a9645f-f732-490c-ad84-d7195c32966b req-7bed93c2-816b-4832-afcd-94ae7af611c4 service nova] [instance: cb569e8d-e4a5-4076-bb8b-deeefad033e2] Received unexpected event network-vif-plugged-67446920-3741-4542-84c6-493388395f26 for instance with vm_state building and task_state spawning.
Nov 29 03:07:37 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.038s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:07:37 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[e606d630-12d5-4de5-b7fc-5516c8762aab]: (4, ('tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bb98586e-4f75-46be-a626-0b73ebda11eb (non-flash)\ntcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ab278679-6cfd-4c7c-a930-c078843640d3 (non-flash)\ntcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:920d6cad-25ab-4f3d-941b-8ec841d43184 (non-flash)\ntcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b1068fd1-6636-43cd-9725-4a37f708b2bc (non-flash)\ntcp: [6] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0d2c0c22-2b3a-43db-b9af-e60a2514e48e (non-flash)\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:07:37 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bb98586e-4f75-46be-a626-0b73ebda11eb (non-flash)
Nov 29 03:07:37 user nova-compute[75234]: tcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ab278679-6cfd-4c7c-a930-c078843640d3 (non-flash)
Nov 29 03:07:37 user nova-compute[75234]: tcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:920d6cad-25ab-4f3d-941b-8ec841d43184 (non-flash)
Nov 29 03:07:37 user nova-compute[75234]: tcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b1068fd1-6636-43cd-9725-4a37f708b2bc (non-flash)
Nov 29 03:07:37 user nova-compute[75234]: tcp: [6] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0d2c0c22-2b3a-43db-b9af-e60a2514e48e (non-flash)
Nov 29 03:07:37 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:07:37 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] iscsi session list stdout=tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bb98586e-4f75-46be-a626-0b73ebda11eb (non-flash)
Nov 29 03:07:37 user nova-compute[75234]: tcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ab278679-6cfd-4c7c-a930-c078843640d3 (non-flash)
Nov 29 03:07:37 user nova-compute[75234]: tcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:920d6cad-25ab-4f3d-941b-8ec841d43184 (non-flash)
Nov 29 03:07:37 user nova-compute[75234]: tcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b1068fd1-6636-43cd-9725-4a37f708b2bc (non-flash)
Nov 29 03:07:37 user nova-compute[75234]: tcp: [6] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0d2c0c22-2b3a-43db-b9af-e60a2514e48e (non-flash)
Nov 29 03:07:37 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsi_session /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Nov 29 03:07:37 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:0d2c0c22-2b3a-43db-b9af-e60a2514e48e" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.315s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:87}}
Nov 29 03:07:37 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] Connected to 172.16.0.220:3260 {{(pid=75234) _connect_vol /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:644}}
Nov 29 03:07:37 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] HCTL ('37', '-', '-', 0) found on session 6 with lun 0 {{(pid=75234) get_hctl /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:770}}
Nov 29 03:07:37 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] Scanning host 37 c: -, t: -, l: 0) {{(pid=75234) scan_iscsi /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:795}}
Nov 29 03:07:37 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host37/scan {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:07:37 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host37/scan" returned: 0 in 0.017s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:07:37 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[78e9bf9d-a2ad-469e-b535-784c9534e16d]: (4, ('- - 0', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:07:37 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] Searching for a device in session 6 and hctl ['37', '*', '*', 0] yield: sdf {{(pid=75234) device_name_by_hctl /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:789}}
Nov 29 03:07:37 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] Connected to sdf using {'target_iqn': 'iqn.2010-10.org.openstack:0d2c0c22-2b3a-43db-b9af-e60a2514e48e', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'oOMLrjWB', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False} {{(pid=75234) _connect_vol /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:672}}
Nov 29 03:07:37 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:37 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:38 user nova-compute[75234]: DEBUG nova.compute.manager [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] [instance: cb569e8d-e4a5-4076-bb8b-deeefad033e2] Instance event wait completed in 0 seconds for  {{(pid=75234) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Nov 29 03:07:38 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] [instance: cb569e8d-e4a5-4076-bb8b-deeefad033e2] Guest created on hypervisor {{(pid=75234) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4421}}
Nov 29 03:07:38 user nova-compute[75234]: INFO nova.virt.libvirt.driver [-] [instance: cb569e8d-e4a5-4076-bb8b-deeefad033e2] Instance spawned successfully.
Nov 29 03:07:38 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] [instance: cb569e8d-e4a5-4076-bb8b-deeefad033e2] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:904}}
Nov 29 03:07:38 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] [instance: cb569e8d-e4a5-4076-bb8b-deeefad033e2] Found default for hw_cdrom_bus of ide {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:07:38 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] [instance: cb569e8d-e4a5-4076-bb8b-deeefad033e2] Found default for hw_disk_bus of virtio {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:07:38 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] [instance: cb569e8d-e4a5-4076-bb8b-deeefad033e2] Found default for hw_input_bus of None {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:07:38 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] [instance: cb569e8d-e4a5-4076-bb8b-deeefad033e2] Found default for hw_pointer_model of None {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:07:38 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] [instance: cb569e8d-e4a5-4076-bb8b-deeefad033e2] Found default for hw_video_model of virtio {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:07:38 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] [instance: cb569e8d-e4a5-4076-bb8b-deeefad033e2] Found default for hw_vif_model of virtio {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:07:38 user nova-compute[75234]: INFO nova.compute.manager [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] [instance: cb569e8d-e4a5-4076-bb8b-deeefad033e2] Took 11.12 seconds to spawn the instance on the hypervisor.
Nov 29 03:07:38 user nova-compute[75234]: DEBUG nova.compute.manager [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] [instance: cb569e8d-e4a5-4076-bb8b-deeefad033e2] Checking state {{(pid=75234) _get_power_state /opt/stack/nova/nova/compute/manager.py:1782}}
Nov 29 03:07:38 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.354s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:87}}
Nov 29 03:07:38 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] <== connect_volume: return (1355ms) {'type': 'block', 'scsi_wwn': '26363346535303333', 'path': '/dev/sdf'} {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:203}}
Nov 29 03:07:38 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] [instance: 763a3e14-4563-4dc2-99ae-c340edbb93df] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '26363346535303333', 'path': '/dev/sdf'} {{(pid=75234) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Nov 29 03:07:38 user nova-compute[75234]: INFO nova.compute.manager [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] [instance: cb569e8d-e4a5-4076-bb8b-deeefad033e2] Took 39.29 seconds to build instance.
Nov 29 03:07:38 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-3d01211a-449e-4c62-97f6-28f541ac380d tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] Lock "cb569e8d-e4a5-4076-bb8b-deeefad033e2" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 39.415s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:07:38 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] [instance: 763a3e14-4563-4dc2-99ae-c340edbb93df] Calling os-brick to attach iSCSI Volume {{(pid=75234) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Nov 29 03:07:38 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f8a5050c9a0>, 'connection_properties': {'target_iqn': 'iqn.2010-10.org.openstack:5fcd66fc-a479-4613-b1f8-e6ef12ca470a', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'lbfrXqMs', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:176}}
Nov 29 03:07:38 user nova-compute[75234]: WARNING os_brick.initiator.connectors.base [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Nov 29 03:07:38 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:68}}
Nov 29 03:07:38 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:73}}
Nov 29 03:07:38 user nova-compute[75234]: WARNING os_brick.initiator.connectors.base [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Nov 29 03:07:38 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:5fcd66fc-a479-4613-b1f8-e6ef12ca470a" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:68}}
Nov 29 03:07:38 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:5fcd66fc-a479-4613-b1f8-e6ef12ca470a" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:73}}
Nov 29 03:07:38 user nova-compute[75234]: INFO os_brick.initiator.connectors.iscsi [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Nov 29 03:07:38 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:5fcd66fc-a479-4613-b1f8-e6ef12ca470a -p 172.16.0.220:3260 {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:07:38 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:5fcd66fc-a479-4613-b1f8-e6ef12ca470a -p 172.16.0.220:3260" returned: 21 in 0.017s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:07:38 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[ab9b8a52-4585-451e-ab68-77588888fc76]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:07:38 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Nov 29 03:07:38 user nova-compute[75234]:  {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:07:38 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:5fcd66fc-a479-4613-b1f8-e6ef12ca470a -p 172.16.0.220:3260 --interface default --op new {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:07:38 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:5fcd66fc-a479-4613-b1f8-e6ef12ca470a -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.019s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:07:38 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[02f8281d-9072-452e-90ae-63f64d67c4a2]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:5fcd66fc-a479-4613-b1f8-e6ef12ca470a] added\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:07:38 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:5fcd66fc-a479-4613-b1f8-e6ef12ca470a] added
Nov 29 03:07:38 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:07:38 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:5fcd66fc-a479-4613-b1f8-e6ef12ca470a -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:07:38 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:5fcd66fc-a479-4613-b1f8-e6ef12ca470a -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.017s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:07:38 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[2bcba1e3-d22e-4695-93d0-8ad69b7c7bf0]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:07:38 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:07:38 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:5fcd66fc-a479-4613-b1f8-e6ef12ca470a -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:07:38 user nova-compute[75234]: DEBUG nova.compute.manager [req-a948c520-2c4a-4f5e-864d-d969f5140dfc req-298b5b8b-1df5-49e1-9500-1f37da93bf24 service nova] [instance: cb569e8d-e4a5-4076-bb8b-deeefad033e2] Received event network-vif-plugged-06db3012-0042-457a-a3c8-14e3a8d2e89e {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:07:38 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-a948c520-2c4a-4f5e-864d-d969f5140dfc req-298b5b8b-1df5-49e1-9500-1f37da93bf24 service nova] Acquiring lock "cb569e8d-e4a5-4076-bb8b-deeefad033e2-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:07:38 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-a948c520-2c4a-4f5e-864d-d969f5140dfc req-298b5b8b-1df5-49e1-9500-1f37da93bf24 service nova] Lock "cb569e8d-e4a5-4076-bb8b-deeefad033e2-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:07:38 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-a948c520-2c4a-4f5e-864d-d969f5140dfc req-298b5b8b-1df5-49e1-9500-1f37da93bf24 service nova] Lock "cb569e8d-e4a5-4076-bb8b-deeefad033e2-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:07:38 user nova-compute[75234]: DEBUG nova.compute.manager [req-a948c520-2c4a-4f5e-864d-d969f5140dfc req-298b5b8b-1df5-49e1-9500-1f37da93bf24 service nova] [instance: cb569e8d-e4a5-4076-bb8b-deeefad033e2] No waiting events found dispatching network-vif-plugged-06db3012-0042-457a-a3c8-14e3a8d2e89e {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:07:38 user nova-compute[75234]: WARNING nova.compute.manager [req-a948c520-2c4a-4f5e-864d-d969f5140dfc req-298b5b8b-1df5-49e1-9500-1f37da93bf24 service nova] [instance: cb569e8d-e4a5-4076-bb8b-deeefad033e2] Received unexpected event network-vif-plugged-06db3012-0042-457a-a3c8-14e3a8d2e89e for instance with vm_state active and task_state None.
Nov 29 03:07:38 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:5fcd66fc-a479-4613-b1f8-e6ef12ca470a -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.020s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:07:38 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[aba3594d-3eec-484c-98ef-f67b6a1cecff]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:07:38 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:07:38 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:5fcd66fc-a479-4613-b1f8-e6ef12ca470a -p 172.16.0.220:3260 --op update -n node.session.auth.username -v lbfrXqMs {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:07:38 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:5fcd66fc-a479-4613-b1f8-e6ef12ca470a -p 172.16.0.220:3260 --op update -n node.session.auth.username -v lbfrXqMs" returned: 0 in 0.017s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:07:38 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[d75c3507-9dd6-46ff-be18-efdc7c03da0e]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:07:38 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'lbfrXqMs'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:07:38 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:5fcd66fc-a479-4613-b1f8-e6ef12ca470a -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:07:38 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:5fcd66fc-a479-4613-b1f8-e6ef12ca470a -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.019s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:07:38 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[d65cbe69-4ff9-4712-a975-fa18238b0ae8]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:07:38 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:07:38 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:07:38 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.023s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:07:38 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[7c40d154-bf68-4077-8e65-c9bfec7b822c]: (4, ('tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bb98586e-4f75-46be-a626-0b73ebda11eb (non-flash)\ntcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ab278679-6cfd-4c7c-a930-c078843640d3 (non-flash)\ntcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:920d6cad-25ab-4f3d-941b-8ec841d43184 (non-flash)\ntcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b1068fd1-6636-43cd-9725-4a37f708b2bc (non-flash)\ntcp: [6] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0d2c0c22-2b3a-43db-b9af-e60a2514e48e (non-flash)\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:07:38 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bb98586e-4f75-46be-a626-0b73ebda11eb (non-flash)
Nov 29 03:07:38 user nova-compute[75234]: tcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ab278679-6cfd-4c7c-a930-c078843640d3 (non-flash)
Nov 29 03:07:38 user nova-compute[75234]: tcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:920d6cad-25ab-4f3d-941b-8ec841d43184 (non-flash)
Nov 29 03:07:38 user nova-compute[75234]: tcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b1068fd1-6636-43cd-9725-4a37f708b2bc (non-flash)
Nov 29 03:07:38 user nova-compute[75234]: tcp: [6] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0d2c0c22-2b3a-43db-b9af-e60a2514e48e (non-flash)
Nov 29 03:07:38 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:07:38 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] iscsi session list stdout=tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bb98586e-4f75-46be-a626-0b73ebda11eb (non-flash)
Nov 29 03:07:38 user nova-compute[75234]: tcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ab278679-6cfd-4c7c-a930-c078843640d3 (non-flash)
Nov 29 03:07:38 user nova-compute[75234]: tcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:920d6cad-25ab-4f3d-941b-8ec841d43184 (non-flash)
Nov 29 03:07:38 user nova-compute[75234]: tcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b1068fd1-6636-43cd-9725-4a37f708b2bc (non-flash)
Nov 29 03:07:38 user nova-compute[75234]: tcp: [6] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0d2c0c22-2b3a-43db-b9af-e60a2514e48e (non-flash)
Nov 29 03:07:38 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsi_session /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Nov 29 03:07:38 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:5fcd66fc-a479-4613-b1f8-e6ef12ca470a -p 172.16.0.220:3260 --login {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:07:39 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:5fcd66fc-a479-4613-b1f8-e6ef12ca470a -p 172.16.0.220:3260 --login" returned: 0 in 0.089s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:07:39 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[62169623-a359-40e2-8847-18f2392d1973]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:5fcd66fc-a479-4613-b1f8-e6ef12ca470a, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:5fcd66fc-a479-4613-b1f8-e6ef12ca470a, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:07:39 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:5fcd66fc-a479-4613-b1f8-e6ef12ca470a, portal: 172.16.0.220,3260]
Nov 29 03:07:39 user nova-compute[75234]: Login to [iface: default, target: iqn.2010-10.org.openstack:5fcd66fc-a479-4613-b1f8-e6ef12ca470a, portal: 172.16.0.220,3260] successful.
Nov 29 03:07:39 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:07:39 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:5fcd66fc-a479-4613-b1f8-e6ef12ca470a -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:07:39 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:5fcd66fc-a479-4613-b1f8-e6ef12ca470a -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.017s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:07:39 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[b49ac7d2-2f5f-436e-9331-df51f3575ed3]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:07:39 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:07:39 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:07:39 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.025s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:07:39 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[373023b8-64d0-4ade-8f23-8aa11f3d32ac]: (4, ('tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bb98586e-4f75-46be-a626-0b73ebda11eb (non-flash)\ntcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ab278679-6cfd-4c7c-a930-c078843640d3 (non-flash)\ntcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:920d6cad-25ab-4f3d-941b-8ec841d43184 (non-flash)\ntcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b1068fd1-6636-43cd-9725-4a37f708b2bc (non-flash)\ntcp: [6] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0d2c0c22-2b3a-43db-b9af-e60a2514e48e (non-flash)\ntcp: [7] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5fcd66fc-a479-4613-b1f8-e6ef12ca470a (non-flash)\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:07:39 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bb98586e-4f75-46be-a626-0b73ebda11eb (non-flash)
Nov 29 03:07:39 user nova-compute[75234]: tcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ab278679-6cfd-4c7c-a930-c078843640d3 (non-flash)
Nov 29 03:07:39 user nova-compute[75234]: tcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:920d6cad-25ab-4f3d-941b-8ec841d43184 (non-flash)
Nov 29 03:07:39 user nova-compute[75234]: tcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b1068fd1-6636-43cd-9725-4a37f708b2bc (non-flash)
Nov 29 03:07:39 user nova-compute[75234]: tcp: [6] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0d2c0c22-2b3a-43db-b9af-e60a2514e48e (non-flash)
Nov 29 03:07:39 user nova-compute[75234]: tcp: [7] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5fcd66fc-a479-4613-b1f8-e6ef12ca470a (non-flash)
Nov 29 03:07:39 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:07:39 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] iscsi session list stdout=tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bb98586e-4f75-46be-a626-0b73ebda11eb (non-flash)
Nov 29 03:07:39 user nova-compute[75234]: tcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ab278679-6cfd-4c7c-a930-c078843640d3 (non-flash)
Nov 29 03:07:39 user nova-compute[75234]: tcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:920d6cad-25ab-4f3d-941b-8ec841d43184 (non-flash)
Nov 29 03:07:39 user nova-compute[75234]: tcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b1068fd1-6636-43cd-9725-4a37f708b2bc (non-flash)
Nov 29 03:07:39 user nova-compute[75234]: tcp: [6] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0d2c0c22-2b3a-43db-b9af-e60a2514e48e (non-flash)
Nov 29 03:07:39 user nova-compute[75234]: tcp: [7] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5fcd66fc-a479-4613-b1f8-e6ef12ca470a (non-flash)
Nov 29 03:07:39 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsi_session /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Nov 29 03:07:39 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:5fcd66fc-a479-4613-b1f8-e6ef12ca470a" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.305s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:87}}
Nov 29 03:07:39 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] Connected to 172.16.0.220:3260 {{(pid=75234) _connect_vol /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:644}}
Nov 29 03:07:39 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] HCTL ('38', '-', '-', 0) found on session 7 with lun 0 {{(pid=75234) get_hctl /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:770}}
Nov 29 03:07:39 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] Scanning host 38 c: -, t: -, l: 0) {{(pid=75234) scan_iscsi /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:795}}
Nov 29 03:07:39 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host38/scan {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:07:39 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host38/scan" returned: 0 in 0.020s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:07:39 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[694e672a-5552-4dc6-844b-96b4c48880b1]: (4, ('- - 0', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:07:39 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] Searching for a device in session 7 and hctl ['38', '*', '*', 0] yield: sdg {{(pid=75234) device_name_by_hctl /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:789}}
Nov 29 03:07:39 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] Connected to sdg using {'target_iqn': 'iqn.2010-10.org.openstack:5fcd66fc-a479-4613-b1f8-e6ef12ca470a', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'lbfrXqMs', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False} {{(pid=75234) _connect_vol /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:672}}
Nov 29 03:07:40 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.342s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:87}}
Nov 29 03:07:40 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] <== connect_volume: return (1343ms) {'type': 'block', 'scsi_wwn': '26234353862376531', 'path': '/dev/sdg'} {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:203}}
Nov 29 03:07:40 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] [instance: 763a3e14-4563-4dc2-99ae-c340edbb93df] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '26234353862376531', 'path': '/dev/sdg'} {{(pid=75234) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Nov 29 03:07:40 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='True',created_at=2023-11-29T03:07:09Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-1204199434',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-device-tagging-server-1204199434',id=11,image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBDKKjhuu8m4NOi61Oc3wSdtCCjvt/NxrQN0hvbdlmSXVXySXKH03AWmGvbrcGtt87bsX9Od6uMV6aRJbEi/1xuRd54kkoiFrhpLljDB6Y8GHkcy2CQP4u+I0A4lLOYUDEQ==',key_name='tempest-keypair-1416470280',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='325013d2bedc498e91c2145c2b398803',ramdisk_id='',reservation_id='r-7e1n3ow9',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TaggedBootDevicesTest_v242-1560452734',owner_user_name='tempest-TaggedBootDevicesTest_v242-1560452734-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-11-29T03:07:11Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='f2e1fe315fb14410b5c0448525d8b9f7',uuid=763a3e14-4563-4dc2-99ae-c340edbb93df,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "dc96090f-e855-4d3e-854c-50ea700dc8ea", "address": "fa:16:3e:9e:2b:8a", "network": {"id": "ea3baf6d-ba72-4615-bc0d-604e4fcd8e4e", "bridge": "br-int", "label": "tempest-TaggedBootDevicesTest_v242-1704264625-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "325013d2bedc498e91c2145c2b398803", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapdc96090f-e8", "ovs_interfaceid": "dc96090f-e855-4d3e-854c-50ea700dc8ea", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=75234) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Nov 29 03:07:40 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] Converting VIF {"id": "dc96090f-e855-4d3e-854c-50ea700dc8ea", "address": "fa:16:3e:9e:2b:8a", "network": {"id": "ea3baf6d-ba72-4615-bc0d-604e4fcd8e4e", "bridge": "br-int", "label": "tempest-TaggedBootDevicesTest_v242-1704264625-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "325013d2bedc498e91c2145c2b398803", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapdc96090f-e8", "ovs_interfaceid": "dc96090f-e855-4d3e-854c-50ea700dc8ea", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:07:40 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:9e:2b:8a,bridge_name='br-int',has_traffic_filtering=True,id=dc96090f-e855-4d3e-854c-50ea700dc8ea,network=Network(ea3baf6d-ba72-4615-bc0d-604e4fcd8e4e),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapdc96090f-e8') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:07:40 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='True',created_at=2023-11-29T03:07:09Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-1204199434',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-device-tagging-server-1204199434',id=11,image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBDKKjhuu8m4NOi61Oc3wSdtCCjvt/NxrQN0hvbdlmSXVXySXKH03AWmGvbrcGtt87bsX9Od6uMV6aRJbEi/1xuRd54kkoiFrhpLljDB6Y8GHkcy2CQP4u+I0A4lLOYUDEQ==',key_name='tempest-keypair-1416470280',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='325013d2bedc498e91c2145c2b398803',ramdisk_id='',reservation_id='r-7e1n3ow9',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TaggedBootDevicesTest_v242-1560452734',owner_user_name='tempest-TaggedBootDevicesTest_v242-1560452734-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-11-29T03:07:11Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='f2e1fe315fb14410b5c0448525d8b9f7',uuid=763a3e14-4563-4dc2-99ae-c340edbb93df,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "2027f7a1-36e4-4434-a063-61e817758917", "address": "fa:16:3e:4d:66:94", "network": {"id": "60b448f6-90dc-4dac-b24e-9370262e6dac", "bridge": "br-int", "label": "tempest-device-tagging-net1-431494419", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.80", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "325013d2bedc498e91c2145c2b398803", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2027f7a1-36", "ovs_interfaceid": "2027f7a1-36e4-4434-a063-61e817758917", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=75234) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Nov 29 03:07:40 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] Converting VIF {"id": "2027f7a1-36e4-4434-a063-61e817758917", "address": "fa:16:3e:4d:66:94", "network": {"id": "60b448f6-90dc-4dac-b24e-9370262e6dac", "bridge": "br-int", "label": "tempest-device-tagging-net1-431494419", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.80", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "325013d2bedc498e91c2145c2b398803", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2027f7a1-36", "ovs_interfaceid": "2027f7a1-36e4-4434-a063-61e817758917", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:07:40 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:4d:66:94,bridge_name='br-int',has_traffic_filtering=True,id=2027f7a1-36e4-4434-a063-61e817758917,network=Network(60b448f6-90dc-4dac-b24e-9370262e6dac),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=True,vif_name='tap2027f7a1-36') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:07:40 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='True',created_at=2023-11-29T03:07:09Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-1204199434',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-device-tagging-server-1204199434',id=11,image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBDKKjhuu8m4NOi61Oc3wSdtCCjvt/NxrQN0hvbdlmSXVXySXKH03AWmGvbrcGtt87bsX9Od6uMV6aRJbEi/1xuRd54kkoiFrhpLljDB6Y8GHkcy2CQP4u+I0A4lLOYUDEQ==',key_name='tempest-keypair-1416470280',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='325013d2bedc498e91c2145c2b398803',ramdisk_id='',reservation_id='r-7e1n3ow9',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TaggedBootDevicesTest_v242-1560452734',owner_user_name='tempest-TaggedBootDevicesTest_v242-1560452734-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-11-29T03:07:11Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='f2e1fe315fb14410b5c0448525d8b9f7',uuid=763a3e14-4563-4dc2-99ae-c340edbb93df,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "de4ad363-fae0-4d1b-9820-5a91a27d630d", "address": "fa:16:3e:94:49:94", "network": {"id": "60b448f6-90dc-4dac-b24e-9370262e6dac", "bridge": "br-int", "label": "tempest-device-tagging-net1-431494419", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.218", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "325013d2bedc498e91c2145c2b398803", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapde4ad363-fa", "ovs_interfaceid": "de4ad363-fae0-4d1b-9820-5a91a27d630d", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=75234) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Nov 29 03:07:40 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] Converting VIF {"id": "de4ad363-fae0-4d1b-9820-5a91a27d630d", "address": "fa:16:3e:94:49:94", "network": {"id": "60b448f6-90dc-4dac-b24e-9370262e6dac", "bridge": "br-int", "label": "tempest-device-tagging-net1-431494419", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.218", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "325013d2bedc498e91c2145c2b398803", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapde4ad363-fa", "ovs_interfaceid": "de4ad363-fae0-4d1b-9820-5a91a27d630d", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:07:40 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:94:49:94,bridge_name='br-int',has_traffic_filtering=True,id=de4ad363-fae0-4d1b-9820-5a91a27d630d,network=Network(60b448f6-90dc-4dac-b24e-9370262e6dac),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=True,vif_name='tapde4ad363-fa') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:07:40 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='True',created_at=2023-11-29T03:07:09Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-1204199434',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-device-tagging-server-1204199434',id=11,image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBDKKjhuu8m4NOi61Oc3wSdtCCjvt/NxrQN0hvbdlmSXVXySXKH03AWmGvbrcGtt87bsX9Od6uMV6aRJbEi/1xuRd54kkoiFrhpLljDB6Y8GHkcy2CQP4u+I0A4lLOYUDEQ==',key_name='tempest-keypair-1416470280',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='325013d2bedc498e91c2145c2b398803',ramdisk_id='',reservation_id='r-7e1n3ow9',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TaggedBootDevicesTest_v242-1560452734',owner_user_name='tempest-TaggedBootDevicesTest_v242-1560452734-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-11-29T03:07:11Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='f2e1fe315fb14410b5c0448525d8b9f7',uuid=763a3e14-4563-4dc2-99ae-c340edbb93df,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "d520a88d-ec02-45b8-adbc-484d38b9c18c", "address": "fa:16:3e:ac:cd:aa", "network": {"id": "60b448f6-90dc-4dac-b24e-9370262e6dac", "bridge": "br-int", "label": "tempest-device-tagging-net1-431494419", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.226", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "325013d2bedc498e91c2145c2b398803", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapd520a88d-ec", "ovs_interfaceid": "d520a88d-ec02-45b8-adbc-484d38b9c18c", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=75234) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Nov 29 03:07:40 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] Converting VIF {"id": "d520a88d-ec02-45b8-adbc-484d38b9c18c", "address": "fa:16:3e:ac:cd:aa", "network": {"id": "60b448f6-90dc-4dac-b24e-9370262e6dac", "bridge": "br-int", "label": "tempest-device-tagging-net1-431494419", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.226", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "325013d2bedc498e91c2145c2b398803", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapd520a88d-ec", "ovs_interfaceid": "d520a88d-ec02-45b8-adbc-484d38b9c18c", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:07:40 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:ac:cd:aa,bridge_name='br-int',has_traffic_filtering=True,id=d520a88d-ec02-45b8-adbc-484d38b9c18c,network=Network(60b448f6-90dc-4dac-b24e-9370262e6dac),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapd520a88d-ec') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:07:40 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='True',created_at=2023-11-29T03:07:09Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-1204199434',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-device-tagging-server-1204199434',id=11,image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBDKKjhuu8m4NOi61Oc3wSdtCCjvt/NxrQN0hvbdlmSXVXySXKH03AWmGvbrcGtt87bsX9Od6uMV6aRJbEi/1xuRd54kkoiFrhpLljDB6Y8GHkcy2CQP4u+I0A4lLOYUDEQ==',key_name='tempest-keypair-1416470280',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='325013d2bedc498e91c2145c2b398803',ramdisk_id='',reservation_id='r-7e1n3ow9',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TaggedBootDevicesTest_v242-1560452734',owner_user_name='tempest-TaggedBootDevicesTest_v242-1560452734-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-11-29T03:07:11Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='f2e1fe315fb14410b5c0448525d8b9f7',uuid=763a3e14-4563-4dc2-99ae-c340edbb93df,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "c50f64cd-b7fa-4580-bb78-bff756a0773a", "address": "fa:16:3e:36:0b:d9", "network": {"id": "60b448f6-90dc-4dac-b24e-9370262e6dac", "bridge": "br-int", "label": "tempest-device-tagging-net1-431494419", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.18", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "325013d2bedc498e91c2145c2b398803", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc50f64cd-b7", "ovs_interfaceid": "c50f64cd-b7fa-4580-bb78-bff756a0773a", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=75234) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Nov 29 03:07:40 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] Converting VIF {"id": "c50f64cd-b7fa-4580-bb78-bff756a0773a", "address": "fa:16:3e:36:0b:d9", "network": {"id": "60b448f6-90dc-4dac-b24e-9370262e6dac", "bridge": "br-int", "label": "tempest-device-tagging-net1-431494419", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.18", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "325013d2bedc498e91c2145c2b398803", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc50f64cd-b7", "ovs_interfaceid": "c50f64cd-b7fa-4580-bb78-bff756a0773a", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:07:40 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:36:0b:d9,bridge_name='br-int',has_traffic_filtering=True,id=c50f64cd-b7fa-4580-bb78-bff756a0773a,network=Network(60b448f6-90dc-4dac-b24e-9370262e6dac),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapc50f64cd-b7') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:07:40 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='True',created_at=2023-11-29T03:07:09Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-1204199434',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-device-tagging-server-1204199434',id=11,image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBDKKjhuu8m4NOi61Oc3wSdtCCjvt/NxrQN0hvbdlmSXVXySXKH03AWmGvbrcGtt87bsX9Od6uMV6aRJbEi/1xuRd54kkoiFrhpLljDB6Y8GHkcy2CQP4u+I0A4lLOYUDEQ==',key_name='tempest-keypair-1416470280',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='325013d2bedc498e91c2145c2b398803',ramdisk_id='',reservation_id='r-7e1n3ow9',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TaggedBootDevicesTest_v242-1560452734',owner_user_name='tempest-TaggedBootDevicesTest_v242-1560452734-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-11-29T03:07:11Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='f2e1fe315fb14410b5c0448525d8b9f7',uuid=763a3e14-4563-4dc2-99ae-c340edbb93df,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "ef8ba969-6186-4b9a-b30f-ec9a70036bab", "address": "fa:16:3e:0f:ed:74", "network": {"id": "31e25520-c84d-451c-9132-ddc8c1e443a6", "bridge": "br-int", "label": "tempest-device-tagging-net2-653330954", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.100", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "325013d2bedc498e91c2145c2b398803", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapef8ba969-61", "ovs_interfaceid": "ef8ba969-6186-4b9a-b30f-ec9a70036bab", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=75234) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Nov 29 03:07:40 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] Converting VIF {"id": "ef8ba969-6186-4b9a-b30f-ec9a70036bab", "address": "fa:16:3e:0f:ed:74", "network": {"id": "31e25520-c84d-451c-9132-ddc8c1e443a6", "bridge": "br-int", "label": "tempest-device-tagging-net2-653330954", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.100", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "325013d2bedc498e91c2145c2b398803", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapef8ba969-61", "ovs_interfaceid": "ef8ba969-6186-4b9a-b30f-ec9a70036bab", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:07:40 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:0f:ed:74,bridge_name='br-int',has_traffic_filtering=True,id=ef8ba969-6186-4b9a-b30f-ec9a70036bab,network=Network(31e25520-c84d-451c-9132-ddc8c1e443a6),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapef8ba969-61') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:07:40 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='True',created_at=2023-11-29T03:07:09Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-1204199434',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-device-tagging-server-1204199434',id=11,image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBDKKjhuu8m4NOi61Oc3wSdtCCjvt/NxrQN0hvbdlmSXVXySXKH03AWmGvbrcGtt87bsX9Od6uMV6aRJbEi/1xuRd54kkoiFrhpLljDB6Y8GHkcy2CQP4u+I0A4lLOYUDEQ==',key_name='tempest-keypair-1416470280',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='325013d2bedc498e91c2145c2b398803',ramdisk_id='',reservation_id='r-7e1n3ow9',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TaggedBootDevicesTest_v242-1560452734',owner_user_name='tempest-TaggedBootDevicesTest_v242-1560452734-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-11-29T03:07:11Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='f2e1fe315fb14410b5c0448525d8b9f7',uuid=763a3e14-4563-4dc2-99ae-c340edbb93df,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "92917a0d-59ee-4167-8693-262a21f6e156", "address": "fa:16:3e:65:95:01", "network": {"id": "31e25520-c84d-451c-9132-ddc8c1e443a6", "bridge": "br-int", "label": "tempest-device-tagging-net2-653330954", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.200", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "325013d2bedc498e91c2145c2b398803", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap92917a0d-59", "ovs_interfaceid": "92917a0d-59ee-4167-8693-262a21f6e156", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=75234) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Nov 29 03:07:40 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] Converting VIF {"id": "92917a0d-59ee-4167-8693-262a21f6e156", "address": "fa:16:3e:65:95:01", "network": {"id": "31e25520-c84d-451c-9132-ddc8c1e443a6", "bridge": "br-int", "label": "tempest-device-tagging-net2-653330954", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.200", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "325013d2bedc498e91c2145c2b398803", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap92917a0d-59", "ovs_interfaceid": "92917a0d-59ee-4167-8693-262a21f6e156", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:07:40 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:65:95:01,bridge_name='br-int',has_traffic_filtering=True,id=92917a0d-59ee-4167-8693-262a21f6e156,network=Network(31e25520-c84d-451c-9132-ddc8c1e443a6),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap92917a0d-59') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:07:40 user nova-compute[75234]: DEBUG nova.objects.instance [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] Lazy-loading 'pci_devices' on Instance uuid 763a3e14-4563-4dc2-99ae-c340edbb93df {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:07:40 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] [instance: 763a3e14-4563-4dc2-99ae-c340edbb93df] End _get_guest_xml xml=<domain type="kvm">
Nov 29 03:07:40 user nova-compute[75234]:   <uuid>763a3e14-4563-4dc2-99ae-c340edbb93df</uuid>
Nov 29 03:07:40 user nova-compute[75234]:   <name>instance-0000000b</name>
Nov 29 03:07:40 user nova-compute[75234]:   <memory>524288</memory>
Nov 29 03:07:40 user nova-compute[75234]:   <vcpu>1</vcpu>
Nov 29 03:07:40 user nova-compute[75234]:   <metadata>
Nov 29 03:07:40 user nova-compute[75234]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Nov 29 03:07:40 user nova-compute[75234]:       <nova:package version="0.0.0"/>
Nov 29 03:07:40 user nova-compute[75234]:       <nova:name>tempest-device-tagging-server-1204199434</nova:name>
Nov 29 03:07:40 user nova-compute[75234]:       <nova:creationTime>2023-11-29 03:07:35</nova:creationTime>
Nov 29 03:07:40 user nova-compute[75234]:       <nova:flavor name="m1.tiny">
Nov 29 03:07:40 user nova-compute[75234]:         <nova:memory>512</nova:memory>
Nov 29 03:07:40 user nova-compute[75234]:         <nova:disk>1</nova:disk>
Nov 29 03:07:40 user nova-compute[75234]:         <nova:swap>0</nova:swap>
Nov 29 03:07:40 user nova-compute[75234]:         <nova:ephemeral>0</nova:ephemeral>
Nov 29 03:07:40 user nova-compute[75234]:         <nova:vcpus>1</nova:vcpus>
Nov 29 03:07:40 user nova-compute[75234]:       </nova:flavor>
Nov 29 03:07:40 user nova-compute[75234]:       <nova:owner>
Nov 29 03:07:40 user nova-compute[75234]:         <nova:user uuid="f2e1fe315fb14410b5c0448525d8b9f7">tempest-TaggedBootDevicesTest_v242-1560452734-project-member</nova:user>
Nov 29 03:07:40 user nova-compute[75234]:         <nova:project uuid="325013d2bedc498e91c2145c2b398803">tempest-TaggedBootDevicesTest_v242-1560452734</nova:project>
Nov 29 03:07:40 user nova-compute[75234]:       </nova:owner>
Nov 29 03:07:40 user nova-compute[75234]:       <nova:root type="image" uuid="96c30ba7-9b73-4a30-8ba4-7138879573b7"/>
Nov 29 03:07:40 user nova-compute[75234]:       <nova:ports>
Nov 29 03:07:40 user nova-compute[75234]:         <nova:port uuid="dc96090f-e855-4d3e-854c-50ea700dc8ea">
Nov 29 03:07:40 user nova-compute[75234]:           <nova:ip type="fixed" address="10.0.0.10" ipVersion="4"/>
Nov 29 03:07:40 user nova-compute[75234]:         </nova:port>
Nov 29 03:07:40 user nova-compute[75234]:         <nova:port uuid="2027f7a1-36e4-4434-a063-61e817758917">
Nov 29 03:07:40 user nova-compute[75234]:           <nova:ip type="fixed" address="10.1.1.80" ipVersion="4"/>
Nov 29 03:07:40 user nova-compute[75234]:         </nova:port>
Nov 29 03:07:40 user nova-compute[75234]:         <nova:port uuid="de4ad363-fae0-4d1b-9820-5a91a27d630d">
Nov 29 03:07:40 user nova-compute[75234]:           <nova:ip type="fixed" address="10.1.1.218" ipVersion="4"/>
Nov 29 03:07:40 user nova-compute[75234]:         </nova:port>
Nov 29 03:07:40 user nova-compute[75234]:         <nova:port uuid="d520a88d-ec02-45b8-adbc-484d38b9c18c">
Nov 29 03:07:40 user nova-compute[75234]:           <nova:ip type="fixed" address="10.1.1.226" ipVersion="4"/>
Nov 29 03:07:40 user nova-compute[75234]:         </nova:port>
Nov 29 03:07:40 user nova-compute[75234]:         <nova:port uuid="c50f64cd-b7fa-4580-bb78-bff756a0773a">
Nov 29 03:07:40 user nova-compute[75234]:           <nova:ip type="fixed" address="10.1.1.18" ipVersion="4"/>
Nov 29 03:07:40 user nova-compute[75234]:         </nova:port>
Nov 29 03:07:40 user nova-compute[75234]:         <nova:port uuid="ef8ba969-6186-4b9a-b30f-ec9a70036bab">
Nov 29 03:07:40 user nova-compute[75234]:           <nova:ip type="fixed" address="10.2.2.100" ipVersion="4"/>
Nov 29 03:07:40 user nova-compute[75234]:         </nova:port>
Nov 29 03:07:40 user nova-compute[75234]:         <nova:port uuid="92917a0d-59ee-4167-8693-262a21f6e156">
Nov 29 03:07:40 user nova-compute[75234]:           <nova:ip type="fixed" address="10.2.2.200" ipVersion="4"/>
Nov 29 03:07:40 user nova-compute[75234]:         </nova:port>
Nov 29 03:07:40 user nova-compute[75234]:       </nova:ports>
Nov 29 03:07:40 user nova-compute[75234]:     </nova:instance>
Nov 29 03:07:40 user nova-compute[75234]:   </metadata>
Nov 29 03:07:40 user nova-compute[75234]:   <sysinfo type="smbios">
Nov 29 03:07:40 user nova-compute[75234]:     <system>
Nov 29 03:07:40 user nova-compute[75234]:       <entry name="manufacturer">OpenStack Foundation</entry>
Nov 29 03:07:40 user nova-compute[75234]:       <entry name="product">OpenStack Nova</entry>
Nov 29 03:07:40 user nova-compute[75234]:       <entry name="version">0.0.0</entry>
Nov 29 03:07:40 user nova-compute[75234]:       <entry name="serial">763a3e14-4563-4dc2-99ae-c340edbb93df</entry>
Nov 29 03:07:40 user nova-compute[75234]:       <entry name="uuid">763a3e14-4563-4dc2-99ae-c340edbb93df</entry>
Nov 29 03:07:40 user nova-compute[75234]:       <entry name="family">Virtual Machine</entry>
Nov 29 03:07:40 user nova-compute[75234]:     </system>
Nov 29 03:07:40 user nova-compute[75234]:   </sysinfo>
Nov 29 03:07:40 user nova-compute[75234]:   <os>
Nov 29 03:07:40 user nova-compute[75234]:     <type arch="x86_64" machine="pc">hvm</type>
Nov 29 03:07:40 user nova-compute[75234]:     <boot dev="hd"/>
Nov 29 03:07:40 user nova-compute[75234]:     <smbios mode="sysinfo"/>
Nov 29 03:07:40 user nova-compute[75234]:   </os>
Nov 29 03:07:40 user nova-compute[75234]:   <features>
Nov 29 03:07:40 user nova-compute[75234]:     <acpi/>
Nov 29 03:07:40 user nova-compute[75234]:     <apic/>
Nov 29 03:07:40 user nova-compute[75234]:     <vmcoreinfo/>
Nov 29 03:07:40 user nova-compute[75234]:   </features>
Nov 29 03:07:40 user nova-compute[75234]:   <clock offset="utc">
Nov 29 03:07:40 user nova-compute[75234]:     <timer name="pit" tickpolicy="delay"/>
Nov 29 03:07:40 user nova-compute[75234]:     <timer name="rtc" tickpolicy="catchup"/>
Nov 29 03:07:40 user nova-compute[75234]:     <timer name="hpet" present="no"/>
Nov 29 03:07:40 user nova-compute[75234]:   </clock>
Nov 29 03:07:40 user nova-compute[75234]:   <cpu mode="custom" match="exact">
Nov 29 03:07:40 user nova-compute[75234]:     <model>Nehalem</model>
Nov 29 03:07:40 user nova-compute[75234]:     <topology sockets="1" cores="1" threads="1"/>
Nov 29 03:07:40 user nova-compute[75234]:   </cpu>
Nov 29 03:07:40 user nova-compute[75234]:   <devices>
Nov 29 03:07:40 user nova-compute[75234]:     <disk type="file" device="cdrom">
Nov 29 03:07:40 user nova-compute[75234]:       <driver name="qemu" type="raw" cache="none"/>
Nov 29 03:07:40 user nova-compute[75234]:       <source file="/opt/stack/data/nova/instances/763a3e14-4563-4dc2-99ae-c340edbb93df/disk.config"/>
Nov 29 03:07:40 user nova-compute[75234]:       <target dev="hda" bus="ide"/>
Nov 29 03:07:40 user nova-compute[75234]:     </disk>
Nov 29 03:07:40 user nova-compute[75234]:     <disk type="block" device="disk">
Nov 29 03:07:40 user nova-compute[75234]:       <driver name="qemu" type="raw" cache="none" io="native"/>
Nov 29 03:07:40 user nova-compute[75234]:       <source dev="/dev/sde"/>
Nov 29 03:07:40 user nova-compute[75234]:       <target dev="vda" bus="virtio"/>
Nov 29 03:07:40 user nova-compute[75234]:       <serial>b1068fd1-6636-43cd-9725-4a37f708b2bc</serial>
Nov 29 03:07:40 user nova-compute[75234]:     </disk>
Nov 29 03:07:40 user nova-compute[75234]:     <disk type="block" device="disk">
Nov 29 03:07:40 user nova-compute[75234]:       <driver name="qemu" type="raw" cache="none" io="native"/>
Nov 29 03:07:40 user nova-compute[75234]:       <source dev="/dev/sdf"/>
Nov 29 03:07:40 user nova-compute[75234]:       <target dev="vdb" bus="virtio"/>
Nov 29 03:07:40 user nova-compute[75234]:       <serial>0d2c0c22-2b3a-43db-b9af-e60a2514e48e</serial>
Nov 29 03:07:40 user nova-compute[75234]:     </disk>
Nov 29 03:07:40 user nova-compute[75234]:     <disk type="block" device="disk">
Nov 29 03:07:40 user nova-compute[75234]:       <driver name="qemu" type="raw" cache="none" io="native"/>
Nov 29 03:07:40 user nova-compute[75234]:       <source dev="/dev/sdg"/>
Nov 29 03:07:40 user nova-compute[75234]:       <target dev="vdc" bus="virtio"/>
Nov 29 03:07:40 user nova-compute[75234]:       <serial>5fcd66fc-a479-4613-b1f8-e6ef12ca470a</serial>
Nov 29 03:07:40 user nova-compute[75234]:     </disk>
Nov 29 03:07:40 user nova-compute[75234]:     <interface type="ethernet">
Nov 29 03:07:40 user nova-compute[75234]:       <mac address="fa:16:3e:9e:2b:8a"/>
Nov 29 03:07:40 user nova-compute[75234]:       <model type="virtio"/>
Nov 29 03:07:40 user nova-compute[75234]:       <mtu size="1442"/>
Nov 29 03:07:40 user nova-compute[75234]:       <target dev="tapdc96090f-e8"/>
Nov 29 03:07:40 user nova-compute[75234]:     </interface>
Nov 29 03:07:40 user nova-compute[75234]:     <interface type="ethernet">
Nov 29 03:07:40 user nova-compute[75234]:       <mac address="fa:16:3e:4d:66:94"/>
Nov 29 03:07:40 user nova-compute[75234]:       <model type="virtio"/>
Nov 29 03:07:40 user nova-compute[75234]:       <mtu size="1442"/>
Nov 29 03:07:40 user nova-compute[75234]:       <target dev="tap2027f7a1-36"/>
Nov 29 03:07:40 user nova-compute[75234]:     </interface>
Nov 29 03:07:40 user nova-compute[75234]:     <interface type="ethernet">
Nov 29 03:07:40 user nova-compute[75234]:       <mac address="fa:16:3e:94:49:94"/>
Nov 29 03:07:40 user nova-compute[75234]:       <model type="virtio"/>
Nov 29 03:07:40 user nova-compute[75234]:       <mtu size="1442"/>
Nov 29 03:07:40 user nova-compute[75234]:       <target dev="tapde4ad363-fa"/>
Nov 29 03:07:40 user nova-compute[75234]:     </interface>
Nov 29 03:07:40 user nova-compute[75234]:     <interface type="ethernet">
Nov 29 03:07:40 user nova-compute[75234]:       <mac address="fa:16:3e:ac:cd:aa"/>
Nov 29 03:07:40 user nova-compute[75234]:       <model type="virtio"/>
Nov 29 03:07:40 user nova-compute[75234]:       <mtu size="1442"/>
Nov 29 03:07:40 user nova-compute[75234]:       <target dev="tapd520a88d-ec"/>
Nov 29 03:07:40 user nova-compute[75234]:     </interface>
Nov 29 03:07:40 user nova-compute[75234]:     <interface type="ethernet">
Nov 29 03:07:40 user nova-compute[75234]:       <mac address="fa:16:3e:36:0b:d9"/>
Nov 29 03:07:40 user nova-compute[75234]:       <model type="virtio"/>
Nov 29 03:07:40 user nova-compute[75234]:       <mtu size="1442"/>
Nov 29 03:07:40 user nova-compute[75234]:       <target dev="tapc50f64cd-b7"/>
Nov 29 03:07:40 user nova-compute[75234]:     </interface>
Nov 29 03:07:40 user nova-compute[75234]:     <interface type="ethernet">
Nov 29 03:07:40 user nova-compute[75234]:       <mac address="fa:16:3e:0f:ed:74"/>
Nov 29 03:07:40 user nova-compute[75234]:       <model type="virtio"/>
Nov 29 03:07:40 user nova-compute[75234]:       <mtu size="1442"/>
Nov 29 03:07:40 user nova-compute[75234]:       <target dev="tapef8ba969-61"/>
Nov 29 03:07:40 user nova-compute[75234]:     </interface>
Nov 29 03:07:40 user nova-compute[75234]:     <interface type="ethernet">
Nov 29 03:07:40 user nova-compute[75234]:       <mac address="fa:16:3e:65:95:01"/>
Nov 29 03:07:40 user nova-compute[75234]:       <model type="virtio"/>
Nov 29 03:07:40 user nova-compute[75234]:       <mtu size="1442"/>
Nov 29 03:07:40 user nova-compute[75234]:       <target dev="tap92917a0d-59"/>
Nov 29 03:07:40 user nova-compute[75234]:     </interface>
Nov 29 03:07:40 user nova-compute[75234]:     <serial type="pty">
Nov 29 03:07:40 user nova-compute[75234]:       <log file="/opt/stack/data/nova/instances/763a3e14-4563-4dc2-99ae-c340edbb93df/console.log" append="off"/>
Nov 29 03:07:40 user nova-compute[75234]:     </serial>
Nov 29 03:07:40 user nova-compute[75234]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Nov 29 03:07:40 user nova-compute[75234]:     <video>
Nov 29 03:07:40 user nova-compute[75234]:       <model type="virtio"/>
Nov 29 03:07:40 user nova-compute[75234]:     </video>
Nov 29 03:07:40 user nova-compute[75234]:     <rng model="virtio">
Nov 29 03:07:40 user nova-compute[75234]:       <backend model="random">/dev/urandom</backend>
Nov 29 03:07:40 user nova-compute[75234]:     </rng>
Nov 29 03:07:40 user nova-compute[75234]:     <controller type="usb" index="0" model="none"/>
Nov 29 03:07:40 user nova-compute[75234]:     <memballoon model="virtio">
Nov 29 03:07:40 user nova-compute[75234]:       <stats period="10"/>
Nov 29 03:07:40 user nova-compute[75234]:     </memballoon>
Nov 29 03:07:40 user nova-compute[75234]:   </devices>
Nov 29 03:07:40 user nova-compute[75234]: </domain>
Nov 29 03:07:40 user nova-compute[75234]:  {{(pid=75234) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7568}}
Nov 29 03:07:40 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='True',created_at=2023-11-29T03:07:09Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-1204199434',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-device-tagging-server-1204199434',id=11,image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBDKKjhuu8m4NOi61Oc3wSdtCCjvt/NxrQN0hvbdlmSXVXySXKH03AWmGvbrcGtt87bsX9Od6uMV6aRJbEi/1xuRd54kkoiFrhpLljDB6Y8GHkcy2CQP4u+I0A4lLOYUDEQ==',key_name='tempest-keypair-1416470280',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='325013d2bedc498e91c2145c2b398803',ramdisk_id='',reservation_id='r-7e1n3ow9',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TaggedBootDevicesTest_v242-1560452734',owner_user_name='tempest-TaggedBootDevicesTest_v242-1560452734-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-11-29T03:07:11Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='f2e1fe315fb14410b5c0448525d8b9f7',uuid=763a3e14-4563-4dc2-99ae-c340edbb93df,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "dc96090f-e855-4d3e-854c-50ea700dc8ea", "address": "fa:16:3e:9e:2b:8a", "network": {"id": "ea3baf6d-ba72-4615-bc0d-604e4fcd8e4e", "bridge": "br-int", "label": "tempest-TaggedBootDevicesTest_v242-1704264625-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "325013d2bedc498e91c2145c2b398803", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapdc96090f-e8", "ovs_interfaceid": "dc96090f-e855-4d3e-854c-50ea700dc8ea", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Nov 29 03:07:40 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] Converting VIF {"id": "dc96090f-e855-4d3e-854c-50ea700dc8ea", "address": "fa:16:3e:9e:2b:8a", "network": {"id": "ea3baf6d-ba72-4615-bc0d-604e4fcd8e4e", "bridge": "br-int", "label": "tempest-TaggedBootDevicesTest_v242-1704264625-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "325013d2bedc498e91c2145c2b398803", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapdc96090f-e8", "ovs_interfaceid": "dc96090f-e855-4d3e-854c-50ea700dc8ea", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:07:40 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:9e:2b:8a,bridge_name='br-int',has_traffic_filtering=True,id=dc96090f-e855-4d3e-854c-50ea700dc8ea,network=Network(ea3baf6d-ba72-4615-bc0d-604e4fcd8e4e),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapdc96090f-e8') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:07:40 user nova-compute[75234]: DEBUG os_vif [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:9e:2b:8a,bridge_name='br-int',has_traffic_filtering=True,id=dc96090f-e855-4d3e-854c-50ea700dc8ea,network=Network(ea3baf6d-ba72-4615-bc0d-604e4fcd8e4e),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapdc96090f-e8') {{(pid=75234) plug /opt/stack/data/venv/lib/python3.10/site-packages/os_vif/__init__.py:76}}
Nov 29 03:07:40 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:40 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:07:40 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Nov 29 03:07:40 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:40 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbCreateCommand(_result=None, table=QoS, columns={'type': 'linux-noop', 'external_ids': {'id': '4fc80074-a000-59ba-99b3-e89fe8abb7bf', '_type': 'linux-noop'}}, row=False) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:07:40 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:40 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:07:40 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:40 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tapdc96090f-e8, may_exist=True, interface_attrs={}) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:07:40 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Port, record=tapdc96090f-e8, col_values=(('qos', UUID('211b1830-9836-48f1-94bd-37b471f0076c')),), if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:07:40 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DbSetCommand(_result=None, table=Interface, record=tapdc96090f-e8, col_values=(('external_ids', {'iface-id': 'dc96090f-e855-4d3e-854c-50ea700dc8ea', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:9e:2b:8a', 'vm-uuid': '763a3e14-4563-4dc2-99ae-c340edbb93df'}),), if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:07:40 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:40 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:07:40 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:40 user nova-compute[75234]: INFO os_vif [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:9e:2b:8a,bridge_name='br-int',has_traffic_filtering=True,id=dc96090f-e855-4d3e-854c-50ea700dc8ea,network=Network(ea3baf6d-ba72-4615-bc0d-604e4fcd8e4e),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapdc96090f-e8')
Nov 29 03:07:40 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='True',created_at=2023-11-29T03:07:09Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-1204199434',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-device-tagging-server-1204199434',id=11,image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBDKKjhuu8m4NOi61Oc3wSdtCCjvt/NxrQN0hvbdlmSXVXySXKH03AWmGvbrcGtt87bsX9Od6uMV6aRJbEi/1xuRd54kkoiFrhpLljDB6Y8GHkcy2CQP4u+I0A4lLOYUDEQ==',key_name='tempest-keypair-1416470280',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='325013d2bedc498e91c2145c2b398803',ramdisk_id='',reservation_id='r-7e1n3ow9',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TaggedBootDevicesTest_v242-1560452734',owner_user_name='tempest-TaggedBootDevicesTest_v242-1560452734-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-11-29T03:07:11Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='f2e1fe315fb14410b5c0448525d8b9f7',uuid=763a3e14-4563-4dc2-99ae-c340edbb93df,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "2027f7a1-36e4-4434-a063-61e817758917", "address": "fa:16:3e:4d:66:94", "network": {"id": "60b448f6-90dc-4dac-b24e-9370262e6dac", "bridge": "br-int", "label": "tempest-device-tagging-net1-431494419", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.80", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "325013d2bedc498e91c2145c2b398803", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2027f7a1-36", "ovs_interfaceid": "2027f7a1-36e4-4434-a063-61e817758917", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}} {{(pid=75234) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Nov 29 03:07:40 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] Converting VIF {"id": "2027f7a1-36e4-4434-a063-61e817758917", "address": "fa:16:3e:4d:66:94", "network": {"id": "60b448f6-90dc-4dac-b24e-9370262e6dac", "bridge": "br-int", "label": "tempest-device-tagging-net1-431494419", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.80", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "325013d2bedc498e91c2145c2b398803", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2027f7a1-36", "ovs_interfaceid": "2027f7a1-36e4-4434-a063-61e817758917", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:07:40 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:4d:66:94,bridge_name='br-int',has_traffic_filtering=True,id=2027f7a1-36e4-4434-a063-61e817758917,network=Network(60b448f6-90dc-4dac-b24e-9370262e6dac),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=True,vif_name='tap2027f7a1-36') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:07:40 user nova-compute[75234]: DEBUG os_vif [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:4d:66:94,bridge_name='br-int',has_traffic_filtering=True,id=2027f7a1-36e4-4434-a063-61e817758917,network=Network(60b448f6-90dc-4dac-b24e-9370262e6dac),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=True,vif_name='tap2027f7a1-36') {{(pid=75234) plug /opt/stack/data/venv/lib/python3.10/site-packages/os_vif/__init__.py:76}}
Nov 29 03:07:40 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:40 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:07:40 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Nov 29 03:07:40 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:40 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbCreateCommand(_result=None, table=QoS, columns={'type': 'linux-noop', 'external_ids': {'id': '37198dfc-6405-50d0-9aa5-5246a28ba6d5', '_type': 'linux-noop'}}, row=False) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:07:40 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:40 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:07:40 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:40 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:40 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tap2027f7a1-36, may_exist=True, interface_attrs={}) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:07:40 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Port, record=tap2027f7a1-36, col_values=(('qos', UUID('34a61084-de2f-469b-b026-fd0630cf2c70')),), if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:07:40 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DbSetCommand(_result=None, table=Interface, record=tap2027f7a1-36, col_values=(('external_ids', {'iface-id': '2027f7a1-36e4-4434-a063-61e817758917', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:4d:66:94', 'vm-uuid': '763a3e14-4563-4dc2-99ae-c340edbb93df'}),), if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:07:40 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:40 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:07:40 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:40 user nova-compute[75234]: INFO os_vif [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:4d:66:94,bridge_name='br-int',has_traffic_filtering=True,id=2027f7a1-36e4-4434-a063-61e817758917,network=Network(60b448f6-90dc-4dac-b24e-9370262e6dac),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=True,vif_name='tap2027f7a1-36')
Nov 29 03:07:40 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='True',created_at=2023-11-29T03:07:09Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-1204199434',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-device-tagging-server-1204199434',id=11,image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBDKKjhuu8m4NOi61Oc3wSdtCCjvt/NxrQN0hvbdlmSXVXySXKH03AWmGvbrcGtt87bsX9Od6uMV6aRJbEi/1xuRd54kkoiFrhpLljDB6Y8GHkcy2CQP4u+I0A4lLOYUDEQ==',key_name='tempest-keypair-1416470280',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='325013d2bedc498e91c2145c2b398803',ramdisk_id='',reservation_id='r-7e1n3ow9',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TaggedBootDevicesTest_v242-1560452734',owner_user_name='tempest-TaggedBootDevicesTest_v242-1560452734-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-11-29T03:07:11Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='f2e1fe315fb14410b5c0448525d8b9f7',uuid=763a3e14-4563-4dc2-99ae-c340edbb93df,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "de4ad363-fae0-4d1b-9820-5a91a27d630d", "address": "fa:16:3e:94:49:94", "network": {"id": "60b448f6-90dc-4dac-b24e-9370262e6dac", "bridge": "br-int", "label": "tempest-device-tagging-net1-431494419", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.218", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "325013d2bedc498e91c2145c2b398803", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapde4ad363-fa", "ovs_interfaceid": "de4ad363-fae0-4d1b-9820-5a91a27d630d", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}} {{(pid=75234) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Nov 29 03:07:40 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] Converting VIF {"id": "de4ad363-fae0-4d1b-9820-5a91a27d630d", "address": "fa:16:3e:94:49:94", "network": {"id": "60b448f6-90dc-4dac-b24e-9370262e6dac", "bridge": "br-int", "label": "tempest-device-tagging-net1-431494419", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.218", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "325013d2bedc498e91c2145c2b398803", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapde4ad363-fa", "ovs_interfaceid": "de4ad363-fae0-4d1b-9820-5a91a27d630d", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:07:40 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:94:49:94,bridge_name='br-int',has_traffic_filtering=True,id=de4ad363-fae0-4d1b-9820-5a91a27d630d,network=Network(60b448f6-90dc-4dac-b24e-9370262e6dac),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=True,vif_name='tapde4ad363-fa') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:07:40 user nova-compute[75234]: DEBUG os_vif [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:94:49:94,bridge_name='br-int',has_traffic_filtering=True,id=de4ad363-fae0-4d1b-9820-5a91a27d630d,network=Network(60b448f6-90dc-4dac-b24e-9370262e6dac),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=True,vif_name='tapde4ad363-fa') {{(pid=75234) plug /opt/stack/data/venv/lib/python3.10/site-packages/os_vif/__init__.py:76}}
Nov 29 03:07:40 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:40 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:07:40 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Nov 29 03:07:40 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:40 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbCreateCommand(_result=None, table=QoS, columns={'type': 'linux-noop', 'external_ids': {'id': '96a902c1-7391-523c-ab84-5aa388ac5196', '_type': 'linux-noop'}}, row=False) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:07:40 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:40 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:07:40 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:40 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:40 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tapde4ad363-fa, may_exist=True, interface_attrs={}) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:07:40 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Port, record=tapde4ad363-fa, col_values=(('qos', UUID('12e6f356-42d4-45e3-9179-eb757fd47734')),), if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:07:40 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DbSetCommand(_result=None, table=Interface, record=tapde4ad363-fa, col_values=(('external_ids', {'iface-id': 'de4ad363-fae0-4d1b-9820-5a91a27d630d', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:94:49:94', 'vm-uuid': '763a3e14-4563-4dc2-99ae-c340edbb93df'}),), if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:07:40 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:40 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:07:40 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:40 user nova-compute[75234]: INFO os_vif [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:94:49:94,bridge_name='br-int',has_traffic_filtering=True,id=de4ad363-fae0-4d1b-9820-5a91a27d630d,network=Network(60b448f6-90dc-4dac-b24e-9370262e6dac),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=True,vif_name='tapde4ad363-fa')
Nov 29 03:07:40 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='True',created_at=2023-11-29T03:07:09Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-1204199434',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-device-tagging-server-1204199434',id=11,image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBDKKjhuu8m4NOi61Oc3wSdtCCjvt/NxrQN0hvbdlmSXVXySXKH03AWmGvbrcGtt87bsX9Od6uMV6aRJbEi/1xuRd54kkoiFrhpLljDB6Y8GHkcy2CQP4u+I0A4lLOYUDEQ==',key_name='tempest-keypair-1416470280',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='325013d2bedc498e91c2145c2b398803',ramdisk_id='',reservation_id='r-7e1n3ow9',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TaggedBootDevicesTest_v242-1560452734',owner_user_name='tempest-TaggedBootDevicesTest_v242-1560452734-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-11-29T03:07:11Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='f2e1fe315fb14410b5c0448525d8b9f7',uuid=763a3e14-4563-4dc2-99ae-c340edbb93df,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "d520a88d-ec02-45b8-adbc-484d38b9c18c", "address": "fa:16:3e:ac:cd:aa", "network": {"id": "60b448f6-90dc-4dac-b24e-9370262e6dac", "bridge": "br-int", "label": "tempest-device-tagging-net1-431494419", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.226", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "325013d2bedc498e91c2145c2b398803", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapd520a88d-ec", "ovs_interfaceid": "d520a88d-ec02-45b8-adbc-484d38b9c18c", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Nov 29 03:07:40 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] Converting VIF {"id": "d520a88d-ec02-45b8-adbc-484d38b9c18c", "address": "fa:16:3e:ac:cd:aa", "network": {"id": "60b448f6-90dc-4dac-b24e-9370262e6dac", "bridge": "br-int", "label": "tempest-device-tagging-net1-431494419", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.226", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "325013d2bedc498e91c2145c2b398803", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapd520a88d-ec", "ovs_interfaceid": "d520a88d-ec02-45b8-adbc-484d38b9c18c", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:07:40 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:ac:cd:aa,bridge_name='br-int',has_traffic_filtering=True,id=d520a88d-ec02-45b8-adbc-484d38b9c18c,network=Network(60b448f6-90dc-4dac-b24e-9370262e6dac),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapd520a88d-ec') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:07:40 user nova-compute[75234]: DEBUG os_vif [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:ac:cd:aa,bridge_name='br-int',has_traffic_filtering=True,id=d520a88d-ec02-45b8-adbc-484d38b9c18c,network=Network(60b448f6-90dc-4dac-b24e-9370262e6dac),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapd520a88d-ec') {{(pid=75234) plug /opt/stack/data/venv/lib/python3.10/site-packages/os_vif/__init__.py:76}}
Nov 29 03:07:40 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:40 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:07:40 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Nov 29 03:07:40 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:40 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbCreateCommand(_result=None, table=QoS, columns={'type': 'linux-noop', 'external_ids': {'id': 'c85eef75-63e1-536d-8d49-57428582c234', '_type': 'linux-noop'}}, row=False) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:07:40 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:40 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:07:40 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:40 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:40 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tapd520a88d-ec, may_exist=True, interface_attrs={}) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:07:40 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Port, record=tapd520a88d-ec, col_values=(('qos', UUID('572725c7-6fbd-4591-afff-1637331469d5')),), if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:07:40 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DbSetCommand(_result=None, table=Interface, record=tapd520a88d-ec, col_values=(('external_ids', {'iface-id': 'd520a88d-ec02-45b8-adbc-484d38b9c18c', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:ac:cd:aa', 'vm-uuid': '763a3e14-4563-4dc2-99ae-c340edbb93df'}),), if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:07:40 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:40 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:07:40 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:40 user nova-compute[75234]: INFO os_vif [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:ac:cd:aa,bridge_name='br-int',has_traffic_filtering=True,id=d520a88d-ec02-45b8-adbc-484d38b9c18c,network=Network(60b448f6-90dc-4dac-b24e-9370262e6dac),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapd520a88d-ec')
Nov 29 03:07:40 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='True',created_at=2023-11-29T03:07:09Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-1204199434',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-device-tagging-server-1204199434',id=11,image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBDKKjhuu8m4NOi61Oc3wSdtCCjvt/NxrQN0hvbdlmSXVXySXKH03AWmGvbrcGtt87bsX9Od6uMV6aRJbEi/1xuRd54kkoiFrhpLljDB6Y8GHkcy2CQP4u+I0A4lLOYUDEQ==',key_name='tempest-keypair-1416470280',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='325013d2bedc498e91c2145c2b398803',ramdisk_id='',reservation_id='r-7e1n3ow9',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TaggedBootDevicesTest_v242-1560452734',owner_user_name='tempest-TaggedBootDevicesTest_v242-1560452734-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-11-29T03:07:11Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='f2e1fe315fb14410b5c0448525d8b9f7',uuid=763a3e14-4563-4dc2-99ae-c340edbb93df,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "c50f64cd-b7fa-4580-bb78-bff756a0773a", "address": "fa:16:3e:36:0b:d9", "network": {"id": "60b448f6-90dc-4dac-b24e-9370262e6dac", "bridge": "br-int", "label": "tempest-device-tagging-net1-431494419", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.18", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "325013d2bedc498e91c2145c2b398803", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc50f64cd-b7", "ovs_interfaceid": "c50f64cd-b7fa-4580-bb78-bff756a0773a", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Nov 29 03:07:40 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] Converting VIF {"id": "c50f64cd-b7fa-4580-bb78-bff756a0773a", "address": "fa:16:3e:36:0b:d9", "network": {"id": "60b448f6-90dc-4dac-b24e-9370262e6dac", "bridge": "br-int", "label": "tempest-device-tagging-net1-431494419", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.18", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "325013d2bedc498e91c2145c2b398803", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc50f64cd-b7", "ovs_interfaceid": "c50f64cd-b7fa-4580-bb78-bff756a0773a", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:07:40 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:36:0b:d9,bridge_name='br-int',has_traffic_filtering=True,id=c50f64cd-b7fa-4580-bb78-bff756a0773a,network=Network(60b448f6-90dc-4dac-b24e-9370262e6dac),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapc50f64cd-b7') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:07:40 user nova-compute[75234]: DEBUG os_vif [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:36:0b:d9,bridge_name='br-int',has_traffic_filtering=True,id=c50f64cd-b7fa-4580-bb78-bff756a0773a,network=Network(60b448f6-90dc-4dac-b24e-9370262e6dac),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapc50f64cd-b7') {{(pid=75234) plug /opt/stack/data/venv/lib/python3.10/site-packages/os_vif/__init__.py:76}}
Nov 29 03:07:40 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:40 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:07:40 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Nov 29 03:07:40 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:40 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbCreateCommand(_result=None, table=QoS, columns={'type': 'linux-noop', 'external_ids': {'id': '8bdf5951-0946-514c-a8a0-3a1cd8e550a2', '_type': 'linux-noop'}}, row=False) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:07:40 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:40 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:07:40 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:40 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:40 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tapc50f64cd-b7, may_exist=True, interface_attrs={}) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:07:40 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Port, record=tapc50f64cd-b7, col_values=(('qos', UUID('13dec504-f0c0-427b-9b1a-501d753e76b5')),), if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:07:40 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DbSetCommand(_result=None, table=Interface, record=tapc50f64cd-b7, col_values=(('external_ids', {'iface-id': 'c50f64cd-b7fa-4580-bb78-bff756a0773a', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:36:0b:d9', 'vm-uuid': '763a3e14-4563-4dc2-99ae-c340edbb93df'}),), if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:07:40 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:40 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:07:40 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:40 user nova-compute[75234]: INFO os_vif [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:36:0b:d9,bridge_name='br-int',has_traffic_filtering=True,id=c50f64cd-b7fa-4580-bb78-bff756a0773a,network=Network(60b448f6-90dc-4dac-b24e-9370262e6dac),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapc50f64cd-b7')
Nov 29 03:07:40 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='True',created_at=2023-11-29T03:07:09Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-1204199434',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-device-tagging-server-1204199434',id=11,image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBDKKjhuu8m4NOi61Oc3wSdtCCjvt/NxrQN0hvbdlmSXVXySXKH03AWmGvbrcGtt87bsX9Od6uMV6aRJbEi/1xuRd54kkoiFrhpLljDB6Y8GHkcy2CQP4u+I0A4lLOYUDEQ==',key_name='tempest-keypair-1416470280',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='325013d2bedc498e91c2145c2b398803',ramdisk_id='',reservation_id='r-7e1n3ow9',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TaggedBootDevicesTest_v242-1560452734',owner_user_name='tempest-TaggedBootDevicesTest_v242-1560452734-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-11-29T03:07:11Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='f2e1fe315fb14410b5c0448525d8b9f7',uuid=763a3e14-4563-4dc2-99ae-c340edbb93df,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "ef8ba969-6186-4b9a-b30f-ec9a70036bab", "address": "fa:16:3e:0f:ed:74", "network": {"id": "31e25520-c84d-451c-9132-ddc8c1e443a6", "bridge": "br-int", "label": "tempest-device-tagging-net2-653330954", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.100", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "325013d2bedc498e91c2145c2b398803", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapef8ba969-61", "ovs_interfaceid": "ef8ba969-6186-4b9a-b30f-ec9a70036bab", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Nov 29 03:07:40 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] Converting VIF {"id": "ef8ba969-6186-4b9a-b30f-ec9a70036bab", "address": "fa:16:3e:0f:ed:74", "network": {"id": "31e25520-c84d-451c-9132-ddc8c1e443a6", "bridge": "br-int", "label": "tempest-device-tagging-net2-653330954", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.100", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "325013d2bedc498e91c2145c2b398803", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapef8ba969-61", "ovs_interfaceid": "ef8ba969-6186-4b9a-b30f-ec9a70036bab", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:07:40 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:0f:ed:74,bridge_name='br-int',has_traffic_filtering=True,id=ef8ba969-6186-4b9a-b30f-ec9a70036bab,network=Network(31e25520-c84d-451c-9132-ddc8c1e443a6),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapef8ba969-61') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:07:40 user nova-compute[75234]: DEBUG os_vif [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:0f:ed:74,bridge_name='br-int',has_traffic_filtering=True,id=ef8ba969-6186-4b9a-b30f-ec9a70036bab,network=Network(31e25520-c84d-451c-9132-ddc8c1e443a6),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapef8ba969-61') {{(pid=75234) plug /opt/stack/data/venv/lib/python3.10/site-packages/os_vif/__init__.py:76}}
Nov 29 03:07:40 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:40 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:07:40 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Nov 29 03:07:40 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:40 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbCreateCommand(_result=None, table=QoS, columns={'type': 'linux-noop', 'external_ids': {'id': 'c3ff1f77-fa29-555c-8c30-fda1715bd578', '_type': 'linux-noop'}}, row=False) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:07:40 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:40 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:07:40 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:40 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:40 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tapef8ba969-61, may_exist=True, interface_attrs={}) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:07:40 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Port, record=tapef8ba969-61, col_values=(('qos', UUID('5470af18-024a-4f42-82dc-92f47a68c7ef')),), if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:07:40 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DbSetCommand(_result=None, table=Interface, record=tapef8ba969-61, col_values=(('external_ids', {'iface-id': 'ef8ba969-6186-4b9a-b30f-ec9a70036bab', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:0f:ed:74', 'vm-uuid': '763a3e14-4563-4dc2-99ae-c340edbb93df'}),), if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:07:40 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:40 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:07:40 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:40 user nova-compute[75234]: INFO os_vif [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:0f:ed:74,bridge_name='br-int',has_traffic_filtering=True,id=ef8ba969-6186-4b9a-b30f-ec9a70036bab,network=Network(31e25520-c84d-451c-9132-ddc8c1e443a6),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapef8ba969-61')
Nov 29 03:07:40 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='True',created_at=2023-11-29T03:07:09Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-1204199434',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-device-tagging-server-1204199434',id=11,image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBDKKjhuu8m4NOi61Oc3wSdtCCjvt/NxrQN0hvbdlmSXVXySXKH03AWmGvbrcGtt87bsX9Od6uMV6aRJbEi/1xuRd54kkoiFrhpLljDB6Y8GHkcy2CQP4u+I0A4lLOYUDEQ==',key_name='tempest-keypair-1416470280',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='325013d2bedc498e91c2145c2b398803',ramdisk_id='',reservation_id='r-7e1n3ow9',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TaggedBootDevicesTest_v242-1560452734',owner_user_name='tempest-TaggedBootDevicesTest_v242-1560452734-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-11-29T03:07:11Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='f2e1fe315fb14410b5c0448525d8b9f7',uuid=763a3e14-4563-4dc2-99ae-c340edbb93df,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "92917a0d-59ee-4167-8693-262a21f6e156", "address": "fa:16:3e:65:95:01", "network": {"id": "31e25520-c84d-451c-9132-ddc8c1e443a6", "bridge": "br-int", "label": "tempest-device-tagging-net2-653330954", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.200", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "325013d2bedc498e91c2145c2b398803", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap92917a0d-59", "ovs_interfaceid": "92917a0d-59ee-4167-8693-262a21f6e156", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Nov 29 03:07:40 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] Converting VIF {"id": "92917a0d-59ee-4167-8693-262a21f6e156", "address": "fa:16:3e:65:95:01", "network": {"id": "31e25520-c84d-451c-9132-ddc8c1e443a6", "bridge": "br-int", "label": "tempest-device-tagging-net2-653330954", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.200", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "325013d2bedc498e91c2145c2b398803", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap92917a0d-59", "ovs_interfaceid": "92917a0d-59ee-4167-8693-262a21f6e156", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:07:40 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:65:95:01,bridge_name='br-int',has_traffic_filtering=True,id=92917a0d-59ee-4167-8693-262a21f6e156,network=Network(31e25520-c84d-451c-9132-ddc8c1e443a6),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap92917a0d-59') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:07:40 user nova-compute[75234]: DEBUG os_vif [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:65:95:01,bridge_name='br-int',has_traffic_filtering=True,id=92917a0d-59ee-4167-8693-262a21f6e156,network=Network(31e25520-c84d-451c-9132-ddc8c1e443a6),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap92917a0d-59') {{(pid=75234) plug /opt/stack/data/venv/lib/python3.10/site-packages/os_vif/__init__.py:76}}
Nov 29 03:07:40 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:40 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:07:40 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Nov 29 03:07:40 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:40 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbCreateCommand(_result=None, table=QoS, columns={'type': 'linux-noop', 'external_ids': {'id': '681a75d8-27c2-506a-95ca-6ea6738dc071', '_type': 'linux-noop'}}, row=False) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:07:40 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:40 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:07:40 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:40 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:40 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tap92917a0d-59, may_exist=True, interface_attrs={}) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:07:40 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Port, record=tap92917a0d-59, col_values=(('qos', UUID('9d2393d9-d024-42ca-94c4-1524cd361a18')),), if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:07:40 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DbSetCommand(_result=None, table=Interface, record=tap92917a0d-59, col_values=(('external_ids', {'iface-id': '92917a0d-59ee-4167-8693-262a21f6e156', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:65:95:01', 'vm-uuid': '763a3e14-4563-4dc2-99ae-c340edbb93df'}),), if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:07:40 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:40 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:07:40 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:40 user nova-compute[75234]: INFO os_vif [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:65:95:01,bridge_name='br-int',has_traffic_filtering=True,id=92917a0d-59ee-4167-8693-262a21f6e156,network=Network(31e25520-c84d-451c-9132-ddc8c1e443a6),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap92917a0d-59')
Nov 29 03:07:40 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] No BDM found with device name hda, not building metadata. {{(pid=75234) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12089}}
Nov 29 03:07:40 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] No BDM found with device name vdc, not building metadata. {{(pid=75234) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12089}}
Nov 29 03:07:40 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] No VIF found with MAC fa:16:3e:9e:2b:8a, not building metadata {{(pid=75234) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Nov 29 03:07:40 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] No VIF found with MAC fa:16:3e:36:0b:d9, not building metadata {{(pid=75234) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Nov 29 03:07:40 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] [instance: 763a3e14-4563-4dc2-99ae-c340edbb93df] Using config drive
Nov 29 03:07:41 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] [instance: 763a3e14-4563-4dc2-99ae-c340edbb93df] Creating config drive at /opt/stack/data/nova/instances/763a3e14-4563-4dc2-99ae-c340edbb93df/disk.config
Nov 29 03:07:41 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] Running cmd (subprocess): genisoimage -o /opt/stack/data/nova/instances/763a3e14-4563-4dc2-99ae-c340edbb93df/disk.config -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 0.0.0 -quiet -J -r -V config-2 /tmp/tmpsn2cf1qc {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:07:41 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] CMD "genisoimage -o /opt/stack/data/nova/instances/763a3e14-4563-4dc2-99ae-c340edbb93df/disk.config -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 0.0.0 -quiet -J -r -V config-2 /tmp/tmpsn2cf1qc" returned: 0 in 0.051s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:07:41 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:42 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:42 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:42 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:42 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:42 user nova-compute[75234]: DEBUG nova.compute.manager [req-a121c19c-3963-4109-ad71-da41f0abbde0 req-88737f94-ff52-4c76-a2af-155b9a1d589a service nova] [instance: 763a3e14-4563-4dc2-99ae-c340edbb93df] Received event network-vif-plugged-dc96090f-e855-4d3e-854c-50ea700dc8ea {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:07:42 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-a121c19c-3963-4109-ad71-da41f0abbde0 req-88737f94-ff52-4c76-a2af-155b9a1d589a service nova] Acquiring lock "763a3e14-4563-4dc2-99ae-c340edbb93df-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:07:42 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-a121c19c-3963-4109-ad71-da41f0abbde0 req-88737f94-ff52-4c76-a2af-155b9a1d589a service nova] Lock "763a3e14-4563-4dc2-99ae-c340edbb93df-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.002s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:07:42 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-a121c19c-3963-4109-ad71-da41f0abbde0 req-88737f94-ff52-4c76-a2af-155b9a1d589a service nova] Lock "763a3e14-4563-4dc2-99ae-c340edbb93df-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:07:42 user nova-compute[75234]: DEBUG nova.compute.manager [req-a121c19c-3963-4109-ad71-da41f0abbde0 req-88737f94-ff52-4c76-a2af-155b9a1d589a service nova] [instance: 763a3e14-4563-4dc2-99ae-c340edbb93df] No waiting events found dispatching network-vif-plugged-dc96090f-e855-4d3e-854c-50ea700dc8ea {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:07:42 user nova-compute[75234]: WARNING nova.compute.manager [req-a121c19c-3963-4109-ad71-da41f0abbde0 req-88737f94-ff52-4c76-a2af-155b9a1d589a service nova] [instance: 763a3e14-4563-4dc2-99ae-c340edbb93df] Received unexpected event network-vif-plugged-dc96090f-e855-4d3e-854c-50ea700dc8ea for instance with vm_state building and task_state spawning.
Nov 29 03:07:42 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:42 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:43 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:43 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:43 user nova-compute[75234]: DEBUG nova.compute.manager [req-6e8eb3b6-2cfe-435a-9ed8-4e46e2c026fd req-7c4f4ea7-e921-4233-b912-39aebda6aea2 service nova] [instance: 763a3e14-4563-4dc2-99ae-c340edbb93df] Received event network-vif-plugged-2027f7a1-36e4-4434-a063-61e817758917 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:07:43 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-6e8eb3b6-2cfe-435a-9ed8-4e46e2c026fd req-7c4f4ea7-e921-4233-b912-39aebda6aea2 service nova] Acquiring lock "763a3e14-4563-4dc2-99ae-c340edbb93df-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:07:43 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-6e8eb3b6-2cfe-435a-9ed8-4e46e2c026fd req-7c4f4ea7-e921-4233-b912-39aebda6aea2 service nova] Lock "763a3e14-4563-4dc2-99ae-c340edbb93df-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:07:43 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-6e8eb3b6-2cfe-435a-9ed8-4e46e2c026fd req-7c4f4ea7-e921-4233-b912-39aebda6aea2 service nova] Lock "763a3e14-4563-4dc2-99ae-c340edbb93df-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:07:43 user nova-compute[75234]: DEBUG nova.compute.manager [req-6e8eb3b6-2cfe-435a-9ed8-4e46e2c026fd req-7c4f4ea7-e921-4233-b912-39aebda6aea2 service nova] [instance: 763a3e14-4563-4dc2-99ae-c340edbb93df] No waiting events found dispatching network-vif-plugged-2027f7a1-36e4-4434-a063-61e817758917 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:07:43 user nova-compute[75234]: WARNING nova.compute.manager [req-6e8eb3b6-2cfe-435a-9ed8-4e46e2c026fd req-7c4f4ea7-e921-4233-b912-39aebda6aea2 service nova] [instance: 763a3e14-4563-4dc2-99ae-c340edbb93df] Received unexpected event network-vif-plugged-2027f7a1-36e4-4434-a063-61e817758917 for instance with vm_state building and task_state spawning.
Nov 29 03:07:43 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:43 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:43 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:43 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:43 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:43 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:43 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:43 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:43 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:43 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:43 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:43 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:43 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:43 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:43 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:44 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:44 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:44 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:44 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:44 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:44 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:44 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:44 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:44 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:44 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:44 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:44 user nova-compute[75234]: DEBUG nova.compute.manager [req-4058677a-5958-4bf8-9c9a-3a916abd7155 req-fbd35a2c-dc73-47db-b8f3-8b983ecf70fc service nova] [instance: 763a3e14-4563-4dc2-99ae-c340edbb93df] Received event network-vif-plugged-dc96090f-e855-4d3e-854c-50ea700dc8ea {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:07:44 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-4058677a-5958-4bf8-9c9a-3a916abd7155 req-fbd35a2c-dc73-47db-b8f3-8b983ecf70fc service nova] Acquiring lock "763a3e14-4563-4dc2-99ae-c340edbb93df-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:07:44 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-4058677a-5958-4bf8-9c9a-3a916abd7155 req-fbd35a2c-dc73-47db-b8f3-8b983ecf70fc service nova] Lock "763a3e14-4563-4dc2-99ae-c340edbb93df-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:07:44 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-4058677a-5958-4bf8-9c9a-3a916abd7155 req-fbd35a2c-dc73-47db-b8f3-8b983ecf70fc service nova] Lock "763a3e14-4563-4dc2-99ae-c340edbb93df-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:07:44 user nova-compute[75234]: DEBUG nova.compute.manager [req-4058677a-5958-4bf8-9c9a-3a916abd7155 req-fbd35a2c-dc73-47db-b8f3-8b983ecf70fc service nova] [instance: 763a3e14-4563-4dc2-99ae-c340edbb93df] No waiting events found dispatching network-vif-plugged-dc96090f-e855-4d3e-854c-50ea700dc8ea {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:07:44 user nova-compute[75234]: WARNING nova.compute.manager [req-4058677a-5958-4bf8-9c9a-3a916abd7155 req-fbd35a2c-dc73-47db-b8f3-8b983ecf70fc service nova] [instance: 763a3e14-4563-4dc2-99ae-c340edbb93df] Received unexpected event network-vif-plugged-dc96090f-e855-4d3e-854c-50ea700dc8ea for instance with vm_state building and task_state spawning.
Nov 29 03:07:44 user nova-compute[75234]: DEBUG nova.compute.manager [req-4058677a-5958-4bf8-9c9a-3a916abd7155 req-fbd35a2c-dc73-47db-b8f3-8b983ecf70fc service nova] [instance: 763a3e14-4563-4dc2-99ae-c340edbb93df] Received event network-vif-plugged-de4ad363-fae0-4d1b-9820-5a91a27d630d {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:07:44 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-4058677a-5958-4bf8-9c9a-3a916abd7155 req-fbd35a2c-dc73-47db-b8f3-8b983ecf70fc service nova] Acquiring lock "763a3e14-4563-4dc2-99ae-c340edbb93df-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:07:44 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-4058677a-5958-4bf8-9c9a-3a916abd7155 req-fbd35a2c-dc73-47db-b8f3-8b983ecf70fc service nova] Lock "763a3e14-4563-4dc2-99ae-c340edbb93df-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:07:44 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-4058677a-5958-4bf8-9c9a-3a916abd7155 req-fbd35a2c-dc73-47db-b8f3-8b983ecf70fc service nova] Lock "763a3e14-4563-4dc2-99ae-c340edbb93df-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:07:44 user nova-compute[75234]: DEBUG nova.compute.manager [req-4058677a-5958-4bf8-9c9a-3a916abd7155 req-fbd35a2c-dc73-47db-b8f3-8b983ecf70fc service nova] [instance: 763a3e14-4563-4dc2-99ae-c340edbb93df] No waiting events found dispatching network-vif-plugged-de4ad363-fae0-4d1b-9820-5a91a27d630d {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:07:44 user nova-compute[75234]: WARNING nova.compute.manager [req-4058677a-5958-4bf8-9c9a-3a916abd7155 req-fbd35a2c-dc73-47db-b8f3-8b983ecf70fc service nova] [instance: 763a3e14-4563-4dc2-99ae-c340edbb93df] Received unexpected event network-vif-plugged-de4ad363-fae0-4d1b-9820-5a91a27d630d for instance with vm_state building and task_state spawning.
Nov 29 03:07:44 user nova-compute[75234]: DEBUG nova.compute.manager [req-4058677a-5958-4bf8-9c9a-3a916abd7155 req-fbd35a2c-dc73-47db-b8f3-8b983ecf70fc service nova] [instance: 763a3e14-4563-4dc2-99ae-c340edbb93df] Received event network-vif-plugged-de4ad363-fae0-4d1b-9820-5a91a27d630d {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:07:44 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-4058677a-5958-4bf8-9c9a-3a916abd7155 req-fbd35a2c-dc73-47db-b8f3-8b983ecf70fc service nova] Acquiring lock "763a3e14-4563-4dc2-99ae-c340edbb93df-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:07:44 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-4058677a-5958-4bf8-9c9a-3a916abd7155 req-fbd35a2c-dc73-47db-b8f3-8b983ecf70fc service nova] Lock "763a3e14-4563-4dc2-99ae-c340edbb93df-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:07:44 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-4058677a-5958-4bf8-9c9a-3a916abd7155 req-fbd35a2c-dc73-47db-b8f3-8b983ecf70fc service nova] Lock "763a3e14-4563-4dc2-99ae-c340edbb93df-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:07:44 user nova-compute[75234]: DEBUG nova.compute.manager [req-4058677a-5958-4bf8-9c9a-3a916abd7155 req-fbd35a2c-dc73-47db-b8f3-8b983ecf70fc service nova] [instance: 763a3e14-4563-4dc2-99ae-c340edbb93df] No waiting events found dispatching network-vif-plugged-de4ad363-fae0-4d1b-9820-5a91a27d630d {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:07:44 user nova-compute[75234]: WARNING nova.compute.manager [req-4058677a-5958-4bf8-9c9a-3a916abd7155 req-fbd35a2c-dc73-47db-b8f3-8b983ecf70fc service nova] [instance: 763a3e14-4563-4dc2-99ae-c340edbb93df] Received unexpected event network-vif-plugged-de4ad363-fae0-4d1b-9820-5a91a27d630d for instance with vm_state building and task_state spawning.
Nov 29 03:07:44 user nova-compute[75234]: DEBUG nova.compute.manager [req-4058677a-5958-4bf8-9c9a-3a916abd7155 req-fbd35a2c-dc73-47db-b8f3-8b983ecf70fc service nova] [instance: 763a3e14-4563-4dc2-99ae-c340edbb93df] Received event network-vif-plugged-d520a88d-ec02-45b8-adbc-484d38b9c18c {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:07:44 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-4058677a-5958-4bf8-9c9a-3a916abd7155 req-fbd35a2c-dc73-47db-b8f3-8b983ecf70fc service nova] Acquiring lock "763a3e14-4563-4dc2-99ae-c340edbb93df-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:07:44 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-4058677a-5958-4bf8-9c9a-3a916abd7155 req-fbd35a2c-dc73-47db-b8f3-8b983ecf70fc service nova] Lock "763a3e14-4563-4dc2-99ae-c340edbb93df-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:07:44 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-4058677a-5958-4bf8-9c9a-3a916abd7155 req-fbd35a2c-dc73-47db-b8f3-8b983ecf70fc service nova] Lock "763a3e14-4563-4dc2-99ae-c340edbb93df-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:07:44 user nova-compute[75234]: DEBUG nova.compute.manager [req-4058677a-5958-4bf8-9c9a-3a916abd7155 req-fbd35a2c-dc73-47db-b8f3-8b983ecf70fc service nova] [instance: 763a3e14-4563-4dc2-99ae-c340edbb93df] No waiting events found dispatching network-vif-plugged-d520a88d-ec02-45b8-adbc-484d38b9c18c {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:07:44 user nova-compute[75234]: WARNING nova.compute.manager [req-4058677a-5958-4bf8-9c9a-3a916abd7155 req-fbd35a2c-dc73-47db-b8f3-8b983ecf70fc service nova] [instance: 763a3e14-4563-4dc2-99ae-c340edbb93df] Received unexpected event network-vif-plugged-d520a88d-ec02-45b8-adbc-484d38b9c18c for instance with vm_state building and task_state spawning.
Nov 29 03:07:44 user nova-compute[75234]: DEBUG nova.compute.manager [req-4058677a-5958-4bf8-9c9a-3a916abd7155 req-fbd35a2c-dc73-47db-b8f3-8b983ecf70fc service nova] [instance: 763a3e14-4563-4dc2-99ae-c340edbb93df] Received event network-vif-plugged-d520a88d-ec02-45b8-adbc-484d38b9c18c {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:07:44 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-4058677a-5958-4bf8-9c9a-3a916abd7155 req-fbd35a2c-dc73-47db-b8f3-8b983ecf70fc service nova] Acquiring lock "763a3e14-4563-4dc2-99ae-c340edbb93df-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:07:44 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-4058677a-5958-4bf8-9c9a-3a916abd7155 req-fbd35a2c-dc73-47db-b8f3-8b983ecf70fc service nova] Lock "763a3e14-4563-4dc2-99ae-c340edbb93df-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:07:44 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-4058677a-5958-4bf8-9c9a-3a916abd7155 req-fbd35a2c-dc73-47db-b8f3-8b983ecf70fc service nova] Lock "763a3e14-4563-4dc2-99ae-c340edbb93df-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:07:44 user nova-compute[75234]: DEBUG nova.compute.manager [req-4058677a-5958-4bf8-9c9a-3a916abd7155 req-fbd35a2c-dc73-47db-b8f3-8b983ecf70fc service nova] [instance: 763a3e14-4563-4dc2-99ae-c340edbb93df] No waiting events found dispatching network-vif-plugged-d520a88d-ec02-45b8-adbc-484d38b9c18c {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:07:44 user nova-compute[75234]: WARNING nova.compute.manager [req-4058677a-5958-4bf8-9c9a-3a916abd7155 req-fbd35a2c-dc73-47db-b8f3-8b983ecf70fc service nova] [instance: 763a3e14-4563-4dc2-99ae-c340edbb93df] Received unexpected event network-vif-plugged-d520a88d-ec02-45b8-adbc-484d38b9c18c for instance with vm_state building and task_state spawning.
Nov 29 03:07:44 user nova-compute[75234]: DEBUG nova.compute.manager [req-4058677a-5958-4bf8-9c9a-3a916abd7155 req-fbd35a2c-dc73-47db-b8f3-8b983ecf70fc service nova] [instance: 763a3e14-4563-4dc2-99ae-c340edbb93df] Received event network-vif-plugged-c50f64cd-b7fa-4580-bb78-bff756a0773a {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:07:44 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-4058677a-5958-4bf8-9c9a-3a916abd7155 req-fbd35a2c-dc73-47db-b8f3-8b983ecf70fc service nova] Acquiring lock "763a3e14-4563-4dc2-99ae-c340edbb93df-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:07:44 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-4058677a-5958-4bf8-9c9a-3a916abd7155 req-fbd35a2c-dc73-47db-b8f3-8b983ecf70fc service nova] Lock "763a3e14-4563-4dc2-99ae-c340edbb93df-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:07:44 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-4058677a-5958-4bf8-9c9a-3a916abd7155 req-fbd35a2c-dc73-47db-b8f3-8b983ecf70fc service nova] Lock "763a3e14-4563-4dc2-99ae-c340edbb93df-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:07:44 user nova-compute[75234]: DEBUG nova.compute.manager [req-4058677a-5958-4bf8-9c9a-3a916abd7155 req-fbd35a2c-dc73-47db-b8f3-8b983ecf70fc service nova] [instance: 763a3e14-4563-4dc2-99ae-c340edbb93df] No waiting events found dispatching network-vif-plugged-c50f64cd-b7fa-4580-bb78-bff756a0773a {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:07:44 user nova-compute[75234]: WARNING nova.compute.manager [req-4058677a-5958-4bf8-9c9a-3a916abd7155 req-fbd35a2c-dc73-47db-b8f3-8b983ecf70fc service nova] [instance: 763a3e14-4563-4dc2-99ae-c340edbb93df] Received unexpected event network-vif-plugged-c50f64cd-b7fa-4580-bb78-bff756a0773a for instance with vm_state building and task_state spawning.
Nov 29 03:07:44 user nova-compute[75234]: DEBUG nova.compute.manager [req-4058677a-5958-4bf8-9c9a-3a916abd7155 req-fbd35a2c-dc73-47db-b8f3-8b983ecf70fc service nova] [instance: 763a3e14-4563-4dc2-99ae-c340edbb93df] Received event network-vif-plugged-c50f64cd-b7fa-4580-bb78-bff756a0773a {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:07:44 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-4058677a-5958-4bf8-9c9a-3a916abd7155 req-fbd35a2c-dc73-47db-b8f3-8b983ecf70fc service nova] Acquiring lock "763a3e14-4563-4dc2-99ae-c340edbb93df-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:07:44 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-4058677a-5958-4bf8-9c9a-3a916abd7155 req-fbd35a2c-dc73-47db-b8f3-8b983ecf70fc service nova] Lock "763a3e14-4563-4dc2-99ae-c340edbb93df-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:07:44 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-4058677a-5958-4bf8-9c9a-3a916abd7155 req-fbd35a2c-dc73-47db-b8f3-8b983ecf70fc service nova] Lock "763a3e14-4563-4dc2-99ae-c340edbb93df-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:07:44 user nova-compute[75234]: DEBUG nova.compute.manager [req-4058677a-5958-4bf8-9c9a-3a916abd7155 req-fbd35a2c-dc73-47db-b8f3-8b983ecf70fc service nova] [instance: 763a3e14-4563-4dc2-99ae-c340edbb93df] No waiting events found dispatching network-vif-plugged-c50f64cd-b7fa-4580-bb78-bff756a0773a {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:07:44 user nova-compute[75234]: WARNING nova.compute.manager [req-4058677a-5958-4bf8-9c9a-3a916abd7155 req-fbd35a2c-dc73-47db-b8f3-8b983ecf70fc service nova] [instance: 763a3e14-4563-4dc2-99ae-c340edbb93df] Received unexpected event network-vif-plugged-c50f64cd-b7fa-4580-bb78-bff756a0773a for instance with vm_state building and task_state spawning.
Nov 29 03:07:44 user nova-compute[75234]: DEBUG nova.compute.manager [req-4058677a-5958-4bf8-9c9a-3a916abd7155 req-fbd35a2c-dc73-47db-b8f3-8b983ecf70fc service nova] [instance: 763a3e14-4563-4dc2-99ae-c340edbb93df] Received event network-vif-plugged-92917a0d-59ee-4167-8693-262a21f6e156 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:07:44 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-4058677a-5958-4bf8-9c9a-3a916abd7155 req-fbd35a2c-dc73-47db-b8f3-8b983ecf70fc service nova] Acquiring lock "763a3e14-4563-4dc2-99ae-c340edbb93df-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:07:44 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-4058677a-5958-4bf8-9c9a-3a916abd7155 req-fbd35a2c-dc73-47db-b8f3-8b983ecf70fc service nova] Lock "763a3e14-4563-4dc2-99ae-c340edbb93df-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:07:44 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-4058677a-5958-4bf8-9c9a-3a916abd7155 req-fbd35a2c-dc73-47db-b8f3-8b983ecf70fc service nova] Lock "763a3e14-4563-4dc2-99ae-c340edbb93df-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:07:44 user nova-compute[75234]: DEBUG nova.compute.manager [req-4058677a-5958-4bf8-9c9a-3a916abd7155 req-fbd35a2c-dc73-47db-b8f3-8b983ecf70fc service nova] [instance: 763a3e14-4563-4dc2-99ae-c340edbb93df] No waiting events found dispatching network-vif-plugged-92917a0d-59ee-4167-8693-262a21f6e156 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:07:44 user nova-compute[75234]: WARNING nova.compute.manager [req-4058677a-5958-4bf8-9c9a-3a916abd7155 req-fbd35a2c-dc73-47db-b8f3-8b983ecf70fc service nova] [instance: 763a3e14-4563-4dc2-99ae-c340edbb93df] Received unexpected event network-vif-plugged-92917a0d-59ee-4167-8693-262a21f6e156 for instance with vm_state building and task_state spawning.
Nov 29 03:07:44 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:45 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:45 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:45 user nova-compute[75234]: DEBUG nova.compute.manager [req-c7056227-5b25-4732-b48e-8f55d288477d req-20ccdd96-afe0-43f0-8cad-f1fe17243358 service nova] [instance: 763a3e14-4563-4dc2-99ae-c340edbb93df] Received event network-vif-plugged-2027f7a1-36e4-4434-a063-61e817758917 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:07:45 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-c7056227-5b25-4732-b48e-8f55d288477d req-20ccdd96-afe0-43f0-8cad-f1fe17243358 service nova] Acquiring lock "763a3e14-4563-4dc2-99ae-c340edbb93df-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:07:45 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-c7056227-5b25-4732-b48e-8f55d288477d req-20ccdd96-afe0-43f0-8cad-f1fe17243358 service nova] Lock "763a3e14-4563-4dc2-99ae-c340edbb93df-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:07:45 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-c7056227-5b25-4732-b48e-8f55d288477d req-20ccdd96-afe0-43f0-8cad-f1fe17243358 service nova] Lock "763a3e14-4563-4dc2-99ae-c340edbb93df-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:07:45 user nova-compute[75234]: DEBUG nova.compute.manager [req-c7056227-5b25-4732-b48e-8f55d288477d req-20ccdd96-afe0-43f0-8cad-f1fe17243358 service nova] [instance: 763a3e14-4563-4dc2-99ae-c340edbb93df] No waiting events found dispatching network-vif-plugged-2027f7a1-36e4-4434-a063-61e817758917 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:07:45 user nova-compute[75234]: WARNING nova.compute.manager [req-c7056227-5b25-4732-b48e-8f55d288477d req-20ccdd96-afe0-43f0-8cad-f1fe17243358 service nova] [instance: 763a3e14-4563-4dc2-99ae-c340edbb93df] Received unexpected event network-vif-plugged-2027f7a1-36e4-4434-a063-61e817758917 for instance with vm_state building and task_state spawning.
Nov 29 03:07:45 user nova-compute[75234]: DEBUG nova.compute.manager [req-c7056227-5b25-4732-b48e-8f55d288477d req-20ccdd96-afe0-43f0-8cad-f1fe17243358 service nova] [instance: 763a3e14-4563-4dc2-99ae-c340edbb93df] Received event network-vif-plugged-ef8ba969-6186-4b9a-b30f-ec9a70036bab {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:07:45 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-c7056227-5b25-4732-b48e-8f55d288477d req-20ccdd96-afe0-43f0-8cad-f1fe17243358 service nova] Acquiring lock "763a3e14-4563-4dc2-99ae-c340edbb93df-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:07:45 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-c7056227-5b25-4732-b48e-8f55d288477d req-20ccdd96-afe0-43f0-8cad-f1fe17243358 service nova] Lock "763a3e14-4563-4dc2-99ae-c340edbb93df-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:07:45 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-c7056227-5b25-4732-b48e-8f55d288477d req-20ccdd96-afe0-43f0-8cad-f1fe17243358 service nova] Lock "763a3e14-4563-4dc2-99ae-c340edbb93df-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:07:45 user nova-compute[75234]: DEBUG nova.compute.manager [req-c7056227-5b25-4732-b48e-8f55d288477d req-20ccdd96-afe0-43f0-8cad-f1fe17243358 service nova] [instance: 763a3e14-4563-4dc2-99ae-c340edbb93df] No waiting events found dispatching network-vif-plugged-ef8ba969-6186-4b9a-b30f-ec9a70036bab {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:07:45 user nova-compute[75234]: WARNING nova.compute.manager [req-c7056227-5b25-4732-b48e-8f55d288477d req-20ccdd96-afe0-43f0-8cad-f1fe17243358 service nova] [instance: 763a3e14-4563-4dc2-99ae-c340edbb93df] Received unexpected event network-vif-plugged-ef8ba969-6186-4b9a-b30f-ec9a70036bab for instance with vm_state building and task_state spawning.
Nov 29 03:07:45 user nova-compute[75234]: DEBUG nova.compute.manager [req-c7056227-5b25-4732-b48e-8f55d288477d req-20ccdd96-afe0-43f0-8cad-f1fe17243358 service nova] [instance: 763a3e14-4563-4dc2-99ae-c340edbb93df] Received event network-vif-plugged-ef8ba969-6186-4b9a-b30f-ec9a70036bab {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:07:45 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-c7056227-5b25-4732-b48e-8f55d288477d req-20ccdd96-afe0-43f0-8cad-f1fe17243358 service nova] Acquiring lock "763a3e14-4563-4dc2-99ae-c340edbb93df-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:07:45 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-c7056227-5b25-4732-b48e-8f55d288477d req-20ccdd96-afe0-43f0-8cad-f1fe17243358 service nova] Lock "763a3e14-4563-4dc2-99ae-c340edbb93df-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:07:45 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-c7056227-5b25-4732-b48e-8f55d288477d req-20ccdd96-afe0-43f0-8cad-f1fe17243358 service nova] Lock "763a3e14-4563-4dc2-99ae-c340edbb93df-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:07:45 user nova-compute[75234]: DEBUG nova.compute.manager [req-c7056227-5b25-4732-b48e-8f55d288477d req-20ccdd96-afe0-43f0-8cad-f1fe17243358 service nova] [instance: 763a3e14-4563-4dc2-99ae-c340edbb93df] No waiting events found dispatching network-vif-plugged-ef8ba969-6186-4b9a-b30f-ec9a70036bab {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:07:45 user nova-compute[75234]: WARNING nova.compute.manager [req-c7056227-5b25-4732-b48e-8f55d288477d req-20ccdd96-afe0-43f0-8cad-f1fe17243358 service nova] [instance: 763a3e14-4563-4dc2-99ae-c340edbb93df] Received unexpected event network-vif-plugged-ef8ba969-6186-4b9a-b30f-ec9a70036bab for instance with vm_state building and task_state spawning.
Nov 29 03:07:45 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:45 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:45 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:45 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:45 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:46 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:46 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:46 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:46 user nova-compute[75234]: DEBUG nova.compute.manager [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] [instance: 763a3e14-4563-4dc2-99ae-c340edbb93df] Instance event wait completed in 0 seconds for  {{(pid=75234) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Nov 29 03:07:46 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] [instance: 763a3e14-4563-4dc2-99ae-c340edbb93df] Guest created on hypervisor {{(pid=75234) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4421}}
Nov 29 03:07:46 user nova-compute[75234]: INFO nova.virt.libvirt.driver [-] [instance: 763a3e14-4563-4dc2-99ae-c340edbb93df] Instance spawned successfully.
Nov 29 03:07:46 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] [instance: 763a3e14-4563-4dc2-99ae-c340edbb93df] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:904}}
Nov 29 03:07:46 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] [instance: 763a3e14-4563-4dc2-99ae-c340edbb93df] Found default for hw_cdrom_bus of ide {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:07:46 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] [instance: 763a3e14-4563-4dc2-99ae-c340edbb93df] Found default for hw_disk_bus of virtio {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:07:46 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] [instance: 763a3e14-4563-4dc2-99ae-c340edbb93df] Found default for hw_input_bus of None {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:07:46 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] [instance: 763a3e14-4563-4dc2-99ae-c340edbb93df] Found default for hw_pointer_model of None {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:07:46 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] [instance: 763a3e14-4563-4dc2-99ae-c340edbb93df] Found default for hw_video_model of virtio {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:07:46 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] [instance: 763a3e14-4563-4dc2-99ae-c340edbb93df] Found default for hw_vif_model of virtio {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:07:46 user nova-compute[75234]: DEBUG nova.compute.manager [req-edd9ed37-017b-48c8-9f80-5e01c18d66d8 req-c68e264c-8c72-4de8-a79b-6b4256d33d14 service nova] [instance: 763a3e14-4563-4dc2-99ae-c340edbb93df] Received event network-vif-plugged-92917a0d-59ee-4167-8693-262a21f6e156 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:07:46 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-edd9ed37-017b-48c8-9f80-5e01c18d66d8 req-c68e264c-8c72-4de8-a79b-6b4256d33d14 service nova] Acquiring lock "763a3e14-4563-4dc2-99ae-c340edbb93df-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:07:46 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-edd9ed37-017b-48c8-9f80-5e01c18d66d8 req-c68e264c-8c72-4de8-a79b-6b4256d33d14 service nova] Lock "763a3e14-4563-4dc2-99ae-c340edbb93df-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:07:46 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-edd9ed37-017b-48c8-9f80-5e01c18d66d8 req-c68e264c-8c72-4de8-a79b-6b4256d33d14 service nova] Lock "763a3e14-4563-4dc2-99ae-c340edbb93df-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:07:46 user nova-compute[75234]: DEBUG nova.compute.manager [req-edd9ed37-017b-48c8-9f80-5e01c18d66d8 req-c68e264c-8c72-4de8-a79b-6b4256d33d14 service nova] [instance: 763a3e14-4563-4dc2-99ae-c340edbb93df] No waiting events found dispatching network-vif-plugged-92917a0d-59ee-4167-8693-262a21f6e156 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:07:46 user nova-compute[75234]: WARNING nova.compute.manager [req-edd9ed37-017b-48c8-9f80-5e01c18d66d8 req-c68e264c-8c72-4de8-a79b-6b4256d33d14 service nova] [instance: 763a3e14-4563-4dc2-99ae-c340edbb93df] Received unexpected event network-vif-plugged-92917a0d-59ee-4167-8693-262a21f6e156 for instance with vm_state building and task_state spawning.
Nov 29 03:07:46 user nova-compute[75234]: INFO nova.compute.manager [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] [instance: 763a3e14-4563-4dc2-99ae-c340edbb93df] Took 11.19 seconds to spawn the instance on the hypervisor.
Nov 29 03:07:46 user nova-compute[75234]: DEBUG nova.compute.manager [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] [instance: 763a3e14-4563-4dc2-99ae-c340edbb93df] Checking state {{(pid=75234) _get_power_state /opt/stack/nova/nova/compute/manager.py:1782}}
Nov 29 03:07:47 user nova-compute[75234]: INFO nova.compute.manager [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] [instance: 763a3e14-4563-4dc2-99ae-c340edbb93df] Took 36.61 seconds to build instance.
Nov 29 03:07:47 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-d60faab8-7833-4a2c-887f-53e00f9b3059 tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] Lock "763a3e14-4563-4dc2-99ae-c340edbb93df" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 36.714s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:07:47 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-2821575f-11ad-484a-9599-886a9bf4fd58 tempest-DeleteServersTestJSON-1668990756 tempest-DeleteServersTestJSON-1668990756-project-member] Acquiring lock "9f7876ad-0487-426d-8b19-b718dccdc4fe" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:07:47 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-2821575f-11ad-484a-9599-886a9bf4fd58 tempest-DeleteServersTestJSON-1668990756 tempest-DeleteServersTestJSON-1668990756-project-member] Lock "9f7876ad-0487-426d-8b19-b718dccdc4fe" acquired by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:07:48 user nova-compute[75234]: DEBUG nova.objects.instance [None req-2821575f-11ad-484a-9599-886a9bf4fd58 tempest-DeleteServersTestJSON-1668990756 tempest-DeleteServersTestJSON-1668990756-project-member] Lazy-loading 'flavor' on Instance uuid 9f7876ad-0487-426d-8b19-b718dccdc4fe {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:07:48 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-2821575f-11ad-484a-9599-886a9bf4fd58 tempest-DeleteServersTestJSON-1668990756 tempest-DeleteServersTestJSON-1668990756-project-member] Lock "9f7876ad-0487-426d-8b19-b718dccdc4fe" "released" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: held 0.385s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:07:48 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-2821575f-11ad-484a-9599-886a9bf4fd58 tempest-DeleteServersTestJSON-1668990756 tempest-DeleteServersTestJSON-1668990756-project-member] Acquiring lock "9f7876ad-0487-426d-8b19-b718dccdc4fe" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:07:48 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-2821575f-11ad-484a-9599-886a9bf4fd58 tempest-DeleteServersTestJSON-1668990756 tempest-DeleteServersTestJSON-1668990756-project-member] Lock "9f7876ad-0487-426d-8b19-b718dccdc4fe" acquired by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:07:48 user nova-compute[75234]: INFO nova.compute.manager [None req-2821575f-11ad-484a-9599-886a9bf4fd58 tempest-DeleteServersTestJSON-1668990756 tempest-DeleteServersTestJSON-1668990756-project-member] [instance: 9f7876ad-0487-426d-8b19-b718dccdc4fe] Attaching volume be635389-1f4c-4c39-8c36-25d81868770a to /dev/vdb
Nov 29 03:07:48 user nova-compute[75234]: DEBUG os_brick.utils [None req-2821575f-11ad-484a-9599-886a9bf4fd58 tempest-DeleteServersTestJSON-1668990756 tempest-DeleteServersTestJSON-1668990756-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:176}}
Nov 29 03:07:48 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:07:48 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.012s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:07:48 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[28a33a8b-d52c-42e0-9e17-09ede4c458ef]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:dd1368c5ba2c\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:07:48 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt -v / -n -o SOURCE {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:07:48 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt -v / -n -o SOURCE" returned: 0 in 0.016s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:07:48 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[0d232954-01f9-4201-858d-8991caa138b1]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:07:48 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:07:48 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.015s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:07:48 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[1b21e3eb-fdc9-4306-9111-c2731c22b80d]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:07:48 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[7247ad5e-6ec6-4b54-ba95-4fd4b1267641]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:07:48 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-2821575f-11ad-484a-9599-886a9bf4fd58 tempest-DeleteServersTestJSON-1668990756 tempest-DeleteServersTestJSON-1668990756-project-member] Running cmd (subprocess): nvme version {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:07:48 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-2821575f-11ad-484a-9599-886a9bf4fd58 tempest-DeleteServersTestJSON-1668990756 tempest-DeleteServersTestJSON-1668990756-project-member] 'nvme version' failed. Not Retrying. {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:473}}
Nov 29 03:07:48 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.nvmeof [None req-2821575f-11ad-484a-9599-886a9bf4fd58 tempest-DeleteServersTestJSON-1668990756 tempest-DeleteServersTestJSON-1668990756-project-member] nvme not present on system {{(pid=75234) nvme_present /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/nvmeof.py:756}}
Nov 29 03:07:48 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:07:48 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:473}}
Nov 29 03:07:48 user nova-compute[75234]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Nov 29 03:07:48 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[c977268a-1864-4d01-b1af-272e0cc482af]: (4, '') {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:07:48 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.lightos [None req-2821575f-11ad-484a-9599-886a9bf4fd58 tempest-DeleteServersTestJSON-1668990756 tempest-DeleteServersTestJSON-1668990756-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=75234) find_dsc /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/lightos.py:98}}
Nov 29 03:07:48 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.lightos [None req-2821575f-11ad-484a-9599-886a9bf4fd58 tempest-DeleteServersTestJSON-1668990756 tempest-DeleteServersTestJSON-1668990756-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=75234) get_connector_properties /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/lightos.py:76}}
Nov 29 03:07:48 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.lightos [None req-2821575f-11ad-484a-9599-886a9bf4fd58 tempest-DeleteServersTestJSON-1668990756 tempest-DeleteServersTestJSON-1668990756-project-member] LIGHTOS: no hostnqn found. {{(pid=75234) get_connector_properties /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/lightos.py:84}}
Nov 29 03:07:48 user nova-compute[75234]: DEBUG os_brick.utils [None req-2821575f-11ad-484a-9599-886a9bf4fd58 tempest-DeleteServersTestJSON-1668990756 tempest-DeleteServersTestJSON-1668990756-project-member] <== get_connector_properties: return (119ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:dd1368c5ba2c', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:203}}
Nov 29 03:07:48 user nova-compute[75234]: DEBUG nova.virt.block_device [None req-2821575f-11ad-484a-9599-886a9bf4fd58 tempest-DeleteServersTestJSON-1668990756 tempest-DeleteServersTestJSON-1668990756-project-member] [instance: 9f7876ad-0487-426d-8b19-b718dccdc4fe] Updating existing volume attachment record: 4f5bed59-e5e1-4c29-839f-330a2fe2e1a7 {{(pid=75234) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Nov 29 03:07:48 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:07:48 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:07:48 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=75234) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10541}}
Nov 29 03:07:49 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:07:49 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:07:50 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:07:50 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Starting heal instance info cache {{(pid=75234) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9922}}
Nov 29 03:07:50 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Acquiring lock "refresh_cache-2bf9fe41-63c2-4882-bcdd-8e314e734ffe" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:07:50 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Acquired lock "refresh_cache-2bf9fe41-63c2-4882-bcdd-8e314e734ffe" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:07:50 user nova-compute[75234]: DEBUG nova.network.neutron [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] [instance: 2bf9fe41-63c2-4882-bcdd-8e314e734ffe] Forcefully refreshing network info cache for instance {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2003}}
Nov 29 03:07:50 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:51 user nova-compute[75234]: DEBUG nova.network.neutron [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] [instance: 2bf9fe41-63c2-4882-bcdd-8e314e734ffe] Updating instance_info_cache with network_info: [{"id": "c8bb20fd-5c85-4e89-8c61-13a4f17dd4cc", "address": "fa:16:3e:2c:c9:1d", "network": {"id": "b27ac31e-3601-4055-be79-34b0946b0913", "bridge": "br-int", "label": "tempest-ServersNegativeTestJSON-1491620118-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "b877668e02eb441c85dc2a7003a94c7a", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc8bb20fd-5c", "ovs_interfaceid": "c8bb20fd-5c85-4e89-8c61-13a4f17dd4cc", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:07:51 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Releasing lock "refresh_cache-2bf9fe41-63c2-4882-bcdd-8e314e734ffe" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:07:51 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] [instance: 2bf9fe41-63c2-4882-bcdd-8e314e734ffe] Updated the network info_cache for instance {{(pid=75234) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9993}}
Nov 29 03:07:51 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:51 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._sync_scheduler_instance_info {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:07:51 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:07:51 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-b7b37b35-34f8-4611-a2d1-7d64ab4aa82b tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] Acquiring lock "23a31b1e-4005-442f-96e4-9fe82d9e17ba" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:07:51 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-b7b37b35-34f8-4611-a2d1-7d64ab4aa82b tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] Lock "23a31b1e-4005-442f-96e4-9fe82d9e17ba" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:07:51 user nova-compute[75234]: DEBUG nova.compute.manager [None req-b7b37b35-34f8-4611-a2d1-7d64ab4aa82b tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] [instance: 23a31b1e-4005-442f-96e4-9fe82d9e17ba] Starting instance... {{(pid=75234) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2422}}
Nov 29 03:07:51 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-b7b37b35-34f8-4611-a2d1-7d64ab4aa82b tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:07:51 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-b7b37b35-34f8-4611-a2d1-7d64ab4aa82b tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:07:52 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-b7b37b35-34f8-4611-a2d1-7d64ab4aa82b tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=75234) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2366}}
Nov 29 03:07:52 user nova-compute[75234]: INFO nova.compute.claims [None req-b7b37b35-34f8-4611-a2d1-7d64ab4aa82b tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] [instance: 23a31b1e-4005-442f-96e4-9fe82d9e17ba] Claim successful on node user
Nov 29 03:07:52 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager.update_available_resource {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:07:52 user nova-compute[75234]: DEBUG nova.compute.provider_tree [None req-b7b37b35-34f8-4611-a2d1-7d64ab4aa82b tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] Inventory has not changed in ProviderTree for provider: 4d291220-7c37-4839-a0e3-b23c66502b74 {{(pid=75234) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Nov 29 03:07:52 user nova-compute[75234]: DEBUG nova.scheduler.client.report [None req-b7b37b35-34f8-4611-a2d1-7d64ab4aa82b tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] Inventory has not changed for provider 4d291220-7c37-4839-a0e3-b23c66502b74 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=75234) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Nov 29 03:07:52 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:07:52 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-b7b37b35-34f8-4611-a2d1-7d64ab4aa82b tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.542s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:07:52 user nova-compute[75234]: DEBUG nova.compute.manager [None req-b7b37b35-34f8-4611-a2d1-7d64ab4aa82b tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] [instance: 23a31b1e-4005-442f-96e4-9fe82d9e17ba] Start building networks asynchronously for instance. {{(pid=75234) _build_resources /opt/stack/nova/nova/compute/manager.py:2819}}
Nov 29 03:07:52 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.028s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:07:52 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:07:52 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Auditing locally available compute resources for user (node: user) {{(pid=75234) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:907}}
Nov 29 03:07:52 user nova-compute[75234]: DEBUG nova.compute.manager [None req-b7b37b35-34f8-4611-a2d1-7d64ab4aa82b tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] [instance: 23a31b1e-4005-442f-96e4-9fe82d9e17ba] Allocating IP information in the background. {{(pid=75234) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1972}}
Nov 29 03:07:52 user nova-compute[75234]: DEBUG nova.network.neutron [None req-b7b37b35-34f8-4611-a2d1-7d64ab4aa82b tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] [instance: 23a31b1e-4005-442f-96e4-9fe82d9e17ba] allocate_for_instance() {{(pid=75234) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1156}}
Nov 29 03:07:52 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-b7b37b35-34f8-4611-a2d1-7d64ab4aa82b tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] [instance: 23a31b1e-4005-442f-96e4-9fe82d9e17ba] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Nov 29 03:07:52 user nova-compute[75234]: DEBUG nova.compute.manager [None req-b7b37b35-34f8-4611-a2d1-7d64ab4aa82b tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] [instance: 23a31b1e-4005-442f-96e4-9fe82d9e17ba] Start building block device mappings for instance. {{(pid=75234) _build_resources /opt/stack/nova/nova/compute/manager.py:2854}}
Nov 29 03:07:52 user nova-compute[75234]: DEBUG nova.policy [None req-b7b37b35-34f8-4611-a2d1-7d64ab4aa82b tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '178bfe0ea37e45d8abd2d6b45bd5660c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b877668e02eb441c85dc2a7003a94c7a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=75234) authorize /opt/stack/nova/nova/policy.py:203}}
Nov 29 03:07:52 user nova-compute[75234]: DEBUG nova.compute.manager [None req-b7b37b35-34f8-4611-a2d1-7d64ab4aa82b tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] [instance: 23a31b1e-4005-442f-96e4-9fe82d9e17ba] Start spawning the instance on the hypervisor. {{(pid=75234) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2628}}
Nov 29 03:07:52 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-b7b37b35-34f8-4611-a2d1-7d64ab4aa82b tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] [instance: 23a31b1e-4005-442f-96e4-9fe82d9e17ba] Creating instance directory {{(pid=75234) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4727}}
Nov 29 03:07:52 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-b7b37b35-34f8-4611-a2d1-7d64ab4aa82b tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] [instance: 23a31b1e-4005-442f-96e4-9fe82d9e17ba] Creating image(s)
Nov 29 03:07:52 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-b7b37b35-34f8-4611-a2d1-7d64ab4aa82b tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] Acquiring lock "/opt/stack/data/nova/instances/23a31b1e-4005-442f-96e4-9fe82d9e17ba/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:07:52 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-b7b37b35-34f8-4611-a2d1-7d64ab4aa82b tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] Lock "/opt/stack/data/nova/instances/23a31b1e-4005-442f-96e4-9fe82d9e17ba/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:07:52 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-b7b37b35-34f8-4611-a2d1-7d64ab4aa82b tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] Lock "/opt/stack/data/nova/instances/23a31b1e-4005-442f-96e4-9fe82d9e17ba/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.002s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:07:52 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-b7b37b35-34f8-4611-a2d1-7d64ab4aa82b tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:07:52 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] skipping disk /dev/sde (vda) as it is a volume {{(pid=75234) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11213}}
Nov 29 03:07:52 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] skipping disk /dev/sdf (vdb) as it is a volume {{(pid=75234) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11213}}
Nov 29 03:07:52 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] skipping disk /dev/sdg (vdc) as it is a volume {{(pid=75234) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11213}}
Nov 29 03:07:52 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a9350a18-14e6-42c6-a150-0a1487f20ad9/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:07:53 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-b7b37b35-34f8-4611-a2d1-7d64ab4aa82b tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json" returned: 0 in 0.236s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:07:53 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-b7b37b35-34f8-4611-a2d1-7d64ab4aa82b tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] Acquiring lock "80686e29dc9db6600d03e85cb62af298e12d7918" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:07:53 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-b7b37b35-34f8-4611-a2d1-7d64ab4aa82b tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] Lock "80686e29dc9db6600d03e85cb62af298e12d7918" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.003s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:07:53 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-b7b37b35-34f8-4611-a2d1-7d64ab4aa82b tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:07:53 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a9350a18-14e6-42c6-a150-0a1487f20ad9/disk --force-share --output=json" returned: 0 in 0.199s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:07:53 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a9350a18-14e6-42c6-a150-0a1487f20ad9/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:07:53 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-b7b37b35-34f8-4611-a2d1-7d64ab4aa82b tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json" returned: 0 in 0.183s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:07:53 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-b7b37b35-34f8-4611-a2d1-7d64ab4aa82b tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918,backing_fmt=raw /opt/stack/data/nova/instances/23a31b1e-4005-442f-96e4-9fe82d9e17ba/disk 1073741824 {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:07:53 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a9350a18-14e6-42c6-a150-0a1487f20ad9/disk --force-share --output=json" returned: 0 in 0.193s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:07:53 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-b7b37b35-34f8-4611-a2d1-7d64ab4aa82b tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918,backing_fmt=raw /opt/stack/data/nova/instances/23a31b1e-4005-442f-96e4-9fe82d9e17ba/disk 1073741824" returned: 0 in 0.058s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:07:53 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-b7b37b35-34f8-4611-a2d1-7d64ab4aa82b tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] Lock "80686e29dc9db6600d03e85cb62af298e12d7918" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.248s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:07:53 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-b7b37b35-34f8-4611-a2d1-7d64ab4aa82b tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:07:53 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/8297c264-a213-44ee-be46-5c08eabda9f7/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:07:53 user nova-compute[75234]: DEBUG nova.network.neutron [None req-b7b37b35-34f8-4611-a2d1-7d64ab4aa82b tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] [instance: 23a31b1e-4005-442f-96e4-9fe82d9e17ba] Successfully created port: 5a8d4829-f681-4e2c-83d7-91e5ea0b1fd2 {{(pid=75234) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:548}}
Nov 29 03:07:53 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-b7b37b35-34f8-4611-a2d1-7d64ab4aa82b tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json" returned: 0 in 0.191s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:07:53 user nova-compute[75234]: DEBUG nova.virt.disk.api [None req-b7b37b35-34f8-4611-a2d1-7d64ab4aa82b tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] Checking if we can resize image /opt/stack/data/nova/instances/23a31b1e-4005-442f-96e4-9fe82d9e17ba/disk. size=1073741824 {{(pid=75234) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:166}}
Nov 29 03:07:53 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-b7b37b35-34f8-4611-a2d1-7d64ab4aa82b tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/23a31b1e-4005-442f-96e4-9fe82d9e17ba/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:07:53 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/8297c264-a213-44ee-be46-5c08eabda9f7/disk --force-share --output=json" returned: 0 in 0.192s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:07:53 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/8297c264-a213-44ee-be46-5c08eabda9f7/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:07:53 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-b7b37b35-34f8-4611-a2d1-7d64ab4aa82b tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/23a31b1e-4005-442f-96e4-9fe82d9e17ba/disk --force-share --output=json" returned: 0 in 0.222s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:07:53 user nova-compute[75234]: DEBUG nova.virt.disk.api [None req-b7b37b35-34f8-4611-a2d1-7d64ab4aa82b tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] Cannot resize image /opt/stack/data/nova/instances/23a31b1e-4005-442f-96e4-9fe82d9e17ba/disk to a smaller size. {{(pid=75234) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:172}}
Nov 29 03:07:53 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-b7b37b35-34f8-4611-a2d1-7d64ab4aa82b tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] [instance: 23a31b1e-4005-442f-96e4-9fe82d9e17ba] Created local disks {{(pid=75234) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4861}}
Nov 29 03:07:53 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-b7b37b35-34f8-4611-a2d1-7d64ab4aa82b tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] [instance: 23a31b1e-4005-442f-96e4-9fe82d9e17ba] Ensure instance console log exists: /opt/stack/data/nova/instances/23a31b1e-4005-442f-96e4-9fe82d9e17ba/console.log {{(pid=75234) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4613}}
Nov 29 03:07:53 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-b7b37b35-34f8-4611-a2d1-7d64ab4aa82b tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:07:53 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-b7b37b35-34f8-4611-a2d1-7d64ab4aa82b tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:07:53 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-b7b37b35-34f8-4611-a2d1-7d64ab4aa82b tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:07:53 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/8297c264-a213-44ee-be46-5c08eabda9f7/disk --force-share --output=json" returned: 0 in 0.261s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:07:53 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/da625f9c-b7ed-4afc-875c-6436bf58e9cc/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:07:53 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/da625f9c-b7ed-4afc-875c-6436bf58e9cc/disk --force-share --output=json" returned: 0 in 0.207s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:07:53 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/da625f9c-b7ed-4afc-875c-6436bf58e9cc/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:07:54 user nova-compute[75234]: DEBUG nova.network.neutron [None req-b7b37b35-34f8-4611-a2d1-7d64ab4aa82b tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] [instance: 23a31b1e-4005-442f-96e4-9fe82d9e17ba] Successfully updated port: 5a8d4829-f681-4e2c-83d7-91e5ea0b1fd2 {{(pid=75234) _update_port /opt/stack/nova/nova/network/neutron.py:586}}
Nov 29 03:07:54 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-b7b37b35-34f8-4611-a2d1-7d64ab4aa82b tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] Acquiring lock "refresh_cache-23a31b1e-4005-442f-96e4-9fe82d9e17ba" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:07:54 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-b7b37b35-34f8-4611-a2d1-7d64ab4aa82b tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] Acquired lock "refresh_cache-23a31b1e-4005-442f-96e4-9fe82d9e17ba" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:07:54 user nova-compute[75234]: DEBUG nova.network.neutron [None req-b7b37b35-34f8-4611-a2d1-7d64ab4aa82b tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] [instance: 23a31b1e-4005-442f-96e4-9fe82d9e17ba] Building network info cache for instance {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2009}}
Nov 29 03:07:54 user nova-compute[75234]: DEBUG nova.compute.manager [req-53982eeb-b07b-438e-b4ac-bcd555166c2e req-fac5c8b1-2b6f-4fd5-99a3-8d91b531feea service nova] [instance: 23a31b1e-4005-442f-96e4-9fe82d9e17ba] Received event network-changed-5a8d4829-f681-4e2c-83d7-91e5ea0b1fd2 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:07:54 user nova-compute[75234]: DEBUG nova.compute.manager [req-53982eeb-b07b-438e-b4ac-bcd555166c2e req-fac5c8b1-2b6f-4fd5-99a3-8d91b531feea service nova] [instance: 23a31b1e-4005-442f-96e4-9fe82d9e17ba] Refreshing instance network info cache due to event network-changed-5a8d4829-f681-4e2c-83d7-91e5ea0b1fd2. {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11117}}
Nov 29 03:07:54 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-53982eeb-b07b-438e-b4ac-bcd555166c2e req-fac5c8b1-2b6f-4fd5-99a3-8d91b531feea service nova] Acquiring lock "refresh_cache-23a31b1e-4005-442f-96e4-9fe82d9e17ba" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:07:54 user nova-compute[75234]: DEBUG nova.network.neutron [None req-b7b37b35-34f8-4611-a2d1-7d64ab4aa82b tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] [instance: 23a31b1e-4005-442f-96e4-9fe82d9e17ba] Instance cache missing network info. {{(pid=75234) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3322}}
Nov 29 03:07:54 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/da625f9c-b7ed-4afc-875c-6436bf58e9cc/disk --force-share --output=json" returned: 0 in 0.215s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:07:54 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2bf9fe41-63c2-4882-bcdd-8e314e734ffe/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:07:54 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2bf9fe41-63c2-4882-bcdd-8e314e734ffe/disk --force-share --output=json" returned: 0 in 0.170s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:07:54 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2bf9fe41-63c2-4882-bcdd-8e314e734ffe/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:07:54 user nova-compute[75234]: DEBUG nova.network.neutron [None req-b7b37b35-34f8-4611-a2d1-7d64ab4aa82b tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] [instance: 23a31b1e-4005-442f-96e4-9fe82d9e17ba] Updating instance_info_cache with network_info: [{"id": "5a8d4829-f681-4e2c-83d7-91e5ea0b1fd2", "address": "fa:16:3e:1b:04:c4", "network": {"id": "b27ac31e-3601-4055-be79-34b0946b0913", "bridge": "br-int", "label": "tempest-ServersNegativeTestJSON-1491620118-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "b877668e02eb441c85dc2a7003a94c7a", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap5a8d4829-f6", "ovs_interfaceid": "5a8d4829-f681-4e2c-83d7-91e5ea0b1fd2", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:07:54 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-b7b37b35-34f8-4611-a2d1-7d64ab4aa82b tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] Releasing lock "refresh_cache-23a31b1e-4005-442f-96e4-9fe82d9e17ba" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:07:54 user nova-compute[75234]: DEBUG nova.compute.manager [None req-b7b37b35-34f8-4611-a2d1-7d64ab4aa82b tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] [instance: 23a31b1e-4005-442f-96e4-9fe82d9e17ba] Instance network_info: |[{"id": "5a8d4829-f681-4e2c-83d7-91e5ea0b1fd2", "address": "fa:16:3e:1b:04:c4", "network": {"id": "b27ac31e-3601-4055-be79-34b0946b0913", "bridge": "br-int", "label": "tempest-ServersNegativeTestJSON-1491620118-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "b877668e02eb441c85dc2a7003a94c7a", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap5a8d4829-f6", "ovs_interfaceid": "5a8d4829-f681-4e2c-83d7-91e5ea0b1fd2", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=75234) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1987}}
Nov 29 03:07:54 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-53982eeb-b07b-438e-b4ac-bcd555166c2e req-fac5c8b1-2b6f-4fd5-99a3-8d91b531feea service nova] Acquired lock "refresh_cache-23a31b1e-4005-442f-96e4-9fe82d9e17ba" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:07:54 user nova-compute[75234]: DEBUG nova.network.neutron [req-53982eeb-b07b-438e-b4ac-bcd555166c2e req-fac5c8b1-2b6f-4fd5-99a3-8d91b531feea service nova] [instance: 23a31b1e-4005-442f-96e4-9fe82d9e17ba] Refreshing network info cache for port 5a8d4829-f681-4e2c-83d7-91e5ea0b1fd2 {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2006}}
Nov 29 03:07:54 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-b7b37b35-34f8-4611-a2d1-7d64ab4aa82b tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] [instance: 23a31b1e-4005-442f-96e4-9fe82d9e17ba] Start _get_guest_xml network_info=[{"id": "5a8d4829-f681-4e2c-83d7-91e5ea0b1fd2", "address": "fa:16:3e:1b:04:c4", "network": {"id": "b27ac31e-3601-4055-be79-34b0946b0913", "bridge": "br-int", "label": "tempest-ServersNegativeTestJSON-1491620118-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "b877668e02eb441c85dc2a7003a94c7a", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap5a8d4829-f6", "ovs_interfaceid": "5a8d4829-f681-4e2c-83d7-91e5ea0b1fd2", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum='c8fc807773e5354afe61636071771906',container_format='bare',created_at=2023-11-29T03:02:15Z,direct_url=<?>,disk_format='qcow2',id=96c30ba7-9b73-4a30-8ba4-7138879573b7,min_disk=0,min_ram=0,name='cirros-0.6.2-x86_64-disk',owner='b18e380921c748b78517b6f1211cd0f1',properties=ImageMetaProps,protected=<?>,size=21430272,status='active',tags=<?>,updated_at=2023-11-29T03:02:17Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [{'device_name': '/dev/vda', 'encryption_format': None, 'encrypted': False, 'device_type': 'disk', 'size': 0, 'disk_bus': 'virtio', 'boot_index': 0, 'encryption_options': None, 'encryption_secret_uuid': None, 'guest_format': None, 'image_id': '96c30ba7-9b73-4a30-8ba4-7138879573b7'}], 'ephemerals': [], 'block_device_mapping': [], 'swap': None} {{(pid=75234) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7562}}
Nov 29 03:07:54 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-b7b37b35-34f8-4611-a2d1-7d64ab4aa82b tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:07:54 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-b7b37b35-34f8-4611-a2d1-7d64ab4aa82b tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:07:54 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-b7b37b35-34f8-4611-a2d1-7d64ab4aa82b tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] Searching host: 'user' for CPU controller through CGroups V1... {{(pid=75234) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1652}}
Nov 29 03:07:54 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-b7b37b35-34f8-4611-a2d1-7d64ab4aa82b tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] CPU controller missing on host. {{(pid=75234) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1662}}
Nov 29 03:07:54 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-b7b37b35-34f8-4611-a2d1-7d64ab4aa82b tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] Searching host: 'user' for CPU controller through CGroups V2... {{(pid=75234) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1671}}
Nov 29 03:07:54 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-b7b37b35-34f8-4611-a2d1-7d64ab4aa82b tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] CPU controller found on host. {{(pid=75234) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1678}}
Nov 29 03:07:54 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-b7b37b35-34f8-4611-a2d1-7d64ab4aa82b tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=75234) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5400}}
Nov 29 03:07:54 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-b7b37b35-34f8-4611-a2d1-7d64ab4aa82b tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-11-29T03:02:04Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='1',id=6,is_public=True,memory_mb=512,name='m1.tiny',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='c8fc807773e5354afe61636071771906',container_format='bare',created_at=2023-11-29T03:02:15Z,direct_url=<?>,disk_format='qcow2',id=96c30ba7-9b73-4a30-8ba4-7138879573b7,min_disk=0,min_ram=0,name='cirros-0.6.2-x86_64-disk',owner='b18e380921c748b78517b6f1211cd0f1',properties=ImageMetaProps,protected=<?>,size=21430272,status='active',tags=<?>,updated_at=2023-11-29T03:02:17Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:563}}
Nov 29 03:07:54 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-b7b37b35-34f8-4611-a2d1-7d64ab4aa82b tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] Flavor limits 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:348}}
Nov 29 03:07:54 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-b7b37b35-34f8-4611-a2d1-7d64ab4aa82b tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] Image limits 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:352}}
Nov 29 03:07:54 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-b7b37b35-34f8-4611-a2d1-7d64ab4aa82b tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] Flavor pref 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:388}}
Nov 29 03:07:54 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-b7b37b35-34f8-4611-a2d1-7d64ab4aa82b tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] Image pref 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:392}}
Nov 29 03:07:54 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-b7b37b35-34f8-4611-a2d1-7d64ab4aa82b tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:430}}
Nov 29 03:07:54 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-b7b37b35-34f8-4611-a2d1-7d64ab4aa82b tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:569}}
Nov 29 03:07:54 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-b7b37b35-34f8-4611-a2d1-7d64ab4aa82b tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=75234) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:471}}
Nov 29 03:07:54 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-b7b37b35-34f8-4611-a2d1-7d64ab4aa82b tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] Got 1 possible topologies {{(pid=75234) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:501}}
Nov 29 03:07:54 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-b7b37b35-34f8-4611-a2d1-7d64ab4aa82b tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:575}}
Nov 29 03:07:54 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-b7b37b35-34f8-4611-a2d1-7d64ab4aa82b tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:577}}
Nov 29 03:07:54 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-b7b37b35-34f8-4611-a2d1-7d64ab4aa82b tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-11-29T03:07:51Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-ServersNegativeTestJSON-server-1398606202',display_name='tempest-ServersNegativeTestJSON-server-1398606202',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-serversnegativetestjson-server-1398606202',id=13,image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data=None,key_name=None,keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='b877668e02eb441c85dc2a7003a94c7a',ramdisk_id='',reservation_id='r-zt0xoplh',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-ServersNegativeTestJSON-20765276',owner_user_name='tempest-ServersNegativeTestJSON-20765276-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-11-29T03:07:53Z,user_data=None,user_id='178bfe0ea37e45d8abd2d6b45bd5660c',uuid=23a31b1e-4005-442f-96e4-9fe82d9e17ba,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "5a8d4829-f681-4e2c-83d7-91e5ea0b1fd2", "address": "fa:16:3e:1b:04:c4", "network": {"id": "b27ac31e-3601-4055-be79-34b0946b0913", "bridge": "br-int", "label": "tempest-ServersNegativeTestJSON-1491620118-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "b877668e02eb441c85dc2a7003a94c7a", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap5a8d4829-f6", "ovs_interfaceid": "5a8d4829-f681-4e2c-83d7-91e5ea0b1fd2", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=75234) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Nov 29 03:07:54 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-b7b37b35-34f8-4611-a2d1-7d64ab4aa82b tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] Converting VIF {"id": "5a8d4829-f681-4e2c-83d7-91e5ea0b1fd2", "address": "fa:16:3e:1b:04:c4", "network": {"id": "b27ac31e-3601-4055-be79-34b0946b0913", "bridge": "br-int", "label": "tempest-ServersNegativeTestJSON-1491620118-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "b877668e02eb441c85dc2a7003a94c7a", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap5a8d4829-f6", "ovs_interfaceid": "5a8d4829-f681-4e2c-83d7-91e5ea0b1fd2", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:07:54 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-b7b37b35-34f8-4611-a2d1-7d64ab4aa82b tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:1b:04:c4,bridge_name='br-int',has_traffic_filtering=True,id=5a8d4829-f681-4e2c-83d7-91e5ea0b1fd2,network=Network(b27ac31e-3601-4055-be79-34b0946b0913),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap5a8d4829-f6') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:07:54 user nova-compute[75234]: DEBUG nova.objects.instance [None req-b7b37b35-34f8-4611-a2d1-7d64ab4aa82b tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] Lazy-loading 'pci_devices' on Instance uuid 23a31b1e-4005-442f-96e4-9fe82d9e17ba {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:07:54 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2bf9fe41-63c2-4882-bcdd-8e314e734ffe/disk --force-share --output=json" returned: 0 in 0.196s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:07:54 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-b7b37b35-34f8-4611-a2d1-7d64ab4aa82b tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] [instance: 23a31b1e-4005-442f-96e4-9fe82d9e17ba] End _get_guest_xml xml=<domain type="kvm">
Nov 29 03:07:54 user nova-compute[75234]:   <uuid>23a31b1e-4005-442f-96e4-9fe82d9e17ba</uuid>
Nov 29 03:07:54 user nova-compute[75234]:   <name>instance-0000000d</name>
Nov 29 03:07:54 user nova-compute[75234]:   <memory>524288</memory>
Nov 29 03:07:54 user nova-compute[75234]:   <vcpu>1</vcpu>
Nov 29 03:07:54 user nova-compute[75234]:   <metadata>
Nov 29 03:07:54 user nova-compute[75234]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Nov 29 03:07:54 user nova-compute[75234]:       <nova:package version="0.0.0"/>
Nov 29 03:07:54 user nova-compute[75234]:       <nova:name>tempest-ServersNegativeTestJSON-server-1398606202</nova:name>
Nov 29 03:07:54 user nova-compute[75234]:       <nova:creationTime>2023-11-29 03:07:54</nova:creationTime>
Nov 29 03:07:54 user nova-compute[75234]:       <nova:flavor name="m1.tiny">
Nov 29 03:07:54 user nova-compute[75234]:         <nova:memory>512</nova:memory>
Nov 29 03:07:54 user nova-compute[75234]:         <nova:disk>1</nova:disk>
Nov 29 03:07:54 user nova-compute[75234]:         <nova:swap>0</nova:swap>
Nov 29 03:07:54 user nova-compute[75234]:         <nova:ephemeral>0</nova:ephemeral>
Nov 29 03:07:54 user nova-compute[75234]:         <nova:vcpus>1</nova:vcpus>
Nov 29 03:07:54 user nova-compute[75234]:       </nova:flavor>
Nov 29 03:07:54 user nova-compute[75234]:       <nova:owner>
Nov 29 03:07:54 user nova-compute[75234]:         <nova:user uuid="178bfe0ea37e45d8abd2d6b45bd5660c">tempest-ServersNegativeTestJSON-20765276-project-member</nova:user>
Nov 29 03:07:54 user nova-compute[75234]:         <nova:project uuid="b877668e02eb441c85dc2a7003a94c7a">tempest-ServersNegativeTestJSON-20765276</nova:project>
Nov 29 03:07:54 user nova-compute[75234]:       </nova:owner>
Nov 29 03:07:54 user nova-compute[75234]:       <nova:root type="image" uuid="96c30ba7-9b73-4a30-8ba4-7138879573b7"/>
Nov 29 03:07:54 user nova-compute[75234]:       <nova:ports>
Nov 29 03:07:54 user nova-compute[75234]:         <nova:port uuid="5a8d4829-f681-4e2c-83d7-91e5ea0b1fd2">
Nov 29 03:07:54 user nova-compute[75234]:           <nova:ip type="fixed" address="10.0.0.3" ipVersion="4"/>
Nov 29 03:07:54 user nova-compute[75234]:         </nova:port>
Nov 29 03:07:54 user nova-compute[75234]:       </nova:ports>
Nov 29 03:07:54 user nova-compute[75234]:     </nova:instance>
Nov 29 03:07:54 user nova-compute[75234]:   </metadata>
Nov 29 03:07:54 user nova-compute[75234]:   <sysinfo type="smbios">
Nov 29 03:07:54 user nova-compute[75234]:     <system>
Nov 29 03:07:54 user nova-compute[75234]:       <entry name="manufacturer">OpenStack Foundation</entry>
Nov 29 03:07:54 user nova-compute[75234]:       <entry name="product">OpenStack Nova</entry>
Nov 29 03:07:54 user nova-compute[75234]:       <entry name="version">0.0.0</entry>
Nov 29 03:07:54 user nova-compute[75234]:       <entry name="serial">23a31b1e-4005-442f-96e4-9fe82d9e17ba</entry>
Nov 29 03:07:54 user nova-compute[75234]:       <entry name="uuid">23a31b1e-4005-442f-96e4-9fe82d9e17ba</entry>
Nov 29 03:07:54 user nova-compute[75234]:       <entry name="family">Virtual Machine</entry>
Nov 29 03:07:54 user nova-compute[75234]:     </system>
Nov 29 03:07:54 user nova-compute[75234]:   </sysinfo>
Nov 29 03:07:54 user nova-compute[75234]:   <os>
Nov 29 03:07:54 user nova-compute[75234]:     <type arch="x86_64" machine="pc">hvm</type>
Nov 29 03:07:54 user nova-compute[75234]:     <boot dev="hd"/>
Nov 29 03:07:54 user nova-compute[75234]:     <smbios mode="sysinfo"/>
Nov 29 03:07:54 user nova-compute[75234]:   </os>
Nov 29 03:07:54 user nova-compute[75234]:   <features>
Nov 29 03:07:54 user nova-compute[75234]:     <acpi/>
Nov 29 03:07:54 user nova-compute[75234]:     <apic/>
Nov 29 03:07:54 user nova-compute[75234]:     <vmcoreinfo/>
Nov 29 03:07:54 user nova-compute[75234]:   </features>
Nov 29 03:07:54 user nova-compute[75234]:   <clock offset="utc">
Nov 29 03:07:54 user nova-compute[75234]:     <timer name="pit" tickpolicy="delay"/>
Nov 29 03:07:54 user nova-compute[75234]:     <timer name="rtc" tickpolicy="catchup"/>
Nov 29 03:07:54 user nova-compute[75234]:     <timer name="hpet" present="no"/>
Nov 29 03:07:54 user nova-compute[75234]:   </clock>
Nov 29 03:07:54 user nova-compute[75234]:   <cpu mode="custom" match="exact">
Nov 29 03:07:54 user nova-compute[75234]:     <model>Nehalem</model>
Nov 29 03:07:54 user nova-compute[75234]:     <topology sockets="1" cores="1" threads="1"/>
Nov 29 03:07:54 user nova-compute[75234]:   </cpu>
Nov 29 03:07:54 user nova-compute[75234]:   <devices>
Nov 29 03:07:54 user nova-compute[75234]:     <disk type="file" device="disk">
Nov 29 03:07:54 user nova-compute[75234]:       <driver name="qemu" type="qcow2" cache="none"/>
Nov 29 03:07:54 user nova-compute[75234]:       <source file="/opt/stack/data/nova/instances/23a31b1e-4005-442f-96e4-9fe82d9e17ba/disk"/>
Nov 29 03:07:54 user nova-compute[75234]:       <target dev="vda" bus="virtio"/>
Nov 29 03:07:54 user nova-compute[75234]:     </disk>
Nov 29 03:07:54 user nova-compute[75234]:     <interface type="ethernet">
Nov 29 03:07:54 user nova-compute[75234]:       <mac address="fa:16:3e:1b:04:c4"/>
Nov 29 03:07:54 user nova-compute[75234]:       <model type="virtio"/>
Nov 29 03:07:54 user nova-compute[75234]:       <mtu size="1442"/>
Nov 29 03:07:54 user nova-compute[75234]:       <target dev="tap5a8d4829-f6"/>
Nov 29 03:07:54 user nova-compute[75234]:     </interface>
Nov 29 03:07:54 user nova-compute[75234]:     <serial type="pty">
Nov 29 03:07:54 user nova-compute[75234]:       <log file="/opt/stack/data/nova/instances/23a31b1e-4005-442f-96e4-9fe82d9e17ba/console.log" append="off"/>
Nov 29 03:07:54 user nova-compute[75234]:     </serial>
Nov 29 03:07:54 user nova-compute[75234]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Nov 29 03:07:54 user nova-compute[75234]:     <video>
Nov 29 03:07:54 user nova-compute[75234]:       <model type="virtio"/>
Nov 29 03:07:54 user nova-compute[75234]:     </video>
Nov 29 03:07:54 user nova-compute[75234]:     <rng model="virtio">
Nov 29 03:07:54 user nova-compute[75234]:       <backend model="random">/dev/urandom</backend>
Nov 29 03:07:54 user nova-compute[75234]:     </rng>
Nov 29 03:07:54 user nova-compute[75234]:     <controller type="usb" index="0" model="none"/>
Nov 29 03:07:54 user nova-compute[75234]:     <memballoon model="virtio">
Nov 29 03:07:54 user nova-compute[75234]:       <stats period="10"/>
Nov 29 03:07:54 user nova-compute[75234]:     </memballoon>
Nov 29 03:07:54 user nova-compute[75234]:   </devices>
Nov 29 03:07:54 user nova-compute[75234]: </domain>
Nov 29 03:07:54 user nova-compute[75234]:  {{(pid=75234) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7568}}
Nov 29 03:07:54 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-b7b37b35-34f8-4611-a2d1-7d64ab4aa82b tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-11-29T03:07:51Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-ServersNegativeTestJSON-server-1398606202',display_name='tempest-ServersNegativeTestJSON-server-1398606202',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-serversnegativetestjson-server-1398606202',id=13,image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data=None,key_name=None,keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='b877668e02eb441c85dc2a7003a94c7a',ramdisk_id='',reservation_id='r-zt0xoplh',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-ServersNegativeTestJSON-20765276',owner_user_name='tempest-ServersNegativeTestJSON-20765276-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-11-29T03:07:53Z,user_data=None,user_id='178bfe0ea37e45d8abd2d6b45bd5660c',uuid=23a31b1e-4005-442f-96e4-9fe82d9e17ba,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "5a8d4829-f681-4e2c-83d7-91e5ea0b1fd2", "address": "fa:16:3e:1b:04:c4", "network": {"id": "b27ac31e-3601-4055-be79-34b0946b0913", "bridge": "br-int", "label": "tempest-ServersNegativeTestJSON-1491620118-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "b877668e02eb441c85dc2a7003a94c7a", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap5a8d4829-f6", "ovs_interfaceid": "5a8d4829-f681-4e2c-83d7-91e5ea0b1fd2", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Nov 29 03:07:54 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-b7b37b35-34f8-4611-a2d1-7d64ab4aa82b tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] Converting VIF {"id": "5a8d4829-f681-4e2c-83d7-91e5ea0b1fd2", "address": "fa:16:3e:1b:04:c4", "network": {"id": "b27ac31e-3601-4055-be79-34b0946b0913", "bridge": "br-int", "label": "tempest-ServersNegativeTestJSON-1491620118-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "b877668e02eb441c85dc2a7003a94c7a", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap5a8d4829-f6", "ovs_interfaceid": "5a8d4829-f681-4e2c-83d7-91e5ea0b1fd2", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:07:54 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-b7b37b35-34f8-4611-a2d1-7d64ab4aa82b tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:1b:04:c4,bridge_name='br-int',has_traffic_filtering=True,id=5a8d4829-f681-4e2c-83d7-91e5ea0b1fd2,network=Network(b27ac31e-3601-4055-be79-34b0946b0913),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap5a8d4829-f6') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:07:54 user nova-compute[75234]: DEBUG os_vif [None req-b7b37b35-34f8-4611-a2d1-7d64ab4aa82b tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:1b:04:c4,bridge_name='br-int',has_traffic_filtering=True,id=5a8d4829-f681-4e2c-83d7-91e5ea0b1fd2,network=Network(b27ac31e-3601-4055-be79-34b0946b0913),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap5a8d4829-f6') {{(pid=75234) plug /opt/stack/data/venv/lib/python3.10/site-packages/os_vif/__init__.py:76}}
Nov 29 03:07:54 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:54 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:07:54 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Nov 29 03:07:54 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:54 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbCreateCommand(_result=None, table=QoS, columns={'type': 'linux-noop', 'external_ids': {'id': '9cbcaf3b-fa51-5fcd-978e-debac9379f97', '_type': 'linux-noop'}}, row=False) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:07:54 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:54 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:07:54 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/9f7876ad-0487-426d-8b19-b718dccdc4fe/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:07:54 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:54 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tap5a8d4829-f6, may_exist=True, interface_attrs={}) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:07:54 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Port, record=tap5a8d4829-f6, col_values=(('qos', UUID('d337d65a-a20d-45a4-89b3-ddbad22fb34f')),), if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:07:54 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DbSetCommand(_result=None, table=Interface, record=tap5a8d4829-f6, col_values=(('external_ids', {'iface-id': '5a8d4829-f681-4e2c-83d7-91e5ea0b1fd2', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:1b:04:c4', 'vm-uuid': '23a31b1e-4005-442f-96e4-9fe82d9e17ba'}),), if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:07:54 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:54 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:07:54 user nova-compute[75234]: INFO os_vif [None req-b7b37b35-34f8-4611-a2d1-7d64ab4aa82b tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:1b:04:c4,bridge_name='br-int',has_traffic_filtering=True,id=5a8d4829-f681-4e2c-83d7-91e5ea0b1fd2,network=Network(b27ac31e-3601-4055-be79-34b0946b0913),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap5a8d4829-f6')
Nov 29 03:07:54 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-b7b37b35-34f8-4611-a2d1-7d64ab4aa82b tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] No BDM found with device name vda, not building metadata. {{(pid=75234) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12089}}
Nov 29 03:07:54 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-b7b37b35-34f8-4611-a2d1-7d64ab4aa82b tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] No VIF found with MAC fa:16:3e:1b:04:c4, not building metadata {{(pid=75234) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Nov 29 03:07:54 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/9f7876ad-0487-426d-8b19-b718dccdc4fe/disk --force-share --output=json" returned: 0 in 0.233s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:07:54 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/9f7876ad-0487-426d-8b19-b718dccdc4fe/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:07:55 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/9f7876ad-0487-426d-8b19-b718dccdc4fe/disk --force-share --output=json" returned: 0 in 0.176s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:07:55 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/21e9c6cd-c39f-41c0-8768-5dd64589c3f3/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:07:55 user nova-compute[75234]: DEBUG nova.network.neutron [req-53982eeb-b07b-438e-b4ac-bcd555166c2e req-fac5c8b1-2b6f-4fd5-99a3-8d91b531feea service nova] [instance: 23a31b1e-4005-442f-96e4-9fe82d9e17ba] Updated VIF entry in instance network info cache for port 5a8d4829-f681-4e2c-83d7-91e5ea0b1fd2. {{(pid=75234) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3481}}
Nov 29 03:07:55 user nova-compute[75234]: DEBUG nova.network.neutron [req-53982eeb-b07b-438e-b4ac-bcd555166c2e req-fac5c8b1-2b6f-4fd5-99a3-8d91b531feea service nova] [instance: 23a31b1e-4005-442f-96e4-9fe82d9e17ba] Updating instance_info_cache with network_info: [{"id": "5a8d4829-f681-4e2c-83d7-91e5ea0b1fd2", "address": "fa:16:3e:1b:04:c4", "network": {"id": "b27ac31e-3601-4055-be79-34b0946b0913", "bridge": "br-int", "label": "tempest-ServersNegativeTestJSON-1491620118-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "b877668e02eb441c85dc2a7003a94c7a", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap5a8d4829-f6", "ovs_interfaceid": "5a8d4829-f681-4e2c-83d7-91e5ea0b1fd2", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:07:55 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-53982eeb-b07b-438e-b4ac-bcd555166c2e req-fac5c8b1-2b6f-4fd5-99a3-8d91b531feea service nova] Releasing lock "refresh_cache-23a31b1e-4005-442f-96e4-9fe82d9e17ba" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:07:55 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/21e9c6cd-c39f-41c0-8768-5dd64589c3f3/disk --force-share --output=json" returned: 0 in 0.191s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:07:55 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/21e9c6cd-c39f-41c0-8768-5dd64589c3f3/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:07:55 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/21e9c6cd-c39f-41c0-8768-5dd64589c3f3/disk --force-share --output=json" returned: 0 in 0.191s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:07:55 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f609a7b7-1be2-44b4-9177-7e72a4d4c060/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:07:55 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f609a7b7-1be2-44b4-9177-7e72a4d4c060/disk --force-share --output=json" returned: 0 in 0.221s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:07:55 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f609a7b7-1be2-44b4-9177-7e72a4d4c060/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:07:55 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f609a7b7-1be2-44b4-9177-7e72a4d4c060/disk --force-share --output=json" returned: 0 in 0.182s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:07:55 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/c9663ab3-3374-400e-ab0a-769731b78cca/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:07:56 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/c9663ab3-3374-400e-ab0a-769731b78cca/disk --force-share --output=json" returned: 0 in 0.200s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:07:56 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/c9663ab3-3374-400e-ab0a-769731b78cca/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:07:56 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:56 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/c9663ab3-3374-400e-ab0a-769731b78cca/disk --force-share --output=json" returned: 0 in 0.179s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:07:56 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f7d0bdeb-6a05-47ac-87f2-c8380a9a4286/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:07:56 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:56 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:56 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:56 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f7d0bdeb-6a05-47ac-87f2-c8380a9a4286/disk --force-share --output=json" returned: 0 in 0.374s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:07:56 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f7d0bdeb-6a05-47ac-87f2-c8380a9a4286/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:07:56 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f7d0bdeb-6a05-47ac-87f2-c8380a9a4286/disk --force-share --output=json" returned: 0 in 0.251s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:07:56 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/aa3c7432-bf15-408e-9fbb-993135734161/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:07:57 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:57 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/aa3c7432-bf15-408e-9fbb-993135734161/disk --force-share --output=json" returned: 0 in 0.240s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:07:57 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/aa3c7432-bf15-408e-9fbb-993135734161/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:07:57 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-2821575f-11ad-484a-9599-886a9bf4fd58 tempest-DeleteServersTestJSON-1668990756 tempest-DeleteServersTestJSON-1668990756-project-member] [instance: 9f7876ad-0487-426d-8b19-b718dccdc4fe] Calling os-brick to attach iSCSI Volume {{(pid=75234) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Nov 29 03:07:57 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-2821575f-11ad-484a-9599-886a9bf4fd58 tempest-DeleteServersTestJSON-1668990756 tempest-DeleteServersTestJSON-1668990756-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f8a5050c9a0>, 'connection_properties': {'target_iqn': 'iqn.2010-10.org.openstack:be635389-1f4c-4c39-8c36-25d81868770a', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'vMCHaGOW', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:176}}
Nov 29 03:07:57 user nova-compute[75234]: WARNING os_brick.initiator.connectors.base [None req-2821575f-11ad-484a-9599-886a9bf4fd58 tempest-DeleteServersTestJSON-1668990756 tempest-DeleteServersTestJSON-1668990756-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Nov 29 03:07:57 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-2821575f-11ad-484a-9599-886a9bf4fd58 tempest-DeleteServersTestJSON-1668990756 tempest-DeleteServersTestJSON-1668990756-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:68}}
Nov 29 03:07:57 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-2821575f-11ad-484a-9599-886a9bf4fd58 tempest-DeleteServersTestJSON-1668990756 tempest-DeleteServersTestJSON-1668990756-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.002s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:73}}
Nov 29 03:07:57 user nova-compute[75234]: WARNING os_brick.initiator.connectors.base [None req-2821575f-11ad-484a-9599-886a9bf4fd58 tempest-DeleteServersTestJSON-1668990756 tempest-DeleteServersTestJSON-1668990756-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Nov 29 03:07:57 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-2821575f-11ad-484a-9599-886a9bf4fd58 tempest-DeleteServersTestJSON-1668990756 tempest-DeleteServersTestJSON-1668990756-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:be635389-1f4c-4c39-8c36-25d81868770a" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:68}}
Nov 29 03:07:57 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-2821575f-11ad-484a-9599-886a9bf4fd58 tempest-DeleteServersTestJSON-1668990756 tempest-DeleteServersTestJSON-1668990756-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:be635389-1f4c-4c39-8c36-25d81868770a" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:73}}
Nov 29 03:07:57 user nova-compute[75234]: INFO os_brick.initiator.connectors.iscsi [None req-2821575f-11ad-484a-9599-886a9bf4fd58 tempest-DeleteServersTestJSON-1668990756 tempest-DeleteServersTestJSON-1668990756-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Nov 29 03:07:57 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:be635389-1f4c-4c39-8c36-25d81868770a -p 172.16.0.220:3260 {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:07:57 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:be635389-1f4c-4c39-8c36-25d81868770a -p 172.16.0.220:3260" returned: 21 in 0.020s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:07:57 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[78a0e58f-f016-4c40-bd09-d7ac0e1a794c]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:07:57 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-2821575f-11ad-484a-9599-886a9bf4fd58 tempest-DeleteServersTestJSON-1668990756 tempest-DeleteServersTestJSON-1668990756-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Nov 29 03:07:57 user nova-compute[75234]:  {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:07:57 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:be635389-1f4c-4c39-8c36-25d81868770a -p 172.16.0.220:3260 --interface default --op new {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:07:57 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:be635389-1f4c-4c39-8c36-25d81868770a -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.020s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:07:57 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[f1cf83f6-23f4-43f1-9e8a-9379ed8f4974]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:be635389-1f4c-4c39-8c36-25d81868770a] added\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:07:57 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-2821575f-11ad-484a-9599-886a9bf4fd58 tempest-DeleteServersTestJSON-1668990756 tempest-DeleteServersTestJSON-1668990756-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:be635389-1f4c-4c39-8c36-25d81868770a] added
Nov 29 03:07:57 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:07:57 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:be635389-1f4c-4c39-8c36-25d81868770a -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:07:57 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:be635389-1f4c-4c39-8c36-25d81868770a -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.027s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:07:57 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[d4180e58-0a54-4bcc-ae54-3d6e9b5df6f7]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:07:57 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-2821575f-11ad-484a-9599-886a9bf4fd58 tempest-DeleteServersTestJSON-1668990756 tempest-DeleteServersTestJSON-1668990756-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:07:57 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:be635389-1f4c-4c39-8c36-25d81868770a -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:07:57 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:be635389-1f4c-4c39-8c36-25d81868770a -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.023s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:07:57 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[045541c8-3579-45d9-9ce8-dc0d00c6b8d8]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:07:57 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-2821575f-11ad-484a-9599-886a9bf4fd58 tempest-DeleteServersTestJSON-1668990756 tempest-DeleteServersTestJSON-1668990756-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:07:57 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:be635389-1f4c-4c39-8c36-25d81868770a -p 172.16.0.220:3260 --op update -n node.session.auth.username -v vMCHaGOW {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:07:57 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/aa3c7432-bf15-408e-9fbb-993135734161/disk --force-share --output=json" returned: 0 in 0.268s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:07:57 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:be635389-1f4c-4c39-8c36-25d81868770a -p 172.16.0.220:3260 --op update -n node.session.auth.username -v vMCHaGOW" returned: 0 in 0.020s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:07:57 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[ca4fd1d3-1373-4d09-b7d1-bc9946857c38]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:07:57 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-2821575f-11ad-484a-9599-886a9bf4fd58 tempest-DeleteServersTestJSON-1668990756 tempest-DeleteServersTestJSON-1668990756-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'vMCHaGOW'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:07:57 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] skipping disk /dev/sdb (vda) as it is a volume {{(pid=75234) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11213}}
Nov 29 03:07:57 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] skipping disk /dev/sdc (vdb) as it is a volume {{(pid=75234) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11213}}
Nov 29 03:07:57 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] skipping disk /dev/sdd (vdc) as it is a volume {{(pid=75234) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11213}}
Nov 29 03:07:57 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:be635389-1f4c-4c39-8c36-25d81868770a -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:07:57 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:be635389-1f4c-4c39-8c36-25d81868770a -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.017s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:07:57 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[aa220883-ec0c-468e-80c3-ccb148368d0f]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:07:57 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-2821575f-11ad-484a-9599-886a9bf4fd58 tempest-DeleteServersTestJSON-1668990756 tempest-DeleteServersTestJSON-1668990756-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:07:57 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:07:59 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:07:59 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.029s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:07:59 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[2eaa8120-7e06-4514-8db7-cb99d2b3cf61]: (4, ('tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bb98586e-4f75-46be-a626-0b73ebda11eb (non-flash)\ntcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ab278679-6cfd-4c7c-a930-c078843640d3 (non-flash)\ntcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:920d6cad-25ab-4f3d-941b-8ec841d43184 (non-flash)\ntcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b1068fd1-6636-43cd-9725-4a37f708b2bc (non-flash)\ntcp: [6] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0d2c0c22-2b3a-43db-b9af-e60a2514e48e (non-flash)\ntcp: [7] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5fcd66fc-a479-4613-b1f8-e6ef12ca470a (non-flash)\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:07:59 user nova-compute[75234]: DEBUG nova.compute.manager [None req-b7b37b35-34f8-4611-a2d1-7d64ab4aa82b tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] [instance: 23a31b1e-4005-442f-96e4-9fe82d9e17ba] Instance event wait completed in 0 seconds for  {{(pid=75234) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Nov 29 03:07:59 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-b7b37b35-34f8-4611-a2d1-7d64ab4aa82b tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] [instance: 23a31b1e-4005-442f-96e4-9fe82d9e17ba] Guest created on hypervisor {{(pid=75234) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4421}}
Nov 29 03:07:59 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-2821575f-11ad-484a-9599-886a9bf4fd58 tempest-DeleteServersTestJSON-1668990756 tempest-DeleteServersTestJSON-1668990756-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bb98586e-4f75-46be-a626-0b73ebda11eb (non-flash)
Nov 29 03:07:59 user nova-compute[75234]: tcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ab278679-6cfd-4c7c-a930-c078843640d3 (non-flash)
Nov 29 03:07:59 user nova-compute[75234]: tcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:920d6cad-25ab-4f3d-941b-8ec841d43184 (non-flash)
Nov 29 03:07:59 user nova-compute[75234]: tcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b1068fd1-6636-43cd-9725-4a37f708b2bc (non-flash)
Nov 29 03:07:59 user nova-compute[75234]: tcp: [6] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0d2c0c22-2b3a-43db-b9af-e60a2514e48e (non-flash)
Nov 29 03:07:59 user nova-compute[75234]: tcp: [7] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5fcd66fc-a479-4613-b1f8-e6ef12ca470a (non-flash)
Nov 29 03:07:59 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:07:59 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-2821575f-11ad-484a-9599-886a9bf4fd58 tempest-DeleteServersTestJSON-1668990756 tempest-DeleteServersTestJSON-1668990756-project-member] iscsi session list stdout=tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bb98586e-4f75-46be-a626-0b73ebda11eb (non-flash)
Nov 29 03:07:59 user nova-compute[75234]: tcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ab278679-6cfd-4c7c-a930-c078843640d3 (non-flash)
Nov 29 03:07:59 user nova-compute[75234]: tcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:920d6cad-25ab-4f3d-941b-8ec841d43184 (non-flash)
Nov 29 03:07:59 user nova-compute[75234]: tcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b1068fd1-6636-43cd-9725-4a37f708b2bc (non-flash)
Nov 29 03:07:59 user nova-compute[75234]: tcp: [6] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0d2c0c22-2b3a-43db-b9af-e60a2514e48e (non-flash)
Nov 29 03:07:59 user nova-compute[75234]: tcp: [7] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5fcd66fc-a479-4613-b1f8-e6ef12ca470a (non-flash)
Nov 29 03:07:59 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsi_session /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Nov 29 03:07:59 user nova-compute[75234]: DEBUG nova.compute.manager [req-9f0fb700-b869-45f7-bdf3-5b1b21110d69 req-351c8285-7dac-4c58-ba46-f0d93140bfa2 service nova] [instance: 23a31b1e-4005-442f-96e4-9fe82d9e17ba] Received event network-vif-plugged-5a8d4829-f681-4e2c-83d7-91e5ea0b1fd2 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:07:59 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-9f0fb700-b869-45f7-bdf3-5b1b21110d69 req-351c8285-7dac-4c58-ba46-f0d93140bfa2 service nova] Acquiring lock "23a31b1e-4005-442f-96e4-9fe82d9e17ba-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:07:59 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-9f0fb700-b869-45f7-bdf3-5b1b21110d69 req-351c8285-7dac-4c58-ba46-f0d93140bfa2 service nova] Lock "23a31b1e-4005-442f-96e4-9fe82d9e17ba-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:07:59 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-9f0fb700-b869-45f7-bdf3-5b1b21110d69 req-351c8285-7dac-4c58-ba46-f0d93140bfa2 service nova] Lock "23a31b1e-4005-442f-96e4-9fe82d9e17ba-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:07:59 user nova-compute[75234]: DEBUG nova.compute.manager [req-9f0fb700-b869-45f7-bdf3-5b1b21110d69 req-351c8285-7dac-4c58-ba46-f0d93140bfa2 service nova] [instance: 23a31b1e-4005-442f-96e4-9fe82d9e17ba] No waiting events found dispatching network-vif-plugged-5a8d4829-f681-4e2c-83d7-91e5ea0b1fd2 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:07:59 user nova-compute[75234]: WARNING nova.compute.manager [req-9f0fb700-b869-45f7-bdf3-5b1b21110d69 req-351c8285-7dac-4c58-ba46-f0d93140bfa2 service nova] [instance: 23a31b1e-4005-442f-96e4-9fe82d9e17ba] Received unexpected event network-vif-plugged-5a8d4829-f681-4e2c-83d7-91e5ea0b1fd2 for instance with vm_state building and task_state spawning.
Nov 29 03:07:59 user nova-compute[75234]: DEBUG nova.compute.manager [req-a1384c8a-c341-41aa-b315-7c8f2cf8f752 req-3f79578d-e29f-4463-b74a-e3bca4e35803 service nova] [instance: a9350a18-14e6-42c6-a150-0a1487f20ad9] Received event network-changed-52024efc-8cde-4610-b12c-198f19d75ccf {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:07:59 user nova-compute[75234]: DEBUG nova.compute.manager [req-a1384c8a-c341-41aa-b315-7c8f2cf8f752 req-3f79578d-e29f-4463-b74a-e3bca4e35803 service nova] [instance: a9350a18-14e6-42c6-a150-0a1487f20ad9] Refreshing instance network info cache due to event network-changed-52024efc-8cde-4610-b12c-198f19d75ccf. {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11117}}
Nov 29 03:07:59 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-a1384c8a-c341-41aa-b315-7c8f2cf8f752 req-3f79578d-e29f-4463-b74a-e3bca4e35803 service nova] Acquiring lock "refresh_cache-a9350a18-14e6-42c6-a150-0a1487f20ad9" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:07:59 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-a1384c8a-c341-41aa-b315-7c8f2cf8f752 req-3f79578d-e29f-4463-b74a-e3bca4e35803 service nova] Acquired lock "refresh_cache-a9350a18-14e6-42c6-a150-0a1487f20ad9" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:07:59 user nova-compute[75234]: DEBUG nova.network.neutron [req-a1384c8a-c341-41aa-b315-7c8f2cf8f752 req-3f79578d-e29f-4463-b74a-e3bca4e35803 service nova] [instance: a9350a18-14e6-42c6-a150-0a1487f20ad9] Refreshing network info cache for port 52024efc-8cde-4610-b12c-198f19d75ccf {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2006}}
Nov 29 03:07:59 user nova-compute[75234]: DEBUG nova.compute.manager [req-c0b89aad-e8de-4a3d-8144-d84e42ee6680 req-a0ca8ed4-68c7-42bb-b535-8c7519aefb2d service nova] [instance: f609a7b7-1be2-44b4-9177-7e72a4d4c060] Received event network-changed-5543f379-2b3a-4dc2-bfe9-52f66a942257 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:07:59 user nova-compute[75234]: DEBUG nova.compute.manager [req-c0b89aad-e8de-4a3d-8144-d84e42ee6680 req-a0ca8ed4-68c7-42bb-b535-8c7519aefb2d service nova] [instance: f609a7b7-1be2-44b4-9177-7e72a4d4c060] Refreshing instance network info cache due to event network-changed-5543f379-2b3a-4dc2-bfe9-52f66a942257. {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11117}}
Nov 29 03:07:59 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-c0b89aad-e8de-4a3d-8144-d84e42ee6680 req-a0ca8ed4-68c7-42bb-b535-8c7519aefb2d service nova] Acquiring lock "refresh_cache-f609a7b7-1be2-44b4-9177-7e72a4d4c060" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:07:59 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-c0b89aad-e8de-4a3d-8144-d84e42ee6680 req-a0ca8ed4-68c7-42bb-b535-8c7519aefb2d service nova] Acquired lock "refresh_cache-f609a7b7-1be2-44b4-9177-7e72a4d4c060" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:07:59 user nova-compute[75234]: DEBUG nova.network.neutron [req-c0b89aad-e8de-4a3d-8144-d84e42ee6680 req-a0ca8ed4-68c7-42bb-b535-8c7519aefb2d service nova] [instance: f609a7b7-1be2-44b4-9177-7e72a4d4c060] Refreshing network info cache for port 5543f379-2b3a-4dc2-bfe9-52f66a942257 {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2006}}
Nov 29 03:07:59 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:be635389-1f4c-4c39-8c36-25d81868770a -p 172.16.0.220:3260 --login {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:07:59 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:07:59 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:07:59 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Hypervisor/Node resource view: name=user free_ram=6744MB free_disk=25.569095611572266GB free_vcpus=0 pci_devices=[{"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}] {{(pid=75234) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1106}}
Nov 29 03:07:59 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:07:59 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:07:59 user nova-compute[75234]: INFO nova.virt.libvirt.driver [-] [instance: 23a31b1e-4005-442f-96e4-9fe82d9e17ba] Instance spawned successfully.
Nov 29 03:07:59 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-b7b37b35-34f8-4611-a2d1-7d64ab4aa82b tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] [instance: 23a31b1e-4005-442f-96e4-9fe82d9e17ba] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:904}}
Nov 29 03:08:00 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-b7b37b35-34f8-4611-a2d1-7d64ab4aa82b tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] [instance: 23a31b1e-4005-442f-96e4-9fe82d9e17ba] Found default for hw_cdrom_bus of ide {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:08:00 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-b7b37b35-34f8-4611-a2d1-7d64ab4aa82b tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] [instance: 23a31b1e-4005-442f-96e4-9fe82d9e17ba] Found default for hw_disk_bus of virtio {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:08:00 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-b7b37b35-34f8-4611-a2d1-7d64ab4aa82b tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] [instance: 23a31b1e-4005-442f-96e4-9fe82d9e17ba] Found default for hw_input_bus of None {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:08:00 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-b7b37b35-34f8-4611-a2d1-7d64ab4aa82b tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] [instance: 23a31b1e-4005-442f-96e4-9fe82d9e17ba] Found default for hw_pointer_model of None {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:08:00 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-b7b37b35-34f8-4611-a2d1-7d64ab4aa82b tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] [instance: 23a31b1e-4005-442f-96e4-9fe82d9e17ba] Found default for hw_video_model of virtio {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:08:00 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-b7b37b35-34f8-4611-a2d1-7d64ab4aa82b tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] [instance: 23a31b1e-4005-442f-96e4-9fe82d9e17ba] Found default for hw_vif_model of virtio {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:08:00 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:be635389-1f4c-4c39-8c36-25d81868770a -p 172.16.0.220:3260 --login" returned: 0 in 0.102s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:08:00 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[64b20849-15b7-44cd-ba50-df80c9c28a24]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:be635389-1f4c-4c39-8c36-25d81868770a, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:be635389-1f4c-4c39-8c36-25d81868770a, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:08:00 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-2821575f-11ad-484a-9599-886a9bf4fd58 tempest-DeleteServersTestJSON-1668990756 tempest-DeleteServersTestJSON-1668990756-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:be635389-1f4c-4c39-8c36-25d81868770a, portal: 172.16.0.220,3260]
Nov 29 03:08:00 user nova-compute[75234]: Login to [iface: default, target: iqn.2010-10.org.openstack:be635389-1f4c-4c39-8c36-25d81868770a, portal: 172.16.0.220,3260] successful.
Nov 29 03:08:00 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:08:00 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:be635389-1f4c-4c39-8c36-25d81868770a -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:08:00 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:be635389-1f4c-4c39-8c36-25d81868770a -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.030s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:08:00 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[4595bc3a-e739-4436-aa0e-80b5938b7d88]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:08:00 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-2821575f-11ad-484a-9599-886a9bf4fd58 tempest-DeleteServersTestJSON-1668990756 tempest-DeleteServersTestJSON-1668990756-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:08:00 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:08:00 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.032s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:08:00 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[ef50c041-9e37-4f6a-a7bb-715fd745f538]: (4, ('tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bb98586e-4f75-46be-a626-0b73ebda11eb (non-flash)\ntcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ab278679-6cfd-4c7c-a930-c078843640d3 (non-flash)\ntcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:920d6cad-25ab-4f3d-941b-8ec841d43184 (non-flash)\ntcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b1068fd1-6636-43cd-9725-4a37f708b2bc (non-flash)\ntcp: [6] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0d2c0c22-2b3a-43db-b9af-e60a2514e48e (non-flash)\ntcp: [7] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5fcd66fc-a479-4613-b1f8-e6ef12ca470a (non-flash)\ntcp: [8] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:be635389-1f4c-4c39-8c36-25d81868770a (non-flash)\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:08:00 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-2821575f-11ad-484a-9599-886a9bf4fd58 tempest-DeleteServersTestJSON-1668990756 tempest-DeleteServersTestJSON-1668990756-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bb98586e-4f75-46be-a626-0b73ebda11eb (non-flash)
Nov 29 03:08:00 user nova-compute[75234]: tcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ab278679-6cfd-4c7c-a930-c078843640d3 (non-flash)
Nov 29 03:08:00 user nova-compute[75234]: tcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:920d6cad-25ab-4f3d-941b-8ec841d43184 (non-flash)
Nov 29 03:08:00 user nova-compute[75234]: tcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b1068fd1-6636-43cd-9725-4a37f708b2bc (non-flash)
Nov 29 03:08:00 user nova-compute[75234]: tcp: [6] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0d2c0c22-2b3a-43db-b9af-e60a2514e48e (non-flash)
Nov 29 03:08:00 user nova-compute[75234]: tcp: [7] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5fcd66fc-a479-4613-b1f8-e6ef12ca470a (non-flash)
Nov 29 03:08:00 user nova-compute[75234]: tcp: [8] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:be635389-1f4c-4c39-8c36-25d81868770a (non-flash)
Nov 29 03:08:00 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:08:00 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-2821575f-11ad-484a-9599-886a9bf4fd58 tempest-DeleteServersTestJSON-1668990756 tempest-DeleteServersTestJSON-1668990756-project-member] iscsi session list stdout=tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bb98586e-4f75-46be-a626-0b73ebda11eb (non-flash)
Nov 29 03:08:00 user nova-compute[75234]: tcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ab278679-6cfd-4c7c-a930-c078843640d3 (non-flash)
Nov 29 03:08:00 user nova-compute[75234]: tcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:920d6cad-25ab-4f3d-941b-8ec841d43184 (non-flash)
Nov 29 03:08:00 user nova-compute[75234]: tcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b1068fd1-6636-43cd-9725-4a37f708b2bc (non-flash)
Nov 29 03:08:00 user nova-compute[75234]: tcp: [6] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0d2c0c22-2b3a-43db-b9af-e60a2514e48e (non-flash)
Nov 29 03:08:00 user nova-compute[75234]: tcp: [7] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5fcd66fc-a479-4613-b1f8-e6ef12ca470a (non-flash)
Nov 29 03:08:00 user nova-compute[75234]: tcp: [8] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:be635389-1f4c-4c39-8c36-25d81868770a (non-flash)
Nov 29 03:08:00 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsi_session /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Nov 29 03:08:00 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-2821575f-11ad-484a-9599-886a9bf4fd58 tempest-DeleteServersTestJSON-1668990756 tempest-DeleteServersTestJSON-1668990756-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:be635389-1f4c-4c39-8c36-25d81868770a" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 2.806s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:87}}
Nov 29 03:08:00 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-2821575f-11ad-484a-9599-886a9bf4fd58 tempest-DeleteServersTestJSON-1668990756 tempest-DeleteServersTestJSON-1668990756-project-member] Connected to 172.16.0.220:3260 {{(pid=75234) _connect_vol /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:644}}
Nov 29 03:08:00 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-2821575f-11ad-484a-9599-886a9bf4fd58 tempest-DeleteServersTestJSON-1668990756 tempest-DeleteServersTestJSON-1668990756-project-member] HCTL ('39', '-', '-', 0) found on session 8 with lun 0 {{(pid=75234) get_hctl /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:770}}
Nov 29 03:08:00 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-2821575f-11ad-484a-9599-886a9bf4fd58 tempest-DeleteServersTestJSON-1668990756 tempest-DeleteServersTestJSON-1668990756-project-member] Scanning host 39 c: -, t: -, l: 0) {{(pid=75234) scan_iscsi /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:795}}
Nov 29 03:08:00 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host39/scan {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:08:00 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host39/scan" returned: 0 in 0.031s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:08:00 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[ccd667a5-b51d-470a-9ddd-3ecaa1b065ca]: (4, ('- - 0', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:08:00 user nova-compute[75234]: INFO nova.compute.manager [None req-b7b37b35-34f8-4611-a2d1-7d64ab4aa82b tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] [instance: 23a31b1e-4005-442f-96e4-9fe82d9e17ba] Took 7.41 seconds to spawn the instance on the hypervisor.
Nov 29 03:08:00 user nova-compute[75234]: DEBUG nova.compute.manager [None req-b7b37b35-34f8-4611-a2d1-7d64ab4aa82b tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] [instance: 23a31b1e-4005-442f-96e4-9fe82d9e17ba] Checking state {{(pid=75234) _get_power_state /opt/stack/nova/nova/compute/manager.py:1782}}
Nov 29 03:08:00 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-2821575f-11ad-484a-9599-886a9bf4fd58 tempest-DeleteServersTestJSON-1668990756 tempest-DeleteServersTestJSON-1668990756-project-member] Searching for a device in session 8 and hctl ['39', '*', '*', 0] yield: sdh {{(pid=75234) device_name_by_hctl /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:789}}
Nov 29 03:08:00 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-2821575f-11ad-484a-9599-886a9bf4fd58 tempest-DeleteServersTestJSON-1668990756 tempest-DeleteServersTestJSON-1668990756-project-member] Connected to sdh using {'target_iqn': 'iqn.2010-10.org.openstack:be635389-1f4c-4c39-8c36-25d81868770a', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'vMCHaGOW', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False} {{(pid=75234) _connect_vol /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:672}}
Nov 29 03:08:00 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance 9f7876ad-0487-426d-8b19-b718dccdc4fe actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:08:00 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance 2bf9fe41-63c2-4882-bcdd-8e314e734ffe actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:08:00 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance 21e9c6cd-c39f-41c0-8768-5dd64589c3f3 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:08:00 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance a9350a18-14e6-42c6-a150-0a1487f20ad9 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:08:00 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance f609a7b7-1be2-44b4-9177-7e72a4d4c060 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:08:00 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance 8297c264-a213-44ee-be46-5c08eabda9f7 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:08:00 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance da625f9c-b7ed-4afc-875c-6436bf58e9cc actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:08:00 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance aa3c7432-bf15-408e-9fbb-993135734161 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:08:00 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance c9663ab3-3374-400e-ab0a-769731b78cca actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:08:00 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance cb569e8d-e4a5-4076-bb8b-deeefad033e2 actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:08:00 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance 763a3e14-4563-4dc2-99ae-c340edbb93df actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:08:00 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance f7d0bdeb-6a05-47ac-87f2-c8380a9a4286 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:08:00 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance 23a31b1e-4005-442f-96e4-9fe82d9e17ba actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:08:00 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Total usable vcpus: 12, total allocated vcpus: 13 {{(pid=75234) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1129}}
Nov 29 03:08:00 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Final resource view: name=user phys_ram=16011MB used_ram=7168MB phys_disk=40GB used_disk=11GB total_vcpus=12 used_vcpus=13 pci_stats=[] {{(pid=75234) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1138}}
Nov 29 03:08:00 user nova-compute[75234]: DEBUG nova.compute.manager [req-0bb5553e-4845-4ac3-a097-1e96bc7315e7 req-065e99c8-9635-45a4-b501-5c4e1f015ff0 service nova] [instance: 21e9c6cd-c39f-41c0-8768-5dd64589c3f3] Received event network-changed-87f1e736-6665-4945-bd19-0cf93bff905e {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:08:00 user nova-compute[75234]: DEBUG nova.compute.manager [req-0bb5553e-4845-4ac3-a097-1e96bc7315e7 req-065e99c8-9635-45a4-b501-5c4e1f015ff0 service nova] [instance: 21e9c6cd-c39f-41c0-8768-5dd64589c3f3] Refreshing instance network info cache due to event network-changed-87f1e736-6665-4945-bd19-0cf93bff905e. {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11117}}
Nov 29 03:08:00 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-0bb5553e-4845-4ac3-a097-1e96bc7315e7 req-065e99c8-9635-45a4-b501-5c4e1f015ff0 service nova] Acquiring lock "refresh_cache-21e9c6cd-c39f-41c0-8768-5dd64589c3f3" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:08:00 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-0bb5553e-4845-4ac3-a097-1e96bc7315e7 req-065e99c8-9635-45a4-b501-5c4e1f015ff0 service nova] Acquired lock "refresh_cache-21e9c6cd-c39f-41c0-8768-5dd64589c3f3" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:08:00 user nova-compute[75234]: DEBUG nova.network.neutron [req-0bb5553e-4845-4ac3-a097-1e96bc7315e7 req-065e99c8-9635-45a4-b501-5c4e1f015ff0 service nova] [instance: 21e9c6cd-c39f-41c0-8768-5dd64589c3f3] Refreshing network info cache for port 87f1e736-6665-4945-bd19-0cf93bff905e {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2006}}
Nov 29 03:08:00 user nova-compute[75234]: INFO nova.compute.manager [None req-b7b37b35-34f8-4611-a2d1-7d64ab4aa82b tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] [instance: 23a31b1e-4005-442f-96e4-9fe82d9e17ba] Took 8.41 seconds to build instance.
Nov 29 03:08:00 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-b7b37b35-34f8-4611-a2d1-7d64ab4aa82b tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] Lock "23a31b1e-4005-442f-96e4-9fe82d9e17ba" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 8.540s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:08:00 user nova-compute[75234]: DEBUG nova.network.neutron [req-a1384c8a-c341-41aa-b315-7c8f2cf8f752 req-3f79578d-e29f-4463-b74a-e3bca4e35803 service nova] [instance: a9350a18-14e6-42c6-a150-0a1487f20ad9] Updated VIF entry in instance network info cache for port 52024efc-8cde-4610-b12c-198f19d75ccf. {{(pid=75234) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3481}}
Nov 29 03:08:00 user nova-compute[75234]: DEBUG nova.network.neutron [req-a1384c8a-c341-41aa-b315-7c8f2cf8f752 req-3f79578d-e29f-4463-b74a-e3bca4e35803 service nova] [instance: a9350a18-14e6-42c6-a150-0a1487f20ad9] Updating instance_info_cache with network_info: [{"id": "52024efc-8cde-4610-b12c-198f19d75ccf", "address": "fa:16:3e:09:b8:70", "network": {"id": "31c4ca3b-3a2f-47df-87fa-4ad095ee5cf9", "bridge": "br-int", "label": "tempest-ServerActionsTestOtherB-1680745008-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.9", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "ad5ea22c63f04f91a3f2fa665e4fc8a1", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap52024efc-8c", "ovs_interfaceid": "52024efc-8cde-4610-b12c-198f19d75ccf", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:08:00 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-a1384c8a-c341-41aa-b315-7c8f2cf8f752 req-3f79578d-e29f-4463-b74a-e3bca4e35803 service nova] Releasing lock "refresh_cache-a9350a18-14e6-42c6-a150-0a1487f20ad9" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:08:00 user nova-compute[75234]: DEBUG nova.compute.provider_tree [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Inventory has not changed in ProviderTree for provider: 4d291220-7c37-4839-a0e3-b23c66502b74 {{(pid=75234) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Nov 29 03:08:00 user nova-compute[75234]: DEBUG nova.scheduler.client.report [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Inventory has not changed for provider 4d291220-7c37-4839-a0e3-b23c66502b74 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=75234) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Nov 29 03:08:00 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Compute_service record updated for user:user {{(pid=75234) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:1067}}
Nov 29 03:08:00 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.847s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:08:01 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:08:01 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-2821575f-11ad-484a-9599-886a9bf4fd58 tempest-DeleteServersTestJSON-1668990756 tempest-DeleteServersTestJSON-1668990756-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 3.891s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:87}}
Nov 29 03:08:01 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-2821575f-11ad-484a-9599-886a9bf4fd58 tempest-DeleteServersTestJSON-1668990756 tempest-DeleteServersTestJSON-1668990756-project-member] <== connect_volume: return (3893ms) {'type': 'block', 'scsi_wwn': '23531396232316561', 'path': '/dev/sdh'} {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:203}}
Nov 29 03:08:01 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-2821575f-11ad-484a-9599-886a9bf4fd58 tempest-DeleteServersTestJSON-1668990756 tempest-DeleteServersTestJSON-1668990756-project-member] [instance: 9f7876ad-0487-426d-8b19-b718dccdc4fe] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '23531396232316561', 'path': '/dev/sdh'} {{(pid=75234) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Nov 29 03:08:01 user nova-compute[75234]: DEBUG nova.objects.instance [None req-2821575f-11ad-484a-9599-886a9bf4fd58 tempest-DeleteServersTestJSON-1668990756 tempest-DeleteServersTestJSON-1668990756-project-member] Lazy-loading 'flavor' on Instance uuid 9f7876ad-0487-426d-8b19-b718dccdc4fe {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:08:01 user nova-compute[75234]: DEBUG nova.virt.libvirt.guest [None req-2821575f-11ad-484a-9599-886a9bf4fd58 tempest-DeleteServersTestJSON-1668990756 tempest-DeleteServersTestJSON-1668990756-project-member] attach device xml: <disk type="block" device="disk">
Nov 29 03:08:01 user nova-compute[75234]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Nov 29 03:08:01 user nova-compute[75234]:   <source dev="/dev/sdh"/>
Nov 29 03:08:01 user nova-compute[75234]:   <target dev="vdb" bus="virtio"/>
Nov 29 03:08:01 user nova-compute[75234]:   <serial>be635389-1f4c-4c39-8c36-25d81868770a</serial>
Nov 29 03:08:01 user nova-compute[75234]: </disk>
Nov 29 03:08:01 user nova-compute[75234]:  {{(pid=75234) attach_device /opt/stack/nova/nova/virt/libvirt/guest.py:339}}
Nov 29 03:08:01 user nova-compute[75234]: DEBUG nova.network.neutron [req-0bb5553e-4845-4ac3-a097-1e96bc7315e7 req-065e99c8-9635-45a4-b501-5c4e1f015ff0 service nova] [instance: 21e9c6cd-c39f-41c0-8768-5dd64589c3f3] Updated VIF entry in instance network info cache for port 87f1e736-6665-4945-bd19-0cf93bff905e. {{(pid=75234) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3481}}
Nov 29 03:08:01 user nova-compute[75234]: DEBUG nova.network.neutron [req-0bb5553e-4845-4ac3-a097-1e96bc7315e7 req-065e99c8-9635-45a4-b501-5c4e1f015ff0 service nova] [instance: 21e9c6cd-c39f-41c0-8768-5dd64589c3f3] Updating instance_info_cache with network_info: [{"id": "87f1e736-6665-4945-bd19-0cf93bff905e", "address": "fa:16:3e:40:14:df", "network": {"id": "1f946e69-dbec-429c-83f8-a186bad4f2cc", "bridge": "br-int", "label": "tempest-ServerActionsTestOtherA-486369423-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.64", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "c1cb3d1f12f242a785a688d59dbe3cd6", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap87f1e736-66", "ovs_interfaceid": "87f1e736-6665-4945-bd19-0cf93bff905e", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:08:01 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-0bb5553e-4845-4ac3-a097-1e96bc7315e7 req-065e99c8-9635-45a4-b501-5c4e1f015ff0 service nova] Releasing lock "refresh_cache-21e9c6cd-c39f-41c0-8768-5dd64589c3f3" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:08:01 user nova-compute[75234]: DEBUG nova.compute.manager [req-0bb5553e-4845-4ac3-a097-1e96bc7315e7 req-065e99c8-9635-45a4-b501-5c4e1f015ff0 service nova] [instance: 23a31b1e-4005-442f-96e4-9fe82d9e17ba] Received event network-vif-plugged-5a8d4829-f681-4e2c-83d7-91e5ea0b1fd2 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:08:01 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-0bb5553e-4845-4ac3-a097-1e96bc7315e7 req-065e99c8-9635-45a4-b501-5c4e1f015ff0 service nova] Acquiring lock "23a31b1e-4005-442f-96e4-9fe82d9e17ba-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:08:01 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-0bb5553e-4845-4ac3-a097-1e96bc7315e7 req-065e99c8-9635-45a4-b501-5c4e1f015ff0 service nova] Lock "23a31b1e-4005-442f-96e4-9fe82d9e17ba-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:08:01 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-0bb5553e-4845-4ac3-a097-1e96bc7315e7 req-065e99c8-9635-45a4-b501-5c4e1f015ff0 service nova] Lock "23a31b1e-4005-442f-96e4-9fe82d9e17ba-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.011s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:08:01 user nova-compute[75234]: DEBUG nova.compute.manager [req-0bb5553e-4845-4ac3-a097-1e96bc7315e7 req-065e99c8-9635-45a4-b501-5c4e1f015ff0 service nova] [instance: 23a31b1e-4005-442f-96e4-9fe82d9e17ba] No waiting events found dispatching network-vif-plugged-5a8d4829-f681-4e2c-83d7-91e5ea0b1fd2 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:08:01 user nova-compute[75234]: WARNING nova.compute.manager [req-0bb5553e-4845-4ac3-a097-1e96bc7315e7 req-065e99c8-9635-45a4-b501-5c4e1f015ff0 service nova] [instance: 23a31b1e-4005-442f-96e4-9fe82d9e17ba] Received unexpected event network-vif-plugged-5a8d4829-f681-4e2c-83d7-91e5ea0b1fd2 for instance with vm_state building and task_state spawning.
Nov 29 03:08:01 user nova-compute[75234]: DEBUG nova.network.neutron [req-c0b89aad-e8de-4a3d-8144-d84e42ee6680 req-a0ca8ed4-68c7-42bb-b535-8c7519aefb2d service nova] [instance: f609a7b7-1be2-44b4-9177-7e72a4d4c060] Updated VIF entry in instance network info cache for port 5543f379-2b3a-4dc2-bfe9-52f66a942257. {{(pid=75234) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3481}}
Nov 29 03:08:01 user nova-compute[75234]: DEBUG nova.network.neutron [req-c0b89aad-e8de-4a3d-8144-d84e42ee6680 req-a0ca8ed4-68c7-42bb-b535-8c7519aefb2d service nova] [instance: f609a7b7-1be2-44b4-9177-7e72a4d4c060] Updating instance_info_cache with network_info: [{"id": "5543f379-2b3a-4dc2-bfe9-52f66a942257", "address": "fa:16:3e:04:ae:43", "network": {"id": "294c3aac-8c9b-45f0-8c0a-344e2d7d8b7c", "bridge": "br-int", "label": "tempest-AttachVolumeTestJSON-700098935-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.14", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "e393d23e783b4a40986674d782606c69", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap5543f379-2b", "ovs_interfaceid": "5543f379-2b3a-4dc2-bfe9-52f66a942257", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:08:01 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-c0b89aad-e8de-4a3d-8144-d84e42ee6680 req-a0ca8ed4-68c7-42bb-b535-8c7519aefb2d service nova] Releasing lock "refresh_cache-f609a7b7-1be2-44b4-9177-7e72a4d4c060" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:08:01 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-2821575f-11ad-484a-9599-886a9bf4fd58 tempest-DeleteServersTestJSON-1668990756 tempest-DeleteServersTestJSON-1668990756-project-member] No BDM found with device name vda, not building metadata. {{(pid=75234) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12089}}
Nov 29 03:08:01 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-2821575f-11ad-484a-9599-886a9bf4fd58 tempest-DeleteServersTestJSON-1668990756 tempest-DeleteServersTestJSON-1668990756-project-member] No BDM found with device name vdb, not building metadata. {{(pid=75234) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12089}}
Nov 29 03:08:01 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-2821575f-11ad-484a-9599-886a9bf4fd58 tempest-DeleteServersTestJSON-1668990756 tempest-DeleteServersTestJSON-1668990756-project-member] No VIF found with MAC fa:16:3e:fd:bb:d1, not building metadata {{(pid=75234) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Nov 29 03:08:01 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:08:01 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:08:01 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-2821575f-11ad-484a-9599-886a9bf4fd58 tempest-DeleteServersTestJSON-1668990756 tempest-DeleteServersTestJSON-1668990756-project-member] Lock "9f7876ad-0487-426d-8b19-b718dccdc4fe" "released" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: held 13.677s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:08:02 user nova-compute[75234]: DEBUG nova.compute.manager [None req-e6a4506b-8f08-442e-ab49-ab610a8b34fc tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] [instance: 8297c264-a213-44ee-be46-5c08eabda9f7] Checking state {{(pid=75234) _get_power_state /opt/stack/nova/nova/compute/manager.py:1782}}
Nov 29 03:08:02 user nova-compute[75234]: INFO nova.compute.manager [None req-e6a4506b-8f08-442e-ab49-ab610a8b34fc tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] [instance: 8297c264-a213-44ee-be46-5c08eabda9f7] instance snapshotting
Nov 29 03:08:02 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-e6a4506b-8f08-442e-ab49-ab610a8b34fc tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] [instance: 8297c264-a213-44ee-be46-5c08eabda9f7] Beginning live snapshot process
Nov 29 03:08:02 user nova-compute[75234]: DEBUG nova.compute.manager [req-1a6dbe9a-fe3a-4df9-b2f6-ff3101fd6e7c req-224a6ff3-2ce1-4809-b496-63e4f7ccc86d service nova] [instance: 8297c264-a213-44ee-be46-5c08eabda9f7] Received event network-changed-d26334d2-8040-4022-b658-69be1cf6a025 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:08:02 user nova-compute[75234]: DEBUG nova.compute.manager [req-1a6dbe9a-fe3a-4df9-b2f6-ff3101fd6e7c req-224a6ff3-2ce1-4809-b496-63e4f7ccc86d service nova] [instance: 8297c264-a213-44ee-be46-5c08eabda9f7] Refreshing instance network info cache due to event network-changed-d26334d2-8040-4022-b658-69be1cf6a025. {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11117}}
Nov 29 03:08:02 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-1a6dbe9a-fe3a-4df9-b2f6-ff3101fd6e7c req-224a6ff3-2ce1-4809-b496-63e4f7ccc86d service nova] Acquiring lock "refresh_cache-8297c264-a213-44ee-be46-5c08eabda9f7" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:08:02 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-1a6dbe9a-fe3a-4df9-b2f6-ff3101fd6e7c req-224a6ff3-2ce1-4809-b496-63e4f7ccc86d service nova] Acquired lock "refresh_cache-8297c264-a213-44ee-be46-5c08eabda9f7" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:08:02 user nova-compute[75234]: DEBUG nova.network.neutron [req-1a6dbe9a-fe3a-4df9-b2f6-ff3101fd6e7c req-224a6ff3-2ce1-4809-b496-63e4f7ccc86d service nova] [instance: 8297c264-a213-44ee-be46-5c08eabda9f7] Refreshing network info cache for port d26334d2-8040-4022-b658-69be1cf6a025 {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2006}}
Nov 29 03:08:02 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-e6a4506b-8f08-442e-ab49-ab610a8b34fc tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/8297c264-a213-44ee-be46-5c08eabda9f7/disk --force-share --output=json -f qcow2 {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:08:02 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-e6a4506b-8f08-442e-ab49-ab610a8b34fc tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/8297c264-a213-44ee-be46-5c08eabda9f7/disk --force-share --output=json -f qcow2" returned: 0 in 0.194s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:08:02 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-e6a4506b-8f08-442e-ab49-ab610a8b34fc tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/8297c264-a213-44ee-be46-5c08eabda9f7/disk --force-share --output=json -f qcow2 {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:08:02 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-e91f7847-953e-40f1-a6f9-601a068df520 tempest-DeleteServersTestJSON-1668990756 tempest-DeleteServersTestJSON-1668990756-project-member] Acquiring lock "9f7876ad-0487-426d-8b19-b718dccdc4fe" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:08:02 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-e91f7847-953e-40f1-a6f9-601a068df520 tempest-DeleteServersTestJSON-1668990756 tempest-DeleteServersTestJSON-1668990756-project-member] Lock "9f7876ad-0487-426d-8b19-b718dccdc4fe" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.003s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:08:02 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-e91f7847-953e-40f1-a6f9-601a068df520 tempest-DeleteServersTestJSON-1668990756 tempest-DeleteServersTestJSON-1668990756-project-member] Acquiring lock "9f7876ad-0487-426d-8b19-b718dccdc4fe-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:08:02 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-e91f7847-953e-40f1-a6f9-601a068df520 tempest-DeleteServersTestJSON-1668990756 tempest-DeleteServersTestJSON-1668990756-project-member] Lock "9f7876ad-0487-426d-8b19-b718dccdc4fe-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:08:02 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-e91f7847-953e-40f1-a6f9-601a068df520 tempest-DeleteServersTestJSON-1668990756 tempest-DeleteServersTestJSON-1668990756-project-member] Lock "9f7876ad-0487-426d-8b19-b718dccdc4fe-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:08:02 user nova-compute[75234]: INFO nova.compute.manager [None req-e91f7847-953e-40f1-a6f9-601a068df520 tempest-DeleteServersTestJSON-1668990756 tempest-DeleteServersTestJSON-1668990756-project-member] [instance: 9f7876ad-0487-426d-8b19-b718dccdc4fe] Terminating instance
Nov 29 03:08:02 user nova-compute[75234]: DEBUG nova.compute.manager [None req-e91f7847-953e-40f1-a6f9-601a068df520 tempest-DeleteServersTestJSON-1668990756 tempest-DeleteServersTestJSON-1668990756-project-member] [instance: 9f7876ad-0487-426d-8b19-b718dccdc4fe] Start destroying the instance on the hypervisor. {{(pid=75234) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3140}}
Nov 29 03:08:02 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:08:02 user nova-compute[75234]: DEBUG nova.network.neutron [req-1a6dbe9a-fe3a-4df9-b2f6-ff3101fd6e7c req-224a6ff3-2ce1-4809-b496-63e4f7ccc86d service nova] [instance: 8297c264-a213-44ee-be46-5c08eabda9f7] Updated VIF entry in instance network info cache for port d26334d2-8040-4022-b658-69be1cf6a025. {{(pid=75234) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3481}}
Nov 29 03:08:02 user nova-compute[75234]: DEBUG nova.network.neutron [req-1a6dbe9a-fe3a-4df9-b2f6-ff3101fd6e7c req-224a6ff3-2ce1-4809-b496-63e4f7ccc86d service nova] [instance: 8297c264-a213-44ee-be46-5c08eabda9f7] Updating instance_info_cache with network_info: [{"id": "d26334d2-8040-4022-b658-69be1cf6a025", "address": "fa:16:3e:5f:9d:5a", "network": {"id": "e0366490-9fe2-4429-88c8-6625128e645d", "bridge": "br-int", "label": "tempest-ServerStableDeviceRescueTest-804259918-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.31", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "745bdb3c72794155829dc0a69f35b321", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapd26334d2-80", "ovs_interfaceid": "d26334d2-8040-4022-b658-69be1cf6a025", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:08:02 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-e6a4506b-8f08-442e-ab49-ab610a8b34fc tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/8297c264-a213-44ee-be46-5c08eabda9f7/disk --force-share --output=json -f qcow2" returned: 0 in 0.214s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:08:02 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-e6a4506b-8f08-442e-ab49-ab610a8b34fc tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:08:02 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:08:02 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-1a6dbe9a-fe3a-4df9-b2f6-ff3101fd6e7c req-224a6ff3-2ce1-4809-b496-63e4f7ccc86d service nova] Releasing lock "refresh_cache-8297c264-a213-44ee-be46-5c08eabda9f7" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:08:02 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:08:02 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:08:03 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-e6a4506b-8f08-442e-ab49-ab610a8b34fc tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json" returned: 0 in 0.184s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:08:03 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-e6a4506b-8f08-442e-ab49-ab610a8b34fc tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918,backing_fmt=raw /opt/stack/data/nova/instances/snapshots/tmp2uf1nphc/418355de30db40c1a2ae0dab57923ee5.delta 1073741824 {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:08:03 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-e6a4506b-8f08-442e-ab49-ab610a8b34fc tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918,backing_fmt=raw /opt/stack/data/nova/instances/snapshots/tmp2uf1nphc/418355de30db40c1a2ae0dab57923ee5.delta 1073741824" returned: 0 in 0.073s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:08:03 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-e6a4506b-8f08-442e-ab49-ab610a8b34fc tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] [instance: 8297c264-a213-44ee-be46-5c08eabda9f7] Quiescing instance not available: QEMU guest agent is not enabled.
Nov 29 03:08:03 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-e6a4506b-8f08-442e-ab49-ab610a8b34fc tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] Checking UEFI support for host arch (x86_64) {{(pid=75234) supports_uefi /opt/stack/nova/nova/virt/libvirt/host.py:1753}}
Nov 29 03:08:03 user nova-compute[75234]: INFO nova.virt.libvirt.host [None req-e6a4506b-8f08-442e-ab49-ab610a8b34fc tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] UEFI support detected
Nov 29 03:08:03 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-5137a17a-1a49-4901-baa4-df00da03fc6d tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Acquiring lock "f609a7b7-1be2-44b4-9177-7e72a4d4c060" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:08:03 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-5137a17a-1a49-4901-baa4-df00da03fc6d tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Lock "f609a7b7-1be2-44b4-9177-7e72a4d4c060" acquired by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:08:03 user nova-compute[75234]: DEBUG nova.objects.instance [None req-5137a17a-1a49-4901-baa4-df00da03fc6d tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Lazy-loading 'flavor' on Instance uuid f609a7b7-1be2-44b4-9177-7e72a4d4c060 {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:08:03 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:08:03 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-5137a17a-1a49-4901-baa4-df00da03fc6d tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Lock "f609a7b7-1be2-44b4-9177-7e72a4d4c060" "released" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: held 0.092s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:08:03 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:08:03 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:08:03 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:08:03 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-5137a17a-1a49-4901-baa4-df00da03fc6d tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Acquiring lock "f609a7b7-1be2-44b4-9177-7e72a4d4c060" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:08:03 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-5137a17a-1a49-4901-baa4-df00da03fc6d tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Lock "f609a7b7-1be2-44b4-9177-7e72a4d4c060" acquired by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:08:03 user nova-compute[75234]: INFO nova.compute.manager [None req-5137a17a-1a49-4901-baa4-df00da03fc6d tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] [instance: f609a7b7-1be2-44b4-9177-7e72a4d4c060] Attaching volume 33601b89-9deb-4aa4-bd5b-1654f1237672 to /dev/vdb
Nov 29 03:08:03 user nova-compute[75234]: DEBUG os_brick.utils [None req-5137a17a-1a49-4901-baa4-df00da03fc6d tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:176}}
Nov 29 03:08:03 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:08:03 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.017s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:08:03 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[12896b0f-7957-4cf1-a42e-3da612f2f1ef]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:dd1368c5ba2c\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:08:04 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt -v / -n -o SOURCE {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:08:04 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt -v / -n -o SOURCE" returned: 0 in 0.129s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:08:04 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[adc1d01d-052b-459e-8e52-3a3fcf6b212d]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:08:04 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:08:04 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.022s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:08:04 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[c47d1970-c726-4f81-a6f8-28cfc73b8ae2]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:08:04 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[7c461698-4a62-431b-adea-0a7999376451]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:08:04 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-5137a17a-1a49-4901-baa4-df00da03fc6d tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Running cmd (subprocess): nvme version {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:08:04 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-5137a17a-1a49-4901-baa4-df00da03fc6d tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] 'nvme version' failed. Not Retrying. {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:473}}
Nov 29 03:08:04 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.nvmeof [None req-5137a17a-1a49-4901-baa4-df00da03fc6d tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] nvme not present on system {{(pid=75234) nvme_present /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/nvmeof.py:756}}
Nov 29 03:08:04 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:08:04 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:473}}
Nov 29 03:08:04 user nova-compute[75234]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Nov 29 03:08:04 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[9223790c-aee1-4479-a3b6-923943a87891]: (4, '') {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:08:04 user nova-compute[75234]: DEBUG nova.virt.libvirt.guest [None req-e6a4506b-8f08-442e-ab49-ab610a8b34fc tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] COPY block job progress, current cursor: 0 final cursor: 75497472 {{(pid=75234) is_job_complete /opt/stack/nova/nova/virt/libvirt/guest.py:846}}
Nov 29 03:08:04 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.lightos [None req-5137a17a-1a49-4901-baa4-df00da03fc6d tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=75234) find_dsc /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/lightos.py:98}}
Nov 29 03:08:04 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.lightos [None req-5137a17a-1a49-4901-baa4-df00da03fc6d tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=75234) get_connector_properties /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/lightos.py:76}}
Nov 29 03:08:04 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.lightos [None req-5137a17a-1a49-4901-baa4-df00da03fc6d tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] LIGHTOS: no hostnqn found. {{(pid=75234) get_connector_properties /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/lightos.py:84}}
Nov 29 03:08:04 user nova-compute[75234]: DEBUG os_brick.utils [None req-5137a17a-1a49-4901-baa4-df00da03fc6d tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] <== get_connector_properties: return (263ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:dd1368c5ba2c', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:203}}
Nov 29 03:08:04 user nova-compute[75234]: DEBUG nova.virt.block_device [None req-5137a17a-1a49-4901-baa4-df00da03fc6d tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] [instance: f609a7b7-1be2-44b4-9177-7e72a4d4c060] Updating existing volume attachment record: fbaf802f-e156-481f-8810-8093ac2287a3 {{(pid=75234) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Nov 29 03:08:04 user nova-compute[75234]: INFO nova.virt.libvirt.driver [-] [instance: 9f7876ad-0487-426d-8b19-b718dccdc4fe] Instance destroyed successfully.
Nov 29 03:08:04 user nova-compute[75234]: DEBUG nova.objects.instance [None req-e91f7847-953e-40f1-a6f9-601a068df520 tempest-DeleteServersTestJSON-1668990756 tempest-DeleteServersTestJSON-1668990756-project-member] Lazy-loading 'resources' on Instance uuid 9f7876ad-0487-426d-8b19-b718dccdc4fe {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:08:04 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-e91f7847-953e-40f1-a6f9-601a068df520 tempest-DeleteServersTestJSON-1668990756 tempest-DeleteServersTestJSON-1668990756-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-11-29T03:05:53Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-DeleteServersTestJSON-server-1027639586',display_name='tempest-DeleteServersTestJSON-server-1027639586',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-deleteserverstestjson-server-1027639586',id=1,image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data=None,key_name=None,keypairs=<?>,launch_index=0,launched_at=2023-11-29T03:06:09Z,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='a21183b0b4f947e39218bb8e7c857e50',ramdisk_id='',reservation_id='r-0a0qd15t',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-DeleteServersTestJSON-1668990756',owner_user_name='tempest-DeleteServersTestJSON-1668990756-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-11-29T03:06:09Z,user_data=None,user_id='7a95c076ff434b43ac1264b5098b94fc',uuid=9f7876ad-0487-426d-8b19-b718dccdc4fe,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "7dd048ce-2c9b-46ac-8e9d-8c6a2ab201f5", "address": "fa:16:3e:fd:bb:d1", "network": {"id": "c810861d-949e-423d-a3b5-a37f66f29342", "bridge": "br-int", "label": "tempest-DeleteServersTestJSON-2024350588-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a21183b0b4f947e39218bb8e7c857e50", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap7dd048ce-2c", "ovs_interfaceid": "7dd048ce-2c9b-46ac-8e9d-8c6a2ab201f5", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Nov 29 03:08:04 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-e91f7847-953e-40f1-a6f9-601a068df520 tempest-DeleteServersTestJSON-1668990756 tempest-DeleteServersTestJSON-1668990756-project-member] Converting VIF {"id": "7dd048ce-2c9b-46ac-8e9d-8c6a2ab201f5", "address": "fa:16:3e:fd:bb:d1", "network": {"id": "c810861d-949e-423d-a3b5-a37f66f29342", "bridge": "br-int", "label": "tempest-DeleteServersTestJSON-2024350588-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "a21183b0b4f947e39218bb8e7c857e50", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap7dd048ce-2c", "ovs_interfaceid": "7dd048ce-2c9b-46ac-8e9d-8c6a2ab201f5", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:08:04 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-e91f7847-953e-40f1-a6f9-601a068df520 tempest-DeleteServersTestJSON-1668990756 tempest-DeleteServersTestJSON-1668990756-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:fd:bb:d1,bridge_name='br-int',has_traffic_filtering=True,id=7dd048ce-2c9b-46ac-8e9d-8c6a2ab201f5,network=Network(c810861d-949e-423d-a3b5-a37f66f29342),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap7dd048ce-2c') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:08:04 user nova-compute[75234]: DEBUG os_vif [None req-e91f7847-953e-40f1-a6f9-601a068df520 tempest-DeleteServersTestJSON-1668990756 tempest-DeleteServersTestJSON-1668990756-project-member] Unplugging vif VIFOpenVSwitch(active=True,address=fa:16:3e:fd:bb:d1,bridge_name='br-int',has_traffic_filtering=True,id=7dd048ce-2c9b-46ac-8e9d-8c6a2ab201f5,network=Network(c810861d-949e-423d-a3b5-a37f66f29342),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap7dd048ce-2c') {{(pid=75234) unplug /opt/stack/data/venv/lib/python3.10/site-packages/os_vif/__init__.py:109}}
Nov 29 03:08:04 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:08:04 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tap7dd048ce-2c, bridge=br-int, if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:08:04 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:08:04 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:08:04 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:08:04 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbDestroyCommand(_result=None, table=QoS, record=ca3ebb37-3ef3-4d9d-8787-1e1608b3fbac) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:08:04 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:08:04 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:08:04 user nova-compute[75234]: INFO os_vif [None req-e91f7847-953e-40f1-a6f9-601a068df520 tempest-DeleteServersTestJSON-1668990756 tempest-DeleteServersTestJSON-1668990756-project-member] Successfully unplugged vif VIFOpenVSwitch(active=True,address=fa:16:3e:fd:bb:d1,bridge_name='br-int',has_traffic_filtering=True,id=7dd048ce-2c9b-46ac-8e9d-8c6a2ab201f5,network=Network(c810861d-949e-423d-a3b5-a37f66f29342),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap7dd048ce-2c')
Nov 29 03:08:04 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-e91f7847-953e-40f1-a6f9-601a068df520 tempest-DeleteServersTestJSON-1668990756 tempest-DeleteServersTestJSON-1668990756-project-member] [instance: 9f7876ad-0487-426d-8b19-b718dccdc4fe] calling os-brick to detach iSCSI Volume {{(pid=75234) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Nov 29 03:08:04 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-e91f7847-953e-40f1-a6f9-601a068df520 tempest-DeleteServersTestJSON-1668990756 tempest-DeleteServersTestJSON-1668990756-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f8a5050c9a0>, {'target_iqn': 'iqn.2010-10.org.openstack:be635389-1f4c-4c39-8c36-25d81868770a', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'vMCHaGOW', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdh'}, None), 'kwargs': True}}" {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:176}}
Nov 29 03:08:04 user nova-compute[75234]: WARNING os_brick.initiator.connectors.base [None req-e91f7847-953e-40f1-a6f9-601a068df520 tempest-DeleteServersTestJSON-1668990756 tempest-DeleteServersTestJSON-1668990756-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Nov 29 03:08:04 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-e91f7847-953e-40f1-a6f9-601a068df520 tempest-DeleteServersTestJSON-1668990756 tempest-DeleteServersTestJSON-1668990756-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:68}}
Nov 29 03:08:04 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-e91f7847-953e-40f1-a6f9-601a068df520 tempest-DeleteServersTestJSON-1668990756 tempest-DeleteServersTestJSON-1668990756-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.002s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:73}}
Nov 29 03:08:04 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:08:04 user nova-compute[75234]: DEBUG nova.compute.manager [req-8f91485f-4847-4bad-9a39-a9e5e75b8094 req-e980bed9-6dba-4231-a34a-b58948e63a9b service nova] [instance: 9f7876ad-0487-426d-8b19-b718dccdc4fe] Received event network-vif-unplugged-7dd048ce-2c9b-46ac-8e9d-8c6a2ab201f5 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:08:04 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-8f91485f-4847-4bad-9a39-a9e5e75b8094 req-e980bed9-6dba-4231-a34a-b58948e63a9b service nova] Acquiring lock "9f7876ad-0487-426d-8b19-b718dccdc4fe-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:08:04 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-8f91485f-4847-4bad-9a39-a9e5e75b8094 req-e980bed9-6dba-4231-a34a-b58948e63a9b service nova] Lock "9f7876ad-0487-426d-8b19-b718dccdc4fe-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:08:04 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-8f91485f-4847-4bad-9a39-a9e5e75b8094 req-e980bed9-6dba-4231-a34a-b58948e63a9b service nova] Lock "9f7876ad-0487-426d-8b19-b718dccdc4fe-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:08:04 user nova-compute[75234]: DEBUG nova.compute.manager [req-8f91485f-4847-4bad-9a39-a9e5e75b8094 req-e980bed9-6dba-4231-a34a-b58948e63a9b service nova] [instance: 9f7876ad-0487-426d-8b19-b718dccdc4fe] No waiting events found dispatching network-vif-unplugged-7dd048ce-2c9b-46ac-8e9d-8c6a2ab201f5 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:08:04 user nova-compute[75234]: DEBUG nova.compute.manager [req-8f91485f-4847-4bad-9a39-a9e5e75b8094 req-e980bed9-6dba-4231-a34a-b58948e63a9b service nova] [instance: 9f7876ad-0487-426d-8b19-b718dccdc4fe] Received event network-vif-unplugged-7dd048ce-2c9b-46ac-8e9d-8c6a2ab201f5 for instance with task_state deleting. {{(pid=75234) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10890}}
Nov 29 03:08:04 user nova-compute[75234]: DEBUG nova.compute.manager [req-8f91485f-4847-4bad-9a39-a9e5e75b8094 req-e980bed9-6dba-4231-a34a-b58948e63a9b service nova] [instance: 9f7876ad-0487-426d-8b19-b718dccdc4fe] Received event network-vif-plugged-7dd048ce-2c9b-46ac-8e9d-8c6a2ab201f5 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:08:04 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-8f91485f-4847-4bad-9a39-a9e5e75b8094 req-e980bed9-6dba-4231-a34a-b58948e63a9b service nova] Acquiring lock "9f7876ad-0487-426d-8b19-b718dccdc4fe-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:08:04 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-8f91485f-4847-4bad-9a39-a9e5e75b8094 req-e980bed9-6dba-4231-a34a-b58948e63a9b service nova] Lock "9f7876ad-0487-426d-8b19-b718dccdc4fe-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:08:04 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-8f91485f-4847-4bad-9a39-a9e5e75b8094 req-e980bed9-6dba-4231-a34a-b58948e63a9b service nova] Lock "9f7876ad-0487-426d-8b19-b718dccdc4fe-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:08:04 user nova-compute[75234]: DEBUG nova.compute.manager [req-8f91485f-4847-4bad-9a39-a9e5e75b8094 req-e980bed9-6dba-4231-a34a-b58948e63a9b service nova] [instance: 9f7876ad-0487-426d-8b19-b718dccdc4fe] No waiting events found dispatching network-vif-plugged-7dd048ce-2c9b-46ac-8e9d-8c6a2ab201f5 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:08:04 user nova-compute[75234]: WARNING nova.compute.manager [req-8f91485f-4847-4bad-9a39-a9e5e75b8094 req-e980bed9-6dba-4231-a34a-b58948e63a9b service nova] [instance: 9f7876ad-0487-426d-8b19-b718dccdc4fe] Received unexpected event network-vif-plugged-7dd048ce-2c9b-46ac-8e9d-8c6a2ab201f5 for instance with vm_state active and task_state deleting.
Nov 29 03:08:04 user nova-compute[75234]: DEBUG nova.compute.manager [req-8f91485f-4847-4bad-9a39-a9e5e75b8094 req-e980bed9-6dba-4231-a34a-b58948e63a9b service nova] [instance: 9f7876ad-0487-426d-8b19-b718dccdc4fe] Received event network-vif-plugged-7dd048ce-2c9b-46ac-8e9d-8c6a2ab201f5 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:08:04 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-8f91485f-4847-4bad-9a39-a9e5e75b8094 req-e980bed9-6dba-4231-a34a-b58948e63a9b service nova] Acquiring lock "9f7876ad-0487-426d-8b19-b718dccdc4fe-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:08:04 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-8f91485f-4847-4bad-9a39-a9e5e75b8094 req-e980bed9-6dba-4231-a34a-b58948e63a9b service nova] Lock "9f7876ad-0487-426d-8b19-b718dccdc4fe-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:08:04 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-8f91485f-4847-4bad-9a39-a9e5e75b8094 req-e980bed9-6dba-4231-a34a-b58948e63a9b service nova] Lock "9f7876ad-0487-426d-8b19-b718dccdc4fe-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:08:04 user nova-compute[75234]: DEBUG nova.compute.manager [req-8f91485f-4847-4bad-9a39-a9e5e75b8094 req-e980bed9-6dba-4231-a34a-b58948e63a9b service nova] [instance: 9f7876ad-0487-426d-8b19-b718dccdc4fe] No waiting events found dispatching network-vif-plugged-7dd048ce-2c9b-46ac-8e9d-8c6a2ab201f5 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:08:04 user nova-compute[75234]: WARNING nova.compute.manager [req-8f91485f-4847-4bad-9a39-a9e5e75b8094 req-e980bed9-6dba-4231-a34a-b58948e63a9b service nova] [instance: 9f7876ad-0487-426d-8b19-b718dccdc4fe] Received unexpected event network-vif-plugged-7dd048ce-2c9b-46ac-8e9d-8c6a2ab201f5 for instance with vm_state active and task_state deleting.
Nov 29 03:08:04 user nova-compute[75234]: DEBUG nova.compute.manager [req-8f91485f-4847-4bad-9a39-a9e5e75b8094 req-e980bed9-6dba-4231-a34a-b58948e63a9b service nova] [instance: 9f7876ad-0487-426d-8b19-b718dccdc4fe] Received event network-vif-plugged-7dd048ce-2c9b-46ac-8e9d-8c6a2ab201f5 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:08:04 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-8f91485f-4847-4bad-9a39-a9e5e75b8094 req-e980bed9-6dba-4231-a34a-b58948e63a9b service nova] Acquiring lock "9f7876ad-0487-426d-8b19-b718dccdc4fe-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:08:04 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-8f91485f-4847-4bad-9a39-a9e5e75b8094 req-e980bed9-6dba-4231-a34a-b58948e63a9b service nova] Lock "9f7876ad-0487-426d-8b19-b718dccdc4fe-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:08:04 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-8f91485f-4847-4bad-9a39-a9e5e75b8094 req-e980bed9-6dba-4231-a34a-b58948e63a9b service nova] Lock "9f7876ad-0487-426d-8b19-b718dccdc4fe-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:08:04 user nova-compute[75234]: DEBUG nova.compute.manager [req-8f91485f-4847-4bad-9a39-a9e5e75b8094 req-e980bed9-6dba-4231-a34a-b58948e63a9b service nova] [instance: 9f7876ad-0487-426d-8b19-b718dccdc4fe] No waiting events found dispatching network-vif-plugged-7dd048ce-2c9b-46ac-8e9d-8c6a2ab201f5 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:08:04 user nova-compute[75234]: WARNING nova.compute.manager [req-8f91485f-4847-4bad-9a39-a9e5e75b8094 req-e980bed9-6dba-4231-a34a-b58948e63a9b service nova] [instance: 9f7876ad-0487-426d-8b19-b718dccdc4fe] Received unexpected event network-vif-plugged-7dd048ce-2c9b-46ac-8e9d-8c6a2ab201f5 for instance with vm_state active and task_state deleting.
Nov 29 03:08:04 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.038s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:08:04 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[548438fe-5ec0-42b7-b25c-3ae2f416c18d]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:0d2c0c22-2b3a-43db-b9af-e60a2514e48e\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:ab278679-6cfd-4c7c-a930-c078843640d3\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:bb98586e-4f75-46be-a626-0b73ebda11eb\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:920d6cad-25ab-4f3d-941b-8ec841d43184\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:5fcd66fc-a479-4613-b1f8-e6ef12ca470a\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:be635389-1f4c-4c39-8c36-25d81868770a\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:b1068fd1-6636-43cd-9725-4a37f708b2bc\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:08:04 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-e91f7847-953e-40f1-a6f9-601a068df520 tempest-DeleteServersTestJSON-1668990756 tempest-DeleteServersTestJSON-1668990756-project-member] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS:
Nov 29 03:08:04 user nova-compute[75234]: No targets found.
Nov 29 03:08:04 user nova-compute[75234]: iSNS:
Nov 29 03:08:04 user nova-compute[75234]: No targets found.
Nov 29 03:08:04 user nova-compute[75234]: STATIC:
Nov 29 03:08:04 user nova-compute[75234]: Target: iqn.2010-10.org.openstack:0d2c0c22-2b3a-43db-b9af-e60a2514e48e
Nov 29 03:08:04 user nova-compute[75234]:         Portal: 172.16.0.220:3260,-1
Nov 29 03:08:04 user nova-compute[75234]:                 Iface Name: default
Nov 29 03:08:04 user nova-compute[75234]: Target: iqn.2010-10.org.openstack:ab278679-6cfd-4c7c-a930-c078843640d3
Nov 29 03:08:04 user nova-compute[75234]:         Portal: 172.16.0.220:3260,-1
Nov 29 03:08:04 user nova-compute[75234]:                 Iface Name: default
Nov 29 03:08:04 user nova-compute[75234]: Target: iqn.2010-10.org.openstack:bb98586e-4f75-46be-a626-0b73ebda11eb
Nov 29 03:08:04 user nova-compute[75234]:         Portal: 172.16.0.220:3260,-1
Nov 29 03:08:04 user nova-compute[75234]:                 Iface Name: default
Nov 29 03:08:04 user nova-compute[75234]: Target: iqn.2010-10.org.openstack:920d6cad-25ab-4f3d-941b-8ec841d43184
Nov 29 03:08:04 user nova-compute[75234]:         Portal: 172.16.0.220:3260,-1
Nov 29 03:08:04 user nova-compute[75234]:                 Iface Name: default
Nov 29 03:08:04 user nova-compute[75234]: Target: iqn.2010-10.org.openstack:5fcd66fc-a479-4613-b1f8-e6ef12ca470a
Nov 29 03:08:04 user nova-compute[75234]:         Portal: 172.16.0.220:3260,-1
Nov 29 03:08:04 user nova-compute[75234]:                 Iface Name: default
Nov 29 03:08:04 user nova-compute[75234]: Target: iqn.2010-10.org.openstack:be635389-1f4c-4c39-8c36-25d81868770a
Nov 29 03:08:04 user nova-compute[75234]:         Portal: 172.16.0.220:3260,-1
Nov 29 03:08:04 user nova-compute[75234]:                 Iface Name: default
Nov 29 03:08:04 user nova-compute[75234]: Target: iqn.2010-10.org.openstack:b1068fd1-6636-43cd-9725-4a37f708b2bc
Nov 29 03:08:04 user nova-compute[75234]:         Portal: 172.16.0.220:3260,-1
Nov 29 03:08:04 user nova-compute[75234]:                 Iface Name: default
Nov 29 03:08:04 user nova-compute[75234]: FIRMWARE:
Nov 29 03:08:04 user nova-compute[75234]: No targets found.
Nov 29 03:08:04 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:08:04 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-e91f7847-953e-40f1-a6f9-601a068df520 tempest-DeleteServersTestJSON-1668990756 tempest-DeleteServersTestJSON-1668990756-project-member] Regex to get portals from discoverydb: ^SENDTARGETS:
Nov 29 03:08:04 user nova-compute[75234]: .*?^DiscoveryAddress: 172.16.0.220,3260.*?
Nov 29 03:08:04 user nova-compute[75234]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=75234) _get_discoverydb_portals /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:382}}
Nov 29 03:08:04 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-e91f7847-953e-40f1-a6f9-601a068df520 tempest-DeleteServersTestJSON-1668990756 tempest-DeleteServersTestJSON-1668990756-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:be635389-1f4c-4c39-8c36-25d81868770a', 0)] {{(pid=75234) _get_connection_devices /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:830}}
Nov 29 03:08:04 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:08:04 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.023s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:08:04 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[67acb1cf-652d-4fbe-a9d3-39491b30e89e]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:0d2c0c22-2b3a-43db-b9af-e60a2514e48e\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:5fcd66fc-a479-4613-b1f8-e6ef12ca470a\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:920d6cad-25ab-4f3d-941b-8ec841d43184\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:ab278679-6cfd-4c7c-a930-c078843640d3\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:b1068fd1-6636-43cd-9725-4a37f708b2bc\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:bb98586e-4f75-46be-a626-0b73ebda11eb\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:be635389-1f4c-4c39-8c36-25d81868770a\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:08:04 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:08:04 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.044s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:08:04 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[c1294b35-92cc-4111-a5f1-2fa5040c1d71]: (4, ('tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bb98586e-4f75-46be-a626-0b73ebda11eb (non-flash)\ntcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ab278679-6cfd-4c7c-a930-c078843640d3 (non-flash)\ntcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:920d6cad-25ab-4f3d-941b-8ec841d43184 (non-flash)\ntcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b1068fd1-6636-43cd-9725-4a37f708b2bc (non-flash)\ntcp: [6] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0d2c0c22-2b3a-43db-b9af-e60a2514e48e (non-flash)\ntcp: [7] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5fcd66fc-a479-4613-b1f8-e6ef12ca470a (non-flash)\ntcp: [8] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:be635389-1f4c-4c39-8c36-25d81868770a (non-flash)\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:08:04 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-e91f7847-953e-40f1-a6f9-601a068df520 tempest-DeleteServersTestJSON-1668990756 tempest-DeleteServersTestJSON-1668990756-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bb98586e-4f75-46be-a626-0b73ebda11eb (non-flash)
Nov 29 03:08:04 user nova-compute[75234]: tcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ab278679-6cfd-4c7c-a930-c078843640d3 (non-flash)
Nov 29 03:08:04 user nova-compute[75234]: tcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:920d6cad-25ab-4f3d-941b-8ec841d43184 (non-flash)
Nov 29 03:08:04 user nova-compute[75234]: tcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b1068fd1-6636-43cd-9725-4a37f708b2bc (non-flash)
Nov 29 03:08:04 user nova-compute[75234]: tcp: [6] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0d2c0c22-2b3a-43db-b9af-e60a2514e48e (non-flash)
Nov 29 03:08:04 user nova-compute[75234]: tcp: [7] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5fcd66fc-a479-4613-b1f8-e6ef12ca470a (non-flash)
Nov 29 03:08:04 user nova-compute[75234]: tcp: [8] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:be635389-1f4c-4c39-8c36-25d81868770a (non-flash)
Nov 29 03:08:04 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:08:04 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-e91f7847-953e-40f1-a6f9-601a068df520 tempest-DeleteServersTestJSON-1668990756 tempest-DeleteServersTestJSON-1668990756-project-member] iscsi session list stdout=tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bb98586e-4f75-46be-a626-0b73ebda11eb (non-flash)
Nov 29 03:08:04 user nova-compute[75234]: tcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ab278679-6cfd-4c7c-a930-c078843640d3 (non-flash)
Nov 29 03:08:04 user nova-compute[75234]: tcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:920d6cad-25ab-4f3d-941b-8ec841d43184 (non-flash)
Nov 29 03:08:04 user nova-compute[75234]: tcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b1068fd1-6636-43cd-9725-4a37f708b2bc (non-flash)
Nov 29 03:08:04 user nova-compute[75234]: tcp: [6] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0d2c0c22-2b3a-43db-b9af-e60a2514e48e (non-flash)
Nov 29 03:08:04 user nova-compute[75234]: tcp: [7] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5fcd66fc-a479-4613-b1f8-e6ef12ca470a (non-flash)
Nov 29 03:08:04 user nova-compute[75234]: tcp: [8] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:be635389-1f4c-4c39-8c36-25d81868770a (non-flash)
Nov 29 03:08:04 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsi_session /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Nov 29 03:08:04 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-e91f7847-953e-40f1-a6f9-601a068df520 tempest-DeleteServersTestJSON-1668990756 tempest-DeleteServersTestJSON-1668990756-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7f8a506d9d80>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:be635389-1f4c-4c39-8c36-25d81868770a'): ({'sdh'}, set())}) {{(pid=75234) _get_connection_devices /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:863}}
Nov 29 03:08:04 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-e91f7847-953e-40f1-a6f9-601a068df520 tempest-DeleteServersTestJSON-1668990756 tempest-DeleteServersTestJSON-1668990756-project-member] Removing single pathed devices sdh {{(pid=75234) remove_connection /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:374}}
Nov 29 03:08:04 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:08:04 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.020s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:08:04 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[be7d1081-b3bd-431b-8bad-8e24c370e174]: (4, ('path checker states:\nup                  7\n\npaths: 0\nbusy: False\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:08:04 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdh {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:08:04 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:08:04 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdh" returned: 0 in 0.025s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:08:04 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[2eec5713-4088-429d-80e6-4981afcac1ea]: (4, ('ok\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:08:04 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-e91f7847-953e-40f1-a6f9-601a068df520 tempest-DeleteServersTestJSON-1668990756 tempest-DeleteServersTestJSON-1668990756-project-member] Flushing IO for device /dev/sdh {{(pid=75234) flush_device_io /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:434}}
Nov 29 03:08:04 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:08:04 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdh {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:08:04 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdh" returned: 0 in 0.016s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:08:04 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[ffd1569b-54dd-4033-8e5b-71fbcc7dfdc3]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:08:04 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-e91f7847-953e-40f1-a6f9-601a068df520 tempest-DeleteServersTestJSON-1668990756 tempest-DeleteServersTestJSON-1668990756-project-member] Remove SCSI device /dev/sdh with /sys/block/sdh/device/delete {{(pid=75234) remove_scsi_device /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:148}}
Nov 29 03:08:04 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdh/device/delete {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:08:04 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:08:04 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:08:04 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdh/device/delete" returned: 0 in 0.047s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:08:04 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[742c13ae-aa6c-44a5-83f0-d46e15fbbdc4]: (4, ('1', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:08:04 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-e91f7847-953e-40f1-a6f9-601a068df520 tempest-DeleteServersTestJSON-1668990756 tempest-DeleteServersTestJSON-1668990756-project-member] Checking to see if SCSI volumes sdh have been removed. {{(pid=75234) wait_for_volumes_removal /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:156}}
Nov 29 03:08:04 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-e91f7847-953e-40f1-a6f9-601a068df520 tempest-DeleteServersTestJSON-1668990756 tempest-DeleteServersTestJSON-1668990756-project-member] SCSI volumes sdh have been removed. {{(pid=75234) wait_for_volumes_removal /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:166}}
Nov 29 03:08:04 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-e91f7847-953e-40f1-a6f9-601a068df520 tempest-DeleteServersTestJSON-1668990756 tempest-DeleteServersTestJSON-1668990756-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:be635389-1f4c-4c39-8c36-25d81868770a')] {{(pid=75234) _disconnect_connection /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Nov 29 03:08:04 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:be635389-1f4c-4c39-8c36-25d81868770a -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:08:04 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:be635389-1f4c-4c39-8c36-25d81868770a -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.022s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:08:04 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[d911d419-72eb-4e51-8328-a551c3830d05]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:08:04 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-e91f7847-953e-40f1-a6f9-601a068df520 tempest-DeleteServersTestJSON-1668990756 tempest-DeleteServersTestJSON-1668990756-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:08:04 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:be635389-1f4c-4c39-8c36-25d81868770a -p 172.16.0.220:3260 --logout {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:08:04 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:08:04 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:08:04 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:be635389-1f4c-4c39-8c36-25d81868770a -p 172.16.0.220:3260 --logout" returned: 0 in 0.072s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:08:04 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[b74ff96a-4d25-4e6e-906e-76a11c7d4a05]: (4, ('Logging out of session [sid: 8, target: iqn.2010-10.org.openstack:be635389-1f4c-4c39-8c36-25d81868770a, portal: 172.16.0.220,3260]\nLogout of [sid: 8, target: iqn.2010-10.org.openstack:be635389-1f4c-4c39-8c36-25d81868770a, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:08:04 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-e91f7847-953e-40f1-a6f9-601a068df520 tempest-DeleteServersTestJSON-1668990756 tempest-DeleteServersTestJSON-1668990756-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 8, target: iqn.2010-10.org.openstack:be635389-1f4c-4c39-8c36-25d81868770a, portal: 172.16.0.220,3260]
Nov 29 03:08:04 user nova-compute[75234]: Logout of [sid: 8, target: iqn.2010-10.org.openstack:be635389-1f4c-4c39-8c36-25d81868770a, portal: 172.16.0.220,3260] successful.
Nov 29 03:08:04 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:08:04 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:be635389-1f4c-4c39-8c36-25d81868770a -p 172.16.0.220:3260 --op delete {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:08:04 user nova-compute[75234]: DEBUG nova.virt.libvirt.guest [None req-e6a4506b-8f08-442e-ab49-ab610a8b34fc tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] COPY block job progress, current cursor: 75497472 final cursor: 75497472 {{(pid=75234) is_job_complete /opt/stack/nova/nova/virt/libvirt/guest.py:846}}
Nov 29 03:08:04 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-e6a4506b-8f08-442e-ab49-ab610a8b34fc tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] [instance: 8297c264-a213-44ee-be46-5c08eabda9f7] Skipping quiescing instance: QEMU guest agent is not enabled.
Nov 29 03:08:04 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:be635389-1f4c-4c39-8c36-25d81868770a -p 172.16.0.220:3260 --op delete" returned: 0 in 0.040s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:08:04 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[ede7133f-bc30-4042-91d9-918907df5d93]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:08:04 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-e91f7847-953e-40f1-a6f9-601a068df520 tempest-DeleteServersTestJSON-1668990756 tempest-DeleteServersTestJSON-1668990756-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:08:04 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-e91f7847-953e-40f1-a6f9-601a068df520 tempest-DeleteServersTestJSON-1668990756 tempest-DeleteServersTestJSON-1668990756-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.443s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:87}}
Nov 29 03:08:04 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-e91f7847-953e-40f1-a6f9-601a068df520 tempest-DeleteServersTestJSON-1668990756 tempest-DeleteServersTestJSON-1668990756-project-member] <== disconnect_volume: return (445ms) None {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:203}}
Nov 29 03:08:04 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-e91f7847-953e-40f1-a6f9-601a068df520 tempest-DeleteServersTestJSON-1668990756 tempest-DeleteServersTestJSON-1668990756-project-member] [instance: 9f7876ad-0487-426d-8b19-b718dccdc4fe] Disconnected iSCSI Volume {{(pid=75234) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:79}}
Nov 29 03:08:04 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-e91f7847-953e-40f1-a6f9-601a068df520 tempest-DeleteServersTestJSON-1668990756 tempest-DeleteServersTestJSON-1668990756-project-member] [instance: 9f7876ad-0487-426d-8b19-b718dccdc4fe] Deleting instance files /opt/stack/data/nova/instances/9f7876ad-0487-426d-8b19-b718dccdc4fe_del
Nov 29 03:08:04 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-e91f7847-953e-40f1-a6f9-601a068df520 tempest-DeleteServersTestJSON-1668990756 tempest-DeleteServersTestJSON-1668990756-project-member] [instance: 9f7876ad-0487-426d-8b19-b718dccdc4fe] Deletion of /opt/stack/data/nova/instances/9f7876ad-0487-426d-8b19-b718dccdc4fe_del complete
Nov 29 03:08:04 user nova-compute[75234]: INFO nova.compute.manager [None req-e91f7847-953e-40f1-a6f9-601a068df520 tempest-DeleteServersTestJSON-1668990756 tempest-DeleteServersTestJSON-1668990756-project-member] [instance: 9f7876ad-0487-426d-8b19-b718dccdc4fe] Took 2.18 seconds to destroy the instance on the hypervisor.
Nov 29 03:08:04 user nova-compute[75234]: DEBUG oslo.service.loopingcall [None req-e91f7847-953e-40f1-a6f9-601a068df520 tempest-DeleteServersTestJSON-1668990756 tempest-DeleteServersTestJSON-1668990756-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=75234) func /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/loopingcall.py:435}}
Nov 29 03:08:04 user nova-compute[75234]: DEBUG nova.compute.manager [-] [instance: 9f7876ad-0487-426d-8b19-b718dccdc4fe] Deallocating network for instance {{(pid=75234) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2279}}
Nov 29 03:08:04 user nova-compute[75234]: DEBUG nova.network.neutron [-] [instance: 9f7876ad-0487-426d-8b19-b718dccdc4fe] deallocate_for_instance() {{(pid=75234) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1802}}
Nov 29 03:08:04 user nova-compute[75234]: DEBUG nova.privsep.utils [None req-e6a4506b-8f08-442e-ab49-ab610a8b34fc tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] Path '/opt/stack/data/nova/instances' supports direct I/O {{(pid=75234) supports_direct_io /opt/stack/nova/nova/privsep/utils.py:63}}
Nov 29 03:08:04 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-e6a4506b-8f08-442e-ab49-ab610a8b34fc tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] Running cmd (subprocess): qemu-img convert -t none -O qcow2 -f qcow2 /opt/stack/data/nova/instances/snapshots/tmp2uf1nphc/418355de30db40c1a2ae0dab57923ee5.delta /opt/stack/data/nova/instances/snapshots/tmp2uf1nphc/418355de30db40c1a2ae0dab57923ee5 {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:08:05 user nova-compute[75234]: DEBUG nova.network.neutron [-] [instance: 9f7876ad-0487-426d-8b19-b718dccdc4fe] Updating instance_info_cache with network_info: [] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:08:05 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-e6a4506b-8f08-442e-ab49-ab610a8b34fc tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] CMD "qemu-img convert -t none -O qcow2 -f qcow2 /opt/stack/data/nova/instances/snapshots/tmp2uf1nphc/418355de30db40c1a2ae0dab57923ee5.delta /opt/stack/data/nova/instances/snapshots/tmp2uf1nphc/418355de30db40c1a2ae0dab57923ee5" returned: 0 in 0.673s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:08:05 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-e6a4506b-8f08-442e-ab49-ab610a8b34fc tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] [instance: 8297c264-a213-44ee-be46-5c08eabda9f7] Snapshot extracted, beginning image upload
Nov 29 03:08:05 user nova-compute[75234]: INFO nova.compute.manager [-] [instance: 9f7876ad-0487-426d-8b19-b718dccdc4fe] Took 0.70 seconds to deallocate network for instance.
Nov 29 03:08:06 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:08:06 user nova-compute[75234]: DEBUG nova.compute.manager [req-dd45a51b-b80c-45a4-8852-7cc359d217d1 req-96847b02-4e7d-4da7-a4fb-7a2d830c9a16 service nova] [instance: 9f7876ad-0487-426d-8b19-b718dccdc4fe] Received event network-vif-unplugged-7dd048ce-2c9b-46ac-8e9d-8c6a2ab201f5 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:08:06 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-dd45a51b-b80c-45a4-8852-7cc359d217d1 req-96847b02-4e7d-4da7-a4fb-7a2d830c9a16 service nova] Acquiring lock "9f7876ad-0487-426d-8b19-b718dccdc4fe-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:08:06 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-dd45a51b-b80c-45a4-8852-7cc359d217d1 req-96847b02-4e7d-4da7-a4fb-7a2d830c9a16 service nova] Lock "9f7876ad-0487-426d-8b19-b718dccdc4fe-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:08:06 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-dd45a51b-b80c-45a4-8852-7cc359d217d1 req-96847b02-4e7d-4da7-a4fb-7a2d830c9a16 service nova] Lock "9f7876ad-0487-426d-8b19-b718dccdc4fe-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:08:06 user nova-compute[75234]: DEBUG nova.compute.manager [req-dd45a51b-b80c-45a4-8852-7cc359d217d1 req-96847b02-4e7d-4da7-a4fb-7a2d830c9a16 service nova] [instance: 9f7876ad-0487-426d-8b19-b718dccdc4fe] No waiting events found dispatching network-vif-unplugged-7dd048ce-2c9b-46ac-8e9d-8c6a2ab201f5 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:08:06 user nova-compute[75234]: DEBUG nova.compute.manager [req-dd45a51b-b80c-45a4-8852-7cc359d217d1 req-96847b02-4e7d-4da7-a4fb-7a2d830c9a16 service nova] [instance: 9f7876ad-0487-426d-8b19-b718dccdc4fe] Received event network-vif-unplugged-7dd048ce-2c9b-46ac-8e9d-8c6a2ab201f5 for instance with task_state deleting. {{(pid=75234) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10890}}
Nov 29 03:08:06 user nova-compute[75234]: DEBUG nova.compute.manager [req-dd45a51b-b80c-45a4-8852-7cc359d217d1 req-96847b02-4e7d-4da7-a4fb-7a2d830c9a16 service nova] [instance: 9f7876ad-0487-426d-8b19-b718dccdc4fe] Received event network-vif-plugged-7dd048ce-2c9b-46ac-8e9d-8c6a2ab201f5 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:08:06 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-dd45a51b-b80c-45a4-8852-7cc359d217d1 req-96847b02-4e7d-4da7-a4fb-7a2d830c9a16 service nova] Acquiring lock "9f7876ad-0487-426d-8b19-b718dccdc4fe-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:08:06 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-dd45a51b-b80c-45a4-8852-7cc359d217d1 req-96847b02-4e7d-4da7-a4fb-7a2d830c9a16 service nova] Lock "9f7876ad-0487-426d-8b19-b718dccdc4fe-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:08:06 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-dd45a51b-b80c-45a4-8852-7cc359d217d1 req-96847b02-4e7d-4da7-a4fb-7a2d830c9a16 service nova] Lock "9f7876ad-0487-426d-8b19-b718dccdc4fe-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:08:06 user nova-compute[75234]: DEBUG nova.compute.manager [req-dd45a51b-b80c-45a4-8852-7cc359d217d1 req-96847b02-4e7d-4da7-a4fb-7a2d830c9a16 service nova] [instance: 9f7876ad-0487-426d-8b19-b718dccdc4fe] No waiting events found dispatching network-vif-plugged-7dd048ce-2c9b-46ac-8e9d-8c6a2ab201f5 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:08:06 user nova-compute[75234]: WARNING nova.compute.manager [req-dd45a51b-b80c-45a4-8852-7cc359d217d1 req-96847b02-4e7d-4da7-a4fb-7a2d830c9a16 service nova] [instance: 9f7876ad-0487-426d-8b19-b718dccdc4fe] Received unexpected event network-vif-plugged-7dd048ce-2c9b-46ac-8e9d-8c6a2ab201f5 for instance with vm_state active and task_state deleting.
Nov 29 03:08:06 user nova-compute[75234]: DEBUG nova.compute.manager [req-dd45a51b-b80c-45a4-8852-7cc359d217d1 req-96847b02-4e7d-4da7-a4fb-7a2d830c9a16 service nova] [instance: 9f7876ad-0487-426d-8b19-b718dccdc4fe] Received event network-vif-deleted-7dd048ce-2c9b-46ac-8e9d-8c6a2ab201f5 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:08:06 user nova-compute[75234]: INFO nova.compute.manager [None req-e91f7847-953e-40f1-a6f9-601a068df520 tempest-DeleteServersTestJSON-1668990756 tempest-DeleteServersTestJSON-1668990756-project-member] [instance: 9f7876ad-0487-426d-8b19-b718dccdc4fe] Took 0.87 seconds to detach 1 volumes for instance.
Nov 29 03:08:06 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-e91f7847-953e-40f1-a6f9-601a068df520 tempest-DeleteServersTestJSON-1668990756 tempest-DeleteServersTestJSON-1668990756-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:08:06 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-e91f7847-953e-40f1-a6f9-601a068df520 tempest-DeleteServersTestJSON-1668990756 tempest-DeleteServersTestJSON-1668990756-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:08:06 user nova-compute[75234]: DEBUG nova.compute.provider_tree [None req-e91f7847-953e-40f1-a6f9-601a068df520 tempest-DeleteServersTestJSON-1668990756 tempest-DeleteServersTestJSON-1668990756-project-member] Inventory has not changed in ProviderTree for provider: 4d291220-7c37-4839-a0e3-b23c66502b74 {{(pid=75234) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Nov 29 03:08:06 user nova-compute[75234]: DEBUG nova.scheduler.client.report [None req-e91f7847-953e-40f1-a6f9-601a068df520 tempest-DeleteServersTestJSON-1668990756 tempest-DeleteServersTestJSON-1668990756-project-member] Inventory has not changed for provider 4d291220-7c37-4839-a0e3-b23c66502b74 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=75234) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Nov 29 03:08:06 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-e91f7847-953e-40f1-a6f9-601a068df520 tempest-DeleteServersTestJSON-1668990756 tempest-DeleteServersTestJSON-1668990756-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.413s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:08:06 user nova-compute[75234]: INFO nova.scheduler.client.report [None req-e91f7847-953e-40f1-a6f9-601a068df520 tempest-DeleteServersTestJSON-1668990756 tempest-DeleteServersTestJSON-1668990756-project-member] Deleted allocations for instance 9f7876ad-0487-426d-8b19-b718dccdc4fe
Nov 29 03:08:07 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-e91f7847-953e-40f1-a6f9-601a068df520 tempest-DeleteServersTestJSON-1668990756 tempest-DeleteServersTestJSON-1668990756-project-member] Lock "9f7876ad-0487-426d-8b19-b718dccdc4fe" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 4.411s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:08:08 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-e6a4506b-8f08-442e-ab49-ab610a8b34fc tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] [instance: 8297c264-a213-44ee-be46-5c08eabda9f7] Snapshot image upload complete
Nov 29 03:08:08 user nova-compute[75234]: INFO nova.compute.manager [None req-e6a4506b-8f08-442e-ab49-ab610a8b34fc tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] [instance: 8297c264-a213-44ee-be46-5c08eabda9f7] Took 6.56 seconds to snapshot the instance on the hypervisor.
Nov 29 03:08:09 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:08:11 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:08:12 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-5137a17a-1a49-4901-baa4-df00da03fc6d tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] [instance: f609a7b7-1be2-44b4-9177-7e72a4d4c060] Calling os-brick to attach iSCSI Volume {{(pid=75234) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Nov 29 03:08:12 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-5137a17a-1a49-4901-baa4-df00da03fc6d tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f8a5050c9a0>, 'connection_properties': {'target_iqn': 'iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'frKqcBuE', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:176}}
Nov 29 03:08:12 user nova-compute[75234]: WARNING os_brick.initiator.connectors.base [None req-5137a17a-1a49-4901-baa4-df00da03fc6d tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Nov 29 03:08:12 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-5137a17a-1a49-4901-baa4-df00da03fc6d tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:68}}
Nov 29 03:08:12 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-5137a17a-1a49-4901-baa4-df00da03fc6d tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:73}}
Nov 29 03:08:12 user nova-compute[75234]: WARNING os_brick.initiator.connectors.base [None req-5137a17a-1a49-4901-baa4-df00da03fc6d tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Nov 29 03:08:12 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-5137a17a-1a49-4901-baa4-df00da03fc6d tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:68}}
Nov 29 03:08:12 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-5137a17a-1a49-4901-baa4-df00da03fc6d tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:73}}
Nov 29 03:08:12 user nova-compute[75234]: INFO os_brick.initiator.connectors.iscsi [None req-5137a17a-1a49-4901-baa4-df00da03fc6d tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Nov 29 03:08:12 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672 -p 172.16.0.220:3260 {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:08:12 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672 -p 172.16.0.220:3260" returned: 21 in 0.017s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:08:12 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[7503d659-e4cb-4d81-b630-7704b646fdd4]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:08:12 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-5137a17a-1a49-4901-baa4-df00da03fc6d tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Nov 29 03:08:12 user nova-compute[75234]:  {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:08:12 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672 -p 172.16.0.220:3260 --interface default --op new {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:08:12 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672 -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.018s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:08:12 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[552e8edc-3996-4de9-8b53-1800841b1116]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672] added\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:08:12 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-5137a17a-1a49-4901-baa4-df00da03fc6d tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672] added
Nov 29 03:08:12 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:08:12 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:08:12 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.021s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:08:12 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[2d132491-75b7-4284-a7c7-47d878331590]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:08:12 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-5137a17a-1a49-4901-baa4-df00da03fc6d tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:08:12 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:08:12 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.017s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:08:12 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[b01d38d1-b0cb-423e-b9ec-d83cf0333808]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:08:12 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-5137a17a-1a49-4901-baa4-df00da03fc6d tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:08:12 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v frKqcBuE {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:08:12 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v frKqcBuE" returned: 0 in 0.018s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:08:12 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[d9e2ecfe-3cce-4e75-9049-a6e5873b4620]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:08:12 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-5137a17a-1a49-4901-baa4-df00da03fc6d tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'frKqcBuE'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:08:12 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:08:12 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.023s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:08:12 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[b68beaef-dfb7-4f10-a2ab-89b9bcb46e5f]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:08:12 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-5137a17a-1a49-4901-baa4-df00da03fc6d tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:08:12 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:08:12 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.026s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:08:12 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[eede1499-4bf7-47a7-aae9-1cd7cd729cab]: (4, ('tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bb98586e-4f75-46be-a626-0b73ebda11eb (non-flash)\ntcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ab278679-6cfd-4c7c-a930-c078843640d3 (non-flash)\ntcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:920d6cad-25ab-4f3d-941b-8ec841d43184 (non-flash)\ntcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b1068fd1-6636-43cd-9725-4a37f708b2bc (non-flash)\ntcp: [6] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0d2c0c22-2b3a-43db-b9af-e60a2514e48e (non-flash)\ntcp: [7] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5fcd66fc-a479-4613-b1f8-e6ef12ca470a (non-flash)\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:08:12 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-5137a17a-1a49-4901-baa4-df00da03fc6d tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bb98586e-4f75-46be-a626-0b73ebda11eb (non-flash)
Nov 29 03:08:12 user nova-compute[75234]: tcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ab278679-6cfd-4c7c-a930-c078843640d3 (non-flash)
Nov 29 03:08:12 user nova-compute[75234]: tcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:920d6cad-25ab-4f3d-941b-8ec841d43184 (non-flash)
Nov 29 03:08:12 user nova-compute[75234]: tcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b1068fd1-6636-43cd-9725-4a37f708b2bc (non-flash)
Nov 29 03:08:12 user nova-compute[75234]: tcp: [6] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0d2c0c22-2b3a-43db-b9af-e60a2514e48e (non-flash)
Nov 29 03:08:12 user nova-compute[75234]: tcp: [7] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5fcd66fc-a479-4613-b1f8-e6ef12ca470a (non-flash)
Nov 29 03:08:12 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:08:12 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-5137a17a-1a49-4901-baa4-df00da03fc6d tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] iscsi session list stdout=tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bb98586e-4f75-46be-a626-0b73ebda11eb (non-flash)
Nov 29 03:08:12 user nova-compute[75234]: tcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ab278679-6cfd-4c7c-a930-c078843640d3 (non-flash)
Nov 29 03:08:12 user nova-compute[75234]: tcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:920d6cad-25ab-4f3d-941b-8ec841d43184 (non-flash)
Nov 29 03:08:12 user nova-compute[75234]: tcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b1068fd1-6636-43cd-9725-4a37f708b2bc (non-flash)
Nov 29 03:08:12 user nova-compute[75234]: tcp: [6] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0d2c0c22-2b3a-43db-b9af-e60a2514e48e (non-flash)
Nov 29 03:08:12 user nova-compute[75234]: tcp: [7] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5fcd66fc-a479-4613-b1f8-e6ef12ca470a (non-flash)
Nov 29 03:08:12 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsi_session /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Nov 29 03:08:12 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672 -p 172.16.0.220:3260 --login {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:08:12 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672 -p 172.16.0.220:3260 --login" returned: 0 in 0.075s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:08:12 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[d76f2a6a-15c7-4004-98ff-c213dedc2219]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:08:12 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-5137a17a-1a49-4901-baa4-df00da03fc6d tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672, portal: 172.16.0.220,3260]
Nov 29 03:08:12 user nova-compute[75234]: Login to [iface: default, target: iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672, portal: 172.16.0.220,3260] successful.
Nov 29 03:08:12 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:08:12 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672 -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:08:12 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672 -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.018s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:08:12 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[7d71b915-c6ec-46eb-a7ef-f307595bd16d]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:08:12 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-5137a17a-1a49-4901-baa4-df00da03fc6d tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:08:12 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:08:12 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.030s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:08:12 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[43d5d683-b9f7-483e-a880-dbb34a1bc359]: (4, ('tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bb98586e-4f75-46be-a626-0b73ebda11eb (non-flash)\ntcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ab278679-6cfd-4c7c-a930-c078843640d3 (non-flash)\ntcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:920d6cad-25ab-4f3d-941b-8ec841d43184 (non-flash)\ntcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b1068fd1-6636-43cd-9725-4a37f708b2bc (non-flash)\ntcp: [6] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0d2c0c22-2b3a-43db-b9af-e60a2514e48e (non-flash)\ntcp: [7] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5fcd66fc-a479-4613-b1f8-e6ef12ca470a (non-flash)\ntcp: [9] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672 (non-flash)\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:08:12 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-5137a17a-1a49-4901-baa4-df00da03fc6d tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bb98586e-4f75-46be-a626-0b73ebda11eb (non-flash)
Nov 29 03:08:12 user nova-compute[75234]: tcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ab278679-6cfd-4c7c-a930-c078843640d3 (non-flash)
Nov 29 03:08:12 user nova-compute[75234]: tcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:920d6cad-25ab-4f3d-941b-8ec841d43184 (non-flash)
Nov 29 03:08:12 user nova-compute[75234]: tcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b1068fd1-6636-43cd-9725-4a37f708b2bc (non-flash)
Nov 29 03:08:12 user nova-compute[75234]: tcp: [6] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0d2c0c22-2b3a-43db-b9af-e60a2514e48e (non-flash)
Nov 29 03:08:12 user nova-compute[75234]: tcp: [7] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5fcd66fc-a479-4613-b1f8-e6ef12ca470a (non-flash)
Nov 29 03:08:12 user nova-compute[75234]: tcp: [9] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672 (non-flash)
Nov 29 03:08:12 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:08:12 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-5137a17a-1a49-4901-baa4-df00da03fc6d tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] iscsi session list stdout=tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bb98586e-4f75-46be-a626-0b73ebda11eb (non-flash)
Nov 29 03:08:12 user nova-compute[75234]: tcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ab278679-6cfd-4c7c-a930-c078843640d3 (non-flash)
Nov 29 03:08:12 user nova-compute[75234]: tcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:920d6cad-25ab-4f3d-941b-8ec841d43184 (non-flash)
Nov 29 03:08:12 user nova-compute[75234]: tcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b1068fd1-6636-43cd-9725-4a37f708b2bc (non-flash)
Nov 29 03:08:12 user nova-compute[75234]: tcp: [6] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0d2c0c22-2b3a-43db-b9af-e60a2514e48e (non-flash)
Nov 29 03:08:12 user nova-compute[75234]: tcp: [7] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5fcd66fc-a479-4613-b1f8-e6ef12ca470a (non-flash)
Nov 29 03:08:12 user nova-compute[75234]: tcp: [9] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672 (non-flash)
Nov 29 03:08:12 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsi_session /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Nov 29 03:08:12 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-5137a17a-1a49-4901-baa4-df00da03fc6d tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.300s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:87}}
Nov 29 03:08:12 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-5137a17a-1a49-4901-baa4-df00da03fc6d tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Connected to 172.16.0.220:3260 {{(pid=75234) _connect_vol /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:644}}
Nov 29 03:08:12 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-5137a17a-1a49-4901-baa4-df00da03fc6d tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] HCTL ('39', '-', '-', 0) found on session 9 with lun 0 {{(pid=75234) get_hctl /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:770}}
Nov 29 03:08:12 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-5137a17a-1a49-4901-baa4-df00da03fc6d tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Scanning host 39 c: -, t: -, l: 0) {{(pid=75234) scan_iscsi /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:795}}
Nov 29 03:08:12 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host39/scan {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:08:12 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host39/scan" returned: 0 in 0.015s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:08:12 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[537c01a5-352e-4262-a08f-5b6d185284dc]: (4, ('- - 0', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:08:12 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-5137a17a-1a49-4901-baa4-df00da03fc6d tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Searching for a device in session 9 and hctl ['39', '*', '*', 0] yield: None {{(pid=75234) device_name_by_hctl /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:789}}
Nov 29 03:08:13 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-5137a17a-1a49-4901-baa4-df00da03fc6d tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Searching for a device in session 9 and hctl ['39', '*', '*', 0] yield: sdh {{(pid=75234) device_name_by_hctl /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:789}}
Nov 29 03:08:13 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-5137a17a-1a49-4901-baa4-df00da03fc6d tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Connected to sdh using {'target_iqn': 'iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'frKqcBuE', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False} {{(pid=75234) _connect_vol /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:672}}
Nov 29 03:08:13 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-5137a17a-1a49-4901-baa4-df00da03fc6d tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.330s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:87}}
Nov 29 03:08:13 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-5137a17a-1a49-4901-baa4-df00da03fc6d tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] <== connect_volume: return (1331ms) {'type': 'block', 'scsi_wwn': '23865363536313666', 'path': '/dev/sdh'} {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:203}}
Nov 29 03:08:13 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-5137a17a-1a49-4901-baa4-df00da03fc6d tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] [instance: f609a7b7-1be2-44b4-9177-7e72a4d4c060] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '23865363536313666', 'path': '/dev/sdh'} {{(pid=75234) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Nov 29 03:08:13 user nova-compute[75234]: DEBUG nova.objects.instance [None req-5137a17a-1a49-4901-baa4-df00da03fc6d tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Lazy-loading 'flavor' on Instance uuid f609a7b7-1be2-44b4-9177-7e72a4d4c060 {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:08:13 user nova-compute[75234]: DEBUG nova.virt.libvirt.guest [None req-5137a17a-1a49-4901-baa4-df00da03fc6d tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] attach device xml: <disk type="block" device="disk">
Nov 29 03:08:13 user nova-compute[75234]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Nov 29 03:08:13 user nova-compute[75234]:   <source dev="/dev/sdh"/>
Nov 29 03:08:13 user nova-compute[75234]:   <target dev="vdb" bus="virtio"/>
Nov 29 03:08:13 user nova-compute[75234]:   <serial>33601b89-9deb-4aa4-bd5b-1654f1237672</serial>
Nov 29 03:08:13 user nova-compute[75234]: </disk>
Nov 29 03:08:13 user nova-compute[75234]:  {{(pid=75234) attach_device /opt/stack/nova/nova/virt/libvirt/guest.py:339}}
Nov 29 03:08:14 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-5137a17a-1a49-4901-baa4-df00da03fc6d tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] No BDM found with device name vda, not building metadata. {{(pid=75234) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12089}}
Nov 29 03:08:14 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-5137a17a-1a49-4901-baa4-df00da03fc6d tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] No BDM found with device name vdb, not building metadata. {{(pid=75234) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12089}}
Nov 29 03:08:14 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-5137a17a-1a49-4901-baa4-df00da03fc6d tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] No VIF found with MAC fa:16:3e:04:ae:43, not building metadata {{(pid=75234) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Nov 29 03:08:14 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-5137a17a-1a49-4901-baa4-df00da03fc6d tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Lock "f609a7b7-1be2-44b4-9177-7e72a4d4c060" "released" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: held 10.426s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:08:14 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:08:15 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-03d920f5-b2ad-4728-b8c7-f3fa7ceda447 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Acquiring lock "f609a7b7-1be2-44b4-9177-7e72a4d4c060" by "nova.compute.manager.ComputeManager.stop_instance.<locals>.do_stop_instance" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:08:15 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-03d920f5-b2ad-4728-b8c7-f3fa7ceda447 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Lock "f609a7b7-1be2-44b4-9177-7e72a4d4c060" acquired by "nova.compute.manager.ComputeManager.stop_instance.<locals>.do_stop_instance" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:08:15 user nova-compute[75234]: DEBUG nova.compute.manager [None req-03d920f5-b2ad-4728-b8c7-f3fa7ceda447 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] [instance: f609a7b7-1be2-44b4-9177-7e72a4d4c060] Checking state {{(pid=75234) _get_power_state /opt/stack/nova/nova/compute/manager.py:1782}}
Nov 29 03:08:15 user nova-compute[75234]: DEBUG nova.compute.manager [None req-03d920f5-b2ad-4728-b8c7-f3fa7ceda447 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] [instance: f609a7b7-1be2-44b4-9177-7e72a4d4c060] Stopping instance; current vm_state: active, current task_state: powering-off, current DB power_state: 1, current VM power_state: 1 {{(pid=75234) do_stop_instance /opt/stack/nova/nova/compute/manager.py:3358}}
Nov 29 03:08:15 user nova-compute[75234]: DEBUG nova.objects.instance [None req-03d920f5-b2ad-4728-b8c7-f3fa7ceda447 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Lazy-loading 'flavor' on Instance uuid f609a7b7-1be2-44b4-9177-7e72a4d4c060 {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:08:15 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:08:15 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:08:15 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:08:15 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:08:15 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:08:15 user nova-compute[75234]: DEBUG nova.compute.manager [req-64c4dbe0-4031-4dbb-8be6-39252d436ee0 req-97f104ad-f35c-4a1b-9f7b-af817c6a825e service nova] [instance: f609a7b7-1be2-44b4-9177-7e72a4d4c060] Received event network-vif-unplugged-5543f379-2b3a-4dc2-bfe9-52f66a942257 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:08:15 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-64c4dbe0-4031-4dbb-8be6-39252d436ee0 req-97f104ad-f35c-4a1b-9f7b-af817c6a825e service nova] Acquiring lock "f609a7b7-1be2-44b4-9177-7e72a4d4c060-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:08:15 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-64c4dbe0-4031-4dbb-8be6-39252d436ee0 req-97f104ad-f35c-4a1b-9f7b-af817c6a825e service nova] Lock "f609a7b7-1be2-44b4-9177-7e72a4d4c060-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:08:15 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-64c4dbe0-4031-4dbb-8be6-39252d436ee0 req-97f104ad-f35c-4a1b-9f7b-af817c6a825e service nova] Lock "f609a7b7-1be2-44b4-9177-7e72a4d4c060-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:08:15 user nova-compute[75234]: DEBUG nova.compute.manager [req-64c4dbe0-4031-4dbb-8be6-39252d436ee0 req-97f104ad-f35c-4a1b-9f7b-af817c6a825e service nova] [instance: f609a7b7-1be2-44b4-9177-7e72a4d4c060] No waiting events found dispatching network-vif-unplugged-5543f379-2b3a-4dc2-bfe9-52f66a942257 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:08:15 user nova-compute[75234]: WARNING nova.compute.manager [req-64c4dbe0-4031-4dbb-8be6-39252d436ee0 req-97f104ad-f35c-4a1b-9f7b-af817c6a825e service nova] [instance: f609a7b7-1be2-44b4-9177-7e72a4d4c060] Received unexpected event network-vif-unplugged-5543f379-2b3a-4dc2-bfe9-52f66a942257 for instance with vm_state active and task_state powering-off.
Nov 29 03:08:15 user nova-compute[75234]: INFO nova.virt.libvirt.driver [-] [instance: f609a7b7-1be2-44b4-9177-7e72a4d4c060] Instance destroyed successfully.
Nov 29 03:08:15 user nova-compute[75234]: DEBUG nova.compute.manager [None req-03d920f5-b2ad-4728-b8c7-f3fa7ceda447 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] [instance: f609a7b7-1be2-44b4-9177-7e72a4d4c060] Checking state {{(pid=75234) _get_power_state /opt/stack/nova/nova/compute/manager.py:1782}}
Nov 29 03:08:15 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-03d920f5-b2ad-4728-b8c7-f3fa7ceda447 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Lock "f609a7b7-1be2-44b4-9177-7e72a4d4c060" "released" by "nova.compute.manager.ComputeManager.stop_instance.<locals>.do_stop_instance" :: held 0.915s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:08:16 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:08:17 user nova-compute[75234]: DEBUG nova.compute.manager [req-714e743c-5da4-4e57-b51d-8ae5ef0d5a6b req-bd70c971-8a74-4113-afe4-3236738b647a service nova] [instance: f609a7b7-1be2-44b4-9177-7e72a4d4c060] Received event network-vif-plugged-5543f379-2b3a-4dc2-bfe9-52f66a942257 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:08:17 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-714e743c-5da4-4e57-b51d-8ae5ef0d5a6b req-bd70c971-8a74-4113-afe4-3236738b647a service nova] Acquiring lock "f609a7b7-1be2-44b4-9177-7e72a4d4c060-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:08:17 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-714e743c-5da4-4e57-b51d-8ae5ef0d5a6b req-bd70c971-8a74-4113-afe4-3236738b647a service nova] Lock "f609a7b7-1be2-44b4-9177-7e72a4d4c060-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:08:17 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-714e743c-5da4-4e57-b51d-8ae5ef0d5a6b req-bd70c971-8a74-4113-afe4-3236738b647a service nova] Lock "f609a7b7-1be2-44b4-9177-7e72a4d4c060-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:08:17 user nova-compute[75234]: DEBUG nova.compute.manager [req-714e743c-5da4-4e57-b51d-8ae5ef0d5a6b req-bd70c971-8a74-4113-afe4-3236738b647a service nova] [instance: f609a7b7-1be2-44b4-9177-7e72a4d4c060] No waiting events found dispatching network-vif-plugged-5543f379-2b3a-4dc2-bfe9-52f66a942257 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:08:17 user nova-compute[75234]: WARNING nova.compute.manager [req-714e743c-5da4-4e57-b51d-8ae5ef0d5a6b req-bd70c971-8a74-4113-afe4-3236738b647a service nova] [instance: f609a7b7-1be2-44b4-9177-7e72a4d4c060] Received unexpected event network-vif-plugged-5543f379-2b3a-4dc2-bfe9-52f66a942257 for instance with vm_state stopped and task_state None.
Nov 29 03:08:18 user nova-compute[75234]: DEBUG nova.compute.manager [req-061b3674-dc38-43ff-bfa2-7225781e6020 req-5cee08b7-d9bc-4d83-8b55-b9b1b35db013 service nova] [instance: da625f9c-b7ed-4afc-875c-6436bf58e9cc] Received event network-changed-a23b8769-d871-4d44-8cea-5d899f514579 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:08:18 user nova-compute[75234]: DEBUG nova.compute.manager [req-061b3674-dc38-43ff-bfa2-7225781e6020 req-5cee08b7-d9bc-4d83-8b55-b9b1b35db013 service nova] [instance: da625f9c-b7ed-4afc-875c-6436bf58e9cc] Refreshing instance network info cache due to event network-changed-a23b8769-d871-4d44-8cea-5d899f514579. {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11117}}
Nov 29 03:08:18 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-061b3674-dc38-43ff-bfa2-7225781e6020 req-5cee08b7-d9bc-4d83-8b55-b9b1b35db013 service nova] Acquiring lock "refresh_cache-da625f9c-b7ed-4afc-875c-6436bf58e9cc" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:08:18 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-061b3674-dc38-43ff-bfa2-7225781e6020 req-5cee08b7-d9bc-4d83-8b55-b9b1b35db013 service nova] Acquired lock "refresh_cache-da625f9c-b7ed-4afc-875c-6436bf58e9cc" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:08:18 user nova-compute[75234]: DEBUG nova.network.neutron [req-061b3674-dc38-43ff-bfa2-7225781e6020 req-5cee08b7-d9bc-4d83-8b55-b9b1b35db013 service nova] [instance: da625f9c-b7ed-4afc-875c-6436bf58e9cc] Refreshing network info cache for port a23b8769-d871-4d44-8cea-5d899f514579 {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2006}}
Nov 29 03:08:19 user nova-compute[75234]: DEBUG nova.network.neutron [req-061b3674-dc38-43ff-bfa2-7225781e6020 req-5cee08b7-d9bc-4d83-8b55-b9b1b35db013 service nova] [instance: da625f9c-b7ed-4afc-875c-6436bf58e9cc] Updated VIF entry in instance network info cache for port a23b8769-d871-4d44-8cea-5d899f514579. {{(pid=75234) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3481}}
Nov 29 03:08:19 user nova-compute[75234]: DEBUG nova.network.neutron [req-061b3674-dc38-43ff-bfa2-7225781e6020 req-5cee08b7-d9bc-4d83-8b55-b9b1b35db013 service nova] [instance: da625f9c-b7ed-4afc-875c-6436bf58e9cc] Updating instance_info_cache with network_info: [{"id": "a23b8769-d871-4d44-8cea-5d899f514579", "address": "fa:16:3e:7f:17:4d", "network": {"id": "5bbe9946-1c42-4c9e-8cf7-5ee0a54ebc8d", "bridge": "br-int", "label": "tempest-AttachSCSIVolumeTestJSON-411635514-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.75", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "df2e94062c154555ab1ee0c3bf8e0f84", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapa23b8769-d8", "ovs_interfaceid": "a23b8769-d871-4d44-8cea-5d899f514579", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:08:19 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-061b3674-dc38-43ff-bfa2-7225781e6020 req-5cee08b7-d9bc-4d83-8b55-b9b1b35db013 service nova] Releasing lock "refresh_cache-da625f9c-b7ed-4afc-875c-6436bf58e9cc" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:08:19 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:08:21 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:08:21 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-12860ead-b910-4f1a-962a-4df814258f3f tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] Acquiring lock "aa3c7432-bf15-408e-9fbb-993135734161" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:08:21 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-12860ead-b910-4f1a-962a-4df814258f3f tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] Lock "aa3c7432-bf15-408e-9fbb-993135734161" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:08:21 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-12860ead-b910-4f1a-962a-4df814258f3f tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] Acquiring lock "aa3c7432-bf15-408e-9fbb-993135734161-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:08:21 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-12860ead-b910-4f1a-962a-4df814258f3f tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] Lock "aa3c7432-bf15-408e-9fbb-993135734161-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:08:21 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-12860ead-b910-4f1a-962a-4df814258f3f tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] Lock "aa3c7432-bf15-408e-9fbb-993135734161-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:08:21 user nova-compute[75234]: INFO nova.compute.manager [None req-12860ead-b910-4f1a-962a-4df814258f3f tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] [instance: aa3c7432-bf15-408e-9fbb-993135734161] Terminating instance
Nov 29 03:08:21 user nova-compute[75234]: DEBUG nova.compute.manager [None req-12860ead-b910-4f1a-962a-4df814258f3f tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] [instance: aa3c7432-bf15-408e-9fbb-993135734161] Start destroying the instance on the hypervisor. {{(pid=75234) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3140}}
Nov 29 03:08:21 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:08:21 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:08:21 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-cec4caa8-f3c7-43da-8baf-520dbd9abf1c tempest-AttachSCSIVolumeTestJSON-719577128 tempest-AttachSCSIVolumeTestJSON-719577128-project-member] Acquiring lock "da625f9c-b7ed-4afc-875c-6436bf58e9cc" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:08:21 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-cec4caa8-f3c7-43da-8baf-520dbd9abf1c tempest-AttachSCSIVolumeTestJSON-719577128 tempest-AttachSCSIVolumeTestJSON-719577128-project-member] Lock "da625f9c-b7ed-4afc-875c-6436bf58e9cc" acquired by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:08:21 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:08:21 user nova-compute[75234]: DEBUG nova.objects.instance [None req-cec4caa8-f3c7-43da-8baf-520dbd9abf1c tempest-AttachSCSIVolumeTestJSON-719577128 tempest-AttachSCSIVolumeTestJSON-719577128-project-member] Lazy-loading 'flavor' on Instance uuid da625f9c-b7ed-4afc-875c-6436bf58e9cc {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:08:21 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:08:21 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-cec4caa8-f3c7-43da-8baf-520dbd9abf1c tempest-AttachSCSIVolumeTestJSON-719577128 tempest-AttachSCSIVolumeTestJSON-719577128-project-member] Lock "da625f9c-b7ed-4afc-875c-6436bf58e9cc" "released" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: held 0.078s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:08:21 user nova-compute[75234]: DEBUG nova.compute.manager [req-c4c388b8-dafc-4aa7-baa5-403b320bf509 req-a4ef4365-06f8-4d67-89c5-14487dfe1e16 service nova] [instance: aa3c7432-bf15-408e-9fbb-993135734161] Received event network-vif-unplugged-aaac2504-1067-43c5-b3bf-316996e0f56c {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:08:21 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-c4c388b8-dafc-4aa7-baa5-403b320bf509 req-a4ef4365-06f8-4d67-89c5-14487dfe1e16 service nova] Acquiring lock "aa3c7432-bf15-408e-9fbb-993135734161-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:08:21 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-c4c388b8-dafc-4aa7-baa5-403b320bf509 req-a4ef4365-06f8-4d67-89c5-14487dfe1e16 service nova] Lock "aa3c7432-bf15-408e-9fbb-993135734161-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:08:21 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-c4c388b8-dafc-4aa7-baa5-403b320bf509 req-a4ef4365-06f8-4d67-89c5-14487dfe1e16 service nova] Lock "aa3c7432-bf15-408e-9fbb-993135734161-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:08:21 user nova-compute[75234]: DEBUG nova.compute.manager [req-c4c388b8-dafc-4aa7-baa5-403b320bf509 req-a4ef4365-06f8-4d67-89c5-14487dfe1e16 service nova] [instance: aa3c7432-bf15-408e-9fbb-993135734161] No waiting events found dispatching network-vif-unplugged-aaac2504-1067-43c5-b3bf-316996e0f56c {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:08:21 user nova-compute[75234]: DEBUG nova.compute.manager [req-c4c388b8-dafc-4aa7-baa5-403b320bf509 req-a4ef4365-06f8-4d67-89c5-14487dfe1e16 service nova] [instance: aa3c7432-bf15-408e-9fbb-993135734161] Received event network-vif-unplugged-aaac2504-1067-43c5-b3bf-316996e0f56c for instance with task_state deleting. {{(pid=75234) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10890}}
Nov 29 03:08:21 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-cec4caa8-f3c7-43da-8baf-520dbd9abf1c tempest-AttachSCSIVolumeTestJSON-719577128 tempest-AttachSCSIVolumeTestJSON-719577128-project-member] Acquiring lock "da625f9c-b7ed-4afc-875c-6436bf58e9cc" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:08:21 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-cec4caa8-f3c7-43da-8baf-520dbd9abf1c tempest-AttachSCSIVolumeTestJSON-719577128 tempest-AttachSCSIVolumeTestJSON-719577128-project-member] Lock "da625f9c-b7ed-4afc-875c-6436bf58e9cc" acquired by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:08:21 user nova-compute[75234]: INFO nova.compute.manager [None req-cec4caa8-f3c7-43da-8baf-520dbd9abf1c tempest-AttachSCSIVolumeTestJSON-719577128 tempest-AttachSCSIVolumeTestJSON-719577128-project-member] [instance: da625f9c-b7ed-4afc-875c-6436bf58e9cc] Attaching volume ae86b2d0-7343-40f1-9624-ddc92be1340d to /dev/sdc
Nov 29 03:08:21 user nova-compute[75234]: DEBUG os_brick.utils [None req-cec4caa8-f3c7-43da-8baf-520dbd9abf1c tempest-AttachSCSIVolumeTestJSON-719577128 tempest-AttachSCSIVolumeTestJSON-719577128-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:176}}
Nov 29 03:08:21 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:08:21 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.016s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:08:21 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[49e4f2c6-9b2b-440a-bf66-4c8b89106a99]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:dd1368c5ba2c\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:08:21 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt -v / -n -o SOURCE {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:08:21 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt -v / -n -o SOURCE" returned: 0 in 0.016s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:08:21 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[65f00ff6-1f66-4c0f-a998-235238edd276]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:08:21 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:08:21 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.020s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:08:21 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[2704894c-3838-4b0e-920e-c9f0c33ec301]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:08:21 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[6daa75f7-fdfc-4248-be00-4b865ccb161e]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:08:21 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-cec4caa8-f3c7-43da-8baf-520dbd9abf1c tempest-AttachSCSIVolumeTestJSON-719577128 tempest-AttachSCSIVolumeTestJSON-719577128-project-member] Running cmd (subprocess): nvme version {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:08:21 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-cec4caa8-f3c7-43da-8baf-520dbd9abf1c tempest-AttachSCSIVolumeTestJSON-719577128 tempest-AttachSCSIVolumeTestJSON-719577128-project-member] 'nvme version' failed. Not Retrying. {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:473}}
Nov 29 03:08:21 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.nvmeof [None req-cec4caa8-f3c7-43da-8baf-520dbd9abf1c tempest-AttachSCSIVolumeTestJSON-719577128 tempest-AttachSCSIVolumeTestJSON-719577128-project-member] nvme not present on system {{(pid=75234) nvme_present /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/nvmeof.py:756}}
Nov 29 03:08:21 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:08:21 user nova-compute[75234]: INFO nova.virt.libvirt.driver [-] [instance: aa3c7432-bf15-408e-9fbb-993135734161] Instance destroyed successfully.
Nov 29 03:08:21 user nova-compute[75234]: DEBUG nova.objects.instance [None req-12860ead-b910-4f1a-962a-4df814258f3f tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] Lazy-loading 'resources' on Instance uuid aa3c7432-bf15-408e-9fbb-993135734161 {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:08:21 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:473}}
Nov 29 03:08:21 user nova-compute[75234]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Nov 29 03:08:21 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[04e0b9a2-36b9-486f-b0b1-fd6fb88a7c0f]: (4, '') {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:08:21 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.lightos [None req-cec4caa8-f3c7-43da-8baf-520dbd9abf1c tempest-AttachSCSIVolumeTestJSON-719577128 tempest-AttachSCSIVolumeTestJSON-719577128-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=75234) find_dsc /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/lightos.py:98}}
Nov 29 03:08:21 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.lightos [None req-cec4caa8-f3c7-43da-8baf-520dbd9abf1c tempest-AttachSCSIVolumeTestJSON-719577128 tempest-AttachSCSIVolumeTestJSON-719577128-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=75234) get_connector_properties /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/lightos.py:76}}
Nov 29 03:08:21 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.lightos [None req-cec4caa8-f3c7-43da-8baf-520dbd9abf1c tempest-AttachSCSIVolumeTestJSON-719577128 tempest-AttachSCSIVolumeTestJSON-719577128-project-member] LIGHTOS: no hostnqn found. {{(pid=75234) get_connector_properties /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/lightos.py:84}}
Nov 29 03:08:21 user nova-compute[75234]: DEBUG os_brick.utils [None req-cec4caa8-f3c7-43da-8baf-520dbd9abf1c tempest-AttachSCSIVolumeTestJSON-719577128 tempest-AttachSCSIVolumeTestJSON-719577128-project-member] <== get_connector_properties: return (132ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:dd1368c5ba2c', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:203}}
Nov 29 03:08:21 user nova-compute[75234]: DEBUG nova.virt.block_device [None req-cec4caa8-f3c7-43da-8baf-520dbd9abf1c tempest-AttachSCSIVolumeTestJSON-719577128 tempest-AttachSCSIVolumeTestJSON-719577128-project-member] [instance: da625f9c-b7ed-4afc-875c-6436bf58e9cc] Updating existing volume attachment record: e1d99ad5-18bc-49ec-a911-5a7eb97e006d {{(pid=75234) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Nov 29 03:08:21 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-12860ead-b910-4f1a-962a-4df814258f3f tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-11-29T03:06:28Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-VolumesActionsTest-instance-958290084',display_name='tempest-VolumesActionsTest-instance-958290084',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-volumesactionstest-instance-958290084',id=8,image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data=None,key_name=None,keypairs=<?>,launch_index=0,launched_at=2023-11-29T03:06:41Z,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='46577f787c244fb2a4db052a0c42eceb',ramdisk_id='',reservation_id='r-xwyt8hlf',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-VolumesActionsTest-196322865',owner_user_name='tempest-VolumesActionsTest-196322865-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-11-29T03:06:41Z,user_data=None,user_id='940a684aa3b6419798bd87562bbc5d23',uuid=aa3c7432-bf15-408e-9fbb-993135734161,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "aaac2504-1067-43c5-b3bf-316996e0f56c", "address": "fa:16:3e:2a:34:9b", "network": {"id": "68a1f282-91d0-47c8-82f3-5ebf1c0f572c", "bridge": "br-int", "label": "tempest-VolumesActionsTest-570720740-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "46577f787c244fb2a4db052a0c42eceb", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapaaac2504-10", "ovs_interfaceid": "aaac2504-1067-43c5-b3bf-316996e0f56c", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Nov 29 03:08:21 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-12860ead-b910-4f1a-962a-4df814258f3f tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] Converting VIF {"id": "aaac2504-1067-43c5-b3bf-316996e0f56c", "address": "fa:16:3e:2a:34:9b", "network": {"id": "68a1f282-91d0-47c8-82f3-5ebf1c0f572c", "bridge": "br-int", "label": "tempest-VolumesActionsTest-570720740-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "46577f787c244fb2a4db052a0c42eceb", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapaaac2504-10", "ovs_interfaceid": "aaac2504-1067-43c5-b3bf-316996e0f56c", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:08:21 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-12860ead-b910-4f1a-962a-4df814258f3f tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:2a:34:9b,bridge_name='br-int',has_traffic_filtering=True,id=aaac2504-1067-43c5-b3bf-316996e0f56c,network=Network(68a1f282-91d0-47c8-82f3-5ebf1c0f572c),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapaaac2504-10') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:08:21 user nova-compute[75234]: DEBUG os_vif [None req-12860ead-b910-4f1a-962a-4df814258f3f tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] Unplugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:2a:34:9b,bridge_name='br-int',has_traffic_filtering=True,id=aaac2504-1067-43c5-b3bf-316996e0f56c,network=Network(68a1f282-91d0-47c8-82f3-5ebf1c0f572c),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapaaac2504-10') {{(pid=75234) unplug /opt/stack/data/venv/lib/python3.10/site-packages/os_vif/__init__.py:109}}
Nov 29 03:08:21 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:08:21 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tapaaac2504-10, bridge=br-int, if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:08:21 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:08:21 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:08:21 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:08:21 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbDestroyCommand(_result=None, table=QoS, record=4f364bff-bc4b-43d9-b56e-5597796afeb0) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:08:21 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:08:21 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:08:21 user nova-compute[75234]: INFO os_vif [None req-12860ead-b910-4f1a-962a-4df814258f3f tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] Successfully unplugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:2a:34:9b,bridge_name='br-int',has_traffic_filtering=True,id=aaac2504-1067-43c5-b3bf-316996e0f56c,network=Network(68a1f282-91d0-47c8-82f3-5ebf1c0f572c),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapaaac2504-10')
Nov 29 03:08:21 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-12860ead-b910-4f1a-962a-4df814258f3f tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] [instance: aa3c7432-bf15-408e-9fbb-993135734161] Deleting instance files /opt/stack/data/nova/instances/aa3c7432-bf15-408e-9fbb-993135734161_del
Nov 29 03:08:21 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-12860ead-b910-4f1a-962a-4df814258f3f tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] [instance: aa3c7432-bf15-408e-9fbb-993135734161] Deletion of /opt/stack/data/nova/instances/aa3c7432-bf15-408e-9fbb-993135734161_del complete
Nov 29 03:08:22 user nova-compute[75234]: INFO nova.compute.manager [None req-12860ead-b910-4f1a-962a-4df814258f3f tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] [instance: aa3c7432-bf15-408e-9fbb-993135734161] Took 0.80 seconds to destroy the instance on the hypervisor.
Nov 29 03:08:22 user nova-compute[75234]: DEBUG oslo.service.loopingcall [None req-12860ead-b910-4f1a-962a-4df814258f3f tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=75234) func /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/loopingcall.py:435}}
Nov 29 03:08:22 user nova-compute[75234]: DEBUG nova.compute.manager [-] [instance: aa3c7432-bf15-408e-9fbb-993135734161] Deallocating network for instance {{(pid=75234) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2279}}
Nov 29 03:08:22 user nova-compute[75234]: DEBUG nova.network.neutron [-] [instance: aa3c7432-bf15-408e-9fbb-993135734161] deallocate_for_instance() {{(pid=75234) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1802}}
Nov 29 03:08:22 user nova-compute[75234]: DEBUG nova.network.neutron [-] [instance: aa3c7432-bf15-408e-9fbb-993135734161] Updating instance_info_cache with network_info: [] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:08:22 user nova-compute[75234]: INFO nova.compute.manager [-] [instance: aa3c7432-bf15-408e-9fbb-993135734161] Took 0.73 seconds to deallocate network for instance.
Nov 29 03:08:22 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-12860ead-b910-4f1a-962a-4df814258f3f tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:08:22 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-12860ead-b910-4f1a-962a-4df814258f3f tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:08:23 user nova-compute[75234]: DEBUG nova.compute.provider_tree [None req-12860ead-b910-4f1a-962a-4df814258f3f tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] Inventory has not changed in ProviderTree for provider: 4d291220-7c37-4839-a0e3-b23c66502b74 {{(pid=75234) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Nov 29 03:08:23 user nova-compute[75234]: DEBUG nova.scheduler.client.report [None req-12860ead-b910-4f1a-962a-4df814258f3f tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] Inventory has not changed for provider 4d291220-7c37-4839-a0e3-b23c66502b74 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=75234) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Nov 29 03:08:23 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-12860ead-b910-4f1a-962a-4df814258f3f tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.408s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:08:23 user nova-compute[75234]: INFO nova.scheduler.client.report [None req-12860ead-b910-4f1a-962a-4df814258f3f tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] Deleted allocations for instance aa3c7432-bf15-408e-9fbb-993135734161
Nov 29 03:08:23 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-12860ead-b910-4f1a-962a-4df814258f3f tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] Lock "aa3c7432-bf15-408e-9fbb-993135734161" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 2.139s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:08:23 user nova-compute[75234]: DEBUG nova.compute.manager [req-e2310da8-e303-427c-b543-d3e349176085 req-918572e0-0291-46d4-b04f-ce9a8efebfcf service nova] [instance: aa3c7432-bf15-408e-9fbb-993135734161] Received event network-vif-plugged-aaac2504-1067-43c5-b3bf-316996e0f56c {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:08:23 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-e2310da8-e303-427c-b543-d3e349176085 req-918572e0-0291-46d4-b04f-ce9a8efebfcf service nova] Acquiring lock "aa3c7432-bf15-408e-9fbb-993135734161-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:08:23 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-e2310da8-e303-427c-b543-d3e349176085 req-918572e0-0291-46d4-b04f-ce9a8efebfcf service nova] Lock "aa3c7432-bf15-408e-9fbb-993135734161-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:08:23 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-e2310da8-e303-427c-b543-d3e349176085 req-918572e0-0291-46d4-b04f-ce9a8efebfcf service nova] Lock "aa3c7432-bf15-408e-9fbb-993135734161-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:08:23 user nova-compute[75234]: DEBUG nova.compute.manager [req-e2310da8-e303-427c-b543-d3e349176085 req-918572e0-0291-46d4-b04f-ce9a8efebfcf service nova] [instance: aa3c7432-bf15-408e-9fbb-993135734161] No waiting events found dispatching network-vif-plugged-aaac2504-1067-43c5-b3bf-316996e0f56c {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:08:23 user nova-compute[75234]: WARNING nova.compute.manager [req-e2310da8-e303-427c-b543-d3e349176085 req-918572e0-0291-46d4-b04f-ce9a8efebfcf service nova] [instance: aa3c7432-bf15-408e-9fbb-993135734161] Received unexpected event network-vif-plugged-aaac2504-1067-43c5-b3bf-316996e0f56c for instance with vm_state deleted and task_state None.
Nov 29 03:08:23 user nova-compute[75234]: DEBUG nova.compute.manager [req-e2310da8-e303-427c-b543-d3e349176085 req-918572e0-0291-46d4-b04f-ce9a8efebfcf service nova] [instance: aa3c7432-bf15-408e-9fbb-993135734161] Received event network-vif-deleted-aaac2504-1067-43c5-b3bf-316996e0f56c {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:08:26 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:08:26 user nova-compute[75234]: DEBUG nova.compute.manager [req-df8be8cd-dbcd-4518-8e8c-5c047fd8d1bb req-de61f1c7-7968-4896-b6f0-948c3f3fab43 service nova] [instance: c9663ab3-3374-400e-ab0a-769731b78cca] Received event network-changed-b55d6eaa-b0d4-4a67-8407-3a8b69558e43 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:08:26 user nova-compute[75234]: DEBUG nova.compute.manager [req-df8be8cd-dbcd-4518-8e8c-5c047fd8d1bb req-de61f1c7-7968-4896-b6f0-948c3f3fab43 service nova] [instance: c9663ab3-3374-400e-ab0a-769731b78cca] Refreshing instance network info cache due to event network-changed-b55d6eaa-b0d4-4a67-8407-3a8b69558e43. {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11117}}
Nov 29 03:08:26 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-df8be8cd-dbcd-4518-8e8c-5c047fd8d1bb req-de61f1c7-7968-4896-b6f0-948c3f3fab43 service nova] Acquiring lock "refresh_cache-c9663ab3-3374-400e-ab0a-769731b78cca" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:08:26 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-df8be8cd-dbcd-4518-8e8c-5c047fd8d1bb req-de61f1c7-7968-4896-b6f0-948c3f3fab43 service nova] Acquired lock "refresh_cache-c9663ab3-3374-400e-ab0a-769731b78cca" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:08:26 user nova-compute[75234]: DEBUG nova.network.neutron [req-df8be8cd-dbcd-4518-8e8c-5c047fd8d1bb req-de61f1c7-7968-4896-b6f0-948c3f3fab43 service nova] [instance: c9663ab3-3374-400e-ab0a-769731b78cca] Refreshing network info cache for port b55d6eaa-b0d4-4a67-8407-3a8b69558e43 {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2006}}
Nov 29 03:08:26 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:08:27 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-1d96cd55-5818-4800-991e-7d3373b14d2d tempest-VolumesAssistedSnapshotsTest-1119739890 tempest-VolumesAssistedSnapshotsTest-1119739890-project-admin] Acquiring lock "c9663ab3-3374-400e-ab0a-769731b78cca" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:08:27 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-1d96cd55-5818-4800-991e-7d3373b14d2d tempest-VolumesAssistedSnapshotsTest-1119739890 tempest-VolumesAssistedSnapshotsTest-1119739890-project-admin] Lock "c9663ab3-3374-400e-ab0a-769731b78cca" acquired by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: waited 0.002s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:08:27 user nova-compute[75234]: DEBUG nova.objects.instance [None req-1d96cd55-5818-4800-991e-7d3373b14d2d tempest-VolumesAssistedSnapshotsTest-1119739890 tempest-VolumesAssistedSnapshotsTest-1119739890-project-admin] Lazy-loading 'flavor' on Instance uuid c9663ab3-3374-400e-ab0a-769731b78cca {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:08:27 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-1d96cd55-5818-4800-991e-7d3373b14d2d tempest-VolumesAssistedSnapshotsTest-1119739890 tempest-VolumesAssistedSnapshotsTest-1119739890-project-admin] Lock "c9663ab3-3374-400e-ab0a-769731b78cca" "released" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: held 0.120s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:08:27 user nova-compute[75234]: DEBUG nova.network.neutron [req-df8be8cd-dbcd-4518-8e8c-5c047fd8d1bb req-de61f1c7-7968-4896-b6f0-948c3f3fab43 service nova] [instance: c9663ab3-3374-400e-ab0a-769731b78cca] Updated VIF entry in instance network info cache for port b55d6eaa-b0d4-4a67-8407-3a8b69558e43. {{(pid=75234) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3481}}
Nov 29 03:08:27 user nova-compute[75234]: DEBUG nova.network.neutron [req-df8be8cd-dbcd-4518-8e8c-5c047fd8d1bb req-de61f1c7-7968-4896-b6f0-948c3f3fab43 service nova] [instance: c9663ab3-3374-400e-ab0a-769731b78cca] Updating instance_info_cache with network_info: [{"id": "b55d6eaa-b0d4-4a67-8407-3a8b69558e43", "address": "fa:16:3e:3e:b6:0b", "network": {"id": "8d7bb68b-4e8e-42c9-8d1a-3d0e301a7b73", "bridge": "br-int", "label": "tempest-VolumesAssistedSnapshotsTest-1523028903-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.204", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "d59c913723894ba1938fb8177c97e627", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapb55d6eaa-b0", "ovs_interfaceid": "b55d6eaa-b0d4-4a67-8407-3a8b69558e43", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:08:27 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-df8be8cd-dbcd-4518-8e8c-5c047fd8d1bb req-de61f1c7-7968-4896-b6f0-948c3f3fab43 service nova] Releasing lock "refresh_cache-c9663ab3-3374-400e-ab0a-769731b78cca" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:08:27 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-1d96cd55-5818-4800-991e-7d3373b14d2d tempest-VolumesAssistedSnapshotsTest-1119739890 tempest-VolumesAssistedSnapshotsTest-1119739890-project-admin] Acquiring lock "c9663ab3-3374-400e-ab0a-769731b78cca" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:08:27 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-1d96cd55-5818-4800-991e-7d3373b14d2d tempest-VolumesAssistedSnapshotsTest-1119739890 tempest-VolumesAssistedSnapshotsTest-1119739890-project-admin] Lock "c9663ab3-3374-400e-ab0a-769731b78cca" acquired by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:08:27 user nova-compute[75234]: INFO nova.compute.manager [None req-1d96cd55-5818-4800-991e-7d3373b14d2d tempest-VolumesAssistedSnapshotsTest-1119739890 tempest-VolumesAssistedSnapshotsTest-1119739890-project-admin] [instance: c9663ab3-3374-400e-ab0a-769731b78cca] Attaching volume a4cf29d0-2ce2-423f-a20c-a7a19a452088 to /dev/vdb
Nov 29 03:08:27 user nova-compute[75234]: DEBUG os_brick.utils [None req-1d96cd55-5818-4800-991e-7d3373b14d2d tempest-VolumesAssistedSnapshotsTest-1119739890 tempest-VolumesAssistedSnapshotsTest-1119739890-project-admin] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:176}}
Nov 29 03:08:27 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:08:27 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.014s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:08:27 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[4c055469-fc8e-497c-bd44-83562b58e0ce]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:dd1368c5ba2c\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:08:27 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt -v / -n -o SOURCE {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:08:27 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt -v / -n -o SOURCE" returned: 0 in 0.015s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:08:27 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[1ddb8c07-0f75-4471-85ec-4c99148d0bbb]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:08:27 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:08:27 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.016s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:08:27 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[b5fcfec1-8329-4d28-8390-2c7719fcf749]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:08:27 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[574dd163-3d0a-421d-b2fa-29d190dd1f19]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:08:27 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-1d96cd55-5818-4800-991e-7d3373b14d2d tempest-VolumesAssistedSnapshotsTest-1119739890 tempest-VolumesAssistedSnapshotsTest-1119739890-project-admin] Running cmd (subprocess): nvme version {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:08:27 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-1d96cd55-5818-4800-991e-7d3373b14d2d tempest-VolumesAssistedSnapshotsTest-1119739890 tempest-VolumesAssistedSnapshotsTest-1119739890-project-admin] 'nvme version' failed. Not Retrying. {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:473}}
Nov 29 03:08:27 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.nvmeof [None req-1d96cd55-5818-4800-991e-7d3373b14d2d tempest-VolumesAssistedSnapshotsTest-1119739890 tempest-VolumesAssistedSnapshotsTest-1119739890-project-admin] nvme not present on system {{(pid=75234) nvme_present /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/nvmeof.py:756}}
Nov 29 03:08:27 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:08:27 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:473}}
Nov 29 03:08:27 user nova-compute[75234]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Nov 29 03:08:27 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[dbbb325c-fa7c-484c-aa2a-c2ad3f3bf2dc]: (4, '') {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:08:27 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.lightos [None req-1d96cd55-5818-4800-991e-7d3373b14d2d tempest-VolumesAssistedSnapshotsTest-1119739890 tempest-VolumesAssistedSnapshotsTest-1119739890-project-admin] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=75234) find_dsc /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/lightos.py:98}}
Nov 29 03:08:27 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.lightos [None req-1d96cd55-5818-4800-991e-7d3373b14d2d tempest-VolumesAssistedSnapshotsTest-1119739890 tempest-VolumesAssistedSnapshotsTest-1119739890-project-admin] LIGHTOS: did not find dsc, continuing anyway. {{(pid=75234) get_connector_properties /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/lightos.py:76}}
Nov 29 03:08:27 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.lightos [None req-1d96cd55-5818-4800-991e-7d3373b14d2d tempest-VolumesAssistedSnapshotsTest-1119739890 tempest-VolumesAssistedSnapshotsTest-1119739890-project-admin] LIGHTOS: no hostnqn found. {{(pid=75234) get_connector_properties /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/lightos.py:84}}
Nov 29 03:08:27 user nova-compute[75234]: DEBUG os_brick.utils [None req-1d96cd55-5818-4800-991e-7d3373b14d2d tempest-VolumesAssistedSnapshotsTest-1119739890 tempest-VolumesAssistedSnapshotsTest-1119739890-project-admin] <== get_connector_properties: return (116ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:dd1368c5ba2c', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:203}}
Nov 29 03:08:27 user nova-compute[75234]: DEBUG nova.virt.block_device [None req-1d96cd55-5818-4800-991e-7d3373b14d2d tempest-VolumesAssistedSnapshotsTest-1119739890 tempest-VolumesAssistedSnapshotsTest-1119739890-project-admin] [instance: c9663ab3-3374-400e-ab0a-769731b78cca] Updating existing volume attachment record: 99dc88b1-1803-4dcf-a1a3-7746b294b87f {{(pid=75234) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Nov 29 03:08:29 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-cec4caa8-f3c7-43da-8baf-520dbd9abf1c tempest-AttachSCSIVolumeTestJSON-719577128 tempest-AttachSCSIVolumeTestJSON-719577128-project-member] [instance: da625f9c-b7ed-4afc-875c-6436bf58e9cc] Calling os-brick to attach iSCSI Volume {{(pid=75234) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Nov 29 03:08:29 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-cec4caa8-f3c7-43da-8baf-520dbd9abf1c tempest-AttachSCSIVolumeTestJSON-719577128 tempest-AttachSCSIVolumeTestJSON-719577128-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f8a5050c9a0>, 'connection_properties': {'target_iqn': 'iqn.2010-10.org.openstack:ae86b2d0-7343-40f1-9624-ddc92be1340d', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'SrlLHTRV', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:176}}
Nov 29 03:08:29 user nova-compute[75234]: WARNING os_brick.initiator.connectors.base [None req-cec4caa8-f3c7-43da-8baf-520dbd9abf1c tempest-AttachSCSIVolumeTestJSON-719577128 tempest-AttachSCSIVolumeTestJSON-719577128-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Nov 29 03:08:29 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-cec4caa8-f3c7-43da-8baf-520dbd9abf1c tempest-AttachSCSIVolumeTestJSON-719577128 tempest-AttachSCSIVolumeTestJSON-719577128-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:68}}
Nov 29 03:08:29 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-cec4caa8-f3c7-43da-8baf-520dbd9abf1c tempest-AttachSCSIVolumeTestJSON-719577128 tempest-AttachSCSIVolumeTestJSON-719577128-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.002s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:73}}
Nov 29 03:08:29 user nova-compute[75234]: WARNING os_brick.initiator.connectors.base [None req-cec4caa8-f3c7-43da-8baf-520dbd9abf1c tempest-AttachSCSIVolumeTestJSON-719577128 tempest-AttachSCSIVolumeTestJSON-719577128-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Nov 29 03:08:29 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-cec4caa8-f3c7-43da-8baf-520dbd9abf1c tempest-AttachSCSIVolumeTestJSON-719577128 tempest-AttachSCSIVolumeTestJSON-719577128-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:ae86b2d0-7343-40f1-9624-ddc92be1340d" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:68}}
Nov 29 03:08:29 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-cec4caa8-f3c7-43da-8baf-520dbd9abf1c tempest-AttachSCSIVolumeTestJSON-719577128 tempest-AttachSCSIVolumeTestJSON-719577128-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:ae86b2d0-7343-40f1-9624-ddc92be1340d" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:73}}
Nov 29 03:08:29 user nova-compute[75234]: INFO os_brick.initiator.connectors.iscsi [None req-cec4caa8-f3c7-43da-8baf-520dbd9abf1c tempest-AttachSCSIVolumeTestJSON-719577128 tempest-AttachSCSIVolumeTestJSON-719577128-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Nov 29 03:08:29 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:ae86b2d0-7343-40f1-9624-ddc92be1340d -p 172.16.0.220:3260 {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:08:29 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:ae86b2d0-7343-40f1-9624-ddc92be1340d -p 172.16.0.220:3260" returned: 21 in 0.018s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:08:29 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[b9600c51-4914-47ad-b2b1-979eb5f05c0d]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:08:29 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-cec4caa8-f3c7-43da-8baf-520dbd9abf1c tempest-AttachSCSIVolumeTestJSON-719577128 tempest-AttachSCSIVolumeTestJSON-719577128-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Nov 29 03:08:29 user nova-compute[75234]:  {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:08:29 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:ae86b2d0-7343-40f1-9624-ddc92be1340d -p 172.16.0.220:3260 --interface default --op new {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:08:29 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:ae86b2d0-7343-40f1-9624-ddc92be1340d -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.018s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:08:29 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[69872162-61f0-4068-964c-252d3f35d2a2]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:ae86b2d0-7343-40f1-9624-ddc92be1340d] added\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:08:29 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-cec4caa8-f3c7-43da-8baf-520dbd9abf1c tempest-AttachSCSIVolumeTestJSON-719577128 tempest-AttachSCSIVolumeTestJSON-719577128-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:ae86b2d0-7343-40f1-9624-ddc92be1340d] added
Nov 29 03:08:29 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:08:29 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:ae86b2d0-7343-40f1-9624-ddc92be1340d -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:08:29 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:ae86b2d0-7343-40f1-9624-ddc92be1340d -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.019s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:08:29 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[2528e305-d193-4cb7-9e6c-c697b8601fa0]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:08:30 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-cec4caa8-f3c7-43da-8baf-520dbd9abf1c tempest-AttachSCSIVolumeTestJSON-719577128 tempest-AttachSCSIVolumeTestJSON-719577128-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:08:30 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:ae86b2d0-7343-40f1-9624-ddc92be1340d -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:08:30 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:ae86b2d0-7343-40f1-9624-ddc92be1340d -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.019s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:08:30 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[f6bf83ee-1e82-4649-972b-6f5f4ac4e90a]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:08:30 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-cec4caa8-f3c7-43da-8baf-520dbd9abf1c tempest-AttachSCSIVolumeTestJSON-719577128 tempest-AttachSCSIVolumeTestJSON-719577128-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:08:30 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:ae86b2d0-7343-40f1-9624-ddc92be1340d -p 172.16.0.220:3260 --op update -n node.session.auth.username -v SrlLHTRV {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:08:30 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:ae86b2d0-7343-40f1-9624-ddc92be1340d -p 172.16.0.220:3260 --op update -n node.session.auth.username -v SrlLHTRV" returned: 0 in 0.019s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:08:30 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[8c647436-a323-442c-ba04-052b2b881c1d]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:08:30 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-cec4caa8-f3c7-43da-8baf-520dbd9abf1c tempest-AttachSCSIVolumeTestJSON-719577128 tempest-AttachSCSIVolumeTestJSON-719577128-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'SrlLHTRV'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:08:30 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:ae86b2d0-7343-40f1-9624-ddc92be1340d -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:08:30 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:ae86b2d0-7343-40f1-9624-ddc92be1340d -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.019s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:08:30 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[8f915a0f-5121-4d7e-a2da-7321cabd219e]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:08:30 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-cec4caa8-f3c7-43da-8baf-520dbd9abf1c tempest-AttachSCSIVolumeTestJSON-719577128 tempest-AttachSCSIVolumeTestJSON-719577128-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:08:30 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:08:30 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.028s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:08:30 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[70573819-7824-4335-abd1-194bbb2e16f3]: (4, ('tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bb98586e-4f75-46be-a626-0b73ebda11eb (non-flash)\ntcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ab278679-6cfd-4c7c-a930-c078843640d3 (non-flash)\ntcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:920d6cad-25ab-4f3d-941b-8ec841d43184 (non-flash)\ntcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b1068fd1-6636-43cd-9725-4a37f708b2bc (non-flash)\ntcp: [6] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0d2c0c22-2b3a-43db-b9af-e60a2514e48e (non-flash)\ntcp: [7] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5fcd66fc-a479-4613-b1f8-e6ef12ca470a (non-flash)\ntcp: [9] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672 (non-flash)\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:08:30 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-cec4caa8-f3c7-43da-8baf-520dbd9abf1c tempest-AttachSCSIVolumeTestJSON-719577128 tempest-AttachSCSIVolumeTestJSON-719577128-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bb98586e-4f75-46be-a626-0b73ebda11eb (non-flash)
Nov 29 03:08:30 user nova-compute[75234]: tcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ab278679-6cfd-4c7c-a930-c078843640d3 (non-flash)
Nov 29 03:08:30 user nova-compute[75234]: tcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:920d6cad-25ab-4f3d-941b-8ec841d43184 (non-flash)
Nov 29 03:08:30 user nova-compute[75234]: tcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b1068fd1-6636-43cd-9725-4a37f708b2bc (non-flash)
Nov 29 03:08:30 user nova-compute[75234]: tcp: [6] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0d2c0c22-2b3a-43db-b9af-e60a2514e48e (non-flash)
Nov 29 03:08:30 user nova-compute[75234]: tcp: [7] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5fcd66fc-a479-4613-b1f8-e6ef12ca470a (non-flash)
Nov 29 03:08:30 user nova-compute[75234]: tcp: [9] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672 (non-flash)
Nov 29 03:08:30 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:08:30 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-cec4caa8-f3c7-43da-8baf-520dbd9abf1c tempest-AttachSCSIVolumeTestJSON-719577128 tempest-AttachSCSIVolumeTestJSON-719577128-project-member] iscsi session list stdout=tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bb98586e-4f75-46be-a626-0b73ebda11eb (non-flash)
Nov 29 03:08:30 user nova-compute[75234]: tcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ab278679-6cfd-4c7c-a930-c078843640d3 (non-flash)
Nov 29 03:08:30 user nova-compute[75234]: tcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:920d6cad-25ab-4f3d-941b-8ec841d43184 (non-flash)
Nov 29 03:08:30 user nova-compute[75234]: tcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b1068fd1-6636-43cd-9725-4a37f708b2bc (non-flash)
Nov 29 03:08:30 user nova-compute[75234]: tcp: [6] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0d2c0c22-2b3a-43db-b9af-e60a2514e48e (non-flash)
Nov 29 03:08:30 user nova-compute[75234]: tcp: [7] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5fcd66fc-a479-4613-b1f8-e6ef12ca470a (non-flash)
Nov 29 03:08:30 user nova-compute[75234]: tcp: [9] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672 (non-flash)
Nov 29 03:08:30 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsi_session /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Nov 29 03:08:30 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:ae86b2d0-7343-40f1-9624-ddc92be1340d -p 172.16.0.220:3260 --login {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:08:30 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:ae86b2d0-7343-40f1-9624-ddc92be1340d -p 172.16.0.220:3260 --login" returned: 0 in 0.079s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:08:30 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[e9206afb-e3d3-4087-8356-595a2b9d9891]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:ae86b2d0-7343-40f1-9624-ddc92be1340d, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:ae86b2d0-7343-40f1-9624-ddc92be1340d, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:08:30 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-cec4caa8-f3c7-43da-8baf-520dbd9abf1c tempest-AttachSCSIVolumeTestJSON-719577128 tempest-AttachSCSIVolumeTestJSON-719577128-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:ae86b2d0-7343-40f1-9624-ddc92be1340d, portal: 172.16.0.220,3260]
Nov 29 03:08:30 user nova-compute[75234]: Login to [iface: default, target: iqn.2010-10.org.openstack:ae86b2d0-7343-40f1-9624-ddc92be1340d, portal: 172.16.0.220,3260] successful.
Nov 29 03:08:30 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:08:30 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:ae86b2d0-7343-40f1-9624-ddc92be1340d -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:08:30 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:ae86b2d0-7343-40f1-9624-ddc92be1340d -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.018s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:08:30 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[cb9aa71d-5dd6-4696-a5c8-d7650b570a20]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:08:30 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-cec4caa8-f3c7-43da-8baf-520dbd9abf1c tempest-AttachSCSIVolumeTestJSON-719577128 tempest-AttachSCSIVolumeTestJSON-719577128-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:08:30 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:08:30 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.030s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:08:30 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[68a76831-f019-44fe-953a-91159cdf4243]: (4, ('tcp: [10] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ae86b2d0-7343-40f1-9624-ddc92be1340d (non-flash)\ntcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bb98586e-4f75-46be-a626-0b73ebda11eb (non-flash)\ntcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ab278679-6cfd-4c7c-a930-c078843640d3 (non-flash)\ntcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:920d6cad-25ab-4f3d-941b-8ec841d43184 (non-flash)\ntcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b1068fd1-6636-43cd-9725-4a37f708b2bc (non-flash)\ntcp: [6] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0d2c0c22-2b3a-43db-b9af-e60a2514e48e (non-flash)\ntcp: [7] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5fcd66fc-a479-4613-b1f8-e6ef12ca470a (non-flash)\ntcp: [9] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672 (non-flash)\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:08:30 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-cec4caa8-f3c7-43da-8baf-520dbd9abf1c tempest-AttachSCSIVolumeTestJSON-719577128 tempest-AttachSCSIVolumeTestJSON-719577128-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [10] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ae86b2d0-7343-40f1-9624-ddc92be1340d (non-flash)
Nov 29 03:08:30 user nova-compute[75234]: tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bb98586e-4f75-46be-a626-0b73ebda11eb (non-flash)
Nov 29 03:08:30 user nova-compute[75234]: tcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ab278679-6cfd-4c7c-a930-c078843640d3 (non-flash)
Nov 29 03:08:30 user nova-compute[75234]: tcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:920d6cad-25ab-4f3d-941b-8ec841d43184 (non-flash)
Nov 29 03:08:30 user nova-compute[75234]: tcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b1068fd1-6636-43cd-9725-4a37f708b2bc (non-flash)
Nov 29 03:08:30 user nova-compute[75234]: tcp: [6] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0d2c0c22-2b3a-43db-b9af-e60a2514e48e (non-flash)
Nov 29 03:08:30 user nova-compute[75234]: tcp: [7] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5fcd66fc-a479-4613-b1f8-e6ef12ca470a (non-flash)
Nov 29 03:08:30 user nova-compute[75234]: tcp: [9] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672 (non-flash)
Nov 29 03:08:30 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:08:30 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-cec4caa8-f3c7-43da-8baf-520dbd9abf1c tempest-AttachSCSIVolumeTestJSON-719577128 tempest-AttachSCSIVolumeTestJSON-719577128-project-member] iscsi session list stdout=tcp: [10] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ae86b2d0-7343-40f1-9624-ddc92be1340d (non-flash)
Nov 29 03:08:30 user nova-compute[75234]: tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bb98586e-4f75-46be-a626-0b73ebda11eb (non-flash)
Nov 29 03:08:30 user nova-compute[75234]: tcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ab278679-6cfd-4c7c-a930-c078843640d3 (non-flash)
Nov 29 03:08:30 user nova-compute[75234]: tcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:920d6cad-25ab-4f3d-941b-8ec841d43184 (non-flash)
Nov 29 03:08:30 user nova-compute[75234]: tcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b1068fd1-6636-43cd-9725-4a37f708b2bc (non-flash)
Nov 29 03:08:30 user nova-compute[75234]: tcp: [6] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0d2c0c22-2b3a-43db-b9af-e60a2514e48e (non-flash)
Nov 29 03:08:30 user nova-compute[75234]: tcp: [7] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5fcd66fc-a479-4613-b1f8-e6ef12ca470a (non-flash)
Nov 29 03:08:30 user nova-compute[75234]: tcp: [9] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672 (non-flash)
Nov 29 03:08:30 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsi_session /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Nov 29 03:08:30 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-cec4caa8-f3c7-43da-8baf-520dbd9abf1c tempest-AttachSCSIVolumeTestJSON-719577128 tempest-AttachSCSIVolumeTestJSON-719577128-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:ae86b2d0-7343-40f1-9624-ddc92be1340d" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.317s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:87}}
Nov 29 03:08:30 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-cec4caa8-f3c7-43da-8baf-520dbd9abf1c tempest-AttachSCSIVolumeTestJSON-719577128 tempest-AttachSCSIVolumeTestJSON-719577128-project-member] Connected to 172.16.0.220:3260 {{(pid=75234) _connect_vol /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:644}}
Nov 29 03:08:30 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-cec4caa8-f3c7-43da-8baf-520dbd9abf1c tempest-AttachSCSIVolumeTestJSON-719577128 tempest-AttachSCSIVolumeTestJSON-719577128-project-member] HCTL ('40', '-', '-', 0) found on session 10 with lun 0 {{(pid=75234) get_hctl /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:770}}
Nov 29 03:08:30 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-cec4caa8-f3c7-43da-8baf-520dbd9abf1c tempest-AttachSCSIVolumeTestJSON-719577128 tempest-AttachSCSIVolumeTestJSON-719577128-project-member] Scanning host 40 c: -, t: -, l: 0) {{(pid=75234) scan_iscsi /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:795}}
Nov 29 03:08:30 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host40/scan {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:08:30 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host40/scan" returned: 0 in 0.028s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:08:30 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[812b8c9f-119f-4b7e-8ee1-50a36795152c]: (4, ('- - 0', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:08:30 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-cec4caa8-f3c7-43da-8baf-520dbd9abf1c tempest-AttachSCSIVolumeTestJSON-719577128 tempest-AttachSCSIVolumeTestJSON-719577128-project-member] Searching for a device in session 10 and hctl ['40', '*', '*', 0] yield: sdi {{(pid=75234) device_name_by_hctl /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:789}}
Nov 29 03:08:30 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-cec4caa8-f3c7-43da-8baf-520dbd9abf1c tempest-AttachSCSIVolumeTestJSON-719577128 tempest-AttachSCSIVolumeTestJSON-719577128-project-member] Connected to sdi using {'target_iqn': 'iqn.2010-10.org.openstack:ae86b2d0-7343-40f1-9624-ddc92be1340d', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'SrlLHTRV', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False} {{(pid=75234) _connect_vol /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:672}}
Nov 29 03:08:31 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:08:31 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-cec4caa8-f3c7-43da-8baf-520dbd9abf1c tempest-AttachSCSIVolumeTestJSON-719577128 tempest-AttachSCSIVolumeTestJSON-719577128-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.366s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:87}}
Nov 29 03:08:31 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-cec4caa8-f3c7-43da-8baf-520dbd9abf1c tempest-AttachSCSIVolumeTestJSON-719577128 tempest-AttachSCSIVolumeTestJSON-719577128-project-member] <== connect_volume: return (1369ms) {'type': 'block', 'scsi_wwn': '26466613831323536', 'path': '/dev/sdi'} {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:203}}
Nov 29 03:08:31 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-cec4caa8-f3c7-43da-8baf-520dbd9abf1c tempest-AttachSCSIVolumeTestJSON-719577128 tempest-AttachSCSIVolumeTestJSON-719577128-project-member] [instance: da625f9c-b7ed-4afc-875c-6436bf58e9cc] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '26466613831323536', 'path': '/dev/sdi'} {{(pid=75234) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Nov 29 03:08:31 user nova-compute[75234]: DEBUG nova.objects.instance [None req-cec4caa8-f3c7-43da-8baf-520dbd9abf1c tempest-AttachSCSIVolumeTestJSON-719577128 tempest-AttachSCSIVolumeTestJSON-719577128-project-member] Lazy-loading 'flavor' on Instance uuid da625f9c-b7ed-4afc-875c-6436bf58e9cc {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:08:31 user nova-compute[75234]: DEBUG nova.virt.libvirt.guest [None req-cec4caa8-f3c7-43da-8baf-520dbd9abf1c tempest-AttachSCSIVolumeTestJSON-719577128 tempest-AttachSCSIVolumeTestJSON-719577128-project-member] attach device xml: <disk type="block" device="disk">
Nov 29 03:08:31 user nova-compute[75234]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Nov 29 03:08:31 user nova-compute[75234]:   <source dev="/dev/sdi"/>
Nov 29 03:08:31 user nova-compute[75234]:   <target dev="sdc" bus="scsi"/>
Nov 29 03:08:31 user nova-compute[75234]:   <serial>ae86b2d0-7343-40f1-9624-ddc92be1340d</serial>
Nov 29 03:08:31 user nova-compute[75234]:   <address type="drive" controller="0" unit="2"/>
Nov 29 03:08:31 user nova-compute[75234]: </disk>
Nov 29 03:08:31 user nova-compute[75234]:  {{(pid=75234) attach_device /opt/stack/nova/nova/virt/libvirt/guest.py:339}}
Nov 29 03:08:31 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-cec4caa8-f3c7-43da-8baf-520dbd9abf1c tempest-AttachSCSIVolumeTestJSON-719577128 tempest-AttachSCSIVolumeTestJSON-719577128-project-member] No BDM found with device name sda, not building metadata. {{(pid=75234) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12089}}
Nov 29 03:08:31 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-cec4caa8-f3c7-43da-8baf-520dbd9abf1c tempest-AttachSCSIVolumeTestJSON-719577128 tempest-AttachSCSIVolumeTestJSON-719577128-project-member] No BDM found with device name sdb, not building metadata. {{(pid=75234) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12089}}
Nov 29 03:08:31 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-cec4caa8-f3c7-43da-8baf-520dbd9abf1c tempest-AttachSCSIVolumeTestJSON-719577128 tempest-AttachSCSIVolumeTestJSON-719577128-project-member] No BDM found with device name sdc, not building metadata. {{(pid=75234) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12089}}
Nov 29 03:08:31 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-cec4caa8-f3c7-43da-8baf-520dbd9abf1c tempest-AttachSCSIVolumeTestJSON-719577128 tempest-AttachSCSIVolumeTestJSON-719577128-project-member] No VIF found with MAC fa:16:3e:7f:17:4d, not building metadata {{(pid=75234) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Nov 29 03:08:31 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:08:31 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-cec4caa8-f3c7-43da-8baf-520dbd9abf1c tempest-AttachSCSIVolumeTestJSON-719577128 tempest-AttachSCSIVolumeTestJSON-719577128-project-member] Lock "da625f9c-b7ed-4afc-875c-6436bf58e9cc" "released" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: held 10.273s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:08:32 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-01a31e51-5891-44ff-ba1a-33093249f28c tempest-AttachSCSIVolumeTestJSON-719577128 tempest-AttachSCSIVolumeTestJSON-719577128-project-member] Acquiring lock "da625f9c-b7ed-4afc-875c-6436bf58e9cc" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:08:32 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-01a31e51-5891-44ff-ba1a-33093249f28c tempest-AttachSCSIVolumeTestJSON-719577128 tempest-AttachSCSIVolumeTestJSON-719577128-project-member] Lock "da625f9c-b7ed-4afc-875c-6436bf58e9cc" acquired by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:08:32 user nova-compute[75234]: INFO nova.compute.manager [None req-01a31e51-5891-44ff-ba1a-33093249f28c tempest-AttachSCSIVolumeTestJSON-719577128 tempest-AttachSCSIVolumeTestJSON-719577128-project-member] [instance: da625f9c-b7ed-4afc-875c-6436bf58e9cc] Detaching volume ae86b2d0-7343-40f1-9624-ddc92be1340d
Nov 29 03:08:32 user nova-compute[75234]: INFO nova.virt.block_device [None req-01a31e51-5891-44ff-ba1a-33093249f28c tempest-AttachSCSIVolumeTestJSON-719577128 tempest-AttachSCSIVolumeTestJSON-719577128-project-member] [instance: da625f9c-b7ed-4afc-875c-6436bf58e9cc] Attempting to driver detach volume ae86b2d0-7343-40f1-9624-ddc92be1340d from mountpoint /dev/sdc
Nov 29 03:08:32 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-01a31e51-5891-44ff-ba1a-33093249f28c tempest-AttachSCSIVolumeTestJSON-719577128 tempest-AttachSCSIVolumeTestJSON-719577128-project-member] Attempting to detach device sdc from instance da625f9c-b7ed-4afc-875c-6436bf58e9cc from the persistent domain config. {{(pid=75234) _detach_from_persistent /opt/stack/nova/nova/virt/libvirt/driver.py:2487}}
Nov 29 03:08:32 user nova-compute[75234]: DEBUG nova.virt.libvirt.guest [None req-01a31e51-5891-44ff-ba1a-33093249f28c tempest-AttachSCSIVolumeTestJSON-719577128 tempest-AttachSCSIVolumeTestJSON-719577128-project-member] detach device xml: <disk type="block" device="disk">
Nov 29 03:08:32 user nova-compute[75234]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Nov 29 03:08:32 user nova-compute[75234]:   <source dev="/dev/sdi"/>
Nov 29 03:08:32 user nova-compute[75234]:   <target dev="sdc" bus="scsi"/>
Nov 29 03:08:32 user nova-compute[75234]:   <serial>ae86b2d0-7343-40f1-9624-ddc92be1340d</serial>
Nov 29 03:08:32 user nova-compute[75234]:   <address type="drive" controller="0" bus="0" target="0" unit="2"/>
Nov 29 03:08:32 user nova-compute[75234]: </disk>
Nov 29 03:08:32 user nova-compute[75234]:  {{(pid=75234) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Nov 29 03:08:32 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-01a31e51-5891-44ff-ba1a-33093249f28c tempest-AttachSCSIVolumeTestJSON-719577128 tempest-AttachSCSIVolumeTestJSON-719577128-project-member] Successfully detached device sdc from instance da625f9c-b7ed-4afc-875c-6436bf58e9cc from the persistent domain config.
Nov 29 03:08:32 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-01a31e51-5891-44ff-ba1a-33093249f28c tempest-AttachSCSIVolumeTestJSON-719577128 tempest-AttachSCSIVolumeTestJSON-719577128-project-member] (1/8): Attempting to detach device sdc with device alias scsi0-0-0-2 from instance da625f9c-b7ed-4afc-875c-6436bf58e9cc from the live domain config. {{(pid=75234) _detach_from_live_with_retry /opt/stack/nova/nova/virt/libvirt/driver.py:2523}}
Nov 29 03:08:32 user nova-compute[75234]: DEBUG nova.virt.libvirt.guest [None req-01a31e51-5891-44ff-ba1a-33093249f28c tempest-AttachSCSIVolumeTestJSON-719577128 tempest-AttachSCSIVolumeTestJSON-719577128-project-member] detach device xml: <disk type="block" device="disk">
Nov 29 03:08:32 user nova-compute[75234]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Nov 29 03:08:32 user nova-compute[75234]:   <source dev="/dev/sdi"/>
Nov 29 03:08:32 user nova-compute[75234]:   <target dev="sdc" bus="scsi"/>
Nov 29 03:08:32 user nova-compute[75234]:   <serial>ae86b2d0-7343-40f1-9624-ddc92be1340d</serial>
Nov 29 03:08:32 user nova-compute[75234]:   <address type="drive" controller="0" bus="0" target="0" unit="2"/>
Nov 29 03:08:32 user nova-compute[75234]: </disk>
Nov 29 03:08:32 user nova-compute[75234]:  {{(pid=75234) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Nov 29 03:08:33 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-01a31e51-5891-44ff-ba1a-33093249f28c tempest-AttachSCSIVolumeTestJSON-719577128 tempest-AttachSCSIVolumeTestJSON-719577128-project-member] Start waiting for the detach event from libvirt for device sdc with device alias scsi0-0-0-2 for instance da625f9c-b7ed-4afc-875c-6436bf58e9cc {{(pid=75234) _detach_from_live_and_wait_for_event /opt/stack/nova/nova/virt/libvirt/driver.py:2599}}
Nov 29 03:08:35 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-1d96cd55-5818-4800-991e-7d3373b14d2d tempest-VolumesAssistedSnapshotsTest-1119739890 tempest-VolumesAssistedSnapshotsTest-1119739890-project-admin] [instance: c9663ab3-3374-400e-ab0a-769731b78cca] Calling os-brick to attach iSCSI Volume {{(pid=75234) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Nov 29 03:08:35 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-1d96cd55-5818-4800-991e-7d3373b14d2d tempest-VolumesAssistedSnapshotsTest-1119739890 tempest-VolumesAssistedSnapshotsTest-1119739890-project-admin] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f8a5050c9a0>, 'connection_properties': {'target_iqn': 'iqn.2010-10.org.openstack:a4cf29d0-2ce2-423f-a20c-a7a19a452088', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'kAVxiylD', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:176}}
Nov 29 03:08:35 user nova-compute[75234]: WARNING os_brick.initiator.connectors.base [None req-1d96cd55-5818-4800-991e-7d3373b14d2d tempest-VolumesAssistedSnapshotsTest-1119739890 tempest-VolumesAssistedSnapshotsTest-1119739890-project-admin] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Nov 29 03:08:35 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-1d96cd55-5818-4800-991e-7d3373b14d2d tempest-VolumesAssistedSnapshotsTest-1119739890 tempest-VolumesAssistedSnapshotsTest-1119739890-project-admin] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:68}}
Nov 29 03:08:35 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-1d96cd55-5818-4800-991e-7d3373b14d2d tempest-VolumesAssistedSnapshotsTest-1119739890 tempest-VolumesAssistedSnapshotsTest-1119739890-project-admin] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:73}}
Nov 29 03:08:35 user nova-compute[75234]: WARNING os_brick.initiator.connectors.base [None req-1d96cd55-5818-4800-991e-7d3373b14d2d tempest-VolumesAssistedSnapshotsTest-1119739890 tempest-VolumesAssistedSnapshotsTest-1119739890-project-admin] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Nov 29 03:08:35 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-1d96cd55-5818-4800-991e-7d3373b14d2d tempest-VolumesAssistedSnapshotsTest-1119739890 tempest-VolumesAssistedSnapshotsTest-1119739890-project-admin] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:a4cf29d0-2ce2-423f-a20c-a7a19a452088" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:68}}
Nov 29 03:08:35 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-1d96cd55-5818-4800-991e-7d3373b14d2d tempest-VolumesAssistedSnapshotsTest-1119739890 tempest-VolumesAssistedSnapshotsTest-1119739890-project-admin] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:a4cf29d0-2ce2-423f-a20c-a7a19a452088" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:73}}
Nov 29 03:08:35 user nova-compute[75234]: INFO os_brick.initiator.connectors.iscsi [None req-1d96cd55-5818-4800-991e-7d3373b14d2d tempest-VolumesAssistedSnapshotsTest-1119739890 tempest-VolumesAssistedSnapshotsTest-1119739890-project-admin] Trying to connect to iSCSI portal 172.16.0.220:3260
Nov 29 03:08:35 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:a4cf29d0-2ce2-423f-a20c-a7a19a452088 -p 172.16.0.220:3260 {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:08:35 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:a4cf29d0-2ce2-423f-a20c-a7a19a452088 -p 172.16.0.220:3260" returned: 21 in 0.018s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:08:35 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[160696f4-faad-450f-96ab-f8c9008d9669]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:08:35 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-1d96cd55-5818-4800-991e-7d3373b14d2d tempest-VolumesAssistedSnapshotsTest-1119739890 tempest-VolumesAssistedSnapshotsTest-1119739890-project-admin] iscsiadm (): stdout= stderr=iscsiadm: No records found
Nov 29 03:08:35 user nova-compute[75234]:  {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:08:35 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:a4cf29d0-2ce2-423f-a20c-a7a19a452088 -p 172.16.0.220:3260 --interface default --op new {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:08:35 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:a4cf29d0-2ce2-423f-a20c-a7a19a452088 -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.018s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:08:35 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[24ec0483-8cad-4e5b-9dfd-b2d2ae156589]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:a4cf29d0-2ce2-423f-a20c-a7a19a452088] added\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:08:35 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-1d96cd55-5818-4800-991e-7d3373b14d2d tempest-VolumesAssistedSnapshotsTest-1119739890 tempest-VolumesAssistedSnapshotsTest-1119739890-project-admin] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:a4cf29d0-2ce2-423f-a20c-a7a19a452088] added
Nov 29 03:08:35 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:08:35 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:a4cf29d0-2ce2-423f-a20c-a7a19a452088 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:08:35 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:a4cf29d0-2ce2-423f-a20c-a7a19a452088 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.018s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:08:35 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[092e04fe-d844-48b9-8fb6-af3df50ad6fc]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:08:35 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-1d96cd55-5818-4800-991e-7d3373b14d2d tempest-VolumesAssistedSnapshotsTest-1119739890 tempest-VolumesAssistedSnapshotsTest-1119739890-project-admin] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:08:35 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:a4cf29d0-2ce2-423f-a20c-a7a19a452088 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:08:35 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:a4cf29d0-2ce2-423f-a20c-a7a19a452088 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.018s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:08:35 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[217a8fba-1e69-4dca-8657-24cf3bbca484]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:08:35 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-1d96cd55-5818-4800-991e-7d3373b14d2d tempest-VolumesAssistedSnapshotsTest-1119739890 tempest-VolumesAssistedSnapshotsTest-1119739890-project-admin] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:08:35 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:a4cf29d0-2ce2-423f-a20c-a7a19a452088 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v kAVxiylD {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:08:35 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:a4cf29d0-2ce2-423f-a20c-a7a19a452088 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v kAVxiylD" returned: 0 in 0.025s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:08:35 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[6d0413cc-6cb0-43d3-97cb-d3ef90fbcb44]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:08:35 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-1d96cd55-5818-4800-991e-7d3373b14d2d tempest-VolumesAssistedSnapshotsTest-1119739890 tempest-VolumesAssistedSnapshotsTest-1119739890-project-admin] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'kAVxiylD'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:08:35 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:a4cf29d0-2ce2-423f-a20c-a7a19a452088 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:08:35 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:a4cf29d0-2ce2-423f-a20c-a7a19a452088 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.018s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:08:35 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[4aa99d2a-afe1-4ca7-826e-5bef7450ae60]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:08:35 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-1d96cd55-5818-4800-991e-7d3373b14d2d tempest-VolumesAssistedSnapshotsTest-1119739890 tempest-VolumesAssistedSnapshotsTest-1119739890-project-admin] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:08:35 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:08:35 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.029s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:08:35 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[22d3e6d1-0963-458f-9946-292cfbf73345]: (4, ('tcp: [10] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ae86b2d0-7343-40f1-9624-ddc92be1340d (non-flash)\ntcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bb98586e-4f75-46be-a626-0b73ebda11eb (non-flash)\ntcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ab278679-6cfd-4c7c-a930-c078843640d3 (non-flash)\ntcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:920d6cad-25ab-4f3d-941b-8ec841d43184 (non-flash)\ntcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b1068fd1-6636-43cd-9725-4a37f708b2bc (non-flash)\ntcp: [6] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0d2c0c22-2b3a-43db-b9af-e60a2514e48e (non-flash)\ntcp: [7] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5fcd66fc-a479-4613-b1f8-e6ef12ca470a (non-flash)\ntcp: [9] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672 (non-flash)\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:08:35 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-1d96cd55-5818-4800-991e-7d3373b14d2d tempest-VolumesAssistedSnapshotsTest-1119739890 tempest-VolumesAssistedSnapshotsTest-1119739890-project-admin] iscsiadm ('-m', 'session'): stdout=tcp: [10] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ae86b2d0-7343-40f1-9624-ddc92be1340d (non-flash)
Nov 29 03:08:35 user nova-compute[75234]: tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bb98586e-4f75-46be-a626-0b73ebda11eb (non-flash)
Nov 29 03:08:35 user nova-compute[75234]: tcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ab278679-6cfd-4c7c-a930-c078843640d3 (non-flash)
Nov 29 03:08:35 user nova-compute[75234]: tcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:920d6cad-25ab-4f3d-941b-8ec841d43184 (non-flash)
Nov 29 03:08:35 user nova-compute[75234]: tcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b1068fd1-6636-43cd-9725-4a37f708b2bc (non-flash)
Nov 29 03:08:35 user nova-compute[75234]: tcp: [6] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0d2c0c22-2b3a-43db-b9af-e60a2514e48e (non-flash)
Nov 29 03:08:35 user nova-compute[75234]: tcp: [7] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5fcd66fc-a479-4613-b1f8-e6ef12ca470a (non-flash)
Nov 29 03:08:35 user nova-compute[75234]: tcp: [9] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672 (non-flash)
Nov 29 03:08:35 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:08:35 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-1d96cd55-5818-4800-991e-7d3373b14d2d tempest-VolumesAssistedSnapshotsTest-1119739890 tempest-VolumesAssistedSnapshotsTest-1119739890-project-admin] iscsi session list stdout=tcp: [10] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ae86b2d0-7343-40f1-9624-ddc92be1340d (non-flash)
Nov 29 03:08:35 user nova-compute[75234]: tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bb98586e-4f75-46be-a626-0b73ebda11eb (non-flash)
Nov 29 03:08:35 user nova-compute[75234]: tcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ab278679-6cfd-4c7c-a930-c078843640d3 (non-flash)
Nov 29 03:08:35 user nova-compute[75234]: tcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:920d6cad-25ab-4f3d-941b-8ec841d43184 (non-flash)
Nov 29 03:08:35 user nova-compute[75234]: tcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b1068fd1-6636-43cd-9725-4a37f708b2bc (non-flash)
Nov 29 03:08:35 user nova-compute[75234]: tcp: [6] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0d2c0c22-2b3a-43db-b9af-e60a2514e48e (non-flash)
Nov 29 03:08:35 user nova-compute[75234]: tcp: [7] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5fcd66fc-a479-4613-b1f8-e6ef12ca470a (non-flash)
Nov 29 03:08:35 user nova-compute[75234]: tcp: [9] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672 (non-flash)
Nov 29 03:08:35 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsi_session /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Nov 29 03:08:35 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:a4cf29d0-2ce2-423f-a20c-a7a19a452088 -p 172.16.0.220:3260 --login {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:08:35 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:a4cf29d0-2ce2-423f-a20c-a7a19a452088 -p 172.16.0.220:3260 --login" returned: 0 in 0.084s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:08:35 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[cd124bfe-6376-4e2c-9595-791b3d347a89]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:a4cf29d0-2ce2-423f-a20c-a7a19a452088, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:a4cf29d0-2ce2-423f-a20c-a7a19a452088, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:08:35 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-1d96cd55-5818-4800-991e-7d3373b14d2d tempest-VolumesAssistedSnapshotsTest-1119739890 tempest-VolumesAssistedSnapshotsTest-1119739890-project-admin] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:a4cf29d0-2ce2-423f-a20c-a7a19a452088, portal: 172.16.0.220,3260]
Nov 29 03:08:35 user nova-compute[75234]: Login to [iface: default, target: iqn.2010-10.org.openstack:a4cf29d0-2ce2-423f-a20c-a7a19a452088, portal: 172.16.0.220,3260] successful.
Nov 29 03:08:35 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:08:35 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:a4cf29d0-2ce2-423f-a20c-a7a19a452088 -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:08:35 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:a4cf29d0-2ce2-423f-a20c-a7a19a452088 -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.019s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:08:35 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[f0fca8ee-dee8-4ac1-afb4-969be82c98f0]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:08:35 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-1d96cd55-5818-4800-991e-7d3373b14d2d tempest-VolumesAssistedSnapshotsTest-1119739890 tempest-VolumesAssistedSnapshotsTest-1119739890-project-admin] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:08:35 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:08:36 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.036s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:08:36 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[2002c5ef-1b2a-4b70-8561-a1b3f7c1ae01]: (4, ('tcp: [10] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ae86b2d0-7343-40f1-9624-ddc92be1340d (non-flash)\ntcp: [11] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a4cf29d0-2ce2-423f-a20c-a7a19a452088 (non-flash)\ntcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bb98586e-4f75-46be-a626-0b73ebda11eb (non-flash)\ntcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ab278679-6cfd-4c7c-a930-c078843640d3 (non-flash)\ntcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:920d6cad-25ab-4f3d-941b-8ec841d43184 (non-flash)\ntcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b1068fd1-6636-43cd-9725-4a37f708b2bc (non-flash)\ntcp: [6] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0d2c0c22-2b3a-43db-b9af-e60a2514e48e (non-flash)\ntcp: [7] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5fcd66fc-a479-4613-b1f8-e6ef12ca470a (non-flash)\ntcp: [9] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672 (non-flash)\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:08:36 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-1d96cd55-5818-4800-991e-7d3373b14d2d tempest-VolumesAssistedSnapshotsTest-1119739890 tempest-VolumesAssistedSnapshotsTest-1119739890-project-admin] iscsiadm ('-m', 'session'): stdout=tcp: [10] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ae86b2d0-7343-40f1-9624-ddc92be1340d (non-flash)
Nov 29 03:08:36 user nova-compute[75234]: tcp: [11] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a4cf29d0-2ce2-423f-a20c-a7a19a452088 (non-flash)
Nov 29 03:08:36 user nova-compute[75234]: tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bb98586e-4f75-46be-a626-0b73ebda11eb (non-flash)
Nov 29 03:08:36 user nova-compute[75234]: tcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ab278679-6cfd-4c7c-a930-c078843640d3 (non-flash)
Nov 29 03:08:36 user nova-compute[75234]: tcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:920d6cad-25ab-4f3d-941b-8ec841d43184 (non-flash)
Nov 29 03:08:36 user nova-compute[75234]: tcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b1068fd1-6636-43cd-9725-4a37f708b2bc (non-flash)
Nov 29 03:08:36 user nova-compute[75234]: tcp: [6] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0d2c0c22-2b3a-43db-b9af-e60a2514e48e (non-flash)
Nov 29 03:08:36 user nova-compute[75234]: tcp: [7] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5fcd66fc-a479-4613-b1f8-e6ef12ca470a (non-flash)
Nov 29 03:08:36 user nova-compute[75234]: tcp: [9] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672 (non-flash)
Nov 29 03:08:36 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:08:36 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-1d96cd55-5818-4800-991e-7d3373b14d2d tempest-VolumesAssistedSnapshotsTest-1119739890 tempest-VolumesAssistedSnapshotsTest-1119739890-project-admin] iscsi session list stdout=tcp: [10] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ae86b2d0-7343-40f1-9624-ddc92be1340d (non-flash)
Nov 29 03:08:36 user nova-compute[75234]: tcp: [11] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a4cf29d0-2ce2-423f-a20c-a7a19a452088 (non-flash)
Nov 29 03:08:36 user nova-compute[75234]: tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bb98586e-4f75-46be-a626-0b73ebda11eb (non-flash)
Nov 29 03:08:36 user nova-compute[75234]: tcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ab278679-6cfd-4c7c-a930-c078843640d3 (non-flash)
Nov 29 03:08:36 user nova-compute[75234]: tcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:920d6cad-25ab-4f3d-941b-8ec841d43184 (non-flash)
Nov 29 03:08:36 user nova-compute[75234]: tcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b1068fd1-6636-43cd-9725-4a37f708b2bc (non-flash)
Nov 29 03:08:36 user nova-compute[75234]: tcp: [6] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0d2c0c22-2b3a-43db-b9af-e60a2514e48e (non-flash)
Nov 29 03:08:36 user nova-compute[75234]: tcp: [7] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5fcd66fc-a479-4613-b1f8-e6ef12ca470a (non-flash)
Nov 29 03:08:36 user nova-compute[75234]: tcp: [9] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672 (non-flash)
Nov 29 03:08:36 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsi_session /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Nov 29 03:08:36 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-1d96cd55-5818-4800-991e-7d3373b14d2d tempest-VolumesAssistedSnapshotsTest-1119739890 tempest-VolumesAssistedSnapshotsTest-1119739890-project-admin] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:a4cf29d0-2ce2-423f-a20c-a7a19a452088" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.322s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:87}}
Nov 29 03:08:36 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-1d96cd55-5818-4800-991e-7d3373b14d2d tempest-VolumesAssistedSnapshotsTest-1119739890 tempest-VolumesAssistedSnapshotsTest-1119739890-project-admin] Connected to 172.16.0.220:3260 {{(pid=75234) _connect_vol /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:644}}
Nov 29 03:08:36 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-1d96cd55-5818-4800-991e-7d3373b14d2d tempest-VolumesAssistedSnapshotsTest-1119739890 tempest-VolumesAssistedSnapshotsTest-1119739890-project-admin] HCTL ('41', '-', '-', 0) found on session 11 with lun 0 {{(pid=75234) get_hctl /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:770}}
Nov 29 03:08:36 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-1d96cd55-5818-4800-991e-7d3373b14d2d tempest-VolumesAssistedSnapshotsTest-1119739890 tempest-VolumesAssistedSnapshotsTest-1119739890-project-admin] Scanning host 41 c: -, t: -, l: 0) {{(pid=75234) scan_iscsi /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:795}}
Nov 29 03:08:36 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host41/scan {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:08:36 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host41/scan" returned: 0 in 0.016s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:08:36 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[3e906e52-819e-443a-9c8d-ebd8452f31e5]: (4, ('- - 0', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:08:36 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-1d96cd55-5818-4800-991e-7d3373b14d2d tempest-VolumesAssistedSnapshotsTest-1119739890 tempest-VolumesAssistedSnapshotsTest-1119739890-project-admin] Searching for a device in session 11 and hctl ['41', '*', '*', 0] yield: sdj {{(pid=75234) device_name_by_hctl /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:789}}
Nov 29 03:08:36 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-1d96cd55-5818-4800-991e-7d3373b14d2d tempest-VolumesAssistedSnapshotsTest-1119739890 tempest-VolumesAssistedSnapshotsTest-1119739890-project-admin] Connected to sdj using {'target_iqn': 'iqn.2010-10.org.openstack:a4cf29d0-2ce2-423f-a20c-a7a19a452088', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'kAVxiylD', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False} {{(pid=75234) _connect_vol /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:672}}
Nov 29 03:08:36 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:08:36 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:08:37 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-1d96cd55-5818-4800-991e-7d3373b14d2d tempest-VolumesAssistedSnapshotsTest-1119739890 tempest-VolumesAssistedSnapshotsTest-1119739890-project-admin] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.356s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:87}}
Nov 29 03:08:37 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-1d96cd55-5818-4800-991e-7d3373b14d2d tempest-VolumesAssistedSnapshotsTest-1119739890 tempest-VolumesAssistedSnapshotsTest-1119739890-project-admin] <== connect_volume: return (1357ms) {'type': 'block', 'scsi_wwn': '23938326334643537', 'path': '/dev/sdj'} {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:203}}
Nov 29 03:08:37 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-1d96cd55-5818-4800-991e-7d3373b14d2d tempest-VolumesAssistedSnapshotsTest-1119739890 tempest-VolumesAssistedSnapshotsTest-1119739890-project-admin] [instance: c9663ab3-3374-400e-ab0a-769731b78cca] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '23938326334643537', 'path': '/dev/sdj'} {{(pid=75234) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Nov 29 03:08:37 user nova-compute[75234]: DEBUG nova.objects.instance [None req-1d96cd55-5818-4800-991e-7d3373b14d2d tempest-VolumesAssistedSnapshotsTest-1119739890 tempest-VolumesAssistedSnapshotsTest-1119739890-project-admin] Lazy-loading 'flavor' on Instance uuid c9663ab3-3374-400e-ab0a-769731b78cca {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:08:37 user nova-compute[75234]: DEBUG nova.virt.libvirt.guest [None req-1d96cd55-5818-4800-991e-7d3373b14d2d tempest-VolumesAssistedSnapshotsTest-1119739890 tempest-VolumesAssistedSnapshotsTest-1119739890-project-admin] attach device xml: <disk type="block" device="disk">
Nov 29 03:08:37 user nova-compute[75234]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Nov 29 03:08:37 user nova-compute[75234]:   <source dev="/dev/sdj"/>
Nov 29 03:08:37 user nova-compute[75234]:   <target dev="vdb" bus="virtio"/>
Nov 29 03:08:37 user nova-compute[75234]:   <serial>a4cf29d0-2ce2-423f-a20c-a7a19a452088</serial>
Nov 29 03:08:37 user nova-compute[75234]: </disk>
Nov 29 03:08:37 user nova-compute[75234]:  {{(pid=75234) attach_device /opt/stack/nova/nova/virt/libvirt/guest.py:339}}
Nov 29 03:08:37 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-1d96cd55-5818-4800-991e-7d3373b14d2d tempest-VolumesAssistedSnapshotsTest-1119739890 tempest-VolumesAssistedSnapshotsTest-1119739890-project-admin] No BDM found with device name vda, not building metadata. {{(pid=75234) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12089}}
Nov 29 03:08:37 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-1d96cd55-5818-4800-991e-7d3373b14d2d tempest-VolumesAssistedSnapshotsTest-1119739890 tempest-VolumesAssistedSnapshotsTest-1119739890-project-admin] No BDM found with device name vdb, not building metadata. {{(pid=75234) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12089}}
Nov 29 03:08:37 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-1d96cd55-5818-4800-991e-7d3373b14d2d tempest-VolumesAssistedSnapshotsTest-1119739890 tempest-VolumesAssistedSnapshotsTest-1119739890-project-admin] No VIF found with MAC fa:16:3e:3e:b6:0b, not building metadata {{(pid=75234) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Nov 29 03:08:37 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-1d96cd55-5818-4800-991e-7d3373b14d2d tempest-VolumesAssistedSnapshotsTest-1119739890 tempest-VolumesAssistedSnapshotsTest-1119739890-project-admin] Lock "c9663ab3-3374-400e-ab0a-769731b78cca" "released" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: held 10.163s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:08:38 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-83d62bf6-1454-424f-a05b-50e404419735 tempest-VolumesAssistedSnapshotsTest-1691523836 tempest-VolumesAssistedSnapshotsTest-1691523836-project] [instance: c9663ab3-3374-400e-ab0a-769731b78cca] volume_snapshot_create: create_info: {'snapshot_id': '59222e85-b132-4ba8-886e-55fbed5b38d7', 'type': 'qcow2', 'new_file': 'new_file'} {{(pid=75234) volume_snapshot_create /opt/stack/nova/nova/virt/libvirt/driver.py:3576}}
Nov 29 03:08:38 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-83d62bf6-1454-424f-a05b-50e404419735 tempest-VolumesAssistedSnapshotsTest-1691523836 tempest-VolumesAssistedSnapshotsTest-1691523836-project] [instance: c9663ab3-3374-400e-ab0a-769731b78cca] snap xml: <domainsnapshot>
Nov 29 03:08:38 user nova-compute[75234]:   <disks>
Nov 29 03:08:38 user nova-compute[75234]:     <disk name="/dev/sdj" snapshot="external" type="file">
Nov 29 03:08:38 user nova-compute[75234]:       <source file="/dev/new_file"/>
Nov 29 03:08:38 user nova-compute[75234]:     </disk>
Nov 29 03:08:38 user nova-compute[75234]:     <disk name="vda" snapshot="no"/>
Nov 29 03:08:38 user nova-compute[75234]:   </disks>
Nov 29 03:08:38 user nova-compute[75234]: </domainsnapshot>
Nov 29 03:08:38 user nova-compute[75234]:  {{(pid=75234) _volume_snapshot_create /opt/stack/nova/nova/virt/libvirt/driver.py:3517}}
Nov 29 03:08:38 user nova-compute[75234]: DEBUG nova.objects.instance [None req-83d62bf6-1454-424f-a05b-50e404419735 tempest-VolumesAssistedSnapshotsTest-1691523836 tempest-VolumesAssistedSnapshotsTest-1691523836-project] Lazy-loading 'system_metadata' on Instance uuid c9663ab3-3374-400e-ab0a-769731b78cca {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:08:38 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-83d62bf6-1454-424f-a05b-50e404419735 tempest-VolumesAssistedSnapshotsTest-1691523836 tempest-VolumesAssistedSnapshotsTest-1691523836-project] [instance: c9663ab3-3374-400e-ab0a-769731b78cca] Skipping quiescing instance: QEMU guest agent is not enabled.
Nov 29 03:08:38 user nova-compute[75234]: ERROR nova.virt.libvirt.driver [None req-83d62bf6-1454-424f-a05b-50e404419735 tempest-VolumesAssistedSnapshotsTest-1691523836 tempest-VolumesAssistedSnapshotsTest-1691523836-project] [instance: c9663ab3-3374-400e-ab0a-769731b78cca] Unable to create VM snapshot, failing volume_snapshot operation.: libvirt.libvirtError: missing existing file for disk vdb: /dev/new_file: No such file or directory
Nov 29 03:08:38 user nova-compute[75234]: ERROR nova.virt.libvirt.driver [instance: c9663ab3-3374-400e-ab0a-769731b78cca] Traceback (most recent call last):
Nov 29 03:08:38 user nova-compute[75234]: ERROR nova.virt.libvirt.driver [instance: c9663ab3-3374-400e-ab0a-769731b78cca]   File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 3544, in _volume_snapshot_create
Nov 29 03:08:38 user nova-compute[75234]: ERROR nova.virt.libvirt.driver [instance: c9663ab3-3374-400e-ab0a-769731b78cca]     guest.snapshot(snapshot, no_metadata=True, disk_only=True,
Nov 29 03:08:38 user nova-compute[75234]: ERROR nova.virt.libvirt.driver [instance: c9663ab3-3374-400e-ab0a-769731b78cca]   File "/opt/stack/nova/nova/virt/libvirt/guest.py", line 569, in snapshot
Nov 29 03:08:38 user nova-compute[75234]: ERROR nova.virt.libvirt.driver [instance: c9663ab3-3374-400e-ab0a-769731b78cca]     self._domain.snapshotCreateXML(device_xml, flags=flags)
Nov 29 03:08:38 user nova-compute[75234]: ERROR nova.virt.libvirt.driver [instance: c9663ab3-3374-400e-ab0a-769731b78cca]   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/tpool.py", line 193, in doit
Nov 29 03:08:38 user nova-compute[75234]: ERROR nova.virt.libvirt.driver [instance: c9663ab3-3374-400e-ab0a-769731b78cca]     result = proxy_call(self._autowrap, f, *args, **kwargs)
Nov 29 03:08:38 user nova-compute[75234]: ERROR nova.virt.libvirt.driver [instance: c9663ab3-3374-400e-ab0a-769731b78cca]   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/tpool.py", line 151, in proxy_call
Nov 29 03:08:38 user nova-compute[75234]: ERROR nova.virt.libvirt.driver [instance: c9663ab3-3374-400e-ab0a-769731b78cca]     rv = execute(f, *args, **kwargs)
Nov 29 03:08:38 user nova-compute[75234]: ERROR nova.virt.libvirt.driver [instance: c9663ab3-3374-400e-ab0a-769731b78cca]   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/tpool.py", line 132, in execute
Nov 29 03:08:38 user nova-compute[75234]: ERROR nova.virt.libvirt.driver [instance: c9663ab3-3374-400e-ab0a-769731b78cca]     six.reraise(c, e, tb)
Nov 29 03:08:38 user nova-compute[75234]: ERROR nova.virt.libvirt.driver [instance: c9663ab3-3374-400e-ab0a-769731b78cca]   File "/usr/lib/python3/dist-packages/six.py", line 719, in reraise
Nov 29 03:08:38 user nova-compute[75234]: ERROR nova.virt.libvirt.driver [instance: c9663ab3-3374-400e-ab0a-769731b78cca]     raise value
Nov 29 03:08:38 user nova-compute[75234]: ERROR nova.virt.libvirt.driver [instance: c9663ab3-3374-400e-ab0a-769731b78cca]   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/tpool.py", line 86, in tworker
Nov 29 03:08:38 user nova-compute[75234]: ERROR nova.virt.libvirt.driver [instance: c9663ab3-3374-400e-ab0a-769731b78cca]     rv = meth(*args, **kwargs)
Nov 29 03:08:38 user nova-compute[75234]: ERROR nova.virt.libvirt.driver [instance: c9663ab3-3374-400e-ab0a-769731b78cca]   File "/usr/lib/python3/dist-packages/libvirt.py", line 3070, in snapshotCreateXML
Nov 29 03:08:38 user nova-compute[75234]: ERROR nova.virt.libvirt.driver [instance: c9663ab3-3374-400e-ab0a-769731b78cca]     raise libvirtError('virDomainSnapshotCreateXML() failed')
Nov 29 03:08:38 user nova-compute[75234]: ERROR nova.virt.libvirt.driver [instance: c9663ab3-3374-400e-ab0a-769731b78cca] libvirt.libvirtError: missing existing file for disk vdb: /dev/new_file: No such file or directory
Nov 29 03:08:38 user nova-compute[75234]: ERROR nova.virt.libvirt.driver [instance: c9663ab3-3374-400e-ab0a-769731b78cca] 
Nov 29 03:08:38 user nova-compute[75234]: ERROR nova.virt.libvirt.driver [None req-83d62bf6-1454-424f-a05b-50e404419735 tempest-VolumesAssistedSnapshotsTest-1691523836 tempest-VolumesAssistedSnapshotsTest-1691523836-project] [instance: c9663ab3-3374-400e-ab0a-769731b78cca] Error occurred during volume_snapshot_create, sending error status to Cinder.: libvirt.libvirtError: missing existing file for disk vdb: /dev/new_file: No such file or directory
Nov 29 03:08:38 user nova-compute[75234]: ERROR nova.virt.libvirt.driver [instance: c9663ab3-3374-400e-ab0a-769731b78cca] Traceback (most recent call last):
Nov 29 03:08:38 user nova-compute[75234]: ERROR nova.virt.libvirt.driver [instance: c9663ab3-3374-400e-ab0a-769731b78cca]   File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 3594, in volume_snapshot_create
Nov 29 03:08:38 user nova-compute[75234]: ERROR nova.virt.libvirt.driver [instance: c9663ab3-3374-400e-ab0a-769731b78cca]     self._volume_snapshot_create(context, instance, guest,
Nov 29 03:08:38 user nova-compute[75234]: ERROR nova.virt.libvirt.driver [instance: c9663ab3-3374-400e-ab0a-769731b78cca]   File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 3544, in _volume_snapshot_create
Nov 29 03:08:38 user nova-compute[75234]: ERROR nova.virt.libvirt.driver [instance: c9663ab3-3374-400e-ab0a-769731b78cca]     guest.snapshot(snapshot, no_metadata=True, disk_only=True,
Nov 29 03:08:38 user nova-compute[75234]: ERROR nova.virt.libvirt.driver [instance: c9663ab3-3374-400e-ab0a-769731b78cca]   File "/opt/stack/nova/nova/virt/libvirt/guest.py", line 569, in snapshot
Nov 29 03:08:38 user nova-compute[75234]: ERROR nova.virt.libvirt.driver [instance: c9663ab3-3374-400e-ab0a-769731b78cca]     self._domain.snapshotCreateXML(device_xml, flags=flags)
Nov 29 03:08:38 user nova-compute[75234]: ERROR nova.virt.libvirt.driver [instance: c9663ab3-3374-400e-ab0a-769731b78cca]   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/tpool.py", line 193, in doit
Nov 29 03:08:38 user nova-compute[75234]: ERROR nova.virt.libvirt.driver [instance: c9663ab3-3374-400e-ab0a-769731b78cca]     result = proxy_call(self._autowrap, f, *args, **kwargs)
Nov 29 03:08:38 user nova-compute[75234]: ERROR nova.virt.libvirt.driver [instance: c9663ab3-3374-400e-ab0a-769731b78cca]   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/tpool.py", line 151, in proxy_call
Nov 29 03:08:38 user nova-compute[75234]: ERROR nova.virt.libvirt.driver [instance: c9663ab3-3374-400e-ab0a-769731b78cca]     rv = execute(f, *args, **kwargs)
Nov 29 03:08:38 user nova-compute[75234]: ERROR nova.virt.libvirt.driver [instance: c9663ab3-3374-400e-ab0a-769731b78cca]   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/tpool.py", line 132, in execute
Nov 29 03:08:38 user nova-compute[75234]: ERROR nova.virt.libvirt.driver [instance: c9663ab3-3374-400e-ab0a-769731b78cca]     six.reraise(c, e, tb)
Nov 29 03:08:38 user nova-compute[75234]: ERROR nova.virt.libvirt.driver [instance: c9663ab3-3374-400e-ab0a-769731b78cca]   File "/usr/lib/python3/dist-packages/six.py", line 719, in reraise
Nov 29 03:08:38 user nova-compute[75234]: ERROR nova.virt.libvirt.driver [instance: c9663ab3-3374-400e-ab0a-769731b78cca]     raise value
Nov 29 03:08:38 user nova-compute[75234]: ERROR nova.virt.libvirt.driver [instance: c9663ab3-3374-400e-ab0a-769731b78cca]   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/tpool.py", line 86, in tworker
Nov 29 03:08:38 user nova-compute[75234]: ERROR nova.virt.libvirt.driver [instance: c9663ab3-3374-400e-ab0a-769731b78cca]     rv = meth(*args, **kwargs)
Nov 29 03:08:38 user nova-compute[75234]: ERROR nova.virt.libvirt.driver [instance: c9663ab3-3374-400e-ab0a-769731b78cca]   File "/usr/lib/python3/dist-packages/libvirt.py", line 3070, in snapshotCreateXML
Nov 29 03:08:38 user nova-compute[75234]: ERROR nova.virt.libvirt.driver [instance: c9663ab3-3374-400e-ab0a-769731b78cca]     raise libvirtError('virDomainSnapshotCreateXML() failed')
Nov 29 03:08:38 user nova-compute[75234]: ERROR nova.virt.libvirt.driver [instance: c9663ab3-3374-400e-ab0a-769731b78cca] libvirt.libvirtError: missing existing file for disk vdb: /dev/new_file: No such file or directory
Nov 29 03:08:38 user nova-compute[75234]: ERROR nova.virt.libvirt.driver [instance: c9663ab3-3374-400e-ab0a-769731b78cca] 
Nov 29 03:08:38 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-65d16d35-2605-4555-b23f-f7b512e92b81 tempest-VolumesAssistedSnapshotsTest-1691523836 tempest-VolumesAssistedSnapshotsTest-1691523836-project] [instance: c9663ab3-3374-400e-ab0a-769731b78cca] volume_snapshot_delete: delete_info: {'volume_id': 'a4cf29d0-2ce2-423f-a20c-a7a19a452088'} {{(pid=75234) _volume_snapshot_delete /opt/stack/nova/nova/virt/libvirt/driver.py:3677}}
Nov 29 03:08:38 user nova-compute[75234]: ERROR nova.virt.libvirt.driver [None req-65d16d35-2605-4555-b23f-f7b512e92b81 tempest-VolumesAssistedSnapshotsTest-1691523836 tempest-VolumesAssistedSnapshotsTest-1691523836-project] [instance: c9663ab3-3374-400e-ab0a-769731b78cca] Error occurred during volume_snapshot_delete, sending error status to Cinder.: KeyError: 'type'
Nov 29 03:08:38 user nova-compute[75234]: ERROR nova.virt.libvirt.driver [instance: c9663ab3-3374-400e-ab0a-769731b78cca] Traceback (most recent call last):
Nov 29 03:08:38 user nova-compute[75234]: ERROR nova.virt.libvirt.driver [instance: c9663ab3-3374-400e-ab0a-769731b78cca]   File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 3850, in volume_snapshot_delete
Nov 29 03:08:38 user nova-compute[75234]: ERROR nova.virt.libvirt.driver [instance: c9663ab3-3374-400e-ab0a-769731b78cca]     self._volume_snapshot_delete(context, instance, volume_id,
Nov 29 03:08:38 user nova-compute[75234]: ERROR nova.virt.libvirt.driver [instance: c9663ab3-3374-400e-ab0a-769731b78cca]   File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 3680, in _volume_snapshot_delete
Nov 29 03:08:38 user nova-compute[75234]: ERROR nova.virt.libvirt.driver [instance: c9663ab3-3374-400e-ab0a-769731b78cca]     if delete_info['type'] != 'qcow2':
Nov 29 03:08:38 user nova-compute[75234]: ERROR nova.virt.libvirt.driver [instance: c9663ab3-3374-400e-ab0a-769731b78cca] KeyError: 'type'
Nov 29 03:08:38 user nova-compute[75234]: ERROR nova.virt.libvirt.driver [instance: c9663ab3-3374-400e-ab0a-769731b78cca] 
Nov 29 03:08:38 user nova-compute[75234]: ERROR nova.virt.libvirt.driver [None req-83d62bf6-1454-424f-a05b-50e404419735 tempest-VolumesAssistedSnapshotsTest-1691523836 tempest-VolumesAssistedSnapshotsTest-1691523836-project] Failed to send updated snapshot status to volume service.: nova.exception.SnapshotNotFound: Snapshot 59222e85-b132-4ba8-886e-55fbed5b38d7 could not be found.
Nov 29 03:08:38 user nova-compute[75234]: ERROR nova.virt.libvirt.driver Traceback (most recent call last):
Nov 29 03:08:38 user nova-compute[75234]: ERROR nova.virt.libvirt.driver   File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 3594, in volume_snapshot_create
Nov 29 03:08:38 user nova-compute[75234]: ERROR nova.virt.libvirt.driver     self._volume_snapshot_create(context, instance, guest,
Nov 29 03:08:38 user nova-compute[75234]: ERROR nova.virt.libvirt.driver   File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 3544, in _volume_snapshot_create
Nov 29 03:08:38 user nova-compute[75234]: ERROR nova.virt.libvirt.driver     guest.snapshot(snapshot, no_metadata=True, disk_only=True,
Nov 29 03:08:38 user nova-compute[75234]: ERROR nova.virt.libvirt.driver   File "/opt/stack/nova/nova/virt/libvirt/guest.py", line 569, in snapshot
Nov 29 03:08:38 user nova-compute[75234]: ERROR nova.virt.libvirt.driver     self._domain.snapshotCreateXML(device_xml, flags=flags)
Nov 29 03:08:38 user nova-compute[75234]: ERROR nova.virt.libvirt.driver   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/tpool.py", line 193, in doit
Nov 29 03:08:38 user nova-compute[75234]: ERROR nova.virt.libvirt.driver     result = proxy_call(self._autowrap, f, *args, **kwargs)
Nov 29 03:08:38 user nova-compute[75234]: ERROR nova.virt.libvirt.driver   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/tpool.py", line 151, in proxy_call
Nov 29 03:08:38 user nova-compute[75234]: ERROR nova.virt.libvirt.driver     rv = execute(f, *args, **kwargs)
Nov 29 03:08:38 user nova-compute[75234]: ERROR nova.virt.libvirt.driver   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/tpool.py", line 132, in execute
Nov 29 03:08:38 user nova-compute[75234]: ERROR nova.virt.libvirt.driver     six.reraise(c, e, tb)
Nov 29 03:08:38 user nova-compute[75234]: ERROR nova.virt.libvirt.driver   File "/usr/lib/python3/dist-packages/six.py", line 719, in reraise
Nov 29 03:08:38 user nova-compute[75234]: ERROR nova.virt.libvirt.driver     raise value
Nov 29 03:08:38 user nova-compute[75234]: ERROR nova.virt.libvirt.driver   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/tpool.py", line 86, in tworker
Nov 29 03:08:38 user nova-compute[75234]: ERROR nova.virt.libvirt.driver     rv = meth(*args, **kwargs)
Nov 29 03:08:38 user nova-compute[75234]: ERROR nova.virt.libvirt.driver   File "/usr/lib/python3/dist-packages/libvirt.py", line 3070, in snapshotCreateXML
Nov 29 03:08:38 user nova-compute[75234]: ERROR nova.virt.libvirt.driver     raise libvirtError('virDomainSnapshotCreateXML() failed')
Nov 29 03:08:38 user nova-compute[75234]: ERROR nova.virt.libvirt.driver libvirt.libvirtError: missing existing file for disk vdb: /dev/new_file: No such file or directory
Nov 29 03:08:38 user nova-compute[75234]: ERROR nova.virt.libvirt.driver 
Nov 29 03:08:38 user nova-compute[75234]: ERROR nova.virt.libvirt.driver During handling of the above exception, another exception occurred:
Nov 29 03:08:38 user nova-compute[75234]: ERROR nova.virt.libvirt.driver 
Nov 29 03:08:38 user nova-compute[75234]: ERROR nova.virt.libvirt.driver Traceback (most recent call last):
Nov 29 03:08:38 user nova-compute[75234]: ERROR nova.virt.libvirt.driver   File "/opt/stack/nova/nova/volume/cinder.py", line 466, in wrapper
Nov 29 03:08:38 user nova-compute[75234]: ERROR nova.virt.libvirt.driver     res = method(self, ctx, snapshot_id, *args, **kwargs)
Nov 29 03:08:38 user nova-compute[75234]: ERROR nova.virt.libvirt.driver   File "/opt/stack/nova/nova/volume/cinder.py", line 761, in update_snapshot_status
Nov 29 03:08:38 user nova-compute[75234]: ERROR nova.virt.libvirt.driver     vs.update_snapshot_status(
Nov 29 03:08:38 user nova-compute[75234]: ERROR nova.virt.libvirt.driver   File "/opt/stack/data/venv/lib/python3.10/site-packages/cinderclient/v3/volume_snapshots.py", line 225, in update_snapshot_status
Nov 29 03:08:38 user nova-compute[75234]: ERROR nova.virt.libvirt.driver     return self._action('os-update_snapshot_status',
Nov 29 03:08:38 user nova-compute[75234]: ERROR nova.virt.libvirt.driver   File "/opt/stack/data/venv/lib/python3.10/site-packages/cinderclient/v3/volume_snapshots.py", line 221, in _action
Nov 29 03:08:38 user nova-compute[75234]: ERROR nova.virt.libvirt.driver     resp, body = self.api.client.post(url, body=body)
Nov 29 03:08:38 user nova-compute[75234]: ERROR nova.virt.libvirt.driver   File "/opt/stack/data/venv/lib/python3.10/site-packages/cinderclient/client.py", line 223, in post
Nov 29 03:08:38 user nova-compute[75234]: ERROR nova.virt.libvirt.driver     return self._cs_request(url, 'POST', **kwargs)
Nov 29 03:08:38 user nova-compute[75234]: ERROR nova.virt.libvirt.driver   File "/opt/stack/data/venv/lib/python3.10/site-packages/cinderclient/client.py", line 211, in _cs_request
Nov 29 03:08:38 user nova-compute[75234]: ERROR nova.virt.libvirt.driver     return self.request(url, method, **kwargs)
Nov 29 03:08:38 user nova-compute[75234]: ERROR nova.virt.libvirt.driver   File "/opt/stack/data/venv/lib/python3.10/site-packages/cinderclient/client.py", line 197, in request
Nov 29 03:08:38 user nova-compute[75234]: ERROR nova.virt.libvirt.driver     raise exceptions.from_response(resp, body)
Nov 29 03:08:38 user nova-compute[75234]: ERROR nova.virt.libvirt.driver cinderclient.exceptions.NotFound: Snapshot 59222e85-b132-4ba8-886e-55fbed5b38d7 could not be found. (HTTP 404) (Request-ID: req-d0579b8d-304d-4e87-b687-2dbf50fdaf3a)
Nov 29 03:08:38 user nova-compute[75234]: ERROR nova.virt.libvirt.driver 
Nov 29 03:08:38 user nova-compute[75234]: ERROR nova.virt.libvirt.driver During handling of the above exception, another exception occurred:
Nov 29 03:08:38 user nova-compute[75234]: ERROR nova.virt.libvirt.driver 
Nov 29 03:08:38 user nova-compute[75234]: ERROR nova.virt.libvirt.driver Traceback (most recent call last):
Nov 29 03:08:38 user nova-compute[75234]: ERROR nova.virt.libvirt.driver   File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 3416, in _volume_snapshot_update_status
Nov 29 03:08:38 user nova-compute[75234]: ERROR nova.virt.libvirt.driver     self._volume_api.update_snapshot_status(context,
Nov 29 03:08:38 user nova-compute[75234]: ERROR nova.virt.libvirt.driver   File "/opt/stack/nova/nova/volume/cinder.py", line 397, in wrapper
Nov 29 03:08:38 user nova-compute[75234]: ERROR nova.virt.libvirt.driver     res = method(self, ctx, *args, **kwargs)
Nov 29 03:08:38 user nova-compute[75234]: ERROR nova.virt.libvirt.driver   File "/opt/stack/nova/nova/volume/cinder.py", line 468, in wrapper
Nov 29 03:08:38 user nova-compute[75234]: ERROR nova.virt.libvirt.driver     _reraise(exception.SnapshotNotFound(snapshot_id=snapshot_id))
Nov 29 03:08:38 user nova-compute[75234]: ERROR nova.virt.libvirt.driver   File "/opt/stack/nova/nova/volume/cinder.py", line 488, in _reraise
Nov 29 03:08:38 user nova-compute[75234]: ERROR nova.virt.libvirt.driver     raise desired_exc.with_traceback(sys.exc_info()[2])
Nov 29 03:08:38 user nova-compute[75234]: ERROR nova.virt.libvirt.driver   File "/opt/stack/nova/nova/volume/cinder.py", line 466, in wrapper
Nov 29 03:08:38 user nova-compute[75234]: ERROR nova.virt.libvirt.driver     res = method(self, ctx, snapshot_id, *args, **kwargs)
Nov 29 03:08:38 user nova-compute[75234]: ERROR nova.virt.libvirt.driver   File "/opt/stack/nova/nova/volume/cinder.py", line 761, in update_snapshot_status
Nov 29 03:08:38 user nova-compute[75234]: ERROR nova.virt.libvirt.driver     vs.update_snapshot_status(
Nov 29 03:08:38 user nova-compute[75234]: ERROR nova.virt.libvirt.driver   File "/opt/stack/data/venv/lib/python3.10/site-packages/cinderclient/v3/volume_snapshots.py", line 225, in update_snapshot_status
Nov 29 03:08:38 user nova-compute[75234]: ERROR nova.virt.libvirt.driver     return self._action('os-update_snapshot_status',
Nov 29 03:08:38 user nova-compute[75234]: ERROR nova.virt.libvirt.driver   File "/opt/stack/data/venv/lib/python3.10/site-packages/cinderclient/v3/volume_snapshots.py", line 221, in _action
Nov 29 03:08:38 user nova-compute[75234]: ERROR nova.virt.libvirt.driver     resp, body = self.api.client.post(url, body=body)
Nov 29 03:08:38 user nova-compute[75234]: ERROR nova.virt.libvirt.driver   File "/opt/stack/data/venv/lib/python3.10/site-packages/cinderclient/client.py", line 223, in post
Nov 29 03:08:38 user nova-compute[75234]: ERROR nova.virt.libvirt.driver     return self._cs_request(url, 'POST', **kwargs)
Nov 29 03:08:38 user nova-compute[75234]: ERROR nova.virt.libvirt.driver   File "/opt/stack/data/venv/lib/python3.10/site-packages/cinderclient/client.py", line 211, in _cs_request
Nov 29 03:08:38 user nova-compute[75234]: ERROR nova.virt.libvirt.driver     return self.request(url, method, **kwargs)
Nov 29 03:08:38 user nova-compute[75234]: ERROR nova.virt.libvirt.driver   File "/opt/stack/data/venv/lib/python3.10/site-packages/cinderclient/client.py", line 197, in request
Nov 29 03:08:38 user nova-compute[75234]: ERROR nova.virt.libvirt.driver     raise exceptions.from_response(resp, body)
Nov 29 03:08:38 user nova-compute[75234]: ERROR nova.virt.libvirt.driver nova.exception.SnapshotNotFound: Snapshot 59222e85-b132-4ba8-886e-55fbed5b38d7 could not be found.
Nov 29 03:08:38 user nova-compute[75234]: ERROR nova.virt.libvirt.driver 
Nov 29 03:08:38 user nova-compute[75234]: ERROR nova.virt.libvirt.driver [None req-65d16d35-2605-4555-b23f-f7b512e92b81 tempest-VolumesAssistedSnapshotsTest-1691523836 tempest-VolumesAssistedSnapshotsTest-1691523836-project] Failed to send updated snapshot status to volume service.: nova.exception.SnapshotNotFound: Snapshot None could not be found.
Nov 29 03:08:38 user nova-compute[75234]: ERROR nova.virt.libvirt.driver Traceback (most recent call last):
Nov 29 03:08:38 user nova-compute[75234]: ERROR nova.virt.libvirt.driver   File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 3850, in volume_snapshot_delete
Nov 29 03:08:38 user nova-compute[75234]: ERROR nova.virt.libvirt.driver     self._volume_snapshot_delete(context, instance, volume_id,
Nov 29 03:08:38 user nova-compute[75234]: ERROR nova.virt.libvirt.driver   File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 3680, in _volume_snapshot_delete
Nov 29 03:08:38 user nova-compute[75234]: ERROR nova.virt.libvirt.driver     if delete_info['type'] != 'qcow2':
Nov 29 03:08:38 user nova-compute[75234]: ERROR nova.virt.libvirt.driver KeyError: 'type'
Nov 29 03:08:38 user nova-compute[75234]: ERROR nova.virt.libvirt.driver 
Nov 29 03:08:38 user nova-compute[75234]: ERROR nova.virt.libvirt.driver During handling of the above exception, another exception occurred:
Nov 29 03:08:38 user nova-compute[75234]: ERROR nova.virt.libvirt.driver 
Nov 29 03:08:38 user nova-compute[75234]: ERROR nova.virt.libvirt.driver Traceback (most recent call last):
Nov 29 03:08:38 user nova-compute[75234]: ERROR nova.virt.libvirt.driver   File "/opt/stack/nova/nova/volume/cinder.py", line 466, in wrapper
Nov 29 03:08:38 user nova-compute[75234]: ERROR nova.virt.libvirt.driver     res = method(self, ctx, snapshot_id, *args, **kwargs)
Nov 29 03:08:38 user nova-compute[75234]: ERROR nova.virt.libvirt.driver   File "/opt/stack/nova/nova/volume/cinder.py", line 761, in update_snapshot_status
Nov 29 03:08:38 user nova-compute[75234]: ERROR nova.virt.libvirt.driver     vs.update_snapshot_status(
Nov 29 03:08:38 user nova-compute[75234]: ERROR nova.virt.libvirt.driver   File "/opt/stack/data/venv/lib/python3.10/site-packages/cinderclient/v3/volume_snapshots.py", line 225, in update_snapshot_status
Nov 29 03:08:38 user nova-compute[75234]: ERROR nova.virt.libvirt.driver     return self._action('os-update_snapshot_status',
Nov 29 03:08:38 user nova-compute[75234]: ERROR nova.virt.libvirt.driver   File "/opt/stack/data/venv/lib/python3.10/site-packages/cinderclient/v3/volume_snapshots.py", line 221, in _action
Nov 29 03:08:38 user nova-compute[75234]: ERROR nova.virt.libvirt.driver     resp, body = self.api.client.post(url, body=body)
Nov 29 03:08:38 user nova-compute[75234]: ERROR nova.virt.libvirt.driver   File "/opt/stack/data/venv/lib/python3.10/site-packages/cinderclient/client.py", line 223, in post
Nov 29 03:08:38 user nova-compute[75234]: ERROR nova.virt.libvirt.driver     return self._cs_request(url, 'POST', **kwargs)
Nov 29 03:08:38 user nova-compute[75234]: ERROR nova.virt.libvirt.driver   File "/opt/stack/data/venv/lib/python3.10/site-packages/cinderclient/client.py", line 211, in _cs_request
Nov 29 03:08:38 user nova-compute[75234]: ERROR nova.virt.libvirt.driver     return self.request(url, method, **kwargs)
Nov 29 03:08:38 user nova-compute[75234]: ERROR nova.virt.libvirt.driver   File "/opt/stack/data/venv/lib/python3.10/site-packages/cinderclient/client.py", line 197, in request
Nov 29 03:08:38 user nova-compute[75234]: ERROR nova.virt.libvirt.driver     raise exceptions.from_response(resp, body)
Nov 29 03:08:38 user nova-compute[75234]: ERROR nova.virt.libvirt.driver cinderclient.exceptions.NotFound: Snapshot None could not be found. (HTTP 404) (Request-ID: req-a7a1631d-9629-455e-9cec-96d077acb8cc)
Nov 29 03:08:38 user nova-compute[75234]: ERROR nova.virt.libvirt.driver 
Nov 29 03:08:38 user nova-compute[75234]: ERROR nova.virt.libvirt.driver During handling of the above exception, another exception occurred:
Nov 29 03:08:38 user nova-compute[75234]: ERROR nova.virt.libvirt.driver 
Nov 29 03:08:38 user nova-compute[75234]: ERROR nova.virt.libvirt.driver Traceback (most recent call last):
Nov 29 03:08:38 user nova-compute[75234]: ERROR nova.virt.libvirt.driver   File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 3416, in _volume_snapshot_update_status
Nov 29 03:08:38 user nova-compute[75234]: ERROR nova.virt.libvirt.driver     self._volume_api.update_snapshot_status(context,
Nov 29 03:08:38 user nova-compute[75234]: ERROR nova.virt.libvirt.driver   File "/opt/stack/nova/nova/volume/cinder.py", line 397, in wrapper
Nov 29 03:08:38 user nova-compute[75234]: ERROR nova.virt.libvirt.driver     res = method(self, ctx, *args, **kwargs)
Nov 29 03:08:38 user nova-compute[75234]: ERROR nova.virt.libvirt.driver   File "/opt/stack/nova/nova/volume/cinder.py", line 468, in wrapper
Nov 29 03:08:38 user nova-compute[75234]: ERROR nova.virt.libvirt.driver     _reraise(exception.SnapshotNotFound(snapshot_id=snapshot_id))
Nov 29 03:08:38 user nova-compute[75234]: ERROR nova.virt.libvirt.driver   File "/opt/stack/nova/nova/volume/cinder.py", line 488, in _reraise
Nov 29 03:08:38 user nova-compute[75234]: ERROR nova.virt.libvirt.driver     raise desired_exc.with_traceback(sys.exc_info()[2])
Nov 29 03:08:38 user nova-compute[75234]: ERROR nova.virt.libvirt.driver   File "/opt/stack/nova/nova/volume/cinder.py", line 466, in wrapper
Nov 29 03:08:38 user nova-compute[75234]: ERROR nova.virt.libvirt.driver     res = method(self, ctx, snapshot_id, *args, **kwargs)
Nov 29 03:08:38 user nova-compute[75234]: ERROR nova.virt.libvirt.driver   File "/opt/stack/nova/nova/volume/cinder.py", line 761, in update_snapshot_status
Nov 29 03:08:38 user nova-compute[75234]: ERROR nova.virt.libvirt.driver     vs.update_snapshot_status(
Nov 29 03:08:38 user nova-compute[75234]: ERROR nova.virt.libvirt.driver   File "/opt/stack/data/venv/lib/python3.10/site-packages/cinderclient/v3/volume_snapshots.py", line 225, in update_snapshot_status
Nov 29 03:08:38 user nova-compute[75234]: ERROR nova.virt.libvirt.driver     return self._action('os-update_snapshot_status',
Nov 29 03:08:38 user nova-compute[75234]: ERROR nova.virt.libvirt.driver   File "/opt/stack/data/venv/lib/python3.10/site-packages/cinderclient/v3/volume_snapshots.py", line 221, in _action
Nov 29 03:08:38 user nova-compute[75234]: ERROR nova.virt.libvirt.driver     resp, body = self.api.client.post(url, body=body)
Nov 29 03:08:38 user nova-compute[75234]: ERROR nova.virt.libvirt.driver   File "/opt/stack/data/venv/lib/python3.10/site-packages/cinderclient/client.py", line 223, in post
Nov 29 03:08:38 user nova-compute[75234]: ERROR nova.virt.libvirt.driver     return self._cs_request(url, 'POST', **kwargs)
Nov 29 03:08:38 user nova-compute[75234]: ERROR nova.virt.libvirt.driver   File "/opt/stack/data/venv/lib/python3.10/site-packages/cinderclient/client.py", line 211, in _cs_request
Nov 29 03:08:38 user nova-compute[75234]: ERROR nova.virt.libvirt.driver     return self.request(url, method, **kwargs)
Nov 29 03:08:38 user nova-compute[75234]: ERROR nova.virt.libvirt.driver   File "/opt/stack/data/venv/lib/python3.10/site-packages/cinderclient/client.py", line 197, in request
Nov 29 03:08:38 user nova-compute[75234]: ERROR nova.virt.libvirt.driver     raise exceptions.from_response(resp, body)
Nov 29 03:08:38 user nova-compute[75234]: ERROR nova.virt.libvirt.driver nova.exception.SnapshotNotFound: Snapshot None could not be found.
Nov 29 03:08:38 user nova-compute[75234]: ERROR nova.virt.libvirt.driver 
Nov 29 03:08:38 user nova-compute[75234]: ERROR oslo_messaging.rpc.server [None req-83d62bf6-1454-424f-a05b-50e404419735 tempest-VolumesAssistedSnapshotsTest-1691523836 tempest-VolumesAssistedSnapshotsTest-1691523836-project] Exception during message handling: libvirt.libvirtError: missing existing file for disk vdb: /dev/new_file: No such file or directory
Nov 29 03:08:38 user nova-compute[75234]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
Nov 29 03:08:38 user nova-compute[75234]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
Nov 29 03:08:38 user nova-compute[75234]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
Nov 29 03:08:38 user nova-compute[75234]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
Nov 29 03:08:38 user nova-compute[75234]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
Nov 29 03:08:38 user nova-compute[75234]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
Nov 29 03:08:38 user nova-compute[75234]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
Nov 29 03:08:38 user nova-compute[75234]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 244, in inner
Nov 29 03:08:38 user nova-compute[75234]: ERROR oslo_messaging.rpc.server     return func(*args, **kwargs)
Nov 29 03:08:38 user nova-compute[75234]: ERROR oslo_messaging.rpc.server   File "/opt/stack/nova/nova/exception_wrapper.py", line 65, in wrapped
Nov 29 03:08:38 user nova-compute[75234]: ERROR oslo_messaging.rpc.server     with excutils.save_and_reraise_exception():
Nov 29 03:08:38 user nova-compute[75234]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__
Nov 29 03:08:38 user nova-compute[75234]: ERROR oslo_messaging.rpc.server     self.force_reraise()
Nov 29 03:08:38 user nova-compute[75234]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise
Nov 29 03:08:38 user nova-compute[75234]: ERROR oslo_messaging.rpc.server     raise self.value
Nov 29 03:08:38 user nova-compute[75234]: ERROR oslo_messaging.rpc.server   File "/opt/stack/nova/nova/exception_wrapper.py", line 63, in wrapped
Nov 29 03:08:38 user nova-compute[75234]: ERROR oslo_messaging.rpc.server     return f(self, context, *args, **kw)
Nov 29 03:08:38 user nova-compute[75234]: ERROR oslo_messaging.rpc.server   File "/opt/stack/nova/nova/compute/manager.py", line 4505, in volume_snapshot_create
Nov 29 03:08:38 user nova-compute[75234]: ERROR oslo_messaging.rpc.server     self.driver.volume_snapshot_create(context, instance, volume_id,
Nov 29 03:08:38 user nova-compute[75234]: ERROR oslo_messaging.rpc.server   File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 3597, in volume_snapshot_create
Nov 29 03:08:38 user nova-compute[75234]: ERROR oslo_messaging.rpc.server     with excutils.save_and_reraise_exception():
Nov 29 03:08:38 user nova-compute[75234]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__
Nov 29 03:08:38 user nova-compute[75234]: ERROR oslo_messaging.rpc.server     self.force_reraise()
Nov 29 03:08:38 user nova-compute[75234]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise
Nov 29 03:08:38 user nova-compute[75234]: ERROR oslo_messaging.rpc.server     raise self.value
Nov 29 03:08:38 user nova-compute[75234]: ERROR oslo_messaging.rpc.server   File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 3594, in volume_snapshot_create
Nov 29 03:08:38 user nova-compute[75234]: ERROR oslo_messaging.rpc.server     self._volume_snapshot_create(context, instance, guest,
Nov 29 03:08:38 user nova-compute[75234]: ERROR oslo_messaging.rpc.server   File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 3544, in _volume_snapshot_create
Nov 29 03:08:38 user nova-compute[75234]: ERROR oslo_messaging.rpc.server     guest.snapshot(snapshot, no_metadata=True, disk_only=True,
Nov 29 03:08:38 user nova-compute[75234]: ERROR oslo_messaging.rpc.server   File "/opt/stack/nova/nova/virt/libvirt/guest.py", line 569, in snapshot
Nov 29 03:08:38 user nova-compute[75234]: ERROR oslo_messaging.rpc.server     self._domain.snapshotCreateXML(device_xml, flags=flags)
Nov 29 03:08:38 user nova-compute[75234]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/tpool.py", line 193, in doit
Nov 29 03:08:38 user nova-compute[75234]: ERROR oslo_messaging.rpc.server     result = proxy_call(self._autowrap, f, *args, **kwargs)
Nov 29 03:08:38 user nova-compute[75234]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/tpool.py", line 151, in proxy_call
Nov 29 03:08:38 user nova-compute[75234]: ERROR oslo_messaging.rpc.server     rv = execute(f, *args, **kwargs)
Nov 29 03:08:38 user nova-compute[75234]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/tpool.py", line 132, in execute
Nov 29 03:08:38 user nova-compute[75234]: ERROR oslo_messaging.rpc.server     six.reraise(c, e, tb)
Nov 29 03:08:38 user nova-compute[75234]: ERROR oslo_messaging.rpc.server   File "/usr/lib/python3/dist-packages/six.py", line 719, in reraise
Nov 29 03:08:38 user nova-compute[75234]: ERROR oslo_messaging.rpc.server     raise value
Nov 29 03:08:38 user nova-compute[75234]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/tpool.py", line 86, in tworker
Nov 29 03:08:38 user nova-compute[75234]: ERROR oslo_messaging.rpc.server     rv = meth(*args, **kwargs)
Nov 29 03:08:38 user nova-compute[75234]: ERROR oslo_messaging.rpc.server   File "/usr/lib/python3/dist-packages/libvirt.py", line 3070, in snapshotCreateXML
Nov 29 03:08:38 user nova-compute[75234]: ERROR oslo_messaging.rpc.server     raise libvirtError('virDomainSnapshotCreateXML() failed')
Nov 29 03:08:38 user nova-compute[75234]: ERROR oslo_messaging.rpc.server libvirt.libvirtError: missing existing file for disk vdb: /dev/new_file: No such file or directory
Nov 29 03:08:38 user nova-compute[75234]: ERROR oslo_messaging.rpc.server 
Nov 29 03:08:38 user nova-compute[75234]: ERROR oslo_messaging.rpc.server [None req-65d16d35-2605-4555-b23f-f7b512e92b81 tempest-VolumesAssistedSnapshotsTest-1691523836 tempest-VolumesAssistedSnapshotsTest-1691523836-project] Exception during message handling: KeyError: 'type'
Nov 29 03:08:38 user nova-compute[75234]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
Nov 29 03:08:38 user nova-compute[75234]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
Nov 29 03:08:38 user nova-compute[75234]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
Nov 29 03:08:38 user nova-compute[75234]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
Nov 29 03:08:38 user nova-compute[75234]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
Nov 29 03:08:38 user nova-compute[75234]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
Nov 29 03:08:38 user nova-compute[75234]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
Nov 29 03:08:38 user nova-compute[75234]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 244, in inner
Nov 29 03:08:38 user nova-compute[75234]: ERROR oslo_messaging.rpc.server     return func(*args, **kwargs)
Nov 29 03:08:38 user nova-compute[75234]: ERROR oslo_messaging.rpc.server   File "/opt/stack/nova/nova/exception_wrapper.py", line 65, in wrapped
Nov 29 03:08:38 user nova-compute[75234]: ERROR oslo_messaging.rpc.server     with excutils.save_and_reraise_exception():
Nov 29 03:08:38 user nova-compute[75234]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__
Nov 29 03:08:38 user nova-compute[75234]: ERROR oslo_messaging.rpc.server     self.force_reraise()
Nov 29 03:08:38 user nova-compute[75234]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise
Nov 29 03:08:38 user nova-compute[75234]: ERROR oslo_messaging.rpc.server     raise self.value
Nov 29 03:08:38 user nova-compute[75234]: ERROR oslo_messaging.rpc.server   File "/opt/stack/nova/nova/exception_wrapper.py", line 63, in wrapped
Nov 29 03:08:38 user nova-compute[75234]: ERROR oslo_messaging.rpc.server     return f(self, context, *args, **kw)
Nov 29 03:08:38 user nova-compute[75234]: ERROR oslo_messaging.rpc.server   File "/opt/stack/nova/nova/compute/manager.py", line 4517, in volume_snapshot_delete
Nov 29 03:08:38 user nova-compute[75234]: ERROR oslo_messaging.rpc.server     self.driver.volume_snapshot_delete(context, instance, volume_id,
Nov 29 03:08:38 user nova-compute[75234]: ERROR oslo_messaging.rpc.server   File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 3853, in volume_snapshot_delete
Nov 29 03:08:38 user nova-compute[75234]: ERROR oslo_messaging.rpc.server     with excutils.save_and_reraise_exception():
Nov 29 03:08:38 user nova-compute[75234]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__
Nov 29 03:08:38 user nova-compute[75234]: ERROR oslo_messaging.rpc.server     self.force_reraise()
Nov 29 03:08:38 user nova-compute[75234]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise
Nov 29 03:08:38 user nova-compute[75234]: ERROR oslo_messaging.rpc.server     raise self.value
Nov 29 03:08:38 user nova-compute[75234]: ERROR oslo_messaging.rpc.server   File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 3850, in volume_snapshot_delete
Nov 29 03:08:38 user nova-compute[75234]: ERROR oslo_messaging.rpc.server     self._volume_snapshot_delete(context, instance, volume_id,
Nov 29 03:08:38 user nova-compute[75234]: ERROR oslo_messaging.rpc.server   File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 3680, in _volume_snapshot_delete
Nov 29 03:08:38 user nova-compute[75234]: ERROR oslo_messaging.rpc.server     if delete_info['type'] != 'qcow2':
Nov 29 03:08:38 user nova-compute[75234]: ERROR oslo_messaging.rpc.server KeyError: 'type'
Nov 29 03:08:38 user nova-compute[75234]: ERROR oslo_messaging.rpc.server 
Nov 29 03:08:38 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-b485989f-0c2c-4b36-8f02-1aa1f0b630cc tempest-VolumesAssistedSnapshotsTest-1119739890 tempest-VolumesAssistedSnapshotsTest-1119739890-project-admin] Acquiring lock "c9663ab3-3374-400e-ab0a-769731b78cca" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:08:38 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-b485989f-0c2c-4b36-8f02-1aa1f0b630cc tempest-VolumesAssistedSnapshotsTest-1119739890 tempest-VolumesAssistedSnapshotsTest-1119739890-project-admin] Lock "c9663ab3-3374-400e-ab0a-769731b78cca" acquired by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:08:38 user nova-compute[75234]: INFO nova.compute.manager [None req-b485989f-0c2c-4b36-8f02-1aa1f0b630cc tempest-VolumesAssistedSnapshotsTest-1119739890 tempest-VolumesAssistedSnapshotsTest-1119739890-project-admin] [instance: c9663ab3-3374-400e-ab0a-769731b78cca] Detaching volume a4cf29d0-2ce2-423f-a20c-a7a19a452088
Nov 29 03:08:38 user nova-compute[75234]: INFO nova.virt.block_device [None req-b485989f-0c2c-4b36-8f02-1aa1f0b630cc tempest-VolumesAssistedSnapshotsTest-1119739890 tempest-VolumesAssistedSnapshotsTest-1119739890-project-admin] [instance: c9663ab3-3374-400e-ab0a-769731b78cca] Attempting to driver detach volume a4cf29d0-2ce2-423f-a20c-a7a19a452088 from mountpoint /dev/vdb
Nov 29 03:08:38 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-b485989f-0c2c-4b36-8f02-1aa1f0b630cc tempest-VolumesAssistedSnapshotsTest-1119739890 tempest-VolumesAssistedSnapshotsTest-1119739890-project-admin] Attempting to detach device vdb from instance c9663ab3-3374-400e-ab0a-769731b78cca from the persistent domain config. {{(pid=75234) _detach_from_persistent /opt/stack/nova/nova/virt/libvirt/driver.py:2487}}
Nov 29 03:08:38 user nova-compute[75234]: DEBUG nova.virt.libvirt.guest [None req-b485989f-0c2c-4b36-8f02-1aa1f0b630cc tempest-VolumesAssistedSnapshotsTest-1119739890 tempest-VolumesAssistedSnapshotsTest-1119739890-project-admin] detach device xml: <disk type="block" device="disk">
Nov 29 03:08:38 user nova-compute[75234]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Nov 29 03:08:38 user nova-compute[75234]:   <source dev="/dev/sdj"/>
Nov 29 03:08:38 user nova-compute[75234]:   <target dev="vdb" bus="virtio"/>
Nov 29 03:08:38 user nova-compute[75234]:   <serial>a4cf29d0-2ce2-423f-a20c-a7a19a452088</serial>
Nov 29 03:08:38 user nova-compute[75234]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x07" function="0x0"/>
Nov 29 03:08:38 user nova-compute[75234]: </disk>
Nov 29 03:08:38 user nova-compute[75234]:  {{(pid=75234) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Nov 29 03:08:38 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-b485989f-0c2c-4b36-8f02-1aa1f0b630cc tempest-VolumesAssistedSnapshotsTest-1119739890 tempest-VolumesAssistedSnapshotsTest-1119739890-project-admin] Successfully detached device vdb from instance c9663ab3-3374-400e-ab0a-769731b78cca from the persistent domain config.
Nov 29 03:08:38 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-b485989f-0c2c-4b36-8f02-1aa1f0b630cc tempest-VolumesAssistedSnapshotsTest-1119739890 tempest-VolumesAssistedSnapshotsTest-1119739890-project-admin] (1/8): Attempting to detach device vdb with device alias virtio-disk1 from instance c9663ab3-3374-400e-ab0a-769731b78cca from the live domain config. {{(pid=75234) _detach_from_live_with_retry /opt/stack/nova/nova/virt/libvirt/driver.py:2523}}
Nov 29 03:08:38 user nova-compute[75234]: DEBUG nova.virt.libvirt.guest [None req-b485989f-0c2c-4b36-8f02-1aa1f0b630cc tempest-VolumesAssistedSnapshotsTest-1119739890 tempest-VolumesAssistedSnapshotsTest-1119739890-project-admin] detach device xml: <disk type="block" device="disk">
Nov 29 03:08:38 user nova-compute[75234]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Nov 29 03:08:38 user nova-compute[75234]:   <source dev="/dev/sdj"/>
Nov 29 03:08:38 user nova-compute[75234]:   <target dev="vdb" bus="virtio"/>
Nov 29 03:08:38 user nova-compute[75234]:   <serial>a4cf29d0-2ce2-423f-a20c-a7a19a452088</serial>
Nov 29 03:08:38 user nova-compute[75234]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x07" function="0x0"/>
Nov 29 03:08:38 user nova-compute[75234]: </disk>
Nov 29 03:08:38 user nova-compute[75234]:  {{(pid=75234) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Nov 29 03:08:39 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-b485989f-0c2c-4b36-8f02-1aa1f0b630cc tempest-VolumesAssistedSnapshotsTest-1119739890 tempest-VolumesAssistedSnapshotsTest-1119739890-project-admin] Start waiting for the detach event from libvirt for device vdb with device alias virtio-disk1 for instance c9663ab3-3374-400e-ab0a-769731b78cca {{(pid=75234) _detach_from_live_and_wait_for_event /opt/stack/nova/nova/virt/libvirt/driver.py:2599}}
Nov 29 03:08:41 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:08:46 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:08:46 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:08:49 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:08:49 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:08:49 user nova-compute[75234]: DEBUG nova.compute.manager [req-e8e1cf94-d319-4bf0-9692-334186430cc3 req-9dea5fc5-5521-4e42-b71b-fa728a987a2f service nova] [instance: cb569e8d-e4a5-4076-bb8b-deeefad033e2] Received event network-changed-2b12ec96-26fa-4974-8a7a-22e85ec20544 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:08:49 user nova-compute[75234]: DEBUG nova.compute.manager [req-e8e1cf94-d319-4bf0-9692-334186430cc3 req-9dea5fc5-5521-4e42-b71b-fa728a987a2f service nova] [instance: cb569e8d-e4a5-4076-bb8b-deeefad033e2] Refreshing instance network info cache due to event network-changed-2b12ec96-26fa-4974-8a7a-22e85ec20544. {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11117}}
Nov 29 03:08:49 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-e8e1cf94-d319-4bf0-9692-334186430cc3 req-9dea5fc5-5521-4e42-b71b-fa728a987a2f service nova] Acquiring lock "refresh_cache-cb569e8d-e4a5-4076-bb8b-deeefad033e2" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:08:49 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-e8e1cf94-d319-4bf0-9692-334186430cc3 req-9dea5fc5-5521-4e42-b71b-fa728a987a2f service nova] Acquired lock "refresh_cache-cb569e8d-e4a5-4076-bb8b-deeefad033e2" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:08:49 user nova-compute[75234]: DEBUG nova.network.neutron [req-e8e1cf94-d319-4bf0-9692-334186430cc3 req-9dea5fc5-5521-4e42-b71b-fa728a987a2f service nova] [instance: cb569e8d-e4a5-4076-bb8b-deeefad033e2] Refreshing network info cache for port 2b12ec96-26fa-4974-8a7a-22e85ec20544 {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2006}}
Nov 29 03:08:50 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:08:50 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:08:50 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=75234) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10541}}
Nov 29 03:08:50 user nova-compute[75234]: DEBUG nova.network.neutron [req-e8e1cf94-d319-4bf0-9692-334186430cc3 req-9dea5fc5-5521-4e42-b71b-fa728a987a2f service nova] [instance: cb569e8d-e4a5-4076-bb8b-deeefad033e2] Updated VIF entry in instance network info cache for port 2b12ec96-26fa-4974-8a7a-22e85ec20544. {{(pid=75234) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3481}}
Nov 29 03:08:50 user nova-compute[75234]: DEBUG nova.network.neutron [req-e8e1cf94-d319-4bf0-9692-334186430cc3 req-9dea5fc5-5521-4e42-b71b-fa728a987a2f service nova] [instance: cb569e8d-e4a5-4076-bb8b-deeefad033e2] Updating instance_info_cache with network_info: [{"id": "2b12ec96-26fa-4974-8a7a-22e85ec20544", "address": "fa:16:3e:fd:00:a6", "network": {"id": "1869abec-0b45-446a-9dc8-5871de9dbfc5", "bridge": "br-int", "label": "tempest-TaggedBootDevicesTest-1812676341-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.181", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "ee1117bd3511400f9f45cc8f9bb9247d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2b12ec96-26", "ovs_interfaceid": "2b12ec96-26fa-4974-8a7a-22e85ec20544", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "0bd5d735-f566-4a6d-b49f-b24fd403a87a", "address": "fa:16:3e:aa:c2:ae", "network": {"id": "005139f3-3686-493a-aa8f-0cdc455befa0", "bridge": "br-int", "label": "tempest-device-tagging-net1-1510949192", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.85", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "ee1117bd3511400f9f45cc8f9bb9247d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap0bd5d735-f5", "ovs_interfaceid": "0bd5d735-f566-4a6d-b49f-b24fd403a87a", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}}, {"id": "27e72461-f85b-42c9-8824-0d760b82a725", "address": "fa:16:3e:8e:29:fd", "network": {"id": "005139f3-3686-493a-aa8f-0cdc455befa0", "bridge": "br-int", "label": "tempest-device-tagging-net1-1510949192", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.127", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "ee1117bd3511400f9f45cc8f9bb9247d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap27e72461-f8", "ovs_interfaceid": "27e72461-f85b-42c9-8824-0d760b82a725", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}}, {"id": "026f606a-534c-4cdd-992a-9f33268b374f", "address": "fa:16:3e:3e:3b:29", "network": {"id": "005139f3-3686-493a-aa8f-0cdc455befa0", "bridge": "br-int", "label": "tempest-device-tagging-net1-1510949192", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.41", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "ee1117bd3511400f9f45cc8f9bb9247d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap026f606a-53", "ovs_interfaceid": "026f606a-534c-4cdd-992a-9f33268b374f", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "67446920-3741-4542-84c6-493388395f26", "address": "fa:16:3e:f6:7a:76", "network": {"id": "005139f3-3686-493a-aa8f-0cdc455befa0", "bridge": "br-int", "label": "tempest-device-tagging-net1-1510949192", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.230", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "ee1117bd3511400f9f45cc8f9bb9247d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap67446920-37", "ovs_interfaceid": "67446920-3741-4542-84c6-493388395f26", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "d7e92753-492b-4fc0-8048-f4792cd3b4b6", "address": "fa:16:3e:68:b2:33", "network": {"id": "cc6c7cdf-8c81-4906-9fd8-de815fea3374", "bridge": "br-int", "label": "tempest-device-tagging-net2-1248280764", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.100", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "ee1117bd3511400f9f45cc8f9bb9247d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapd7e92753-49", "ovs_interfaceid": "d7e92753-492b-4fc0-8048-f4792cd3b4b6", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "06db3012-0042-457a-a3c8-14e3a8d2e89e", "address": "fa:16:3e:27:54:30", "network": {"id": "cc6c7cdf-8c81-4906-9fd8-de815fea3374", "bridge": "br-int", "label": "tempest-device-tagging-net2-1248280764", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.200", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "ee1117bd3511400f9f45cc8f9bb9247d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap06db3012-00", "ovs_interfaceid": "06db3012-0042-457a-a3c8-14e3a8d2e89e", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:08:50 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-e8e1cf94-d319-4bf0-9692-334186430cc3 req-9dea5fc5-5521-4e42-b71b-fa728a987a2f service nova] Releasing lock "refresh_cache-cb569e8d-e4a5-4076-bb8b-deeefad033e2" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:08:51 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:08:51 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:08:51 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Starting heal instance info cache {{(pid=75234) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9922}}
Nov 29 03:08:51 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Acquiring lock "refresh_cache-21e9c6cd-c39f-41c0-8768-5dd64589c3f3" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:08:51 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Acquired lock "refresh_cache-21e9c6cd-c39f-41c0-8768-5dd64589c3f3" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:08:51 user nova-compute[75234]: DEBUG nova.network.neutron [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] [instance: 21e9c6cd-c39f-41c0-8768-5dd64589c3f3] Forcefully refreshing network info cache for instance {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2003}}
Nov 29 03:08:51 user nova-compute[75234]: DEBUG nova.network.neutron [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] [instance: 21e9c6cd-c39f-41c0-8768-5dd64589c3f3] Updating instance_info_cache with network_info: [{"id": "87f1e736-6665-4945-bd19-0cf93bff905e", "address": "fa:16:3e:40:14:df", "network": {"id": "1f946e69-dbec-429c-83f8-a186bad4f2cc", "bridge": "br-int", "label": "tempest-ServerActionsTestOtherA-486369423-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.64", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "c1cb3d1f12f242a785a688d59dbe3cd6", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap87f1e736-66", "ovs_interfaceid": "87f1e736-6665-4945-bd19-0cf93bff905e", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:08:51 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Releasing lock "refresh_cache-21e9c6cd-c39f-41c0-8768-5dd64589c3f3" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:08:51 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] [instance: 21e9c6cd-c39f-41c0-8768-5dd64589c3f3] Updated the network info_cache for instance {{(pid=75234) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9993}}
Nov 29 03:08:51 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:08:51 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:08:52 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-88bf7696-da01-415d-80cd-dd7361a45c4c tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] Acquiring lock "cb569e8d-e4a5-4076-bb8b-deeefad033e2" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:08:52 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-88bf7696-da01-415d-80cd-dd7361a45c4c tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] Lock "cb569e8d-e4a5-4076-bb8b-deeefad033e2" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:08:52 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-88bf7696-da01-415d-80cd-dd7361a45c4c tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] Acquiring lock "cb569e8d-e4a5-4076-bb8b-deeefad033e2-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:08:52 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-88bf7696-da01-415d-80cd-dd7361a45c4c tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] Lock "cb569e8d-e4a5-4076-bb8b-deeefad033e2-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:08:52 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-88bf7696-da01-415d-80cd-dd7361a45c4c tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] Lock "cb569e8d-e4a5-4076-bb8b-deeefad033e2-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:08:52 user nova-compute[75234]: INFO nova.compute.manager [None req-88bf7696-da01-415d-80cd-dd7361a45c4c tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] [instance: cb569e8d-e4a5-4076-bb8b-deeefad033e2] Terminating instance
Nov 29 03:08:52 user nova-compute[75234]: DEBUG nova.compute.manager [None req-88bf7696-da01-415d-80cd-dd7361a45c4c tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] [instance: cb569e8d-e4a5-4076-bb8b-deeefad033e2] Start destroying the instance on the hypervisor. {{(pid=75234) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3140}}
Nov 29 03:08:52 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:08:52 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:08:52 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:08:52 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:08:52 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:08:52 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:08:52 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:08:52 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:08:52 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:08:52 user nova-compute[75234]: DEBUG nova.compute.manager [req-0282a088-8381-4e7b-8907-deb42041676d req-a1ccaa34-9788-44b7-a2d0-74a801484c26 service nova] [instance: cb569e8d-e4a5-4076-bb8b-deeefad033e2] Received event network-vif-unplugged-0bd5d735-f566-4a6d-b49f-b24fd403a87a {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:08:52 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-0282a088-8381-4e7b-8907-deb42041676d req-a1ccaa34-9788-44b7-a2d0-74a801484c26 service nova] Acquiring lock "cb569e8d-e4a5-4076-bb8b-deeefad033e2-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:08:52 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-0282a088-8381-4e7b-8907-deb42041676d req-a1ccaa34-9788-44b7-a2d0-74a801484c26 service nova] Lock "cb569e8d-e4a5-4076-bb8b-deeefad033e2-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:08:52 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-0282a088-8381-4e7b-8907-deb42041676d req-a1ccaa34-9788-44b7-a2d0-74a801484c26 service nova] Lock "cb569e8d-e4a5-4076-bb8b-deeefad033e2-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:08:52 user nova-compute[75234]: DEBUG nova.compute.manager [req-0282a088-8381-4e7b-8907-deb42041676d req-a1ccaa34-9788-44b7-a2d0-74a801484c26 service nova] [instance: cb569e8d-e4a5-4076-bb8b-deeefad033e2] No waiting events found dispatching network-vif-unplugged-0bd5d735-f566-4a6d-b49f-b24fd403a87a {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:08:52 user nova-compute[75234]: DEBUG nova.compute.manager [req-0282a088-8381-4e7b-8907-deb42041676d req-a1ccaa34-9788-44b7-a2d0-74a801484c26 service nova] [instance: cb569e8d-e4a5-4076-bb8b-deeefad033e2] Received event network-vif-unplugged-0bd5d735-f566-4a6d-b49f-b24fd403a87a for instance with task_state deleting. {{(pid=75234) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10890}}
Nov 29 03:08:52 user nova-compute[75234]: DEBUG nova.compute.manager [req-29e29927-c693-4b9d-923d-e7dbda54bfc7 req-e29475f7-dfc6-4134-b199-87c19fb15391 service nova] [instance: cb569e8d-e4a5-4076-bb8b-deeefad033e2] Received event network-vif-unplugged-2b12ec96-26fa-4974-8a7a-22e85ec20544 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:08:52 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-29e29927-c693-4b9d-923d-e7dbda54bfc7 req-e29475f7-dfc6-4134-b199-87c19fb15391 service nova] Acquiring lock "cb569e8d-e4a5-4076-bb8b-deeefad033e2-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:08:52 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-29e29927-c693-4b9d-923d-e7dbda54bfc7 req-e29475f7-dfc6-4134-b199-87c19fb15391 service nova] Lock "cb569e8d-e4a5-4076-bb8b-deeefad033e2-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:08:52 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-29e29927-c693-4b9d-923d-e7dbda54bfc7 req-e29475f7-dfc6-4134-b199-87c19fb15391 service nova] Lock "cb569e8d-e4a5-4076-bb8b-deeefad033e2-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:08:52 user nova-compute[75234]: DEBUG nova.compute.manager [req-29e29927-c693-4b9d-923d-e7dbda54bfc7 req-e29475f7-dfc6-4134-b199-87c19fb15391 service nova] [instance: cb569e8d-e4a5-4076-bb8b-deeefad033e2] No waiting events found dispatching network-vif-unplugged-2b12ec96-26fa-4974-8a7a-22e85ec20544 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:08:52 user nova-compute[75234]: DEBUG nova.compute.manager [req-29e29927-c693-4b9d-923d-e7dbda54bfc7 req-e29475f7-dfc6-4134-b199-87c19fb15391 service nova] [instance: cb569e8d-e4a5-4076-bb8b-deeefad033e2] Received event network-vif-unplugged-2b12ec96-26fa-4974-8a7a-22e85ec20544 for instance with task_state deleting. {{(pid=75234) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10890}}
Nov 29 03:08:52 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:08:52 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:08:52 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:08:52 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:08:53 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:08:53 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:08:53 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:08:53 user nova-compute[75234]: DEBUG nova.compute.manager [req-7221572f-6b87-4503-a2e1-5d3aa2cd0326 req-855f5c8b-3317-4f64-8d10-b844642a086e service nova] [instance: cb569e8d-e4a5-4076-bb8b-deeefad033e2] Received event network-vif-unplugged-026f606a-534c-4cdd-992a-9f33268b374f {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:08:53 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-7221572f-6b87-4503-a2e1-5d3aa2cd0326 req-855f5c8b-3317-4f64-8d10-b844642a086e service nova] Acquiring lock "cb569e8d-e4a5-4076-bb8b-deeefad033e2-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:08:53 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-7221572f-6b87-4503-a2e1-5d3aa2cd0326 req-855f5c8b-3317-4f64-8d10-b844642a086e service nova] Lock "cb569e8d-e4a5-4076-bb8b-deeefad033e2-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:08:53 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-7221572f-6b87-4503-a2e1-5d3aa2cd0326 req-855f5c8b-3317-4f64-8d10-b844642a086e service nova] Lock "cb569e8d-e4a5-4076-bb8b-deeefad033e2-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:08:53 user nova-compute[75234]: DEBUG nova.compute.manager [req-7221572f-6b87-4503-a2e1-5d3aa2cd0326 req-855f5c8b-3317-4f64-8d10-b844642a086e service nova] [instance: cb569e8d-e4a5-4076-bb8b-deeefad033e2] No waiting events found dispatching network-vif-unplugged-026f606a-534c-4cdd-992a-9f33268b374f {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:08:53 user nova-compute[75234]: DEBUG nova.compute.manager [req-7221572f-6b87-4503-a2e1-5d3aa2cd0326 req-855f5c8b-3317-4f64-8d10-b844642a086e service nova] [instance: cb569e8d-e4a5-4076-bb8b-deeefad033e2] Received event network-vif-unplugged-026f606a-534c-4cdd-992a-9f33268b374f for instance with task_state deleting. {{(pid=75234) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10890}}
Nov 29 03:08:53 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:08:53 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:08:53 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:08:53 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-01a31e51-5891-44ff-ba1a-33093249f28c tempest-AttachSCSIVolumeTestJSON-719577128 tempest-AttachSCSIVolumeTestJSON-719577128-project-member] Waiting for libvirt event about the detach of device sdc with device alias scsi0-0-0-2 from instance da625f9c-b7ed-4afc-875c-6436bf58e9cc is timed out.
Nov 29 03:08:53 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-01a31e51-5891-44ff-ba1a-33093249f28c tempest-AttachSCSIVolumeTestJSON-719577128 tempest-AttachSCSIVolumeTestJSON-719577128-project-member] Successfully detached device sdc from instance da625f9c-b7ed-4afc-875c-6436bf58e9cc from the live domain config.
Nov 29 03:08:53 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-01a31e51-5891-44ff-ba1a-33093249f28c tempest-AttachSCSIVolumeTestJSON-719577128 tempest-AttachSCSIVolumeTestJSON-719577128-project-member] [instance: da625f9c-b7ed-4afc-875c-6436bf58e9cc] calling os-brick to detach iSCSI Volume {{(pid=75234) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Nov 29 03:08:53 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-01a31e51-5891-44ff-ba1a-33093249f28c tempest-AttachSCSIVolumeTestJSON-719577128 tempest-AttachSCSIVolumeTestJSON-719577128-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f8a5050c9a0>, {'target_iqn': 'iqn.2010-10.org.openstack:ae86b2d0-7343-40f1-9624-ddc92be1340d', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'SrlLHTRV', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdi'}, None), 'kwargs': False}}" {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:176}}
Nov 29 03:08:53 user nova-compute[75234]: WARNING os_brick.initiator.connectors.base [None req-01a31e51-5891-44ff-ba1a-33093249f28c tempest-AttachSCSIVolumeTestJSON-719577128 tempest-AttachSCSIVolumeTestJSON-719577128-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Nov 29 03:08:53 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-01a31e51-5891-44ff-ba1a-33093249f28c tempest-AttachSCSIVolumeTestJSON-719577128 tempest-AttachSCSIVolumeTestJSON-719577128-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:68}}
Nov 29 03:08:53 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-01a31e51-5891-44ff-ba1a-33093249f28c tempest-AttachSCSIVolumeTestJSON-719577128 tempest-AttachSCSIVolumeTestJSON-719577128-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.002s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:73}}
Nov 29 03:08:53 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:08:53 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.037s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:08:53 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[c9586503-d4f3-4141-8c86-b89a9bbf4b19]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:0d2c0c22-2b3a-43db-b9af-e60a2514e48e\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:ab278679-6cfd-4c7c-a930-c078843640d3\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:a4cf29d0-2ce2-423f-a20c-a7a19a452088\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:ae86b2d0-7343-40f1-9624-ddc92be1340d\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:bb98586e-4f75-46be-a626-0b73ebda11eb\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:920d6cad-25ab-4f3d-941b-8ec841d43184\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:5fcd66fc-a479-4613-b1f8-e6ef12ca470a\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:b1068fd1-6636-43cd-9725-4a37f708b2bc\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:08:53 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-01a31e51-5891-44ff-ba1a-33093249f28c tempest-AttachSCSIVolumeTestJSON-719577128 tempest-AttachSCSIVolumeTestJSON-719577128-project-member] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS:
Nov 29 03:08:53 user nova-compute[75234]: No targets found.
Nov 29 03:08:53 user nova-compute[75234]: iSNS:
Nov 29 03:08:53 user nova-compute[75234]: No targets found.
Nov 29 03:08:53 user nova-compute[75234]: STATIC:
Nov 29 03:08:53 user nova-compute[75234]: Target: iqn.2010-10.org.openstack:0d2c0c22-2b3a-43db-b9af-e60a2514e48e
Nov 29 03:08:53 user nova-compute[75234]:         Portal: 172.16.0.220:3260,-1
Nov 29 03:08:53 user nova-compute[75234]:                 Iface Name: default
Nov 29 03:08:53 user nova-compute[75234]: Target: iqn.2010-10.org.openstack:ab278679-6cfd-4c7c-a930-c078843640d3
Nov 29 03:08:53 user nova-compute[75234]:         Portal: 172.16.0.220:3260,-1
Nov 29 03:08:53 user nova-compute[75234]:                 Iface Name: default
Nov 29 03:08:53 user nova-compute[75234]: Target: iqn.2010-10.org.openstack:a4cf29d0-2ce2-423f-a20c-a7a19a452088
Nov 29 03:08:53 user nova-compute[75234]:         Portal: 172.16.0.220:3260,-1
Nov 29 03:08:53 user nova-compute[75234]:                 Iface Name: default
Nov 29 03:08:53 user nova-compute[75234]: Target: iqn.2010-10.org.openstack:ae86b2d0-7343-40f1-9624-ddc92be1340d
Nov 29 03:08:53 user nova-compute[75234]:         Portal: 172.16.0.220:3260,-1
Nov 29 03:08:53 user nova-compute[75234]:                 Iface Name: default
Nov 29 03:08:53 user nova-compute[75234]: Target: iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672
Nov 29 03:08:53 user nova-compute[75234]:         Portal: 172.16.0.220:3260,-1
Nov 29 03:08:53 user nova-compute[75234]:                 Iface Name: default
Nov 29 03:08:53 user nova-compute[75234]: Target: iqn.2010-10.org.openstack:bb98586e-4f75-46be-a626-0b73ebda11eb
Nov 29 03:08:53 user nova-compute[75234]:         Portal: 172.16.0.220:3260,-1
Nov 29 03:08:53 user nova-compute[75234]:                 Iface Name: default
Nov 29 03:08:53 user nova-compute[75234]: Target: iqn.2010-10.org.openstack:920d6cad-25ab-4f3d-941b-8ec841d43184
Nov 29 03:08:53 user nova-compute[75234]:         Portal: 172.16.0.220:3260,-1
Nov 29 03:08:53 user nova-compute[75234]:                 Iface Name: default
Nov 29 03:08:53 user nova-compute[75234]: Target: iqn.2010-10.org.openstack:5fcd66fc-a479-4613-b1f8-e6ef12ca470a
Nov 29 03:08:53 user nova-compute[75234]:         Portal: 172.16.0.220:3260,-1
Nov 29 03:08:53 user nova-compute[75234]:                 Iface Name: default
Nov 29 03:08:53 user nova-compute[75234]: Target: iqn.2010-10.org.openstack:b1068fd1-6636-43cd-9725-4a37f708b2bc
Nov 29 03:08:53 user nova-compute[75234]:         Portal: 172.16.0.220:3260,-1
Nov 29 03:08:53 user nova-compute[75234]:                 Iface Name: default
Nov 29 03:08:53 user nova-compute[75234]: FIRMWARE:
Nov 29 03:08:53 user nova-compute[75234]: No targets found.
Nov 29 03:08:53 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:08:53 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-01a31e51-5891-44ff-ba1a-33093249f28c tempest-AttachSCSIVolumeTestJSON-719577128 tempest-AttachSCSIVolumeTestJSON-719577128-project-member] Regex to get portals from discoverydb: ^SENDTARGETS:
Nov 29 03:08:53 user nova-compute[75234]: .*?^DiscoveryAddress: 172.16.0.220,3260.*?
Nov 29 03:08:53 user nova-compute[75234]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=75234) _get_discoverydb_portals /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:382}}
Nov 29 03:08:53 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-01a31e51-5891-44ff-ba1a-33093249f28c tempest-AttachSCSIVolumeTestJSON-719577128 tempest-AttachSCSIVolumeTestJSON-719577128-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:ae86b2d0-7343-40f1-9624-ddc92be1340d', 0)] {{(pid=75234) _get_connection_devices /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:830}}
Nov 29 03:08:53 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:08:53 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.026s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:08:53 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[58e24da8-961f-4cbb-b8b3-ed0a28145918]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:0d2c0c22-2b3a-43db-b9af-e60a2514e48e\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:5fcd66fc-a479-4613-b1f8-e6ef12ca470a\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:920d6cad-25ab-4f3d-941b-8ec841d43184\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:a4cf29d0-2ce2-423f-a20c-a7a19a452088\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:ab278679-6cfd-4c7c-a930-c078843640d3\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:ae86b2d0-7343-40f1-9624-ddc92be1340d\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:b1068fd1-6636-43cd-9725-4a37f708b2bc\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:bb98586e-4f75-46be-a626-0b73ebda11eb\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:08:53 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:08:53 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:08:53 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager.update_available_resource {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:08:53 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:08:53 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:08:53 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:08:53 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Auditing locally available compute resources for user (node: user) {{(pid=75234) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:907}}
Nov 29 03:08:53 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.053s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:08:53 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[372d01ea-729d-4b78-a8d0-66b918168230]: (4, ('tcp: [10] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ae86b2d0-7343-40f1-9624-ddc92be1340d (non-flash)\ntcp: [11] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a4cf29d0-2ce2-423f-a20c-a7a19a452088 (non-flash)\ntcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bb98586e-4f75-46be-a626-0b73ebda11eb (non-flash)\ntcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ab278679-6cfd-4c7c-a930-c078843640d3 (non-flash)\ntcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:920d6cad-25ab-4f3d-941b-8ec841d43184 (non-flash)\ntcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b1068fd1-6636-43cd-9725-4a37f708b2bc (non-flash)\ntcp: [6] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0d2c0c22-2b3a-43db-b9af-e60a2514e48e (non-flash)\ntcp: [7] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5fcd66fc-a479-4613-b1f8-e6ef12ca470a (non-flash)\ntcp: [9] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672 (non-flash)\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:08:53 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-01a31e51-5891-44ff-ba1a-33093249f28c tempest-AttachSCSIVolumeTestJSON-719577128 tempest-AttachSCSIVolumeTestJSON-719577128-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [10] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ae86b2d0-7343-40f1-9624-ddc92be1340d (non-flash)
Nov 29 03:08:53 user nova-compute[75234]: tcp: [11] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a4cf29d0-2ce2-423f-a20c-a7a19a452088 (non-flash)
Nov 29 03:08:53 user nova-compute[75234]: tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bb98586e-4f75-46be-a626-0b73ebda11eb (non-flash)
Nov 29 03:08:53 user nova-compute[75234]: tcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ab278679-6cfd-4c7c-a930-c078843640d3 (non-flash)
Nov 29 03:08:53 user nova-compute[75234]: tcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:920d6cad-25ab-4f3d-941b-8ec841d43184 (non-flash)
Nov 29 03:08:53 user nova-compute[75234]: tcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b1068fd1-6636-43cd-9725-4a37f708b2bc (non-flash)
Nov 29 03:08:53 user nova-compute[75234]: tcp: [6] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0d2c0c22-2b3a-43db-b9af-e60a2514e48e (non-flash)
Nov 29 03:08:53 user nova-compute[75234]: tcp: [7] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5fcd66fc-a479-4613-b1f8-e6ef12ca470a (non-flash)
Nov 29 03:08:53 user nova-compute[75234]: tcp: [9] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672 (non-flash)
Nov 29 03:08:53 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:08:53 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-01a31e51-5891-44ff-ba1a-33093249f28c tempest-AttachSCSIVolumeTestJSON-719577128 tempest-AttachSCSIVolumeTestJSON-719577128-project-member] iscsi session list stdout=tcp: [10] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ae86b2d0-7343-40f1-9624-ddc92be1340d (non-flash)
Nov 29 03:08:53 user nova-compute[75234]: tcp: [11] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a4cf29d0-2ce2-423f-a20c-a7a19a452088 (non-flash)
Nov 29 03:08:53 user nova-compute[75234]: tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bb98586e-4f75-46be-a626-0b73ebda11eb (non-flash)
Nov 29 03:08:53 user nova-compute[75234]: tcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ab278679-6cfd-4c7c-a930-c078843640d3 (non-flash)
Nov 29 03:08:53 user nova-compute[75234]: tcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:920d6cad-25ab-4f3d-941b-8ec841d43184 (non-flash)
Nov 29 03:08:53 user nova-compute[75234]: tcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b1068fd1-6636-43cd-9725-4a37f708b2bc (non-flash)
Nov 29 03:08:53 user nova-compute[75234]: tcp: [6] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0d2c0c22-2b3a-43db-b9af-e60a2514e48e (non-flash)
Nov 29 03:08:53 user nova-compute[75234]: tcp: [7] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5fcd66fc-a479-4613-b1f8-e6ef12ca470a (non-flash)
Nov 29 03:08:53 user nova-compute[75234]: tcp: [9] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672 (non-flash)
Nov 29 03:08:53 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsi_session /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Nov 29 03:08:53 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-01a31e51-5891-44ff-ba1a-33093249f28c tempest-AttachSCSIVolumeTestJSON-719577128 tempest-AttachSCSIVolumeTestJSON-719577128-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7f8a502bed40>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:ae86b2d0-7343-40f1-9624-ddc92be1340d'): ({'sdi'}, set())}) {{(pid=75234) _get_connection_devices /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:863}}
Nov 29 03:08:53 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-01a31e51-5891-44ff-ba1a-33093249f28c tempest-AttachSCSIVolumeTestJSON-719577128 tempest-AttachSCSIVolumeTestJSON-719577128-project-member] Removing single pathed devices sdi {{(pid=75234) remove_connection /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:374}}
Nov 29 03:08:53 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:08:53 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.031s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:08:53 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[797998dd-7e91-4909-8678-38be719dfcff]: (4, ('path checker states:\nup                  9\n\npaths: 0\nbusy: False\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:08:53 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdi {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:08:53 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdi" returned: 0 in 0.026s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:08:53 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[1883b0e1-7fd4-41cf-8538-44c202026e45]: (4, ('ok\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:08:53 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-01a31e51-5891-44ff-ba1a-33093249f28c tempest-AttachSCSIVolumeTestJSON-719577128 tempest-AttachSCSIVolumeTestJSON-719577128-project-member] Flushing IO for device /dev/sdi {{(pid=75234) flush_device_io /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:434}}
Nov 29 03:08:53 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:08:53 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdi {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:08:53 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdi" returned: 0 in 0.017s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:08:53 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[98990ea3-cb5f-4f7b-9670-22521b077fa5]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:08:53 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-01a31e51-5891-44ff-ba1a-33093249f28c tempest-AttachSCSIVolumeTestJSON-719577128 tempest-AttachSCSIVolumeTestJSON-719577128-project-member] Remove SCSI device /dev/sdi with /sys/block/sdi/device/delete {{(pid=75234) remove_scsi_device /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:148}}
Nov 29 03:08:53 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdi/device/delete {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:08:53 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdi/device/delete" returned: 0 in 0.043s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:08:53 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[28cdfd9f-829d-442f-9627-dc75827e4091]: (4, ('1', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:08:53 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-01a31e51-5891-44ff-ba1a-33093249f28c tempest-AttachSCSIVolumeTestJSON-719577128 tempest-AttachSCSIVolumeTestJSON-719577128-project-member] Checking to see if SCSI volumes sdi have been removed. {{(pid=75234) wait_for_volumes_removal /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:156}}
Nov 29 03:08:53 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-01a31e51-5891-44ff-ba1a-33093249f28c tempest-AttachSCSIVolumeTestJSON-719577128 tempest-AttachSCSIVolumeTestJSON-719577128-project-member] SCSI volumes sdi have been removed. {{(pid=75234) wait_for_volumes_removal /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:166}}
Nov 29 03:08:53 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-01a31e51-5891-44ff-ba1a-33093249f28c tempest-AttachSCSIVolumeTestJSON-719577128 tempest-AttachSCSIVolumeTestJSON-719577128-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:ae86b2d0-7343-40f1-9624-ddc92be1340d')] {{(pid=75234) _disconnect_connection /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Nov 29 03:08:53 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:ae86b2d0-7343-40f1-9624-ddc92be1340d -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:08:53 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:08:53 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:ae86b2d0-7343-40f1-9624-ddc92be1340d -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.025s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:08:53 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[1c6a1258-4322-49c5-8e04-39f62d5104c6]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:08:53 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-01a31e51-5891-44ff-ba1a-33093249f28c tempest-AttachSCSIVolumeTestJSON-719577128 tempest-AttachSCSIVolumeTestJSON-719577128-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:08:53 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:ae86b2d0-7343-40f1-9624-ddc92be1340d -p 172.16.0.220:3260 --logout {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:08:53 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:ae86b2d0-7343-40f1-9624-ddc92be1340d -p 172.16.0.220:3260 --logout" returned: 0 in 0.117s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:08:53 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[ade1d213-7494-4b77-aeb1-4ecaceae58e9]: (4, ('Logging out of session [sid: 10, target: iqn.2010-10.org.openstack:ae86b2d0-7343-40f1-9624-ddc92be1340d, portal: 172.16.0.220,3260]\nLogout of [sid: 10, target: iqn.2010-10.org.openstack:ae86b2d0-7343-40f1-9624-ddc92be1340d, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:08:53 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-01a31e51-5891-44ff-ba1a-33093249f28c tempest-AttachSCSIVolumeTestJSON-719577128 tempest-AttachSCSIVolumeTestJSON-719577128-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 10, target: iqn.2010-10.org.openstack:ae86b2d0-7343-40f1-9624-ddc92be1340d, portal: 172.16.0.220,3260]
Nov 29 03:08:53 user nova-compute[75234]: Logout of [sid: 10, target: iqn.2010-10.org.openstack:ae86b2d0-7343-40f1-9624-ddc92be1340d, portal: 172.16.0.220,3260] successful.
Nov 29 03:08:53 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:08:53 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:ae86b2d0-7343-40f1-9624-ddc92be1340d -p 172.16.0.220:3260 --op delete {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:08:53 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:08:53 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:ae86b2d0-7343-40f1-9624-ddc92be1340d -p 172.16.0.220:3260 --op delete" returned: 0 in 0.045s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:08:53 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[22e364b1-a286-407d-94a6-72bad267bd81]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:08:53 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-01a31e51-5891-44ff-ba1a-33093249f28c tempest-AttachSCSIVolumeTestJSON-719577128 tempest-AttachSCSIVolumeTestJSON-719577128-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:08:53 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-01a31e51-5891-44ff-ba1a-33093249f28c tempest-AttachSCSIVolumeTestJSON-719577128 tempest-AttachSCSIVolumeTestJSON-719577128-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.503s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:87}}
Nov 29 03:08:53 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-01a31e51-5891-44ff-ba1a-33093249f28c tempest-AttachSCSIVolumeTestJSON-719577128 tempest-AttachSCSIVolumeTestJSON-719577128-project-member] <== disconnect_volume: return (505ms) None {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:203}}
Nov 29 03:08:53 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-01a31e51-5891-44ff-ba1a-33093249f28c tempest-AttachSCSIVolumeTestJSON-719577128 tempest-AttachSCSIVolumeTestJSON-719577128-project-member] [instance: da625f9c-b7ed-4afc-875c-6436bf58e9cc] Disconnected iSCSI Volume {{(pid=75234) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:79}}
Nov 29 03:08:53 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:08:54 user nova-compute[75234]: INFO nova.virt.libvirt.driver [-] [instance: cb569e8d-e4a5-4076-bb8b-deeefad033e2] Instance destroyed successfully.
Nov 29 03:08:54 user nova-compute[75234]: DEBUG nova.objects.instance [None req-88bf7696-da01-415d-80cd-dd7361a45c4c tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] Lazy-loading 'resources' on Instance uuid cb569e8d-e4a5-4076-bb8b-deeefad033e2 {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:08:54 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-88bf7696-da01-415d-80cd-dd7361a45c4c tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='True',created_at=2023-11-29T03:06:58Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-1051091393',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-device-tagging-server-1051091393',id=10,image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBOxFHXB9/btSEa91bTsp1yfkF2UsLgRU7yInw9joeBa8Nj3E7aswz+RinNuyzQo33k8c3J3qTezyAtFU7X0/nxAqdQ5O2Sv0+97iAivTta7NtcdKSix5ZmkHgQqMMjKOGw==',key_name='tempest-keypair-1672591556',keypairs=<?>,launch_index=0,launched_at=2023-11-29T03:07:38Z,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='ee1117bd3511400f9f45cc8f9bb9247d',ramdisk_id='',reservation_id='r-936jf1h5',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-TaggedBootDevicesTest-1858653514',owner_user_name='tempest-TaggedBootDevicesTest-1858653514-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-11-29T03:07:39Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='36c915d13a4943549549177c756da93b',uuid=cb569e8d-e4a5-4076-bb8b-deeefad033e2,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "2b12ec96-26fa-4974-8a7a-22e85ec20544", "address": "fa:16:3e:fd:00:a6", "network": {"id": "1869abec-0b45-446a-9dc8-5871de9dbfc5", "bridge": "br-int", "label": "tempest-TaggedBootDevicesTest-1812676341-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.181", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "ee1117bd3511400f9f45cc8f9bb9247d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2b12ec96-26", "ovs_interfaceid": "2b12ec96-26fa-4974-8a7a-22e85ec20544", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Nov 29 03:08:54 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-88bf7696-da01-415d-80cd-dd7361a45c4c tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] Converting VIF {"id": "2b12ec96-26fa-4974-8a7a-22e85ec20544", "address": "fa:16:3e:fd:00:a6", "network": {"id": "1869abec-0b45-446a-9dc8-5871de9dbfc5", "bridge": "br-int", "label": "tempest-TaggedBootDevicesTest-1812676341-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.181", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "ee1117bd3511400f9f45cc8f9bb9247d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2b12ec96-26", "ovs_interfaceid": "2b12ec96-26fa-4974-8a7a-22e85ec20544", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:08:54 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-88bf7696-da01-415d-80cd-dd7361a45c4c tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:fd:00:a6,bridge_name='br-int',has_traffic_filtering=True,id=2b12ec96-26fa-4974-8a7a-22e85ec20544,network=Network(1869abec-0b45-446a-9dc8-5871de9dbfc5),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap2b12ec96-26') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:08:54 user nova-compute[75234]: DEBUG os_vif [None req-88bf7696-da01-415d-80cd-dd7361a45c4c tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] Unplugging vif VIFOpenVSwitch(active=True,address=fa:16:3e:fd:00:a6,bridge_name='br-int',has_traffic_filtering=True,id=2b12ec96-26fa-4974-8a7a-22e85ec20544,network=Network(1869abec-0b45-446a-9dc8-5871de9dbfc5),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap2b12ec96-26') {{(pid=75234) unplug /opt/stack/data/venv/lib/python3.10/site-packages/os_vif/__init__.py:109}}
Nov 29 03:08:54 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:08:54 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tap2b12ec96-26, bridge=br-int, if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:08:54 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:08:54 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:08:54 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:08:54 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:08:54 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbDestroyCommand(_result=None, table=QoS, record=969108ac-b9d9-4aad-8009-826f457ca50f) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:08:54 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:08:54 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:08:54 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] skipping disk /dev/sde (vda) as it is a volume {{(pid=75234) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11213}}
Nov 29 03:08:54 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] skipping disk /dev/sdf (vdb) as it is a volume {{(pid=75234) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11213}}
Nov 29 03:08:54 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] skipping disk /dev/sdg (vdc) as it is a volume {{(pid=75234) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11213}}
Nov 29 03:08:54 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a9350a18-14e6-42c6-a150-0a1487f20ad9/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:08:54 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:08:54 user nova-compute[75234]: INFO os_vif [None req-88bf7696-da01-415d-80cd-dd7361a45c4c tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] Successfully unplugged vif VIFOpenVSwitch(active=True,address=fa:16:3e:fd:00:a6,bridge_name='br-int',has_traffic_filtering=True,id=2b12ec96-26fa-4974-8a7a-22e85ec20544,network=Network(1869abec-0b45-446a-9dc8-5871de9dbfc5),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap2b12ec96-26')
Nov 29 03:08:54 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-88bf7696-da01-415d-80cd-dd7361a45c4c tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='True',created_at=2023-11-29T03:06:58Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-1051091393',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-device-tagging-server-1051091393',id=10,image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBOxFHXB9/btSEa91bTsp1yfkF2UsLgRU7yInw9joeBa8Nj3E7aswz+RinNuyzQo33k8c3J3qTezyAtFU7X0/nxAqdQ5O2Sv0+97iAivTta7NtcdKSix5ZmkHgQqMMjKOGw==',key_name='tempest-keypair-1672591556',keypairs=<?>,launch_index=0,launched_at=2023-11-29T03:07:38Z,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='ee1117bd3511400f9f45cc8f9bb9247d',ramdisk_id='',reservation_id='r-936jf1h5',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-TaggedBootDevicesTest-1858653514',owner_user_name='tempest-TaggedBootDevicesTest-1858653514-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-11-29T03:07:39Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='36c915d13a4943549549177c756da93b',uuid=cb569e8d-e4a5-4076-bb8b-deeefad033e2,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "0bd5d735-f566-4a6d-b49f-b24fd403a87a", "address": "fa:16:3e:aa:c2:ae", "network": {"id": "005139f3-3686-493a-aa8f-0cdc455befa0", "bridge": "br-int", "label": "tempest-device-tagging-net1-1510949192", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.85", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "ee1117bd3511400f9f45cc8f9bb9247d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap0bd5d735-f5", "ovs_interfaceid": "0bd5d735-f566-4a6d-b49f-b24fd403a87a", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}} {{(pid=75234) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Nov 29 03:08:54 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-88bf7696-da01-415d-80cd-dd7361a45c4c tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] Converting VIF {"id": "0bd5d735-f566-4a6d-b49f-b24fd403a87a", "address": "fa:16:3e:aa:c2:ae", "network": {"id": "005139f3-3686-493a-aa8f-0cdc455befa0", "bridge": "br-int", "label": "tempest-device-tagging-net1-1510949192", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.85", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "ee1117bd3511400f9f45cc8f9bb9247d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap0bd5d735-f5", "ovs_interfaceid": "0bd5d735-f566-4a6d-b49f-b24fd403a87a", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:08:54 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-88bf7696-da01-415d-80cd-dd7361a45c4c tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:aa:c2:ae,bridge_name='br-int',has_traffic_filtering=True,id=0bd5d735-f566-4a6d-b49f-b24fd403a87a,network=Network(005139f3-3686-493a-aa8f-0cdc455befa0),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=True,vif_name='tap0bd5d735-f5') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:08:54 user nova-compute[75234]: DEBUG os_vif [None req-88bf7696-da01-415d-80cd-dd7361a45c4c tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] Unplugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:aa:c2:ae,bridge_name='br-int',has_traffic_filtering=True,id=0bd5d735-f566-4a6d-b49f-b24fd403a87a,network=Network(005139f3-3686-493a-aa8f-0cdc455befa0),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=True,vif_name='tap0bd5d735-f5') {{(pid=75234) unplug /opt/stack/data/venv/lib/python3.10/site-packages/os_vif/__init__.py:109}}
Nov 29 03:08:54 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:08:54 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tap0bd5d735-f5, bridge=br-int, if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:08:54 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:08:54 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:08:54 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:08:54 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:08:54 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbDestroyCommand(_result=None, table=QoS, record=d93826bd-c224-4a03-8c14-2a2bba2568ac) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:08:54 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:08:54 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:08:54 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:08:54 user nova-compute[75234]: INFO os_vif [None req-88bf7696-da01-415d-80cd-dd7361a45c4c tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] Successfully unplugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:aa:c2:ae,bridge_name='br-int',has_traffic_filtering=True,id=0bd5d735-f566-4a6d-b49f-b24fd403a87a,network=Network(005139f3-3686-493a-aa8f-0cdc455befa0),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=True,vif_name='tap0bd5d735-f5')
Nov 29 03:08:54 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-88bf7696-da01-415d-80cd-dd7361a45c4c tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='True',created_at=2023-11-29T03:06:58Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-1051091393',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-device-tagging-server-1051091393',id=10,image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBOxFHXB9/btSEa91bTsp1yfkF2UsLgRU7yInw9joeBa8Nj3E7aswz+RinNuyzQo33k8c3J3qTezyAtFU7X0/nxAqdQ5O2Sv0+97iAivTta7NtcdKSix5ZmkHgQqMMjKOGw==',key_name='tempest-keypair-1672591556',keypairs=<?>,launch_index=0,launched_at=2023-11-29T03:07:38Z,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='ee1117bd3511400f9f45cc8f9bb9247d',ramdisk_id='',reservation_id='r-936jf1h5',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-TaggedBootDevicesTest-1858653514',owner_user_name='tempest-TaggedBootDevicesTest-1858653514-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-11-29T03:07:39Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='36c915d13a4943549549177c756da93b',uuid=cb569e8d-e4a5-4076-bb8b-deeefad033e2,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "27e72461-f85b-42c9-8824-0d760b82a725", "address": "fa:16:3e:8e:29:fd", "network": {"id": "005139f3-3686-493a-aa8f-0cdc455befa0", "bridge": "br-int", "label": "tempest-device-tagging-net1-1510949192", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.127", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "ee1117bd3511400f9f45cc8f9bb9247d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap27e72461-f8", "ovs_interfaceid": "27e72461-f85b-42c9-8824-0d760b82a725", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}} {{(pid=75234) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Nov 29 03:08:54 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-88bf7696-da01-415d-80cd-dd7361a45c4c tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] Converting VIF {"id": "27e72461-f85b-42c9-8824-0d760b82a725", "address": "fa:16:3e:8e:29:fd", "network": {"id": "005139f3-3686-493a-aa8f-0cdc455befa0", "bridge": "br-int", "label": "tempest-device-tagging-net1-1510949192", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.127", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "ee1117bd3511400f9f45cc8f9bb9247d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap27e72461-f8", "ovs_interfaceid": "27e72461-f85b-42c9-8824-0d760b82a725", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:08:54 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-88bf7696-da01-415d-80cd-dd7361a45c4c tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:8e:29:fd,bridge_name='br-int',has_traffic_filtering=True,id=27e72461-f85b-42c9-8824-0d760b82a725,network=Network(005139f3-3686-493a-aa8f-0cdc455befa0),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=True,vif_name='tap27e72461-f8') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:08:54 user nova-compute[75234]: DEBUG os_vif [None req-88bf7696-da01-415d-80cd-dd7361a45c4c tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] Unplugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:8e:29:fd,bridge_name='br-int',has_traffic_filtering=True,id=27e72461-f85b-42c9-8824-0d760b82a725,network=Network(005139f3-3686-493a-aa8f-0cdc455befa0),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=True,vif_name='tap27e72461-f8') {{(pid=75234) unplug /opt/stack/data/venv/lib/python3.10/site-packages/os_vif/__init__.py:109}}
Nov 29 03:08:54 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:08:54 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tap27e72461-f8, bridge=br-int, if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:08:54 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:08:54 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:08:54 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a9350a18-14e6-42c6-a150-0a1487f20ad9/disk --force-share --output=json" returned: 0 in 0.235s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:08:54 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a9350a18-14e6-42c6-a150-0a1487f20ad9/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:08:54 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:08:54 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:08:54 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbDestroyCommand(_result=None, table=QoS, record=5d91513d-f0d4-4351-a36a-2225af2b7c52) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:08:54 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:08:54 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:08:54 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:08:54 user nova-compute[75234]: INFO os_vif [None req-88bf7696-da01-415d-80cd-dd7361a45c4c tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] Successfully unplugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:8e:29:fd,bridge_name='br-int',has_traffic_filtering=True,id=27e72461-f85b-42c9-8824-0d760b82a725,network=Network(005139f3-3686-493a-aa8f-0cdc455befa0),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=True,vif_name='tap27e72461-f8')
Nov 29 03:08:54 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-88bf7696-da01-415d-80cd-dd7361a45c4c tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='True',created_at=2023-11-29T03:06:58Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-1051091393',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-device-tagging-server-1051091393',id=10,image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBOxFHXB9/btSEa91bTsp1yfkF2UsLgRU7yInw9joeBa8Nj3E7aswz+RinNuyzQo33k8c3J3qTezyAtFU7X0/nxAqdQ5O2Sv0+97iAivTta7NtcdKSix5ZmkHgQqMMjKOGw==',key_name='tempest-keypair-1672591556',keypairs=<?>,launch_index=0,launched_at=2023-11-29T03:07:38Z,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='ee1117bd3511400f9f45cc8f9bb9247d',ramdisk_id='',reservation_id='r-936jf1h5',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-TaggedBootDevicesTest-1858653514',owner_user_name='tempest-TaggedBootDevicesTest-1858653514-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-11-29T03:07:39Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='36c915d13a4943549549177c756da93b',uuid=cb569e8d-e4a5-4076-bb8b-deeefad033e2,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "026f606a-534c-4cdd-992a-9f33268b374f", "address": "fa:16:3e:3e:3b:29", "network": {"id": "005139f3-3686-493a-aa8f-0cdc455befa0", "bridge": "br-int", "label": "tempest-device-tagging-net1-1510949192", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.41", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "ee1117bd3511400f9f45cc8f9bb9247d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap026f606a-53", "ovs_interfaceid": "026f606a-534c-4cdd-992a-9f33268b374f", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Nov 29 03:08:54 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-88bf7696-da01-415d-80cd-dd7361a45c4c tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] Converting VIF {"id": "026f606a-534c-4cdd-992a-9f33268b374f", "address": "fa:16:3e:3e:3b:29", "network": {"id": "005139f3-3686-493a-aa8f-0cdc455befa0", "bridge": "br-int", "label": "tempest-device-tagging-net1-1510949192", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.41", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "ee1117bd3511400f9f45cc8f9bb9247d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap026f606a-53", "ovs_interfaceid": "026f606a-534c-4cdd-992a-9f33268b374f", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:08:54 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-88bf7696-da01-415d-80cd-dd7361a45c4c tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:3e:3b:29,bridge_name='br-int',has_traffic_filtering=True,id=026f606a-534c-4cdd-992a-9f33268b374f,network=Network(005139f3-3686-493a-aa8f-0cdc455befa0),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap026f606a-53') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:08:54 user nova-compute[75234]: DEBUG os_vif [None req-88bf7696-da01-415d-80cd-dd7361a45c4c tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] Unplugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:3e:3b:29,bridge_name='br-int',has_traffic_filtering=True,id=026f606a-534c-4cdd-992a-9f33268b374f,network=Network(005139f3-3686-493a-aa8f-0cdc455befa0),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap026f606a-53') {{(pid=75234) unplug /opt/stack/data/venv/lib/python3.10/site-packages/os_vif/__init__.py:109}}
Nov 29 03:08:54 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:08:54 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tap026f606a-53, bridge=br-int, if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:08:54 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:08:54 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:08:54 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:08:54 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:08:54 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbDestroyCommand(_result=None, table=QoS, record=3a7aa241-830c-48d4-b27d-c679a692a4cb) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:08:54 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:08:54 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:08:54 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a9350a18-14e6-42c6-a150-0a1487f20ad9/disk --force-share --output=json" returned: 0 in 0.174s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:08:54 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/8297c264-a213-44ee-be46-5c08eabda9f7/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:08:54 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:08:54 user nova-compute[75234]: INFO os_vif [None req-88bf7696-da01-415d-80cd-dd7361a45c4c tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] Successfully unplugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:3e:3b:29,bridge_name='br-int',has_traffic_filtering=True,id=026f606a-534c-4cdd-992a-9f33268b374f,network=Network(005139f3-3686-493a-aa8f-0cdc455befa0),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap026f606a-53')
Nov 29 03:08:54 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-88bf7696-da01-415d-80cd-dd7361a45c4c tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='True',created_at=2023-11-29T03:06:58Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-1051091393',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-device-tagging-server-1051091393',id=10,image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBOxFHXB9/btSEa91bTsp1yfkF2UsLgRU7yInw9joeBa8Nj3E7aswz+RinNuyzQo33k8c3J3qTezyAtFU7X0/nxAqdQ5O2Sv0+97iAivTta7NtcdKSix5ZmkHgQqMMjKOGw==',key_name='tempest-keypair-1672591556',keypairs=<?>,launch_index=0,launched_at=2023-11-29T03:07:38Z,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='ee1117bd3511400f9f45cc8f9bb9247d',ramdisk_id='',reservation_id='r-936jf1h5',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-TaggedBootDevicesTest-1858653514',owner_user_name='tempest-TaggedBootDevicesTest-1858653514-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-11-29T03:07:39Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='36c915d13a4943549549177c756da93b',uuid=cb569e8d-e4a5-4076-bb8b-deeefad033e2,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "67446920-3741-4542-84c6-493388395f26", "address": "fa:16:3e:f6:7a:76", "network": {"id": "005139f3-3686-493a-aa8f-0cdc455befa0", "bridge": "br-int", "label": "tempest-device-tagging-net1-1510949192", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.230", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "ee1117bd3511400f9f45cc8f9bb9247d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap67446920-37", "ovs_interfaceid": "67446920-3741-4542-84c6-493388395f26", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Nov 29 03:08:54 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-88bf7696-da01-415d-80cd-dd7361a45c4c tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] Converting VIF {"id": "67446920-3741-4542-84c6-493388395f26", "address": "fa:16:3e:f6:7a:76", "network": {"id": "005139f3-3686-493a-aa8f-0cdc455befa0", "bridge": "br-int", "label": "tempest-device-tagging-net1-1510949192", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.230", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "ee1117bd3511400f9f45cc8f9bb9247d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap67446920-37", "ovs_interfaceid": "67446920-3741-4542-84c6-493388395f26", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:08:54 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-88bf7696-da01-415d-80cd-dd7361a45c4c tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:f6:7a:76,bridge_name='br-int',has_traffic_filtering=True,id=67446920-3741-4542-84c6-493388395f26,network=Network(005139f3-3686-493a-aa8f-0cdc455befa0),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap67446920-37') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:08:54 user nova-compute[75234]: DEBUG os_vif [None req-88bf7696-da01-415d-80cd-dd7361a45c4c tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] Unplugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:f6:7a:76,bridge_name='br-int',has_traffic_filtering=True,id=67446920-3741-4542-84c6-493388395f26,network=Network(005139f3-3686-493a-aa8f-0cdc455befa0),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap67446920-37') {{(pid=75234) unplug /opt/stack/data/venv/lib/python3.10/site-packages/os_vif/__init__.py:109}}
Nov 29 03:08:54 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:08:54 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tap67446920-37, bridge=br-int, if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:08:54 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:08:54 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:08:54 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:08:54 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:08:54 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbDestroyCommand(_result=None, table=QoS, record=29a4a37a-747b-40e3-beca-787c66f2ee25) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:08:54 user nova-compute[75234]: DEBUG nova.compute.manager [req-8f076f91-1e27-41a4-87f0-42c4da2ad5b5 req-6b04a6a4-e836-4288-801c-7c9c67f25d4d service nova] [instance: cb569e8d-e4a5-4076-bb8b-deeefad033e2] Received event network-vif-plugged-2b12ec96-26fa-4974-8a7a-22e85ec20544 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:08:54 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-8f076f91-1e27-41a4-87f0-42c4da2ad5b5 req-6b04a6a4-e836-4288-801c-7c9c67f25d4d service nova] Acquiring lock "cb569e8d-e4a5-4076-bb8b-deeefad033e2-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:08:54 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-8f076f91-1e27-41a4-87f0-42c4da2ad5b5 req-6b04a6a4-e836-4288-801c-7c9c67f25d4d service nova] Lock "cb569e8d-e4a5-4076-bb8b-deeefad033e2-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:08:54 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-8f076f91-1e27-41a4-87f0-42c4da2ad5b5 req-6b04a6a4-e836-4288-801c-7c9c67f25d4d service nova] Lock "cb569e8d-e4a5-4076-bb8b-deeefad033e2-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:08:54 user nova-compute[75234]: DEBUG nova.compute.manager [req-8f076f91-1e27-41a4-87f0-42c4da2ad5b5 req-6b04a6a4-e836-4288-801c-7c9c67f25d4d service nova] [instance: cb569e8d-e4a5-4076-bb8b-deeefad033e2] No waiting events found dispatching network-vif-plugged-2b12ec96-26fa-4974-8a7a-22e85ec20544 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:08:54 user nova-compute[75234]: WARNING nova.compute.manager [req-8f076f91-1e27-41a4-87f0-42c4da2ad5b5 req-6b04a6a4-e836-4288-801c-7c9c67f25d4d service nova] [instance: cb569e8d-e4a5-4076-bb8b-deeefad033e2] Received unexpected event network-vif-plugged-2b12ec96-26fa-4974-8a7a-22e85ec20544 for instance with vm_state active and task_state deleting.
Nov 29 03:08:54 user nova-compute[75234]: DEBUG nova.compute.manager [req-8f076f91-1e27-41a4-87f0-42c4da2ad5b5 req-6b04a6a4-e836-4288-801c-7c9c67f25d4d service nova] [instance: cb569e8d-e4a5-4076-bb8b-deeefad033e2] Received event network-vif-unplugged-d7e92753-492b-4fc0-8048-f4792cd3b4b6 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:08:54 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-8f076f91-1e27-41a4-87f0-42c4da2ad5b5 req-6b04a6a4-e836-4288-801c-7c9c67f25d4d service nova] Acquiring lock "cb569e8d-e4a5-4076-bb8b-deeefad033e2-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:08:54 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-8f076f91-1e27-41a4-87f0-42c4da2ad5b5 req-6b04a6a4-e836-4288-801c-7c9c67f25d4d service nova] Lock "cb569e8d-e4a5-4076-bb8b-deeefad033e2-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:08:54 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-8f076f91-1e27-41a4-87f0-42c4da2ad5b5 req-6b04a6a4-e836-4288-801c-7c9c67f25d4d service nova] Lock "cb569e8d-e4a5-4076-bb8b-deeefad033e2-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:08:54 user nova-compute[75234]: DEBUG nova.compute.manager [req-8f076f91-1e27-41a4-87f0-42c4da2ad5b5 req-6b04a6a4-e836-4288-801c-7c9c67f25d4d service nova] [instance: cb569e8d-e4a5-4076-bb8b-deeefad033e2] No waiting events found dispatching network-vif-unplugged-d7e92753-492b-4fc0-8048-f4792cd3b4b6 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:08:54 user nova-compute[75234]: DEBUG nova.compute.manager [req-8f076f91-1e27-41a4-87f0-42c4da2ad5b5 req-6b04a6a4-e836-4288-801c-7c9c67f25d4d service nova] [instance: cb569e8d-e4a5-4076-bb8b-deeefad033e2] Received event network-vif-unplugged-d7e92753-492b-4fc0-8048-f4792cd3b4b6 for instance with task_state deleting. {{(pid=75234) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10890}}
Nov 29 03:08:54 user nova-compute[75234]: DEBUG nova.compute.manager [req-8f076f91-1e27-41a4-87f0-42c4da2ad5b5 req-6b04a6a4-e836-4288-801c-7c9c67f25d4d service nova] [instance: cb569e8d-e4a5-4076-bb8b-deeefad033e2] Received event network-vif-plugged-d7e92753-492b-4fc0-8048-f4792cd3b4b6 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:08:54 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-8f076f91-1e27-41a4-87f0-42c4da2ad5b5 req-6b04a6a4-e836-4288-801c-7c9c67f25d4d service nova] Acquiring lock "cb569e8d-e4a5-4076-bb8b-deeefad033e2-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:08:54 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-8f076f91-1e27-41a4-87f0-42c4da2ad5b5 req-6b04a6a4-e836-4288-801c-7c9c67f25d4d service nova] Lock "cb569e8d-e4a5-4076-bb8b-deeefad033e2-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:08:54 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-8f076f91-1e27-41a4-87f0-42c4da2ad5b5 req-6b04a6a4-e836-4288-801c-7c9c67f25d4d service nova] Lock "cb569e8d-e4a5-4076-bb8b-deeefad033e2-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:08:54 user nova-compute[75234]: DEBUG nova.compute.manager [req-8f076f91-1e27-41a4-87f0-42c4da2ad5b5 req-6b04a6a4-e836-4288-801c-7c9c67f25d4d service nova] [instance: cb569e8d-e4a5-4076-bb8b-deeefad033e2] No waiting events found dispatching network-vif-plugged-d7e92753-492b-4fc0-8048-f4792cd3b4b6 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:08:54 user nova-compute[75234]: WARNING nova.compute.manager [req-8f076f91-1e27-41a4-87f0-42c4da2ad5b5 req-6b04a6a4-e836-4288-801c-7c9c67f25d4d service nova] [instance: cb569e8d-e4a5-4076-bb8b-deeefad033e2] Received unexpected event network-vif-plugged-d7e92753-492b-4fc0-8048-f4792cd3b4b6 for instance with vm_state active and task_state deleting.
Nov 29 03:08:54 user nova-compute[75234]: DEBUG nova.compute.manager [req-8f076f91-1e27-41a4-87f0-42c4da2ad5b5 req-6b04a6a4-e836-4288-801c-7c9c67f25d4d service nova] [instance: cb569e8d-e4a5-4076-bb8b-deeefad033e2] Received event network-vif-unplugged-06db3012-0042-457a-a3c8-14e3a8d2e89e {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:08:54 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-8f076f91-1e27-41a4-87f0-42c4da2ad5b5 req-6b04a6a4-e836-4288-801c-7c9c67f25d4d service nova] Acquiring lock "cb569e8d-e4a5-4076-bb8b-deeefad033e2-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:08:54 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-8f076f91-1e27-41a4-87f0-42c4da2ad5b5 req-6b04a6a4-e836-4288-801c-7c9c67f25d4d service nova] Lock "cb569e8d-e4a5-4076-bb8b-deeefad033e2-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:08:54 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-8f076f91-1e27-41a4-87f0-42c4da2ad5b5 req-6b04a6a4-e836-4288-801c-7c9c67f25d4d service nova] Lock "cb569e8d-e4a5-4076-bb8b-deeefad033e2-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:08:54 user nova-compute[75234]: DEBUG nova.compute.manager [req-8f076f91-1e27-41a4-87f0-42c4da2ad5b5 req-6b04a6a4-e836-4288-801c-7c9c67f25d4d service nova] [instance: cb569e8d-e4a5-4076-bb8b-deeefad033e2] No waiting events found dispatching network-vif-unplugged-06db3012-0042-457a-a3c8-14e3a8d2e89e {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:08:54 user nova-compute[75234]: DEBUG nova.compute.manager [req-8f076f91-1e27-41a4-87f0-42c4da2ad5b5 req-6b04a6a4-e836-4288-801c-7c9c67f25d4d service nova] [instance: cb569e8d-e4a5-4076-bb8b-deeefad033e2] Received event network-vif-unplugged-06db3012-0042-457a-a3c8-14e3a8d2e89e for instance with task_state deleting. {{(pid=75234) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10890}}
Nov 29 03:08:54 user nova-compute[75234]: DEBUG nova.compute.manager [req-8f076f91-1e27-41a4-87f0-42c4da2ad5b5 req-6b04a6a4-e836-4288-801c-7c9c67f25d4d service nova] [instance: cb569e8d-e4a5-4076-bb8b-deeefad033e2] Received event network-vif-plugged-06db3012-0042-457a-a3c8-14e3a8d2e89e {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:08:54 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-8f076f91-1e27-41a4-87f0-42c4da2ad5b5 req-6b04a6a4-e836-4288-801c-7c9c67f25d4d service nova] Acquiring lock "cb569e8d-e4a5-4076-bb8b-deeefad033e2-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:08:54 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-8f076f91-1e27-41a4-87f0-42c4da2ad5b5 req-6b04a6a4-e836-4288-801c-7c9c67f25d4d service nova] Lock "cb569e8d-e4a5-4076-bb8b-deeefad033e2-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:08:54 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-8f076f91-1e27-41a4-87f0-42c4da2ad5b5 req-6b04a6a4-e836-4288-801c-7c9c67f25d4d service nova] Lock "cb569e8d-e4a5-4076-bb8b-deeefad033e2-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:08:54 user nova-compute[75234]: DEBUG nova.compute.manager [req-8f076f91-1e27-41a4-87f0-42c4da2ad5b5 req-6b04a6a4-e836-4288-801c-7c9c67f25d4d service nova] [instance: cb569e8d-e4a5-4076-bb8b-deeefad033e2] No waiting events found dispatching network-vif-plugged-06db3012-0042-457a-a3c8-14e3a8d2e89e {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:08:54 user nova-compute[75234]: WARNING nova.compute.manager [req-8f076f91-1e27-41a4-87f0-42c4da2ad5b5 req-6b04a6a4-e836-4288-801c-7c9c67f25d4d service nova] [instance: cb569e8d-e4a5-4076-bb8b-deeefad033e2] Received unexpected event network-vif-plugged-06db3012-0042-457a-a3c8-14e3a8d2e89e for instance with vm_state active and task_state deleting.
Nov 29 03:08:54 user nova-compute[75234]: DEBUG nova.compute.manager [req-4e76cf05-fd89-412f-a204-2f5cda6a281e req-1bbfc762-064e-4f7e-9c1d-b01e01801383 service nova] [instance: cb569e8d-e4a5-4076-bb8b-deeefad033e2] Received event network-vif-plugged-0bd5d735-f566-4a6d-b49f-b24fd403a87a {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:08:54 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-4e76cf05-fd89-412f-a204-2f5cda6a281e req-1bbfc762-064e-4f7e-9c1d-b01e01801383 service nova] Acquiring lock "cb569e8d-e4a5-4076-bb8b-deeefad033e2-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:08:54 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-4e76cf05-fd89-412f-a204-2f5cda6a281e req-1bbfc762-064e-4f7e-9c1d-b01e01801383 service nova] Lock "cb569e8d-e4a5-4076-bb8b-deeefad033e2-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:08:54 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-4e76cf05-fd89-412f-a204-2f5cda6a281e req-1bbfc762-064e-4f7e-9c1d-b01e01801383 service nova] Lock "cb569e8d-e4a5-4076-bb8b-deeefad033e2-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:08:54 user nova-compute[75234]: DEBUG nova.compute.manager [req-4e76cf05-fd89-412f-a204-2f5cda6a281e req-1bbfc762-064e-4f7e-9c1d-b01e01801383 service nova] [instance: cb569e8d-e4a5-4076-bb8b-deeefad033e2] No waiting events found dispatching network-vif-plugged-0bd5d735-f566-4a6d-b49f-b24fd403a87a {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:08:54 user nova-compute[75234]: WARNING nova.compute.manager [req-4e76cf05-fd89-412f-a204-2f5cda6a281e req-1bbfc762-064e-4f7e-9c1d-b01e01801383 service nova] [instance: cb569e8d-e4a5-4076-bb8b-deeefad033e2] Received unexpected event network-vif-plugged-0bd5d735-f566-4a6d-b49f-b24fd403a87a for instance with vm_state active and task_state deleting.
Nov 29 03:08:54 user nova-compute[75234]: DEBUG nova.compute.manager [req-4e76cf05-fd89-412f-a204-2f5cda6a281e req-1bbfc762-064e-4f7e-9c1d-b01e01801383 service nova] [instance: cb569e8d-e4a5-4076-bb8b-deeefad033e2] Received event network-vif-unplugged-27e72461-f85b-42c9-8824-0d760b82a725 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:08:54 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-4e76cf05-fd89-412f-a204-2f5cda6a281e req-1bbfc762-064e-4f7e-9c1d-b01e01801383 service nova] Acquiring lock "cb569e8d-e4a5-4076-bb8b-deeefad033e2-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:08:54 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-4e76cf05-fd89-412f-a204-2f5cda6a281e req-1bbfc762-064e-4f7e-9c1d-b01e01801383 service nova] Lock "cb569e8d-e4a5-4076-bb8b-deeefad033e2-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:08:54 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-4e76cf05-fd89-412f-a204-2f5cda6a281e req-1bbfc762-064e-4f7e-9c1d-b01e01801383 service nova] Lock "cb569e8d-e4a5-4076-bb8b-deeefad033e2-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:08:54 user nova-compute[75234]: DEBUG nova.compute.manager [req-4e76cf05-fd89-412f-a204-2f5cda6a281e req-1bbfc762-064e-4f7e-9c1d-b01e01801383 service nova] [instance: cb569e8d-e4a5-4076-bb8b-deeefad033e2] No waiting events found dispatching network-vif-unplugged-27e72461-f85b-42c9-8824-0d760b82a725 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:08:54 user nova-compute[75234]: DEBUG nova.compute.manager [req-4e76cf05-fd89-412f-a204-2f5cda6a281e req-1bbfc762-064e-4f7e-9c1d-b01e01801383 service nova] [instance: cb569e8d-e4a5-4076-bb8b-deeefad033e2] Received event network-vif-unplugged-27e72461-f85b-42c9-8824-0d760b82a725 for instance with task_state deleting. {{(pid=75234) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10890}}
Nov 29 03:08:54 user nova-compute[75234]: DEBUG nova.compute.manager [req-4e76cf05-fd89-412f-a204-2f5cda6a281e req-1bbfc762-064e-4f7e-9c1d-b01e01801383 service nova] [instance: cb569e8d-e4a5-4076-bb8b-deeefad033e2] Received event network-vif-plugged-27e72461-f85b-42c9-8824-0d760b82a725 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:08:54 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-4e76cf05-fd89-412f-a204-2f5cda6a281e req-1bbfc762-064e-4f7e-9c1d-b01e01801383 service nova] Acquiring lock "cb569e8d-e4a5-4076-bb8b-deeefad033e2-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:08:54 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-4e76cf05-fd89-412f-a204-2f5cda6a281e req-1bbfc762-064e-4f7e-9c1d-b01e01801383 service nova] Lock "cb569e8d-e4a5-4076-bb8b-deeefad033e2-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:08:54 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-4e76cf05-fd89-412f-a204-2f5cda6a281e req-1bbfc762-064e-4f7e-9c1d-b01e01801383 service nova] Lock "cb569e8d-e4a5-4076-bb8b-deeefad033e2-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:08:54 user nova-compute[75234]: DEBUG nova.compute.manager [req-4e76cf05-fd89-412f-a204-2f5cda6a281e req-1bbfc762-064e-4f7e-9c1d-b01e01801383 service nova] [instance: cb569e8d-e4a5-4076-bb8b-deeefad033e2] No waiting events found dispatching network-vif-plugged-27e72461-f85b-42c9-8824-0d760b82a725 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:08:54 user nova-compute[75234]: WARNING nova.compute.manager [req-4e76cf05-fd89-412f-a204-2f5cda6a281e req-1bbfc762-064e-4f7e-9c1d-b01e01801383 service nova] [instance: cb569e8d-e4a5-4076-bb8b-deeefad033e2] Received unexpected event network-vif-plugged-27e72461-f85b-42c9-8824-0d760b82a725 for instance with vm_state active and task_state deleting.
Nov 29 03:08:54 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:08:54 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:08:54 user nova-compute[75234]: INFO os_vif [None req-88bf7696-da01-415d-80cd-dd7361a45c4c tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] Successfully unplugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:f6:7a:76,bridge_name='br-int',has_traffic_filtering=True,id=67446920-3741-4542-84c6-493388395f26,network=Network(005139f3-3686-493a-aa8f-0cdc455befa0),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap67446920-37')
Nov 29 03:08:54 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-88bf7696-da01-415d-80cd-dd7361a45c4c tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='True',created_at=2023-11-29T03:06:58Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-1051091393',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-device-tagging-server-1051091393',id=10,image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBOxFHXB9/btSEa91bTsp1yfkF2UsLgRU7yInw9joeBa8Nj3E7aswz+RinNuyzQo33k8c3J3qTezyAtFU7X0/nxAqdQ5O2Sv0+97iAivTta7NtcdKSix5ZmkHgQqMMjKOGw==',key_name='tempest-keypair-1672591556',keypairs=<?>,launch_index=0,launched_at=2023-11-29T03:07:38Z,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='ee1117bd3511400f9f45cc8f9bb9247d',ramdisk_id='',reservation_id='r-936jf1h5',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-TaggedBootDevicesTest-1858653514',owner_user_name='tempest-TaggedBootDevicesTest-1858653514-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-11-29T03:07:39Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='36c915d13a4943549549177c756da93b',uuid=cb569e8d-e4a5-4076-bb8b-deeefad033e2,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "d7e92753-492b-4fc0-8048-f4792cd3b4b6", "address": "fa:16:3e:68:b2:33", "network": {"id": "cc6c7cdf-8c81-4906-9fd8-de815fea3374", "bridge": "br-int", "label": "tempest-device-tagging-net2-1248280764", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.100", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "ee1117bd3511400f9f45cc8f9bb9247d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapd7e92753-49", "ovs_interfaceid": "d7e92753-492b-4fc0-8048-f4792cd3b4b6", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Nov 29 03:08:54 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-88bf7696-da01-415d-80cd-dd7361a45c4c tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] Converting VIF {"id": "d7e92753-492b-4fc0-8048-f4792cd3b4b6", "address": "fa:16:3e:68:b2:33", "network": {"id": "cc6c7cdf-8c81-4906-9fd8-de815fea3374", "bridge": "br-int", "label": "tempest-device-tagging-net2-1248280764", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.100", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "ee1117bd3511400f9f45cc8f9bb9247d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapd7e92753-49", "ovs_interfaceid": "d7e92753-492b-4fc0-8048-f4792cd3b4b6", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:08:54 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-88bf7696-da01-415d-80cd-dd7361a45c4c tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:68:b2:33,bridge_name='br-int',has_traffic_filtering=True,id=d7e92753-492b-4fc0-8048-f4792cd3b4b6,network=Network(cc6c7cdf-8c81-4906-9fd8-de815fea3374),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapd7e92753-49') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:08:54 user nova-compute[75234]: DEBUG os_vif [None req-88bf7696-da01-415d-80cd-dd7361a45c4c tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] Unplugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:68:b2:33,bridge_name='br-int',has_traffic_filtering=True,id=d7e92753-492b-4fc0-8048-f4792cd3b4b6,network=Network(cc6c7cdf-8c81-4906-9fd8-de815fea3374),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapd7e92753-49') {{(pid=75234) unplug /opt/stack/data/venv/lib/python3.10/site-packages/os_vif/__init__.py:109}}
Nov 29 03:08:54 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:08:54 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tapd7e92753-49, bridge=br-int, if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:08:54 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:08:54 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:08:54 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:08:54 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:08:54 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbDestroyCommand(_result=None, table=QoS, record=7ec833be-07f5-430a-83d6-e92b4b25db40) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:08:54 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:08:54 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:08:54 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/8297c264-a213-44ee-be46-5c08eabda9f7/disk --force-share --output=json" returned: 0 in 0.183s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:08:54 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/8297c264-a213-44ee-be46-5c08eabda9f7/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:08:54 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:08:54 user nova-compute[75234]: INFO os_vif [None req-88bf7696-da01-415d-80cd-dd7361a45c4c tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] Successfully unplugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:68:b2:33,bridge_name='br-int',has_traffic_filtering=True,id=d7e92753-492b-4fc0-8048-f4792cd3b4b6,network=Network(cc6c7cdf-8c81-4906-9fd8-de815fea3374),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapd7e92753-49')
Nov 29 03:08:54 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-88bf7696-da01-415d-80cd-dd7361a45c4c tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='True',created_at=2023-11-29T03:06:58Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-1051091393',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-device-tagging-server-1051091393',id=10,image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBOxFHXB9/btSEa91bTsp1yfkF2UsLgRU7yInw9joeBa8Nj3E7aswz+RinNuyzQo33k8c3J3qTezyAtFU7X0/nxAqdQ5O2Sv0+97iAivTta7NtcdKSix5ZmkHgQqMMjKOGw==',key_name='tempest-keypair-1672591556',keypairs=<?>,launch_index=0,launched_at=2023-11-29T03:07:38Z,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='ee1117bd3511400f9f45cc8f9bb9247d',ramdisk_id='',reservation_id='r-936jf1h5',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-TaggedBootDevicesTest-1858653514',owner_user_name='tempest-TaggedBootDevicesTest-1858653514-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-11-29T03:07:39Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='36c915d13a4943549549177c756da93b',uuid=cb569e8d-e4a5-4076-bb8b-deeefad033e2,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "06db3012-0042-457a-a3c8-14e3a8d2e89e", "address": "fa:16:3e:27:54:30", "network": {"id": "cc6c7cdf-8c81-4906-9fd8-de815fea3374", "bridge": "br-int", "label": "tempest-device-tagging-net2-1248280764", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.200", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "ee1117bd3511400f9f45cc8f9bb9247d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap06db3012-00", "ovs_interfaceid": "06db3012-0042-457a-a3c8-14e3a8d2e89e", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Nov 29 03:08:54 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-88bf7696-da01-415d-80cd-dd7361a45c4c tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] Converting VIF {"id": "06db3012-0042-457a-a3c8-14e3a8d2e89e", "address": "fa:16:3e:27:54:30", "network": {"id": "cc6c7cdf-8c81-4906-9fd8-de815fea3374", "bridge": "br-int", "label": "tempest-device-tagging-net2-1248280764", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.200", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "ee1117bd3511400f9f45cc8f9bb9247d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap06db3012-00", "ovs_interfaceid": "06db3012-0042-457a-a3c8-14e3a8d2e89e", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:08:54 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-88bf7696-da01-415d-80cd-dd7361a45c4c tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:27:54:30,bridge_name='br-int',has_traffic_filtering=True,id=06db3012-0042-457a-a3c8-14e3a8d2e89e,network=Network(cc6c7cdf-8c81-4906-9fd8-de815fea3374),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap06db3012-00') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:08:54 user nova-compute[75234]: DEBUG os_vif [None req-88bf7696-da01-415d-80cd-dd7361a45c4c tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] Unplugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:27:54:30,bridge_name='br-int',has_traffic_filtering=True,id=06db3012-0042-457a-a3c8-14e3a8d2e89e,network=Network(cc6c7cdf-8c81-4906-9fd8-de815fea3374),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap06db3012-00') {{(pid=75234) unplug /opt/stack/data/venv/lib/python3.10/site-packages/os_vif/__init__.py:109}}
Nov 29 03:08:54 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:08:54 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tap06db3012-00, bridge=br-int, if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:08:54 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:08:54 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:08:54 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:08:54 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:08:54 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbDestroyCommand(_result=None, table=QoS, record=320e66cf-bfaa-4eb7-a6a6-636644483966) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:08:54 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:08:55 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:08:55 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:08:55 user nova-compute[75234]: INFO os_vif [None req-88bf7696-da01-415d-80cd-dd7361a45c4c tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] Successfully unplugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:27:54:30,bridge_name='br-int',has_traffic_filtering=True,id=06db3012-0042-457a-a3c8-14e3a8d2e89e,network=Network(cc6c7cdf-8c81-4906-9fd8-de815fea3374),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap06db3012-00')
Nov 29 03:08:55 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-88bf7696-da01-415d-80cd-dd7361a45c4c tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] [instance: cb569e8d-e4a5-4076-bb8b-deeefad033e2] calling os-brick to detach iSCSI Volume {{(pid=75234) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Nov 29 03:08:55 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-88bf7696-da01-415d-80cd-dd7361a45c4c tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f8a5050c9a0>, {'target_iqn': 'iqn.2010-10.org.openstack:bb98586e-4f75-46be-a626-0b73ebda11eb', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'hngCANBF', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdb'}, None), 'kwargs': True}}" {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:176}}
Nov 29 03:08:55 user nova-compute[75234]: WARNING os_brick.initiator.connectors.base [None req-88bf7696-da01-415d-80cd-dd7361a45c4c tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Nov 29 03:08:55 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-88bf7696-da01-415d-80cd-dd7361a45c4c tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:68}}
Nov 29 03:08:55 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-88bf7696-da01-415d-80cd-dd7361a45c4c tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.002s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:73}}
Nov 29 03:08:55 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:08:55 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.022s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:08:55 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[91f4065b-65c7-4284-bd90-4f515867dcf6]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:0d2c0c22-2b3a-43db-b9af-e60a2514e48e\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:ab278679-6cfd-4c7c-a930-c078843640d3\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:a4cf29d0-2ce2-423f-a20c-a7a19a452088\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:bb98586e-4f75-46be-a626-0b73ebda11eb\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:920d6cad-25ab-4f3d-941b-8ec841d43184\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:5fcd66fc-a479-4613-b1f8-e6ef12ca470a\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:b1068fd1-6636-43cd-9725-4a37f708b2bc\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:08:55 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-88bf7696-da01-415d-80cd-dd7361a45c4c tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS:
Nov 29 03:08:55 user nova-compute[75234]: No targets found.
Nov 29 03:08:55 user nova-compute[75234]: iSNS:
Nov 29 03:08:55 user nova-compute[75234]: No targets found.
Nov 29 03:08:55 user nova-compute[75234]: STATIC:
Nov 29 03:08:55 user nova-compute[75234]: Target: iqn.2010-10.org.openstack:0d2c0c22-2b3a-43db-b9af-e60a2514e48e
Nov 29 03:08:55 user nova-compute[75234]:         Portal: 172.16.0.220:3260,-1
Nov 29 03:08:55 user nova-compute[75234]:                 Iface Name: default
Nov 29 03:08:55 user nova-compute[75234]: Target: iqn.2010-10.org.openstack:ab278679-6cfd-4c7c-a930-c078843640d3
Nov 29 03:08:55 user nova-compute[75234]:         Portal: 172.16.0.220:3260,-1
Nov 29 03:08:55 user nova-compute[75234]:                 Iface Name: default
Nov 29 03:08:55 user nova-compute[75234]: Target: iqn.2010-10.org.openstack:a4cf29d0-2ce2-423f-a20c-a7a19a452088
Nov 29 03:08:55 user nova-compute[75234]:         Portal: 172.16.0.220:3260,-1
Nov 29 03:08:55 user nova-compute[75234]:                 Iface Name: default
Nov 29 03:08:55 user nova-compute[75234]: Target: iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672
Nov 29 03:08:55 user nova-compute[75234]:         Portal: 172.16.0.220:3260,-1
Nov 29 03:08:55 user nova-compute[75234]:                 Iface Name: default
Nov 29 03:08:55 user nova-compute[75234]: Target: iqn.2010-10.org.openstack:bb98586e-4f75-46be-a626-0b73ebda11eb
Nov 29 03:08:55 user nova-compute[75234]:         Portal: 172.16.0.220:3260,-1
Nov 29 03:08:55 user nova-compute[75234]:                 Iface Name: default
Nov 29 03:08:55 user nova-compute[75234]: Target: iqn.2010-10.org.openstack:920d6cad-25ab-4f3d-941b-8ec841d43184
Nov 29 03:08:55 user nova-compute[75234]:         Portal: 172.16.0.220:3260,-1
Nov 29 03:08:55 user nova-compute[75234]:                 Iface Name: default
Nov 29 03:08:55 user nova-compute[75234]: Target: iqn.2010-10.org.openstack:5fcd66fc-a479-4613-b1f8-e6ef12ca470a
Nov 29 03:08:55 user nova-compute[75234]:         Portal: 172.16.0.220:3260,-1
Nov 29 03:08:55 user nova-compute[75234]:                 Iface Name: default
Nov 29 03:08:55 user nova-compute[75234]: Target: iqn.2010-10.org.openstack:b1068fd1-6636-43cd-9725-4a37f708b2bc
Nov 29 03:08:55 user nova-compute[75234]:         Portal: 172.16.0.220:3260,-1
Nov 29 03:08:55 user nova-compute[75234]:                 Iface Name: default
Nov 29 03:08:55 user nova-compute[75234]: FIRMWARE:
Nov 29 03:08:55 user nova-compute[75234]: No targets found.
Nov 29 03:08:55 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:08:55 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-88bf7696-da01-415d-80cd-dd7361a45c4c tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] Regex to get portals from discoverydb: ^SENDTARGETS:
Nov 29 03:08:55 user nova-compute[75234]: .*?^DiscoveryAddress: 172.16.0.220,3260.*?
Nov 29 03:08:55 user nova-compute[75234]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=75234) _get_discoverydb_portals /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:382}}
Nov 29 03:08:55 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-88bf7696-da01-415d-80cd-dd7361a45c4c tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:bb98586e-4f75-46be-a626-0b73ebda11eb', 0)] {{(pid=75234) _get_connection_devices /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:830}}
Nov 29 03:08:55 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:08:55 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.020s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:08:55 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[bb597de3-78d3-4819-98c6-e2ab9eb42fed]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:0d2c0c22-2b3a-43db-b9af-e60a2514e48e\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:5fcd66fc-a479-4613-b1f8-e6ef12ca470a\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:920d6cad-25ab-4f3d-941b-8ec841d43184\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:a4cf29d0-2ce2-423f-a20c-a7a19a452088\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:ab278679-6cfd-4c7c-a930-c078843640d3\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:b1068fd1-6636-43cd-9725-4a37f708b2bc\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:bb98586e-4f75-46be-a626-0b73ebda11eb\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:08:55 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:08:55 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/8297c264-a213-44ee-be46-5c08eabda9f7/disk --force-share --output=json" returned: 0 in 0.174s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:08:55 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.029s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:08:55 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[4f23f407-d042-4576-84a7-348ad6d0a6b0]: (4, ('tcp: [11] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a4cf29d0-2ce2-423f-a20c-a7a19a452088 (non-flash)\ntcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bb98586e-4f75-46be-a626-0b73ebda11eb (non-flash)\ntcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ab278679-6cfd-4c7c-a930-c078843640d3 (non-flash)\ntcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:920d6cad-25ab-4f3d-941b-8ec841d43184 (non-flash)\ntcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b1068fd1-6636-43cd-9725-4a37f708b2bc (non-flash)\ntcp: [6] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0d2c0c22-2b3a-43db-b9af-e60a2514e48e (non-flash)\ntcp: [7] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5fcd66fc-a479-4613-b1f8-e6ef12ca470a (non-flash)\ntcp: [9] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672 (non-flash)\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:08:55 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-88bf7696-da01-415d-80cd-dd7361a45c4c tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [11] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a4cf29d0-2ce2-423f-a20c-a7a19a452088 (non-flash)
Nov 29 03:08:55 user nova-compute[75234]: tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bb98586e-4f75-46be-a626-0b73ebda11eb (non-flash)
Nov 29 03:08:55 user nova-compute[75234]: tcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ab278679-6cfd-4c7c-a930-c078843640d3 (non-flash)
Nov 29 03:08:55 user nova-compute[75234]: tcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:920d6cad-25ab-4f3d-941b-8ec841d43184 (non-flash)
Nov 29 03:08:55 user nova-compute[75234]: tcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b1068fd1-6636-43cd-9725-4a37f708b2bc (non-flash)
Nov 29 03:08:55 user nova-compute[75234]: tcp: [6] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0d2c0c22-2b3a-43db-b9af-e60a2514e48e (non-flash)
Nov 29 03:08:55 user nova-compute[75234]: tcp: [7] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5fcd66fc-a479-4613-b1f8-e6ef12ca470a (non-flash)
Nov 29 03:08:55 user nova-compute[75234]: tcp: [9] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672 (non-flash)
Nov 29 03:08:55 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:08:55 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-88bf7696-da01-415d-80cd-dd7361a45c4c tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] iscsi session list stdout=tcp: [11] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a4cf29d0-2ce2-423f-a20c-a7a19a452088 (non-flash)
Nov 29 03:08:55 user nova-compute[75234]: tcp: [2] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bb98586e-4f75-46be-a626-0b73ebda11eb (non-flash)
Nov 29 03:08:55 user nova-compute[75234]: tcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ab278679-6cfd-4c7c-a930-c078843640d3 (non-flash)
Nov 29 03:08:55 user nova-compute[75234]: tcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:920d6cad-25ab-4f3d-941b-8ec841d43184 (non-flash)
Nov 29 03:08:55 user nova-compute[75234]: tcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b1068fd1-6636-43cd-9725-4a37f708b2bc (non-flash)
Nov 29 03:08:55 user nova-compute[75234]: tcp: [6] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0d2c0c22-2b3a-43db-b9af-e60a2514e48e (non-flash)
Nov 29 03:08:55 user nova-compute[75234]: tcp: [7] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5fcd66fc-a479-4613-b1f8-e6ef12ca470a (non-flash)
Nov 29 03:08:55 user nova-compute[75234]: tcp: [9] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672 (non-flash)
Nov 29 03:08:55 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsi_session /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Nov 29 03:08:55 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-88bf7696-da01-415d-80cd-dd7361a45c4c tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7f8a6035e440>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:bb98586e-4f75-46be-a626-0b73ebda11eb'): ({'sdb'}, set())}) {{(pid=75234) _get_connection_devices /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:863}}
Nov 29 03:08:55 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-88bf7696-da01-415d-80cd-dd7361a45c4c tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] Removing single pathed devices sdb {{(pid=75234) remove_connection /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:374}}
Nov 29 03:08:55 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:08:55 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/da625f9c-b7ed-4afc-875c-6436bf58e9cc/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:08:55 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.018s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:08:55 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[20b9e2ba-2e43-40da-8f70-89de85f8c74e]: (4, ('path checker states:\nup                  8\n\npaths: 0\nbusy: False\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:08:55 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdb {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:08:55 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdb" returned: 0 in 0.018s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:08:55 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[0f85f77e-697a-4067-b5a4-73f8ff690d29]: (4, ('ok\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:08:55 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-88bf7696-da01-415d-80cd-dd7361a45c4c tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] Flushing IO for device /dev/sdb {{(pid=75234) flush_device_io /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:434}}
Nov 29 03:08:55 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdb {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:08:55 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdb" returned: 0 in 0.012s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:08:55 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[f121b36c-54f1-4167-8c07-2684e61d792f]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:08:55 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-88bf7696-da01-415d-80cd-dd7361a45c4c tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] Remove SCSI device /dev/sdb with /sys/block/sdb/device/delete {{(pid=75234) remove_scsi_device /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:148}}
Nov 29 03:08:55 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdb/device/delete {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:08:55 user nova-compute[75234]: DEBUG nova.compute.manager [req-534775a7-03ed-494d-8dd2-5f1f4a65d632 req-74ef00b4-c362-439d-b905-ab40703e9f9f service nova] [instance: cb569e8d-e4a5-4076-bb8b-deeefad033e2] Received event network-vif-plugged-026f606a-534c-4cdd-992a-9f33268b374f {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:08:55 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-534775a7-03ed-494d-8dd2-5f1f4a65d632 req-74ef00b4-c362-439d-b905-ab40703e9f9f service nova] Acquiring lock "cb569e8d-e4a5-4076-bb8b-deeefad033e2-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:08:55 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-534775a7-03ed-494d-8dd2-5f1f4a65d632 req-74ef00b4-c362-439d-b905-ab40703e9f9f service nova] Lock "cb569e8d-e4a5-4076-bb8b-deeefad033e2-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:08:55 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-534775a7-03ed-494d-8dd2-5f1f4a65d632 req-74ef00b4-c362-439d-b905-ab40703e9f9f service nova] Lock "cb569e8d-e4a5-4076-bb8b-deeefad033e2-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:08:55 user nova-compute[75234]: DEBUG nova.compute.manager [req-534775a7-03ed-494d-8dd2-5f1f4a65d632 req-74ef00b4-c362-439d-b905-ab40703e9f9f service nova] [instance: cb569e8d-e4a5-4076-bb8b-deeefad033e2] No waiting events found dispatching network-vif-plugged-026f606a-534c-4cdd-992a-9f33268b374f {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:08:55 user nova-compute[75234]: WARNING nova.compute.manager [req-534775a7-03ed-494d-8dd2-5f1f4a65d632 req-74ef00b4-c362-439d-b905-ab40703e9f9f service nova] [instance: cb569e8d-e4a5-4076-bb8b-deeefad033e2] Received unexpected event network-vif-plugged-026f606a-534c-4cdd-992a-9f33268b374f for instance with vm_state active and task_state deleting.
Nov 29 03:08:55 user nova-compute[75234]: DEBUG nova.compute.manager [req-534775a7-03ed-494d-8dd2-5f1f4a65d632 req-74ef00b4-c362-439d-b905-ab40703e9f9f service nova] [instance: cb569e8d-e4a5-4076-bb8b-deeefad033e2] Received event network-vif-unplugged-67446920-3741-4542-84c6-493388395f26 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:08:55 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-534775a7-03ed-494d-8dd2-5f1f4a65d632 req-74ef00b4-c362-439d-b905-ab40703e9f9f service nova] Acquiring lock "cb569e8d-e4a5-4076-bb8b-deeefad033e2-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:08:55 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-534775a7-03ed-494d-8dd2-5f1f4a65d632 req-74ef00b4-c362-439d-b905-ab40703e9f9f service nova] Lock "cb569e8d-e4a5-4076-bb8b-deeefad033e2-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:08:55 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-534775a7-03ed-494d-8dd2-5f1f4a65d632 req-74ef00b4-c362-439d-b905-ab40703e9f9f service nova] Lock "cb569e8d-e4a5-4076-bb8b-deeefad033e2-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:08:55 user nova-compute[75234]: DEBUG nova.compute.manager [req-534775a7-03ed-494d-8dd2-5f1f4a65d632 req-74ef00b4-c362-439d-b905-ab40703e9f9f service nova] [instance: cb569e8d-e4a5-4076-bb8b-deeefad033e2] No waiting events found dispatching network-vif-unplugged-67446920-3741-4542-84c6-493388395f26 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:08:55 user nova-compute[75234]: DEBUG nova.compute.manager [req-534775a7-03ed-494d-8dd2-5f1f4a65d632 req-74ef00b4-c362-439d-b905-ab40703e9f9f service nova] [instance: cb569e8d-e4a5-4076-bb8b-deeefad033e2] Received event network-vif-unplugged-67446920-3741-4542-84c6-493388395f26 for instance with task_state deleting. {{(pid=75234) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10890}}
Nov 29 03:08:55 user nova-compute[75234]: DEBUG nova.compute.manager [req-534775a7-03ed-494d-8dd2-5f1f4a65d632 req-74ef00b4-c362-439d-b905-ab40703e9f9f service nova] [instance: cb569e8d-e4a5-4076-bb8b-deeefad033e2] Received event network-vif-plugged-67446920-3741-4542-84c6-493388395f26 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:08:55 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-534775a7-03ed-494d-8dd2-5f1f4a65d632 req-74ef00b4-c362-439d-b905-ab40703e9f9f service nova] Acquiring lock "cb569e8d-e4a5-4076-bb8b-deeefad033e2-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:08:55 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-534775a7-03ed-494d-8dd2-5f1f4a65d632 req-74ef00b4-c362-439d-b905-ab40703e9f9f service nova] Lock "cb569e8d-e4a5-4076-bb8b-deeefad033e2-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:08:55 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-534775a7-03ed-494d-8dd2-5f1f4a65d632 req-74ef00b4-c362-439d-b905-ab40703e9f9f service nova] Lock "cb569e8d-e4a5-4076-bb8b-deeefad033e2-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:08:55 user nova-compute[75234]: DEBUG nova.compute.manager [req-534775a7-03ed-494d-8dd2-5f1f4a65d632 req-74ef00b4-c362-439d-b905-ab40703e9f9f service nova] [instance: cb569e8d-e4a5-4076-bb8b-deeefad033e2] No waiting events found dispatching network-vif-plugged-67446920-3741-4542-84c6-493388395f26 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:08:55 user nova-compute[75234]: WARNING nova.compute.manager [req-534775a7-03ed-494d-8dd2-5f1f4a65d632 req-74ef00b4-c362-439d-b905-ab40703e9f9f service nova] [instance: cb569e8d-e4a5-4076-bb8b-deeefad033e2] Received unexpected event network-vif-plugged-67446920-3741-4542-84c6-493388395f26 for instance with vm_state active and task_state deleting.
Nov 29 03:08:55 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdb/device/delete" returned: 0 in 0.055s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:08:55 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[67a9dc16-510b-42c4-8596-7e537a42aa3f]: (4, ('1', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:08:55 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-88bf7696-da01-415d-80cd-dd7361a45c4c tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] Checking to see if SCSI volumes sdb have been removed. {{(pid=75234) wait_for_volumes_removal /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:156}}
Nov 29 03:08:55 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-88bf7696-da01-415d-80cd-dd7361a45c4c tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] SCSI volumes sdb have been removed. {{(pid=75234) wait_for_volumes_removal /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:166}}
Nov 29 03:08:55 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-88bf7696-da01-415d-80cd-dd7361a45c4c tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:bb98586e-4f75-46be-a626-0b73ebda11eb')] {{(pid=75234) _disconnect_connection /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Nov 29 03:08:55 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:bb98586e-4f75-46be-a626-0b73ebda11eb -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:08:55 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:bb98586e-4f75-46be-a626-0b73ebda11eb -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.020s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:08:55 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[50892927-324f-4e70-b69c-55c861b389f0]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:08:55 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-88bf7696-da01-415d-80cd-dd7361a45c4c tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:08:55 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:bb98586e-4f75-46be-a626-0b73ebda11eb -p 172.16.0.220:3260 --logout {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:08:55 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/da625f9c-b7ed-4afc-875c-6436bf58e9cc/disk --force-share --output=json" returned: 0 in 0.185s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:08:55 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/da625f9c-b7ed-4afc-875c-6436bf58e9cc/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:08:55 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:bb98586e-4f75-46be-a626-0b73ebda11eb -p 172.16.0.220:3260 --logout" returned: 0 in 0.071s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:08:55 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[322b2671-daaa-4e9e-ad3f-d6c12b4b3e0b]: (4, ('Logging out of session [sid: 2, target: iqn.2010-10.org.openstack:bb98586e-4f75-46be-a626-0b73ebda11eb, portal: 172.16.0.220,3260]\nLogout of [sid: 2, target: iqn.2010-10.org.openstack:bb98586e-4f75-46be-a626-0b73ebda11eb, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:08:55 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-88bf7696-da01-415d-80cd-dd7361a45c4c tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 2, target: iqn.2010-10.org.openstack:bb98586e-4f75-46be-a626-0b73ebda11eb, portal: 172.16.0.220,3260]
Nov 29 03:08:55 user nova-compute[75234]: Logout of [sid: 2, target: iqn.2010-10.org.openstack:bb98586e-4f75-46be-a626-0b73ebda11eb, portal: 172.16.0.220,3260] successful.
Nov 29 03:08:55 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:08:55 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:bb98586e-4f75-46be-a626-0b73ebda11eb -p 172.16.0.220:3260 --op delete {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:08:55 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:bb98586e-4f75-46be-a626-0b73ebda11eb -p 172.16.0.220:3260 --op delete" returned: 0 in 0.033s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:08:55 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[337d9fb5-03c0-410c-b0bd-42e08fbb301e]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:08:55 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-88bf7696-da01-415d-80cd-dd7361a45c4c tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:08:55 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-88bf7696-da01-415d-80cd-dd7361a45c4c tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.387s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:87}}
Nov 29 03:08:55 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-88bf7696-da01-415d-80cd-dd7361a45c4c tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] <== disconnect_volume: return (389ms) None {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:203}}
Nov 29 03:08:55 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-88bf7696-da01-415d-80cd-dd7361a45c4c tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] [instance: cb569e8d-e4a5-4076-bb8b-deeefad033e2] Disconnected iSCSI Volume {{(pid=75234) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:79}}
Nov 29 03:08:55 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-88bf7696-da01-415d-80cd-dd7361a45c4c tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] [instance: cb569e8d-e4a5-4076-bb8b-deeefad033e2] calling os-brick to detach iSCSI Volume {{(pid=75234) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Nov 29 03:08:55 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-88bf7696-da01-415d-80cd-dd7361a45c4c tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f8a5050c9a0>, {'target_iqn': 'iqn.2010-10.org.openstack:ab278679-6cfd-4c7c-a930-c078843640d3', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'JhsSAlap', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdc'}, None), 'kwargs': True}}" {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:176}}
Nov 29 03:08:55 user nova-compute[75234]: WARNING os_brick.initiator.connectors.base [None req-88bf7696-da01-415d-80cd-dd7361a45c4c tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Nov 29 03:08:55 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-88bf7696-da01-415d-80cd-dd7361a45c4c tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:68}}
Nov 29 03:08:55 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-88bf7696-da01-415d-80cd-dd7361a45c4c tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.002s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:73}}
Nov 29 03:08:55 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:08:55 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.021s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:08:55 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[16bc0940-fd3a-48d9-8c05-02424674e86b]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:0d2c0c22-2b3a-43db-b9af-e60a2514e48e\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:ab278679-6cfd-4c7c-a930-c078843640d3\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:a4cf29d0-2ce2-423f-a20c-a7a19a452088\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:920d6cad-25ab-4f3d-941b-8ec841d43184\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:5fcd66fc-a479-4613-b1f8-e6ef12ca470a\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:b1068fd1-6636-43cd-9725-4a37f708b2bc\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:08:55 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-88bf7696-da01-415d-80cd-dd7361a45c4c tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS:
Nov 29 03:08:55 user nova-compute[75234]: No targets found.
Nov 29 03:08:55 user nova-compute[75234]: iSNS:
Nov 29 03:08:55 user nova-compute[75234]: No targets found.
Nov 29 03:08:55 user nova-compute[75234]: STATIC:
Nov 29 03:08:55 user nova-compute[75234]: Target: iqn.2010-10.org.openstack:0d2c0c22-2b3a-43db-b9af-e60a2514e48e
Nov 29 03:08:55 user nova-compute[75234]:         Portal: 172.16.0.220:3260,-1
Nov 29 03:08:55 user nova-compute[75234]:                 Iface Name: default
Nov 29 03:08:55 user nova-compute[75234]: Target: iqn.2010-10.org.openstack:ab278679-6cfd-4c7c-a930-c078843640d3
Nov 29 03:08:55 user nova-compute[75234]:         Portal: 172.16.0.220:3260,-1
Nov 29 03:08:55 user nova-compute[75234]:                 Iface Name: default
Nov 29 03:08:55 user nova-compute[75234]: Target: iqn.2010-10.org.openstack:a4cf29d0-2ce2-423f-a20c-a7a19a452088
Nov 29 03:08:55 user nova-compute[75234]:         Portal: 172.16.0.220:3260,-1
Nov 29 03:08:55 user nova-compute[75234]:                 Iface Name: default
Nov 29 03:08:55 user nova-compute[75234]: Target: iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672
Nov 29 03:08:55 user nova-compute[75234]:         Portal: 172.16.0.220:3260,-1
Nov 29 03:08:55 user nova-compute[75234]:                 Iface Name: default
Nov 29 03:08:55 user nova-compute[75234]: Target: iqn.2010-10.org.openstack:920d6cad-25ab-4f3d-941b-8ec841d43184
Nov 29 03:08:55 user nova-compute[75234]:         Portal: 172.16.0.220:3260,-1
Nov 29 03:08:55 user nova-compute[75234]:                 Iface Name: default
Nov 29 03:08:55 user nova-compute[75234]: Target: iqn.2010-10.org.openstack:5fcd66fc-a479-4613-b1f8-e6ef12ca470a
Nov 29 03:08:55 user nova-compute[75234]:         Portal: 172.16.0.220:3260,-1
Nov 29 03:08:55 user nova-compute[75234]:                 Iface Name: default
Nov 29 03:08:55 user nova-compute[75234]: Target: iqn.2010-10.org.openstack:b1068fd1-6636-43cd-9725-4a37f708b2bc
Nov 29 03:08:55 user nova-compute[75234]:         Portal: 172.16.0.220:3260,-1
Nov 29 03:08:55 user nova-compute[75234]:                 Iface Name: default
Nov 29 03:08:55 user nova-compute[75234]: FIRMWARE:
Nov 29 03:08:55 user nova-compute[75234]: No targets found.
Nov 29 03:08:55 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:08:55 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-88bf7696-da01-415d-80cd-dd7361a45c4c tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] Regex to get portals from discoverydb: ^SENDTARGETS:
Nov 29 03:08:55 user nova-compute[75234]: .*?^DiscoveryAddress: 172.16.0.220,3260.*?
Nov 29 03:08:55 user nova-compute[75234]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=75234) _get_discoverydb_portals /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:382}}
Nov 29 03:08:55 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-88bf7696-da01-415d-80cd-dd7361a45c4c tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:ab278679-6cfd-4c7c-a930-c078843640d3', 0)] {{(pid=75234) _get_connection_devices /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:830}}
Nov 29 03:08:55 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:08:55 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.019s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:08:55 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[f4f74a06-2a6f-4e18-96c7-251009c2a8fe]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:0d2c0c22-2b3a-43db-b9af-e60a2514e48e\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:5fcd66fc-a479-4613-b1f8-e6ef12ca470a\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:920d6cad-25ab-4f3d-941b-8ec841d43184\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:a4cf29d0-2ce2-423f-a20c-a7a19a452088\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:ab278679-6cfd-4c7c-a930-c078843640d3\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:b1068fd1-6636-43cd-9725-4a37f708b2bc\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:08:55 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:08:55 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/da625f9c-b7ed-4afc-875c-6436bf58e9cc/disk --force-share --output=json" returned: 0 in 0.174s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:08:55 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2bf9fe41-63c2-4882-bcdd-8e314e734ffe/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:08:55 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.027s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:08:55 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[3776f187-97b5-48b6-8cf9-21468c00038d]: (4, ('tcp: [11] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a4cf29d0-2ce2-423f-a20c-a7a19a452088 (non-flash)\ntcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ab278679-6cfd-4c7c-a930-c078843640d3 (non-flash)\ntcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:920d6cad-25ab-4f3d-941b-8ec841d43184 (non-flash)\ntcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b1068fd1-6636-43cd-9725-4a37f708b2bc (non-flash)\ntcp: [6] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0d2c0c22-2b3a-43db-b9af-e60a2514e48e (non-flash)\ntcp: [7] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5fcd66fc-a479-4613-b1f8-e6ef12ca470a (non-flash)\ntcp: [9] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672 (non-flash)\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:08:55 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-88bf7696-da01-415d-80cd-dd7361a45c4c tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [11] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a4cf29d0-2ce2-423f-a20c-a7a19a452088 (non-flash)
Nov 29 03:08:55 user nova-compute[75234]: tcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ab278679-6cfd-4c7c-a930-c078843640d3 (non-flash)
Nov 29 03:08:55 user nova-compute[75234]: tcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:920d6cad-25ab-4f3d-941b-8ec841d43184 (non-flash)
Nov 29 03:08:55 user nova-compute[75234]: tcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b1068fd1-6636-43cd-9725-4a37f708b2bc (non-flash)
Nov 29 03:08:55 user nova-compute[75234]: tcp: [6] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0d2c0c22-2b3a-43db-b9af-e60a2514e48e (non-flash)
Nov 29 03:08:55 user nova-compute[75234]: tcp: [7] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5fcd66fc-a479-4613-b1f8-e6ef12ca470a (non-flash)
Nov 29 03:08:55 user nova-compute[75234]: tcp: [9] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672 (non-flash)
Nov 29 03:08:55 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:08:55 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-88bf7696-da01-415d-80cd-dd7361a45c4c tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] iscsi session list stdout=tcp: [11] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a4cf29d0-2ce2-423f-a20c-a7a19a452088 (non-flash)
Nov 29 03:08:55 user nova-compute[75234]: tcp: [3] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ab278679-6cfd-4c7c-a930-c078843640d3 (non-flash)
Nov 29 03:08:55 user nova-compute[75234]: tcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:920d6cad-25ab-4f3d-941b-8ec841d43184 (non-flash)
Nov 29 03:08:55 user nova-compute[75234]: tcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b1068fd1-6636-43cd-9725-4a37f708b2bc (non-flash)
Nov 29 03:08:55 user nova-compute[75234]: tcp: [6] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0d2c0c22-2b3a-43db-b9af-e60a2514e48e (non-flash)
Nov 29 03:08:55 user nova-compute[75234]: tcp: [7] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5fcd66fc-a479-4613-b1f8-e6ef12ca470a (non-flash)
Nov 29 03:08:55 user nova-compute[75234]: tcp: [9] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672 (non-flash)
Nov 29 03:08:55 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsi_session /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Nov 29 03:08:55 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-88bf7696-da01-415d-80cd-dd7361a45c4c tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7f8a5032eb00>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:ab278679-6cfd-4c7c-a930-c078843640d3'): ({'sdc'}, set())}) {{(pid=75234) _get_connection_devices /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:863}}
Nov 29 03:08:55 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-88bf7696-da01-415d-80cd-dd7361a45c4c tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] Removing single pathed devices sdc {{(pid=75234) remove_connection /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:374}}
Nov 29 03:08:55 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:08:55 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.018s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:08:55 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[b252cd38-6951-47f8-afac-a9daa5e9f3e0]: (4, ('path checker states:\nup                  7\n\npaths: 0\nbusy: False\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:08:55 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdc {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:08:55 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdc" returned: 0 in 0.017s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:08:55 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[128bf3d9-2090-427e-a777-027dd958b6b4]: (4, ('ok\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:08:55 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-88bf7696-da01-415d-80cd-dd7361a45c4c tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] Flushing IO for device /dev/sdc {{(pid=75234) flush_device_io /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:434}}
Nov 29 03:08:55 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdc {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:08:55 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdc" returned: 0 in 0.016s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:08:55 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[235e82c2-b2a6-486d-9b59-25f85818ff04]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:08:55 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-88bf7696-da01-415d-80cd-dd7361a45c4c tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] Remove SCSI device /dev/sdc with /sys/block/sdc/device/delete {{(pid=75234) remove_scsi_device /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:148}}
Nov 29 03:08:55 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdc/device/delete {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:08:55 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdc/device/delete" returned: 0 in 0.029s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:08:55 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[f5922950-6566-4906-8b3c-9a5bc80812d8]: (4, ('1', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:08:55 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-88bf7696-da01-415d-80cd-dd7361a45c4c tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] Checking to see if SCSI volumes sdc have been removed. {{(pid=75234) wait_for_volumes_removal /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:156}}
Nov 29 03:08:55 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-88bf7696-da01-415d-80cd-dd7361a45c4c tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] SCSI volumes sdc have been removed. {{(pid=75234) wait_for_volumes_removal /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:166}}
Nov 29 03:08:55 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-88bf7696-da01-415d-80cd-dd7361a45c4c tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:ab278679-6cfd-4c7c-a930-c078843640d3')] {{(pid=75234) _disconnect_connection /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Nov 29 03:08:55 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:ab278679-6cfd-4c7c-a930-c078843640d3 -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:08:55 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:ab278679-6cfd-4c7c-a930-c078843640d3 -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.024s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:08:55 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[6a953023-6915-42ad-aaa6-a3ffcfb0d15d]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:08:55 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-88bf7696-da01-415d-80cd-dd7361a45c4c tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:08:55 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:ab278679-6cfd-4c7c-a930-c078843640d3 -p 172.16.0.220:3260 --logout {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:08:55 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2bf9fe41-63c2-4882-bcdd-8e314e734ffe/disk --force-share --output=json" returned: 0 in 0.210s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:08:55 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2bf9fe41-63c2-4882-bcdd-8e314e734ffe/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:08:55 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:ab278679-6cfd-4c7c-a930-c078843640d3 -p 172.16.0.220:3260 --logout" returned: 0 in 0.073s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:08:55 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[889b6752-11a3-4ac6-aea3-5c5f57131702]: (4, ('Logging out of session [sid: 3, target: iqn.2010-10.org.openstack:ab278679-6cfd-4c7c-a930-c078843640d3, portal: 172.16.0.220,3260]\nLogout of [sid: 3, target: iqn.2010-10.org.openstack:ab278679-6cfd-4c7c-a930-c078843640d3, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:08:55 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-88bf7696-da01-415d-80cd-dd7361a45c4c tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 3, target: iqn.2010-10.org.openstack:ab278679-6cfd-4c7c-a930-c078843640d3, portal: 172.16.0.220,3260]
Nov 29 03:08:55 user nova-compute[75234]: Logout of [sid: 3, target: iqn.2010-10.org.openstack:ab278679-6cfd-4c7c-a930-c078843640d3, portal: 172.16.0.220,3260] successful.
Nov 29 03:08:55 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:08:55 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:ab278679-6cfd-4c7c-a930-c078843640d3 -p 172.16.0.220:3260 --op delete {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:08:55 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:ab278679-6cfd-4c7c-a930-c078843640d3 -p 172.16.0.220:3260 --op delete" returned: 0 in 0.035s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:08:55 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[f0e0ab6d-c677-4c1b-91a9-25cd7e1af480]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:08:55 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-88bf7696-da01-415d-80cd-dd7361a45c4c tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:08:55 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-88bf7696-da01-415d-80cd-dd7361a45c4c tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.375s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:87}}
Nov 29 03:08:55 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-88bf7696-da01-415d-80cd-dd7361a45c4c tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] <== disconnect_volume: return (377ms) None {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:203}}
Nov 29 03:08:55 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-88bf7696-da01-415d-80cd-dd7361a45c4c tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] [instance: cb569e8d-e4a5-4076-bb8b-deeefad033e2] Disconnected iSCSI Volume {{(pid=75234) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:79}}
Nov 29 03:08:55 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-88bf7696-da01-415d-80cd-dd7361a45c4c tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] [instance: cb569e8d-e4a5-4076-bb8b-deeefad033e2] calling os-brick to detach iSCSI Volume {{(pid=75234) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Nov 29 03:08:55 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-88bf7696-da01-415d-80cd-dd7361a45c4c tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f8a5050c9a0>, {'target_iqn': 'iqn.2010-10.org.openstack:920d6cad-25ab-4f3d-941b-8ec841d43184', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'fynrLYlU', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdd'}, None), 'kwargs': True}}" {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:176}}
Nov 29 03:08:55 user nova-compute[75234]: WARNING os_brick.initiator.connectors.base [None req-88bf7696-da01-415d-80cd-dd7361a45c4c tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Nov 29 03:08:55 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-88bf7696-da01-415d-80cd-dd7361a45c4c tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:68}}
Nov 29 03:08:55 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-88bf7696-da01-415d-80cd-dd7361a45c4c tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.002s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:73}}
Nov 29 03:08:55 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:08:55 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.025s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:08:55 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[747204b6-62d6-48c7-b18d-8a1336cde08f]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:0d2c0c22-2b3a-43db-b9af-e60a2514e48e\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:a4cf29d0-2ce2-423f-a20c-a7a19a452088\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:920d6cad-25ab-4f3d-941b-8ec841d43184\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:5fcd66fc-a479-4613-b1f8-e6ef12ca470a\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:b1068fd1-6636-43cd-9725-4a37f708b2bc\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:08:55 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-88bf7696-da01-415d-80cd-dd7361a45c4c tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS:
Nov 29 03:08:55 user nova-compute[75234]: No targets found.
Nov 29 03:08:55 user nova-compute[75234]: iSNS:
Nov 29 03:08:55 user nova-compute[75234]: No targets found.
Nov 29 03:08:55 user nova-compute[75234]: STATIC:
Nov 29 03:08:55 user nova-compute[75234]: Target: iqn.2010-10.org.openstack:0d2c0c22-2b3a-43db-b9af-e60a2514e48e
Nov 29 03:08:55 user nova-compute[75234]:         Portal: 172.16.0.220:3260,-1
Nov 29 03:08:55 user nova-compute[75234]:                 Iface Name: default
Nov 29 03:08:55 user nova-compute[75234]: Target: iqn.2010-10.org.openstack:a4cf29d0-2ce2-423f-a20c-a7a19a452088
Nov 29 03:08:55 user nova-compute[75234]:         Portal: 172.16.0.220:3260,-1
Nov 29 03:08:55 user nova-compute[75234]:                 Iface Name: default
Nov 29 03:08:55 user nova-compute[75234]: Target: iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672
Nov 29 03:08:55 user nova-compute[75234]:         Portal: 172.16.0.220:3260,-1
Nov 29 03:08:55 user nova-compute[75234]:                 Iface Name: default
Nov 29 03:08:55 user nova-compute[75234]: Target: iqn.2010-10.org.openstack:920d6cad-25ab-4f3d-941b-8ec841d43184
Nov 29 03:08:55 user nova-compute[75234]:         Portal: 172.16.0.220:3260,-1
Nov 29 03:08:55 user nova-compute[75234]:                 Iface Name: default
Nov 29 03:08:55 user nova-compute[75234]: Target: iqn.2010-10.org.openstack:5fcd66fc-a479-4613-b1f8-e6ef12ca470a
Nov 29 03:08:55 user nova-compute[75234]:         Portal: 172.16.0.220:3260,-1
Nov 29 03:08:55 user nova-compute[75234]:                 Iface Name: default
Nov 29 03:08:55 user nova-compute[75234]: Target: iqn.2010-10.org.openstack:b1068fd1-6636-43cd-9725-4a37f708b2bc
Nov 29 03:08:55 user nova-compute[75234]:         Portal: 172.16.0.220:3260,-1
Nov 29 03:08:55 user nova-compute[75234]:                 Iface Name: default
Nov 29 03:08:55 user nova-compute[75234]: FIRMWARE:
Nov 29 03:08:55 user nova-compute[75234]: No targets found.
Nov 29 03:08:55 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:08:55 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-88bf7696-da01-415d-80cd-dd7361a45c4c tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] Regex to get portals from discoverydb: ^SENDTARGETS:
Nov 29 03:08:55 user nova-compute[75234]: .*?^DiscoveryAddress: 172.16.0.220,3260.*?
Nov 29 03:08:55 user nova-compute[75234]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=75234) _get_discoverydb_portals /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:382}}
Nov 29 03:08:55 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-88bf7696-da01-415d-80cd-dd7361a45c4c tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:920d6cad-25ab-4f3d-941b-8ec841d43184', 0)] {{(pid=75234) _get_connection_devices /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:830}}
Nov 29 03:08:55 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:08:55 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.018s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:08:55 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[051f83c1-06db-42d8-b74c-4a0d4c75dbdb]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:0d2c0c22-2b3a-43db-b9af-e60a2514e48e\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:5fcd66fc-a479-4613-b1f8-e6ef12ca470a\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:920d6cad-25ab-4f3d-941b-8ec841d43184\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:a4cf29d0-2ce2-423f-a20c-a7a19a452088\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:b1068fd1-6636-43cd-9725-4a37f708b2bc\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:08:55 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:08:55 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2bf9fe41-63c2-4882-bcdd-8e314e734ffe/disk --force-share --output=json" returned: 0 in 0.177s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:08:55 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/21e9c6cd-c39f-41c0-8768-5dd64589c3f3/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:08:55 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.025s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:08:55 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[8d1ec50b-ecb3-4ce2-b373-7cbdf52f174d]: (4, ('tcp: [11] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a4cf29d0-2ce2-423f-a20c-a7a19a452088 (non-flash)\ntcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:920d6cad-25ab-4f3d-941b-8ec841d43184 (non-flash)\ntcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b1068fd1-6636-43cd-9725-4a37f708b2bc (non-flash)\ntcp: [6] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0d2c0c22-2b3a-43db-b9af-e60a2514e48e (non-flash)\ntcp: [7] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5fcd66fc-a479-4613-b1f8-e6ef12ca470a (non-flash)\ntcp: [9] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672 (non-flash)\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:08:55 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-88bf7696-da01-415d-80cd-dd7361a45c4c tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [11] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a4cf29d0-2ce2-423f-a20c-a7a19a452088 (non-flash)
Nov 29 03:08:55 user nova-compute[75234]: tcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:920d6cad-25ab-4f3d-941b-8ec841d43184 (non-flash)
Nov 29 03:08:55 user nova-compute[75234]: tcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b1068fd1-6636-43cd-9725-4a37f708b2bc (non-flash)
Nov 29 03:08:55 user nova-compute[75234]: tcp: [6] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0d2c0c22-2b3a-43db-b9af-e60a2514e48e (non-flash)
Nov 29 03:08:55 user nova-compute[75234]: tcp: [7] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5fcd66fc-a479-4613-b1f8-e6ef12ca470a (non-flash)
Nov 29 03:08:55 user nova-compute[75234]: tcp: [9] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672 (non-flash)
Nov 29 03:08:55 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:08:55 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-88bf7696-da01-415d-80cd-dd7361a45c4c tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] iscsi session list stdout=tcp: [11] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a4cf29d0-2ce2-423f-a20c-a7a19a452088 (non-flash)
Nov 29 03:08:55 user nova-compute[75234]: tcp: [4] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:920d6cad-25ab-4f3d-941b-8ec841d43184 (non-flash)
Nov 29 03:08:55 user nova-compute[75234]: tcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b1068fd1-6636-43cd-9725-4a37f708b2bc (non-flash)
Nov 29 03:08:55 user nova-compute[75234]: tcp: [6] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0d2c0c22-2b3a-43db-b9af-e60a2514e48e (non-flash)
Nov 29 03:08:55 user nova-compute[75234]: tcp: [7] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5fcd66fc-a479-4613-b1f8-e6ef12ca470a (non-flash)
Nov 29 03:08:55 user nova-compute[75234]: tcp: [9] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672 (non-flash)
Nov 29 03:08:55 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsi_session /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Nov 29 03:08:55 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-88bf7696-da01-415d-80cd-dd7361a45c4c tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7f8a5032eb00>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:920d6cad-25ab-4f3d-941b-8ec841d43184'): ({'sdd'}, set())}) {{(pid=75234) _get_connection_devices /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:863}}
Nov 29 03:08:55 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-88bf7696-da01-415d-80cd-dd7361a45c4c tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] Removing single pathed devices sdd {{(pid=75234) remove_connection /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:374}}
Nov 29 03:08:55 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:08:55 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.019s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:08:55 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[bcd1b0cc-ae66-42bd-ad77-9e6054ef9e19]: (4, ('path checker states:\nup                  6\n\npaths: 0\nbusy: False\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:08:55 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdd {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:08:55 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdd" returned: 0 in 0.019s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:08:55 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[7e90388a-c8ff-4dcc-92bf-c2df11691909]: (4, ('ok\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:08:55 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-88bf7696-da01-415d-80cd-dd7361a45c4c tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] Flushing IO for device /dev/sdd {{(pid=75234) flush_device_io /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:434}}
Nov 29 03:08:55 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdd {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:08:55 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdd" returned: 0 in 0.012s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:08:55 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[5c575837-4769-4cb9-8065-2b43f5a26fe4]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:08:55 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-88bf7696-da01-415d-80cd-dd7361a45c4c tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] Remove SCSI device /dev/sdd with /sys/block/sdd/device/delete {{(pid=75234) remove_scsi_device /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:148}}
Nov 29 03:08:55 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdd/device/delete {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:08:56 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdd/device/delete" returned: 0 in 0.036s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:08:56 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[bd28eb86-8ad0-4f20-9c3a-e4a7abd09ac0]: (4, ('1', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:08:56 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-88bf7696-da01-415d-80cd-dd7361a45c4c tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] Checking to see if SCSI volumes sdd have been removed. {{(pid=75234) wait_for_volumes_removal /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:156}}
Nov 29 03:08:56 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-88bf7696-da01-415d-80cd-dd7361a45c4c tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] SCSI volumes sdd have been removed. {{(pid=75234) wait_for_volumes_removal /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:166}}
Nov 29 03:08:56 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-88bf7696-da01-415d-80cd-dd7361a45c4c tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:920d6cad-25ab-4f3d-941b-8ec841d43184')] {{(pid=75234) _disconnect_connection /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Nov 29 03:08:56 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:920d6cad-25ab-4f3d-941b-8ec841d43184 -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:08:56 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:920d6cad-25ab-4f3d-941b-8ec841d43184 -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.018s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:08:56 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[121e61f1-d71d-4609-9b75-cff661ae758c]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:08:56 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-88bf7696-da01-415d-80cd-dd7361a45c4c tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:08:56 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:920d6cad-25ab-4f3d-941b-8ec841d43184 -p 172.16.0.220:3260 --logout {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:08:56 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/21e9c6cd-c39f-41c0-8768-5dd64589c3f3/disk --force-share --output=json" returned: 0 in 0.178s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:08:56 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/21e9c6cd-c39f-41c0-8768-5dd64589c3f3/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:08:56 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:920d6cad-25ab-4f3d-941b-8ec841d43184 -p 172.16.0.220:3260 --logout" returned: 0 in 0.069s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:08:56 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[369289e3-e36e-4155-9bb5-c1450c9766b5]: (4, ('Logging out of session [sid: 4, target: iqn.2010-10.org.openstack:920d6cad-25ab-4f3d-941b-8ec841d43184, portal: 172.16.0.220,3260]\nLogout of [sid: 4, target: iqn.2010-10.org.openstack:920d6cad-25ab-4f3d-941b-8ec841d43184, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:08:56 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-88bf7696-da01-415d-80cd-dd7361a45c4c tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 4, target: iqn.2010-10.org.openstack:920d6cad-25ab-4f3d-941b-8ec841d43184, portal: 172.16.0.220,3260]
Nov 29 03:08:56 user nova-compute[75234]: Logout of [sid: 4, target: iqn.2010-10.org.openstack:920d6cad-25ab-4f3d-941b-8ec841d43184, portal: 172.16.0.220,3260] successful.
Nov 29 03:08:56 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:08:56 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:920d6cad-25ab-4f3d-941b-8ec841d43184 -p 172.16.0.220:3260 --op delete {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:08:56 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:920d6cad-25ab-4f3d-941b-8ec841d43184 -p 172.16.0.220:3260 --op delete" returned: 0 in 0.029s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:08:56 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[3827e34f-2d90-4621-bd04-325a690f34c0]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:08:56 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-88bf7696-da01-415d-80cd-dd7361a45c4c tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:08:56 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-88bf7696-da01-415d-80cd-dd7361a45c4c tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.358s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:87}}
Nov 29 03:08:56 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-88bf7696-da01-415d-80cd-dd7361a45c4c tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] <== disconnect_volume: return (359ms) None {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:203}}
Nov 29 03:08:56 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-88bf7696-da01-415d-80cd-dd7361a45c4c tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] [instance: cb569e8d-e4a5-4076-bb8b-deeefad033e2] Disconnected iSCSI Volume {{(pid=75234) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:79}}
Nov 29 03:08:56 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-88bf7696-da01-415d-80cd-dd7361a45c4c tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] [instance: cb569e8d-e4a5-4076-bb8b-deeefad033e2] Deleting instance files /opt/stack/data/nova/instances/cb569e8d-e4a5-4076-bb8b-deeefad033e2_del
Nov 29 03:08:56 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-88bf7696-da01-415d-80cd-dd7361a45c4c tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] [instance: cb569e8d-e4a5-4076-bb8b-deeefad033e2] Deletion of /opt/stack/data/nova/instances/cb569e8d-e4a5-4076-bb8b-deeefad033e2_del complete
Nov 29 03:08:56 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:08:56 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/21e9c6cd-c39f-41c0-8768-5dd64589c3f3/disk --force-share --output=json" returned: 0 in 0.177s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:08:56 user nova-compute[75234]: INFO nova.compute.manager [None req-88bf7696-da01-415d-80cd-dd7361a45c4c tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] [instance: cb569e8d-e4a5-4076-bb8b-deeefad033e2] Took 3.93 seconds to destroy the instance on the hypervisor.
Nov 29 03:08:56 user nova-compute[75234]: DEBUG oslo.service.loopingcall [None req-88bf7696-da01-415d-80cd-dd7361a45c4c tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=75234) func /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/loopingcall.py:435}}
Nov 29 03:08:56 user nova-compute[75234]: DEBUG nova.compute.manager [-] [instance: cb569e8d-e4a5-4076-bb8b-deeefad033e2] Deallocating network for instance {{(pid=75234) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2279}}
Nov 29 03:08:56 user nova-compute[75234]: DEBUG nova.network.neutron [-] [instance: cb569e8d-e4a5-4076-bb8b-deeefad033e2] deallocate_for_instance() {{(pid=75234) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1802}}
Nov 29 03:08:56 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f609a7b7-1be2-44b4-9177-7e72a4d4c060/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:08:56 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f609a7b7-1be2-44b4-9177-7e72a4d4c060/disk --force-share --output=json" returned: 0 in 0.168s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:08:56 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f609a7b7-1be2-44b4-9177-7e72a4d4c060/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:08:56 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f609a7b7-1be2-44b4-9177-7e72a4d4c060/disk --force-share --output=json" returned: 0 in 0.173s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:08:56 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] skipping disk /dev/sdh (vdb) as it is a volume {{(pid=75234) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11213}}
Nov 29 03:08:56 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/c9663ab3-3374-400e-ab0a-769731b78cca/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:08:56 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/c9663ab3-3374-400e-ab0a-769731b78cca/disk --force-share --output=json" returned: 0 in 0.187s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:08:56 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/c9663ab3-3374-400e-ab0a-769731b78cca/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:08:57 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/c9663ab3-3374-400e-ab0a-769731b78cca/disk --force-share --output=json" returned: 0 in 0.199s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:08:57 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f7d0bdeb-6a05-47ac-87f2-c8380a9a4286/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:08:57 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f7d0bdeb-6a05-47ac-87f2-c8380a9a4286/disk --force-share --output=json" returned: 0 in 0.177s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:08:57 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f7d0bdeb-6a05-47ac-87f2-c8380a9a4286/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:08:57 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f7d0bdeb-6a05-47ac-87f2-c8380a9a4286/disk --force-share --output=json" returned: 0 in 0.235s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:08:57 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/23a31b1e-4005-442f-96e4-9fe82d9e17ba/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:08:57 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/23a31b1e-4005-442f-96e4-9fe82d9e17ba/disk --force-share --output=json" returned: 0 in 0.176s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:08:57 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/23a31b1e-4005-442f-96e4-9fe82d9e17ba/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:08:57 user nova-compute[75234]: DEBUG nova.objects.instance [None req-01a31e51-5891-44ff-ba1a-33093249f28c tempest-AttachSCSIVolumeTestJSON-719577128 tempest-AttachSCSIVolumeTestJSON-719577128-project-member] Lazy-loading 'flavor' on Instance uuid da625f9c-b7ed-4afc-875c-6436bf58e9cc {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:08:57 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-01a31e51-5891-44ff-ba1a-33093249f28c tempest-AttachSCSIVolumeTestJSON-719577128 tempest-AttachSCSIVolumeTestJSON-719577128-project-member] Lock "da625f9c-b7ed-4afc-875c-6436bf58e9cc" "released" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: held 24.868s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:08:57 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/23a31b1e-4005-442f-96e4-9fe82d9e17ba/disk --force-share --output=json" returned: 0 in 0.168s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:08:57 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Error from libvirt while getting description of instance-0000000a: [Error Code 42] Domain not found: no domain with matching uuid 'cb569e8d-e4a5-4076-bb8b-deeefad033e2' (instance-0000000a): libvirt.libvirtError: Domain not found: no domain with matching uuid 'cb569e8d-e4a5-4076-bb8b-deeefad033e2' (instance-0000000a)
Nov 29 03:08:58 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:08:58 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:08:58 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Hypervisor/Node resource view: name=user free_ram=7509MB free_disk=25.527572631835938GB free_vcpus=3 pci_devices=[{"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}] {{(pid=75234) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1106}}
Nov 29 03:08:58 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:08:58 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:08:58 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance 2bf9fe41-63c2-4882-bcdd-8e314e734ffe actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:08:58 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance 21e9c6cd-c39f-41c0-8768-5dd64589c3f3 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:08:58 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance a9350a18-14e6-42c6-a150-0a1487f20ad9 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:08:58 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance f609a7b7-1be2-44b4-9177-7e72a4d4c060 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:08:58 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance 8297c264-a213-44ee-be46-5c08eabda9f7 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:08:58 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance da625f9c-b7ed-4afc-875c-6436bf58e9cc actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:08:58 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance c9663ab3-3374-400e-ab0a-769731b78cca actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:08:58 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance cb569e8d-e4a5-4076-bb8b-deeefad033e2 actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:08:58 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance 763a3e14-4563-4dc2-99ae-c340edbb93df actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:08:58 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance f7d0bdeb-6a05-47ac-87f2-c8380a9a4286 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:08:58 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance 23a31b1e-4005-442f-96e4-9fe82d9e17ba actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:08:58 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Total usable vcpus: 12, total allocated vcpus: 11 {{(pid=75234) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1129}}
Nov 29 03:08:58 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Final resource view: name=user phys_ram=16011MB used_ram=6144MB phys_disk=40GB used_disk=9GB total_vcpus=12 used_vcpus=11 pci_stats=[] {{(pid=75234) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1138}}
Nov 29 03:08:58 user nova-compute[75234]: DEBUG nova.compute.provider_tree [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Inventory has not changed in ProviderTree for provider: 4d291220-7c37-4839-a0e3-b23c66502b74 {{(pid=75234) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Nov 29 03:08:58 user nova-compute[75234]: DEBUG nova.scheduler.client.report [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Inventory has not changed for provider 4d291220-7c37-4839-a0e3-b23c66502b74 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=75234) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Nov 29 03:08:58 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Compute_service record updated for user:user {{(pid=75234) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:1067}}
Nov 29 03:08:58 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.598s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:08:58 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-f169a941-a5e2-4136-b6b9-a4f0e9e55047 tempest-AttachSCSIVolumeTestJSON-719577128 tempest-AttachSCSIVolumeTestJSON-719577128-project-member] Acquiring lock "da625f9c-b7ed-4afc-875c-6436bf58e9cc" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:08:58 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-f169a941-a5e2-4136-b6b9-a4f0e9e55047 tempest-AttachSCSIVolumeTestJSON-719577128 tempest-AttachSCSIVolumeTestJSON-719577128-project-member] Lock "da625f9c-b7ed-4afc-875c-6436bf58e9cc" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:08:58 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-f169a941-a5e2-4136-b6b9-a4f0e9e55047 tempest-AttachSCSIVolumeTestJSON-719577128 tempest-AttachSCSIVolumeTestJSON-719577128-project-member] Acquiring lock "da625f9c-b7ed-4afc-875c-6436bf58e9cc-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:08:58 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-f169a941-a5e2-4136-b6b9-a4f0e9e55047 tempest-AttachSCSIVolumeTestJSON-719577128 tempest-AttachSCSIVolumeTestJSON-719577128-project-member] Lock "da625f9c-b7ed-4afc-875c-6436bf58e9cc-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:08:58 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-f169a941-a5e2-4136-b6b9-a4f0e9e55047 tempest-AttachSCSIVolumeTestJSON-719577128 tempest-AttachSCSIVolumeTestJSON-719577128-project-member] Lock "da625f9c-b7ed-4afc-875c-6436bf58e9cc-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:08:58 user nova-compute[75234]: INFO nova.compute.manager [None req-f169a941-a5e2-4136-b6b9-a4f0e9e55047 tempest-AttachSCSIVolumeTestJSON-719577128 tempest-AttachSCSIVolumeTestJSON-719577128-project-member] [instance: da625f9c-b7ed-4afc-875c-6436bf58e9cc] Terminating instance
Nov 29 03:08:58 user nova-compute[75234]: DEBUG nova.compute.manager [None req-f169a941-a5e2-4136-b6b9-a4f0e9e55047 tempest-AttachSCSIVolumeTestJSON-719577128 tempest-AttachSCSIVolumeTestJSON-719577128-project-member] [instance: da625f9c-b7ed-4afc-875c-6436bf58e9cc] Start destroying the instance on the hypervisor. {{(pid=75234) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3140}}
Nov 29 03:08:59 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:08:59 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:08:59 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:08:59 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:08:59 user nova-compute[75234]: DEBUG nova.compute.manager [req-5c01a72c-a7db-4a06-a300-1f5199023e28 req-8905b5a9-e857-4588-92b3-cc63331b5774 service nova] [instance: da625f9c-b7ed-4afc-875c-6436bf58e9cc] Received event network-vif-unplugged-a23b8769-d871-4d44-8cea-5d899f514579 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:08:59 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-5c01a72c-a7db-4a06-a300-1f5199023e28 req-8905b5a9-e857-4588-92b3-cc63331b5774 service nova] Acquiring lock "da625f9c-b7ed-4afc-875c-6436bf58e9cc-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:08:59 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-5c01a72c-a7db-4a06-a300-1f5199023e28 req-8905b5a9-e857-4588-92b3-cc63331b5774 service nova] Lock "da625f9c-b7ed-4afc-875c-6436bf58e9cc-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:08:59 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-5c01a72c-a7db-4a06-a300-1f5199023e28 req-8905b5a9-e857-4588-92b3-cc63331b5774 service nova] Lock "da625f9c-b7ed-4afc-875c-6436bf58e9cc-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:08:59 user nova-compute[75234]: DEBUG nova.compute.manager [req-5c01a72c-a7db-4a06-a300-1f5199023e28 req-8905b5a9-e857-4588-92b3-cc63331b5774 service nova] [instance: da625f9c-b7ed-4afc-875c-6436bf58e9cc] No waiting events found dispatching network-vif-unplugged-a23b8769-d871-4d44-8cea-5d899f514579 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:08:59 user nova-compute[75234]: DEBUG nova.compute.manager [req-5c01a72c-a7db-4a06-a300-1f5199023e28 req-8905b5a9-e857-4588-92b3-cc63331b5774 service nova] [instance: da625f9c-b7ed-4afc-875c-6436bf58e9cc] Received event network-vif-unplugged-a23b8769-d871-4d44-8cea-5d899f514579 for instance with task_state deleting. {{(pid=75234) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10890}}
Nov 29 03:08:59 user nova-compute[75234]: DEBUG nova.compute.manager [req-6fe70808-6021-4966-a2b2-23e4b9b88ee2 req-50a5515e-3cb7-4f3d-8999-9682af62655a service nova] [instance: cb569e8d-e4a5-4076-bb8b-deeefad033e2] Received event network-vif-deleted-2b12ec96-26fa-4974-8a7a-22e85ec20544 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:08:59 user nova-compute[75234]: INFO nova.compute.manager [req-6fe70808-6021-4966-a2b2-23e4b9b88ee2 req-50a5515e-3cb7-4f3d-8999-9682af62655a service nova] [instance: cb569e8d-e4a5-4076-bb8b-deeefad033e2] Neutron deleted interface 2b12ec96-26fa-4974-8a7a-22e85ec20544; detaching it from the instance and deleting it from the info cache
Nov 29 03:08:59 user nova-compute[75234]: DEBUG nova.network.neutron [req-6fe70808-6021-4966-a2b2-23e4b9b88ee2 req-50a5515e-3cb7-4f3d-8999-9682af62655a service nova] [instance: cb569e8d-e4a5-4076-bb8b-deeefad033e2] Updating instance_info_cache with network_info: [{"id": "0bd5d735-f566-4a6d-b49f-b24fd403a87a", "address": "fa:16:3e:aa:c2:ae", "network": {"id": "005139f3-3686-493a-aa8f-0cdc455befa0", "bridge": "br-int", "label": "tempest-device-tagging-net1-1510949192", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.85", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "ee1117bd3511400f9f45cc8f9bb9247d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap0bd5d735-f5", "ovs_interfaceid": "0bd5d735-f566-4a6d-b49f-b24fd403a87a", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}}, {"id": "27e72461-f85b-42c9-8824-0d760b82a725", "address": "fa:16:3e:8e:29:fd", "network": {"id": "005139f3-3686-493a-aa8f-0cdc455befa0", "bridge": "br-int", "label": "tempest-device-tagging-net1-1510949192", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.127", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "ee1117bd3511400f9f45cc8f9bb9247d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap27e72461-f8", "ovs_interfaceid": "27e72461-f85b-42c9-8824-0d760b82a725", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}}, {"id": "026f606a-534c-4cdd-992a-9f33268b374f", "address": "fa:16:3e:3e:3b:29", "network": {"id": "005139f3-3686-493a-aa8f-0cdc455befa0", "bridge": "br-int", "label": "tempest-device-tagging-net1-1510949192", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.41", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "ee1117bd3511400f9f45cc8f9bb9247d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap026f606a-53", "ovs_interfaceid": "026f606a-534c-4cdd-992a-9f33268b374f", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "67446920-3741-4542-84c6-493388395f26", "address": "fa:16:3e:f6:7a:76", "network": {"id": "005139f3-3686-493a-aa8f-0cdc455befa0", "bridge": "br-int", "label": "tempest-device-tagging-net1-1510949192", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.230", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "ee1117bd3511400f9f45cc8f9bb9247d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap67446920-37", "ovs_interfaceid": "67446920-3741-4542-84c6-493388395f26", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "d7e92753-492b-4fc0-8048-f4792cd3b4b6", "address": "fa:16:3e:68:b2:33", "network": {"id": "cc6c7cdf-8c81-4906-9fd8-de815fea3374", "bridge": "br-int", "label": "tempest-device-tagging-net2-1248280764", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.100", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "ee1117bd3511400f9f45cc8f9bb9247d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapd7e92753-49", "ovs_interfaceid": "d7e92753-492b-4fc0-8048-f4792cd3b4b6", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "06db3012-0042-457a-a3c8-14e3a8d2e89e", "address": "fa:16:3e:27:54:30", "network": {"id": "cc6c7cdf-8c81-4906-9fd8-de815fea3374", "bridge": "br-int", "label": "tempest-device-tagging-net2-1248280764", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.200", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "ee1117bd3511400f9f45cc8f9bb9247d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap06db3012-00", "ovs_interfaceid": "06db3012-0042-457a-a3c8-14e3a8d2e89e", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:08:59 user nova-compute[75234]: DEBUG nova.compute.manager [req-6fe70808-6021-4966-a2b2-23e4b9b88ee2 req-50a5515e-3cb7-4f3d-8999-9682af62655a service nova] [instance: cb569e8d-e4a5-4076-bb8b-deeefad033e2] Detach interface failed, port_id=2b12ec96-26fa-4974-8a7a-22e85ec20544, reason: Instance cb569e8d-e4a5-4076-bb8b-deeefad033e2 could not be found. {{(pid=75234) _process_instance_vif_deleted_event /opt/stack/nova/nova/compute/manager.py:10946}}
Nov 29 03:08:59 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-b485989f-0c2c-4b36-8f02-1aa1f0b630cc tempest-VolumesAssistedSnapshotsTest-1119739890 tempest-VolumesAssistedSnapshotsTest-1119739890-project-admin] Waiting for libvirt event about the detach of device vdb with device alias virtio-disk1 from instance c9663ab3-3374-400e-ab0a-769731b78cca is timed out.
Nov 29 03:08:59 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-b485989f-0c2c-4b36-8f02-1aa1f0b630cc tempest-VolumesAssistedSnapshotsTest-1119739890 tempest-VolumesAssistedSnapshotsTest-1119739890-project-admin] Successfully detached device vdb from instance c9663ab3-3374-400e-ab0a-769731b78cca from the live domain config.
Nov 29 03:08:59 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-b485989f-0c2c-4b36-8f02-1aa1f0b630cc tempest-VolumesAssistedSnapshotsTest-1119739890 tempest-VolumesAssistedSnapshotsTest-1119739890-project-admin] [instance: c9663ab3-3374-400e-ab0a-769731b78cca] calling os-brick to detach iSCSI Volume {{(pid=75234) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Nov 29 03:08:59 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-b485989f-0c2c-4b36-8f02-1aa1f0b630cc tempest-VolumesAssistedSnapshotsTest-1119739890 tempest-VolumesAssistedSnapshotsTest-1119739890-project-admin] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f8a5050c9a0>, {'target_iqn': 'iqn.2010-10.org.openstack:a4cf29d0-2ce2-423f-a20c-a7a19a452088', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'kAVxiylD', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdj'}, None), 'kwargs': False}}" {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:176}}
Nov 29 03:08:59 user nova-compute[75234]: WARNING os_brick.initiator.connectors.base [None req-b485989f-0c2c-4b36-8f02-1aa1f0b630cc tempest-VolumesAssistedSnapshotsTest-1119739890 tempest-VolumesAssistedSnapshotsTest-1119739890-project-admin] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Nov 29 03:08:59 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-b485989f-0c2c-4b36-8f02-1aa1f0b630cc tempest-VolumesAssistedSnapshotsTest-1119739890 tempest-VolumesAssistedSnapshotsTest-1119739890-project-admin] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:68}}
Nov 29 03:08:59 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-b485989f-0c2c-4b36-8f02-1aa1f0b630cc tempest-VolumesAssistedSnapshotsTest-1119739890 tempest-VolumesAssistedSnapshotsTest-1119739890-project-admin] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.002s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:73}}
Nov 29 03:08:59 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:08:59 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.022s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:08:59 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[16b3540b-3596-4c6f-b026-fb9d2312c1f2]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:0d2c0c22-2b3a-43db-b9af-e60a2514e48e\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:a4cf29d0-2ce2-423f-a20c-a7a19a452088\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:5fcd66fc-a479-4613-b1f8-e6ef12ca470a\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:b1068fd1-6636-43cd-9725-4a37f708b2bc\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:08:59 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-b485989f-0c2c-4b36-8f02-1aa1f0b630cc tempest-VolumesAssistedSnapshotsTest-1119739890 tempest-VolumesAssistedSnapshotsTest-1119739890-project-admin] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS:
Nov 29 03:08:59 user nova-compute[75234]: No targets found.
Nov 29 03:08:59 user nova-compute[75234]: iSNS:
Nov 29 03:08:59 user nova-compute[75234]: No targets found.
Nov 29 03:08:59 user nova-compute[75234]: STATIC:
Nov 29 03:08:59 user nova-compute[75234]: Target: iqn.2010-10.org.openstack:0d2c0c22-2b3a-43db-b9af-e60a2514e48e
Nov 29 03:08:59 user nova-compute[75234]:         Portal: 172.16.0.220:3260,-1
Nov 29 03:08:59 user nova-compute[75234]:                 Iface Name: default
Nov 29 03:08:59 user nova-compute[75234]: Target: iqn.2010-10.org.openstack:a4cf29d0-2ce2-423f-a20c-a7a19a452088
Nov 29 03:08:59 user nova-compute[75234]:         Portal: 172.16.0.220:3260,-1
Nov 29 03:08:59 user nova-compute[75234]:                 Iface Name: default
Nov 29 03:08:59 user nova-compute[75234]: Target: iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672
Nov 29 03:08:59 user nova-compute[75234]:         Portal: 172.16.0.220:3260,-1
Nov 29 03:08:59 user nova-compute[75234]:                 Iface Name: default
Nov 29 03:08:59 user nova-compute[75234]: Target: iqn.2010-10.org.openstack:5fcd66fc-a479-4613-b1f8-e6ef12ca470a
Nov 29 03:08:59 user nova-compute[75234]:         Portal: 172.16.0.220:3260,-1
Nov 29 03:08:59 user nova-compute[75234]:                 Iface Name: default
Nov 29 03:08:59 user nova-compute[75234]: Target: iqn.2010-10.org.openstack:b1068fd1-6636-43cd-9725-4a37f708b2bc
Nov 29 03:08:59 user nova-compute[75234]:         Portal: 172.16.0.220:3260,-1
Nov 29 03:08:59 user nova-compute[75234]:                 Iface Name: default
Nov 29 03:08:59 user nova-compute[75234]: FIRMWARE:
Nov 29 03:08:59 user nova-compute[75234]: No targets found.
Nov 29 03:08:59 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:08:59 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-b485989f-0c2c-4b36-8f02-1aa1f0b630cc tempest-VolumesAssistedSnapshotsTest-1119739890 tempest-VolumesAssistedSnapshotsTest-1119739890-project-admin] Regex to get portals from discoverydb: ^SENDTARGETS:
Nov 29 03:08:59 user nova-compute[75234]: .*?^DiscoveryAddress: 172.16.0.220,3260.*?
Nov 29 03:08:59 user nova-compute[75234]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=75234) _get_discoverydb_portals /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:382}}
Nov 29 03:08:59 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-b485989f-0c2c-4b36-8f02-1aa1f0b630cc tempest-VolumesAssistedSnapshotsTest-1119739890 tempest-VolumesAssistedSnapshotsTest-1119739890-project-admin] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:a4cf29d0-2ce2-423f-a20c-a7a19a452088', 0)] {{(pid=75234) _get_connection_devices /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:830}}
Nov 29 03:08:59 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:08:59 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.018s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:08:59 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[c7d0a8e2-ad63-4806-b120-433ef04f93db]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:0d2c0c22-2b3a-43db-b9af-e60a2514e48e\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:5fcd66fc-a479-4613-b1f8-e6ef12ca470a\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:a4cf29d0-2ce2-423f-a20c-a7a19a452088\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:b1068fd1-6636-43cd-9725-4a37f708b2bc\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:08:59 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:08:59 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.027s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:08:59 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[e2d3d0a1-4792-408b-ae18-b0fcce198d6f]: (4, ('tcp: [11] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a4cf29d0-2ce2-423f-a20c-a7a19a452088 (non-flash)\ntcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b1068fd1-6636-43cd-9725-4a37f708b2bc (non-flash)\ntcp: [6] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0d2c0c22-2b3a-43db-b9af-e60a2514e48e (non-flash)\ntcp: [7] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5fcd66fc-a479-4613-b1f8-e6ef12ca470a (non-flash)\ntcp: [9] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672 (non-flash)\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:08:59 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-b485989f-0c2c-4b36-8f02-1aa1f0b630cc tempest-VolumesAssistedSnapshotsTest-1119739890 tempest-VolumesAssistedSnapshotsTest-1119739890-project-admin] iscsiadm ('-m', 'session'): stdout=tcp: [11] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a4cf29d0-2ce2-423f-a20c-a7a19a452088 (non-flash)
Nov 29 03:08:59 user nova-compute[75234]: tcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b1068fd1-6636-43cd-9725-4a37f708b2bc (non-flash)
Nov 29 03:08:59 user nova-compute[75234]: tcp: [6] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0d2c0c22-2b3a-43db-b9af-e60a2514e48e (non-flash)
Nov 29 03:08:59 user nova-compute[75234]: tcp: [7] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5fcd66fc-a479-4613-b1f8-e6ef12ca470a (non-flash)
Nov 29 03:08:59 user nova-compute[75234]: tcp: [9] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672 (non-flash)
Nov 29 03:08:59 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:08:59 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-b485989f-0c2c-4b36-8f02-1aa1f0b630cc tempest-VolumesAssistedSnapshotsTest-1119739890 tempest-VolumesAssistedSnapshotsTest-1119739890-project-admin] iscsi session list stdout=tcp: [11] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a4cf29d0-2ce2-423f-a20c-a7a19a452088 (non-flash)
Nov 29 03:08:59 user nova-compute[75234]: tcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b1068fd1-6636-43cd-9725-4a37f708b2bc (non-flash)
Nov 29 03:08:59 user nova-compute[75234]: tcp: [6] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0d2c0c22-2b3a-43db-b9af-e60a2514e48e (non-flash)
Nov 29 03:08:59 user nova-compute[75234]: tcp: [7] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5fcd66fc-a479-4613-b1f8-e6ef12ca470a (non-flash)
Nov 29 03:08:59 user nova-compute[75234]: tcp: [9] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672 (non-flash)
Nov 29 03:08:59 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsi_session /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Nov 29 03:08:59 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-b485989f-0c2c-4b36-8f02-1aa1f0b630cc tempest-VolumesAssistedSnapshotsTest-1119739890 tempest-VolumesAssistedSnapshotsTest-1119739890-project-admin] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7f8a502bd360>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:a4cf29d0-2ce2-423f-a20c-a7a19a452088'): ({'sdj'}, set())}) {{(pid=75234) _get_connection_devices /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:863}}
Nov 29 03:08:59 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-b485989f-0c2c-4b36-8f02-1aa1f0b630cc tempest-VolumesAssistedSnapshotsTest-1119739890 tempest-VolumesAssistedSnapshotsTest-1119739890-project-admin] Removing single pathed devices sdj {{(pid=75234) remove_connection /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:374}}
Nov 29 03:08:59 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:08:59 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.018s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:08:59 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[17b58eac-4077-4591-b360-0aabe44331e4]: (4, ('path checker states:\nup                  5\n\npaths: 0\nbusy: False\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:08:59 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdj {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:08:59 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdj" returned: 0 in 0.019s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:08:59 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[c0d6b700-cba8-4b06-948c-97b1daeeb101]: (4, ('ok\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:08:59 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-b485989f-0c2c-4b36-8f02-1aa1f0b630cc tempest-VolumesAssistedSnapshotsTest-1119739890 tempest-VolumesAssistedSnapshotsTest-1119739890-project-admin] Flushing IO for device /dev/sdj {{(pid=75234) flush_device_io /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:434}}
Nov 29 03:08:59 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdj {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:08:59 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdj" returned: 0 in 0.012s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:08:59 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[2cea566c-7cb7-4c09-bfb9-2991ac6f954b]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:08:59 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-b485989f-0c2c-4b36-8f02-1aa1f0b630cc tempest-VolumesAssistedSnapshotsTest-1119739890 tempest-VolumesAssistedSnapshotsTest-1119739890-project-admin] Remove SCSI device /dev/sdj with /sys/block/sdj/device/delete {{(pid=75234) remove_scsi_device /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:148}}
Nov 29 03:08:59 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdj/device/delete {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:08:59 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdj/device/delete" returned: 0 in 0.038s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:08:59 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[f098a37e-b633-4374-a632-f671fdef7e05]: (4, ('1', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:08:59 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-b485989f-0c2c-4b36-8f02-1aa1f0b630cc tempest-VolumesAssistedSnapshotsTest-1119739890 tempest-VolumesAssistedSnapshotsTest-1119739890-project-admin] Checking to see if SCSI volumes sdj have been removed. {{(pid=75234) wait_for_volumes_removal /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:156}}
Nov 29 03:08:59 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-b485989f-0c2c-4b36-8f02-1aa1f0b630cc tempest-VolumesAssistedSnapshotsTest-1119739890 tempest-VolumesAssistedSnapshotsTest-1119739890-project-admin] SCSI volumes sdj have been removed. {{(pid=75234) wait_for_volumes_removal /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:166}}
Nov 29 03:08:59 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-b485989f-0c2c-4b36-8f02-1aa1f0b630cc tempest-VolumesAssistedSnapshotsTest-1119739890 tempest-VolumesAssistedSnapshotsTest-1119739890-project-admin] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:a4cf29d0-2ce2-423f-a20c-a7a19a452088')] {{(pid=75234) _disconnect_connection /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Nov 29 03:08:59 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:a4cf29d0-2ce2-423f-a20c-a7a19a452088 -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:08:59 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:a4cf29d0-2ce2-423f-a20c-a7a19a452088 -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.020s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:08:59 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[4d2ffd21-654e-4a16-8c37-39a9545e811a]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:08:59 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-b485989f-0c2c-4b36-8f02-1aa1f0b630cc tempest-VolumesAssistedSnapshotsTest-1119739890 tempest-VolumesAssistedSnapshotsTest-1119739890-project-admin] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:08:59 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:a4cf29d0-2ce2-423f-a20c-a7a19a452088 -p 172.16.0.220:3260 --logout {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:08:59 user nova-compute[75234]: INFO nova.virt.libvirt.driver [-] [instance: da625f9c-b7ed-4afc-875c-6436bf58e9cc] Instance destroyed successfully.
Nov 29 03:08:59 user nova-compute[75234]: DEBUG nova.objects.instance [None req-f169a941-a5e2-4136-b6b9-a4f0e9e55047 tempest-AttachSCSIVolumeTestJSON-719577128 tempest-AttachSCSIVolumeTestJSON-719577128-project-member] Lazy-loading 'resources' on Instance uuid da625f9c-b7ed-4afc-875c-6436bf58e9cc {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:08:59 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:a4cf29d0-2ce2-423f-a20c-a7a19a452088 -p 172.16.0.220:3260 --logout" returned: 0 in 0.073s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:08:59 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[899acfd3-7a7d-4d88-8c75-0db1f70871af]: (4, ('Logging out of session [sid: 11, target: iqn.2010-10.org.openstack:a4cf29d0-2ce2-423f-a20c-a7a19a452088, portal: 172.16.0.220,3260]\nLogout of [sid: 11, target: iqn.2010-10.org.openstack:a4cf29d0-2ce2-423f-a20c-a7a19a452088, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:08:59 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-b485989f-0c2c-4b36-8f02-1aa1f0b630cc tempest-VolumesAssistedSnapshotsTest-1119739890 tempest-VolumesAssistedSnapshotsTest-1119739890-project-admin] iscsiadm ('--logout',): stdout=Logging out of session [sid: 11, target: iqn.2010-10.org.openstack:a4cf29d0-2ce2-423f-a20c-a7a19a452088, portal: 172.16.0.220,3260]
Nov 29 03:08:59 user nova-compute[75234]: Logout of [sid: 11, target: iqn.2010-10.org.openstack:a4cf29d0-2ce2-423f-a20c-a7a19a452088, portal: 172.16.0.220,3260] successful.
Nov 29 03:08:59 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:08:59 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-f169a941-a5e2-4136-b6b9-a4f0e9e55047 tempest-AttachSCSIVolumeTestJSON-719577128 tempest-AttachSCSIVolumeTestJSON-719577128-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='True',created_at=2023-11-29T03:06:27Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-AttachSCSIVolumeTestJSON-server-171828016',display_name='tempest-AttachSCSIVolumeTestJSON-server-171828016',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-attachscsivolumetestjson-server-171828016',id=7,image_ref='b25373f9-ce4e-4a59-b733-b4caaf9c34c6',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBC4uploT/5M0XHJIlQuZn+27+ncVPyCTxw0QjUV1DHWW3WslgbLdQdc14yzb4DwiiAP8Omme0YQgb6GOuaNR43ED2DaxENLJ5i6sS1MBovu0WHlXf/5HMiLederK2qneAQ==',key_name='tempest-keypair-473777431',keypairs=<?>,launch_index=0,launched_at=2023-11-29T03:06:40Z,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='df2e94062c154555ab1ee0c3bf8e0f84',ramdisk_id='',reservation_id='r-801mj20k',resources=None,root_device_name='/dev/sda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='b25373f9-ce4e-4a59-b733-b4caaf9c34c6',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='scsi',image_hw_disk_bus='scsi',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_scsi_model='virtio-scsi',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',owner_project_name='tempest-AttachSCSIVolumeTestJSON-719577128',owner_user_name='tempest-AttachSCSIVolumeTestJSON-719577128-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-11-29T03:06:41Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='ef10980c66b44933bc463320af40dfca',uuid=da625f9c-b7ed-4afc-875c-6436bf58e9cc,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "a23b8769-d871-4d44-8cea-5d899f514579", "address": "fa:16:3e:7f:17:4d", "network": {"id": "5bbe9946-1c42-4c9e-8cf7-5ee0a54ebc8d", "bridge": "br-int", "label": "tempest-AttachSCSIVolumeTestJSON-411635514-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.75", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "df2e94062c154555ab1ee0c3bf8e0f84", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapa23b8769-d8", "ovs_interfaceid": "a23b8769-d871-4d44-8cea-5d899f514579", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Nov 29 03:08:59 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-f169a941-a5e2-4136-b6b9-a4f0e9e55047 tempest-AttachSCSIVolumeTestJSON-719577128 tempest-AttachSCSIVolumeTestJSON-719577128-project-member] Converting VIF {"id": "a23b8769-d871-4d44-8cea-5d899f514579", "address": "fa:16:3e:7f:17:4d", "network": {"id": "5bbe9946-1c42-4c9e-8cf7-5ee0a54ebc8d", "bridge": "br-int", "label": "tempest-AttachSCSIVolumeTestJSON-411635514-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.75", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "df2e94062c154555ab1ee0c3bf8e0f84", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapa23b8769-d8", "ovs_interfaceid": "a23b8769-d871-4d44-8cea-5d899f514579", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:08:59 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-f169a941-a5e2-4136-b6b9-a4f0e9e55047 tempest-AttachSCSIVolumeTestJSON-719577128 tempest-AttachSCSIVolumeTestJSON-719577128-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:7f:17:4d,bridge_name='br-int',has_traffic_filtering=True,id=a23b8769-d871-4d44-8cea-5d899f514579,network=Network(5bbe9946-1c42-4c9e-8cf7-5ee0a54ebc8d),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapa23b8769-d8') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:08:59 user nova-compute[75234]: DEBUG os_vif [None req-f169a941-a5e2-4136-b6b9-a4f0e9e55047 tempest-AttachSCSIVolumeTestJSON-719577128 tempest-AttachSCSIVolumeTestJSON-719577128-project-member] Unplugging vif VIFOpenVSwitch(active=True,address=fa:16:3e:7f:17:4d,bridge_name='br-int',has_traffic_filtering=True,id=a23b8769-d871-4d44-8cea-5d899f514579,network=Network(5bbe9946-1c42-4c9e-8cf7-5ee0a54ebc8d),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapa23b8769-d8') {{(pid=75234) unplug /opt/stack/data/venv/lib/python3.10/site-packages/os_vif/__init__.py:109}}
Nov 29 03:08:59 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:a4cf29d0-2ce2-423f-a20c-a7a19a452088 -p 172.16.0.220:3260 --op delete {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:08:59 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:08:59 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tapa23b8769-d8, bridge=br-int, if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:08:59 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:08:59 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:08:59 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:a4cf29d0-2ce2-423f-a20c-a7a19a452088 -p 172.16.0.220:3260 --op delete" returned: 0 in 0.029s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:08:59 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[24129af7-55a4-4dff-a567-9a91ec417ddd]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:08:59 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-b485989f-0c2c-4b36-8f02-1aa1f0b630cc tempest-VolumesAssistedSnapshotsTest-1119739890 tempest-VolumesAssistedSnapshotsTest-1119739890-project-admin] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:08:59 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-b485989f-0c2c-4b36-8f02-1aa1f0b630cc tempest-VolumesAssistedSnapshotsTest-1119739890 tempest-VolumesAssistedSnapshotsTest-1119739890-project-admin] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.346s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:87}}
Nov 29 03:08:59 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-b485989f-0c2c-4b36-8f02-1aa1f0b630cc tempest-VolumesAssistedSnapshotsTest-1119739890 tempest-VolumesAssistedSnapshotsTest-1119739890-project-admin] <== disconnect_volume: return (349ms) None {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:203}}
Nov 29 03:08:59 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-b485989f-0c2c-4b36-8f02-1aa1f0b630cc tempest-VolumesAssistedSnapshotsTest-1119739890 tempest-VolumesAssistedSnapshotsTest-1119739890-project-admin] [instance: c9663ab3-3374-400e-ab0a-769731b78cca] Disconnected iSCSI Volume {{(pid=75234) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:79}}
Nov 29 03:08:59 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:08:59 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:08:59 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbDestroyCommand(_result=None, table=QoS, record=df5d30c3-fd7c-45e1-91d0-c284d514245e) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:08:59 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:08:59 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:08:59 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:08:59 user nova-compute[75234]: INFO os_vif [None req-f169a941-a5e2-4136-b6b9-a4f0e9e55047 tempest-AttachSCSIVolumeTestJSON-719577128 tempest-AttachSCSIVolumeTestJSON-719577128-project-member] Successfully unplugged vif VIFOpenVSwitch(active=True,address=fa:16:3e:7f:17:4d,bridge_name='br-int',has_traffic_filtering=True,id=a23b8769-d871-4d44-8cea-5d899f514579,network=Network(5bbe9946-1c42-4c9e-8cf7-5ee0a54ebc8d),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapa23b8769-d8')
Nov 29 03:08:59 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-f169a941-a5e2-4136-b6b9-a4f0e9e55047 tempest-AttachSCSIVolumeTestJSON-719577128 tempest-AttachSCSIVolumeTestJSON-719577128-project-member] [instance: da625f9c-b7ed-4afc-875c-6436bf58e9cc] Deleting instance files /opt/stack/data/nova/instances/da625f9c-b7ed-4afc-875c-6436bf58e9cc_del
Nov 29 03:08:59 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-f169a941-a5e2-4136-b6b9-a4f0e9e55047 tempest-AttachSCSIVolumeTestJSON-719577128 tempest-AttachSCSIVolumeTestJSON-719577128-project-member] [instance: da625f9c-b7ed-4afc-875c-6436bf58e9cc] Deletion of /opt/stack/data/nova/instances/da625f9c-b7ed-4afc-875c-6436bf58e9cc_del complete
Nov 29 03:08:59 user nova-compute[75234]: INFO nova.compute.manager [None req-f169a941-a5e2-4136-b6b9-a4f0e9e55047 tempest-AttachSCSIVolumeTestJSON-719577128 tempest-AttachSCSIVolumeTestJSON-719577128-project-member] [instance: da625f9c-b7ed-4afc-875c-6436bf58e9cc] Took 0.93 seconds to destroy the instance on the hypervisor.
Nov 29 03:08:59 user nova-compute[75234]: DEBUG oslo.service.loopingcall [None req-f169a941-a5e2-4136-b6b9-a4f0e9e55047 tempest-AttachSCSIVolumeTestJSON-719577128 tempest-AttachSCSIVolumeTestJSON-719577128-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=75234) func /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/loopingcall.py:435}}
Nov 29 03:08:59 user nova-compute[75234]: DEBUG nova.compute.manager [-] [instance: da625f9c-b7ed-4afc-875c-6436bf58e9cc] Deallocating network for instance {{(pid=75234) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2279}}
Nov 29 03:08:59 user nova-compute[75234]: DEBUG nova.network.neutron [-] [instance: da625f9c-b7ed-4afc-875c-6436bf58e9cc] deallocate_for_instance() {{(pid=75234) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1802}}
Nov 29 03:08:59 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:09:00 user nova-compute[75234]: DEBUG nova.objects.instance [None req-b485989f-0c2c-4b36-8f02-1aa1f0b630cc tempest-VolumesAssistedSnapshotsTest-1119739890 tempest-VolumesAssistedSnapshotsTest-1119739890-project-admin] Lazy-loading 'flavor' on Instance uuid c9663ab3-3374-400e-ab0a-769731b78cca {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:09:00 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-b485989f-0c2c-4b36-8f02-1aa1f0b630cc tempest-VolumesAssistedSnapshotsTest-1119739890 tempest-VolumesAssistedSnapshotsTest-1119739890-project-admin] Lock "c9663ab3-3374-400e-ab0a-769731b78cca" "released" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: held 21.742s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:09:00 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:09:00 user nova-compute[75234]: DEBUG nova.network.neutron [-] [instance: da625f9c-b7ed-4afc-875c-6436bf58e9cc] Updating instance_info_cache with network_info: [] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:09:01 user nova-compute[75234]: INFO nova.compute.manager [-] [instance: da625f9c-b7ed-4afc-875c-6436bf58e9cc] Took 1.16 seconds to deallocate network for instance.
Nov 29 03:09:01 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-f169a941-a5e2-4136-b6b9-a4f0e9e55047 tempest-AttachSCSIVolumeTestJSON-719577128 tempest-AttachSCSIVolumeTestJSON-719577128-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:09:01 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-f169a941-a5e2-4136-b6b9-a4f0e9e55047 tempest-AttachSCSIVolumeTestJSON-719577128 tempest-AttachSCSIVolumeTestJSON-719577128-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.002s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:09:01 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:09:01 user nova-compute[75234]: DEBUG nova.compute.manager [req-37841c8f-d6ac-419b-84e4-330850714f42 req-2aec6793-f01e-4482-852d-b87f4ff5112c service nova] [instance: da625f9c-b7ed-4afc-875c-6436bf58e9cc] Received event network-vif-plugged-a23b8769-d871-4d44-8cea-5d899f514579 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:09:01 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-37841c8f-d6ac-419b-84e4-330850714f42 req-2aec6793-f01e-4482-852d-b87f4ff5112c service nova] Acquiring lock "da625f9c-b7ed-4afc-875c-6436bf58e9cc-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:09:01 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-37841c8f-d6ac-419b-84e4-330850714f42 req-2aec6793-f01e-4482-852d-b87f4ff5112c service nova] Lock "da625f9c-b7ed-4afc-875c-6436bf58e9cc-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:09:01 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-37841c8f-d6ac-419b-84e4-330850714f42 req-2aec6793-f01e-4482-852d-b87f4ff5112c service nova] Lock "da625f9c-b7ed-4afc-875c-6436bf58e9cc-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:09:01 user nova-compute[75234]: DEBUG nova.compute.manager [req-37841c8f-d6ac-419b-84e4-330850714f42 req-2aec6793-f01e-4482-852d-b87f4ff5112c service nova] [instance: da625f9c-b7ed-4afc-875c-6436bf58e9cc] No waiting events found dispatching network-vif-plugged-a23b8769-d871-4d44-8cea-5d899f514579 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:09:01 user nova-compute[75234]: WARNING nova.compute.manager [req-37841c8f-d6ac-419b-84e4-330850714f42 req-2aec6793-f01e-4482-852d-b87f4ff5112c service nova] [instance: da625f9c-b7ed-4afc-875c-6436bf58e9cc] Received unexpected event network-vif-plugged-a23b8769-d871-4d44-8cea-5d899f514579 for instance with vm_state deleted and task_state None.
Nov 29 03:09:01 user nova-compute[75234]: DEBUG nova.compute.manager [req-df18194e-a19f-4d08-b4a3-56e3077933d5 req-403be565-638c-49c5-800b-65bd14ccbee8 service nova] [instance: cb569e8d-e4a5-4076-bb8b-deeefad033e2] Received event network-vif-deleted-06db3012-0042-457a-a3c8-14e3a8d2e89e {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:09:01 user nova-compute[75234]: INFO nova.compute.manager [req-df18194e-a19f-4d08-b4a3-56e3077933d5 req-403be565-638c-49c5-800b-65bd14ccbee8 service nova] [instance: cb569e8d-e4a5-4076-bb8b-deeefad033e2] Neutron deleted interface 06db3012-0042-457a-a3c8-14e3a8d2e89e; detaching it from the instance and deleting it from the info cache
Nov 29 03:09:01 user nova-compute[75234]: DEBUG nova.network.neutron [req-df18194e-a19f-4d08-b4a3-56e3077933d5 req-403be565-638c-49c5-800b-65bd14ccbee8 service nova] [instance: cb569e8d-e4a5-4076-bb8b-deeefad033e2] Updating instance_info_cache with network_info: [{"id": "0bd5d735-f566-4a6d-b49f-b24fd403a87a", "address": "fa:16:3e:aa:c2:ae", "network": {"id": "005139f3-3686-493a-aa8f-0cdc455befa0", "bridge": "br-int", "label": "tempest-device-tagging-net1-1510949192", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.85", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "ee1117bd3511400f9f45cc8f9bb9247d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap0bd5d735-f5", "ovs_interfaceid": "0bd5d735-f566-4a6d-b49f-b24fd403a87a", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}}, {"id": "27e72461-f85b-42c9-8824-0d760b82a725", "address": "fa:16:3e:8e:29:fd", "network": {"id": "005139f3-3686-493a-aa8f-0cdc455befa0", "bridge": "br-int", "label": "tempest-device-tagging-net1-1510949192", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.127", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "ee1117bd3511400f9f45cc8f9bb9247d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap27e72461-f8", "ovs_interfaceid": "27e72461-f85b-42c9-8824-0d760b82a725", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}}, {"id": "026f606a-534c-4cdd-992a-9f33268b374f", "address": "fa:16:3e:3e:3b:29", "network": {"id": "005139f3-3686-493a-aa8f-0cdc455befa0", "bridge": "br-int", "label": "tempest-device-tagging-net1-1510949192", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.41", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "ee1117bd3511400f9f45cc8f9bb9247d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap026f606a-53", "ovs_interfaceid": "026f606a-534c-4cdd-992a-9f33268b374f", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "67446920-3741-4542-84c6-493388395f26", "address": "fa:16:3e:f6:7a:76", "network": {"id": "005139f3-3686-493a-aa8f-0cdc455befa0", "bridge": "br-int", "label": "tempest-device-tagging-net1-1510949192", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.230", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "ee1117bd3511400f9f45cc8f9bb9247d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap67446920-37", "ovs_interfaceid": "67446920-3741-4542-84c6-493388395f26", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "d7e92753-492b-4fc0-8048-f4792cd3b4b6", "address": "fa:16:3e:68:b2:33", "network": {"id": "cc6c7cdf-8c81-4906-9fd8-de815fea3374", "bridge": "br-int", "label": "tempest-device-tagging-net2-1248280764", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.100", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "ee1117bd3511400f9f45cc8f9bb9247d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapd7e92753-49", "ovs_interfaceid": "d7e92753-492b-4fc0-8048-f4792cd3b4b6", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:09:01 user nova-compute[75234]: DEBUG nova.compute.manager [req-df18194e-a19f-4d08-b4a3-56e3077933d5 req-403be565-638c-49c5-800b-65bd14ccbee8 service nova] [instance: cb569e8d-e4a5-4076-bb8b-deeefad033e2] Detach interface failed, port_id=06db3012-0042-457a-a3c8-14e3a8d2e89e, reason: Instance cb569e8d-e4a5-4076-bb8b-deeefad033e2 could not be found. {{(pid=75234) _process_instance_vif_deleted_event /opt/stack/nova/nova/compute/manager.py:10946}}
Nov 29 03:09:01 user nova-compute[75234]: DEBUG nova.compute.manager [req-df18194e-a19f-4d08-b4a3-56e3077933d5 req-403be565-638c-49c5-800b-65bd14ccbee8 service nova] [instance: cb569e8d-e4a5-4076-bb8b-deeefad033e2] Received event network-vif-deleted-d7e92753-492b-4fc0-8048-f4792cd3b4b6 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:09:01 user nova-compute[75234]: INFO nova.compute.manager [req-df18194e-a19f-4d08-b4a3-56e3077933d5 req-403be565-638c-49c5-800b-65bd14ccbee8 service nova] [instance: cb569e8d-e4a5-4076-bb8b-deeefad033e2] Neutron deleted interface d7e92753-492b-4fc0-8048-f4792cd3b4b6; detaching it from the instance and deleting it from the info cache
Nov 29 03:09:01 user nova-compute[75234]: DEBUG nova.network.neutron [req-df18194e-a19f-4d08-b4a3-56e3077933d5 req-403be565-638c-49c5-800b-65bd14ccbee8 service nova] [instance: cb569e8d-e4a5-4076-bb8b-deeefad033e2] Updating instance_info_cache with network_info: [{"id": "0bd5d735-f566-4a6d-b49f-b24fd403a87a", "address": "fa:16:3e:aa:c2:ae", "network": {"id": "005139f3-3686-493a-aa8f-0cdc455befa0", "bridge": "br-int", "label": "tempest-device-tagging-net1-1510949192", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.85", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "ee1117bd3511400f9f45cc8f9bb9247d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap0bd5d735-f5", "ovs_interfaceid": "0bd5d735-f566-4a6d-b49f-b24fd403a87a", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}}, {"id": "27e72461-f85b-42c9-8824-0d760b82a725", "address": "fa:16:3e:8e:29:fd", "network": {"id": "005139f3-3686-493a-aa8f-0cdc455befa0", "bridge": "br-int", "label": "tempest-device-tagging-net1-1510949192", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.127", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "ee1117bd3511400f9f45cc8f9bb9247d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap27e72461-f8", "ovs_interfaceid": "27e72461-f85b-42c9-8824-0d760b82a725", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}}, {"id": "026f606a-534c-4cdd-992a-9f33268b374f", "address": "fa:16:3e:3e:3b:29", "network": {"id": "005139f3-3686-493a-aa8f-0cdc455befa0", "bridge": "br-int", "label": "tempest-device-tagging-net1-1510949192", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.41", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "ee1117bd3511400f9f45cc8f9bb9247d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap026f606a-53", "ovs_interfaceid": "026f606a-534c-4cdd-992a-9f33268b374f", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "67446920-3741-4542-84c6-493388395f26", "address": "fa:16:3e:f6:7a:76", "network": {"id": "005139f3-3686-493a-aa8f-0cdc455befa0", "bridge": "br-int", "label": "tempest-device-tagging-net1-1510949192", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.230", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "ee1117bd3511400f9f45cc8f9bb9247d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap67446920-37", "ovs_interfaceid": "67446920-3741-4542-84c6-493388395f26", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:09:01 user nova-compute[75234]: DEBUG nova.compute.provider_tree [None req-f169a941-a5e2-4136-b6b9-a4f0e9e55047 tempest-AttachSCSIVolumeTestJSON-719577128 tempest-AttachSCSIVolumeTestJSON-719577128-project-member] Inventory has not changed in ProviderTree for provider: 4d291220-7c37-4839-a0e3-b23c66502b74 {{(pid=75234) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Nov 29 03:09:01 user nova-compute[75234]: DEBUG nova.scheduler.client.report [None req-f169a941-a5e2-4136-b6b9-a4f0e9e55047 tempest-AttachSCSIVolumeTestJSON-719577128 tempest-AttachSCSIVolumeTestJSON-719577128-project-member] Inventory has not changed for provider 4d291220-7c37-4839-a0e3-b23c66502b74 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=75234) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Nov 29 03:09:01 user nova-compute[75234]: DEBUG nova.compute.manager [req-df18194e-a19f-4d08-b4a3-56e3077933d5 req-403be565-638c-49c5-800b-65bd14ccbee8 service nova] [instance: cb569e8d-e4a5-4076-bb8b-deeefad033e2] Detach interface failed, port_id=d7e92753-492b-4fc0-8048-f4792cd3b4b6, reason: Instance cb569e8d-e4a5-4076-bb8b-deeefad033e2 could not be found. {{(pid=75234) _process_instance_vif_deleted_event /opt/stack/nova/nova/compute/manager.py:10946}}
Nov 29 03:09:01 user nova-compute[75234]: DEBUG nova.compute.manager [req-df18194e-a19f-4d08-b4a3-56e3077933d5 req-403be565-638c-49c5-800b-65bd14ccbee8 service nova] [instance: da625f9c-b7ed-4afc-875c-6436bf58e9cc] Received event network-vif-deleted-a23b8769-d871-4d44-8cea-5d899f514579 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:09:01 user nova-compute[75234]: DEBUG nova.compute.manager [req-df18194e-a19f-4d08-b4a3-56e3077933d5 req-403be565-638c-49c5-800b-65bd14ccbee8 service nova] [instance: cb569e8d-e4a5-4076-bb8b-deeefad033e2] Received event network-vif-deleted-67446920-3741-4542-84c6-493388395f26 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:09:01 user nova-compute[75234]: INFO nova.compute.manager [req-df18194e-a19f-4d08-b4a3-56e3077933d5 req-403be565-638c-49c5-800b-65bd14ccbee8 service nova] [instance: cb569e8d-e4a5-4076-bb8b-deeefad033e2] Neutron deleted interface 67446920-3741-4542-84c6-493388395f26; detaching it from the instance and deleting it from the info cache
Nov 29 03:09:01 user nova-compute[75234]: DEBUG nova.network.neutron [req-df18194e-a19f-4d08-b4a3-56e3077933d5 req-403be565-638c-49c5-800b-65bd14ccbee8 service nova] [instance: cb569e8d-e4a5-4076-bb8b-deeefad033e2] Updating instance_info_cache with network_info: [{"id": "0bd5d735-f566-4a6d-b49f-b24fd403a87a", "address": "fa:16:3e:aa:c2:ae", "network": {"id": "005139f3-3686-493a-aa8f-0cdc455befa0", "bridge": "br-int", "label": "tempest-device-tagging-net1-1510949192", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.85", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "ee1117bd3511400f9f45cc8f9bb9247d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap0bd5d735-f5", "ovs_interfaceid": "0bd5d735-f566-4a6d-b49f-b24fd403a87a", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}}, {"id": "27e72461-f85b-42c9-8824-0d760b82a725", "address": "fa:16:3e:8e:29:fd", "network": {"id": "005139f3-3686-493a-aa8f-0cdc455befa0", "bridge": "br-int", "label": "tempest-device-tagging-net1-1510949192", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.127", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "ee1117bd3511400f9f45cc8f9bb9247d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap27e72461-f8", "ovs_interfaceid": "27e72461-f85b-42c9-8824-0d760b82a725", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}}, {"id": "026f606a-534c-4cdd-992a-9f33268b374f", "address": "fa:16:3e:3e:3b:29", "network": {"id": "005139f3-3686-493a-aa8f-0cdc455befa0", "bridge": "br-int", "label": "tempest-device-tagging-net1-1510949192", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.41", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "ee1117bd3511400f9f45cc8f9bb9247d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap026f606a-53", "ovs_interfaceid": "026f606a-534c-4cdd-992a-9f33268b374f", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:09:01 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-f169a941-a5e2-4136-b6b9-a4f0e9e55047 tempest-AttachSCSIVolumeTestJSON-719577128 tempest-AttachSCSIVolumeTestJSON-719577128-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.370s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:09:01 user nova-compute[75234]: DEBUG nova.compute.manager [req-df18194e-a19f-4d08-b4a3-56e3077933d5 req-403be565-638c-49c5-800b-65bd14ccbee8 service nova] [instance: cb569e8d-e4a5-4076-bb8b-deeefad033e2] Detach interface failed, port_id=67446920-3741-4542-84c6-493388395f26, reason: Instance cb569e8d-e4a5-4076-bb8b-deeefad033e2 could not be found. {{(pid=75234) _process_instance_vif_deleted_event /opt/stack/nova/nova/compute/manager.py:10946}}
Nov 29 03:09:01 user nova-compute[75234]: INFO nova.scheduler.client.report [None req-f169a941-a5e2-4136-b6b9-a4f0e9e55047 tempest-AttachSCSIVolumeTestJSON-719577128 tempest-AttachSCSIVolumeTestJSON-719577128-project-member] Deleted allocations for instance da625f9c-b7ed-4afc-875c-6436bf58e9cc
Nov 29 03:09:01 user nova-compute[75234]: DEBUG nova.network.neutron [-] [instance: cb569e8d-e4a5-4076-bb8b-deeefad033e2] Updating instance_info_cache with network_info: [] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:09:01 user nova-compute[75234]: INFO nova.compute.manager [-] [instance: cb569e8d-e4a5-4076-bb8b-deeefad033e2] Took 5.37 seconds to deallocate network for instance.
Nov 29 03:09:01 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-f169a941-a5e2-4136-b6b9-a4f0e9e55047 tempest-AttachSCSIVolumeTestJSON-719577128 tempest-AttachSCSIVolumeTestJSON-719577128-project-member] Lock "da625f9c-b7ed-4afc-875c-6436bf58e9cc" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 2.710s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:09:02 user nova-compute[75234]: DEBUG nova.compute.manager [req-841f7688-363b-493d-8eb8-7b69142841d3 req-8fcbe0a3-66a9-4f9c-b6c1-bcfe0df00678 service nova] [instance: 763a3e14-4563-4dc2-99ae-c340edbb93df] Received event network-changed-dc96090f-e855-4d3e-854c-50ea700dc8ea {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:09:02 user nova-compute[75234]: DEBUG nova.compute.manager [req-841f7688-363b-493d-8eb8-7b69142841d3 req-8fcbe0a3-66a9-4f9c-b6c1-bcfe0df00678 service nova] [instance: 763a3e14-4563-4dc2-99ae-c340edbb93df] Refreshing instance network info cache due to event network-changed-dc96090f-e855-4d3e-854c-50ea700dc8ea. {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11117}}
Nov 29 03:09:02 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-841f7688-363b-493d-8eb8-7b69142841d3 req-8fcbe0a3-66a9-4f9c-b6c1-bcfe0df00678 service nova] Acquiring lock "refresh_cache-763a3e14-4563-4dc2-99ae-c340edbb93df" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:09:02 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-841f7688-363b-493d-8eb8-7b69142841d3 req-8fcbe0a3-66a9-4f9c-b6c1-bcfe0df00678 service nova] Acquired lock "refresh_cache-763a3e14-4563-4dc2-99ae-c340edbb93df" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:09:02 user nova-compute[75234]: DEBUG nova.network.neutron [req-841f7688-363b-493d-8eb8-7b69142841d3 req-8fcbe0a3-66a9-4f9c-b6c1-bcfe0df00678 service nova] [instance: 763a3e14-4563-4dc2-99ae-c340edbb93df] Refreshing network info cache for port dc96090f-e855-4d3e-854c-50ea700dc8ea {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2006}}
Nov 29 03:09:02 user nova-compute[75234]: DEBUG nova.network.neutron [req-841f7688-363b-493d-8eb8-7b69142841d3 req-8fcbe0a3-66a9-4f9c-b6c1-bcfe0df00678 service nova] [instance: 763a3e14-4563-4dc2-99ae-c340edbb93df] Updated VIF entry in instance network info cache for port dc96090f-e855-4d3e-854c-50ea700dc8ea. {{(pid=75234) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3481}}
Nov 29 03:09:02 user nova-compute[75234]: DEBUG nova.network.neutron [req-841f7688-363b-493d-8eb8-7b69142841d3 req-8fcbe0a3-66a9-4f9c-b6c1-bcfe0df00678 service nova] [instance: 763a3e14-4563-4dc2-99ae-c340edbb93df] Updating instance_info_cache with network_info: [{"id": "dc96090f-e855-4d3e-854c-50ea700dc8ea", "address": "fa:16:3e:9e:2b:8a", "network": {"id": "ea3baf6d-ba72-4615-bc0d-604e4fcd8e4e", "bridge": "br-int", "label": "tempest-TaggedBootDevicesTest_v242-1704264625-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.206", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "325013d2bedc498e91c2145c2b398803", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapdc96090f-e8", "ovs_interfaceid": "dc96090f-e855-4d3e-854c-50ea700dc8ea", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "2027f7a1-36e4-4434-a063-61e817758917", "address": "fa:16:3e:4d:66:94", "network": {"id": "60b448f6-90dc-4dac-b24e-9370262e6dac", "bridge": "br-int", "label": "tempest-device-tagging-net1-431494419", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.80", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "325013d2bedc498e91c2145c2b398803", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2027f7a1-36", "ovs_interfaceid": "2027f7a1-36e4-4434-a063-61e817758917", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}}, {"id": "de4ad363-fae0-4d1b-9820-5a91a27d630d", "address": "fa:16:3e:94:49:94", "network": {"id": "60b448f6-90dc-4dac-b24e-9370262e6dac", "bridge": "br-int", "label": "tempest-device-tagging-net1-431494419", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.218", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "325013d2bedc498e91c2145c2b398803", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapde4ad363-fa", "ovs_interfaceid": "de4ad363-fae0-4d1b-9820-5a91a27d630d", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}}, {"id": "d520a88d-ec02-45b8-adbc-484d38b9c18c", "address": "fa:16:3e:ac:cd:aa", "network": {"id": "60b448f6-90dc-4dac-b24e-9370262e6dac", "bridge": "br-int", "label": "tempest-device-tagging-net1-431494419", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.226", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "325013d2bedc498e91c2145c2b398803", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapd520a88d-ec", "ovs_interfaceid": "d520a88d-ec02-45b8-adbc-484d38b9c18c", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "c50f64cd-b7fa-4580-bb78-bff756a0773a", "address": "fa:16:3e:36:0b:d9", "network": {"id": "60b448f6-90dc-4dac-b24e-9370262e6dac", "bridge": "br-int", "label": "tempest-device-tagging-net1-431494419", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.18", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "325013d2bedc498e91c2145c2b398803", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc50f64cd-b7", "ovs_interfaceid": "c50f64cd-b7fa-4580-bb78-bff756a0773a", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "ef8ba969-6186-4b9a-b30f-ec9a70036bab", "address": "fa:16:3e:0f:ed:74", "network": {"id": "31e25520-c84d-451c-9132-ddc8c1e443a6", "bridge": "br-int", "label": "tempest-device-tagging-net2-653330954", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.100", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "325013d2bedc498e91c2145c2b398803", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapef8ba969-61", "ovs_interfaceid": "ef8ba969-6186-4b9a-b30f-ec9a70036bab", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "92917a0d-59ee-4167-8693-262a21f6e156", "address": "fa:16:3e:65:95:01", "network": {"id": "31e25520-c84d-451c-9132-ddc8c1e443a6", "bridge": "br-int", "label": "tempest-device-tagging-net2-653330954", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.200", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "325013d2bedc498e91c2145c2b398803", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap92917a0d-59", "ovs_interfaceid": "92917a0d-59ee-4167-8693-262a21f6e156", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:09:03 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-841f7688-363b-493d-8eb8-7b69142841d3 req-8fcbe0a3-66a9-4f9c-b6c1-bcfe0df00678 service nova] Releasing lock "refresh_cache-763a3e14-4563-4dc2-99ae-c340edbb93df" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:09:03 user nova-compute[75234]: DEBUG nova.compute.manager [req-b144ab70-13ff-4289-b33d-b323306237ce req-e2246988-9786-459d-a6d3-a9119c04ecb7 service nova] [instance: cb569e8d-e4a5-4076-bb8b-deeefad033e2] Received event network-vif-deleted-026f606a-534c-4cdd-992a-9f33268b374f {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:09:04 user nova-compute[75234]: INFO nova.compute.manager [None req-88bf7696-da01-415d-80cd-dd7361a45c4c tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] [instance: cb569e8d-e4a5-4076-bb8b-deeefad033e2] Took 2.48 seconds to detach 3 volumes for instance.
Nov 29 03:09:04 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-88bf7696-da01-415d-80cd-dd7361a45c4c tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:09:04 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-88bf7696-da01-415d-80cd-dd7361a45c4c tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:09:04 user nova-compute[75234]: DEBUG nova.compute.provider_tree [None req-88bf7696-da01-415d-80cd-dd7361a45c4c tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] Inventory has not changed in ProviderTree for provider: 4d291220-7c37-4839-a0e3-b23c66502b74 {{(pid=75234) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Nov 29 03:09:04 user nova-compute[75234]: DEBUG nova.scheduler.client.report [None req-88bf7696-da01-415d-80cd-dd7361a45c4c tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] Inventory has not changed for provider 4d291220-7c37-4839-a0e3-b23c66502b74 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=75234) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Nov 29 03:09:04 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-88bf7696-da01-415d-80cd-dd7361a45c4c tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.318s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:09:04 user nova-compute[75234]: INFO nova.scheduler.client.report [None req-88bf7696-da01-415d-80cd-dd7361a45c4c tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] Deleted allocations for instance cb569e8d-e4a5-4076-bb8b-deeefad033e2
Nov 29 03:09:04 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-88bf7696-da01-415d-80cd-dd7361a45c4c tempest-TaggedBootDevicesTest-1858653514 tempest-TaggedBootDevicesTest-1858653514-project-member] Lock "cb569e8d-e4a5-4076-bb8b-deeefad033e2" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 12.284s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:09:04 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:09:04 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-8437735b-1338-4943-bef4-6f6d4f9cbbba tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] Acquiring lock "763a3e14-4563-4dc2-99ae-c340edbb93df" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:09:04 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-8437735b-1338-4943-bef4-6f6d4f9cbbba tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] Lock "763a3e14-4563-4dc2-99ae-c340edbb93df" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:09:04 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-8437735b-1338-4943-bef4-6f6d4f9cbbba tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] Acquiring lock "763a3e14-4563-4dc2-99ae-c340edbb93df-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:09:04 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-8437735b-1338-4943-bef4-6f6d4f9cbbba tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] Lock "763a3e14-4563-4dc2-99ae-c340edbb93df-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:09:04 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-8437735b-1338-4943-bef4-6f6d4f9cbbba tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] Lock "763a3e14-4563-4dc2-99ae-c340edbb93df-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:09:04 user nova-compute[75234]: INFO nova.compute.manager [None req-8437735b-1338-4943-bef4-6f6d4f9cbbba tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] [instance: 763a3e14-4563-4dc2-99ae-c340edbb93df] Terminating instance
Nov 29 03:09:04 user nova-compute[75234]: DEBUG nova.compute.manager [None req-8437735b-1338-4943-bef4-6f6d4f9cbbba tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] [instance: 763a3e14-4563-4dc2-99ae-c340edbb93df] Start destroying the instance on the hypervisor. {{(pid=75234) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3140}}
Nov 29 03:09:04 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:09:04 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:09:04 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:09:04 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:09:05 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:09:05 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:09:05 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:09:05 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:09:05 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:09:05 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:09:05 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:09:05 user nova-compute[75234]: DEBUG nova.compute.manager [req-d728a0b6-7473-466e-a750-926f13112390 req-22ccb69d-e0fa-4c96-bb86-291cb9b86eca service nova] [instance: 763a3e14-4563-4dc2-99ae-c340edbb93df] Received event network-vif-unplugged-2027f7a1-36e4-4434-a063-61e817758917 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:09:05 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-d728a0b6-7473-466e-a750-926f13112390 req-22ccb69d-e0fa-4c96-bb86-291cb9b86eca service nova] Acquiring lock "763a3e14-4563-4dc2-99ae-c340edbb93df-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:09:05 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-d728a0b6-7473-466e-a750-926f13112390 req-22ccb69d-e0fa-4c96-bb86-291cb9b86eca service nova] Lock "763a3e14-4563-4dc2-99ae-c340edbb93df-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:09:05 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-d728a0b6-7473-466e-a750-926f13112390 req-22ccb69d-e0fa-4c96-bb86-291cb9b86eca service nova] Lock "763a3e14-4563-4dc2-99ae-c340edbb93df-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:09:05 user nova-compute[75234]: DEBUG nova.compute.manager [req-d728a0b6-7473-466e-a750-926f13112390 req-22ccb69d-e0fa-4c96-bb86-291cb9b86eca service nova] [instance: 763a3e14-4563-4dc2-99ae-c340edbb93df] No waiting events found dispatching network-vif-unplugged-2027f7a1-36e4-4434-a063-61e817758917 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:09:05 user nova-compute[75234]: DEBUG nova.compute.manager [req-d728a0b6-7473-466e-a750-926f13112390 req-22ccb69d-e0fa-4c96-bb86-291cb9b86eca service nova] [instance: 763a3e14-4563-4dc2-99ae-c340edbb93df] Received event network-vif-unplugged-2027f7a1-36e4-4434-a063-61e817758917 for instance with task_state deleting. {{(pid=75234) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10890}}
Nov 29 03:09:05 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:09:05 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:09:05 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:09:05 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:09:05 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:09:05 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:09:05 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:09:05 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:09:05 user nova-compute[75234]: DEBUG nova.compute.manager [req-0cc12258-5502-41cc-aefc-c982c29c2dd4 req-cc8fc2e0-82c3-4b17-849b-887fbda3225b service nova] [instance: 763a3e14-4563-4dc2-99ae-c340edbb93df] Received event network-vif-unplugged-dc96090f-e855-4d3e-854c-50ea700dc8ea {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:09:05 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-0cc12258-5502-41cc-aefc-c982c29c2dd4 req-cc8fc2e0-82c3-4b17-849b-887fbda3225b service nova] Acquiring lock "763a3e14-4563-4dc2-99ae-c340edbb93df-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:09:05 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-0cc12258-5502-41cc-aefc-c982c29c2dd4 req-cc8fc2e0-82c3-4b17-849b-887fbda3225b service nova] Lock "763a3e14-4563-4dc2-99ae-c340edbb93df-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:09:05 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-0cc12258-5502-41cc-aefc-c982c29c2dd4 req-cc8fc2e0-82c3-4b17-849b-887fbda3225b service nova] Lock "763a3e14-4563-4dc2-99ae-c340edbb93df-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:09:05 user nova-compute[75234]: DEBUG nova.compute.manager [req-0cc12258-5502-41cc-aefc-c982c29c2dd4 req-cc8fc2e0-82c3-4b17-849b-887fbda3225b service nova] [instance: 763a3e14-4563-4dc2-99ae-c340edbb93df] No waiting events found dispatching network-vif-unplugged-dc96090f-e855-4d3e-854c-50ea700dc8ea {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:09:05 user nova-compute[75234]: DEBUG nova.compute.manager [req-0cc12258-5502-41cc-aefc-c982c29c2dd4 req-cc8fc2e0-82c3-4b17-849b-887fbda3225b service nova] [instance: 763a3e14-4563-4dc2-99ae-c340edbb93df] Received event network-vif-unplugged-dc96090f-e855-4d3e-854c-50ea700dc8ea for instance with task_state deleting. {{(pid=75234) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10890}}
Nov 29 03:09:05 user nova-compute[75234]: DEBUG nova.compute.manager [req-0cc12258-5502-41cc-aefc-c982c29c2dd4 req-cc8fc2e0-82c3-4b17-849b-887fbda3225b service nova] [instance: 763a3e14-4563-4dc2-99ae-c340edbb93df] Received event network-vif-plugged-dc96090f-e855-4d3e-854c-50ea700dc8ea {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:09:05 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-0cc12258-5502-41cc-aefc-c982c29c2dd4 req-cc8fc2e0-82c3-4b17-849b-887fbda3225b service nova] Acquiring lock "763a3e14-4563-4dc2-99ae-c340edbb93df-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:09:05 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-0cc12258-5502-41cc-aefc-c982c29c2dd4 req-cc8fc2e0-82c3-4b17-849b-887fbda3225b service nova] Lock "763a3e14-4563-4dc2-99ae-c340edbb93df-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:09:05 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-0cc12258-5502-41cc-aefc-c982c29c2dd4 req-cc8fc2e0-82c3-4b17-849b-887fbda3225b service nova] Lock "763a3e14-4563-4dc2-99ae-c340edbb93df-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:09:05 user nova-compute[75234]: DEBUG nova.compute.manager [req-0cc12258-5502-41cc-aefc-c982c29c2dd4 req-cc8fc2e0-82c3-4b17-849b-887fbda3225b service nova] [instance: 763a3e14-4563-4dc2-99ae-c340edbb93df] No waiting events found dispatching network-vif-plugged-dc96090f-e855-4d3e-854c-50ea700dc8ea {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:09:05 user nova-compute[75234]: WARNING nova.compute.manager [req-0cc12258-5502-41cc-aefc-c982c29c2dd4 req-cc8fc2e0-82c3-4b17-849b-887fbda3225b service nova] [instance: 763a3e14-4563-4dc2-99ae-c340edbb93df] Received unexpected event network-vif-plugged-dc96090f-e855-4d3e-854c-50ea700dc8ea for instance with vm_state active and task_state deleting.
Nov 29 03:09:05 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:09:06 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:09:06 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:09:06 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:09:06 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:09:06 user nova-compute[75234]: INFO nova.virt.libvirt.driver [-] [instance: 763a3e14-4563-4dc2-99ae-c340edbb93df] Instance destroyed successfully.
Nov 29 03:09:06 user nova-compute[75234]: DEBUG nova.objects.instance [None req-8437735b-1338-4943-bef4-6f6d4f9cbbba tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] Lazy-loading 'resources' on Instance uuid 763a3e14-4563-4dc2-99ae-c340edbb93df {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:09:06 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-8437735b-1338-4943-bef4-6f6d4f9cbbba tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='True',created_at=2023-11-29T03:07:09Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-1204199434',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-device-tagging-server-1204199434',id=11,image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBDKKjhuu8m4NOi61Oc3wSdtCCjvt/NxrQN0hvbdlmSXVXySXKH03AWmGvbrcGtt87bsX9Od6uMV6aRJbEi/1xuRd54kkoiFrhpLljDB6Y8GHkcy2CQP4u+I0A4lLOYUDEQ==',key_name='tempest-keypair-1416470280',keypairs=<?>,launch_index=0,launched_at=2023-11-29T03:07:46Z,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='325013d2bedc498e91c2145c2b398803',ramdisk_id='',reservation_id='r-7e1n3ow9',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-TaggedBootDevicesTest_v242-1560452734',owner_user_name='tempest-TaggedBootDevicesTest_v242-1560452734-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-11-29T03:07:47Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='f2e1fe315fb14410b5c0448525d8b9f7',uuid=763a3e14-4563-4dc2-99ae-c340edbb93df,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "dc96090f-e855-4d3e-854c-50ea700dc8ea", "address": "fa:16:3e:9e:2b:8a", "network": {"id": "ea3baf6d-ba72-4615-bc0d-604e4fcd8e4e", "bridge": "br-int", "label": "tempest-TaggedBootDevicesTest_v242-1704264625-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.206", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "325013d2bedc498e91c2145c2b398803", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapdc96090f-e8", "ovs_interfaceid": "dc96090f-e855-4d3e-854c-50ea700dc8ea", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Nov 29 03:09:06 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-8437735b-1338-4943-bef4-6f6d4f9cbbba tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] Converting VIF {"id": "dc96090f-e855-4d3e-854c-50ea700dc8ea", "address": "fa:16:3e:9e:2b:8a", "network": {"id": "ea3baf6d-ba72-4615-bc0d-604e4fcd8e4e", "bridge": "br-int", "label": "tempest-TaggedBootDevicesTest_v242-1704264625-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.206", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "325013d2bedc498e91c2145c2b398803", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapdc96090f-e8", "ovs_interfaceid": "dc96090f-e855-4d3e-854c-50ea700dc8ea", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:09:06 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-8437735b-1338-4943-bef4-6f6d4f9cbbba tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:9e:2b:8a,bridge_name='br-int',has_traffic_filtering=True,id=dc96090f-e855-4d3e-854c-50ea700dc8ea,network=Network(ea3baf6d-ba72-4615-bc0d-604e4fcd8e4e),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapdc96090f-e8') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:09:06 user nova-compute[75234]: DEBUG os_vif [None req-8437735b-1338-4943-bef4-6f6d4f9cbbba tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] Unplugging vif VIFOpenVSwitch(active=True,address=fa:16:3e:9e:2b:8a,bridge_name='br-int',has_traffic_filtering=True,id=dc96090f-e855-4d3e-854c-50ea700dc8ea,network=Network(ea3baf6d-ba72-4615-bc0d-604e4fcd8e4e),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapdc96090f-e8') {{(pid=75234) unplug /opt/stack/data/venv/lib/python3.10/site-packages/os_vif/__init__.py:109}}
Nov 29 03:09:06 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:09:06 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tapdc96090f-e8, bridge=br-int, if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:09:06 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:09:06 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:09:06 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:09:06 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:09:06 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbDestroyCommand(_result=None, table=QoS, record=211b1830-9836-48f1-94bd-37b471f0076c) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:09:06 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:09:06 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:09:06 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:09:06 user nova-compute[75234]: INFO os_vif [None req-8437735b-1338-4943-bef4-6f6d4f9cbbba tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] Successfully unplugged vif VIFOpenVSwitch(active=True,address=fa:16:3e:9e:2b:8a,bridge_name='br-int',has_traffic_filtering=True,id=dc96090f-e855-4d3e-854c-50ea700dc8ea,network=Network(ea3baf6d-ba72-4615-bc0d-604e4fcd8e4e),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapdc96090f-e8')
Nov 29 03:09:06 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-8437735b-1338-4943-bef4-6f6d4f9cbbba tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='True',created_at=2023-11-29T03:07:09Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-1204199434',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-device-tagging-server-1204199434',id=11,image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBDKKjhuu8m4NOi61Oc3wSdtCCjvt/NxrQN0hvbdlmSXVXySXKH03AWmGvbrcGtt87bsX9Od6uMV6aRJbEi/1xuRd54kkoiFrhpLljDB6Y8GHkcy2CQP4u+I0A4lLOYUDEQ==',key_name='tempest-keypair-1416470280',keypairs=<?>,launch_index=0,launched_at=2023-11-29T03:07:46Z,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='325013d2bedc498e91c2145c2b398803',ramdisk_id='',reservation_id='r-7e1n3ow9',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-TaggedBootDevicesTest_v242-1560452734',owner_user_name='tempest-TaggedBootDevicesTest_v242-1560452734-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-11-29T03:07:47Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='f2e1fe315fb14410b5c0448525d8b9f7',uuid=763a3e14-4563-4dc2-99ae-c340edbb93df,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "2027f7a1-36e4-4434-a063-61e817758917", "address": "fa:16:3e:4d:66:94", "network": {"id": "60b448f6-90dc-4dac-b24e-9370262e6dac", "bridge": "br-int", "label": "tempest-device-tagging-net1-431494419", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.80", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "325013d2bedc498e91c2145c2b398803", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2027f7a1-36", "ovs_interfaceid": "2027f7a1-36e4-4434-a063-61e817758917", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}} {{(pid=75234) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Nov 29 03:09:06 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-8437735b-1338-4943-bef4-6f6d4f9cbbba tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] Converting VIF {"id": "2027f7a1-36e4-4434-a063-61e817758917", "address": "fa:16:3e:4d:66:94", "network": {"id": "60b448f6-90dc-4dac-b24e-9370262e6dac", "bridge": "br-int", "label": "tempest-device-tagging-net1-431494419", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.80", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "325013d2bedc498e91c2145c2b398803", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2027f7a1-36", "ovs_interfaceid": "2027f7a1-36e4-4434-a063-61e817758917", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:09:06 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-8437735b-1338-4943-bef4-6f6d4f9cbbba tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:4d:66:94,bridge_name='br-int',has_traffic_filtering=True,id=2027f7a1-36e4-4434-a063-61e817758917,network=Network(60b448f6-90dc-4dac-b24e-9370262e6dac),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=True,vif_name='tap2027f7a1-36') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:09:06 user nova-compute[75234]: DEBUG os_vif [None req-8437735b-1338-4943-bef4-6f6d4f9cbbba tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] Unplugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:4d:66:94,bridge_name='br-int',has_traffic_filtering=True,id=2027f7a1-36e4-4434-a063-61e817758917,network=Network(60b448f6-90dc-4dac-b24e-9370262e6dac),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=True,vif_name='tap2027f7a1-36') {{(pid=75234) unplug /opt/stack/data/venv/lib/python3.10/site-packages/os_vif/__init__.py:109}}
Nov 29 03:09:06 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:09:06 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tap2027f7a1-36, bridge=br-int, if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:09:06 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:09:06 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:09:06 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:09:06 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:09:06 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbDestroyCommand(_result=None, table=QoS, record=34a61084-de2f-469b-b026-fd0630cf2c70) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:09:06 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:09:06 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:09:06 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:09:06 user nova-compute[75234]: INFO os_vif [None req-8437735b-1338-4943-bef4-6f6d4f9cbbba tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] Successfully unplugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:4d:66:94,bridge_name='br-int',has_traffic_filtering=True,id=2027f7a1-36e4-4434-a063-61e817758917,network=Network(60b448f6-90dc-4dac-b24e-9370262e6dac),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=True,vif_name='tap2027f7a1-36')
Nov 29 03:09:06 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-8437735b-1338-4943-bef4-6f6d4f9cbbba tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='True',created_at=2023-11-29T03:07:09Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-1204199434',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-device-tagging-server-1204199434',id=11,image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBDKKjhuu8m4NOi61Oc3wSdtCCjvt/NxrQN0hvbdlmSXVXySXKH03AWmGvbrcGtt87bsX9Od6uMV6aRJbEi/1xuRd54kkoiFrhpLljDB6Y8GHkcy2CQP4u+I0A4lLOYUDEQ==',key_name='tempest-keypair-1416470280',keypairs=<?>,launch_index=0,launched_at=2023-11-29T03:07:46Z,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='325013d2bedc498e91c2145c2b398803',ramdisk_id='',reservation_id='r-7e1n3ow9',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-TaggedBootDevicesTest_v242-1560452734',owner_user_name='tempest-TaggedBootDevicesTest_v242-1560452734-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-11-29T03:07:47Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='f2e1fe315fb14410b5c0448525d8b9f7',uuid=763a3e14-4563-4dc2-99ae-c340edbb93df,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "de4ad363-fae0-4d1b-9820-5a91a27d630d", "address": "fa:16:3e:94:49:94", "network": {"id": "60b448f6-90dc-4dac-b24e-9370262e6dac", "bridge": "br-int", "label": "tempest-device-tagging-net1-431494419", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.218", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "325013d2bedc498e91c2145c2b398803", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapde4ad363-fa", "ovs_interfaceid": "de4ad363-fae0-4d1b-9820-5a91a27d630d", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}} {{(pid=75234) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Nov 29 03:09:06 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-8437735b-1338-4943-bef4-6f6d4f9cbbba tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] Converting VIF {"id": "de4ad363-fae0-4d1b-9820-5a91a27d630d", "address": "fa:16:3e:94:49:94", "network": {"id": "60b448f6-90dc-4dac-b24e-9370262e6dac", "bridge": "br-int", "label": "tempest-device-tagging-net1-431494419", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.218", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "325013d2bedc498e91c2145c2b398803", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapde4ad363-fa", "ovs_interfaceid": "de4ad363-fae0-4d1b-9820-5a91a27d630d", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:09:06 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-8437735b-1338-4943-bef4-6f6d4f9cbbba tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:94:49:94,bridge_name='br-int',has_traffic_filtering=True,id=de4ad363-fae0-4d1b-9820-5a91a27d630d,network=Network(60b448f6-90dc-4dac-b24e-9370262e6dac),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=True,vif_name='tapde4ad363-fa') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:09:06 user nova-compute[75234]: DEBUG os_vif [None req-8437735b-1338-4943-bef4-6f6d4f9cbbba tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] Unplugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:94:49:94,bridge_name='br-int',has_traffic_filtering=True,id=de4ad363-fae0-4d1b-9820-5a91a27d630d,network=Network(60b448f6-90dc-4dac-b24e-9370262e6dac),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=True,vif_name='tapde4ad363-fa') {{(pid=75234) unplug /opt/stack/data/venv/lib/python3.10/site-packages/os_vif/__init__.py:109}}
Nov 29 03:09:06 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:09:06 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tapde4ad363-fa, bridge=br-int, if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:09:06 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:09:06 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:09:06 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:09:06 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:09:06 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbDestroyCommand(_result=None, table=QoS, record=12e6f356-42d4-45e3-9179-eb757fd47734) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:09:06 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:09:07 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:09:07 user nova-compute[75234]: INFO os_vif [None req-8437735b-1338-4943-bef4-6f6d4f9cbbba tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] Successfully unplugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:94:49:94,bridge_name='br-int',has_traffic_filtering=True,id=de4ad363-fae0-4d1b-9820-5a91a27d630d,network=Network(60b448f6-90dc-4dac-b24e-9370262e6dac),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=True,vif_name='tapde4ad363-fa')
Nov 29 03:09:07 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-8437735b-1338-4943-bef4-6f6d4f9cbbba tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='True',created_at=2023-11-29T03:07:09Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-1204199434',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-device-tagging-server-1204199434',id=11,image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBDKKjhuu8m4NOi61Oc3wSdtCCjvt/NxrQN0hvbdlmSXVXySXKH03AWmGvbrcGtt87bsX9Od6uMV6aRJbEi/1xuRd54kkoiFrhpLljDB6Y8GHkcy2CQP4u+I0A4lLOYUDEQ==',key_name='tempest-keypair-1416470280',keypairs=<?>,launch_index=0,launched_at=2023-11-29T03:07:46Z,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='325013d2bedc498e91c2145c2b398803',ramdisk_id='',reservation_id='r-7e1n3ow9',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-TaggedBootDevicesTest_v242-1560452734',owner_user_name='tempest-TaggedBootDevicesTest_v242-1560452734-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-11-29T03:07:47Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='f2e1fe315fb14410b5c0448525d8b9f7',uuid=763a3e14-4563-4dc2-99ae-c340edbb93df,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "d520a88d-ec02-45b8-adbc-484d38b9c18c", "address": "fa:16:3e:ac:cd:aa", "network": {"id": "60b448f6-90dc-4dac-b24e-9370262e6dac", "bridge": "br-int", "label": "tempest-device-tagging-net1-431494419", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.226", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "325013d2bedc498e91c2145c2b398803", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapd520a88d-ec", "ovs_interfaceid": "d520a88d-ec02-45b8-adbc-484d38b9c18c", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Nov 29 03:09:07 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-8437735b-1338-4943-bef4-6f6d4f9cbbba tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] Converting VIF {"id": "d520a88d-ec02-45b8-adbc-484d38b9c18c", "address": "fa:16:3e:ac:cd:aa", "network": {"id": "60b448f6-90dc-4dac-b24e-9370262e6dac", "bridge": "br-int", "label": "tempest-device-tagging-net1-431494419", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.226", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "325013d2bedc498e91c2145c2b398803", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapd520a88d-ec", "ovs_interfaceid": "d520a88d-ec02-45b8-adbc-484d38b9c18c", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:09:07 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-8437735b-1338-4943-bef4-6f6d4f9cbbba tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:ac:cd:aa,bridge_name='br-int',has_traffic_filtering=True,id=d520a88d-ec02-45b8-adbc-484d38b9c18c,network=Network(60b448f6-90dc-4dac-b24e-9370262e6dac),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapd520a88d-ec') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:09:07 user nova-compute[75234]: DEBUG os_vif [None req-8437735b-1338-4943-bef4-6f6d4f9cbbba tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] Unplugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:ac:cd:aa,bridge_name='br-int',has_traffic_filtering=True,id=d520a88d-ec02-45b8-adbc-484d38b9c18c,network=Network(60b448f6-90dc-4dac-b24e-9370262e6dac),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapd520a88d-ec') {{(pid=75234) unplug /opt/stack/data/venv/lib/python3.10/site-packages/os_vif/__init__.py:109}}
Nov 29 03:09:07 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:09:07 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tapd520a88d-ec, bridge=br-int, if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:09:07 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:09:07 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:09:07 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:09:07 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbDestroyCommand(_result=None, table=QoS, record=572725c7-6fbd-4591-afff-1637331469d5) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:09:07 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:09:07 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:09:07 user nova-compute[75234]: INFO os_vif [None req-8437735b-1338-4943-bef4-6f6d4f9cbbba tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] Successfully unplugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:ac:cd:aa,bridge_name='br-int',has_traffic_filtering=True,id=d520a88d-ec02-45b8-adbc-484d38b9c18c,network=Network(60b448f6-90dc-4dac-b24e-9370262e6dac),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapd520a88d-ec')
Nov 29 03:09:07 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-8437735b-1338-4943-bef4-6f6d4f9cbbba tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='True',created_at=2023-11-29T03:07:09Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-1204199434',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-device-tagging-server-1204199434',id=11,image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBDKKjhuu8m4NOi61Oc3wSdtCCjvt/NxrQN0hvbdlmSXVXySXKH03AWmGvbrcGtt87bsX9Od6uMV6aRJbEi/1xuRd54kkoiFrhpLljDB6Y8GHkcy2CQP4u+I0A4lLOYUDEQ==',key_name='tempest-keypair-1416470280',keypairs=<?>,launch_index=0,launched_at=2023-11-29T03:07:46Z,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='325013d2bedc498e91c2145c2b398803',ramdisk_id='',reservation_id='r-7e1n3ow9',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-TaggedBootDevicesTest_v242-1560452734',owner_user_name='tempest-TaggedBootDevicesTest_v242-1560452734-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-11-29T03:07:47Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='f2e1fe315fb14410b5c0448525d8b9f7',uuid=763a3e14-4563-4dc2-99ae-c340edbb93df,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "c50f64cd-b7fa-4580-bb78-bff756a0773a", "address": "fa:16:3e:36:0b:d9", "network": {"id": "60b448f6-90dc-4dac-b24e-9370262e6dac", "bridge": "br-int", "label": "tempest-device-tagging-net1-431494419", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.18", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "325013d2bedc498e91c2145c2b398803", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc50f64cd-b7", "ovs_interfaceid": "c50f64cd-b7fa-4580-bb78-bff756a0773a", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Nov 29 03:09:07 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-8437735b-1338-4943-bef4-6f6d4f9cbbba tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] Converting VIF {"id": "c50f64cd-b7fa-4580-bb78-bff756a0773a", "address": "fa:16:3e:36:0b:d9", "network": {"id": "60b448f6-90dc-4dac-b24e-9370262e6dac", "bridge": "br-int", "label": "tempest-device-tagging-net1-431494419", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.18", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "325013d2bedc498e91c2145c2b398803", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc50f64cd-b7", "ovs_interfaceid": "c50f64cd-b7fa-4580-bb78-bff756a0773a", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:09:07 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-8437735b-1338-4943-bef4-6f6d4f9cbbba tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:36:0b:d9,bridge_name='br-int',has_traffic_filtering=True,id=c50f64cd-b7fa-4580-bb78-bff756a0773a,network=Network(60b448f6-90dc-4dac-b24e-9370262e6dac),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapc50f64cd-b7') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:09:07 user nova-compute[75234]: DEBUG os_vif [None req-8437735b-1338-4943-bef4-6f6d4f9cbbba tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] Unplugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:36:0b:d9,bridge_name='br-int',has_traffic_filtering=True,id=c50f64cd-b7fa-4580-bb78-bff756a0773a,network=Network(60b448f6-90dc-4dac-b24e-9370262e6dac),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapc50f64cd-b7') {{(pid=75234) unplug /opt/stack/data/venv/lib/python3.10/site-packages/os_vif/__init__.py:109}}
Nov 29 03:09:07 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:09:07 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tapc50f64cd-b7, bridge=br-int, if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:09:07 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:09:07 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:09:07 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:09:07 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbDestroyCommand(_result=None, table=QoS, record=13dec504-f0c0-427b-9b1a-501d753e76b5) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:09:07 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:09:07 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:09:07 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:09:07 user nova-compute[75234]: INFO os_vif [None req-8437735b-1338-4943-bef4-6f6d4f9cbbba tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] Successfully unplugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:36:0b:d9,bridge_name='br-int',has_traffic_filtering=True,id=c50f64cd-b7fa-4580-bb78-bff756a0773a,network=Network(60b448f6-90dc-4dac-b24e-9370262e6dac),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapc50f64cd-b7')
Nov 29 03:09:07 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-8437735b-1338-4943-bef4-6f6d4f9cbbba tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='True',created_at=2023-11-29T03:07:09Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-1204199434',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-device-tagging-server-1204199434',id=11,image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBDKKjhuu8m4NOi61Oc3wSdtCCjvt/NxrQN0hvbdlmSXVXySXKH03AWmGvbrcGtt87bsX9Od6uMV6aRJbEi/1xuRd54kkoiFrhpLljDB6Y8GHkcy2CQP4u+I0A4lLOYUDEQ==',key_name='tempest-keypair-1416470280',keypairs=<?>,launch_index=0,launched_at=2023-11-29T03:07:46Z,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='325013d2bedc498e91c2145c2b398803',ramdisk_id='',reservation_id='r-7e1n3ow9',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-TaggedBootDevicesTest_v242-1560452734',owner_user_name='tempest-TaggedBootDevicesTest_v242-1560452734-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-11-29T03:07:47Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='f2e1fe315fb14410b5c0448525d8b9f7',uuid=763a3e14-4563-4dc2-99ae-c340edbb93df,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "ef8ba969-6186-4b9a-b30f-ec9a70036bab", "address": "fa:16:3e:0f:ed:74", "network": {"id": "31e25520-c84d-451c-9132-ddc8c1e443a6", "bridge": "br-int", "label": "tempest-device-tagging-net2-653330954", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.100", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "325013d2bedc498e91c2145c2b398803", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapef8ba969-61", "ovs_interfaceid": "ef8ba969-6186-4b9a-b30f-ec9a70036bab", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Nov 29 03:09:07 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-8437735b-1338-4943-bef4-6f6d4f9cbbba tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] Converting VIF {"id": "ef8ba969-6186-4b9a-b30f-ec9a70036bab", "address": "fa:16:3e:0f:ed:74", "network": {"id": "31e25520-c84d-451c-9132-ddc8c1e443a6", "bridge": "br-int", "label": "tempest-device-tagging-net2-653330954", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.100", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "325013d2bedc498e91c2145c2b398803", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapef8ba969-61", "ovs_interfaceid": "ef8ba969-6186-4b9a-b30f-ec9a70036bab", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:09:07 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-8437735b-1338-4943-bef4-6f6d4f9cbbba tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:0f:ed:74,bridge_name='br-int',has_traffic_filtering=True,id=ef8ba969-6186-4b9a-b30f-ec9a70036bab,network=Network(31e25520-c84d-451c-9132-ddc8c1e443a6),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapef8ba969-61') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:09:07 user nova-compute[75234]: DEBUG os_vif [None req-8437735b-1338-4943-bef4-6f6d4f9cbbba tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] Unplugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:0f:ed:74,bridge_name='br-int',has_traffic_filtering=True,id=ef8ba969-6186-4b9a-b30f-ec9a70036bab,network=Network(31e25520-c84d-451c-9132-ddc8c1e443a6),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapef8ba969-61') {{(pid=75234) unplug /opt/stack/data/venv/lib/python3.10/site-packages/os_vif/__init__.py:109}}
Nov 29 03:09:07 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:09:07 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tapef8ba969-61, bridge=br-int, if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:09:07 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:09:07 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:09:07 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:09:07 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:09:07 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbDestroyCommand(_result=None, table=QoS, record=5470af18-024a-4f42-82dc-92f47a68c7ef) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:09:07 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:09:07 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:09:07 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:09:07 user nova-compute[75234]: INFO os_vif [None req-8437735b-1338-4943-bef4-6f6d4f9cbbba tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] Successfully unplugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:0f:ed:74,bridge_name='br-int',has_traffic_filtering=True,id=ef8ba969-6186-4b9a-b30f-ec9a70036bab,network=Network(31e25520-c84d-451c-9132-ddc8c1e443a6),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapef8ba969-61')
Nov 29 03:09:07 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-8437735b-1338-4943-bef4-6f6d4f9cbbba tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='True',created_at=2023-11-29T03:07:09Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-1204199434',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-device-tagging-server-1204199434',id=11,image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBDKKjhuu8m4NOi61Oc3wSdtCCjvt/NxrQN0hvbdlmSXVXySXKH03AWmGvbrcGtt87bsX9Od6uMV6aRJbEi/1xuRd54kkoiFrhpLljDB6Y8GHkcy2CQP4u+I0A4lLOYUDEQ==',key_name='tempest-keypair-1416470280',keypairs=<?>,launch_index=0,launched_at=2023-11-29T03:07:46Z,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='325013d2bedc498e91c2145c2b398803',ramdisk_id='',reservation_id='r-7e1n3ow9',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-TaggedBootDevicesTest_v242-1560452734',owner_user_name='tempest-TaggedBootDevicesTest_v242-1560452734-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-11-29T03:07:47Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='f2e1fe315fb14410b5c0448525d8b9f7',uuid=763a3e14-4563-4dc2-99ae-c340edbb93df,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "92917a0d-59ee-4167-8693-262a21f6e156", "address": "fa:16:3e:65:95:01", "network": {"id": "31e25520-c84d-451c-9132-ddc8c1e443a6", "bridge": "br-int", "label": "tempest-device-tagging-net2-653330954", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.200", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "325013d2bedc498e91c2145c2b398803", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap92917a0d-59", "ovs_interfaceid": "92917a0d-59ee-4167-8693-262a21f6e156", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Nov 29 03:09:07 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-8437735b-1338-4943-bef4-6f6d4f9cbbba tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] Converting VIF {"id": "92917a0d-59ee-4167-8693-262a21f6e156", "address": "fa:16:3e:65:95:01", "network": {"id": "31e25520-c84d-451c-9132-ddc8c1e443a6", "bridge": "br-int", "label": "tempest-device-tagging-net2-653330954", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.200", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "325013d2bedc498e91c2145c2b398803", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap92917a0d-59", "ovs_interfaceid": "92917a0d-59ee-4167-8693-262a21f6e156", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:09:07 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-8437735b-1338-4943-bef4-6f6d4f9cbbba tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:65:95:01,bridge_name='br-int',has_traffic_filtering=True,id=92917a0d-59ee-4167-8693-262a21f6e156,network=Network(31e25520-c84d-451c-9132-ddc8c1e443a6),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap92917a0d-59') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:09:07 user nova-compute[75234]: DEBUG os_vif [None req-8437735b-1338-4943-bef4-6f6d4f9cbbba tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] Unplugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:65:95:01,bridge_name='br-int',has_traffic_filtering=True,id=92917a0d-59ee-4167-8693-262a21f6e156,network=Network(31e25520-c84d-451c-9132-ddc8c1e443a6),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap92917a0d-59') {{(pid=75234) unplug /opt/stack/data/venv/lib/python3.10/site-packages/os_vif/__init__.py:109}}
Nov 29 03:09:07 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:09:07 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tap92917a0d-59, bridge=br-int, if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:09:07 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:09:07 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:09:07 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:09:07 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:09:07 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbDestroyCommand(_result=None, table=QoS, record=9d2393d9-d024-42ca-94c4-1524cd361a18) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:09:07 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:09:07 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:09:07 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:09:07 user nova-compute[75234]: INFO os_vif [None req-8437735b-1338-4943-bef4-6f6d4f9cbbba tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] Successfully unplugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:65:95:01,bridge_name='br-int',has_traffic_filtering=True,id=92917a0d-59ee-4167-8693-262a21f6e156,network=Network(31e25520-c84d-451c-9132-ddc8c1e443a6),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap92917a0d-59')
Nov 29 03:09:07 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-8437735b-1338-4943-bef4-6f6d4f9cbbba tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] [instance: 763a3e14-4563-4dc2-99ae-c340edbb93df] calling os-brick to detach iSCSI Volume {{(pid=75234) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Nov 29 03:09:07 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-8437735b-1338-4943-bef4-6f6d4f9cbbba tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f8a5050c9a0>, {'target_iqn': 'iqn.2010-10.org.openstack:b1068fd1-6636-43cd-9725-4a37f708b2bc', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'LpdSswjf', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sde'}, None), 'kwargs': True}}" {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:176}}
Nov 29 03:09:07 user nova-compute[75234]: WARNING os_brick.initiator.connectors.base [None req-8437735b-1338-4943-bef4-6f6d4f9cbbba tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Nov 29 03:09:07 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-8437735b-1338-4943-bef4-6f6d4f9cbbba tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:68}}
Nov 29 03:09:07 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-8437735b-1338-4943-bef4-6f6d4f9cbbba tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:73}}
Nov 29 03:09:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:09:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.019s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:09:07 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[bc659d50-4c8f-4f8d-bb16-3af3ec32f5d1]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:0d2c0c22-2b3a-43db-b9af-e60a2514e48e\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:5fcd66fc-a479-4613-b1f8-e6ef12ca470a\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:b1068fd1-6636-43cd-9725-4a37f708b2bc\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:09:07 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-8437735b-1338-4943-bef4-6f6d4f9cbbba tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS:
Nov 29 03:09:07 user nova-compute[75234]: No targets found.
Nov 29 03:09:07 user nova-compute[75234]: iSNS:
Nov 29 03:09:07 user nova-compute[75234]: No targets found.
Nov 29 03:09:07 user nova-compute[75234]: STATIC:
Nov 29 03:09:07 user nova-compute[75234]: Target: iqn.2010-10.org.openstack:0d2c0c22-2b3a-43db-b9af-e60a2514e48e
Nov 29 03:09:07 user nova-compute[75234]:         Portal: 172.16.0.220:3260,-1
Nov 29 03:09:07 user nova-compute[75234]:                 Iface Name: default
Nov 29 03:09:07 user nova-compute[75234]: Target: iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672
Nov 29 03:09:07 user nova-compute[75234]:         Portal: 172.16.0.220:3260,-1
Nov 29 03:09:07 user nova-compute[75234]:                 Iface Name: default
Nov 29 03:09:07 user nova-compute[75234]: Target: iqn.2010-10.org.openstack:5fcd66fc-a479-4613-b1f8-e6ef12ca470a
Nov 29 03:09:07 user nova-compute[75234]:         Portal: 172.16.0.220:3260,-1
Nov 29 03:09:07 user nova-compute[75234]:                 Iface Name: default
Nov 29 03:09:07 user nova-compute[75234]: Target: iqn.2010-10.org.openstack:b1068fd1-6636-43cd-9725-4a37f708b2bc
Nov 29 03:09:07 user nova-compute[75234]:         Portal: 172.16.0.220:3260,-1
Nov 29 03:09:07 user nova-compute[75234]:                 Iface Name: default
Nov 29 03:09:07 user nova-compute[75234]: FIRMWARE:
Nov 29 03:09:07 user nova-compute[75234]: No targets found.
Nov 29 03:09:07 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:09:07 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-8437735b-1338-4943-bef4-6f6d4f9cbbba tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] Regex to get portals from discoverydb: ^SENDTARGETS:
Nov 29 03:09:07 user nova-compute[75234]: .*?^DiscoveryAddress: 172.16.0.220,3260.*?
Nov 29 03:09:07 user nova-compute[75234]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=75234) _get_discoverydb_portals /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:382}}
Nov 29 03:09:07 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-8437735b-1338-4943-bef4-6f6d4f9cbbba tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:b1068fd1-6636-43cd-9725-4a37f708b2bc', 0)] {{(pid=75234) _get_connection_devices /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:830}}
Nov 29 03:09:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:09:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.022s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:09:07 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[e5a2d63f-72a2-4400-8be9-9e967f7fcc38]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:0d2c0c22-2b3a-43db-b9af-e60a2514e48e\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:5fcd66fc-a479-4613-b1f8-e6ef12ca470a\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:b1068fd1-6636-43cd-9725-4a37f708b2bc\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:09:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:09:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.028s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:09:07 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[bbc2c42e-5f94-4d8d-a0cb-452b1079f7aa]: (4, ('tcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b1068fd1-6636-43cd-9725-4a37f708b2bc (non-flash)\ntcp: [6] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0d2c0c22-2b3a-43db-b9af-e60a2514e48e (non-flash)\ntcp: [7] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5fcd66fc-a479-4613-b1f8-e6ef12ca470a (non-flash)\ntcp: [9] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672 (non-flash)\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:09:07 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-8437735b-1338-4943-bef4-6f6d4f9cbbba tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b1068fd1-6636-43cd-9725-4a37f708b2bc (non-flash)
Nov 29 03:09:07 user nova-compute[75234]: tcp: [6] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0d2c0c22-2b3a-43db-b9af-e60a2514e48e (non-flash)
Nov 29 03:09:07 user nova-compute[75234]: tcp: [7] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5fcd66fc-a479-4613-b1f8-e6ef12ca470a (non-flash)
Nov 29 03:09:07 user nova-compute[75234]: tcp: [9] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672 (non-flash)
Nov 29 03:09:07 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:09:07 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-8437735b-1338-4943-bef4-6f6d4f9cbbba tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] iscsi session list stdout=tcp: [5] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:b1068fd1-6636-43cd-9725-4a37f708b2bc (non-flash)
Nov 29 03:09:07 user nova-compute[75234]: tcp: [6] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0d2c0c22-2b3a-43db-b9af-e60a2514e48e (non-flash)
Nov 29 03:09:07 user nova-compute[75234]: tcp: [7] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5fcd66fc-a479-4613-b1f8-e6ef12ca470a (non-flash)
Nov 29 03:09:07 user nova-compute[75234]: tcp: [9] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672 (non-flash)
Nov 29 03:09:07 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsi_session /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Nov 29 03:09:07 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-8437735b-1338-4943-bef4-6f6d4f9cbbba tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7f8a50374a60>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:b1068fd1-6636-43cd-9725-4a37f708b2bc'): ({'sde'}, set())}) {{(pid=75234) _get_connection_devices /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:863}}
Nov 29 03:09:07 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-8437735b-1338-4943-bef4-6f6d4f9cbbba tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] Removing single pathed devices sde {{(pid=75234) remove_connection /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:374}}
Nov 29 03:09:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:09:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.020s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:09:07 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[2d971d4c-b2d0-4e0e-bd31-58814226ddd3]: (4, ('path checker states:\nup                  4\n\npaths: 0\nbusy: False\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:09:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sde {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:09:07 user nova-compute[75234]: DEBUG nova.compute.manager [req-cd6c73f3-ba5f-411c-8d73-ebd0ff9df9bc req-f82fe18d-e1d6-4e96-89f6-699912b61760 service nova] [instance: 763a3e14-4563-4dc2-99ae-c340edbb93df] Received event network-vif-plugged-2027f7a1-36e4-4434-a063-61e817758917 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:09:07 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-cd6c73f3-ba5f-411c-8d73-ebd0ff9df9bc req-f82fe18d-e1d6-4e96-89f6-699912b61760 service nova] Acquiring lock "763a3e14-4563-4dc2-99ae-c340edbb93df-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:09:07 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-cd6c73f3-ba5f-411c-8d73-ebd0ff9df9bc req-f82fe18d-e1d6-4e96-89f6-699912b61760 service nova] Lock "763a3e14-4563-4dc2-99ae-c340edbb93df-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:09:07 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-cd6c73f3-ba5f-411c-8d73-ebd0ff9df9bc req-f82fe18d-e1d6-4e96-89f6-699912b61760 service nova] Lock "763a3e14-4563-4dc2-99ae-c340edbb93df-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:09:07 user nova-compute[75234]: DEBUG nova.compute.manager [req-cd6c73f3-ba5f-411c-8d73-ebd0ff9df9bc req-f82fe18d-e1d6-4e96-89f6-699912b61760 service nova] [instance: 763a3e14-4563-4dc2-99ae-c340edbb93df] No waiting events found dispatching network-vif-plugged-2027f7a1-36e4-4434-a063-61e817758917 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:09:07 user nova-compute[75234]: WARNING nova.compute.manager [req-cd6c73f3-ba5f-411c-8d73-ebd0ff9df9bc req-f82fe18d-e1d6-4e96-89f6-699912b61760 service nova] [instance: 763a3e14-4563-4dc2-99ae-c340edbb93df] Received unexpected event network-vif-plugged-2027f7a1-36e4-4434-a063-61e817758917 for instance with vm_state active and task_state deleting.
Nov 29 03:09:07 user nova-compute[75234]: DEBUG nova.compute.manager [req-cd6c73f3-ba5f-411c-8d73-ebd0ff9df9bc req-f82fe18d-e1d6-4e96-89f6-699912b61760 service nova] [instance: 763a3e14-4563-4dc2-99ae-c340edbb93df] Received event network-vif-unplugged-ef8ba969-6186-4b9a-b30f-ec9a70036bab {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:09:07 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-cd6c73f3-ba5f-411c-8d73-ebd0ff9df9bc req-f82fe18d-e1d6-4e96-89f6-699912b61760 service nova] Acquiring lock "763a3e14-4563-4dc2-99ae-c340edbb93df-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:09:07 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-cd6c73f3-ba5f-411c-8d73-ebd0ff9df9bc req-f82fe18d-e1d6-4e96-89f6-699912b61760 service nova] Lock "763a3e14-4563-4dc2-99ae-c340edbb93df-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:09:07 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-cd6c73f3-ba5f-411c-8d73-ebd0ff9df9bc req-f82fe18d-e1d6-4e96-89f6-699912b61760 service nova] Lock "763a3e14-4563-4dc2-99ae-c340edbb93df-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:09:07 user nova-compute[75234]: DEBUG nova.compute.manager [req-cd6c73f3-ba5f-411c-8d73-ebd0ff9df9bc req-f82fe18d-e1d6-4e96-89f6-699912b61760 service nova] [instance: 763a3e14-4563-4dc2-99ae-c340edbb93df] No waiting events found dispatching network-vif-unplugged-ef8ba969-6186-4b9a-b30f-ec9a70036bab {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:09:07 user nova-compute[75234]: DEBUG nova.compute.manager [req-cd6c73f3-ba5f-411c-8d73-ebd0ff9df9bc req-f82fe18d-e1d6-4e96-89f6-699912b61760 service nova] [instance: 763a3e14-4563-4dc2-99ae-c340edbb93df] Received event network-vif-unplugged-ef8ba969-6186-4b9a-b30f-ec9a70036bab for instance with task_state deleting. {{(pid=75234) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10890}}
Nov 29 03:09:07 user nova-compute[75234]: DEBUG nova.compute.manager [req-cd6c73f3-ba5f-411c-8d73-ebd0ff9df9bc req-f82fe18d-e1d6-4e96-89f6-699912b61760 service nova] [instance: 763a3e14-4563-4dc2-99ae-c340edbb93df] Received event network-vif-plugged-ef8ba969-6186-4b9a-b30f-ec9a70036bab {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:09:07 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-cd6c73f3-ba5f-411c-8d73-ebd0ff9df9bc req-f82fe18d-e1d6-4e96-89f6-699912b61760 service nova] Acquiring lock "763a3e14-4563-4dc2-99ae-c340edbb93df-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:09:07 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-cd6c73f3-ba5f-411c-8d73-ebd0ff9df9bc req-f82fe18d-e1d6-4e96-89f6-699912b61760 service nova] Lock "763a3e14-4563-4dc2-99ae-c340edbb93df-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:09:07 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-cd6c73f3-ba5f-411c-8d73-ebd0ff9df9bc req-f82fe18d-e1d6-4e96-89f6-699912b61760 service nova] Lock "763a3e14-4563-4dc2-99ae-c340edbb93df-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:09:07 user nova-compute[75234]: DEBUG nova.compute.manager [req-cd6c73f3-ba5f-411c-8d73-ebd0ff9df9bc req-f82fe18d-e1d6-4e96-89f6-699912b61760 service nova] [instance: 763a3e14-4563-4dc2-99ae-c340edbb93df] No waiting events found dispatching network-vif-plugged-ef8ba969-6186-4b9a-b30f-ec9a70036bab {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:09:07 user nova-compute[75234]: WARNING nova.compute.manager [req-cd6c73f3-ba5f-411c-8d73-ebd0ff9df9bc req-f82fe18d-e1d6-4e96-89f6-699912b61760 service nova] [instance: 763a3e14-4563-4dc2-99ae-c340edbb93df] Received unexpected event network-vif-plugged-ef8ba969-6186-4b9a-b30f-ec9a70036bab for instance with vm_state active and task_state deleting.
Nov 29 03:09:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sde" returned: 0 in 0.020s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:09:07 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[a0d1d592-17d6-4ad2-a143-1272588cfbaf]: (4, ('ok\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:09:07 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-8437735b-1338-4943-bef4-6f6d4f9cbbba tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] Flushing IO for device /dev/sde {{(pid=75234) flush_device_io /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:434}}
Nov 29 03:09:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sde {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:09:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sde" returned: 0 in 0.014s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:09:07 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[2405830f-daca-4a4c-9865-636b089c4980]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:09:07 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-8437735b-1338-4943-bef4-6f6d4f9cbbba tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] Remove SCSI device /dev/sde with /sys/block/sde/device/delete {{(pid=75234) remove_scsi_device /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:148}}
Nov 29 03:09:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sde/device/delete {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:09:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sde/device/delete" returned: 0 in 0.046s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:09:07 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[c87483b4-195e-4982-ba77-9195254734a7]: (4, ('1', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:09:07 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-8437735b-1338-4943-bef4-6f6d4f9cbbba tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] Checking to see if SCSI volumes sde have been removed. {{(pid=75234) wait_for_volumes_removal /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:156}}
Nov 29 03:09:07 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-8437735b-1338-4943-bef4-6f6d4f9cbbba tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] SCSI volumes sde have been removed. {{(pid=75234) wait_for_volumes_removal /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:166}}
Nov 29 03:09:07 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-8437735b-1338-4943-bef4-6f6d4f9cbbba tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:b1068fd1-6636-43cd-9725-4a37f708b2bc')] {{(pid=75234) _disconnect_connection /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Nov 29 03:09:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:b1068fd1-6636-43cd-9725-4a37f708b2bc -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:09:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:b1068fd1-6636-43cd-9725-4a37f708b2bc -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.019s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:09:07 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[bc4166f7-5a3c-495f-8a5d-87f1b35cd6c0]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:09:07 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-8437735b-1338-4943-bef4-6f6d4f9cbbba tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:09:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:b1068fd1-6636-43cd-9725-4a37f708b2bc -p 172.16.0.220:3260 --logout {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:09:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:b1068fd1-6636-43cd-9725-4a37f708b2bc -p 172.16.0.220:3260 --logout" returned: 0 in 0.063s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:09:07 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[936a66d4-2189-4227-abd6-026fcc97e4f1]: (4, ('Logging out of session [sid: 5, target: iqn.2010-10.org.openstack:b1068fd1-6636-43cd-9725-4a37f708b2bc, portal: 172.16.0.220,3260]\nLogout of [sid: 5, target: iqn.2010-10.org.openstack:b1068fd1-6636-43cd-9725-4a37f708b2bc, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:09:07 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-8437735b-1338-4943-bef4-6f6d4f9cbbba tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 5, target: iqn.2010-10.org.openstack:b1068fd1-6636-43cd-9725-4a37f708b2bc, portal: 172.16.0.220,3260]
Nov 29 03:09:07 user nova-compute[75234]: Logout of [sid: 5, target: iqn.2010-10.org.openstack:b1068fd1-6636-43cd-9725-4a37f708b2bc, portal: 172.16.0.220,3260] successful.
Nov 29 03:09:07 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:09:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:b1068fd1-6636-43cd-9725-4a37f708b2bc -p 172.16.0.220:3260 --op delete {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:09:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:b1068fd1-6636-43cd-9725-4a37f708b2bc -p 172.16.0.220:3260 --op delete" returned: 0 in 0.026s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:09:07 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[5887cc31-5836-451a-8c07-c1faa2b1731f]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:09:07 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-8437735b-1338-4943-bef4-6f6d4f9cbbba tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:09:07 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-8437735b-1338-4943-bef4-6f6d4f9cbbba tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.322s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:87}}
Nov 29 03:09:07 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-8437735b-1338-4943-bef4-6f6d4f9cbbba tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] <== disconnect_volume: return (323ms) None {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:203}}
Nov 29 03:09:07 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-8437735b-1338-4943-bef4-6f6d4f9cbbba tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] [instance: 763a3e14-4563-4dc2-99ae-c340edbb93df] Disconnected iSCSI Volume {{(pid=75234) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:79}}
Nov 29 03:09:07 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-8437735b-1338-4943-bef4-6f6d4f9cbbba tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] [instance: 763a3e14-4563-4dc2-99ae-c340edbb93df] calling os-brick to detach iSCSI Volume {{(pid=75234) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Nov 29 03:09:07 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-8437735b-1338-4943-bef4-6f6d4f9cbbba tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f8a5050c9a0>, {'target_iqn': 'iqn.2010-10.org.openstack:0d2c0c22-2b3a-43db-b9af-e60a2514e48e', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'oOMLrjWB', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdf'}, None), 'kwargs': True}}" {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:176}}
Nov 29 03:09:07 user nova-compute[75234]: WARNING os_brick.initiator.connectors.base [None req-8437735b-1338-4943-bef4-6f6d4f9cbbba tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Nov 29 03:09:07 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-8437735b-1338-4943-bef4-6f6d4f9cbbba tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:68}}
Nov 29 03:09:07 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-8437735b-1338-4943-bef4-6f6d4f9cbbba tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:73}}
Nov 29 03:09:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:09:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.019s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:09:07 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[cd3c3254-d514-45e8-8716-3d08f9ad2a57]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:0d2c0c22-2b3a-43db-b9af-e60a2514e48e\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:5fcd66fc-a479-4613-b1f8-e6ef12ca470a\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:09:07 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-8437735b-1338-4943-bef4-6f6d4f9cbbba tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS:
Nov 29 03:09:07 user nova-compute[75234]: No targets found.
Nov 29 03:09:07 user nova-compute[75234]: iSNS:
Nov 29 03:09:07 user nova-compute[75234]: No targets found.
Nov 29 03:09:07 user nova-compute[75234]: STATIC:
Nov 29 03:09:07 user nova-compute[75234]: Target: iqn.2010-10.org.openstack:0d2c0c22-2b3a-43db-b9af-e60a2514e48e
Nov 29 03:09:07 user nova-compute[75234]:         Portal: 172.16.0.220:3260,-1
Nov 29 03:09:07 user nova-compute[75234]:                 Iface Name: default
Nov 29 03:09:07 user nova-compute[75234]: Target: iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672
Nov 29 03:09:07 user nova-compute[75234]:         Portal: 172.16.0.220:3260,-1
Nov 29 03:09:07 user nova-compute[75234]:                 Iface Name: default
Nov 29 03:09:07 user nova-compute[75234]: Target: iqn.2010-10.org.openstack:5fcd66fc-a479-4613-b1f8-e6ef12ca470a
Nov 29 03:09:07 user nova-compute[75234]:         Portal: 172.16.0.220:3260,-1
Nov 29 03:09:07 user nova-compute[75234]:                 Iface Name: default
Nov 29 03:09:07 user nova-compute[75234]: FIRMWARE:
Nov 29 03:09:07 user nova-compute[75234]: No targets found.
Nov 29 03:09:07 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:09:07 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-8437735b-1338-4943-bef4-6f6d4f9cbbba tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] Regex to get portals from discoverydb: ^SENDTARGETS:
Nov 29 03:09:07 user nova-compute[75234]: .*?^DiscoveryAddress: 172.16.0.220,3260.*?
Nov 29 03:09:07 user nova-compute[75234]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=75234) _get_discoverydb_portals /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:382}}
Nov 29 03:09:07 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-8437735b-1338-4943-bef4-6f6d4f9cbbba tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:0d2c0c22-2b3a-43db-b9af-e60a2514e48e', 0)] {{(pid=75234) _get_connection_devices /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:830}}
Nov 29 03:09:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:09:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.019s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:09:07 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[36101b03-4417-4cfd-a5e2-333f5ad5bbbe]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:0d2c0c22-2b3a-43db-b9af-e60a2514e48e\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:5fcd66fc-a479-4613-b1f8-e6ef12ca470a\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:09:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:09:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.023s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:09:07 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[c97a25b9-d38b-49aa-a971-b21f6cf9b157]: (4, ('tcp: [6] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0d2c0c22-2b3a-43db-b9af-e60a2514e48e (non-flash)\ntcp: [7] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5fcd66fc-a479-4613-b1f8-e6ef12ca470a (non-flash)\ntcp: [9] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672 (non-flash)\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:09:07 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-8437735b-1338-4943-bef4-6f6d4f9cbbba tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [6] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0d2c0c22-2b3a-43db-b9af-e60a2514e48e (non-flash)
Nov 29 03:09:07 user nova-compute[75234]: tcp: [7] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5fcd66fc-a479-4613-b1f8-e6ef12ca470a (non-flash)
Nov 29 03:09:07 user nova-compute[75234]: tcp: [9] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672 (non-flash)
Nov 29 03:09:07 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:09:07 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-8437735b-1338-4943-bef4-6f6d4f9cbbba tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] iscsi session list stdout=tcp: [6] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0d2c0c22-2b3a-43db-b9af-e60a2514e48e (non-flash)
Nov 29 03:09:07 user nova-compute[75234]: tcp: [7] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5fcd66fc-a479-4613-b1f8-e6ef12ca470a (non-flash)
Nov 29 03:09:07 user nova-compute[75234]: tcp: [9] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672 (non-flash)
Nov 29 03:09:07 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsi_session /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Nov 29 03:09:07 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-8437735b-1338-4943-bef4-6f6d4f9cbbba tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7f8a502bf880>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:0d2c0c22-2b3a-43db-b9af-e60a2514e48e'): ({'sdf'}, set())}) {{(pid=75234) _get_connection_devices /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:863}}
Nov 29 03:09:07 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-8437735b-1338-4943-bef4-6f6d4f9cbbba tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] Removing single pathed devices sdf {{(pid=75234) remove_connection /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:374}}
Nov 29 03:09:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:09:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.019s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:09:07 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[11114c73-126a-4d2a-bca8-c97b91176aba]: (4, ('path checker states:\nup                  3\n\npaths: 0\nbusy: False\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:09:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdf {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:09:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdf" returned: 0 in 0.020s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:09:07 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[62a573ea-0273-4a32-8613-5bb69a81b31d]: (4, ('ok\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:09:07 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-8437735b-1338-4943-bef4-6f6d4f9cbbba tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] Flushing IO for device /dev/sdf {{(pid=75234) flush_device_io /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:434}}
Nov 29 03:09:07 user nova-compute[75234]: DEBUG nova.compute.manager [req-04e70842-b59e-486a-b93b-c8b67eb05008 req-e3fef5b3-6ddb-4fbe-a502-a4439d959ebb service nova] [instance: 763a3e14-4563-4dc2-99ae-c340edbb93df] Received event network-vif-unplugged-de4ad363-fae0-4d1b-9820-5a91a27d630d {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:09:07 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-04e70842-b59e-486a-b93b-c8b67eb05008 req-e3fef5b3-6ddb-4fbe-a502-a4439d959ebb service nova] Acquiring lock "763a3e14-4563-4dc2-99ae-c340edbb93df-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:09:07 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-04e70842-b59e-486a-b93b-c8b67eb05008 req-e3fef5b3-6ddb-4fbe-a502-a4439d959ebb service nova] Lock "763a3e14-4563-4dc2-99ae-c340edbb93df-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:09:07 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-04e70842-b59e-486a-b93b-c8b67eb05008 req-e3fef5b3-6ddb-4fbe-a502-a4439d959ebb service nova] Lock "763a3e14-4563-4dc2-99ae-c340edbb93df-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:09:07 user nova-compute[75234]: DEBUG nova.compute.manager [req-04e70842-b59e-486a-b93b-c8b67eb05008 req-e3fef5b3-6ddb-4fbe-a502-a4439d959ebb service nova] [instance: 763a3e14-4563-4dc2-99ae-c340edbb93df] No waiting events found dispatching network-vif-unplugged-de4ad363-fae0-4d1b-9820-5a91a27d630d {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:09:07 user nova-compute[75234]: DEBUG nova.compute.manager [req-04e70842-b59e-486a-b93b-c8b67eb05008 req-e3fef5b3-6ddb-4fbe-a502-a4439d959ebb service nova] [instance: 763a3e14-4563-4dc2-99ae-c340edbb93df] Received event network-vif-unplugged-de4ad363-fae0-4d1b-9820-5a91a27d630d for instance with task_state deleting. {{(pid=75234) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10890}}
Nov 29 03:09:07 user nova-compute[75234]: DEBUG nova.compute.manager [req-04e70842-b59e-486a-b93b-c8b67eb05008 req-e3fef5b3-6ddb-4fbe-a502-a4439d959ebb service nova] [instance: 763a3e14-4563-4dc2-99ae-c340edbb93df] Received event network-vif-plugged-de4ad363-fae0-4d1b-9820-5a91a27d630d {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:09:07 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-04e70842-b59e-486a-b93b-c8b67eb05008 req-e3fef5b3-6ddb-4fbe-a502-a4439d959ebb service nova] Acquiring lock "763a3e14-4563-4dc2-99ae-c340edbb93df-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:09:07 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-04e70842-b59e-486a-b93b-c8b67eb05008 req-e3fef5b3-6ddb-4fbe-a502-a4439d959ebb service nova] Lock "763a3e14-4563-4dc2-99ae-c340edbb93df-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:09:07 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-04e70842-b59e-486a-b93b-c8b67eb05008 req-e3fef5b3-6ddb-4fbe-a502-a4439d959ebb service nova] Lock "763a3e14-4563-4dc2-99ae-c340edbb93df-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:09:07 user nova-compute[75234]: DEBUG nova.compute.manager [req-04e70842-b59e-486a-b93b-c8b67eb05008 req-e3fef5b3-6ddb-4fbe-a502-a4439d959ebb service nova] [instance: 763a3e14-4563-4dc2-99ae-c340edbb93df] No waiting events found dispatching network-vif-plugged-de4ad363-fae0-4d1b-9820-5a91a27d630d {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:09:07 user nova-compute[75234]: WARNING nova.compute.manager [req-04e70842-b59e-486a-b93b-c8b67eb05008 req-e3fef5b3-6ddb-4fbe-a502-a4439d959ebb service nova] [instance: 763a3e14-4563-4dc2-99ae-c340edbb93df] Received unexpected event network-vif-plugged-de4ad363-fae0-4d1b-9820-5a91a27d630d for instance with vm_state active and task_state deleting.
Nov 29 03:09:07 user nova-compute[75234]: DEBUG nova.compute.manager [req-04e70842-b59e-486a-b93b-c8b67eb05008 req-e3fef5b3-6ddb-4fbe-a502-a4439d959ebb service nova] [instance: 763a3e14-4563-4dc2-99ae-c340edbb93df] Received event network-vif-unplugged-d520a88d-ec02-45b8-adbc-484d38b9c18c {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:09:07 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-04e70842-b59e-486a-b93b-c8b67eb05008 req-e3fef5b3-6ddb-4fbe-a502-a4439d959ebb service nova] Acquiring lock "763a3e14-4563-4dc2-99ae-c340edbb93df-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:09:07 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-04e70842-b59e-486a-b93b-c8b67eb05008 req-e3fef5b3-6ddb-4fbe-a502-a4439d959ebb service nova] Lock "763a3e14-4563-4dc2-99ae-c340edbb93df-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:09:07 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-04e70842-b59e-486a-b93b-c8b67eb05008 req-e3fef5b3-6ddb-4fbe-a502-a4439d959ebb service nova] Lock "763a3e14-4563-4dc2-99ae-c340edbb93df-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:09:07 user nova-compute[75234]: DEBUG nova.compute.manager [req-04e70842-b59e-486a-b93b-c8b67eb05008 req-e3fef5b3-6ddb-4fbe-a502-a4439d959ebb service nova] [instance: 763a3e14-4563-4dc2-99ae-c340edbb93df] No waiting events found dispatching network-vif-unplugged-d520a88d-ec02-45b8-adbc-484d38b9c18c {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:09:07 user nova-compute[75234]: DEBUG nova.compute.manager [req-04e70842-b59e-486a-b93b-c8b67eb05008 req-e3fef5b3-6ddb-4fbe-a502-a4439d959ebb service nova] [instance: 763a3e14-4563-4dc2-99ae-c340edbb93df] Received event network-vif-unplugged-d520a88d-ec02-45b8-adbc-484d38b9c18c for instance with task_state deleting. {{(pid=75234) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10890}}
Nov 29 03:09:07 user nova-compute[75234]: DEBUG nova.compute.manager [req-04e70842-b59e-486a-b93b-c8b67eb05008 req-e3fef5b3-6ddb-4fbe-a502-a4439d959ebb service nova] [instance: 763a3e14-4563-4dc2-99ae-c340edbb93df] Received event network-vif-plugged-d520a88d-ec02-45b8-adbc-484d38b9c18c {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:09:07 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-04e70842-b59e-486a-b93b-c8b67eb05008 req-e3fef5b3-6ddb-4fbe-a502-a4439d959ebb service nova] Acquiring lock "763a3e14-4563-4dc2-99ae-c340edbb93df-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:09:07 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-04e70842-b59e-486a-b93b-c8b67eb05008 req-e3fef5b3-6ddb-4fbe-a502-a4439d959ebb service nova] Lock "763a3e14-4563-4dc2-99ae-c340edbb93df-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:09:07 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-04e70842-b59e-486a-b93b-c8b67eb05008 req-e3fef5b3-6ddb-4fbe-a502-a4439d959ebb service nova] Lock "763a3e14-4563-4dc2-99ae-c340edbb93df-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:09:07 user nova-compute[75234]: DEBUG nova.compute.manager [req-04e70842-b59e-486a-b93b-c8b67eb05008 req-e3fef5b3-6ddb-4fbe-a502-a4439d959ebb service nova] [instance: 763a3e14-4563-4dc2-99ae-c340edbb93df] No waiting events found dispatching network-vif-plugged-d520a88d-ec02-45b8-adbc-484d38b9c18c {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:09:07 user nova-compute[75234]: WARNING nova.compute.manager [req-04e70842-b59e-486a-b93b-c8b67eb05008 req-e3fef5b3-6ddb-4fbe-a502-a4439d959ebb service nova] [instance: 763a3e14-4563-4dc2-99ae-c340edbb93df] Received unexpected event network-vif-plugged-d520a88d-ec02-45b8-adbc-484d38b9c18c for instance with vm_state active and task_state deleting.
Nov 29 03:09:07 user nova-compute[75234]: DEBUG nova.compute.manager [req-04e70842-b59e-486a-b93b-c8b67eb05008 req-e3fef5b3-6ddb-4fbe-a502-a4439d959ebb service nova] [instance: 763a3e14-4563-4dc2-99ae-c340edbb93df] Received event network-vif-unplugged-c50f64cd-b7fa-4580-bb78-bff756a0773a {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:09:07 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-04e70842-b59e-486a-b93b-c8b67eb05008 req-e3fef5b3-6ddb-4fbe-a502-a4439d959ebb service nova] Acquiring lock "763a3e14-4563-4dc2-99ae-c340edbb93df-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:09:07 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-04e70842-b59e-486a-b93b-c8b67eb05008 req-e3fef5b3-6ddb-4fbe-a502-a4439d959ebb service nova] Lock "763a3e14-4563-4dc2-99ae-c340edbb93df-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:09:07 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-04e70842-b59e-486a-b93b-c8b67eb05008 req-e3fef5b3-6ddb-4fbe-a502-a4439d959ebb service nova] Lock "763a3e14-4563-4dc2-99ae-c340edbb93df-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:09:07 user nova-compute[75234]: DEBUG nova.compute.manager [req-04e70842-b59e-486a-b93b-c8b67eb05008 req-e3fef5b3-6ddb-4fbe-a502-a4439d959ebb service nova] [instance: 763a3e14-4563-4dc2-99ae-c340edbb93df] No waiting events found dispatching network-vif-unplugged-c50f64cd-b7fa-4580-bb78-bff756a0773a {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:09:07 user nova-compute[75234]: DEBUG nova.compute.manager [req-04e70842-b59e-486a-b93b-c8b67eb05008 req-e3fef5b3-6ddb-4fbe-a502-a4439d959ebb service nova] [instance: 763a3e14-4563-4dc2-99ae-c340edbb93df] Received event network-vif-unplugged-c50f64cd-b7fa-4580-bb78-bff756a0773a for instance with task_state deleting. {{(pid=75234) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10890}}
Nov 29 03:09:07 user nova-compute[75234]: DEBUG nova.compute.manager [req-04e70842-b59e-486a-b93b-c8b67eb05008 req-e3fef5b3-6ddb-4fbe-a502-a4439d959ebb service nova] [instance: 763a3e14-4563-4dc2-99ae-c340edbb93df] Received event network-vif-plugged-c50f64cd-b7fa-4580-bb78-bff756a0773a {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:09:07 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-04e70842-b59e-486a-b93b-c8b67eb05008 req-e3fef5b3-6ddb-4fbe-a502-a4439d959ebb service nova] Acquiring lock "763a3e14-4563-4dc2-99ae-c340edbb93df-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:09:07 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-04e70842-b59e-486a-b93b-c8b67eb05008 req-e3fef5b3-6ddb-4fbe-a502-a4439d959ebb service nova] Lock "763a3e14-4563-4dc2-99ae-c340edbb93df-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:09:07 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-04e70842-b59e-486a-b93b-c8b67eb05008 req-e3fef5b3-6ddb-4fbe-a502-a4439d959ebb service nova] Lock "763a3e14-4563-4dc2-99ae-c340edbb93df-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:09:07 user nova-compute[75234]: DEBUG nova.compute.manager [req-04e70842-b59e-486a-b93b-c8b67eb05008 req-e3fef5b3-6ddb-4fbe-a502-a4439d959ebb service nova] [instance: 763a3e14-4563-4dc2-99ae-c340edbb93df] No waiting events found dispatching network-vif-plugged-c50f64cd-b7fa-4580-bb78-bff756a0773a {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:09:07 user nova-compute[75234]: WARNING nova.compute.manager [req-04e70842-b59e-486a-b93b-c8b67eb05008 req-e3fef5b3-6ddb-4fbe-a502-a4439d959ebb service nova] [instance: 763a3e14-4563-4dc2-99ae-c340edbb93df] Received unexpected event network-vif-plugged-c50f64cd-b7fa-4580-bb78-bff756a0773a for instance with vm_state active and task_state deleting.
Nov 29 03:09:07 user nova-compute[75234]: DEBUG nova.compute.manager [req-04e70842-b59e-486a-b93b-c8b67eb05008 req-e3fef5b3-6ddb-4fbe-a502-a4439d959ebb service nova] [instance: 763a3e14-4563-4dc2-99ae-c340edbb93df] Received event network-vif-unplugged-92917a0d-59ee-4167-8693-262a21f6e156 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:09:07 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-04e70842-b59e-486a-b93b-c8b67eb05008 req-e3fef5b3-6ddb-4fbe-a502-a4439d959ebb service nova] Acquiring lock "763a3e14-4563-4dc2-99ae-c340edbb93df-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:09:07 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-04e70842-b59e-486a-b93b-c8b67eb05008 req-e3fef5b3-6ddb-4fbe-a502-a4439d959ebb service nova] Lock "763a3e14-4563-4dc2-99ae-c340edbb93df-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:09:07 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-04e70842-b59e-486a-b93b-c8b67eb05008 req-e3fef5b3-6ddb-4fbe-a502-a4439d959ebb service nova] Lock "763a3e14-4563-4dc2-99ae-c340edbb93df-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:09:07 user nova-compute[75234]: DEBUG nova.compute.manager [req-04e70842-b59e-486a-b93b-c8b67eb05008 req-e3fef5b3-6ddb-4fbe-a502-a4439d959ebb service nova] [instance: 763a3e14-4563-4dc2-99ae-c340edbb93df] No waiting events found dispatching network-vif-unplugged-92917a0d-59ee-4167-8693-262a21f6e156 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:09:07 user nova-compute[75234]: DEBUG nova.compute.manager [req-04e70842-b59e-486a-b93b-c8b67eb05008 req-e3fef5b3-6ddb-4fbe-a502-a4439d959ebb service nova] [instance: 763a3e14-4563-4dc2-99ae-c340edbb93df] Received event network-vif-unplugged-92917a0d-59ee-4167-8693-262a21f6e156 for instance with task_state deleting. {{(pid=75234) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10890}}
Nov 29 03:09:07 user nova-compute[75234]: DEBUG nova.compute.manager [req-04e70842-b59e-486a-b93b-c8b67eb05008 req-e3fef5b3-6ddb-4fbe-a502-a4439d959ebb service nova] [instance: 763a3e14-4563-4dc2-99ae-c340edbb93df] Received event network-vif-plugged-92917a0d-59ee-4167-8693-262a21f6e156 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:09:07 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-04e70842-b59e-486a-b93b-c8b67eb05008 req-e3fef5b3-6ddb-4fbe-a502-a4439d959ebb service nova] Acquiring lock "763a3e14-4563-4dc2-99ae-c340edbb93df-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:09:07 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-04e70842-b59e-486a-b93b-c8b67eb05008 req-e3fef5b3-6ddb-4fbe-a502-a4439d959ebb service nova] Lock "763a3e14-4563-4dc2-99ae-c340edbb93df-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:09:07 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-04e70842-b59e-486a-b93b-c8b67eb05008 req-e3fef5b3-6ddb-4fbe-a502-a4439d959ebb service nova] Lock "763a3e14-4563-4dc2-99ae-c340edbb93df-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:09:07 user nova-compute[75234]: DEBUG nova.compute.manager [req-04e70842-b59e-486a-b93b-c8b67eb05008 req-e3fef5b3-6ddb-4fbe-a502-a4439d959ebb service nova] [instance: 763a3e14-4563-4dc2-99ae-c340edbb93df] No waiting events found dispatching network-vif-plugged-92917a0d-59ee-4167-8693-262a21f6e156 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:09:07 user nova-compute[75234]: WARNING nova.compute.manager [req-04e70842-b59e-486a-b93b-c8b67eb05008 req-e3fef5b3-6ddb-4fbe-a502-a4439d959ebb service nova] [instance: 763a3e14-4563-4dc2-99ae-c340edbb93df] Received unexpected event network-vif-plugged-92917a0d-59ee-4167-8693-262a21f6e156 for instance with vm_state active and task_state deleting.
Nov 29 03:09:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdf {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:09:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdf" returned: 0 in 0.015s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:09:07 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[191f720d-04be-40da-834f-08205c3e5618]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:09:07 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-8437735b-1338-4943-bef4-6f6d4f9cbbba tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] Remove SCSI device /dev/sdf with /sys/block/sdf/device/delete {{(pid=75234) remove_scsi_device /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:148}}
Nov 29 03:09:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdf/device/delete {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:09:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdf/device/delete" returned: 0 in 0.051s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:09:07 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[4e2aff8e-de8f-4127-9ac6-5645a4f8a3e8]: (4, ('1', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:09:07 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-8437735b-1338-4943-bef4-6f6d4f9cbbba tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] Checking to see if SCSI volumes sdf have been removed. {{(pid=75234) wait_for_volumes_removal /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:156}}
Nov 29 03:09:07 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-8437735b-1338-4943-bef4-6f6d4f9cbbba tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] SCSI volumes sdf have been removed. {{(pid=75234) wait_for_volumes_removal /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:166}}
Nov 29 03:09:07 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-8437735b-1338-4943-bef4-6f6d4f9cbbba tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:0d2c0c22-2b3a-43db-b9af-e60a2514e48e')] {{(pid=75234) _disconnect_connection /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Nov 29 03:09:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:0d2c0c22-2b3a-43db-b9af-e60a2514e48e -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:09:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:0d2c0c22-2b3a-43db-b9af-e60a2514e48e -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.023s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:09:07 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[31ee8d12-f25a-4379-96db-6dfdf252df35]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:09:07 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-8437735b-1338-4943-bef4-6f6d4f9cbbba tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:09:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:0d2c0c22-2b3a-43db-b9af-e60a2514e48e -p 172.16.0.220:3260 --logout {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:09:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:0d2c0c22-2b3a-43db-b9af-e60a2514e48e -p 172.16.0.220:3260 --logout" returned: 0 in 0.062s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:09:07 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[86cc24cf-2903-4337-b588-d43d872d5986]: (4, ('Logging out of session [sid: 6, target: iqn.2010-10.org.openstack:0d2c0c22-2b3a-43db-b9af-e60a2514e48e, portal: 172.16.0.220,3260]\nLogout of [sid: 6, target: iqn.2010-10.org.openstack:0d2c0c22-2b3a-43db-b9af-e60a2514e48e, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:09:07 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-8437735b-1338-4943-bef4-6f6d4f9cbbba tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 6, target: iqn.2010-10.org.openstack:0d2c0c22-2b3a-43db-b9af-e60a2514e48e, portal: 172.16.0.220,3260]
Nov 29 03:09:07 user nova-compute[75234]: Logout of [sid: 6, target: iqn.2010-10.org.openstack:0d2c0c22-2b3a-43db-b9af-e60a2514e48e, portal: 172.16.0.220,3260] successful.
Nov 29 03:09:07 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:09:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:0d2c0c22-2b3a-43db-b9af-e60a2514e48e -p 172.16.0.220:3260 --op delete {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:09:08 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:0d2c0c22-2b3a-43db-b9af-e60a2514e48e -p 172.16.0.220:3260 --op delete" returned: 0 in 0.030s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:09:08 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[1863cc42-8ae3-45c7-bf5f-d1eb6e00a2a8]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:09:08 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-8437735b-1338-4943-bef4-6f6d4f9cbbba tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:09:08 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-8437735b-1338-4943-bef4-6f6d4f9cbbba tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.367s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:87}}
Nov 29 03:09:08 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-8437735b-1338-4943-bef4-6f6d4f9cbbba tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] <== disconnect_volume: return (368ms) None {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:203}}
Nov 29 03:09:08 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-8437735b-1338-4943-bef4-6f6d4f9cbbba tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] [instance: 763a3e14-4563-4dc2-99ae-c340edbb93df] Disconnected iSCSI Volume {{(pid=75234) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:79}}
Nov 29 03:09:08 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-8437735b-1338-4943-bef4-6f6d4f9cbbba tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] [instance: 763a3e14-4563-4dc2-99ae-c340edbb93df] calling os-brick to detach iSCSI Volume {{(pid=75234) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Nov 29 03:09:08 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-8437735b-1338-4943-bef4-6f6d4f9cbbba tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f8a5050c9a0>, {'target_iqn': 'iqn.2010-10.org.openstack:5fcd66fc-a479-4613-b1f8-e6ef12ca470a', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'lbfrXqMs', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdg'}, None), 'kwargs': True}}" {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:176}}
Nov 29 03:09:08 user nova-compute[75234]: WARNING os_brick.initiator.connectors.base [None req-8437735b-1338-4943-bef4-6f6d4f9cbbba tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Nov 29 03:09:08 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-8437735b-1338-4943-bef4-6f6d4f9cbbba tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:68}}
Nov 29 03:09:08 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-8437735b-1338-4943-bef4-6f6d4f9cbbba tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:73}}
Nov 29 03:09:08 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:09:08 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.019s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:09:08 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[703ed4ea-5877-4856-a252-87300df3aa59]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:5fcd66fc-a479-4613-b1f8-e6ef12ca470a\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:09:08 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-8437735b-1338-4943-bef4-6f6d4f9cbbba tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS:
Nov 29 03:09:08 user nova-compute[75234]: No targets found.
Nov 29 03:09:08 user nova-compute[75234]: iSNS:
Nov 29 03:09:08 user nova-compute[75234]: No targets found.
Nov 29 03:09:08 user nova-compute[75234]: STATIC:
Nov 29 03:09:08 user nova-compute[75234]: Target: iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672
Nov 29 03:09:08 user nova-compute[75234]:         Portal: 172.16.0.220:3260,-1
Nov 29 03:09:08 user nova-compute[75234]:                 Iface Name: default
Nov 29 03:09:08 user nova-compute[75234]: Target: iqn.2010-10.org.openstack:5fcd66fc-a479-4613-b1f8-e6ef12ca470a
Nov 29 03:09:08 user nova-compute[75234]:         Portal: 172.16.0.220:3260,-1
Nov 29 03:09:08 user nova-compute[75234]:                 Iface Name: default
Nov 29 03:09:08 user nova-compute[75234]: FIRMWARE:
Nov 29 03:09:08 user nova-compute[75234]: No targets found.
Nov 29 03:09:08 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:09:08 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-8437735b-1338-4943-bef4-6f6d4f9cbbba tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] Regex to get portals from discoverydb: ^SENDTARGETS:
Nov 29 03:09:08 user nova-compute[75234]: .*?^DiscoveryAddress: 172.16.0.220,3260.*?
Nov 29 03:09:08 user nova-compute[75234]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=75234) _get_discoverydb_portals /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:382}}
Nov 29 03:09:08 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-8437735b-1338-4943-bef4-6f6d4f9cbbba tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:5fcd66fc-a479-4613-b1f8-e6ef12ca470a', 0)] {{(pid=75234) _get_connection_devices /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:830}}
Nov 29 03:09:08 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:09:08 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.018s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:09:08 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[0a05f8df-d75a-4321-bbb7-75325aaee44a]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:5fcd66fc-a479-4613-b1f8-e6ef12ca470a\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:09:08 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:09:08 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.020s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:09:08 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[7981a433-d7be-4e88-b7b3-08403af35061]: (4, ('tcp: [7] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5fcd66fc-a479-4613-b1f8-e6ef12ca470a (non-flash)\ntcp: [9] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672 (non-flash)\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:09:08 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-8437735b-1338-4943-bef4-6f6d4f9cbbba tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [7] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5fcd66fc-a479-4613-b1f8-e6ef12ca470a (non-flash)
Nov 29 03:09:08 user nova-compute[75234]: tcp: [9] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672 (non-flash)
Nov 29 03:09:08 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:09:08 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-8437735b-1338-4943-bef4-6f6d4f9cbbba tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] iscsi session list stdout=tcp: [7] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5fcd66fc-a479-4613-b1f8-e6ef12ca470a (non-flash)
Nov 29 03:09:08 user nova-compute[75234]: tcp: [9] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672 (non-flash)
Nov 29 03:09:08 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsi_session /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Nov 29 03:09:08 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-8437735b-1338-4943-bef4-6f6d4f9cbbba tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7f8a50374b80>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:5fcd66fc-a479-4613-b1f8-e6ef12ca470a'): ({'sdg'}, set())}) {{(pid=75234) _get_connection_devices /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:863}}
Nov 29 03:09:08 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-8437735b-1338-4943-bef4-6f6d4f9cbbba tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] Removing single pathed devices sdg {{(pid=75234) remove_connection /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:374}}
Nov 29 03:09:08 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:09:08 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.020s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:09:08 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[e9aed8b4-4492-41f1-90e6-856bb0cb6013]: (4, ('path checker states:\nup                  2\n\npaths: 0\nbusy: False\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:09:08 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdg {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:09:08 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdg" returned: 0 in 0.018s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:09:08 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[60e4b9be-cf21-4a01-8610-c7b7bc133981]: (4, ('ok\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:09:08 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-8437735b-1338-4943-bef4-6f6d4f9cbbba tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] Flushing IO for device /dev/sdg {{(pid=75234) flush_device_io /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:434}}
Nov 29 03:09:08 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdg {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:09:08 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdg" returned: 0 in 0.012s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:09:08 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[549adf9a-e031-4ba4-95de-ed0e9cd53fb3]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:09:08 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-8437735b-1338-4943-bef4-6f6d4f9cbbba tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] Remove SCSI device /dev/sdg with /sys/block/sdg/device/delete {{(pid=75234) remove_scsi_device /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:148}}
Nov 29 03:09:08 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdg/device/delete {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:09:08 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdg/device/delete" returned: 0 in 0.047s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:09:08 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[69bb3ff6-7d7a-4929-bbb5-139354a76a72]: (4, ('1', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:09:08 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-8437735b-1338-4943-bef4-6f6d4f9cbbba tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] Checking to see if SCSI volumes sdg have been removed. {{(pid=75234) wait_for_volumes_removal /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:156}}
Nov 29 03:09:08 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-8437735b-1338-4943-bef4-6f6d4f9cbbba tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] SCSI volumes sdg have been removed. {{(pid=75234) wait_for_volumes_removal /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:166}}
Nov 29 03:09:08 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-8437735b-1338-4943-bef4-6f6d4f9cbbba tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:5fcd66fc-a479-4613-b1f8-e6ef12ca470a')] {{(pid=75234) _disconnect_connection /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Nov 29 03:09:08 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:5fcd66fc-a479-4613-b1f8-e6ef12ca470a -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:09:08 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:5fcd66fc-a479-4613-b1f8-e6ef12ca470a -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.025s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:09:08 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[0187c20d-b97d-433c-981a-0c541b374cf5]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:09:08 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-8437735b-1338-4943-bef4-6f6d4f9cbbba tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:09:08 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:5fcd66fc-a479-4613-b1f8-e6ef12ca470a -p 172.16.0.220:3260 --logout {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:09:08 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:5fcd66fc-a479-4613-b1f8-e6ef12ca470a -p 172.16.0.220:3260 --logout" returned: 0 in 0.048s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:09:08 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[05e6b4b6-7b21-4123-85ba-a92d9e907dc6]: (4, ('Logging out of session [sid: 7, target: iqn.2010-10.org.openstack:5fcd66fc-a479-4613-b1f8-e6ef12ca470a, portal: 172.16.0.220,3260]\nLogout of [sid: 7, target: iqn.2010-10.org.openstack:5fcd66fc-a479-4613-b1f8-e6ef12ca470a, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:09:08 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-8437735b-1338-4943-bef4-6f6d4f9cbbba tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 7, target: iqn.2010-10.org.openstack:5fcd66fc-a479-4613-b1f8-e6ef12ca470a, portal: 172.16.0.220,3260]
Nov 29 03:09:08 user nova-compute[75234]: Logout of [sid: 7, target: iqn.2010-10.org.openstack:5fcd66fc-a479-4613-b1f8-e6ef12ca470a, portal: 172.16.0.220,3260] successful.
Nov 29 03:09:08 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:09:08 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:5fcd66fc-a479-4613-b1f8-e6ef12ca470a -p 172.16.0.220:3260 --op delete {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:09:08 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:5fcd66fc-a479-4613-b1f8-e6ef12ca470a -p 172.16.0.220:3260 --op delete" returned: 0 in 0.020s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:09:08 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[da9ea439-6768-42b4-95df-94ae716f3996]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:09:08 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-8437735b-1338-4943-bef4-6f6d4f9cbbba tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:09:08 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-8437735b-1338-4943-bef4-6f6d4f9cbbba tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.300s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:87}}
Nov 29 03:09:08 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-8437735b-1338-4943-bef4-6f6d4f9cbbba tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] <== disconnect_volume: return (302ms) None {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:203}}
Nov 29 03:09:08 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-8437735b-1338-4943-bef4-6f6d4f9cbbba tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] [instance: 763a3e14-4563-4dc2-99ae-c340edbb93df] Disconnected iSCSI Volume {{(pid=75234) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:79}}
Nov 29 03:09:08 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-8437735b-1338-4943-bef4-6f6d4f9cbbba tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] [instance: 763a3e14-4563-4dc2-99ae-c340edbb93df] Deleting instance files /opt/stack/data/nova/instances/763a3e14-4563-4dc2-99ae-c340edbb93df_del
Nov 29 03:09:08 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-8437735b-1338-4943-bef4-6f6d4f9cbbba tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] [instance: 763a3e14-4563-4dc2-99ae-c340edbb93df] Deletion of /opt/stack/data/nova/instances/763a3e14-4563-4dc2-99ae-c340edbb93df_del complete
Nov 29 03:09:08 user nova-compute[75234]: INFO nova.compute.manager [None req-8437735b-1338-4943-bef4-6f6d4f9cbbba tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] [instance: 763a3e14-4563-4dc2-99ae-c340edbb93df] Took 3.62 seconds to destroy the instance on the hypervisor.
Nov 29 03:09:08 user nova-compute[75234]: DEBUG oslo.service.loopingcall [None req-8437735b-1338-4943-bef4-6f6d4f9cbbba tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=75234) func /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/loopingcall.py:435}}
Nov 29 03:09:08 user nova-compute[75234]: DEBUG nova.compute.manager [-] [instance: 763a3e14-4563-4dc2-99ae-c340edbb93df] Deallocating network for instance {{(pid=75234) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2279}}
Nov 29 03:09:08 user nova-compute[75234]: DEBUG nova.network.neutron [-] [instance: 763a3e14-4563-4dc2-99ae-c340edbb93df] deallocate_for_instance() {{(pid=75234) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1802}}
Nov 29 03:09:08 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:09:09 user nova-compute[75234]: DEBUG nova.compute.manager [req-08ee075f-80e8-40fd-beb6-6b2e44d3aa4d req-c5fc948b-8900-4a54-95ff-75b40b09a8fa service nova] [instance: 763a3e14-4563-4dc2-99ae-c340edbb93df] Received event network-vif-deleted-92917a0d-59ee-4167-8693-262a21f6e156 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:09:09 user nova-compute[75234]: INFO nova.compute.manager [req-08ee075f-80e8-40fd-beb6-6b2e44d3aa4d req-c5fc948b-8900-4a54-95ff-75b40b09a8fa service nova] [instance: 763a3e14-4563-4dc2-99ae-c340edbb93df] Neutron deleted interface 92917a0d-59ee-4167-8693-262a21f6e156; detaching it from the instance and deleting it from the info cache
Nov 29 03:09:09 user nova-compute[75234]: DEBUG nova.network.neutron [req-08ee075f-80e8-40fd-beb6-6b2e44d3aa4d req-c5fc948b-8900-4a54-95ff-75b40b09a8fa service nova] [instance: 763a3e14-4563-4dc2-99ae-c340edbb93df] Updating instance_info_cache with network_info: [{"id": "dc96090f-e855-4d3e-854c-50ea700dc8ea", "address": "fa:16:3e:9e:2b:8a", "network": {"id": "ea3baf6d-ba72-4615-bc0d-604e4fcd8e4e", "bridge": "br-int", "label": "tempest-TaggedBootDevicesTest_v242-1704264625-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.206", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "325013d2bedc498e91c2145c2b398803", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapdc96090f-e8", "ovs_interfaceid": "dc96090f-e855-4d3e-854c-50ea700dc8ea", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "2027f7a1-36e4-4434-a063-61e817758917", "address": "fa:16:3e:4d:66:94", "network": {"id": "60b448f6-90dc-4dac-b24e-9370262e6dac", "bridge": "br-int", "label": "tempest-device-tagging-net1-431494419", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.80", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "325013d2bedc498e91c2145c2b398803", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2027f7a1-36", "ovs_interfaceid": "2027f7a1-36e4-4434-a063-61e817758917", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}}, {"id": "de4ad363-fae0-4d1b-9820-5a91a27d630d", "address": "fa:16:3e:94:49:94", "network": {"id": "60b448f6-90dc-4dac-b24e-9370262e6dac", "bridge": "br-int", "label": "tempest-device-tagging-net1-431494419", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.218", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "325013d2bedc498e91c2145c2b398803", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapde4ad363-fa", "ovs_interfaceid": "de4ad363-fae0-4d1b-9820-5a91a27d630d", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}}, {"id": "d520a88d-ec02-45b8-adbc-484d38b9c18c", "address": "fa:16:3e:ac:cd:aa", "network": {"id": "60b448f6-90dc-4dac-b24e-9370262e6dac", "bridge": "br-int", "label": "tempest-device-tagging-net1-431494419", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.226", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "325013d2bedc498e91c2145c2b398803", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapd520a88d-ec", "ovs_interfaceid": "d520a88d-ec02-45b8-adbc-484d38b9c18c", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "c50f64cd-b7fa-4580-bb78-bff756a0773a", "address": "fa:16:3e:36:0b:d9", "network": {"id": "60b448f6-90dc-4dac-b24e-9370262e6dac", "bridge": "br-int", "label": "tempest-device-tagging-net1-431494419", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.18", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "325013d2bedc498e91c2145c2b398803", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc50f64cd-b7", "ovs_interfaceid": "c50f64cd-b7fa-4580-bb78-bff756a0773a", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "ef8ba969-6186-4b9a-b30f-ec9a70036bab", "address": "fa:16:3e:0f:ed:74", "network": {"id": "31e25520-c84d-451c-9132-ddc8c1e443a6", "bridge": "br-int", "label": "tempest-device-tagging-net2-653330954", "subnets": [{"cidr": "10.2.2.0/24", "dns": [], "gateway": {"address": "10.2.2.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.2.2.100", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.2.2.1"}}], "meta": {"injected": false, "tenant_id": "325013d2bedc498e91c2145c2b398803", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapef8ba969-61", "ovs_interfaceid": "ef8ba969-6186-4b9a-b30f-ec9a70036bab", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:09:09 user nova-compute[75234]: DEBUG nova.compute.manager [req-08ee075f-80e8-40fd-beb6-6b2e44d3aa4d req-c5fc948b-8900-4a54-95ff-75b40b09a8fa service nova] [instance: 763a3e14-4563-4dc2-99ae-c340edbb93df] Detach interface failed, port_id=92917a0d-59ee-4167-8693-262a21f6e156, reason: Instance 763a3e14-4563-4dc2-99ae-c340edbb93df could not be found. {{(pid=75234) _process_instance_vif_deleted_event /opt/stack/nova/nova/compute/manager.py:10946}}
Nov 29 03:09:11 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:09:11 user nova-compute[75234]: DEBUG nova.compute.manager [req-5ee117b1-7032-4aea-bce8-c55ecf4da6c1 req-6ab5ea12-ca2a-46f7-8727-6249cf0fa375 service nova] [instance: f7d0bdeb-6a05-47ac-87f2-c8380a9a4286] Received event network-changed-b5192c04-0323-48b8-a326-ddf6b21f9019 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:09:11 user nova-compute[75234]: DEBUG nova.compute.manager [req-5ee117b1-7032-4aea-bce8-c55ecf4da6c1 req-6ab5ea12-ca2a-46f7-8727-6249cf0fa375 service nova] [instance: f7d0bdeb-6a05-47ac-87f2-c8380a9a4286] Refreshing instance network info cache due to event network-changed-b5192c04-0323-48b8-a326-ddf6b21f9019. {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11117}}
Nov 29 03:09:11 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-5ee117b1-7032-4aea-bce8-c55ecf4da6c1 req-6ab5ea12-ca2a-46f7-8727-6249cf0fa375 service nova] Acquiring lock "refresh_cache-f7d0bdeb-6a05-47ac-87f2-c8380a9a4286" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:09:11 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-5ee117b1-7032-4aea-bce8-c55ecf4da6c1 req-6ab5ea12-ca2a-46f7-8727-6249cf0fa375 service nova] Acquired lock "refresh_cache-f7d0bdeb-6a05-47ac-87f2-c8380a9a4286" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:09:11 user nova-compute[75234]: DEBUG nova.network.neutron [req-5ee117b1-7032-4aea-bce8-c55ecf4da6c1 req-6ab5ea12-ca2a-46f7-8727-6249cf0fa375 service nova] [instance: f7d0bdeb-6a05-47ac-87f2-c8380a9a4286] Refreshing network info cache for port b5192c04-0323-48b8-a326-ddf6b21f9019 {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2006}}
Nov 29 03:09:12 user nova-compute[75234]: DEBUG nova.compute.manager [req-ad3e05ca-f4da-4ad0-bab7-7168cd059527 req-6f668b30-3680-4d66-acbf-9c55f280eb73 service nova] [instance: 763a3e14-4563-4dc2-99ae-c340edbb93df] Received event network-vif-deleted-ef8ba969-6186-4b9a-b30f-ec9a70036bab {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:09:12 user nova-compute[75234]: INFO nova.compute.manager [req-ad3e05ca-f4da-4ad0-bab7-7168cd059527 req-6f668b30-3680-4d66-acbf-9c55f280eb73 service nova] [instance: 763a3e14-4563-4dc2-99ae-c340edbb93df] Neutron deleted interface ef8ba969-6186-4b9a-b30f-ec9a70036bab; detaching it from the instance and deleting it from the info cache
Nov 29 03:09:12 user nova-compute[75234]: DEBUG nova.network.neutron [req-ad3e05ca-f4da-4ad0-bab7-7168cd059527 req-6f668b30-3680-4d66-acbf-9c55f280eb73 service nova] [instance: 763a3e14-4563-4dc2-99ae-c340edbb93df] Updating instance_info_cache with network_info: [{"id": "dc96090f-e855-4d3e-854c-50ea700dc8ea", "address": "fa:16:3e:9e:2b:8a", "network": {"id": "ea3baf6d-ba72-4615-bc0d-604e4fcd8e4e", "bridge": "br-int", "label": "tempest-TaggedBootDevicesTest_v242-1704264625-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.206", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "325013d2bedc498e91c2145c2b398803", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapdc96090f-e8", "ovs_interfaceid": "dc96090f-e855-4d3e-854c-50ea700dc8ea", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "2027f7a1-36e4-4434-a063-61e817758917", "address": "fa:16:3e:4d:66:94", "network": {"id": "60b448f6-90dc-4dac-b24e-9370262e6dac", "bridge": "br-int", "label": "tempest-device-tagging-net1-431494419", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.80", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "325013d2bedc498e91c2145c2b398803", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2027f7a1-36", "ovs_interfaceid": "2027f7a1-36e4-4434-a063-61e817758917", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}}, {"id": "de4ad363-fae0-4d1b-9820-5a91a27d630d", "address": "fa:16:3e:94:49:94", "network": {"id": "60b448f6-90dc-4dac-b24e-9370262e6dac", "bridge": "br-int", "label": "tempest-device-tagging-net1-431494419", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.218", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "325013d2bedc498e91c2145c2b398803", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapde4ad363-fa", "ovs_interfaceid": "de4ad363-fae0-4d1b-9820-5a91a27d630d", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}}, {"id": "d520a88d-ec02-45b8-adbc-484d38b9c18c", "address": "fa:16:3e:ac:cd:aa", "network": {"id": "60b448f6-90dc-4dac-b24e-9370262e6dac", "bridge": "br-int", "label": "tempest-device-tagging-net1-431494419", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.226", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "325013d2bedc498e91c2145c2b398803", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapd520a88d-ec", "ovs_interfaceid": "d520a88d-ec02-45b8-adbc-484d38b9c18c", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "c50f64cd-b7fa-4580-bb78-bff756a0773a", "address": "fa:16:3e:36:0b:d9", "network": {"id": "60b448f6-90dc-4dac-b24e-9370262e6dac", "bridge": "br-int", "label": "tempest-device-tagging-net1-431494419", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.18", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "325013d2bedc498e91c2145c2b398803", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc50f64cd-b7", "ovs_interfaceid": "c50f64cd-b7fa-4580-bb78-bff756a0773a", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:09:12 user nova-compute[75234]: DEBUG nova.compute.manager [req-ad3e05ca-f4da-4ad0-bab7-7168cd059527 req-6f668b30-3680-4d66-acbf-9c55f280eb73 service nova] [instance: 763a3e14-4563-4dc2-99ae-c340edbb93df] Detach interface failed, port_id=ef8ba969-6186-4b9a-b30f-ec9a70036bab, reason: Instance 763a3e14-4563-4dc2-99ae-c340edbb93df could not be found. {{(pid=75234) _process_instance_vif_deleted_event /opt/stack/nova/nova/compute/manager.py:10946}}
Nov 29 03:09:12 user nova-compute[75234]: DEBUG nova.compute.manager [req-ad3e05ca-f4da-4ad0-bab7-7168cd059527 req-6f668b30-3680-4d66-acbf-9c55f280eb73 service nova] [instance: 763a3e14-4563-4dc2-99ae-c340edbb93df] Received event network-vif-deleted-dc96090f-e855-4d3e-854c-50ea700dc8ea {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:09:12 user nova-compute[75234]: INFO nova.compute.manager [req-ad3e05ca-f4da-4ad0-bab7-7168cd059527 req-6f668b30-3680-4d66-acbf-9c55f280eb73 service nova] [instance: 763a3e14-4563-4dc2-99ae-c340edbb93df] Neutron deleted interface dc96090f-e855-4d3e-854c-50ea700dc8ea; detaching it from the instance and deleting it from the info cache
Nov 29 03:09:12 user nova-compute[75234]: DEBUG nova.network.neutron [req-ad3e05ca-f4da-4ad0-bab7-7168cd059527 req-6f668b30-3680-4d66-acbf-9c55f280eb73 service nova] [instance: 763a3e14-4563-4dc2-99ae-c340edbb93df] Updating instance_info_cache with network_info: [{"id": "2027f7a1-36e4-4434-a063-61e817758917", "address": "fa:16:3e:4d:66:94", "network": {"id": "60b448f6-90dc-4dac-b24e-9370262e6dac", "bridge": "br-int", "label": "tempest-device-tagging-net1-431494419", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.80", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "325013d2bedc498e91c2145c2b398803", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2027f7a1-36", "ovs_interfaceid": "2027f7a1-36e4-4434-a063-61e817758917", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}}, {"id": "de4ad363-fae0-4d1b-9820-5a91a27d630d", "address": "fa:16:3e:94:49:94", "network": {"id": "60b448f6-90dc-4dac-b24e-9370262e6dac", "bridge": "br-int", "label": "tempest-device-tagging-net1-431494419", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.218", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "325013d2bedc498e91c2145c2b398803", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapde4ad363-fa", "ovs_interfaceid": "de4ad363-fae0-4d1b-9820-5a91a27d630d", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": true, "delegate_create": true, "meta": {}}, {"id": "d520a88d-ec02-45b8-adbc-484d38b9c18c", "address": "fa:16:3e:ac:cd:aa", "network": {"id": "60b448f6-90dc-4dac-b24e-9370262e6dac", "bridge": "br-int", "label": "tempest-device-tagging-net1-431494419", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.226", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "325013d2bedc498e91c2145c2b398803", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapd520a88d-ec", "ovs_interfaceid": "d520a88d-ec02-45b8-adbc-484d38b9c18c", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "c50f64cd-b7fa-4580-bb78-bff756a0773a", "address": "fa:16:3e:36:0b:d9", "network": {"id": "60b448f6-90dc-4dac-b24e-9370262e6dac", "bridge": "br-int", "label": "tempest-device-tagging-net1-431494419", "subnets": [{"cidr": "10.1.1.0/24", "dns": [], "gateway": {"address": "10.1.1.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.1.1.18", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.1.1.1"}}], "meta": {"injected": false, "tenant_id": "325013d2bedc498e91c2145c2b398803", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc50f64cd-b7", "ovs_interfaceid": "c50f64cd-b7fa-4580-bb78-bff756a0773a", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:09:12 user nova-compute[75234]: DEBUG nova.compute.manager [req-ad3e05ca-f4da-4ad0-bab7-7168cd059527 req-6f668b30-3680-4d66-acbf-9c55f280eb73 service nova] [instance: 763a3e14-4563-4dc2-99ae-c340edbb93df] Detach interface failed, port_id=dc96090f-e855-4d3e-854c-50ea700dc8ea, reason: Instance 763a3e14-4563-4dc2-99ae-c340edbb93df could not be found. {{(pid=75234) _process_instance_vif_deleted_event /opt/stack/nova/nova/compute/manager.py:10946}}
Nov 29 03:09:12 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-117159fd-f2d8-4878-ada7-68e5f6e62872 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Acquiring lock "062753a6-8f39-4e80-a549-9f0f5a4b877a" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:09:12 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-117159fd-f2d8-4878-ada7-68e5f6e62872 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Lock "062753a6-8f39-4e80-a549-9f0f5a4b877a" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:09:12 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:09:12 user nova-compute[75234]: DEBUG nova.compute.manager [None req-117159fd-f2d8-4878-ada7-68e5f6e62872 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] [instance: 062753a6-8f39-4e80-a549-9f0f5a4b877a] Starting instance... {{(pid=75234) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2422}}
Nov 29 03:09:12 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-117159fd-f2d8-4878-ada7-68e5f6e62872 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:09:12 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-117159fd-f2d8-4878-ada7-68e5f6e62872 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:09:12 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-117159fd-f2d8-4878-ada7-68e5f6e62872 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=75234) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2366}}
Nov 29 03:09:12 user nova-compute[75234]: INFO nova.compute.claims [None req-117159fd-f2d8-4878-ada7-68e5f6e62872 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] [instance: 062753a6-8f39-4e80-a549-9f0f5a4b877a] Claim successful on node user
Nov 29 03:09:12 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-cf00e939-67db-4afa-b7b4-2ac67871e737 tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] Acquiring lock "f7d0bdeb-6a05-47ac-87f2-c8380a9a4286" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:09:12 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-cf00e939-67db-4afa-b7b4-2ac67871e737 tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] Lock "f7d0bdeb-6a05-47ac-87f2-c8380a9a4286" acquired by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: waited 0.002s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:09:12 user nova-compute[75234]: DEBUG nova.objects.instance [None req-cf00e939-67db-4afa-b7b4-2ac67871e737 tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] Lazy-loading 'flavor' on Instance uuid f7d0bdeb-6a05-47ac-87f2-c8380a9a4286 {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:09:12 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-cf00e939-67db-4afa-b7b4-2ac67871e737 tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] Lock "f7d0bdeb-6a05-47ac-87f2-c8380a9a4286" "released" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: held 0.054s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:09:12 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-cf00e939-67db-4afa-b7b4-2ac67871e737 tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] Acquiring lock "f7d0bdeb-6a05-47ac-87f2-c8380a9a4286" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:09:12 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-cf00e939-67db-4afa-b7b4-2ac67871e737 tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] Lock "f7d0bdeb-6a05-47ac-87f2-c8380a9a4286" acquired by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:09:12 user nova-compute[75234]: INFO nova.compute.manager [None req-cf00e939-67db-4afa-b7b4-2ac67871e737 tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] [instance: f7d0bdeb-6a05-47ac-87f2-c8380a9a4286] Attaching volume ada91c26-ca97-4b8b-98bb-a310d0627691 to /dev/vdb
Nov 29 03:09:12 user nova-compute[75234]: DEBUG os_brick.utils [None req-cf00e939-67db-4afa-b7b4-2ac67871e737 tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:176}}
Nov 29 03:09:12 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:09:12 user nova-compute[75234]: DEBUG nova.compute.provider_tree [None req-117159fd-f2d8-4878-ada7-68e5f6e62872 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Inventory has not changed in ProviderTree for provider: 4d291220-7c37-4839-a0e3-b23c66502b74 {{(pid=75234) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Nov 29 03:09:12 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.013s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:09:12 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[d498411d-87a9-44aa-868d-3e7cb69144a6]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:dd1368c5ba2c\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:09:12 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt -v / -n -o SOURCE {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:09:12 user nova-compute[75234]: DEBUG nova.scheduler.client.report [None req-117159fd-f2d8-4878-ada7-68e5f6e62872 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Inventory has not changed for provider 4d291220-7c37-4839-a0e3-b23c66502b74 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=75234) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Nov 29 03:09:12 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt -v / -n -o SOURCE" returned: 0 in 0.016s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:09:12 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[c37266b9-46ea-4e7b-9a88-d51a6540af96]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:09:12 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:09:12 user nova-compute[75234]: DEBUG nova.network.neutron [req-5ee117b1-7032-4aea-bce8-c55ecf4da6c1 req-6ab5ea12-ca2a-46f7-8727-6249cf0fa375 service nova] [instance: f7d0bdeb-6a05-47ac-87f2-c8380a9a4286] Updated VIF entry in instance network info cache for port b5192c04-0323-48b8-a326-ddf6b21f9019. {{(pid=75234) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3481}}
Nov 29 03:09:12 user nova-compute[75234]: DEBUG nova.network.neutron [req-5ee117b1-7032-4aea-bce8-c55ecf4da6c1 req-6ab5ea12-ca2a-46f7-8727-6249cf0fa375 service nova] [instance: f7d0bdeb-6a05-47ac-87f2-c8380a9a4286] Updating instance_info_cache with network_info: [{"id": "b5192c04-0323-48b8-a326-ddf6b21f9019", "address": "fa:16:3e:59:dd:b5", "network": {"id": "22b1262f-108b-45c1-91cb-a0f77f9bfccf", "bridge": "br-int", "label": "tempest-TestServerVolumeAttachmentScenario-1154356954-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.139", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "b793707804a24961968ac432a37d9581", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapb5192c04-03", "ovs_interfaceid": "b5192c04-0323-48b8-a326-ddf6b21f9019", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:09:12 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.019s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:09:12 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[7316ec90-6d43-4460-9560-a9f9df08e673]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:09:12 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-117159fd-f2d8-4878-ada7-68e5f6e62872 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.406s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:09:12 user nova-compute[75234]: DEBUG nova.compute.manager [None req-117159fd-f2d8-4878-ada7-68e5f6e62872 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] [instance: 062753a6-8f39-4e80-a549-9f0f5a4b877a] Start building networks asynchronously for instance. {{(pid=75234) _build_resources /opt/stack/nova/nova/compute/manager.py:2819}}
Nov 29 03:09:12 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-5ee117b1-7032-4aea-bce8-c55ecf4da6c1 req-6ab5ea12-ca2a-46f7-8727-6249cf0fa375 service nova] Releasing lock "refresh_cache-f7d0bdeb-6a05-47ac-87f2-c8380a9a4286" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:09:12 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[ed7b1841-6ab2-447c-9edd-acdbe003e0f6]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:09:12 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-cf00e939-67db-4afa-b7b4-2ac67871e737 tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] Running cmd (subprocess): nvme version {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:09:12 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-cf00e939-67db-4afa-b7b4-2ac67871e737 tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] 'nvme version' failed. Not Retrying. {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:473}}
Nov 29 03:09:12 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.nvmeof [None req-cf00e939-67db-4afa-b7b4-2ac67871e737 tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] nvme not present on system {{(pid=75234) nvme_present /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/nvmeof.py:756}}
Nov 29 03:09:12 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:09:12 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:473}}
Nov 29 03:09:12 user nova-compute[75234]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Nov 29 03:09:12 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[b7694e1f-3bb9-4e4e-8ec4-4f67b65efdb3]: (4, '') {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:09:12 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.lightos [None req-cf00e939-67db-4afa-b7b4-2ac67871e737 tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=75234) find_dsc /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/lightos.py:98}}
Nov 29 03:09:12 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.lightos [None req-cf00e939-67db-4afa-b7b4-2ac67871e737 tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=75234) get_connector_properties /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/lightos.py:76}}
Nov 29 03:09:12 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.lightos [None req-cf00e939-67db-4afa-b7b4-2ac67871e737 tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] LIGHTOS: no hostnqn found. {{(pid=75234) get_connector_properties /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/lightos.py:84}}
Nov 29 03:09:12 user nova-compute[75234]: DEBUG os_brick.utils [None req-cf00e939-67db-4afa-b7b4-2ac67871e737 tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] <== get_connector_properties: return (184ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:dd1368c5ba2c', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:203}}
Nov 29 03:09:12 user nova-compute[75234]: DEBUG nova.virt.block_device [None req-cf00e939-67db-4afa-b7b4-2ac67871e737 tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] [instance: f7d0bdeb-6a05-47ac-87f2-c8380a9a4286] Updating existing volume attachment record: 76e68ee9-f0b0-4266-8d7a-988c10d96ea7 {{(pid=75234) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Nov 29 03:09:13 user nova-compute[75234]: DEBUG nova.compute.manager [None req-117159fd-f2d8-4878-ada7-68e5f6e62872 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] [instance: 062753a6-8f39-4e80-a549-9f0f5a4b877a] Allocating IP information in the background. {{(pid=75234) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1972}}
Nov 29 03:09:13 user nova-compute[75234]: DEBUG nova.network.neutron [None req-117159fd-f2d8-4878-ada7-68e5f6e62872 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] [instance: 062753a6-8f39-4e80-a549-9f0f5a4b877a] allocate_for_instance() {{(pid=75234) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1156}}
Nov 29 03:09:13 user nova-compute[75234]: DEBUG nova.policy [None req-117159fd-f2d8-4878-ada7-68e5f6e62872 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '154f3f4f57514450baf8f22d67768bc9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e17150005a9a449f9373b91e1c00083b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=75234) authorize /opt/stack/nova/nova/policy.py:203}}
Nov 29 03:09:13 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-117159fd-f2d8-4878-ada7-68e5f6e62872 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] [instance: 062753a6-8f39-4e80-a549-9f0f5a4b877a] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Nov 29 03:09:13 user nova-compute[75234]: DEBUG nova.compute.manager [None req-117159fd-f2d8-4878-ada7-68e5f6e62872 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] [instance: 062753a6-8f39-4e80-a549-9f0f5a4b877a] Start building block device mappings for instance. {{(pid=75234) _build_resources /opt/stack/nova/nova/compute/manager.py:2854}}
Nov 29 03:09:13 user nova-compute[75234]: DEBUG nova.network.neutron [-] [instance: 763a3e14-4563-4dc2-99ae-c340edbb93df] Updating instance_info_cache with network_info: [] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:09:13 user nova-compute[75234]: INFO nova.compute.manager [-] [instance: 763a3e14-4563-4dc2-99ae-c340edbb93df] Took 4.88 seconds to deallocate network for instance.
Nov 29 03:09:13 user nova-compute[75234]: DEBUG nova.compute.manager [None req-117159fd-f2d8-4878-ada7-68e5f6e62872 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] [instance: 062753a6-8f39-4e80-a549-9f0f5a4b877a] Start spawning the instance on the hypervisor. {{(pid=75234) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2628}}
Nov 29 03:09:13 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-117159fd-f2d8-4878-ada7-68e5f6e62872 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] [instance: 062753a6-8f39-4e80-a549-9f0f5a4b877a] Creating instance directory {{(pid=75234) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4727}}
Nov 29 03:09:13 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-117159fd-f2d8-4878-ada7-68e5f6e62872 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] [instance: 062753a6-8f39-4e80-a549-9f0f5a4b877a] Creating image(s)
Nov 29 03:09:13 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-117159fd-f2d8-4878-ada7-68e5f6e62872 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Acquiring lock "/opt/stack/data/nova/instances/062753a6-8f39-4e80-a549-9f0f5a4b877a/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:09:13 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-117159fd-f2d8-4878-ada7-68e5f6e62872 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Lock "/opt/stack/data/nova/instances/062753a6-8f39-4e80-a549-9f0f5a4b877a/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:09:13 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-117159fd-f2d8-4878-ada7-68e5f6e62872 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Lock "/opt/stack/data/nova/instances/062753a6-8f39-4e80-a549-9f0f5a4b877a/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.002s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:09:13 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-117159fd-f2d8-4878-ada7-68e5f6e62872 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:09:13 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-117159fd-f2d8-4878-ada7-68e5f6e62872 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json" returned: 0 in 0.182s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:09:13 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-117159fd-f2d8-4878-ada7-68e5f6e62872 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Acquiring lock "80686e29dc9db6600d03e85cb62af298e12d7918" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:09:13 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-117159fd-f2d8-4878-ada7-68e5f6e62872 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Lock "80686e29dc9db6600d03e85cb62af298e12d7918" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.002s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:09:13 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-117159fd-f2d8-4878-ada7-68e5f6e62872 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:09:13 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-117159fd-f2d8-4878-ada7-68e5f6e62872 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json" returned: 0 in 0.195s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:09:13 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-117159fd-f2d8-4878-ada7-68e5f6e62872 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918,backing_fmt=raw /opt/stack/data/nova/instances/062753a6-8f39-4e80-a549-9f0f5a4b877a/disk 1073741824 {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:09:13 user nova-compute[75234]: DEBUG nova.network.neutron [None req-117159fd-f2d8-4878-ada7-68e5f6e62872 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] [instance: 062753a6-8f39-4e80-a549-9f0f5a4b877a] Successfully created port: 8f6751f1-fc65-4842-b77f-1dfe26912afb {{(pid=75234) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:548}}
Nov 29 03:09:13 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-65aec05b-a901-4f17-a4e0-2e822560de93 tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] Acquiring lock "ac620533-8274-4674-b241-f48526bed3c3" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:09:13 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-65aec05b-a901-4f17-a4e0-2e822560de93 tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] Lock "ac620533-8274-4674-b241-f48526bed3c3" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:09:13 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-117159fd-f2d8-4878-ada7-68e5f6e62872 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918,backing_fmt=raw /opt/stack/data/nova/instances/062753a6-8f39-4e80-a549-9f0f5a4b877a/disk 1073741824" returned: 0 in 0.068s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:09:13 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-117159fd-f2d8-4878-ada7-68e5f6e62872 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Lock "80686e29dc9db6600d03e85cb62af298e12d7918" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.269s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:09:13 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-117159fd-f2d8-4878-ada7-68e5f6e62872 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:09:13 user nova-compute[75234]: DEBUG nova.compute.manager [None req-65aec05b-a901-4f17-a4e0-2e822560de93 tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] [instance: ac620533-8274-4674-b241-f48526bed3c3] Starting instance... {{(pid=75234) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2422}}
Nov 29 03:09:13 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-65aec05b-a901-4f17-a4e0-2e822560de93 tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:09:13 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-65aec05b-a901-4f17-a4e0-2e822560de93 tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:09:13 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-65aec05b-a901-4f17-a4e0-2e822560de93 tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=75234) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2366}}
Nov 29 03:09:13 user nova-compute[75234]: INFO nova.compute.claims [None req-65aec05b-a901-4f17-a4e0-2e822560de93 tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] [instance: ac620533-8274-4674-b241-f48526bed3c3] Claim successful on node user
Nov 29 03:09:13 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-117159fd-f2d8-4878-ada7-68e5f6e62872 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json" returned: 0 in 0.173s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:09:13 user nova-compute[75234]: DEBUG nova.virt.disk.api [None req-117159fd-f2d8-4878-ada7-68e5f6e62872 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Checking if we can resize image /opt/stack/data/nova/instances/062753a6-8f39-4e80-a549-9f0f5a4b877a/disk. size=1073741824 {{(pid=75234) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:166}}
Nov 29 03:09:13 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-117159fd-f2d8-4878-ada7-68e5f6e62872 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/062753a6-8f39-4e80-a549-9f0f5a4b877a/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:09:14 user nova-compute[75234]: DEBUG nova.compute.manager [req-6eaa8de3-5062-4e74-b10a-afd973453465 req-d7ab6f58-eb6c-49ef-ac7e-84a4731c3459 service nova] [instance: 763a3e14-4563-4dc2-99ae-c340edbb93df] Received event network-vif-deleted-d520a88d-ec02-45b8-adbc-484d38b9c18c {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:09:14 user nova-compute[75234]: DEBUG nova.compute.manager [req-6eaa8de3-5062-4e74-b10a-afd973453465 req-d7ab6f58-eb6c-49ef-ac7e-84a4731c3459 service nova] [instance: 763a3e14-4563-4dc2-99ae-c340edbb93df] Received event network-vif-deleted-c50f64cd-b7fa-4580-bb78-bff756a0773a {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:09:14 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-117159fd-f2d8-4878-ada7-68e5f6e62872 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/062753a6-8f39-4e80-a549-9f0f5a4b877a/disk --force-share --output=json" returned: 0 in 0.187s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:09:14 user nova-compute[75234]: DEBUG nova.virt.disk.api [None req-117159fd-f2d8-4878-ada7-68e5f6e62872 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Cannot resize image /opt/stack/data/nova/instances/062753a6-8f39-4e80-a549-9f0f5a4b877a/disk to a smaller size. {{(pid=75234) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:172}}
Nov 29 03:09:14 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-117159fd-f2d8-4878-ada7-68e5f6e62872 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] [instance: 062753a6-8f39-4e80-a549-9f0f5a4b877a] Created local disks {{(pid=75234) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4861}}
Nov 29 03:09:14 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-117159fd-f2d8-4878-ada7-68e5f6e62872 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] [instance: 062753a6-8f39-4e80-a549-9f0f5a4b877a] Ensure instance console log exists: /opt/stack/data/nova/instances/062753a6-8f39-4e80-a549-9f0f5a4b877a/console.log {{(pid=75234) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4613}}
Nov 29 03:09:14 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-117159fd-f2d8-4878-ada7-68e5f6e62872 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:09:14 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-117159fd-f2d8-4878-ada7-68e5f6e62872 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:09:14 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-117159fd-f2d8-4878-ada7-68e5f6e62872 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:09:14 user nova-compute[75234]: DEBUG nova.compute.provider_tree [None req-65aec05b-a901-4f17-a4e0-2e822560de93 tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] Inventory has not changed in ProviderTree for provider: 4d291220-7c37-4839-a0e3-b23c66502b74 {{(pid=75234) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Nov 29 03:09:14 user nova-compute[75234]: DEBUG nova.scheduler.client.report [None req-65aec05b-a901-4f17-a4e0-2e822560de93 tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] Inventory has not changed for provider 4d291220-7c37-4839-a0e3-b23c66502b74 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=75234) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Nov 29 03:09:14 user nova-compute[75234]: DEBUG nova.network.neutron [None req-117159fd-f2d8-4878-ada7-68e5f6e62872 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] [instance: 062753a6-8f39-4e80-a549-9f0f5a4b877a] Successfully updated port: 8f6751f1-fc65-4842-b77f-1dfe26912afb {{(pid=75234) _update_port /opt/stack/nova/nova/network/neutron.py:586}}
Nov 29 03:09:14 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-65aec05b-a901-4f17-a4e0-2e822560de93 tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.530s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:09:14 user nova-compute[75234]: DEBUG nova.compute.manager [None req-65aec05b-a901-4f17-a4e0-2e822560de93 tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] [instance: ac620533-8274-4674-b241-f48526bed3c3] Start building networks asynchronously for instance. {{(pid=75234) _build_resources /opt/stack/nova/nova/compute/manager.py:2819}}
Nov 29 03:09:14 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-117159fd-f2d8-4878-ada7-68e5f6e62872 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Acquiring lock "refresh_cache-062753a6-8f39-4e80-a549-9f0f5a4b877a" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:09:14 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-117159fd-f2d8-4878-ada7-68e5f6e62872 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Acquired lock "refresh_cache-062753a6-8f39-4e80-a549-9f0f5a4b877a" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:09:14 user nova-compute[75234]: DEBUG nova.network.neutron [None req-117159fd-f2d8-4878-ada7-68e5f6e62872 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] [instance: 062753a6-8f39-4e80-a549-9f0f5a4b877a] Building network info cache for instance {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2009}}
Nov 29 03:09:14 user nova-compute[75234]: DEBUG nova.compute.manager [None req-65aec05b-a901-4f17-a4e0-2e822560de93 tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] [instance: ac620533-8274-4674-b241-f48526bed3c3] Allocating IP information in the background. {{(pid=75234) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1972}}
Nov 29 03:09:14 user nova-compute[75234]: DEBUG nova.network.neutron [None req-65aec05b-a901-4f17-a4e0-2e822560de93 tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] [instance: ac620533-8274-4674-b241-f48526bed3c3] allocate_for_instance() {{(pid=75234) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1156}}
Nov 29 03:09:14 user nova-compute[75234]: DEBUG nova.network.neutron [None req-117159fd-f2d8-4878-ada7-68e5f6e62872 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] [instance: 062753a6-8f39-4e80-a549-9f0f5a4b877a] Instance cache missing network info. {{(pid=75234) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3322}}
Nov 29 03:09:14 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-65aec05b-a901-4f17-a4e0-2e822560de93 tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] [instance: ac620533-8274-4674-b241-f48526bed3c3] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Nov 29 03:09:14 user nova-compute[75234]: DEBUG nova.compute.manager [None req-65aec05b-a901-4f17-a4e0-2e822560de93 tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] [instance: ac620533-8274-4674-b241-f48526bed3c3] Start building block device mappings for instance. {{(pid=75234) _build_resources /opt/stack/nova/nova/compute/manager.py:2854}}
Nov 29 03:09:14 user nova-compute[75234]: DEBUG nova.compute.manager [None req-65aec05b-a901-4f17-a4e0-2e822560de93 tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] [instance: ac620533-8274-4674-b241-f48526bed3c3] Start spawning the instance on the hypervisor. {{(pid=75234) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2628}}
Nov 29 03:09:14 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-65aec05b-a901-4f17-a4e0-2e822560de93 tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] [instance: ac620533-8274-4674-b241-f48526bed3c3] Creating instance directory {{(pid=75234) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4727}}
Nov 29 03:09:14 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-65aec05b-a901-4f17-a4e0-2e822560de93 tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] [instance: ac620533-8274-4674-b241-f48526bed3c3] Creating image(s)
Nov 29 03:09:14 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-65aec05b-a901-4f17-a4e0-2e822560de93 tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] Acquiring lock "/opt/stack/data/nova/instances/ac620533-8274-4674-b241-f48526bed3c3/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:09:14 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-65aec05b-a901-4f17-a4e0-2e822560de93 tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] Lock "/opt/stack/data/nova/instances/ac620533-8274-4674-b241-f48526bed3c3/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:09:14 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-65aec05b-a901-4f17-a4e0-2e822560de93 tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] Lock "/opt/stack/data/nova/instances/ac620533-8274-4674-b241-f48526bed3c3/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.002s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:09:14 user nova-compute[75234]: DEBUG nova.policy [None req-65aec05b-a901-4f17-a4e0-2e822560de93 tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '940a684aa3b6419798bd87562bbc5d23', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '46577f787c244fb2a4db052a0c42eceb', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=75234) authorize /opt/stack/nova/nova/policy.py:203}}
Nov 29 03:09:14 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-65aec05b-a901-4f17-a4e0-2e822560de93 tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:09:14 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-65aec05b-a901-4f17-a4e0-2e822560de93 tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json" returned: 0 in 0.175s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:09:14 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-65aec05b-a901-4f17-a4e0-2e822560de93 tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] Acquiring lock "80686e29dc9db6600d03e85cb62af298e12d7918" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:09:14 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-65aec05b-a901-4f17-a4e0-2e822560de93 tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] Lock "80686e29dc9db6600d03e85cb62af298e12d7918" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.002s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:09:14 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-65aec05b-a901-4f17-a4e0-2e822560de93 tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:09:15 user nova-compute[75234]: DEBUG nova.network.neutron [None req-117159fd-f2d8-4878-ada7-68e5f6e62872 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] [instance: 062753a6-8f39-4e80-a549-9f0f5a4b877a] Updating instance_info_cache with network_info: [{"id": "8f6751f1-fc65-4842-b77f-1dfe26912afb", "address": "fa:16:3e:62:4a:5a", "network": {"id": "aad085f7-106b-4de6-93f5-f86bb66ec962", "bridge": "br-int", "label": "tempest-AttachVolumeShelveTestJSON-775984268-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "e17150005a9a449f9373b91e1c00083b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap8f6751f1-fc", "ovs_interfaceid": "8f6751f1-fc65-4842-b77f-1dfe26912afb", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:09:15 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-117159fd-f2d8-4878-ada7-68e5f6e62872 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Releasing lock "refresh_cache-062753a6-8f39-4e80-a549-9f0f5a4b877a" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:09:15 user nova-compute[75234]: DEBUG nova.compute.manager [None req-117159fd-f2d8-4878-ada7-68e5f6e62872 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] [instance: 062753a6-8f39-4e80-a549-9f0f5a4b877a] Instance network_info: |[{"id": "8f6751f1-fc65-4842-b77f-1dfe26912afb", "address": "fa:16:3e:62:4a:5a", "network": {"id": "aad085f7-106b-4de6-93f5-f86bb66ec962", "bridge": "br-int", "label": "tempest-AttachVolumeShelveTestJSON-775984268-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "e17150005a9a449f9373b91e1c00083b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap8f6751f1-fc", "ovs_interfaceid": "8f6751f1-fc65-4842-b77f-1dfe26912afb", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=75234) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1987}}
Nov 29 03:09:15 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-117159fd-f2d8-4878-ada7-68e5f6e62872 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] [instance: 062753a6-8f39-4e80-a549-9f0f5a4b877a] Start _get_guest_xml network_info=[{"id": "8f6751f1-fc65-4842-b77f-1dfe26912afb", "address": "fa:16:3e:62:4a:5a", "network": {"id": "aad085f7-106b-4de6-93f5-f86bb66ec962", "bridge": "br-int", "label": "tempest-AttachVolumeShelveTestJSON-775984268-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "e17150005a9a449f9373b91e1c00083b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap8f6751f1-fc", "ovs_interfaceid": "8f6751f1-fc65-4842-b77f-1dfe26912afb", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum='c8fc807773e5354afe61636071771906',container_format='bare',created_at=2023-11-29T03:02:15Z,direct_url=<?>,disk_format='qcow2',id=96c30ba7-9b73-4a30-8ba4-7138879573b7,min_disk=0,min_ram=0,name='cirros-0.6.2-x86_64-disk',owner='b18e380921c748b78517b6f1211cd0f1',properties=ImageMetaProps,protected=<?>,size=21430272,status='active',tags=<?>,updated_at=2023-11-29T03:02:17Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [{'device_name': '/dev/vda', 'encryption_format': None, 'encrypted': False, 'device_type': 'disk', 'size': 0, 'disk_bus': 'virtio', 'boot_index': 0, 'encryption_options': None, 'encryption_secret_uuid': None, 'guest_format': None, 'image_id': '96c30ba7-9b73-4a30-8ba4-7138879573b7'}], 'ephemerals': [], 'block_device_mapping': [], 'swap': None} {{(pid=75234) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7562}}
Nov 29 03:09:15 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-117159fd-f2d8-4878-ada7-68e5f6e62872 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:09:15 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-117159fd-f2d8-4878-ada7-68e5f6e62872 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:09:15 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-117159fd-f2d8-4878-ada7-68e5f6e62872 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Searching host: 'user' for CPU controller through CGroups V1... {{(pid=75234) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1652}}
Nov 29 03:09:15 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-117159fd-f2d8-4878-ada7-68e5f6e62872 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] CPU controller missing on host. {{(pid=75234) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1662}}
Nov 29 03:09:15 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-65aec05b-a901-4f17-a4e0-2e822560de93 tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json" returned: 0 in 0.170s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:09:15 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-65aec05b-a901-4f17-a4e0-2e822560de93 tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918,backing_fmt=raw /opt/stack/data/nova/instances/ac620533-8274-4674-b241-f48526bed3c3/disk 1073741824 {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:09:15 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-65aec05b-a901-4f17-a4e0-2e822560de93 tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918,backing_fmt=raw /opt/stack/data/nova/instances/ac620533-8274-4674-b241-f48526bed3c3/disk 1073741824" returned: 0 in 0.053s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:09:15 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-65aec05b-a901-4f17-a4e0-2e822560de93 tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] Lock "80686e29dc9db6600d03e85cb62af298e12d7918" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.227s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:09:15 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-65aec05b-a901-4f17-a4e0-2e822560de93 tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:09:15 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-117159fd-f2d8-4878-ada7-68e5f6e62872 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Searching host: 'user' for CPU controller through CGroups V2... {{(pid=75234) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1671}}
Nov 29 03:09:15 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-117159fd-f2d8-4878-ada7-68e5f6e62872 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] CPU controller found on host. {{(pid=75234) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1678}}
Nov 29 03:09:15 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-117159fd-f2d8-4878-ada7-68e5f6e62872 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=75234) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5400}}
Nov 29 03:09:15 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-117159fd-f2d8-4878-ada7-68e5f6e62872 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-11-29T03:02:04Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='1',id=6,is_public=True,memory_mb=512,name='m1.tiny',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='c8fc807773e5354afe61636071771906',container_format='bare',created_at=2023-11-29T03:02:15Z,direct_url=<?>,disk_format='qcow2',id=96c30ba7-9b73-4a30-8ba4-7138879573b7,min_disk=0,min_ram=0,name='cirros-0.6.2-x86_64-disk',owner='b18e380921c748b78517b6f1211cd0f1',properties=ImageMetaProps,protected=<?>,size=21430272,status='active',tags=<?>,updated_at=2023-11-29T03:02:17Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:563}}
Nov 29 03:09:15 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-117159fd-f2d8-4878-ada7-68e5f6e62872 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Flavor limits 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:348}}
Nov 29 03:09:15 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-117159fd-f2d8-4878-ada7-68e5f6e62872 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Image limits 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:352}}
Nov 29 03:09:15 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-117159fd-f2d8-4878-ada7-68e5f6e62872 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Flavor pref 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:388}}
Nov 29 03:09:15 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-117159fd-f2d8-4878-ada7-68e5f6e62872 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Image pref 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:392}}
Nov 29 03:09:15 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-117159fd-f2d8-4878-ada7-68e5f6e62872 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:430}}
Nov 29 03:09:15 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-117159fd-f2d8-4878-ada7-68e5f6e62872 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:569}}
Nov 29 03:09:15 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-117159fd-f2d8-4878-ada7-68e5f6e62872 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=75234) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:471}}
Nov 29 03:09:15 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-117159fd-f2d8-4878-ada7-68e5f6e62872 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Got 1 possible topologies {{(pid=75234) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:501}}
Nov 29 03:09:15 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-117159fd-f2d8-4878-ada7-68e5f6e62872 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:575}}
Nov 29 03:09:15 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-117159fd-f2d8-4878-ada7-68e5f6e62872 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:577}}
Nov 29 03:09:15 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-117159fd-f2d8-4878-ada7-68e5f6e62872 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-11-29T03:09:11Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description=None,display_name='tempest-AttachVolumeShelveTestJSON-server-2002337175',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-attachvolumeshelvetestjson-server-2002337175',id=14,image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBJL1BPeb1pVpN5A24HF0gsy7slhuswFxR+GxX0R0LvMX9QS8GL1hvY7OFzvrLw6S/VLmgqgKQ7HFXhrJDiWyf/i/jTyWtVuorLQtfVudKMIFZjmVfY2JDvM7kwU/YbL6jQ==',key_name='tempest-keypair-1961974262',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='e17150005a9a449f9373b91e1c00083b',ramdisk_id='',reservation_id='r-p3yfidye',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-AttachVolumeShelveTestJSON-1180535246',owner_user_name='tempest-AttachVolumeShelveTestJSON-1180535246-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-11-29T03:09:13Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='154f3f4f57514450baf8f22d67768bc9',uuid=062753a6-8f39-4e80-a549-9f0f5a4b877a,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "8f6751f1-fc65-4842-b77f-1dfe26912afb", "address": "fa:16:3e:62:4a:5a", "network": {"id": "aad085f7-106b-4de6-93f5-f86bb66ec962", "bridge": "br-int", "label": "tempest-AttachVolumeShelveTestJSON-775984268-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "e17150005a9a449f9373b91e1c00083b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap8f6751f1-fc", "ovs_interfaceid": "8f6751f1-fc65-4842-b77f-1dfe26912afb", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=75234) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Nov 29 03:09:15 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-117159fd-f2d8-4878-ada7-68e5f6e62872 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Converting VIF {"id": "8f6751f1-fc65-4842-b77f-1dfe26912afb", "address": "fa:16:3e:62:4a:5a", "network": {"id": "aad085f7-106b-4de6-93f5-f86bb66ec962", "bridge": "br-int", "label": "tempest-AttachVolumeShelveTestJSON-775984268-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "e17150005a9a449f9373b91e1c00083b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap8f6751f1-fc", "ovs_interfaceid": "8f6751f1-fc65-4842-b77f-1dfe26912afb", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:09:15 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-117159fd-f2d8-4878-ada7-68e5f6e62872 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:62:4a:5a,bridge_name='br-int',has_traffic_filtering=True,id=8f6751f1-fc65-4842-b77f-1dfe26912afb,network=Network(aad085f7-106b-4de6-93f5-f86bb66ec962),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap8f6751f1-fc') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:09:15 user nova-compute[75234]: DEBUG nova.objects.instance [None req-117159fd-f2d8-4878-ada7-68e5f6e62872 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Lazy-loading 'pci_devices' on Instance uuid 062753a6-8f39-4e80-a549-9f0f5a4b877a {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:09:15 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-117159fd-f2d8-4878-ada7-68e5f6e62872 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] [instance: 062753a6-8f39-4e80-a549-9f0f5a4b877a] End _get_guest_xml xml=<domain type="kvm">
Nov 29 03:09:15 user nova-compute[75234]:   <uuid>062753a6-8f39-4e80-a549-9f0f5a4b877a</uuid>
Nov 29 03:09:15 user nova-compute[75234]:   <name>instance-0000000e</name>
Nov 29 03:09:15 user nova-compute[75234]:   <memory>524288</memory>
Nov 29 03:09:15 user nova-compute[75234]:   <vcpu>1</vcpu>
Nov 29 03:09:15 user nova-compute[75234]:   <metadata>
Nov 29 03:09:15 user nova-compute[75234]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Nov 29 03:09:15 user nova-compute[75234]:       <nova:package version="0.0.0"/>
Nov 29 03:09:15 user nova-compute[75234]:       <nova:name>tempest-AttachVolumeShelveTestJSON-server-2002337175</nova:name>
Nov 29 03:09:15 user nova-compute[75234]:       <nova:creationTime>2023-11-29 03:09:15</nova:creationTime>
Nov 29 03:09:15 user nova-compute[75234]:       <nova:flavor name="m1.tiny">
Nov 29 03:09:15 user nova-compute[75234]:         <nova:memory>512</nova:memory>
Nov 29 03:09:15 user nova-compute[75234]:         <nova:disk>1</nova:disk>
Nov 29 03:09:15 user nova-compute[75234]:         <nova:swap>0</nova:swap>
Nov 29 03:09:15 user nova-compute[75234]:         <nova:ephemeral>0</nova:ephemeral>
Nov 29 03:09:15 user nova-compute[75234]:         <nova:vcpus>1</nova:vcpus>
Nov 29 03:09:15 user nova-compute[75234]:       </nova:flavor>
Nov 29 03:09:15 user nova-compute[75234]:       <nova:owner>
Nov 29 03:09:15 user nova-compute[75234]:         <nova:user uuid="154f3f4f57514450baf8f22d67768bc9">tempest-AttachVolumeShelveTestJSON-1180535246-project-member</nova:user>
Nov 29 03:09:15 user nova-compute[75234]:         <nova:project uuid="e17150005a9a449f9373b91e1c00083b">tempest-AttachVolumeShelveTestJSON-1180535246</nova:project>
Nov 29 03:09:15 user nova-compute[75234]:       </nova:owner>
Nov 29 03:09:15 user nova-compute[75234]:       <nova:root type="image" uuid="96c30ba7-9b73-4a30-8ba4-7138879573b7"/>
Nov 29 03:09:15 user nova-compute[75234]:       <nova:ports>
Nov 29 03:09:15 user nova-compute[75234]:         <nova:port uuid="8f6751f1-fc65-4842-b77f-1dfe26912afb">
Nov 29 03:09:15 user nova-compute[75234]:           <nova:ip type="fixed" address="10.0.0.3" ipVersion="4"/>
Nov 29 03:09:15 user nova-compute[75234]:         </nova:port>
Nov 29 03:09:15 user nova-compute[75234]:       </nova:ports>
Nov 29 03:09:15 user nova-compute[75234]:     </nova:instance>
Nov 29 03:09:15 user nova-compute[75234]:   </metadata>
Nov 29 03:09:15 user nova-compute[75234]:   <sysinfo type="smbios">
Nov 29 03:09:15 user nova-compute[75234]:     <system>
Nov 29 03:09:15 user nova-compute[75234]:       <entry name="manufacturer">OpenStack Foundation</entry>
Nov 29 03:09:15 user nova-compute[75234]:       <entry name="product">OpenStack Nova</entry>
Nov 29 03:09:15 user nova-compute[75234]:       <entry name="version">0.0.0</entry>
Nov 29 03:09:15 user nova-compute[75234]:       <entry name="serial">062753a6-8f39-4e80-a549-9f0f5a4b877a</entry>
Nov 29 03:09:15 user nova-compute[75234]:       <entry name="uuid">062753a6-8f39-4e80-a549-9f0f5a4b877a</entry>
Nov 29 03:09:15 user nova-compute[75234]:       <entry name="family">Virtual Machine</entry>
Nov 29 03:09:15 user nova-compute[75234]:     </system>
Nov 29 03:09:15 user nova-compute[75234]:   </sysinfo>
Nov 29 03:09:15 user nova-compute[75234]:   <os>
Nov 29 03:09:15 user nova-compute[75234]:     <type arch="x86_64" machine="pc">hvm</type>
Nov 29 03:09:15 user nova-compute[75234]:     <boot dev="hd"/>
Nov 29 03:09:15 user nova-compute[75234]:     <smbios mode="sysinfo"/>
Nov 29 03:09:15 user nova-compute[75234]:   </os>
Nov 29 03:09:15 user nova-compute[75234]:   <features>
Nov 29 03:09:15 user nova-compute[75234]:     <acpi/>
Nov 29 03:09:15 user nova-compute[75234]:     <apic/>
Nov 29 03:09:15 user nova-compute[75234]:     <vmcoreinfo/>
Nov 29 03:09:15 user nova-compute[75234]:   </features>
Nov 29 03:09:15 user nova-compute[75234]:   <clock offset="utc">
Nov 29 03:09:15 user nova-compute[75234]:     <timer name="pit" tickpolicy="delay"/>
Nov 29 03:09:15 user nova-compute[75234]:     <timer name="rtc" tickpolicy="catchup"/>
Nov 29 03:09:15 user nova-compute[75234]:     <timer name="hpet" present="no"/>
Nov 29 03:09:15 user nova-compute[75234]:   </clock>
Nov 29 03:09:15 user nova-compute[75234]:   <cpu mode="custom" match="exact">
Nov 29 03:09:15 user nova-compute[75234]:     <model>Nehalem</model>
Nov 29 03:09:15 user nova-compute[75234]:     <topology sockets="1" cores="1" threads="1"/>
Nov 29 03:09:15 user nova-compute[75234]:   </cpu>
Nov 29 03:09:15 user nova-compute[75234]:   <devices>
Nov 29 03:09:15 user nova-compute[75234]:     <disk type="file" device="disk">
Nov 29 03:09:15 user nova-compute[75234]:       <driver name="qemu" type="qcow2" cache="none"/>
Nov 29 03:09:15 user nova-compute[75234]:       <source file="/opt/stack/data/nova/instances/062753a6-8f39-4e80-a549-9f0f5a4b877a/disk"/>
Nov 29 03:09:15 user nova-compute[75234]:       <target dev="vda" bus="virtio"/>
Nov 29 03:09:15 user nova-compute[75234]:     </disk>
Nov 29 03:09:15 user nova-compute[75234]:     <interface type="ethernet">
Nov 29 03:09:15 user nova-compute[75234]:       <mac address="fa:16:3e:62:4a:5a"/>
Nov 29 03:09:15 user nova-compute[75234]:       <model type="virtio"/>
Nov 29 03:09:15 user nova-compute[75234]:       <mtu size="1442"/>
Nov 29 03:09:15 user nova-compute[75234]:       <target dev="tap8f6751f1-fc"/>
Nov 29 03:09:15 user nova-compute[75234]:     </interface>
Nov 29 03:09:15 user nova-compute[75234]:     <serial type="pty">
Nov 29 03:09:15 user nova-compute[75234]:       <log file="/opt/stack/data/nova/instances/062753a6-8f39-4e80-a549-9f0f5a4b877a/console.log" append="off"/>
Nov 29 03:09:15 user nova-compute[75234]:     </serial>
Nov 29 03:09:15 user nova-compute[75234]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Nov 29 03:09:15 user nova-compute[75234]:     <video>
Nov 29 03:09:15 user nova-compute[75234]:       <model type="virtio"/>
Nov 29 03:09:15 user nova-compute[75234]:     </video>
Nov 29 03:09:15 user nova-compute[75234]:     <rng model="virtio">
Nov 29 03:09:15 user nova-compute[75234]:       <backend model="random">/dev/urandom</backend>
Nov 29 03:09:15 user nova-compute[75234]:     </rng>
Nov 29 03:09:15 user nova-compute[75234]:     <controller type="usb" index="0" model="none"/>
Nov 29 03:09:15 user nova-compute[75234]:     <memballoon model="virtio">
Nov 29 03:09:15 user nova-compute[75234]:       <stats period="10"/>
Nov 29 03:09:15 user nova-compute[75234]:     </memballoon>
Nov 29 03:09:15 user nova-compute[75234]:   </devices>
Nov 29 03:09:15 user nova-compute[75234]: </domain>
Nov 29 03:09:15 user nova-compute[75234]:  {{(pid=75234) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7568}}
Nov 29 03:09:15 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-117159fd-f2d8-4878-ada7-68e5f6e62872 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-11-29T03:09:11Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description=None,display_name='tempest-AttachVolumeShelveTestJSON-server-2002337175',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-attachvolumeshelvetestjson-server-2002337175',id=14,image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBJL1BPeb1pVpN5A24HF0gsy7slhuswFxR+GxX0R0LvMX9QS8GL1hvY7OFzvrLw6S/VLmgqgKQ7HFXhrJDiWyf/i/jTyWtVuorLQtfVudKMIFZjmVfY2JDvM7kwU/YbL6jQ==',key_name='tempest-keypair-1961974262',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='e17150005a9a449f9373b91e1c00083b',ramdisk_id='',reservation_id='r-p3yfidye',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-AttachVolumeShelveTestJSON-1180535246',owner_user_name='tempest-AttachVolumeShelveTestJSON-1180535246-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-11-29T03:09:13Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='154f3f4f57514450baf8f22d67768bc9',uuid=062753a6-8f39-4e80-a549-9f0f5a4b877a,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "8f6751f1-fc65-4842-b77f-1dfe26912afb", "address": "fa:16:3e:62:4a:5a", "network": {"id": "aad085f7-106b-4de6-93f5-f86bb66ec962", "bridge": "br-int", "label": "tempest-AttachVolumeShelveTestJSON-775984268-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "e17150005a9a449f9373b91e1c00083b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap8f6751f1-fc", "ovs_interfaceid": "8f6751f1-fc65-4842-b77f-1dfe26912afb", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Nov 29 03:09:15 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-117159fd-f2d8-4878-ada7-68e5f6e62872 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Converting VIF {"id": "8f6751f1-fc65-4842-b77f-1dfe26912afb", "address": "fa:16:3e:62:4a:5a", "network": {"id": "aad085f7-106b-4de6-93f5-f86bb66ec962", "bridge": "br-int", "label": "tempest-AttachVolumeShelveTestJSON-775984268-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "e17150005a9a449f9373b91e1c00083b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap8f6751f1-fc", "ovs_interfaceid": "8f6751f1-fc65-4842-b77f-1dfe26912afb", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:09:15 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-117159fd-f2d8-4878-ada7-68e5f6e62872 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:62:4a:5a,bridge_name='br-int',has_traffic_filtering=True,id=8f6751f1-fc65-4842-b77f-1dfe26912afb,network=Network(aad085f7-106b-4de6-93f5-f86bb66ec962),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap8f6751f1-fc') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:09:15 user nova-compute[75234]: DEBUG os_vif [None req-117159fd-f2d8-4878-ada7-68e5f6e62872 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:62:4a:5a,bridge_name='br-int',has_traffic_filtering=True,id=8f6751f1-fc65-4842-b77f-1dfe26912afb,network=Network(aad085f7-106b-4de6-93f5-f86bb66ec962),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap8f6751f1-fc') {{(pid=75234) plug /opt/stack/data/venv/lib/python3.10/site-packages/os_vif/__init__.py:76}}
Nov 29 03:09:15 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:09:15 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:09:15 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Nov 29 03:09:15 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:09:15 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbCreateCommand(_result=None, table=QoS, columns={'type': 'linux-noop', 'external_ids': {'id': 'ef7de737-0cc6-5031-aedb-46e6cda2e0b1', '_type': 'linux-noop'}}, row=False) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:09:15 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:09:15 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:09:15 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:09:15 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:09:15 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tap8f6751f1-fc, may_exist=True, interface_attrs={}) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:09:15 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Port, record=tap8f6751f1-fc, col_values=(('qos', UUID('ee6d3ae0-8482-47cf-9121-ee12da13da9d')),), if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:09:15 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DbSetCommand(_result=None, table=Interface, record=tap8f6751f1-fc, col_values=(('external_ids', {'iface-id': '8f6751f1-fc65-4842-b77f-1dfe26912afb', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:62:4a:5a', 'vm-uuid': '062753a6-8f39-4e80-a549-9f0f5a4b877a'}),), if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:09:15 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:09:15 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:09:15 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:09:15 user nova-compute[75234]: INFO os_vif [None req-117159fd-f2d8-4878-ada7-68e5f6e62872 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:62:4a:5a,bridge_name='br-int',has_traffic_filtering=True,id=8f6751f1-fc65-4842-b77f-1dfe26912afb,network=Network(aad085f7-106b-4de6-93f5-f86bb66ec962),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap8f6751f1-fc')
Nov 29 03:09:15 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-65aec05b-a901-4f17-a4e0-2e822560de93 tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json" returned: 0 in 0.189s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:09:15 user nova-compute[75234]: DEBUG nova.virt.disk.api [None req-65aec05b-a901-4f17-a4e0-2e822560de93 tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] Checking if we can resize image /opt/stack/data/nova/instances/ac620533-8274-4674-b241-f48526bed3c3/disk. size=1073741824 {{(pid=75234) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:166}}
Nov 29 03:09:15 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-65aec05b-a901-4f17-a4e0-2e822560de93 tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/ac620533-8274-4674-b241-f48526bed3c3/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:09:15 user nova-compute[75234]: DEBUG nova.network.neutron [None req-65aec05b-a901-4f17-a4e0-2e822560de93 tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] [instance: ac620533-8274-4674-b241-f48526bed3c3] Successfully created port: f96869cf-1585-47e8-a04e-40ad5f212130 {{(pid=75234) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:548}}
Nov 29 03:09:15 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-117159fd-f2d8-4878-ada7-68e5f6e62872 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] No BDM found with device name vda, not building metadata. {{(pid=75234) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12089}}
Nov 29 03:09:15 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-117159fd-f2d8-4878-ada7-68e5f6e62872 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] No VIF found with MAC fa:16:3e:62:4a:5a, not building metadata {{(pid=75234) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Nov 29 03:09:15 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-65aec05b-a901-4f17-a4e0-2e822560de93 tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/ac620533-8274-4674-b241-f48526bed3c3/disk --force-share --output=json" returned: 0 in 0.199s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:09:15 user nova-compute[75234]: DEBUG nova.virt.disk.api [None req-65aec05b-a901-4f17-a4e0-2e822560de93 tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] Cannot resize image /opt/stack/data/nova/instances/ac620533-8274-4674-b241-f48526bed3c3/disk to a smaller size. {{(pid=75234) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:172}}
Nov 29 03:09:15 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-65aec05b-a901-4f17-a4e0-2e822560de93 tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] [instance: ac620533-8274-4674-b241-f48526bed3c3] Created local disks {{(pid=75234) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4861}}
Nov 29 03:09:15 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-65aec05b-a901-4f17-a4e0-2e822560de93 tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] [instance: ac620533-8274-4674-b241-f48526bed3c3] Ensure instance console log exists: /opt/stack/data/nova/instances/ac620533-8274-4674-b241-f48526bed3c3/console.log {{(pid=75234) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4613}}
Nov 29 03:09:15 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-65aec05b-a901-4f17-a4e0-2e822560de93 tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:09:15 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-65aec05b-a901-4f17-a4e0-2e822560de93 tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:09:15 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-65aec05b-a901-4f17-a4e0-2e822560de93 tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:09:15 user nova-compute[75234]: INFO nova.compute.manager [None req-8437735b-1338-4943-bef4-6f6d4f9cbbba tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] [instance: 763a3e14-4563-4dc2-99ae-c340edbb93df] Took 2.71 seconds to detach 3 volumes for instance.
Nov 29 03:09:16 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-8437735b-1338-4943-bef4-6f6d4f9cbbba tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:09:16 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-8437735b-1338-4943-bef4-6f6d4f9cbbba tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:09:16 user nova-compute[75234]: DEBUG nova.network.neutron [None req-65aec05b-a901-4f17-a4e0-2e822560de93 tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] [instance: ac620533-8274-4674-b241-f48526bed3c3] Successfully updated port: f96869cf-1585-47e8-a04e-40ad5f212130 {{(pid=75234) _update_port /opt/stack/nova/nova/network/neutron.py:586}}
Nov 29 03:09:16 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-65aec05b-a901-4f17-a4e0-2e822560de93 tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] Acquiring lock "refresh_cache-ac620533-8274-4674-b241-f48526bed3c3" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:09:16 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-65aec05b-a901-4f17-a4e0-2e822560de93 tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] Acquired lock "refresh_cache-ac620533-8274-4674-b241-f48526bed3c3" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:09:16 user nova-compute[75234]: DEBUG nova.network.neutron [None req-65aec05b-a901-4f17-a4e0-2e822560de93 tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] [instance: ac620533-8274-4674-b241-f48526bed3c3] Building network info cache for instance {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2009}}
Nov 29 03:09:16 user nova-compute[75234]: DEBUG nova.network.neutron [None req-65aec05b-a901-4f17-a4e0-2e822560de93 tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] [instance: ac620533-8274-4674-b241-f48526bed3c3] Instance cache missing network info. {{(pid=75234) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3322}}
Nov 29 03:09:16 user nova-compute[75234]: DEBUG nova.compute.manager [req-87b50e83-9f3f-4b42-b726-988a11f5662d req-20bd2df2-581c-49fe-9874-8a2e97fee518 service nova] [instance: 062753a6-8f39-4e80-a549-9f0f5a4b877a] Received event network-changed-8f6751f1-fc65-4842-b77f-1dfe26912afb {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:09:16 user nova-compute[75234]: DEBUG nova.compute.manager [req-87b50e83-9f3f-4b42-b726-988a11f5662d req-20bd2df2-581c-49fe-9874-8a2e97fee518 service nova] [instance: 062753a6-8f39-4e80-a549-9f0f5a4b877a] Refreshing instance network info cache due to event network-changed-8f6751f1-fc65-4842-b77f-1dfe26912afb. {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11117}}
Nov 29 03:09:16 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-87b50e83-9f3f-4b42-b726-988a11f5662d req-20bd2df2-581c-49fe-9874-8a2e97fee518 service nova] Acquiring lock "refresh_cache-062753a6-8f39-4e80-a549-9f0f5a4b877a" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:09:16 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-87b50e83-9f3f-4b42-b726-988a11f5662d req-20bd2df2-581c-49fe-9874-8a2e97fee518 service nova] Acquired lock "refresh_cache-062753a6-8f39-4e80-a549-9f0f5a4b877a" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:09:16 user nova-compute[75234]: DEBUG nova.network.neutron [req-87b50e83-9f3f-4b42-b726-988a11f5662d req-20bd2df2-581c-49fe-9874-8a2e97fee518 service nova] [instance: 062753a6-8f39-4e80-a549-9f0f5a4b877a] Refreshing network info cache for port 8f6751f1-fc65-4842-b77f-1dfe26912afb {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2006}}
Nov 29 03:09:16 user nova-compute[75234]: DEBUG nova.compute.provider_tree [None req-8437735b-1338-4943-bef4-6f6d4f9cbbba tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] Inventory has not changed in ProviderTree for provider: 4d291220-7c37-4839-a0e3-b23c66502b74 {{(pid=75234) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Nov 29 03:09:16 user nova-compute[75234]: DEBUG nova.scheduler.client.report [None req-8437735b-1338-4943-bef4-6f6d4f9cbbba tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] Inventory has not changed for provider 4d291220-7c37-4839-a0e3-b23c66502b74 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=75234) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Nov 29 03:09:16 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:09:16 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-8437735b-1338-4943-bef4-6f6d4f9cbbba tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.420s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:09:16 user nova-compute[75234]: INFO nova.scheduler.client.report [None req-8437735b-1338-4943-bef4-6f6d4f9cbbba tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] Deleted allocations for instance 763a3e14-4563-4dc2-99ae-c340edbb93df
Nov 29 03:09:16 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-8437735b-1338-4943-bef4-6f6d4f9cbbba tempest-TaggedBootDevicesTest_v242-1560452734 tempest-TaggedBootDevicesTest_v242-1560452734-project-member] Lock "763a3e14-4563-4dc2-99ae-c340edbb93df" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 11.826s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:09:16 user nova-compute[75234]: DEBUG nova.network.neutron [None req-65aec05b-a901-4f17-a4e0-2e822560de93 tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] [instance: ac620533-8274-4674-b241-f48526bed3c3] Updating instance_info_cache with network_info: [{"id": "f96869cf-1585-47e8-a04e-40ad5f212130", "address": "fa:16:3e:e6:9b:b1", "network": {"id": "68a1f282-91d0-47c8-82f3-5ebf1c0f572c", "bridge": "br-int", "label": "tempest-VolumesActionsTest-570720740-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "46577f787c244fb2a4db052a0c42eceb", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapf96869cf-15", "ovs_interfaceid": "f96869cf-1585-47e8-a04e-40ad5f212130", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:09:16 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-65aec05b-a901-4f17-a4e0-2e822560de93 tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] Releasing lock "refresh_cache-ac620533-8274-4674-b241-f48526bed3c3" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:09:16 user nova-compute[75234]: DEBUG nova.compute.manager [None req-65aec05b-a901-4f17-a4e0-2e822560de93 tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] [instance: ac620533-8274-4674-b241-f48526bed3c3] Instance network_info: |[{"id": "f96869cf-1585-47e8-a04e-40ad5f212130", "address": "fa:16:3e:e6:9b:b1", "network": {"id": "68a1f282-91d0-47c8-82f3-5ebf1c0f572c", "bridge": "br-int", "label": "tempest-VolumesActionsTest-570720740-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "46577f787c244fb2a4db052a0c42eceb", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapf96869cf-15", "ovs_interfaceid": "f96869cf-1585-47e8-a04e-40ad5f212130", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=75234) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1987}}
Nov 29 03:09:16 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-65aec05b-a901-4f17-a4e0-2e822560de93 tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] [instance: ac620533-8274-4674-b241-f48526bed3c3] Start _get_guest_xml network_info=[{"id": "f96869cf-1585-47e8-a04e-40ad5f212130", "address": "fa:16:3e:e6:9b:b1", "network": {"id": "68a1f282-91d0-47c8-82f3-5ebf1c0f572c", "bridge": "br-int", "label": "tempest-VolumesActionsTest-570720740-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "46577f787c244fb2a4db052a0c42eceb", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapf96869cf-15", "ovs_interfaceid": "f96869cf-1585-47e8-a04e-40ad5f212130", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum='c8fc807773e5354afe61636071771906',container_format='bare',created_at=2023-11-29T03:02:15Z,direct_url=<?>,disk_format='qcow2',id=96c30ba7-9b73-4a30-8ba4-7138879573b7,min_disk=0,min_ram=0,name='cirros-0.6.2-x86_64-disk',owner='b18e380921c748b78517b6f1211cd0f1',properties=ImageMetaProps,protected=<?>,size=21430272,status='active',tags=<?>,updated_at=2023-11-29T03:02:17Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [{'device_name': '/dev/vda', 'encryption_format': None, 'encrypted': False, 'device_type': 'disk', 'size': 0, 'disk_bus': 'virtio', 'boot_index': 0, 'encryption_options': None, 'encryption_secret_uuid': None, 'guest_format': None, 'image_id': '96c30ba7-9b73-4a30-8ba4-7138879573b7'}], 'ephemerals': [], 'block_device_mapping': [], 'swap': None} {{(pid=75234) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7562}}
Nov 29 03:09:16 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-65aec05b-a901-4f17-a4e0-2e822560de93 tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:09:16 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-65aec05b-a901-4f17-a4e0-2e822560de93 tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:09:16 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-65aec05b-a901-4f17-a4e0-2e822560de93 tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] Searching host: 'user' for CPU controller through CGroups V1... {{(pid=75234) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1652}}
Nov 29 03:09:16 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-65aec05b-a901-4f17-a4e0-2e822560de93 tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] CPU controller missing on host. {{(pid=75234) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1662}}
Nov 29 03:09:16 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-65aec05b-a901-4f17-a4e0-2e822560de93 tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] Searching host: 'user' for CPU controller through CGroups V2... {{(pid=75234) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1671}}
Nov 29 03:09:16 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-65aec05b-a901-4f17-a4e0-2e822560de93 tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] CPU controller found on host. {{(pid=75234) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1678}}
Nov 29 03:09:16 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-65aec05b-a901-4f17-a4e0-2e822560de93 tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=75234) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5400}}
Nov 29 03:09:16 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-65aec05b-a901-4f17-a4e0-2e822560de93 tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-11-29T03:02:04Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='1',id=6,is_public=True,memory_mb=512,name='m1.tiny',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='c8fc807773e5354afe61636071771906',container_format='bare',created_at=2023-11-29T03:02:15Z,direct_url=<?>,disk_format='qcow2',id=96c30ba7-9b73-4a30-8ba4-7138879573b7,min_disk=0,min_ram=0,name='cirros-0.6.2-x86_64-disk',owner='b18e380921c748b78517b6f1211cd0f1',properties=ImageMetaProps,protected=<?>,size=21430272,status='active',tags=<?>,updated_at=2023-11-29T03:02:17Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:563}}
Nov 29 03:09:16 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-65aec05b-a901-4f17-a4e0-2e822560de93 tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] Flavor limits 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:348}}
Nov 29 03:09:16 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-65aec05b-a901-4f17-a4e0-2e822560de93 tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] Image limits 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:352}}
Nov 29 03:09:16 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-65aec05b-a901-4f17-a4e0-2e822560de93 tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] Flavor pref 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:388}}
Nov 29 03:09:16 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-65aec05b-a901-4f17-a4e0-2e822560de93 tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] Image pref 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:392}}
Nov 29 03:09:16 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-65aec05b-a901-4f17-a4e0-2e822560de93 tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:430}}
Nov 29 03:09:16 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-65aec05b-a901-4f17-a4e0-2e822560de93 tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:569}}
Nov 29 03:09:16 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-65aec05b-a901-4f17-a4e0-2e822560de93 tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=75234) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:471}}
Nov 29 03:09:16 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-65aec05b-a901-4f17-a4e0-2e822560de93 tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] Got 1 possible topologies {{(pid=75234) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:501}}
Nov 29 03:09:16 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-65aec05b-a901-4f17-a4e0-2e822560de93 tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:575}}
Nov 29 03:09:16 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-65aec05b-a901-4f17-a4e0-2e822560de93 tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:577}}
Nov 29 03:09:16 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-65aec05b-a901-4f17-a4e0-2e822560de93 tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-11-29T03:09:13Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-VolumesActionsTest-instance-134859871',display_name='tempest-VolumesActionsTest-instance-134859871',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-volumesactionstest-instance-134859871',id=15,image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data=None,key_name=None,keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='46577f787c244fb2a4db052a0c42eceb',ramdisk_id='',reservation_id='r-oalbemin',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-VolumesActionsTest-196322865',owner_user_name='tempest-VolumesActionsTest-196322865-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-11-29T03:09:15Z,user_data=None,user_id='940a684aa3b6419798bd87562bbc5d23',uuid=ac620533-8274-4674-b241-f48526bed3c3,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "f96869cf-1585-47e8-a04e-40ad5f212130", "address": "fa:16:3e:e6:9b:b1", "network": {"id": "68a1f282-91d0-47c8-82f3-5ebf1c0f572c", "bridge": "br-int", "label": "tempest-VolumesActionsTest-570720740-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "46577f787c244fb2a4db052a0c42eceb", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapf96869cf-15", "ovs_interfaceid": "f96869cf-1585-47e8-a04e-40ad5f212130", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=75234) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Nov 29 03:09:16 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-65aec05b-a901-4f17-a4e0-2e822560de93 tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] Converting VIF {"id": "f96869cf-1585-47e8-a04e-40ad5f212130", "address": "fa:16:3e:e6:9b:b1", "network": {"id": "68a1f282-91d0-47c8-82f3-5ebf1c0f572c", "bridge": "br-int", "label": "tempest-VolumesActionsTest-570720740-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "46577f787c244fb2a4db052a0c42eceb", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapf96869cf-15", "ovs_interfaceid": "f96869cf-1585-47e8-a04e-40ad5f212130", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:09:16 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-65aec05b-a901-4f17-a4e0-2e822560de93 tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:e6:9b:b1,bridge_name='br-int',has_traffic_filtering=True,id=f96869cf-1585-47e8-a04e-40ad5f212130,network=Network(68a1f282-91d0-47c8-82f3-5ebf1c0f572c),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapf96869cf-15') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:09:16 user nova-compute[75234]: DEBUG nova.objects.instance [None req-65aec05b-a901-4f17-a4e0-2e822560de93 tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] Lazy-loading 'pci_devices' on Instance uuid ac620533-8274-4674-b241-f48526bed3c3 {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:09:16 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-65aec05b-a901-4f17-a4e0-2e822560de93 tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] [instance: ac620533-8274-4674-b241-f48526bed3c3] End _get_guest_xml xml=<domain type="kvm">
Nov 29 03:09:16 user nova-compute[75234]:   <uuid>ac620533-8274-4674-b241-f48526bed3c3</uuid>
Nov 29 03:09:16 user nova-compute[75234]:   <name>instance-0000000f</name>
Nov 29 03:09:16 user nova-compute[75234]:   <memory>524288</memory>
Nov 29 03:09:16 user nova-compute[75234]:   <vcpu>1</vcpu>
Nov 29 03:09:16 user nova-compute[75234]:   <metadata>
Nov 29 03:09:16 user nova-compute[75234]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Nov 29 03:09:16 user nova-compute[75234]:       <nova:package version="0.0.0"/>
Nov 29 03:09:16 user nova-compute[75234]:       <nova:name>tempest-VolumesActionsTest-instance-134859871</nova:name>
Nov 29 03:09:16 user nova-compute[75234]:       <nova:creationTime>2023-11-29 03:09:16</nova:creationTime>
Nov 29 03:09:16 user nova-compute[75234]:       <nova:flavor name="m1.tiny">
Nov 29 03:09:16 user nova-compute[75234]:         <nova:memory>512</nova:memory>
Nov 29 03:09:16 user nova-compute[75234]:         <nova:disk>1</nova:disk>
Nov 29 03:09:16 user nova-compute[75234]:         <nova:swap>0</nova:swap>
Nov 29 03:09:16 user nova-compute[75234]:         <nova:ephemeral>0</nova:ephemeral>
Nov 29 03:09:16 user nova-compute[75234]:         <nova:vcpus>1</nova:vcpus>
Nov 29 03:09:16 user nova-compute[75234]:       </nova:flavor>
Nov 29 03:09:16 user nova-compute[75234]:       <nova:owner>
Nov 29 03:09:16 user nova-compute[75234]:         <nova:user uuid="940a684aa3b6419798bd87562bbc5d23">tempest-VolumesActionsTest-196322865-project-member</nova:user>
Nov 29 03:09:16 user nova-compute[75234]:         <nova:project uuid="46577f787c244fb2a4db052a0c42eceb">tempest-VolumesActionsTest-196322865</nova:project>
Nov 29 03:09:16 user nova-compute[75234]:       </nova:owner>
Nov 29 03:09:16 user nova-compute[75234]:       <nova:root type="image" uuid="96c30ba7-9b73-4a30-8ba4-7138879573b7"/>
Nov 29 03:09:16 user nova-compute[75234]:       <nova:ports>
Nov 29 03:09:16 user nova-compute[75234]:         <nova:port uuid="f96869cf-1585-47e8-a04e-40ad5f212130">
Nov 29 03:09:16 user nova-compute[75234]:           <nova:ip type="fixed" address="10.0.0.13" ipVersion="4"/>
Nov 29 03:09:16 user nova-compute[75234]:         </nova:port>
Nov 29 03:09:16 user nova-compute[75234]:       </nova:ports>
Nov 29 03:09:16 user nova-compute[75234]:     </nova:instance>
Nov 29 03:09:16 user nova-compute[75234]:   </metadata>
Nov 29 03:09:16 user nova-compute[75234]:   <sysinfo type="smbios">
Nov 29 03:09:16 user nova-compute[75234]:     <system>
Nov 29 03:09:16 user nova-compute[75234]:       <entry name="manufacturer">OpenStack Foundation</entry>
Nov 29 03:09:16 user nova-compute[75234]:       <entry name="product">OpenStack Nova</entry>
Nov 29 03:09:16 user nova-compute[75234]:       <entry name="version">0.0.0</entry>
Nov 29 03:09:16 user nova-compute[75234]:       <entry name="serial">ac620533-8274-4674-b241-f48526bed3c3</entry>
Nov 29 03:09:16 user nova-compute[75234]:       <entry name="uuid">ac620533-8274-4674-b241-f48526bed3c3</entry>
Nov 29 03:09:16 user nova-compute[75234]:       <entry name="family">Virtual Machine</entry>
Nov 29 03:09:16 user nova-compute[75234]:     </system>
Nov 29 03:09:16 user nova-compute[75234]:   </sysinfo>
Nov 29 03:09:16 user nova-compute[75234]:   <os>
Nov 29 03:09:16 user nova-compute[75234]:     <type arch="x86_64" machine="pc">hvm</type>
Nov 29 03:09:16 user nova-compute[75234]:     <boot dev="hd"/>
Nov 29 03:09:16 user nova-compute[75234]:     <smbios mode="sysinfo"/>
Nov 29 03:09:16 user nova-compute[75234]:   </os>
Nov 29 03:09:16 user nova-compute[75234]:   <features>
Nov 29 03:09:16 user nova-compute[75234]:     <acpi/>
Nov 29 03:09:16 user nova-compute[75234]:     <apic/>
Nov 29 03:09:16 user nova-compute[75234]:     <vmcoreinfo/>
Nov 29 03:09:16 user nova-compute[75234]:   </features>
Nov 29 03:09:16 user nova-compute[75234]:   <clock offset="utc">
Nov 29 03:09:16 user nova-compute[75234]:     <timer name="pit" tickpolicy="delay"/>
Nov 29 03:09:16 user nova-compute[75234]:     <timer name="rtc" tickpolicy="catchup"/>
Nov 29 03:09:16 user nova-compute[75234]:     <timer name="hpet" present="no"/>
Nov 29 03:09:16 user nova-compute[75234]:   </clock>
Nov 29 03:09:16 user nova-compute[75234]:   <cpu mode="custom" match="exact">
Nov 29 03:09:16 user nova-compute[75234]:     <model>Nehalem</model>
Nov 29 03:09:16 user nova-compute[75234]:     <topology sockets="1" cores="1" threads="1"/>
Nov 29 03:09:16 user nova-compute[75234]:   </cpu>
Nov 29 03:09:16 user nova-compute[75234]:   <devices>
Nov 29 03:09:16 user nova-compute[75234]:     <disk type="file" device="disk">
Nov 29 03:09:16 user nova-compute[75234]:       <driver name="qemu" type="qcow2" cache="none"/>
Nov 29 03:09:16 user nova-compute[75234]:       <source file="/opt/stack/data/nova/instances/ac620533-8274-4674-b241-f48526bed3c3/disk"/>
Nov 29 03:09:16 user nova-compute[75234]:       <target dev="vda" bus="virtio"/>
Nov 29 03:09:16 user nova-compute[75234]:     </disk>
Nov 29 03:09:16 user nova-compute[75234]:     <interface type="ethernet">
Nov 29 03:09:16 user nova-compute[75234]:       <mac address="fa:16:3e:e6:9b:b1"/>
Nov 29 03:09:16 user nova-compute[75234]:       <model type="virtio"/>
Nov 29 03:09:16 user nova-compute[75234]:       <mtu size="1442"/>
Nov 29 03:09:16 user nova-compute[75234]:       <target dev="tapf96869cf-15"/>
Nov 29 03:09:16 user nova-compute[75234]:     </interface>
Nov 29 03:09:16 user nova-compute[75234]:     <serial type="pty">
Nov 29 03:09:16 user nova-compute[75234]:       <log file="/opt/stack/data/nova/instances/ac620533-8274-4674-b241-f48526bed3c3/console.log" append="off"/>
Nov 29 03:09:16 user nova-compute[75234]:     </serial>
Nov 29 03:09:16 user nova-compute[75234]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Nov 29 03:09:16 user nova-compute[75234]:     <video>
Nov 29 03:09:16 user nova-compute[75234]:       <model type="virtio"/>
Nov 29 03:09:16 user nova-compute[75234]:     </video>
Nov 29 03:09:16 user nova-compute[75234]:     <rng model="virtio">
Nov 29 03:09:16 user nova-compute[75234]:       <backend model="random">/dev/urandom</backend>
Nov 29 03:09:16 user nova-compute[75234]:     </rng>
Nov 29 03:09:16 user nova-compute[75234]:     <controller type="usb" index="0" model="none"/>
Nov 29 03:09:16 user nova-compute[75234]:     <memballoon model="virtio">
Nov 29 03:09:16 user nova-compute[75234]:       <stats period="10"/>
Nov 29 03:09:16 user nova-compute[75234]:     </memballoon>
Nov 29 03:09:16 user nova-compute[75234]:   </devices>
Nov 29 03:09:16 user nova-compute[75234]: </domain>
Nov 29 03:09:16 user nova-compute[75234]:  {{(pid=75234) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7568}}
Nov 29 03:09:16 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-65aec05b-a901-4f17-a4e0-2e822560de93 tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-11-29T03:09:13Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-VolumesActionsTest-instance-134859871',display_name='tempest-VolumesActionsTest-instance-134859871',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-volumesactionstest-instance-134859871',id=15,image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data=None,key_name=None,keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='46577f787c244fb2a4db052a0c42eceb',ramdisk_id='',reservation_id='r-oalbemin',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-VolumesActionsTest-196322865',owner_user_name='tempest-VolumesActionsTest-196322865-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-11-29T03:09:15Z,user_data=None,user_id='940a684aa3b6419798bd87562bbc5d23',uuid=ac620533-8274-4674-b241-f48526bed3c3,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "f96869cf-1585-47e8-a04e-40ad5f212130", "address": "fa:16:3e:e6:9b:b1", "network": {"id": "68a1f282-91d0-47c8-82f3-5ebf1c0f572c", "bridge": "br-int", "label": "tempest-VolumesActionsTest-570720740-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "46577f787c244fb2a4db052a0c42eceb", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapf96869cf-15", "ovs_interfaceid": "f96869cf-1585-47e8-a04e-40ad5f212130", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Nov 29 03:09:16 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-65aec05b-a901-4f17-a4e0-2e822560de93 tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] Converting VIF {"id": "f96869cf-1585-47e8-a04e-40ad5f212130", "address": "fa:16:3e:e6:9b:b1", "network": {"id": "68a1f282-91d0-47c8-82f3-5ebf1c0f572c", "bridge": "br-int", "label": "tempest-VolumesActionsTest-570720740-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "46577f787c244fb2a4db052a0c42eceb", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapf96869cf-15", "ovs_interfaceid": "f96869cf-1585-47e8-a04e-40ad5f212130", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:09:16 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-65aec05b-a901-4f17-a4e0-2e822560de93 tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:e6:9b:b1,bridge_name='br-int',has_traffic_filtering=True,id=f96869cf-1585-47e8-a04e-40ad5f212130,network=Network(68a1f282-91d0-47c8-82f3-5ebf1c0f572c),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapf96869cf-15') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:09:16 user nova-compute[75234]: DEBUG os_vif [None req-65aec05b-a901-4f17-a4e0-2e822560de93 tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:e6:9b:b1,bridge_name='br-int',has_traffic_filtering=True,id=f96869cf-1585-47e8-a04e-40ad5f212130,network=Network(68a1f282-91d0-47c8-82f3-5ebf1c0f572c),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapf96869cf-15') {{(pid=75234) plug /opt/stack/data/venv/lib/python3.10/site-packages/os_vif/__init__.py:76}}
Nov 29 03:09:16 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:09:16 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:09:16 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Nov 29 03:09:16 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:09:16 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbCreateCommand(_result=None, table=QoS, columns={'type': 'linux-noop', 'external_ids': {'id': '3257cd1e-0665-5f1e-878f-ab959bd5ce05', '_type': 'linux-noop'}}, row=False) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:09:16 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:09:16 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:09:16 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:09:16 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:09:16 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tapf96869cf-15, may_exist=True, interface_attrs={}) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:09:16 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Port, record=tapf96869cf-15, col_values=(('qos', UUID('9404b288-ffa1-4472-968a-cdaa548ad21b')),), if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:09:16 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DbSetCommand(_result=None, table=Interface, record=tapf96869cf-15, col_values=(('external_ids', {'iface-id': 'f96869cf-1585-47e8-a04e-40ad5f212130', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:e6:9b:b1', 'vm-uuid': 'ac620533-8274-4674-b241-f48526bed3c3'}),), if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:09:16 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:09:16 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:09:16 user nova-compute[75234]: DEBUG nova.network.neutron [req-87b50e83-9f3f-4b42-b726-988a11f5662d req-20bd2df2-581c-49fe-9874-8a2e97fee518 service nova] [instance: 062753a6-8f39-4e80-a549-9f0f5a4b877a] Updated VIF entry in instance network info cache for port 8f6751f1-fc65-4842-b77f-1dfe26912afb. {{(pid=75234) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3481}}
Nov 29 03:09:16 user nova-compute[75234]: DEBUG nova.network.neutron [req-87b50e83-9f3f-4b42-b726-988a11f5662d req-20bd2df2-581c-49fe-9874-8a2e97fee518 service nova] [instance: 062753a6-8f39-4e80-a549-9f0f5a4b877a] Updating instance_info_cache with network_info: [{"id": "8f6751f1-fc65-4842-b77f-1dfe26912afb", "address": "fa:16:3e:62:4a:5a", "network": {"id": "aad085f7-106b-4de6-93f5-f86bb66ec962", "bridge": "br-int", "label": "tempest-AttachVolumeShelveTestJSON-775984268-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "e17150005a9a449f9373b91e1c00083b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap8f6751f1-fc", "ovs_interfaceid": "8f6751f1-fc65-4842-b77f-1dfe26912afb", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:09:16 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:09:16 user nova-compute[75234]: INFO os_vif [None req-65aec05b-a901-4f17-a4e0-2e822560de93 tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:e6:9b:b1,bridge_name='br-int',has_traffic_filtering=True,id=f96869cf-1585-47e8-a04e-40ad5f212130,network=Network(68a1f282-91d0-47c8-82f3-5ebf1c0f572c),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapf96869cf-15')
Nov 29 03:09:17 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-87b50e83-9f3f-4b42-b726-988a11f5662d req-20bd2df2-581c-49fe-9874-8a2e97fee518 service nova] Releasing lock "refresh_cache-062753a6-8f39-4e80-a549-9f0f5a4b877a" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:09:17 user nova-compute[75234]: DEBUG nova.compute.manager [req-87b50e83-9f3f-4b42-b726-988a11f5662d req-20bd2df2-581c-49fe-9874-8a2e97fee518 service nova] [instance: ac620533-8274-4674-b241-f48526bed3c3] Received event network-changed-f96869cf-1585-47e8-a04e-40ad5f212130 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:09:17 user nova-compute[75234]: DEBUG nova.compute.manager [req-87b50e83-9f3f-4b42-b726-988a11f5662d req-20bd2df2-581c-49fe-9874-8a2e97fee518 service nova] [instance: ac620533-8274-4674-b241-f48526bed3c3] Refreshing instance network info cache due to event network-changed-f96869cf-1585-47e8-a04e-40ad5f212130. {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11117}}
Nov 29 03:09:17 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-87b50e83-9f3f-4b42-b726-988a11f5662d req-20bd2df2-581c-49fe-9874-8a2e97fee518 service nova] Acquiring lock "refresh_cache-ac620533-8274-4674-b241-f48526bed3c3" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:09:17 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-87b50e83-9f3f-4b42-b726-988a11f5662d req-20bd2df2-581c-49fe-9874-8a2e97fee518 service nova] Acquired lock "refresh_cache-ac620533-8274-4674-b241-f48526bed3c3" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:09:17 user nova-compute[75234]: DEBUG nova.network.neutron [req-87b50e83-9f3f-4b42-b726-988a11f5662d req-20bd2df2-581c-49fe-9874-8a2e97fee518 service nova] [instance: ac620533-8274-4674-b241-f48526bed3c3] Refreshing network info cache for port f96869cf-1585-47e8-a04e-40ad5f212130 {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2006}}
Nov 29 03:09:17 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:09:17 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:09:17 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:09:17 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-65aec05b-a901-4f17-a4e0-2e822560de93 tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] No BDM found with device name vda, not building metadata. {{(pid=75234) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12089}}
Nov 29 03:09:17 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-65aec05b-a901-4f17-a4e0-2e822560de93 tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] No VIF found with MAC fa:16:3e:e6:9b:b1, not building metadata {{(pid=75234) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Nov 29 03:09:17 user nova-compute[75234]: DEBUG nova.network.neutron [req-87b50e83-9f3f-4b42-b726-988a11f5662d req-20bd2df2-581c-49fe-9874-8a2e97fee518 service nova] [instance: ac620533-8274-4674-b241-f48526bed3c3] Updated VIF entry in instance network info cache for port f96869cf-1585-47e8-a04e-40ad5f212130. {{(pid=75234) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3481}}
Nov 29 03:09:17 user nova-compute[75234]: DEBUG nova.network.neutron [req-87b50e83-9f3f-4b42-b726-988a11f5662d req-20bd2df2-581c-49fe-9874-8a2e97fee518 service nova] [instance: ac620533-8274-4674-b241-f48526bed3c3] Updating instance_info_cache with network_info: [{"id": "f96869cf-1585-47e8-a04e-40ad5f212130", "address": "fa:16:3e:e6:9b:b1", "network": {"id": "68a1f282-91d0-47c8-82f3-5ebf1c0f572c", "bridge": "br-int", "label": "tempest-VolumesActionsTest-570720740-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "46577f787c244fb2a4db052a0c42eceb", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapf96869cf-15", "ovs_interfaceid": "f96869cf-1585-47e8-a04e-40ad5f212130", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:09:17 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-87b50e83-9f3f-4b42-b726-988a11f5662d req-20bd2df2-581c-49fe-9874-8a2e97fee518 service nova] Releasing lock "refresh_cache-ac620533-8274-4674-b241-f48526bed3c3" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:09:17 user nova-compute[75234]: DEBUG nova.compute.manager [req-cbf71c81-cc8f-4b2b-a7aa-110614e5edb9 req-1f8e6428-36a7-431e-ad83-1dabb86fa440 service nova] [instance: 062753a6-8f39-4e80-a549-9f0f5a4b877a] Received event network-vif-plugged-8f6751f1-fc65-4842-b77f-1dfe26912afb {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:09:17 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-cbf71c81-cc8f-4b2b-a7aa-110614e5edb9 req-1f8e6428-36a7-431e-ad83-1dabb86fa440 service nova] Acquiring lock "062753a6-8f39-4e80-a549-9f0f5a4b877a-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:09:17 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-cbf71c81-cc8f-4b2b-a7aa-110614e5edb9 req-1f8e6428-36a7-431e-ad83-1dabb86fa440 service nova] Lock "062753a6-8f39-4e80-a549-9f0f5a4b877a-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:09:17 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-cbf71c81-cc8f-4b2b-a7aa-110614e5edb9 req-1f8e6428-36a7-431e-ad83-1dabb86fa440 service nova] Lock "062753a6-8f39-4e80-a549-9f0f5a4b877a-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:09:17 user nova-compute[75234]: DEBUG nova.compute.manager [req-cbf71c81-cc8f-4b2b-a7aa-110614e5edb9 req-1f8e6428-36a7-431e-ad83-1dabb86fa440 service nova] [instance: 062753a6-8f39-4e80-a549-9f0f5a4b877a] No waiting events found dispatching network-vif-plugged-8f6751f1-fc65-4842-b77f-1dfe26912afb {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:09:17 user nova-compute[75234]: WARNING nova.compute.manager [req-cbf71c81-cc8f-4b2b-a7aa-110614e5edb9 req-1f8e6428-36a7-431e-ad83-1dabb86fa440 service nova] [instance: 062753a6-8f39-4e80-a549-9f0f5a4b877a] Received unexpected event network-vif-plugged-8f6751f1-fc65-4842-b77f-1dfe26912afb for instance with vm_state building and task_state spawning.
Nov 29 03:09:17 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:09:17 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:09:17 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:09:17 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:09:18 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:09:18 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:09:18 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:09:18 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:09:19 user nova-compute[75234]: DEBUG nova.compute.manager [req-03800870-8404-4d42-b2ba-5a67b809dd31 req-1bd7f7e9-45f8-4168-bc2d-264857395e84 service nova] [instance: ac620533-8274-4674-b241-f48526bed3c3] Received event network-vif-plugged-f96869cf-1585-47e8-a04e-40ad5f212130 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:09:19 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-03800870-8404-4d42-b2ba-5a67b809dd31 req-1bd7f7e9-45f8-4168-bc2d-264857395e84 service nova] Acquiring lock "ac620533-8274-4674-b241-f48526bed3c3-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:09:19 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-03800870-8404-4d42-b2ba-5a67b809dd31 req-1bd7f7e9-45f8-4168-bc2d-264857395e84 service nova] Lock "ac620533-8274-4674-b241-f48526bed3c3-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:09:19 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-03800870-8404-4d42-b2ba-5a67b809dd31 req-1bd7f7e9-45f8-4168-bc2d-264857395e84 service nova] Lock "ac620533-8274-4674-b241-f48526bed3c3-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:09:19 user nova-compute[75234]: DEBUG nova.compute.manager [req-03800870-8404-4d42-b2ba-5a67b809dd31 req-1bd7f7e9-45f8-4168-bc2d-264857395e84 service nova] [instance: ac620533-8274-4674-b241-f48526bed3c3] No waiting events found dispatching network-vif-plugged-f96869cf-1585-47e8-a04e-40ad5f212130 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:09:19 user nova-compute[75234]: WARNING nova.compute.manager [req-03800870-8404-4d42-b2ba-5a67b809dd31 req-1bd7f7e9-45f8-4168-bc2d-264857395e84 service nova] [instance: ac620533-8274-4674-b241-f48526bed3c3] Received unexpected event network-vif-plugged-f96869cf-1585-47e8-a04e-40ad5f212130 for instance with vm_state building and task_state spawning.
Nov 29 03:09:19 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:09:19 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:09:19 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:09:19 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:09:19 user nova-compute[75234]: DEBUG nova.compute.manager [req-37ed6bdb-66de-4383-b076-cc35f0e7cc4d req-2244091e-81da-4340-bd40-7e37bc698b65 service nova] [instance: 062753a6-8f39-4e80-a549-9f0f5a4b877a] Received event network-vif-plugged-8f6751f1-fc65-4842-b77f-1dfe26912afb {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:09:19 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-37ed6bdb-66de-4383-b076-cc35f0e7cc4d req-2244091e-81da-4340-bd40-7e37bc698b65 service nova] Acquiring lock "062753a6-8f39-4e80-a549-9f0f5a4b877a-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:09:19 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-37ed6bdb-66de-4383-b076-cc35f0e7cc4d req-2244091e-81da-4340-bd40-7e37bc698b65 service nova] Lock "062753a6-8f39-4e80-a549-9f0f5a4b877a-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:09:19 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-37ed6bdb-66de-4383-b076-cc35f0e7cc4d req-2244091e-81da-4340-bd40-7e37bc698b65 service nova] Lock "062753a6-8f39-4e80-a549-9f0f5a4b877a-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:09:19 user nova-compute[75234]: DEBUG nova.compute.manager [req-37ed6bdb-66de-4383-b076-cc35f0e7cc4d req-2244091e-81da-4340-bd40-7e37bc698b65 service nova] [instance: 062753a6-8f39-4e80-a549-9f0f5a4b877a] No waiting events found dispatching network-vif-plugged-8f6751f1-fc65-4842-b77f-1dfe26912afb {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:09:19 user nova-compute[75234]: WARNING nova.compute.manager [req-37ed6bdb-66de-4383-b076-cc35f0e7cc4d req-2244091e-81da-4340-bd40-7e37bc698b65 service nova] [instance: 062753a6-8f39-4e80-a549-9f0f5a4b877a] Received unexpected event network-vif-plugged-8f6751f1-fc65-4842-b77f-1dfe26912afb for instance with vm_state building and task_state spawning.
Nov 29 03:09:20 user nova-compute[75234]: DEBUG nova.compute.manager [None req-117159fd-f2d8-4878-ada7-68e5f6e62872 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] [instance: 062753a6-8f39-4e80-a549-9f0f5a4b877a] Instance event wait completed in 0 seconds for  {{(pid=75234) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Nov 29 03:09:20 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-117159fd-f2d8-4878-ada7-68e5f6e62872 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] [instance: 062753a6-8f39-4e80-a549-9f0f5a4b877a] Guest created on hypervisor {{(pid=75234) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4421}}
Nov 29 03:09:20 user nova-compute[75234]: INFO nova.virt.libvirt.driver [-] [instance: 062753a6-8f39-4e80-a549-9f0f5a4b877a] Instance spawned successfully.
Nov 29 03:09:20 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-117159fd-f2d8-4878-ada7-68e5f6e62872 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] [instance: 062753a6-8f39-4e80-a549-9f0f5a4b877a] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:904}}
Nov 29 03:09:20 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-117159fd-f2d8-4878-ada7-68e5f6e62872 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] [instance: 062753a6-8f39-4e80-a549-9f0f5a4b877a] Found default for hw_cdrom_bus of ide {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:09:20 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-117159fd-f2d8-4878-ada7-68e5f6e62872 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] [instance: 062753a6-8f39-4e80-a549-9f0f5a4b877a] Found default for hw_disk_bus of virtio {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:09:20 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-117159fd-f2d8-4878-ada7-68e5f6e62872 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] [instance: 062753a6-8f39-4e80-a549-9f0f5a4b877a] Found default for hw_input_bus of None {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:09:20 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-117159fd-f2d8-4878-ada7-68e5f6e62872 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] [instance: 062753a6-8f39-4e80-a549-9f0f5a4b877a] Found default for hw_pointer_model of None {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:09:20 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-117159fd-f2d8-4878-ada7-68e5f6e62872 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] [instance: 062753a6-8f39-4e80-a549-9f0f5a4b877a] Found default for hw_video_model of virtio {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:09:20 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-117159fd-f2d8-4878-ada7-68e5f6e62872 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] [instance: 062753a6-8f39-4e80-a549-9f0f5a4b877a] Found default for hw_vif_model of virtio {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:09:20 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-cf00e939-67db-4afa-b7b4-2ac67871e737 tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] [instance: f7d0bdeb-6a05-47ac-87f2-c8380a9a4286] Calling os-brick to attach iSCSI Volume {{(pid=75234) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Nov 29 03:09:20 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-cf00e939-67db-4afa-b7b4-2ac67871e737 tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f8a5050c9a0>, 'connection_properties': {'target_iqn': 'iqn.2010-10.org.openstack:ada91c26-ca97-4b8b-98bb-a310d0627691', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'pbUzSvGT', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:176}}
Nov 29 03:09:20 user nova-compute[75234]: WARNING os_brick.initiator.connectors.base [None req-cf00e939-67db-4afa-b7b4-2ac67871e737 tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Nov 29 03:09:20 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-cf00e939-67db-4afa-b7b4-2ac67871e737 tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:68}}
Nov 29 03:09:20 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-cf00e939-67db-4afa-b7b4-2ac67871e737 tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:73}}
Nov 29 03:09:20 user nova-compute[75234]: WARNING os_brick.initiator.connectors.base [None req-cf00e939-67db-4afa-b7b4-2ac67871e737 tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Nov 29 03:09:20 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-cf00e939-67db-4afa-b7b4-2ac67871e737 tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:ada91c26-ca97-4b8b-98bb-a310d0627691" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:68}}
Nov 29 03:09:20 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-cf00e939-67db-4afa-b7b4-2ac67871e737 tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:ada91c26-ca97-4b8b-98bb-a310d0627691" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:73}}
Nov 29 03:09:20 user nova-compute[75234]: INFO os_brick.initiator.connectors.iscsi [None req-cf00e939-67db-4afa-b7b4-2ac67871e737 tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Nov 29 03:09:20 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:ada91c26-ca97-4b8b-98bb-a310d0627691 -p 172.16.0.220:3260 {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:09:20 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:ada91c26-ca97-4b8b-98bb-a310d0627691 -p 172.16.0.220:3260" returned: 21 in 0.021s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:09:20 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[d99caf86-7821-46cb-9da6-3af53a438874]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:09:20 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-cf00e939-67db-4afa-b7b4-2ac67871e737 tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Nov 29 03:09:20 user nova-compute[75234]:  {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:09:20 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:ada91c26-ca97-4b8b-98bb-a310d0627691 -p 172.16.0.220:3260 --interface default --op new {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:09:20 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:ada91c26-ca97-4b8b-98bb-a310d0627691 -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.019s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:09:20 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[a790e583-e1af-4234-8fce-8542fff4ed9c]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:ada91c26-ca97-4b8b-98bb-a310d0627691] added\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:09:20 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-cf00e939-67db-4afa-b7b4-2ac67871e737 tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:ada91c26-ca97-4b8b-98bb-a310d0627691] added
Nov 29 03:09:20 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:09:20 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:ada91c26-ca97-4b8b-98bb-a310d0627691 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:09:20 user nova-compute[75234]: INFO nova.compute.manager [None req-117159fd-f2d8-4878-ada7-68e5f6e62872 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] [instance: 062753a6-8f39-4e80-a549-9f0f5a4b877a] Took 7.39 seconds to spawn the instance on the hypervisor.
Nov 29 03:09:20 user nova-compute[75234]: DEBUG nova.compute.manager [None req-117159fd-f2d8-4878-ada7-68e5f6e62872 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] [instance: 062753a6-8f39-4e80-a549-9f0f5a4b877a] Checking state {{(pid=75234) _get_power_state /opt/stack/nova/nova/compute/manager.py:1782}}
Nov 29 03:09:20 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:ada91c26-ca97-4b8b-98bb-a310d0627691 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.020s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:09:20 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[fe7d0abd-04b5-4dad-8aa0-c57899acf030]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:09:20 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-cf00e939-67db-4afa-b7b4-2ac67871e737 tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:09:20 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:ada91c26-ca97-4b8b-98bb-a310d0627691 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:09:20 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:ada91c26-ca97-4b8b-98bb-a310d0627691 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.017s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:09:20 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[4813d469-8303-4c15-b34b-a9d5e1b66628]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:09:20 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-cf00e939-67db-4afa-b7b4-2ac67871e737 tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:09:20 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:ada91c26-ca97-4b8b-98bb-a310d0627691 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v pbUzSvGT {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:09:20 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:ada91c26-ca97-4b8b-98bb-a310d0627691 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v pbUzSvGT" returned: 0 in 0.019s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:09:20 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[60ae8574-f68c-40f5-b722-06f19b1adf5e]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:09:20 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-cf00e939-67db-4afa-b7b4-2ac67871e737 tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'pbUzSvGT'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:09:20 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:ada91c26-ca97-4b8b-98bb-a310d0627691 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:09:20 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:ada91c26-ca97-4b8b-98bb-a310d0627691 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.019s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:09:20 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[d629bcb6-8694-4ec3-b5a0-2a443aaf598e]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:09:20 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-cf00e939-67db-4afa-b7b4-2ac67871e737 tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:09:20 user nova-compute[75234]: INFO nova.compute.manager [None req-117159fd-f2d8-4878-ada7-68e5f6e62872 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] [instance: 062753a6-8f39-4e80-a549-9f0f5a4b877a] Took 8.43 seconds to build instance.
Nov 29 03:09:20 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:09:20 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-117159fd-f2d8-4878-ada7-68e5f6e62872 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Lock "062753a6-8f39-4e80-a549-9f0f5a4b877a" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 8.529s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:09:20 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.020s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:09:20 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[718e189e-a534-4f09-9ba1-f054f6fdd45f]: (4, ('tcp: [9] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672 (non-flash)\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:09:20 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-cf00e939-67db-4afa-b7b4-2ac67871e737 tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [9] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672 (non-flash)
Nov 29 03:09:20 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:09:20 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-cf00e939-67db-4afa-b7b4-2ac67871e737 tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] iscsi session list stdout=tcp: [9] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672 (non-flash)
Nov 29 03:09:20 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsi_session /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Nov 29 03:09:20 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:ada91c26-ca97-4b8b-98bb-a310d0627691 -p 172.16.0.220:3260 --login {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:09:20 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:ada91c26-ca97-4b8b-98bb-a310d0627691 -p 172.16.0.220:3260 --login" returned: 0 in 0.066s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:09:20 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[2bdec2d9-e51f-4bb0-abde-f35432834220]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:ada91c26-ca97-4b8b-98bb-a310d0627691, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:ada91c26-ca97-4b8b-98bb-a310d0627691, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:09:20 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-cf00e939-67db-4afa-b7b4-2ac67871e737 tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:ada91c26-ca97-4b8b-98bb-a310d0627691, portal: 172.16.0.220,3260]
Nov 29 03:09:20 user nova-compute[75234]: Login to [iface: default, target: iqn.2010-10.org.openstack:ada91c26-ca97-4b8b-98bb-a310d0627691, portal: 172.16.0.220,3260] successful.
Nov 29 03:09:20 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:09:20 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:ada91c26-ca97-4b8b-98bb-a310d0627691 -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:09:20 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:ada91c26-ca97-4b8b-98bb-a310d0627691 -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.020s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:09:20 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[e5334c0d-135d-472f-a097-27b57aa3edd2]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:09:20 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-cf00e939-67db-4afa-b7b4-2ac67871e737 tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:09:20 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:09:20 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.020s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:09:20 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[759dd667-e8c9-4213-8b9f-b5ed84f241d0]: (4, ('tcp: [12] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ada91c26-ca97-4b8b-98bb-a310d0627691 (non-flash)\ntcp: [9] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672 (non-flash)\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:09:20 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-cf00e939-67db-4afa-b7b4-2ac67871e737 tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [12] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ada91c26-ca97-4b8b-98bb-a310d0627691 (non-flash)
Nov 29 03:09:20 user nova-compute[75234]: tcp: [9] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672 (non-flash)
Nov 29 03:09:20 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:09:20 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-cf00e939-67db-4afa-b7b4-2ac67871e737 tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] iscsi session list stdout=tcp: [12] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ada91c26-ca97-4b8b-98bb-a310d0627691 (non-flash)
Nov 29 03:09:20 user nova-compute[75234]: tcp: [9] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672 (non-flash)
Nov 29 03:09:20 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsi_session /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Nov 29 03:09:20 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-cf00e939-67db-4afa-b7b4-2ac67871e737 tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:ada91c26-ca97-4b8b-98bb-a310d0627691" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.296s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:87}}
Nov 29 03:09:20 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-cf00e939-67db-4afa-b7b4-2ac67871e737 tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] Connected to 172.16.0.220:3260 {{(pid=75234) _connect_vol /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:644}}
Nov 29 03:09:20 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-cf00e939-67db-4afa-b7b4-2ac67871e737 tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] HCTL ('33', '-', '-', 0) found on session 12 with lun 0 {{(pid=75234) get_hctl /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:770}}
Nov 29 03:09:20 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-cf00e939-67db-4afa-b7b4-2ac67871e737 tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] Scanning host 33 c: -, t: -, l: 0) {{(pid=75234) scan_iscsi /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:795}}
Nov 29 03:09:20 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host33/scan {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:09:21 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host33/scan" returned: 0 in 0.017s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:09:21 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[41e14b2a-f9ec-4337-b688-c3b3c5d8bd87]: (4, ('- - 0', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:09:21 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-cf00e939-67db-4afa-b7b4-2ac67871e737 tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] Searching for a device in session 12 and hctl ['33', '*', '*', 0] yield: None {{(pid=75234) device_name_by_hctl /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:789}}
Nov 29 03:09:21 user nova-compute[75234]: DEBUG nova.compute.manager [None req-65aec05b-a901-4f17-a4e0-2e822560de93 tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] [instance: ac620533-8274-4674-b241-f48526bed3c3] Instance event wait completed in 0 seconds for  {{(pid=75234) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Nov 29 03:09:21 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-65aec05b-a901-4f17-a4e0-2e822560de93 tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] [instance: ac620533-8274-4674-b241-f48526bed3c3] Guest created on hypervisor {{(pid=75234) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4421}}
Nov 29 03:09:21 user nova-compute[75234]: INFO nova.virt.libvirt.driver [-] [instance: ac620533-8274-4674-b241-f48526bed3c3] Instance spawned successfully.
Nov 29 03:09:21 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-65aec05b-a901-4f17-a4e0-2e822560de93 tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] [instance: ac620533-8274-4674-b241-f48526bed3c3] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:904}}
Nov 29 03:09:21 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-65aec05b-a901-4f17-a4e0-2e822560de93 tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] [instance: ac620533-8274-4674-b241-f48526bed3c3] Found default for hw_cdrom_bus of ide {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:09:21 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-65aec05b-a901-4f17-a4e0-2e822560de93 tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] [instance: ac620533-8274-4674-b241-f48526bed3c3] Found default for hw_disk_bus of virtio {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:09:21 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-65aec05b-a901-4f17-a4e0-2e822560de93 tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] [instance: ac620533-8274-4674-b241-f48526bed3c3] Found default for hw_input_bus of None {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:09:21 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-65aec05b-a901-4f17-a4e0-2e822560de93 tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] [instance: ac620533-8274-4674-b241-f48526bed3c3] Found default for hw_pointer_model of None {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:09:21 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-65aec05b-a901-4f17-a4e0-2e822560de93 tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] [instance: ac620533-8274-4674-b241-f48526bed3c3] Found default for hw_video_model of virtio {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:09:21 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-65aec05b-a901-4f17-a4e0-2e822560de93 tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] [instance: ac620533-8274-4674-b241-f48526bed3c3] Found default for hw_vif_model of virtio {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:09:21 user nova-compute[75234]: INFO nova.compute.manager [None req-65aec05b-a901-4f17-a4e0-2e822560de93 tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] [instance: ac620533-8274-4674-b241-f48526bed3c3] Took 6.46 seconds to spawn the instance on the hypervisor.
Nov 29 03:09:21 user nova-compute[75234]: DEBUG nova.compute.manager [None req-65aec05b-a901-4f17-a4e0-2e822560de93 tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] [instance: ac620533-8274-4674-b241-f48526bed3c3] Checking state {{(pid=75234) _get_power_state /opt/stack/nova/nova/compute/manager.py:1782}}
Nov 29 03:09:21 user nova-compute[75234]: DEBUG nova.compute.manager [req-9d6be6e3-9cbe-41dd-b602-897934b4ee5f req-68efa4ca-7339-451d-83b7-bc3d016c5b0f service nova] [instance: ac620533-8274-4674-b241-f48526bed3c3] Received event network-vif-plugged-f96869cf-1585-47e8-a04e-40ad5f212130 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:09:21 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-9d6be6e3-9cbe-41dd-b602-897934b4ee5f req-68efa4ca-7339-451d-83b7-bc3d016c5b0f service nova] Acquiring lock "ac620533-8274-4674-b241-f48526bed3c3-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:09:21 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-9d6be6e3-9cbe-41dd-b602-897934b4ee5f req-68efa4ca-7339-451d-83b7-bc3d016c5b0f service nova] Lock "ac620533-8274-4674-b241-f48526bed3c3-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:09:21 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-9d6be6e3-9cbe-41dd-b602-897934b4ee5f req-68efa4ca-7339-451d-83b7-bc3d016c5b0f service nova] Lock "ac620533-8274-4674-b241-f48526bed3c3-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:09:21 user nova-compute[75234]: DEBUG nova.compute.manager [req-9d6be6e3-9cbe-41dd-b602-897934b4ee5f req-68efa4ca-7339-451d-83b7-bc3d016c5b0f service nova] [instance: ac620533-8274-4674-b241-f48526bed3c3] No waiting events found dispatching network-vif-plugged-f96869cf-1585-47e8-a04e-40ad5f212130 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:09:21 user nova-compute[75234]: WARNING nova.compute.manager [req-9d6be6e3-9cbe-41dd-b602-897934b4ee5f req-68efa4ca-7339-451d-83b7-bc3d016c5b0f service nova] [instance: ac620533-8274-4674-b241-f48526bed3c3] Received unexpected event network-vif-plugged-f96869cf-1585-47e8-a04e-40ad5f212130 for instance with vm_state building and task_state spawning.
Nov 29 03:09:21 user nova-compute[75234]: INFO nova.compute.manager [None req-65aec05b-a901-4f17-a4e0-2e822560de93 tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] [instance: ac620533-8274-4674-b241-f48526bed3c3] Took 7.33 seconds to build instance.
Nov 29 03:09:21 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-65aec05b-a901-4f17-a4e0-2e822560de93 tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] Lock "ac620533-8274-4674-b241-f48526bed3c3" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 7.458s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:09:21 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:09:21 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:09:21 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-cf00e939-67db-4afa-b7b4-2ac67871e737 tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] Searching for a device in session 12 and hctl ['33', '*', '*', 0] yield: sdb {{(pid=75234) device_name_by_hctl /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:789}}
Nov 29 03:09:21 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-cf00e939-67db-4afa-b7b4-2ac67871e737 tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] Connected to sdb using {'target_iqn': 'iqn.2010-10.org.openstack:ada91c26-ca97-4b8b-98bb-a310d0627691', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'pbUzSvGT', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False} {{(pid=75234) _connect_vol /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:672}}
Nov 29 03:09:21 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-cf00e939-67db-4afa-b7b4-2ac67871e737 tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.326s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:87}}
Nov 29 03:09:21 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-cf00e939-67db-4afa-b7b4-2ac67871e737 tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] <== connect_volume: return (1327ms) {'type': 'block', 'scsi_wwn': '26339663765653931', 'path': '/dev/sdb'} {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:203}}
Nov 29 03:09:21 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-cf00e939-67db-4afa-b7b4-2ac67871e737 tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] [instance: f7d0bdeb-6a05-47ac-87f2-c8380a9a4286] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '26339663765653931', 'path': '/dev/sdb'} {{(pid=75234) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Nov 29 03:09:22 user nova-compute[75234]: DEBUG nova.objects.instance [None req-cf00e939-67db-4afa-b7b4-2ac67871e737 tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] Lazy-loading 'flavor' on Instance uuid f7d0bdeb-6a05-47ac-87f2-c8380a9a4286 {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:09:22 user nova-compute[75234]: DEBUG nova.virt.libvirt.guest [None req-cf00e939-67db-4afa-b7b4-2ac67871e737 tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] attach device xml: <disk type="block" device="disk">
Nov 29 03:09:22 user nova-compute[75234]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Nov 29 03:09:22 user nova-compute[75234]:   <source dev="/dev/sdb"/>
Nov 29 03:09:22 user nova-compute[75234]:   <target dev="vdb" bus="virtio"/>
Nov 29 03:09:22 user nova-compute[75234]:   <serial>ada91c26-ca97-4b8b-98bb-a310d0627691</serial>
Nov 29 03:09:22 user nova-compute[75234]: </disk>
Nov 29 03:09:22 user nova-compute[75234]:  {{(pid=75234) attach_device /opt/stack/nova/nova/virt/libvirt/guest.py:339}}
Nov 29 03:09:22 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-cf00e939-67db-4afa-b7b4-2ac67871e737 tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] No BDM found with device name vda, not building metadata. {{(pid=75234) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12089}}
Nov 29 03:09:22 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-cf00e939-67db-4afa-b7b4-2ac67871e737 tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] No BDM found with device name vdb, not building metadata. {{(pid=75234) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12089}}
Nov 29 03:09:22 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-cf00e939-67db-4afa-b7b4-2ac67871e737 tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] No VIF found with MAC fa:16:3e:59:dd:b5, not building metadata {{(pid=75234) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Nov 29 03:09:22 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-cf00e939-67db-4afa-b7b4-2ac67871e737 tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] Lock "f7d0bdeb-6a05-47ac-87f2-c8380a9a4286" "released" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: held 9.913s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:09:26 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:09:26 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:09:26 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-52323a77-5f22-48d6-8fcf-24998492e5ac tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] Acquiring lock "f7d0bdeb-6a05-47ac-87f2-c8380a9a4286" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:09:26 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-52323a77-5f22-48d6-8fcf-24998492e5ac tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] Lock "f7d0bdeb-6a05-47ac-87f2-c8380a9a4286" acquired by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:09:26 user nova-compute[75234]: DEBUG nova.objects.instance [None req-52323a77-5f22-48d6-8fcf-24998492e5ac tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] Lazy-loading 'flavor' on Instance uuid f7d0bdeb-6a05-47ac-87f2-c8380a9a4286 {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:09:26 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-52323a77-5f22-48d6-8fcf-24998492e5ac tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] Lock "f7d0bdeb-6a05-47ac-87f2-c8380a9a4286" "released" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: held 0.053s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:09:27 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-52323a77-5f22-48d6-8fcf-24998492e5ac tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] Acquiring lock "f7d0bdeb-6a05-47ac-87f2-c8380a9a4286" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:09:27 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-52323a77-5f22-48d6-8fcf-24998492e5ac tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] Lock "f7d0bdeb-6a05-47ac-87f2-c8380a9a4286" acquired by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:09:27 user nova-compute[75234]: INFO nova.compute.manager [None req-52323a77-5f22-48d6-8fcf-24998492e5ac tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] [instance: f7d0bdeb-6a05-47ac-87f2-c8380a9a4286] Attaching volume 6e883290-42f6-40fd-854d-c42f9aca8992 to /dev/vdc
Nov 29 03:09:27 user nova-compute[75234]: DEBUG os_brick.utils [None req-52323a77-5f22-48d6-8fcf-24998492e5ac tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:176}}
Nov 29 03:09:27 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:09:27 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.016s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:09:27 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[68a66f68-4b30-4491-a3a7-865948d11123]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:dd1368c5ba2c\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:09:27 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt -v / -n -o SOURCE {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:09:27 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt -v / -n -o SOURCE" returned: 0 in 0.016s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:09:27 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[c477e1b3-b10a-4b4c-81b3-aad2aafc4bbb]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:09:27 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:09:27 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.018s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:09:27 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[4d90b685-b69e-4918-b830-aa0205605b95]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:09:27 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[bbcf6d54-78ef-4027-921e-b1ffb072a897]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:09:27 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-52323a77-5f22-48d6-8fcf-24998492e5ac tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] Running cmd (subprocess): nvme version {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:09:27 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-52323a77-5f22-48d6-8fcf-24998492e5ac tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] 'nvme version' failed. Not Retrying. {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:473}}
Nov 29 03:09:27 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.nvmeof [None req-52323a77-5f22-48d6-8fcf-24998492e5ac tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] nvme not present on system {{(pid=75234) nvme_present /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/nvmeof.py:756}}
Nov 29 03:09:27 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:09:27 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:473}}
Nov 29 03:09:27 user nova-compute[75234]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Nov 29 03:09:27 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[002d1852-3575-4dba-8dc0-d853e14204f3]: (4, '') {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:09:27 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.lightos [None req-52323a77-5f22-48d6-8fcf-24998492e5ac tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=75234) find_dsc /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/lightos.py:98}}
Nov 29 03:09:27 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.lightos [None req-52323a77-5f22-48d6-8fcf-24998492e5ac tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=75234) get_connector_properties /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/lightos.py:76}}
Nov 29 03:09:27 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.lightos [None req-52323a77-5f22-48d6-8fcf-24998492e5ac tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] LIGHTOS: no hostnqn found. {{(pid=75234) get_connector_properties /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/lightos.py:84}}
Nov 29 03:09:27 user nova-compute[75234]: DEBUG os_brick.utils [None req-52323a77-5f22-48d6-8fcf-24998492e5ac tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] <== get_connector_properties: return (125ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:dd1368c5ba2c', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:203}}
Nov 29 03:09:27 user nova-compute[75234]: DEBUG nova.virt.block_device [None req-52323a77-5f22-48d6-8fcf-24998492e5ac tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] [instance: f7d0bdeb-6a05-47ac-87f2-c8380a9a4286] Updating existing volume attachment record: 10514319-8875-4117-9dae-d1198bda40d2 {{(pid=75234) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Nov 29 03:09:31 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:09:31 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:09:31 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5003 ms, sending inactivity probe {{(pid=75234) run /opt/stack/data/venv/lib/python3.10/site-packages/ovs/reconnect.py:117}}
Nov 29 03:09:31 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=75234) _transition /opt/stack/data/venv/lib/python3.10/site-packages/ovs/reconnect.py:519}}
Nov 29 03:09:31 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=75234) _transition /opt/stack/data/venv/lib/python3.10/site-packages/ovs/reconnect.py:519}}
Nov 29 03:09:31 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:09:33 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-92401ae9-0e20-4861-8c4a-6e78a4efc497 tempest-VolumesAssistedSnapshotsTest-1323460801 tempest-VolumesAssistedSnapshotsTest-1323460801-project-member] Acquiring lock "c9663ab3-3374-400e-ab0a-769731b78cca" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:09:33 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-92401ae9-0e20-4861-8c4a-6e78a4efc497 tempest-VolumesAssistedSnapshotsTest-1323460801 tempest-VolumesAssistedSnapshotsTest-1323460801-project-member] Lock "c9663ab3-3374-400e-ab0a-769731b78cca" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:09:33 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-92401ae9-0e20-4861-8c4a-6e78a4efc497 tempest-VolumesAssistedSnapshotsTest-1323460801 tempest-VolumesAssistedSnapshotsTest-1323460801-project-member] Acquiring lock "c9663ab3-3374-400e-ab0a-769731b78cca-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:09:33 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-92401ae9-0e20-4861-8c4a-6e78a4efc497 tempest-VolumesAssistedSnapshotsTest-1323460801 tempest-VolumesAssistedSnapshotsTest-1323460801-project-member] Lock "c9663ab3-3374-400e-ab0a-769731b78cca-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:09:33 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-92401ae9-0e20-4861-8c4a-6e78a4efc497 tempest-VolumesAssistedSnapshotsTest-1323460801 tempest-VolumesAssistedSnapshotsTest-1323460801-project-member] Lock "c9663ab3-3374-400e-ab0a-769731b78cca-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:09:33 user nova-compute[75234]: INFO nova.compute.manager [None req-92401ae9-0e20-4861-8c4a-6e78a4efc497 tempest-VolumesAssistedSnapshotsTest-1323460801 tempest-VolumesAssistedSnapshotsTest-1323460801-project-member] [instance: c9663ab3-3374-400e-ab0a-769731b78cca] Terminating instance
Nov 29 03:09:33 user nova-compute[75234]: DEBUG nova.compute.manager [None req-92401ae9-0e20-4861-8c4a-6e78a4efc497 tempest-VolumesAssistedSnapshotsTest-1323460801 tempest-VolumesAssistedSnapshotsTest-1323460801-project-member] [instance: c9663ab3-3374-400e-ab0a-769731b78cca] Start destroying the instance on the hypervisor. {{(pid=75234) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3140}}
Nov 29 03:09:33 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:09:33 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:09:33 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:09:34 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:09:34 user nova-compute[75234]: DEBUG nova.compute.manager [req-fbe38206-afe9-4ece-90e0-d414baf6aa69 req-93b3c6e0-f7ef-4054-9435-2c247490ebdf service nova] [instance: c9663ab3-3374-400e-ab0a-769731b78cca] Received event network-vif-unplugged-b55d6eaa-b0d4-4a67-8407-3a8b69558e43 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:09:34 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-fbe38206-afe9-4ece-90e0-d414baf6aa69 req-93b3c6e0-f7ef-4054-9435-2c247490ebdf service nova] Acquiring lock "c9663ab3-3374-400e-ab0a-769731b78cca-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:09:34 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-fbe38206-afe9-4ece-90e0-d414baf6aa69 req-93b3c6e0-f7ef-4054-9435-2c247490ebdf service nova] Lock "c9663ab3-3374-400e-ab0a-769731b78cca-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:09:34 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-fbe38206-afe9-4ece-90e0-d414baf6aa69 req-93b3c6e0-f7ef-4054-9435-2c247490ebdf service nova] Lock "c9663ab3-3374-400e-ab0a-769731b78cca-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:09:34 user nova-compute[75234]: DEBUG nova.compute.manager [req-fbe38206-afe9-4ece-90e0-d414baf6aa69 req-93b3c6e0-f7ef-4054-9435-2c247490ebdf service nova] [instance: c9663ab3-3374-400e-ab0a-769731b78cca] No waiting events found dispatching network-vif-unplugged-b55d6eaa-b0d4-4a67-8407-3a8b69558e43 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:09:34 user nova-compute[75234]: DEBUG nova.compute.manager [req-fbe38206-afe9-4ece-90e0-d414baf6aa69 req-93b3c6e0-f7ef-4054-9435-2c247490ebdf service nova] [instance: c9663ab3-3374-400e-ab0a-769731b78cca] Received event network-vif-unplugged-b55d6eaa-b0d4-4a67-8407-3a8b69558e43 for instance with task_state deleting. {{(pid=75234) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10890}}
Nov 29 03:09:34 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:09:34 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:09:34 user nova-compute[75234]: INFO nova.virt.libvirt.driver [-] [instance: c9663ab3-3374-400e-ab0a-769731b78cca] Instance destroyed successfully.
Nov 29 03:09:34 user nova-compute[75234]: DEBUG nova.objects.instance [None req-92401ae9-0e20-4861-8c4a-6e78a4efc497 tempest-VolumesAssistedSnapshotsTest-1323460801 tempest-VolumesAssistedSnapshotsTest-1323460801-project-member] Lazy-loading 'resources' on Instance uuid c9663ab3-3374-400e-ab0a-769731b78cca {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:09:34 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-92401ae9-0e20-4861-8c4a-6e78a4efc497 tempest-VolumesAssistedSnapshotsTest-1323460801 tempest-VolumesAssistedSnapshotsTest-1323460801-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-11-29T03:06:35Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-VolumesAssistedSnapshotsTest-server-15749010',display_name='tempest-VolumesAssistedSnapshotsTest-server-15749010',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-volumesassistedsnapshotstest-server-15749010',id=9,image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBOpV1r9kDBNAejJiIxQHnv5nT3IVZ/wlovVaZh1jY/ciYwn121v+8yMvPVcNbJUyt62m/+e6PHsXJeMqoN77/bu6BfTHWz76/IxLZ6WUZ95en78z4N4sjKvEYdHAkzx67g==',key_name='tempest-keypair-973383639',keypairs=<?>,launch_index=0,launched_at=2023-11-29T03:06:44Z,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='d59c913723894ba1938fb8177c97e627',ramdisk_id='',reservation_id='r-3beeznuu',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-VolumesAssistedSnapshotsTest-1323460801',owner_user_name='tempest-VolumesAssistedSnapshotsTest-1323460801-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-11-29T03:06:45Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='f90b22e80de849c3b1ca951f7650fe84',uuid=c9663ab3-3374-400e-ab0a-769731b78cca,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "b55d6eaa-b0d4-4a67-8407-3a8b69558e43", "address": "fa:16:3e:3e:b6:0b", "network": {"id": "8d7bb68b-4e8e-42c9-8d1a-3d0e301a7b73", "bridge": "br-int", "label": "tempest-VolumesAssistedSnapshotsTest-1523028903-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.204", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "d59c913723894ba1938fb8177c97e627", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapb55d6eaa-b0", "ovs_interfaceid": "b55d6eaa-b0d4-4a67-8407-3a8b69558e43", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Nov 29 03:09:34 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-92401ae9-0e20-4861-8c4a-6e78a4efc497 tempest-VolumesAssistedSnapshotsTest-1323460801 tempest-VolumesAssistedSnapshotsTest-1323460801-project-member] Converting VIF {"id": "b55d6eaa-b0d4-4a67-8407-3a8b69558e43", "address": "fa:16:3e:3e:b6:0b", "network": {"id": "8d7bb68b-4e8e-42c9-8d1a-3d0e301a7b73", "bridge": "br-int", "label": "tempest-VolumesAssistedSnapshotsTest-1523028903-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.204", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "d59c913723894ba1938fb8177c97e627", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapb55d6eaa-b0", "ovs_interfaceid": "b55d6eaa-b0d4-4a67-8407-3a8b69558e43", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:09:34 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-92401ae9-0e20-4861-8c4a-6e78a4efc497 tempest-VolumesAssistedSnapshotsTest-1323460801 tempest-VolumesAssistedSnapshotsTest-1323460801-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:3e:b6:0b,bridge_name='br-int',has_traffic_filtering=True,id=b55d6eaa-b0d4-4a67-8407-3a8b69558e43,network=Network(8d7bb68b-4e8e-42c9-8d1a-3d0e301a7b73),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapb55d6eaa-b0') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:09:34 user nova-compute[75234]: DEBUG os_vif [None req-92401ae9-0e20-4861-8c4a-6e78a4efc497 tempest-VolumesAssistedSnapshotsTest-1323460801 tempest-VolumesAssistedSnapshotsTest-1323460801-project-member] Unplugging vif VIFOpenVSwitch(active=True,address=fa:16:3e:3e:b6:0b,bridge_name='br-int',has_traffic_filtering=True,id=b55d6eaa-b0d4-4a67-8407-3a8b69558e43,network=Network(8d7bb68b-4e8e-42c9-8d1a-3d0e301a7b73),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapb55d6eaa-b0') {{(pid=75234) unplug /opt/stack/data/venv/lib/python3.10/site-packages/os_vif/__init__.py:109}}
Nov 29 03:09:34 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:09:34 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tapb55d6eaa-b0, bridge=br-int, if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:09:34 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:09:34 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:09:34 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:09:34 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:09:34 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbDestroyCommand(_result=None, table=QoS, record=c27d0110-659d-48ee-917e-82dd53a5bc62) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:09:34 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:09:34 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:09:34 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:09:34 user nova-compute[75234]: INFO os_vif [None req-92401ae9-0e20-4861-8c4a-6e78a4efc497 tempest-VolumesAssistedSnapshotsTest-1323460801 tempest-VolumesAssistedSnapshotsTest-1323460801-project-member] Successfully unplugged vif VIFOpenVSwitch(active=True,address=fa:16:3e:3e:b6:0b,bridge_name='br-int',has_traffic_filtering=True,id=b55d6eaa-b0d4-4a67-8407-3a8b69558e43,network=Network(8d7bb68b-4e8e-42c9-8d1a-3d0e301a7b73),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapb55d6eaa-b0')
Nov 29 03:09:34 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-92401ae9-0e20-4861-8c4a-6e78a4efc497 tempest-VolumesAssistedSnapshotsTest-1323460801 tempest-VolumesAssistedSnapshotsTest-1323460801-project-member] [instance: c9663ab3-3374-400e-ab0a-769731b78cca] Deleting instance files /opt/stack/data/nova/instances/c9663ab3-3374-400e-ab0a-769731b78cca_del
Nov 29 03:09:34 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-92401ae9-0e20-4861-8c4a-6e78a4efc497 tempest-VolumesAssistedSnapshotsTest-1323460801 tempest-VolumesAssistedSnapshotsTest-1323460801-project-member] [instance: c9663ab3-3374-400e-ab0a-769731b78cca] Deletion of /opt/stack/data/nova/instances/c9663ab3-3374-400e-ab0a-769731b78cca_del complete
Nov 29 03:09:34 user nova-compute[75234]: INFO nova.compute.manager [None req-92401ae9-0e20-4861-8c4a-6e78a4efc497 tempest-VolumesAssistedSnapshotsTest-1323460801 tempest-VolumesAssistedSnapshotsTest-1323460801-project-member] [instance: c9663ab3-3374-400e-ab0a-769731b78cca] Took 1.10 seconds to destroy the instance on the hypervisor.
Nov 29 03:09:34 user nova-compute[75234]: DEBUG oslo.service.loopingcall [None req-92401ae9-0e20-4861-8c4a-6e78a4efc497 tempest-VolumesAssistedSnapshotsTest-1323460801 tempest-VolumesAssistedSnapshotsTest-1323460801-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=75234) func /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/loopingcall.py:435}}
Nov 29 03:09:34 user nova-compute[75234]: DEBUG nova.compute.manager [-] [instance: c9663ab3-3374-400e-ab0a-769731b78cca] Deallocating network for instance {{(pid=75234) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2279}}
Nov 29 03:09:34 user nova-compute[75234]: DEBUG nova.network.neutron [-] [instance: c9663ab3-3374-400e-ab0a-769731b78cca] deallocate_for_instance() {{(pid=75234) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1802}}
Nov 29 03:09:35 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-52323a77-5f22-48d6-8fcf-24998492e5ac tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] [instance: f7d0bdeb-6a05-47ac-87f2-c8380a9a4286] Calling os-brick to attach iSCSI Volume {{(pid=75234) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Nov 29 03:09:35 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-52323a77-5f22-48d6-8fcf-24998492e5ac tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f8a5050c9a0>, 'connection_properties': {'target_iqn': 'iqn.2010-10.org.openstack:6e883290-42f6-40fd-854d-c42f9aca8992', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'IAXcliLh', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:176}}
Nov 29 03:09:35 user nova-compute[75234]: WARNING os_brick.initiator.connectors.base [None req-52323a77-5f22-48d6-8fcf-24998492e5ac tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Nov 29 03:09:35 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-52323a77-5f22-48d6-8fcf-24998492e5ac tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:68}}
Nov 29 03:09:35 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-52323a77-5f22-48d6-8fcf-24998492e5ac tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:73}}
Nov 29 03:09:35 user nova-compute[75234]: WARNING os_brick.initiator.connectors.base [None req-52323a77-5f22-48d6-8fcf-24998492e5ac tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Nov 29 03:09:35 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-52323a77-5f22-48d6-8fcf-24998492e5ac tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:6e883290-42f6-40fd-854d-c42f9aca8992" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:68}}
Nov 29 03:09:35 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-52323a77-5f22-48d6-8fcf-24998492e5ac tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:6e883290-42f6-40fd-854d-c42f9aca8992" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:73}}
Nov 29 03:09:35 user nova-compute[75234]: INFO os_brick.initiator.connectors.iscsi [None req-52323a77-5f22-48d6-8fcf-24998492e5ac tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Nov 29 03:09:35 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:6e883290-42f6-40fd-854d-c42f9aca8992 -p 172.16.0.220:3260 {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:09:35 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:6e883290-42f6-40fd-854d-c42f9aca8992 -p 172.16.0.220:3260" returned: 21 in 0.023s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:09:35 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[06ae59a2-b13d-4f89-a4bb-b4e9897109b4]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:09:35 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-52323a77-5f22-48d6-8fcf-24998492e5ac tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Nov 29 03:09:35 user nova-compute[75234]:  {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:09:35 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:6e883290-42f6-40fd-854d-c42f9aca8992 -p 172.16.0.220:3260 --interface default --op new {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:09:35 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:6e883290-42f6-40fd-854d-c42f9aca8992 -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.019s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:09:35 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[fae0c6b2-f40f-4117-9630-d1e8f2dc5116]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:6e883290-42f6-40fd-854d-c42f9aca8992] added\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:09:35 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-52323a77-5f22-48d6-8fcf-24998492e5ac tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:6e883290-42f6-40fd-854d-c42f9aca8992] added
Nov 29 03:09:35 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:09:35 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:6e883290-42f6-40fd-854d-c42f9aca8992 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:09:35 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:6e883290-42f6-40fd-854d-c42f9aca8992 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.023s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:09:35 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[36410902-dee5-4de5-94d1-498445c727df]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:09:35 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-52323a77-5f22-48d6-8fcf-24998492e5ac tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:09:35 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:6e883290-42f6-40fd-854d-c42f9aca8992 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:09:35 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:6e883290-42f6-40fd-854d-c42f9aca8992 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.017s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:09:35 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[913af4cf-2cb0-412b-ad88-90594a893099]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:09:35 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-52323a77-5f22-48d6-8fcf-24998492e5ac tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:09:35 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:6e883290-42f6-40fd-854d-c42f9aca8992 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v IAXcliLh {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:09:35 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:6e883290-42f6-40fd-854d-c42f9aca8992 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v IAXcliLh" returned: 0 in 0.018s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:09:35 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[11b14d72-9c13-4bdd-832c-b54327d4b9ec]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:09:35 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-52323a77-5f22-48d6-8fcf-24998492e5ac tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'IAXcliLh'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:09:35 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:6e883290-42f6-40fd-854d-c42f9aca8992 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:09:35 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:6e883290-42f6-40fd-854d-c42f9aca8992 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.018s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:09:35 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[453f739c-7aeb-4d99-8db2-f702095e35d6]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:09:35 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-52323a77-5f22-48d6-8fcf-24998492e5ac tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:09:35 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:09:35 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.020s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:09:35 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[4a76006c-498c-49d0-a8ac-b1409f49679b]: (4, ('tcp: [12] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ada91c26-ca97-4b8b-98bb-a310d0627691 (non-flash)\ntcp: [9] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672 (non-flash)\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:09:35 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-52323a77-5f22-48d6-8fcf-24998492e5ac tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [12] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ada91c26-ca97-4b8b-98bb-a310d0627691 (non-flash)
Nov 29 03:09:35 user nova-compute[75234]: tcp: [9] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672 (non-flash)
Nov 29 03:09:35 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:09:35 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-52323a77-5f22-48d6-8fcf-24998492e5ac tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] iscsi session list stdout=tcp: [12] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ada91c26-ca97-4b8b-98bb-a310d0627691 (non-flash)
Nov 29 03:09:35 user nova-compute[75234]: tcp: [9] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672 (non-flash)
Nov 29 03:09:35 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsi_session /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Nov 29 03:09:35 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:6e883290-42f6-40fd-854d-c42f9aca8992 -p 172.16.0.220:3260 --login {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:09:35 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:6e883290-42f6-40fd-854d-c42f9aca8992 -p 172.16.0.220:3260 --login" returned: 0 in 0.060s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:09:35 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[f8b78a75-0f69-40bd-9f6e-820aa966086d]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:6e883290-42f6-40fd-854d-c42f9aca8992, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:6e883290-42f6-40fd-854d-c42f9aca8992, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:09:35 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-52323a77-5f22-48d6-8fcf-24998492e5ac tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:6e883290-42f6-40fd-854d-c42f9aca8992, portal: 172.16.0.220,3260]
Nov 29 03:09:35 user nova-compute[75234]: Login to [iface: default, target: iqn.2010-10.org.openstack:6e883290-42f6-40fd-854d-c42f9aca8992, portal: 172.16.0.220,3260] successful.
Nov 29 03:09:35 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:09:35 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:6e883290-42f6-40fd-854d-c42f9aca8992 -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:09:35 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:6e883290-42f6-40fd-854d-c42f9aca8992 -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.018s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:09:35 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[15dab59b-88ac-4499-963e-360612347985]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:09:35 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-52323a77-5f22-48d6-8fcf-24998492e5ac tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:09:35 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:09:35 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.021s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:09:35 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[6401d920-f788-43a1-87ac-dc5f7e52be5d]: (4, ('tcp: [12] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ada91c26-ca97-4b8b-98bb-a310d0627691 (non-flash)\ntcp: [13] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e883290-42f6-40fd-854d-c42f9aca8992 (non-flash)\ntcp: [9] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672 (non-flash)\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:09:35 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-52323a77-5f22-48d6-8fcf-24998492e5ac tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [12] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ada91c26-ca97-4b8b-98bb-a310d0627691 (non-flash)
Nov 29 03:09:35 user nova-compute[75234]: tcp: [13] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e883290-42f6-40fd-854d-c42f9aca8992 (non-flash)
Nov 29 03:09:35 user nova-compute[75234]: tcp: [9] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672 (non-flash)
Nov 29 03:09:35 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:09:35 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-52323a77-5f22-48d6-8fcf-24998492e5ac tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] iscsi session list stdout=tcp: [12] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ada91c26-ca97-4b8b-98bb-a310d0627691 (non-flash)
Nov 29 03:09:35 user nova-compute[75234]: tcp: [13] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e883290-42f6-40fd-854d-c42f9aca8992 (non-flash)
Nov 29 03:09:35 user nova-compute[75234]: tcp: [9] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672 (non-flash)
Nov 29 03:09:35 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsi_session /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Nov 29 03:09:35 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-52323a77-5f22-48d6-8fcf-24998492e5ac tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:6e883290-42f6-40fd-854d-c42f9aca8992" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.280s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:87}}
Nov 29 03:09:35 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-52323a77-5f22-48d6-8fcf-24998492e5ac tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] Connected to 172.16.0.220:3260 {{(pid=75234) _connect_vol /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:644}}
Nov 29 03:09:35 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-52323a77-5f22-48d6-8fcf-24998492e5ac tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] HCTL ('34', '-', '-', 0) found on session 13 with lun 0 {{(pid=75234) get_hctl /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:770}}
Nov 29 03:09:35 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-52323a77-5f22-48d6-8fcf-24998492e5ac tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] Scanning host 34 c: -, t: -, l: 0) {{(pid=75234) scan_iscsi /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:795}}
Nov 29 03:09:35 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host34/scan {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:09:35 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host34/scan" returned: 0 in 0.018s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:09:35 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[f62074e4-8bcb-415c-989d-a4457362cd27]: (4, ('- - 0', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:09:35 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-52323a77-5f22-48d6-8fcf-24998492e5ac tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] Searching for a device in session 13 and hctl ['34', '*', '*', 0] yield: None {{(pid=75234) device_name_by_hctl /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:789}}
Nov 29 03:09:36 user nova-compute[75234]: DEBUG nova.network.neutron [-] [instance: c9663ab3-3374-400e-ab0a-769731b78cca] Updating instance_info_cache with network_info: [] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:09:36 user nova-compute[75234]: INFO nova.compute.manager [-] [instance: c9663ab3-3374-400e-ab0a-769731b78cca] Took 1.46 seconds to deallocate network for instance.
Nov 29 03:09:36 user nova-compute[75234]: DEBUG nova.compute.manager [req-979ad78a-7cdb-4517-b301-cec17ba57ef6 req-3197fa5c-388b-41ca-8897-22d2e94603c6 service nova] [instance: c9663ab3-3374-400e-ab0a-769731b78cca] Received event network-vif-deleted-b55d6eaa-b0d4-4a67-8407-3a8b69558e43 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:09:36 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-92401ae9-0e20-4861-8c4a-6e78a4efc497 tempest-VolumesAssistedSnapshotsTest-1323460801 tempest-VolumesAssistedSnapshotsTest-1323460801-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:09:36 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-92401ae9-0e20-4861-8c4a-6e78a4efc497 tempest-VolumesAssistedSnapshotsTest-1323460801 tempest-VolumesAssistedSnapshotsTest-1323460801-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:09:36 user nova-compute[75234]: DEBUG nova.compute.manager [req-f4f5967e-eedc-4a3f-98f0-23a2398780ff req-e33322dc-acde-47b0-9ff1-8306a560c6a1 service nova] [instance: c9663ab3-3374-400e-ab0a-769731b78cca] Received event network-vif-plugged-b55d6eaa-b0d4-4a67-8407-3a8b69558e43 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:09:36 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-f4f5967e-eedc-4a3f-98f0-23a2398780ff req-e33322dc-acde-47b0-9ff1-8306a560c6a1 service nova] Acquiring lock "c9663ab3-3374-400e-ab0a-769731b78cca-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:09:36 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-f4f5967e-eedc-4a3f-98f0-23a2398780ff req-e33322dc-acde-47b0-9ff1-8306a560c6a1 service nova] Lock "c9663ab3-3374-400e-ab0a-769731b78cca-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:09:36 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-f4f5967e-eedc-4a3f-98f0-23a2398780ff req-e33322dc-acde-47b0-9ff1-8306a560c6a1 service nova] Lock "c9663ab3-3374-400e-ab0a-769731b78cca-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:09:36 user nova-compute[75234]: DEBUG nova.compute.manager [req-f4f5967e-eedc-4a3f-98f0-23a2398780ff req-e33322dc-acde-47b0-9ff1-8306a560c6a1 service nova] [instance: c9663ab3-3374-400e-ab0a-769731b78cca] No waiting events found dispatching network-vif-plugged-b55d6eaa-b0d4-4a67-8407-3a8b69558e43 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:09:36 user nova-compute[75234]: WARNING nova.compute.manager [req-f4f5967e-eedc-4a3f-98f0-23a2398780ff req-e33322dc-acde-47b0-9ff1-8306a560c6a1 service nova] [instance: c9663ab3-3374-400e-ab0a-769731b78cca] Received unexpected event network-vif-plugged-b55d6eaa-b0d4-4a67-8407-3a8b69558e43 for instance with vm_state deleted and task_state None.
Nov 29 03:09:36 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:09:36 user nova-compute[75234]: DEBUG nova.compute.provider_tree [None req-92401ae9-0e20-4861-8c4a-6e78a4efc497 tempest-VolumesAssistedSnapshotsTest-1323460801 tempest-VolumesAssistedSnapshotsTest-1323460801-project-member] Inventory has not changed in ProviderTree for provider: 4d291220-7c37-4839-a0e3-b23c66502b74 {{(pid=75234) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Nov 29 03:09:36 user nova-compute[75234]: DEBUG nova.scheduler.client.report [None req-92401ae9-0e20-4861-8c4a-6e78a4efc497 tempest-VolumesAssistedSnapshotsTest-1323460801 tempest-VolumesAssistedSnapshotsTest-1323460801-project-member] Inventory has not changed for provider 4d291220-7c37-4839-a0e3-b23c66502b74 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=75234) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Nov 29 03:09:36 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-92401ae9-0e20-4861-8c4a-6e78a4efc497 tempest-VolumesAssistedSnapshotsTest-1323460801 tempest-VolumesAssistedSnapshotsTest-1323460801-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.332s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:09:36 user nova-compute[75234]: INFO nova.scheduler.client.report [None req-92401ae9-0e20-4861-8c4a-6e78a4efc497 tempest-VolumesAssistedSnapshotsTest-1323460801 tempest-VolumesAssistedSnapshotsTest-1323460801-project-member] Deleted allocations for instance c9663ab3-3374-400e-ab0a-769731b78cca
Nov 29 03:09:36 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-92401ae9-0e20-4861-8c4a-6e78a4efc497 tempest-VolumesAssistedSnapshotsTest-1323460801 tempest-VolumesAssistedSnapshotsTest-1323460801-project-member] Lock "c9663ab3-3374-400e-ab0a-769731b78cca" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 3.102s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:09:36 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-52323a77-5f22-48d6-8fcf-24998492e5ac tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] Searching for a device in session 13 and hctl ['34', '*', '*', 0] yield: sdc {{(pid=75234) device_name_by_hctl /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:789}}
Nov 29 03:09:36 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-52323a77-5f22-48d6-8fcf-24998492e5ac tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] Connected to sdc using {'target_iqn': 'iqn.2010-10.org.openstack:6e883290-42f6-40fd-854d-c42f9aca8992', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'IAXcliLh', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False} {{(pid=75234) _connect_vol /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:672}}
Nov 29 03:09:36 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-52323a77-5f22-48d6-8fcf-24998492e5ac tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.312s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:87}}
Nov 29 03:09:36 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-52323a77-5f22-48d6-8fcf-24998492e5ac tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] <== connect_volume: return (1314ms) {'type': 'block', 'scsi_wwn': '23530333766313264', 'path': '/dev/sdc'} {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:203}}
Nov 29 03:09:36 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-52323a77-5f22-48d6-8fcf-24998492e5ac tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] [instance: f7d0bdeb-6a05-47ac-87f2-c8380a9a4286] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '23530333766313264', 'path': '/dev/sdc'} {{(pid=75234) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Nov 29 03:09:36 user nova-compute[75234]: DEBUG nova.objects.instance [None req-52323a77-5f22-48d6-8fcf-24998492e5ac tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] Lazy-loading 'flavor' on Instance uuid f7d0bdeb-6a05-47ac-87f2-c8380a9a4286 {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:09:36 user nova-compute[75234]: DEBUG nova.virt.libvirt.guest [None req-52323a77-5f22-48d6-8fcf-24998492e5ac tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] attach device xml: <disk type="block" device="disk">
Nov 29 03:09:36 user nova-compute[75234]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Nov 29 03:09:36 user nova-compute[75234]:   <source dev="/dev/sdc"/>
Nov 29 03:09:36 user nova-compute[75234]:   <target dev="vdc" bus="virtio"/>
Nov 29 03:09:36 user nova-compute[75234]:   <serial>6e883290-42f6-40fd-854d-c42f9aca8992</serial>
Nov 29 03:09:36 user nova-compute[75234]: </disk>
Nov 29 03:09:36 user nova-compute[75234]:  {{(pid=75234) attach_device /opt/stack/nova/nova/virt/libvirt/guest.py:339}}
Nov 29 03:09:37 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-52323a77-5f22-48d6-8fcf-24998492e5ac tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] No BDM found with device name vda, not building metadata. {{(pid=75234) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12089}}
Nov 29 03:09:37 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-52323a77-5f22-48d6-8fcf-24998492e5ac tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] No BDM found with device name vdb, not building metadata. {{(pid=75234) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12089}}
Nov 29 03:09:37 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-52323a77-5f22-48d6-8fcf-24998492e5ac tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] No BDM found with device name vdc, not building metadata. {{(pid=75234) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12089}}
Nov 29 03:09:37 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-52323a77-5f22-48d6-8fcf-24998492e5ac tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] No VIF found with MAC fa:16:3e:59:dd:b5, not building metadata {{(pid=75234) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Nov 29 03:09:37 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-52323a77-5f22-48d6-8fcf-24998492e5ac tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] Lock "f7d0bdeb-6a05-47ac-87f2-c8380a9a4286" "released" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: held 10.445s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:09:38 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-4badf7dd-4675-4725-b5e9-a99187ebb992 tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] Acquiring lock "f7d0bdeb-6a05-47ac-87f2-c8380a9a4286" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:09:38 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-4badf7dd-4675-4725-b5e9-a99187ebb992 tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] Lock "f7d0bdeb-6a05-47ac-87f2-c8380a9a4286" acquired by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: waited 0.002s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:09:38 user nova-compute[75234]: INFO nova.compute.manager [None req-4badf7dd-4675-4725-b5e9-a99187ebb992 tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] [instance: f7d0bdeb-6a05-47ac-87f2-c8380a9a4286] Detaching volume 6e883290-42f6-40fd-854d-c42f9aca8992
Nov 29 03:09:38 user nova-compute[75234]: INFO nova.virt.block_device [None req-4badf7dd-4675-4725-b5e9-a99187ebb992 tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] [instance: f7d0bdeb-6a05-47ac-87f2-c8380a9a4286] Attempting to driver detach volume 6e883290-42f6-40fd-854d-c42f9aca8992 from mountpoint /dev/vdc
Nov 29 03:09:38 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-4badf7dd-4675-4725-b5e9-a99187ebb992 tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] Attempting to detach device vdc from instance f7d0bdeb-6a05-47ac-87f2-c8380a9a4286 from the persistent domain config. {{(pid=75234) _detach_from_persistent /opt/stack/nova/nova/virt/libvirt/driver.py:2487}}
Nov 29 03:09:38 user nova-compute[75234]: DEBUG nova.virt.libvirt.guest [None req-4badf7dd-4675-4725-b5e9-a99187ebb992 tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] detach device xml: <disk type="block" device="disk">
Nov 29 03:09:38 user nova-compute[75234]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Nov 29 03:09:38 user nova-compute[75234]:   <source dev="/dev/sdc"/>
Nov 29 03:09:38 user nova-compute[75234]:   <target dev="vdc" bus="virtio"/>
Nov 29 03:09:38 user nova-compute[75234]:   <serial>6e883290-42f6-40fd-854d-c42f9aca8992</serial>
Nov 29 03:09:38 user nova-compute[75234]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x08" function="0x0"/>
Nov 29 03:09:38 user nova-compute[75234]: </disk>
Nov 29 03:09:38 user nova-compute[75234]:  {{(pid=75234) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Nov 29 03:09:38 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-4badf7dd-4675-4725-b5e9-a99187ebb992 tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] Successfully detached device vdc from instance f7d0bdeb-6a05-47ac-87f2-c8380a9a4286 from the persistent domain config.
Nov 29 03:09:38 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-4badf7dd-4675-4725-b5e9-a99187ebb992 tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] (1/8): Attempting to detach device vdc with device alias virtio-disk2 from instance f7d0bdeb-6a05-47ac-87f2-c8380a9a4286 from the live domain config. {{(pid=75234) _detach_from_live_with_retry /opt/stack/nova/nova/virt/libvirt/driver.py:2523}}
Nov 29 03:09:38 user nova-compute[75234]: DEBUG nova.virt.libvirt.guest [None req-4badf7dd-4675-4725-b5e9-a99187ebb992 tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] detach device xml: <disk type="block" device="disk">
Nov 29 03:09:38 user nova-compute[75234]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Nov 29 03:09:38 user nova-compute[75234]:   <source dev="/dev/sdc"/>
Nov 29 03:09:38 user nova-compute[75234]:   <target dev="vdc" bus="virtio"/>
Nov 29 03:09:38 user nova-compute[75234]:   <serial>6e883290-42f6-40fd-854d-c42f9aca8992</serial>
Nov 29 03:09:38 user nova-compute[75234]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x08" function="0x0"/>
Nov 29 03:09:38 user nova-compute[75234]: </disk>
Nov 29 03:09:38 user nova-compute[75234]:  {{(pid=75234) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Nov 29 03:09:38 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-4badf7dd-4675-4725-b5e9-a99187ebb992 tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] Start waiting for the detach event from libvirt for device vdc with device alias virtio-disk2 for instance f7d0bdeb-6a05-47ac-87f2-c8380a9a4286 {{(pid=75234) _detach_from_live_and_wait_for_event /opt/stack/nova/nova/virt/libvirt/driver.py:2599}}
Nov 29 03:09:39 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:09:41 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:09:42 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-1806247b-6283-409e-9863-9bb966ef4e2a tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] Acquiring lock "23a31b1e-4005-442f-96e4-9fe82d9e17ba" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:09:42 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-1806247b-6283-409e-9863-9bb966ef4e2a tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] Lock "23a31b1e-4005-442f-96e4-9fe82d9e17ba" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:09:42 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-1806247b-6283-409e-9863-9bb966ef4e2a tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] Acquiring lock "23a31b1e-4005-442f-96e4-9fe82d9e17ba-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:09:42 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-1806247b-6283-409e-9863-9bb966ef4e2a tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] Lock "23a31b1e-4005-442f-96e4-9fe82d9e17ba-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:09:42 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-1806247b-6283-409e-9863-9bb966ef4e2a tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] Lock "23a31b1e-4005-442f-96e4-9fe82d9e17ba-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:09:42 user nova-compute[75234]: INFO nova.compute.manager [None req-1806247b-6283-409e-9863-9bb966ef4e2a tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] [instance: 23a31b1e-4005-442f-96e4-9fe82d9e17ba] Terminating instance
Nov 29 03:09:42 user nova-compute[75234]: DEBUG nova.compute.manager [None req-1806247b-6283-409e-9863-9bb966ef4e2a tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] [instance: 23a31b1e-4005-442f-96e4-9fe82d9e17ba] Start destroying the instance on the hypervisor. {{(pid=75234) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3140}}
Nov 29 03:09:42 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:09:42 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:09:42 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:09:42 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:09:42 user nova-compute[75234]: DEBUG nova.compute.manager [req-6b97117a-af80-4344-8cff-d7996a1a78cb req-887b3168-4325-41e8-8f27-2f7ea66130c7 service nova] [instance: 23a31b1e-4005-442f-96e4-9fe82d9e17ba] Received event network-vif-unplugged-5a8d4829-f681-4e2c-83d7-91e5ea0b1fd2 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:09:42 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-6b97117a-af80-4344-8cff-d7996a1a78cb req-887b3168-4325-41e8-8f27-2f7ea66130c7 service nova] Acquiring lock "23a31b1e-4005-442f-96e4-9fe82d9e17ba-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:09:42 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-6b97117a-af80-4344-8cff-d7996a1a78cb req-887b3168-4325-41e8-8f27-2f7ea66130c7 service nova] Lock "23a31b1e-4005-442f-96e4-9fe82d9e17ba-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:09:42 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-6b97117a-af80-4344-8cff-d7996a1a78cb req-887b3168-4325-41e8-8f27-2f7ea66130c7 service nova] Lock "23a31b1e-4005-442f-96e4-9fe82d9e17ba-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:09:42 user nova-compute[75234]: DEBUG nova.compute.manager [req-6b97117a-af80-4344-8cff-d7996a1a78cb req-887b3168-4325-41e8-8f27-2f7ea66130c7 service nova] [instance: 23a31b1e-4005-442f-96e4-9fe82d9e17ba] No waiting events found dispatching network-vif-unplugged-5a8d4829-f681-4e2c-83d7-91e5ea0b1fd2 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:09:42 user nova-compute[75234]: DEBUG nova.compute.manager [req-6b97117a-af80-4344-8cff-d7996a1a78cb req-887b3168-4325-41e8-8f27-2f7ea66130c7 service nova] [instance: 23a31b1e-4005-442f-96e4-9fe82d9e17ba] Received event network-vif-unplugged-5a8d4829-f681-4e2c-83d7-91e5ea0b1fd2 for instance with task_state deleting. {{(pid=75234) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10890}}
Nov 29 03:09:42 user nova-compute[75234]: INFO nova.virt.libvirt.driver [-] [instance: 23a31b1e-4005-442f-96e4-9fe82d9e17ba] Instance destroyed successfully.
Nov 29 03:09:42 user nova-compute[75234]: DEBUG nova.objects.instance [None req-1806247b-6283-409e-9863-9bb966ef4e2a tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] Lazy-loading 'resources' on Instance uuid 23a31b1e-4005-442f-96e4-9fe82d9e17ba {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:09:42 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-1806247b-6283-409e-9863-9bb966ef4e2a tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-11-29T03:07:51Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-ServersNegativeTestJSON-server-1398606202',display_name='tempest-ServersNegativeTestJSON-server-1398606202',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-serversnegativetestjson-server-1398606202',id=13,image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data=None,key_name=None,keypairs=<?>,launch_index=0,launched_at=2023-11-29T03:08:00Z,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='b877668e02eb441c85dc2a7003a94c7a',ramdisk_id='',reservation_id='r-zt0xoplh',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-ServersNegativeTestJSON-20765276',owner_user_name='tempest-ServersNegativeTestJSON-20765276-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-11-29T03:08:00Z,user_data=None,user_id='178bfe0ea37e45d8abd2d6b45bd5660c',uuid=23a31b1e-4005-442f-96e4-9fe82d9e17ba,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "5a8d4829-f681-4e2c-83d7-91e5ea0b1fd2", "address": "fa:16:3e:1b:04:c4", "network": {"id": "b27ac31e-3601-4055-be79-34b0946b0913", "bridge": "br-int", "label": "tempest-ServersNegativeTestJSON-1491620118-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "b877668e02eb441c85dc2a7003a94c7a", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap5a8d4829-f6", "ovs_interfaceid": "5a8d4829-f681-4e2c-83d7-91e5ea0b1fd2", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Nov 29 03:09:42 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-1806247b-6283-409e-9863-9bb966ef4e2a tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] Converting VIF {"id": "5a8d4829-f681-4e2c-83d7-91e5ea0b1fd2", "address": "fa:16:3e:1b:04:c4", "network": {"id": "b27ac31e-3601-4055-be79-34b0946b0913", "bridge": "br-int", "label": "tempest-ServersNegativeTestJSON-1491620118-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "b877668e02eb441c85dc2a7003a94c7a", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap5a8d4829-f6", "ovs_interfaceid": "5a8d4829-f681-4e2c-83d7-91e5ea0b1fd2", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:09:42 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-1806247b-6283-409e-9863-9bb966ef4e2a tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:1b:04:c4,bridge_name='br-int',has_traffic_filtering=True,id=5a8d4829-f681-4e2c-83d7-91e5ea0b1fd2,network=Network(b27ac31e-3601-4055-be79-34b0946b0913),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap5a8d4829-f6') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:09:42 user nova-compute[75234]: DEBUG os_vif [None req-1806247b-6283-409e-9863-9bb966ef4e2a tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] Unplugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:1b:04:c4,bridge_name='br-int',has_traffic_filtering=True,id=5a8d4829-f681-4e2c-83d7-91e5ea0b1fd2,network=Network(b27ac31e-3601-4055-be79-34b0946b0913),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap5a8d4829-f6') {{(pid=75234) unplug /opt/stack/data/venv/lib/python3.10/site-packages/os_vif/__init__.py:109}}
Nov 29 03:09:42 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:09:42 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tap5a8d4829-f6, bridge=br-int, if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:09:42 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:09:42 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:09:42 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:09:42 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:09:42 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbDestroyCommand(_result=None, table=QoS, record=d337d65a-a20d-45a4-89b3-ddbad22fb34f) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:09:42 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:09:42 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:09:42 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:09:42 user nova-compute[75234]: INFO os_vif [None req-1806247b-6283-409e-9863-9bb966ef4e2a tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] Successfully unplugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:1b:04:c4,bridge_name='br-int',has_traffic_filtering=True,id=5a8d4829-f681-4e2c-83d7-91e5ea0b1fd2,network=Network(b27ac31e-3601-4055-be79-34b0946b0913),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap5a8d4829-f6')
Nov 29 03:09:42 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-1806247b-6283-409e-9863-9bb966ef4e2a tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] [instance: 23a31b1e-4005-442f-96e4-9fe82d9e17ba] Deleting instance files /opt/stack/data/nova/instances/23a31b1e-4005-442f-96e4-9fe82d9e17ba_del
Nov 29 03:09:42 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-1806247b-6283-409e-9863-9bb966ef4e2a tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] [instance: 23a31b1e-4005-442f-96e4-9fe82d9e17ba] Deletion of /opt/stack/data/nova/instances/23a31b1e-4005-442f-96e4-9fe82d9e17ba_del complete
Nov 29 03:09:42 user nova-compute[75234]: INFO nova.compute.manager [None req-1806247b-6283-409e-9863-9bb966ef4e2a tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] [instance: 23a31b1e-4005-442f-96e4-9fe82d9e17ba] Took 0.70 seconds to destroy the instance on the hypervisor.
Nov 29 03:09:42 user nova-compute[75234]: DEBUG oslo.service.loopingcall [None req-1806247b-6283-409e-9863-9bb966ef4e2a tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=75234) func /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/loopingcall.py:435}}
Nov 29 03:09:42 user nova-compute[75234]: DEBUG nova.compute.manager [-] [instance: 23a31b1e-4005-442f-96e4-9fe82d9e17ba] Deallocating network for instance {{(pid=75234) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2279}}
Nov 29 03:09:42 user nova-compute[75234]: DEBUG nova.network.neutron [-] [instance: 23a31b1e-4005-442f-96e4-9fe82d9e17ba] deallocate_for_instance() {{(pid=75234) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1802}}
Nov 29 03:09:43 user nova-compute[75234]: DEBUG nova.network.neutron [-] [instance: 23a31b1e-4005-442f-96e4-9fe82d9e17ba] Updating instance_info_cache with network_info: [] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:09:43 user nova-compute[75234]: INFO nova.compute.manager [-] [instance: 23a31b1e-4005-442f-96e4-9fe82d9e17ba] Took 0.43 seconds to deallocate network for instance.
Nov 29 03:09:43 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-1806247b-6283-409e-9863-9bb966ef4e2a tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:09:43 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-1806247b-6283-409e-9863-9bb966ef4e2a tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:09:43 user nova-compute[75234]: DEBUG nova.compute.provider_tree [None req-1806247b-6283-409e-9863-9bb966ef4e2a tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] Inventory has not changed in ProviderTree for provider: 4d291220-7c37-4839-a0e3-b23c66502b74 {{(pid=75234) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Nov 29 03:09:43 user nova-compute[75234]: DEBUG nova.scheduler.client.report [None req-1806247b-6283-409e-9863-9bb966ef4e2a tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] Inventory has not changed for provider 4d291220-7c37-4839-a0e3-b23c66502b74 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=75234) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Nov 29 03:09:43 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-1806247b-6283-409e-9863-9bb966ef4e2a tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.328s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:09:43 user nova-compute[75234]: INFO nova.scheduler.client.report [None req-1806247b-6283-409e-9863-9bb966ef4e2a tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] Deleted allocations for instance 23a31b1e-4005-442f-96e4-9fe82d9e17ba
Nov 29 03:09:43 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-1806247b-6283-409e-9863-9bb966ef4e2a tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] Lock "23a31b1e-4005-442f-96e4-9fe82d9e17ba" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 1.652s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:09:44 user nova-compute[75234]: DEBUG nova.compute.manager [req-690e5a34-6a46-48b6-8e65-4665656cf4e7 req-611c06d4-c02e-4e72-ac95-424d2d555042 service nova] [instance: 23a31b1e-4005-442f-96e4-9fe82d9e17ba] Received event network-vif-plugged-5a8d4829-f681-4e2c-83d7-91e5ea0b1fd2 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:09:44 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-690e5a34-6a46-48b6-8e65-4665656cf4e7 req-611c06d4-c02e-4e72-ac95-424d2d555042 service nova] Acquiring lock "23a31b1e-4005-442f-96e4-9fe82d9e17ba-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:09:44 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-690e5a34-6a46-48b6-8e65-4665656cf4e7 req-611c06d4-c02e-4e72-ac95-424d2d555042 service nova] Lock "23a31b1e-4005-442f-96e4-9fe82d9e17ba-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:09:44 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-690e5a34-6a46-48b6-8e65-4665656cf4e7 req-611c06d4-c02e-4e72-ac95-424d2d555042 service nova] Lock "23a31b1e-4005-442f-96e4-9fe82d9e17ba-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:09:44 user nova-compute[75234]: DEBUG nova.compute.manager [req-690e5a34-6a46-48b6-8e65-4665656cf4e7 req-611c06d4-c02e-4e72-ac95-424d2d555042 service nova] [instance: 23a31b1e-4005-442f-96e4-9fe82d9e17ba] No waiting events found dispatching network-vif-plugged-5a8d4829-f681-4e2c-83d7-91e5ea0b1fd2 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:09:44 user nova-compute[75234]: WARNING nova.compute.manager [req-690e5a34-6a46-48b6-8e65-4665656cf4e7 req-611c06d4-c02e-4e72-ac95-424d2d555042 service nova] [instance: 23a31b1e-4005-442f-96e4-9fe82d9e17ba] Received unexpected event network-vif-plugged-5a8d4829-f681-4e2c-83d7-91e5ea0b1fd2 for instance with vm_state deleted and task_state None.
Nov 29 03:09:44 user nova-compute[75234]: DEBUG nova.compute.manager [req-690e5a34-6a46-48b6-8e65-4665656cf4e7 req-611c06d4-c02e-4e72-ac95-424d2d555042 service nova] [instance: 23a31b1e-4005-442f-96e4-9fe82d9e17ba] Received event network-vif-deleted-5a8d4829-f681-4e2c-83d7-91e5ea0b1fd2 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:09:44 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:09:46 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:09:47 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:09:49 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:09:50 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:09:50 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:09:50 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=75234) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10541}}
Nov 29 03:09:51 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:09:51 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:09:51 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Starting heal instance info cache {{(pid=75234) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9922}}
Nov 29 03:09:51 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:09:51 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Acquiring lock "refresh_cache-f609a7b7-1be2-44b4-9177-7e72a4d4c060" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:09:51 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Acquired lock "refresh_cache-f609a7b7-1be2-44b4-9177-7e72a4d4c060" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:09:51 user nova-compute[75234]: DEBUG nova.network.neutron [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] [instance: f609a7b7-1be2-44b4-9177-7e72a4d4c060] Forcefully refreshing network info cache for instance {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2003}}
Nov 29 03:09:52 user nova-compute[75234]: DEBUG nova.network.neutron [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] [instance: f609a7b7-1be2-44b4-9177-7e72a4d4c060] Updating instance_info_cache with network_info: [{"id": "5543f379-2b3a-4dc2-bfe9-52f66a942257", "address": "fa:16:3e:04:ae:43", "network": {"id": "294c3aac-8c9b-45f0-8c0a-344e2d7d8b7c", "bridge": "br-int", "label": "tempest-AttachVolumeTestJSON-700098935-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.14", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "e393d23e783b4a40986674d782606c69", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap5543f379-2b", "ovs_interfaceid": "5543f379-2b3a-4dc2-bfe9-52f66a942257", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:09:52 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Releasing lock "refresh_cache-f609a7b7-1be2-44b4-9177-7e72a4d4c060" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:09:52 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] [instance: f609a7b7-1be2-44b4-9177-7e72a4d4c060] Updated the network info_cache for instance {{(pid=75234) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9993}}
Nov 29 03:09:52 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._sync_scheduler_instance_info {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:09:52 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:09:52 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:09:53 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-24b81732-961d-4591-ba9d-d1bd04899fa4 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Acquiring lock "6c0e0184-ca3a-48ec-a6e3-a0a1172260d6" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:09:53 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-24b81732-961d-4591-ba9d-d1bd04899fa4 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Lock "6c0e0184-ca3a-48ec-a6e3-a0a1172260d6" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:09:53 user nova-compute[75234]: DEBUG nova.compute.manager [None req-24b81732-961d-4591-ba9d-d1bd04899fa4 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] [instance: 6c0e0184-ca3a-48ec-a6e3-a0a1172260d6] Starting instance... {{(pid=75234) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2422}}
Nov 29 03:09:53 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-24b81732-961d-4591-ba9d-d1bd04899fa4 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:09:53 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-24b81732-961d-4591-ba9d-d1bd04899fa4 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:09:53 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-24b81732-961d-4591-ba9d-d1bd04899fa4 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=75234) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2366}}
Nov 29 03:09:53 user nova-compute[75234]: INFO nova.compute.claims [None req-24b81732-961d-4591-ba9d-d1bd04899fa4 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] [instance: 6c0e0184-ca3a-48ec-a6e3-a0a1172260d6] Claim successful on node user
Nov 29 03:09:53 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-8a8b0aa3-ddc3-41cf-8db9-0f46a39943f7 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] Acquiring lock "8297c264-a213-44ee-be46-5c08eabda9f7" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:09:53 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-8a8b0aa3-ddc3-41cf-8db9-0f46a39943f7 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] Lock "8297c264-a213-44ee-be46-5c08eabda9f7" acquired by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:09:53 user nova-compute[75234]: DEBUG nova.objects.instance [None req-8a8b0aa3-ddc3-41cf-8db9-0f46a39943f7 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] Lazy-loading 'flavor' on Instance uuid 8297c264-a213-44ee-be46-5c08eabda9f7 {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:09:53 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-fd53cdf9-c639-44ea-869c-801a1a7b6505 tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Acquiring lock "a9350a18-14e6-42c6-a150-0a1487f20ad9" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:09:53 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-fd53cdf9-c639-44ea-869c-801a1a7b6505 tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Lock "a9350a18-14e6-42c6-a150-0a1487f20ad9" acquired by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: waited 0.003s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:09:53 user nova-compute[75234]: DEBUG nova.objects.instance [None req-fd53cdf9-c639-44ea-869c-801a1a7b6505 tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Lazy-loading 'flavor' on Instance uuid a9350a18-14e6-42c6-a150-0a1487f20ad9 {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:09:53 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-8a8b0aa3-ddc3-41cf-8db9-0f46a39943f7 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] Lock "8297c264-a213-44ee-be46-5c08eabda9f7" "released" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: held 0.105s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:09:53 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager.update_available_resource {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:09:53 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-fd53cdf9-c639-44ea-869c-801a1a7b6505 tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Lock "a9350a18-14e6-42c6-a150-0a1487f20ad9" "released" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: held 0.069s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:09:53 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:09:53 user nova-compute[75234]: DEBUG nova.compute.provider_tree [None req-24b81732-961d-4591-ba9d-d1bd04899fa4 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Inventory has not changed in ProviderTree for provider: 4d291220-7c37-4839-a0e3-b23c66502b74 {{(pid=75234) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Nov 29 03:09:53 user nova-compute[75234]: DEBUG nova.scheduler.client.report [None req-24b81732-961d-4591-ba9d-d1bd04899fa4 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Inventory has not changed for provider 4d291220-7c37-4839-a0e3-b23c66502b74 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=75234) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Nov 29 03:09:53 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-24b81732-961d-4591-ba9d-d1bd04899fa4 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.451s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:09:53 user nova-compute[75234]: DEBUG nova.compute.manager [None req-24b81732-961d-4591-ba9d-d1bd04899fa4 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] [instance: 6c0e0184-ca3a-48ec-a6e3-a0a1172260d6] Start building networks asynchronously for instance. {{(pid=75234) _build_resources /opt/stack/nova/nova/compute/manager.py:2819}}
Nov 29 03:09:53 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.098s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:09:53 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:09:53 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Auditing locally available compute resources for user (node: user) {{(pid=75234) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:907}}
Nov 29 03:09:53 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-fd53cdf9-c639-44ea-869c-801a1a7b6505 tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Acquiring lock "a9350a18-14e6-42c6-a150-0a1487f20ad9" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:09:53 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-fd53cdf9-c639-44ea-869c-801a1a7b6505 tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Lock "a9350a18-14e6-42c6-a150-0a1487f20ad9" acquired by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:09:53 user nova-compute[75234]: INFO nova.compute.manager [None req-fd53cdf9-c639-44ea-869c-801a1a7b6505 tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] [instance: a9350a18-14e6-42c6-a150-0a1487f20ad9] Attaching volume f4130d3c-b329-4ce1-8b6d-d9e270a43b0b to /dev/vdb
Nov 29 03:09:53 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-8a8b0aa3-ddc3-41cf-8db9-0f46a39943f7 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] Acquiring lock "8297c264-a213-44ee-be46-5c08eabda9f7" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:09:53 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-8a8b0aa3-ddc3-41cf-8db9-0f46a39943f7 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] Lock "8297c264-a213-44ee-be46-5c08eabda9f7" acquired by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:09:53 user nova-compute[75234]: INFO nova.compute.manager [None req-8a8b0aa3-ddc3-41cf-8db9-0f46a39943f7 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] [instance: 8297c264-a213-44ee-be46-5c08eabda9f7] Attaching volume a7ef35bf-017a-4fd9-9182-79c1b2ab4f98 to /dev/vdb
Nov 29 03:09:53 user nova-compute[75234]: DEBUG nova.compute.manager [None req-24b81732-961d-4591-ba9d-d1bd04899fa4 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] [instance: 6c0e0184-ca3a-48ec-a6e3-a0a1172260d6] Allocating IP information in the background. {{(pid=75234) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1972}}
Nov 29 03:09:53 user nova-compute[75234]: DEBUG nova.network.neutron [None req-24b81732-961d-4591-ba9d-d1bd04899fa4 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] [instance: 6c0e0184-ca3a-48ec-a6e3-a0a1172260d6] allocate_for_instance() {{(pid=75234) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1156}}
Nov 29 03:09:53 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-24b81732-961d-4591-ba9d-d1bd04899fa4 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] [instance: 6c0e0184-ca3a-48ec-a6e3-a0a1172260d6] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Nov 29 03:09:53 user nova-compute[75234]: DEBUG nova.compute.manager [None req-24b81732-961d-4591-ba9d-d1bd04899fa4 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] [instance: 6c0e0184-ca3a-48ec-a6e3-a0a1172260d6] Start building block device mappings for instance. {{(pid=75234) _build_resources /opt/stack/nova/nova/compute/manager.py:2854}}
Nov 29 03:09:53 user nova-compute[75234]: DEBUG os_brick.utils [None req-fd53cdf9-c639-44ea-869c-801a1a7b6505 tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:176}}
Nov 29 03:09:53 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:09:53 user nova-compute[75234]: DEBUG os_brick.utils [None req-8a8b0aa3-ddc3-41cf-8db9-0f46a39943f7 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:176}}
Nov 29 03:09:53 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:09:53 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.019s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:09:53 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[662ba7fb-cddd-4808-bfca-507482f1f21e]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:dd1368c5ba2c\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:09:53 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt -v / -n -o SOURCE {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:09:53 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.022s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:09:53 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[734036a9-e5b8-42d7-bc65-4e53685fe2ff]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:dd1368c5ba2c\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:09:53 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt -v / -n -o SOURCE {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:09:53 user nova-compute[75234]: DEBUG nova.policy [None req-24b81732-961d-4591-ba9d-d1bd04899fa4 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': 'd611826f3be947c0947b99c763c26d44', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c1cb3d1f12f242a785a688d59dbe3cd6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=75234) authorize /opt/stack/nova/nova/policy.py:203}}
Nov 29 03:09:53 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt -v / -n -o SOURCE" returned: 0 in 0.020s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:09:53 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[21e2c230-6e50-4bbb-9ab9-7608f3ce0332]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:09:53 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:09:53 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt -v / -n -o SOURCE" returned: 0 in 0.020s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:09:53 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[457f7194-c1e0-4382-b159-870f9668b8b6]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:09:53 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:09:53 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.020s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:09:53 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[b848ab56-43c1-4117-bc44-27d310c05344]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:09:53 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.019s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:09:53 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[097fb2ac-1df5-498b-849a-307f34fd8439]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:09:53 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[7f9e516f-4605-4b59-818a-0dc6603a8b24]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:09:53 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[74f1514a-d30f-42cc-9fa4-818514e7890d]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:09:53 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-8a8b0aa3-ddc3-41cf-8db9-0f46a39943f7 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] Running cmd (subprocess): nvme version {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:09:53 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-8a8b0aa3-ddc3-41cf-8db9-0f46a39943f7 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] 'nvme version' failed. Not Retrying. {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:473}}
Nov 29 03:09:53 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-fd53cdf9-c639-44ea-869c-801a1a7b6505 tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Running cmd (subprocess): nvme version {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:09:53 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-fd53cdf9-c639-44ea-869c-801a1a7b6505 tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] 'nvme version' failed. Not Retrying. {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:473}}
Nov 29 03:09:53 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.nvmeof [None req-8a8b0aa3-ddc3-41cf-8db9-0f46a39943f7 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] nvme not present on system {{(pid=75234) nvme_present /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/nvmeof.py:756}}
Nov 29 03:09:53 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.nvmeof [None req-fd53cdf9-c639-44ea-869c-801a1a7b6505 tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] nvme not present on system {{(pid=75234) nvme_present /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/nvmeof.py:756}}
Nov 29 03:09:53 user nova-compute[75234]: DEBUG nova.compute.manager [None req-24b81732-961d-4591-ba9d-d1bd04899fa4 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] [instance: 6c0e0184-ca3a-48ec-a6e3-a0a1172260d6] Start spawning the instance on the hypervisor. {{(pid=75234) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2628}}
Nov 29 03:09:53 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-24b81732-961d-4591-ba9d-d1bd04899fa4 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] [instance: 6c0e0184-ca3a-48ec-a6e3-a0a1172260d6] Creating instance directory {{(pid=75234) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4727}}
Nov 29 03:09:53 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-24b81732-961d-4591-ba9d-d1bd04899fa4 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] [instance: 6c0e0184-ca3a-48ec-a6e3-a0a1172260d6] Creating image(s)
Nov 29 03:09:53 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-24b81732-961d-4591-ba9d-d1bd04899fa4 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Acquiring lock "/opt/stack/data/nova/instances/6c0e0184-ca3a-48ec-a6e3-a0a1172260d6/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:09:53 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-24b81732-961d-4591-ba9d-d1bd04899fa4 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Lock "/opt/stack/data/nova/instances/6c0e0184-ca3a-48ec-a6e3-a0a1172260d6/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:09:53 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-24b81732-961d-4591-ba9d-d1bd04899fa4 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Lock "/opt/stack/data/nova/instances/6c0e0184-ca3a-48ec-a6e3-a0a1172260d6/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.002s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:09:53 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:09:53 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:09:53 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-24b81732-961d-4591-ba9d-d1bd04899fa4 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:09:53 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:473}}
Nov 29 03:09:53 user nova-compute[75234]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Nov 29 03:09:53 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[039f2d29-e103-4e0b-8053-bbd46b3dd3a6]: (4, '') {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:09:53 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:473}}
Nov 29 03:09:53 user nova-compute[75234]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Nov 29 03:09:53 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[f896a9a6-c68f-4cab-b18b-93e9f5ad3d9b]: (4, '') {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:09:53 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.lightos [None req-fd53cdf9-c639-44ea-869c-801a1a7b6505 tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=75234) find_dsc /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/lightos.py:98}}
Nov 29 03:09:53 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.lightos [None req-fd53cdf9-c639-44ea-869c-801a1a7b6505 tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=75234) get_connector_properties /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/lightos.py:76}}
Nov 29 03:09:53 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.lightos [None req-fd53cdf9-c639-44ea-869c-801a1a7b6505 tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] LIGHTOS: no hostnqn found. {{(pid=75234) get_connector_properties /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/lightos.py:84}}
Nov 29 03:09:53 user nova-compute[75234]: DEBUG os_brick.utils [None req-fd53cdf9-c639-44ea-869c-801a1a7b6505 tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] <== get_connector_properties: return (239ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:dd1368c5ba2c', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:203}}
Nov 29 03:09:53 user nova-compute[75234]: DEBUG nova.virt.block_device [None req-fd53cdf9-c639-44ea-869c-801a1a7b6505 tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] [instance: a9350a18-14e6-42c6-a150-0a1487f20ad9] Updating existing volume attachment record: c0be8671-b64f-4464-a1f8-795546f89c90 {{(pid=75234) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Nov 29 03:09:53 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.lightos [None req-8a8b0aa3-ddc3-41cf-8db9-0f46a39943f7 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=75234) find_dsc /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/lightos.py:98}}
Nov 29 03:09:53 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.lightos [None req-8a8b0aa3-ddc3-41cf-8db9-0f46a39943f7 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=75234) get_connector_properties /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/lightos.py:76}}
Nov 29 03:09:53 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.lightos [None req-8a8b0aa3-ddc3-41cf-8db9-0f46a39943f7 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] LIGHTOS: no hostnqn found. {{(pid=75234) get_connector_properties /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/lightos.py:84}}
Nov 29 03:09:53 user nova-compute[75234]: DEBUG os_brick.utils [None req-8a8b0aa3-ddc3-41cf-8db9-0f46a39943f7 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] <== get_connector_properties: return (242ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:dd1368c5ba2c', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:203}}
Nov 29 03:09:53 user nova-compute[75234]: DEBUG nova.virt.block_device [None req-8a8b0aa3-ddc3-41cf-8db9-0f46a39943f7 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] [instance: 8297c264-a213-44ee-be46-5c08eabda9f7] Updating existing volume attachment record: eb51d5aa-7828-4fae-beb6-6698a6b605ab {{(pid=75234) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Nov 29 03:09:53 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a9350a18-14e6-42c6-a150-0a1487f20ad9/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:09:54 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-24b81732-961d-4591-ba9d-d1bd04899fa4 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json" returned: 0 in 0.172s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:09:54 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-24b81732-961d-4591-ba9d-d1bd04899fa4 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Acquiring lock "80686e29dc9db6600d03e85cb62af298e12d7918" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:09:54 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-24b81732-961d-4591-ba9d-d1bd04899fa4 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Lock "80686e29dc9db6600d03e85cb62af298e12d7918" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.002s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:09:54 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-24b81732-961d-4591-ba9d-d1bd04899fa4 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:09:54 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a9350a18-14e6-42c6-a150-0a1487f20ad9/disk --force-share --output=json" returned: 0 in 0.186s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:09:54 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a9350a18-14e6-42c6-a150-0a1487f20ad9/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:09:54 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-24b81732-961d-4591-ba9d-d1bd04899fa4 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json" returned: 0 in 0.224s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:09:54 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-24b81732-961d-4591-ba9d-d1bd04899fa4 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918,backing_fmt=raw /opt/stack/data/nova/instances/6c0e0184-ca3a-48ec-a6e3-a0a1172260d6/disk 1073741824 {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:09:54 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a9350a18-14e6-42c6-a150-0a1487f20ad9/disk --force-share --output=json" returned: 0 in 0.193s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:09:54 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-24b81732-961d-4591-ba9d-d1bd04899fa4 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918,backing_fmt=raw /opt/stack/data/nova/instances/6c0e0184-ca3a-48ec-a6e3-a0a1172260d6/disk 1073741824" returned: 0 in 0.050s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:09:54 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-24b81732-961d-4591-ba9d-d1bd04899fa4 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Lock "80686e29dc9db6600d03e85cb62af298e12d7918" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.281s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:09:54 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-24b81732-961d-4591-ba9d-d1bd04899fa4 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:09:54 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/8297c264-a213-44ee-be46-5c08eabda9f7/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:09:54 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-24b81732-961d-4591-ba9d-d1bd04899fa4 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json" returned: 0 in 0.220s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:09:54 user nova-compute[75234]: DEBUG nova.virt.disk.api [None req-24b81732-961d-4591-ba9d-d1bd04899fa4 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Checking if we can resize image /opt/stack/data/nova/instances/6c0e0184-ca3a-48ec-a6e3-a0a1172260d6/disk. size=1073741824 {{(pid=75234) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:166}}
Nov 29 03:09:54 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-24b81732-961d-4591-ba9d-d1bd04899fa4 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6c0e0184-ca3a-48ec-a6e3-a0a1172260d6/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:09:54 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/8297c264-a213-44ee-be46-5c08eabda9f7/disk --force-share --output=json" returned: 0 in 0.220s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:09:54 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/8297c264-a213-44ee-be46-5c08eabda9f7/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:09:54 user nova-compute[75234]: DEBUG nova.network.neutron [None req-24b81732-961d-4591-ba9d-d1bd04899fa4 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] [instance: 6c0e0184-ca3a-48ec-a6e3-a0a1172260d6] Successfully created port: 50593025-edf1-44ec-b5f0-c92d6a807c23 {{(pid=75234) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:548}}
Nov 29 03:09:54 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-24b81732-961d-4591-ba9d-d1bd04899fa4 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6c0e0184-ca3a-48ec-a6e3-a0a1172260d6/disk --force-share --output=json" returned: 0 in 0.173s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:09:54 user nova-compute[75234]: DEBUG nova.virt.disk.api [None req-24b81732-961d-4591-ba9d-d1bd04899fa4 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Cannot resize image /opt/stack/data/nova/instances/6c0e0184-ca3a-48ec-a6e3-a0a1172260d6/disk to a smaller size. {{(pid=75234) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:172}}
Nov 29 03:09:54 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-24b81732-961d-4591-ba9d-d1bd04899fa4 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] [instance: 6c0e0184-ca3a-48ec-a6e3-a0a1172260d6] Created local disks {{(pid=75234) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4861}}
Nov 29 03:09:54 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-24b81732-961d-4591-ba9d-d1bd04899fa4 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] [instance: 6c0e0184-ca3a-48ec-a6e3-a0a1172260d6] Ensure instance console log exists: /opt/stack/data/nova/instances/6c0e0184-ca3a-48ec-a6e3-a0a1172260d6/console.log {{(pid=75234) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4613}}
Nov 29 03:09:54 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-24b81732-961d-4591-ba9d-d1bd04899fa4 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:09:54 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-24b81732-961d-4591-ba9d-d1bd04899fa4 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:09:54 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-24b81732-961d-4591-ba9d-d1bd04899fa4 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:09:54 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/8297c264-a213-44ee-be46-5c08eabda9f7/disk --force-share --output=json" returned: 0 in 0.170s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:09:54 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/ac620533-8274-4674-b241-f48526bed3c3/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:09:54 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/ac620533-8274-4674-b241-f48526bed3c3/disk --force-share --output=json" returned: 0 in 0.170s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:09:54 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/ac620533-8274-4674-b241-f48526bed3c3/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:09:55 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/ac620533-8274-4674-b241-f48526bed3c3/disk --force-share --output=json" returned: 0 in 0.168s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:09:55 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2bf9fe41-63c2-4882-bcdd-8e314e734ffe/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:09:55 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2bf9fe41-63c2-4882-bcdd-8e314e734ffe/disk --force-share --output=json" returned: 0 in 0.193s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:09:55 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2bf9fe41-63c2-4882-bcdd-8e314e734ffe/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:09:55 user nova-compute[75234]: DEBUG nova.network.neutron [None req-24b81732-961d-4591-ba9d-d1bd04899fa4 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] [instance: 6c0e0184-ca3a-48ec-a6e3-a0a1172260d6] Successfully updated port: 50593025-edf1-44ec-b5f0-c92d6a807c23 {{(pid=75234) _update_port /opt/stack/nova/nova/network/neutron.py:586}}
Nov 29 03:09:55 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-24b81732-961d-4591-ba9d-d1bd04899fa4 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Acquiring lock "refresh_cache-6c0e0184-ca3a-48ec-a6e3-a0a1172260d6" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:09:55 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-24b81732-961d-4591-ba9d-d1bd04899fa4 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Acquired lock "refresh_cache-6c0e0184-ca3a-48ec-a6e3-a0a1172260d6" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:09:55 user nova-compute[75234]: DEBUG nova.network.neutron [None req-24b81732-961d-4591-ba9d-d1bd04899fa4 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] [instance: 6c0e0184-ca3a-48ec-a6e3-a0a1172260d6] Building network info cache for instance {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2009}}
Nov 29 03:09:55 user nova-compute[75234]: DEBUG nova.compute.manager [req-d084dabe-c120-4658-ac01-afe1e7adc9c4 req-6acdb4e3-efe8-4895-b322-c7f63f057286 service nova] [instance: 6c0e0184-ca3a-48ec-a6e3-a0a1172260d6] Received event network-changed-50593025-edf1-44ec-b5f0-c92d6a807c23 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:09:55 user nova-compute[75234]: DEBUG nova.compute.manager [req-d084dabe-c120-4658-ac01-afe1e7adc9c4 req-6acdb4e3-efe8-4895-b322-c7f63f057286 service nova] [instance: 6c0e0184-ca3a-48ec-a6e3-a0a1172260d6] Refreshing instance network info cache due to event network-changed-50593025-edf1-44ec-b5f0-c92d6a807c23. {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11117}}
Nov 29 03:09:55 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-d084dabe-c120-4658-ac01-afe1e7adc9c4 req-6acdb4e3-efe8-4895-b322-c7f63f057286 service nova] Acquiring lock "refresh_cache-6c0e0184-ca3a-48ec-a6e3-a0a1172260d6" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:09:55 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2bf9fe41-63c2-4882-bcdd-8e314e734ffe/disk --force-share --output=json" returned: 0 in 0.202s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:09:55 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/21e9c6cd-c39f-41c0-8768-5dd64589c3f3/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:09:55 user nova-compute[75234]: DEBUG nova.network.neutron [None req-24b81732-961d-4591-ba9d-d1bd04899fa4 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] [instance: 6c0e0184-ca3a-48ec-a6e3-a0a1172260d6] Instance cache missing network info. {{(pid=75234) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3322}}
Nov 29 03:09:55 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/21e9c6cd-c39f-41c0-8768-5dd64589c3f3/disk --force-share --output=json" returned: 0 in 0.191s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:09:55 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/21e9c6cd-c39f-41c0-8768-5dd64589c3f3/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:09:55 user nova-compute[75234]: DEBUG nova.network.neutron [None req-24b81732-961d-4591-ba9d-d1bd04899fa4 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] [instance: 6c0e0184-ca3a-48ec-a6e3-a0a1172260d6] Updating instance_info_cache with network_info: [{"id": "50593025-edf1-44ec-b5f0-c92d6a807c23", "address": "fa:16:3e:d8:a2:e7", "network": {"id": "1f946e69-dbec-429c-83f8-a186bad4f2cc", "bridge": "br-int", "label": "tempest-ServerActionsTestOtherA-486369423-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "c1cb3d1f12f242a785a688d59dbe3cd6", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap50593025-ed", "ovs_interfaceid": "50593025-edf1-44ec-b5f0-c92d6a807c23", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:09:55 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-24b81732-961d-4591-ba9d-d1bd04899fa4 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Releasing lock "refresh_cache-6c0e0184-ca3a-48ec-a6e3-a0a1172260d6" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:09:55 user nova-compute[75234]: DEBUG nova.compute.manager [None req-24b81732-961d-4591-ba9d-d1bd04899fa4 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] [instance: 6c0e0184-ca3a-48ec-a6e3-a0a1172260d6] Instance network_info: |[{"id": "50593025-edf1-44ec-b5f0-c92d6a807c23", "address": "fa:16:3e:d8:a2:e7", "network": {"id": "1f946e69-dbec-429c-83f8-a186bad4f2cc", "bridge": "br-int", "label": "tempest-ServerActionsTestOtherA-486369423-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "c1cb3d1f12f242a785a688d59dbe3cd6", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap50593025-ed", "ovs_interfaceid": "50593025-edf1-44ec-b5f0-c92d6a807c23", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=75234) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1987}}
Nov 29 03:09:55 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-d084dabe-c120-4658-ac01-afe1e7adc9c4 req-6acdb4e3-efe8-4895-b322-c7f63f057286 service nova] Acquired lock "refresh_cache-6c0e0184-ca3a-48ec-a6e3-a0a1172260d6" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:09:55 user nova-compute[75234]: DEBUG nova.network.neutron [req-d084dabe-c120-4658-ac01-afe1e7adc9c4 req-6acdb4e3-efe8-4895-b322-c7f63f057286 service nova] [instance: 6c0e0184-ca3a-48ec-a6e3-a0a1172260d6] Refreshing network info cache for port 50593025-edf1-44ec-b5f0-c92d6a807c23 {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2006}}
Nov 29 03:09:55 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-24b81732-961d-4591-ba9d-d1bd04899fa4 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] [instance: 6c0e0184-ca3a-48ec-a6e3-a0a1172260d6] Start _get_guest_xml network_info=[{"id": "50593025-edf1-44ec-b5f0-c92d6a807c23", "address": "fa:16:3e:d8:a2:e7", "network": {"id": "1f946e69-dbec-429c-83f8-a186bad4f2cc", "bridge": "br-int", "label": "tempest-ServerActionsTestOtherA-486369423-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "c1cb3d1f12f242a785a688d59dbe3cd6", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap50593025-ed", "ovs_interfaceid": "50593025-edf1-44ec-b5f0-c92d6a807c23", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum='c8fc807773e5354afe61636071771906',container_format='bare',created_at=2023-11-29T03:02:15Z,direct_url=<?>,disk_format='qcow2',id=96c30ba7-9b73-4a30-8ba4-7138879573b7,min_disk=0,min_ram=0,name='cirros-0.6.2-x86_64-disk',owner='b18e380921c748b78517b6f1211cd0f1',properties=ImageMetaProps,protected=<?>,size=21430272,status='active',tags=<?>,updated_at=2023-11-29T03:02:17Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [{'device_name': '/dev/vda', 'encryption_format': None, 'encrypted': False, 'device_type': 'disk', 'size': 0, 'disk_bus': 'virtio', 'boot_index': 0, 'encryption_options': None, 'encryption_secret_uuid': None, 'guest_format': None, 'image_id': '96c30ba7-9b73-4a30-8ba4-7138879573b7'}], 'ephemerals': [], 'block_device_mapping': [], 'swap': None} {{(pid=75234) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7562}}
Nov 29 03:09:56 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-24b81732-961d-4591-ba9d-d1bd04899fa4 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:09:56 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-24b81732-961d-4591-ba9d-d1bd04899fa4 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:09:56 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-24b81732-961d-4591-ba9d-d1bd04899fa4 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Searching host: 'user' for CPU controller through CGroups V1... {{(pid=75234) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1652}}
Nov 29 03:09:56 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-24b81732-961d-4591-ba9d-d1bd04899fa4 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] CPU controller missing on host. {{(pid=75234) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1662}}
Nov 29 03:09:56 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/21e9c6cd-c39f-41c0-8768-5dd64589c3f3/disk --force-share --output=json" returned: 0 in 0.237s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:09:56 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-24b81732-961d-4591-ba9d-d1bd04899fa4 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Searching host: 'user' for CPU controller through CGroups V2... {{(pid=75234) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1671}}
Nov 29 03:09:56 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-24b81732-961d-4591-ba9d-d1bd04899fa4 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] CPU controller found on host. {{(pid=75234) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1678}}
Nov 29 03:09:56 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-24b81732-961d-4591-ba9d-d1bd04899fa4 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=75234) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5400}}
Nov 29 03:09:56 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-24b81732-961d-4591-ba9d-d1bd04899fa4 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-11-29T03:02:04Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='1',id=6,is_public=True,memory_mb=512,name='m1.tiny',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='c8fc807773e5354afe61636071771906',container_format='bare',created_at=2023-11-29T03:02:15Z,direct_url=<?>,disk_format='qcow2',id=96c30ba7-9b73-4a30-8ba4-7138879573b7,min_disk=0,min_ram=0,name='cirros-0.6.2-x86_64-disk',owner='b18e380921c748b78517b6f1211cd0f1',properties=ImageMetaProps,protected=<?>,size=21430272,status='active',tags=<?>,updated_at=2023-11-29T03:02:17Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:563}}
Nov 29 03:09:56 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-24b81732-961d-4591-ba9d-d1bd04899fa4 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Flavor limits 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:348}}
Nov 29 03:09:56 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-24b81732-961d-4591-ba9d-d1bd04899fa4 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Image limits 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:352}}
Nov 29 03:09:56 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-24b81732-961d-4591-ba9d-d1bd04899fa4 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Flavor pref 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:388}}
Nov 29 03:09:56 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-24b81732-961d-4591-ba9d-d1bd04899fa4 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Image pref 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:392}}
Nov 29 03:09:56 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-24b81732-961d-4591-ba9d-d1bd04899fa4 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:430}}
Nov 29 03:09:56 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-24b81732-961d-4591-ba9d-d1bd04899fa4 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:569}}
Nov 29 03:09:56 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-24b81732-961d-4591-ba9d-d1bd04899fa4 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=75234) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:471}}
Nov 29 03:09:56 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-24b81732-961d-4591-ba9d-d1bd04899fa4 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Got 1 possible topologies {{(pid=75234) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:501}}
Nov 29 03:09:56 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-24b81732-961d-4591-ba9d-d1bd04899fa4 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:575}}
Nov 29 03:09:56 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-24b81732-961d-4591-ba9d-d1bd04899fa4 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:577}}
Nov 29 03:09:56 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-24b81732-961d-4591-ba9d-d1bd04899fa4 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-11-29T03:09:52Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-tempest.common.compute-instance-176407456',display_name='tempest-tempest.common.compute-instance-176407456',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-tempest-common-compute-instance-176407456',id=16,image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBFVgvqBN7/vaAFxNi0MSZxI04yEwk9FFUfhV/l7Fr5QumrLMJI6JbWuh0vhY2F6+RLBsF4qtCGuui27rDdawR2ALy+mo/mA7ZKd/Mr3qfGMSgFbF3GghJD1rMovi6Hk9rQ==',key_name='tempest-keypair-382073294',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='c1cb3d1f12f242a785a688d59dbe3cd6',ramdisk_id='',reservation_id='r-7poirwds',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-ServerActionsTestOtherA-1495067208',owner_user_name='tempest-ServerActionsTestOtherA-1495067208-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-11-29T03:09:54Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='d611826f3be947c0947b99c763c26d44',uuid=6c0e0184-ca3a-48ec-a6e3-a0a1172260d6,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "50593025-edf1-44ec-b5f0-c92d6a807c23", "address": "fa:16:3e:d8:a2:e7", "network": {"id": "1f946e69-dbec-429c-83f8-a186bad4f2cc", "bridge": "br-int", "label": "tempest-ServerActionsTestOtherA-486369423-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "c1cb3d1f12f242a785a688d59dbe3cd6", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap50593025-ed", "ovs_interfaceid": "50593025-edf1-44ec-b5f0-c92d6a807c23", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=75234) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Nov 29 03:09:56 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-24b81732-961d-4591-ba9d-d1bd04899fa4 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Converting VIF {"id": "50593025-edf1-44ec-b5f0-c92d6a807c23", "address": "fa:16:3e:d8:a2:e7", "network": {"id": "1f946e69-dbec-429c-83f8-a186bad4f2cc", "bridge": "br-int", "label": "tempest-ServerActionsTestOtherA-486369423-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "c1cb3d1f12f242a785a688d59dbe3cd6", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap50593025-ed", "ovs_interfaceid": "50593025-edf1-44ec-b5f0-c92d6a807c23", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:09:56 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-24b81732-961d-4591-ba9d-d1bd04899fa4 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:d8:a2:e7,bridge_name='br-int',has_traffic_filtering=True,id=50593025-edf1-44ec-b5f0-c92d6a807c23,network=Network(1f946e69-dbec-429c-83f8-a186bad4f2cc),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap50593025-ed') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:09:56 user nova-compute[75234]: DEBUG nova.objects.instance [None req-24b81732-961d-4591-ba9d-d1bd04899fa4 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Lazy-loading 'pci_devices' on Instance uuid 6c0e0184-ca3a-48ec-a6e3-a0a1172260d6 {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:09:56 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-24b81732-961d-4591-ba9d-d1bd04899fa4 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] [instance: 6c0e0184-ca3a-48ec-a6e3-a0a1172260d6] End _get_guest_xml xml=<domain type="kvm">
Nov 29 03:09:56 user nova-compute[75234]:   <uuid>6c0e0184-ca3a-48ec-a6e3-a0a1172260d6</uuid>
Nov 29 03:09:56 user nova-compute[75234]:   <name>instance-00000010</name>
Nov 29 03:09:56 user nova-compute[75234]:   <memory>524288</memory>
Nov 29 03:09:56 user nova-compute[75234]:   <vcpu>1</vcpu>
Nov 29 03:09:56 user nova-compute[75234]:   <metadata>
Nov 29 03:09:56 user nova-compute[75234]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Nov 29 03:09:56 user nova-compute[75234]:       <nova:package version="0.0.0"/>
Nov 29 03:09:56 user nova-compute[75234]:       <nova:name>tempest-tempest.common.compute-instance-176407456</nova:name>
Nov 29 03:09:56 user nova-compute[75234]:       <nova:creationTime>2023-11-29 03:09:56</nova:creationTime>
Nov 29 03:09:56 user nova-compute[75234]:       <nova:flavor name="m1.tiny">
Nov 29 03:09:56 user nova-compute[75234]:         <nova:memory>512</nova:memory>
Nov 29 03:09:56 user nova-compute[75234]:         <nova:disk>1</nova:disk>
Nov 29 03:09:56 user nova-compute[75234]:         <nova:swap>0</nova:swap>
Nov 29 03:09:56 user nova-compute[75234]:         <nova:ephemeral>0</nova:ephemeral>
Nov 29 03:09:56 user nova-compute[75234]:         <nova:vcpus>1</nova:vcpus>
Nov 29 03:09:56 user nova-compute[75234]:       </nova:flavor>
Nov 29 03:09:56 user nova-compute[75234]:       <nova:owner>
Nov 29 03:09:56 user nova-compute[75234]:         <nova:user uuid="d611826f3be947c0947b99c763c26d44">tempest-ServerActionsTestOtherA-1495067208-project-member</nova:user>
Nov 29 03:09:56 user nova-compute[75234]:         <nova:project uuid="c1cb3d1f12f242a785a688d59dbe3cd6">tempest-ServerActionsTestOtherA-1495067208</nova:project>
Nov 29 03:09:56 user nova-compute[75234]:       </nova:owner>
Nov 29 03:09:56 user nova-compute[75234]:       <nova:root type="image" uuid="96c30ba7-9b73-4a30-8ba4-7138879573b7"/>
Nov 29 03:09:56 user nova-compute[75234]:       <nova:ports>
Nov 29 03:09:56 user nova-compute[75234]:         <nova:port uuid="50593025-edf1-44ec-b5f0-c92d6a807c23">
Nov 29 03:09:56 user nova-compute[75234]:           <nova:ip type="fixed" address="10.0.0.14" ipVersion="4"/>
Nov 29 03:09:56 user nova-compute[75234]:         </nova:port>
Nov 29 03:09:56 user nova-compute[75234]:       </nova:ports>
Nov 29 03:09:56 user nova-compute[75234]:     </nova:instance>
Nov 29 03:09:56 user nova-compute[75234]:   </metadata>
Nov 29 03:09:56 user nova-compute[75234]:   <sysinfo type="smbios">
Nov 29 03:09:56 user nova-compute[75234]:     <system>
Nov 29 03:09:56 user nova-compute[75234]:       <entry name="manufacturer">OpenStack Foundation</entry>
Nov 29 03:09:56 user nova-compute[75234]:       <entry name="product">OpenStack Nova</entry>
Nov 29 03:09:56 user nova-compute[75234]:       <entry name="version">0.0.0</entry>
Nov 29 03:09:56 user nova-compute[75234]:       <entry name="serial">6c0e0184-ca3a-48ec-a6e3-a0a1172260d6</entry>
Nov 29 03:09:56 user nova-compute[75234]:       <entry name="uuid">6c0e0184-ca3a-48ec-a6e3-a0a1172260d6</entry>
Nov 29 03:09:56 user nova-compute[75234]:       <entry name="family">Virtual Machine</entry>
Nov 29 03:09:56 user nova-compute[75234]:     </system>
Nov 29 03:09:56 user nova-compute[75234]:   </sysinfo>
Nov 29 03:09:56 user nova-compute[75234]:   <os>
Nov 29 03:09:56 user nova-compute[75234]:     <type arch="x86_64" machine="pc">hvm</type>
Nov 29 03:09:56 user nova-compute[75234]:     <boot dev="hd"/>
Nov 29 03:09:56 user nova-compute[75234]:     <smbios mode="sysinfo"/>
Nov 29 03:09:56 user nova-compute[75234]:   </os>
Nov 29 03:09:56 user nova-compute[75234]:   <features>
Nov 29 03:09:56 user nova-compute[75234]:     <acpi/>
Nov 29 03:09:56 user nova-compute[75234]:     <apic/>
Nov 29 03:09:56 user nova-compute[75234]:     <vmcoreinfo/>
Nov 29 03:09:56 user nova-compute[75234]:   </features>
Nov 29 03:09:56 user nova-compute[75234]:   <clock offset="utc">
Nov 29 03:09:56 user nova-compute[75234]:     <timer name="pit" tickpolicy="delay"/>
Nov 29 03:09:56 user nova-compute[75234]:     <timer name="rtc" tickpolicy="catchup"/>
Nov 29 03:09:56 user nova-compute[75234]:     <timer name="hpet" present="no"/>
Nov 29 03:09:56 user nova-compute[75234]:   </clock>
Nov 29 03:09:56 user nova-compute[75234]:   <cpu mode="custom" match="exact">
Nov 29 03:09:56 user nova-compute[75234]:     <model>Nehalem</model>
Nov 29 03:09:56 user nova-compute[75234]:     <topology sockets="1" cores="1" threads="1"/>
Nov 29 03:09:56 user nova-compute[75234]:   </cpu>
Nov 29 03:09:56 user nova-compute[75234]:   <devices>
Nov 29 03:09:56 user nova-compute[75234]:     <disk type="file" device="disk">
Nov 29 03:09:56 user nova-compute[75234]:       <driver name="qemu" type="qcow2" cache="none"/>
Nov 29 03:09:56 user nova-compute[75234]:       <source file="/opt/stack/data/nova/instances/6c0e0184-ca3a-48ec-a6e3-a0a1172260d6/disk"/>
Nov 29 03:09:56 user nova-compute[75234]:       <target dev="vda" bus="virtio"/>
Nov 29 03:09:56 user nova-compute[75234]:     </disk>
Nov 29 03:09:56 user nova-compute[75234]:     <interface type="ethernet">
Nov 29 03:09:56 user nova-compute[75234]:       <mac address="fa:16:3e:d8:a2:e7"/>
Nov 29 03:09:56 user nova-compute[75234]:       <model type="virtio"/>
Nov 29 03:09:56 user nova-compute[75234]:       <mtu size="1442"/>
Nov 29 03:09:56 user nova-compute[75234]:       <target dev="tap50593025-ed"/>
Nov 29 03:09:56 user nova-compute[75234]:     </interface>
Nov 29 03:09:56 user nova-compute[75234]:     <serial type="pty">
Nov 29 03:09:56 user nova-compute[75234]:       <log file="/opt/stack/data/nova/instances/6c0e0184-ca3a-48ec-a6e3-a0a1172260d6/console.log" append="off"/>
Nov 29 03:09:56 user nova-compute[75234]:     </serial>
Nov 29 03:09:56 user nova-compute[75234]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Nov 29 03:09:56 user nova-compute[75234]:     <video>
Nov 29 03:09:56 user nova-compute[75234]:       <model type="virtio"/>
Nov 29 03:09:56 user nova-compute[75234]:     </video>
Nov 29 03:09:56 user nova-compute[75234]:     <rng model="virtio">
Nov 29 03:09:56 user nova-compute[75234]:       <backend model="random">/dev/urandom</backend>
Nov 29 03:09:56 user nova-compute[75234]:     </rng>
Nov 29 03:09:56 user nova-compute[75234]:     <controller type="usb" index="0" model="none"/>
Nov 29 03:09:56 user nova-compute[75234]:     <memballoon model="virtio">
Nov 29 03:09:56 user nova-compute[75234]:       <stats period="10"/>
Nov 29 03:09:56 user nova-compute[75234]:     </memballoon>
Nov 29 03:09:56 user nova-compute[75234]:   </devices>
Nov 29 03:09:56 user nova-compute[75234]: </domain>
Nov 29 03:09:56 user nova-compute[75234]:  {{(pid=75234) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7568}}
Nov 29 03:09:56 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-24b81732-961d-4591-ba9d-d1bd04899fa4 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-11-29T03:09:52Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-tempest.common.compute-instance-176407456',display_name='tempest-tempest.common.compute-instance-176407456',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-tempest-common-compute-instance-176407456',id=16,image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBFVgvqBN7/vaAFxNi0MSZxI04yEwk9FFUfhV/l7Fr5QumrLMJI6JbWuh0vhY2F6+RLBsF4qtCGuui27rDdawR2ALy+mo/mA7ZKd/Mr3qfGMSgFbF3GghJD1rMovi6Hk9rQ==',key_name='tempest-keypair-382073294',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='c1cb3d1f12f242a785a688d59dbe3cd6',ramdisk_id='',reservation_id='r-7poirwds',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-ServerActionsTestOtherA-1495067208',owner_user_name='tempest-ServerActionsTestOtherA-1495067208-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-11-29T03:09:54Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='d611826f3be947c0947b99c763c26d44',uuid=6c0e0184-ca3a-48ec-a6e3-a0a1172260d6,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "50593025-edf1-44ec-b5f0-c92d6a807c23", "address": "fa:16:3e:d8:a2:e7", "network": {"id": "1f946e69-dbec-429c-83f8-a186bad4f2cc", "bridge": "br-int", "label": "tempest-ServerActionsTestOtherA-486369423-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "c1cb3d1f12f242a785a688d59dbe3cd6", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap50593025-ed", "ovs_interfaceid": "50593025-edf1-44ec-b5f0-c92d6a807c23", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Nov 29 03:09:56 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-24b81732-961d-4591-ba9d-d1bd04899fa4 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Converting VIF {"id": "50593025-edf1-44ec-b5f0-c92d6a807c23", "address": "fa:16:3e:d8:a2:e7", "network": {"id": "1f946e69-dbec-429c-83f8-a186bad4f2cc", "bridge": "br-int", "label": "tempest-ServerActionsTestOtherA-486369423-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "c1cb3d1f12f242a785a688d59dbe3cd6", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap50593025-ed", "ovs_interfaceid": "50593025-edf1-44ec-b5f0-c92d6a807c23", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:09:56 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-24b81732-961d-4591-ba9d-d1bd04899fa4 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:d8:a2:e7,bridge_name='br-int',has_traffic_filtering=True,id=50593025-edf1-44ec-b5f0-c92d6a807c23,network=Network(1f946e69-dbec-429c-83f8-a186bad4f2cc),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap50593025-ed') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:09:56 user nova-compute[75234]: DEBUG os_vif [None req-24b81732-961d-4591-ba9d-d1bd04899fa4 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:d8:a2:e7,bridge_name='br-int',has_traffic_filtering=True,id=50593025-edf1-44ec-b5f0-c92d6a807c23,network=Network(1f946e69-dbec-429c-83f8-a186bad4f2cc),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap50593025-ed') {{(pid=75234) plug /opt/stack/data/venv/lib/python3.10/site-packages/os_vif/__init__.py:76}}
Nov 29 03:09:56 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:09:56 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:09:56 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Nov 29 03:09:56 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:09:56 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbCreateCommand(_result=None, table=QoS, columns={'type': 'linux-noop', 'external_ids': {'id': '5f332f4a-3db6-5b3a-9153-840969a09f3a', '_type': 'linux-noop'}}, row=False) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:09:56 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f609a7b7-1be2-44b4-9177-7e72a4d4c060/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:09:56 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:09:56 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:09:56 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:09:56 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tap50593025-ed, may_exist=True, interface_attrs={}) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:09:56 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Port, record=tap50593025-ed, col_values=(('qos', UUID('7ad029b0-ed26-4b2c-a8e9-5b9037b94574')),), if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:09:56 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DbSetCommand(_result=None, table=Interface, record=tap50593025-ed, col_values=(('external_ids', {'iface-id': '50593025-edf1-44ec-b5f0-c92d6a807c23', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:d8:a2:e7', 'vm-uuid': '6c0e0184-ca3a-48ec-a6e3-a0a1172260d6'}),), if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:09:56 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:09:56 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:09:56 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:09:56 user nova-compute[75234]: INFO os_vif [None req-24b81732-961d-4591-ba9d-d1bd04899fa4 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:d8:a2:e7,bridge_name='br-int',has_traffic_filtering=True,id=50593025-edf1-44ec-b5f0-c92d6a807c23,network=Network(1f946e69-dbec-429c-83f8-a186bad4f2cc),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap50593025-ed')
Nov 29 03:09:56 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-24b81732-961d-4591-ba9d-d1bd04899fa4 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] No BDM found with device name vda, not building metadata. {{(pid=75234) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12089}}
Nov 29 03:09:56 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-24b81732-961d-4591-ba9d-d1bd04899fa4 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] No VIF found with MAC fa:16:3e:d8:a2:e7, not building metadata {{(pid=75234) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Nov 29 03:09:56 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f609a7b7-1be2-44b4-9177-7e72a4d4c060/disk --force-share --output=json" returned: 0 in 0.202s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:09:56 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f609a7b7-1be2-44b4-9177-7e72a4d4c060/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:09:56 user nova-compute[75234]: DEBUG nova.network.neutron [req-d084dabe-c120-4658-ac01-afe1e7adc9c4 req-6acdb4e3-efe8-4895-b322-c7f63f057286 service nova] [instance: 6c0e0184-ca3a-48ec-a6e3-a0a1172260d6] Updated VIF entry in instance network info cache for port 50593025-edf1-44ec-b5f0-c92d6a807c23. {{(pid=75234) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3481}}
Nov 29 03:09:56 user nova-compute[75234]: DEBUG nova.network.neutron [req-d084dabe-c120-4658-ac01-afe1e7adc9c4 req-6acdb4e3-efe8-4895-b322-c7f63f057286 service nova] [instance: 6c0e0184-ca3a-48ec-a6e3-a0a1172260d6] Updating instance_info_cache with network_info: [{"id": "50593025-edf1-44ec-b5f0-c92d6a807c23", "address": "fa:16:3e:d8:a2:e7", "network": {"id": "1f946e69-dbec-429c-83f8-a186bad4f2cc", "bridge": "br-int", "label": "tempest-ServerActionsTestOtherA-486369423-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "c1cb3d1f12f242a785a688d59dbe3cd6", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap50593025-ed", "ovs_interfaceid": "50593025-edf1-44ec-b5f0-c92d6a807c23", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:09:56 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-d084dabe-c120-4658-ac01-afe1e7adc9c4 req-6acdb4e3-efe8-4895-b322-c7f63f057286 service nova] Releasing lock "refresh_cache-6c0e0184-ca3a-48ec-a6e3-a0a1172260d6" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:09:56 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f609a7b7-1be2-44b4-9177-7e72a4d4c060/disk --force-share --output=json" returned: 0 in 0.177s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:09:56 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] skipping disk /dev/sdh (vdb) as it is a volume {{(pid=75234) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11213}}
Nov 29 03:09:56 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f7d0bdeb-6a05-47ac-87f2-c8380a9a4286/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:09:56 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f7d0bdeb-6a05-47ac-87f2-c8380a9a4286/disk --force-share --output=json" returned: 0 in 0.215s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:09:56 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f7d0bdeb-6a05-47ac-87f2-c8380a9a4286/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:09:56 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f7d0bdeb-6a05-47ac-87f2-c8380a9a4286/disk --force-share --output=json" returned: 0 in 0.225s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:09:56 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] skipping disk /dev/sdb (vdb) as it is a volume {{(pid=75234) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11213}}
Nov 29 03:09:57 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/062753a6-8f39-4e80-a549-9f0f5a4b877a/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:09:57 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/062753a6-8f39-4e80-a549-9f0f5a4b877a/disk --force-share --output=json" returned: 0 in 0.193s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:09:57 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/062753a6-8f39-4e80-a549-9f0f5a4b877a/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:09:57 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/062753a6-8f39-4e80-a549-9f0f5a4b877a/disk --force-share --output=json" returned: 0 in 0.224s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:10:00 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:10:00 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-4badf7dd-4675-4725-b5e9-a99187ebb992 tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] Waiting for libvirt event about the detach of device vdc with device alias virtio-disk2 from instance f7d0bdeb-6a05-47ac-87f2-c8380a9a4286 is timed out.
Nov 29 03:10:00 user nova-compute[75234]: DEBUG nova.compute.manager [None req-24b81732-961d-4591-ba9d-d1bd04899fa4 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] [instance: 6c0e0184-ca3a-48ec-a6e3-a0a1172260d6] Instance event wait completed in 0 seconds for  {{(pid=75234) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Nov 29 03:10:00 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-24b81732-961d-4591-ba9d-d1bd04899fa4 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] [instance: 6c0e0184-ca3a-48ec-a6e3-a0a1172260d6] Guest created on hypervisor {{(pid=75234) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4421}}
Nov 29 03:10:00 user nova-compute[75234]: DEBUG nova.compute.manager [req-4be3e68a-2c09-440b-a228-25218896b0af req-9a1ab70e-76a1-482f-8fd2-7151c78146c5 service nova] [instance: 6c0e0184-ca3a-48ec-a6e3-a0a1172260d6] Received event network-vif-plugged-50593025-edf1-44ec-b5f0-c92d6a807c23 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:10:00 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-4be3e68a-2c09-440b-a228-25218896b0af req-9a1ab70e-76a1-482f-8fd2-7151c78146c5 service nova] Acquiring lock "6c0e0184-ca3a-48ec-a6e3-a0a1172260d6-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:10:00 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-4be3e68a-2c09-440b-a228-25218896b0af req-9a1ab70e-76a1-482f-8fd2-7151c78146c5 service nova] Lock "6c0e0184-ca3a-48ec-a6e3-a0a1172260d6-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:10:00 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-4be3e68a-2c09-440b-a228-25218896b0af req-9a1ab70e-76a1-482f-8fd2-7151c78146c5 service nova] Lock "6c0e0184-ca3a-48ec-a6e3-a0a1172260d6-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:10:00 user nova-compute[75234]: DEBUG nova.compute.manager [req-4be3e68a-2c09-440b-a228-25218896b0af req-9a1ab70e-76a1-482f-8fd2-7151c78146c5 service nova] [instance: 6c0e0184-ca3a-48ec-a6e3-a0a1172260d6] No waiting events found dispatching network-vif-plugged-50593025-edf1-44ec-b5f0-c92d6a807c23 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:10:00 user nova-compute[75234]: WARNING nova.compute.manager [req-4be3e68a-2c09-440b-a228-25218896b0af req-9a1ab70e-76a1-482f-8fd2-7151c78146c5 service nova] [instance: 6c0e0184-ca3a-48ec-a6e3-a0a1172260d6] Received unexpected event network-vif-plugged-50593025-edf1-44ec-b5f0-c92d6a807c23 for instance with vm_state building and task_state spawning.
Nov 29 03:10:00 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-4badf7dd-4675-4725-b5e9-a99187ebb992 tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] Successfully detached device vdc from instance f7d0bdeb-6a05-47ac-87f2-c8380a9a4286 from the live domain config.
Nov 29 03:10:00 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-4badf7dd-4675-4725-b5e9-a99187ebb992 tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] [instance: f7d0bdeb-6a05-47ac-87f2-c8380a9a4286] calling os-brick to detach iSCSI Volume {{(pid=75234) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Nov 29 03:10:00 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-4badf7dd-4675-4725-b5e9-a99187ebb992 tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f8a5050c9a0>, {'target_iqn': 'iqn.2010-10.org.openstack:6e883290-42f6-40fd-854d-c42f9aca8992', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'IAXcliLh', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdc'}, None), 'kwargs': False}}" {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:176}}
Nov 29 03:10:00 user nova-compute[75234]: WARNING os_brick.initiator.connectors.base [None req-4badf7dd-4675-4725-b5e9-a99187ebb992 tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Nov 29 03:10:00 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-4badf7dd-4675-4725-b5e9-a99187ebb992 tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:68}}
Nov 29 03:10:00 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-4badf7dd-4675-4725-b5e9-a99187ebb992 tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.003s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:73}}
Nov 29 03:10:00 user nova-compute[75234]: INFO nova.virt.libvirt.driver [-] [instance: 6c0e0184-ca3a-48ec-a6e3-a0a1172260d6] Instance spawned successfully.
Nov 29 03:10:00 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:10:00 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-24b81732-961d-4591-ba9d-d1bd04899fa4 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] [instance: 6c0e0184-ca3a-48ec-a6e3-a0a1172260d6] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:904}}
Nov 29 03:10:00 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:10:00 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:10:00 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Hypervisor/Node resource view: name=user free_ram=7894MB free_disk=25.581321716308594GB free_vcpus=5 pci_devices=[{"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}] {{(pid=75234) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1106}}
Nov 29 03:10:00 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:10:00 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:10:00 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.021s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:10:00 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[6f381848-9e5a-40fc-a3f8-64ffec4a6978]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:6e883290-42f6-40fd-854d-c42f9aca8992\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:ada91c26-ca97-4b8b-98bb-a310d0627691\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:10:00 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-4badf7dd-4675-4725-b5e9-a99187ebb992 tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS:
Nov 29 03:10:00 user nova-compute[75234]: No targets found.
Nov 29 03:10:00 user nova-compute[75234]: iSNS:
Nov 29 03:10:00 user nova-compute[75234]: No targets found.
Nov 29 03:10:00 user nova-compute[75234]: STATIC:
Nov 29 03:10:00 user nova-compute[75234]: Target: iqn.2010-10.org.openstack:6e883290-42f6-40fd-854d-c42f9aca8992
Nov 29 03:10:00 user nova-compute[75234]:         Portal: 172.16.0.220:3260,-1
Nov 29 03:10:00 user nova-compute[75234]:                 Iface Name: default
Nov 29 03:10:00 user nova-compute[75234]: Target: iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672
Nov 29 03:10:00 user nova-compute[75234]:         Portal: 172.16.0.220:3260,-1
Nov 29 03:10:00 user nova-compute[75234]:                 Iface Name: default
Nov 29 03:10:00 user nova-compute[75234]: Target: iqn.2010-10.org.openstack:ada91c26-ca97-4b8b-98bb-a310d0627691
Nov 29 03:10:00 user nova-compute[75234]:         Portal: 172.16.0.220:3260,-1
Nov 29 03:10:00 user nova-compute[75234]:                 Iface Name: default
Nov 29 03:10:00 user nova-compute[75234]: FIRMWARE:
Nov 29 03:10:00 user nova-compute[75234]: No targets found.
Nov 29 03:10:00 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:10:00 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-4badf7dd-4675-4725-b5e9-a99187ebb992 tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] Regex to get portals from discoverydb: ^SENDTARGETS:
Nov 29 03:10:00 user nova-compute[75234]: .*?^DiscoveryAddress: 172.16.0.220,3260.*?
Nov 29 03:10:00 user nova-compute[75234]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=75234) _get_discoverydb_portals /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:382}}
Nov 29 03:10:00 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-4badf7dd-4675-4725-b5e9-a99187ebb992 tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:6e883290-42f6-40fd-854d-c42f9aca8992', 0)] {{(pid=75234) _get_connection_devices /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:830}}
Nov 29 03:10:00 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:10:00 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-24b81732-961d-4591-ba9d-d1bd04899fa4 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] [instance: 6c0e0184-ca3a-48ec-a6e3-a0a1172260d6] Found default for hw_cdrom_bus of ide {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:10:00 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-24b81732-961d-4591-ba9d-d1bd04899fa4 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] [instance: 6c0e0184-ca3a-48ec-a6e3-a0a1172260d6] Found default for hw_disk_bus of virtio {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:10:00 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-24b81732-961d-4591-ba9d-d1bd04899fa4 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] [instance: 6c0e0184-ca3a-48ec-a6e3-a0a1172260d6] Found default for hw_input_bus of None {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:10:00 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-24b81732-961d-4591-ba9d-d1bd04899fa4 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] [instance: 6c0e0184-ca3a-48ec-a6e3-a0a1172260d6] Found default for hw_pointer_model of None {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:10:00 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-24b81732-961d-4591-ba9d-d1bd04899fa4 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] [instance: 6c0e0184-ca3a-48ec-a6e3-a0a1172260d6] Found default for hw_video_model of virtio {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:10:00 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-24b81732-961d-4591-ba9d-d1bd04899fa4 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] [instance: 6c0e0184-ca3a-48ec-a6e3-a0a1172260d6] Found default for hw_vif_model of virtio {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:10:00 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.025s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:10:00 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[811bbee9-d550-4ecf-bd2d-3f1653e290e3]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:6e883290-42f6-40fd-854d-c42f9aca8992\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:ada91c26-ca97-4b8b-98bb-a310d0627691\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:10:00 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:10:00 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.029s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:10:00 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[263ea1b6-1485-427d-9147-a0df57e02410]: (4, ('tcp: [12] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ada91c26-ca97-4b8b-98bb-a310d0627691 (non-flash)\ntcp: [13] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e883290-42f6-40fd-854d-c42f9aca8992 (non-flash)\ntcp: [9] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672 (non-flash)\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:10:00 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-4badf7dd-4675-4725-b5e9-a99187ebb992 tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [12] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ada91c26-ca97-4b8b-98bb-a310d0627691 (non-flash)
Nov 29 03:10:00 user nova-compute[75234]: tcp: [13] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e883290-42f6-40fd-854d-c42f9aca8992 (non-flash)
Nov 29 03:10:00 user nova-compute[75234]: tcp: [9] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672 (non-flash)
Nov 29 03:10:00 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:10:00 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-4badf7dd-4675-4725-b5e9-a99187ebb992 tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] iscsi session list stdout=tcp: [12] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ada91c26-ca97-4b8b-98bb-a310d0627691 (non-flash)
Nov 29 03:10:00 user nova-compute[75234]: tcp: [13] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6e883290-42f6-40fd-854d-c42f9aca8992 (non-flash)
Nov 29 03:10:00 user nova-compute[75234]: tcp: [9] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672 (non-flash)
Nov 29 03:10:00 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsi_session /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Nov 29 03:10:00 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-4badf7dd-4675-4725-b5e9-a99187ebb992 tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7f8a502c6e60>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:6e883290-42f6-40fd-854d-c42f9aca8992'): ({'sdc'}, set())}) {{(pid=75234) _get_connection_devices /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:863}}
Nov 29 03:10:00 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-4badf7dd-4675-4725-b5e9-a99187ebb992 tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] Removing single pathed devices sdc {{(pid=75234) remove_connection /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:374}}
Nov 29 03:10:00 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:10:00 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.029s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:10:00 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[f7370257-204b-4fbe-8fca-cd484d521134]: (4, ('path checker states:\nup                  3\n\npaths: 0\nbusy: False\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:10:00 user nova-compute[75234]: INFO nova.compute.manager [None req-24b81732-961d-4591-ba9d-d1bd04899fa4 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] [instance: 6c0e0184-ca3a-48ec-a6e3-a0a1172260d6] Took 6.67 seconds to spawn the instance on the hypervisor.
Nov 29 03:10:00 user nova-compute[75234]: DEBUG nova.compute.manager [None req-24b81732-961d-4591-ba9d-d1bd04899fa4 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] [instance: 6c0e0184-ca3a-48ec-a6e3-a0a1172260d6] Checking state {{(pid=75234) _get_power_state /opt/stack/nova/nova/compute/manager.py:1782}}
Nov 29 03:10:00 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdc {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:10:00 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdc" returned: 0 in 0.044s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:10:00 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[807b45ef-82df-4f41-af26-64d8cfa82aee]: (4, ('ok\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:10:00 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-4badf7dd-4675-4725-b5e9-a99187ebb992 tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] Flushing IO for device /dev/sdc {{(pid=75234) flush_device_io /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:434}}
Nov 29 03:10:00 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdc {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:10:00 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance 2bf9fe41-63c2-4882-bcdd-8e314e734ffe actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:10:00 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance 21e9c6cd-c39f-41c0-8768-5dd64589c3f3 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:10:00 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance a9350a18-14e6-42c6-a150-0a1487f20ad9 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:10:00 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance f609a7b7-1be2-44b4-9177-7e72a4d4c060 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:10:00 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance 8297c264-a213-44ee-be46-5c08eabda9f7 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:10:00 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance f7d0bdeb-6a05-47ac-87f2-c8380a9a4286 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:10:00 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance 062753a6-8f39-4e80-a549-9f0f5a4b877a actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:10:00 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance ac620533-8274-4674-b241-f48526bed3c3 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:10:00 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance 6c0e0184-ca3a-48ec-a6e3-a0a1172260d6 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:10:00 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Total usable vcpus: 12, total allocated vcpus: 9 {{(pid=75234) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1129}}
Nov 29 03:10:00 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Final resource view: name=user phys_ram=16011MB used_ram=5120MB phys_disk=40GB used_disk=9GB total_vcpus=12 used_vcpus=9 pci_stats=[] {{(pid=75234) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1138}}
Nov 29 03:10:00 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdc" returned: 0 in 0.019s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:10:00 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[9d6a2153-57c5-4697-b974-906a1c3ee43a]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:10:00 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-4badf7dd-4675-4725-b5e9-a99187ebb992 tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] Remove SCSI device /dev/sdc with /sys/block/sdc/device/delete {{(pid=75234) remove_scsi_device /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:148}}
Nov 29 03:10:00 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdc/device/delete {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:10:00 user nova-compute[75234]: INFO nova.compute.manager [None req-24b81732-961d-4591-ba9d-d1bd04899fa4 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] [instance: 6c0e0184-ca3a-48ec-a6e3-a0a1172260d6] Took 7.61 seconds to build instance.
Nov 29 03:10:00 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdc/device/delete" returned: 0 in 0.045s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:10:00 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[bd6118f9-a3b5-4dcb-9cb5-f5267d938300]: (4, ('1', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:10:00 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-4badf7dd-4675-4725-b5e9-a99187ebb992 tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] Checking to see if SCSI volumes sdc have been removed. {{(pid=75234) wait_for_volumes_removal /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:156}}
Nov 29 03:10:00 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-4badf7dd-4675-4725-b5e9-a99187ebb992 tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] SCSI volumes sdc have been removed. {{(pid=75234) wait_for_volumes_removal /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:166}}
Nov 29 03:10:00 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-4badf7dd-4675-4725-b5e9-a99187ebb992 tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:6e883290-42f6-40fd-854d-c42f9aca8992')] {{(pid=75234) _disconnect_connection /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Nov 29 03:10:00 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:6e883290-42f6-40fd-854d-c42f9aca8992 -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:10:00 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:6e883290-42f6-40fd-854d-c42f9aca8992 -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.020s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:10:00 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[f27e708c-8aef-4a68-90a1-3efa10c84a37]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:10:00 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-24b81732-961d-4591-ba9d-d1bd04899fa4 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Lock "6c0e0184-ca3a-48ec-a6e3-a0a1172260d6" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 7.720s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:10:00 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-4badf7dd-4675-4725-b5e9-a99187ebb992 tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:10:00 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:6e883290-42f6-40fd-854d-c42f9aca8992 -p 172.16.0.220:3260 --logout {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:10:00 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:6e883290-42f6-40fd-854d-c42f9aca8992 -p 172.16.0.220:3260 --logout" returned: 0 in 0.071s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:10:00 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[3d8be0eb-ed46-4763-a098-59481f1f9dd4]: (4, ('Logging out of session [sid: 13, target: iqn.2010-10.org.openstack:6e883290-42f6-40fd-854d-c42f9aca8992, portal: 172.16.0.220,3260]\nLogout of [sid: 13, target: iqn.2010-10.org.openstack:6e883290-42f6-40fd-854d-c42f9aca8992, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:10:00 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-4badf7dd-4675-4725-b5e9-a99187ebb992 tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 13, target: iqn.2010-10.org.openstack:6e883290-42f6-40fd-854d-c42f9aca8992, portal: 172.16.0.220,3260]
Nov 29 03:10:00 user nova-compute[75234]: Logout of [sid: 13, target: iqn.2010-10.org.openstack:6e883290-42f6-40fd-854d-c42f9aca8992, portal: 172.16.0.220,3260] successful.
Nov 29 03:10:00 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:10:00 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:6e883290-42f6-40fd-854d-c42f9aca8992 -p 172.16.0.220:3260 --op delete {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:10:00 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:6e883290-42f6-40fd-854d-c42f9aca8992 -p 172.16.0.220:3260 --op delete" returned: 0 in 0.032s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:10:00 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[7faf5e04-6c80-4459-95cb-237715be7160]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:10:00 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-4badf7dd-4675-4725-b5e9-a99187ebb992 tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:10:00 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-4badf7dd-4675-4725-b5e9-a99187ebb992 tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.444s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:87}}
Nov 29 03:10:00 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-4badf7dd-4675-4725-b5e9-a99187ebb992 tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] <== disconnect_volume: return (448ms) None {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:203}}
Nov 29 03:10:00 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-4badf7dd-4675-4725-b5e9-a99187ebb992 tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] [instance: f7d0bdeb-6a05-47ac-87f2-c8380a9a4286] Disconnected iSCSI Volume {{(pid=75234) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:79}}
Nov 29 03:10:00 user nova-compute[75234]: DEBUG nova.compute.manager [req-76ba6b80-de70-49a9-a89d-ed753acb0b70 req-7625e8e9-6585-40be-8269-1da305ce2ebc service nova] [instance: 6c0e0184-ca3a-48ec-a6e3-a0a1172260d6] Received event network-vif-plugged-50593025-edf1-44ec-b5f0-c92d6a807c23 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:10:00 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-76ba6b80-de70-49a9-a89d-ed753acb0b70 req-7625e8e9-6585-40be-8269-1da305ce2ebc service nova] Acquiring lock "6c0e0184-ca3a-48ec-a6e3-a0a1172260d6-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:10:00 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-76ba6b80-de70-49a9-a89d-ed753acb0b70 req-7625e8e9-6585-40be-8269-1da305ce2ebc service nova] Lock "6c0e0184-ca3a-48ec-a6e3-a0a1172260d6-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:10:00 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-76ba6b80-de70-49a9-a89d-ed753acb0b70 req-7625e8e9-6585-40be-8269-1da305ce2ebc service nova] Lock "6c0e0184-ca3a-48ec-a6e3-a0a1172260d6-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:10:00 user nova-compute[75234]: DEBUG nova.compute.manager [req-76ba6b80-de70-49a9-a89d-ed753acb0b70 req-7625e8e9-6585-40be-8269-1da305ce2ebc service nova] [instance: 6c0e0184-ca3a-48ec-a6e3-a0a1172260d6] No waiting events found dispatching network-vif-plugged-50593025-edf1-44ec-b5f0-c92d6a807c23 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:10:00 user nova-compute[75234]: WARNING nova.compute.manager [req-76ba6b80-de70-49a9-a89d-ed753acb0b70 req-7625e8e9-6585-40be-8269-1da305ce2ebc service nova] [instance: 6c0e0184-ca3a-48ec-a6e3-a0a1172260d6] Received unexpected event network-vif-plugged-50593025-edf1-44ec-b5f0-c92d6a807c23 for instance with vm_state active and task_state None.
Nov 29 03:10:01 user nova-compute[75234]: DEBUG nova.compute.provider_tree [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Inventory has not changed in ProviderTree for provider: 4d291220-7c37-4839-a0e3-b23c66502b74 {{(pid=75234) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Nov 29 03:10:01 user nova-compute[75234]: DEBUG nova.scheduler.client.report [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Inventory has not changed for provider 4d291220-7c37-4839-a0e3-b23c66502b74 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=75234) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Nov 29 03:10:01 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Compute_service record updated for user:user {{(pid=75234) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:1067}}
Nov 29 03:10:01 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.641s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:10:01 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:10:01 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:10:01 user nova-compute[75234]: DEBUG nova.objects.instance [None req-4badf7dd-4675-4725-b5e9-a99187ebb992 tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] Lazy-loading 'flavor' on Instance uuid f7d0bdeb-6a05-47ac-87f2-c8380a9a4286 {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:10:01 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-4badf7dd-4675-4725-b5e9-a99187ebb992 tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] Lock "f7d0bdeb-6a05-47ac-87f2-c8380a9a4286" "released" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: held 23.453s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:10:02 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:10:02 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-fd53cdf9-c639-44ea-869c-801a1a7b6505 tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] [instance: a9350a18-14e6-42c6-a150-0a1487f20ad9] Calling os-brick to attach iSCSI Volume {{(pid=75234) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Nov 29 03:10:02 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-fd53cdf9-c639-44ea-869c-801a1a7b6505 tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f8a5050c9a0>, 'connection_properties': {'target_iqn': 'iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'NFCsGdbh', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:176}}
Nov 29 03:10:02 user nova-compute[75234]: WARNING os_brick.initiator.connectors.base [None req-fd53cdf9-c639-44ea-869c-801a1a7b6505 tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Nov 29 03:10:02 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-fd53cdf9-c639-44ea-869c-801a1a7b6505 tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:68}}
Nov 29 03:10:02 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-fd53cdf9-c639-44ea-869c-801a1a7b6505 tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.002s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:73}}
Nov 29 03:10:02 user nova-compute[75234]: WARNING os_brick.initiator.connectors.base [None req-fd53cdf9-c639-44ea-869c-801a1a7b6505 tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Nov 29 03:10:02 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-fd53cdf9-c639-44ea-869c-801a1a7b6505 tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:68}}
Nov 29 03:10:02 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-fd53cdf9-c639-44ea-869c-801a1a7b6505 tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.002s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:73}}
Nov 29 03:10:02 user nova-compute[75234]: INFO os_brick.initiator.connectors.iscsi [None req-fd53cdf9-c639-44ea-869c-801a1a7b6505 tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Nov 29 03:10:02 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b -p 172.16.0.220:3260 {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:10:02 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b -p 172.16.0.220:3260" returned: 21 in 0.019s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:10:02 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[0b98ea0f-3197-4494-8c85-98a2a7ff4051]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:10:02 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-fd53cdf9-c639-44ea-869c-801a1a7b6505 tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Nov 29 03:10:02 user nova-compute[75234]:  {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:10:02 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b -p 172.16.0.220:3260 --interface default --op new {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:10:02 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.021s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:10:02 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[70f5c88f-438f-4a0f-99ba-5e9b03e80df7]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b] added\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:10:02 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-fd53cdf9-c639-44ea-869c-801a1a7b6505 tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b] added
Nov 29 03:10:02 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:10:02 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:10:02 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.026s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:10:02 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[4e123f9d-8149-4b6e-a504-4cd6bb86f197]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:10:02 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-fd53cdf9-c639-44ea-869c-801a1a7b6505 tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:10:02 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:10:02 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.021s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:10:02 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[66361255-2e89-4f46-b164-53d65101b6df]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:10:02 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-fd53cdf9-c639-44ea-869c-801a1a7b6505 tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:10:02 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b -p 172.16.0.220:3260 --op update -n node.session.auth.username -v NFCsGdbh {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:10:02 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b -p 172.16.0.220:3260 --op update -n node.session.auth.username -v NFCsGdbh" returned: 0 in 0.020s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:10:02 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[c0e9082b-d7ae-4059-acda-68879c58f001]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:10:02 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-fd53cdf9-c639-44ea-869c-801a1a7b6505 tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'NFCsGdbh'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:10:02 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:10:02 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.026s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:10:02 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[aee4267a-5587-454b-ace6-76ee20e0b84c]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:10:02 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-fd53cdf9-c639-44ea-869c-801a1a7b6505 tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:10:02 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:10:02 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.021s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:10:02 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[a8e5ee10-ce8a-4ba5-a0bf-239d6591e8cc]: (4, ('tcp: [12] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ada91c26-ca97-4b8b-98bb-a310d0627691 (non-flash)\ntcp: [9] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672 (non-flash)\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:10:02 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-fd53cdf9-c639-44ea-869c-801a1a7b6505 tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [12] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ada91c26-ca97-4b8b-98bb-a310d0627691 (non-flash)
Nov 29 03:10:02 user nova-compute[75234]: tcp: [9] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672 (non-flash)
Nov 29 03:10:02 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:10:02 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-fd53cdf9-c639-44ea-869c-801a1a7b6505 tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] iscsi session list stdout=tcp: [12] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ada91c26-ca97-4b8b-98bb-a310d0627691 (non-flash)
Nov 29 03:10:02 user nova-compute[75234]: tcp: [9] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672 (non-flash)
Nov 29 03:10:02 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsi_session /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Nov 29 03:10:02 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b -p 172.16.0.220:3260 --login {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:10:02 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b -p 172.16.0.220:3260 --login" returned: 0 in 0.080s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:10:02 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[d26aceff-a346-405c-8dec-ca4a74d92638]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:10:02 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-fd53cdf9-c639-44ea-869c-801a1a7b6505 tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b, portal: 172.16.0.220,3260]
Nov 29 03:10:02 user nova-compute[75234]: Login to [iface: default, target: iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b, portal: 172.16.0.220,3260] successful.
Nov 29 03:10:02 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:10:02 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:10:02 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.020s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:10:02 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[f5929aa2-60b8-488d-9930-398d48568a46]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:10:02 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-fd53cdf9-c639-44ea-869c-801a1a7b6505 tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:10:02 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:10:02 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.028s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:10:02 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[655860b9-78ab-45e9-bc3a-9f7512b17dd1]: (4, ('tcp: [12] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ada91c26-ca97-4b8b-98bb-a310d0627691 (non-flash)\ntcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b (non-flash)\ntcp: [9] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672 (non-flash)\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:10:02 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-fd53cdf9-c639-44ea-869c-801a1a7b6505 tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [12] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ada91c26-ca97-4b8b-98bb-a310d0627691 (non-flash)
Nov 29 03:10:02 user nova-compute[75234]: tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b (non-flash)
Nov 29 03:10:02 user nova-compute[75234]: tcp: [9] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672 (non-flash)
Nov 29 03:10:02 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:10:02 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-fd53cdf9-c639-44ea-869c-801a1a7b6505 tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] iscsi session list stdout=tcp: [12] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ada91c26-ca97-4b8b-98bb-a310d0627691 (non-flash)
Nov 29 03:10:02 user nova-compute[75234]: tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b (non-flash)
Nov 29 03:10:02 user nova-compute[75234]: tcp: [9] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672 (non-flash)
Nov 29 03:10:02 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsi_session /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Nov 29 03:10:02 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-fd53cdf9-c639-44ea-869c-801a1a7b6505 tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.335s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:87}}
Nov 29 03:10:02 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-fd53cdf9-c639-44ea-869c-801a1a7b6505 tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Connected to 172.16.0.220:3260 {{(pid=75234) _connect_vol /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:644}}
Nov 29 03:10:02 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-fd53cdf9-c639-44ea-869c-801a1a7b6505 tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] HCTL ('34', '-', '-', 0) found on session 14 with lun 0 {{(pid=75234) get_hctl /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:770}}
Nov 29 03:10:02 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-fd53cdf9-c639-44ea-869c-801a1a7b6505 tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Scanning host 34 c: -, t: -, l: 0) {{(pid=75234) scan_iscsi /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:795}}
Nov 29 03:10:02 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host34/scan {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:10:02 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host34/scan" returned: 0 in 0.019s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:10:02 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[d24c0bc6-b250-4697-b2f0-5a3ce3089408]: (4, ('- - 0', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:10:02 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-fd53cdf9-c639-44ea-869c-801a1a7b6505 tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Searching for a device in session 14 and hctl ['34', '*', '*', 0] yield: sdc {{(pid=75234) device_name_by_hctl /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:789}}
Nov 29 03:10:02 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-fd53cdf9-c639-44ea-869c-801a1a7b6505 tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Connected to sdc using {'target_iqn': 'iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'NFCsGdbh', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False} {{(pid=75234) _connect_vol /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:672}}
Nov 29 03:10:02 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-8a8b0aa3-ddc3-41cf-8db9-0f46a39943f7 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] [instance: 8297c264-a213-44ee-be46-5c08eabda9f7] Calling os-brick to attach iSCSI Volume {{(pid=75234) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Nov 29 03:10:02 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-8a8b0aa3-ddc3-41cf-8db9-0f46a39943f7 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f8a5050c9a0>, 'connection_properties': {'target_iqn': 'iqn.2010-10.org.openstack:a7ef35bf-017a-4fd9-9182-79c1b2ab4f98', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'iSOJqDFN', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:176}}
Nov 29 03:10:02 user nova-compute[75234]: WARNING os_brick.initiator.connectors.base [None req-8a8b0aa3-ddc3-41cf-8db9-0f46a39943f7 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Nov 29 03:10:02 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-8a8b0aa3-ddc3-41cf-8db9-0f46a39943f7 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:68}}
Nov 29 03:10:03 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:10:03 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:10:03 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-fd53cdf9-c639-44ea-869c-801a1a7b6505 tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.385s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:87}}
Nov 29 03:10:03 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-fd53cdf9-c639-44ea-869c-801a1a7b6505 tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] <== connect_volume: return (1387ms) {'type': 'block', 'scsi_wwn': '26539313062336265', 'path': '/dev/sdc'} {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:203}}
Nov 29 03:10:03 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-fd53cdf9-c639-44ea-869c-801a1a7b6505 tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] [instance: a9350a18-14e6-42c6-a150-0a1487f20ad9] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '26539313062336265', 'path': '/dev/sdc'} {{(pid=75234) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Nov 29 03:10:03 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-8a8b0aa3-ddc3-41cf-8db9-0f46a39943f7 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.910s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:73}}
Nov 29 03:10:03 user nova-compute[75234]: WARNING os_brick.initiator.connectors.base [None req-8a8b0aa3-ddc3-41cf-8db9-0f46a39943f7 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Nov 29 03:10:03 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-8a8b0aa3-ddc3-41cf-8db9-0f46a39943f7 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:a7ef35bf-017a-4fd9-9182-79c1b2ab4f98" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:68}}
Nov 29 03:10:03 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-8a8b0aa3-ddc3-41cf-8db9-0f46a39943f7 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:a7ef35bf-017a-4fd9-9182-79c1b2ab4f98" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:73}}
Nov 29 03:10:03 user nova-compute[75234]: INFO os_brick.initiator.connectors.iscsi [None req-8a8b0aa3-ddc3-41cf-8db9-0f46a39943f7 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Nov 29 03:10:03 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:a7ef35bf-017a-4fd9-9182-79c1b2ab4f98 -p 172.16.0.220:3260 {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:10:03 user nova-compute[75234]: DEBUG nova.objects.instance [None req-fd53cdf9-c639-44ea-869c-801a1a7b6505 tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Lazy-loading 'flavor' on Instance uuid a9350a18-14e6-42c6-a150-0a1487f20ad9 {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:10:03 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:a7ef35bf-017a-4fd9-9182-79c1b2ab4f98 -p 172.16.0.220:3260" returned: 21 in 0.019s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:10:03 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[d7ef5969-3ec6-4aff-8eaa-7582f8b06b77]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:10:03 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-8a8b0aa3-ddc3-41cf-8db9-0f46a39943f7 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Nov 29 03:10:03 user nova-compute[75234]:  {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:10:03 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:a7ef35bf-017a-4fd9-9182-79c1b2ab4f98 -p 172.16.0.220:3260 --interface default --op new {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:10:03 user nova-compute[75234]: DEBUG nova.virt.libvirt.guest [None req-fd53cdf9-c639-44ea-869c-801a1a7b6505 tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] attach device xml: <disk type="block" device="disk">
Nov 29 03:10:03 user nova-compute[75234]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Nov 29 03:10:03 user nova-compute[75234]:   <source dev="/dev/sdc"/>
Nov 29 03:10:03 user nova-compute[75234]:   <target dev="vdb" bus="virtio"/>
Nov 29 03:10:03 user nova-compute[75234]:   <serial>f4130d3c-b329-4ce1-8b6d-d9e270a43b0b</serial>
Nov 29 03:10:03 user nova-compute[75234]: </disk>
Nov 29 03:10:03 user nova-compute[75234]:  {{(pid=75234) attach_device /opt/stack/nova/nova/virt/libvirt/guest.py:339}}
Nov 29 03:10:03 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:a7ef35bf-017a-4fd9-9182-79c1b2ab4f98 -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.025s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:10:03 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[64401b9f-b83e-4b2d-844a-216fa2800891]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:a7ef35bf-017a-4fd9-9182-79c1b2ab4f98] added\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:10:03 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-8a8b0aa3-ddc3-41cf-8db9-0f46a39943f7 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:a7ef35bf-017a-4fd9-9182-79c1b2ab4f98] added
Nov 29 03:10:03 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:10:03 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:a7ef35bf-017a-4fd9-9182-79c1b2ab4f98 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:10:03 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:a7ef35bf-017a-4fd9-9182-79c1b2ab4f98 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.023s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:10:03 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[74fc79c3-65ef-4a34-9fc7-d446673b3e6c]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:10:03 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-8a8b0aa3-ddc3-41cf-8db9-0f46a39943f7 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:10:03 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:a7ef35bf-017a-4fd9-9182-79c1b2ab4f98 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:10:03 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:a7ef35bf-017a-4fd9-9182-79c1b2ab4f98 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.019s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:10:03 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[11bf63e9-1193-4638-b1be-77091d8c199f]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:10:03 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-8a8b0aa3-ddc3-41cf-8db9-0f46a39943f7 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:10:03 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:a7ef35bf-017a-4fd9-9182-79c1b2ab4f98 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v iSOJqDFN {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:10:03 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:a7ef35bf-017a-4fd9-9182-79c1b2ab4f98 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v iSOJqDFN" returned: 0 in 0.021s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:10:03 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[2269b631-eb8b-4a61-bd22-24bf8490fca2]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:10:03 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-8a8b0aa3-ddc3-41cf-8db9-0f46a39943f7 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'iSOJqDFN'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:10:03 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:a7ef35bf-017a-4fd9-9182-79c1b2ab4f98 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:10:03 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:a7ef35bf-017a-4fd9-9182-79c1b2ab4f98 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.029s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:10:03 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[ccfbd85a-29ea-4458-91ea-8538297666f1]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:10:03 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-8a8b0aa3-ddc3-41cf-8db9-0f46a39943f7 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:10:03 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:10:03 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.027s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:10:03 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[40c72205-a604-4e74-8c97-0d1f69ca1135]: (4, ('tcp: [12] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ada91c26-ca97-4b8b-98bb-a310d0627691 (non-flash)\ntcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b (non-flash)\ntcp: [9] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672 (non-flash)\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:10:03 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-8a8b0aa3-ddc3-41cf-8db9-0f46a39943f7 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [12] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ada91c26-ca97-4b8b-98bb-a310d0627691 (non-flash)
Nov 29 03:10:03 user nova-compute[75234]: tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b (non-flash)
Nov 29 03:10:03 user nova-compute[75234]: tcp: [9] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672 (non-flash)
Nov 29 03:10:03 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:10:03 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-8a8b0aa3-ddc3-41cf-8db9-0f46a39943f7 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] iscsi session list stdout=tcp: [12] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ada91c26-ca97-4b8b-98bb-a310d0627691 (non-flash)
Nov 29 03:10:03 user nova-compute[75234]: tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b (non-flash)
Nov 29 03:10:03 user nova-compute[75234]: tcp: [9] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672 (non-flash)
Nov 29 03:10:03 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsi_session /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Nov 29 03:10:03 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:a7ef35bf-017a-4fd9-9182-79c1b2ab4f98 -p 172.16.0.220:3260 --login {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:10:04 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:a7ef35bf-017a-4fd9-9182-79c1b2ab4f98 -p 172.16.0.220:3260 --login" returned: 0 in 0.092s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:10:04 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[f7a8e702-2807-45f5-bf27-0fe3131d33ad]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:a7ef35bf-017a-4fd9-9182-79c1b2ab4f98, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:a7ef35bf-017a-4fd9-9182-79c1b2ab4f98, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:10:04 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-8a8b0aa3-ddc3-41cf-8db9-0f46a39943f7 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:a7ef35bf-017a-4fd9-9182-79c1b2ab4f98, portal: 172.16.0.220,3260]
Nov 29 03:10:04 user nova-compute[75234]: Login to [iface: default, target: iqn.2010-10.org.openstack:a7ef35bf-017a-4fd9-9182-79c1b2ab4f98, portal: 172.16.0.220,3260] successful.
Nov 29 03:10:04 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:10:04 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:a7ef35bf-017a-4fd9-9182-79c1b2ab4f98 -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:10:04 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:a7ef35bf-017a-4fd9-9182-79c1b2ab4f98 -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.018s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:10:04 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[0f4b39a4-c0fa-4af9-a9b7-7e52c82185b8]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:10:04 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-8a8b0aa3-ddc3-41cf-8db9-0f46a39943f7 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:10:04 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:10:04 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.036s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:10:04 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[b1dab103-59c4-4e59-a3f8-98d15bfe345a]: (4, ('tcp: [12] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ada91c26-ca97-4b8b-98bb-a310d0627691 (non-flash)\ntcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b (non-flash)\ntcp: [15] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a7ef35bf-017a-4fd9-9182-79c1b2ab4f98 (non-flash)\ntcp: [9] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672 (non-flash)\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:10:04 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-8a8b0aa3-ddc3-41cf-8db9-0f46a39943f7 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [12] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ada91c26-ca97-4b8b-98bb-a310d0627691 (non-flash)
Nov 29 03:10:04 user nova-compute[75234]: tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b (non-flash)
Nov 29 03:10:04 user nova-compute[75234]: tcp: [15] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a7ef35bf-017a-4fd9-9182-79c1b2ab4f98 (non-flash)
Nov 29 03:10:04 user nova-compute[75234]: tcp: [9] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672 (non-flash)
Nov 29 03:10:04 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:10:04 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-8a8b0aa3-ddc3-41cf-8db9-0f46a39943f7 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] iscsi session list stdout=tcp: [12] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ada91c26-ca97-4b8b-98bb-a310d0627691 (non-flash)
Nov 29 03:10:04 user nova-compute[75234]: tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b (non-flash)
Nov 29 03:10:04 user nova-compute[75234]: tcp: [15] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a7ef35bf-017a-4fd9-9182-79c1b2ab4f98 (non-flash)
Nov 29 03:10:04 user nova-compute[75234]: tcp: [9] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672 (non-flash)
Nov 29 03:10:04 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsi_session /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Nov 29 03:10:04 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-8a8b0aa3-ddc3-41cf-8db9-0f46a39943f7 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:a7ef35bf-017a-4fd9-9182-79c1b2ab4f98" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.366s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:87}}
Nov 29 03:10:04 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-8a8b0aa3-ddc3-41cf-8db9-0f46a39943f7 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] Connected to 172.16.0.220:3260 {{(pid=75234) _connect_vol /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:644}}
Nov 29 03:10:04 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-8a8b0aa3-ddc3-41cf-8db9-0f46a39943f7 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] HCTL ('35', '-', '-', 0) found on session 15 with lun 0 {{(pid=75234) get_hctl /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:770}}
Nov 29 03:10:04 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-8a8b0aa3-ddc3-41cf-8db9-0f46a39943f7 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] Scanning host 35 c: -, t: -, l: 0) {{(pid=75234) scan_iscsi /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:795}}
Nov 29 03:10:04 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host35/scan {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:10:04 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host35/scan" returned: 0 in 0.027s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:10:04 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[924a01f0-db57-4a62-95dc-fbfab7ce5193]: (4, ('- - 0', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:10:04 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-8a8b0aa3-ddc3-41cf-8db9-0f46a39943f7 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] Searching for a device in session 15 and hctl ['35', '*', '*', 0] yield: sdd {{(pid=75234) device_name_by_hctl /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:789}}
Nov 29 03:10:04 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-8a8b0aa3-ddc3-41cf-8db9-0f46a39943f7 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] Connected to sdd using {'target_iqn': 'iqn.2010-10.org.openstack:a7ef35bf-017a-4fd9-9182-79c1b2ab4f98', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'iSOJqDFN', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False} {{(pid=75234) _connect_vol /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:672}}
Nov 29 03:10:04 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-fd53cdf9-c639-44ea-869c-801a1a7b6505 tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] No BDM found with device name vda, not building metadata. {{(pid=75234) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12089}}
Nov 29 03:10:04 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-fd53cdf9-c639-44ea-869c-801a1a7b6505 tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] No BDM found with device name vdb, not building metadata. {{(pid=75234) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12089}}
Nov 29 03:10:04 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-fd53cdf9-c639-44ea-869c-801a1a7b6505 tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] No VIF found with MAC fa:16:3e:09:b8:70, not building metadata {{(pid=75234) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Nov 29 03:10:04 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-fd53cdf9-c639-44ea-869c-801a1a7b6505 tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Lock "a9350a18-14e6-42c6-a150-0a1487f20ad9" "released" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: held 10.818s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:10:05 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-8a8b0aa3-ddc3-41cf-8db9-0f46a39943f7 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.432s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:87}}
Nov 29 03:10:05 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-8a8b0aa3-ddc3-41cf-8db9-0f46a39943f7 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] <== connect_volume: return (2342ms) {'type': 'block', 'scsi_wwn': '23261333730333830', 'path': '/dev/sdd'} {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:203}}
Nov 29 03:10:05 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-8a8b0aa3-ddc3-41cf-8db9-0f46a39943f7 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] [instance: 8297c264-a213-44ee-be46-5c08eabda9f7] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '23261333730333830', 'path': '/dev/sdd'} {{(pid=75234) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Nov 29 03:10:05 user nova-compute[75234]: DEBUG nova.objects.instance [None req-8a8b0aa3-ddc3-41cf-8db9-0f46a39943f7 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] Lazy-loading 'flavor' on Instance uuid 8297c264-a213-44ee-be46-5c08eabda9f7 {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:10:05 user nova-compute[75234]: DEBUG nova.virt.libvirt.guest [None req-8a8b0aa3-ddc3-41cf-8db9-0f46a39943f7 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] attach device xml: <disk type="block" device="disk">
Nov 29 03:10:05 user nova-compute[75234]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Nov 29 03:10:05 user nova-compute[75234]:   <source dev="/dev/sdd"/>
Nov 29 03:10:05 user nova-compute[75234]:   <target dev="vdb" bus="virtio"/>
Nov 29 03:10:05 user nova-compute[75234]:   <serial>a7ef35bf-017a-4fd9-9182-79c1b2ab4f98</serial>
Nov 29 03:10:05 user nova-compute[75234]: </disk>
Nov 29 03:10:05 user nova-compute[75234]:  {{(pid=75234) attach_device /opt/stack/nova/nova/virt/libvirt/guest.py:339}}
Nov 29 03:10:05 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-8a8b0aa3-ddc3-41cf-8db9-0f46a39943f7 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] No BDM found with device name vda, not building metadata. {{(pid=75234) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12089}}
Nov 29 03:10:05 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-8a8b0aa3-ddc3-41cf-8db9-0f46a39943f7 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] No BDM found with device name vdb, not building metadata. {{(pid=75234) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12089}}
Nov 29 03:10:05 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-8a8b0aa3-ddc3-41cf-8db9-0f46a39943f7 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] No VIF found with MAC fa:16:3e:5f:9d:5a, not building metadata {{(pid=75234) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Nov 29 03:10:06 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-8a8b0aa3-ddc3-41cf-8db9-0f46a39943f7 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] Lock "8297c264-a213-44ee-be46-5c08eabda9f7" "released" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: held 12.404s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:10:06 user nova-compute[75234]: DEBUG nova.objects.instance [None req-e0b104d6-8283-44f5-a150-3606c697b81a tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Lazy-loading 'flavor' on Instance uuid f609a7b7-1be2-44b4-9177-7e72a4d4c060 {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:10:06 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:10:06 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-e0b104d6-8283-44f5-a150-3606c697b81a tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Acquiring lock "refresh_cache-f609a7b7-1be2-44b4-9177-7e72a4d4c060" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:10:06 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-e0b104d6-8283-44f5-a150-3606c697b81a tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Acquired lock "refresh_cache-f609a7b7-1be2-44b4-9177-7e72a4d4c060" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:10:06 user nova-compute[75234]: DEBUG nova.network.neutron [None req-e0b104d6-8283-44f5-a150-3606c697b81a tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] [instance: f609a7b7-1be2-44b4-9177-7e72a4d4c060] Building network info cache for instance {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2009}}
Nov 29 03:10:06 user nova-compute[75234]: DEBUG nova.objects.instance [None req-e0b104d6-8283-44f5-a150-3606c697b81a tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Lazy-loading 'info_cache' on Instance uuid f609a7b7-1be2-44b4-9177-7e72a4d4c060 {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:10:06 user nova-compute[75234]: DEBUG nova.objects.base [None req-e0b104d6-8283-44f5-a150-3606c697b81a tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Object Instance<f609a7b7-1be2-44b4-9177-7e72a4d4c060> lazy-loaded attributes: flavor,info_cache {{(pid=75234) wrapper /opt/stack/nova/nova/objects/base.py:126}}
Nov 29 03:10:06 user nova-compute[75234]: INFO nova.compute.manager [None req-413e3555-a1c6-47b1-9af6-79aba9489343 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] [instance: 8297c264-a213-44ee-be46-5c08eabda9f7] Rescuing
Nov 29 03:10:06 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-413e3555-a1c6-47b1-9af6-79aba9489343 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] Acquiring lock "refresh_cache-8297c264-a213-44ee-be46-5c08eabda9f7" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:10:06 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-413e3555-a1c6-47b1-9af6-79aba9489343 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] Acquired lock "refresh_cache-8297c264-a213-44ee-be46-5c08eabda9f7" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:10:06 user nova-compute[75234]: DEBUG nova.network.neutron [None req-413e3555-a1c6-47b1-9af6-79aba9489343 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] [instance: 8297c264-a213-44ee-be46-5c08eabda9f7] Building network info cache for instance {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2009}}
Nov 29 03:10:06 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:10:07 user nova-compute[75234]: DEBUG nova.network.neutron [None req-e0b104d6-8283-44f5-a150-3606c697b81a tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] [instance: f609a7b7-1be2-44b4-9177-7e72a4d4c060] Updating instance_info_cache with network_info: [{"id": "5543f379-2b3a-4dc2-bfe9-52f66a942257", "address": "fa:16:3e:04:ae:43", "network": {"id": "294c3aac-8c9b-45f0-8c0a-344e2d7d8b7c", "bridge": "br-int", "label": "tempest-AttachVolumeTestJSON-700098935-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.14", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "e393d23e783b4a40986674d782606c69", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap5543f379-2b", "ovs_interfaceid": "5543f379-2b3a-4dc2-bfe9-52f66a942257", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:10:07 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-e0b104d6-8283-44f5-a150-3606c697b81a tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Releasing lock "refresh_cache-f609a7b7-1be2-44b4-9177-7e72a4d4c060" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:10:07 user nova-compute[75234]: DEBUG nova.compute.manager [None req-221919a0-9cdd-44ec-881f-d6c82383d31d tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] [instance: a9350a18-14e6-42c6-a150-0a1487f20ad9] Stashing vm_state: active {{(pid=75234) _prep_resize /opt/stack/nova/nova/compute/manager.py:5615}}
Nov 29 03:10:07 user nova-compute[75234]: INFO nova.virt.libvirt.driver [-] [instance: f609a7b7-1be2-44b4-9177-7e72a4d4c060] Instance destroyed successfully.
Nov 29 03:10:07 user nova-compute[75234]: DEBUG nova.objects.instance [None req-e0b104d6-8283-44f5-a150-3606c697b81a tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Lazy-loading 'resources' on Instance uuid f609a7b7-1be2-44b4-9177-7e72a4d4c060 {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:10:07 user nova-compute[75234]: DEBUG nova.objects.base [None req-e0b104d6-8283-44f5-a150-3606c697b81a tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Object Instance<f609a7b7-1be2-44b4-9177-7e72a4d4c060> lazy-loaded attributes: flavor,info_cache,resources {{(pid=75234) wrapper /opt/stack/nova/nova/objects/base.py:126}}
Nov 29 03:10:07 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-e0b104d6-8283-44f5-a150-3606c697b81a tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-11-29T03:06:06Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-AttachVolumeTestJSON-server-394712256',display_name='tempest-AttachVolumeTestJSON-server-394712256',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-attachvolumetestjson-server-394712256',id=4,image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBGphX+MWMhY0YZ/wlOvEQFKFHTIX4g9KMhxkTe+ErHeVHCk7d+KpOlTDVK9edUWFFxjgJKboch3pp1xrAYGa/rIEWyUV4YOfCGPFoAeneAuweb6ekmwg4qgLxfML0aXASQ==',key_name='tempest-keypair-213174705',keypairs=<?>,launch_index=0,launched_at=2023-11-29T03:06:23Z,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=<?>,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=4,progress=0,project_id='e393d23e783b4a40986674d782606c69',ramdisk_id='',reservation_id='r-9o6ggnbe',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=<?>,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-AttachVolumeTestJSON-1911423306',owner_user_name='tempest-AttachVolumeTestJSON-1911423306-project-member'},tags=<?>,task_state='powering-on',terminated_at=None,trusted_certs=<?>,updated_at=2023-11-29T03:08:16Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='bb1126c998ee430baac2e2494f30c0aa',uuid=f609a7b7-1be2-44b4-9177-7e72a4d4c060,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='stopped') vif={"id": "5543f379-2b3a-4dc2-bfe9-52f66a942257", "address": "fa:16:3e:04:ae:43", "network": {"id": "294c3aac-8c9b-45f0-8c0a-344e2d7d8b7c", "bridge": "br-int", "label": "tempest-AttachVolumeTestJSON-700098935-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.14", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "e393d23e783b4a40986674d782606c69", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap5543f379-2b", "ovs_interfaceid": "5543f379-2b3a-4dc2-bfe9-52f66a942257", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Nov 29 03:10:07 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-e0b104d6-8283-44f5-a150-3606c697b81a tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Converting VIF {"id": "5543f379-2b3a-4dc2-bfe9-52f66a942257", "address": "fa:16:3e:04:ae:43", "network": {"id": "294c3aac-8c9b-45f0-8c0a-344e2d7d8b7c", "bridge": "br-int", "label": "tempest-AttachVolumeTestJSON-700098935-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.14", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "e393d23e783b4a40986674d782606c69", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap5543f379-2b", "ovs_interfaceid": "5543f379-2b3a-4dc2-bfe9-52f66a942257", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:10:07 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-e0b104d6-8283-44f5-a150-3606c697b81a tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:04:ae:43,bridge_name='br-int',has_traffic_filtering=True,id=5543f379-2b3a-4dc2-bfe9-52f66a942257,network=Network(294c3aac-8c9b-45f0-8c0a-344e2d7d8b7c),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap5543f379-2b') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:10:07 user nova-compute[75234]: DEBUG os_vif [None req-e0b104d6-8283-44f5-a150-3606c697b81a tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Unplugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:04:ae:43,bridge_name='br-int',has_traffic_filtering=True,id=5543f379-2b3a-4dc2-bfe9-52f66a942257,network=Network(294c3aac-8c9b-45f0-8c0a-344e2d7d8b7c),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap5543f379-2b') {{(pid=75234) unplug /opt/stack/data/venv/lib/python3.10/site-packages/os_vif/__init__.py:109}}
Nov 29 03:10:07 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:10:07 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tap5543f379-2b, bridge=br-int, if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:10:07 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:10:07 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:10:07 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:10:07 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbDestroyCommand(_result=None, table=QoS, record=8f761051-aad1-489a-9d20-44261a252c60) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:10:07 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:10:07 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:10:07 user nova-compute[75234]: INFO os_vif [None req-e0b104d6-8283-44f5-a150-3606c697b81a tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Successfully unplugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:04:ae:43,bridge_name='br-int',has_traffic_filtering=True,id=5543f379-2b3a-4dc2-bfe9-52f66a942257,network=Network(294c3aac-8c9b-45f0-8c0a-344e2d7d8b7c),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap5543f379-2b')
Nov 29 03:10:07 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-e0b104d6-8283-44f5-a150-3606c697b81a tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] [instance: f609a7b7-1be2-44b4-9177-7e72a4d4c060] calling os-brick to detach iSCSI Volume {{(pid=75234) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Nov 29 03:10:07 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-e0b104d6-8283-44f5-a150-3606c697b81a tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f8a5050c9a0>, {'target_iqn': 'iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'frKqcBuE', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdh'}, None), 'kwargs': True}}" {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:176}}
Nov 29 03:10:07 user nova-compute[75234]: WARNING os_brick.initiator.connectors.base [None req-e0b104d6-8283-44f5-a150-3606c697b81a tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Nov 29 03:10:07 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-e0b104d6-8283-44f5-a150-3606c697b81a tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:68}}
Nov 29 03:10:07 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-e0b104d6-8283-44f5-a150-3606c697b81a tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:73}}
Nov 29 03:10:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:10:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.030s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:10:07 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[e6823d19-2fe0-48a3-9945-e505bb3c7eb9]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:a7ef35bf-017a-4fd9-9182-79c1b2ab4f98\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:ada91c26-ca97-4b8b-98bb-a310d0627691\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:10:07 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-e0b104d6-8283-44f5-a150-3606c697b81a tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS:
Nov 29 03:10:07 user nova-compute[75234]: No targets found.
Nov 29 03:10:07 user nova-compute[75234]: iSNS:
Nov 29 03:10:07 user nova-compute[75234]: No targets found.
Nov 29 03:10:07 user nova-compute[75234]: STATIC:
Nov 29 03:10:07 user nova-compute[75234]: Target: iqn.2010-10.org.openstack:a7ef35bf-017a-4fd9-9182-79c1b2ab4f98
Nov 29 03:10:07 user nova-compute[75234]:         Portal: 172.16.0.220:3260,-1
Nov 29 03:10:07 user nova-compute[75234]:                 Iface Name: default
Nov 29 03:10:07 user nova-compute[75234]: Target: iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672
Nov 29 03:10:07 user nova-compute[75234]:         Portal: 172.16.0.220:3260,-1
Nov 29 03:10:07 user nova-compute[75234]:                 Iface Name: default
Nov 29 03:10:07 user nova-compute[75234]: Target: iqn.2010-10.org.openstack:ada91c26-ca97-4b8b-98bb-a310d0627691
Nov 29 03:10:07 user nova-compute[75234]:         Portal: 172.16.0.220:3260,-1
Nov 29 03:10:07 user nova-compute[75234]:                 Iface Name: default
Nov 29 03:10:07 user nova-compute[75234]: Target: iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b
Nov 29 03:10:07 user nova-compute[75234]:         Portal: 172.16.0.220:3260,-1
Nov 29 03:10:07 user nova-compute[75234]:                 Iface Name: default
Nov 29 03:10:07 user nova-compute[75234]: FIRMWARE:
Nov 29 03:10:07 user nova-compute[75234]: No targets found.
Nov 29 03:10:07 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:10:07 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-e0b104d6-8283-44f5-a150-3606c697b81a tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Regex to get portals from discoverydb: ^SENDTARGETS:
Nov 29 03:10:07 user nova-compute[75234]: .*?^DiscoveryAddress: 172.16.0.220,3260.*?
Nov 29 03:10:07 user nova-compute[75234]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=75234) _get_discoverydb_portals /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:382}}
Nov 29 03:10:07 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-e0b104d6-8283-44f5-a150-3606c697b81a tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672', 0)] {{(pid=75234) _get_connection_devices /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:830}}
Nov 29 03:10:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:10:07 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-221919a0-9cdd-44ec-881f-d6c82383d31d tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.resize_claim" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:10:07 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-221919a0-9cdd-44ec-881f-d6c82383d31d tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.resize_claim" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:10:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.020s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:10:07 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[908c82f2-dd73-4bea-a23c-1de8c001dceb]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:a7ef35bf-017a-4fd9-9182-79c1b2ab4f98\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:ada91c26-ca97-4b8b-98bb-a310d0627691\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:10:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:10:07 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-221919a0-9cdd-44ec-881f-d6c82383d31d tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=75234) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2366}}
Nov 29 03:10:07 user nova-compute[75234]: INFO nova.compute.claims [None req-221919a0-9cdd-44ec-881f-d6c82383d31d tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] [instance: a9350a18-14e6-42c6-a150-0a1487f20ad9] Claim successful on node user
Nov 29 03:10:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.026s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:10:07 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[d261c172-a5db-41f8-a011-aaadbd17bf0d]: (4, ('tcp: [12] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ada91c26-ca97-4b8b-98bb-a310d0627691 (non-flash)\ntcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b (non-flash)\ntcp: [15] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a7ef35bf-017a-4fd9-9182-79c1b2ab4f98 (non-flash)\ntcp: [9] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672 (non-flash)\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:10:07 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-e0b104d6-8283-44f5-a150-3606c697b81a tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [12] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ada91c26-ca97-4b8b-98bb-a310d0627691 (non-flash)
Nov 29 03:10:07 user nova-compute[75234]: tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b (non-flash)
Nov 29 03:10:07 user nova-compute[75234]: tcp: [15] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a7ef35bf-017a-4fd9-9182-79c1b2ab4f98 (non-flash)
Nov 29 03:10:07 user nova-compute[75234]: tcp: [9] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672 (non-flash)
Nov 29 03:10:07 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:10:07 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-e0b104d6-8283-44f5-a150-3606c697b81a tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] iscsi session list stdout=tcp: [12] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ada91c26-ca97-4b8b-98bb-a310d0627691 (non-flash)
Nov 29 03:10:07 user nova-compute[75234]: tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b (non-flash)
Nov 29 03:10:07 user nova-compute[75234]: tcp: [15] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a7ef35bf-017a-4fd9-9182-79c1b2ab4f98 (non-flash)
Nov 29 03:10:07 user nova-compute[75234]: tcp: [9] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672 (non-flash)
Nov 29 03:10:07 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsi_session /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Nov 29 03:10:07 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-e0b104d6-8283-44f5-a150-3606c697b81a tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7f8a502c8430>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672'): ({'sdh'}, set())}) {{(pid=75234) _get_connection_devices /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:863}}
Nov 29 03:10:07 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-e0b104d6-8283-44f5-a150-3606c697b81a tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Removing single pathed devices sdh {{(pid=75234) remove_connection /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:374}}
Nov 29 03:10:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:10:07 user nova-compute[75234]: DEBUG nova.network.neutron [None req-413e3555-a1c6-47b1-9af6-79aba9489343 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] [instance: 8297c264-a213-44ee-be46-5c08eabda9f7] Updating instance_info_cache with network_info: [{"id": "d26334d2-8040-4022-b658-69be1cf6a025", "address": "fa:16:3e:5f:9d:5a", "network": {"id": "e0366490-9fe2-4429-88c8-6625128e645d", "bridge": "br-int", "label": "tempest-ServerStableDeviceRescueTest-804259918-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.31", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "745bdb3c72794155829dc0a69f35b321", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapd26334d2-80", "ovs_interfaceid": "d26334d2-8040-4022-b658-69be1cf6a025", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:10:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.019s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:10:07 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[fd787bc9-ceeb-4ce8-bace-95074f108779]: (4, ('path checker states:\nup                  4\n\npaths: 0\nbusy: False\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:10:07 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-413e3555-a1c6-47b1-9af6-79aba9489343 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] Releasing lock "refresh_cache-8297c264-a213-44ee-be46-5c08eabda9f7" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:10:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdh {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:10:07 user nova-compute[75234]: INFO nova.compute.resource_tracker [None req-221919a0-9cdd-44ec-881f-d6c82383d31d tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] [instance: a9350a18-14e6-42c6-a150-0a1487f20ad9] Updating resource usage from migration 29b9709a-732b-4af0-8f8f-3b10fbde30dc
Nov 29 03:10:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdh" returned: 0 in 0.022s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:10:07 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[5de1477b-34d7-4a97-bd95-b9a12df294a2]: (4, ('ok\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:10:07 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-e0b104d6-8283-44f5-a150-3606c697b81a tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Flushing IO for device /dev/sdh {{(pid=75234) flush_device_io /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:434}}
Nov 29 03:10:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdh {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:10:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdh" returned: 0 in 0.016s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:10:07 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[75e4def4-3a7e-48e1-a14b-a89b003c6334]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:10:07 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-e0b104d6-8283-44f5-a150-3606c697b81a tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Remove SCSI device /dev/sdh with /sys/block/sdh/device/delete {{(pid=75234) remove_scsi_device /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:148}}
Nov 29 03:10:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdh/device/delete {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:10:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdh/device/delete" returned: 0 in 0.066s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:10:07 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[c6a0be2e-a50a-4395-908d-4b32b7fa4c1c]: (4, ('1', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:10:07 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-e0b104d6-8283-44f5-a150-3606c697b81a tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Checking to see if SCSI volumes sdh have been removed. {{(pid=75234) wait_for_volumes_removal /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:156}}
Nov 29 03:10:07 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-e0b104d6-8283-44f5-a150-3606c697b81a tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] SCSI volumes sdh have been removed. {{(pid=75234) wait_for_volumes_removal /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:166}}
Nov 29 03:10:07 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-e0b104d6-8283-44f5-a150-3606c697b81a tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672')] {{(pid=75234) _disconnect_connection /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Nov 29 03:10:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672 -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:10:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672 -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.025s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:10:07 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[62793bd2-6aa0-4281-ac26-7bf2aa1ed444]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:10:07 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-e0b104d6-8283-44f5-a150-3606c697b81a tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:10:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672 -p 172.16.0.220:3260 --logout {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:10:07 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:10:07 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:10:07 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:10:07 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:10:07 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:10:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672 -p 172.16.0.220:3260 --logout" returned: 0 in 0.107s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:10:07 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[04b30a7b-e9c3-44e8-b6ce-cf3a87bfb615]: (4, ('Logging out of session [sid: 9, target: iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672, portal: 172.16.0.220,3260]\nLogout of [sid: 9, target: iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:10:07 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-e0b104d6-8283-44f5-a150-3606c697b81a tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 9, target: iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672, portal: 172.16.0.220,3260]
Nov 29 03:10:07 user nova-compute[75234]: Logout of [sid: 9, target: iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672, portal: 172.16.0.220,3260] successful.
Nov 29 03:10:07 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:10:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672 -p 172.16.0.220:3260 --op delete {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:10:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672 -p 172.16.0.220:3260 --op delete" returned: 0 in 0.025s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:10:07 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[fd112d9a-48d2-47d2-bd30-b1831ec3bbe1]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:10:07 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-e0b104d6-8283-44f5-a150-3606c697b81a tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:10:07 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-e0b104d6-8283-44f5-a150-3606c697b81a tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.444s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:87}}
Nov 29 03:10:07 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-e0b104d6-8283-44f5-a150-3606c697b81a tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] <== disconnect_volume: return (445ms) None {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:203}}
Nov 29 03:10:07 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-e0b104d6-8283-44f5-a150-3606c697b81a tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] [instance: f609a7b7-1be2-44b4-9177-7e72a4d4c060] Disconnected iSCSI Volume {{(pid=75234) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:79}}
Nov 29 03:10:07 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-e0b104d6-8283-44f5-a150-3606c697b81a tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] [instance: f609a7b7-1be2-44b4-9177-7e72a4d4c060] Start _get_guest_xml network_info=[{"id": "5543f379-2b3a-4dc2-bfe9-52f66a942257", "address": "fa:16:3e:04:ae:43", "network": {"id": "294c3aac-8c9b-45f0-8c0a-344e2d7d8b7c", "bridge": "br-int", "label": "tempest-AttachVolumeTestJSON-700098935-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.14", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "e393d23e783b4a40986674d782606c69", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap5543f379-2b", "ovs_interfaceid": "5543f379-2b3a-4dc2-bfe9-52f66a942257", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, '/dev/vdb': {'bus': 'virtio', 'dev': 'vdb', 'type': 'disk'}}} image_meta=ImageMeta(checksum=<?>,container_format='bare',created_at=<?>,direct_url=<?>,disk_format='qcow2',id=96c30ba7-9b73-4a30-8ba4-7138879573b7,min_disk=1,min_ram=0,name=<?>,owner=<?>,properties=ImageMetaProps,protected=<?>,size=<?>,status=<?>,tags=<?>,updated_at=<?>,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [{'device_name': '/dev/vda', 'encryption_format': None, 'encrypted': False, 'device_type': 'disk', 'size': 0, 'disk_bus': 'virtio', 'boot_index': 0, 'encryption_options': None, 'encryption_secret_uuid': None, 'guest_format': None, 'image_id': '96c30ba7-9b73-4a30-8ba4-7138879573b7'}], 'ephemerals': [], 'block_device_mapping': [{'mount_device': '/dev/vdb', 'attachment_id': 'fbaf802f-e156-481f-8810-8093ac2287a3', 'disk_bus': 'virtio', 'device_type': 'disk', 'boot_index': None, 'guest_format': None, 'delete_on_termination': False, 'connection_info': {'driver_volume_type': 'iscsi', 'data': {'target_iqn': 'iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'frKqcBuE', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdh'}, 'status': 'reserved', 'instance': 'f609a7b7-1be2-44b4-9177-7e72a4d4c060', 'attached_at': '', 'detached_at': '', 'volume_id': '33601b89-9deb-4aa4-bd5b-1654f1237672', 'serial': '33601b89-9deb-4aa4-bd5b-1654f1237672'}, 'volume_type': None} {{(pid=75234) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7562}}
Nov 29 03:10:07 user nova-compute[75234]: DEBUG nova.objects.instance [None req-e0b104d6-8283-44f5-a150-3606c697b81a tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Lazy-loading 'numa_topology' on Instance uuid f609a7b7-1be2-44b4-9177-7e72a4d4c060 {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:10:07 user nova-compute[75234]: DEBUG nova.objects.base [None req-e0b104d6-8283-44f5-a150-3606c697b81a tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Object Instance<f609a7b7-1be2-44b4-9177-7e72a4d4c060> lazy-loaded attributes: flavor,info_cache,resources,numa_topology {{(pid=75234) wrapper /opt/stack/nova/nova/objects/base.py:126}}
Nov 29 03:10:08 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-e0b104d6-8283-44f5-a150-3606c697b81a tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:10:08 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-e0b104d6-8283-44f5-a150-3606c697b81a tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:10:08 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-e0b104d6-8283-44f5-a150-3606c697b81a tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Searching host: 'user' for CPU controller through CGroups V1... {{(pid=75234) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1652}}
Nov 29 03:10:08 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-e0b104d6-8283-44f5-a150-3606c697b81a tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] CPU controller missing on host. {{(pid=75234) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1662}}
Nov 29 03:10:08 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-e0b104d6-8283-44f5-a150-3606c697b81a tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Searching host: 'user' for CPU controller through CGroups V2... {{(pid=75234) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1671}}
Nov 29 03:10:08 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-e0b104d6-8283-44f5-a150-3606c697b81a tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] CPU controller found on host. {{(pid=75234) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1678}}
Nov 29 03:10:08 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-e0b104d6-8283-44f5-a150-3606c697b81a tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=75234) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5400}}
Nov 29 03:10:08 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-e0b104d6-8283-44f5-a150-3606c697b81a tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-11-29T03:02:04Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='1',id=6,is_public=True,memory_mb=512,name='m1.tiny',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum=<?>,container_format='bare',created_at=<?>,direct_url=<?>,disk_format='qcow2',id=96c30ba7-9b73-4a30-8ba4-7138879573b7,min_disk=1,min_ram=0,name=<?>,owner=<?>,properties=ImageMetaProps,protected=<?>,size=<?>,status=<?>,tags=<?>,updated_at=<?>,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:563}}
Nov 29 03:10:08 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-e0b104d6-8283-44f5-a150-3606c697b81a tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Flavor limits 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:348}}
Nov 29 03:10:08 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-e0b104d6-8283-44f5-a150-3606c697b81a tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Image limits 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:352}}
Nov 29 03:10:08 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-e0b104d6-8283-44f5-a150-3606c697b81a tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Flavor pref 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:388}}
Nov 29 03:10:08 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-e0b104d6-8283-44f5-a150-3606c697b81a tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Image pref 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:392}}
Nov 29 03:10:08 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-e0b104d6-8283-44f5-a150-3606c697b81a tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:430}}
Nov 29 03:10:08 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-e0b104d6-8283-44f5-a150-3606c697b81a tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:569}}
Nov 29 03:10:08 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-e0b104d6-8283-44f5-a150-3606c697b81a tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=75234) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:471}}
Nov 29 03:10:08 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-e0b104d6-8283-44f5-a150-3606c697b81a tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Got 1 possible topologies {{(pid=75234) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:501}}
Nov 29 03:10:08 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-e0b104d6-8283-44f5-a150-3606c697b81a tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:575}}
Nov 29 03:10:08 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-e0b104d6-8283-44f5-a150-3606c697b81a tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:577}}
Nov 29 03:10:08 user nova-compute[75234]: DEBUG nova.objects.instance [None req-e0b104d6-8283-44f5-a150-3606c697b81a tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Lazy-loading 'vcpu_model' on Instance uuid f609a7b7-1be2-44b4-9177-7e72a4d4c060 {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:10:08 user nova-compute[75234]: DEBUG nova.objects.base [None req-e0b104d6-8283-44f5-a150-3606c697b81a tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Object Instance<f609a7b7-1be2-44b4-9177-7e72a4d4c060> lazy-loaded attributes: flavor,info_cache,resources,numa_topology,vcpu_model {{(pid=75234) wrapper /opt/stack/nova/nova/objects/base.py:126}}
Nov 29 03:10:08 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-e0b104d6-8283-44f5-a150-3606c697b81a tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] [instance: f609a7b7-1be2-44b4-9177-7e72a4d4c060] Calling os-brick to attach iSCSI Volume {{(pid=75234) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Nov 29 03:10:08 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-e0b104d6-8283-44f5-a150-3606c697b81a tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f8a5050c9a0>, 'connection_properties': {'target_iqn': 'iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'frKqcBuE', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path'}}" {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:176}}
Nov 29 03:10:08 user nova-compute[75234]: WARNING os_brick.initiator.connectors.base [None req-e0b104d6-8283-44f5-a150-3606c697b81a tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Nov 29 03:10:08 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-e0b104d6-8283-44f5-a150-3606c697b81a tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:68}}
Nov 29 03:10:08 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-e0b104d6-8283-44f5-a150-3606c697b81a tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:73}}
Nov 29 03:10:08 user nova-compute[75234]: WARNING os_brick.initiator.connectors.base [None req-e0b104d6-8283-44f5-a150-3606c697b81a tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Nov 29 03:10:08 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-e0b104d6-8283-44f5-a150-3606c697b81a tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:68}}
Nov 29 03:10:08 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-e0b104d6-8283-44f5-a150-3606c697b81a tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:73}}
Nov 29 03:10:08 user nova-compute[75234]: INFO os_brick.initiator.connectors.iscsi [None req-e0b104d6-8283-44f5-a150-3606c697b81a tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Nov 29 03:10:08 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672 -p 172.16.0.220:3260 {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:10:08 user nova-compute[75234]: DEBUG nova.compute.provider_tree [None req-221919a0-9cdd-44ec-881f-d6c82383d31d tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Inventory has not changed in ProviderTree for provider: 4d291220-7c37-4839-a0e3-b23c66502b74 {{(pid=75234) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Nov 29 03:10:08 user nova-compute[75234]: DEBUG nova.scheduler.client.report [None req-221919a0-9cdd-44ec-881f-d6c82383d31d tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Inventory has not changed for provider 4d291220-7c37-4839-a0e3-b23c66502b74 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=75234) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Nov 29 03:10:08 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672 -p 172.16.0.220:3260" returned: 21 in 0.037s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:10:08 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[3a732fd2-5032-4bfe-8054-707bb61de179]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:10:08 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-e0b104d6-8283-44f5-a150-3606c697b81a tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Nov 29 03:10:08 user nova-compute[75234]:  {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:10:08 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672 -p 172.16.0.220:3260 --interface default --op new {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:10:08 user nova-compute[75234]: DEBUG nova.compute.manager [req-9afc4708-a975-4dc6-98fa-c90caf5e0ed6 req-8f02d7cb-8499-4fd9-97e0-dd7349eba545 service nova] [instance: 8297c264-a213-44ee-be46-5c08eabda9f7] Received event network-vif-unplugged-d26334d2-8040-4022-b658-69be1cf6a025 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:10:08 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-9afc4708-a975-4dc6-98fa-c90caf5e0ed6 req-8f02d7cb-8499-4fd9-97e0-dd7349eba545 service nova] Acquiring lock "8297c264-a213-44ee-be46-5c08eabda9f7-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:10:08 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-9afc4708-a975-4dc6-98fa-c90caf5e0ed6 req-8f02d7cb-8499-4fd9-97e0-dd7349eba545 service nova] Lock "8297c264-a213-44ee-be46-5c08eabda9f7-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:10:08 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-9afc4708-a975-4dc6-98fa-c90caf5e0ed6 req-8f02d7cb-8499-4fd9-97e0-dd7349eba545 service nova] Lock "8297c264-a213-44ee-be46-5c08eabda9f7-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:10:08 user nova-compute[75234]: DEBUG nova.compute.manager [req-9afc4708-a975-4dc6-98fa-c90caf5e0ed6 req-8f02d7cb-8499-4fd9-97e0-dd7349eba545 service nova] [instance: 8297c264-a213-44ee-be46-5c08eabda9f7] No waiting events found dispatching network-vif-unplugged-d26334d2-8040-4022-b658-69be1cf6a025 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:10:08 user nova-compute[75234]: WARNING nova.compute.manager [req-9afc4708-a975-4dc6-98fa-c90caf5e0ed6 req-8f02d7cb-8499-4fd9-97e0-dd7349eba545 service nova] [instance: 8297c264-a213-44ee-be46-5c08eabda9f7] Received unexpected event network-vif-unplugged-d26334d2-8040-4022-b658-69be1cf6a025 for instance with vm_state active and task_state rescuing.
Nov 29 03:10:08 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672 -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.020s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:10:08 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[14940f5b-f921-476b-bd59-b62257d03503]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672] added\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:10:08 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-e0b104d6-8283-44f5-a150-3606c697b81a tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672] added
Nov 29 03:10:08 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:10:08 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:10:08 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-221919a0-9cdd-44ec-881f-d6c82383d31d tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.resize_claim" :: held 0.633s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:10:08 user nova-compute[75234]: INFO nova.compute.manager [None req-221919a0-9cdd-44ec-881f-d6c82383d31d tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] [instance: a9350a18-14e6-42c6-a150-0a1487f20ad9] Migrating
Nov 29 03:10:08 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-221919a0-9cdd-44ec-881f-d6c82383d31d tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Acquiring lock "compute-rpcapi-router" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:10:08 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-221919a0-9cdd-44ec-881f-d6c82383d31d tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Acquired lock "compute-rpcapi-router" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:10:08 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.022s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:10:08 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[624a7815-2fd1-4120-b3c2-ec4aacafdf2c]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:10:08 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-e0b104d6-8283-44f5-a150-3606c697b81a tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:10:08 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:10:08 user nova-compute[75234]: INFO nova.compute.rpcapi [None req-221919a0-9cdd-44ec-881f-d6c82383d31d tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Automatically selected compute RPC version 6.2 from minimum service version 66
Nov 29 03:10:08 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-221919a0-9cdd-44ec-881f-d6c82383d31d tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Releasing lock "compute-rpcapi-router" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:10:08 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.028s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:10:08 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[00f1a9d9-8dcc-46aa-93c5-c0e84c9814be]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:10:08 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-e0b104d6-8283-44f5-a150-3606c697b81a tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:10:08 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v frKqcBuE {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:10:08 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v frKqcBuE" returned: 0 in 0.020s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:10:08 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[2cd0fca0-c6a2-47b9-8f68-47b3b5abf09d]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:10:08 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-e0b104d6-8283-44f5-a150-3606c697b81a tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'frKqcBuE'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:10:08 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:10:08 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-221919a0-9cdd-44ec-881f-d6c82383d31d tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Acquiring lock "refresh_cache-a9350a18-14e6-42c6-a150-0a1487f20ad9" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:10:08 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-221919a0-9cdd-44ec-881f-d6c82383d31d tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Acquired lock "refresh_cache-a9350a18-14e6-42c6-a150-0a1487f20ad9" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:10:08 user nova-compute[75234]: DEBUG nova.network.neutron [None req-221919a0-9cdd-44ec-881f-d6c82383d31d tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] [instance: a9350a18-14e6-42c6-a150-0a1487f20ad9] Building network info cache for instance {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2009}}
Nov 29 03:10:08 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.021s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:10:08 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[0b5cd89c-ff75-4688-bafb-2fbfd22f0f7a]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:10:08 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-e0b104d6-8283-44f5-a150-3606c697b81a tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:10:08 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:10:08 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.027s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:10:08 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[bbc5a337-4b11-4daa-b8de-c1e81582a39e]: (4, ('tcp: [12] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ada91c26-ca97-4b8b-98bb-a310d0627691 (non-flash)\ntcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b (non-flash)\ntcp: [15] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a7ef35bf-017a-4fd9-9182-79c1b2ab4f98 (non-flash)\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:10:08 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-e0b104d6-8283-44f5-a150-3606c697b81a tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [12] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ada91c26-ca97-4b8b-98bb-a310d0627691 (non-flash)
Nov 29 03:10:08 user nova-compute[75234]: tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b (non-flash)
Nov 29 03:10:08 user nova-compute[75234]: tcp: [15] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a7ef35bf-017a-4fd9-9182-79c1b2ab4f98 (non-flash)
Nov 29 03:10:08 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:10:08 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-e0b104d6-8283-44f5-a150-3606c697b81a tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] iscsi session list stdout=tcp: [12] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ada91c26-ca97-4b8b-98bb-a310d0627691 (non-flash)
Nov 29 03:10:08 user nova-compute[75234]: tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b (non-flash)
Nov 29 03:10:08 user nova-compute[75234]: tcp: [15] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a7ef35bf-017a-4fd9-9182-79c1b2ab4f98 (non-flash)
Nov 29 03:10:08 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsi_session /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Nov 29 03:10:08 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672 -p 172.16.0.220:3260 --login {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:10:08 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672 -p 172.16.0.220:3260 --login" returned: 0 in 0.082s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:10:08 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[4752b6c7-b336-4216-b34c-6081287335ca]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:10:08 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-e0b104d6-8283-44f5-a150-3606c697b81a tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672, portal: 172.16.0.220,3260]
Nov 29 03:10:08 user nova-compute[75234]: Login to [iface: default, target: iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672, portal: 172.16.0.220,3260] successful.
Nov 29 03:10:08 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:10:08 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672 -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:10:08 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672 -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.027s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:10:08 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[8f12773b-0fe1-4911-8e9d-5f3da0bc4429]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:10:08 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-e0b104d6-8283-44f5-a150-3606c697b81a tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:10:08 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:10:08 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:10:08 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.030s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:10:08 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[63d97b6d-dc91-4b5f-b25c-84132935c3b3]: (4, ('tcp: [12] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ada91c26-ca97-4b8b-98bb-a310d0627691 (non-flash)\ntcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b (non-flash)\ntcp: [15] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a7ef35bf-017a-4fd9-9182-79c1b2ab4f98 (non-flash)\ntcp: [16] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672 (non-flash)\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:10:08 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-e0b104d6-8283-44f5-a150-3606c697b81a tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [12] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ada91c26-ca97-4b8b-98bb-a310d0627691 (non-flash)
Nov 29 03:10:08 user nova-compute[75234]: tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b (non-flash)
Nov 29 03:10:08 user nova-compute[75234]: tcp: [15] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a7ef35bf-017a-4fd9-9182-79c1b2ab4f98 (non-flash)
Nov 29 03:10:08 user nova-compute[75234]: tcp: [16] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672 (non-flash)
Nov 29 03:10:08 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:10:08 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-e0b104d6-8283-44f5-a150-3606c697b81a tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] iscsi session list stdout=tcp: [12] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ada91c26-ca97-4b8b-98bb-a310d0627691 (non-flash)
Nov 29 03:10:08 user nova-compute[75234]: tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b (non-flash)
Nov 29 03:10:08 user nova-compute[75234]: tcp: [15] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a7ef35bf-017a-4fd9-9182-79c1b2ab4f98 (non-flash)
Nov 29 03:10:08 user nova-compute[75234]: tcp: [16] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672 (non-flash)
Nov 29 03:10:08 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsi_session /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Nov 29 03:10:08 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-e0b104d6-8283-44f5-a150-3606c697b81a tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.382s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:87}}
Nov 29 03:10:08 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-e0b104d6-8283-44f5-a150-3606c697b81a tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Connected to 172.16.0.220:3260 {{(pid=75234) _connect_vol /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:644}}
Nov 29 03:10:08 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-e0b104d6-8283-44f5-a150-3606c697b81a tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] HCTL ('36', '-', '-', 0) found on session 16 with lun 0 {{(pid=75234) get_hctl /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:770}}
Nov 29 03:10:08 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-e0b104d6-8283-44f5-a150-3606c697b81a tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Scanning host 36 c: -, t: -, l: 0) {{(pid=75234) scan_iscsi /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:795}}
Nov 29 03:10:08 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:10:08 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host36/scan {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:10:08 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host36/scan" returned: 0 in 0.021s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:10:08 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[c5b418f8-cbf1-47ae-9e3f-38f4c8dca4af]: (4, ('- - 0', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:10:08 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-e0b104d6-8283-44f5-a150-3606c697b81a tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Searching for a device in session 16 and hctl ['36', '*', '*', 0] yield: sde {{(pid=75234) device_name_by_hctl /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:789}}
Nov 29 03:10:08 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-e0b104d6-8283-44f5-a150-3606c697b81a tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Connected to sde using {'target_iqn': 'iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'frKqcBuE', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path'} {{(pid=75234) _connect_vol /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:672}}
Nov 29 03:10:08 user nova-compute[75234]: INFO nova.virt.libvirt.driver [-] [instance: 8297c264-a213-44ee-be46-5c08eabda9f7] Instance destroyed successfully.
Nov 29 03:10:08 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-413e3555-a1c6-47b1-9af6-79aba9489343 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] [instance: 8297c264-a213-44ee-be46-5c08eabda9f7] Attempting a stable device rescue
Nov 29 03:10:08 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-413e3555-a1c6-47b1-9af6-79aba9489343 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] rescue generated disk_info: {'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, '/dev/vdb': {'bus': 'virtio', 'dev': 'vdb', 'type': 'disk'}, 'disk.rescue': {'bus': 'virtio', 'dev': 'vdc', 'type': 'disk'}}} {{(pid=75234) rescue /opt/stack/nova/nova/virt/libvirt/driver.py:4318}}
Nov 29 03:10:08 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-413e3555-a1c6-47b1-9af6-79aba9489343 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] [instance: 8297c264-a213-44ee-be46-5c08eabda9f7] Instance directory exists: not creating {{(pid=75234) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4723}}
Nov 29 03:10:08 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-413e3555-a1c6-47b1-9af6-79aba9489343 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] [instance: 8297c264-a213-44ee-be46-5c08eabda9f7] Creating image(s)
Nov 29 03:10:08 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-413e3555-a1c6-47b1-9af6-79aba9489343 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] Acquiring lock "/opt/stack/data/nova/instances/8297c264-a213-44ee-be46-5c08eabda9f7/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:10:08 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-413e3555-a1c6-47b1-9af6-79aba9489343 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] Lock "/opt/stack/data/nova/instances/8297c264-a213-44ee-be46-5c08eabda9f7/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:10:08 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-413e3555-a1c6-47b1-9af6-79aba9489343 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] Lock "/opt/stack/data/nova/instances/8297c264-a213-44ee-be46-5c08eabda9f7/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.005s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:10:08 user nova-compute[75234]: DEBUG nova.objects.instance [None req-413e3555-a1c6-47b1-9af6-79aba9489343 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] Lazy-loading 'trusted_certs' on Instance uuid 8297c264-a213-44ee-be46-5c08eabda9f7 {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:10:08 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-413e3555-a1c6-47b1-9af6-79aba9489343 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] Acquiring lock "10d9c8f4c9d1fbded8da4c61ee5f1f19a245d0ae" by "nova.virt.libvirt.imagebackend.Image.cache.<locals>.fetch_func_sync" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:10:08 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-413e3555-a1c6-47b1-9af6-79aba9489343 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] Lock "10d9c8f4c9d1fbded8da4c61ee5f1f19a245d0ae" acquired by "nova.virt.libvirt.imagebackend.Image.cache.<locals>.fetch_func_sync" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:10:09 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-413e3555-a1c6-47b1-9af6-79aba9489343 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/10d9c8f4c9d1fbded8da4c61ee5f1f19a245d0ae.part --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:10:09 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-e0b104d6-8283-44f5-a150-3606c697b81a tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.618s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:87}}
Nov 29 03:10:09 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-e0b104d6-8283-44f5-a150-3606c697b81a tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] <== connect_volume: return (1621ms) {'type': 'block', 'scsi_wwn': '23865363536313666', 'path': '/dev/sde'} {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:203}}
Nov 29 03:10:09 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-e0b104d6-8283-44f5-a150-3606c697b81a tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] [instance: f609a7b7-1be2-44b4-9177-7e72a4d4c060] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '23865363536313666', 'path': '/dev/sde'} {{(pid=75234) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Nov 29 03:10:09 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:10:09 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-e0b104d6-8283-44f5-a150-3606c697b81a tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-11-29T03:06:06Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-AttachVolumeTestJSON-server-394712256',display_name='tempest-AttachVolumeTestJSON-server-394712256',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-attachvolumetestjson-server-394712256',id=4,image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBGphX+MWMhY0YZ/wlOvEQFKFHTIX4g9KMhxkTe+ErHeVHCk7d+KpOlTDVK9edUWFFxjgJKboch3pp1xrAYGa/rIEWyUV4YOfCGPFoAeneAuweb6ekmwg4qgLxfML0aXASQ==',key_name='tempest-keypair-213174705',keypairs=<?>,launch_index=0,launched_at=2023-11-29T03:06:23Z,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=4,progress=0,project_id='e393d23e783b4a40986674d782606c69',ramdisk_id='',reservation_id='r-9o6ggnbe',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=<?>,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-AttachVolumeTestJSON-1911423306',owner_user_name='tempest-AttachVolumeTestJSON-1911423306-project-member'},tags=<?>,task_state='powering-on',terminated_at=None,trusted_certs=<?>,updated_at=2023-11-29T03:08:16Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='bb1126c998ee430baac2e2494f30c0aa',uuid=f609a7b7-1be2-44b4-9177-7e72a4d4c060,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='stopped') vif={"id": "5543f379-2b3a-4dc2-bfe9-52f66a942257", "address": "fa:16:3e:04:ae:43", "network": {"id": "294c3aac-8c9b-45f0-8c0a-344e2d7d8b7c", "bridge": "br-int", "label": "tempest-AttachVolumeTestJSON-700098935-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.14", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "e393d23e783b4a40986674d782606c69", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap5543f379-2b", "ovs_interfaceid": "5543f379-2b3a-4dc2-bfe9-52f66a942257", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=75234) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Nov 29 03:10:09 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-e0b104d6-8283-44f5-a150-3606c697b81a tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Converting VIF {"id": "5543f379-2b3a-4dc2-bfe9-52f66a942257", "address": "fa:16:3e:04:ae:43", "network": {"id": "294c3aac-8c9b-45f0-8c0a-344e2d7d8b7c", "bridge": "br-int", "label": "tempest-AttachVolumeTestJSON-700098935-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.14", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "e393d23e783b4a40986674d782606c69", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap5543f379-2b", "ovs_interfaceid": "5543f379-2b3a-4dc2-bfe9-52f66a942257", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:10:09 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-e0b104d6-8283-44f5-a150-3606c697b81a tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:04:ae:43,bridge_name='br-int',has_traffic_filtering=True,id=5543f379-2b3a-4dc2-bfe9-52f66a942257,network=Network(294c3aac-8c9b-45f0-8c0a-344e2d7d8b7c),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap5543f379-2b') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:10:09 user nova-compute[75234]: DEBUG nova.objects.instance [None req-e0b104d6-8283-44f5-a150-3606c697b81a tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Lazy-loading 'pci_devices' on Instance uuid f609a7b7-1be2-44b4-9177-7e72a4d4c060 {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:10:09 user nova-compute[75234]: DEBUG nova.objects.base [None req-e0b104d6-8283-44f5-a150-3606c697b81a tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Object Instance<f609a7b7-1be2-44b4-9177-7e72a4d4c060> lazy-loaded attributes: flavor,info_cache,resources,numa_topology,vcpu_model,pci_devices {{(pid=75234) wrapper /opt/stack/nova/nova/objects/base.py:126}}
Nov 29 03:10:09 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-e0b104d6-8283-44f5-a150-3606c697b81a tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] [instance: f609a7b7-1be2-44b4-9177-7e72a4d4c060] End _get_guest_xml xml=<domain type="kvm">
Nov 29 03:10:09 user nova-compute[75234]:   <uuid>f609a7b7-1be2-44b4-9177-7e72a4d4c060</uuid>
Nov 29 03:10:09 user nova-compute[75234]:   <name>instance-00000004</name>
Nov 29 03:10:09 user nova-compute[75234]:   <memory>524288</memory>
Nov 29 03:10:09 user nova-compute[75234]:   <vcpu>1</vcpu>
Nov 29 03:10:09 user nova-compute[75234]:   <metadata>
Nov 29 03:10:09 user nova-compute[75234]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Nov 29 03:10:09 user nova-compute[75234]:       <nova:package version="0.0.0"/>
Nov 29 03:10:09 user nova-compute[75234]:       <nova:name>tempest-AttachVolumeTestJSON-server-394712256</nova:name>
Nov 29 03:10:09 user nova-compute[75234]:       <nova:creationTime>2023-11-29 03:10:08</nova:creationTime>
Nov 29 03:10:09 user nova-compute[75234]:       <nova:flavor name="m1.tiny">
Nov 29 03:10:09 user nova-compute[75234]:         <nova:memory>512</nova:memory>
Nov 29 03:10:09 user nova-compute[75234]:         <nova:disk>1</nova:disk>
Nov 29 03:10:09 user nova-compute[75234]:         <nova:swap>0</nova:swap>
Nov 29 03:10:09 user nova-compute[75234]:         <nova:ephemeral>0</nova:ephemeral>
Nov 29 03:10:09 user nova-compute[75234]:         <nova:vcpus>1</nova:vcpus>
Nov 29 03:10:09 user nova-compute[75234]:       </nova:flavor>
Nov 29 03:10:09 user nova-compute[75234]:       <nova:owner>
Nov 29 03:10:09 user nova-compute[75234]:         <nova:user uuid="bb1126c998ee430baac2e2494f30c0aa">tempest-AttachVolumeTestJSON-1911423306-project-member</nova:user>
Nov 29 03:10:09 user nova-compute[75234]:         <nova:project uuid="e393d23e783b4a40986674d782606c69">tempest-AttachVolumeTestJSON-1911423306</nova:project>
Nov 29 03:10:09 user nova-compute[75234]:       </nova:owner>
Nov 29 03:10:09 user nova-compute[75234]:       <nova:root type="image" uuid="96c30ba7-9b73-4a30-8ba4-7138879573b7"/>
Nov 29 03:10:09 user nova-compute[75234]:       <nova:ports>
Nov 29 03:10:09 user nova-compute[75234]:         <nova:port uuid="5543f379-2b3a-4dc2-bfe9-52f66a942257">
Nov 29 03:10:09 user nova-compute[75234]:           <nova:ip type="fixed" address="10.0.0.3" ipVersion="4"/>
Nov 29 03:10:09 user nova-compute[75234]:         </nova:port>
Nov 29 03:10:09 user nova-compute[75234]:       </nova:ports>
Nov 29 03:10:09 user nova-compute[75234]:     </nova:instance>
Nov 29 03:10:09 user nova-compute[75234]:   </metadata>
Nov 29 03:10:09 user nova-compute[75234]:   <sysinfo type="smbios">
Nov 29 03:10:09 user nova-compute[75234]:     <system>
Nov 29 03:10:09 user nova-compute[75234]:       <entry name="manufacturer">OpenStack Foundation</entry>
Nov 29 03:10:09 user nova-compute[75234]:       <entry name="product">OpenStack Nova</entry>
Nov 29 03:10:09 user nova-compute[75234]:       <entry name="version">0.0.0</entry>
Nov 29 03:10:09 user nova-compute[75234]:       <entry name="serial">f609a7b7-1be2-44b4-9177-7e72a4d4c060</entry>
Nov 29 03:10:09 user nova-compute[75234]:       <entry name="uuid">f609a7b7-1be2-44b4-9177-7e72a4d4c060</entry>
Nov 29 03:10:09 user nova-compute[75234]:       <entry name="family">Virtual Machine</entry>
Nov 29 03:10:09 user nova-compute[75234]:     </system>
Nov 29 03:10:09 user nova-compute[75234]:   </sysinfo>
Nov 29 03:10:09 user nova-compute[75234]:   <os>
Nov 29 03:10:09 user nova-compute[75234]:     <type arch="x86_64" machine="pc">hvm</type>
Nov 29 03:10:09 user nova-compute[75234]:     <boot dev="hd"/>
Nov 29 03:10:09 user nova-compute[75234]:     <smbios mode="sysinfo"/>
Nov 29 03:10:09 user nova-compute[75234]:   </os>
Nov 29 03:10:09 user nova-compute[75234]:   <features>
Nov 29 03:10:09 user nova-compute[75234]:     <acpi/>
Nov 29 03:10:09 user nova-compute[75234]:     <apic/>
Nov 29 03:10:09 user nova-compute[75234]:     <vmcoreinfo/>
Nov 29 03:10:09 user nova-compute[75234]:   </features>
Nov 29 03:10:09 user nova-compute[75234]:   <clock offset="utc">
Nov 29 03:10:09 user nova-compute[75234]:     <timer name="pit" tickpolicy="delay"/>
Nov 29 03:10:09 user nova-compute[75234]:     <timer name="rtc" tickpolicy="catchup"/>
Nov 29 03:10:09 user nova-compute[75234]:     <timer name="hpet" present="no"/>
Nov 29 03:10:09 user nova-compute[75234]:   </clock>
Nov 29 03:10:09 user nova-compute[75234]:   <cpu mode="custom" match="exact">
Nov 29 03:10:09 user nova-compute[75234]:     <model>Nehalem</model>
Nov 29 03:10:09 user nova-compute[75234]:     <topology sockets="1" cores="1" threads="1"/>
Nov 29 03:10:09 user nova-compute[75234]:   </cpu>
Nov 29 03:10:09 user nova-compute[75234]:   <devices>
Nov 29 03:10:09 user nova-compute[75234]:     <disk type="file" device="disk">
Nov 29 03:10:09 user nova-compute[75234]:       <driver name="qemu" type="qcow2" cache="none"/>
Nov 29 03:10:09 user nova-compute[75234]:       <source file="/opt/stack/data/nova/instances/f609a7b7-1be2-44b4-9177-7e72a4d4c060/disk"/>
Nov 29 03:10:09 user nova-compute[75234]:       <target dev="vda" bus="virtio"/>
Nov 29 03:10:09 user nova-compute[75234]:     </disk>
Nov 29 03:10:09 user nova-compute[75234]:     <disk type="block" device="disk">
Nov 29 03:10:09 user nova-compute[75234]:       <driver name="qemu" type="raw" cache="none" io="native"/>
Nov 29 03:10:09 user nova-compute[75234]:       <source dev="/dev/sde"/>
Nov 29 03:10:09 user nova-compute[75234]:       <target dev="vdb" bus="virtio"/>
Nov 29 03:10:09 user nova-compute[75234]:       <serial>33601b89-9deb-4aa4-bd5b-1654f1237672</serial>
Nov 29 03:10:09 user nova-compute[75234]:     </disk>
Nov 29 03:10:09 user nova-compute[75234]:     <interface type="ethernet">
Nov 29 03:10:09 user nova-compute[75234]:       <mac address="fa:16:3e:04:ae:43"/>
Nov 29 03:10:09 user nova-compute[75234]:       <model type="virtio"/>
Nov 29 03:10:09 user nova-compute[75234]:       <mtu size="1442"/>
Nov 29 03:10:09 user nova-compute[75234]:       <target dev="tap5543f379-2b"/>
Nov 29 03:10:09 user nova-compute[75234]:     </interface>
Nov 29 03:10:09 user nova-compute[75234]:     <serial type="pty">
Nov 29 03:10:09 user nova-compute[75234]:       <log file="/opt/stack/data/nova/instances/f609a7b7-1be2-44b4-9177-7e72a4d4c060/console.log" append="off"/>
Nov 29 03:10:09 user nova-compute[75234]:     </serial>
Nov 29 03:10:09 user nova-compute[75234]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Nov 29 03:10:09 user nova-compute[75234]:     <video>
Nov 29 03:10:09 user nova-compute[75234]:       <model type="virtio"/>
Nov 29 03:10:09 user nova-compute[75234]:     </video>
Nov 29 03:10:09 user nova-compute[75234]:     <rng model="virtio">
Nov 29 03:10:09 user nova-compute[75234]:       <backend model="random">/dev/urandom</backend>
Nov 29 03:10:09 user nova-compute[75234]:     </rng>
Nov 29 03:10:09 user nova-compute[75234]:     <controller type="usb" index="0" model="none"/>
Nov 29 03:10:09 user nova-compute[75234]:     <memballoon model="virtio">
Nov 29 03:10:09 user nova-compute[75234]:       <stats period="10"/>
Nov 29 03:10:09 user nova-compute[75234]:     </memballoon>
Nov 29 03:10:09 user nova-compute[75234]:   </devices>
Nov 29 03:10:09 user nova-compute[75234]: </domain>
Nov 29 03:10:09 user nova-compute[75234]:  {{(pid=75234) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7568}}
Nov 29 03:10:09 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-e0b104d6-8283-44f5-a150-3606c697b81a tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f609a7b7-1be2-44b4-9177-7e72a4d4c060/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:10:09 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-413e3555-a1c6-47b1-9af6-79aba9489343 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/10d9c8f4c9d1fbded8da4c61ee5f1f19a245d0ae.part --force-share --output=json" returned: 0 in 0.240s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:10:09 user nova-compute[75234]: DEBUG nova.virt.images [None req-413e3555-a1c6-47b1-9af6-79aba9489343 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] 86e6a165-1d2a-4d4f-8ab8-66bf88539337 was qcow2, converting to raw {{(pid=75234) fetch_to_raw /opt/stack/nova/nova/virt/images.py:165}}
Nov 29 03:10:09 user nova-compute[75234]: DEBUG nova.privsep.utils [None req-413e3555-a1c6-47b1-9af6-79aba9489343 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] Path '/opt/stack/data/nova/instances' supports direct I/O {{(pid=75234) supports_direct_io /opt/stack/nova/nova/privsep/utils.py:63}}
Nov 29 03:10:09 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-413e3555-a1c6-47b1-9af6-79aba9489343 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] Running cmd (subprocess): qemu-img convert -t none -O raw -f qcow2 /opt/stack/data/nova/instances/_base/10d9c8f4c9d1fbded8da4c61ee5f1f19a245d0ae.part /opt/stack/data/nova/instances/_base/10d9c8f4c9d1fbded8da4c61ee5f1f19a245d0ae.converted {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:10:10 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-e0b104d6-8283-44f5-a150-3606c697b81a tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f609a7b7-1be2-44b4-9177-7e72a4d4c060/disk --force-share --output=json" returned: 0 in 0.199s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:10:10 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-e0b104d6-8283-44f5-a150-3606c697b81a tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f609a7b7-1be2-44b4-9177-7e72a4d4c060/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:10:10 user nova-compute[75234]: DEBUG nova.compute.manager [req-101f2a29-922e-4b5d-b163-cb841f275238 req-1cdd4211-48a6-4863-a489-3bb82b0840cb service nova] [instance: 8297c264-a213-44ee-be46-5c08eabda9f7] Received event network-vif-plugged-d26334d2-8040-4022-b658-69be1cf6a025 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:10:10 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-101f2a29-922e-4b5d-b163-cb841f275238 req-1cdd4211-48a6-4863-a489-3bb82b0840cb service nova] Acquiring lock "8297c264-a213-44ee-be46-5c08eabda9f7-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:10:10 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-101f2a29-922e-4b5d-b163-cb841f275238 req-1cdd4211-48a6-4863-a489-3bb82b0840cb service nova] Lock "8297c264-a213-44ee-be46-5c08eabda9f7-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.002s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:10:10 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-101f2a29-922e-4b5d-b163-cb841f275238 req-1cdd4211-48a6-4863-a489-3bb82b0840cb service nova] Lock "8297c264-a213-44ee-be46-5c08eabda9f7-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:10:10 user nova-compute[75234]: DEBUG nova.compute.manager [req-101f2a29-922e-4b5d-b163-cb841f275238 req-1cdd4211-48a6-4863-a489-3bb82b0840cb service nova] [instance: 8297c264-a213-44ee-be46-5c08eabda9f7] No waiting events found dispatching network-vif-plugged-d26334d2-8040-4022-b658-69be1cf6a025 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:10:10 user nova-compute[75234]: WARNING nova.compute.manager [req-101f2a29-922e-4b5d-b163-cb841f275238 req-1cdd4211-48a6-4863-a489-3bb82b0840cb service nova] [instance: 8297c264-a213-44ee-be46-5c08eabda9f7] Received unexpected event network-vif-plugged-d26334d2-8040-4022-b658-69be1cf6a025 for instance with vm_state active and task_state rescuing.
Nov 29 03:10:10 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-e0b104d6-8283-44f5-a150-3606c697b81a tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f609a7b7-1be2-44b4-9177-7e72a4d4c060/disk --force-share --output=json" returned: 0 in 0.197s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:10:10 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-e0b104d6-8283-44f5-a150-3606c697b81a tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] skipping disk /dev/sde (vdb) as it is a volume {{(pid=75234) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11213}}
Nov 29 03:10:10 user nova-compute[75234]: DEBUG nova.objects.instance [None req-e0b104d6-8283-44f5-a150-3606c697b81a tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Lazy-loading 'trusted_certs' on Instance uuid f609a7b7-1be2-44b4-9177-7e72a4d4c060 {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:10:10 user nova-compute[75234]: DEBUG nova.objects.base [None req-e0b104d6-8283-44f5-a150-3606c697b81a tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Object Instance<f609a7b7-1be2-44b4-9177-7e72a4d4c060> lazy-loaded attributes: flavor,info_cache,resources,numa_topology,vcpu_model,pci_devices,trusted_certs {{(pid=75234) wrapper /opt/stack/nova/nova/objects/base.py:126}}
Nov 29 03:10:10 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-e0b104d6-8283-44f5-a150-3606c697b81a tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:10:10 user nova-compute[75234]: DEBUG nova.network.neutron [None req-221919a0-9cdd-44ec-881f-d6c82383d31d tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] [instance: a9350a18-14e6-42c6-a150-0a1487f20ad9] Updating instance_info_cache with network_info: [{"id": "52024efc-8cde-4610-b12c-198f19d75ccf", "address": "fa:16:3e:09:b8:70", "network": {"id": "31c4ca3b-3a2f-47df-87fa-4ad095ee5cf9", "bridge": "br-int", "label": "tempest-ServerActionsTestOtherB-1680745008-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.9", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "ad5ea22c63f04f91a3f2fa665e4fc8a1", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap52024efc-8c", "ovs_interfaceid": "52024efc-8cde-4610-b12c-198f19d75ccf", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:10:10 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-413e3555-a1c6-47b1-9af6-79aba9489343 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] CMD "qemu-img convert -t none -O raw -f qcow2 /opt/stack/data/nova/instances/_base/10d9c8f4c9d1fbded8da4c61ee5f1f19a245d0ae.part /opt/stack/data/nova/instances/_base/10d9c8f4c9d1fbded8da4c61ee5f1f19a245d0ae.converted" returned: 0 in 0.602s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:10:10 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-413e3555-a1c6-47b1-9af6-79aba9489343 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/10d9c8f4c9d1fbded8da4c61ee5f1f19a245d0ae.converted --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:10:10 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-221919a0-9cdd-44ec-881f-d6c82383d31d tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Releasing lock "refresh_cache-a9350a18-14e6-42c6-a150-0a1487f20ad9" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:10:10 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-e0b104d6-8283-44f5-a150-3606c697b81a tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json" returned: 0 in 0.251s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:10:10 user nova-compute[75234]: DEBUG nova.virt.disk.api [None req-e0b104d6-8283-44f5-a150-3606c697b81a tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Checking if we can resize image /opt/stack/data/nova/instances/f609a7b7-1be2-44b4-9177-7e72a4d4c060/disk. size=1073741824 {{(pid=75234) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:166}}
Nov 29 03:10:10 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-e0b104d6-8283-44f5-a150-3606c697b81a tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f609a7b7-1be2-44b4-9177-7e72a4d4c060/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:10:10 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-413e3555-a1c6-47b1-9af6-79aba9489343 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/10d9c8f4c9d1fbded8da4c61ee5f1f19a245d0ae.converted --force-share --output=json" returned: 0 in 0.200s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:10:10 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-413e3555-a1c6-47b1-9af6-79aba9489343 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] Lock "10d9c8f4c9d1fbded8da4c61ee5f1f19a245d0ae" "released" by "nova.virt.libvirt.imagebackend.Image.cache.<locals>.fetch_func_sync" :: held 2.096s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:10:10 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-413e3555-a1c6-47b1-9af6-79aba9489343 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] Acquiring lock "10d9c8f4c9d1fbded8da4c61ee5f1f19a245d0ae" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:10:10 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-413e3555-a1c6-47b1-9af6-79aba9489343 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] Lock "10d9c8f4c9d1fbded8da4c61ee5f1f19a245d0ae" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:10:10 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-413e3555-a1c6-47b1-9af6-79aba9489343 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/10d9c8f4c9d1fbded8da4c61ee5f1f19a245d0ae --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:10:10 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-e0b104d6-8283-44f5-a150-3606c697b81a tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f609a7b7-1be2-44b4-9177-7e72a4d4c060/disk --force-share --output=json" returned: 0 in 0.219s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:10:10 user nova-compute[75234]: DEBUG nova.virt.disk.api [None req-e0b104d6-8283-44f5-a150-3606c697b81a tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Cannot resize image /opt/stack/data/nova/instances/f609a7b7-1be2-44b4-9177-7e72a4d4c060/disk to a smaller size. {{(pid=75234) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:172}}
Nov 29 03:10:10 user nova-compute[75234]: DEBUG nova.objects.instance [None req-e0b104d6-8283-44f5-a150-3606c697b81a tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Lazy-loading 'migration_context' on Instance uuid f609a7b7-1be2-44b4-9177-7e72a4d4c060 {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:10:10 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-221919a0-9cdd-44ec-881f-d6c82383d31d tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] [instance: a9350a18-14e6-42c6-a150-0a1487f20ad9] Starting migrate_disk_and_power_off {{(pid=75234) migrate_disk_and_power_off /opt/stack/nova/nova/virt/libvirt/driver.py:11484}}
Nov 29 03:10:10 user nova-compute[75234]: DEBUG nova.objects.base [None req-e0b104d6-8283-44f5-a150-3606c697b81a tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Object Instance<f609a7b7-1be2-44b4-9177-7e72a4d4c060> lazy-loaded attributes: flavor,info_cache,resources,numa_topology,vcpu_model,pci_devices,trusted_certs,migration_context {{(pid=75234) wrapper /opt/stack/nova/nova/objects/base.py:126}}
Nov 29 03:10:10 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-e0b104d6-8283-44f5-a150-3606c697b81a tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-11-29T03:06:06Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-AttachVolumeTestJSON-server-394712256',display_name='tempest-AttachVolumeTestJSON-server-394712256',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-attachvolumetestjson-server-394712256',id=4,image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBGphX+MWMhY0YZ/wlOvEQFKFHTIX4g9KMhxkTe+ErHeVHCk7d+KpOlTDVK9edUWFFxjgJKboch3pp1xrAYGa/rIEWyUV4YOfCGPFoAeneAuweb6ekmwg4qgLxfML0aXASQ==',key_name='tempest-keypair-213174705',keypairs=<?>,launch_index=0,launched_at=2023-11-29T03:06:23Z,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=<?>,power_state=4,progress=0,project_id='e393d23e783b4a40986674d782606c69',ramdisk_id='',reservation_id='r-9o6ggnbe',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=<?>,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-AttachVolumeTestJSON-1911423306',owner_user_name='tempest-AttachVolumeTestJSON-1911423306-project-member'},tags=<?>,task_state='powering-on',terminated_at=None,trusted_certs=None,updated_at=2023-11-29T03:08:16Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='bb1126c998ee430baac2e2494f30c0aa',uuid=f609a7b7-1be2-44b4-9177-7e72a4d4c060,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='stopped') vif={"id": "5543f379-2b3a-4dc2-bfe9-52f66a942257", "address": "fa:16:3e:04:ae:43", "network": {"id": "294c3aac-8c9b-45f0-8c0a-344e2d7d8b7c", "bridge": "br-int", "label": "tempest-AttachVolumeTestJSON-700098935-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.14", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "e393d23e783b4a40986674d782606c69", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap5543f379-2b", "ovs_interfaceid": "5543f379-2b3a-4dc2-bfe9-52f66a942257", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Nov 29 03:10:10 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-e0b104d6-8283-44f5-a150-3606c697b81a tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Converting VIF {"id": "5543f379-2b3a-4dc2-bfe9-52f66a942257", "address": "fa:16:3e:04:ae:43", "network": {"id": "294c3aac-8c9b-45f0-8c0a-344e2d7d8b7c", "bridge": "br-int", "label": "tempest-AttachVolumeTestJSON-700098935-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.14", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "e393d23e783b4a40986674d782606c69", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap5543f379-2b", "ovs_interfaceid": "5543f379-2b3a-4dc2-bfe9-52f66a942257", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:10:10 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-e0b104d6-8283-44f5-a150-3606c697b81a tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:04:ae:43,bridge_name='br-int',has_traffic_filtering=True,id=5543f379-2b3a-4dc2-bfe9-52f66a942257,network=Network(294c3aac-8c9b-45f0-8c0a-344e2d7d8b7c),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap5543f379-2b') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:10:10 user nova-compute[75234]: DEBUG os_vif [None req-e0b104d6-8283-44f5-a150-3606c697b81a tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:04:ae:43,bridge_name='br-int',has_traffic_filtering=True,id=5543f379-2b3a-4dc2-bfe9-52f66a942257,network=Network(294c3aac-8c9b-45f0-8c0a-344e2d7d8b7c),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap5543f379-2b') {{(pid=75234) plug /opt/stack/data/venv/lib/python3.10/site-packages/os_vif/__init__.py:76}}
Nov 29 03:10:10 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:10:10 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:10:10 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Nov 29 03:10:10 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:10:10 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbCreateCommand(_result=None, table=QoS, columns={'type': 'linux-noop', 'external_ids': {'id': 'abd56afa-ec17-518b-948a-b5862316fd9e', '_type': 'linux-noop'}}, row=False) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:10:10 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:10:10 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:10:10 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:10:10 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:10:10 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tap5543f379-2b, may_exist=True, interface_attrs={}) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:10:10 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Port, record=tap5543f379-2b, col_values=(('qos', UUID('de5d95e4-4f1c-4481-b7b3-c4df9ebda4e9')),), if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:10:10 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DbSetCommand(_result=None, table=Interface, record=tap5543f379-2b, col_values=(('external_ids', {'iface-id': '5543f379-2b3a-4dc2-bfe9-52f66a942257', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:04:ae:43', 'vm-uuid': 'f609a7b7-1be2-44b4-9177-7e72a4d4c060'}),), if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:10:10 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:10:10 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:10:10 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:10:10 user nova-compute[75234]: INFO os_vif [None req-e0b104d6-8283-44f5-a150-3606c697b81a tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:04:ae:43,bridge_name='br-int',has_traffic_filtering=True,id=5543f379-2b3a-4dc2-bfe9-52f66a942257,network=Network(294c3aac-8c9b-45f0-8c0a-344e2d7d8b7c),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap5543f379-2b')
Nov 29 03:10:10 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:10:10 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:10:11 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-413e3555-a1c6-47b1-9af6-79aba9489343 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/10d9c8f4c9d1fbded8da4c61ee5f1f19a245d0ae --force-share --output=json" returned: 0 in 0.249s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:10:11 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-413e3555-a1c6-47b1-9af6-79aba9489343 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/10d9c8f4c9d1fbded8da4c61ee5f1f19a245d0ae,backing_fmt=raw /opt/stack/data/nova/instances/8297c264-a213-44ee-be46-5c08eabda9f7/disk.rescue {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:10:11 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:10:11 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-413e3555-a1c6-47b1-9af6-79aba9489343 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/10d9c8f4c9d1fbded8da4c61ee5f1f19a245d0ae,backing_fmt=raw /opt/stack/data/nova/instances/8297c264-a213-44ee-be46-5c08eabda9f7/disk.rescue" returned: 0 in 0.079s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:10:11 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-413e3555-a1c6-47b1-9af6-79aba9489343 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] Lock "10d9c8f4c9d1fbded8da4c61ee5f1f19a245d0ae" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.335s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:10:11 user nova-compute[75234]: DEBUG nova.objects.instance [None req-413e3555-a1c6-47b1-9af6-79aba9489343 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] Lazy-loading 'migration_context' on Instance uuid 8297c264-a213-44ee-be46-5c08eabda9f7 {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:10:11 user nova-compute[75234]: DEBUG nova.objects.base [None req-413e3555-a1c6-47b1-9af6-79aba9489343 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] Object Instance<8297c264-a213-44ee-be46-5c08eabda9f7> lazy-loaded attributes: trusted_certs,migration_context {{(pid=75234) wrapper /opt/stack/nova/nova/objects/base.py:126}}
Nov 29 03:10:11 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-413e3555-a1c6-47b1-9af6-79aba9489343 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] [instance: 8297c264-a213-44ee-be46-5c08eabda9f7] Created local disks {{(pid=75234) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4861}}
Nov 29 03:10:11 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-413e3555-a1c6-47b1-9af6-79aba9489343 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] [instance: 8297c264-a213-44ee-be46-5c08eabda9f7] Start _get_guest_xml network_info=[{"id": "d26334d2-8040-4022-b658-69be1cf6a025", "address": "fa:16:3e:5f:9d:5a", "network": {"id": "e0366490-9fe2-4429-88c8-6625128e645d", "bridge": "br-int", "label": "tempest-ServerStableDeviceRescueTest-804259918-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.31", "type": "floating", "version": 4, "meta": {}}], "label": "tempest-ServerStableDeviceRescueTest-804259918-network", "vif_mac": "fa:16:3e:5f:9d:5a"}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "745bdb3c72794155829dc0a69f35b321", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapd26334d2-80", "ovs_interfaceid": "d26334d2-8040-4022-b658-69be1cf6a025", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, '/dev/vdb': {'bus': 'virtio', 'dev': 'vdb', 'type': 'disk'}, 'disk.rescue': {'bus': 'virtio', 'dev': 'vdc', 'type': 'disk'}}} image_meta=ImageMeta(checksum='c8fc807773e5354afe61636071771906',container_format='bare',created_at=2023-11-29T03:02:15Z,direct_url=<?>,disk_format='qcow2',id=96c30ba7-9b73-4a30-8ba4-7138879573b7,min_disk=0,min_ram=0,name='cirros-0.6.2-x86_64-disk',owner='b18e380921c748b78517b6f1211cd0f1',properties=ImageMetaProps,protected=<?>,size=21430272,status='active',tags=<?>,updated_at=2023-11-29T03:02:17Z,virtual_size=<?>,visibility=<?>) rescue={'image_id': '86e6a165-1d2a-4d4f-8ab8-66bf88539337', 'kernel_id': '', 'ramdisk_id': ''} block_device_info={'root_device_name': '/dev/vda', 'image': [{'device_name': '/dev/vda', 'encryption_format': None, 'encrypted': False, 'device_type': 'disk', 'size': 0, 'disk_bus': 'virtio', 'boot_index': 0, 'encryption_options': None, 'encryption_secret_uuid': None, 'guest_format': None, 'image_id': '96c30ba7-9b73-4a30-8ba4-7138879573b7'}], 'ephemerals': [], 'block_device_mapping': [{'mount_device': '/dev/vdb', 'attachment_id': 'eb51d5aa-7828-4fae-beb6-6698a6b605ab', 'disk_bus': 'virtio', 'device_type': 'disk', 'boot_index': None, 'guest_format': None, 'delete_on_termination': False, 'connection_info': {'driver_volume_type': 'iscsi', 'data': {'target_iqn': 'iqn.2010-10.org.openstack:a7ef35bf-017a-4fd9-9182-79c1b2ab4f98', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'iSOJqDFN', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdd'}, 'status': 'reserved', 'instance': '8297c264-a213-44ee-be46-5c08eabda9f7', 'attached_at': '', 'detached_at': '', 'volume_id': 'a7ef35bf-017a-4fd9-9182-79c1b2ab4f98', 'serial': 'a7ef35bf-017a-4fd9-9182-79c1b2ab4f98'}, 'volume_type': None} {{(pid=75234) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7562}}
Nov 29 03:10:11 user nova-compute[75234]: DEBUG nova.objects.instance [None req-413e3555-a1c6-47b1-9af6-79aba9489343 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] Lazy-loading 'resources' on Instance uuid 8297c264-a213-44ee-be46-5c08eabda9f7 {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:10:11 user nova-compute[75234]: DEBUG nova.objects.base [None req-413e3555-a1c6-47b1-9af6-79aba9489343 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] Object Instance<8297c264-a213-44ee-be46-5c08eabda9f7> lazy-loaded attributes: trusted_certs,migration_context,resources {{(pid=75234) wrapper /opt/stack/nova/nova/objects/base.py:126}}
Nov 29 03:10:11 user nova-compute[75234]: DEBUG nova.objects.instance [None req-413e3555-a1c6-47b1-9af6-79aba9489343 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] Lazy-loading 'numa_topology' on Instance uuid 8297c264-a213-44ee-be46-5c08eabda9f7 {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:10:11 user nova-compute[75234]: DEBUG nova.objects.base [None req-413e3555-a1c6-47b1-9af6-79aba9489343 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] Object Instance<8297c264-a213-44ee-be46-5c08eabda9f7> lazy-loaded attributes: trusted_certs,migration_context,resources,numa_topology {{(pid=75234) wrapper /opt/stack/nova/nova/objects/base.py:126}}
Nov 29 03:10:11 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-413e3555-a1c6-47b1-9af6-79aba9489343 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:10:11 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-413e3555-a1c6-47b1-9af6-79aba9489343 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:10:11 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-413e3555-a1c6-47b1-9af6-79aba9489343 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] Searching host: 'user' for CPU controller through CGroups V1... {{(pid=75234) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1652}}
Nov 29 03:10:11 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-413e3555-a1c6-47b1-9af6-79aba9489343 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] CPU controller missing on host. {{(pid=75234) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1662}}
Nov 29 03:10:11 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-413e3555-a1c6-47b1-9af6-79aba9489343 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] Searching host: 'user' for CPU controller through CGroups V2... {{(pid=75234) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1671}}
Nov 29 03:10:11 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-413e3555-a1c6-47b1-9af6-79aba9489343 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] CPU controller found on host. {{(pid=75234) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1678}}
Nov 29 03:10:11 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-413e3555-a1c6-47b1-9af6-79aba9489343 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=75234) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5400}}
Nov 29 03:10:11 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-413e3555-a1c6-47b1-9af6-79aba9489343 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-11-29T03:02:04Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='1',id=6,is_public=True,memory_mb=512,name='m1.tiny',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='c8fc807773e5354afe61636071771906',container_format='bare',created_at=2023-11-29T03:02:15Z,direct_url=<?>,disk_format='qcow2',id=96c30ba7-9b73-4a30-8ba4-7138879573b7,min_disk=0,min_ram=0,name='cirros-0.6.2-x86_64-disk',owner='b18e380921c748b78517b6f1211cd0f1',properties=ImageMetaProps,protected=<?>,size=21430272,status='active',tags=<?>,updated_at=2023-11-29T03:02:17Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:563}}
Nov 29 03:10:11 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-413e3555-a1c6-47b1-9af6-79aba9489343 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] Flavor limits 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:348}}
Nov 29 03:10:11 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-413e3555-a1c6-47b1-9af6-79aba9489343 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] Image limits 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:352}}
Nov 29 03:10:11 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-413e3555-a1c6-47b1-9af6-79aba9489343 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] Flavor pref 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:388}}
Nov 29 03:10:11 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-413e3555-a1c6-47b1-9af6-79aba9489343 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] Image pref 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:392}}
Nov 29 03:10:11 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-413e3555-a1c6-47b1-9af6-79aba9489343 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:430}}
Nov 29 03:10:11 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-413e3555-a1c6-47b1-9af6-79aba9489343 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:569}}
Nov 29 03:10:11 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-413e3555-a1c6-47b1-9af6-79aba9489343 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=75234) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:471}}
Nov 29 03:10:11 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-413e3555-a1c6-47b1-9af6-79aba9489343 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] Got 1 possible topologies {{(pid=75234) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:501}}
Nov 29 03:10:11 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-413e3555-a1c6-47b1-9af6-79aba9489343 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:575}}
Nov 29 03:10:11 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-413e3555-a1c6-47b1-9af6-79aba9489343 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:577}}
Nov 29 03:10:11 user nova-compute[75234]: DEBUG nova.objects.instance [None req-413e3555-a1c6-47b1-9af6-79aba9489343 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] Lazy-loading 'vcpu_model' on Instance uuid 8297c264-a213-44ee-be46-5c08eabda9f7 {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:10:11 user nova-compute[75234]: DEBUG nova.objects.base [None req-413e3555-a1c6-47b1-9af6-79aba9489343 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] Object Instance<8297c264-a213-44ee-be46-5c08eabda9f7> lazy-loaded attributes: trusted_certs,migration_context,resources,numa_topology,vcpu_model {{(pid=75234) wrapper /opt/stack/nova/nova/objects/base.py:126}}
Nov 29 03:10:11 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-413e3555-a1c6-47b1-9af6-79aba9489343 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] [instance: 8297c264-a213-44ee-be46-5c08eabda9f7] Calling os-brick to attach iSCSI Volume {{(pid=75234) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Nov 29 03:10:11 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-413e3555-a1c6-47b1-9af6-79aba9489343 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f8a5050c9a0>, 'connection_properties': {'target_iqn': 'iqn.2010-10.org.openstack:a7ef35bf-017a-4fd9-9182-79c1b2ab4f98', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'iSOJqDFN', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path'}}" {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:176}}
Nov 29 03:10:11 user nova-compute[75234]: WARNING os_brick.initiator.connectors.base [None req-413e3555-a1c6-47b1-9af6-79aba9489343 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Nov 29 03:10:11 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-413e3555-a1c6-47b1-9af6-79aba9489343 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:68}}
Nov 29 03:10:11 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-413e3555-a1c6-47b1-9af6-79aba9489343 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.002s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:73}}
Nov 29 03:10:11 user nova-compute[75234]: WARNING os_brick.initiator.connectors.base [None req-413e3555-a1c6-47b1-9af6-79aba9489343 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Nov 29 03:10:11 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-413e3555-a1c6-47b1-9af6-79aba9489343 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:a7ef35bf-017a-4fd9-9182-79c1b2ab4f98" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:68}}
Nov 29 03:10:11 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-413e3555-a1c6-47b1-9af6-79aba9489343 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:a7ef35bf-017a-4fd9-9182-79c1b2ab4f98" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:73}}
Nov 29 03:10:11 user nova-compute[75234]: INFO os_brick.initiator.connectors.iscsi [None req-413e3555-a1c6-47b1-9af6-79aba9489343 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Nov 29 03:10:11 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:a7ef35bf-017a-4fd9-9182-79c1b2ab4f98 -p 172.16.0.220:3260 {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:10:11 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:a7ef35bf-017a-4fd9-9182-79c1b2ab4f98 -p 172.16.0.220:3260" returned: 0 in 0.018s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:10:11 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[be6eaff3-687f-4445-9de4-bd133aa83324]: (4, ('# BEGIN RECORD 2.1.5\nnode.name = iqn.2010-10.org.openstack:a7ef35bf-017a-4fd9-9182-79c1b2ab4f98\nnode.tpgt = -1\nnode.startup = automatic\nnode.leading_login = No\niface.iscsi_ifacename = default\niface.net_ifacename = <empty>\niface.ipaddress = <empty>\niface.prefix_len = 0\niface.hwaddress = <empty>\niface.transport_name = tcp\niface.initiatorname = <empty>\niface.state = <empty>\niface.vlan_id = 0\niface.vlan_priority = 0\niface.vlan_state = <empty>\niface.iface_num = 0\niface.mtu = 0\niface.port = 0\niface.bootproto = <empty>\niface.subnet_mask = <empty>\niface.gateway = <empty>\niface.dhcp_alt_client_id_state = <empty>\niface.dhcp_alt_client_id = <empty>\niface.dhcp_dns = <empty>\niface.dhcp_learn_iqn = <empty>\niface.dhcp_req_vendor_id_state = <empty>\niface.dhcp_vendor_id_state = <empty>\niface.dhcp_vendor_id = <empty>\niface.dhcp_slp_da = <empty>\niface.fragmentation = <empty>\niface.gratuitous_arp = <empty>\niface.incoming_forwarding = <empty>\niface.tos_state = <empty>\niface.tos = 0\niface.ttl = 0\niface.delayed_ack = <empty>\niface.tcp_nagle = <empty>\niface.tcp_wsf_state = <empty>\niface.tcp_wsf = 0\niface.tcp_timer_scale = 0\niface.tcp_timestamp = <empty>\niface.redirect = <empty>\niface.def_task_mgmt_timeout = 0\niface.header_digest = <empty>\niface.data_digest = <empty>\niface.immediate_data = <empty>\niface.initial_r2t = <empty>\niface.data_seq_inorder = <empty>\niface.data_pdu_inorder = <empty>\niface.erl = 0\niface.max_receive_data_len = 0\niface.first_burst_len = 0\niface.max_outstanding_r2t = 0\niface.max_burst_len = 0\niface.chap_auth = <empty>\niface.bidi_chap = <empty>\niface.strict_login_compliance = <empty>\niface.discovery_auth = <empty>\niface.discovery_logout = <empty>\nnode.discovery_address = <empty>\nnode.discovery_port = 0\nnode.discovery_type = static\nnode.session.initial_cmdsn = 0\nnode.session.initial_login_retry_max = 8\nnode.session.xmit_thread_priority = -20\nnode.session.cmds_max = 128\nnode.session.queue_depth = 32\nnode.session.nr_sessions = 1\nnode.session.auth.authmethod = CHAP\nnode.session.auth.username = iSOJqDFN\nnode.session.auth.password = ********\nnode.session.auth.username_in = <empty>\nnode.session.auth.password_in = <empty>\nnode.session.auth.chap_algs = MD5\nnode.session.timeo.replacement_timeout = 120\nnode.session.err_timeo.abort_timeout = 15\nnode.session.err_timeo.lu_reset_timeout = 30\nnode.session.err_timeo.tgt_reset_timeout = 30\nnode.session.err_timeo.host_reset_timeout = 60\nnode.session.iscsi.FastAbort = Yes\nnode.session.iscsi.InitialR2T = No\nnode.session.iscsi.ImmediateData = Yes\nnode.session.iscsi.FirstBurstLength = 262144\nnode.session.iscsi.MaxBurstLength = 16776192\nnode.session.iscsi.DefaultTime2Retain = 0\nnode.session.iscsi.DefaultTime2Wait = 2\nnode.session.iscsi.MaxConnections = 1\nnode.session.iscsi.MaxOutstandingR2T = 1\nnode.session.iscsi.ERL = 0\nnode.session.scan = manual\nnode.session.reopen_max = 0\nnode.conn[0].address = 172.16.0.220\nnode.conn[0].port = 3260\nnode.conn[0].startup = manual\nnode.conn[0].tcp.window_size = 524288\nnode.conn[0].tcp.type_of_service = 0\nnode.conn[0].timeo.logout_timeout = 15\nnode.conn[0].timeo.login_timeout = 15\nnode.conn[0].timeo.auth_timeout = 45\nnode.conn[0].timeo.noop_out_interval = 5\nnode.conn[0].timeo.noop_out_timeout = 5\nnode.conn[0].iscsi.MaxXmitDataSegmentLength = 0\nnode.conn[0].iscsi.MaxRecvDataSegmentLength = 262144\nnode.conn[0].iscsi.HeaderDigest = None\nnode.conn[0].iscsi.DataDigest = None\nnode.conn[0].iscsi.IFMarker = No\nnode.conn[0].iscsi.OFMarker = No\n# END RECORD\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:10:11 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-413e3555-a1c6-47b1-9af6-79aba9489343 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] iscsiadm (): stdout=# BEGIN RECORD 2.1.5
Nov 29 03:10:11 user nova-compute[75234]: node.name = iqn.2010-10.org.openstack:a7ef35bf-017a-4fd9-9182-79c1b2ab4f98
Nov 29 03:10:11 user nova-compute[75234]: node.tpgt = -1
Nov 29 03:10:11 user nova-compute[75234]: node.startup = automatic
Nov 29 03:10:11 user nova-compute[75234]: node.leading_login = No
Nov 29 03:10:11 user nova-compute[75234]: iface.iscsi_ifacename = default
Nov 29 03:10:11 user nova-compute[75234]: iface.net_ifacename = <empty>
Nov 29 03:10:11 user nova-compute[75234]: iface.ipaddress = <empty>
Nov 29 03:10:11 user nova-compute[75234]: iface.prefix_len = 0
Nov 29 03:10:11 user nova-compute[75234]: iface.hwaddress = <empty>
Nov 29 03:10:11 user nova-compute[75234]: iface.transport_name = tcp
Nov 29 03:10:11 user nova-compute[75234]: iface.initiatorname = <empty>
Nov 29 03:10:11 user nova-compute[75234]: iface.state = <empty>
Nov 29 03:10:11 user nova-compute[75234]: iface.vlan_id = 0
Nov 29 03:10:11 user nova-compute[75234]: iface.vlan_priority = 0
Nov 29 03:10:11 user nova-compute[75234]: iface.vlan_state = <empty>
Nov 29 03:10:11 user nova-compute[75234]: iface.iface_num = 0
Nov 29 03:10:11 user nova-compute[75234]: iface.mtu = 0
Nov 29 03:10:11 user nova-compute[75234]: iface.port = 0
Nov 29 03:10:11 user nova-compute[75234]: iface.bootproto = <empty>
Nov 29 03:10:11 user nova-compute[75234]: iface.subnet_mask = <empty>
Nov 29 03:10:11 user nova-compute[75234]: iface.gateway = <empty>
Nov 29 03:10:11 user nova-compute[75234]: iface.dhcp_alt_client_id_state = <empty>
Nov 29 03:10:11 user nova-compute[75234]: iface.dhcp_alt_client_id = <empty>
Nov 29 03:10:11 user nova-compute[75234]: iface.dhcp_dns = <empty>
Nov 29 03:10:11 user nova-compute[75234]: iface.dhcp_learn_iqn = <empty>
Nov 29 03:10:11 user nova-compute[75234]: iface.dhcp_req_vendor_id_state = <empty>
Nov 29 03:10:11 user nova-compute[75234]: iface.dhcp_vendor_id_state = <empty>
Nov 29 03:10:11 user nova-compute[75234]: iface.dhcp_vendor_id = <empty>
Nov 29 03:10:11 user nova-compute[75234]: iface.dhcp_slp_da = <empty>
Nov 29 03:10:11 user nova-compute[75234]: iface.fragmentation = <empty>
Nov 29 03:10:11 user nova-compute[75234]: iface.gratuitous_arp = <empty>
Nov 29 03:10:11 user nova-compute[75234]: iface.incoming_forwarding = <empty>
Nov 29 03:10:11 user nova-compute[75234]: iface.tos_state = <empty>
Nov 29 03:10:11 user nova-compute[75234]: iface.tos = 0
Nov 29 03:10:11 user nova-compute[75234]: iface.ttl = 0
Nov 29 03:10:11 user nova-compute[75234]: iface.delayed_ack = <empty>
Nov 29 03:10:11 user nova-compute[75234]: iface.tcp_nagle = <empty>
Nov 29 03:10:11 user nova-compute[75234]: iface.tcp_wsf_state = <empty>
Nov 29 03:10:11 user nova-compute[75234]: iface.tcp_wsf = 0
Nov 29 03:10:11 user nova-compute[75234]: iface.tcp_timer_scale = 0
Nov 29 03:10:11 user nova-compute[75234]: iface.tcp_timestamp = <empty>
Nov 29 03:10:11 user nova-compute[75234]: iface.redirect = <empty>
Nov 29 03:10:11 user nova-compute[75234]: iface.def_task_mgmt_timeout = 0
Nov 29 03:10:11 user nova-compute[75234]: iface.header_digest = <empty>
Nov 29 03:10:11 user nova-compute[75234]: iface.data_digest = <empty>
Nov 29 03:10:11 user nova-compute[75234]: iface.immediate_data = <empty>
Nov 29 03:10:11 user nova-compute[75234]: iface.initial_r2t = <empty>
Nov 29 03:10:11 user nova-compute[75234]: iface.data_seq_inorder = <empty>
Nov 29 03:10:11 user nova-compute[75234]: iface.data_pdu_inorder = <empty>
Nov 29 03:10:11 user nova-compute[75234]: iface.erl = 0
Nov 29 03:10:11 user nova-compute[75234]: iface.max_receive_data_len = 0
Nov 29 03:10:11 user nova-compute[75234]: iface.first_burst_len = 0
Nov 29 03:10:11 user nova-compute[75234]: iface.max_outstanding_r2t = 0
Nov 29 03:10:11 user nova-compute[75234]: iface.max_burst_len = 0
Nov 29 03:10:11 user nova-compute[75234]: iface.chap_auth = <empty>
Nov 29 03:10:11 user nova-compute[75234]: iface.bidi_chap = <empty>
Nov 29 03:10:11 user nova-compute[75234]: iface.strict_login_compliance = <empty>
Nov 29 03:10:11 user nova-compute[75234]: iface.discovery_auth = <empty>
Nov 29 03:10:11 user nova-compute[75234]: iface.discovery_logout = <empty>
Nov 29 03:10:11 user nova-compute[75234]: node.discovery_address = <empty>
Nov 29 03:10:11 user nova-compute[75234]: node.discovery_port = 0
Nov 29 03:10:11 user nova-compute[75234]: node.discovery_type = static
Nov 29 03:10:11 user nova-compute[75234]: node.session.initial_cmdsn = 0
Nov 29 03:10:11 user nova-compute[75234]: node.session.initial_login_retry_max = 8
Nov 29 03:10:11 user nova-compute[75234]: node.session.xmit_thread_priority = -20
Nov 29 03:10:11 user nova-compute[75234]: node.session.cmds_max = 128
Nov 29 03:10:11 user nova-compute[75234]: node.session.queue_depth = 32
Nov 29 03:10:11 user nova-compute[75234]: node.session.nr_sessions = 1
Nov 29 03:10:11 user nova-compute[75234]: node.session.auth.authmethod = CHAP
Nov 29 03:10:11 user nova-compute[75234]: node.session.auth.username = iSOJqDFN
Nov 29 03:10:11 user nova-compute[75234]: node.session.auth.password = ***
Nov 29 03:10:11 user nova-compute[75234]: node.session.auth.username_in = <empty>
Nov 29 03:10:11 user nova-compute[75234]: node.session.auth.password_in = <empty>
Nov 29 03:10:11 user nova-compute[75234]: node.session.auth.chap_algs = MD5
Nov 29 03:10:11 user nova-compute[75234]: node.session.timeo.replacement_timeout = 120
Nov 29 03:10:11 user nova-compute[75234]: node.session.err_timeo.abort_timeout = 15
Nov 29 03:10:11 user nova-compute[75234]: node.session.err_timeo.lu_reset_timeout = 30
Nov 29 03:10:11 user nova-compute[75234]: node.session.err_timeo.tgt_reset_timeout = 30
Nov 29 03:10:11 user nova-compute[75234]: node.session.err_timeo.host_reset_timeout = 60
Nov 29 03:10:11 user nova-compute[75234]: node.session.iscsi.FastAbort = Yes
Nov 29 03:10:11 user nova-compute[75234]: node.session.iscsi.InitialR2T = No
Nov 29 03:10:11 user nova-compute[75234]: node.session.iscsi.ImmediateData = Yes
Nov 29 03:10:11 user nova-compute[75234]: node.session.iscsi.FirstBurstLength = 262144
Nov 29 03:10:11 user nova-compute[75234]: node.session.iscsi.MaxBurstLength = 16776192
Nov 29 03:10:11 user nova-compute[75234]: node.session.iscsi.DefaultTime2Retain = 0
Nov 29 03:10:11 user nova-compute[75234]: node.session.iscsi.DefaultTime2Wait = 2
Nov 29 03:10:11 user nova-compute[75234]: node.session.iscsi.MaxConnections = 1
Nov 29 03:10:11 user nova-compute[75234]: node.session.iscsi.MaxOutstandingR2T = 1
Nov 29 03:10:11 user nova-compute[75234]: node.session.iscsi.ERL = 0
Nov 29 03:10:11 user nova-compute[75234]: node.session.scan = manual
Nov 29 03:10:11 user nova-compute[75234]: node.session.reopen_max = 0
Nov 29 03:10:11 user nova-compute[75234]: node.conn[0].address = 172.16.0.220
Nov 29 03:10:11 user nova-compute[75234]: node.conn[0].port = 3260
Nov 29 03:10:11 user nova-compute[75234]: node.conn[0].startup = manual
Nov 29 03:10:11 user nova-compute[75234]: node.conn[0].tcp.window_size = 524288
Nov 29 03:10:11 user nova-compute[75234]: node.conn[0].tcp.type_of_service = 0
Nov 29 03:10:11 user nova-compute[75234]: node.conn[0].timeo.logout_timeout = 15
Nov 29 03:10:11 user nova-compute[75234]: node.conn[0].timeo.login_timeout = 15
Nov 29 03:10:11 user nova-compute[75234]: node.conn[0].timeo.auth_timeout = 45
Nov 29 03:10:11 user nova-compute[75234]: node.conn[0].timeo.noop_out_interval = 5
Nov 29 03:10:11 user nova-compute[75234]: node.conn[0].timeo.noop_out_timeout = 5
Nov 29 03:10:11 user nova-compute[75234]: node.conn[0].iscsi.MaxXmitDataSegmentLength = 0
Nov 29 03:10:11 user nova-compute[75234]: node.conn[0].iscsi.MaxRecvDataSegmentLength = 262144
Nov 29 03:10:11 user nova-compute[75234]: node.conn[0].iscsi.HeaderDigest = None
Nov 29 03:10:11 user nova-compute[75234]: node.conn[0].iscsi.DataDigest = None
Nov 29 03:10:11 user nova-compute[75234]: node.conn[0].iscsi.IFMarker = No
Nov 29 03:10:11 user nova-compute[75234]: node.conn[0].iscsi.OFMarker = No
Nov 29 03:10:11 user nova-compute[75234]: # END RECORD
Nov 29 03:10:11 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:10:11 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:a7ef35bf-017a-4fd9-9182-79c1b2ab4f98 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:10:11 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:a7ef35bf-017a-4fd9-9182-79c1b2ab4f98 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.022s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:10:11 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[74c05b9f-2b46-470f-b5a9-18e4eb3176cf]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:10:11 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-413e3555-a1c6-47b1-9af6-79aba9489343 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:10:11 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:a7ef35bf-017a-4fd9-9182-79c1b2ab4f98 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:10:11 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:a7ef35bf-017a-4fd9-9182-79c1b2ab4f98 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.020s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:10:11 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[c06d4f16-62c8-46d3-a67b-35be3eb857fd]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:10:11 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-413e3555-a1c6-47b1-9af6-79aba9489343 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:10:11 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:a7ef35bf-017a-4fd9-9182-79c1b2ab4f98 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v iSOJqDFN {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:10:11 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:a7ef35bf-017a-4fd9-9182-79c1b2ab4f98 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v iSOJqDFN" returned: 0 in 0.020s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:10:11 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[2feeed17-4a24-4fc4-921e-c6617ad2259f]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:10:11 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-413e3555-a1c6-47b1-9af6-79aba9489343 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'iSOJqDFN'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:10:11 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:a7ef35bf-017a-4fd9-9182-79c1b2ab4f98 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:10:11 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:a7ef35bf-017a-4fd9-9182-79c1b2ab4f98 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.020s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:10:11 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[84a60db8-042a-46ba-9bfc-12dbb48eca09]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:10:11 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-413e3555-a1c6-47b1-9af6-79aba9489343 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:10:11 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:10:11 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.022s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:10:11 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[67499880-bea0-4194-b20a-83190328510f]: (4, ('tcp: [12] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ada91c26-ca97-4b8b-98bb-a310d0627691 (non-flash)\ntcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b (non-flash)\ntcp: [15] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a7ef35bf-017a-4fd9-9182-79c1b2ab4f98 (non-flash)\ntcp: [16] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672 (non-flash)\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:10:11 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-413e3555-a1c6-47b1-9af6-79aba9489343 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [12] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ada91c26-ca97-4b8b-98bb-a310d0627691 (non-flash)
Nov 29 03:10:11 user nova-compute[75234]: tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b (non-flash)
Nov 29 03:10:11 user nova-compute[75234]: tcp: [15] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a7ef35bf-017a-4fd9-9182-79c1b2ab4f98 (non-flash)
Nov 29 03:10:11 user nova-compute[75234]: tcp: [16] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672 (non-flash)
Nov 29 03:10:11 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:10:11 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-413e3555-a1c6-47b1-9af6-79aba9489343 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] iscsi session list stdout=tcp: [12] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ada91c26-ca97-4b8b-98bb-a310d0627691 (non-flash)
Nov 29 03:10:11 user nova-compute[75234]: tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b (non-flash)
Nov 29 03:10:11 user nova-compute[75234]: tcp: [15] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a7ef35bf-017a-4fd9-9182-79c1b2ab4f98 (non-flash)
Nov 29 03:10:11 user nova-compute[75234]: tcp: [16] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672 (non-flash)
Nov 29 03:10:11 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsi_session /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Nov 29 03:10:11 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-413e3555-a1c6-47b1-9af6-79aba9489343 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:a7ef35bf-017a-4fd9-9182-79c1b2ab4f98" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.160s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:87}}
Nov 29 03:10:11 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-413e3555-a1c6-47b1-9af6-79aba9489343 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] Connected to 172.16.0.220:3260 {{(pid=75234) _connect_vol /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:644}}
Nov 29 03:10:11 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-413e3555-a1c6-47b1-9af6-79aba9489343 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] HCTL ('35', '0', '0', 0) found on session 15 with lun 0 {{(pid=75234) get_hctl /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:770}}
Nov 29 03:10:11 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-413e3555-a1c6-47b1-9af6-79aba9489343 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] Scanning host 35 c: 0, t: 0, l: 0) {{(pid=75234) scan_iscsi /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:795}}
Nov 29 03:10:11 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host35/scan {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:10:11 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host35/scan" returned: 0 in 0.013s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:10:11 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[9fe83539-e12b-44f3-969c-4833c02f9e7f]: (4, ('0 0 0', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:10:11 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-413e3555-a1c6-47b1-9af6-79aba9489343 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] Searching for a device in session 15 and hctl ('35', '0', '0', 0) yield: sdd {{(pid=75234) device_name_by_hctl /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:789}}
Nov 29 03:10:11 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-413e3555-a1c6-47b1-9af6-79aba9489343 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] Connected to sdd using {'target_iqn': 'iqn.2010-10.org.openstack:a7ef35bf-017a-4fd9-9182-79c1b2ab4f98', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'iSOJqDFN', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path'} {{(pid=75234) _connect_vol /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:672}}
Nov 29 03:10:11 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-413e3555-a1c6-47b1-9af6-79aba9489343 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 0.187s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:87}}
Nov 29 03:10:11 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-413e3555-a1c6-47b1-9af6-79aba9489343 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] <== connect_volume: return (189ms) {'type': 'block', 'scsi_wwn': '23261333730333830', 'path': '/dev/sdd'} {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:203}}
Nov 29 03:10:11 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-413e3555-a1c6-47b1-9af6-79aba9489343 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] [instance: 8297c264-a213-44ee-be46-5c08eabda9f7] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '23261333730333830', 'path': '/dev/sdd'} {{(pid=75234) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Nov 29 03:10:11 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:10:11 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-413e3555-a1c6-47b1-9af6-79aba9489343 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-11-29T03:06:07Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-ServerStableDeviceRescueTest-server-428917867',display_name='tempest-ServerStableDeviceRescueTest-server-428917867',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-serverstabledevicerescuetest-server-428917867',id=6,image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBPAGeJQ8OrWhDv/r5lktd6O/obaXRK7/7VEiU/V0MksOr2sSmn5M84mkyhNPvWnFo3k1J68udxxAQOPj9YvQEkVpNgH7z7grcEsUVqlgMVin9R9cWVCI3LAQJjKBdTQkeA==',key_name='tempest-keypair-1167119775',keypairs=<?>,launch_index=0,launched_at=2023-11-29T03:06:23Z,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='745bdb3c72794155829dc0a69f35b321',ramdisk_id='',reservation_id='r-tt096l76',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-ServerStableDeviceRescueTest-1509102897',owner_user_name='tempest-ServerStableDeviceRescueTest-1509102897-project-member'},tags=<?>,task_state='rescuing',terminated_at=None,trusted_certs=None,updated_at=2023-11-29T03:08:09Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='566e3dc84bbb40d9badcacbcaec9b987',uuid=8297c264-a213-44ee-be46-5c08eabda9f7,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "d26334d2-8040-4022-b658-69be1cf6a025", "address": "fa:16:3e:5f:9d:5a", "network": {"id": "e0366490-9fe2-4429-88c8-6625128e645d", "bridge": "br-int", "label": "tempest-ServerStableDeviceRescueTest-804259918-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.31", "type": "floating", "version": 4, "meta": {}}], "label": "tempest-ServerStableDeviceRescueTest-804259918-network", "vif_mac": "fa:16:3e:5f:9d:5a"}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "745bdb3c72794155829dc0a69f35b321", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapd26334d2-80", "ovs_interfaceid": "d26334d2-8040-4022-b658-69be1cf6a025", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=75234) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Nov 29 03:10:11 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-413e3555-a1c6-47b1-9af6-79aba9489343 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] Converting VIF {"id": "d26334d2-8040-4022-b658-69be1cf6a025", "address": "fa:16:3e:5f:9d:5a", "network": {"id": "e0366490-9fe2-4429-88c8-6625128e645d", "bridge": "br-int", "label": "tempest-ServerStableDeviceRescueTest-804259918-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.31", "type": "floating", "version": 4, "meta": {}}], "label": "tempest-ServerStableDeviceRescueTest-804259918-network", "vif_mac": "fa:16:3e:5f:9d:5a"}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "745bdb3c72794155829dc0a69f35b321", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapd26334d2-80", "ovs_interfaceid": "d26334d2-8040-4022-b658-69be1cf6a025", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:10:11 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-413e3555-a1c6-47b1-9af6-79aba9489343 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:5f:9d:5a,bridge_name='br-int',has_traffic_filtering=True,id=d26334d2-8040-4022-b658-69be1cf6a025,network=Network(e0366490-9fe2-4429-88c8-6625128e645d),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapd26334d2-80') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:10:11 user nova-compute[75234]: DEBUG nova.objects.instance [None req-413e3555-a1c6-47b1-9af6-79aba9489343 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] Lazy-loading 'pci_devices' on Instance uuid 8297c264-a213-44ee-be46-5c08eabda9f7 {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:10:11 user nova-compute[75234]: DEBUG nova.objects.base [None req-413e3555-a1c6-47b1-9af6-79aba9489343 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] Object Instance<8297c264-a213-44ee-be46-5c08eabda9f7> lazy-loaded attributes: trusted_certs,migration_context,resources,numa_topology,vcpu_model,pci_devices {{(pid=75234) wrapper /opt/stack/nova/nova/objects/base.py:126}}
Nov 29 03:10:11 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-413e3555-a1c6-47b1-9af6-79aba9489343 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] [instance: 8297c264-a213-44ee-be46-5c08eabda9f7] End _get_guest_xml xml=<domain type="kvm">
Nov 29 03:10:11 user nova-compute[75234]:   <uuid>8297c264-a213-44ee-be46-5c08eabda9f7</uuid>
Nov 29 03:10:11 user nova-compute[75234]:   <name>instance-00000006</name>
Nov 29 03:10:11 user nova-compute[75234]:   <memory>524288</memory>
Nov 29 03:10:11 user nova-compute[75234]:   <vcpu>1</vcpu>
Nov 29 03:10:11 user nova-compute[75234]:   <metadata>
Nov 29 03:10:11 user nova-compute[75234]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Nov 29 03:10:11 user nova-compute[75234]:       <nova:package version="0.0.0"/>
Nov 29 03:10:11 user nova-compute[75234]:       <nova:name>tempest-ServerStableDeviceRescueTest-server-428917867</nova:name>
Nov 29 03:10:11 user nova-compute[75234]:       <nova:creationTime>2023-11-29 03:10:11</nova:creationTime>
Nov 29 03:10:11 user nova-compute[75234]:       <nova:flavor name="m1.tiny">
Nov 29 03:10:11 user nova-compute[75234]:         <nova:memory>512</nova:memory>
Nov 29 03:10:11 user nova-compute[75234]:         <nova:disk>1</nova:disk>
Nov 29 03:10:11 user nova-compute[75234]:         <nova:swap>0</nova:swap>
Nov 29 03:10:11 user nova-compute[75234]:         <nova:ephemeral>0</nova:ephemeral>
Nov 29 03:10:11 user nova-compute[75234]:         <nova:vcpus>1</nova:vcpus>
Nov 29 03:10:11 user nova-compute[75234]:       </nova:flavor>
Nov 29 03:10:11 user nova-compute[75234]:       <nova:owner>
Nov 29 03:10:11 user nova-compute[75234]:         <nova:user uuid="566e3dc84bbb40d9badcacbcaec9b987">tempest-ServerStableDeviceRescueTest-1509102897-project-member</nova:user>
Nov 29 03:10:11 user nova-compute[75234]:         <nova:project uuid="745bdb3c72794155829dc0a69f35b321">tempest-ServerStableDeviceRescueTest-1509102897</nova:project>
Nov 29 03:10:11 user nova-compute[75234]:       </nova:owner>
Nov 29 03:10:11 user nova-compute[75234]:       <nova:root type="image" uuid="96c30ba7-9b73-4a30-8ba4-7138879573b7"/>
Nov 29 03:10:11 user nova-compute[75234]:       <nova:ports>
Nov 29 03:10:11 user nova-compute[75234]:         <nova:port uuid="d26334d2-8040-4022-b658-69be1cf6a025">
Nov 29 03:10:11 user nova-compute[75234]:           <nova:ip type="fixed" address="10.0.0.12" ipVersion="4"/>
Nov 29 03:10:11 user nova-compute[75234]:         </nova:port>
Nov 29 03:10:11 user nova-compute[75234]:       </nova:ports>
Nov 29 03:10:11 user nova-compute[75234]:     </nova:instance>
Nov 29 03:10:11 user nova-compute[75234]:   </metadata>
Nov 29 03:10:11 user nova-compute[75234]:   <sysinfo type="smbios">
Nov 29 03:10:11 user nova-compute[75234]:     <system>
Nov 29 03:10:11 user nova-compute[75234]:       <entry name="manufacturer">OpenStack Foundation</entry>
Nov 29 03:10:11 user nova-compute[75234]:       <entry name="product">OpenStack Nova</entry>
Nov 29 03:10:11 user nova-compute[75234]:       <entry name="version">0.0.0</entry>
Nov 29 03:10:11 user nova-compute[75234]:       <entry name="serial">8297c264-a213-44ee-be46-5c08eabda9f7</entry>
Nov 29 03:10:11 user nova-compute[75234]:       <entry name="uuid">8297c264-a213-44ee-be46-5c08eabda9f7</entry>
Nov 29 03:10:11 user nova-compute[75234]:       <entry name="family">Virtual Machine</entry>
Nov 29 03:10:11 user nova-compute[75234]:     </system>
Nov 29 03:10:11 user nova-compute[75234]:   </sysinfo>
Nov 29 03:10:11 user nova-compute[75234]:   <os>
Nov 29 03:10:11 user nova-compute[75234]:     <type arch="x86_64" machine="pc">hvm</type>
Nov 29 03:10:11 user nova-compute[75234]:     <smbios mode="sysinfo"/>
Nov 29 03:10:11 user nova-compute[75234]:   </os>
Nov 29 03:10:11 user nova-compute[75234]:   <features>
Nov 29 03:10:11 user nova-compute[75234]:     <acpi/>
Nov 29 03:10:11 user nova-compute[75234]:     <apic/>
Nov 29 03:10:11 user nova-compute[75234]:     <vmcoreinfo/>
Nov 29 03:10:11 user nova-compute[75234]:   </features>
Nov 29 03:10:11 user nova-compute[75234]:   <clock offset="utc">
Nov 29 03:10:11 user nova-compute[75234]:     <timer name="pit" tickpolicy="delay"/>
Nov 29 03:10:11 user nova-compute[75234]:     <timer name="rtc" tickpolicy="catchup"/>
Nov 29 03:10:11 user nova-compute[75234]:     <timer name="hpet" present="no"/>
Nov 29 03:10:11 user nova-compute[75234]:   </clock>
Nov 29 03:10:11 user nova-compute[75234]:   <cpu mode="custom" match="exact">
Nov 29 03:10:11 user nova-compute[75234]:     <model>Nehalem</model>
Nov 29 03:10:11 user nova-compute[75234]:     <topology sockets="1" cores="1" threads="1"/>
Nov 29 03:10:11 user nova-compute[75234]:   </cpu>
Nov 29 03:10:11 user nova-compute[75234]:   <devices>
Nov 29 03:10:11 user nova-compute[75234]:     <disk type="file" device="disk">
Nov 29 03:10:11 user nova-compute[75234]:       <driver name="qemu" type="qcow2" cache="none"/>
Nov 29 03:10:11 user nova-compute[75234]:       <source file="/opt/stack/data/nova/instances/8297c264-a213-44ee-be46-5c08eabda9f7/disk"/>
Nov 29 03:10:11 user nova-compute[75234]:       <target dev="vda" bus="virtio"/>
Nov 29 03:10:11 user nova-compute[75234]:     </disk>
Nov 29 03:10:11 user nova-compute[75234]:     <disk type="block" device="disk">
Nov 29 03:10:11 user nova-compute[75234]:       <driver name="qemu" type="raw" cache="none" io="native"/>
Nov 29 03:10:11 user nova-compute[75234]:       <source dev="/dev/sdd"/>
Nov 29 03:10:11 user nova-compute[75234]:       <target dev="vdb" bus="virtio"/>
Nov 29 03:10:11 user nova-compute[75234]:       <serial>a7ef35bf-017a-4fd9-9182-79c1b2ab4f98</serial>
Nov 29 03:10:11 user nova-compute[75234]:     </disk>
Nov 29 03:10:11 user nova-compute[75234]:     <disk type="file" device="disk">
Nov 29 03:10:11 user nova-compute[75234]:       <driver name="qemu" type="qcow2" cache="none"/>
Nov 29 03:10:11 user nova-compute[75234]:       <source file="/opt/stack/data/nova/instances/8297c264-a213-44ee-be46-5c08eabda9f7/disk.rescue"/>
Nov 29 03:10:11 user nova-compute[75234]:       <target dev="vdc" bus="virtio"/>
Nov 29 03:10:11 user nova-compute[75234]:       <boot order="1"/>
Nov 29 03:10:11 user nova-compute[75234]:     </disk>
Nov 29 03:10:11 user nova-compute[75234]:     <interface type="ethernet">
Nov 29 03:10:11 user nova-compute[75234]:       <mac address="fa:16:3e:5f:9d:5a"/>
Nov 29 03:10:11 user nova-compute[75234]:       <model type="virtio"/>
Nov 29 03:10:11 user nova-compute[75234]:       <mtu size="1442"/>
Nov 29 03:10:11 user nova-compute[75234]:       <target dev="tapd26334d2-80"/>
Nov 29 03:10:11 user nova-compute[75234]:     </interface>
Nov 29 03:10:11 user nova-compute[75234]:     <serial type="pty">
Nov 29 03:10:11 user nova-compute[75234]:       <log file="/opt/stack/data/nova/instances/8297c264-a213-44ee-be46-5c08eabda9f7/console.log" append="off"/>
Nov 29 03:10:11 user nova-compute[75234]:     </serial>
Nov 29 03:10:11 user nova-compute[75234]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Nov 29 03:10:11 user nova-compute[75234]:     <video>
Nov 29 03:10:11 user nova-compute[75234]:       <model type="virtio"/>
Nov 29 03:10:11 user nova-compute[75234]:     </video>
Nov 29 03:10:11 user nova-compute[75234]:     <rng model="virtio">
Nov 29 03:10:11 user nova-compute[75234]:       <backend model="random">/dev/urandom</backend>
Nov 29 03:10:11 user nova-compute[75234]:     </rng>
Nov 29 03:10:11 user nova-compute[75234]:     <controller type="usb" index="0" model="none"/>
Nov 29 03:10:11 user nova-compute[75234]:     <memballoon model="virtio">
Nov 29 03:10:11 user nova-compute[75234]:       <stats period="10"/>
Nov 29 03:10:11 user nova-compute[75234]:     </memballoon>
Nov 29 03:10:11 user nova-compute[75234]:   </devices>
Nov 29 03:10:11 user nova-compute[75234]: </domain>
Nov 29 03:10:11 user nova-compute[75234]:  {{(pid=75234) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7568}}
Nov 29 03:10:11 user nova-compute[75234]: INFO nova.virt.libvirt.driver [-] [instance: 8297c264-a213-44ee-be46-5c08eabda9f7] Instance destroyed successfully.
Nov 29 03:10:11 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-413e3555-a1c6-47b1-9af6-79aba9489343 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] No BDM found with device name vda, not building metadata. {{(pid=75234) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12089}}
Nov 29 03:10:11 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-413e3555-a1c6-47b1-9af6-79aba9489343 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] No BDM found with device name vdb, not building metadata. {{(pid=75234) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12089}}
Nov 29 03:10:11 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-413e3555-a1c6-47b1-9af6-79aba9489343 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] No BDM found with device name vdc, not building metadata. {{(pid=75234) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12089}}
Nov 29 03:10:11 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-413e3555-a1c6-47b1-9af6-79aba9489343 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] No VIF found with MAC fa:16:3e:5f:9d:5a, not building metadata {{(pid=75234) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Nov 29 03:10:12 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:10:12 user nova-compute[75234]: INFO nova.virt.libvirt.driver [-] [instance: a9350a18-14e6-42c6-a150-0a1487f20ad9] Instance destroyed successfully.
Nov 29 03:10:12 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-221919a0-9cdd-44ec-881f-d6c82383d31d tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-11-29T03:06:06Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-ServerActionsTestOtherB-server-372321920',display_name='tempest-ServerActionsTestOtherB-server-372321920',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-serveractionstestotherb-server-372321920',id=5,image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBFOggqgO1QPxMZPb2P6SX24mx6LOsfQfrzx4GSpNnJiixHqqOn61ObLwI9/TG/r6plC8w1huVCtu+OqAyk9QXUlEr0+VsPNunPSN9WPXBAM4+U+3iCOgHXExdr++34pXkw==',key_name='tempest-keypair-1692341699',keypairs=<?>,launch_index=0,launched_at=2023-11-29T03:06:23Z,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=MigrationContext,new_flavor=Flavor(7),node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=1,progress=0,project_id='ad5ea22c63f04f91a3f2fa665e4fc8a1',ramdisk_id='',reservation_id='r-f5lkid4e',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=ServiceList,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',old_vm_state='active',owner_project_name='tempest-ServerActionsTestOtherB-70688183',owner_user_name='tempest-ServerActionsTestOtherB-70688183-project-member'},tags=<?>,task_state='resize_migrating',terminated_at=None,trusted_certs=None,updated_at=2023-11-29T03:10:08Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='bdddba0ef9c44b6cb43c32a9ab0e29f6',uuid=a9350a18-14e6-42c6-a150-0a1487f20ad9,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "52024efc-8cde-4610-b12c-198f19d75ccf", "address": "fa:16:3e:09:b8:70", "network": {"id": "31c4ca3b-3a2f-47df-87fa-4ad095ee5cf9", "bridge": "br-int", "label": "tempest-ServerActionsTestOtherB-1680745008-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.9", "type": "floating", "version": 4, "meta": {}}], "label": "tempest-ServerActionsTestOtherB-1680745008-network", "vif_mac": "fa:16:3e:09:b8:70"}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "ad5ea22c63f04f91a3f2fa665e4fc8a1", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap52024efc-8c", "ovs_interfaceid": "52024efc-8cde-4610-b12c-198f19d75ccf", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Nov 29 03:10:12 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-221919a0-9cdd-44ec-881f-d6c82383d31d tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Converting VIF {"id": "52024efc-8cde-4610-b12c-198f19d75ccf", "address": "fa:16:3e:09:b8:70", "network": {"id": "31c4ca3b-3a2f-47df-87fa-4ad095ee5cf9", "bridge": "br-int", "label": "tempest-ServerActionsTestOtherB-1680745008-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.9", "type": "floating", "version": 4, "meta": {}}], "label": "tempest-ServerActionsTestOtherB-1680745008-network", "vif_mac": "fa:16:3e:09:b8:70"}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "ad5ea22c63f04f91a3f2fa665e4fc8a1", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap52024efc-8c", "ovs_interfaceid": "52024efc-8cde-4610-b12c-198f19d75ccf", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:10:12 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-221919a0-9cdd-44ec-881f-d6c82383d31d tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:09:b8:70,bridge_name='br-int',has_traffic_filtering=True,id=52024efc-8cde-4610-b12c-198f19d75ccf,network=Network(31c4ca3b-3a2f-47df-87fa-4ad095ee5cf9),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap52024efc-8c') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:10:12 user nova-compute[75234]: DEBUG os_vif [None req-221919a0-9cdd-44ec-881f-d6c82383d31d tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Unplugging vif VIFOpenVSwitch(active=True,address=fa:16:3e:09:b8:70,bridge_name='br-int',has_traffic_filtering=True,id=52024efc-8cde-4610-b12c-198f19d75ccf,network=Network(31c4ca3b-3a2f-47df-87fa-4ad095ee5cf9),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap52024efc-8c') {{(pid=75234) unplug /opt/stack/data/venv/lib/python3.10/site-packages/os_vif/__init__.py:109}}
Nov 29 03:10:12 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:10:12 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tap52024efc-8c, bridge=br-int, if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:10:12 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:10:12 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:10:12 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:10:12 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:10:12 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbDestroyCommand(_result=None, table=QoS, record=1df22596-2c4c-4f7a-952f-593c7370f497) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:10:12 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:10:12 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:10:12 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:10:12 user nova-compute[75234]: INFO os_vif [None req-221919a0-9cdd-44ec-881f-d6c82383d31d tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Successfully unplugged vif VIFOpenVSwitch(active=True,address=fa:16:3e:09:b8:70,bridge_name='br-int',has_traffic_filtering=True,id=52024efc-8cde-4610-b12c-198f19d75ccf,network=Network(31c4ca3b-3a2f-47df-87fa-4ad095ee5cf9),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap52024efc-8c')
Nov 29 03:10:12 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-221919a0-9cdd-44ec-881f-d6c82383d31d tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] [instance: a9350a18-14e6-42c6-a150-0a1487f20ad9] calling os-brick to detach iSCSI Volume {{(pid=75234) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Nov 29 03:10:12 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-221919a0-9cdd-44ec-881f-d6c82383d31d tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f8a5050c9a0>, {'target_iqn': 'iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'NFCsGdbh', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdc'}, None), 'kwargs': False}}" {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:176}}
Nov 29 03:10:12 user nova-compute[75234]: WARNING os_brick.initiator.connectors.base [None req-221919a0-9cdd-44ec-881f-d6c82383d31d tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Nov 29 03:10:12 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-221919a0-9cdd-44ec-881f-d6c82383d31d tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:68}}
Nov 29 03:10:12 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-221919a0-9cdd-44ec-881f-d6c82383d31d tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.002s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:73}}
Nov 29 03:10:12 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:10:12 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.031s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:10:12 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[8da62ed2-30ac-4cb4-88d3-74cfa6631df2]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:a7ef35bf-017a-4fd9-9182-79c1b2ab4f98\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:ada91c26-ca97-4b8b-98bb-a310d0627691\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:10:12 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-221919a0-9cdd-44ec-881f-d6c82383d31d tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS:
Nov 29 03:10:12 user nova-compute[75234]: No targets found.
Nov 29 03:10:12 user nova-compute[75234]: iSNS:
Nov 29 03:10:12 user nova-compute[75234]: No targets found.
Nov 29 03:10:12 user nova-compute[75234]: STATIC:
Nov 29 03:10:12 user nova-compute[75234]: Target: iqn.2010-10.org.openstack:a7ef35bf-017a-4fd9-9182-79c1b2ab4f98
Nov 29 03:10:12 user nova-compute[75234]:         Portal: 172.16.0.220:3260,-1
Nov 29 03:10:12 user nova-compute[75234]:                 Iface Name: default
Nov 29 03:10:12 user nova-compute[75234]: Target: iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672
Nov 29 03:10:12 user nova-compute[75234]:         Portal: 172.16.0.220:3260,-1
Nov 29 03:10:12 user nova-compute[75234]:                 Iface Name: default
Nov 29 03:10:12 user nova-compute[75234]: Target: iqn.2010-10.org.openstack:ada91c26-ca97-4b8b-98bb-a310d0627691
Nov 29 03:10:12 user nova-compute[75234]:         Portal: 172.16.0.220:3260,-1
Nov 29 03:10:12 user nova-compute[75234]:                 Iface Name: default
Nov 29 03:10:12 user nova-compute[75234]: Target: iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b
Nov 29 03:10:12 user nova-compute[75234]:         Portal: 172.16.0.220:3260,-1
Nov 29 03:10:12 user nova-compute[75234]:                 Iface Name: default
Nov 29 03:10:12 user nova-compute[75234]: FIRMWARE:
Nov 29 03:10:12 user nova-compute[75234]: No targets found.
Nov 29 03:10:12 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:10:12 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-221919a0-9cdd-44ec-881f-d6c82383d31d tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Regex to get portals from discoverydb: ^SENDTARGETS:
Nov 29 03:10:12 user nova-compute[75234]: .*?^DiscoveryAddress: 172.16.0.220,3260.*?
Nov 29 03:10:12 user nova-compute[75234]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=75234) _get_discoverydb_portals /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:382}}
Nov 29 03:10:12 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-221919a0-9cdd-44ec-881f-d6c82383d31d tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b', 0)] {{(pid=75234) _get_connection_devices /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:830}}
Nov 29 03:10:12 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:10:12 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.020s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:10:12 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[bfb3c0e4-874a-4502-b185-3a424cdcb0d5]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:a7ef35bf-017a-4fd9-9182-79c1b2ab4f98\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:ada91c26-ca97-4b8b-98bb-a310d0627691\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:10:12 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:10:12 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.024s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:10:12 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[737384f3-8bcb-4d99-98c4-3197b5eb69cf]: (4, ('tcp: [12] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ada91c26-ca97-4b8b-98bb-a310d0627691 (non-flash)\ntcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b (non-flash)\ntcp: [15] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a7ef35bf-017a-4fd9-9182-79c1b2ab4f98 (non-flash)\ntcp: [16] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672 (non-flash)\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:10:12 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-221919a0-9cdd-44ec-881f-d6c82383d31d tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [12] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ada91c26-ca97-4b8b-98bb-a310d0627691 (non-flash)
Nov 29 03:10:12 user nova-compute[75234]: tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b (non-flash)
Nov 29 03:10:12 user nova-compute[75234]: tcp: [15] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a7ef35bf-017a-4fd9-9182-79c1b2ab4f98 (non-flash)
Nov 29 03:10:12 user nova-compute[75234]: tcp: [16] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672 (non-flash)
Nov 29 03:10:12 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:10:12 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-221919a0-9cdd-44ec-881f-d6c82383d31d tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] iscsi session list stdout=tcp: [12] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ada91c26-ca97-4b8b-98bb-a310d0627691 (non-flash)
Nov 29 03:10:12 user nova-compute[75234]: tcp: [14] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b (non-flash)
Nov 29 03:10:12 user nova-compute[75234]: tcp: [15] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a7ef35bf-017a-4fd9-9182-79c1b2ab4f98 (non-flash)
Nov 29 03:10:12 user nova-compute[75234]: tcp: [16] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672 (non-flash)
Nov 29 03:10:12 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsi_session /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Nov 29 03:10:12 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-221919a0-9cdd-44ec-881f-d6c82383d31d tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7f8a502bfd90>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b'): ({'sdc'}, set())}) {{(pid=75234) _get_connection_devices /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:863}}
Nov 29 03:10:12 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-221919a0-9cdd-44ec-881f-d6c82383d31d tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Removing single pathed devices sdc {{(pid=75234) remove_connection /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:374}}
Nov 29 03:10:12 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:10:12 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.019s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:10:12 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[375509df-38c6-4ff7-be44-2820ce550b64]: (4, ('path checker states:\nup                  4\n\npaths: 0\nbusy: False\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:10:12 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdc {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:10:12 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdc" returned: 0 in 0.023s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:10:12 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[61376e57-7a8e-4de0-824d-04d8b1a60633]: (4, ('ok\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:10:12 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-221919a0-9cdd-44ec-881f-d6c82383d31d tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Flushing IO for device /dev/sdc {{(pid=75234) flush_device_io /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:434}}
Nov 29 03:10:12 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdc {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:10:12 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdc" returned: 0 in 0.014s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:10:12 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[568154b9-20df-4f86-a3dc-d29299e340df]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:10:12 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:10:12 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-221919a0-9cdd-44ec-881f-d6c82383d31d tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Remove SCSI device /dev/sdc with /sys/block/sdc/device/delete {{(pid=75234) remove_scsi_device /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:148}}
Nov 29 03:10:12 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdc/device/delete {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:10:12 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdc/device/delete" returned: 0 in 0.046s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:10:12 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[a75878d9-1759-4ca2-b390-d826cbb2d4a0]: (4, ('1', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:10:12 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-221919a0-9cdd-44ec-881f-d6c82383d31d tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Checking to see if SCSI volumes sdc have been removed. {{(pid=75234) wait_for_volumes_removal /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:156}}
Nov 29 03:10:12 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-221919a0-9cdd-44ec-881f-d6c82383d31d tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] SCSI volumes sdc have been removed. {{(pid=75234) wait_for_volumes_removal /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:166}}
Nov 29 03:10:12 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-221919a0-9cdd-44ec-881f-d6c82383d31d tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b')] {{(pid=75234) _disconnect_connection /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Nov 29 03:10:12 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:10:12 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.020s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:10:12 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[621b1ac7-542c-4839-9d62-f9e43589a859]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:10:12 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-221919a0-9cdd-44ec-881f-d6c82383d31d tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:10:12 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b -p 172.16.0.220:3260 --logout {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:10:12 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b -p 172.16.0.220:3260 --logout" returned: 0 in 0.069s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:10:12 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[5492e53b-61a5-4bb9-8e68-735690c35184]: (4, ('Logging out of session [sid: 14, target: iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b, portal: 172.16.0.220,3260]\nLogout of [sid: 14, target: iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:10:12 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-221919a0-9cdd-44ec-881f-d6c82383d31d tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 14, target: iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b, portal: 172.16.0.220,3260]
Nov 29 03:10:12 user nova-compute[75234]: Logout of [sid: 14, target: iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b, portal: 172.16.0.220,3260] successful.
Nov 29 03:10:12 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:10:12 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b -p 172.16.0.220:3260 --op delete {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:10:12 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b -p 172.16.0.220:3260 --op delete" returned: 0 in 0.032s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:10:12 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[df18fdf9-bfdd-43f9-a4af-6dff001852f5]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:10:12 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-221919a0-9cdd-44ec-881f-d6c82383d31d tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:10:12 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-221919a0-9cdd-44ec-881f-d6c82383d31d tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.362s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:87}}
Nov 29 03:10:12 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-221919a0-9cdd-44ec-881f-d6c82383d31d tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] <== disconnect_volume: return (363ms) None {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:203}}
Nov 29 03:10:12 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-221919a0-9cdd-44ec-881f-d6c82383d31d tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] [instance: a9350a18-14e6-42c6-a150-0a1487f20ad9] Disconnected iSCSI Volume {{(pid=75234) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:79}}
Nov 29 03:10:12 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-221919a0-9cdd-44ec-881f-d6c82383d31d tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a9350a18-14e6-42c6-a150-0a1487f20ad9/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:10:13 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-221919a0-9cdd-44ec-881f-d6c82383d31d tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a9350a18-14e6-42c6-a150-0a1487f20ad9/disk --force-share --output=json" returned: 0 in 0.212s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:10:13 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-221919a0-9cdd-44ec-881f-d6c82383d31d tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a9350a18-14e6-42c6-a150-0a1487f20ad9/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:10:13 user nova-compute[75234]: DEBUG nova.compute.manager [req-54f895d9-28ab-49aa-b03c-dad7f48c26d3 req-346ff44f-89dd-4892-b3f6-48733feda473 service nova] [instance: a9350a18-14e6-42c6-a150-0a1487f20ad9] Received event network-vif-unplugged-52024efc-8cde-4610-b12c-198f19d75ccf {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:10:13 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-54f895d9-28ab-49aa-b03c-dad7f48c26d3 req-346ff44f-89dd-4892-b3f6-48733feda473 service nova] Acquiring lock "a9350a18-14e6-42c6-a150-0a1487f20ad9-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:10:13 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-54f895d9-28ab-49aa-b03c-dad7f48c26d3 req-346ff44f-89dd-4892-b3f6-48733feda473 service nova] Lock "a9350a18-14e6-42c6-a150-0a1487f20ad9-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:10:13 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-54f895d9-28ab-49aa-b03c-dad7f48c26d3 req-346ff44f-89dd-4892-b3f6-48733feda473 service nova] Lock "a9350a18-14e6-42c6-a150-0a1487f20ad9-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:10:13 user nova-compute[75234]: DEBUG nova.compute.manager [req-54f895d9-28ab-49aa-b03c-dad7f48c26d3 req-346ff44f-89dd-4892-b3f6-48733feda473 service nova] [instance: a9350a18-14e6-42c6-a150-0a1487f20ad9] No waiting events found dispatching network-vif-unplugged-52024efc-8cde-4610-b12c-198f19d75ccf {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:10:13 user nova-compute[75234]: WARNING nova.compute.manager [req-54f895d9-28ab-49aa-b03c-dad7f48c26d3 req-346ff44f-89dd-4892-b3f6-48733feda473 service nova] [instance: a9350a18-14e6-42c6-a150-0a1487f20ad9] Received unexpected event network-vif-unplugged-52024efc-8cde-4610-b12c-198f19d75ccf for instance with vm_state active and task_state resize_migrating.
Nov 29 03:10:13 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-221919a0-9cdd-44ec-881f-d6c82383d31d tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a9350a18-14e6-42c6-a150-0a1487f20ad9/disk --force-share --output=json" returned: 0 in 0.181s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:10:13 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-221919a0-9cdd-44ec-881f-d6c82383d31d tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] skipping disk /dev/sdc (vdb) as it is a volume {{(pid=75234) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11213}}
Nov 29 03:10:13 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-221919a0-9cdd-44ec-881f-d6c82383d31d tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Running cmd (subprocess): cp -r /opt/stack/data/nova/instances/a9350a18-14e6-42c6-a150-0a1487f20ad9_resize/disk /opt/stack/data/nova/instances/a9350a18-14e6-42c6-a150-0a1487f20ad9/disk {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:10:13 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-221919a0-9cdd-44ec-881f-d6c82383d31d tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] CMD "cp -r /opt/stack/data/nova/instances/a9350a18-14e6-42c6-a150-0a1487f20ad9_resize/disk /opt/stack/data/nova/instances/a9350a18-14e6-42c6-a150-0a1487f20ad9/disk" returned: 0 in 0.483s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:10:13 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-221919a0-9cdd-44ec-881f-d6c82383d31d tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Running cmd (subprocess): cp -r /opt/stack/data/nova/instances/a9350a18-14e6-42c6-a150-0a1487f20ad9_resize/disk.info /opt/stack/data/nova/instances/a9350a18-14e6-42c6-a150-0a1487f20ad9/disk.info {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:10:13 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-221919a0-9cdd-44ec-881f-d6c82383d31d tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] CMD "cp -r /opt/stack/data/nova/instances/a9350a18-14e6-42c6-a150-0a1487f20ad9_resize/disk.info /opt/stack/data/nova/instances/a9350a18-14e6-42c6-a150-0a1487f20ad9/disk.info" returned: 0 in 0.061s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:10:13 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:10:13 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:10:13 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:10:13 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:10:14 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:10:14 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:10:14 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:10:14 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:10:14 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:10:14 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:10:15 user nova-compute[75234]: DEBUG nova.compute.manager [req-6e6fb8ae-3277-4942-b5f6-4b31a18ed676 req-8cc16043-b65d-4324-830a-f8f379ce46d6 service nova] [instance: a9350a18-14e6-42c6-a150-0a1487f20ad9] Received event network-vif-plugged-52024efc-8cde-4610-b12c-198f19d75ccf {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:10:15 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-6e6fb8ae-3277-4942-b5f6-4b31a18ed676 req-8cc16043-b65d-4324-830a-f8f379ce46d6 service nova] Acquiring lock "a9350a18-14e6-42c6-a150-0a1487f20ad9-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:10:15 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-6e6fb8ae-3277-4942-b5f6-4b31a18ed676 req-8cc16043-b65d-4324-830a-f8f379ce46d6 service nova] Lock "a9350a18-14e6-42c6-a150-0a1487f20ad9-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:10:15 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-6e6fb8ae-3277-4942-b5f6-4b31a18ed676 req-8cc16043-b65d-4324-830a-f8f379ce46d6 service nova] Lock "a9350a18-14e6-42c6-a150-0a1487f20ad9-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:10:15 user nova-compute[75234]: DEBUG nova.compute.manager [req-6e6fb8ae-3277-4942-b5f6-4b31a18ed676 req-8cc16043-b65d-4324-830a-f8f379ce46d6 service nova] [instance: a9350a18-14e6-42c6-a150-0a1487f20ad9] No waiting events found dispatching network-vif-plugged-52024efc-8cde-4610-b12c-198f19d75ccf {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:10:15 user nova-compute[75234]: WARNING nova.compute.manager [req-6e6fb8ae-3277-4942-b5f6-4b31a18ed676 req-8cc16043-b65d-4324-830a-f8f379ce46d6 service nova] [instance: a9350a18-14e6-42c6-a150-0a1487f20ad9] Received unexpected event network-vif-plugged-52024efc-8cde-4610-b12c-198f19d75ccf for instance with vm_state active and task_state resize_migrating.
Nov 29 03:10:15 user nova-compute[75234]: DEBUG nova.compute.manager [req-6e6fb8ae-3277-4942-b5f6-4b31a18ed676 req-8cc16043-b65d-4324-830a-f8f379ce46d6 service nova] [instance: 8297c264-a213-44ee-be46-5c08eabda9f7] Received event network-vif-plugged-d26334d2-8040-4022-b658-69be1cf6a025 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:10:15 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-6e6fb8ae-3277-4942-b5f6-4b31a18ed676 req-8cc16043-b65d-4324-830a-f8f379ce46d6 service nova] Acquiring lock "8297c264-a213-44ee-be46-5c08eabda9f7-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:10:15 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-6e6fb8ae-3277-4942-b5f6-4b31a18ed676 req-8cc16043-b65d-4324-830a-f8f379ce46d6 service nova] Lock "8297c264-a213-44ee-be46-5c08eabda9f7-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:10:15 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-6e6fb8ae-3277-4942-b5f6-4b31a18ed676 req-8cc16043-b65d-4324-830a-f8f379ce46d6 service nova] Lock "8297c264-a213-44ee-be46-5c08eabda9f7-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:10:15 user nova-compute[75234]: DEBUG nova.compute.manager [req-6e6fb8ae-3277-4942-b5f6-4b31a18ed676 req-8cc16043-b65d-4324-830a-f8f379ce46d6 service nova] [instance: 8297c264-a213-44ee-be46-5c08eabda9f7] No waiting events found dispatching network-vif-plugged-d26334d2-8040-4022-b658-69be1cf6a025 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:10:15 user nova-compute[75234]: WARNING nova.compute.manager [req-6e6fb8ae-3277-4942-b5f6-4b31a18ed676 req-8cc16043-b65d-4324-830a-f8f379ce46d6 service nova] [instance: 8297c264-a213-44ee-be46-5c08eabda9f7] Received unexpected event network-vif-plugged-d26334d2-8040-4022-b658-69be1cf6a025 for instance with vm_state active and task_state rescuing.
Nov 29 03:10:15 user nova-compute[75234]: DEBUG nova.compute.manager [req-6e6fb8ae-3277-4942-b5f6-4b31a18ed676 req-8cc16043-b65d-4324-830a-f8f379ce46d6 service nova] [instance: 8297c264-a213-44ee-be46-5c08eabda9f7] Received event network-vif-plugged-d26334d2-8040-4022-b658-69be1cf6a025 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:10:15 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-6e6fb8ae-3277-4942-b5f6-4b31a18ed676 req-8cc16043-b65d-4324-830a-f8f379ce46d6 service nova] Acquiring lock "8297c264-a213-44ee-be46-5c08eabda9f7-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:10:15 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-6e6fb8ae-3277-4942-b5f6-4b31a18ed676 req-8cc16043-b65d-4324-830a-f8f379ce46d6 service nova] Lock "8297c264-a213-44ee-be46-5c08eabda9f7-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:10:15 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-6e6fb8ae-3277-4942-b5f6-4b31a18ed676 req-8cc16043-b65d-4324-830a-f8f379ce46d6 service nova] Lock "8297c264-a213-44ee-be46-5c08eabda9f7-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:10:15 user nova-compute[75234]: DEBUG nova.compute.manager [req-6e6fb8ae-3277-4942-b5f6-4b31a18ed676 req-8cc16043-b65d-4324-830a-f8f379ce46d6 service nova] [instance: 8297c264-a213-44ee-be46-5c08eabda9f7] No waiting events found dispatching network-vif-plugged-d26334d2-8040-4022-b658-69be1cf6a025 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:10:15 user nova-compute[75234]: WARNING nova.compute.manager [req-6e6fb8ae-3277-4942-b5f6-4b31a18ed676 req-8cc16043-b65d-4324-830a-f8f379ce46d6 service nova] [instance: 8297c264-a213-44ee-be46-5c08eabda9f7] Received unexpected event network-vif-plugged-d26334d2-8040-4022-b658-69be1cf6a025 for instance with vm_state active and task_state rescuing.
Nov 29 03:10:15 user nova-compute[75234]: DEBUG nova.compute.manager [req-6e6fb8ae-3277-4942-b5f6-4b31a18ed676 req-8cc16043-b65d-4324-830a-f8f379ce46d6 service nova] [instance: f609a7b7-1be2-44b4-9177-7e72a4d4c060] Received event network-vif-plugged-5543f379-2b3a-4dc2-bfe9-52f66a942257 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:10:15 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-6e6fb8ae-3277-4942-b5f6-4b31a18ed676 req-8cc16043-b65d-4324-830a-f8f379ce46d6 service nova] Acquiring lock "f609a7b7-1be2-44b4-9177-7e72a4d4c060-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:10:15 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-6e6fb8ae-3277-4942-b5f6-4b31a18ed676 req-8cc16043-b65d-4324-830a-f8f379ce46d6 service nova] Lock "f609a7b7-1be2-44b4-9177-7e72a4d4c060-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:10:15 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-6e6fb8ae-3277-4942-b5f6-4b31a18ed676 req-8cc16043-b65d-4324-830a-f8f379ce46d6 service nova] Lock "f609a7b7-1be2-44b4-9177-7e72a4d4c060-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:10:15 user nova-compute[75234]: DEBUG nova.compute.manager [req-6e6fb8ae-3277-4942-b5f6-4b31a18ed676 req-8cc16043-b65d-4324-830a-f8f379ce46d6 service nova] [instance: f609a7b7-1be2-44b4-9177-7e72a4d4c060] No waiting events found dispatching network-vif-plugged-5543f379-2b3a-4dc2-bfe9-52f66a942257 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:10:15 user nova-compute[75234]: WARNING nova.compute.manager [req-6e6fb8ae-3277-4942-b5f6-4b31a18ed676 req-8cc16043-b65d-4324-830a-f8f379ce46d6 service nova] [instance: f609a7b7-1be2-44b4-9177-7e72a4d4c060] Received unexpected event network-vif-plugged-5543f379-2b3a-4dc2-bfe9-52f66a942257 for instance with vm_state stopped and task_state powering-on.
Nov 29 03:10:15 user nova-compute[75234]: DEBUG nova.network.neutron [None req-221919a0-9cdd-44ec-881f-d6c82383d31d tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] [instance: a9350a18-14e6-42c6-a150-0a1487f20ad9] Port 52024efc-8cde-4610-b12c-198f19d75ccf binding to destination host user is already ACTIVE {{(pid=75234) migrate_instance_start /opt/stack/nova/nova/network/neutron.py:3170}}
Nov 29 03:10:15 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-221919a0-9cdd-44ec-881f-d6c82383d31d tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Acquiring lock "a9350a18-14e6-42c6-a150-0a1487f20ad9-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:10:15 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-221919a0-9cdd-44ec-881f-d6c82383d31d tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Lock "a9350a18-14e6-42c6-a150-0a1487f20ad9-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.003s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:10:15 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-221919a0-9cdd-44ec-881f-d6c82383d31d tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Lock "a9350a18-14e6-42c6-a150-0a1487f20ad9-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:10:15 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:10:15 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:10:15 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-221919a0-9cdd-44ec-881f-d6c82383d31d tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Acquiring lock "refresh_cache-a9350a18-14e6-42c6-a150-0a1487f20ad9" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:10:15 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-221919a0-9cdd-44ec-881f-d6c82383d31d tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Acquired lock "refresh_cache-a9350a18-14e6-42c6-a150-0a1487f20ad9" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:10:15 user nova-compute[75234]: DEBUG nova.network.neutron [None req-221919a0-9cdd-44ec-881f-d6c82383d31d tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] [instance: a9350a18-14e6-42c6-a150-0a1487f20ad9] Building network info cache for instance {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2009}}
Nov 29 03:10:16 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:10:16 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:10:16 user nova-compute[75234]: DEBUG nova.network.neutron [None req-221919a0-9cdd-44ec-881f-d6c82383d31d tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] [instance: a9350a18-14e6-42c6-a150-0a1487f20ad9] Updating instance_info_cache with network_info: [{"id": "52024efc-8cde-4610-b12c-198f19d75ccf", "address": "fa:16:3e:09:b8:70", "network": {"id": "31c4ca3b-3a2f-47df-87fa-4ad095ee5cf9", "bridge": "br-int", "label": "tempest-ServerActionsTestOtherB-1680745008-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.9", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "ad5ea22c63f04f91a3f2fa665e4fc8a1", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap52024efc-8c", "ovs_interfaceid": "52024efc-8cde-4610-b12c-198f19d75ccf", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:10:16 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-221919a0-9cdd-44ec-881f-d6c82383d31d tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Releasing lock "refresh_cache-a9350a18-14e6-42c6-a150-0a1487f20ad9" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:10:16 user nova-compute[75234]: DEBUG os_brick.utils [None req-221919a0-9cdd-44ec-881f-d6c82383d31d tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:176}}
Nov 29 03:10:16 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:10:16 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.019s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:10:16 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[5efc4c07-d5bb-4f27-860d-9e098390da31]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:dd1368c5ba2c\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:10:16 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt -v / -n -o SOURCE {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:10:16 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt -v / -n -o SOURCE" returned: 0 in 0.023s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:10:16 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[052ca95c-75da-4a4e-906d-81d8ce588c23]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:10:16 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:10:16 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.021s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:10:16 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[62a7fa4e-a4e2-489d-a060-eeecf5f9ea8f]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:10:16 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[38752ad4-4768-40b1-aa3a-a347aace1005]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:10:16 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-221919a0-9cdd-44ec-881f-d6c82383d31d tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Running cmd (subprocess): nvme version {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:10:16 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-221919a0-9cdd-44ec-881f-d6c82383d31d tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] 'nvme version' failed. Not Retrying. {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:473}}
Nov 29 03:10:16 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.nvmeof [None req-221919a0-9cdd-44ec-881f-d6c82383d31d tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] nvme not present on system {{(pid=75234) nvme_present /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/nvmeof.py:756}}
Nov 29 03:10:16 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:10:16 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:473}}
Nov 29 03:10:16 user nova-compute[75234]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Nov 29 03:10:16 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[bc81b28c-f089-4a7c-bdf2-cfd7ab8230d3]: (4, '') {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:10:16 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.lightos [None req-221919a0-9cdd-44ec-881f-d6c82383d31d tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=75234) find_dsc /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/lightos.py:98}}
Nov 29 03:10:16 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.lightos [None req-221919a0-9cdd-44ec-881f-d6c82383d31d tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=75234) get_connector_properties /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/lightos.py:76}}
Nov 29 03:10:16 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.lightos [None req-221919a0-9cdd-44ec-881f-d6c82383d31d tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] LIGHTOS: no hostnqn found. {{(pid=75234) get_connector_properties /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/lightos.py:84}}
Nov 29 03:10:16 user nova-compute[75234]: DEBUG os_brick.utils [None req-221919a0-9cdd-44ec-881f-d6c82383d31d tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] <== get_connector_properties: return (144ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:dd1368c5ba2c', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:203}}
Nov 29 03:10:16 user nova-compute[75234]: DEBUG nova.compute.manager [None req-413e3555-a1c6-47b1-9af6-79aba9489343 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] [instance: 8297c264-a213-44ee-be46-5c08eabda9f7] Checking state {{(pid=75234) _get_power_state /opt/stack/nova/nova/compute/manager.py:1782}}
Nov 29 03:10:17 user nova-compute[75234]: DEBUG nova.compute.manager [None req-e0b104d6-8283-44f5-a150-3606c697b81a tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] [instance: f609a7b7-1be2-44b4-9177-7e72a4d4c060] Instance event wait completed in 0 seconds for  {{(pid=75234) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Nov 29 03:10:17 user nova-compute[75234]: INFO nova.virt.libvirt.driver [-] [instance: f609a7b7-1be2-44b4-9177-7e72a4d4c060] Instance rebooted successfully.
Nov 29 03:10:17 user nova-compute[75234]: DEBUG nova.compute.manager [None req-e0b104d6-8283-44f5-a150-3606c697b81a tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] [instance: f609a7b7-1be2-44b4-9177-7e72a4d4c060] Checking state {{(pid=75234) _get_power_state /opt/stack/nova/nova/compute/manager.py:1782}}
Nov 29 03:10:17 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:10:17 user nova-compute[75234]: DEBUG nova.compute.manager [req-3089c54b-f9c1-4308-a6a1-eb49aa2a965a req-e468dcc7-50be-4e41-993c-f452d2caeecc service nova] [instance: f609a7b7-1be2-44b4-9177-7e72a4d4c060] Received event network-vif-plugged-5543f379-2b3a-4dc2-bfe9-52f66a942257 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:10:17 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-3089c54b-f9c1-4308-a6a1-eb49aa2a965a req-e468dcc7-50be-4e41-993c-f452d2caeecc service nova] Acquiring lock "f609a7b7-1be2-44b4-9177-7e72a4d4c060-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:10:17 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-3089c54b-f9c1-4308-a6a1-eb49aa2a965a req-e468dcc7-50be-4e41-993c-f452d2caeecc service nova] Lock "f609a7b7-1be2-44b4-9177-7e72a4d4c060-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:10:17 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-3089c54b-f9c1-4308-a6a1-eb49aa2a965a req-e468dcc7-50be-4e41-993c-f452d2caeecc service nova] Lock "f609a7b7-1be2-44b4-9177-7e72a4d4c060-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:10:17 user nova-compute[75234]: DEBUG nova.compute.manager [req-3089c54b-f9c1-4308-a6a1-eb49aa2a965a req-e468dcc7-50be-4e41-993c-f452d2caeecc service nova] [instance: f609a7b7-1be2-44b4-9177-7e72a4d4c060] No waiting events found dispatching network-vif-plugged-5543f379-2b3a-4dc2-bfe9-52f66a942257 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:10:17 user nova-compute[75234]: WARNING nova.compute.manager [req-3089c54b-f9c1-4308-a6a1-eb49aa2a965a req-e468dcc7-50be-4e41-993c-f452d2caeecc service nova] [instance: f609a7b7-1be2-44b4-9177-7e72a4d4c060] Received unexpected event network-vif-plugged-5543f379-2b3a-4dc2-bfe9-52f66a942257 for instance with vm_state active and task_state None.
Nov 29 03:10:17 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:10:20 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:10:21 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:10:21 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:10:22 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:10:22 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:10:24 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:10:25 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-046208e3-2011-4422-97ac-ba2e4f78cd46 tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] Acquiring lock "d3b90f39-4b91-4152-b820-bd0e263eec89" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:10:25 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-046208e3-2011-4422-97ac-ba2e4f78cd46 tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] Lock "d3b90f39-4b91-4152-b820-bd0e263eec89" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:10:25 user nova-compute[75234]: DEBUG nova.compute.manager [None req-046208e3-2011-4422-97ac-ba2e4f78cd46 tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] [instance: d3b90f39-4b91-4152-b820-bd0e263eec89] Starting instance... {{(pid=75234) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2422}}
Nov 29 03:10:25 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-221919a0-9cdd-44ec-881f-d6c82383d31d tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] [instance: a9350a18-14e6-42c6-a150-0a1487f20ad9] Starting finish_migration {{(pid=75234) finish_migration /opt/stack/nova/nova/virt/libvirt/driver.py:11671}}
Nov 29 03:10:25 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-221919a0-9cdd-44ec-881f-d6c82383d31d tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] [instance: a9350a18-14e6-42c6-a150-0a1487f20ad9] Instance directory exists: not creating {{(pid=75234) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4723}}
Nov 29 03:10:25 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-221919a0-9cdd-44ec-881f-d6c82383d31d tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] [instance: a9350a18-14e6-42c6-a150-0a1487f20ad9] Creating image(s)
Nov 29 03:10:25 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-221919a0-9cdd-44ec-881f-d6c82383d31d tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:10:25 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-221919a0-9cdd-44ec-881f-d6c82383d31d tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json" returned: 0 in 0.177s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:10:25 user nova-compute[75234]: DEBUG nova.virt.disk.api [None req-221919a0-9cdd-44ec-881f-d6c82383d31d tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Checking if we can resize image /opt/stack/data/nova/instances/a9350a18-14e6-42c6-a150-0a1487f20ad9/disk. size=21474836480 {{(pid=75234) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:166}}
Nov 29 03:10:25 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-221919a0-9cdd-44ec-881f-d6c82383d31d tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a9350a18-14e6-42c6-a150-0a1487f20ad9/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:10:25 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-046208e3-2011-4422-97ac-ba2e4f78cd46 tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:10:25 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-046208e3-2011-4422-97ac-ba2e4f78cd46 tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.002s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:10:25 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-046208e3-2011-4422-97ac-ba2e4f78cd46 tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=75234) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2366}}
Nov 29 03:10:25 user nova-compute[75234]: INFO nova.compute.claims [None req-046208e3-2011-4422-97ac-ba2e4f78cd46 tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] [instance: d3b90f39-4b91-4152-b820-bd0e263eec89] Claim successful on node user
Nov 29 03:10:25 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-221919a0-9cdd-44ec-881f-d6c82383d31d tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a9350a18-14e6-42c6-a150-0a1487f20ad9/disk --force-share --output=json" returned: 0 in 0.212s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:10:25 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-221919a0-9cdd-44ec-881f-d6c82383d31d tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Running cmd (subprocess): qemu-img resize /opt/stack/data/nova/instances/a9350a18-14e6-42c6-a150-0a1487f20ad9/disk 21474836480 {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:10:25 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-221919a0-9cdd-44ec-881f-d6c82383d31d tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] CMD "qemu-img resize /opt/stack/data/nova/instances/a9350a18-14e6-42c6-a150-0a1487f20ad9/disk 21474836480" returned: 0 in 0.104s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:10:25 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-221919a0-9cdd-44ec-881f-d6c82383d31d tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] [instance: a9350a18-14e6-42c6-a150-0a1487f20ad9] Did not create local disks {{(pid=75234) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4863}}
Nov 29 03:10:25 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-221919a0-9cdd-44ec-881f-d6c82383d31d tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] [instance: a9350a18-14e6-42c6-a150-0a1487f20ad9] Ensure instance console log exists: /opt/stack/data/nova/instances/a9350a18-14e6-42c6-a150-0a1487f20ad9/console.log {{(pid=75234) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4613}}
Nov 29 03:10:25 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-221919a0-9cdd-44ec-881f-d6c82383d31d tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:10:25 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-221919a0-9cdd-44ec-881f-d6c82383d31d tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:10:25 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-221919a0-9cdd-44ec-881f-d6c82383d31d tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:10:25 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-221919a0-9cdd-44ec-881f-d6c82383d31d tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] [instance: a9350a18-14e6-42c6-a150-0a1487f20ad9] Start _get_guest_xml network_info=[{"id": "52024efc-8cde-4610-b12c-198f19d75ccf", "address": "fa:16:3e:09:b8:70", "network": {"id": "31c4ca3b-3a2f-47df-87fa-4ad095ee5cf9", "bridge": "br-int", "label": "tempest-ServerActionsTestOtherB-1680745008-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.9", "type": "floating", "version": 4, "meta": {}}], "label": "tempest-ServerActionsTestOtherB-1680745008-network", "vif_mac": "fa:16:3e:09:b8:70"}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "ad5ea22c63f04f91a3f2fa665e4fc8a1", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap52024efc-8c", "ovs_interfaceid": "52024efc-8cde-4610-b12c-198f19d75ccf", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, '/dev/vdb': {'bus': 'virtio', 'dev': 'vdb', 'type': 'disk'}}} image_meta=ImageMeta(checksum='c8fc807773e5354afe61636071771906',container_format='bare',created_at=2023-11-29T03:02:15Z,direct_url=<?>,disk_format='qcow2',id=96c30ba7-9b73-4a30-8ba4-7138879573b7,min_disk=0,min_ram=0,name='cirros-0.6.2-x86_64-disk',owner='b18e380921c748b78517b6f1211cd0f1',properties=ImageMetaProps,protected=<?>,size=21430272,status='active',tags=<?>,updated_at=2023-11-29T03:02:17Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [{'device_name': '/dev/vda', 'encryption_format': None, 'encrypted': False, 'device_type': 'disk', 'size': 0, 'disk_bus': 'virtio', 'boot_index': 0, 'encryption_options': None, 'encryption_secret_uuid': None, 'guest_format': None, 'image_id': '96c30ba7-9b73-4a30-8ba4-7138879573b7'}], 'ephemerals': [], 'block_device_mapping': [{'mount_device': '/dev/vdb', 'attachment_id': 'adda2d45-58ff-4b48-b5dd-2486177875b3', 'disk_bus': 'virtio', 'device_type': 'disk', 'boot_index': None, 'guest_format': None, 'delete_on_termination': False, 'connection_info': {'driver_volume_type': 'iscsi', 'data': {'target_iqn': 'iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'NFCsGdbh', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}, 'status': 'attaching', 'instance': 'a9350a18-14e6-42c6-a150-0a1487f20ad9', 'attached_at': '2023-11-29T03:10:25.000000', 'detached_at': '', 'volume_id': 'f4130d3c-b329-4ce1-8b6d-d9e270a43b0b', 'serial': 'f4130d3c-b329-4ce1-8b6d-d9e270a43b0b'}, 'volume_type': None} {{(pid=75234) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7562}}
Nov 29 03:10:25 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-221919a0-9cdd-44ec-881f-d6c82383d31d tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:10:25 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-221919a0-9cdd-44ec-881f-d6c82383d31d tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:10:25 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-221919a0-9cdd-44ec-881f-d6c82383d31d tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Searching host: 'user' for CPU controller through CGroups V1... {{(pid=75234) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1652}}
Nov 29 03:10:25 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-221919a0-9cdd-44ec-881f-d6c82383d31d tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] CPU controller missing on host. {{(pid=75234) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1662}}
Nov 29 03:10:25 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-221919a0-9cdd-44ec-881f-d6c82383d31d tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Searching host: 'user' for CPU controller through CGroups V2... {{(pid=75234) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1671}}
Nov 29 03:10:25 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-221919a0-9cdd-44ec-881f-d6c82383d31d tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] CPU controller found on host. {{(pid=75234) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1678}}
Nov 29 03:10:25 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-221919a0-9cdd-44ec-881f-d6c82383d31d tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=75234) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5400}}
Nov 29 03:10:25 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-221919a0-9cdd-44ec-881f-d6c82383d31d tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-11-29T03:02:06Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='2',id=7,is_public=True,memory_mb=2048,name='m1.small',projects=<?>,root_gb=20,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='c8fc807773e5354afe61636071771906',container_format='bare',created_at=2023-11-29T03:02:15Z,direct_url=<?>,disk_format='qcow2',id=96c30ba7-9b73-4a30-8ba4-7138879573b7,min_disk=0,min_ram=0,name='cirros-0.6.2-x86_64-disk',owner='b18e380921c748b78517b6f1211cd0f1',properties=ImageMetaProps,protected=<?>,size=21430272,status='active',tags=<?>,updated_at=2023-11-29T03:02:17Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:563}}
Nov 29 03:10:25 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-221919a0-9cdd-44ec-881f-d6c82383d31d tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Flavor limits 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:348}}
Nov 29 03:10:25 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-221919a0-9cdd-44ec-881f-d6c82383d31d tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Image limits 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:352}}
Nov 29 03:10:25 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-221919a0-9cdd-44ec-881f-d6c82383d31d tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Flavor pref 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:388}}
Nov 29 03:10:25 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-221919a0-9cdd-44ec-881f-d6c82383d31d tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Image pref 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:392}}
Nov 29 03:10:25 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-221919a0-9cdd-44ec-881f-d6c82383d31d tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:430}}
Nov 29 03:10:25 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-221919a0-9cdd-44ec-881f-d6c82383d31d tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:569}}
Nov 29 03:10:25 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-221919a0-9cdd-44ec-881f-d6c82383d31d tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=75234) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:471}}
Nov 29 03:10:25 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-221919a0-9cdd-44ec-881f-d6c82383d31d tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Got 1 possible topologies {{(pid=75234) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:501}}
Nov 29 03:10:25 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-221919a0-9cdd-44ec-881f-d6c82383d31d tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:575}}
Nov 29 03:10:25 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-221919a0-9cdd-44ec-881f-d6c82383d31d tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:577}}
Nov 29 03:10:25 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-221919a0-9cdd-44ec-881f-d6c82383d31d tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] [instance: a9350a18-14e6-42c6-a150-0a1487f20ad9] Calling os-brick to attach iSCSI Volume {{(pid=75234) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Nov 29 03:10:25 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-221919a0-9cdd-44ec-881f-d6c82383d31d tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f8a5050c9a0>, 'connection_properties': {'target_iqn': 'iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'NFCsGdbh', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:176}}
Nov 29 03:10:25 user nova-compute[75234]: WARNING os_brick.initiator.connectors.base [None req-221919a0-9cdd-44ec-881f-d6c82383d31d tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Nov 29 03:10:25 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-221919a0-9cdd-44ec-881f-d6c82383d31d tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:68}}
Nov 29 03:10:25 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-221919a0-9cdd-44ec-881f-d6c82383d31d tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:73}}
Nov 29 03:10:25 user nova-compute[75234]: WARNING os_brick.initiator.connectors.base [None req-221919a0-9cdd-44ec-881f-d6c82383d31d tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Nov 29 03:10:25 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-221919a0-9cdd-44ec-881f-d6c82383d31d tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:68}}
Nov 29 03:10:25 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-221919a0-9cdd-44ec-881f-d6c82383d31d tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:73}}
Nov 29 03:10:25 user nova-compute[75234]: INFO os_brick.initiator.connectors.iscsi [None req-221919a0-9cdd-44ec-881f-d6c82383d31d tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Nov 29 03:10:25 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b -p 172.16.0.220:3260 {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:10:25 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b -p 172.16.0.220:3260" returned: 21 in 0.023s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:10:25 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[6f5fb780-7c7a-4bcb-89cd-8cab82b1e2d4]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:10:25 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-221919a0-9cdd-44ec-881f-d6c82383d31d tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Nov 29 03:10:25 user nova-compute[75234]:  {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:10:25 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b -p 172.16.0.220:3260 --interface default --op new {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:10:25 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.024s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:10:25 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[56231c85-db37-4eb8-a41d-55ca9bf71ffa]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b] added\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:10:25 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-221919a0-9cdd-44ec-881f-d6c82383d31d tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b] added
Nov 29 03:10:25 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:10:25 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:10:25 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.023s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:10:25 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[01408f24-8eaa-4b4c-ba27-4be35de04411]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:10:25 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-221919a0-9cdd-44ec-881f-d6c82383d31d tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:10:25 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:10:26 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.024s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:10:26 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[910786d9-22ee-4ea2-bc1a-8432cdbf3ab2]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:10:26 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-221919a0-9cdd-44ec-881f-d6c82383d31d tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:10:26 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b -p 172.16.0.220:3260 --op update -n node.session.auth.username -v NFCsGdbh {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:10:26 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b -p 172.16.0.220:3260 --op update -n node.session.auth.username -v NFCsGdbh" returned: 0 in 0.033s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:10:26 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[37d04341-42b5-40e2-9f1a-8d7e7f62766b]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:10:26 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-221919a0-9cdd-44ec-881f-d6c82383d31d tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'NFCsGdbh'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:10:26 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:10:26 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.023s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:10:26 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[60a6efd6-43b7-4ba5-ac5d-9db66005f420]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:10:26 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-221919a0-9cdd-44ec-881f-d6c82383d31d tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:10:26 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:10:26 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.038s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:10:26 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[e5fc355c-f396-476c-b828-e80e33ef298d]: (4, ('tcp: [12] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ada91c26-ca97-4b8b-98bb-a310d0627691 (non-flash)\ntcp: [15] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a7ef35bf-017a-4fd9-9182-79c1b2ab4f98 (non-flash)\ntcp: [16] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672 (non-flash)\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:10:26 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-221919a0-9cdd-44ec-881f-d6c82383d31d tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [12] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ada91c26-ca97-4b8b-98bb-a310d0627691 (non-flash)
Nov 29 03:10:26 user nova-compute[75234]: tcp: [15] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a7ef35bf-017a-4fd9-9182-79c1b2ab4f98 (non-flash)
Nov 29 03:10:26 user nova-compute[75234]: tcp: [16] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672 (non-flash)
Nov 29 03:10:26 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:10:26 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-221919a0-9cdd-44ec-881f-d6c82383d31d tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] iscsi session list stdout=tcp: [12] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ada91c26-ca97-4b8b-98bb-a310d0627691 (non-flash)
Nov 29 03:10:26 user nova-compute[75234]: tcp: [15] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a7ef35bf-017a-4fd9-9182-79c1b2ab4f98 (non-flash)
Nov 29 03:10:26 user nova-compute[75234]: tcp: [16] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672 (non-flash)
Nov 29 03:10:26 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsi_session /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Nov 29 03:10:26 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b -p 172.16.0.220:3260 --login {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:10:26 user nova-compute[75234]: DEBUG nova.compute.provider_tree [None req-046208e3-2011-4422-97ac-ba2e4f78cd46 tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] Inventory has not changed in ProviderTree for provider: 4d291220-7c37-4839-a0e3-b23c66502b74 {{(pid=75234) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Nov 29 03:10:26 user nova-compute[75234]: DEBUG nova.scheduler.client.report [None req-046208e3-2011-4422-97ac-ba2e4f78cd46 tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] Inventory has not changed for provider 4d291220-7c37-4839-a0e3-b23c66502b74 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=75234) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Nov 29 03:10:26 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-046208e3-2011-4422-97ac-ba2e4f78cd46 tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.703s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:10:26 user nova-compute[75234]: DEBUG nova.compute.manager [None req-046208e3-2011-4422-97ac-ba2e4f78cd46 tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] [instance: d3b90f39-4b91-4152-b820-bd0e263eec89] Start building networks asynchronously for instance. {{(pid=75234) _build_resources /opt/stack/nova/nova/compute/manager.py:2819}}
Nov 29 03:10:26 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b -p 172.16.0.220:3260 --login" returned: 0 in 0.099s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:10:26 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[d260e95e-4004-42a2-83c7-e2124d1d32ed]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:10:26 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-221919a0-9cdd-44ec-881f-d6c82383d31d tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b, portal: 172.16.0.220,3260]
Nov 29 03:10:26 user nova-compute[75234]: Login to [iface: default, target: iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b, portal: 172.16.0.220,3260] successful.
Nov 29 03:10:26 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:10:26 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:10:26 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.024s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:10:26 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[075eee56-36e3-4631-9cc0-0e5fca806be7]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:10:26 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-221919a0-9cdd-44ec-881f-d6c82383d31d tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:10:26 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:10:26 user nova-compute[75234]: DEBUG nova.compute.manager [None req-046208e3-2011-4422-97ac-ba2e4f78cd46 tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] [instance: d3b90f39-4b91-4152-b820-bd0e263eec89] Allocating IP information in the background. {{(pid=75234) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1972}}
Nov 29 03:10:26 user nova-compute[75234]: DEBUG nova.network.neutron [None req-046208e3-2011-4422-97ac-ba2e4f78cd46 tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] [instance: d3b90f39-4b91-4152-b820-bd0e263eec89] allocate_for_instance() {{(pid=75234) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1156}}
Nov 29 03:10:26 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.027s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:10:26 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[e6264b27-2065-496e-8cd3-1c08131f2058]: (4, ('tcp: [12] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ada91c26-ca97-4b8b-98bb-a310d0627691 (non-flash)\ntcp: [15] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a7ef35bf-017a-4fd9-9182-79c1b2ab4f98 (non-flash)\ntcp: [16] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672 (non-flash)\ntcp: [17] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b (non-flash)\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:10:26 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-221919a0-9cdd-44ec-881f-d6c82383d31d tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [12] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ada91c26-ca97-4b8b-98bb-a310d0627691 (non-flash)
Nov 29 03:10:26 user nova-compute[75234]: tcp: [15] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a7ef35bf-017a-4fd9-9182-79c1b2ab4f98 (non-flash)
Nov 29 03:10:26 user nova-compute[75234]: tcp: [16] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672 (non-flash)
Nov 29 03:10:26 user nova-compute[75234]: tcp: [17] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b (non-flash)
Nov 29 03:10:26 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:10:26 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-221919a0-9cdd-44ec-881f-d6c82383d31d tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] iscsi session list stdout=tcp: [12] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ada91c26-ca97-4b8b-98bb-a310d0627691 (non-flash)
Nov 29 03:10:26 user nova-compute[75234]: tcp: [15] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a7ef35bf-017a-4fd9-9182-79c1b2ab4f98 (non-flash)
Nov 29 03:10:26 user nova-compute[75234]: tcp: [16] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672 (non-flash)
Nov 29 03:10:26 user nova-compute[75234]: tcp: [17] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b (non-flash)
Nov 29 03:10:26 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsi_session /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Nov 29 03:10:26 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-221919a0-9cdd-44ec-881f-d6c82383d31d tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.386s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:87}}
Nov 29 03:10:26 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-221919a0-9cdd-44ec-881f-d6c82383d31d tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Connected to 172.16.0.220:3260 {{(pid=75234) _connect_vol /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:644}}
Nov 29 03:10:26 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-221919a0-9cdd-44ec-881f-d6c82383d31d tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] HCTL ('34', '-', '-', 0) found on session 17 with lun 0 {{(pid=75234) get_hctl /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:770}}
Nov 29 03:10:26 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-221919a0-9cdd-44ec-881f-d6c82383d31d tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Scanning host 34 c: -, t: -, l: 0) {{(pid=75234) scan_iscsi /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:795}}
Nov 29 03:10:26 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host34/scan {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:10:26 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-97e2cf32-f7d5-4ab9-a932-35fa0c7cd310 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Acquiring lock "13e5da64-9adc-4479-81c7-d329e9b5f08f" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:10:26 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-97e2cf32-f7d5-4ab9-a932-35fa0c7cd310 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Lock "13e5da64-9adc-4479-81c7-d329e9b5f08f" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:10:26 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host34/scan" returned: 0 in 0.025s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:10:26 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[cff2d36e-78e1-48df-9c24-1bdab6fb649c]: (4, ('- - 0', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:10:26 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-221919a0-9cdd-44ec-881f-d6c82383d31d tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Searching for a device in session 17 and hctl ['34', '*', '*', 0] yield: sdc {{(pid=75234) device_name_by_hctl /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:789}}
Nov 29 03:10:26 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-221919a0-9cdd-44ec-881f-d6c82383d31d tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Connected to sdc using {'target_iqn': 'iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'NFCsGdbh', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False} {{(pid=75234) _connect_vol /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:672}}
Nov 29 03:10:26 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-046208e3-2011-4422-97ac-ba2e4f78cd46 tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] [instance: d3b90f39-4b91-4152-b820-bd0e263eec89] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Nov 29 03:10:26 user nova-compute[75234]: DEBUG nova.compute.manager [None req-97e2cf32-f7d5-4ab9-a932-35fa0c7cd310 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] [instance: 13e5da64-9adc-4479-81c7-d329e9b5f08f] Starting instance... {{(pid=75234) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2422}}
Nov 29 03:10:26 user nova-compute[75234]: DEBUG nova.compute.manager [None req-046208e3-2011-4422-97ac-ba2e4f78cd46 tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] [instance: d3b90f39-4b91-4152-b820-bd0e263eec89] Start building block device mappings for instance. {{(pid=75234) _build_resources /opt/stack/nova/nova/compute/manager.py:2854}}
Nov 29 03:10:26 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-97e2cf32-f7d5-4ab9-a932-35fa0c7cd310 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:10:26 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-97e2cf32-f7d5-4ab9-a932-35fa0c7cd310 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:10:26 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-97e2cf32-f7d5-4ab9-a932-35fa0c7cd310 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=75234) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2366}}
Nov 29 03:10:26 user nova-compute[75234]: INFO nova.compute.claims [None req-97e2cf32-f7d5-4ab9-a932-35fa0c7cd310 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] [instance: 13e5da64-9adc-4479-81c7-d329e9b5f08f] Claim successful on node user
Nov 29 03:10:26 user nova-compute[75234]: DEBUG nova.policy [None req-046208e3-2011-4422-97ac-ba2e4f78cd46 tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '460854be8e05408a80790620f64cf8e1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5eaa91ebc2e64f2eba987f02bcd1a9f4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=75234) authorize /opt/stack/nova/nova/policy.py:203}}
Nov 29 03:10:26 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:10:26 user nova-compute[75234]: DEBUG nova.compute.manager [None req-046208e3-2011-4422-97ac-ba2e4f78cd46 tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] [instance: d3b90f39-4b91-4152-b820-bd0e263eec89] Start spawning the instance on the hypervisor. {{(pid=75234) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2628}}
Nov 29 03:10:26 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-046208e3-2011-4422-97ac-ba2e4f78cd46 tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] [instance: d3b90f39-4b91-4152-b820-bd0e263eec89] Creating instance directory {{(pid=75234) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4727}}
Nov 29 03:10:26 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-046208e3-2011-4422-97ac-ba2e4f78cd46 tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] [instance: d3b90f39-4b91-4152-b820-bd0e263eec89] Creating image(s)
Nov 29 03:10:26 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-046208e3-2011-4422-97ac-ba2e4f78cd46 tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] Acquiring lock "/opt/stack/data/nova/instances/d3b90f39-4b91-4152-b820-bd0e263eec89/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:10:26 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-046208e3-2011-4422-97ac-ba2e4f78cd46 tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] Lock "/opt/stack/data/nova/instances/d3b90f39-4b91-4152-b820-bd0e263eec89/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:10:26 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-046208e3-2011-4422-97ac-ba2e4f78cd46 tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] Lock "/opt/stack/data/nova/instances/d3b90f39-4b91-4152-b820-bd0e263eec89/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.006s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:10:26 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-046208e3-2011-4422-97ac-ba2e4f78cd46 tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:10:26 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-046208e3-2011-4422-97ac-ba2e4f78cd46 tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json" returned: 0 in 0.279s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:10:26 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-046208e3-2011-4422-97ac-ba2e4f78cd46 tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] Acquiring lock "80686e29dc9db6600d03e85cb62af298e12d7918" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:10:26 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-046208e3-2011-4422-97ac-ba2e4f78cd46 tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] Lock "80686e29dc9db6600d03e85cb62af298e12d7918" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:10:26 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-046208e3-2011-4422-97ac-ba2e4f78cd46 tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:10:27 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-046208e3-2011-4422-97ac-ba2e4f78cd46 tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json" returned: 0 in 0.204s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:10:27 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-046208e3-2011-4422-97ac-ba2e4f78cd46 tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918,backing_fmt=raw /opt/stack/data/nova/instances/d3b90f39-4b91-4152-b820-bd0e263eec89/disk 1073741824 {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:10:27 user nova-compute[75234]: DEBUG nova.compute.provider_tree [None req-97e2cf32-f7d5-4ab9-a932-35fa0c7cd310 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Inventory has not changed in ProviderTree for provider: 4d291220-7c37-4839-a0e3-b23c66502b74 {{(pid=75234) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Nov 29 03:10:27 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-046208e3-2011-4422-97ac-ba2e4f78cd46 tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918,backing_fmt=raw /opt/stack/data/nova/instances/d3b90f39-4b91-4152-b820-bd0e263eec89/disk 1073741824" returned: 0 in 0.072s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:10:27 user nova-compute[75234]: DEBUG nova.scheduler.client.report [None req-97e2cf32-f7d5-4ab9-a932-35fa0c7cd310 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Inventory has not changed for provider 4d291220-7c37-4839-a0e3-b23c66502b74 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=75234) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Nov 29 03:10:27 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-046208e3-2011-4422-97ac-ba2e4f78cd46 tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] Lock "80686e29dc9db6600d03e85cb62af298e12d7918" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.297s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:10:27 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-046208e3-2011-4422-97ac-ba2e4f78cd46 tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:10:27 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-97e2cf32-f7d5-4ab9-a932-35fa0c7cd310 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.697s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:10:27 user nova-compute[75234]: DEBUG nova.compute.manager [None req-97e2cf32-f7d5-4ab9-a932-35fa0c7cd310 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] [instance: 13e5da64-9adc-4479-81c7-d329e9b5f08f] Start building networks asynchronously for instance. {{(pid=75234) _build_resources /opt/stack/nova/nova/compute/manager.py:2819}}
Nov 29 03:10:27 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ddae26ab-aba2-44e2-9f96-a517489c193b tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Acquiring lock "a082c96e-e384-40b6-bcc8-fc5e0e104669" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:10:27 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ddae26ab-aba2-44e2-9f96-a517489c193b tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Lock "a082c96e-e384-40b6-bcc8-fc5e0e104669" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:10:27 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ddae26ab-aba2-44e2-9f96-a517489c193b tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] [instance: a082c96e-e384-40b6-bcc8-fc5e0e104669] Starting instance... {{(pid=75234) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2422}}
Nov 29 03:10:27 user nova-compute[75234]: DEBUG nova.compute.manager [None req-97e2cf32-f7d5-4ab9-a932-35fa0c7cd310 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] [instance: 13e5da64-9adc-4479-81c7-d329e9b5f08f] Allocating IP information in the background. {{(pid=75234) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1972}}
Nov 29 03:10:27 user nova-compute[75234]: DEBUG nova.network.neutron [None req-97e2cf32-f7d5-4ab9-a932-35fa0c7cd310 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] [instance: 13e5da64-9adc-4479-81c7-d329e9b5f08f] allocate_for_instance() {{(pid=75234) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1156}}
Nov 29 03:10:27 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-97e2cf32-f7d5-4ab9-a932-35fa0c7cd310 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] [instance: 13e5da64-9adc-4479-81c7-d329e9b5f08f] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Nov 29 03:10:27 user nova-compute[75234]: DEBUG nova.compute.manager [None req-97e2cf32-f7d5-4ab9-a932-35fa0c7cd310 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] [instance: 13e5da64-9adc-4479-81c7-d329e9b5f08f] Start building block device mappings for instance. {{(pid=75234) _build_resources /opt/stack/nova/nova/compute/manager.py:2854}}
Nov 29 03:10:27 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-221919a0-9cdd-44ec-881f-d6c82383d31d tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.444s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:87}}
Nov 29 03:10:27 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-221919a0-9cdd-44ec-881f-d6c82383d31d tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] <== connect_volume: return (1445ms) {'type': 'block', 'scsi_wwn': '26539313062336265', 'path': '/dev/sdc'} {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:203}}
Nov 29 03:10:27 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-221919a0-9cdd-44ec-881f-d6c82383d31d tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] [instance: a9350a18-14e6-42c6-a150-0a1487f20ad9] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '26539313062336265', 'path': '/dev/sdc'} {{(pid=75234) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Nov 29 03:10:27 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-046208e3-2011-4422-97ac-ba2e4f78cd46 tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json" returned: 0 in 0.213s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:10:27 user nova-compute[75234]: DEBUG nova.virt.disk.api [None req-046208e3-2011-4422-97ac-ba2e4f78cd46 tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] Checking if we can resize image /opt/stack/data/nova/instances/d3b90f39-4b91-4152-b820-bd0e263eec89/disk. size=1073741824 {{(pid=75234) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:166}}
Nov 29 03:10:27 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-046208e3-2011-4422-97ac-ba2e4f78cd46 tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/d3b90f39-4b91-4152-b820-bd0e263eec89/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:10:27 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ddae26ab-aba2-44e2-9f96-a517489c193b tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:10:27 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ddae26ab-aba2-44e2-9f96-a517489c193b tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.002s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:10:27 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:10:27 user nova-compute[75234]: DEBUG nova.policy [None req-97e2cf32-f7d5-4ab9-a932-35fa0c7cd310 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '6792e866677645c598ddb7d2c1723a1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f8a7884867c944e1ba1024270e4902f3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=75234) authorize /opt/stack/nova/nova/policy.py:203}}
Nov 29 03:10:27 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-221919a0-9cdd-44ec-881f-d6c82383d31d tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-11-29T03:06:06Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-ServerActionsTestOtherB-server-372321920',display_name='tempest-ServerActionsTestOtherB-server-372321920',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(7),hidden=False,host='user',hostname='tempest-serveractionstestotherb-server-372321920',id=5,image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',info_cache=InstanceInfoCache,instance_type_id=7,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBFOggqgO1QPxMZPb2P6SX24mx6LOsfQfrzx4GSpNnJiixHqqOn61ObLwI9/TG/r6plC8w1huVCtu+OqAyk9QXUlEr0+VsPNunPSN9WPXBAM4+U+3iCOgHXExdr++34pXkw==',key_name='tempest-keypair-1692341699',keypairs=<?>,launch_index=0,launched_at=2023-11-29T03:06:23Z,launched_on='user',locked=False,locked_by=None,memory_mb=2048,metadata={},migration_context=MigrationContext,new_flavor=Flavor(7),node='user',numa_topology=None,old_flavor=Flavor(6),os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=1,progress=0,project_id='ad5ea22c63f04f91a3f2fa665e4fc8a1',ramdisk_id='',reservation_id='r-f5lkid4e',resources=None,root_device_name='/dev/vda',root_gb=20,security_groups=SecurityGroupList,services=ServiceList,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',old_vm_state='active',owner_project_name='tempest-ServerActionsTestOtherB-70688183',owner_user_name='tempest-ServerActionsTestOtherB-70688183-project-member'},tags=<?>,task_state='resize_finish',terminated_at=None,trusted_certs=None,updated_at=2023-11-29T03:10:15Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='bdddba0ef9c44b6cb43c32a9ab0e29f6',uuid=a9350a18-14e6-42c6-a150-0a1487f20ad9,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "52024efc-8cde-4610-b12c-198f19d75ccf", "address": "fa:16:3e:09:b8:70", "network": {"id": "31c4ca3b-3a2f-47df-87fa-4ad095ee5cf9", "bridge": "br-int", "label": "tempest-ServerActionsTestOtherB-1680745008-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.9", "type": "floating", "version": 4, "meta": {}}], "label": "tempest-ServerActionsTestOtherB-1680745008-network", "vif_mac": "fa:16:3e:09:b8:70"}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "ad5ea22c63f04f91a3f2fa665e4fc8a1", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap52024efc-8c", "ovs_interfaceid": "52024efc-8cde-4610-b12c-198f19d75ccf", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=75234) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Nov 29 03:10:27 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-221919a0-9cdd-44ec-881f-d6c82383d31d tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Converting VIF {"id": "52024efc-8cde-4610-b12c-198f19d75ccf", "address": "fa:16:3e:09:b8:70", "network": {"id": "31c4ca3b-3a2f-47df-87fa-4ad095ee5cf9", "bridge": "br-int", "label": "tempest-ServerActionsTestOtherB-1680745008-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.9", "type": "floating", "version": 4, "meta": {}}], "label": "tempest-ServerActionsTestOtherB-1680745008-network", "vif_mac": "fa:16:3e:09:b8:70"}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "ad5ea22c63f04f91a3f2fa665e4fc8a1", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap52024efc-8c", "ovs_interfaceid": "52024efc-8cde-4610-b12c-198f19d75ccf", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:10:27 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-221919a0-9cdd-44ec-881f-d6c82383d31d tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:09:b8:70,bridge_name='br-int',has_traffic_filtering=True,id=52024efc-8cde-4610-b12c-198f19d75ccf,network=Network(31c4ca3b-3a2f-47df-87fa-4ad095ee5cf9),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap52024efc-8c') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:10:27 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-221919a0-9cdd-44ec-881f-d6c82383d31d tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] [instance: a9350a18-14e6-42c6-a150-0a1487f20ad9] End _get_guest_xml xml=<domain type="kvm">
Nov 29 03:10:27 user nova-compute[75234]:   <uuid>a9350a18-14e6-42c6-a150-0a1487f20ad9</uuid>
Nov 29 03:10:27 user nova-compute[75234]:   <name>instance-00000005</name>
Nov 29 03:10:27 user nova-compute[75234]:   <memory>2097152</memory>
Nov 29 03:10:27 user nova-compute[75234]:   <vcpu>1</vcpu>
Nov 29 03:10:27 user nova-compute[75234]:   <metadata>
Nov 29 03:10:27 user nova-compute[75234]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Nov 29 03:10:27 user nova-compute[75234]:       <nova:package version="0.0.0"/>
Nov 29 03:10:27 user nova-compute[75234]:       <nova:name>tempest-ServerActionsTestOtherB-server-372321920</nova:name>
Nov 29 03:10:27 user nova-compute[75234]:       <nova:creationTime>2023-11-29 03:10:25</nova:creationTime>
Nov 29 03:10:27 user nova-compute[75234]:       <nova:flavor name="m1.small">
Nov 29 03:10:27 user nova-compute[75234]:         <nova:memory>2048</nova:memory>
Nov 29 03:10:27 user nova-compute[75234]:         <nova:disk>20</nova:disk>
Nov 29 03:10:27 user nova-compute[75234]:         <nova:swap>0</nova:swap>
Nov 29 03:10:27 user nova-compute[75234]:         <nova:ephemeral>0</nova:ephemeral>
Nov 29 03:10:27 user nova-compute[75234]:         <nova:vcpus>1</nova:vcpus>
Nov 29 03:10:27 user nova-compute[75234]:       </nova:flavor>
Nov 29 03:10:27 user nova-compute[75234]:       <nova:owner>
Nov 29 03:10:27 user nova-compute[75234]:         <nova:user uuid="bdddba0ef9c44b6cb43c32a9ab0e29f6">tempest-ServerActionsTestOtherB-70688183-project-member</nova:user>
Nov 29 03:10:27 user nova-compute[75234]:         <nova:project uuid="ad5ea22c63f04f91a3f2fa665e4fc8a1">tempest-ServerActionsTestOtherB-70688183</nova:project>
Nov 29 03:10:27 user nova-compute[75234]:       </nova:owner>
Nov 29 03:10:27 user nova-compute[75234]:       <nova:root type="image" uuid="96c30ba7-9b73-4a30-8ba4-7138879573b7"/>
Nov 29 03:10:27 user nova-compute[75234]:       <nova:ports>
Nov 29 03:10:27 user nova-compute[75234]:         <nova:port uuid="52024efc-8cde-4610-b12c-198f19d75ccf">
Nov 29 03:10:27 user nova-compute[75234]:           <nova:ip type="fixed" address="10.0.0.5" ipVersion="4"/>
Nov 29 03:10:27 user nova-compute[75234]:         </nova:port>
Nov 29 03:10:27 user nova-compute[75234]:       </nova:ports>
Nov 29 03:10:27 user nova-compute[75234]:     </nova:instance>
Nov 29 03:10:27 user nova-compute[75234]:   </metadata>
Nov 29 03:10:27 user nova-compute[75234]:   <sysinfo type="smbios">
Nov 29 03:10:27 user nova-compute[75234]:     <system>
Nov 29 03:10:27 user nova-compute[75234]:       <entry name="manufacturer">OpenStack Foundation</entry>
Nov 29 03:10:27 user nova-compute[75234]:       <entry name="product">OpenStack Nova</entry>
Nov 29 03:10:27 user nova-compute[75234]:       <entry name="version">0.0.0</entry>
Nov 29 03:10:27 user nova-compute[75234]:       <entry name="serial">a9350a18-14e6-42c6-a150-0a1487f20ad9</entry>
Nov 29 03:10:27 user nova-compute[75234]:       <entry name="uuid">a9350a18-14e6-42c6-a150-0a1487f20ad9</entry>
Nov 29 03:10:27 user nova-compute[75234]:       <entry name="family">Virtual Machine</entry>
Nov 29 03:10:27 user nova-compute[75234]:     </system>
Nov 29 03:10:27 user nova-compute[75234]:   </sysinfo>
Nov 29 03:10:27 user nova-compute[75234]:   <os>
Nov 29 03:10:27 user nova-compute[75234]:     <type arch="x86_64" machine="pc">hvm</type>
Nov 29 03:10:27 user nova-compute[75234]:     <boot dev="hd"/>
Nov 29 03:10:27 user nova-compute[75234]:     <smbios mode="sysinfo"/>
Nov 29 03:10:27 user nova-compute[75234]:   </os>
Nov 29 03:10:27 user nova-compute[75234]:   <features>
Nov 29 03:10:27 user nova-compute[75234]:     <acpi/>
Nov 29 03:10:27 user nova-compute[75234]:     <apic/>
Nov 29 03:10:27 user nova-compute[75234]:     <vmcoreinfo/>
Nov 29 03:10:27 user nova-compute[75234]:   </features>
Nov 29 03:10:27 user nova-compute[75234]:   <clock offset="utc">
Nov 29 03:10:27 user nova-compute[75234]:     <timer name="pit" tickpolicy="delay"/>
Nov 29 03:10:27 user nova-compute[75234]:     <timer name="rtc" tickpolicy="catchup"/>
Nov 29 03:10:27 user nova-compute[75234]:     <timer name="hpet" present="no"/>
Nov 29 03:10:27 user nova-compute[75234]:   </clock>
Nov 29 03:10:27 user nova-compute[75234]:   <cpu mode="custom" match="exact">
Nov 29 03:10:27 user nova-compute[75234]:     <model>Nehalem</model>
Nov 29 03:10:27 user nova-compute[75234]:     <topology sockets="1" cores="1" threads="1"/>
Nov 29 03:10:27 user nova-compute[75234]:   </cpu>
Nov 29 03:10:27 user nova-compute[75234]:   <devices>
Nov 29 03:10:27 user nova-compute[75234]:     <disk type="file" device="disk">
Nov 29 03:10:27 user nova-compute[75234]:       <driver name="qemu" type="qcow2" cache="none"/>
Nov 29 03:10:27 user nova-compute[75234]:       <source file="/opt/stack/data/nova/instances/a9350a18-14e6-42c6-a150-0a1487f20ad9/disk"/>
Nov 29 03:10:27 user nova-compute[75234]:       <target dev="vda" bus="virtio"/>
Nov 29 03:10:27 user nova-compute[75234]:     </disk>
Nov 29 03:10:27 user nova-compute[75234]:     <disk type="block" device="disk">
Nov 29 03:10:27 user nova-compute[75234]:       <driver name="qemu" type="raw" cache="none" io="native"/>
Nov 29 03:10:27 user nova-compute[75234]:       <source dev="/dev/sdc"/>
Nov 29 03:10:27 user nova-compute[75234]:       <target dev="vdb" bus="virtio"/>
Nov 29 03:10:27 user nova-compute[75234]:       <serial>f4130d3c-b329-4ce1-8b6d-d9e270a43b0b</serial>
Nov 29 03:10:27 user nova-compute[75234]:     </disk>
Nov 29 03:10:27 user nova-compute[75234]:     <interface type="ethernet">
Nov 29 03:10:27 user nova-compute[75234]:       <mac address="fa:16:3e:09:b8:70"/>
Nov 29 03:10:27 user nova-compute[75234]:       <model type="virtio"/>
Nov 29 03:10:27 user nova-compute[75234]:       <mtu size="1442"/>
Nov 29 03:10:27 user nova-compute[75234]:       <target dev="tap52024efc-8c"/>
Nov 29 03:10:27 user nova-compute[75234]:     </interface>
Nov 29 03:10:27 user nova-compute[75234]:     <serial type="pty">
Nov 29 03:10:27 user nova-compute[75234]:       <log file="/opt/stack/data/nova/instances/a9350a18-14e6-42c6-a150-0a1487f20ad9/console.log" append="off"/>
Nov 29 03:10:27 user nova-compute[75234]:     </serial>
Nov 29 03:10:27 user nova-compute[75234]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Nov 29 03:10:27 user nova-compute[75234]:     <video>
Nov 29 03:10:27 user nova-compute[75234]:       <model type="virtio"/>
Nov 29 03:10:27 user nova-compute[75234]:     </video>
Nov 29 03:10:27 user nova-compute[75234]:     <rng model="virtio">
Nov 29 03:10:27 user nova-compute[75234]:       <backend model="random">/dev/urandom</backend>
Nov 29 03:10:27 user nova-compute[75234]:     </rng>
Nov 29 03:10:27 user nova-compute[75234]:     <controller type="usb" index="0" model="none"/>
Nov 29 03:10:27 user nova-compute[75234]:     <memballoon model="virtio">
Nov 29 03:10:27 user nova-compute[75234]:       <stats period="10"/>
Nov 29 03:10:27 user nova-compute[75234]:     </memballoon>
Nov 29 03:10:27 user nova-compute[75234]:   </devices>
Nov 29 03:10:27 user nova-compute[75234]: </domain>
Nov 29 03:10:27 user nova-compute[75234]:  {{(pid=75234) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7568}}
Nov 29 03:10:27 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-221919a0-9cdd-44ec-881f-d6c82383d31d tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-11-29T03:06:06Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-ServerActionsTestOtherB-server-372321920',display_name='tempest-ServerActionsTestOtherB-server-372321920',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(7),hidden=False,host='user',hostname='tempest-serveractionstestotherb-server-372321920',id=5,image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',info_cache=InstanceInfoCache,instance_type_id=7,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBFOggqgO1QPxMZPb2P6SX24mx6LOsfQfrzx4GSpNnJiixHqqOn61ObLwI9/TG/r6plC8w1huVCtu+OqAyk9QXUlEr0+VsPNunPSN9WPXBAM4+U+3iCOgHXExdr++34pXkw==',key_name='tempest-keypair-1692341699',keypairs=<?>,launch_index=0,launched_at=2023-11-29T03:06:23Z,launched_on='user',locked=False,locked_by=None,memory_mb=2048,metadata={},migration_context=MigrationContext,new_flavor=Flavor(7),node='user',numa_topology=None,old_flavor=Flavor(6),os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=1,progress=0,project_id='ad5ea22c63f04f91a3f2fa665e4fc8a1',ramdisk_id='',reservation_id='r-f5lkid4e',resources=None,root_device_name='/dev/vda',root_gb=20,security_groups=SecurityGroupList,services=ServiceList,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',old_vm_state='active',owner_project_name='tempest-ServerActionsTestOtherB-70688183',owner_user_name='tempest-ServerActionsTestOtherB-70688183-project-member'},tags=<?>,task_state='resize_finish',terminated_at=None,trusted_certs=None,updated_at=2023-11-29T03:10:15Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='bdddba0ef9c44b6cb43c32a9ab0e29f6',uuid=a9350a18-14e6-42c6-a150-0a1487f20ad9,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "52024efc-8cde-4610-b12c-198f19d75ccf", "address": "fa:16:3e:09:b8:70", "network": {"id": "31c4ca3b-3a2f-47df-87fa-4ad095ee5cf9", "bridge": "br-int", "label": "tempest-ServerActionsTestOtherB-1680745008-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.9", "type": "floating", "version": 4, "meta": {}}], "label": "tempest-ServerActionsTestOtherB-1680745008-network", "vif_mac": "fa:16:3e:09:b8:70"}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "ad5ea22c63f04f91a3f2fa665e4fc8a1", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap52024efc-8c", "ovs_interfaceid": "52024efc-8cde-4610-b12c-198f19d75ccf", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Nov 29 03:10:27 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-221919a0-9cdd-44ec-881f-d6c82383d31d tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Converting VIF {"id": "52024efc-8cde-4610-b12c-198f19d75ccf", "address": "fa:16:3e:09:b8:70", "network": {"id": "31c4ca3b-3a2f-47df-87fa-4ad095ee5cf9", "bridge": "br-int", "label": "tempest-ServerActionsTestOtherB-1680745008-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.9", "type": "floating", "version": 4, "meta": {}}], "label": "tempest-ServerActionsTestOtherB-1680745008-network", "vif_mac": "fa:16:3e:09:b8:70"}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "ad5ea22c63f04f91a3f2fa665e4fc8a1", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap52024efc-8c", "ovs_interfaceid": "52024efc-8cde-4610-b12c-198f19d75ccf", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:10:27 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-221919a0-9cdd-44ec-881f-d6c82383d31d tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:09:b8:70,bridge_name='br-int',has_traffic_filtering=True,id=52024efc-8cde-4610-b12c-198f19d75ccf,network=Network(31c4ca3b-3a2f-47df-87fa-4ad095ee5cf9),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap52024efc-8c') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:10:27 user nova-compute[75234]: DEBUG os_vif [None req-221919a0-9cdd-44ec-881f-d6c82383d31d tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:09:b8:70,bridge_name='br-int',has_traffic_filtering=True,id=52024efc-8cde-4610-b12c-198f19d75ccf,network=Network(31c4ca3b-3a2f-47df-87fa-4ad095ee5cf9),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap52024efc-8c') {{(pid=75234) plug /opt/stack/data/venv/lib/python3.10/site-packages/os_vif/__init__.py:76}}
Nov 29 03:10:27 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:10:27 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:10:27 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Nov 29 03:10:27 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:10:27 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbCreateCommand(_result=None, table=QoS, columns={'type': 'linux-noop', 'external_ids': {'id': '2ef8c24f-0c97-504b-852e-38bf52f37f86', '_type': 'linux-noop'}}, row=False) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:10:27 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-ddae26ab-aba2-44e2-9f96-a517489c193b tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=75234) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2366}}
Nov 29 03:10:27 user nova-compute[75234]: INFO nova.compute.claims [None req-ddae26ab-aba2-44e2-9f96-a517489c193b tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] [instance: a082c96e-e384-40b6-bcc8-fc5e0e104669] Claim successful on node user
Nov 29 03:10:27 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:10:27 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:10:27 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:10:27 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tap52024efc-8c, may_exist=True, interface_attrs={}) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:10:27 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Port, record=tap52024efc-8c, col_values=(('qos', UUID('3804a230-08ac-480e-a716-2663f3c8b1cc')),), if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:10:27 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DbSetCommand(_result=None, table=Interface, record=tap52024efc-8c, col_values=(('external_ids', {'iface-id': '52024efc-8cde-4610-b12c-198f19d75ccf', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:09:b8:70', 'vm-uuid': 'a9350a18-14e6-42c6-a150-0a1487f20ad9'}),), if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:10:27 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:10:27 user nova-compute[75234]: DEBUG nova.network.neutron [None req-046208e3-2011-4422-97ac-ba2e4f78cd46 tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] [instance: d3b90f39-4b91-4152-b820-bd0e263eec89] Successfully created port: 83eb52bb-5be5-49d6-81ec-86a21dde3df2 {{(pid=75234) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:548}}
Nov 29 03:10:27 user nova-compute[75234]: DEBUG nova.compute.manager [None req-97e2cf32-f7d5-4ab9-a932-35fa0c7cd310 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] [instance: 13e5da64-9adc-4479-81c7-d329e9b5f08f] Start spawning the instance on the hypervisor. {{(pid=75234) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2628}}
Nov 29 03:10:27 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-97e2cf32-f7d5-4ab9-a932-35fa0c7cd310 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] [instance: 13e5da64-9adc-4479-81c7-d329e9b5f08f] Creating instance directory {{(pid=75234) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4727}}
Nov 29 03:10:27 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-97e2cf32-f7d5-4ab9-a932-35fa0c7cd310 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] [instance: 13e5da64-9adc-4479-81c7-d329e9b5f08f] Creating image(s)
Nov 29 03:10:27 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-97e2cf32-f7d5-4ab9-a932-35fa0c7cd310 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Acquiring lock "/opt/stack/data/nova/instances/13e5da64-9adc-4479-81c7-d329e9b5f08f/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:10:27 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-97e2cf32-f7d5-4ab9-a932-35fa0c7cd310 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Lock "/opt/stack/data/nova/instances/13e5da64-9adc-4479-81c7-d329e9b5f08f/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:10:27 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-97e2cf32-f7d5-4ab9-a932-35fa0c7cd310 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Lock "/opt/stack/data/nova/instances/13e5da64-9adc-4479-81c7-d329e9b5f08f/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.002s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:10:27 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:10:27 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-97e2cf32-f7d5-4ab9-a932-35fa0c7cd310 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:10:27 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:10:27 user nova-compute[75234]: INFO os_vif [None req-221919a0-9cdd-44ec-881f-d6c82383d31d tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:09:b8:70,bridge_name='br-int',has_traffic_filtering=True,id=52024efc-8cde-4610-b12c-198f19d75ccf,network=Network(31c4ca3b-3a2f-47df-87fa-4ad095ee5cf9),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap52024efc-8c')
Nov 29 03:10:27 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-046208e3-2011-4422-97ac-ba2e4f78cd46 tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/d3b90f39-4b91-4152-b820-bd0e263eec89/disk --force-share --output=json" returned: 0 in 0.269s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:10:27 user nova-compute[75234]: DEBUG nova.virt.disk.api [None req-046208e3-2011-4422-97ac-ba2e4f78cd46 tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] Cannot resize image /opt/stack/data/nova/instances/d3b90f39-4b91-4152-b820-bd0e263eec89/disk to a smaller size. {{(pid=75234) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:172}}
Nov 29 03:10:27 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-046208e3-2011-4422-97ac-ba2e4f78cd46 tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] [instance: d3b90f39-4b91-4152-b820-bd0e263eec89] Created local disks {{(pid=75234) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4861}}
Nov 29 03:10:27 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-046208e3-2011-4422-97ac-ba2e4f78cd46 tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] [instance: d3b90f39-4b91-4152-b820-bd0e263eec89] Ensure instance console log exists: /opt/stack/data/nova/instances/d3b90f39-4b91-4152-b820-bd0e263eec89/console.log {{(pid=75234) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4613}}
Nov 29 03:10:27 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-046208e3-2011-4422-97ac-ba2e4f78cd46 tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:10:27 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-046208e3-2011-4422-97ac-ba2e4f78cd46 tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:10:27 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-046208e3-2011-4422-97ac-ba2e4f78cd46 tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:10:27 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-221919a0-9cdd-44ec-881f-d6c82383d31d tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] No BDM found with device name vda, not building metadata. {{(pid=75234) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12089}}
Nov 29 03:10:27 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-221919a0-9cdd-44ec-881f-d6c82383d31d tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] No BDM found with device name vdb, not building metadata. {{(pid=75234) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12089}}
Nov 29 03:10:27 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-221919a0-9cdd-44ec-881f-d6c82383d31d tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] No VIF found with MAC fa:16:3e:09:b8:70, not building metadata {{(pid=75234) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Nov 29 03:10:27 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-97e2cf32-f7d5-4ab9-a932-35fa0c7cd310 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json" returned: 0 in 0.243s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:10:27 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-97e2cf32-f7d5-4ab9-a932-35fa0c7cd310 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Acquiring lock "80686e29dc9db6600d03e85cb62af298e12d7918" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:10:27 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-97e2cf32-f7d5-4ab9-a932-35fa0c7cd310 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Lock "80686e29dc9db6600d03e85cb62af298e12d7918" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.002s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:10:27 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-97e2cf32-f7d5-4ab9-a932-35fa0c7cd310 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:10:27 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-97e2cf32-f7d5-4ab9-a932-35fa0c7cd310 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json" returned: 0 in 0.189s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:10:27 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-97e2cf32-f7d5-4ab9-a932-35fa0c7cd310 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918,backing_fmt=raw /opt/stack/data/nova/instances/13e5da64-9adc-4479-81c7-d329e9b5f08f/disk 1073741824 {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:10:28 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-97e2cf32-f7d5-4ab9-a932-35fa0c7cd310 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918,backing_fmt=raw /opt/stack/data/nova/instances/13e5da64-9adc-4479-81c7-d329e9b5f08f/disk 1073741824" returned: 0 in 0.062s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:10:28 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-97e2cf32-f7d5-4ab9-a932-35fa0c7cd310 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Lock "80686e29dc9db6600d03e85cb62af298e12d7918" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.271s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:10:28 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-97e2cf32-f7d5-4ab9-a932-35fa0c7cd310 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:10:28 user nova-compute[75234]: DEBUG nova.compute.provider_tree [None req-ddae26ab-aba2-44e2-9f96-a517489c193b tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Inventory has not changed in ProviderTree for provider: 4d291220-7c37-4839-a0e3-b23c66502b74 {{(pid=75234) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Nov 29 03:10:28 user nova-compute[75234]: DEBUG nova.scheduler.client.report [None req-ddae26ab-aba2-44e2-9f96-a517489c193b tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Inventory has not changed for provider 4d291220-7c37-4839-a0e3-b23c66502b74 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=75234) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Nov 29 03:10:28 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-97e2cf32-f7d5-4ab9-a932-35fa0c7cd310 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json" returned: 0 in 0.208s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:10:28 user nova-compute[75234]: DEBUG nova.virt.disk.api [None req-97e2cf32-f7d5-4ab9-a932-35fa0c7cd310 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Checking if we can resize image /opt/stack/data/nova/instances/13e5da64-9adc-4479-81c7-d329e9b5f08f/disk. size=1073741824 {{(pid=75234) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:166}}
Nov 29 03:10:28 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-97e2cf32-f7d5-4ab9-a932-35fa0c7cd310 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/13e5da64-9adc-4479-81c7-d329e9b5f08f/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:10:28 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ddae26ab-aba2-44e2-9f96-a517489c193b tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.889s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:10:28 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ddae26ab-aba2-44e2-9f96-a517489c193b tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] [instance: a082c96e-e384-40b6-bcc8-fc5e0e104669] Start building networks asynchronously for instance. {{(pid=75234) _build_resources /opt/stack/nova/nova/compute/manager.py:2819}}
Nov 29 03:10:28 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ddae26ab-aba2-44e2-9f96-a517489c193b tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] [instance: a082c96e-e384-40b6-bcc8-fc5e0e104669] Allocating IP information in the background. {{(pid=75234) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1972}}
Nov 29 03:10:28 user nova-compute[75234]: DEBUG nova.network.neutron [None req-ddae26ab-aba2-44e2-9f96-a517489c193b tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] [instance: a082c96e-e384-40b6-bcc8-fc5e0e104669] allocate_for_instance() {{(pid=75234) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1156}}
Nov 29 03:10:28 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-ddae26ab-aba2-44e2-9f96-a517489c193b tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] [instance: a082c96e-e384-40b6-bcc8-fc5e0e104669] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Nov 29 03:10:28 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ddae26ab-aba2-44e2-9f96-a517489c193b tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] [instance: a082c96e-e384-40b6-bcc8-fc5e0e104669] Start building block device mappings for instance. {{(pid=75234) _build_resources /opt/stack/nova/nova/compute/manager.py:2854}}
Nov 29 03:10:28 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-97e2cf32-f7d5-4ab9-a932-35fa0c7cd310 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/13e5da64-9adc-4479-81c7-d329e9b5f08f/disk --force-share --output=json" returned: 0 in 0.265s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:10:28 user nova-compute[75234]: DEBUG nova.virt.disk.api [None req-97e2cf32-f7d5-4ab9-a932-35fa0c7cd310 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Cannot resize image /opt/stack/data/nova/instances/13e5da64-9adc-4479-81c7-d329e9b5f08f/disk to a smaller size. {{(pid=75234) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:172}}
Nov 29 03:10:28 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-97e2cf32-f7d5-4ab9-a932-35fa0c7cd310 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] [instance: 13e5da64-9adc-4479-81c7-d329e9b5f08f] Created local disks {{(pid=75234) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4861}}
Nov 29 03:10:28 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-97e2cf32-f7d5-4ab9-a932-35fa0c7cd310 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] [instance: 13e5da64-9adc-4479-81c7-d329e9b5f08f] Ensure instance console log exists: /opt/stack/data/nova/instances/13e5da64-9adc-4479-81c7-d329e9b5f08f/console.log {{(pid=75234) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4613}}
Nov 29 03:10:28 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-97e2cf32-f7d5-4ab9-a932-35fa0c7cd310 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:10:28 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-97e2cf32-f7d5-4ab9-a932-35fa0c7cd310 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:10:28 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-97e2cf32-f7d5-4ab9-a932-35fa0c7cd310 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:10:28 user nova-compute[75234]: DEBUG nova.policy [None req-ddae26ab-aba2-44e2-9f96-a517489c193b tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '6792e866677645c598ddb7d2c1723a1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f8a7884867c944e1ba1024270e4902f3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=75234) authorize /opt/stack/nova/nova/policy.py:203}}
Nov 29 03:10:28 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ddae26ab-aba2-44e2-9f96-a517489c193b tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] [instance: a082c96e-e384-40b6-bcc8-fc5e0e104669] Start spawning the instance on the hypervisor. {{(pid=75234) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2628}}
Nov 29 03:10:28 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-ddae26ab-aba2-44e2-9f96-a517489c193b tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] [instance: a082c96e-e384-40b6-bcc8-fc5e0e104669] Creating instance directory {{(pid=75234) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4727}}
Nov 29 03:10:28 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-ddae26ab-aba2-44e2-9f96-a517489c193b tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] [instance: a082c96e-e384-40b6-bcc8-fc5e0e104669] Creating image(s)
Nov 29 03:10:28 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ddae26ab-aba2-44e2-9f96-a517489c193b tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Acquiring lock "/opt/stack/data/nova/instances/a082c96e-e384-40b6-bcc8-fc5e0e104669/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:10:28 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ddae26ab-aba2-44e2-9f96-a517489c193b tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Lock "/opt/stack/data/nova/instances/a082c96e-e384-40b6-bcc8-fc5e0e104669/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:10:28 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ddae26ab-aba2-44e2-9f96-a517489c193b tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Lock "/opt/stack/data/nova/instances/a082c96e-e384-40b6-bcc8-fc5e0e104669/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.002s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:10:28 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ddae26ab-aba2-44e2-9f96-a517489c193b tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:10:28 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:10:28 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ddae26ab-aba2-44e2-9f96-a517489c193b tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json" returned: 0 in 0.226s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:10:28 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ddae26ab-aba2-44e2-9f96-a517489c193b tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Acquiring lock "80686e29dc9db6600d03e85cb62af298e12d7918" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:10:28 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ddae26ab-aba2-44e2-9f96-a517489c193b tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Lock "80686e29dc9db6600d03e85cb62af298e12d7918" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.004s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:10:28 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ddae26ab-aba2-44e2-9f96-a517489c193b tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:10:29 user nova-compute[75234]: DEBUG nova.network.neutron [None req-97e2cf32-f7d5-4ab9-a932-35fa0c7cd310 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] [instance: 13e5da64-9adc-4479-81c7-d329e9b5f08f] Successfully created port: 2abf9e25-d1d9-4893-899e-912d41b7c38f {{(pid=75234) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:548}}
Nov 29 03:10:29 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ddae26ab-aba2-44e2-9f96-a517489c193b tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json" returned: 0 in 0.238s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:10:29 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ddae26ab-aba2-44e2-9f96-a517489c193b tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918,backing_fmt=raw /opt/stack/data/nova/instances/a082c96e-e384-40b6-bcc8-fc5e0e104669/disk 1073741824 {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:10:29 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ef90f75a-4af4-4a58-bb0e-eaf1f32e9b1c tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] Acquiring lock "f7d0bdeb-6a05-47ac-87f2-c8380a9a4286" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:10:29 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ef90f75a-4af4-4a58-bb0e-eaf1f32e9b1c tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] Lock "f7d0bdeb-6a05-47ac-87f2-c8380a9a4286" acquired by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: waited 0.003s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:10:29 user nova-compute[75234]: INFO nova.compute.manager [None req-ef90f75a-4af4-4a58-bb0e-eaf1f32e9b1c tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] [instance: f7d0bdeb-6a05-47ac-87f2-c8380a9a4286] Detaching volume ada91c26-ca97-4b8b-98bb-a310d0627691
Nov 29 03:10:29 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:10:29 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:10:29 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:10:29 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:10:29 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ddae26ab-aba2-44e2-9f96-a517489c193b tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918,backing_fmt=raw /opt/stack/data/nova/instances/a082c96e-e384-40b6-bcc8-fc5e0e104669/disk 1073741824" returned: 0 in 0.191s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:10:29 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ddae26ab-aba2-44e2-9f96-a517489c193b tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Lock "80686e29dc9db6600d03e85cb62af298e12d7918" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.437s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:10:29 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ddae26ab-aba2-44e2-9f96-a517489c193b tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:10:29 user nova-compute[75234]: INFO nova.virt.block_device [None req-ef90f75a-4af4-4a58-bb0e-eaf1f32e9b1c tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] [instance: f7d0bdeb-6a05-47ac-87f2-c8380a9a4286] Attempting to driver detach volume ada91c26-ca97-4b8b-98bb-a310d0627691 from mountpoint /dev/vdb
Nov 29 03:10:29 user nova-compute[75234]: DEBUG nova.network.neutron [None req-046208e3-2011-4422-97ac-ba2e4f78cd46 tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] [instance: d3b90f39-4b91-4152-b820-bd0e263eec89] Successfully updated port: 83eb52bb-5be5-49d6-81ec-86a21dde3df2 {{(pid=75234) _update_port /opt/stack/nova/nova/network/neutron.py:586}}
Nov 29 03:10:29 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-ef90f75a-4af4-4a58-bb0e-eaf1f32e9b1c tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] Attempting to detach device vdb from instance f7d0bdeb-6a05-47ac-87f2-c8380a9a4286 from the persistent domain config. {{(pid=75234) _detach_from_persistent /opt/stack/nova/nova/virt/libvirt/driver.py:2487}}
Nov 29 03:10:29 user nova-compute[75234]: DEBUG nova.virt.libvirt.guest [None req-ef90f75a-4af4-4a58-bb0e-eaf1f32e9b1c tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] detach device xml: <disk type="block" device="disk">
Nov 29 03:10:29 user nova-compute[75234]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Nov 29 03:10:29 user nova-compute[75234]:   <source dev="/dev/sdb"/>
Nov 29 03:10:29 user nova-compute[75234]:   <target dev="vdb" bus="virtio"/>
Nov 29 03:10:29 user nova-compute[75234]:   <serial>ada91c26-ca97-4b8b-98bb-a310d0627691</serial>
Nov 29 03:10:29 user nova-compute[75234]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x07" function="0x0"/>
Nov 29 03:10:29 user nova-compute[75234]: </disk>
Nov 29 03:10:29 user nova-compute[75234]:  {{(pid=75234) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Nov 29 03:10:29 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-ef90f75a-4af4-4a58-bb0e-eaf1f32e9b1c tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] Successfully detached device vdb from instance f7d0bdeb-6a05-47ac-87f2-c8380a9a4286 from the persistent domain config.
Nov 29 03:10:29 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-ef90f75a-4af4-4a58-bb0e-eaf1f32e9b1c tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] (1/8): Attempting to detach device vdb with device alias virtio-disk1 from instance f7d0bdeb-6a05-47ac-87f2-c8380a9a4286 from the live domain config. {{(pid=75234) _detach_from_live_with_retry /opt/stack/nova/nova/virt/libvirt/driver.py:2523}}
Nov 29 03:10:29 user nova-compute[75234]: DEBUG nova.virt.libvirt.guest [None req-ef90f75a-4af4-4a58-bb0e-eaf1f32e9b1c tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] detach device xml: <disk type="block" device="disk">
Nov 29 03:10:29 user nova-compute[75234]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Nov 29 03:10:29 user nova-compute[75234]:   <source dev="/dev/sdb"/>
Nov 29 03:10:29 user nova-compute[75234]:   <target dev="vdb" bus="virtio"/>
Nov 29 03:10:29 user nova-compute[75234]:   <serial>ada91c26-ca97-4b8b-98bb-a310d0627691</serial>
Nov 29 03:10:29 user nova-compute[75234]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x07" function="0x0"/>
Nov 29 03:10:29 user nova-compute[75234]: </disk>
Nov 29 03:10:29 user nova-compute[75234]:  {{(pid=75234) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Nov 29 03:10:29 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-046208e3-2011-4422-97ac-ba2e4f78cd46 tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] Acquiring lock "refresh_cache-d3b90f39-4b91-4152-b820-bd0e263eec89" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:10:29 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-046208e3-2011-4422-97ac-ba2e4f78cd46 tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] Acquired lock "refresh_cache-d3b90f39-4b91-4152-b820-bd0e263eec89" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:10:29 user nova-compute[75234]: DEBUG nova.network.neutron [None req-046208e3-2011-4422-97ac-ba2e4f78cd46 tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] [instance: d3b90f39-4b91-4152-b820-bd0e263eec89] Building network info cache for instance {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2009}}
Nov 29 03:10:29 user nova-compute[75234]: DEBUG nova.compute.manager [req-661586d9-8d0a-45d0-9daa-b722e86406b1 req-815a28ed-41ba-4ba4-9eab-5a6661eb375a service nova] [instance: d3b90f39-4b91-4152-b820-bd0e263eec89] Received event network-changed-83eb52bb-5be5-49d6-81ec-86a21dde3df2 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:10:29 user nova-compute[75234]: DEBUG nova.compute.manager [req-661586d9-8d0a-45d0-9daa-b722e86406b1 req-815a28ed-41ba-4ba4-9eab-5a6661eb375a service nova] [instance: d3b90f39-4b91-4152-b820-bd0e263eec89] Refreshing instance network info cache due to event network-changed-83eb52bb-5be5-49d6-81ec-86a21dde3df2. {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11117}}
Nov 29 03:10:29 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-661586d9-8d0a-45d0-9daa-b722e86406b1 req-815a28ed-41ba-4ba4-9eab-5a6661eb375a service nova] Acquiring lock "refresh_cache-d3b90f39-4b91-4152-b820-bd0e263eec89" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:10:29 user nova-compute[75234]: DEBUG nova.network.neutron [None req-046208e3-2011-4422-97ac-ba2e4f78cd46 tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] [instance: d3b90f39-4b91-4152-b820-bd0e263eec89] Instance cache missing network info. {{(pid=75234) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3322}}
Nov 29 03:10:29 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ddae26ab-aba2-44e2-9f96-a517489c193b tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json" returned: 0 in 0.368s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:10:29 user nova-compute[75234]: DEBUG nova.virt.disk.api [None req-ddae26ab-aba2-44e2-9f96-a517489c193b tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Checking if we can resize image /opt/stack/data/nova/instances/a082c96e-e384-40b6-bcc8-fc5e0e104669/disk. size=1073741824 {{(pid=75234) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:166}}
Nov 29 03:10:29 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ddae26ab-aba2-44e2-9f96-a517489c193b tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a082c96e-e384-40b6-bcc8-fc5e0e104669/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:10:29 user nova-compute[75234]: DEBUG nova.compute.manager [req-ffba9743-80e9-40a4-8212-362da567a2e3 req-08efe86b-1241-4b81-b020-7c6cd7d698d8 service nova] [instance: a9350a18-14e6-42c6-a150-0a1487f20ad9] Received event network-vif-plugged-52024efc-8cde-4610-b12c-198f19d75ccf {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:10:29 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-ffba9743-80e9-40a4-8212-362da567a2e3 req-08efe86b-1241-4b81-b020-7c6cd7d698d8 service nova] Acquiring lock "a9350a18-14e6-42c6-a150-0a1487f20ad9-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:10:29 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-ffba9743-80e9-40a4-8212-362da567a2e3 req-08efe86b-1241-4b81-b020-7c6cd7d698d8 service nova] Lock "a9350a18-14e6-42c6-a150-0a1487f20ad9-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:10:29 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-ffba9743-80e9-40a4-8212-362da567a2e3 req-08efe86b-1241-4b81-b020-7c6cd7d698d8 service nova] Lock "a9350a18-14e6-42c6-a150-0a1487f20ad9-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:10:29 user nova-compute[75234]: DEBUG nova.compute.manager [req-ffba9743-80e9-40a4-8212-362da567a2e3 req-08efe86b-1241-4b81-b020-7c6cd7d698d8 service nova] [instance: a9350a18-14e6-42c6-a150-0a1487f20ad9] No waiting events found dispatching network-vif-plugged-52024efc-8cde-4610-b12c-198f19d75ccf {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:10:29 user nova-compute[75234]: WARNING nova.compute.manager [req-ffba9743-80e9-40a4-8212-362da567a2e3 req-08efe86b-1241-4b81-b020-7c6cd7d698d8 service nova] [instance: a9350a18-14e6-42c6-a150-0a1487f20ad9] Received unexpected event network-vif-plugged-52024efc-8cde-4610-b12c-198f19d75ccf for instance with vm_state active and task_state resize_finish.
Nov 29 03:10:29 user nova-compute[75234]: DEBUG nova.network.neutron [None req-ddae26ab-aba2-44e2-9f96-a517489c193b tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] [instance: a082c96e-e384-40b6-bcc8-fc5e0e104669] Successfully created port: 0d131138-51b6-49f9-9a11-6b64e32e5796 {{(pid=75234) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:548}}
Nov 29 03:10:30 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ddae26ab-aba2-44e2-9f96-a517489c193b tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a082c96e-e384-40b6-bcc8-fc5e0e104669/disk --force-share --output=json" returned: 0 in 0.265s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:10:30 user nova-compute[75234]: DEBUG nova.virt.disk.api [None req-ddae26ab-aba2-44e2-9f96-a517489c193b tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Cannot resize image /opt/stack/data/nova/instances/a082c96e-e384-40b6-bcc8-fc5e0e104669/disk to a smaller size. {{(pid=75234) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:172}}
Nov 29 03:10:30 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-ddae26ab-aba2-44e2-9f96-a517489c193b tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] [instance: a082c96e-e384-40b6-bcc8-fc5e0e104669] Created local disks {{(pid=75234) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4861}}
Nov 29 03:10:30 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-ddae26ab-aba2-44e2-9f96-a517489c193b tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] [instance: a082c96e-e384-40b6-bcc8-fc5e0e104669] Ensure instance console log exists: /opt/stack/data/nova/instances/a082c96e-e384-40b6-bcc8-fc5e0e104669/console.log {{(pid=75234) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4613}}
Nov 29 03:10:30 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ddae26ab-aba2-44e2-9f96-a517489c193b tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:10:30 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ddae26ab-aba2-44e2-9f96-a517489c193b tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:10:30 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ddae26ab-aba2-44e2-9f96-a517489c193b tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:10:30 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-ef90f75a-4af4-4a58-bb0e-eaf1f32e9b1c tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] Start waiting for the detach event from libvirt for device vdb with device alias virtio-disk1 for instance f7d0bdeb-6a05-47ac-87f2-c8380a9a4286 {{(pid=75234) _detach_from_live_and_wait_for_event /opt/stack/nova/nova/virt/libvirt/driver.py:2599}}
Nov 29 03:10:30 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:10:30 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:10:30 user nova-compute[75234]: DEBUG nova.network.neutron [None req-046208e3-2011-4422-97ac-ba2e4f78cd46 tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] [instance: d3b90f39-4b91-4152-b820-bd0e263eec89] Updating instance_info_cache with network_info: [{"id": "83eb52bb-5be5-49d6-81ec-86a21dde3df2", "address": "fa:16:3e:9d:70:dd", "network": {"id": "49d62bf6-34ee-4de8-ab85-6ef489d957c8", "bridge": "br-int", "label": "tempest-VolumesAdminNegativeTest-1304144113-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "5eaa91ebc2e64f2eba987f02bcd1a9f4", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap83eb52bb-5b", "ovs_interfaceid": "83eb52bb-5be5-49d6-81ec-86a21dde3df2", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:10:30 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:10:30 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-046208e3-2011-4422-97ac-ba2e4f78cd46 tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] Releasing lock "refresh_cache-d3b90f39-4b91-4152-b820-bd0e263eec89" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:10:30 user nova-compute[75234]: DEBUG nova.compute.manager [None req-046208e3-2011-4422-97ac-ba2e4f78cd46 tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] [instance: d3b90f39-4b91-4152-b820-bd0e263eec89] Instance network_info: |[{"id": "83eb52bb-5be5-49d6-81ec-86a21dde3df2", "address": "fa:16:3e:9d:70:dd", "network": {"id": "49d62bf6-34ee-4de8-ab85-6ef489d957c8", "bridge": "br-int", "label": "tempest-VolumesAdminNegativeTest-1304144113-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "5eaa91ebc2e64f2eba987f02bcd1a9f4", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap83eb52bb-5b", "ovs_interfaceid": "83eb52bb-5be5-49d6-81ec-86a21dde3df2", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=75234) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1987}}
Nov 29 03:10:30 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-661586d9-8d0a-45d0-9daa-b722e86406b1 req-815a28ed-41ba-4ba4-9eab-5a6661eb375a service nova] Acquired lock "refresh_cache-d3b90f39-4b91-4152-b820-bd0e263eec89" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:10:30 user nova-compute[75234]: DEBUG nova.network.neutron [req-661586d9-8d0a-45d0-9daa-b722e86406b1 req-815a28ed-41ba-4ba4-9eab-5a6661eb375a service nova] [instance: d3b90f39-4b91-4152-b820-bd0e263eec89] Refreshing network info cache for port 83eb52bb-5be5-49d6-81ec-86a21dde3df2 {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2006}}
Nov 29 03:10:30 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-046208e3-2011-4422-97ac-ba2e4f78cd46 tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] [instance: d3b90f39-4b91-4152-b820-bd0e263eec89] Start _get_guest_xml network_info=[{"id": "83eb52bb-5be5-49d6-81ec-86a21dde3df2", "address": "fa:16:3e:9d:70:dd", "network": {"id": "49d62bf6-34ee-4de8-ab85-6ef489d957c8", "bridge": "br-int", "label": "tempest-VolumesAdminNegativeTest-1304144113-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "5eaa91ebc2e64f2eba987f02bcd1a9f4", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap83eb52bb-5b", "ovs_interfaceid": "83eb52bb-5be5-49d6-81ec-86a21dde3df2", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum='c8fc807773e5354afe61636071771906',container_format='bare',created_at=2023-11-29T03:02:15Z,direct_url=<?>,disk_format='qcow2',id=96c30ba7-9b73-4a30-8ba4-7138879573b7,min_disk=0,min_ram=0,name='cirros-0.6.2-x86_64-disk',owner='b18e380921c748b78517b6f1211cd0f1',properties=ImageMetaProps,protected=<?>,size=21430272,status='active',tags=<?>,updated_at=2023-11-29T03:02:17Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [{'device_name': '/dev/vda', 'encryption_format': None, 'encrypted': False, 'device_type': 'disk', 'size': 0, 'disk_bus': 'virtio', 'boot_index': 0, 'encryption_options': None, 'encryption_secret_uuid': None, 'guest_format': None, 'image_id': '96c30ba7-9b73-4a30-8ba4-7138879573b7'}], 'ephemerals': [], 'block_device_mapping': [], 'swap': None} {{(pid=75234) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7562}}
Nov 29 03:10:30 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-046208e3-2011-4422-97ac-ba2e4f78cd46 tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:10:30 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-046208e3-2011-4422-97ac-ba2e4f78cd46 tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:10:30 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-046208e3-2011-4422-97ac-ba2e4f78cd46 tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] Searching host: 'user' for CPU controller through CGroups V1... {{(pid=75234) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1652}}
Nov 29 03:10:30 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-046208e3-2011-4422-97ac-ba2e4f78cd46 tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] CPU controller missing on host. {{(pid=75234) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1662}}
Nov 29 03:10:30 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-046208e3-2011-4422-97ac-ba2e4f78cd46 tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] Searching host: 'user' for CPU controller through CGroups V2... {{(pid=75234) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1671}}
Nov 29 03:10:30 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-046208e3-2011-4422-97ac-ba2e4f78cd46 tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] CPU controller found on host. {{(pid=75234) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1678}}
Nov 29 03:10:30 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-046208e3-2011-4422-97ac-ba2e4f78cd46 tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=75234) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5400}}
Nov 29 03:10:30 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-046208e3-2011-4422-97ac-ba2e4f78cd46 tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-11-29T03:02:04Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='1',id=6,is_public=True,memory_mb=512,name='m1.tiny',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='c8fc807773e5354afe61636071771906',container_format='bare',created_at=2023-11-29T03:02:15Z,direct_url=<?>,disk_format='qcow2',id=96c30ba7-9b73-4a30-8ba4-7138879573b7,min_disk=0,min_ram=0,name='cirros-0.6.2-x86_64-disk',owner='b18e380921c748b78517b6f1211cd0f1',properties=ImageMetaProps,protected=<?>,size=21430272,status='active',tags=<?>,updated_at=2023-11-29T03:02:17Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:563}}
Nov 29 03:10:30 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-046208e3-2011-4422-97ac-ba2e4f78cd46 tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] Flavor limits 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:348}}
Nov 29 03:10:30 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-046208e3-2011-4422-97ac-ba2e4f78cd46 tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] Image limits 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:352}}
Nov 29 03:10:30 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-046208e3-2011-4422-97ac-ba2e4f78cd46 tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] Flavor pref 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:388}}
Nov 29 03:10:30 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-046208e3-2011-4422-97ac-ba2e4f78cd46 tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] Image pref 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:392}}
Nov 29 03:10:30 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-046208e3-2011-4422-97ac-ba2e4f78cd46 tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:430}}
Nov 29 03:10:30 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-046208e3-2011-4422-97ac-ba2e4f78cd46 tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:569}}
Nov 29 03:10:30 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-046208e3-2011-4422-97ac-ba2e4f78cd46 tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=75234) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:471}}
Nov 29 03:10:30 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-046208e3-2011-4422-97ac-ba2e4f78cd46 tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] Got 1 possible topologies {{(pid=75234) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:501}}
Nov 29 03:10:30 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-046208e3-2011-4422-97ac-ba2e4f78cd46 tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:575}}
Nov 29 03:10:30 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-046208e3-2011-4422-97ac-ba2e4f78cd46 tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:577}}
Nov 29 03:10:30 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-046208e3-2011-4422-97ac-ba2e4f78cd46 tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-11-29T03:10:24Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-VolumesAdminNegativeTest-server-846124845',display_name='tempest-VolumesAdminNegativeTest-server-846124845',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-volumesadminnegativetest-server-846124845',id=17,image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBOhY8TGmPVW2eahQ/7NwxuWatHHhSisv4kKa49hvwZ8KZoBzAoVyERO5tPD8b1ELtbNUxo/5KbqYKeCg6v3iRzJlbCyhkM5yEZuV+qUhHx8TePthXUgsgJEbpNvQr1IWCw==',key_name='tempest-keypair-534984900',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='5eaa91ebc2e64f2eba987f02bcd1a9f4',ramdisk_id='',reservation_id='r-2l9d0jow',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-VolumesAdminNegativeTest-1445338761',owner_user_name='tempest-VolumesAdminNegativeTest-1445338761-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-11-29T03:10:26Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='460854be8e05408a80790620f64cf8e1',uuid=d3b90f39-4b91-4152-b820-bd0e263eec89,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "83eb52bb-5be5-49d6-81ec-86a21dde3df2", "address": "fa:16:3e:9d:70:dd", "network": {"id": "49d62bf6-34ee-4de8-ab85-6ef489d957c8", "bridge": "br-int", "label": "tempest-VolumesAdminNegativeTest-1304144113-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "5eaa91ebc2e64f2eba987f02bcd1a9f4", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap83eb52bb-5b", "ovs_interfaceid": "83eb52bb-5be5-49d6-81ec-86a21dde3df2", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=75234) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Nov 29 03:10:30 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-046208e3-2011-4422-97ac-ba2e4f78cd46 tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] Converting VIF {"id": "83eb52bb-5be5-49d6-81ec-86a21dde3df2", "address": "fa:16:3e:9d:70:dd", "network": {"id": "49d62bf6-34ee-4de8-ab85-6ef489d957c8", "bridge": "br-int", "label": "tempest-VolumesAdminNegativeTest-1304144113-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "5eaa91ebc2e64f2eba987f02bcd1a9f4", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap83eb52bb-5b", "ovs_interfaceid": "83eb52bb-5be5-49d6-81ec-86a21dde3df2", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:10:30 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-046208e3-2011-4422-97ac-ba2e4f78cd46 tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:9d:70:dd,bridge_name='br-int',has_traffic_filtering=True,id=83eb52bb-5be5-49d6-81ec-86a21dde3df2,network=Network(49d62bf6-34ee-4de8-ab85-6ef489d957c8),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap83eb52bb-5b') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:10:30 user nova-compute[75234]: DEBUG nova.objects.instance [None req-046208e3-2011-4422-97ac-ba2e4f78cd46 tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] Lazy-loading 'pci_devices' on Instance uuid d3b90f39-4b91-4152-b820-bd0e263eec89 {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:10:30 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-046208e3-2011-4422-97ac-ba2e4f78cd46 tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] [instance: d3b90f39-4b91-4152-b820-bd0e263eec89] End _get_guest_xml xml=<domain type="kvm">
Nov 29 03:10:30 user nova-compute[75234]:   <uuid>d3b90f39-4b91-4152-b820-bd0e263eec89</uuid>
Nov 29 03:10:30 user nova-compute[75234]:   <name>instance-00000011</name>
Nov 29 03:10:30 user nova-compute[75234]:   <memory>524288</memory>
Nov 29 03:10:30 user nova-compute[75234]:   <vcpu>1</vcpu>
Nov 29 03:10:30 user nova-compute[75234]:   <metadata>
Nov 29 03:10:30 user nova-compute[75234]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Nov 29 03:10:30 user nova-compute[75234]:       <nova:package version="0.0.0"/>
Nov 29 03:10:30 user nova-compute[75234]:       <nova:name>tempest-VolumesAdminNegativeTest-server-846124845</nova:name>
Nov 29 03:10:30 user nova-compute[75234]:       <nova:creationTime>2023-11-29 03:10:30</nova:creationTime>
Nov 29 03:10:30 user nova-compute[75234]:       <nova:flavor name="m1.tiny">
Nov 29 03:10:30 user nova-compute[75234]:         <nova:memory>512</nova:memory>
Nov 29 03:10:30 user nova-compute[75234]:         <nova:disk>1</nova:disk>
Nov 29 03:10:30 user nova-compute[75234]:         <nova:swap>0</nova:swap>
Nov 29 03:10:30 user nova-compute[75234]:         <nova:ephemeral>0</nova:ephemeral>
Nov 29 03:10:30 user nova-compute[75234]:         <nova:vcpus>1</nova:vcpus>
Nov 29 03:10:30 user nova-compute[75234]:       </nova:flavor>
Nov 29 03:10:30 user nova-compute[75234]:       <nova:owner>
Nov 29 03:10:30 user nova-compute[75234]:         <nova:user uuid="460854be8e05408a80790620f64cf8e1">tempest-VolumesAdminNegativeTest-1445338761-project-member</nova:user>
Nov 29 03:10:30 user nova-compute[75234]:         <nova:project uuid="5eaa91ebc2e64f2eba987f02bcd1a9f4">tempest-VolumesAdminNegativeTest-1445338761</nova:project>
Nov 29 03:10:30 user nova-compute[75234]:       </nova:owner>
Nov 29 03:10:30 user nova-compute[75234]:       <nova:root type="image" uuid="96c30ba7-9b73-4a30-8ba4-7138879573b7"/>
Nov 29 03:10:30 user nova-compute[75234]:       <nova:ports>
Nov 29 03:10:30 user nova-compute[75234]:         <nova:port uuid="83eb52bb-5be5-49d6-81ec-86a21dde3df2">
Nov 29 03:10:30 user nova-compute[75234]:           <nova:ip type="fixed" address="10.0.0.12" ipVersion="4"/>
Nov 29 03:10:30 user nova-compute[75234]:         </nova:port>
Nov 29 03:10:30 user nova-compute[75234]:       </nova:ports>
Nov 29 03:10:30 user nova-compute[75234]:     </nova:instance>
Nov 29 03:10:30 user nova-compute[75234]:   </metadata>
Nov 29 03:10:30 user nova-compute[75234]:   <sysinfo type="smbios">
Nov 29 03:10:30 user nova-compute[75234]:     <system>
Nov 29 03:10:30 user nova-compute[75234]:       <entry name="manufacturer">OpenStack Foundation</entry>
Nov 29 03:10:30 user nova-compute[75234]:       <entry name="product">OpenStack Nova</entry>
Nov 29 03:10:30 user nova-compute[75234]:       <entry name="version">0.0.0</entry>
Nov 29 03:10:30 user nova-compute[75234]:       <entry name="serial">d3b90f39-4b91-4152-b820-bd0e263eec89</entry>
Nov 29 03:10:30 user nova-compute[75234]:       <entry name="uuid">d3b90f39-4b91-4152-b820-bd0e263eec89</entry>
Nov 29 03:10:30 user nova-compute[75234]:       <entry name="family">Virtual Machine</entry>
Nov 29 03:10:30 user nova-compute[75234]:     </system>
Nov 29 03:10:30 user nova-compute[75234]:   </sysinfo>
Nov 29 03:10:30 user nova-compute[75234]:   <os>
Nov 29 03:10:30 user nova-compute[75234]:     <type arch="x86_64" machine="pc">hvm</type>
Nov 29 03:10:30 user nova-compute[75234]:     <boot dev="hd"/>
Nov 29 03:10:30 user nova-compute[75234]:     <smbios mode="sysinfo"/>
Nov 29 03:10:30 user nova-compute[75234]:   </os>
Nov 29 03:10:30 user nova-compute[75234]:   <features>
Nov 29 03:10:30 user nova-compute[75234]:     <acpi/>
Nov 29 03:10:30 user nova-compute[75234]:     <apic/>
Nov 29 03:10:30 user nova-compute[75234]:     <vmcoreinfo/>
Nov 29 03:10:30 user nova-compute[75234]:   </features>
Nov 29 03:10:30 user nova-compute[75234]:   <clock offset="utc">
Nov 29 03:10:30 user nova-compute[75234]:     <timer name="pit" tickpolicy="delay"/>
Nov 29 03:10:30 user nova-compute[75234]:     <timer name="rtc" tickpolicy="catchup"/>
Nov 29 03:10:30 user nova-compute[75234]:     <timer name="hpet" present="no"/>
Nov 29 03:10:30 user nova-compute[75234]:   </clock>
Nov 29 03:10:30 user nova-compute[75234]:   <cpu mode="custom" match="exact">
Nov 29 03:10:30 user nova-compute[75234]:     <model>Nehalem</model>
Nov 29 03:10:30 user nova-compute[75234]:     <topology sockets="1" cores="1" threads="1"/>
Nov 29 03:10:30 user nova-compute[75234]:   </cpu>
Nov 29 03:10:30 user nova-compute[75234]:   <devices>
Nov 29 03:10:30 user nova-compute[75234]:     <disk type="file" device="disk">
Nov 29 03:10:30 user nova-compute[75234]:       <driver name="qemu" type="qcow2" cache="none"/>
Nov 29 03:10:30 user nova-compute[75234]:       <source file="/opt/stack/data/nova/instances/d3b90f39-4b91-4152-b820-bd0e263eec89/disk"/>
Nov 29 03:10:30 user nova-compute[75234]:       <target dev="vda" bus="virtio"/>
Nov 29 03:10:30 user nova-compute[75234]:     </disk>
Nov 29 03:10:30 user nova-compute[75234]:     <interface type="ethernet">
Nov 29 03:10:30 user nova-compute[75234]:       <mac address="fa:16:3e:9d:70:dd"/>
Nov 29 03:10:30 user nova-compute[75234]:       <model type="virtio"/>
Nov 29 03:10:30 user nova-compute[75234]:       <mtu size="1442"/>
Nov 29 03:10:30 user nova-compute[75234]:       <target dev="tap83eb52bb-5b"/>
Nov 29 03:10:30 user nova-compute[75234]:     </interface>
Nov 29 03:10:30 user nova-compute[75234]:     <serial type="pty">
Nov 29 03:10:30 user nova-compute[75234]:       <log file="/opt/stack/data/nova/instances/d3b90f39-4b91-4152-b820-bd0e263eec89/console.log" append="off"/>
Nov 29 03:10:30 user nova-compute[75234]:     </serial>
Nov 29 03:10:30 user nova-compute[75234]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Nov 29 03:10:30 user nova-compute[75234]:     <video>
Nov 29 03:10:30 user nova-compute[75234]:       <model type="virtio"/>
Nov 29 03:10:30 user nova-compute[75234]:     </video>
Nov 29 03:10:30 user nova-compute[75234]:     <rng model="virtio">
Nov 29 03:10:30 user nova-compute[75234]:       <backend model="random">/dev/urandom</backend>
Nov 29 03:10:30 user nova-compute[75234]:     </rng>
Nov 29 03:10:30 user nova-compute[75234]:     <controller type="usb" index="0" model="none"/>
Nov 29 03:10:30 user nova-compute[75234]:     <memballoon model="virtio">
Nov 29 03:10:30 user nova-compute[75234]:       <stats period="10"/>
Nov 29 03:10:30 user nova-compute[75234]:     </memballoon>
Nov 29 03:10:30 user nova-compute[75234]:   </devices>
Nov 29 03:10:30 user nova-compute[75234]: </domain>
Nov 29 03:10:30 user nova-compute[75234]:  {{(pid=75234) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7568}}
Nov 29 03:10:30 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-046208e3-2011-4422-97ac-ba2e4f78cd46 tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-11-29T03:10:24Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-VolumesAdminNegativeTest-server-846124845',display_name='tempest-VolumesAdminNegativeTest-server-846124845',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-volumesadminnegativetest-server-846124845',id=17,image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBOhY8TGmPVW2eahQ/7NwxuWatHHhSisv4kKa49hvwZ8KZoBzAoVyERO5tPD8b1ELtbNUxo/5KbqYKeCg6v3iRzJlbCyhkM5yEZuV+qUhHx8TePthXUgsgJEbpNvQr1IWCw==',key_name='tempest-keypair-534984900',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='5eaa91ebc2e64f2eba987f02bcd1a9f4',ramdisk_id='',reservation_id='r-2l9d0jow',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-VolumesAdminNegativeTest-1445338761',owner_user_name='tempest-VolumesAdminNegativeTest-1445338761-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-11-29T03:10:26Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='460854be8e05408a80790620f64cf8e1',uuid=d3b90f39-4b91-4152-b820-bd0e263eec89,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "83eb52bb-5be5-49d6-81ec-86a21dde3df2", "address": "fa:16:3e:9d:70:dd", "network": {"id": "49d62bf6-34ee-4de8-ab85-6ef489d957c8", "bridge": "br-int", "label": "tempest-VolumesAdminNegativeTest-1304144113-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "5eaa91ebc2e64f2eba987f02bcd1a9f4", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap83eb52bb-5b", "ovs_interfaceid": "83eb52bb-5be5-49d6-81ec-86a21dde3df2", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Nov 29 03:10:30 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-046208e3-2011-4422-97ac-ba2e4f78cd46 tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] Converting VIF {"id": "83eb52bb-5be5-49d6-81ec-86a21dde3df2", "address": "fa:16:3e:9d:70:dd", "network": {"id": "49d62bf6-34ee-4de8-ab85-6ef489d957c8", "bridge": "br-int", "label": "tempest-VolumesAdminNegativeTest-1304144113-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "5eaa91ebc2e64f2eba987f02bcd1a9f4", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap83eb52bb-5b", "ovs_interfaceid": "83eb52bb-5be5-49d6-81ec-86a21dde3df2", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:10:30 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-046208e3-2011-4422-97ac-ba2e4f78cd46 tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:9d:70:dd,bridge_name='br-int',has_traffic_filtering=True,id=83eb52bb-5be5-49d6-81ec-86a21dde3df2,network=Network(49d62bf6-34ee-4de8-ab85-6ef489d957c8),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap83eb52bb-5b') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:10:30 user nova-compute[75234]: DEBUG os_vif [None req-046208e3-2011-4422-97ac-ba2e4f78cd46 tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:9d:70:dd,bridge_name='br-int',has_traffic_filtering=True,id=83eb52bb-5be5-49d6-81ec-86a21dde3df2,network=Network(49d62bf6-34ee-4de8-ab85-6ef489d957c8),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap83eb52bb-5b') {{(pid=75234) plug /opt/stack/data/venv/lib/python3.10/site-packages/os_vif/__init__.py:76}}
Nov 29 03:10:30 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:10:30 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:10:30 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Nov 29 03:10:30 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:10:30 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbCreateCommand(_result=None, table=QoS, columns={'type': 'linux-noop', 'external_ids': {'id': '8948d378-15f0-5f34-848c-836f9263d5bd', '_type': 'linux-noop'}}, row=False) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:10:30 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:10:30 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:10:30 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:10:30 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tap83eb52bb-5b, may_exist=True, interface_attrs={}) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:10:30 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Port, record=tap83eb52bb-5b, col_values=(('qos', UUID('c5f89983-4f48-4826-b11b-d1c063cdaea7')),), if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:10:30 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DbSetCommand(_result=None, table=Interface, record=tap83eb52bb-5b, col_values=(('external_ids', {'iface-id': '83eb52bb-5be5-49d6-81ec-86a21dde3df2', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:9d:70:dd', 'vm-uuid': 'd3b90f39-4b91-4152-b820-bd0e263eec89'}),), if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:10:30 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:10:30 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:10:30 user nova-compute[75234]: INFO os_vif [None req-046208e3-2011-4422-97ac-ba2e4f78cd46 tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:9d:70:dd,bridge_name='br-int',has_traffic_filtering=True,id=83eb52bb-5be5-49d6-81ec-86a21dde3df2,network=Network(49d62bf6-34ee-4de8-ab85-6ef489d957c8),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap83eb52bb-5b')
Nov 29 03:10:30 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-046208e3-2011-4422-97ac-ba2e4f78cd46 tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] No BDM found with device name vda, not building metadata. {{(pid=75234) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12089}}
Nov 29 03:10:30 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-046208e3-2011-4422-97ac-ba2e4f78cd46 tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] No VIF found with MAC fa:16:3e:9d:70:dd, not building metadata {{(pid=75234) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Nov 29 03:10:30 user nova-compute[75234]: DEBUG nova.network.neutron [None req-97e2cf32-f7d5-4ab9-a932-35fa0c7cd310 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] [instance: 13e5da64-9adc-4479-81c7-d329e9b5f08f] Successfully updated port: 2abf9e25-d1d9-4893-899e-912d41b7c38f {{(pid=75234) _update_port /opt/stack/nova/nova/network/neutron.py:586}}
Nov 29 03:10:30 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-97e2cf32-f7d5-4ab9-a932-35fa0c7cd310 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Acquiring lock "refresh_cache-13e5da64-9adc-4479-81c7-d329e9b5f08f" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:10:30 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-97e2cf32-f7d5-4ab9-a932-35fa0c7cd310 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Acquired lock "refresh_cache-13e5da64-9adc-4479-81c7-d329e9b5f08f" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:10:30 user nova-compute[75234]: DEBUG nova.network.neutron [None req-97e2cf32-f7d5-4ab9-a932-35fa0c7cd310 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] [instance: 13e5da64-9adc-4479-81c7-d329e9b5f08f] Building network info cache for instance {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2009}}
Nov 29 03:10:31 user nova-compute[75234]: DEBUG nova.network.neutron [None req-ddae26ab-aba2-44e2-9f96-a517489c193b tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] [instance: a082c96e-e384-40b6-bcc8-fc5e0e104669] Successfully updated port: 0d131138-51b6-49f9-9a11-6b64e32e5796 {{(pid=75234) _update_port /opt/stack/nova/nova/network/neutron.py:586}}
Nov 29 03:10:31 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ddae26ab-aba2-44e2-9f96-a517489c193b tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Acquiring lock "refresh_cache-a082c96e-e384-40b6-bcc8-fc5e0e104669" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:10:31 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ddae26ab-aba2-44e2-9f96-a517489c193b tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Acquired lock "refresh_cache-a082c96e-e384-40b6-bcc8-fc5e0e104669" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:10:31 user nova-compute[75234]: DEBUG nova.network.neutron [None req-ddae26ab-aba2-44e2-9f96-a517489c193b tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] [instance: a082c96e-e384-40b6-bcc8-fc5e0e104669] Building network info cache for instance {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2009}}
Nov 29 03:10:31 user nova-compute[75234]: DEBUG nova.network.neutron [None req-97e2cf32-f7d5-4ab9-a932-35fa0c7cd310 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] [instance: 13e5da64-9adc-4479-81c7-d329e9b5f08f] Instance cache missing network info. {{(pid=75234) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3322}}
Nov 29 03:10:31 user nova-compute[75234]: DEBUG nova.network.neutron [req-661586d9-8d0a-45d0-9daa-b722e86406b1 req-815a28ed-41ba-4ba4-9eab-5a6661eb375a service nova] [instance: d3b90f39-4b91-4152-b820-bd0e263eec89] Updated VIF entry in instance network info cache for port 83eb52bb-5be5-49d6-81ec-86a21dde3df2. {{(pid=75234) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3481}}
Nov 29 03:10:31 user nova-compute[75234]: DEBUG nova.network.neutron [req-661586d9-8d0a-45d0-9daa-b722e86406b1 req-815a28ed-41ba-4ba4-9eab-5a6661eb375a service nova] [instance: d3b90f39-4b91-4152-b820-bd0e263eec89] Updating instance_info_cache with network_info: [{"id": "83eb52bb-5be5-49d6-81ec-86a21dde3df2", "address": "fa:16:3e:9d:70:dd", "network": {"id": "49d62bf6-34ee-4de8-ab85-6ef489d957c8", "bridge": "br-int", "label": "tempest-VolumesAdminNegativeTest-1304144113-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "5eaa91ebc2e64f2eba987f02bcd1a9f4", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap83eb52bb-5b", "ovs_interfaceid": "83eb52bb-5be5-49d6-81ec-86a21dde3df2", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:10:31 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-661586d9-8d0a-45d0-9daa-b722e86406b1 req-815a28ed-41ba-4ba4-9eab-5a6661eb375a service nova] Releasing lock "refresh_cache-d3b90f39-4b91-4152-b820-bd0e263eec89" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:10:31 user nova-compute[75234]: DEBUG nova.network.neutron [None req-ddae26ab-aba2-44e2-9f96-a517489c193b tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] [instance: a082c96e-e384-40b6-bcc8-fc5e0e104669] Instance cache missing network info. {{(pid=75234) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3322}}
Nov 29 03:10:31 user nova-compute[75234]: DEBUG nova.network.neutron [None req-ddae26ab-aba2-44e2-9f96-a517489c193b tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] [instance: a082c96e-e384-40b6-bcc8-fc5e0e104669] Updating instance_info_cache with network_info: [{"id": "0d131138-51b6-49f9-9a11-6b64e32e5796", "address": "fa:16:3e:ee:2e:82", "network": {"id": "2b3c3140-f3b6-45da-8202-bb4d7f0dd130", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-2043420778-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "f8a7884867c944e1ba1024270e4902f3", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap0d131138-51", "ovs_interfaceid": "0d131138-51b6-49f9-9a11-6b64e32e5796", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:10:31 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ddae26ab-aba2-44e2-9f96-a517489c193b tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Releasing lock "refresh_cache-a082c96e-e384-40b6-bcc8-fc5e0e104669" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:10:31 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ddae26ab-aba2-44e2-9f96-a517489c193b tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] [instance: a082c96e-e384-40b6-bcc8-fc5e0e104669] Instance network_info: |[{"id": "0d131138-51b6-49f9-9a11-6b64e32e5796", "address": "fa:16:3e:ee:2e:82", "network": {"id": "2b3c3140-f3b6-45da-8202-bb4d7f0dd130", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-2043420778-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "f8a7884867c944e1ba1024270e4902f3", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap0d131138-51", "ovs_interfaceid": "0d131138-51b6-49f9-9a11-6b64e32e5796", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=75234) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1987}}
Nov 29 03:10:31 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-ddae26ab-aba2-44e2-9f96-a517489c193b tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] [instance: a082c96e-e384-40b6-bcc8-fc5e0e104669] Start _get_guest_xml network_info=[{"id": "0d131138-51b6-49f9-9a11-6b64e32e5796", "address": "fa:16:3e:ee:2e:82", "network": {"id": "2b3c3140-f3b6-45da-8202-bb4d7f0dd130", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-2043420778-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "f8a7884867c944e1ba1024270e4902f3", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap0d131138-51", "ovs_interfaceid": "0d131138-51b6-49f9-9a11-6b64e32e5796", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum='c8fc807773e5354afe61636071771906',container_format='bare',created_at=2023-11-29T03:02:15Z,direct_url=<?>,disk_format='qcow2',id=96c30ba7-9b73-4a30-8ba4-7138879573b7,min_disk=0,min_ram=0,name='cirros-0.6.2-x86_64-disk',owner='b18e380921c748b78517b6f1211cd0f1',properties=ImageMetaProps,protected=<?>,size=21430272,status='active',tags=<?>,updated_at=2023-11-29T03:02:17Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [{'device_name': '/dev/vda', 'encryption_format': None, 'encrypted': False, 'device_type': 'disk', 'size': 0, 'disk_bus': 'virtio', 'boot_index': 0, 'encryption_options': None, 'encryption_secret_uuid': None, 'guest_format': None, 'image_id': '96c30ba7-9b73-4a30-8ba4-7138879573b7'}], 'ephemerals': [], 'block_device_mapping': [], 'swap': None} {{(pid=75234) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7562}}
Nov 29 03:10:31 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-ddae26ab-aba2-44e2-9f96-a517489c193b tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:10:31 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-ddae26ab-aba2-44e2-9f96-a517489c193b tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:10:31 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-ddae26ab-aba2-44e2-9f96-a517489c193b tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Searching host: 'user' for CPU controller through CGroups V1... {{(pid=75234) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1652}}
Nov 29 03:10:31 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-ddae26ab-aba2-44e2-9f96-a517489c193b tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] CPU controller missing on host. {{(pid=75234) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1662}}
Nov 29 03:10:31 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:10:31 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-ddae26ab-aba2-44e2-9f96-a517489c193b tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Searching host: 'user' for CPU controller through CGroups V2... {{(pid=75234) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1671}}
Nov 29 03:10:31 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-ddae26ab-aba2-44e2-9f96-a517489c193b tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] CPU controller found on host. {{(pid=75234) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1678}}
Nov 29 03:10:31 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-ddae26ab-aba2-44e2-9f96-a517489c193b tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=75234) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5400}}
Nov 29 03:10:31 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-ddae26ab-aba2-44e2-9f96-a517489c193b tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-11-29T03:02:04Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='1',id=6,is_public=True,memory_mb=512,name='m1.tiny',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='c8fc807773e5354afe61636071771906',container_format='bare',created_at=2023-11-29T03:02:15Z,direct_url=<?>,disk_format='qcow2',id=96c30ba7-9b73-4a30-8ba4-7138879573b7,min_disk=0,min_ram=0,name='cirros-0.6.2-x86_64-disk',owner='b18e380921c748b78517b6f1211cd0f1',properties=ImageMetaProps,protected=<?>,size=21430272,status='active',tags=<?>,updated_at=2023-11-29T03:02:17Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:563}}
Nov 29 03:10:31 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-ddae26ab-aba2-44e2-9f96-a517489c193b tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Flavor limits 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:348}}
Nov 29 03:10:31 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-ddae26ab-aba2-44e2-9f96-a517489c193b tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Image limits 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:352}}
Nov 29 03:10:31 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-ddae26ab-aba2-44e2-9f96-a517489c193b tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Flavor pref 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:388}}
Nov 29 03:10:31 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-ddae26ab-aba2-44e2-9f96-a517489c193b tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Image pref 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:392}}
Nov 29 03:10:31 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-ddae26ab-aba2-44e2-9f96-a517489c193b tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:430}}
Nov 29 03:10:31 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-ddae26ab-aba2-44e2-9f96-a517489c193b tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:569}}
Nov 29 03:10:31 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-ddae26ab-aba2-44e2-9f96-a517489c193b tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=75234) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:471}}
Nov 29 03:10:31 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-ddae26ab-aba2-44e2-9f96-a517489c193b tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Got 1 possible topologies {{(pid=75234) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:501}}
Nov 29 03:10:31 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-ddae26ab-aba2-44e2-9f96-a517489c193b tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:575}}
Nov 29 03:10:31 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-ddae26ab-aba2-44e2-9f96-a517489c193b tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:577}}
Nov 29 03:10:31 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-ddae26ab-aba2-44e2-9f96-a517489c193b tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-11-29T03:10:26Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-ServerRescueNegativeTestJSON-server-2058873758',display_name='tempest-ServerRescueNegativeTestJSON-server-2058873758',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-serverrescuenegativetestjson-server-2058873758',id=19,image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data=None,key_name=None,keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='f8a7884867c944e1ba1024270e4902f3',ramdisk_id='',reservation_id='r-58ukk6ag',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-ServerRescueNegativeTestJSON-1182706601',owner_user_name='tempest-ServerRescueNegativeTestJSON-1182706601-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-11-29T03:10:29Z,user_data=None,user_id='6792e866677645c598ddb7d2c1723a1a',uuid=a082c96e-e384-40b6-bcc8-fc5e0e104669,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "0d131138-51b6-49f9-9a11-6b64e32e5796", "address": "fa:16:3e:ee:2e:82", "network": {"id": "2b3c3140-f3b6-45da-8202-bb4d7f0dd130", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-2043420778-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "f8a7884867c944e1ba1024270e4902f3", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap0d131138-51", "ovs_interfaceid": "0d131138-51b6-49f9-9a11-6b64e32e5796", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=75234) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Nov 29 03:10:31 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-ddae26ab-aba2-44e2-9f96-a517489c193b tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Converting VIF {"id": "0d131138-51b6-49f9-9a11-6b64e32e5796", "address": "fa:16:3e:ee:2e:82", "network": {"id": "2b3c3140-f3b6-45da-8202-bb4d7f0dd130", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-2043420778-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "f8a7884867c944e1ba1024270e4902f3", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap0d131138-51", "ovs_interfaceid": "0d131138-51b6-49f9-9a11-6b64e32e5796", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:10:31 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-ddae26ab-aba2-44e2-9f96-a517489c193b tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:ee:2e:82,bridge_name='br-int',has_traffic_filtering=True,id=0d131138-51b6-49f9-9a11-6b64e32e5796,network=Network(2b3c3140-f3b6-45da-8202-bb4d7f0dd130),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap0d131138-51') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:10:31 user nova-compute[75234]: DEBUG nova.objects.instance [None req-ddae26ab-aba2-44e2-9f96-a517489c193b tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Lazy-loading 'pci_devices' on Instance uuid a082c96e-e384-40b6-bcc8-fc5e0e104669 {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:10:31 user nova-compute[75234]: DEBUG nova.network.neutron [None req-97e2cf32-f7d5-4ab9-a932-35fa0c7cd310 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] [instance: 13e5da64-9adc-4479-81c7-d329e9b5f08f] Updating instance_info_cache with network_info: [{"id": "2abf9e25-d1d9-4893-899e-912d41b7c38f", "address": "fa:16:3e:59:79:35", "network": {"id": "2b3c3140-f3b6-45da-8202-bb4d7f0dd130", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-2043420778-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "f8a7884867c944e1ba1024270e4902f3", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2abf9e25-d1", "ovs_interfaceid": "2abf9e25-d1d9-4893-899e-912d41b7c38f", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:10:31 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-ddae26ab-aba2-44e2-9f96-a517489c193b tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] [instance: a082c96e-e384-40b6-bcc8-fc5e0e104669] End _get_guest_xml xml=<domain type="kvm">
Nov 29 03:10:31 user nova-compute[75234]:   <uuid>a082c96e-e384-40b6-bcc8-fc5e0e104669</uuid>
Nov 29 03:10:31 user nova-compute[75234]:   <name>instance-00000013</name>
Nov 29 03:10:31 user nova-compute[75234]:   <memory>524288</memory>
Nov 29 03:10:31 user nova-compute[75234]:   <vcpu>1</vcpu>
Nov 29 03:10:31 user nova-compute[75234]:   <metadata>
Nov 29 03:10:31 user nova-compute[75234]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Nov 29 03:10:31 user nova-compute[75234]:       <nova:package version="0.0.0"/>
Nov 29 03:10:31 user nova-compute[75234]:       <nova:name>tempest-ServerRescueNegativeTestJSON-server-2058873758</nova:name>
Nov 29 03:10:31 user nova-compute[75234]:       <nova:creationTime>2023-11-29 03:10:31</nova:creationTime>
Nov 29 03:10:31 user nova-compute[75234]:       <nova:flavor name="m1.tiny">
Nov 29 03:10:31 user nova-compute[75234]:         <nova:memory>512</nova:memory>
Nov 29 03:10:31 user nova-compute[75234]:         <nova:disk>1</nova:disk>
Nov 29 03:10:31 user nova-compute[75234]:         <nova:swap>0</nova:swap>
Nov 29 03:10:31 user nova-compute[75234]:         <nova:ephemeral>0</nova:ephemeral>
Nov 29 03:10:31 user nova-compute[75234]:         <nova:vcpus>1</nova:vcpus>
Nov 29 03:10:31 user nova-compute[75234]:       </nova:flavor>
Nov 29 03:10:31 user nova-compute[75234]:       <nova:owner>
Nov 29 03:10:31 user nova-compute[75234]:         <nova:user uuid="6792e866677645c598ddb7d2c1723a1a">tempest-ServerRescueNegativeTestJSON-1182706601-project-member</nova:user>
Nov 29 03:10:31 user nova-compute[75234]:         <nova:project uuid="f8a7884867c944e1ba1024270e4902f3">tempest-ServerRescueNegativeTestJSON-1182706601</nova:project>
Nov 29 03:10:31 user nova-compute[75234]:       </nova:owner>
Nov 29 03:10:31 user nova-compute[75234]:       <nova:root type="image" uuid="96c30ba7-9b73-4a30-8ba4-7138879573b7"/>
Nov 29 03:10:31 user nova-compute[75234]:       <nova:ports>
Nov 29 03:10:31 user nova-compute[75234]:         <nova:port uuid="0d131138-51b6-49f9-9a11-6b64e32e5796">
Nov 29 03:10:31 user nova-compute[75234]:           <nova:ip type="fixed" address="10.0.0.10" ipVersion="4"/>
Nov 29 03:10:31 user nova-compute[75234]:         </nova:port>
Nov 29 03:10:31 user nova-compute[75234]:       </nova:ports>
Nov 29 03:10:31 user nova-compute[75234]:     </nova:instance>
Nov 29 03:10:31 user nova-compute[75234]:   </metadata>
Nov 29 03:10:31 user nova-compute[75234]:   <sysinfo type="smbios">
Nov 29 03:10:31 user nova-compute[75234]:     <system>
Nov 29 03:10:31 user nova-compute[75234]:       <entry name="manufacturer">OpenStack Foundation</entry>
Nov 29 03:10:31 user nova-compute[75234]:       <entry name="product">OpenStack Nova</entry>
Nov 29 03:10:31 user nova-compute[75234]:       <entry name="version">0.0.0</entry>
Nov 29 03:10:31 user nova-compute[75234]:       <entry name="serial">a082c96e-e384-40b6-bcc8-fc5e0e104669</entry>
Nov 29 03:10:31 user nova-compute[75234]:       <entry name="uuid">a082c96e-e384-40b6-bcc8-fc5e0e104669</entry>
Nov 29 03:10:31 user nova-compute[75234]:       <entry name="family">Virtual Machine</entry>
Nov 29 03:10:31 user nova-compute[75234]:     </system>
Nov 29 03:10:31 user nova-compute[75234]:   </sysinfo>
Nov 29 03:10:31 user nova-compute[75234]:   <os>
Nov 29 03:10:31 user nova-compute[75234]:     <type arch="x86_64" machine="pc">hvm</type>
Nov 29 03:10:31 user nova-compute[75234]:     <boot dev="hd"/>
Nov 29 03:10:31 user nova-compute[75234]:     <smbios mode="sysinfo"/>
Nov 29 03:10:31 user nova-compute[75234]:   </os>
Nov 29 03:10:31 user nova-compute[75234]:   <features>
Nov 29 03:10:31 user nova-compute[75234]:     <acpi/>
Nov 29 03:10:31 user nova-compute[75234]:     <apic/>
Nov 29 03:10:31 user nova-compute[75234]:     <vmcoreinfo/>
Nov 29 03:10:31 user nova-compute[75234]:   </features>
Nov 29 03:10:31 user nova-compute[75234]:   <clock offset="utc">
Nov 29 03:10:31 user nova-compute[75234]:     <timer name="pit" tickpolicy="delay"/>
Nov 29 03:10:31 user nova-compute[75234]:     <timer name="rtc" tickpolicy="catchup"/>
Nov 29 03:10:31 user nova-compute[75234]:     <timer name="hpet" present="no"/>
Nov 29 03:10:31 user nova-compute[75234]:   </clock>
Nov 29 03:10:31 user nova-compute[75234]:   <cpu mode="custom" match="exact">
Nov 29 03:10:31 user nova-compute[75234]:     <model>Nehalem</model>
Nov 29 03:10:31 user nova-compute[75234]:     <topology sockets="1" cores="1" threads="1"/>
Nov 29 03:10:31 user nova-compute[75234]:   </cpu>
Nov 29 03:10:31 user nova-compute[75234]:   <devices>
Nov 29 03:10:31 user nova-compute[75234]:     <disk type="file" device="disk">
Nov 29 03:10:31 user nova-compute[75234]:       <driver name="qemu" type="qcow2" cache="none"/>
Nov 29 03:10:31 user nova-compute[75234]:       <source file="/opt/stack/data/nova/instances/a082c96e-e384-40b6-bcc8-fc5e0e104669/disk"/>
Nov 29 03:10:31 user nova-compute[75234]:       <target dev="vda" bus="virtio"/>
Nov 29 03:10:31 user nova-compute[75234]:     </disk>
Nov 29 03:10:31 user nova-compute[75234]:     <interface type="ethernet">
Nov 29 03:10:31 user nova-compute[75234]:       <mac address="fa:16:3e:ee:2e:82"/>
Nov 29 03:10:31 user nova-compute[75234]:       <model type="virtio"/>
Nov 29 03:10:31 user nova-compute[75234]:       <mtu size="1442"/>
Nov 29 03:10:31 user nova-compute[75234]:       <target dev="tap0d131138-51"/>
Nov 29 03:10:31 user nova-compute[75234]:     </interface>
Nov 29 03:10:31 user nova-compute[75234]:     <serial type="pty">
Nov 29 03:10:31 user nova-compute[75234]:       <log file="/opt/stack/data/nova/instances/a082c96e-e384-40b6-bcc8-fc5e0e104669/console.log" append="off"/>
Nov 29 03:10:31 user nova-compute[75234]:     </serial>
Nov 29 03:10:31 user nova-compute[75234]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Nov 29 03:10:31 user nova-compute[75234]:     <video>
Nov 29 03:10:31 user nova-compute[75234]:       <model type="virtio"/>
Nov 29 03:10:31 user nova-compute[75234]:     </video>
Nov 29 03:10:31 user nova-compute[75234]:     <rng model="virtio">
Nov 29 03:10:31 user nova-compute[75234]:       <backend model="random">/dev/urandom</backend>
Nov 29 03:10:31 user nova-compute[75234]:     </rng>
Nov 29 03:10:31 user nova-compute[75234]:     <controller type="usb" index="0" model="none"/>
Nov 29 03:10:31 user nova-compute[75234]:     <memballoon model="virtio">
Nov 29 03:10:31 user nova-compute[75234]:       <stats period="10"/>
Nov 29 03:10:31 user nova-compute[75234]:     </memballoon>
Nov 29 03:10:31 user nova-compute[75234]:   </devices>
Nov 29 03:10:31 user nova-compute[75234]: </domain>
Nov 29 03:10:31 user nova-compute[75234]:  {{(pid=75234) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7568}}
Nov 29 03:10:31 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-ddae26ab-aba2-44e2-9f96-a517489c193b tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-11-29T03:10:26Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-ServerRescueNegativeTestJSON-server-2058873758',display_name='tempest-ServerRescueNegativeTestJSON-server-2058873758',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-serverrescuenegativetestjson-server-2058873758',id=19,image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data=None,key_name=None,keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='f8a7884867c944e1ba1024270e4902f3',ramdisk_id='',reservation_id='r-58ukk6ag',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-ServerRescueNegativeTestJSON-1182706601',owner_user_name='tempest-ServerRescueNegativeTestJSON-1182706601-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-11-29T03:10:29Z,user_data=None,user_id='6792e866677645c598ddb7d2c1723a1a',uuid=a082c96e-e384-40b6-bcc8-fc5e0e104669,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "0d131138-51b6-49f9-9a11-6b64e32e5796", "address": "fa:16:3e:ee:2e:82", "network": {"id": "2b3c3140-f3b6-45da-8202-bb4d7f0dd130", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-2043420778-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "f8a7884867c944e1ba1024270e4902f3", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap0d131138-51", "ovs_interfaceid": "0d131138-51b6-49f9-9a11-6b64e32e5796", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Nov 29 03:10:31 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-ddae26ab-aba2-44e2-9f96-a517489c193b tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Converting VIF {"id": "0d131138-51b6-49f9-9a11-6b64e32e5796", "address": "fa:16:3e:ee:2e:82", "network": {"id": "2b3c3140-f3b6-45da-8202-bb4d7f0dd130", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-2043420778-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "f8a7884867c944e1ba1024270e4902f3", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap0d131138-51", "ovs_interfaceid": "0d131138-51b6-49f9-9a11-6b64e32e5796", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:10:31 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-ddae26ab-aba2-44e2-9f96-a517489c193b tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:ee:2e:82,bridge_name='br-int',has_traffic_filtering=True,id=0d131138-51b6-49f9-9a11-6b64e32e5796,network=Network(2b3c3140-f3b6-45da-8202-bb4d7f0dd130),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap0d131138-51') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:10:31 user nova-compute[75234]: DEBUG os_vif [None req-ddae26ab-aba2-44e2-9f96-a517489c193b tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:ee:2e:82,bridge_name='br-int',has_traffic_filtering=True,id=0d131138-51b6-49f9-9a11-6b64e32e5796,network=Network(2b3c3140-f3b6-45da-8202-bb4d7f0dd130),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap0d131138-51') {{(pid=75234) plug /opt/stack/data/venv/lib/python3.10/site-packages/os_vif/__init__.py:76}}
Nov 29 03:10:31 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:10:31 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:10:31 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Nov 29 03:10:31 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-97e2cf32-f7d5-4ab9-a932-35fa0c7cd310 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Releasing lock "refresh_cache-13e5da64-9adc-4479-81c7-d329e9b5f08f" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:10:31 user nova-compute[75234]: DEBUG nova.compute.manager [None req-97e2cf32-f7d5-4ab9-a932-35fa0c7cd310 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] [instance: 13e5da64-9adc-4479-81c7-d329e9b5f08f] Instance network_info: |[{"id": "2abf9e25-d1d9-4893-899e-912d41b7c38f", "address": "fa:16:3e:59:79:35", "network": {"id": "2b3c3140-f3b6-45da-8202-bb4d7f0dd130", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-2043420778-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "f8a7884867c944e1ba1024270e4902f3", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2abf9e25-d1", "ovs_interfaceid": "2abf9e25-d1d9-4893-899e-912d41b7c38f", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=75234) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1987}}
Nov 29 03:10:31 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:10:31 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbCreateCommand(_result=None, table=QoS, columns={'type': 'linux-noop', 'external_ids': {'id': '56f7785d-39a2-53f2-ae5e-8e430123d891', '_type': 'linux-noop'}}, row=False) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:10:31 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-97e2cf32-f7d5-4ab9-a932-35fa0c7cd310 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] [instance: 13e5da64-9adc-4479-81c7-d329e9b5f08f] Start _get_guest_xml network_info=[{"id": "2abf9e25-d1d9-4893-899e-912d41b7c38f", "address": "fa:16:3e:59:79:35", "network": {"id": "2b3c3140-f3b6-45da-8202-bb4d7f0dd130", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-2043420778-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "f8a7884867c944e1ba1024270e4902f3", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2abf9e25-d1", "ovs_interfaceid": "2abf9e25-d1d9-4893-899e-912d41b7c38f", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum='c8fc807773e5354afe61636071771906',container_format='bare',created_at=2023-11-29T03:02:15Z,direct_url=<?>,disk_format='qcow2',id=96c30ba7-9b73-4a30-8ba4-7138879573b7,min_disk=0,min_ram=0,name='cirros-0.6.2-x86_64-disk',owner='b18e380921c748b78517b6f1211cd0f1',properties=ImageMetaProps,protected=<?>,size=21430272,status='active',tags=<?>,updated_at=2023-11-29T03:02:17Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [{'device_name': '/dev/vda', 'encryption_format': None, 'encrypted': False, 'device_type': 'disk', 'size': 0, 'disk_bus': 'virtio', 'boot_index': 0, 'encryption_options': None, 'encryption_secret_uuid': None, 'guest_format': None, 'image_id': '96c30ba7-9b73-4a30-8ba4-7138879573b7'}], 'ephemerals': [], 'block_device_mapping': [], 'swap': None} {{(pid=75234) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7562}}
Nov 29 03:10:31 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:10:31 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:10:31 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:10:31 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-97e2cf32-f7d5-4ab9-a932-35fa0c7cd310 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:10:31 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-97e2cf32-f7d5-4ab9-a932-35fa0c7cd310 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:10:31 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:10:31 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tap0d131138-51, may_exist=True, interface_attrs={}) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:10:31 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Port, record=tap0d131138-51, col_values=(('qos', UUID('0651645b-eb78-4bea-9b8d-25f0c14ee539')),), if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:10:31 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DbSetCommand(_result=None, table=Interface, record=tap0d131138-51, col_values=(('external_ids', {'iface-id': '0d131138-51b6-49f9-9a11-6b64e32e5796', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:ee:2e:82', 'vm-uuid': 'a082c96e-e384-40b6-bcc8-fc5e0e104669'}),), if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:10:31 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:10:31 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-97e2cf32-f7d5-4ab9-a932-35fa0c7cd310 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Searching host: 'user' for CPU controller through CGroups V1... {{(pid=75234) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1652}}
Nov 29 03:10:31 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-97e2cf32-f7d5-4ab9-a932-35fa0c7cd310 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] CPU controller missing on host. {{(pid=75234) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1662}}
Nov 29 03:10:31 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:10:31 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-97e2cf32-f7d5-4ab9-a932-35fa0c7cd310 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Searching host: 'user' for CPU controller through CGroups V2... {{(pid=75234) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1671}}
Nov 29 03:10:31 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-97e2cf32-f7d5-4ab9-a932-35fa0c7cd310 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] CPU controller found on host. {{(pid=75234) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1678}}
Nov 29 03:10:31 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-97e2cf32-f7d5-4ab9-a932-35fa0c7cd310 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=75234) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5400}}
Nov 29 03:10:31 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-97e2cf32-f7d5-4ab9-a932-35fa0c7cd310 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-11-29T03:02:04Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='1',id=6,is_public=True,memory_mb=512,name='m1.tiny',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='c8fc807773e5354afe61636071771906',container_format='bare',created_at=2023-11-29T03:02:15Z,direct_url=<?>,disk_format='qcow2',id=96c30ba7-9b73-4a30-8ba4-7138879573b7,min_disk=0,min_ram=0,name='cirros-0.6.2-x86_64-disk',owner='b18e380921c748b78517b6f1211cd0f1',properties=ImageMetaProps,protected=<?>,size=21430272,status='active',tags=<?>,updated_at=2023-11-29T03:02:17Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:563}}
Nov 29 03:10:31 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-97e2cf32-f7d5-4ab9-a932-35fa0c7cd310 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Flavor limits 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:348}}
Nov 29 03:10:31 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-97e2cf32-f7d5-4ab9-a932-35fa0c7cd310 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Image limits 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:352}}
Nov 29 03:10:31 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-97e2cf32-f7d5-4ab9-a932-35fa0c7cd310 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Flavor pref 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:388}}
Nov 29 03:10:31 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-97e2cf32-f7d5-4ab9-a932-35fa0c7cd310 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Image pref 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:392}}
Nov 29 03:10:31 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-97e2cf32-f7d5-4ab9-a932-35fa0c7cd310 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:430}}
Nov 29 03:10:31 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-97e2cf32-f7d5-4ab9-a932-35fa0c7cd310 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:569}}
Nov 29 03:10:31 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-97e2cf32-f7d5-4ab9-a932-35fa0c7cd310 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=75234) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:471}}
Nov 29 03:10:31 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-97e2cf32-f7d5-4ab9-a932-35fa0c7cd310 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Got 1 possible topologies {{(pid=75234) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:501}}
Nov 29 03:10:31 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-97e2cf32-f7d5-4ab9-a932-35fa0c7cd310 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:575}}
Nov 29 03:10:31 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-97e2cf32-f7d5-4ab9-a932-35fa0c7cd310 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:577}}
Nov 29 03:10:31 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-97e2cf32-f7d5-4ab9-a932-35fa0c7cd310 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-11-29T03:10:25Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-ServerRescueNegativeTestJSON-server-715053465',display_name='tempest-ServerRescueNegativeTestJSON-server-715053465',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-serverrescuenegativetestjson-server-715053465',id=18,image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data=None,key_name=None,keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='f8a7884867c944e1ba1024270e4902f3',ramdisk_id='',reservation_id='r-e0thim3u',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-ServerRescueNegativeTestJSON-1182706601',owner_user_name='tempest-ServerRescueNegativeTestJSON-1182706601-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-11-29T03:10:27Z,user_data=None,user_id='6792e866677645c598ddb7d2c1723a1a',uuid=13e5da64-9adc-4479-81c7-d329e9b5f08f,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "2abf9e25-d1d9-4893-899e-912d41b7c38f", "address": "fa:16:3e:59:79:35", "network": {"id": "2b3c3140-f3b6-45da-8202-bb4d7f0dd130", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-2043420778-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "f8a7884867c944e1ba1024270e4902f3", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2abf9e25-d1", "ovs_interfaceid": "2abf9e25-d1d9-4893-899e-912d41b7c38f", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=75234) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Nov 29 03:10:31 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-97e2cf32-f7d5-4ab9-a932-35fa0c7cd310 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Converting VIF {"id": "2abf9e25-d1d9-4893-899e-912d41b7c38f", "address": "fa:16:3e:59:79:35", "network": {"id": "2b3c3140-f3b6-45da-8202-bb4d7f0dd130", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-2043420778-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "f8a7884867c944e1ba1024270e4902f3", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2abf9e25-d1", "ovs_interfaceid": "2abf9e25-d1d9-4893-899e-912d41b7c38f", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:10:31 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-97e2cf32-f7d5-4ab9-a932-35fa0c7cd310 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:59:79:35,bridge_name='br-int',has_traffic_filtering=True,id=2abf9e25-d1d9-4893-899e-912d41b7c38f,network=Network(2b3c3140-f3b6-45da-8202-bb4d7f0dd130),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap2abf9e25-d1') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:10:31 user nova-compute[75234]: DEBUG nova.objects.instance [None req-97e2cf32-f7d5-4ab9-a932-35fa0c7cd310 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Lazy-loading 'pci_devices' on Instance uuid 13e5da64-9adc-4479-81c7-d329e9b5f08f {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:10:31 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:10:31 user nova-compute[75234]: INFO os_vif [None req-ddae26ab-aba2-44e2-9f96-a517489c193b tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:ee:2e:82,bridge_name='br-int',has_traffic_filtering=True,id=0d131138-51b6-49f9-9a11-6b64e32e5796,network=Network(2b3c3140-f3b6-45da-8202-bb4d7f0dd130),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap0d131138-51')
Nov 29 03:10:31 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-97e2cf32-f7d5-4ab9-a932-35fa0c7cd310 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] [instance: 13e5da64-9adc-4479-81c7-d329e9b5f08f] End _get_guest_xml xml=<domain type="kvm">
Nov 29 03:10:31 user nova-compute[75234]:   <uuid>13e5da64-9adc-4479-81c7-d329e9b5f08f</uuid>
Nov 29 03:10:31 user nova-compute[75234]:   <name>instance-00000012</name>
Nov 29 03:10:31 user nova-compute[75234]:   <memory>524288</memory>
Nov 29 03:10:31 user nova-compute[75234]:   <vcpu>1</vcpu>
Nov 29 03:10:31 user nova-compute[75234]:   <metadata>
Nov 29 03:10:31 user nova-compute[75234]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Nov 29 03:10:31 user nova-compute[75234]:       <nova:package version="0.0.0"/>
Nov 29 03:10:31 user nova-compute[75234]:       <nova:name>tempest-ServerRescueNegativeTestJSON-server-715053465</nova:name>
Nov 29 03:10:31 user nova-compute[75234]:       <nova:creationTime>2023-11-29 03:10:31</nova:creationTime>
Nov 29 03:10:31 user nova-compute[75234]:       <nova:flavor name="m1.tiny">
Nov 29 03:10:31 user nova-compute[75234]:         <nova:memory>512</nova:memory>
Nov 29 03:10:31 user nova-compute[75234]:         <nova:disk>1</nova:disk>
Nov 29 03:10:31 user nova-compute[75234]:         <nova:swap>0</nova:swap>
Nov 29 03:10:31 user nova-compute[75234]:         <nova:ephemeral>0</nova:ephemeral>
Nov 29 03:10:31 user nova-compute[75234]:         <nova:vcpus>1</nova:vcpus>
Nov 29 03:10:31 user nova-compute[75234]:       </nova:flavor>
Nov 29 03:10:31 user nova-compute[75234]:       <nova:owner>
Nov 29 03:10:31 user nova-compute[75234]:         <nova:user uuid="6792e866677645c598ddb7d2c1723a1a">tempest-ServerRescueNegativeTestJSON-1182706601-project-member</nova:user>
Nov 29 03:10:31 user nova-compute[75234]:         <nova:project uuid="f8a7884867c944e1ba1024270e4902f3">tempest-ServerRescueNegativeTestJSON-1182706601</nova:project>
Nov 29 03:10:31 user nova-compute[75234]:       </nova:owner>
Nov 29 03:10:31 user nova-compute[75234]:       <nova:root type="image" uuid="96c30ba7-9b73-4a30-8ba4-7138879573b7"/>
Nov 29 03:10:31 user nova-compute[75234]:       <nova:ports>
Nov 29 03:10:31 user nova-compute[75234]:         <nova:port uuid="2abf9e25-d1d9-4893-899e-912d41b7c38f">
Nov 29 03:10:31 user nova-compute[75234]:           <nova:ip type="fixed" address="10.0.0.4" ipVersion="4"/>
Nov 29 03:10:31 user nova-compute[75234]:         </nova:port>
Nov 29 03:10:31 user nova-compute[75234]:       </nova:ports>
Nov 29 03:10:31 user nova-compute[75234]:     </nova:instance>
Nov 29 03:10:31 user nova-compute[75234]:   </metadata>
Nov 29 03:10:31 user nova-compute[75234]:   <sysinfo type="smbios">
Nov 29 03:10:31 user nova-compute[75234]:     <system>
Nov 29 03:10:31 user nova-compute[75234]:       <entry name="manufacturer">OpenStack Foundation</entry>
Nov 29 03:10:31 user nova-compute[75234]:       <entry name="product">OpenStack Nova</entry>
Nov 29 03:10:31 user nova-compute[75234]:       <entry name="version">0.0.0</entry>
Nov 29 03:10:31 user nova-compute[75234]:       <entry name="serial">13e5da64-9adc-4479-81c7-d329e9b5f08f</entry>
Nov 29 03:10:31 user nova-compute[75234]:       <entry name="uuid">13e5da64-9adc-4479-81c7-d329e9b5f08f</entry>
Nov 29 03:10:31 user nova-compute[75234]:       <entry name="family">Virtual Machine</entry>
Nov 29 03:10:31 user nova-compute[75234]:     </system>
Nov 29 03:10:31 user nova-compute[75234]:   </sysinfo>
Nov 29 03:10:31 user nova-compute[75234]:   <os>
Nov 29 03:10:31 user nova-compute[75234]:     <type arch="x86_64" machine="pc">hvm</type>
Nov 29 03:10:31 user nova-compute[75234]:     <boot dev="hd"/>
Nov 29 03:10:31 user nova-compute[75234]:     <smbios mode="sysinfo"/>
Nov 29 03:10:31 user nova-compute[75234]:   </os>
Nov 29 03:10:31 user nova-compute[75234]:   <features>
Nov 29 03:10:31 user nova-compute[75234]:     <acpi/>
Nov 29 03:10:31 user nova-compute[75234]:     <apic/>
Nov 29 03:10:31 user nova-compute[75234]:     <vmcoreinfo/>
Nov 29 03:10:31 user nova-compute[75234]:   </features>
Nov 29 03:10:31 user nova-compute[75234]:   <clock offset="utc">
Nov 29 03:10:31 user nova-compute[75234]:     <timer name="pit" tickpolicy="delay"/>
Nov 29 03:10:31 user nova-compute[75234]:     <timer name="rtc" tickpolicy="catchup"/>
Nov 29 03:10:31 user nova-compute[75234]:     <timer name="hpet" present="no"/>
Nov 29 03:10:31 user nova-compute[75234]:   </clock>
Nov 29 03:10:31 user nova-compute[75234]:   <cpu mode="custom" match="exact">
Nov 29 03:10:31 user nova-compute[75234]:     <model>Nehalem</model>
Nov 29 03:10:31 user nova-compute[75234]:     <topology sockets="1" cores="1" threads="1"/>
Nov 29 03:10:31 user nova-compute[75234]:   </cpu>
Nov 29 03:10:31 user nova-compute[75234]:   <devices>
Nov 29 03:10:31 user nova-compute[75234]:     <disk type="file" device="disk">
Nov 29 03:10:31 user nova-compute[75234]:       <driver name="qemu" type="qcow2" cache="none"/>
Nov 29 03:10:31 user nova-compute[75234]:       <source file="/opt/stack/data/nova/instances/13e5da64-9adc-4479-81c7-d329e9b5f08f/disk"/>
Nov 29 03:10:31 user nova-compute[75234]:       <target dev="vda" bus="virtio"/>
Nov 29 03:10:31 user nova-compute[75234]:     </disk>
Nov 29 03:10:31 user nova-compute[75234]:     <interface type="ethernet">
Nov 29 03:10:31 user nova-compute[75234]:       <mac address="fa:16:3e:59:79:35"/>
Nov 29 03:10:31 user nova-compute[75234]:       <model type="virtio"/>
Nov 29 03:10:31 user nova-compute[75234]:       <mtu size="1442"/>
Nov 29 03:10:31 user nova-compute[75234]:       <target dev="tap2abf9e25-d1"/>
Nov 29 03:10:31 user nova-compute[75234]:     </interface>
Nov 29 03:10:31 user nova-compute[75234]:     <serial type="pty">
Nov 29 03:10:31 user nova-compute[75234]:       <log file="/opt/stack/data/nova/instances/13e5da64-9adc-4479-81c7-d329e9b5f08f/console.log" append="off"/>
Nov 29 03:10:31 user nova-compute[75234]:     </serial>
Nov 29 03:10:31 user nova-compute[75234]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Nov 29 03:10:31 user nova-compute[75234]:     <video>
Nov 29 03:10:31 user nova-compute[75234]:       <model type="virtio"/>
Nov 29 03:10:31 user nova-compute[75234]:     </video>
Nov 29 03:10:31 user nova-compute[75234]:     <rng model="virtio">
Nov 29 03:10:31 user nova-compute[75234]:       <backend model="random">/dev/urandom</backend>
Nov 29 03:10:31 user nova-compute[75234]:     </rng>
Nov 29 03:10:31 user nova-compute[75234]:     <controller type="usb" index="0" model="none"/>
Nov 29 03:10:31 user nova-compute[75234]:     <memballoon model="virtio">
Nov 29 03:10:31 user nova-compute[75234]:       <stats period="10"/>
Nov 29 03:10:31 user nova-compute[75234]:     </memballoon>
Nov 29 03:10:31 user nova-compute[75234]:   </devices>
Nov 29 03:10:31 user nova-compute[75234]: </domain>
Nov 29 03:10:31 user nova-compute[75234]:  {{(pid=75234) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7568}}
Nov 29 03:10:31 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-97e2cf32-f7d5-4ab9-a932-35fa0c7cd310 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-11-29T03:10:25Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-ServerRescueNegativeTestJSON-server-715053465',display_name='tempest-ServerRescueNegativeTestJSON-server-715053465',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-serverrescuenegativetestjson-server-715053465',id=18,image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data=None,key_name=None,keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='f8a7884867c944e1ba1024270e4902f3',ramdisk_id='',reservation_id='r-e0thim3u',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-ServerRescueNegativeTestJSON-1182706601',owner_user_name='tempest-ServerRescueNegativeTestJSON-1182706601-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-11-29T03:10:27Z,user_data=None,user_id='6792e866677645c598ddb7d2c1723a1a',uuid=13e5da64-9adc-4479-81c7-d329e9b5f08f,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "2abf9e25-d1d9-4893-899e-912d41b7c38f", "address": "fa:16:3e:59:79:35", "network": {"id": "2b3c3140-f3b6-45da-8202-bb4d7f0dd130", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-2043420778-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "f8a7884867c944e1ba1024270e4902f3", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2abf9e25-d1", "ovs_interfaceid": "2abf9e25-d1d9-4893-899e-912d41b7c38f", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Nov 29 03:10:31 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-97e2cf32-f7d5-4ab9-a932-35fa0c7cd310 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Converting VIF {"id": "2abf9e25-d1d9-4893-899e-912d41b7c38f", "address": "fa:16:3e:59:79:35", "network": {"id": "2b3c3140-f3b6-45da-8202-bb4d7f0dd130", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-2043420778-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "f8a7884867c944e1ba1024270e4902f3", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2abf9e25-d1", "ovs_interfaceid": "2abf9e25-d1d9-4893-899e-912d41b7c38f", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:10:31 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-97e2cf32-f7d5-4ab9-a932-35fa0c7cd310 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:59:79:35,bridge_name='br-int',has_traffic_filtering=True,id=2abf9e25-d1d9-4893-899e-912d41b7c38f,network=Network(2b3c3140-f3b6-45da-8202-bb4d7f0dd130),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap2abf9e25-d1') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:10:31 user nova-compute[75234]: DEBUG os_vif [None req-97e2cf32-f7d5-4ab9-a932-35fa0c7cd310 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:59:79:35,bridge_name='br-int',has_traffic_filtering=True,id=2abf9e25-d1d9-4893-899e-912d41b7c38f,network=Network(2b3c3140-f3b6-45da-8202-bb4d7f0dd130),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap2abf9e25-d1') {{(pid=75234) plug /opt/stack/data/venv/lib/python3.10/site-packages/os_vif/__init__.py:76}}
Nov 29 03:10:31 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:10:31 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:10:31 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Nov 29 03:10:31 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:10:31 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbCreateCommand(_result=None, table=QoS, columns={'type': 'linux-noop', 'external_ids': {'id': '6fa9fef7-a74e-579d-a0ab-5e4edadbff31', '_type': 'linux-noop'}}, row=False) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:10:31 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:10:31 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:10:31 user nova-compute[75234]: DEBUG nova.compute.manager [req-2e6d9a32-e339-4bf1-a988-5ab473ea3eb1 req-c13ee422-f50a-43a8-9520-7e636546cd21 service nova] [instance: 13e5da64-9adc-4479-81c7-d329e9b5f08f] Received event network-changed-2abf9e25-d1d9-4893-899e-912d41b7c38f {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:10:31 user nova-compute[75234]: DEBUG nova.compute.manager [req-2e6d9a32-e339-4bf1-a988-5ab473ea3eb1 req-c13ee422-f50a-43a8-9520-7e636546cd21 service nova] [instance: 13e5da64-9adc-4479-81c7-d329e9b5f08f] Refreshing instance network info cache due to event network-changed-2abf9e25-d1d9-4893-899e-912d41b7c38f. {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11117}}
Nov 29 03:10:31 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-2e6d9a32-e339-4bf1-a988-5ab473ea3eb1 req-c13ee422-f50a-43a8-9520-7e636546cd21 service nova] Acquiring lock "refresh_cache-13e5da64-9adc-4479-81c7-d329e9b5f08f" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:10:31 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-2e6d9a32-e339-4bf1-a988-5ab473ea3eb1 req-c13ee422-f50a-43a8-9520-7e636546cd21 service nova] Acquired lock "refresh_cache-13e5da64-9adc-4479-81c7-d329e9b5f08f" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:10:31 user nova-compute[75234]: DEBUG nova.network.neutron [req-2e6d9a32-e339-4bf1-a988-5ab473ea3eb1 req-c13ee422-f50a-43a8-9520-7e636546cd21 service nova] [instance: 13e5da64-9adc-4479-81c7-d329e9b5f08f] Refreshing network info cache for port 2abf9e25-d1d9-4893-899e-912d41b7c38f {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2006}}
Nov 29 03:10:31 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:10:31 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:10:31 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tap2abf9e25-d1, may_exist=True, interface_attrs={}) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:10:31 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Port, record=tap2abf9e25-d1, col_values=(('qos', UUID('268ca68b-d061-4990-a3d9-a154adf42b8b')),), if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:10:31 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DbSetCommand(_result=None, table=Interface, record=tap2abf9e25-d1, col_values=(('external_ids', {'iface-id': '2abf9e25-d1d9-4893-899e-912d41b7c38f', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:59:79:35', 'vm-uuid': '13e5da64-9adc-4479-81c7-d329e9b5f08f'}),), if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:10:31 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:10:31 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:10:31 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-ddae26ab-aba2-44e2-9f96-a517489c193b tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] No BDM found with device name vda, not building metadata. {{(pid=75234) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12089}}
Nov 29 03:10:31 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-ddae26ab-aba2-44e2-9f96-a517489c193b tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] No VIF found with MAC fa:16:3e:ee:2e:82, not building metadata {{(pid=75234) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Nov 29 03:10:31 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:10:31 user nova-compute[75234]: INFO os_vif [None req-97e2cf32-f7d5-4ab9-a932-35fa0c7cd310 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:59:79:35,bridge_name='br-int',has_traffic_filtering=True,id=2abf9e25-d1d9-4893-899e-912d41b7c38f,network=Network(2b3c3140-f3b6-45da-8202-bb4d7f0dd130),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap2abf9e25-d1')
Nov 29 03:10:31 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-97e2cf32-f7d5-4ab9-a932-35fa0c7cd310 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] No BDM found with device name vda, not building metadata. {{(pid=75234) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12089}}
Nov 29 03:10:31 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-97e2cf32-f7d5-4ab9-a932-35fa0c7cd310 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] No VIF found with MAC fa:16:3e:59:79:35, not building metadata {{(pid=75234) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Nov 29 03:10:31 user nova-compute[75234]: DEBUG nova.compute.manager [req-0e2b8869-f1bd-44b8-92e5-60274446389e req-160b25d4-532e-4a90-bdd3-1afe86b236c3 service nova] [instance: a9350a18-14e6-42c6-a150-0a1487f20ad9] Received event network-vif-plugged-52024efc-8cde-4610-b12c-198f19d75ccf {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:10:31 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-0e2b8869-f1bd-44b8-92e5-60274446389e req-160b25d4-532e-4a90-bdd3-1afe86b236c3 service nova] Acquiring lock "a9350a18-14e6-42c6-a150-0a1487f20ad9-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:10:31 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-0e2b8869-f1bd-44b8-92e5-60274446389e req-160b25d4-532e-4a90-bdd3-1afe86b236c3 service nova] Lock "a9350a18-14e6-42c6-a150-0a1487f20ad9-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:10:31 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-0e2b8869-f1bd-44b8-92e5-60274446389e req-160b25d4-532e-4a90-bdd3-1afe86b236c3 service nova] Lock "a9350a18-14e6-42c6-a150-0a1487f20ad9-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:10:31 user nova-compute[75234]: DEBUG nova.compute.manager [req-0e2b8869-f1bd-44b8-92e5-60274446389e req-160b25d4-532e-4a90-bdd3-1afe86b236c3 service nova] [instance: a9350a18-14e6-42c6-a150-0a1487f20ad9] No waiting events found dispatching network-vif-plugged-52024efc-8cde-4610-b12c-198f19d75ccf {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:10:31 user nova-compute[75234]: WARNING nova.compute.manager [req-0e2b8869-f1bd-44b8-92e5-60274446389e req-160b25d4-532e-4a90-bdd3-1afe86b236c3 service nova] [instance: a9350a18-14e6-42c6-a150-0a1487f20ad9] Received unexpected event network-vif-plugged-52024efc-8cde-4610-b12c-198f19d75ccf for instance with vm_state active and task_state resize_finish.
Nov 29 03:10:31 user nova-compute[75234]: DEBUG nova.compute.manager [req-0e2b8869-f1bd-44b8-92e5-60274446389e req-160b25d4-532e-4a90-bdd3-1afe86b236c3 service nova] [instance: a082c96e-e384-40b6-bcc8-fc5e0e104669] Received event network-changed-0d131138-51b6-49f9-9a11-6b64e32e5796 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:10:31 user nova-compute[75234]: DEBUG nova.compute.manager [req-0e2b8869-f1bd-44b8-92e5-60274446389e req-160b25d4-532e-4a90-bdd3-1afe86b236c3 service nova] [instance: a082c96e-e384-40b6-bcc8-fc5e0e104669] Refreshing instance network info cache due to event network-changed-0d131138-51b6-49f9-9a11-6b64e32e5796. {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11117}}
Nov 29 03:10:31 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-0e2b8869-f1bd-44b8-92e5-60274446389e req-160b25d4-532e-4a90-bdd3-1afe86b236c3 service nova] Acquiring lock "refresh_cache-a082c96e-e384-40b6-bcc8-fc5e0e104669" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:10:31 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-0e2b8869-f1bd-44b8-92e5-60274446389e req-160b25d4-532e-4a90-bdd3-1afe86b236c3 service nova] Acquired lock "refresh_cache-a082c96e-e384-40b6-bcc8-fc5e0e104669" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:10:31 user nova-compute[75234]: DEBUG nova.network.neutron [req-0e2b8869-f1bd-44b8-92e5-60274446389e req-160b25d4-532e-4a90-bdd3-1afe86b236c3 service nova] [instance: a082c96e-e384-40b6-bcc8-fc5e0e104669] Refreshing network info cache for port 0d131138-51b6-49f9-9a11-6b64e32e5796 {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2006}}
Nov 29 03:10:32 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:10:32 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:10:32 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:10:32 user nova-compute[75234]: DEBUG nova.compute.manager [None req-221919a0-9cdd-44ec-881f-d6c82383d31d tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] [instance: a9350a18-14e6-42c6-a150-0a1487f20ad9] Instance event wait completed in 0 seconds for  {{(pid=75234) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Nov 29 03:10:32 user nova-compute[75234]: INFO nova.virt.libvirt.driver [-] [instance: a9350a18-14e6-42c6-a150-0a1487f20ad9] Instance running successfully.
Nov 29 03:10:32 user nova-compute[75234]: DEBUG nova.virt.libvirt.guest [None req-221919a0-9cdd-44ec-881f-d6c82383d31d tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] [instance: a9350a18-14e6-42c6-a150-0a1487f20ad9] Failed to set time: agent not configured {{(pid=75234) sync_guest_time /opt/stack/nova/nova/virt/libvirt/guest.py:200}}
Nov 29 03:10:32 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-221919a0-9cdd-44ec-881f-d6c82383d31d tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] [instance: a9350a18-14e6-42c6-a150-0a1487f20ad9] finish_migration finished successfully. {{(pid=75234) finish_migration /opt/stack/nova/nova/virt/libvirt/driver.py:11766}}
Nov 29 03:10:32 user nova-compute[75234]: DEBUG nova.network.neutron [req-2e6d9a32-e339-4bf1-a988-5ab473ea3eb1 req-c13ee422-f50a-43a8-9520-7e636546cd21 service nova] [instance: 13e5da64-9adc-4479-81c7-d329e9b5f08f] Updated VIF entry in instance network info cache for port 2abf9e25-d1d9-4893-899e-912d41b7c38f. {{(pid=75234) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3481}}
Nov 29 03:10:32 user nova-compute[75234]: DEBUG nova.network.neutron [req-2e6d9a32-e339-4bf1-a988-5ab473ea3eb1 req-c13ee422-f50a-43a8-9520-7e636546cd21 service nova] [instance: 13e5da64-9adc-4479-81c7-d329e9b5f08f] Updating instance_info_cache with network_info: [{"id": "2abf9e25-d1d9-4893-899e-912d41b7c38f", "address": "fa:16:3e:59:79:35", "network": {"id": "2b3c3140-f3b6-45da-8202-bb4d7f0dd130", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-2043420778-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "f8a7884867c944e1ba1024270e4902f3", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2abf9e25-d1", "ovs_interfaceid": "2abf9e25-d1d9-4893-899e-912d41b7c38f", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:10:32 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-2e6d9a32-e339-4bf1-a988-5ab473ea3eb1 req-c13ee422-f50a-43a8-9520-7e636546cd21 service nova] Releasing lock "refresh_cache-13e5da64-9adc-4479-81c7-d329e9b5f08f" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:10:32 user nova-compute[75234]: DEBUG nova.network.neutron [req-0e2b8869-f1bd-44b8-92e5-60274446389e req-160b25d4-532e-4a90-bdd3-1afe86b236c3 service nova] [instance: a082c96e-e384-40b6-bcc8-fc5e0e104669] Updated VIF entry in instance network info cache for port 0d131138-51b6-49f9-9a11-6b64e32e5796. {{(pid=75234) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3481}}
Nov 29 03:10:32 user nova-compute[75234]: DEBUG nova.network.neutron [req-0e2b8869-f1bd-44b8-92e5-60274446389e req-160b25d4-532e-4a90-bdd3-1afe86b236c3 service nova] [instance: a082c96e-e384-40b6-bcc8-fc5e0e104669] Updating instance_info_cache with network_info: [{"id": "0d131138-51b6-49f9-9a11-6b64e32e5796", "address": "fa:16:3e:ee:2e:82", "network": {"id": "2b3c3140-f3b6-45da-8202-bb4d7f0dd130", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-2043420778-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "f8a7884867c944e1ba1024270e4902f3", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap0d131138-51", "ovs_interfaceid": "0d131138-51b6-49f9-9a11-6b64e32e5796", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:10:32 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-0e2b8869-f1bd-44b8-92e5-60274446389e req-160b25d4-532e-4a90-bdd3-1afe86b236c3 service nova] Releasing lock "refresh_cache-a082c96e-e384-40b6-bcc8-fc5e0e104669" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:10:32 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:10:32 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:10:32 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:10:32 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:10:32 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:10:33 user nova-compute[75234]: DEBUG nova.compute.manager [req-e01e0aa1-9108-48f0-a583-fa7695efa27b req-1155c564-8e18-4a7c-bf4a-9e7d17bbf943 service nova] [instance: d3b90f39-4b91-4152-b820-bd0e263eec89] Received event network-vif-plugged-83eb52bb-5be5-49d6-81ec-86a21dde3df2 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:10:33 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-e01e0aa1-9108-48f0-a583-fa7695efa27b req-1155c564-8e18-4a7c-bf4a-9e7d17bbf943 service nova] Acquiring lock "d3b90f39-4b91-4152-b820-bd0e263eec89-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:10:33 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-e01e0aa1-9108-48f0-a583-fa7695efa27b req-1155c564-8e18-4a7c-bf4a-9e7d17bbf943 service nova] Lock "d3b90f39-4b91-4152-b820-bd0e263eec89-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:10:33 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-e01e0aa1-9108-48f0-a583-fa7695efa27b req-1155c564-8e18-4a7c-bf4a-9e7d17bbf943 service nova] Lock "d3b90f39-4b91-4152-b820-bd0e263eec89-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:10:33 user nova-compute[75234]: DEBUG nova.compute.manager [req-e01e0aa1-9108-48f0-a583-fa7695efa27b req-1155c564-8e18-4a7c-bf4a-9e7d17bbf943 service nova] [instance: d3b90f39-4b91-4152-b820-bd0e263eec89] No waiting events found dispatching network-vif-plugged-83eb52bb-5be5-49d6-81ec-86a21dde3df2 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:10:34 user nova-compute[75234]: WARNING nova.compute.manager [req-e01e0aa1-9108-48f0-a583-fa7695efa27b req-1155c564-8e18-4a7c-bf4a-9e7d17bbf943 service nova] [instance: d3b90f39-4b91-4152-b820-bd0e263eec89] Received unexpected event network-vif-plugged-83eb52bb-5be5-49d6-81ec-86a21dde3df2 for instance with vm_state building and task_state spawning.
Nov 29 03:10:34 user nova-compute[75234]: DEBUG nova.compute.manager [req-e01e0aa1-9108-48f0-a583-fa7695efa27b req-1155c564-8e18-4a7c-bf4a-9e7d17bbf943 service nova] [instance: d3b90f39-4b91-4152-b820-bd0e263eec89] Received event network-vif-plugged-83eb52bb-5be5-49d6-81ec-86a21dde3df2 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:10:34 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-e01e0aa1-9108-48f0-a583-fa7695efa27b req-1155c564-8e18-4a7c-bf4a-9e7d17bbf943 service nova] Acquiring lock "d3b90f39-4b91-4152-b820-bd0e263eec89-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:10:34 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-e01e0aa1-9108-48f0-a583-fa7695efa27b req-1155c564-8e18-4a7c-bf4a-9e7d17bbf943 service nova] Lock "d3b90f39-4b91-4152-b820-bd0e263eec89-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:10:34 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-e01e0aa1-9108-48f0-a583-fa7695efa27b req-1155c564-8e18-4a7c-bf4a-9e7d17bbf943 service nova] Lock "d3b90f39-4b91-4152-b820-bd0e263eec89-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:10:34 user nova-compute[75234]: DEBUG nova.compute.manager [req-e01e0aa1-9108-48f0-a583-fa7695efa27b req-1155c564-8e18-4a7c-bf4a-9e7d17bbf943 service nova] [instance: d3b90f39-4b91-4152-b820-bd0e263eec89] No waiting events found dispatching network-vif-plugged-83eb52bb-5be5-49d6-81ec-86a21dde3df2 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:10:34 user nova-compute[75234]: WARNING nova.compute.manager [req-e01e0aa1-9108-48f0-a583-fa7695efa27b req-1155c564-8e18-4a7c-bf4a-9e7d17bbf943 service nova] [instance: d3b90f39-4b91-4152-b820-bd0e263eec89] Received unexpected event network-vif-plugged-83eb52bb-5be5-49d6-81ec-86a21dde3df2 for instance with vm_state building and task_state spawning.
Nov 29 03:10:34 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:10:35 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:10:35 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:10:35 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:10:35 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:10:35 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:10:35 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:10:35 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:10:35 user nova-compute[75234]: DEBUG nova.compute.manager [req-8400a474-9e7d-40f4-9b88-59a7ca3f127d req-2e36d956-c9be-4a36-b728-d9477bc8ea11 service nova] [instance: 13e5da64-9adc-4479-81c7-d329e9b5f08f] Received event network-vif-plugged-2abf9e25-d1d9-4893-899e-912d41b7c38f {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:10:35 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-8400a474-9e7d-40f4-9b88-59a7ca3f127d req-2e36d956-c9be-4a36-b728-d9477bc8ea11 service nova] Acquiring lock "13e5da64-9adc-4479-81c7-d329e9b5f08f-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:10:35 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-8400a474-9e7d-40f4-9b88-59a7ca3f127d req-2e36d956-c9be-4a36-b728-d9477bc8ea11 service nova] Lock "13e5da64-9adc-4479-81c7-d329e9b5f08f-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:10:35 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-8400a474-9e7d-40f4-9b88-59a7ca3f127d req-2e36d956-c9be-4a36-b728-d9477bc8ea11 service nova] Lock "13e5da64-9adc-4479-81c7-d329e9b5f08f-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:10:35 user nova-compute[75234]: DEBUG nova.compute.manager [req-8400a474-9e7d-40f4-9b88-59a7ca3f127d req-2e36d956-c9be-4a36-b728-d9477bc8ea11 service nova] [instance: 13e5da64-9adc-4479-81c7-d329e9b5f08f] No waiting events found dispatching network-vif-plugged-2abf9e25-d1d9-4893-899e-912d41b7c38f {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:10:35 user nova-compute[75234]: WARNING nova.compute.manager [req-8400a474-9e7d-40f4-9b88-59a7ca3f127d req-2e36d956-c9be-4a36-b728-d9477bc8ea11 service nova] [instance: 13e5da64-9adc-4479-81c7-d329e9b5f08f] Received unexpected event network-vif-plugged-2abf9e25-d1d9-4893-899e-912d41b7c38f for instance with vm_state building and task_state spawning.
Nov 29 03:10:35 user nova-compute[75234]: DEBUG nova.compute.manager [req-b7920b0f-dbfa-4699-9d80-63809cc24a96 req-6fad1cda-8876-4912-a7a4-e7942547b20e service nova] [instance: a082c96e-e384-40b6-bcc8-fc5e0e104669] Received event network-vif-plugged-0d131138-51b6-49f9-9a11-6b64e32e5796 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:10:35 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-b7920b0f-dbfa-4699-9d80-63809cc24a96 req-6fad1cda-8876-4912-a7a4-e7942547b20e service nova] Acquiring lock "a082c96e-e384-40b6-bcc8-fc5e0e104669-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:10:35 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-b7920b0f-dbfa-4699-9d80-63809cc24a96 req-6fad1cda-8876-4912-a7a4-e7942547b20e service nova] Lock "a082c96e-e384-40b6-bcc8-fc5e0e104669-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:10:35 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-b7920b0f-dbfa-4699-9d80-63809cc24a96 req-6fad1cda-8876-4912-a7a4-e7942547b20e service nova] Lock "a082c96e-e384-40b6-bcc8-fc5e0e104669-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:10:35 user nova-compute[75234]: DEBUG nova.compute.manager [req-b7920b0f-dbfa-4699-9d80-63809cc24a96 req-6fad1cda-8876-4912-a7a4-e7942547b20e service nova] [instance: a082c96e-e384-40b6-bcc8-fc5e0e104669] No waiting events found dispatching network-vif-plugged-0d131138-51b6-49f9-9a11-6b64e32e5796 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:10:35 user nova-compute[75234]: WARNING nova.compute.manager [req-b7920b0f-dbfa-4699-9d80-63809cc24a96 req-6fad1cda-8876-4912-a7a4-e7942547b20e service nova] [instance: a082c96e-e384-40b6-bcc8-fc5e0e104669] Received unexpected event network-vif-plugged-0d131138-51b6-49f9-9a11-6b64e32e5796 for instance with vm_state building and task_state spawning.
Nov 29 03:10:36 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:10:36 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:10:36 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:10:36 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:10:36 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:10:36 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:10:36 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:10:36 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:10:37 user nova-compute[75234]: DEBUG nova.compute.manager [None req-046208e3-2011-4422-97ac-ba2e4f78cd46 tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] [instance: d3b90f39-4b91-4152-b820-bd0e263eec89] Instance event wait completed in 0 seconds for  {{(pid=75234) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Nov 29 03:10:37 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-046208e3-2011-4422-97ac-ba2e4f78cd46 tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] [instance: d3b90f39-4b91-4152-b820-bd0e263eec89] Guest created on hypervisor {{(pid=75234) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4421}}
Nov 29 03:10:37 user nova-compute[75234]: INFO nova.virt.libvirt.driver [-] [instance: d3b90f39-4b91-4152-b820-bd0e263eec89] Instance spawned successfully.
Nov 29 03:10:37 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-046208e3-2011-4422-97ac-ba2e4f78cd46 tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] [instance: d3b90f39-4b91-4152-b820-bd0e263eec89] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:904}}
Nov 29 03:10:37 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-046208e3-2011-4422-97ac-ba2e4f78cd46 tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] [instance: d3b90f39-4b91-4152-b820-bd0e263eec89] Found default for hw_cdrom_bus of ide {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:10:37 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-046208e3-2011-4422-97ac-ba2e4f78cd46 tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] [instance: d3b90f39-4b91-4152-b820-bd0e263eec89] Found default for hw_disk_bus of virtio {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:10:37 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-046208e3-2011-4422-97ac-ba2e4f78cd46 tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] [instance: d3b90f39-4b91-4152-b820-bd0e263eec89] Found default for hw_input_bus of None {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:10:37 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-046208e3-2011-4422-97ac-ba2e4f78cd46 tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] [instance: d3b90f39-4b91-4152-b820-bd0e263eec89] Found default for hw_pointer_model of None {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:10:37 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-046208e3-2011-4422-97ac-ba2e4f78cd46 tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] [instance: d3b90f39-4b91-4152-b820-bd0e263eec89] Found default for hw_video_model of virtio {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:10:37 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-046208e3-2011-4422-97ac-ba2e4f78cd46 tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] [instance: d3b90f39-4b91-4152-b820-bd0e263eec89] Found default for hw_vif_model of virtio {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:10:37 user nova-compute[75234]: INFO nova.compute.manager [None req-046208e3-2011-4422-97ac-ba2e4f78cd46 tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] [instance: d3b90f39-4b91-4152-b820-bd0e263eec89] Took 10.89 seconds to spawn the instance on the hypervisor.
Nov 29 03:10:37 user nova-compute[75234]: DEBUG nova.compute.manager [None req-046208e3-2011-4422-97ac-ba2e4f78cd46 tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] [instance: d3b90f39-4b91-4152-b820-bd0e263eec89] Checking state {{(pid=75234) _get_power_state /opt/stack/nova/nova/compute/manager.py:1782}}
Nov 29 03:10:37 user nova-compute[75234]: INFO nova.compute.manager [None req-046208e3-2011-4422-97ac-ba2e4f78cd46 tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] [instance: d3b90f39-4b91-4152-b820-bd0e263eec89] Took 12.14 seconds to build instance.
Nov 29 03:10:37 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-046208e3-2011-4422-97ac-ba2e4f78cd46 tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] Lock "d3b90f39-4b91-4152-b820-bd0e263eec89" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 12.384s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:10:37 user nova-compute[75234]: DEBUG nova.compute.manager [req-20f94c8c-f02e-41cc-af22-cee5ab1d2573 req-48ae6d9e-41b7-4324-a001-b9c28f89327f service nova] [instance: 13e5da64-9adc-4479-81c7-d329e9b5f08f] Received event network-vif-plugged-2abf9e25-d1d9-4893-899e-912d41b7c38f {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:10:37 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-20f94c8c-f02e-41cc-af22-cee5ab1d2573 req-48ae6d9e-41b7-4324-a001-b9c28f89327f service nova] Acquiring lock "13e5da64-9adc-4479-81c7-d329e9b5f08f-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:10:37 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-20f94c8c-f02e-41cc-af22-cee5ab1d2573 req-48ae6d9e-41b7-4324-a001-b9c28f89327f service nova] Lock "13e5da64-9adc-4479-81c7-d329e9b5f08f-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:10:37 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-20f94c8c-f02e-41cc-af22-cee5ab1d2573 req-48ae6d9e-41b7-4324-a001-b9c28f89327f service nova] Lock "13e5da64-9adc-4479-81c7-d329e9b5f08f-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:10:37 user nova-compute[75234]: DEBUG nova.compute.manager [req-20f94c8c-f02e-41cc-af22-cee5ab1d2573 req-48ae6d9e-41b7-4324-a001-b9c28f89327f service nova] [instance: 13e5da64-9adc-4479-81c7-d329e9b5f08f] No waiting events found dispatching network-vif-plugged-2abf9e25-d1d9-4893-899e-912d41b7c38f {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:10:37 user nova-compute[75234]: WARNING nova.compute.manager [req-20f94c8c-f02e-41cc-af22-cee5ab1d2573 req-48ae6d9e-41b7-4324-a001-b9c28f89327f service nova] [instance: 13e5da64-9adc-4479-81c7-d329e9b5f08f] Received unexpected event network-vif-plugged-2abf9e25-d1d9-4893-899e-912d41b7c38f for instance with vm_state building and task_state spawning.
Nov 29 03:10:37 user nova-compute[75234]: DEBUG nova.compute.manager [req-e3cc0b0c-e3c4-4cd7-b407-e3f942c5633c req-3dfe521b-aa56-4bc2-ba98-9b429a638be2 service nova] [instance: a082c96e-e384-40b6-bcc8-fc5e0e104669] Received event network-vif-plugged-0d131138-51b6-49f9-9a11-6b64e32e5796 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:10:37 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-e3cc0b0c-e3c4-4cd7-b407-e3f942c5633c req-3dfe521b-aa56-4bc2-ba98-9b429a638be2 service nova] Acquiring lock "a082c96e-e384-40b6-bcc8-fc5e0e104669-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:10:37 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-e3cc0b0c-e3c4-4cd7-b407-e3f942c5633c req-3dfe521b-aa56-4bc2-ba98-9b429a638be2 service nova] Lock "a082c96e-e384-40b6-bcc8-fc5e0e104669-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:10:37 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-e3cc0b0c-e3c4-4cd7-b407-e3f942c5633c req-3dfe521b-aa56-4bc2-ba98-9b429a638be2 service nova] Lock "a082c96e-e384-40b6-bcc8-fc5e0e104669-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:10:37 user nova-compute[75234]: DEBUG nova.compute.manager [req-e3cc0b0c-e3c4-4cd7-b407-e3f942c5633c req-3dfe521b-aa56-4bc2-ba98-9b429a638be2 service nova] [instance: a082c96e-e384-40b6-bcc8-fc5e0e104669] No waiting events found dispatching network-vif-plugged-0d131138-51b6-49f9-9a11-6b64e32e5796 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:10:37 user nova-compute[75234]: WARNING nova.compute.manager [req-e3cc0b0c-e3c4-4cd7-b407-e3f942c5633c req-3dfe521b-aa56-4bc2-ba98-9b429a638be2 service nova] [instance: a082c96e-e384-40b6-bcc8-fc5e0e104669] Received unexpected event network-vif-plugged-0d131138-51b6-49f9-9a11-6b64e32e5796 for instance with vm_state building and task_state spawning.
Nov 29 03:10:38 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:10:38 user nova-compute[75234]: DEBUG nova.compute.manager [None req-97e2cf32-f7d5-4ab9-a932-35fa0c7cd310 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] [instance: 13e5da64-9adc-4479-81c7-d329e9b5f08f] Instance event wait completed in 0 seconds for  {{(pid=75234) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Nov 29 03:10:38 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-97e2cf32-f7d5-4ab9-a932-35fa0c7cd310 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] [instance: 13e5da64-9adc-4479-81c7-d329e9b5f08f] Guest created on hypervisor {{(pid=75234) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4421}}
Nov 29 03:10:38 user nova-compute[75234]: INFO nova.virt.libvirt.driver [-] [instance: 13e5da64-9adc-4479-81c7-d329e9b5f08f] Instance spawned successfully.
Nov 29 03:10:38 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-97e2cf32-f7d5-4ab9-a932-35fa0c7cd310 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] [instance: 13e5da64-9adc-4479-81c7-d329e9b5f08f] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:904}}
Nov 29 03:10:38 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ddae26ab-aba2-44e2-9f96-a517489c193b tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] [instance: a082c96e-e384-40b6-bcc8-fc5e0e104669] Instance event wait completed in 0 seconds for  {{(pid=75234) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Nov 29 03:10:38 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-ddae26ab-aba2-44e2-9f96-a517489c193b tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] [instance: a082c96e-e384-40b6-bcc8-fc5e0e104669] Guest created on hypervisor {{(pid=75234) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4421}}
Nov 29 03:10:38 user nova-compute[75234]: INFO nova.virt.libvirt.driver [-] [instance: a082c96e-e384-40b6-bcc8-fc5e0e104669] Instance spawned successfully.
Nov 29 03:10:38 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-ddae26ab-aba2-44e2-9f96-a517489c193b tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] [instance: a082c96e-e384-40b6-bcc8-fc5e0e104669] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:904}}
Nov 29 03:10:38 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-97e2cf32-f7d5-4ab9-a932-35fa0c7cd310 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] [instance: 13e5da64-9adc-4479-81c7-d329e9b5f08f] Found default for hw_cdrom_bus of ide {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:10:38 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-97e2cf32-f7d5-4ab9-a932-35fa0c7cd310 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] [instance: 13e5da64-9adc-4479-81c7-d329e9b5f08f] Found default for hw_disk_bus of virtio {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:10:38 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-97e2cf32-f7d5-4ab9-a932-35fa0c7cd310 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] [instance: 13e5da64-9adc-4479-81c7-d329e9b5f08f] Found default for hw_input_bus of None {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:10:38 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-97e2cf32-f7d5-4ab9-a932-35fa0c7cd310 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] [instance: 13e5da64-9adc-4479-81c7-d329e9b5f08f] Found default for hw_pointer_model of None {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:10:38 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-97e2cf32-f7d5-4ab9-a932-35fa0c7cd310 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] [instance: 13e5da64-9adc-4479-81c7-d329e9b5f08f] Found default for hw_video_model of virtio {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:10:38 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-97e2cf32-f7d5-4ab9-a932-35fa0c7cd310 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] [instance: 13e5da64-9adc-4479-81c7-d329e9b5f08f] Found default for hw_vif_model of virtio {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:10:38 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-ddae26ab-aba2-44e2-9f96-a517489c193b tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] [instance: a082c96e-e384-40b6-bcc8-fc5e0e104669] Found default for hw_cdrom_bus of ide {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:10:38 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-ddae26ab-aba2-44e2-9f96-a517489c193b tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] [instance: a082c96e-e384-40b6-bcc8-fc5e0e104669] Found default for hw_disk_bus of virtio {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:10:38 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-ddae26ab-aba2-44e2-9f96-a517489c193b tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] [instance: a082c96e-e384-40b6-bcc8-fc5e0e104669] Found default for hw_input_bus of None {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:10:38 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-ddae26ab-aba2-44e2-9f96-a517489c193b tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] [instance: a082c96e-e384-40b6-bcc8-fc5e0e104669] Found default for hw_pointer_model of None {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:10:38 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-ddae26ab-aba2-44e2-9f96-a517489c193b tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] [instance: a082c96e-e384-40b6-bcc8-fc5e0e104669] Found default for hw_video_model of virtio {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:10:38 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-ddae26ab-aba2-44e2-9f96-a517489c193b tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] [instance: a082c96e-e384-40b6-bcc8-fc5e0e104669] Found default for hw_vif_model of virtio {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:10:38 user nova-compute[75234]: INFO nova.compute.manager [None req-97e2cf32-f7d5-4ab9-a932-35fa0c7cd310 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] [instance: 13e5da64-9adc-4479-81c7-d329e9b5f08f] Took 11.07 seconds to spawn the instance on the hypervisor.
Nov 29 03:10:38 user nova-compute[75234]: DEBUG nova.compute.manager [None req-97e2cf32-f7d5-4ab9-a932-35fa0c7cd310 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] [instance: 13e5da64-9adc-4479-81c7-d329e9b5f08f] Checking state {{(pid=75234) _get_power_state /opt/stack/nova/nova/compute/manager.py:1782}}
Nov 29 03:10:38 user nova-compute[75234]: INFO nova.compute.manager [None req-ddae26ab-aba2-44e2-9f96-a517489c193b tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] [instance: a082c96e-e384-40b6-bcc8-fc5e0e104669] Took 9.95 seconds to spawn the instance on the hypervisor.
Nov 29 03:10:38 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ddae26ab-aba2-44e2-9f96-a517489c193b tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] [instance: a082c96e-e384-40b6-bcc8-fc5e0e104669] Checking state {{(pid=75234) _get_power_state /opt/stack/nova/nova/compute/manager.py:1782}}
Nov 29 03:10:38 user nova-compute[75234]: INFO nova.compute.manager [None req-97e2cf32-f7d5-4ab9-a932-35fa0c7cd310 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] [instance: 13e5da64-9adc-4479-81c7-d329e9b5f08f] Took 12.43 seconds to build instance.
Nov 29 03:10:38 user nova-compute[75234]: INFO nova.compute.manager [None req-ddae26ab-aba2-44e2-9f96-a517489c193b tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] [instance: a082c96e-e384-40b6-bcc8-fc5e0e104669] Took 11.47 seconds to build instance.
Nov 29 03:10:38 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-97e2cf32-f7d5-4ab9-a932-35fa0c7cd310 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Lock "13e5da64-9adc-4479-81c7-d329e9b5f08f" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 12.598s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:10:38 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ddae26ab-aba2-44e2-9f96-a517489c193b tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Lock "a082c96e-e384-40b6-bcc8-fc5e0e104669" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 11.669s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:10:41 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:10:41 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:10:46 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:10:46 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:10:47 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:10:50 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-ef90f75a-4af4-4a58-bb0e-eaf1f32e9b1c tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] Waiting for libvirt event about the detach of device vdb with device alias virtio-disk1 from instance f7d0bdeb-6a05-47ac-87f2-c8380a9a4286 is timed out.
Nov 29 03:10:50 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-ef90f75a-4af4-4a58-bb0e-eaf1f32e9b1c tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] Successfully detached device vdb from instance f7d0bdeb-6a05-47ac-87f2-c8380a9a4286 from the live domain config.
Nov 29 03:10:50 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-ef90f75a-4af4-4a58-bb0e-eaf1f32e9b1c tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] [instance: f7d0bdeb-6a05-47ac-87f2-c8380a9a4286] calling os-brick to detach iSCSI Volume {{(pid=75234) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Nov 29 03:10:50 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-ef90f75a-4af4-4a58-bb0e-eaf1f32e9b1c tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f8a5050c9a0>, {'target_iqn': 'iqn.2010-10.org.openstack:ada91c26-ca97-4b8b-98bb-a310d0627691', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'pbUzSvGT', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdb'}, None), 'kwargs': False}}" {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:176}}
Nov 29 03:10:50 user nova-compute[75234]: WARNING os_brick.initiator.connectors.base [None req-ef90f75a-4af4-4a58-bb0e-eaf1f32e9b1c tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Nov 29 03:10:50 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-ef90f75a-4af4-4a58-bb0e-eaf1f32e9b1c tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:68}}
Nov 29 03:10:50 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-ef90f75a-4af4-4a58-bb0e-eaf1f32e9b1c tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:73}}
Nov 29 03:10:50 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:10:50 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.028s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:10:50 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[410ca81d-1258-46a8-b119-caa0197a52c1]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:a7ef35bf-017a-4fd9-9182-79c1b2ab4f98\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:ada91c26-ca97-4b8b-98bb-a310d0627691\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:10:50 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-ef90f75a-4af4-4a58-bb0e-eaf1f32e9b1c tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS:
Nov 29 03:10:50 user nova-compute[75234]: No targets found.
Nov 29 03:10:50 user nova-compute[75234]: iSNS:
Nov 29 03:10:50 user nova-compute[75234]: No targets found.
Nov 29 03:10:50 user nova-compute[75234]: STATIC:
Nov 29 03:10:50 user nova-compute[75234]: Target: iqn.2010-10.org.openstack:a7ef35bf-017a-4fd9-9182-79c1b2ab4f98
Nov 29 03:10:50 user nova-compute[75234]:         Portal: 172.16.0.220:3260,-1
Nov 29 03:10:50 user nova-compute[75234]:                 Iface Name: default
Nov 29 03:10:50 user nova-compute[75234]: Target: iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672
Nov 29 03:10:50 user nova-compute[75234]:         Portal: 172.16.0.220:3260,-1
Nov 29 03:10:50 user nova-compute[75234]:                 Iface Name: default
Nov 29 03:10:50 user nova-compute[75234]: Target: iqn.2010-10.org.openstack:ada91c26-ca97-4b8b-98bb-a310d0627691
Nov 29 03:10:50 user nova-compute[75234]:         Portal: 172.16.0.220:3260,-1
Nov 29 03:10:50 user nova-compute[75234]:                 Iface Name: default
Nov 29 03:10:50 user nova-compute[75234]: Target: iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b
Nov 29 03:10:50 user nova-compute[75234]:         Portal: 172.16.0.220:3260,-1
Nov 29 03:10:50 user nova-compute[75234]:                 Iface Name: default
Nov 29 03:10:50 user nova-compute[75234]: FIRMWARE:
Nov 29 03:10:50 user nova-compute[75234]: No targets found.
Nov 29 03:10:50 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:10:50 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-ef90f75a-4af4-4a58-bb0e-eaf1f32e9b1c tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] Regex to get portals from discoverydb: ^SENDTARGETS:
Nov 29 03:10:50 user nova-compute[75234]: .*?^DiscoveryAddress: 172.16.0.220,3260.*?
Nov 29 03:10:50 user nova-compute[75234]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=75234) _get_discoverydb_portals /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:382}}
Nov 29 03:10:50 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-ef90f75a-4af4-4a58-bb0e-eaf1f32e9b1c tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:ada91c26-ca97-4b8b-98bb-a310d0627691', 0)] {{(pid=75234) _get_connection_devices /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:830}}
Nov 29 03:10:50 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:10:50 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.017s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:10:50 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[44abaec6-bd3d-4bac-8b42-f423c9369004]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:a7ef35bf-017a-4fd9-9182-79c1b2ab4f98\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:ada91c26-ca97-4b8b-98bb-a310d0627691\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:10:50 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:10:50 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.030s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:10:50 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[51b0b4fe-acc2-43bf-a884-26ed0c36dd64]: (4, ('tcp: [12] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ada91c26-ca97-4b8b-98bb-a310d0627691 (non-flash)\ntcp: [15] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a7ef35bf-017a-4fd9-9182-79c1b2ab4f98 (non-flash)\ntcp: [16] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672 (non-flash)\ntcp: [17] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b (non-flash)\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:10:50 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-ef90f75a-4af4-4a58-bb0e-eaf1f32e9b1c tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [12] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ada91c26-ca97-4b8b-98bb-a310d0627691 (non-flash)
Nov 29 03:10:50 user nova-compute[75234]: tcp: [15] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a7ef35bf-017a-4fd9-9182-79c1b2ab4f98 (non-flash)
Nov 29 03:10:50 user nova-compute[75234]: tcp: [16] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672 (non-flash)
Nov 29 03:10:50 user nova-compute[75234]: tcp: [17] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b (non-flash)
Nov 29 03:10:50 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:10:50 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-ef90f75a-4af4-4a58-bb0e-eaf1f32e9b1c tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] iscsi session list stdout=tcp: [12] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ada91c26-ca97-4b8b-98bb-a310d0627691 (non-flash)
Nov 29 03:10:50 user nova-compute[75234]: tcp: [15] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a7ef35bf-017a-4fd9-9182-79c1b2ab4f98 (non-flash)
Nov 29 03:10:50 user nova-compute[75234]: tcp: [16] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672 (non-flash)
Nov 29 03:10:50 user nova-compute[75234]: tcp: [17] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b (non-flash)
Nov 29 03:10:50 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsi_session /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Nov 29 03:10:50 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-ef90f75a-4af4-4a58-bb0e-eaf1f32e9b1c tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7f8a50163d90>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:ada91c26-ca97-4b8b-98bb-a310d0627691'): ({'sdb'}, set())}) {{(pid=75234) _get_connection_devices /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:863}}
Nov 29 03:10:50 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-ef90f75a-4af4-4a58-bb0e-eaf1f32e9b1c tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] Removing single pathed devices sdb {{(pid=75234) remove_connection /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:374}}
Nov 29 03:10:50 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:10:50 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.028s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:10:50 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[ead63a72-f83b-463f-b6ac-202092849c74]: (4, ('path checker states:\nup                  4\n\npaths: 0\nbusy: False\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:10:50 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdb {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:10:50 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdb" returned: 0 in 0.036s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:10:50 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[8c04564b-db0e-425b-8ea4-059e42a15bf8]: (4, ('ok\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:10:50 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-ef90f75a-4af4-4a58-bb0e-eaf1f32e9b1c tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] Flushing IO for device /dev/sdb {{(pid=75234) flush_device_io /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:434}}
Nov 29 03:10:50 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdb {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:10:50 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdb" returned: 0 in 0.014s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:10:50 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[d368022a-72bb-4640-b264-dab43f089a29]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:10:50 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-ef90f75a-4af4-4a58-bb0e-eaf1f32e9b1c tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] Remove SCSI device /dev/sdb with /sys/block/sdb/device/delete {{(pid=75234) remove_scsi_device /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:148}}
Nov 29 03:10:50 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdb/device/delete {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:10:50 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdb/device/delete" returned: 0 in 0.066s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:10:50 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[91dbcfba-4c0d-4599-9aed-a7c164992888]: (4, ('1', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:10:50 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-ef90f75a-4af4-4a58-bb0e-eaf1f32e9b1c tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] Checking to see if SCSI volumes sdb have been removed. {{(pid=75234) wait_for_volumes_removal /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:156}}
Nov 29 03:10:50 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-ef90f75a-4af4-4a58-bb0e-eaf1f32e9b1c tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] SCSI volumes sdb have been removed. {{(pid=75234) wait_for_volumes_removal /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:166}}
Nov 29 03:10:50 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-ef90f75a-4af4-4a58-bb0e-eaf1f32e9b1c tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:ada91c26-ca97-4b8b-98bb-a310d0627691')] {{(pid=75234) _disconnect_connection /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Nov 29 03:10:50 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:ada91c26-ca97-4b8b-98bb-a310d0627691 -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:10:50 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:ada91c26-ca97-4b8b-98bb-a310d0627691 -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.028s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:10:50 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[39adc1b9-45af-401e-94f7-f866da1d94f2]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:10:50 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-ef90f75a-4af4-4a58-bb0e-eaf1f32e9b1c tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:10:50 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:ada91c26-ca97-4b8b-98bb-a310d0627691 -p 172.16.0.220:3260 --logout {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:10:50 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:ada91c26-ca97-4b8b-98bb-a310d0627691 -p 172.16.0.220:3260 --logout" returned: 0 in 0.061s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:10:50 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[0384d886-f6b4-442a-9ed8-3a1a2946201d]: (4, ('Logging out of session [sid: 12, target: iqn.2010-10.org.openstack:ada91c26-ca97-4b8b-98bb-a310d0627691, portal: 172.16.0.220,3260]\nLogout of [sid: 12, target: iqn.2010-10.org.openstack:ada91c26-ca97-4b8b-98bb-a310d0627691, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:10:50 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-ef90f75a-4af4-4a58-bb0e-eaf1f32e9b1c tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 12, target: iqn.2010-10.org.openstack:ada91c26-ca97-4b8b-98bb-a310d0627691, portal: 172.16.0.220,3260]
Nov 29 03:10:50 user nova-compute[75234]: Logout of [sid: 12, target: iqn.2010-10.org.openstack:ada91c26-ca97-4b8b-98bb-a310d0627691, portal: 172.16.0.220,3260] successful.
Nov 29 03:10:50 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:10:50 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:ada91c26-ca97-4b8b-98bb-a310d0627691 -p 172.16.0.220:3260 --op delete {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:10:50 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:ada91c26-ca97-4b8b-98bb-a310d0627691 -p 172.16.0.220:3260 --op delete" returned: 0 in 0.035s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:10:50 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[8b3cb4e3-f55a-4ac5-88bf-4a88c817b09c]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:10:50 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-ef90f75a-4af4-4a58-bb0e-eaf1f32e9b1c tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:10:50 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-ef90f75a-4af4-4a58-bb0e-eaf1f32e9b1c tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.408s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:87}}
Nov 29 03:10:50 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-ef90f75a-4af4-4a58-bb0e-eaf1f32e9b1c tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] <== disconnect_volume: return (410ms) None {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:203}}
Nov 29 03:10:50 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-ef90f75a-4af4-4a58-bb0e-eaf1f32e9b1c tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] [instance: f7d0bdeb-6a05-47ac-87f2-c8380a9a4286] Disconnected iSCSI Volume {{(pid=75234) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:79}}
Nov 29 03:10:51 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:10:51 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:10:51 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:10:51 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Starting heal instance info cache {{(pid=75234) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9922}}
Nov 29 03:10:51 user nova-compute[75234]: DEBUG nova.objects.instance [None req-ef90f75a-4af4-4a58-bb0e-eaf1f32e9b1c tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] Lazy-loading 'flavor' on Instance uuid f7d0bdeb-6a05-47ac-87f2-c8380a9a4286 {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:10:51 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:10:51 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ef90f75a-4af4-4a58-bb0e-eaf1f32e9b1c tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] Lock "f7d0bdeb-6a05-47ac-87f2-c8380a9a4286" "released" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: held 22.325s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:10:51 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Acquiring lock "refresh_cache-a9350a18-14e6-42c6-a150-0a1487f20ad9" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:10:51 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Acquired lock "refresh_cache-a9350a18-14e6-42c6-a150-0a1487f20ad9" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:10:51 user nova-compute[75234]: DEBUG nova.network.neutron [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] [instance: a9350a18-14e6-42c6-a150-0a1487f20ad9] Forcefully refreshing network info cache for instance {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2003}}
Nov 29 03:10:51 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:10:52 user nova-compute[75234]: DEBUG nova.network.neutron [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] [instance: a9350a18-14e6-42c6-a150-0a1487f20ad9] Updating instance_info_cache with network_info: [{"id": "52024efc-8cde-4610-b12c-198f19d75ccf", "address": "fa:16:3e:09:b8:70", "network": {"id": "31c4ca3b-3a2f-47df-87fa-4ad095ee5cf9", "bridge": "br-int", "label": "tempest-ServerActionsTestOtherB-1680745008-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.9", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "ad5ea22c63f04f91a3f2fa665e4fc8a1", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap52024efc-8c", "ovs_interfaceid": "52024efc-8cde-4610-b12c-198f19d75ccf", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:10:52 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Releasing lock "refresh_cache-a9350a18-14e6-42c6-a150-0a1487f20ad9" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:10:52 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] [instance: a9350a18-14e6-42c6-a150-0a1487f20ad9] Updated the network info_cache for instance {{(pid=75234) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9993}}
Nov 29 03:10:52 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:10:52 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:10:52 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=75234) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10541}}
Nov 29 03:10:52 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:10:52 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:10:54 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:10:54 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:10:55 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager.update_available_resource {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:10:55 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:10:55 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:10:55 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:10:55 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Auditing locally available compute resources for user (node: user) {{(pid=75234) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:907}}
Nov 29 03:10:55 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a9350a18-14e6-42c6-a150-0a1487f20ad9/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:10:55 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a9350a18-14e6-42c6-a150-0a1487f20ad9/disk --force-share --output=json" returned: 0 in 0.238s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:10:55 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a9350a18-14e6-42c6-a150-0a1487f20ad9/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:10:56 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a9350a18-14e6-42c6-a150-0a1487f20ad9/disk --force-share --output=json" returned: 0 in 0.213s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:10:56 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] skipping disk /dev/sdc (vdb) as it is a volume {{(pid=75234) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11213}}
Nov 29 03:10:56 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/8297c264-a213-44ee-be46-5c08eabda9f7/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:10:56 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/8297c264-a213-44ee-be46-5c08eabda9f7/disk --force-share --output=json" returned: 0 in 0.191s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:10:56 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/8297c264-a213-44ee-be46-5c08eabda9f7/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:10:56 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:10:56 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/8297c264-a213-44ee-be46-5c08eabda9f7/disk --force-share --output=json" returned: 0 in 0.195s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:10:56 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] skipping disk /dev/sdd (vdb) as it is a volume {{(pid=75234) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11213}}
Nov 29 03:10:56 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/8297c264-a213-44ee-be46-5c08eabda9f7/disk.rescue --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:10:56 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:10:56 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/8297c264-a213-44ee-be46-5c08eabda9f7/disk.rescue --force-share --output=json" returned: 0 in 0.198s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:10:56 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/8297c264-a213-44ee-be46-5c08eabda9f7/disk.rescue --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:10:56 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/8297c264-a213-44ee-be46-5c08eabda9f7/disk.rescue --force-share --output=json" returned: 0 in 0.202s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:10:56 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/ac620533-8274-4674-b241-f48526bed3c3/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:10:57 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/ac620533-8274-4674-b241-f48526bed3c3/disk --force-share --output=json" returned: 0 in 0.188s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:10:57 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/ac620533-8274-4674-b241-f48526bed3c3/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:10:57 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/ac620533-8274-4674-b241-f48526bed3c3/disk --force-share --output=json" returned: 0 in 0.200s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:10:57 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a082c96e-e384-40b6-bcc8-fc5e0e104669/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:10:57 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:10:57 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a082c96e-e384-40b6-bcc8-fc5e0e104669/disk --force-share --output=json" returned: 0 in 0.194s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:10:57 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a082c96e-e384-40b6-bcc8-fc5e0e104669/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:10:57 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a082c96e-e384-40b6-bcc8-fc5e0e104669/disk --force-share --output=json" returned: 0 in 0.193s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:10:57 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2bf9fe41-63c2-4882-bcdd-8e314e734ffe/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:10:57 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2bf9fe41-63c2-4882-bcdd-8e314e734ffe/disk --force-share --output=json" returned: 0 in 0.183s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:10:57 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2bf9fe41-63c2-4882-bcdd-8e314e734ffe/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:10:58 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2bf9fe41-63c2-4882-bcdd-8e314e734ffe/disk --force-share --output=json" returned: 0 in 0.176s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:10:58 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f609a7b7-1be2-44b4-9177-7e72a4d4c060/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:10:58 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f609a7b7-1be2-44b4-9177-7e72a4d4c060/disk --force-share --output=json" returned: 0 in 0.191s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:10:58 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f609a7b7-1be2-44b4-9177-7e72a4d4c060/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:10:58 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f609a7b7-1be2-44b4-9177-7e72a4d4c060/disk --force-share --output=json" returned: 0 in 0.179s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:10:58 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] skipping disk /dev/sde (vdb) as it is a volume {{(pid=75234) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11213}}
Nov 29 03:10:58 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/21e9c6cd-c39f-41c0-8768-5dd64589c3f3/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:10:58 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/21e9c6cd-c39f-41c0-8768-5dd64589c3f3/disk --force-share --output=json" returned: 0 in 0.176s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:10:58 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/21e9c6cd-c39f-41c0-8768-5dd64589c3f3/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:10:58 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/21e9c6cd-c39f-41c0-8768-5dd64589c3f3/disk --force-share --output=json" returned: 0 in 0.194s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:10:58 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/13e5da64-9adc-4479-81c7-d329e9b5f08f/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:10:59 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/13e5da64-9adc-4479-81c7-d329e9b5f08f/disk --force-share --output=json" returned: 0 in 0.176s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:10:59 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/13e5da64-9adc-4479-81c7-d329e9b5f08f/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:10:59 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/13e5da64-9adc-4479-81c7-d329e9b5f08f/disk --force-share --output=json" returned: 0 in 0.195s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:10:59 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6c0e0184-ca3a-48ec-a6e3-a0a1172260d6/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:10:59 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6c0e0184-ca3a-48ec-a6e3-a0a1172260d6/disk --force-share --output=json" returned: 0 in 0.233s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:10:59 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6c0e0184-ca3a-48ec-a6e3-a0a1172260d6/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:10:59 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6c0e0184-ca3a-48ec-a6e3-a0a1172260d6/disk --force-share --output=json" returned: 0 in 0.200s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:10:59 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f7d0bdeb-6a05-47ac-87f2-c8380a9a4286/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:10:59 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f7d0bdeb-6a05-47ac-87f2-c8380a9a4286/disk --force-share --output=json" returned: 0 in 0.181s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:10:59 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f7d0bdeb-6a05-47ac-87f2-c8380a9a4286/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:11:00 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f7d0bdeb-6a05-47ac-87f2-c8380a9a4286/disk --force-share --output=json" returned: 0 in 0.203s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:11:00 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/062753a6-8f39-4e80-a549-9f0f5a4b877a/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:11:00 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/062753a6-8f39-4e80-a549-9f0f5a4b877a/disk --force-share --output=json" returned: 0 in 0.214s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:11:00 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/062753a6-8f39-4e80-a549-9f0f5a4b877a/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:11:00 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/062753a6-8f39-4e80-a549-9f0f5a4b877a/disk --force-share --output=json" returned: 0 in 0.241s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:11:00 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/d3b90f39-4b91-4152-b820-bd0e263eec89/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:11:00 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/d3b90f39-4b91-4152-b820-bd0e263eec89/disk --force-share --output=json" returned: 0 in 0.223s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:11:00 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/d3b90f39-4b91-4152-b820-bd0e263eec89/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:11:01 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/d3b90f39-4b91-4152-b820-bd0e263eec89/disk --force-share --output=json" returned: 0 in 0.172s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:11:01 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:11:01 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:11:01 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:11:01 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Hypervisor/Node resource view: name=user free_ram=6815MB free_disk=25.322723388671875GB free_vcpus=0 pci_devices=[{"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}] {{(pid=75234) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1106}}
Nov 29 03:11:01 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:11:01 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:11:01 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Applying migration context for instance a9350a18-14e6-42c6-a150-0a1487f20ad9 as it has an incoming, in-progress migration 29b9709a-732b-4af0-8f8f-3b10fbde30dc. Migration status is finished {{(pid=75234) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:1016}}
Nov 29 03:11:01 user nova-compute[75234]: INFO nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] [instance: a9350a18-14e6-42c6-a150-0a1487f20ad9] Updating resource usage from migration 29b9709a-732b-4af0-8f8f-3b10fbde30dc
Nov 29 03:11:01 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance 2bf9fe41-63c2-4882-bcdd-8e314e734ffe actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:11:01 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance 21e9c6cd-c39f-41c0-8768-5dd64589c3f3 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:11:01 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance f609a7b7-1be2-44b4-9177-7e72a4d4c060 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:11:01 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance 8297c264-a213-44ee-be46-5c08eabda9f7 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:11:01 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance f7d0bdeb-6a05-47ac-87f2-c8380a9a4286 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:11:01 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance 062753a6-8f39-4e80-a549-9f0f5a4b877a actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:11:01 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance ac620533-8274-4674-b241-f48526bed3c3 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:11:01 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance 6c0e0184-ca3a-48ec-a6e3-a0a1172260d6 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:11:01 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Migration 29b9709a-732b-4af0-8f8f-3b10fbde30dc is active on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 512, 'DISK_GB': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1709}}
Nov 29 03:11:01 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance a9350a18-14e6-42c6-a150-0a1487f20ad9 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 20, 'MEMORY_MB': 2048, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:11:01 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance d3b90f39-4b91-4152-b820-bd0e263eec89 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:11:01 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance 13e5da64-9adc-4479-81c7-d329e9b5f08f actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:11:01 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance a082c96e-e384-40b6-bcc8-fc5e0e104669 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:11:01 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Total usable vcpus: 12, total allocated vcpus: 13 {{(pid=75234) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1129}}
Nov 29 03:11:01 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Final resource view: name=user phys_ram=16011MB used_ram=8704MB phys_disk=40GB used_disk=32GB total_vcpus=12 used_vcpus=13 pci_stats=[] {{(pid=75234) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1138}}
Nov 29 03:11:01 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:11:02 user nova-compute[75234]: DEBUG nova.compute.provider_tree [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Inventory has not changed in ProviderTree for provider: 4d291220-7c37-4839-a0e3-b23c66502b74 {{(pid=75234) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Nov 29 03:11:02 user nova-compute[75234]: DEBUG nova.scheduler.client.report [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Inventory has not changed for provider 4d291220-7c37-4839-a0e3-b23c66502b74 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=75234) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Nov 29 03:11:02 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Compute_service record updated for user:user {{(pid=75234) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:1067}}
Nov 29 03:11:02 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.631s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:11:03 user nova-compute[75234]: DEBUG nova.compute.manager [req-fdc4c285-533c-4f11-8c48-2780e19d30f3 req-a1507be0-cbca-4aa3-994b-a4ac8a14869a service nova] [instance: 062753a6-8f39-4e80-a549-9f0f5a4b877a] Received event network-changed-8f6751f1-fc65-4842-b77f-1dfe26912afb {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:11:03 user nova-compute[75234]: DEBUG nova.compute.manager [req-fdc4c285-533c-4f11-8c48-2780e19d30f3 req-a1507be0-cbca-4aa3-994b-a4ac8a14869a service nova] [instance: 062753a6-8f39-4e80-a549-9f0f5a4b877a] Refreshing instance network info cache due to event network-changed-8f6751f1-fc65-4842-b77f-1dfe26912afb. {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11117}}
Nov 29 03:11:03 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-fdc4c285-533c-4f11-8c48-2780e19d30f3 req-a1507be0-cbca-4aa3-994b-a4ac8a14869a service nova] Acquiring lock "refresh_cache-062753a6-8f39-4e80-a549-9f0f5a4b877a" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:11:03 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-fdc4c285-533c-4f11-8c48-2780e19d30f3 req-a1507be0-cbca-4aa3-994b-a4ac8a14869a service nova] Acquired lock "refresh_cache-062753a6-8f39-4e80-a549-9f0f5a4b877a" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:11:03 user nova-compute[75234]: DEBUG nova.network.neutron [req-fdc4c285-533c-4f11-8c48-2780e19d30f3 req-a1507be0-cbca-4aa3-994b-a4ac8a14869a service nova] [instance: 062753a6-8f39-4e80-a549-9f0f5a4b877a] Refreshing network info cache for port 8f6751f1-fc65-4842-b77f-1dfe26912afb {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2006}}
Nov 29 03:11:04 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:11:04 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:11:04 user nova-compute[75234]: DEBUG nova.network.neutron [req-fdc4c285-533c-4f11-8c48-2780e19d30f3 req-a1507be0-cbca-4aa3-994b-a4ac8a14869a service nova] [instance: 062753a6-8f39-4e80-a549-9f0f5a4b877a] Updated VIF entry in instance network info cache for port 8f6751f1-fc65-4842-b77f-1dfe26912afb. {{(pid=75234) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3481}}
Nov 29 03:11:04 user nova-compute[75234]: DEBUG nova.network.neutron [req-fdc4c285-533c-4f11-8c48-2780e19d30f3 req-a1507be0-cbca-4aa3-994b-a4ac8a14869a service nova] [instance: 062753a6-8f39-4e80-a549-9f0f5a4b877a] Updating instance_info_cache with network_info: [{"id": "8f6751f1-fc65-4842-b77f-1dfe26912afb", "address": "fa:16:3e:62:4a:5a", "network": {"id": "aad085f7-106b-4de6-93f5-f86bb66ec962", "bridge": "br-int", "label": "tempest-AttachVolumeShelveTestJSON-775984268-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.175", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "e17150005a9a449f9373b91e1c00083b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap8f6751f1-fc", "ovs_interfaceid": "8f6751f1-fc65-4842-b77f-1dfe26912afb", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:11:04 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-fdc4c285-533c-4f11-8c48-2780e19d30f3 req-a1507be0-cbca-4aa3-994b-a4ac8a14869a service nova] Releasing lock "refresh_cache-062753a6-8f39-4e80-a549-9f0f5a4b877a" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:11:04 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-14f2419a-4182-44e3-a5d2-3a9299cc69ee tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] Acquiring lock "ac620533-8274-4674-b241-f48526bed3c3" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:11:04 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-14f2419a-4182-44e3-a5d2-3a9299cc69ee tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] Lock "ac620533-8274-4674-b241-f48526bed3c3" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:11:04 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-14f2419a-4182-44e3-a5d2-3a9299cc69ee tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] Acquiring lock "ac620533-8274-4674-b241-f48526bed3c3-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:11:04 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-14f2419a-4182-44e3-a5d2-3a9299cc69ee tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] Lock "ac620533-8274-4674-b241-f48526bed3c3-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:11:04 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-14f2419a-4182-44e3-a5d2-3a9299cc69ee tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] Lock "ac620533-8274-4674-b241-f48526bed3c3-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:11:04 user nova-compute[75234]: INFO nova.compute.manager [None req-14f2419a-4182-44e3-a5d2-3a9299cc69ee tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] [instance: ac620533-8274-4674-b241-f48526bed3c3] Terminating instance
Nov 29 03:11:04 user nova-compute[75234]: DEBUG nova.compute.manager [None req-14f2419a-4182-44e3-a5d2-3a9299cc69ee tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] [instance: ac620533-8274-4674-b241-f48526bed3c3] Start destroying the instance on the hypervisor. {{(pid=75234) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3140}}
Nov 29 03:11:04 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:11:04 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:11:04 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:11:04 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:11:04 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:11:05 user nova-compute[75234]: DEBUG nova.compute.manager [req-62eef49e-7c56-408b-879d-7f669de955f4 req-7406cbd8-c0e2-4cb6-9263-acb19ab6c1ff service nova] [instance: ac620533-8274-4674-b241-f48526bed3c3] Received event network-vif-unplugged-f96869cf-1585-47e8-a04e-40ad5f212130 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:11:05 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-62eef49e-7c56-408b-879d-7f669de955f4 req-7406cbd8-c0e2-4cb6-9263-acb19ab6c1ff service nova] Acquiring lock "ac620533-8274-4674-b241-f48526bed3c3-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:11:05 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-62eef49e-7c56-408b-879d-7f669de955f4 req-7406cbd8-c0e2-4cb6-9263-acb19ab6c1ff service nova] Lock "ac620533-8274-4674-b241-f48526bed3c3-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:11:05 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-62eef49e-7c56-408b-879d-7f669de955f4 req-7406cbd8-c0e2-4cb6-9263-acb19ab6c1ff service nova] Lock "ac620533-8274-4674-b241-f48526bed3c3-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:11:05 user nova-compute[75234]: DEBUG nova.compute.manager [req-62eef49e-7c56-408b-879d-7f669de955f4 req-7406cbd8-c0e2-4cb6-9263-acb19ab6c1ff service nova] [instance: ac620533-8274-4674-b241-f48526bed3c3] No waiting events found dispatching network-vif-unplugged-f96869cf-1585-47e8-a04e-40ad5f212130 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:11:05 user nova-compute[75234]: DEBUG nova.compute.manager [req-62eef49e-7c56-408b-879d-7f669de955f4 req-7406cbd8-c0e2-4cb6-9263-acb19ab6c1ff service nova] [instance: ac620533-8274-4674-b241-f48526bed3c3] Received event network-vif-unplugged-f96869cf-1585-47e8-a04e-40ad5f212130 for instance with task_state deleting. {{(pid=75234) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10890}}
Nov 29 03:11:05 user nova-compute[75234]: INFO nova.virt.libvirt.driver [-] [instance: ac620533-8274-4674-b241-f48526bed3c3] Instance destroyed successfully.
Nov 29 03:11:05 user nova-compute[75234]: DEBUG nova.objects.instance [None req-14f2419a-4182-44e3-a5d2-3a9299cc69ee tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] Lazy-loading 'resources' on Instance uuid ac620533-8274-4674-b241-f48526bed3c3 {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:11:05 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-14f2419a-4182-44e3-a5d2-3a9299cc69ee tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-11-29T03:09:13Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-VolumesActionsTest-instance-134859871',display_name='tempest-VolumesActionsTest-instance-134859871',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-volumesactionstest-instance-134859871',id=15,image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data=None,key_name=None,keypairs=<?>,launch_index=0,launched_at=2023-11-29T03:09:21Z,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='46577f787c244fb2a4db052a0c42eceb',ramdisk_id='',reservation_id='r-oalbemin',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-VolumesActionsTest-196322865',owner_user_name='tempest-VolumesActionsTest-196322865-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-11-29T03:09:21Z,user_data=None,user_id='940a684aa3b6419798bd87562bbc5d23',uuid=ac620533-8274-4674-b241-f48526bed3c3,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "f96869cf-1585-47e8-a04e-40ad5f212130", "address": "fa:16:3e:e6:9b:b1", "network": {"id": "68a1f282-91d0-47c8-82f3-5ebf1c0f572c", "bridge": "br-int", "label": "tempest-VolumesActionsTest-570720740-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "46577f787c244fb2a4db052a0c42eceb", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapf96869cf-15", "ovs_interfaceid": "f96869cf-1585-47e8-a04e-40ad5f212130", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Nov 29 03:11:05 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-14f2419a-4182-44e3-a5d2-3a9299cc69ee tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] Converting VIF {"id": "f96869cf-1585-47e8-a04e-40ad5f212130", "address": "fa:16:3e:e6:9b:b1", "network": {"id": "68a1f282-91d0-47c8-82f3-5ebf1c0f572c", "bridge": "br-int", "label": "tempest-VolumesActionsTest-570720740-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "46577f787c244fb2a4db052a0c42eceb", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapf96869cf-15", "ovs_interfaceid": "f96869cf-1585-47e8-a04e-40ad5f212130", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:11:05 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-14f2419a-4182-44e3-a5d2-3a9299cc69ee tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:e6:9b:b1,bridge_name='br-int',has_traffic_filtering=True,id=f96869cf-1585-47e8-a04e-40ad5f212130,network=Network(68a1f282-91d0-47c8-82f3-5ebf1c0f572c),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapf96869cf-15') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:11:05 user nova-compute[75234]: DEBUG os_vif [None req-14f2419a-4182-44e3-a5d2-3a9299cc69ee tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] Unplugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:e6:9b:b1,bridge_name='br-int',has_traffic_filtering=True,id=f96869cf-1585-47e8-a04e-40ad5f212130,network=Network(68a1f282-91d0-47c8-82f3-5ebf1c0f572c),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapf96869cf-15') {{(pid=75234) unplug /opt/stack/data/venv/lib/python3.10/site-packages/os_vif/__init__.py:109}}
Nov 29 03:11:05 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:11:05 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tapf96869cf-15, bridge=br-int, if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:11:05 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:11:05 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:11:05 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:11:05 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbDestroyCommand(_result=None, table=QoS, record=9404b288-ffa1-4472-968a-cdaa548ad21b) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:11:05 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:11:05 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:11:05 user nova-compute[75234]: INFO os_vif [None req-14f2419a-4182-44e3-a5d2-3a9299cc69ee tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] Successfully unplugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:e6:9b:b1,bridge_name='br-int',has_traffic_filtering=True,id=f96869cf-1585-47e8-a04e-40ad5f212130,network=Network(68a1f282-91d0-47c8-82f3-5ebf1c0f572c),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapf96869cf-15')
Nov 29 03:11:05 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-14f2419a-4182-44e3-a5d2-3a9299cc69ee tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] [instance: ac620533-8274-4674-b241-f48526bed3c3] Deleting instance files /opt/stack/data/nova/instances/ac620533-8274-4674-b241-f48526bed3c3_del
Nov 29 03:11:05 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-14f2419a-4182-44e3-a5d2-3a9299cc69ee tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] [instance: ac620533-8274-4674-b241-f48526bed3c3] Deletion of /opt/stack/data/nova/instances/ac620533-8274-4674-b241-f48526bed3c3_del complete
Nov 29 03:11:05 user nova-compute[75234]: INFO nova.compute.manager [None req-14f2419a-4182-44e3-a5d2-3a9299cc69ee tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] [instance: ac620533-8274-4674-b241-f48526bed3c3] Took 0.96 seconds to destroy the instance on the hypervisor.
Nov 29 03:11:05 user nova-compute[75234]: DEBUG oslo.service.loopingcall [None req-14f2419a-4182-44e3-a5d2-3a9299cc69ee tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=75234) func /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/loopingcall.py:435}}
Nov 29 03:11:05 user nova-compute[75234]: DEBUG nova.compute.manager [-] [instance: ac620533-8274-4674-b241-f48526bed3c3] Deallocating network for instance {{(pid=75234) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2279}}
Nov 29 03:11:05 user nova-compute[75234]: DEBUG nova.network.neutron [-] [instance: ac620533-8274-4674-b241-f48526bed3c3] deallocate_for_instance() {{(pid=75234) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1802}}
Nov 29 03:11:06 user nova-compute[75234]: DEBUG nova.network.neutron [-] [instance: ac620533-8274-4674-b241-f48526bed3c3] Updating instance_info_cache with network_info: [] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:11:06 user nova-compute[75234]: INFO nova.compute.manager [-] [instance: ac620533-8274-4674-b241-f48526bed3c3] Took 0.67 seconds to deallocate network for instance.
Nov 29 03:11:06 user nova-compute[75234]: DEBUG nova.compute.manager [req-3777cfa8-7c95-4577-994f-2cf67b85b447 req-2bcee5de-a7f3-4bb1-97a0-caf44d4664d7 service nova] [instance: ac620533-8274-4674-b241-f48526bed3c3] Received event network-vif-deleted-f96869cf-1585-47e8-a04e-40ad5f212130 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:11:06 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-14f2419a-4182-44e3-a5d2-3a9299cc69ee tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:11:06 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-14f2419a-4182-44e3-a5d2-3a9299cc69ee tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:11:06 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-1afdf47c-9f51-4398-b7f1-e228ee2aba24 tempest-TestVolumeSwap-1089931716 tempest-TestVolumeSwap-1089931716-project-member] Acquiring lock "3e355ae8-aab9-4ae9-b6e3-6de108ec1af9" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:11:06 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-1afdf47c-9f51-4398-b7f1-e228ee2aba24 tempest-TestVolumeSwap-1089931716 tempest-TestVolumeSwap-1089931716-project-member] Lock "3e355ae8-aab9-4ae9-b6e3-6de108ec1af9" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:11:06 user nova-compute[75234]: DEBUG nova.compute.manager [None req-1afdf47c-9f51-4398-b7f1-e228ee2aba24 tempest-TestVolumeSwap-1089931716 tempest-TestVolumeSwap-1089931716-project-member] [instance: 3e355ae8-aab9-4ae9-b6e3-6de108ec1af9] Starting instance... {{(pid=75234) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2422}}
Nov 29 03:11:06 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-1afdf47c-9f51-4398-b7f1-e228ee2aba24 tempest-TestVolumeSwap-1089931716 tempest-TestVolumeSwap-1089931716-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:11:06 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:11:06 user nova-compute[75234]: DEBUG nova.compute.provider_tree [None req-14f2419a-4182-44e3-a5d2-3a9299cc69ee tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] Inventory has not changed in ProviderTree for provider: 4d291220-7c37-4839-a0e3-b23c66502b74 {{(pid=75234) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Nov 29 03:11:06 user nova-compute[75234]: DEBUG nova.scheduler.client.report [None req-14f2419a-4182-44e3-a5d2-3a9299cc69ee tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] Inventory has not changed for provider 4d291220-7c37-4839-a0e3-b23c66502b74 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=75234) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Nov 29 03:11:06 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-14f2419a-4182-44e3-a5d2-3a9299cc69ee tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.477s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:11:06 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-1afdf47c-9f51-4398-b7f1-e228ee2aba24 tempest-TestVolumeSwap-1089931716 tempest-TestVolumeSwap-1089931716-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.351s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:11:06 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-1afdf47c-9f51-4398-b7f1-e228ee2aba24 tempest-TestVolumeSwap-1089931716 tempest-TestVolumeSwap-1089931716-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=75234) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2366}}
Nov 29 03:11:06 user nova-compute[75234]: INFO nova.compute.claims [None req-1afdf47c-9f51-4398-b7f1-e228ee2aba24 tempest-TestVolumeSwap-1089931716 tempest-TestVolumeSwap-1089931716-project-member] [instance: 3e355ae8-aab9-4ae9-b6e3-6de108ec1af9] Claim successful on node user
Nov 29 03:11:06 user nova-compute[75234]: INFO nova.scheduler.client.report [None req-14f2419a-4182-44e3-a5d2-3a9299cc69ee tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] Deleted allocations for instance ac620533-8274-4674-b241-f48526bed3c3
Nov 29 03:11:06 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-14f2419a-4182-44e3-a5d2-3a9299cc69ee tempest-VolumesActionsTest-196322865 tempest-VolumesActionsTest-196322865-project-member] Lock "ac620533-8274-4674-b241-f48526bed3c3" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 2.300s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:11:07 user nova-compute[75234]: DEBUG nova.compute.manager [req-1def89fe-832c-4baf-9b4a-09eb96c984ad req-8d7201e0-043d-40b9-8647-fc5987ed4ce9 service nova] [instance: ac620533-8274-4674-b241-f48526bed3c3] Received event network-vif-plugged-f96869cf-1585-47e8-a04e-40ad5f212130 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:11:07 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-1def89fe-832c-4baf-9b4a-09eb96c984ad req-8d7201e0-043d-40b9-8647-fc5987ed4ce9 service nova] Acquiring lock "ac620533-8274-4674-b241-f48526bed3c3-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:11:07 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-1def89fe-832c-4baf-9b4a-09eb96c984ad req-8d7201e0-043d-40b9-8647-fc5987ed4ce9 service nova] Lock "ac620533-8274-4674-b241-f48526bed3c3-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:11:07 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-1def89fe-832c-4baf-9b4a-09eb96c984ad req-8d7201e0-043d-40b9-8647-fc5987ed4ce9 service nova] Lock "ac620533-8274-4674-b241-f48526bed3c3-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:11:07 user nova-compute[75234]: DEBUG nova.compute.manager [req-1def89fe-832c-4baf-9b4a-09eb96c984ad req-8d7201e0-043d-40b9-8647-fc5987ed4ce9 service nova] [instance: ac620533-8274-4674-b241-f48526bed3c3] No waiting events found dispatching network-vif-plugged-f96869cf-1585-47e8-a04e-40ad5f212130 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:11:07 user nova-compute[75234]: WARNING nova.compute.manager [req-1def89fe-832c-4baf-9b4a-09eb96c984ad req-8d7201e0-043d-40b9-8647-fc5987ed4ce9 service nova] [instance: ac620533-8274-4674-b241-f48526bed3c3] Received unexpected event network-vif-plugged-f96869cf-1585-47e8-a04e-40ad5f212130 for instance with vm_state deleted and task_state None.
Nov 29 03:11:07 user nova-compute[75234]: DEBUG nova.compute.provider_tree [None req-1afdf47c-9f51-4398-b7f1-e228ee2aba24 tempest-TestVolumeSwap-1089931716 tempest-TestVolumeSwap-1089931716-project-member] Inventory has not changed in ProviderTree for provider: 4d291220-7c37-4839-a0e3-b23c66502b74 {{(pid=75234) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Nov 29 03:11:07 user nova-compute[75234]: DEBUG nova.scheduler.client.report [None req-1afdf47c-9f51-4398-b7f1-e228ee2aba24 tempest-TestVolumeSwap-1089931716 tempest-TestVolumeSwap-1089931716-project-member] Inventory has not changed for provider 4d291220-7c37-4839-a0e3-b23c66502b74 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=75234) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Nov 29 03:11:07 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-1afdf47c-9f51-4398-b7f1-e228ee2aba24 tempest-TestVolumeSwap-1089931716 tempest-TestVolumeSwap-1089931716-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.524s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:11:07 user nova-compute[75234]: DEBUG nova.compute.manager [None req-1afdf47c-9f51-4398-b7f1-e228ee2aba24 tempest-TestVolumeSwap-1089931716 tempest-TestVolumeSwap-1089931716-project-member] [instance: 3e355ae8-aab9-4ae9-b6e3-6de108ec1af9] Start building networks asynchronously for instance. {{(pid=75234) _build_resources /opt/stack/nova/nova/compute/manager.py:2819}}
Nov 29 03:11:07 user nova-compute[75234]: DEBUG nova.compute.manager [None req-1afdf47c-9f51-4398-b7f1-e228ee2aba24 tempest-TestVolumeSwap-1089931716 tempest-TestVolumeSwap-1089931716-project-member] [instance: 3e355ae8-aab9-4ae9-b6e3-6de108ec1af9] Allocating IP information in the background. {{(pid=75234) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1972}}
Nov 29 03:11:07 user nova-compute[75234]: DEBUG nova.network.neutron [None req-1afdf47c-9f51-4398-b7f1-e228ee2aba24 tempest-TestVolumeSwap-1089931716 tempest-TestVolumeSwap-1089931716-project-member] [instance: 3e355ae8-aab9-4ae9-b6e3-6de108ec1af9] allocate_for_instance() {{(pid=75234) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1156}}
Nov 29 03:11:07 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-1afdf47c-9f51-4398-b7f1-e228ee2aba24 tempest-TestVolumeSwap-1089931716 tempest-TestVolumeSwap-1089931716-project-member] [instance: 3e355ae8-aab9-4ae9-b6e3-6de108ec1af9] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Nov 29 03:11:07 user nova-compute[75234]: DEBUG nova.compute.manager [None req-1afdf47c-9f51-4398-b7f1-e228ee2aba24 tempest-TestVolumeSwap-1089931716 tempest-TestVolumeSwap-1089931716-project-member] [instance: 3e355ae8-aab9-4ae9-b6e3-6de108ec1af9] Start building block device mappings for instance. {{(pid=75234) _build_resources /opt/stack/nova/nova/compute/manager.py:2854}}
Nov 29 03:11:07 user nova-compute[75234]: DEBUG nova.policy [None req-1afdf47c-9f51-4398-b7f1-e228ee2aba24 tempest-TestVolumeSwap-1089931716 tempest-TestVolumeSwap-1089931716-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '5366feb605674f9a8c7f547ba2beb65c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'cd4a0b5a414044d68524962183d47fca', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=75234) authorize /opt/stack/nova/nova/policy.py:203}}
Nov 29 03:11:07 user nova-compute[75234]: DEBUG nova.compute.manager [None req-1afdf47c-9f51-4398-b7f1-e228ee2aba24 tempest-TestVolumeSwap-1089931716 tempest-TestVolumeSwap-1089931716-project-member] [instance: 3e355ae8-aab9-4ae9-b6e3-6de108ec1af9] Start spawning the instance on the hypervisor. {{(pid=75234) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2628}}
Nov 29 03:11:07 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-1afdf47c-9f51-4398-b7f1-e228ee2aba24 tempest-TestVolumeSwap-1089931716 tempest-TestVolumeSwap-1089931716-project-member] [instance: 3e355ae8-aab9-4ae9-b6e3-6de108ec1af9] Creating instance directory {{(pid=75234) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4727}}
Nov 29 03:11:07 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-1afdf47c-9f51-4398-b7f1-e228ee2aba24 tempest-TestVolumeSwap-1089931716 tempest-TestVolumeSwap-1089931716-project-member] [instance: 3e355ae8-aab9-4ae9-b6e3-6de108ec1af9] Creating image(s)
Nov 29 03:11:07 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-1afdf47c-9f51-4398-b7f1-e228ee2aba24 tempest-TestVolumeSwap-1089931716 tempest-TestVolumeSwap-1089931716-project-member] Acquiring lock "/opt/stack/data/nova/instances/3e355ae8-aab9-4ae9-b6e3-6de108ec1af9/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:11:07 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-1afdf47c-9f51-4398-b7f1-e228ee2aba24 tempest-TestVolumeSwap-1089931716 tempest-TestVolumeSwap-1089931716-project-member] Lock "/opt/stack/data/nova/instances/3e355ae8-aab9-4ae9-b6e3-6de108ec1af9/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:11:07 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-1afdf47c-9f51-4398-b7f1-e228ee2aba24 tempest-TestVolumeSwap-1089931716 tempest-TestVolumeSwap-1089931716-project-member] Lock "/opt/stack/data/nova/instances/3e355ae8-aab9-4ae9-b6e3-6de108ec1af9/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:11:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-1afdf47c-9f51-4398-b7f1-e228ee2aba24 tempest-TestVolumeSwap-1089931716 tempest-TestVolumeSwap-1089931716-project-member] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:11:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-1afdf47c-9f51-4398-b7f1-e228ee2aba24 tempest-TestVolumeSwap-1089931716 tempest-TestVolumeSwap-1089931716-project-member] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json" returned: 0 in 0.199s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:11:07 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-1afdf47c-9f51-4398-b7f1-e228ee2aba24 tempest-TestVolumeSwap-1089931716 tempest-TestVolumeSwap-1089931716-project-member] Acquiring lock "80686e29dc9db6600d03e85cb62af298e12d7918" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:11:07 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-1afdf47c-9f51-4398-b7f1-e228ee2aba24 tempest-TestVolumeSwap-1089931716 tempest-TestVolumeSwap-1089931716-project-member] Lock "80686e29dc9db6600d03e85cb62af298e12d7918" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:11:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-1afdf47c-9f51-4398-b7f1-e228ee2aba24 tempest-TestVolumeSwap-1089931716 tempest-TestVolumeSwap-1089931716-project-member] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:11:07 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-45c0b654-c277-4e79-b2de-2c89bf731fdf tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Acquiring lock "062753a6-8f39-4e80-a549-9f0f5a4b877a" by "nova.compute.manager.ComputeManager.shelve_instance.<locals>.do_shelve_instance" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:11:07 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-45c0b654-c277-4e79-b2de-2c89bf731fdf tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Lock "062753a6-8f39-4e80-a549-9f0f5a4b877a" acquired by "nova.compute.manager.ComputeManager.shelve_instance.<locals>.do_shelve_instance" :: waited 0.004s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:11:07 user nova-compute[75234]: INFO nova.compute.manager [None req-45c0b654-c277-4e79-b2de-2c89bf731fdf tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] [instance: 062753a6-8f39-4e80-a549-9f0f5a4b877a] Shelving
Nov 29 03:11:07 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:11:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-1afdf47c-9f51-4398-b7f1-e228ee2aba24 tempest-TestVolumeSwap-1089931716 tempest-TestVolumeSwap-1089931716-project-member] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json" returned: 0 in 0.183s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:11:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-1afdf47c-9f51-4398-b7f1-e228ee2aba24 tempest-TestVolumeSwap-1089931716 tempest-TestVolumeSwap-1089931716-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918,backing_fmt=raw /opt/stack/data/nova/instances/3e355ae8-aab9-4ae9-b6e3-6de108ec1af9/disk 1073741824 {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:11:07 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:11:07 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:11:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-1afdf47c-9f51-4398-b7f1-e228ee2aba24 tempest-TestVolumeSwap-1089931716 tempest-TestVolumeSwap-1089931716-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918,backing_fmt=raw /opt/stack/data/nova/instances/3e355ae8-aab9-4ae9-b6e3-6de108ec1af9/disk 1073741824" returned: 0 in 0.073s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:11:07 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-1afdf47c-9f51-4398-b7f1-e228ee2aba24 tempest-TestVolumeSwap-1089931716 tempest-TestVolumeSwap-1089931716-project-member] Lock "80686e29dc9db6600d03e85cb62af298e12d7918" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.260s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:11:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-1afdf47c-9f51-4398-b7f1-e228ee2aba24 tempest-TestVolumeSwap-1089931716 tempest-TestVolumeSwap-1089931716-project-member] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:11:08 user nova-compute[75234]: DEBUG nova.compute.manager [req-421690b3-86df-4464-8f42-9efe37390369 req-5f4e7163-e212-4eef-a095-eae9045efcbe service nova] [instance: 062753a6-8f39-4e80-a549-9f0f5a4b877a] Received event network-vif-unplugged-8f6751f1-fc65-4842-b77f-1dfe26912afb {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:11:08 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-421690b3-86df-4464-8f42-9efe37390369 req-5f4e7163-e212-4eef-a095-eae9045efcbe service nova] Acquiring lock "062753a6-8f39-4e80-a549-9f0f5a4b877a-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:11:08 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-421690b3-86df-4464-8f42-9efe37390369 req-5f4e7163-e212-4eef-a095-eae9045efcbe service nova] Lock "062753a6-8f39-4e80-a549-9f0f5a4b877a-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:11:08 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-421690b3-86df-4464-8f42-9efe37390369 req-5f4e7163-e212-4eef-a095-eae9045efcbe service nova] Lock "062753a6-8f39-4e80-a549-9f0f5a4b877a-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:11:08 user nova-compute[75234]: DEBUG nova.compute.manager [req-421690b3-86df-4464-8f42-9efe37390369 req-5f4e7163-e212-4eef-a095-eae9045efcbe service nova] [instance: 062753a6-8f39-4e80-a549-9f0f5a4b877a] No waiting events found dispatching network-vif-unplugged-8f6751f1-fc65-4842-b77f-1dfe26912afb {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:11:08 user nova-compute[75234]: WARNING nova.compute.manager [req-421690b3-86df-4464-8f42-9efe37390369 req-5f4e7163-e212-4eef-a095-eae9045efcbe service nova] [instance: 062753a6-8f39-4e80-a549-9f0f5a4b877a] Received unexpected event network-vif-unplugged-8f6751f1-fc65-4842-b77f-1dfe26912afb for instance with vm_state active and task_state shelving.
Nov 29 03:11:08 user nova-compute[75234]: DEBUG nova.network.neutron [None req-1afdf47c-9f51-4398-b7f1-e228ee2aba24 tempest-TestVolumeSwap-1089931716 tempest-TestVolumeSwap-1089931716-project-member] [instance: 3e355ae8-aab9-4ae9-b6e3-6de108ec1af9] Successfully created port: b70722c1-452e-4b37-9ff3-7d8a1c7a9ba3 {{(pid=75234) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:548}}
Nov 29 03:11:08 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-1afdf47c-9f51-4398-b7f1-e228ee2aba24 tempest-TestVolumeSwap-1089931716 tempest-TestVolumeSwap-1089931716-project-member] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json" returned: 0 in 0.210s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:11:08 user nova-compute[75234]: DEBUG nova.virt.disk.api [None req-1afdf47c-9f51-4398-b7f1-e228ee2aba24 tempest-TestVolumeSwap-1089931716 tempest-TestVolumeSwap-1089931716-project-member] Checking if we can resize image /opt/stack/data/nova/instances/3e355ae8-aab9-4ae9-b6e3-6de108ec1af9/disk. size=1073741824 {{(pid=75234) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:166}}
Nov 29 03:11:08 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-1afdf47c-9f51-4398-b7f1-e228ee2aba24 tempest-TestVolumeSwap-1089931716 tempest-TestVolumeSwap-1089931716-project-member] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/3e355ae8-aab9-4ae9-b6e3-6de108ec1af9/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:11:08 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-1afdf47c-9f51-4398-b7f1-e228ee2aba24 tempest-TestVolumeSwap-1089931716 tempest-TestVolumeSwap-1089931716-project-member] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/3e355ae8-aab9-4ae9-b6e3-6de108ec1af9/disk --force-share --output=json" returned: 0 in 0.168s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:11:08 user nova-compute[75234]: DEBUG nova.virt.disk.api [None req-1afdf47c-9f51-4398-b7f1-e228ee2aba24 tempest-TestVolumeSwap-1089931716 tempest-TestVolumeSwap-1089931716-project-member] Cannot resize image /opt/stack/data/nova/instances/3e355ae8-aab9-4ae9-b6e3-6de108ec1af9/disk to a smaller size. {{(pid=75234) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:172}}
Nov 29 03:11:08 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-1afdf47c-9f51-4398-b7f1-e228ee2aba24 tempest-TestVolumeSwap-1089931716 tempest-TestVolumeSwap-1089931716-project-member] [instance: 3e355ae8-aab9-4ae9-b6e3-6de108ec1af9] Created local disks {{(pid=75234) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4861}}
Nov 29 03:11:08 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-1afdf47c-9f51-4398-b7f1-e228ee2aba24 tempest-TestVolumeSwap-1089931716 tempest-TestVolumeSwap-1089931716-project-member] [instance: 3e355ae8-aab9-4ae9-b6e3-6de108ec1af9] Ensure instance console log exists: /opt/stack/data/nova/instances/3e355ae8-aab9-4ae9-b6e3-6de108ec1af9/console.log {{(pid=75234) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4613}}
Nov 29 03:11:08 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-1afdf47c-9f51-4398-b7f1-e228ee2aba24 tempest-TestVolumeSwap-1089931716 tempest-TestVolumeSwap-1089931716-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:11:08 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-1afdf47c-9f51-4398-b7f1-e228ee2aba24 tempest-TestVolumeSwap-1089931716 tempest-TestVolumeSwap-1089931716-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:11:08 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-1afdf47c-9f51-4398-b7f1-e228ee2aba24 tempest-TestVolumeSwap-1089931716 tempest-TestVolumeSwap-1089931716-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:11:08 user nova-compute[75234]: INFO nova.virt.libvirt.driver [-] [instance: 062753a6-8f39-4e80-a549-9f0f5a4b877a] Instance destroyed successfully.
Nov 29 03:11:08 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-45c0b654-c277-4e79-b2de-2c89bf731fdf tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] [instance: 062753a6-8f39-4e80-a549-9f0f5a4b877a] Beginning cold snapshot process
Nov 29 03:11:08 user nova-compute[75234]: DEBUG nova.network.neutron [None req-1afdf47c-9f51-4398-b7f1-e228ee2aba24 tempest-TestVolumeSwap-1089931716 tempest-TestVolumeSwap-1089931716-project-member] [instance: 3e355ae8-aab9-4ae9-b6e3-6de108ec1af9] Successfully updated port: b70722c1-452e-4b37-9ff3-7d8a1c7a9ba3 {{(pid=75234) _update_port /opt/stack/nova/nova/network/neutron.py:586}}
Nov 29 03:11:08 user nova-compute[75234]: DEBUG nova.compute.manager [req-a2963188-b139-4502-bd9e-8e72f195c1c3 req-7ad07246-2092-4a44-b260-bccfba41a7d6 service nova] [instance: 3e355ae8-aab9-4ae9-b6e3-6de108ec1af9] Received event network-changed-b70722c1-452e-4b37-9ff3-7d8a1c7a9ba3 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:11:08 user nova-compute[75234]: DEBUG nova.compute.manager [req-a2963188-b139-4502-bd9e-8e72f195c1c3 req-7ad07246-2092-4a44-b260-bccfba41a7d6 service nova] [instance: 3e355ae8-aab9-4ae9-b6e3-6de108ec1af9] Refreshing instance network info cache due to event network-changed-b70722c1-452e-4b37-9ff3-7d8a1c7a9ba3. {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11117}}
Nov 29 03:11:08 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-a2963188-b139-4502-bd9e-8e72f195c1c3 req-7ad07246-2092-4a44-b260-bccfba41a7d6 service nova] Acquiring lock "refresh_cache-3e355ae8-aab9-4ae9-b6e3-6de108ec1af9" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:11:08 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-a2963188-b139-4502-bd9e-8e72f195c1c3 req-7ad07246-2092-4a44-b260-bccfba41a7d6 service nova] Acquired lock "refresh_cache-3e355ae8-aab9-4ae9-b6e3-6de108ec1af9" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:11:08 user nova-compute[75234]: DEBUG nova.network.neutron [req-a2963188-b139-4502-bd9e-8e72f195c1c3 req-7ad07246-2092-4a44-b260-bccfba41a7d6 service nova] [instance: 3e355ae8-aab9-4ae9-b6e3-6de108ec1af9] Refreshing network info cache for port b70722c1-452e-4b37-9ff3-7d8a1c7a9ba3 {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2006}}
Nov 29 03:11:08 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-1afdf47c-9f51-4398-b7f1-e228ee2aba24 tempest-TestVolumeSwap-1089931716 tempest-TestVolumeSwap-1089931716-project-member] Acquiring lock "refresh_cache-3e355ae8-aab9-4ae9-b6e3-6de108ec1af9" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:11:08 user nova-compute[75234]: DEBUG nova.network.neutron [req-a2963188-b139-4502-bd9e-8e72f195c1c3 req-7ad07246-2092-4a44-b260-bccfba41a7d6 service nova] [instance: 3e355ae8-aab9-4ae9-b6e3-6de108ec1af9] Instance cache missing network info. {{(pid=75234) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3322}}
Nov 29 03:11:08 user nova-compute[75234]: DEBUG nova.privsep.utils [None req-45c0b654-c277-4e79-b2de-2c89bf731fdf tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Path '/opt/stack/data/nova/instances' supports direct I/O {{(pid=75234) supports_direct_io /opt/stack/nova/nova/privsep/utils.py:63}}
Nov 29 03:11:08 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-45c0b654-c277-4e79-b2de-2c89bf731fdf tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Running cmd (subprocess): qemu-img convert -t none -O qcow2 -f qcow2 /opt/stack/data/nova/instances/062753a6-8f39-4e80-a549-9f0f5a4b877a/disk /opt/stack/data/nova/instances/snapshots/tmplk18vv2m/158e85d2038d44809c5585f24eb14b9f {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:11:09 user nova-compute[75234]: DEBUG nova.network.neutron [req-a2963188-b139-4502-bd9e-8e72f195c1c3 req-7ad07246-2092-4a44-b260-bccfba41a7d6 service nova] [instance: 3e355ae8-aab9-4ae9-b6e3-6de108ec1af9] Updating instance_info_cache with network_info: [] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:11:09 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-a2963188-b139-4502-bd9e-8e72f195c1c3 req-7ad07246-2092-4a44-b260-bccfba41a7d6 service nova] Releasing lock "refresh_cache-3e355ae8-aab9-4ae9-b6e3-6de108ec1af9" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:11:09 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-1afdf47c-9f51-4398-b7f1-e228ee2aba24 tempest-TestVolumeSwap-1089931716 tempest-TestVolumeSwap-1089931716-project-member] Acquired lock "refresh_cache-3e355ae8-aab9-4ae9-b6e3-6de108ec1af9" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:11:09 user nova-compute[75234]: DEBUG nova.network.neutron [None req-1afdf47c-9f51-4398-b7f1-e228ee2aba24 tempest-TestVolumeSwap-1089931716 tempest-TestVolumeSwap-1089931716-project-member] [instance: 3e355ae8-aab9-4ae9-b6e3-6de108ec1af9] Building network info cache for instance {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2009}}
Nov 29 03:11:09 user nova-compute[75234]: DEBUG nova.network.neutron [None req-1afdf47c-9f51-4398-b7f1-e228ee2aba24 tempest-TestVolumeSwap-1089931716 tempest-TestVolumeSwap-1089931716-project-member] [instance: 3e355ae8-aab9-4ae9-b6e3-6de108ec1af9] Instance cache missing network info. {{(pid=75234) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3322}}
Nov 29 03:11:09 user nova-compute[75234]: DEBUG nova.network.neutron [None req-1afdf47c-9f51-4398-b7f1-e228ee2aba24 tempest-TestVolumeSwap-1089931716 tempest-TestVolumeSwap-1089931716-project-member] [instance: 3e355ae8-aab9-4ae9-b6e3-6de108ec1af9] Updating instance_info_cache with network_info: [{"id": "b70722c1-452e-4b37-9ff3-7d8a1c7a9ba3", "address": "fa:16:3e:db:81:b1", "network": {"id": "67d2e671-98cd-4178-8f5f-1ee06dd66006", "bridge": "br-int", "label": "tempest-TestVolumeSwap-1472119110-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "cd4a0b5a414044d68524962183d47fca", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapb70722c1-45", "ovs_interfaceid": "b70722c1-452e-4b37-9ff3-7d8a1c7a9ba3", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:11:09 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-1afdf47c-9f51-4398-b7f1-e228ee2aba24 tempest-TestVolumeSwap-1089931716 tempest-TestVolumeSwap-1089931716-project-member] Releasing lock "refresh_cache-3e355ae8-aab9-4ae9-b6e3-6de108ec1af9" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:11:09 user nova-compute[75234]: DEBUG nova.compute.manager [None req-1afdf47c-9f51-4398-b7f1-e228ee2aba24 tempest-TestVolumeSwap-1089931716 tempest-TestVolumeSwap-1089931716-project-member] [instance: 3e355ae8-aab9-4ae9-b6e3-6de108ec1af9] Instance network_info: |[{"id": "b70722c1-452e-4b37-9ff3-7d8a1c7a9ba3", "address": "fa:16:3e:db:81:b1", "network": {"id": "67d2e671-98cd-4178-8f5f-1ee06dd66006", "bridge": "br-int", "label": "tempest-TestVolumeSwap-1472119110-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "cd4a0b5a414044d68524962183d47fca", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapb70722c1-45", "ovs_interfaceid": "b70722c1-452e-4b37-9ff3-7d8a1c7a9ba3", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=75234) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1987}}
Nov 29 03:11:09 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-1afdf47c-9f51-4398-b7f1-e228ee2aba24 tempest-TestVolumeSwap-1089931716 tempest-TestVolumeSwap-1089931716-project-member] [instance: 3e355ae8-aab9-4ae9-b6e3-6de108ec1af9] Start _get_guest_xml network_info=[{"id": "b70722c1-452e-4b37-9ff3-7d8a1c7a9ba3", "address": "fa:16:3e:db:81:b1", "network": {"id": "67d2e671-98cd-4178-8f5f-1ee06dd66006", "bridge": "br-int", "label": "tempest-TestVolumeSwap-1472119110-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "cd4a0b5a414044d68524962183d47fca", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapb70722c1-45", "ovs_interfaceid": "b70722c1-452e-4b37-9ff3-7d8a1c7a9ba3", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum='c8fc807773e5354afe61636071771906',container_format='bare',created_at=2023-11-29T03:02:15Z,direct_url=<?>,disk_format='qcow2',id=96c30ba7-9b73-4a30-8ba4-7138879573b7,min_disk=0,min_ram=0,name='cirros-0.6.2-x86_64-disk',owner='b18e380921c748b78517b6f1211cd0f1',properties=ImageMetaProps,protected=<?>,size=21430272,status='active',tags=<?>,updated_at=2023-11-29T03:02:17Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [{'device_name': '/dev/vda', 'encryption_format': None, 'encrypted': False, 'device_type': 'disk', 'size': 0, 'disk_bus': 'virtio', 'boot_index': 0, 'encryption_options': None, 'encryption_secret_uuid': None, 'guest_format': None, 'image_id': '96c30ba7-9b73-4a30-8ba4-7138879573b7'}], 'ephemerals': [], 'block_device_mapping': [], 'swap': None} {{(pid=75234) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7562}}
Nov 29 03:11:09 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-1afdf47c-9f51-4398-b7f1-e228ee2aba24 tempest-TestVolumeSwap-1089931716 tempest-TestVolumeSwap-1089931716-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:11:09 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-1afdf47c-9f51-4398-b7f1-e228ee2aba24 tempest-TestVolumeSwap-1089931716 tempest-TestVolumeSwap-1089931716-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:11:09 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-1afdf47c-9f51-4398-b7f1-e228ee2aba24 tempest-TestVolumeSwap-1089931716 tempest-TestVolumeSwap-1089931716-project-member] Searching host: 'user' for CPU controller through CGroups V1... {{(pid=75234) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1652}}
Nov 29 03:11:09 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-1afdf47c-9f51-4398-b7f1-e228ee2aba24 tempest-TestVolumeSwap-1089931716 tempest-TestVolumeSwap-1089931716-project-member] CPU controller missing on host. {{(pid=75234) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1662}}
Nov 29 03:11:09 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-1afdf47c-9f51-4398-b7f1-e228ee2aba24 tempest-TestVolumeSwap-1089931716 tempest-TestVolumeSwap-1089931716-project-member] Searching host: 'user' for CPU controller through CGroups V2... {{(pid=75234) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1671}}
Nov 29 03:11:09 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-1afdf47c-9f51-4398-b7f1-e228ee2aba24 tempest-TestVolumeSwap-1089931716 tempest-TestVolumeSwap-1089931716-project-member] CPU controller found on host. {{(pid=75234) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1678}}
Nov 29 03:11:09 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-1afdf47c-9f51-4398-b7f1-e228ee2aba24 tempest-TestVolumeSwap-1089931716 tempest-TestVolumeSwap-1089931716-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=75234) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5400}}
Nov 29 03:11:09 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-1afdf47c-9f51-4398-b7f1-e228ee2aba24 tempest-TestVolumeSwap-1089931716 tempest-TestVolumeSwap-1089931716-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-11-29T03:02:04Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='1',id=6,is_public=True,memory_mb=512,name='m1.tiny',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='c8fc807773e5354afe61636071771906',container_format='bare',created_at=2023-11-29T03:02:15Z,direct_url=<?>,disk_format='qcow2',id=96c30ba7-9b73-4a30-8ba4-7138879573b7,min_disk=0,min_ram=0,name='cirros-0.6.2-x86_64-disk',owner='b18e380921c748b78517b6f1211cd0f1',properties=ImageMetaProps,protected=<?>,size=21430272,status='active',tags=<?>,updated_at=2023-11-29T03:02:17Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:563}}
Nov 29 03:11:09 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-1afdf47c-9f51-4398-b7f1-e228ee2aba24 tempest-TestVolumeSwap-1089931716 tempest-TestVolumeSwap-1089931716-project-member] Flavor limits 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:348}}
Nov 29 03:11:09 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-1afdf47c-9f51-4398-b7f1-e228ee2aba24 tempest-TestVolumeSwap-1089931716 tempest-TestVolumeSwap-1089931716-project-member] Image limits 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:352}}
Nov 29 03:11:09 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-1afdf47c-9f51-4398-b7f1-e228ee2aba24 tempest-TestVolumeSwap-1089931716 tempest-TestVolumeSwap-1089931716-project-member] Flavor pref 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:388}}
Nov 29 03:11:09 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-1afdf47c-9f51-4398-b7f1-e228ee2aba24 tempest-TestVolumeSwap-1089931716 tempest-TestVolumeSwap-1089931716-project-member] Image pref 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:392}}
Nov 29 03:11:09 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-1afdf47c-9f51-4398-b7f1-e228ee2aba24 tempest-TestVolumeSwap-1089931716 tempest-TestVolumeSwap-1089931716-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:430}}
Nov 29 03:11:09 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-1afdf47c-9f51-4398-b7f1-e228ee2aba24 tempest-TestVolumeSwap-1089931716 tempest-TestVolumeSwap-1089931716-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:569}}
Nov 29 03:11:09 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-1afdf47c-9f51-4398-b7f1-e228ee2aba24 tempest-TestVolumeSwap-1089931716 tempest-TestVolumeSwap-1089931716-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=75234) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:471}}
Nov 29 03:11:09 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-1afdf47c-9f51-4398-b7f1-e228ee2aba24 tempest-TestVolumeSwap-1089931716 tempest-TestVolumeSwap-1089931716-project-member] Got 1 possible topologies {{(pid=75234) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:501}}
Nov 29 03:11:09 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-1afdf47c-9f51-4398-b7f1-e228ee2aba24 tempest-TestVolumeSwap-1089931716 tempest-TestVolumeSwap-1089931716-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:575}}
Nov 29 03:11:09 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-1afdf47c-9f51-4398-b7f1-e228ee2aba24 tempest-TestVolumeSwap-1089931716 tempest-TestVolumeSwap-1089931716-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:577}}
Nov 29 03:11:09 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-1afdf47c-9f51-4398-b7f1-e228ee2aba24 tempest-TestVolumeSwap-1089931716 tempest-TestVolumeSwap-1089931716-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-11-29T03:11:05Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-TestVolumeSwap-server-1891489941',display_name='tempest-TestVolumeSwap-server-1891489941',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-testvolumeswap-server-1891489941',id=20,image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBErO5gpIWBBUnin4dWgri2kRPv0Oc5PIA/2FQHxsJbVhAZPCTRN7Vo3YWPw19dRA90B59GrDhvW0wQP5nRsc6TH8eYdjlY58N/MAEOrS3Qhl7HjnnODcwnB/z6MrUdlMZA==',key_name='tempest-keypair-238289792',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='cd4a0b5a414044d68524962183d47fca',ramdisk_id='',reservation_id='r-k7dv1o4o',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TestVolumeSwap-1089931716',owner_user_name='tempest-TestVolumeSwap-1089931716-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-11-29T03:11:07Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='5366feb605674f9a8c7f547ba2beb65c',uuid=3e355ae8-aab9-4ae9-b6e3-6de108ec1af9,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "b70722c1-452e-4b37-9ff3-7d8a1c7a9ba3", "address": "fa:16:3e:db:81:b1", "network": {"id": "67d2e671-98cd-4178-8f5f-1ee06dd66006", "bridge": "br-int", "label": "tempest-TestVolumeSwap-1472119110-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "cd4a0b5a414044d68524962183d47fca", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapb70722c1-45", "ovs_interfaceid": "b70722c1-452e-4b37-9ff3-7d8a1c7a9ba3", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=75234) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Nov 29 03:11:09 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-1afdf47c-9f51-4398-b7f1-e228ee2aba24 tempest-TestVolumeSwap-1089931716 tempest-TestVolumeSwap-1089931716-project-member] Converting VIF {"id": "b70722c1-452e-4b37-9ff3-7d8a1c7a9ba3", "address": "fa:16:3e:db:81:b1", "network": {"id": "67d2e671-98cd-4178-8f5f-1ee06dd66006", "bridge": "br-int", "label": "tempest-TestVolumeSwap-1472119110-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "cd4a0b5a414044d68524962183d47fca", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapb70722c1-45", "ovs_interfaceid": "b70722c1-452e-4b37-9ff3-7d8a1c7a9ba3", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:11:09 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-1afdf47c-9f51-4398-b7f1-e228ee2aba24 tempest-TestVolumeSwap-1089931716 tempest-TestVolumeSwap-1089931716-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:db:81:b1,bridge_name='br-int',has_traffic_filtering=True,id=b70722c1-452e-4b37-9ff3-7d8a1c7a9ba3,network=Network(67d2e671-98cd-4178-8f5f-1ee06dd66006),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapb70722c1-45') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:11:09 user nova-compute[75234]: DEBUG nova.objects.instance [None req-1afdf47c-9f51-4398-b7f1-e228ee2aba24 tempest-TestVolumeSwap-1089931716 tempest-TestVolumeSwap-1089931716-project-member] Lazy-loading 'pci_devices' on Instance uuid 3e355ae8-aab9-4ae9-b6e3-6de108ec1af9 {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:11:09 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-1afdf47c-9f51-4398-b7f1-e228ee2aba24 tempest-TestVolumeSwap-1089931716 tempest-TestVolumeSwap-1089931716-project-member] [instance: 3e355ae8-aab9-4ae9-b6e3-6de108ec1af9] End _get_guest_xml xml=<domain type="kvm">
Nov 29 03:11:09 user nova-compute[75234]:   <uuid>3e355ae8-aab9-4ae9-b6e3-6de108ec1af9</uuid>
Nov 29 03:11:09 user nova-compute[75234]:   <name>instance-00000014</name>
Nov 29 03:11:09 user nova-compute[75234]:   <memory>524288</memory>
Nov 29 03:11:09 user nova-compute[75234]:   <vcpu>1</vcpu>
Nov 29 03:11:09 user nova-compute[75234]:   <metadata>
Nov 29 03:11:09 user nova-compute[75234]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Nov 29 03:11:09 user nova-compute[75234]:       <nova:package version="0.0.0"/>
Nov 29 03:11:09 user nova-compute[75234]:       <nova:name>tempest-TestVolumeSwap-server-1891489941</nova:name>
Nov 29 03:11:09 user nova-compute[75234]:       <nova:creationTime>2023-11-29 03:11:09</nova:creationTime>
Nov 29 03:11:09 user nova-compute[75234]:       <nova:flavor name="m1.tiny">
Nov 29 03:11:09 user nova-compute[75234]:         <nova:memory>512</nova:memory>
Nov 29 03:11:09 user nova-compute[75234]:         <nova:disk>1</nova:disk>
Nov 29 03:11:09 user nova-compute[75234]:         <nova:swap>0</nova:swap>
Nov 29 03:11:09 user nova-compute[75234]:         <nova:ephemeral>0</nova:ephemeral>
Nov 29 03:11:09 user nova-compute[75234]:         <nova:vcpus>1</nova:vcpus>
Nov 29 03:11:09 user nova-compute[75234]:       </nova:flavor>
Nov 29 03:11:09 user nova-compute[75234]:       <nova:owner>
Nov 29 03:11:09 user nova-compute[75234]:         <nova:user uuid="5366feb605674f9a8c7f547ba2beb65c">tempest-TestVolumeSwap-1089931716-project-member</nova:user>
Nov 29 03:11:09 user nova-compute[75234]:         <nova:project uuid="cd4a0b5a414044d68524962183d47fca">tempest-TestVolumeSwap-1089931716</nova:project>
Nov 29 03:11:09 user nova-compute[75234]:       </nova:owner>
Nov 29 03:11:09 user nova-compute[75234]:       <nova:root type="image" uuid="96c30ba7-9b73-4a30-8ba4-7138879573b7"/>
Nov 29 03:11:09 user nova-compute[75234]:       <nova:ports>
Nov 29 03:11:09 user nova-compute[75234]:         <nova:port uuid="b70722c1-452e-4b37-9ff3-7d8a1c7a9ba3">
Nov 29 03:11:09 user nova-compute[75234]:           <nova:ip type="fixed" address="10.0.0.5" ipVersion="4"/>
Nov 29 03:11:09 user nova-compute[75234]:         </nova:port>
Nov 29 03:11:09 user nova-compute[75234]:       </nova:ports>
Nov 29 03:11:09 user nova-compute[75234]:     </nova:instance>
Nov 29 03:11:09 user nova-compute[75234]:   </metadata>
Nov 29 03:11:09 user nova-compute[75234]:   <sysinfo type="smbios">
Nov 29 03:11:09 user nova-compute[75234]:     <system>
Nov 29 03:11:09 user nova-compute[75234]:       <entry name="manufacturer">OpenStack Foundation</entry>
Nov 29 03:11:09 user nova-compute[75234]:       <entry name="product">OpenStack Nova</entry>
Nov 29 03:11:09 user nova-compute[75234]:       <entry name="version">0.0.0</entry>
Nov 29 03:11:09 user nova-compute[75234]:       <entry name="serial">3e355ae8-aab9-4ae9-b6e3-6de108ec1af9</entry>
Nov 29 03:11:09 user nova-compute[75234]:       <entry name="uuid">3e355ae8-aab9-4ae9-b6e3-6de108ec1af9</entry>
Nov 29 03:11:09 user nova-compute[75234]:       <entry name="family">Virtual Machine</entry>
Nov 29 03:11:09 user nova-compute[75234]:     </system>
Nov 29 03:11:09 user nova-compute[75234]:   </sysinfo>
Nov 29 03:11:09 user nova-compute[75234]:   <os>
Nov 29 03:11:09 user nova-compute[75234]:     <type arch="x86_64" machine="pc">hvm</type>
Nov 29 03:11:09 user nova-compute[75234]:     <boot dev="hd"/>
Nov 29 03:11:09 user nova-compute[75234]:     <smbios mode="sysinfo"/>
Nov 29 03:11:09 user nova-compute[75234]:   </os>
Nov 29 03:11:09 user nova-compute[75234]:   <features>
Nov 29 03:11:09 user nova-compute[75234]:     <acpi/>
Nov 29 03:11:09 user nova-compute[75234]:     <apic/>
Nov 29 03:11:09 user nova-compute[75234]:     <vmcoreinfo/>
Nov 29 03:11:09 user nova-compute[75234]:   </features>
Nov 29 03:11:09 user nova-compute[75234]:   <clock offset="utc">
Nov 29 03:11:09 user nova-compute[75234]:     <timer name="pit" tickpolicy="delay"/>
Nov 29 03:11:09 user nova-compute[75234]:     <timer name="rtc" tickpolicy="catchup"/>
Nov 29 03:11:09 user nova-compute[75234]:     <timer name="hpet" present="no"/>
Nov 29 03:11:09 user nova-compute[75234]:   </clock>
Nov 29 03:11:09 user nova-compute[75234]:   <cpu mode="custom" match="exact">
Nov 29 03:11:09 user nova-compute[75234]:     <model>Nehalem</model>
Nov 29 03:11:09 user nova-compute[75234]:     <topology sockets="1" cores="1" threads="1"/>
Nov 29 03:11:09 user nova-compute[75234]:   </cpu>
Nov 29 03:11:09 user nova-compute[75234]:   <devices>
Nov 29 03:11:09 user nova-compute[75234]:     <disk type="file" device="disk">
Nov 29 03:11:09 user nova-compute[75234]:       <driver name="qemu" type="qcow2" cache="none"/>
Nov 29 03:11:09 user nova-compute[75234]:       <source file="/opt/stack/data/nova/instances/3e355ae8-aab9-4ae9-b6e3-6de108ec1af9/disk"/>
Nov 29 03:11:09 user nova-compute[75234]:       <target dev="vda" bus="virtio"/>
Nov 29 03:11:09 user nova-compute[75234]:     </disk>
Nov 29 03:11:09 user nova-compute[75234]:     <interface type="ethernet">
Nov 29 03:11:09 user nova-compute[75234]:       <mac address="fa:16:3e:db:81:b1"/>
Nov 29 03:11:09 user nova-compute[75234]:       <model type="virtio"/>
Nov 29 03:11:09 user nova-compute[75234]:       <mtu size="1442"/>
Nov 29 03:11:09 user nova-compute[75234]:       <target dev="tapb70722c1-45"/>
Nov 29 03:11:09 user nova-compute[75234]:     </interface>
Nov 29 03:11:09 user nova-compute[75234]:     <serial type="pty">
Nov 29 03:11:09 user nova-compute[75234]:       <log file="/opt/stack/data/nova/instances/3e355ae8-aab9-4ae9-b6e3-6de108ec1af9/console.log" append="off"/>
Nov 29 03:11:09 user nova-compute[75234]:     </serial>
Nov 29 03:11:09 user nova-compute[75234]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Nov 29 03:11:09 user nova-compute[75234]:     <video>
Nov 29 03:11:09 user nova-compute[75234]:       <model type="virtio"/>
Nov 29 03:11:09 user nova-compute[75234]:     </video>
Nov 29 03:11:09 user nova-compute[75234]:     <rng model="virtio">
Nov 29 03:11:09 user nova-compute[75234]:       <backend model="random">/dev/urandom</backend>
Nov 29 03:11:09 user nova-compute[75234]:     </rng>
Nov 29 03:11:09 user nova-compute[75234]:     <controller type="usb" index="0" model="none"/>
Nov 29 03:11:09 user nova-compute[75234]:     <memballoon model="virtio">
Nov 29 03:11:09 user nova-compute[75234]:       <stats period="10"/>
Nov 29 03:11:09 user nova-compute[75234]:     </memballoon>
Nov 29 03:11:09 user nova-compute[75234]:   </devices>
Nov 29 03:11:09 user nova-compute[75234]: </domain>
Nov 29 03:11:09 user nova-compute[75234]:  {{(pid=75234) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7568}}
Nov 29 03:11:09 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-1afdf47c-9f51-4398-b7f1-e228ee2aba24 tempest-TestVolumeSwap-1089931716 tempest-TestVolumeSwap-1089931716-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-11-29T03:11:05Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-TestVolumeSwap-server-1891489941',display_name='tempest-TestVolumeSwap-server-1891489941',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-testvolumeswap-server-1891489941',id=20,image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBErO5gpIWBBUnin4dWgri2kRPv0Oc5PIA/2FQHxsJbVhAZPCTRN7Vo3YWPw19dRA90B59GrDhvW0wQP5nRsc6TH8eYdjlY58N/MAEOrS3Qhl7HjnnODcwnB/z6MrUdlMZA==',key_name='tempest-keypair-238289792',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='cd4a0b5a414044d68524962183d47fca',ramdisk_id='',reservation_id='r-k7dv1o4o',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TestVolumeSwap-1089931716',owner_user_name='tempest-TestVolumeSwap-1089931716-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-11-29T03:11:07Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='5366feb605674f9a8c7f547ba2beb65c',uuid=3e355ae8-aab9-4ae9-b6e3-6de108ec1af9,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "b70722c1-452e-4b37-9ff3-7d8a1c7a9ba3", "address": "fa:16:3e:db:81:b1", "network": {"id": "67d2e671-98cd-4178-8f5f-1ee06dd66006", "bridge": "br-int", "label": "tempest-TestVolumeSwap-1472119110-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "cd4a0b5a414044d68524962183d47fca", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapb70722c1-45", "ovs_interfaceid": "b70722c1-452e-4b37-9ff3-7d8a1c7a9ba3", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Nov 29 03:11:09 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-1afdf47c-9f51-4398-b7f1-e228ee2aba24 tempest-TestVolumeSwap-1089931716 tempest-TestVolumeSwap-1089931716-project-member] Converting VIF {"id": "b70722c1-452e-4b37-9ff3-7d8a1c7a9ba3", "address": "fa:16:3e:db:81:b1", "network": {"id": "67d2e671-98cd-4178-8f5f-1ee06dd66006", "bridge": "br-int", "label": "tempest-TestVolumeSwap-1472119110-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "cd4a0b5a414044d68524962183d47fca", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapb70722c1-45", "ovs_interfaceid": "b70722c1-452e-4b37-9ff3-7d8a1c7a9ba3", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:11:09 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-1afdf47c-9f51-4398-b7f1-e228ee2aba24 tempest-TestVolumeSwap-1089931716 tempest-TestVolumeSwap-1089931716-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:db:81:b1,bridge_name='br-int',has_traffic_filtering=True,id=b70722c1-452e-4b37-9ff3-7d8a1c7a9ba3,network=Network(67d2e671-98cd-4178-8f5f-1ee06dd66006),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapb70722c1-45') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:11:09 user nova-compute[75234]: DEBUG os_vif [None req-1afdf47c-9f51-4398-b7f1-e228ee2aba24 tempest-TestVolumeSwap-1089931716 tempest-TestVolumeSwap-1089931716-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:db:81:b1,bridge_name='br-int',has_traffic_filtering=True,id=b70722c1-452e-4b37-9ff3-7d8a1c7a9ba3,network=Network(67d2e671-98cd-4178-8f5f-1ee06dd66006),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapb70722c1-45') {{(pid=75234) plug /opt/stack/data/venv/lib/python3.10/site-packages/os_vif/__init__.py:76}}
Nov 29 03:11:09 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:11:09 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:11:09 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Nov 29 03:11:09 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-45c0b654-c277-4e79-b2de-2c89bf731fdf tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] CMD "qemu-img convert -t none -O qcow2 -f qcow2 /opt/stack/data/nova/instances/062753a6-8f39-4e80-a549-9f0f5a4b877a/disk /opt/stack/data/nova/instances/snapshots/tmplk18vv2m/158e85d2038d44809c5585f24eb14b9f" returned: 0 in 0.824s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:11:09 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:11:09 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbCreateCommand(_result=None, table=QoS, columns={'type': 'linux-noop', 'external_ids': {'id': '85e2c60c-bf1b-570e-8551-6c3a7cbb52dd', '_type': 'linux-noop'}}, row=False) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:11:09 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-45c0b654-c277-4e79-b2de-2c89bf731fdf tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] [instance: 062753a6-8f39-4e80-a549-9f0f5a4b877a] Snapshot extracted, beginning image upload
Nov 29 03:11:09 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:11:09 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:11:09 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:11:09 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:11:09 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tapb70722c1-45, may_exist=True, interface_attrs={}) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:11:09 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Port, record=tapb70722c1-45, col_values=(('qos', UUID('7aaeed91-7b6e-4ea2-a1f7-fc9e7bd8d604')),), if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:11:09 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DbSetCommand(_result=None, table=Interface, record=tapb70722c1-45, col_values=(('external_ids', {'iface-id': 'b70722c1-452e-4b37-9ff3-7d8a1c7a9ba3', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:db:81:b1', 'vm-uuid': '3e355ae8-aab9-4ae9-b6e3-6de108ec1af9'}),), if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:11:09 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:11:09 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:11:09 user nova-compute[75234]: INFO os_vif [None req-1afdf47c-9f51-4398-b7f1-e228ee2aba24 tempest-TestVolumeSwap-1089931716 tempest-TestVolumeSwap-1089931716-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:db:81:b1,bridge_name='br-int',has_traffic_filtering=True,id=b70722c1-452e-4b37-9ff3-7d8a1c7a9ba3,network=Network(67d2e671-98cd-4178-8f5f-1ee06dd66006),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapb70722c1-45')
Nov 29 03:11:09 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-1afdf47c-9f51-4398-b7f1-e228ee2aba24 tempest-TestVolumeSwap-1089931716 tempest-TestVolumeSwap-1089931716-project-member] No BDM found with device name vda, not building metadata. {{(pid=75234) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12089}}
Nov 29 03:11:09 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-1afdf47c-9f51-4398-b7f1-e228ee2aba24 tempest-TestVolumeSwap-1089931716 tempest-TestVolumeSwap-1089931716-project-member] No VIF found with MAC fa:16:3e:db:81:b1, not building metadata {{(pid=75234) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Nov 29 03:11:10 user nova-compute[75234]: DEBUG nova.compute.manager [req-6685ce0d-95cc-49a8-987b-9e81cb9d621b req-7cdc3c53-ac3b-4e9a-8a3b-168e5e8330f9 service nova] [instance: 062753a6-8f39-4e80-a549-9f0f5a4b877a] Received event network-vif-plugged-8f6751f1-fc65-4842-b77f-1dfe26912afb {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:11:10 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-6685ce0d-95cc-49a8-987b-9e81cb9d621b req-7cdc3c53-ac3b-4e9a-8a3b-168e5e8330f9 service nova] Acquiring lock "062753a6-8f39-4e80-a549-9f0f5a4b877a-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:11:10 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-6685ce0d-95cc-49a8-987b-9e81cb9d621b req-7cdc3c53-ac3b-4e9a-8a3b-168e5e8330f9 service nova] Lock "062753a6-8f39-4e80-a549-9f0f5a4b877a-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:11:10 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-6685ce0d-95cc-49a8-987b-9e81cb9d621b req-7cdc3c53-ac3b-4e9a-8a3b-168e5e8330f9 service nova] Lock "062753a6-8f39-4e80-a549-9f0f5a4b877a-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:11:10 user nova-compute[75234]: DEBUG nova.compute.manager [req-6685ce0d-95cc-49a8-987b-9e81cb9d621b req-7cdc3c53-ac3b-4e9a-8a3b-168e5e8330f9 service nova] [instance: 062753a6-8f39-4e80-a549-9f0f5a4b877a] No waiting events found dispatching network-vif-plugged-8f6751f1-fc65-4842-b77f-1dfe26912afb {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:11:10 user nova-compute[75234]: WARNING nova.compute.manager [req-6685ce0d-95cc-49a8-987b-9e81cb9d621b req-7cdc3c53-ac3b-4e9a-8a3b-168e5e8330f9 service nova] [instance: 062753a6-8f39-4e80-a549-9f0f5a4b877a] Received unexpected event network-vif-plugged-8f6751f1-fc65-4842-b77f-1dfe26912afb for instance with vm_state active and task_state shelving_image_uploading.
Nov 29 03:11:11 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:11:11 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:11:11 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:11:11 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:11:11 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:11:11 user nova-compute[75234]: DEBUG nova.compute.manager [req-e768c0cb-fb3b-4f00-9cf0-8ffd890d1939 req-35eb53e9-bcfb-4685-be59-fa6057fbde16 service nova] [instance: 3e355ae8-aab9-4ae9-b6e3-6de108ec1af9] Received event network-vif-plugged-b70722c1-452e-4b37-9ff3-7d8a1c7a9ba3 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:11:11 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-e768c0cb-fb3b-4f00-9cf0-8ffd890d1939 req-35eb53e9-bcfb-4685-be59-fa6057fbde16 service nova] Acquiring lock "3e355ae8-aab9-4ae9-b6e3-6de108ec1af9-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:11:11 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-e768c0cb-fb3b-4f00-9cf0-8ffd890d1939 req-35eb53e9-bcfb-4685-be59-fa6057fbde16 service nova] Lock "3e355ae8-aab9-4ae9-b6e3-6de108ec1af9-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:11:11 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-e768c0cb-fb3b-4f00-9cf0-8ffd890d1939 req-35eb53e9-bcfb-4685-be59-fa6057fbde16 service nova] Lock "3e355ae8-aab9-4ae9-b6e3-6de108ec1af9-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:11:11 user nova-compute[75234]: DEBUG nova.compute.manager [req-e768c0cb-fb3b-4f00-9cf0-8ffd890d1939 req-35eb53e9-bcfb-4685-be59-fa6057fbde16 service nova] [instance: 3e355ae8-aab9-4ae9-b6e3-6de108ec1af9] No waiting events found dispatching network-vif-plugged-b70722c1-452e-4b37-9ff3-7d8a1c7a9ba3 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:11:11 user nova-compute[75234]: WARNING nova.compute.manager [req-e768c0cb-fb3b-4f00-9cf0-8ffd890d1939 req-35eb53e9-bcfb-4685-be59-fa6057fbde16 service nova] [instance: 3e355ae8-aab9-4ae9-b6e3-6de108ec1af9] Received unexpected event network-vif-plugged-b70722c1-452e-4b37-9ff3-7d8a1c7a9ba3 for instance with vm_state building and task_state spawning.
Nov 29 03:11:12 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:11:12 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:11:12 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:11:12 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:11:12 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:11:13 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-45c0b654-c277-4e79-b2de-2c89bf731fdf tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] [instance: 062753a6-8f39-4e80-a549-9f0f5a4b877a] Snapshot image upload complete
Nov 29 03:11:13 user nova-compute[75234]: DEBUG nova.compute.manager [None req-45c0b654-c277-4e79-b2de-2c89bf731fdf tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] [instance: 062753a6-8f39-4e80-a549-9f0f5a4b877a] Checking state {{(pid=75234) _get_power_state /opt/stack/nova/nova/compute/manager.py:1782}}
Nov 29 03:11:13 user nova-compute[75234]: INFO nova.compute.manager [None req-45c0b654-c277-4e79-b2de-2c89bf731fdf tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] [instance: 062753a6-8f39-4e80-a549-9f0f5a4b877a] Shelve offloading
Nov 29 03:11:13 user nova-compute[75234]: INFO nova.virt.libvirt.driver [-] [instance: 062753a6-8f39-4e80-a549-9f0f5a4b877a] Instance destroyed successfully.
Nov 29 03:11:13 user nova-compute[75234]: DEBUG nova.compute.manager [None req-45c0b654-c277-4e79-b2de-2c89bf731fdf tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] [instance: 062753a6-8f39-4e80-a549-9f0f5a4b877a] Checking state {{(pid=75234) _get_power_state /opt/stack/nova/nova/compute/manager.py:1782}}
Nov 29 03:11:13 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-45c0b654-c277-4e79-b2de-2c89bf731fdf tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Acquiring lock "refresh_cache-062753a6-8f39-4e80-a549-9f0f5a4b877a" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:11:13 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-45c0b654-c277-4e79-b2de-2c89bf731fdf tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Acquired lock "refresh_cache-062753a6-8f39-4e80-a549-9f0f5a4b877a" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:11:13 user nova-compute[75234]: DEBUG nova.network.neutron [None req-45c0b654-c277-4e79-b2de-2c89bf731fdf tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] [instance: 062753a6-8f39-4e80-a549-9f0f5a4b877a] Building network info cache for instance {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2009}}
Nov 29 03:11:13 user nova-compute[75234]: DEBUG nova.compute.manager [None req-1afdf47c-9f51-4398-b7f1-e228ee2aba24 tempest-TestVolumeSwap-1089931716 tempest-TestVolumeSwap-1089931716-project-member] [instance: 3e355ae8-aab9-4ae9-b6e3-6de108ec1af9] Instance event wait completed in 0 seconds for  {{(pid=75234) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Nov 29 03:11:13 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-1afdf47c-9f51-4398-b7f1-e228ee2aba24 tempest-TestVolumeSwap-1089931716 tempest-TestVolumeSwap-1089931716-project-member] [instance: 3e355ae8-aab9-4ae9-b6e3-6de108ec1af9] Guest created on hypervisor {{(pid=75234) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4421}}
Nov 29 03:11:13 user nova-compute[75234]: INFO nova.virt.libvirt.driver [-] [instance: 3e355ae8-aab9-4ae9-b6e3-6de108ec1af9] Instance spawned successfully.
Nov 29 03:11:13 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-1afdf47c-9f51-4398-b7f1-e228ee2aba24 tempest-TestVolumeSwap-1089931716 tempest-TestVolumeSwap-1089931716-project-member] [instance: 3e355ae8-aab9-4ae9-b6e3-6de108ec1af9] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:904}}
Nov 29 03:11:13 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-1afdf47c-9f51-4398-b7f1-e228ee2aba24 tempest-TestVolumeSwap-1089931716 tempest-TestVolumeSwap-1089931716-project-member] [instance: 3e355ae8-aab9-4ae9-b6e3-6de108ec1af9] Found default for hw_cdrom_bus of ide {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:11:13 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-1afdf47c-9f51-4398-b7f1-e228ee2aba24 tempest-TestVolumeSwap-1089931716 tempest-TestVolumeSwap-1089931716-project-member] [instance: 3e355ae8-aab9-4ae9-b6e3-6de108ec1af9] Found default for hw_disk_bus of virtio {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:11:13 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-1afdf47c-9f51-4398-b7f1-e228ee2aba24 tempest-TestVolumeSwap-1089931716 tempest-TestVolumeSwap-1089931716-project-member] [instance: 3e355ae8-aab9-4ae9-b6e3-6de108ec1af9] Found default for hw_input_bus of None {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:11:13 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-1afdf47c-9f51-4398-b7f1-e228ee2aba24 tempest-TestVolumeSwap-1089931716 tempest-TestVolumeSwap-1089931716-project-member] [instance: 3e355ae8-aab9-4ae9-b6e3-6de108ec1af9] Found default for hw_pointer_model of None {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:11:13 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-1afdf47c-9f51-4398-b7f1-e228ee2aba24 tempest-TestVolumeSwap-1089931716 tempest-TestVolumeSwap-1089931716-project-member] [instance: 3e355ae8-aab9-4ae9-b6e3-6de108ec1af9] Found default for hw_video_model of virtio {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:11:13 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-1afdf47c-9f51-4398-b7f1-e228ee2aba24 tempest-TestVolumeSwap-1089931716 tempest-TestVolumeSwap-1089931716-project-member] [instance: 3e355ae8-aab9-4ae9-b6e3-6de108ec1af9] Found default for hw_vif_model of virtio {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:11:13 user nova-compute[75234]: INFO nova.compute.manager [None req-1afdf47c-9f51-4398-b7f1-e228ee2aba24 tempest-TestVolumeSwap-1089931716 tempest-TestVolumeSwap-1089931716-project-member] [instance: 3e355ae8-aab9-4ae9-b6e3-6de108ec1af9] Took 6.01 seconds to spawn the instance on the hypervisor.
Nov 29 03:11:13 user nova-compute[75234]: DEBUG nova.compute.manager [None req-1afdf47c-9f51-4398-b7f1-e228ee2aba24 tempest-TestVolumeSwap-1089931716 tempest-TestVolumeSwap-1089931716-project-member] [instance: 3e355ae8-aab9-4ae9-b6e3-6de108ec1af9] Checking state {{(pid=75234) _get_power_state /opt/stack/nova/nova/compute/manager.py:1782}}
Nov 29 03:11:13 user nova-compute[75234]: DEBUG nova.compute.manager [req-08a0ee37-7623-4c8f-9a5d-fd55106ed46e req-66829266-1258-4a55-9bf3-79fb100cb7df service nova] [instance: 3e355ae8-aab9-4ae9-b6e3-6de108ec1af9] Received event network-vif-plugged-b70722c1-452e-4b37-9ff3-7d8a1c7a9ba3 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:11:13 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-08a0ee37-7623-4c8f-9a5d-fd55106ed46e req-66829266-1258-4a55-9bf3-79fb100cb7df service nova] Acquiring lock "3e355ae8-aab9-4ae9-b6e3-6de108ec1af9-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:11:13 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-08a0ee37-7623-4c8f-9a5d-fd55106ed46e req-66829266-1258-4a55-9bf3-79fb100cb7df service nova] Lock "3e355ae8-aab9-4ae9-b6e3-6de108ec1af9-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:11:13 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-08a0ee37-7623-4c8f-9a5d-fd55106ed46e req-66829266-1258-4a55-9bf3-79fb100cb7df service nova] Lock "3e355ae8-aab9-4ae9-b6e3-6de108ec1af9-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:11:13 user nova-compute[75234]: DEBUG nova.compute.manager [req-08a0ee37-7623-4c8f-9a5d-fd55106ed46e req-66829266-1258-4a55-9bf3-79fb100cb7df service nova] [instance: 3e355ae8-aab9-4ae9-b6e3-6de108ec1af9] No waiting events found dispatching network-vif-plugged-b70722c1-452e-4b37-9ff3-7d8a1c7a9ba3 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:11:13 user nova-compute[75234]: WARNING nova.compute.manager [req-08a0ee37-7623-4c8f-9a5d-fd55106ed46e req-66829266-1258-4a55-9bf3-79fb100cb7df service nova] [instance: 3e355ae8-aab9-4ae9-b6e3-6de108ec1af9] Received unexpected event network-vif-plugged-b70722c1-452e-4b37-9ff3-7d8a1c7a9ba3 for instance with vm_state building and task_state spawning.
Nov 29 03:11:13 user nova-compute[75234]: INFO nova.compute.manager [None req-1afdf47c-9f51-4398-b7f1-e228ee2aba24 tempest-TestVolumeSwap-1089931716 tempest-TestVolumeSwap-1089931716-project-member] [instance: 3e355ae8-aab9-4ae9-b6e3-6de108ec1af9] Took 7.25 seconds to build instance.
Nov 29 03:11:13 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-1afdf47c-9f51-4398-b7f1-e228ee2aba24 tempest-TestVolumeSwap-1089931716 tempest-TestVolumeSwap-1089931716-project-member] Lock "3e355ae8-aab9-4ae9-b6e3-6de108ec1af9" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 7.360s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:11:13 user nova-compute[75234]: DEBUG nova.network.neutron [None req-45c0b654-c277-4e79-b2de-2c89bf731fdf tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] [instance: 062753a6-8f39-4e80-a549-9f0f5a4b877a] Updating instance_info_cache with network_info: [{"id": "8f6751f1-fc65-4842-b77f-1dfe26912afb", "address": "fa:16:3e:62:4a:5a", "network": {"id": "aad085f7-106b-4de6-93f5-f86bb66ec962", "bridge": "br-int", "label": "tempest-AttachVolumeShelveTestJSON-775984268-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.175", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "e17150005a9a449f9373b91e1c00083b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap8f6751f1-fc", "ovs_interfaceid": "8f6751f1-fc65-4842-b77f-1dfe26912afb", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:11:13 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-45c0b654-c277-4e79-b2de-2c89bf731fdf tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Releasing lock "refresh_cache-062753a6-8f39-4e80-a549-9f0f5a4b877a" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:11:14 user nova-compute[75234]: INFO nova.virt.libvirt.driver [-] [instance: 062753a6-8f39-4e80-a549-9f0f5a4b877a] Instance destroyed successfully.
Nov 29 03:11:14 user nova-compute[75234]: DEBUG nova.objects.instance [None req-45c0b654-c277-4e79-b2de-2c89bf731fdf tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Lazy-loading 'resources' on Instance uuid 062753a6-8f39-4e80-a549-9f0f5a4b877a {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:11:14 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-45c0b654-c277-4e79-b2de-2c89bf731fdf tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-11-29T03:09:11Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description=None,display_name='tempest-AttachVolumeShelveTestJSON-server-2002337175',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-attachvolumeshelvetestjson-server-2002337175',id=14,image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBJL1BPeb1pVpN5A24HF0gsy7slhuswFxR+GxX0R0LvMX9QS8GL1hvY7OFzvrLw6S/VLmgqgKQ7HFXhrJDiWyf/i/jTyWtVuorLQtfVudKMIFZjmVfY2JDvM7kwU/YbL6jQ==',key_name='tempest-keypair-1961974262',keypairs=<?>,launch_index=0,launched_at=2023-11-29T03:09:20Z,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=<?>,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=4,progress=0,project_id='e17150005a9a449f9373b91e1c00083b',ramdisk_id='',reservation_id='r-p3yfidye',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-AttachVolumeShelveTestJSON-1180535246',owner_user_name='tempest-AttachVolumeShelveTestJSON-1180535246-project-member',shelved_at='2023-11-29T03:11:13.240549',shelved_host='user',shelved_image_id='b14da3aa-0b00-47e2-a40c-9736f7ce03a1'},tags=<?>,task_state='shelving_offloading',terminated_at=None,trusted_certs=<?>,updated_at=2023-11-29T03:11:10Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='154f3f4f57514450baf8f22d67768bc9',uuid=062753a6-8f39-4e80-a549-9f0f5a4b877a,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='shelved') vif={"id": "8f6751f1-fc65-4842-b77f-1dfe26912afb", "address": "fa:16:3e:62:4a:5a", "network": {"id": "aad085f7-106b-4de6-93f5-f86bb66ec962", "bridge": "br-int", "label": "tempest-AttachVolumeShelveTestJSON-775984268-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.175", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "e17150005a9a449f9373b91e1c00083b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap8f6751f1-fc", "ovs_interfaceid": "8f6751f1-fc65-4842-b77f-1dfe26912afb", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Nov 29 03:11:14 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-45c0b654-c277-4e79-b2de-2c89bf731fdf tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Converting VIF {"id": "8f6751f1-fc65-4842-b77f-1dfe26912afb", "address": "fa:16:3e:62:4a:5a", "network": {"id": "aad085f7-106b-4de6-93f5-f86bb66ec962", "bridge": "br-int", "label": "tempest-AttachVolumeShelveTestJSON-775984268-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.175", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "e17150005a9a449f9373b91e1c00083b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap8f6751f1-fc", "ovs_interfaceid": "8f6751f1-fc65-4842-b77f-1dfe26912afb", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:11:14 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-45c0b654-c277-4e79-b2de-2c89bf731fdf tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:62:4a:5a,bridge_name='br-int',has_traffic_filtering=True,id=8f6751f1-fc65-4842-b77f-1dfe26912afb,network=Network(aad085f7-106b-4de6-93f5-f86bb66ec962),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap8f6751f1-fc') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:11:14 user nova-compute[75234]: DEBUG os_vif [None req-45c0b654-c277-4e79-b2de-2c89bf731fdf tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Unplugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:62:4a:5a,bridge_name='br-int',has_traffic_filtering=True,id=8f6751f1-fc65-4842-b77f-1dfe26912afb,network=Network(aad085f7-106b-4de6-93f5-f86bb66ec962),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap8f6751f1-fc') {{(pid=75234) unplug /opt/stack/data/venv/lib/python3.10/site-packages/os_vif/__init__.py:109}}
Nov 29 03:11:14 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:11:14 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tap8f6751f1-fc, bridge=br-int, if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:11:14 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:11:14 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:11:14 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:11:14 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbDestroyCommand(_result=None, table=QoS, record=ee6d3ae0-8482-47cf-9121-ee12da13da9d) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:11:14 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:11:14 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:11:14 user nova-compute[75234]: INFO os_vif [None req-45c0b654-c277-4e79-b2de-2c89bf731fdf tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Successfully unplugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:62:4a:5a,bridge_name='br-int',has_traffic_filtering=True,id=8f6751f1-fc65-4842-b77f-1dfe26912afb,network=Network(aad085f7-106b-4de6-93f5-f86bb66ec962),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap8f6751f1-fc')
Nov 29 03:11:14 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-45c0b654-c277-4e79-b2de-2c89bf731fdf tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] [instance: 062753a6-8f39-4e80-a549-9f0f5a4b877a] Deleting instance files /opt/stack/data/nova/instances/062753a6-8f39-4e80-a549-9f0f5a4b877a_del
Nov 29 03:11:14 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-45c0b654-c277-4e79-b2de-2c89bf731fdf tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] [instance: 062753a6-8f39-4e80-a549-9f0f5a4b877a] Deletion of /opt/stack/data/nova/instances/062753a6-8f39-4e80-a549-9f0f5a4b877a_del complete
Nov 29 03:11:14 user nova-compute[75234]: DEBUG nova.compute.manager [req-48222a73-1c37-4175-b210-3df32a90da44 req-645aec9c-0b71-4f34-95d2-68e85ff0fdea service nova] [instance: 062753a6-8f39-4e80-a549-9f0f5a4b877a] Received event network-changed-8f6751f1-fc65-4842-b77f-1dfe26912afb {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:11:14 user nova-compute[75234]: DEBUG nova.compute.manager [req-48222a73-1c37-4175-b210-3df32a90da44 req-645aec9c-0b71-4f34-95d2-68e85ff0fdea service nova] [instance: 062753a6-8f39-4e80-a549-9f0f5a4b877a] Refreshing instance network info cache due to event network-changed-8f6751f1-fc65-4842-b77f-1dfe26912afb. {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11117}}
Nov 29 03:11:14 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-48222a73-1c37-4175-b210-3df32a90da44 req-645aec9c-0b71-4f34-95d2-68e85ff0fdea service nova] Acquiring lock "refresh_cache-062753a6-8f39-4e80-a549-9f0f5a4b877a" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:11:14 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-48222a73-1c37-4175-b210-3df32a90da44 req-645aec9c-0b71-4f34-95d2-68e85ff0fdea service nova] Acquired lock "refresh_cache-062753a6-8f39-4e80-a549-9f0f5a4b877a" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:11:14 user nova-compute[75234]: DEBUG nova.network.neutron [req-48222a73-1c37-4175-b210-3df32a90da44 req-645aec9c-0b71-4f34-95d2-68e85ff0fdea service nova] [instance: 062753a6-8f39-4e80-a549-9f0f5a4b877a] Refreshing network info cache for port 8f6751f1-fc65-4842-b77f-1dfe26912afb {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2006}}
Nov 29 03:11:14 user nova-compute[75234]: INFO nova.scheduler.client.report [None req-45c0b654-c277-4e79-b2de-2c89bf731fdf tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Deleted allocations for instance 062753a6-8f39-4e80-a549-9f0f5a4b877a
Nov 29 03:11:14 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-45c0b654-c277-4e79-b2de-2c89bf731fdf tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:11:14 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-45c0b654-c277-4e79-b2de-2c89bf731fdf tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:11:14 user nova-compute[75234]: DEBUG nova.objects.instance [None req-45c0b654-c277-4e79-b2de-2c89bf731fdf tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Lazy-loading 'numa_topology' on Instance uuid 062753a6-8f39-4e80-a549-9f0f5a4b877a {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:11:14 user nova-compute[75234]: DEBUG nova.objects.base [None req-45c0b654-c277-4e79-b2de-2c89bf731fdf tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Object Instance<062753a6-8f39-4e80-a549-9f0f5a4b877a> lazy-loaded attributes: resources,numa_topology {{(pid=75234) wrapper /opt/stack/nova/nova/objects/base.py:126}}
Nov 29 03:11:15 user nova-compute[75234]: DEBUG nova.network.neutron [req-48222a73-1c37-4175-b210-3df32a90da44 req-645aec9c-0b71-4f34-95d2-68e85ff0fdea service nova] [instance: 062753a6-8f39-4e80-a549-9f0f5a4b877a] Updated VIF entry in instance network info cache for port 8f6751f1-fc65-4842-b77f-1dfe26912afb. {{(pid=75234) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3481}}
Nov 29 03:11:15 user nova-compute[75234]: DEBUG nova.network.neutron [req-48222a73-1c37-4175-b210-3df32a90da44 req-645aec9c-0b71-4f34-95d2-68e85ff0fdea service nova] [instance: 062753a6-8f39-4e80-a549-9f0f5a4b877a] Updating instance_info_cache with network_info: [{"id": "8f6751f1-fc65-4842-b77f-1dfe26912afb", "address": "fa:16:3e:62:4a:5a", "network": {"id": "aad085f7-106b-4de6-93f5-f86bb66ec962", "bridge": null, "label": "tempest-AttachVolumeShelveTestJSON-775984268-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.175", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "e17150005a9a449f9373b91e1c00083b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "unbound", "details": {}, "devname": "tap8f6751f1-fc", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:11:15 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-48222a73-1c37-4175-b210-3df32a90da44 req-645aec9c-0b71-4f34-95d2-68e85ff0fdea service nova] Releasing lock "refresh_cache-062753a6-8f39-4e80-a549-9f0f5a4b877a" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:11:15 user nova-compute[75234]: DEBUG nova.compute.provider_tree [None req-45c0b654-c277-4e79-b2de-2c89bf731fdf tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Inventory has not changed in ProviderTree for provider: 4d291220-7c37-4839-a0e3-b23c66502b74 {{(pid=75234) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Nov 29 03:11:15 user nova-compute[75234]: DEBUG nova.scheduler.client.report [None req-45c0b654-c277-4e79-b2de-2c89bf731fdf tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Inventory has not changed for provider 4d291220-7c37-4839-a0e3-b23c66502b74 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=75234) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Nov 29 03:11:15 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-45c0b654-c277-4e79-b2de-2c89bf731fdf tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.716s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:11:15 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-45c0b654-c277-4e79-b2de-2c89bf731fdf tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Lock "062753a6-8f39-4e80-a549-9f0f5a4b877a" "released" by "nova.compute.manager.ComputeManager.shelve_instance.<locals>.do_shelve_instance" :: held 7.903s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:11:16 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:11:19 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-12833e69-8a87-4891-b279-991727f33b3c tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] Acquiring lock "f7d0bdeb-6a05-47ac-87f2-c8380a9a4286" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:11:19 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-12833e69-8a87-4891-b279-991727f33b3c tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] Lock "f7d0bdeb-6a05-47ac-87f2-c8380a9a4286" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:11:19 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-12833e69-8a87-4891-b279-991727f33b3c tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] Acquiring lock "f7d0bdeb-6a05-47ac-87f2-c8380a9a4286-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:11:19 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-12833e69-8a87-4891-b279-991727f33b3c tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] Lock "f7d0bdeb-6a05-47ac-87f2-c8380a9a4286-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:11:19 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-12833e69-8a87-4891-b279-991727f33b3c tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] Lock "f7d0bdeb-6a05-47ac-87f2-c8380a9a4286-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:11:19 user nova-compute[75234]: INFO nova.compute.manager [None req-12833e69-8a87-4891-b279-991727f33b3c tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] [instance: f7d0bdeb-6a05-47ac-87f2-c8380a9a4286] Terminating instance
Nov 29 03:11:19 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:11:19 user nova-compute[75234]: DEBUG nova.compute.manager [None req-12833e69-8a87-4891-b279-991727f33b3c tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] [instance: f7d0bdeb-6a05-47ac-87f2-c8380a9a4286] Start destroying the instance on the hypervisor. {{(pid=75234) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3140}}
Nov 29 03:11:19 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:11:19 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:11:19 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:11:19 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:11:19 user nova-compute[75234]: DEBUG nova.compute.manager [req-6e0542bd-d421-49ce-a94c-e361d5d7d840 req-9a3107bd-d654-4655-a69a-ed021dc0544e service nova] [instance: f7d0bdeb-6a05-47ac-87f2-c8380a9a4286] Received event network-vif-unplugged-b5192c04-0323-48b8-a326-ddf6b21f9019 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:11:19 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-6e0542bd-d421-49ce-a94c-e361d5d7d840 req-9a3107bd-d654-4655-a69a-ed021dc0544e service nova] Acquiring lock "f7d0bdeb-6a05-47ac-87f2-c8380a9a4286-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:11:19 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-6e0542bd-d421-49ce-a94c-e361d5d7d840 req-9a3107bd-d654-4655-a69a-ed021dc0544e service nova] Lock "f7d0bdeb-6a05-47ac-87f2-c8380a9a4286-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:11:19 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-6e0542bd-d421-49ce-a94c-e361d5d7d840 req-9a3107bd-d654-4655-a69a-ed021dc0544e service nova] Lock "f7d0bdeb-6a05-47ac-87f2-c8380a9a4286-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:11:19 user nova-compute[75234]: DEBUG nova.compute.manager [req-6e0542bd-d421-49ce-a94c-e361d5d7d840 req-9a3107bd-d654-4655-a69a-ed021dc0544e service nova] [instance: f7d0bdeb-6a05-47ac-87f2-c8380a9a4286] No waiting events found dispatching network-vif-unplugged-b5192c04-0323-48b8-a326-ddf6b21f9019 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:11:19 user nova-compute[75234]: DEBUG nova.compute.manager [req-6e0542bd-d421-49ce-a94c-e361d5d7d840 req-9a3107bd-d654-4655-a69a-ed021dc0544e service nova] [instance: f7d0bdeb-6a05-47ac-87f2-c8380a9a4286] Received event network-vif-unplugged-b5192c04-0323-48b8-a326-ddf6b21f9019 for instance with task_state deleting. {{(pid=75234) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10890}}
Nov 29 03:11:20 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:11:20 user nova-compute[75234]: INFO nova.virt.libvirt.driver [-] [instance: f7d0bdeb-6a05-47ac-87f2-c8380a9a4286] Instance destroyed successfully.
Nov 29 03:11:20 user nova-compute[75234]: DEBUG nova.objects.instance [None req-12833e69-8a87-4891-b279-991727f33b3c tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] Lazy-loading 'resources' on Instance uuid f7d0bdeb-6a05-47ac-87f2-c8380a9a4286 {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:11:20 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-12833e69-8a87-4891-b279-991727f33b3c tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-11-29T03:07:20Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-TestServerVolumeAttachmentScenario-server-2022761297',display_name='tempest-TestServerVolumeAttachmentScenario-server-2022761297',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-testservervolumeattachmentscenario-server-2022761297',id=12,image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBLSetanuoGY87fz1sA6LqyoFxZXw/IT2g+PnGJrAk3M0CxyfGSx2WYx6YjLRXYZ+sLnP2egb5hufwvhkiC775qvjKUtZ+ivGOwDpgIHPbDOr0aWZsSvtPkGBAHDATSfNHg==',key_name='tempest-keypair-645674634',keypairs=<?>,launch_index=0,launched_at=2023-11-29T03:07:26Z,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='b793707804a24961968ac432a37d9581',ramdisk_id='',reservation_id='r-7jpogwhz',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-TestServerVolumeAttachmentScenario-1219797116',owner_user_name='tempest-TestServerVolumeAttachmentScenario-1219797116-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-11-29T03:07:27Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='36692f8f1e9649c79183c447666364e2',uuid=f7d0bdeb-6a05-47ac-87f2-c8380a9a4286,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "b5192c04-0323-48b8-a326-ddf6b21f9019", "address": "fa:16:3e:59:dd:b5", "network": {"id": "22b1262f-108b-45c1-91cb-a0f77f9bfccf", "bridge": "br-int", "label": "tempest-TestServerVolumeAttachmentScenario-1154356954-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.139", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "b793707804a24961968ac432a37d9581", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapb5192c04-03", "ovs_interfaceid": "b5192c04-0323-48b8-a326-ddf6b21f9019", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Nov 29 03:11:20 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-12833e69-8a87-4891-b279-991727f33b3c tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] Converting VIF {"id": "b5192c04-0323-48b8-a326-ddf6b21f9019", "address": "fa:16:3e:59:dd:b5", "network": {"id": "22b1262f-108b-45c1-91cb-a0f77f9bfccf", "bridge": "br-int", "label": "tempest-TestServerVolumeAttachmentScenario-1154356954-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.139", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "b793707804a24961968ac432a37d9581", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapb5192c04-03", "ovs_interfaceid": "b5192c04-0323-48b8-a326-ddf6b21f9019", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:11:20 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-12833e69-8a87-4891-b279-991727f33b3c tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:59:dd:b5,bridge_name='br-int',has_traffic_filtering=True,id=b5192c04-0323-48b8-a326-ddf6b21f9019,network=Network(22b1262f-108b-45c1-91cb-a0f77f9bfccf),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapb5192c04-03') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:11:20 user nova-compute[75234]: DEBUG os_vif [None req-12833e69-8a87-4891-b279-991727f33b3c tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] Unplugging vif VIFOpenVSwitch(active=True,address=fa:16:3e:59:dd:b5,bridge_name='br-int',has_traffic_filtering=True,id=b5192c04-0323-48b8-a326-ddf6b21f9019,network=Network(22b1262f-108b-45c1-91cb-a0f77f9bfccf),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapb5192c04-03') {{(pid=75234) unplug /opt/stack/data/venv/lib/python3.10/site-packages/os_vif/__init__.py:109}}
Nov 29 03:11:20 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:11:20 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tapb5192c04-03, bridge=br-int, if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:11:20 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:11:20 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:11:20 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:11:20 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbDestroyCommand(_result=None, table=QoS, record=53b88442-2bba-4191-8bad-c0ea61ad9462) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:11:20 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:11:20 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:11:20 user nova-compute[75234]: INFO os_vif [None req-12833e69-8a87-4891-b279-991727f33b3c tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] Successfully unplugged vif VIFOpenVSwitch(active=True,address=fa:16:3e:59:dd:b5,bridge_name='br-int',has_traffic_filtering=True,id=b5192c04-0323-48b8-a326-ddf6b21f9019,network=Network(22b1262f-108b-45c1-91cb-a0f77f9bfccf),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapb5192c04-03')
Nov 29 03:11:20 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-12833e69-8a87-4891-b279-991727f33b3c tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] [instance: f7d0bdeb-6a05-47ac-87f2-c8380a9a4286] Deleting instance files /opt/stack/data/nova/instances/f7d0bdeb-6a05-47ac-87f2-c8380a9a4286_del
Nov 29 03:11:20 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-12833e69-8a87-4891-b279-991727f33b3c tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] [instance: f7d0bdeb-6a05-47ac-87f2-c8380a9a4286] Deletion of /opt/stack/data/nova/instances/f7d0bdeb-6a05-47ac-87f2-c8380a9a4286_del complete
Nov 29 03:11:20 user nova-compute[75234]: INFO nova.compute.manager [None req-12833e69-8a87-4891-b279-991727f33b3c tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] [instance: f7d0bdeb-6a05-47ac-87f2-c8380a9a4286] Took 0.92 seconds to destroy the instance on the hypervisor.
Nov 29 03:11:20 user nova-compute[75234]: DEBUG oslo.service.loopingcall [None req-12833e69-8a87-4891-b279-991727f33b3c tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=75234) func /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/loopingcall.py:435}}
Nov 29 03:11:20 user nova-compute[75234]: DEBUG nova.compute.manager [-] [instance: f7d0bdeb-6a05-47ac-87f2-c8380a9a4286] Deallocating network for instance {{(pid=75234) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2279}}
Nov 29 03:11:20 user nova-compute[75234]: DEBUG nova.network.neutron [-] [instance: f7d0bdeb-6a05-47ac-87f2-c8380a9a4286] deallocate_for_instance() {{(pid=75234) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1802}}
Nov 29 03:11:21 user nova-compute[75234]: DEBUG nova.network.neutron [-] [instance: f7d0bdeb-6a05-47ac-87f2-c8380a9a4286] Updating instance_info_cache with network_info: [] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:11:21 user nova-compute[75234]: INFO nova.compute.manager [-] [instance: f7d0bdeb-6a05-47ac-87f2-c8380a9a4286] Took 0.89 seconds to deallocate network for instance.
Nov 29 03:11:21 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:11:21 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-12833e69-8a87-4891-b279-991727f33b3c tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:11:21 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-12833e69-8a87-4891-b279-991727f33b3c tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:11:21 user nova-compute[75234]: DEBUG nova.compute.provider_tree [None req-12833e69-8a87-4891-b279-991727f33b3c tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] Inventory has not changed in ProviderTree for provider: 4d291220-7c37-4839-a0e3-b23c66502b74 {{(pid=75234) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Nov 29 03:11:21 user nova-compute[75234]: DEBUG nova.scheduler.client.report [None req-12833e69-8a87-4891-b279-991727f33b3c tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] Inventory has not changed for provider 4d291220-7c37-4839-a0e3-b23c66502b74 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=75234) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Nov 29 03:11:21 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-12833e69-8a87-4891-b279-991727f33b3c tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.392s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:11:21 user nova-compute[75234]: INFO nova.scheduler.client.report [None req-12833e69-8a87-4891-b279-991727f33b3c tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] Deleted allocations for instance f7d0bdeb-6a05-47ac-87f2-c8380a9a4286
Nov 29 03:11:22 user nova-compute[75234]: DEBUG nova.compute.manager [req-6f241a55-2b58-42c8-9bc2-ae09f7ff5031 req-87ab1a9b-5e74-498e-af19-b02ce2e9d081 service nova] [instance: f7d0bdeb-6a05-47ac-87f2-c8380a9a4286] Received event network-vif-plugged-b5192c04-0323-48b8-a326-ddf6b21f9019 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:11:22 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-6f241a55-2b58-42c8-9bc2-ae09f7ff5031 req-87ab1a9b-5e74-498e-af19-b02ce2e9d081 service nova] Acquiring lock "f7d0bdeb-6a05-47ac-87f2-c8380a9a4286-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:11:22 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-6f241a55-2b58-42c8-9bc2-ae09f7ff5031 req-87ab1a9b-5e74-498e-af19-b02ce2e9d081 service nova] Lock "f7d0bdeb-6a05-47ac-87f2-c8380a9a4286-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:11:22 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-6f241a55-2b58-42c8-9bc2-ae09f7ff5031 req-87ab1a9b-5e74-498e-af19-b02ce2e9d081 service nova] Lock "f7d0bdeb-6a05-47ac-87f2-c8380a9a4286-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:11:22 user nova-compute[75234]: DEBUG nova.compute.manager [req-6f241a55-2b58-42c8-9bc2-ae09f7ff5031 req-87ab1a9b-5e74-498e-af19-b02ce2e9d081 service nova] [instance: f7d0bdeb-6a05-47ac-87f2-c8380a9a4286] No waiting events found dispatching network-vif-plugged-b5192c04-0323-48b8-a326-ddf6b21f9019 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:11:22 user nova-compute[75234]: WARNING nova.compute.manager [req-6f241a55-2b58-42c8-9bc2-ae09f7ff5031 req-87ab1a9b-5e74-498e-af19-b02ce2e9d081 service nova] [instance: f7d0bdeb-6a05-47ac-87f2-c8380a9a4286] Received unexpected event network-vif-plugged-b5192c04-0323-48b8-a326-ddf6b21f9019 for instance with vm_state deleted and task_state None.
Nov 29 03:11:22 user nova-compute[75234]: DEBUG nova.compute.manager [req-6f241a55-2b58-42c8-9bc2-ae09f7ff5031 req-87ab1a9b-5e74-498e-af19-b02ce2e9d081 service nova] [instance: f7d0bdeb-6a05-47ac-87f2-c8380a9a4286] Received event network-vif-deleted-b5192c04-0323-48b8-a326-ddf6b21f9019 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:11:22 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-12833e69-8a87-4891-b279-991727f33b3c tempest-TestServerVolumeAttachmentScenario-1219797116 tempest-TestServerVolumeAttachmentScenario-1219797116-project-member] Lock "f7d0bdeb-6a05-47ac-87f2-c8380a9a4286" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 2.409s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:11:22 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:11:25 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:11:26 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:11:26 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:11:27 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:11:29 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-f77c1f58-64dc-4b23-9050-987d9f2e6066 tempest-VolumesActionsTest-1495826017 tempest-VolumesActionsTest-1495826017-project-member] Acquiring lock "1eb9b734-2711-4463-9743-e5fd95c6857b" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:11:29 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-f77c1f58-64dc-4b23-9050-987d9f2e6066 tempest-VolumesActionsTest-1495826017 tempest-VolumesActionsTest-1495826017-project-member] Lock "1eb9b734-2711-4463-9743-e5fd95c6857b" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:11:29 user nova-compute[75234]: DEBUG nova.compute.manager [None req-f77c1f58-64dc-4b23-9050-987d9f2e6066 tempest-VolumesActionsTest-1495826017 tempest-VolumesActionsTest-1495826017-project-member] [instance: 1eb9b734-2711-4463-9743-e5fd95c6857b] Starting instance... {{(pid=75234) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2422}}
Nov 29 03:11:29 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-f77c1f58-64dc-4b23-9050-987d9f2e6066 tempest-VolumesActionsTest-1495826017 tempest-VolumesActionsTest-1495826017-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:11:29 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-f77c1f58-64dc-4b23-9050-987d9f2e6066 tempest-VolumesActionsTest-1495826017 tempest-VolumesActionsTest-1495826017-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:11:29 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-f77c1f58-64dc-4b23-9050-987d9f2e6066 tempest-VolumesActionsTest-1495826017 tempest-VolumesActionsTest-1495826017-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=75234) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2366}}
Nov 29 03:11:29 user nova-compute[75234]: INFO nova.compute.claims [None req-f77c1f58-64dc-4b23-9050-987d9f2e6066 tempest-VolumesActionsTest-1495826017 tempest-VolumesActionsTest-1495826017-project-member] [instance: 1eb9b734-2711-4463-9743-e5fd95c6857b] Claim successful on node user
Nov 29 03:11:29 user nova-compute[75234]: DEBUG nova.compute.provider_tree [None req-f77c1f58-64dc-4b23-9050-987d9f2e6066 tempest-VolumesActionsTest-1495826017 tempest-VolumesActionsTest-1495826017-project-member] Inventory has not changed in ProviderTree for provider: 4d291220-7c37-4839-a0e3-b23c66502b74 {{(pid=75234) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Nov 29 03:11:29 user nova-compute[75234]: DEBUG nova.scheduler.client.report [None req-f77c1f58-64dc-4b23-9050-987d9f2e6066 tempest-VolumesActionsTest-1495826017 tempest-VolumesActionsTest-1495826017-project-member] Inventory has not changed for provider 4d291220-7c37-4839-a0e3-b23c66502b74 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=75234) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Nov 29 03:11:29 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-f77c1f58-64dc-4b23-9050-987d9f2e6066 tempest-VolumesActionsTest-1495826017 tempest-VolumesActionsTest-1495826017-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.472s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:11:29 user nova-compute[75234]: DEBUG nova.compute.manager [None req-f77c1f58-64dc-4b23-9050-987d9f2e6066 tempest-VolumesActionsTest-1495826017 tempest-VolumesActionsTest-1495826017-project-member] [instance: 1eb9b734-2711-4463-9743-e5fd95c6857b] Start building networks asynchronously for instance. {{(pid=75234) _build_resources /opt/stack/nova/nova/compute/manager.py:2819}}
Nov 29 03:11:29 user nova-compute[75234]: DEBUG nova.compute.manager [None req-f77c1f58-64dc-4b23-9050-987d9f2e6066 tempest-VolumesActionsTest-1495826017 tempest-VolumesActionsTest-1495826017-project-member] [instance: 1eb9b734-2711-4463-9743-e5fd95c6857b] Allocating IP information in the background. {{(pid=75234) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1972}}
Nov 29 03:11:29 user nova-compute[75234]: DEBUG nova.network.neutron [None req-f77c1f58-64dc-4b23-9050-987d9f2e6066 tempest-VolumesActionsTest-1495826017 tempest-VolumesActionsTest-1495826017-project-member] [instance: 1eb9b734-2711-4463-9743-e5fd95c6857b] allocate_for_instance() {{(pid=75234) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1156}}
Nov 29 03:11:29 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-f77c1f58-64dc-4b23-9050-987d9f2e6066 tempest-VolumesActionsTest-1495826017 tempest-VolumesActionsTest-1495826017-project-member] [instance: 1eb9b734-2711-4463-9743-e5fd95c6857b] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Nov 29 03:11:29 user nova-compute[75234]: DEBUG nova.compute.manager [None req-f77c1f58-64dc-4b23-9050-987d9f2e6066 tempest-VolumesActionsTest-1495826017 tempest-VolumesActionsTest-1495826017-project-member] [instance: 1eb9b734-2711-4463-9743-e5fd95c6857b] Start building block device mappings for instance. {{(pid=75234) _build_resources /opt/stack/nova/nova/compute/manager.py:2854}}
Nov 29 03:11:29 user nova-compute[75234]: DEBUG nova.policy [None req-f77c1f58-64dc-4b23-9050-987d9f2e6066 tempest-VolumesActionsTest-1495826017 tempest-VolumesActionsTest-1495826017-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '324ad29d0b8b443086942e61e2be4ceb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '773aac44d1a74a439f4f750dfe8ac142', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=75234) authorize /opt/stack/nova/nova/policy.py:203}}
Nov 29 03:11:29 user nova-compute[75234]: DEBUG nova.compute.manager [None req-f77c1f58-64dc-4b23-9050-987d9f2e6066 tempest-VolumesActionsTest-1495826017 tempest-VolumesActionsTest-1495826017-project-member] [instance: 1eb9b734-2711-4463-9743-e5fd95c6857b] Start spawning the instance on the hypervisor. {{(pid=75234) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2628}}
Nov 29 03:11:29 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-f77c1f58-64dc-4b23-9050-987d9f2e6066 tempest-VolumesActionsTest-1495826017 tempest-VolumesActionsTest-1495826017-project-member] [instance: 1eb9b734-2711-4463-9743-e5fd95c6857b] Creating instance directory {{(pid=75234) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4727}}
Nov 29 03:11:29 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-f77c1f58-64dc-4b23-9050-987d9f2e6066 tempest-VolumesActionsTest-1495826017 tempest-VolumesActionsTest-1495826017-project-member] [instance: 1eb9b734-2711-4463-9743-e5fd95c6857b] Creating image(s)
Nov 29 03:11:29 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-f77c1f58-64dc-4b23-9050-987d9f2e6066 tempest-VolumesActionsTest-1495826017 tempest-VolumesActionsTest-1495826017-project-member] Acquiring lock "/opt/stack/data/nova/instances/1eb9b734-2711-4463-9743-e5fd95c6857b/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:11:29 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-f77c1f58-64dc-4b23-9050-987d9f2e6066 tempest-VolumesActionsTest-1495826017 tempest-VolumesActionsTest-1495826017-project-member] Lock "/opt/stack/data/nova/instances/1eb9b734-2711-4463-9743-e5fd95c6857b/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:11:29 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-f77c1f58-64dc-4b23-9050-987d9f2e6066 tempest-VolumesActionsTest-1495826017 tempest-VolumesActionsTest-1495826017-project-member] Lock "/opt/stack/data/nova/instances/1eb9b734-2711-4463-9743-e5fd95c6857b/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:11:29 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-f77c1f58-64dc-4b23-9050-987d9f2e6066 tempest-VolumesActionsTest-1495826017 tempest-VolumesActionsTest-1495826017-project-member] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:11:30 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-f77c1f58-64dc-4b23-9050-987d9f2e6066 tempest-VolumesActionsTest-1495826017 tempest-VolumesActionsTest-1495826017-project-member] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json" returned: 0 in 0.184s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:11:30 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-f77c1f58-64dc-4b23-9050-987d9f2e6066 tempest-VolumesActionsTest-1495826017 tempest-VolumesActionsTest-1495826017-project-member] Acquiring lock "80686e29dc9db6600d03e85cb62af298e12d7918" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:11:30 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-f77c1f58-64dc-4b23-9050-987d9f2e6066 tempest-VolumesActionsTest-1495826017 tempest-VolumesActionsTest-1495826017-project-member] Lock "80686e29dc9db6600d03e85cb62af298e12d7918" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.002s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:11:30 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-f77c1f58-64dc-4b23-9050-987d9f2e6066 tempest-VolumesActionsTest-1495826017 tempest-VolumesActionsTest-1495826017-project-member] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:11:30 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-f77c1f58-64dc-4b23-9050-987d9f2e6066 tempest-VolumesActionsTest-1495826017 tempest-VolumesActionsTest-1495826017-project-member] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json" returned: 0 in 0.173s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:11:30 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-f77c1f58-64dc-4b23-9050-987d9f2e6066 tempest-VolumesActionsTest-1495826017 tempest-VolumesActionsTest-1495826017-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918,backing_fmt=raw /opt/stack/data/nova/instances/1eb9b734-2711-4463-9743-e5fd95c6857b/disk 1073741824 {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:11:30 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-f77c1f58-64dc-4b23-9050-987d9f2e6066 tempest-VolumesActionsTest-1495826017 tempest-VolumesActionsTest-1495826017-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918,backing_fmt=raw /opt/stack/data/nova/instances/1eb9b734-2711-4463-9743-e5fd95c6857b/disk 1073741824" returned: 0 in 0.072s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:11:30 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-f77c1f58-64dc-4b23-9050-987d9f2e6066 tempest-VolumesActionsTest-1495826017 tempest-VolumesActionsTest-1495826017-project-member] Lock "80686e29dc9db6600d03e85cb62af298e12d7918" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.253s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:11:30 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-f77c1f58-64dc-4b23-9050-987d9f2e6066 tempest-VolumesActionsTest-1495826017 tempest-VolumesActionsTest-1495826017-project-member] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:11:30 user nova-compute[75234]: DEBUG nova.network.neutron [None req-f77c1f58-64dc-4b23-9050-987d9f2e6066 tempest-VolumesActionsTest-1495826017 tempest-VolumesActionsTest-1495826017-project-member] [instance: 1eb9b734-2711-4463-9743-e5fd95c6857b] Successfully created port: 6d9e1401-d167-4fb3-a59d-ca69b18ed417 {{(pid=75234) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:548}}
Nov 29 03:11:30 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:11:30 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-f77c1f58-64dc-4b23-9050-987d9f2e6066 tempest-VolumesActionsTest-1495826017 tempest-VolumesActionsTest-1495826017-project-member] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json" returned: 0 in 0.169s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:11:30 user nova-compute[75234]: DEBUG nova.virt.disk.api [None req-f77c1f58-64dc-4b23-9050-987d9f2e6066 tempest-VolumesActionsTest-1495826017 tempest-VolumesActionsTest-1495826017-project-member] Checking if we can resize image /opt/stack/data/nova/instances/1eb9b734-2711-4463-9743-e5fd95c6857b/disk. size=1073741824 {{(pid=75234) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:166}}
Nov 29 03:11:30 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-f77c1f58-64dc-4b23-9050-987d9f2e6066 tempest-VolumesActionsTest-1495826017 tempest-VolumesActionsTest-1495826017-project-member] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/1eb9b734-2711-4463-9743-e5fd95c6857b/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:11:30 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-f77c1f58-64dc-4b23-9050-987d9f2e6066 tempest-VolumesActionsTest-1495826017 tempest-VolumesActionsTest-1495826017-project-member] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/1eb9b734-2711-4463-9743-e5fd95c6857b/disk --force-share --output=json" returned: 0 in 0.172s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:11:30 user nova-compute[75234]: DEBUG nova.virt.disk.api [None req-f77c1f58-64dc-4b23-9050-987d9f2e6066 tempest-VolumesActionsTest-1495826017 tempest-VolumesActionsTest-1495826017-project-member] Cannot resize image /opt/stack/data/nova/instances/1eb9b734-2711-4463-9743-e5fd95c6857b/disk to a smaller size. {{(pid=75234) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:172}}
Nov 29 03:11:30 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-f77c1f58-64dc-4b23-9050-987d9f2e6066 tempest-VolumesActionsTest-1495826017 tempest-VolumesActionsTest-1495826017-project-member] [instance: 1eb9b734-2711-4463-9743-e5fd95c6857b] Created local disks {{(pid=75234) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4861}}
Nov 29 03:11:30 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-f77c1f58-64dc-4b23-9050-987d9f2e6066 tempest-VolumesActionsTest-1495826017 tempest-VolumesActionsTest-1495826017-project-member] [instance: 1eb9b734-2711-4463-9743-e5fd95c6857b] Ensure instance console log exists: /opt/stack/data/nova/instances/1eb9b734-2711-4463-9743-e5fd95c6857b/console.log {{(pid=75234) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4613}}
Nov 29 03:11:30 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-f77c1f58-64dc-4b23-9050-987d9f2e6066 tempest-VolumesActionsTest-1495826017 tempest-VolumesActionsTest-1495826017-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:11:30 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-f77c1f58-64dc-4b23-9050-987d9f2e6066 tempest-VolumesActionsTest-1495826017 tempest-VolumesActionsTest-1495826017-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:11:30 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-f77c1f58-64dc-4b23-9050-987d9f2e6066 tempest-VolumesActionsTest-1495826017 tempest-VolumesActionsTest-1495826017-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:11:31 user nova-compute[75234]: DEBUG nova.network.neutron [None req-f77c1f58-64dc-4b23-9050-987d9f2e6066 tempest-VolumesActionsTest-1495826017 tempest-VolumesActionsTest-1495826017-project-member] [instance: 1eb9b734-2711-4463-9743-e5fd95c6857b] Successfully updated port: 6d9e1401-d167-4fb3-a59d-ca69b18ed417 {{(pid=75234) _update_port /opt/stack/nova/nova/network/neutron.py:586}}
Nov 29 03:11:31 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-f77c1f58-64dc-4b23-9050-987d9f2e6066 tempest-VolumesActionsTest-1495826017 tempest-VolumesActionsTest-1495826017-project-member] Acquiring lock "refresh_cache-1eb9b734-2711-4463-9743-e5fd95c6857b" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:11:31 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-f77c1f58-64dc-4b23-9050-987d9f2e6066 tempest-VolumesActionsTest-1495826017 tempest-VolumesActionsTest-1495826017-project-member] Acquired lock "refresh_cache-1eb9b734-2711-4463-9743-e5fd95c6857b" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:11:31 user nova-compute[75234]: DEBUG nova.network.neutron [None req-f77c1f58-64dc-4b23-9050-987d9f2e6066 tempest-VolumesActionsTest-1495826017 tempest-VolumesActionsTest-1495826017-project-member] [instance: 1eb9b734-2711-4463-9743-e5fd95c6857b] Building network info cache for instance {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2009}}
Nov 29 03:11:31 user nova-compute[75234]: DEBUG nova.compute.manager [req-ea56f372-3ef4-411e-b6ad-f855dfaecc91 req-5205e618-552e-49ed-bee6-0e511b9b8584 service nova] [instance: 1eb9b734-2711-4463-9743-e5fd95c6857b] Received event network-changed-6d9e1401-d167-4fb3-a59d-ca69b18ed417 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:11:31 user nova-compute[75234]: DEBUG nova.compute.manager [req-ea56f372-3ef4-411e-b6ad-f855dfaecc91 req-5205e618-552e-49ed-bee6-0e511b9b8584 service nova] [instance: 1eb9b734-2711-4463-9743-e5fd95c6857b] Refreshing instance network info cache due to event network-changed-6d9e1401-d167-4fb3-a59d-ca69b18ed417. {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11117}}
Nov 29 03:11:31 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-ea56f372-3ef4-411e-b6ad-f855dfaecc91 req-5205e618-552e-49ed-bee6-0e511b9b8584 service nova] Acquiring lock "refresh_cache-1eb9b734-2711-4463-9743-e5fd95c6857b" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:11:31 user nova-compute[75234]: DEBUG nova.network.neutron [None req-f77c1f58-64dc-4b23-9050-987d9f2e6066 tempest-VolumesActionsTest-1495826017 tempest-VolumesActionsTest-1495826017-project-member] [instance: 1eb9b734-2711-4463-9743-e5fd95c6857b] Instance cache missing network info. {{(pid=75234) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3322}}
Nov 29 03:11:31 user nova-compute[75234]: DEBUG nova.network.neutron [None req-f77c1f58-64dc-4b23-9050-987d9f2e6066 tempest-VolumesActionsTest-1495826017 tempest-VolumesActionsTest-1495826017-project-member] [instance: 1eb9b734-2711-4463-9743-e5fd95c6857b] Updating instance_info_cache with network_info: [{"id": "6d9e1401-d167-4fb3-a59d-ca69b18ed417", "address": "fa:16:3e:24:48:f1", "network": {"id": "cb9b1cd8-2f38-4fe2-b680-380cff4b4d11", "bridge": "br-int", "label": "tempest-VolumesActionsTest-1501188375-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "773aac44d1a74a439f4f750dfe8ac142", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap6d9e1401-d1", "ovs_interfaceid": "6d9e1401-d167-4fb3-a59d-ca69b18ed417", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:11:31 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-f77c1f58-64dc-4b23-9050-987d9f2e6066 tempest-VolumesActionsTest-1495826017 tempest-VolumesActionsTest-1495826017-project-member] Releasing lock "refresh_cache-1eb9b734-2711-4463-9743-e5fd95c6857b" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:11:31 user nova-compute[75234]: DEBUG nova.compute.manager [None req-f77c1f58-64dc-4b23-9050-987d9f2e6066 tempest-VolumesActionsTest-1495826017 tempest-VolumesActionsTest-1495826017-project-member] [instance: 1eb9b734-2711-4463-9743-e5fd95c6857b] Instance network_info: |[{"id": "6d9e1401-d167-4fb3-a59d-ca69b18ed417", "address": "fa:16:3e:24:48:f1", "network": {"id": "cb9b1cd8-2f38-4fe2-b680-380cff4b4d11", "bridge": "br-int", "label": "tempest-VolumesActionsTest-1501188375-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "773aac44d1a74a439f4f750dfe8ac142", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap6d9e1401-d1", "ovs_interfaceid": "6d9e1401-d167-4fb3-a59d-ca69b18ed417", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=75234) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1987}}
Nov 29 03:11:31 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-ea56f372-3ef4-411e-b6ad-f855dfaecc91 req-5205e618-552e-49ed-bee6-0e511b9b8584 service nova] Acquired lock "refresh_cache-1eb9b734-2711-4463-9743-e5fd95c6857b" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:11:31 user nova-compute[75234]: DEBUG nova.network.neutron [req-ea56f372-3ef4-411e-b6ad-f855dfaecc91 req-5205e618-552e-49ed-bee6-0e511b9b8584 service nova] [instance: 1eb9b734-2711-4463-9743-e5fd95c6857b] Refreshing network info cache for port 6d9e1401-d167-4fb3-a59d-ca69b18ed417 {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2006}}
Nov 29 03:11:31 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-f77c1f58-64dc-4b23-9050-987d9f2e6066 tempest-VolumesActionsTest-1495826017 tempest-VolumesActionsTest-1495826017-project-member] [instance: 1eb9b734-2711-4463-9743-e5fd95c6857b] Start _get_guest_xml network_info=[{"id": "6d9e1401-d167-4fb3-a59d-ca69b18ed417", "address": "fa:16:3e:24:48:f1", "network": {"id": "cb9b1cd8-2f38-4fe2-b680-380cff4b4d11", "bridge": "br-int", "label": "tempest-VolumesActionsTest-1501188375-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "773aac44d1a74a439f4f750dfe8ac142", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap6d9e1401-d1", "ovs_interfaceid": "6d9e1401-d167-4fb3-a59d-ca69b18ed417", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum='c8fc807773e5354afe61636071771906',container_format='bare',created_at=2023-11-29T03:02:15Z,direct_url=<?>,disk_format='qcow2',id=96c30ba7-9b73-4a30-8ba4-7138879573b7,min_disk=0,min_ram=0,name='cirros-0.6.2-x86_64-disk',owner='b18e380921c748b78517b6f1211cd0f1',properties=ImageMetaProps,protected=<?>,size=21430272,status='active',tags=<?>,updated_at=2023-11-29T03:02:17Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [{'device_name': '/dev/vda', 'encryption_format': None, 'encrypted': False, 'device_type': 'disk', 'size': 0, 'disk_bus': 'virtio', 'boot_index': 0, 'encryption_options': None, 'encryption_secret_uuid': None, 'guest_format': None, 'image_id': '96c30ba7-9b73-4a30-8ba4-7138879573b7'}], 'ephemerals': [], 'block_device_mapping': [], 'swap': None} {{(pid=75234) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7562}}
Nov 29 03:11:31 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-f77c1f58-64dc-4b23-9050-987d9f2e6066 tempest-VolumesActionsTest-1495826017 tempest-VolumesActionsTest-1495826017-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:11:31 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-f77c1f58-64dc-4b23-9050-987d9f2e6066 tempest-VolumesActionsTest-1495826017 tempest-VolumesActionsTest-1495826017-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:11:31 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-f77c1f58-64dc-4b23-9050-987d9f2e6066 tempest-VolumesActionsTest-1495826017 tempest-VolumesActionsTest-1495826017-project-member] Searching host: 'user' for CPU controller through CGroups V1... {{(pid=75234) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1652}}
Nov 29 03:11:31 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-f77c1f58-64dc-4b23-9050-987d9f2e6066 tempest-VolumesActionsTest-1495826017 tempest-VolumesActionsTest-1495826017-project-member] CPU controller missing on host. {{(pid=75234) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1662}}
Nov 29 03:11:31 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-f77c1f58-64dc-4b23-9050-987d9f2e6066 tempest-VolumesActionsTest-1495826017 tempest-VolumesActionsTest-1495826017-project-member] Searching host: 'user' for CPU controller through CGroups V2... {{(pid=75234) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1671}}
Nov 29 03:11:31 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-f77c1f58-64dc-4b23-9050-987d9f2e6066 tempest-VolumesActionsTest-1495826017 tempest-VolumesActionsTest-1495826017-project-member] CPU controller found on host. {{(pid=75234) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1678}}
Nov 29 03:11:31 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-f77c1f58-64dc-4b23-9050-987d9f2e6066 tempest-VolumesActionsTest-1495826017 tempest-VolumesActionsTest-1495826017-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=75234) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5400}}
Nov 29 03:11:31 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-f77c1f58-64dc-4b23-9050-987d9f2e6066 tempest-VolumesActionsTest-1495826017 tempest-VolumesActionsTest-1495826017-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-11-29T03:02:04Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='1',id=6,is_public=True,memory_mb=512,name='m1.tiny',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='c8fc807773e5354afe61636071771906',container_format='bare',created_at=2023-11-29T03:02:15Z,direct_url=<?>,disk_format='qcow2',id=96c30ba7-9b73-4a30-8ba4-7138879573b7,min_disk=0,min_ram=0,name='cirros-0.6.2-x86_64-disk',owner='b18e380921c748b78517b6f1211cd0f1',properties=ImageMetaProps,protected=<?>,size=21430272,status='active',tags=<?>,updated_at=2023-11-29T03:02:17Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:563}}
Nov 29 03:11:31 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-f77c1f58-64dc-4b23-9050-987d9f2e6066 tempest-VolumesActionsTest-1495826017 tempest-VolumesActionsTest-1495826017-project-member] Flavor limits 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:348}}
Nov 29 03:11:31 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-f77c1f58-64dc-4b23-9050-987d9f2e6066 tempest-VolumesActionsTest-1495826017 tempest-VolumesActionsTest-1495826017-project-member] Image limits 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:352}}
Nov 29 03:11:31 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-f77c1f58-64dc-4b23-9050-987d9f2e6066 tempest-VolumesActionsTest-1495826017 tempest-VolumesActionsTest-1495826017-project-member] Flavor pref 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:388}}
Nov 29 03:11:31 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-f77c1f58-64dc-4b23-9050-987d9f2e6066 tempest-VolumesActionsTest-1495826017 tempest-VolumesActionsTest-1495826017-project-member] Image pref 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:392}}
Nov 29 03:11:31 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-f77c1f58-64dc-4b23-9050-987d9f2e6066 tempest-VolumesActionsTest-1495826017 tempest-VolumesActionsTest-1495826017-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:430}}
Nov 29 03:11:31 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-f77c1f58-64dc-4b23-9050-987d9f2e6066 tempest-VolumesActionsTest-1495826017 tempest-VolumesActionsTest-1495826017-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:569}}
Nov 29 03:11:31 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-f77c1f58-64dc-4b23-9050-987d9f2e6066 tempest-VolumesActionsTest-1495826017 tempest-VolumesActionsTest-1495826017-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=75234) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:471}}
Nov 29 03:11:31 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-f77c1f58-64dc-4b23-9050-987d9f2e6066 tempest-VolumesActionsTest-1495826017 tempest-VolumesActionsTest-1495826017-project-member] Got 1 possible topologies {{(pid=75234) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:501}}
Nov 29 03:11:31 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-f77c1f58-64dc-4b23-9050-987d9f2e6066 tempest-VolumesActionsTest-1495826017 tempest-VolumesActionsTest-1495826017-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:575}}
Nov 29 03:11:31 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-f77c1f58-64dc-4b23-9050-987d9f2e6066 tempest-VolumesActionsTest-1495826017 tempest-VolumesActionsTest-1495826017-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:577}}
Nov 29 03:11:31 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-f77c1f58-64dc-4b23-9050-987d9f2e6066 tempest-VolumesActionsTest-1495826017 tempest-VolumesActionsTest-1495826017-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-11-29T03:11:28Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-VolumesActionsTest-instance-2001816366',display_name='tempest-VolumesActionsTest-instance-2001816366',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-volumesactionstest-instance-2001816366',id=21,image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data=None,key_name=None,keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='773aac44d1a74a439f4f750dfe8ac142',ramdisk_id='',reservation_id='r-3x0wrwrv',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-VolumesActionsTest-1495826017',owner_user_name='tempest-VolumesActionsTest-1495826017-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-11-29T03:11:30Z,user_data=None,user_id='324ad29d0b8b443086942e61e2be4ceb',uuid=1eb9b734-2711-4463-9743-e5fd95c6857b,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "6d9e1401-d167-4fb3-a59d-ca69b18ed417", "address": "fa:16:3e:24:48:f1", "network": {"id": "cb9b1cd8-2f38-4fe2-b680-380cff4b4d11", "bridge": "br-int", "label": "tempest-VolumesActionsTest-1501188375-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "773aac44d1a74a439f4f750dfe8ac142", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap6d9e1401-d1", "ovs_interfaceid": "6d9e1401-d167-4fb3-a59d-ca69b18ed417", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=75234) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Nov 29 03:11:31 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-f77c1f58-64dc-4b23-9050-987d9f2e6066 tempest-VolumesActionsTest-1495826017 tempest-VolumesActionsTest-1495826017-project-member] Converting VIF {"id": "6d9e1401-d167-4fb3-a59d-ca69b18ed417", "address": "fa:16:3e:24:48:f1", "network": {"id": "cb9b1cd8-2f38-4fe2-b680-380cff4b4d11", "bridge": "br-int", "label": "tempest-VolumesActionsTest-1501188375-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "773aac44d1a74a439f4f750dfe8ac142", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap6d9e1401-d1", "ovs_interfaceid": "6d9e1401-d167-4fb3-a59d-ca69b18ed417", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:11:31 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-f77c1f58-64dc-4b23-9050-987d9f2e6066 tempest-VolumesActionsTest-1495826017 tempest-VolumesActionsTest-1495826017-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:24:48:f1,bridge_name='br-int',has_traffic_filtering=True,id=6d9e1401-d167-4fb3-a59d-ca69b18ed417,network=Network(cb9b1cd8-2f38-4fe2-b680-380cff4b4d11),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap6d9e1401-d1') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:11:31 user nova-compute[75234]: DEBUG nova.objects.instance [None req-f77c1f58-64dc-4b23-9050-987d9f2e6066 tempest-VolumesActionsTest-1495826017 tempest-VolumesActionsTest-1495826017-project-member] Lazy-loading 'pci_devices' on Instance uuid 1eb9b734-2711-4463-9743-e5fd95c6857b {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:11:31 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:11:31 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-f77c1f58-64dc-4b23-9050-987d9f2e6066 tempest-VolumesActionsTest-1495826017 tempest-VolumesActionsTest-1495826017-project-member] [instance: 1eb9b734-2711-4463-9743-e5fd95c6857b] End _get_guest_xml xml=<domain type="kvm">
Nov 29 03:11:31 user nova-compute[75234]:   <uuid>1eb9b734-2711-4463-9743-e5fd95c6857b</uuid>
Nov 29 03:11:31 user nova-compute[75234]:   <name>instance-00000015</name>
Nov 29 03:11:31 user nova-compute[75234]:   <memory>524288</memory>
Nov 29 03:11:31 user nova-compute[75234]:   <vcpu>1</vcpu>
Nov 29 03:11:31 user nova-compute[75234]:   <metadata>
Nov 29 03:11:31 user nova-compute[75234]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Nov 29 03:11:31 user nova-compute[75234]:       <nova:package version="0.0.0"/>
Nov 29 03:11:31 user nova-compute[75234]:       <nova:name>tempest-VolumesActionsTest-instance-2001816366</nova:name>
Nov 29 03:11:31 user nova-compute[75234]:       <nova:creationTime>2023-11-29 03:11:31</nova:creationTime>
Nov 29 03:11:31 user nova-compute[75234]:       <nova:flavor name="m1.tiny">
Nov 29 03:11:31 user nova-compute[75234]:         <nova:memory>512</nova:memory>
Nov 29 03:11:31 user nova-compute[75234]:         <nova:disk>1</nova:disk>
Nov 29 03:11:31 user nova-compute[75234]:         <nova:swap>0</nova:swap>
Nov 29 03:11:31 user nova-compute[75234]:         <nova:ephemeral>0</nova:ephemeral>
Nov 29 03:11:31 user nova-compute[75234]:         <nova:vcpus>1</nova:vcpus>
Nov 29 03:11:31 user nova-compute[75234]:       </nova:flavor>
Nov 29 03:11:31 user nova-compute[75234]:       <nova:owner>
Nov 29 03:11:31 user nova-compute[75234]:         <nova:user uuid="324ad29d0b8b443086942e61e2be4ceb">tempest-VolumesActionsTest-1495826017-project-member</nova:user>
Nov 29 03:11:31 user nova-compute[75234]:         <nova:project uuid="773aac44d1a74a439f4f750dfe8ac142">tempest-VolumesActionsTest-1495826017</nova:project>
Nov 29 03:11:31 user nova-compute[75234]:       </nova:owner>
Nov 29 03:11:31 user nova-compute[75234]:       <nova:root type="image" uuid="96c30ba7-9b73-4a30-8ba4-7138879573b7"/>
Nov 29 03:11:31 user nova-compute[75234]:       <nova:ports>
Nov 29 03:11:31 user nova-compute[75234]:         <nova:port uuid="6d9e1401-d167-4fb3-a59d-ca69b18ed417">
Nov 29 03:11:31 user nova-compute[75234]:           <nova:ip type="fixed" address="10.0.0.9" ipVersion="4"/>
Nov 29 03:11:31 user nova-compute[75234]:         </nova:port>
Nov 29 03:11:31 user nova-compute[75234]:       </nova:ports>
Nov 29 03:11:31 user nova-compute[75234]:     </nova:instance>
Nov 29 03:11:31 user nova-compute[75234]:   </metadata>
Nov 29 03:11:31 user nova-compute[75234]:   <sysinfo type="smbios">
Nov 29 03:11:31 user nova-compute[75234]:     <system>
Nov 29 03:11:31 user nova-compute[75234]:       <entry name="manufacturer">OpenStack Foundation</entry>
Nov 29 03:11:31 user nova-compute[75234]:       <entry name="product">OpenStack Nova</entry>
Nov 29 03:11:31 user nova-compute[75234]:       <entry name="version">0.0.0</entry>
Nov 29 03:11:31 user nova-compute[75234]:       <entry name="serial">1eb9b734-2711-4463-9743-e5fd95c6857b</entry>
Nov 29 03:11:31 user nova-compute[75234]:       <entry name="uuid">1eb9b734-2711-4463-9743-e5fd95c6857b</entry>
Nov 29 03:11:31 user nova-compute[75234]:       <entry name="family">Virtual Machine</entry>
Nov 29 03:11:31 user nova-compute[75234]:     </system>
Nov 29 03:11:31 user nova-compute[75234]:   </sysinfo>
Nov 29 03:11:31 user nova-compute[75234]:   <os>
Nov 29 03:11:31 user nova-compute[75234]:     <type arch="x86_64" machine="pc">hvm</type>
Nov 29 03:11:31 user nova-compute[75234]:     <boot dev="hd"/>
Nov 29 03:11:31 user nova-compute[75234]:     <smbios mode="sysinfo"/>
Nov 29 03:11:31 user nova-compute[75234]:   </os>
Nov 29 03:11:31 user nova-compute[75234]:   <features>
Nov 29 03:11:31 user nova-compute[75234]:     <acpi/>
Nov 29 03:11:31 user nova-compute[75234]:     <apic/>
Nov 29 03:11:31 user nova-compute[75234]:     <vmcoreinfo/>
Nov 29 03:11:31 user nova-compute[75234]:   </features>
Nov 29 03:11:31 user nova-compute[75234]:   <clock offset="utc">
Nov 29 03:11:31 user nova-compute[75234]:     <timer name="pit" tickpolicy="delay"/>
Nov 29 03:11:31 user nova-compute[75234]:     <timer name="rtc" tickpolicy="catchup"/>
Nov 29 03:11:31 user nova-compute[75234]:     <timer name="hpet" present="no"/>
Nov 29 03:11:31 user nova-compute[75234]:   </clock>
Nov 29 03:11:31 user nova-compute[75234]:   <cpu mode="custom" match="exact">
Nov 29 03:11:31 user nova-compute[75234]:     <model>Nehalem</model>
Nov 29 03:11:31 user nova-compute[75234]:     <topology sockets="1" cores="1" threads="1"/>
Nov 29 03:11:31 user nova-compute[75234]:   </cpu>
Nov 29 03:11:31 user nova-compute[75234]:   <devices>
Nov 29 03:11:31 user nova-compute[75234]:     <disk type="file" device="disk">
Nov 29 03:11:31 user nova-compute[75234]:       <driver name="qemu" type="qcow2" cache="none"/>
Nov 29 03:11:31 user nova-compute[75234]:       <source file="/opt/stack/data/nova/instances/1eb9b734-2711-4463-9743-e5fd95c6857b/disk"/>
Nov 29 03:11:31 user nova-compute[75234]:       <target dev="vda" bus="virtio"/>
Nov 29 03:11:31 user nova-compute[75234]:     </disk>
Nov 29 03:11:31 user nova-compute[75234]:     <interface type="ethernet">
Nov 29 03:11:31 user nova-compute[75234]:       <mac address="fa:16:3e:24:48:f1"/>
Nov 29 03:11:31 user nova-compute[75234]:       <model type="virtio"/>
Nov 29 03:11:31 user nova-compute[75234]:       <mtu size="1442"/>
Nov 29 03:11:31 user nova-compute[75234]:       <target dev="tap6d9e1401-d1"/>
Nov 29 03:11:31 user nova-compute[75234]:     </interface>
Nov 29 03:11:31 user nova-compute[75234]:     <serial type="pty">
Nov 29 03:11:31 user nova-compute[75234]:       <log file="/opt/stack/data/nova/instances/1eb9b734-2711-4463-9743-e5fd95c6857b/console.log" append="off"/>
Nov 29 03:11:31 user nova-compute[75234]:     </serial>
Nov 29 03:11:31 user nova-compute[75234]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Nov 29 03:11:31 user nova-compute[75234]:     <video>
Nov 29 03:11:31 user nova-compute[75234]:       <model type="virtio"/>
Nov 29 03:11:31 user nova-compute[75234]:     </video>
Nov 29 03:11:31 user nova-compute[75234]:     <rng model="virtio">
Nov 29 03:11:31 user nova-compute[75234]:       <backend model="random">/dev/urandom</backend>
Nov 29 03:11:31 user nova-compute[75234]:     </rng>
Nov 29 03:11:31 user nova-compute[75234]:     <controller type="usb" index="0" model="none"/>
Nov 29 03:11:31 user nova-compute[75234]:     <memballoon model="virtio">
Nov 29 03:11:31 user nova-compute[75234]:       <stats period="10"/>
Nov 29 03:11:31 user nova-compute[75234]:     </memballoon>
Nov 29 03:11:31 user nova-compute[75234]:   </devices>
Nov 29 03:11:31 user nova-compute[75234]: </domain>
Nov 29 03:11:31 user nova-compute[75234]:  {{(pid=75234) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7568}}
Nov 29 03:11:31 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-f77c1f58-64dc-4b23-9050-987d9f2e6066 tempest-VolumesActionsTest-1495826017 tempest-VolumesActionsTest-1495826017-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-11-29T03:11:28Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-VolumesActionsTest-instance-2001816366',display_name='tempest-VolumesActionsTest-instance-2001816366',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-volumesactionstest-instance-2001816366',id=21,image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data=None,key_name=None,keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='773aac44d1a74a439f4f750dfe8ac142',ramdisk_id='',reservation_id='r-3x0wrwrv',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-VolumesActionsTest-1495826017',owner_user_name='tempest-VolumesActionsTest-1495826017-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-11-29T03:11:30Z,user_data=None,user_id='324ad29d0b8b443086942e61e2be4ceb',uuid=1eb9b734-2711-4463-9743-e5fd95c6857b,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "6d9e1401-d167-4fb3-a59d-ca69b18ed417", "address": "fa:16:3e:24:48:f1", "network": {"id": "cb9b1cd8-2f38-4fe2-b680-380cff4b4d11", "bridge": "br-int", "label": "tempest-VolumesActionsTest-1501188375-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "773aac44d1a74a439f4f750dfe8ac142", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap6d9e1401-d1", "ovs_interfaceid": "6d9e1401-d167-4fb3-a59d-ca69b18ed417", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Nov 29 03:11:31 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-f77c1f58-64dc-4b23-9050-987d9f2e6066 tempest-VolumesActionsTest-1495826017 tempest-VolumesActionsTest-1495826017-project-member] Converting VIF {"id": "6d9e1401-d167-4fb3-a59d-ca69b18ed417", "address": "fa:16:3e:24:48:f1", "network": {"id": "cb9b1cd8-2f38-4fe2-b680-380cff4b4d11", "bridge": "br-int", "label": "tempest-VolumesActionsTest-1501188375-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "773aac44d1a74a439f4f750dfe8ac142", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap6d9e1401-d1", "ovs_interfaceid": "6d9e1401-d167-4fb3-a59d-ca69b18ed417", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:11:31 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-f77c1f58-64dc-4b23-9050-987d9f2e6066 tempest-VolumesActionsTest-1495826017 tempest-VolumesActionsTest-1495826017-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:24:48:f1,bridge_name='br-int',has_traffic_filtering=True,id=6d9e1401-d167-4fb3-a59d-ca69b18ed417,network=Network(cb9b1cd8-2f38-4fe2-b680-380cff4b4d11),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap6d9e1401-d1') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:11:31 user nova-compute[75234]: DEBUG os_vif [None req-f77c1f58-64dc-4b23-9050-987d9f2e6066 tempest-VolumesActionsTest-1495826017 tempest-VolumesActionsTest-1495826017-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:24:48:f1,bridge_name='br-int',has_traffic_filtering=True,id=6d9e1401-d167-4fb3-a59d-ca69b18ed417,network=Network(cb9b1cd8-2f38-4fe2-b680-380cff4b4d11),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap6d9e1401-d1') {{(pid=75234) plug /opt/stack/data/venv/lib/python3.10/site-packages/os_vif/__init__.py:76}}
Nov 29 03:11:31 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:11:31 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:11:31 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Nov 29 03:11:31 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:11:31 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbCreateCommand(_result=None, table=QoS, columns={'type': 'linux-noop', 'external_ids': {'id': '58559eee-fe52-5af9-916d-d86547852866', '_type': 'linux-noop'}}, row=False) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:11:31 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:11:31 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:11:31 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:11:31 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tap6d9e1401-d1, may_exist=True, interface_attrs={}) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:11:31 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Port, record=tap6d9e1401-d1, col_values=(('qos', UUID('8bd11dce-012b-44a1-aea9-7d6c1d6bf63c')),), if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:11:31 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DbSetCommand(_result=None, table=Interface, record=tap6d9e1401-d1, col_values=(('external_ids', {'iface-id': '6d9e1401-d167-4fb3-a59d-ca69b18ed417', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:24:48:f1', 'vm-uuid': '1eb9b734-2711-4463-9743-e5fd95c6857b'}),), if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:11:31 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:11:31 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:11:31 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:11:31 user nova-compute[75234]: INFO os_vif [None req-f77c1f58-64dc-4b23-9050-987d9f2e6066 tempest-VolumesActionsTest-1495826017 tempest-VolumesActionsTest-1495826017-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:24:48:f1,bridge_name='br-int',has_traffic_filtering=True,id=6d9e1401-d167-4fb3-a59d-ca69b18ed417,network=Network(cb9b1cd8-2f38-4fe2-b680-380cff4b4d11),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap6d9e1401-d1')
Nov 29 03:11:31 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-f77c1f58-64dc-4b23-9050-987d9f2e6066 tempest-VolumesActionsTest-1495826017 tempest-VolumesActionsTest-1495826017-project-member] No BDM found with device name vda, not building metadata. {{(pid=75234) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12089}}
Nov 29 03:11:31 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-f77c1f58-64dc-4b23-9050-987d9f2e6066 tempest-VolumesActionsTest-1495826017 tempest-VolumesActionsTest-1495826017-project-member] No VIF found with MAC fa:16:3e:24:48:f1, not building metadata {{(pid=75234) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Nov 29 03:11:31 user nova-compute[75234]: DEBUG nova.network.neutron [req-ea56f372-3ef4-411e-b6ad-f855dfaecc91 req-5205e618-552e-49ed-bee6-0e511b9b8584 service nova] [instance: 1eb9b734-2711-4463-9743-e5fd95c6857b] Updated VIF entry in instance network info cache for port 6d9e1401-d167-4fb3-a59d-ca69b18ed417. {{(pid=75234) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3481}}
Nov 29 03:11:31 user nova-compute[75234]: DEBUG nova.network.neutron [req-ea56f372-3ef4-411e-b6ad-f855dfaecc91 req-5205e618-552e-49ed-bee6-0e511b9b8584 service nova] [instance: 1eb9b734-2711-4463-9743-e5fd95c6857b] Updating instance_info_cache with network_info: [{"id": "6d9e1401-d167-4fb3-a59d-ca69b18ed417", "address": "fa:16:3e:24:48:f1", "network": {"id": "cb9b1cd8-2f38-4fe2-b680-380cff4b4d11", "bridge": "br-int", "label": "tempest-VolumesActionsTest-1501188375-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "773aac44d1a74a439f4f750dfe8ac142", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap6d9e1401-d1", "ovs_interfaceid": "6d9e1401-d167-4fb3-a59d-ca69b18ed417", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:11:31 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-ea56f372-3ef4-411e-b6ad-f855dfaecc91 req-5205e618-552e-49ed-bee6-0e511b9b8584 service nova] Releasing lock "refresh_cache-1eb9b734-2711-4463-9743-e5fd95c6857b" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:11:32 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:11:32 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:11:32 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:11:33 user nova-compute[75234]: DEBUG nova.compute.manager [req-971cb3f5-06ec-4377-b2fc-4d1cddbf40b8 req-94b8accc-8507-429b-9eec-269f03372281 service nova] [instance: 1eb9b734-2711-4463-9743-e5fd95c6857b] Received event network-vif-plugged-6d9e1401-d167-4fb3-a59d-ca69b18ed417 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:11:33 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-971cb3f5-06ec-4377-b2fc-4d1cddbf40b8 req-94b8accc-8507-429b-9eec-269f03372281 service nova] Acquiring lock "1eb9b734-2711-4463-9743-e5fd95c6857b-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:11:33 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-971cb3f5-06ec-4377-b2fc-4d1cddbf40b8 req-94b8accc-8507-429b-9eec-269f03372281 service nova] Lock "1eb9b734-2711-4463-9743-e5fd95c6857b-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:11:33 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-971cb3f5-06ec-4377-b2fc-4d1cddbf40b8 req-94b8accc-8507-429b-9eec-269f03372281 service nova] Lock "1eb9b734-2711-4463-9743-e5fd95c6857b-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:11:33 user nova-compute[75234]: DEBUG nova.compute.manager [req-971cb3f5-06ec-4377-b2fc-4d1cddbf40b8 req-94b8accc-8507-429b-9eec-269f03372281 service nova] [instance: 1eb9b734-2711-4463-9743-e5fd95c6857b] No waiting events found dispatching network-vif-plugged-6d9e1401-d167-4fb3-a59d-ca69b18ed417 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:11:33 user nova-compute[75234]: WARNING nova.compute.manager [req-971cb3f5-06ec-4377-b2fc-4d1cddbf40b8 req-94b8accc-8507-429b-9eec-269f03372281 service nova] [instance: 1eb9b734-2711-4463-9743-e5fd95c6857b] Received unexpected event network-vif-plugged-6d9e1401-d167-4fb3-a59d-ca69b18ed417 for instance with vm_state building and task_state spawning.
Nov 29 03:11:33 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:11:33 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:11:33 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:11:33 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:11:35 user nova-compute[75234]: DEBUG nova.compute.manager [req-56fb5913-056d-4bc2-b92b-69a42713038c req-c9b0dea6-7764-4b37-baa2-e49a6c89a4c6 service nova] [instance: 1eb9b734-2711-4463-9743-e5fd95c6857b] Received event network-vif-plugged-6d9e1401-d167-4fb3-a59d-ca69b18ed417 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:11:35 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-56fb5913-056d-4bc2-b92b-69a42713038c req-c9b0dea6-7764-4b37-baa2-e49a6c89a4c6 service nova] Acquiring lock "1eb9b734-2711-4463-9743-e5fd95c6857b-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:11:35 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-56fb5913-056d-4bc2-b92b-69a42713038c req-c9b0dea6-7764-4b37-baa2-e49a6c89a4c6 service nova] Lock "1eb9b734-2711-4463-9743-e5fd95c6857b-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:11:35 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-56fb5913-056d-4bc2-b92b-69a42713038c req-c9b0dea6-7764-4b37-baa2-e49a6c89a4c6 service nova] Lock "1eb9b734-2711-4463-9743-e5fd95c6857b-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:11:35 user nova-compute[75234]: DEBUG nova.compute.manager [req-56fb5913-056d-4bc2-b92b-69a42713038c req-c9b0dea6-7764-4b37-baa2-e49a6c89a4c6 service nova] [instance: 1eb9b734-2711-4463-9743-e5fd95c6857b] No waiting events found dispatching network-vif-plugged-6d9e1401-d167-4fb3-a59d-ca69b18ed417 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:11:35 user nova-compute[75234]: WARNING nova.compute.manager [req-56fb5913-056d-4bc2-b92b-69a42713038c req-c9b0dea6-7764-4b37-baa2-e49a6c89a4c6 service nova] [instance: 1eb9b734-2711-4463-9743-e5fd95c6857b] Received unexpected event network-vif-plugged-6d9e1401-d167-4fb3-a59d-ca69b18ed417 for instance with vm_state building and task_state spawning.
Nov 29 03:11:35 user nova-compute[75234]: DEBUG nova.compute.manager [None req-f77c1f58-64dc-4b23-9050-987d9f2e6066 tempest-VolumesActionsTest-1495826017 tempest-VolumesActionsTest-1495826017-project-member] [instance: 1eb9b734-2711-4463-9743-e5fd95c6857b] Instance event wait completed in 0 seconds for  {{(pid=75234) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Nov 29 03:11:35 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-f77c1f58-64dc-4b23-9050-987d9f2e6066 tempest-VolumesActionsTest-1495826017 tempest-VolumesActionsTest-1495826017-project-member] [instance: 1eb9b734-2711-4463-9743-e5fd95c6857b] Guest created on hypervisor {{(pid=75234) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4421}}
Nov 29 03:11:35 user nova-compute[75234]: INFO nova.virt.libvirt.driver [-] [instance: 1eb9b734-2711-4463-9743-e5fd95c6857b] Instance spawned successfully.
Nov 29 03:11:35 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-f77c1f58-64dc-4b23-9050-987d9f2e6066 tempest-VolumesActionsTest-1495826017 tempest-VolumesActionsTest-1495826017-project-member] [instance: 1eb9b734-2711-4463-9743-e5fd95c6857b] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:904}}
Nov 29 03:11:35 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-f77c1f58-64dc-4b23-9050-987d9f2e6066 tempest-VolumesActionsTest-1495826017 tempest-VolumesActionsTest-1495826017-project-member] [instance: 1eb9b734-2711-4463-9743-e5fd95c6857b] Found default for hw_cdrom_bus of ide {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:11:35 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-f77c1f58-64dc-4b23-9050-987d9f2e6066 tempest-VolumesActionsTest-1495826017 tempest-VolumesActionsTest-1495826017-project-member] [instance: 1eb9b734-2711-4463-9743-e5fd95c6857b] Found default for hw_disk_bus of virtio {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:11:35 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-f77c1f58-64dc-4b23-9050-987d9f2e6066 tempest-VolumesActionsTest-1495826017 tempest-VolumesActionsTest-1495826017-project-member] [instance: 1eb9b734-2711-4463-9743-e5fd95c6857b] Found default for hw_input_bus of None {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:11:35 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-f77c1f58-64dc-4b23-9050-987d9f2e6066 tempest-VolumesActionsTest-1495826017 tempest-VolumesActionsTest-1495826017-project-member] [instance: 1eb9b734-2711-4463-9743-e5fd95c6857b] Found default for hw_pointer_model of None {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:11:35 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-f77c1f58-64dc-4b23-9050-987d9f2e6066 tempest-VolumesActionsTest-1495826017 tempest-VolumesActionsTest-1495826017-project-member] [instance: 1eb9b734-2711-4463-9743-e5fd95c6857b] Found default for hw_video_model of virtio {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:11:35 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-f77c1f58-64dc-4b23-9050-987d9f2e6066 tempest-VolumesActionsTest-1495826017 tempest-VolumesActionsTest-1495826017-project-member] [instance: 1eb9b734-2711-4463-9743-e5fd95c6857b] Found default for hw_vif_model of virtio {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:11:35 user nova-compute[75234]: INFO nova.compute.manager [None req-f77c1f58-64dc-4b23-9050-987d9f2e6066 tempest-VolumesActionsTest-1495826017 tempest-VolumesActionsTest-1495826017-project-member] [instance: 1eb9b734-2711-4463-9743-e5fd95c6857b] Took 5.48 seconds to spawn the instance on the hypervisor.
Nov 29 03:11:35 user nova-compute[75234]: DEBUG nova.compute.manager [None req-f77c1f58-64dc-4b23-9050-987d9f2e6066 tempest-VolumesActionsTest-1495826017 tempest-VolumesActionsTest-1495826017-project-member] [instance: 1eb9b734-2711-4463-9743-e5fd95c6857b] Checking state {{(pid=75234) _get_power_state /opt/stack/nova/nova/compute/manager.py:1782}}
Nov 29 03:11:35 user nova-compute[75234]: INFO nova.compute.manager [None req-f77c1f58-64dc-4b23-9050-987d9f2e6066 tempest-VolumesActionsTest-1495826017 tempest-VolumesActionsTest-1495826017-project-member] [instance: 1eb9b734-2711-4463-9743-e5fd95c6857b] Took 6.30 seconds to build instance.
Nov 29 03:11:35 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-f77c1f58-64dc-4b23-9050-987d9f2e6066 tempest-VolumesActionsTest-1495826017 tempest-VolumesActionsTest-1495826017-project-member] Lock "1eb9b734-2711-4463-9743-e5fd95c6857b" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 6.389s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:11:36 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:11:36 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:11:41 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:11:41 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:11:44 user nova-compute[75234]: DEBUG nova.compute.manager [req-5471ea87-ade1-4cd3-9b36-e7125c06d5ae req-bdf61c91-9081-4daf-ac2f-17b1b32e7889 service nova] [instance: 6c0e0184-ca3a-48ec-a6e3-a0a1172260d6] Received event network-changed-50593025-edf1-44ec-b5f0-c92d6a807c23 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:11:44 user nova-compute[75234]: DEBUG nova.compute.manager [req-5471ea87-ade1-4cd3-9b36-e7125c06d5ae req-bdf61c91-9081-4daf-ac2f-17b1b32e7889 service nova] [instance: 6c0e0184-ca3a-48ec-a6e3-a0a1172260d6] Refreshing instance network info cache due to event network-changed-50593025-edf1-44ec-b5f0-c92d6a807c23. {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11117}}
Nov 29 03:11:44 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-5471ea87-ade1-4cd3-9b36-e7125c06d5ae req-bdf61c91-9081-4daf-ac2f-17b1b32e7889 service nova] Acquiring lock "refresh_cache-6c0e0184-ca3a-48ec-a6e3-a0a1172260d6" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:11:44 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-5471ea87-ade1-4cd3-9b36-e7125c06d5ae req-bdf61c91-9081-4daf-ac2f-17b1b32e7889 service nova] Acquired lock "refresh_cache-6c0e0184-ca3a-48ec-a6e3-a0a1172260d6" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:11:44 user nova-compute[75234]: DEBUG nova.network.neutron [req-5471ea87-ade1-4cd3-9b36-e7125c06d5ae req-bdf61c91-9081-4daf-ac2f-17b1b32e7889 service nova] [instance: 6c0e0184-ca3a-48ec-a6e3-a0a1172260d6] Refreshing network info cache for port 50593025-edf1-44ec-b5f0-c92d6a807c23 {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2006}}
Nov 29 03:11:45 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-c619ef90-45c7-40a6-b458-0b8b10bae2ef tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Acquiring lock "6c0e0184-ca3a-48ec-a6e3-a0a1172260d6" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:11:45 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-c619ef90-45c7-40a6-b458-0b8b10bae2ef tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Lock "6c0e0184-ca3a-48ec-a6e3-a0a1172260d6" acquired by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:11:45 user nova-compute[75234]: DEBUG nova.objects.instance [None req-c619ef90-45c7-40a6-b458-0b8b10bae2ef tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Lazy-loading 'flavor' on Instance uuid 6c0e0184-ca3a-48ec-a6e3-a0a1172260d6 {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:11:45 user nova-compute[75234]: DEBUG nova.network.neutron [req-5471ea87-ade1-4cd3-9b36-e7125c06d5ae req-bdf61c91-9081-4daf-ac2f-17b1b32e7889 service nova] [instance: 6c0e0184-ca3a-48ec-a6e3-a0a1172260d6] Updated VIF entry in instance network info cache for port 50593025-edf1-44ec-b5f0-c92d6a807c23. {{(pid=75234) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3481}}
Nov 29 03:11:45 user nova-compute[75234]: DEBUG nova.network.neutron [req-5471ea87-ade1-4cd3-9b36-e7125c06d5ae req-bdf61c91-9081-4daf-ac2f-17b1b32e7889 service nova] [instance: 6c0e0184-ca3a-48ec-a6e3-a0a1172260d6] Updating instance_info_cache with network_info: [{"id": "50593025-edf1-44ec-b5f0-c92d6a807c23", "address": "fa:16:3e:d8:a2:e7", "network": {"id": "1f946e69-dbec-429c-83f8-a186bad4f2cc", "bridge": "br-int", "label": "tempest-ServerActionsTestOtherA-486369423-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.146", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "c1cb3d1f12f242a785a688d59dbe3cd6", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap50593025-ed", "ovs_interfaceid": "50593025-edf1-44ec-b5f0-c92d6a807c23", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:11:45 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-5471ea87-ade1-4cd3-9b36-e7125c06d5ae req-bdf61c91-9081-4daf-ac2f-17b1b32e7889 service nova] Releasing lock "refresh_cache-6c0e0184-ca3a-48ec-a6e3-a0a1172260d6" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:11:45 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-c619ef90-45c7-40a6-b458-0b8b10bae2ef tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Lock "6c0e0184-ca3a-48ec-a6e3-a0a1172260d6" "released" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: held 0.051s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:11:45 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-c619ef90-45c7-40a6-b458-0b8b10bae2ef tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Acquiring lock "6c0e0184-ca3a-48ec-a6e3-a0a1172260d6" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:11:45 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-c619ef90-45c7-40a6-b458-0b8b10bae2ef tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Lock "6c0e0184-ca3a-48ec-a6e3-a0a1172260d6" acquired by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:11:45 user nova-compute[75234]: INFO nova.compute.manager [None req-c619ef90-45c7-40a6-b458-0b8b10bae2ef tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] [instance: 6c0e0184-ca3a-48ec-a6e3-a0a1172260d6] Attaching volume 56237f4c-4138-4f50-aafb-9c0f9d4bb1cf to /dev/vdb
Nov 29 03:11:45 user nova-compute[75234]: DEBUG os_brick.utils [None req-c619ef90-45c7-40a6-b458-0b8b10bae2ef tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:176}}
Nov 29 03:11:45 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:11:45 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.017s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:11:45 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[ad3ead26-56ff-44b2-bc02-fc413472f72c]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:dd1368c5ba2c\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:11:45 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt -v / -n -o SOURCE {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:11:45 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt -v / -n -o SOURCE" returned: 0 in 0.016s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:11:45 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[5f46eb90-610c-44cf-bfd4-2668fc33287d]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:11:45 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:11:45 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.018s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:11:45 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[1ce19dd5-9a8f-441b-ae5c-481ae865a7b6]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:11:45 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[bec99cc7-181d-477c-9575-f653b01550f3]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:11:45 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-c619ef90-45c7-40a6-b458-0b8b10bae2ef tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Running cmd (subprocess): nvme version {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:11:45 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-c619ef90-45c7-40a6-b458-0b8b10bae2ef tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] 'nvme version' failed. Not Retrying. {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:473}}
Nov 29 03:11:45 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.nvmeof [None req-c619ef90-45c7-40a6-b458-0b8b10bae2ef tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] nvme not present on system {{(pid=75234) nvme_present /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/nvmeof.py:756}}
Nov 29 03:11:45 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:11:45 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:473}}
Nov 29 03:11:45 user nova-compute[75234]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Nov 29 03:11:45 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[ab6df77f-d34a-4bee-9de5-d50e699ede48]: (4, '') {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:11:45 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.lightos [None req-c619ef90-45c7-40a6-b458-0b8b10bae2ef tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=75234) find_dsc /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/lightos.py:98}}
Nov 29 03:11:45 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.lightos [None req-c619ef90-45c7-40a6-b458-0b8b10bae2ef tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=75234) get_connector_properties /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/lightos.py:76}}
Nov 29 03:11:45 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.lightos [None req-c619ef90-45c7-40a6-b458-0b8b10bae2ef tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] LIGHTOS: no hostnqn found. {{(pid=75234) get_connector_properties /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/lightos.py:84}}
Nov 29 03:11:45 user nova-compute[75234]: DEBUG os_brick.utils [None req-c619ef90-45c7-40a6-b458-0b8b10bae2ef tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] <== get_connector_properties: return (120ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:dd1368c5ba2c', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:203}}
Nov 29 03:11:45 user nova-compute[75234]: DEBUG nova.virt.block_device [None req-c619ef90-45c7-40a6-b458-0b8b10bae2ef tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] [instance: 6c0e0184-ca3a-48ec-a6e3-a0a1172260d6] Updating existing volume attachment record: 60720872-9b7b-4b81-9d15-4363c9d77de8 {{(pid=75234) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Nov 29 03:11:46 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._cleanup_incomplete_migrations {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:11:46 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Cleaning up deleted instances with incomplete migration  {{(pid=75234) _cleanup_incomplete_migrations /opt/stack/nova/nova/compute/manager.py:11247}}
Nov 29 03:11:46 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:11:46 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:11:51 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:11:51 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:11:51 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Starting heal instance info cache {{(pid=75234) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9922}}
Nov 29 03:11:51 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:11:51 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Acquiring lock "refresh_cache-8297c264-a213-44ee-be46-5c08eabda9f7" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:11:51 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Acquired lock "refresh_cache-8297c264-a213-44ee-be46-5c08eabda9f7" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:11:51 user nova-compute[75234]: DEBUG nova.network.neutron [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] [instance: 8297c264-a213-44ee-be46-5c08eabda9f7] Forcefully refreshing network info cache for instance {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2003}}
Nov 29 03:11:51 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:11:52 user nova-compute[75234]: DEBUG nova.network.neutron [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] [instance: 8297c264-a213-44ee-be46-5c08eabda9f7] Updating instance_info_cache with network_info: [{"id": "d26334d2-8040-4022-b658-69be1cf6a025", "address": "fa:16:3e:5f:9d:5a", "network": {"id": "e0366490-9fe2-4429-88c8-6625128e645d", "bridge": "br-int", "label": "tempest-ServerStableDeviceRescueTest-804259918-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.31", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "745bdb3c72794155829dc0a69f35b321", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapd26334d2-80", "ovs_interfaceid": "d26334d2-8040-4022-b658-69be1cf6a025", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:11:52 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Releasing lock "refresh_cache-8297c264-a213-44ee-be46-5c08eabda9f7" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:11:52 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] [instance: 8297c264-a213-44ee-be46-5c08eabda9f7] Updated the network info_cache for instance {{(pid=75234) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9993}}
Nov 29 03:11:52 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._cleanup_expired_console_auth_tokens {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:11:52 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:11:52 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:11:52 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=75234) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10541}}
Nov 29 03:11:52 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-c619ef90-45c7-40a6-b458-0b8b10bae2ef tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] [instance: 6c0e0184-ca3a-48ec-a6e3-a0a1172260d6] Calling os-brick to attach iSCSI Volume {{(pid=75234) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Nov 29 03:11:52 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-c619ef90-45c7-40a6-b458-0b8b10bae2ef tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f8a5050c9a0>, 'connection_properties': {'target_iqn': 'iqn.2010-10.org.openstack:56237f4c-4138-4f50-aafb-9c0f9d4bb1cf', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'hbYaegTr', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:176}}
Nov 29 03:11:52 user nova-compute[75234]: WARNING os_brick.initiator.connectors.base [None req-c619ef90-45c7-40a6-b458-0b8b10bae2ef tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Nov 29 03:11:52 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-c619ef90-45c7-40a6-b458-0b8b10bae2ef tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:68}}
Nov 29 03:11:52 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-c619ef90-45c7-40a6-b458-0b8b10bae2ef tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:73}}
Nov 29 03:11:52 user nova-compute[75234]: WARNING os_brick.initiator.connectors.base [None req-c619ef90-45c7-40a6-b458-0b8b10bae2ef tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Nov 29 03:11:52 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-c619ef90-45c7-40a6-b458-0b8b10bae2ef tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:56237f4c-4138-4f50-aafb-9c0f9d4bb1cf" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:68}}
Nov 29 03:11:52 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-c619ef90-45c7-40a6-b458-0b8b10bae2ef tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:56237f4c-4138-4f50-aafb-9c0f9d4bb1cf" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:73}}
Nov 29 03:11:52 user nova-compute[75234]: INFO os_brick.initiator.connectors.iscsi [None req-c619ef90-45c7-40a6-b458-0b8b10bae2ef tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Nov 29 03:11:52 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:56237f4c-4138-4f50-aafb-9c0f9d4bb1cf -p 172.16.0.220:3260 {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:11:52 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:56237f4c-4138-4f50-aafb-9c0f9d4bb1cf -p 172.16.0.220:3260" returned: 21 in 0.018s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:11:52 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[67b95370-c38a-4b5c-973a-0c4165cad3e3]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:11:52 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-c619ef90-45c7-40a6-b458-0b8b10bae2ef tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Nov 29 03:11:52 user nova-compute[75234]:  {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:11:52 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:56237f4c-4138-4f50-aafb-9c0f9d4bb1cf -p 172.16.0.220:3260 --interface default --op new {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:11:52 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:56237f4c-4138-4f50-aafb-9c0f9d4bb1cf -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.019s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:11:52 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[43cd3ace-8539-4d5a-80ab-893b49add95a]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:56237f4c-4138-4f50-aafb-9c0f9d4bb1cf] added\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:11:52 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-c619ef90-45c7-40a6-b458-0b8b10bae2ef tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:56237f4c-4138-4f50-aafb-9c0f9d4bb1cf] added
Nov 29 03:11:52 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:11:52 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:56237f4c-4138-4f50-aafb-9c0f9d4bb1cf -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:11:52 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:56237f4c-4138-4f50-aafb-9c0f9d4bb1cf -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.018s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:11:52 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[d9ce50bd-382d-4e92-b474-cea4344a7335]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:11:52 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-c619ef90-45c7-40a6-b458-0b8b10bae2ef tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:11:52 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:56237f4c-4138-4f50-aafb-9c0f9d4bb1cf -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:11:52 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:56237f4c-4138-4f50-aafb-9c0f9d4bb1cf -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.018s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:11:52 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[c6454937-9e33-4875-b17a-ee47dd56592b]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:11:52 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-c619ef90-45c7-40a6-b458-0b8b10bae2ef tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:11:52 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:56237f4c-4138-4f50-aafb-9c0f9d4bb1cf -p 172.16.0.220:3260 --op update -n node.session.auth.username -v hbYaegTr {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:11:52 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:56237f4c-4138-4f50-aafb-9c0f9d4bb1cf -p 172.16.0.220:3260 --op update -n node.session.auth.username -v hbYaegTr" returned: 0 in 0.019s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:11:52 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[e1000ba2-7d3a-49ab-b98b-f6b9678c5753]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:11:52 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-c619ef90-45c7-40a6-b458-0b8b10bae2ef tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'hbYaegTr'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:11:52 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:56237f4c-4138-4f50-aafb-9c0f9d4bb1cf -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:11:52 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:56237f4c-4138-4f50-aafb-9c0f9d4bb1cf -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.017s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:11:52 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[264747ec-5d62-4fb3-82b9-eb912a2a0dad]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:11:52 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-c619ef90-45c7-40a6-b458-0b8b10bae2ef tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:11:52 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:11:52 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.021s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:11:52 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[502d00a8-a7c1-4fe9-a7d5-fe9ca1e77c01]: (4, ('tcp: [15] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a7ef35bf-017a-4fd9-9182-79c1b2ab4f98 (non-flash)\ntcp: [16] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672 (non-flash)\ntcp: [17] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b (non-flash)\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:11:52 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-c619ef90-45c7-40a6-b458-0b8b10bae2ef tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [15] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a7ef35bf-017a-4fd9-9182-79c1b2ab4f98 (non-flash)
Nov 29 03:11:52 user nova-compute[75234]: tcp: [16] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672 (non-flash)
Nov 29 03:11:52 user nova-compute[75234]: tcp: [17] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b (non-flash)
Nov 29 03:11:52 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:11:52 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-c619ef90-45c7-40a6-b458-0b8b10bae2ef tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] iscsi session list stdout=tcp: [15] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a7ef35bf-017a-4fd9-9182-79c1b2ab4f98 (non-flash)
Nov 29 03:11:52 user nova-compute[75234]: tcp: [16] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672 (non-flash)
Nov 29 03:11:52 user nova-compute[75234]: tcp: [17] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b (non-flash)
Nov 29 03:11:52 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsi_session /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Nov 29 03:11:52 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:56237f4c-4138-4f50-aafb-9c0f9d4bb1cf -p 172.16.0.220:3260 --login {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:11:52 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:56237f4c-4138-4f50-aafb-9c0f9d4bb1cf -p 172.16.0.220:3260 --login" returned: 0 in 0.064s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:11:52 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[e7d95935-f64b-4c34-bad4-d63545592ac3]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:56237f4c-4138-4f50-aafb-9c0f9d4bb1cf, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:56237f4c-4138-4f50-aafb-9c0f9d4bb1cf, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:11:52 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-c619ef90-45c7-40a6-b458-0b8b10bae2ef tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:56237f4c-4138-4f50-aafb-9c0f9d4bb1cf, portal: 172.16.0.220,3260]
Nov 29 03:11:52 user nova-compute[75234]: Login to [iface: default, target: iqn.2010-10.org.openstack:56237f4c-4138-4f50-aafb-9c0f9d4bb1cf, portal: 172.16.0.220,3260] successful.
Nov 29 03:11:52 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:11:52 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:56237f4c-4138-4f50-aafb-9c0f9d4bb1cf -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:11:52 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:56237f4c-4138-4f50-aafb-9c0f9d4bb1cf -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.019s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:11:52 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[13397228-33ff-48a8-aa70-fc555a3ab4ab]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:11:52 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-c619ef90-45c7-40a6-b458-0b8b10bae2ef tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:11:52 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:11:52 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.024s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:11:52 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[4262b94e-8e8b-43db-8bae-b45bdbca70bd]: (4, ('tcp: [15] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a7ef35bf-017a-4fd9-9182-79c1b2ab4f98 (non-flash)\ntcp: [16] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672 (non-flash)\ntcp: [17] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b (non-flash)\ntcp: [18] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:56237f4c-4138-4f50-aafb-9c0f9d4bb1cf (non-flash)\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:11:52 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-c619ef90-45c7-40a6-b458-0b8b10bae2ef tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [15] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a7ef35bf-017a-4fd9-9182-79c1b2ab4f98 (non-flash)
Nov 29 03:11:52 user nova-compute[75234]: tcp: [16] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672 (non-flash)
Nov 29 03:11:52 user nova-compute[75234]: tcp: [17] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b (non-flash)
Nov 29 03:11:52 user nova-compute[75234]: tcp: [18] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:56237f4c-4138-4f50-aafb-9c0f9d4bb1cf (non-flash)
Nov 29 03:11:52 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:11:52 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-c619ef90-45c7-40a6-b458-0b8b10bae2ef tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] iscsi session list stdout=tcp: [15] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a7ef35bf-017a-4fd9-9182-79c1b2ab4f98 (non-flash)
Nov 29 03:11:52 user nova-compute[75234]: tcp: [16] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672 (non-flash)
Nov 29 03:11:52 user nova-compute[75234]: tcp: [17] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b (non-flash)
Nov 29 03:11:52 user nova-compute[75234]: tcp: [18] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:56237f4c-4138-4f50-aafb-9c0f9d4bb1cf (non-flash)
Nov 29 03:11:52 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsi_session /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Nov 29 03:11:52 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-c619ef90-45c7-40a6-b458-0b8b10bae2ef tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:56237f4c-4138-4f50-aafb-9c0f9d4bb1cf" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.276s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:87}}
Nov 29 03:11:52 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-c619ef90-45c7-40a6-b458-0b8b10bae2ef tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Connected to 172.16.0.220:3260 {{(pid=75234) _connect_vol /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:644}}
Nov 29 03:11:52 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-c619ef90-45c7-40a6-b458-0b8b10bae2ef tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] HCTL ('33', '-', '-', 0) found on session 18 with lun 0 {{(pid=75234) get_hctl /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:770}}
Nov 29 03:11:52 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-c619ef90-45c7-40a6-b458-0b8b10bae2ef tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Scanning host 33 c: -, t: -, l: 0) {{(pid=75234) scan_iscsi /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:795}}
Nov 29 03:11:52 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host33/scan {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:11:52 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host33/scan" returned: 0 in 0.018s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:11:52 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[f6446ed9-1572-4050-b8cd-d6920c542b2b]: (4, ('- - 0', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:11:52 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-c619ef90-45c7-40a6-b458-0b8b10bae2ef tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Searching for a device in session 18 and hctl ['33', '*', '*', 0] yield: None {{(pid=75234) device_name_by_hctl /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:789}}
Nov 29 03:11:53 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-c619ef90-45c7-40a6-b458-0b8b10bae2ef tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Searching for a device in session 18 and hctl ['33', '*', '*', 0] yield: sdb {{(pid=75234) device_name_by_hctl /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:789}}
Nov 29 03:11:53 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-c619ef90-45c7-40a6-b458-0b8b10bae2ef tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Connected to sdb using {'target_iqn': 'iqn.2010-10.org.openstack:56237f4c-4138-4f50-aafb-9c0f9d4bb1cf', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'hbYaegTr', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False} {{(pid=75234) _connect_vol /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:672}}
Nov 29 03:11:53 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-c619ef90-45c7-40a6-b458-0b8b10bae2ef tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.310s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:87}}
Nov 29 03:11:53 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-c619ef90-45c7-40a6-b458-0b8b10bae2ef tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] <== connect_volume: return (1311ms) {'type': 'block', 'scsi_wwn': '23932316239353532', 'path': '/dev/sdb'} {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:203}}
Nov 29 03:11:53 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-c619ef90-45c7-40a6-b458-0b8b10bae2ef tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] [instance: 6c0e0184-ca3a-48ec-a6e3-a0a1172260d6] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '23932316239353532', 'path': '/dev/sdb'} {{(pid=75234) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Nov 29 03:11:53 user nova-compute[75234]: DEBUG nova.objects.instance [None req-c619ef90-45c7-40a6-b458-0b8b10bae2ef tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Lazy-loading 'flavor' on Instance uuid 6c0e0184-ca3a-48ec-a6e3-a0a1172260d6 {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:11:53 user nova-compute[75234]: DEBUG nova.virt.libvirt.guest [None req-c619ef90-45c7-40a6-b458-0b8b10bae2ef tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] attach device xml: <disk type="block" device="disk">
Nov 29 03:11:53 user nova-compute[75234]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Nov 29 03:11:53 user nova-compute[75234]:   <source dev="/dev/sdb"/>
Nov 29 03:11:53 user nova-compute[75234]:   <target dev="vdb" bus="virtio"/>
Nov 29 03:11:53 user nova-compute[75234]:   <serial>56237f4c-4138-4f50-aafb-9c0f9d4bb1cf</serial>
Nov 29 03:11:53 user nova-compute[75234]: </disk>
Nov 29 03:11:53 user nova-compute[75234]:  {{(pid=75234) attach_device /opt/stack/nova/nova/virt/libvirt/guest.py:339}}
Nov 29 03:11:54 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-c619ef90-45c7-40a6-b458-0b8b10bae2ef tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] No BDM found with device name vda, not building metadata. {{(pid=75234) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12089}}
Nov 29 03:11:54 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-c619ef90-45c7-40a6-b458-0b8b10bae2ef tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] No BDM found with device name vdb, not building metadata. {{(pid=75234) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12089}}
Nov 29 03:11:54 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-c619ef90-45c7-40a6-b458-0b8b10bae2ef tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] No VIF found with MAC fa:16:3e:d8:a2:e7, not building metadata {{(pid=75234) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Nov 29 03:11:54 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:11:54 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-c619ef90-45c7-40a6-b458-0b8b10bae2ef tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Lock "6c0e0184-ca3a-48ec-a6e3-a0a1172260d6" "released" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: held 9.198s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:11:54 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:11:55 user nova-compute[75234]: INFO nova.compute.manager [None req-4a795bce-c0be-4707-baff-68c516a2f74c tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] [instance: 6c0e0184-ca3a-48ec-a6e3-a0a1172260d6] Rebuilding instance
Nov 29 03:11:55 user nova-compute[75234]: DEBUG nova.compute.manager [None req-4a795bce-c0be-4707-baff-68c516a2f74c tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] [instance: 6c0e0184-ca3a-48ec-a6e3-a0a1172260d6] Checking state {{(pid=75234) _get_power_state /opt/stack/nova/nova/compute/manager.py:1782}}
Nov 29 03:11:55 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:11:56 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:11:56 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:11:56 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:11:56 user nova-compute[75234]: DEBUG nova.compute.manager [req-97bd42a5-c1f8-4184-a52e-d65a351c9984 req-2defc074-9545-4935-ad7e-aa5f2f3e94f1 service nova] [instance: 6c0e0184-ca3a-48ec-a6e3-a0a1172260d6] Received event network-vif-unplugged-50593025-edf1-44ec-b5f0-c92d6a807c23 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:11:56 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-97bd42a5-c1f8-4184-a52e-d65a351c9984 req-2defc074-9545-4935-ad7e-aa5f2f3e94f1 service nova] Acquiring lock "6c0e0184-ca3a-48ec-a6e3-a0a1172260d6-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:11:56 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-97bd42a5-c1f8-4184-a52e-d65a351c9984 req-2defc074-9545-4935-ad7e-aa5f2f3e94f1 service nova] Lock "6c0e0184-ca3a-48ec-a6e3-a0a1172260d6-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:11:56 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-97bd42a5-c1f8-4184-a52e-d65a351c9984 req-2defc074-9545-4935-ad7e-aa5f2f3e94f1 service nova] Lock "6c0e0184-ca3a-48ec-a6e3-a0a1172260d6-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:11:56 user nova-compute[75234]: DEBUG nova.compute.manager [req-97bd42a5-c1f8-4184-a52e-d65a351c9984 req-2defc074-9545-4935-ad7e-aa5f2f3e94f1 service nova] [instance: 6c0e0184-ca3a-48ec-a6e3-a0a1172260d6] No waiting events found dispatching network-vif-unplugged-50593025-edf1-44ec-b5f0-c92d6a807c23 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:11:56 user nova-compute[75234]: WARNING nova.compute.manager [req-97bd42a5-c1f8-4184-a52e-d65a351c9984 req-2defc074-9545-4935-ad7e-aa5f2f3e94f1 service nova] [instance: 6c0e0184-ca3a-48ec-a6e3-a0a1172260d6] Received unexpected event network-vif-unplugged-50593025-edf1-44ec-b5f0-c92d6a807c23 for instance with vm_state active and task_state rebuilding.
Nov 29 03:11:56 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager.update_available_resource {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:11:56 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:11:56 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:11:56 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:11:56 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Auditing locally available compute resources for user (node: user) {{(pid=75234) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:907}}
Nov 29 03:11:56 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:11:56 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:11:56 user nova-compute[75234]: INFO nova.virt.libvirt.driver [-] [instance: 6c0e0184-ca3a-48ec-a6e3-a0a1172260d6] Instance destroyed successfully.
Nov 29 03:11:56 user nova-compute[75234]: INFO nova.compute.manager [None req-4a795bce-c0be-4707-baff-68c516a2f74c tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] [instance: 6c0e0184-ca3a-48ec-a6e3-a0a1172260d6] Detaching volume 56237f4c-4138-4f50-aafb-9c0f9d4bb1cf
Nov 29 03:11:56 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/1eb9b734-2711-4463-9743-e5fd95c6857b/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:11:56 user nova-compute[75234]: INFO nova.virt.block_device [None req-4a795bce-c0be-4707-baff-68c516a2f74c tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] [instance: 6c0e0184-ca3a-48ec-a6e3-a0a1172260d6] Attempting to driver detach volume 56237f4c-4138-4f50-aafb-9c0f9d4bb1cf from mountpoint /dev/vdb
Nov 29 03:11:56 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-4a795bce-c0be-4707-baff-68c516a2f74c tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Attempting to detach device vdb from instance 6c0e0184-ca3a-48ec-a6e3-a0a1172260d6 from the persistent domain config. {{(pid=75234) _detach_from_persistent /opt/stack/nova/nova/virt/libvirt/driver.py:2487}}
Nov 29 03:11:56 user nova-compute[75234]: DEBUG nova.virt.libvirt.guest [None req-4a795bce-c0be-4707-baff-68c516a2f74c tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] detach device xml: <disk type="block" device="disk">
Nov 29 03:11:56 user nova-compute[75234]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Nov 29 03:11:56 user nova-compute[75234]:   <source dev="/dev/sdb"/>
Nov 29 03:11:56 user nova-compute[75234]:   <target dev="vdb" bus="virtio"/>
Nov 29 03:11:56 user nova-compute[75234]:   <serial>56237f4c-4138-4f50-aafb-9c0f9d4bb1cf</serial>
Nov 29 03:11:56 user nova-compute[75234]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x07" function="0x0"/>
Nov 29 03:11:56 user nova-compute[75234]: </disk>
Nov 29 03:11:56 user nova-compute[75234]:  {{(pid=75234) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Nov 29 03:11:56 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-4a795bce-c0be-4707-baff-68c516a2f74c tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Successfully detached device vdb from instance 6c0e0184-ca3a-48ec-a6e3-a0a1172260d6 from the persistent domain config.
Nov 29 03:11:56 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-4a795bce-c0be-4707-baff-68c516a2f74c tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] [instance: 6c0e0184-ca3a-48ec-a6e3-a0a1172260d6] calling os-brick to detach iSCSI Volume {{(pid=75234) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Nov 29 03:11:56 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-4a795bce-c0be-4707-baff-68c516a2f74c tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f8a5050c9a0>, {'target_iqn': 'iqn.2010-10.org.openstack:56237f4c-4138-4f50-aafb-9c0f9d4bb1cf', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'hbYaegTr', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdb'}, None), 'kwargs': False}}" {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:176}}
Nov 29 03:11:56 user nova-compute[75234]: WARNING os_brick.initiator.connectors.base [None req-4a795bce-c0be-4707-baff-68c516a2f74c tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release: NotImplementedError
Nov 29 03:11:56 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-4a795bce-c0be-4707-baff-68c516a2f74c tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:68}}
Nov 29 03:11:56 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-4a795bce-c0be-4707-baff-68c516a2f74c tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.002s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:73}}
Nov 29 03:11:56 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:11:56 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.020s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:11:56 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[81daf60d-f88b-4e6b-b88a-ec6322fa58a9]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:a7ef35bf-017a-4fd9-9182-79c1b2ab4f98\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:56237f4c-4138-4f50-aafb-9c0f9d4bb1cf\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:11:56 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-4a795bce-c0be-4707-baff-68c516a2f74c tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS:
Nov 29 03:11:56 user nova-compute[75234]: No targets found.
Nov 29 03:11:56 user nova-compute[75234]: iSNS:
Nov 29 03:11:56 user nova-compute[75234]: No targets found.
Nov 29 03:11:56 user nova-compute[75234]: STATIC:
Nov 29 03:11:56 user nova-compute[75234]: Target: iqn.2010-10.org.openstack:a7ef35bf-017a-4fd9-9182-79c1b2ab4f98
Nov 29 03:11:56 user nova-compute[75234]:         Portal: 172.16.0.220:3260,-1
Nov 29 03:11:56 user nova-compute[75234]:                 Iface Name: default
Nov 29 03:11:56 user nova-compute[75234]: Target: iqn.2010-10.org.openstack:56237f4c-4138-4f50-aafb-9c0f9d4bb1cf
Nov 29 03:11:56 user nova-compute[75234]:         Portal: 172.16.0.220:3260,-1
Nov 29 03:11:56 user nova-compute[75234]:                 Iface Name: default
Nov 29 03:11:56 user nova-compute[75234]: Target: iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672
Nov 29 03:11:56 user nova-compute[75234]:         Portal: 172.16.0.220:3260,-1
Nov 29 03:11:56 user nova-compute[75234]:                 Iface Name: default
Nov 29 03:11:56 user nova-compute[75234]: Target: iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b
Nov 29 03:11:56 user nova-compute[75234]:         Portal: 172.16.0.220:3260,-1
Nov 29 03:11:56 user nova-compute[75234]:                 Iface Name: default
Nov 29 03:11:56 user nova-compute[75234]: FIRMWARE:
Nov 29 03:11:56 user nova-compute[75234]: No targets found.
Nov 29 03:11:56 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:11:56 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-4a795bce-c0be-4707-baff-68c516a2f74c tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Regex to get portals from discoverydb: ^SENDTARGETS:
Nov 29 03:11:56 user nova-compute[75234]: .*?^DiscoveryAddress: 172.16.0.220,3260.*?
Nov 29 03:11:56 user nova-compute[75234]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=75234) _get_discoverydb_portals /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:382}}
Nov 29 03:11:56 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-4a795bce-c0be-4707-baff-68c516a2f74c tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:56237f4c-4138-4f50-aafb-9c0f9d4bb1cf', 0)] {{(pid=75234) _get_connection_devices /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:830}}
Nov 29 03:11:56 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:11:56 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.016s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:11:56 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[8300974b-2404-4267-867a-76566afc4d43]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:56237f4c-4138-4f50-aafb-9c0f9d4bb1cf\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:a7ef35bf-017a-4fd9-9182-79c1b2ab4f98\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:11:56 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:11:57 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.025s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:11:57 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[f6bb78c8-569f-40ca-90f1-efbc3a7b99cf]: (4, ('tcp: [15] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a7ef35bf-017a-4fd9-9182-79c1b2ab4f98 (non-flash)\ntcp: [16] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672 (non-flash)\ntcp: [17] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b (non-flash)\ntcp: [18] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:56237f4c-4138-4f50-aafb-9c0f9d4bb1cf (non-flash)\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:11:57 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-4a795bce-c0be-4707-baff-68c516a2f74c tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [15] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a7ef35bf-017a-4fd9-9182-79c1b2ab4f98 (non-flash)
Nov 29 03:11:57 user nova-compute[75234]: tcp: [16] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672 (non-flash)
Nov 29 03:11:57 user nova-compute[75234]: tcp: [17] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b (non-flash)
Nov 29 03:11:57 user nova-compute[75234]: tcp: [18] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:56237f4c-4138-4f50-aafb-9c0f9d4bb1cf (non-flash)
Nov 29 03:11:57 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:11:57 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-4a795bce-c0be-4707-baff-68c516a2f74c tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] iscsi session list stdout=tcp: [15] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a7ef35bf-017a-4fd9-9182-79c1b2ab4f98 (non-flash)
Nov 29 03:11:57 user nova-compute[75234]: tcp: [16] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672 (non-flash)
Nov 29 03:11:57 user nova-compute[75234]: tcp: [17] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b (non-flash)
Nov 29 03:11:57 user nova-compute[75234]: tcp: [18] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:56237f4c-4138-4f50-aafb-9c0f9d4bb1cf (non-flash)
Nov 29 03:11:57 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsi_session /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Nov 29 03:11:57 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-4a795bce-c0be-4707-baff-68c516a2f74c tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7f8a6017c700>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:56237f4c-4138-4f50-aafb-9c0f9d4bb1cf'): ({'sdb'}, set())}) {{(pid=75234) _get_connection_devices /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:863}}
Nov 29 03:11:57 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-4a795bce-c0be-4707-baff-68c516a2f74c tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Removing single pathed devices sdb {{(pid=75234) remove_connection /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:374}}
Nov 29 03:11:57 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:11:57 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/1eb9b734-2711-4463-9743-e5fd95c6857b/disk --force-share --output=json" returned: 0 in 0.190s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:11:57 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/1eb9b734-2711-4463-9743-e5fd95c6857b/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:11:57 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.023s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:11:57 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[c2262f74-ef1c-47c6-9378-80ea7a05889d]: (4, ('path checker states:\nup                  4\n\npaths: 0\nbusy: False\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:11:57 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdb {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:11:57 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdb" returned: 0 in 0.019s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:11:57 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[8c9545bc-8d3a-45ef-8656-6477026ed406]: (4, ('ok\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:11:57 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-4a795bce-c0be-4707-baff-68c516a2f74c tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Flushing IO for device /dev/sdb {{(pid=75234) flush_device_io /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:434}}
Nov 29 03:11:57 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdb {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:11:57 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdb" returned: 0 in 0.013s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:11:57 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[8b298e5b-6dad-4194-8cbc-71ef52128555]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:11:57 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-4a795bce-c0be-4707-baff-68c516a2f74c tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Remove SCSI device /dev/sdb with /sys/block/sdb/device/delete {{(pid=75234) remove_scsi_device /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:148}}
Nov 29 03:11:57 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdb/device/delete {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:11:57 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdb/device/delete" returned: 0 in 0.048s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:11:57 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[a4a9c933-6297-4927-99cc-80358c385c17]: (4, ('1', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:11:57 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-4a795bce-c0be-4707-baff-68c516a2f74c tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Checking to see if SCSI volumes sdb have been removed. {{(pid=75234) wait_for_volumes_removal /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:156}}
Nov 29 03:11:57 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-4a795bce-c0be-4707-baff-68c516a2f74c tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] SCSI volumes sdb have been removed. {{(pid=75234) wait_for_volumes_removal /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:166}}
Nov 29 03:11:57 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-4a795bce-c0be-4707-baff-68c516a2f74c tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:56237f4c-4138-4f50-aafb-9c0f9d4bb1cf')] {{(pid=75234) _disconnect_connection /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Nov 29 03:11:57 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:56237f4c-4138-4f50-aafb-9c0f9d4bb1cf -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:11:57 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:56237f4c-4138-4f50-aafb-9c0f9d4bb1cf -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.022s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:11:57 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[f3c5b40b-6ce0-41f9-a0f0-879a030abd5b]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:11:57 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-4a795bce-c0be-4707-baff-68c516a2f74c tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:11:57 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:56237f4c-4138-4f50-aafb-9c0f9d4bb1cf -p 172.16.0.220:3260 --logout {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:11:57 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/1eb9b734-2711-4463-9743-e5fd95c6857b/disk --force-share --output=json" returned: 0 in 0.194s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:11:57 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a9350a18-14e6-42c6-a150-0a1487f20ad9/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:11:57 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:56237f4c-4138-4f50-aafb-9c0f9d4bb1cf -p 172.16.0.220:3260 --logout" returned: 0 in 0.054s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:11:57 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[a411450e-0742-4bcb-b92c-b11a502f2524]: (4, ('Logging out of session [sid: 18, target: iqn.2010-10.org.openstack:56237f4c-4138-4f50-aafb-9c0f9d4bb1cf, portal: 172.16.0.220,3260]\nLogout of [sid: 18, target: iqn.2010-10.org.openstack:56237f4c-4138-4f50-aafb-9c0f9d4bb1cf, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:11:57 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-4a795bce-c0be-4707-baff-68c516a2f74c tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 18, target: iqn.2010-10.org.openstack:56237f4c-4138-4f50-aafb-9c0f9d4bb1cf, portal: 172.16.0.220,3260]
Nov 29 03:11:57 user nova-compute[75234]: Logout of [sid: 18, target: iqn.2010-10.org.openstack:56237f4c-4138-4f50-aafb-9c0f9d4bb1cf, portal: 172.16.0.220,3260] successful.
Nov 29 03:11:57 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:11:57 user nova-compute[75234]: DEBUG nova.network.neutron [None req-0d805482-05bf-44f8-bb11-e441afd66aac tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] [instance: a9350a18-14e6-42c6-a150-0a1487f20ad9] Port 52024efc-8cde-4610-b12c-198f19d75ccf binding to destination host user is already ACTIVE {{(pid=75234) migrate_instance_start /opt/stack/nova/nova/network/neutron.py:3170}}
Nov 29 03:11:57 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-0d805482-05bf-44f8-bb11-e441afd66aac tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Acquiring lock "refresh_cache-a9350a18-14e6-42c6-a150-0a1487f20ad9" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:11:57 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-0d805482-05bf-44f8-bb11-e441afd66aac tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Acquired lock "refresh_cache-a9350a18-14e6-42c6-a150-0a1487f20ad9" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:11:57 user nova-compute[75234]: DEBUG nova.network.neutron [None req-0d805482-05bf-44f8-bb11-e441afd66aac tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] [instance: a9350a18-14e6-42c6-a150-0a1487f20ad9] Building network info cache for instance {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2009}}
Nov 29 03:11:57 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:56237f4c-4138-4f50-aafb-9c0f9d4bb1cf -p 172.16.0.220:3260 --op delete {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:11:57 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:56237f4c-4138-4f50-aafb-9c0f9d4bb1cf -p 172.16.0.220:3260 --op delete" returned: 0 in 0.027s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:11:57 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[0dbe2451-f8c1-4885-a777-2df24654d82e]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:11:57 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-4a795bce-c0be-4707-baff-68c516a2f74c tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:11:57 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-4a795bce-c0be-4707-baff-68c516a2f74c tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.390s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:87}}
Nov 29 03:11:57 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-4a795bce-c0be-4707-baff-68c516a2f74c tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] <== disconnect_volume: return (392ms) None {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:203}}
Nov 29 03:11:57 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-4a795bce-c0be-4707-baff-68c516a2f74c tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] [instance: 6c0e0184-ca3a-48ec-a6e3-a0a1172260d6] Disconnected iSCSI Volume {{(pid=75234) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:79}}
Nov 29 03:11:57 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a9350a18-14e6-42c6-a150-0a1487f20ad9/disk --force-share --output=json" returned: 0 in 0.182s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:11:57 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a9350a18-14e6-42c6-a150-0a1487f20ad9/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:11:57 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a9350a18-14e6-42c6-a150-0a1487f20ad9/disk --force-share --output=json" returned: 0 in 0.209s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:11:57 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] skipping disk /dev/sdc (vdb) as it is a volume {{(pid=75234) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11213}}
Nov 29 03:11:57 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/8297c264-a213-44ee-be46-5c08eabda9f7/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:11:57 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-eab52431-6501-43e3-9e20-651dd629a8bc tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Acquiring lock "f609a7b7-1be2-44b4-9177-7e72a4d4c060" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:11:57 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-eab52431-6501-43e3-9e20-651dd629a8bc tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Lock "f609a7b7-1be2-44b4-9177-7e72a4d4c060" acquired by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:11:57 user nova-compute[75234]: INFO nova.compute.manager [None req-eab52431-6501-43e3-9e20-651dd629a8bc tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] [instance: f609a7b7-1be2-44b4-9177-7e72a4d4c060] Detaching volume 33601b89-9deb-4aa4-bd5b-1654f1237672
Nov 29 03:11:57 user nova-compute[75234]: DEBUG nova.network.neutron [None req-0d805482-05bf-44f8-bb11-e441afd66aac tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] [instance: a9350a18-14e6-42c6-a150-0a1487f20ad9] Updating instance_info_cache with network_info: [{"id": "52024efc-8cde-4610-b12c-198f19d75ccf", "address": "fa:16:3e:09:b8:70", "network": {"id": "31c4ca3b-3a2f-47df-87fa-4ad095ee5cf9", "bridge": "br-int", "label": "tempest-ServerActionsTestOtherB-1680745008-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.9", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "ad5ea22c63f04f91a3f2fa665e4fc8a1", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap52024efc-8c", "ovs_interfaceid": "52024efc-8cde-4610-b12c-198f19d75ccf", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:11:57 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-0d805482-05bf-44f8-bb11-e441afd66aac tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Releasing lock "refresh_cache-a9350a18-14e6-42c6-a150-0a1487f20ad9" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:11:57 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-0d805482-05bf-44f8-bb11-e441afd66aac tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] [instance: a9350a18-14e6-42c6-a150-0a1487f20ad9] Creating tmpfile /opt/stack/data/nova/instances/a9350a18-14e6-42c6-a150-0a1487f20ad9/tmpklrjnxq7 to verify with other compute node that the instance is on the same shared storage. {{(pid=75234) check_instance_shared_storage_local /opt/stack/nova/nova/virt/libvirt/driver.py:9625}}
Nov 29 03:11:57 user nova-compute[75234]: INFO nova.virt.block_device [None req-eab52431-6501-43e3-9e20-651dd629a8bc tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] [instance: f609a7b7-1be2-44b4-9177-7e72a4d4c060] Attempting to driver detach volume 33601b89-9deb-4aa4-bd5b-1654f1237672 from mountpoint /dev/vdb
Nov 29 03:11:57 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/8297c264-a213-44ee-be46-5c08eabda9f7/disk --force-share --output=json" returned: 0 in 0.179s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:11:57 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/8297c264-a213-44ee-be46-5c08eabda9f7/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:11:57 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-eab52431-6501-43e3-9e20-651dd629a8bc tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Attempting to detach device vdb from instance f609a7b7-1be2-44b4-9177-7e72a4d4c060 from the persistent domain config. {{(pid=75234) _detach_from_persistent /opt/stack/nova/nova/virt/libvirt/driver.py:2487}}
Nov 29 03:11:57 user nova-compute[75234]: DEBUG nova.virt.libvirt.guest [None req-eab52431-6501-43e3-9e20-651dd629a8bc tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] detach device xml: <disk type="block" device="disk">
Nov 29 03:11:57 user nova-compute[75234]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Nov 29 03:11:57 user nova-compute[75234]:   <source dev="/dev/sde"/>
Nov 29 03:11:57 user nova-compute[75234]:   <target dev="vdb" bus="virtio"/>
Nov 29 03:11:57 user nova-compute[75234]:   <serial>33601b89-9deb-4aa4-bd5b-1654f1237672</serial>
Nov 29 03:11:57 user nova-compute[75234]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x05" function="0x0"/>
Nov 29 03:11:57 user nova-compute[75234]: </disk>
Nov 29 03:11:57 user nova-compute[75234]:  {{(pid=75234) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Nov 29 03:11:57 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-eab52431-6501-43e3-9e20-651dd629a8bc tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Successfully detached device vdb from instance f609a7b7-1be2-44b4-9177-7e72a4d4c060 from the persistent domain config.
Nov 29 03:11:57 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-eab52431-6501-43e3-9e20-651dd629a8bc tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] (1/8): Attempting to detach device vdb with device alias virtio-disk1 from instance f609a7b7-1be2-44b4-9177-7e72a4d4c060 from the live domain config. {{(pid=75234) _detach_from_live_with_retry /opt/stack/nova/nova/virt/libvirt/driver.py:2523}}
Nov 29 03:11:57 user nova-compute[75234]: DEBUG nova.virt.libvirt.guest [None req-eab52431-6501-43e3-9e20-651dd629a8bc tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] detach device xml: <disk type="block" device="disk">
Nov 29 03:11:57 user nova-compute[75234]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Nov 29 03:11:57 user nova-compute[75234]:   <source dev="/dev/sde"/>
Nov 29 03:11:57 user nova-compute[75234]:   <target dev="vdb" bus="virtio"/>
Nov 29 03:11:57 user nova-compute[75234]:   <serial>33601b89-9deb-4aa4-bd5b-1654f1237672</serial>
Nov 29 03:11:57 user nova-compute[75234]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x05" function="0x0"/>
Nov 29 03:11:57 user nova-compute[75234]: </disk>
Nov 29 03:11:57 user nova-compute[75234]:  {{(pid=75234) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Nov 29 03:11:57 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:11:57 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:11:57 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:11:57 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:11:57 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:11:58 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/8297c264-a213-44ee-be46-5c08eabda9f7/disk --force-share --output=json" returned: 0 in 0.182s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:11:58 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] skipping disk /dev/sdd (vdb) as it is a volume {{(pid=75234) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11213}}
Nov 29 03:11:58 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/8297c264-a213-44ee-be46-5c08eabda9f7/disk.rescue --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:11:58 user nova-compute[75234]: DEBUG nova.compute.manager [req-083f575c-cab0-4739-8a90-c22ec869ed39 req-152bb1b0-940a-49f8-bd7b-961bf1b06798 service nova] [instance: a9350a18-14e6-42c6-a150-0a1487f20ad9] Received event network-vif-unplugged-52024efc-8cde-4610-b12c-198f19d75ccf {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:11:58 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-083f575c-cab0-4739-8a90-c22ec869ed39 req-152bb1b0-940a-49f8-bd7b-961bf1b06798 service nova] Acquiring lock "a9350a18-14e6-42c6-a150-0a1487f20ad9-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:11:58 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-083f575c-cab0-4739-8a90-c22ec869ed39 req-152bb1b0-940a-49f8-bd7b-961bf1b06798 service nova] Lock "a9350a18-14e6-42c6-a150-0a1487f20ad9-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:11:58 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-083f575c-cab0-4739-8a90-c22ec869ed39 req-152bb1b0-940a-49f8-bd7b-961bf1b06798 service nova] Lock "a9350a18-14e6-42c6-a150-0a1487f20ad9-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:11:58 user nova-compute[75234]: DEBUG nova.compute.manager [req-083f575c-cab0-4739-8a90-c22ec869ed39 req-152bb1b0-940a-49f8-bd7b-961bf1b06798 service nova] [instance: a9350a18-14e6-42c6-a150-0a1487f20ad9] No waiting events found dispatching network-vif-unplugged-52024efc-8cde-4610-b12c-198f19d75ccf {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:11:58 user nova-compute[75234]: WARNING nova.compute.manager [req-083f575c-cab0-4739-8a90-c22ec869ed39 req-152bb1b0-940a-49f8-bd7b-961bf1b06798 service nova] [instance: a9350a18-14e6-42c6-a150-0a1487f20ad9] Received unexpected event network-vif-unplugged-52024efc-8cde-4610-b12c-198f19d75ccf for instance with vm_state resized and task_state resize_reverting.
Nov 29 03:11:58 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/8297c264-a213-44ee-be46-5c08eabda9f7/disk.rescue --force-share --output=json" returned: 0 in 0.172s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:11:58 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/8297c264-a213-44ee-be46-5c08eabda9f7/disk.rescue --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:11:58 user nova-compute[75234]: INFO nova.compute.manager [None req-dff34f7d-3494-4b65-bed9-d18c861141d7 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] [instance: 8297c264-a213-44ee-be46-5c08eabda9f7] Unrescuing
Nov 29 03:11:58 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-dff34f7d-3494-4b65-bed9-d18c861141d7 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] Acquiring lock "refresh_cache-8297c264-a213-44ee-be46-5c08eabda9f7" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:11:58 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-dff34f7d-3494-4b65-bed9-d18c861141d7 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] Acquired lock "refresh_cache-8297c264-a213-44ee-be46-5c08eabda9f7" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:11:58 user nova-compute[75234]: DEBUG nova.network.neutron [None req-dff34f7d-3494-4b65-bed9-d18c861141d7 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] [instance: 8297c264-a213-44ee-be46-5c08eabda9f7] Building network info cache for instance {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2009}}
Nov 29 03:11:58 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/8297c264-a213-44ee-be46-5c08eabda9f7/disk.rescue --force-share --output=json" returned: 0 in 0.195s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:11:58 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:11:58 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/3e355ae8-aab9-4ae9-b6e3-6de108ec1af9/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:11:58 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-eab52431-6501-43e3-9e20-651dd629a8bc tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Start waiting for the detach event from libvirt for device vdb with device alias virtio-disk1 for instance f609a7b7-1be2-44b4-9177-7e72a4d4c060 {{(pid=75234) _detach_from_live_and_wait_for_event /opt/stack/nova/nova/virt/libvirt/driver.py:2599}}
Nov 29 03:11:58 user nova-compute[75234]: DEBUG nova.compute.manager [req-ad72ce23-7fef-4576-9c9a-c0262b66593c req-21ca4847-5fb9-4ffe-844e-04802097d90d service nova] [instance: 6c0e0184-ca3a-48ec-a6e3-a0a1172260d6] Received event network-vif-plugged-50593025-edf1-44ec-b5f0-c92d6a807c23 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:11:58 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-ad72ce23-7fef-4576-9c9a-c0262b66593c req-21ca4847-5fb9-4ffe-844e-04802097d90d service nova] Acquiring lock "6c0e0184-ca3a-48ec-a6e3-a0a1172260d6-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:11:58 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-ad72ce23-7fef-4576-9c9a-c0262b66593c req-21ca4847-5fb9-4ffe-844e-04802097d90d service nova] Lock "6c0e0184-ca3a-48ec-a6e3-a0a1172260d6-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:11:58 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-ad72ce23-7fef-4576-9c9a-c0262b66593c req-21ca4847-5fb9-4ffe-844e-04802097d90d service nova] Lock "6c0e0184-ca3a-48ec-a6e3-a0a1172260d6-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:11:58 user nova-compute[75234]: DEBUG nova.compute.manager [req-ad72ce23-7fef-4576-9c9a-c0262b66593c req-21ca4847-5fb9-4ffe-844e-04802097d90d service nova] [instance: 6c0e0184-ca3a-48ec-a6e3-a0a1172260d6] No waiting events found dispatching network-vif-plugged-50593025-edf1-44ec-b5f0-c92d6a807c23 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:11:58 user nova-compute[75234]: WARNING nova.compute.manager [req-ad72ce23-7fef-4576-9c9a-c0262b66593c req-21ca4847-5fb9-4ffe-844e-04802097d90d service nova] [instance: 6c0e0184-ca3a-48ec-a6e3-a0a1172260d6] Received unexpected event network-vif-plugged-50593025-edf1-44ec-b5f0-c92d6a807c23 for instance with vm_state active and task_state rebuilding.
Nov 29 03:11:58 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/3e355ae8-aab9-4ae9-b6e3-6de108ec1af9/disk --force-share --output=json" returned: 0 in 0.175s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:11:58 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/3e355ae8-aab9-4ae9-b6e3-6de108ec1af9/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:11:58 user nova-compute[75234]: DEBUG nova.network.neutron [None req-dff34f7d-3494-4b65-bed9-d18c861141d7 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] [instance: 8297c264-a213-44ee-be46-5c08eabda9f7] Updating instance_info_cache with network_info: [{"id": "d26334d2-8040-4022-b658-69be1cf6a025", "address": "fa:16:3e:5f:9d:5a", "network": {"id": "e0366490-9fe2-4429-88c8-6625128e645d", "bridge": "br-int", "label": "tempest-ServerStableDeviceRescueTest-804259918-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.31", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "745bdb3c72794155829dc0a69f35b321", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapd26334d2-80", "ovs_interfaceid": "d26334d2-8040-4022-b658-69be1cf6a025", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:11:58 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-dff34f7d-3494-4b65-bed9-d18c861141d7 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] Releasing lock "refresh_cache-8297c264-a213-44ee-be46-5c08eabda9f7" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:11:58 user nova-compute[75234]: DEBUG nova.objects.instance [None req-dff34f7d-3494-4b65-bed9-d18c861141d7 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] Lazy-loading 'flavor' on Instance uuid 8297c264-a213-44ee-be46-5c08eabda9f7 {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:11:58 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/3e355ae8-aab9-4ae9-b6e3-6de108ec1af9/disk --force-share --output=json" returned: 0 in 0.178s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:11:58 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a082c96e-e384-40b6-bcc8-fc5e0e104669/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:11:58 user nova-compute[75234]: INFO nova.virt.libvirt.driver [-] [instance: a9350a18-14e6-42c6-a150-0a1487f20ad9] Instance destroyed successfully.
Nov 29 03:11:58 user nova-compute[75234]: DEBUG nova.objects.instance [None req-0d805482-05bf-44f8-bb11-e441afd66aac tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Lazy-loading 'resources' on Instance uuid a9350a18-14e6-42c6-a150-0a1487f20ad9 {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:11:58 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-0d805482-05bf-44f8-bb11-e441afd66aac tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-11-29T03:06:06Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-ServerActionsTestOtherB-server-372321920',display_name='tempest-ServerActionsTestOtherB-server-372321920',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(7),hidden=False,host='user',hostname='tempest-serveractionstestotherb-server-372321920',id=5,image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',info_cache=InstanceInfoCache,instance_type_id=7,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBFOggqgO1QPxMZPb2P6SX24mx6LOsfQfrzx4GSpNnJiixHqqOn61ObLwI9/TG/r6plC8w1huVCtu+OqAyk9QXUlEr0+VsPNunPSN9WPXBAM4+U+3iCOgHXExdr++34pXkw==',key_name='tempest-keypair-1692341699',keypairs=<?>,launch_index=0,launched_at=2023-11-29T03:10:32Z,launched_on='user',locked=False,locked_by=None,memory_mb=2048,metadata={},migration_context=<?>,new_flavor=Flavor(7),node='user',numa_topology=None,old_flavor=Flavor(6),os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='ad5ea22c63f04f91a3f2fa665e4fc8a1',ramdisk_id='',reservation_id='r-f5lkid4e',resources=None,root_device_name='/dev/vda',root_gb=20,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',old_vm_state='active',owner_project_name='tempest-ServerActionsTestOtherB-70688183',owner_user_name='tempest-ServerActionsTestOtherB-70688183-project-member'},tags=<?>,task_state='resize_reverting',terminated_at=None,trusted_certs=<?>,updated_at=2023-11-29T03:10:33Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='bdddba0ef9c44b6cb43c32a9ab0e29f6',uuid=a9350a18-14e6-42c6-a150-0a1487f20ad9,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='resized') vif={"id": "52024efc-8cde-4610-b12c-198f19d75ccf", "address": "fa:16:3e:09:b8:70", "network": {"id": "31c4ca3b-3a2f-47df-87fa-4ad095ee5cf9", "bridge": "br-int", "label": "tempest-ServerActionsTestOtherB-1680745008-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.9", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "ad5ea22c63f04f91a3f2fa665e4fc8a1", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap52024efc-8c", "ovs_interfaceid": "52024efc-8cde-4610-b12c-198f19d75ccf", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Nov 29 03:11:58 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-0d805482-05bf-44f8-bb11-e441afd66aac tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Converting VIF {"id": "52024efc-8cde-4610-b12c-198f19d75ccf", "address": "fa:16:3e:09:b8:70", "network": {"id": "31c4ca3b-3a2f-47df-87fa-4ad095ee5cf9", "bridge": "br-int", "label": "tempest-ServerActionsTestOtherB-1680745008-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.9", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "ad5ea22c63f04f91a3f2fa665e4fc8a1", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap52024efc-8c", "ovs_interfaceid": "52024efc-8cde-4610-b12c-198f19d75ccf", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:11:58 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-0d805482-05bf-44f8-bb11-e441afd66aac tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:09:b8:70,bridge_name='br-int',has_traffic_filtering=True,id=52024efc-8cde-4610-b12c-198f19d75ccf,network=Network(31c4ca3b-3a2f-47df-87fa-4ad095ee5cf9),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap52024efc-8c') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:11:58 user nova-compute[75234]: DEBUG os_vif [None req-0d805482-05bf-44f8-bb11-e441afd66aac tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Unplugging vif VIFOpenVSwitch(active=True,address=fa:16:3e:09:b8:70,bridge_name='br-int',has_traffic_filtering=True,id=52024efc-8cde-4610-b12c-198f19d75ccf,network=Network(31c4ca3b-3a2f-47df-87fa-4ad095ee5cf9),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap52024efc-8c') {{(pid=75234) unplug /opt/stack/data/venv/lib/python3.10/site-packages/os_vif/__init__.py:109}}
Nov 29 03:11:58 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:11:58 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tap52024efc-8c, bridge=br-int, if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:11:58 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:11:58 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:11:58 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:11:58 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:11:58 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbDestroyCommand(_result=None, table=QoS, record=3804a230-08ac-480e-a716-2663f3c8b1cc) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:11:58 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:11:58 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:11:58 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:11:58 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:11:58 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a082c96e-e384-40b6-bcc8-fc5e0e104669/disk --force-share --output=json" returned: 0 in 0.196s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:11:58 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a082c96e-e384-40b6-bcc8-fc5e0e104669/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:11:58 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:11:59 user nova-compute[75234]: INFO os_vif [None req-0d805482-05bf-44f8-bb11-e441afd66aac tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Successfully unplugged vif VIFOpenVSwitch(active=True,address=fa:16:3e:09:b8:70,bridge_name='br-int',has_traffic_filtering=True,id=52024efc-8cde-4610-b12c-198f19d75ccf,network=Network(31c4ca3b-3a2f-47df-87fa-4ad095ee5cf9),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap52024efc-8c')
Nov 29 03:11:59 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-0d805482-05bf-44f8-bb11-e441afd66aac tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] [instance: a9350a18-14e6-42c6-a150-0a1487f20ad9] calling os-brick to detach iSCSI Volume {{(pid=75234) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Nov 29 03:11:59 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-0d805482-05bf-44f8-bb11-e441afd66aac tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f8a5050c9a0>, {'target_iqn': 'iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'NFCsGdbh', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdc'}, None), 'kwargs': True}}" {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:176}}
Nov 29 03:11:59 user nova-compute[75234]: WARNING os_brick.initiator.connectors.base [None req-0d805482-05bf-44f8-bb11-e441afd66aac tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Nov 29 03:11:59 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-0d805482-05bf-44f8-bb11-e441afd66aac tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:68}}
Nov 29 03:11:59 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-0d805482-05bf-44f8-bb11-e441afd66aac tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.002s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:73}}
Nov 29 03:11:59 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:11:59 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:11:59 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.025s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:11:59 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[4371b50e-85d1-4f6d-b38d-b0005aaef1ae]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:a7ef35bf-017a-4fd9-9182-79c1b2ab4f98\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:11:59 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-0d805482-05bf-44f8-bb11-e441afd66aac tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS:
Nov 29 03:11:59 user nova-compute[75234]: No targets found.
Nov 29 03:11:59 user nova-compute[75234]: iSNS:
Nov 29 03:11:59 user nova-compute[75234]: No targets found.
Nov 29 03:11:59 user nova-compute[75234]: STATIC:
Nov 29 03:11:59 user nova-compute[75234]: Target: iqn.2010-10.org.openstack:a7ef35bf-017a-4fd9-9182-79c1b2ab4f98
Nov 29 03:11:59 user nova-compute[75234]:         Portal: 172.16.0.220:3260,-1
Nov 29 03:11:59 user nova-compute[75234]:                 Iface Name: default
Nov 29 03:11:59 user nova-compute[75234]: Target: iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672
Nov 29 03:11:59 user nova-compute[75234]:         Portal: 172.16.0.220:3260,-1
Nov 29 03:11:59 user nova-compute[75234]:                 Iface Name: default
Nov 29 03:11:59 user nova-compute[75234]: Target: iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b
Nov 29 03:11:59 user nova-compute[75234]:         Portal: 172.16.0.220:3260,-1
Nov 29 03:11:59 user nova-compute[75234]:                 Iface Name: default
Nov 29 03:11:59 user nova-compute[75234]: FIRMWARE:
Nov 29 03:11:59 user nova-compute[75234]: No targets found.
Nov 29 03:11:59 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:11:59 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-0d805482-05bf-44f8-bb11-e441afd66aac tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Regex to get portals from discoverydb: ^SENDTARGETS:
Nov 29 03:11:59 user nova-compute[75234]: .*?^DiscoveryAddress: 172.16.0.220,3260.*?
Nov 29 03:11:59 user nova-compute[75234]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=75234) _get_discoverydb_portals /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:382}}
Nov 29 03:11:59 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-0d805482-05bf-44f8-bb11-e441afd66aac tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b', 0)] {{(pid=75234) _get_connection_devices /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:830}}
Nov 29 03:11:59 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:11:59 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.018s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:11:59 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[a69951e4-e35d-4f98-b96f-45ba8c8b9488]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:a7ef35bf-017a-4fd9-9182-79c1b2ab4f98\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:11:59 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:11:59 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.023s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:11:59 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[ba1e9e8d-d711-4fdf-9b36-24e1367d4f7f]: (4, ('tcp: [15] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a7ef35bf-017a-4fd9-9182-79c1b2ab4f98 (non-flash)\ntcp: [16] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672 (non-flash)\ntcp: [17] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b (non-flash)\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:11:59 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-0d805482-05bf-44f8-bb11-e441afd66aac tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [15] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a7ef35bf-017a-4fd9-9182-79c1b2ab4f98 (non-flash)
Nov 29 03:11:59 user nova-compute[75234]: tcp: [16] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672 (non-flash)
Nov 29 03:11:59 user nova-compute[75234]: tcp: [17] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b (non-flash)
Nov 29 03:11:59 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:11:59 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-0d805482-05bf-44f8-bb11-e441afd66aac tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] iscsi session list stdout=tcp: [15] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a7ef35bf-017a-4fd9-9182-79c1b2ab4f98 (non-flash)
Nov 29 03:11:59 user nova-compute[75234]: tcp: [16] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672 (non-flash)
Nov 29 03:11:59 user nova-compute[75234]: tcp: [17] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b (non-flash)
Nov 29 03:11:59 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsi_session /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Nov 29 03:11:59 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-0d805482-05bf-44f8-bb11-e441afd66aac tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7f8a50748040>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b'): ({'sdc'}, set())}) {{(pid=75234) _get_connection_devices /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:863}}
Nov 29 03:11:59 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-0d805482-05bf-44f8-bb11-e441afd66aac tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Removing single pathed devices sdc {{(pid=75234) remove_connection /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:374}}
Nov 29 03:11:59 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:11:59 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.022s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:11:59 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[e9c305b3-d668-481f-93ea-feaa96cbcb82]: (4, ('path checker states:\nup                  3\n\npaths: 0\nbusy: False\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:11:59 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdc {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:11:59 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a082c96e-e384-40b6-bcc8-fc5e0e104669/disk --force-share --output=json" returned: 0 in 0.186s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:11:59 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdc" returned: 0 in 0.023s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:11:59 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[28d57801-c344-4c74-bc33-039bfb130858]: (4, ('ok\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:11:59 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-0d805482-05bf-44f8-bb11-e441afd66aac tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Flushing IO for device /dev/sdc {{(pid=75234) flush_device_io /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:434}}
Nov 29 03:11:59 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdc {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:11:59 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2bf9fe41-63c2-4882-bcdd-8e314e734ffe/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:11:59 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdc" returned: 0 in 0.018s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:11:59 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[aa055650-b900-42bf-bac3-d1dd4999d69c]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:11:59 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-0d805482-05bf-44f8-bb11-e441afd66aac tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Remove SCSI device /dev/sdc with /sys/block/sdc/device/delete {{(pid=75234) remove_scsi_device /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:148}}
Nov 29 03:11:59 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdc/device/delete {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:11:59 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdc/device/delete" returned: 0 in 0.041s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:11:59 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[d03a2c27-c9c2-4ec3-aeee-ac08886786d6]: (4, ('1', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:11:59 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-0d805482-05bf-44f8-bb11-e441afd66aac tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Checking to see if SCSI volumes sdc have been removed. {{(pid=75234) wait_for_volumes_removal /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:156}}
Nov 29 03:11:59 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-0d805482-05bf-44f8-bb11-e441afd66aac tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] SCSI volumes sdc have been removed. {{(pid=75234) wait_for_volumes_removal /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:166}}
Nov 29 03:11:59 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-0d805482-05bf-44f8-bb11-e441afd66aac tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b')] {{(pid=75234) _disconnect_connection /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Nov 29 03:11:59 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:11:59 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.023s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:11:59 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[a63089da-d2ba-48ca-ad83-2316e55db8fe]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:11:59 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-0d805482-05bf-44f8-bb11-e441afd66aac tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:11:59 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b -p 172.16.0.220:3260 --logout {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:11:59 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2bf9fe41-63c2-4882-bcdd-8e314e734ffe/disk --force-share --output=json" returned: 0 in 0.195s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:11:59 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2bf9fe41-63c2-4882-bcdd-8e314e734ffe/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:11:59 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b -p 172.16.0.220:3260 --logout" returned: 0 in 0.093s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:11:59 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[39042a5d-301e-4162-a64b-4879aea32dac]: (4, ('Logging out of session [sid: 17, target: iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b, portal: 172.16.0.220,3260]\nLogout of [sid: 17, target: iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:11:59 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-0d805482-05bf-44f8-bb11-e441afd66aac tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 17, target: iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b, portal: 172.16.0.220,3260]
Nov 29 03:11:59 user nova-compute[75234]: Logout of [sid: 17, target: iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b, portal: 172.16.0.220,3260] successful.
Nov 29 03:11:59 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:11:59 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b -p 172.16.0.220:3260 --op delete {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:11:59 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b -p 172.16.0.220:3260 --op delete" returned: 0 in 0.028s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:11:59 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[2e1bcc8a-d5b4-40ae-a132-a777710c995d]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:11:59 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-0d805482-05bf-44f8-bb11-e441afd66aac tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:11:59 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-0d805482-05bf-44f8-bb11-e441afd66aac tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.426s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:87}}
Nov 29 03:11:59 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-0d805482-05bf-44f8-bb11-e441afd66aac tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] <== disconnect_volume: return (428ms) None {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:203}}
Nov 29 03:11:59 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-0d805482-05bf-44f8-bb11-e441afd66aac tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] [instance: a9350a18-14e6-42c6-a150-0a1487f20ad9] Disconnected iSCSI Volume {{(pid=75234) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:79}}
Nov 29 03:11:59 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2bf9fe41-63c2-4882-bcdd-8e314e734ffe/disk --force-share --output=json" returned: 0 in 0.200s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:11:59 user nova-compute[75234]: INFO nova.virt.libvirt.driver [-] [instance: 8297c264-a213-44ee-be46-5c08eabda9f7] Instance destroyed successfully.
Nov 29 03:11:59 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f609a7b7-1be2-44b4-9177-7e72a4d4c060/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:11:59 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f609a7b7-1be2-44b4-9177-7e72a4d4c060/disk --force-share --output=json" returned: 0 in 0.172s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:11:59 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f609a7b7-1be2-44b4-9177-7e72a4d4c060/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:11:59 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f609a7b7-1be2-44b4-9177-7e72a4d4c060/disk --force-share --output=json" returned: 0 in 0.172s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:11:59 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/21e9c6cd-c39f-41c0-8768-5dd64589c3f3/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:12:00 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/21e9c6cd-c39f-41c0-8768-5dd64589c3f3/disk --force-share --output=json" returned: 0 in 0.171s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:12:00 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/21e9c6cd-c39f-41c0-8768-5dd64589c3f3/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:12:00 user nova-compute[75234]: DEBUG nova.compute.manager [req-55f81f3d-f228-42af-9cf6-8bf9978e41d1 req-cde5c7d9-19b7-47c6-a4a5-42843aa49c0f service nova] [instance: a9350a18-14e6-42c6-a150-0a1487f20ad9] Received event network-vif-plugged-52024efc-8cde-4610-b12c-198f19d75ccf {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:12:00 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-55f81f3d-f228-42af-9cf6-8bf9978e41d1 req-cde5c7d9-19b7-47c6-a4a5-42843aa49c0f service nova] Acquiring lock "a9350a18-14e6-42c6-a150-0a1487f20ad9-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:12:00 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-55f81f3d-f228-42af-9cf6-8bf9978e41d1 req-cde5c7d9-19b7-47c6-a4a5-42843aa49c0f service nova] Lock "a9350a18-14e6-42c6-a150-0a1487f20ad9-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:12:00 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-55f81f3d-f228-42af-9cf6-8bf9978e41d1 req-cde5c7d9-19b7-47c6-a4a5-42843aa49c0f service nova] Lock "a9350a18-14e6-42c6-a150-0a1487f20ad9-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:12:00 user nova-compute[75234]: DEBUG nova.compute.manager [req-55f81f3d-f228-42af-9cf6-8bf9978e41d1 req-cde5c7d9-19b7-47c6-a4a5-42843aa49c0f service nova] [instance: a9350a18-14e6-42c6-a150-0a1487f20ad9] No waiting events found dispatching network-vif-plugged-52024efc-8cde-4610-b12c-198f19d75ccf {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:12:00 user nova-compute[75234]: WARNING nova.compute.manager [req-55f81f3d-f228-42af-9cf6-8bf9978e41d1 req-cde5c7d9-19b7-47c6-a4a5-42843aa49c0f service nova] [instance: a9350a18-14e6-42c6-a150-0a1487f20ad9] Received unexpected event network-vif-plugged-52024efc-8cde-4610-b12c-198f19d75ccf for instance with vm_state resized and task_state resize_reverting.
Nov 29 03:12:00 user nova-compute[75234]: DEBUG nova.compute.manager [req-55f81f3d-f228-42af-9cf6-8bf9978e41d1 req-cde5c7d9-19b7-47c6-a4a5-42843aa49c0f service nova] [instance: 8297c264-a213-44ee-be46-5c08eabda9f7] Received event network-vif-unplugged-d26334d2-8040-4022-b658-69be1cf6a025 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:12:00 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-55f81f3d-f228-42af-9cf6-8bf9978e41d1 req-cde5c7d9-19b7-47c6-a4a5-42843aa49c0f service nova] Acquiring lock "8297c264-a213-44ee-be46-5c08eabda9f7-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:12:00 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-55f81f3d-f228-42af-9cf6-8bf9978e41d1 req-cde5c7d9-19b7-47c6-a4a5-42843aa49c0f service nova] Lock "8297c264-a213-44ee-be46-5c08eabda9f7-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:12:00 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-55f81f3d-f228-42af-9cf6-8bf9978e41d1 req-cde5c7d9-19b7-47c6-a4a5-42843aa49c0f service nova] Lock "8297c264-a213-44ee-be46-5c08eabda9f7-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:12:00 user nova-compute[75234]: DEBUG nova.compute.manager [req-55f81f3d-f228-42af-9cf6-8bf9978e41d1 req-cde5c7d9-19b7-47c6-a4a5-42843aa49c0f service nova] [instance: 8297c264-a213-44ee-be46-5c08eabda9f7] No waiting events found dispatching network-vif-unplugged-d26334d2-8040-4022-b658-69be1cf6a025 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:12:00 user nova-compute[75234]: WARNING nova.compute.manager [req-55f81f3d-f228-42af-9cf6-8bf9978e41d1 req-cde5c7d9-19b7-47c6-a4a5-42843aa49c0f service nova] [instance: 8297c264-a213-44ee-be46-5c08eabda9f7] Received unexpected event network-vif-unplugged-d26334d2-8040-4022-b658-69be1cf6a025 for instance with vm_state rescued and task_state unrescuing.
Nov 29 03:12:00 user nova-compute[75234]: DEBUG nova.compute.manager [req-55f81f3d-f228-42af-9cf6-8bf9978e41d1 req-cde5c7d9-19b7-47c6-a4a5-42843aa49c0f service nova] [instance: 8297c264-a213-44ee-be46-5c08eabda9f7] Received event network-vif-plugged-d26334d2-8040-4022-b658-69be1cf6a025 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:12:00 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-55f81f3d-f228-42af-9cf6-8bf9978e41d1 req-cde5c7d9-19b7-47c6-a4a5-42843aa49c0f service nova] Acquiring lock "8297c264-a213-44ee-be46-5c08eabda9f7-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:12:00 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-55f81f3d-f228-42af-9cf6-8bf9978e41d1 req-cde5c7d9-19b7-47c6-a4a5-42843aa49c0f service nova] Lock "8297c264-a213-44ee-be46-5c08eabda9f7-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:12:00 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-55f81f3d-f228-42af-9cf6-8bf9978e41d1 req-cde5c7d9-19b7-47c6-a4a5-42843aa49c0f service nova] Lock "8297c264-a213-44ee-be46-5c08eabda9f7-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:12:00 user nova-compute[75234]: DEBUG nova.compute.manager [req-55f81f3d-f228-42af-9cf6-8bf9978e41d1 req-cde5c7d9-19b7-47c6-a4a5-42843aa49c0f service nova] [instance: 8297c264-a213-44ee-be46-5c08eabda9f7] No waiting events found dispatching network-vif-plugged-d26334d2-8040-4022-b658-69be1cf6a025 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:12:00 user nova-compute[75234]: WARNING nova.compute.manager [req-55f81f3d-f228-42af-9cf6-8bf9978e41d1 req-cde5c7d9-19b7-47c6-a4a5-42843aa49c0f service nova] [instance: 8297c264-a213-44ee-be46-5c08eabda9f7] Received unexpected event network-vif-plugged-d26334d2-8040-4022-b658-69be1cf6a025 for instance with vm_state rescued and task_state unrescuing.
Nov 29 03:12:00 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/21e9c6cd-c39f-41c0-8768-5dd64589c3f3/disk --force-share --output=json" returned: 0 in 0.185s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:12:00 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/13e5da64-9adc-4479-81c7-d329e9b5f08f/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:12:00 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-0d805482-05bf-44f8-bb11-e441afd66aac tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.drop_move_claim_at_dest" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:12:00 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-0d805482-05bf-44f8-bb11-e441afd66aac tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.drop_move_claim_at_dest" :: waited 0.004s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:12:00 user nova-compute[75234]: DEBUG nova.objects.instance [None req-0d805482-05bf-44f8-bb11-e441afd66aac tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Lazy-loading 'migration_context' on Instance uuid a9350a18-14e6-42c6-a150-0a1487f20ad9 {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:12:00 user nova-compute[75234]: DEBUG nova.objects.base [None req-0d805482-05bf-44f8-bb11-e441afd66aac tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Object Instance<a9350a18-14e6-42c6-a150-0a1487f20ad9> lazy-loaded attributes: resources,migration_context {{(pid=75234) wrapper /opt/stack/nova/nova/objects/base.py:126}}
Nov 29 03:12:00 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/13e5da64-9adc-4479-81c7-d329e9b5f08f/disk --force-share --output=json" returned: 0 in 0.172s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:12:00 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/13e5da64-9adc-4479-81c7-d329e9b5f08f/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:12:00 user nova-compute[75234]: DEBUG nova.scheduler.client.report [None req-0d805482-05bf-44f8-bb11-e441afd66aac tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Refreshing inventories for resource provider 4d291220-7c37-4839-a0e3-b23c66502b74 {{(pid=75234) _refresh_associations /opt/stack/nova/nova/scheduler/client/report.py:804}}
Nov 29 03:12:00 user nova-compute[75234]: DEBUG nova.scheduler.client.report [None req-0d805482-05bf-44f8-bb11-e441afd66aac tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Updating ProviderTree inventory for provider 4d291220-7c37-4839-a0e3-b23c66502b74 from _refresh_and_get_inventory using data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=75234) _refresh_and_get_inventory /opt/stack/nova/nova/scheduler/client/report.py:768}}
Nov 29 03:12:00 user nova-compute[75234]: DEBUG nova.compute.provider_tree [None req-0d805482-05bf-44f8-bb11-e441afd66aac tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Updating inventory in ProviderTree for provider 4d291220-7c37-4839-a0e3-b23c66502b74 with inventory: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=75234) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:176}}
Nov 29 03:12:00 user nova-compute[75234]: DEBUG nova.compute.manager [req-a1bf16f1-81d7-45d2-ae85-3ab3104a7a99 req-093530ec-115a-445e-85fd-3a057b86b223 service nova] [instance: 8297c264-a213-44ee-be46-5c08eabda9f7] Received event network-changed-d26334d2-8040-4022-b658-69be1cf6a025 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:12:00 user nova-compute[75234]: DEBUG nova.compute.manager [req-a1bf16f1-81d7-45d2-ae85-3ab3104a7a99 req-093530ec-115a-445e-85fd-3a057b86b223 service nova] [instance: 8297c264-a213-44ee-be46-5c08eabda9f7] Refreshing instance network info cache due to event network-changed-d26334d2-8040-4022-b658-69be1cf6a025. {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11117}}
Nov 29 03:12:00 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-a1bf16f1-81d7-45d2-ae85-3ab3104a7a99 req-093530ec-115a-445e-85fd-3a057b86b223 service nova] Acquiring lock "refresh_cache-8297c264-a213-44ee-be46-5c08eabda9f7" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:12:00 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-a1bf16f1-81d7-45d2-ae85-3ab3104a7a99 req-093530ec-115a-445e-85fd-3a057b86b223 service nova] Acquired lock "refresh_cache-8297c264-a213-44ee-be46-5c08eabda9f7" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:12:00 user nova-compute[75234]: DEBUG nova.network.neutron [req-a1bf16f1-81d7-45d2-ae85-3ab3104a7a99 req-093530ec-115a-445e-85fd-3a057b86b223 service nova] [instance: 8297c264-a213-44ee-be46-5c08eabda9f7] Refreshing network info cache for port d26334d2-8040-4022-b658-69be1cf6a025 {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2006}}
Nov 29 03:12:00 user nova-compute[75234]: DEBUG nova.scheduler.client.report [None req-0d805482-05bf-44f8-bb11-e441afd66aac tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Refreshing aggregate associations for resource provider 4d291220-7c37-4839-a0e3-b23c66502b74, aggregates: None {{(pid=75234) _refresh_associations /opt/stack/nova/nova/scheduler/client/report.py:813}}
Nov 29 03:12:00 user nova-compute[75234]: DEBUG nova.scheduler.client.report [None req-0d805482-05bf-44f8-bb11-e441afd66aac tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Refreshing trait associations for resource provider 4d291220-7c37-4839-a0e3-b23c66502b74, traits: COMPUTE_IMAGE_TYPE_AMI,COMPUTE_GRAPHICS_MODEL_CIRRUS,COMPUTE_NET_VIF_MODEL_VMXNET3,COMPUTE_STORAGE_BUS_VIRTIO,COMPUTE_GRAPHICS_MODEL_NONE,COMPUTE_IMAGE_TYPE_ISO,COMPUTE_NET_VIF_MODEL_RTL8139,HW_CPU_X86_SSE,COMPUTE_IMAGE_TYPE_QCOW2,COMPUTE_SOCKET_PCI_NUMA_AFFINITY,COMPUTE_GRAPHICS_MODEL_BOCHS,COMPUTE_VOLUME_ATTACH_WITH_TAG,COMPUTE_ACCELERATORS,HW_CPU_X86_SSE2,COMPUTE_STORAGE_BUS_SCSI,COMPUTE_GRAPHICS_MODEL_VGA,COMPUTE_NET_ATTACH_INTERFACE_WITH_TAG,COMPUTE_SECURITY_UEFI_SECURE_BOOT,HW_CPU_X86_SSE41,COMPUTE_NET_VIF_MODEL_E1000E,COMPUTE_TRUSTED_CERTS,COMPUTE_STORAGE_BUS_FDC,COMPUTE_VIOMMU_MODEL_INTEL,COMPUTE_GRAPHICS_MODEL_VIRTIO,COMPUTE_VIOMMU_MODEL_AUTO,COMPUTE_IMAGE_TYPE_AKI,COMPUTE_NET_VIF_MODEL_PCNET,COMPUTE_GRAPHICS_MODEL_QXL,COMPUTE_NET_VIF_MODEL_VIRTIO,COMPUTE_GRAPHICS_MODEL_VMVGA,COMPUTE_VOLUME_MULTI_ATTACH,COMPUTE_STORAGE_BUS_SATA,HW_CPU_X86_MMX,COMPUTE_NET_VIF_MODEL_E1000,COMPUTE_IMAGE_TYPE_ARI,COMPUTE_STORAGE_BUS_IDE,COMPUTE_NODE,HW_CPU_X86_SSSE3,COMPUTE_RESCUE_BFV,COMPUTE_NET_ATTACH_INTERFACE,COMPUTE_STORAGE_BUS_USB,COMPUTE_NET_VIF_MODEL_SPAPR_VLAN,COMPUTE_IMAGE_TYPE_RAW,COMPUTE_NET_VIF_MODEL_NE2K_PCI,COMPUTE_VOLUME_EXTEND,COMPUTE_DEVICE_TAGGING,HW_CPU_X86_SSE42 {{(pid=75234) _refresh_associations /opt/stack/nova/nova/scheduler/client/report.py:825}}
Nov 29 03:12:00 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/13e5da64-9adc-4479-81c7-d329e9b5f08f/disk --force-share --output=json" returned: 0 in 0.171s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:12:00 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6c0e0184-ca3a-48ec-a6e3-a0a1172260d6/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:12:00 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6c0e0184-ca3a-48ec-a6e3-a0a1172260d6/disk --force-share --output=json" returned: 0 in 0.168s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:12:00 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6c0e0184-ca3a-48ec-a6e3-a0a1172260d6/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:12:00 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:12:00 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:12:00 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:12:00 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:12:01 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6c0e0184-ca3a-48ec-a6e3-a0a1172260d6/disk --force-share --output=json" returned: 0 in 0.190s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:12:01 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/d3b90f39-4b91-4152-b820-bd0e263eec89/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:12:01 user nova-compute[75234]: DEBUG nova.compute.provider_tree [None req-0d805482-05bf-44f8-bb11-e441afd66aac tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Inventory has not changed in ProviderTree for provider: 4d291220-7c37-4839-a0e3-b23c66502b74 {{(pid=75234) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Nov 29 03:12:01 user nova-compute[75234]: DEBUG nova.scheduler.client.report [None req-0d805482-05bf-44f8-bb11-e441afd66aac tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Inventory has not changed for provider 4d291220-7c37-4839-a0e3-b23c66502b74 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=75234) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Nov 29 03:12:01 user nova-compute[75234]: DEBUG nova.network.neutron [req-a1bf16f1-81d7-45d2-ae85-3ab3104a7a99 req-093530ec-115a-445e-85fd-3a057b86b223 service nova] [instance: 8297c264-a213-44ee-be46-5c08eabda9f7] Updated VIF entry in instance network info cache for port d26334d2-8040-4022-b658-69be1cf6a025. {{(pid=75234) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3481}}
Nov 29 03:12:01 user nova-compute[75234]: DEBUG nova.network.neutron [req-a1bf16f1-81d7-45d2-ae85-3ab3104a7a99 req-093530ec-115a-445e-85fd-3a057b86b223 service nova] [instance: 8297c264-a213-44ee-be46-5c08eabda9f7] Updating instance_info_cache with network_info: [{"id": "d26334d2-8040-4022-b658-69be1cf6a025", "address": "fa:16:3e:5f:9d:5a", "network": {"id": "e0366490-9fe2-4429-88c8-6625128e645d", "bridge": "br-int", "label": "tempest-ServerStableDeviceRescueTest-804259918-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.31", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "745bdb3c72794155829dc0a69f35b321", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapd26334d2-80", "ovs_interfaceid": "d26334d2-8040-4022-b658-69be1cf6a025", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:12:01 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-a1bf16f1-81d7-45d2-ae85-3ab3104a7a99 req-093530ec-115a-445e-85fd-3a057b86b223 service nova] Releasing lock "refresh_cache-8297c264-a213-44ee-be46-5c08eabda9f7" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:12:01 user nova-compute[75234]: DEBUG nova.compute.manager [req-a1bf16f1-81d7-45d2-ae85-3ab3104a7a99 req-093530ec-115a-445e-85fd-3a057b86b223 service nova] [instance: 8297c264-a213-44ee-be46-5c08eabda9f7] Received event network-changed-d26334d2-8040-4022-b658-69be1cf6a025 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:12:01 user nova-compute[75234]: DEBUG nova.compute.manager [req-a1bf16f1-81d7-45d2-ae85-3ab3104a7a99 req-093530ec-115a-445e-85fd-3a057b86b223 service nova] [instance: 8297c264-a213-44ee-be46-5c08eabda9f7] Refreshing instance network info cache due to event network-changed-d26334d2-8040-4022-b658-69be1cf6a025. {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11117}}
Nov 29 03:12:01 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-a1bf16f1-81d7-45d2-ae85-3ab3104a7a99 req-093530ec-115a-445e-85fd-3a057b86b223 service nova] Acquiring lock "refresh_cache-8297c264-a213-44ee-be46-5c08eabda9f7" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:12:01 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-a1bf16f1-81d7-45d2-ae85-3ab3104a7a99 req-093530ec-115a-445e-85fd-3a057b86b223 service nova] Acquired lock "refresh_cache-8297c264-a213-44ee-be46-5c08eabda9f7" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:12:01 user nova-compute[75234]: DEBUG nova.network.neutron [req-a1bf16f1-81d7-45d2-ae85-3ab3104a7a99 req-093530ec-115a-445e-85fd-3a057b86b223 service nova] [instance: 8297c264-a213-44ee-be46-5c08eabda9f7] Refreshing network info cache for port d26334d2-8040-4022-b658-69be1cf6a025 {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2006}}
Nov 29 03:12:01 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/d3b90f39-4b91-4152-b820-bd0e263eec89/disk --force-share --output=json" returned: 0 in 0.221s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:12:01 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/d3b90f39-4b91-4152-b820-bd0e263eec89/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:12:01 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-0d805482-05bf-44f8-bb11-e441afd66aac tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.drop_move_claim_at_dest" :: held 0.903s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:12:01 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/d3b90f39-4b91-4152-b820-bd0e263eec89/disk --force-share --output=json" returned: 0 in 0.184s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:12:03 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:12:03 user nova-compute[75234]: INFO nova.compute.manager [None req-0d805482-05bf-44f8-bb11-e441afd66aac tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] [instance: a9350a18-14e6-42c6-a150-0a1487f20ad9] Swapping old allocation on dict_keys(['4d291220-7c37-4839-a0e3-b23c66502b74']) held by migration 29b9709a-732b-4af0-8f8f-3b10fbde30dc for instance
Nov 29 03:12:03 user nova-compute[75234]: DEBUG nova.compute.manager [None req-dff34f7d-3494-4b65-bed9-d18c861141d7 tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] [instance: 8297c264-a213-44ee-be46-5c08eabda9f7] Checking state {{(pid=75234) _get_power_state /opt/stack/nova/nova/compute/manager.py:1782}}
Nov 29 03:12:03 user nova-compute[75234]: DEBUG nova.compute.manager [req-dcb76c8f-e057-4c93-8c95-2be042a54fcc req-14f65f61-1d5f-4930-837f-545e5ba43bb0 service nova] [instance: 8297c264-a213-44ee-be46-5c08eabda9f7] Received event network-vif-plugged-d26334d2-8040-4022-b658-69be1cf6a025 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:12:03 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-dcb76c8f-e057-4c93-8c95-2be042a54fcc req-14f65f61-1d5f-4930-837f-545e5ba43bb0 service nova] Acquiring lock "8297c264-a213-44ee-be46-5c08eabda9f7-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:12:03 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-dcb76c8f-e057-4c93-8c95-2be042a54fcc req-14f65f61-1d5f-4930-837f-545e5ba43bb0 service nova] Lock "8297c264-a213-44ee-be46-5c08eabda9f7-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:12:03 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-dcb76c8f-e057-4c93-8c95-2be042a54fcc req-14f65f61-1d5f-4930-837f-545e5ba43bb0 service nova] Lock "8297c264-a213-44ee-be46-5c08eabda9f7-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:12:03 user nova-compute[75234]: DEBUG nova.compute.manager [req-dcb76c8f-e057-4c93-8c95-2be042a54fcc req-14f65f61-1d5f-4930-837f-545e5ba43bb0 service nova] [instance: 8297c264-a213-44ee-be46-5c08eabda9f7] No waiting events found dispatching network-vif-plugged-d26334d2-8040-4022-b658-69be1cf6a025 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:12:03 user nova-compute[75234]: WARNING nova.compute.manager [req-dcb76c8f-e057-4c93-8c95-2be042a54fcc req-14f65f61-1d5f-4930-837f-545e5ba43bb0 service nova] [instance: 8297c264-a213-44ee-be46-5c08eabda9f7] Received unexpected event network-vif-plugged-d26334d2-8040-4022-b658-69be1cf6a025 for instance with vm_state rescued and task_state unrescuing.
Nov 29 03:12:03 user nova-compute[75234]: DEBUG nova.compute.manager [req-dcb76c8f-e057-4c93-8c95-2be042a54fcc req-14f65f61-1d5f-4930-837f-545e5ba43bb0 service nova] [instance: 8297c264-a213-44ee-be46-5c08eabda9f7] Received event network-vif-plugged-d26334d2-8040-4022-b658-69be1cf6a025 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:12:03 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-dcb76c8f-e057-4c93-8c95-2be042a54fcc req-14f65f61-1d5f-4930-837f-545e5ba43bb0 service nova] Acquiring lock "8297c264-a213-44ee-be46-5c08eabda9f7-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:12:03 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-dcb76c8f-e057-4c93-8c95-2be042a54fcc req-14f65f61-1d5f-4930-837f-545e5ba43bb0 service nova] Lock "8297c264-a213-44ee-be46-5c08eabda9f7-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:12:03 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-dcb76c8f-e057-4c93-8c95-2be042a54fcc req-14f65f61-1d5f-4930-837f-545e5ba43bb0 service nova] Lock "8297c264-a213-44ee-be46-5c08eabda9f7-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:12:03 user nova-compute[75234]: DEBUG nova.compute.manager [req-dcb76c8f-e057-4c93-8c95-2be042a54fcc req-14f65f61-1d5f-4930-837f-545e5ba43bb0 service nova] [instance: 8297c264-a213-44ee-be46-5c08eabda9f7] No waiting events found dispatching network-vif-plugged-d26334d2-8040-4022-b658-69be1cf6a025 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:12:03 user nova-compute[75234]: WARNING nova.compute.manager [req-dcb76c8f-e057-4c93-8c95-2be042a54fcc req-14f65f61-1d5f-4930-837f-545e5ba43bb0 service nova] [instance: 8297c264-a213-44ee-be46-5c08eabda9f7] Received unexpected event network-vif-plugged-d26334d2-8040-4022-b658-69be1cf6a025 for instance with vm_state rescued and task_state unrescuing.
Nov 29 03:12:03 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:12:03 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:12:03 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Hypervisor/Node resource view: name=user free_ram=7038MB free_disk=25.28342056274414GB free_vcpus=2 pci_devices=[{"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}] {{(pid=75234) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1106}}
Nov 29 03:12:03 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:12:03 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:12:03 user nova-compute[75234]: DEBUG nova.scheduler.client.report [None req-0d805482-05bf-44f8-bb11-e441afd66aac tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Overwriting current allocation {'allocations': {'4d291220-7c37-4839-a0e3-b23c66502b74': {'resources': {'VCPU': 1, 'MEMORY_MB': 2048, 'DISK_GB': 20}, 'generation': 25}}, 'project_id': 'ad5ea22c63f04f91a3f2fa665e4fc8a1', 'user_id': 'bdddba0ef9c44b6cb43c32a9ab0e29f6', 'consumer_generation': 1} on consumer a9350a18-14e6-42c6-a150-0a1487f20ad9 {{(pid=75234) move_allocations /opt/stack/nova/nova/scheduler/client/report.py:2018}}
Nov 29 03:12:03 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance 2bf9fe41-63c2-4882-bcdd-8e314e734ffe actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:12:03 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance 21e9c6cd-c39f-41c0-8768-5dd64589c3f3 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:12:03 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance f609a7b7-1be2-44b4-9177-7e72a4d4c060 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:12:03 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance 8297c264-a213-44ee-be46-5c08eabda9f7 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:12:03 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance 6c0e0184-ca3a-48ec-a6e3-a0a1172260d6 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:12:03 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance a9350a18-14e6-42c6-a150-0a1487f20ad9 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 512, 'DISK_GB': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:12:03 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance d3b90f39-4b91-4152-b820-bd0e263eec89 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:12:03 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance 13e5da64-9adc-4479-81c7-d329e9b5f08f actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:12:03 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance a082c96e-e384-40b6-bcc8-fc5e0e104669 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:12:03 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance 3e355ae8-aab9-4ae9-b6e3-6de108ec1af9 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:12:03 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance 1eb9b734-2711-4463-9743-e5fd95c6857b actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:12:03 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Total usable vcpus: 12, total allocated vcpus: 11 {{(pid=75234) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1129}}
Nov 29 03:12:03 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Final resource view: name=user phys_ram=16011MB used_ram=6144MB phys_disk=40GB used_disk=11GB total_vcpus=12 used_vcpus=11 pci_stats=[] {{(pid=75234) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1138}}
Nov 29 03:12:03 user nova-compute[75234]: DEBUG nova.network.neutron [req-a1bf16f1-81d7-45d2-ae85-3ab3104a7a99 req-093530ec-115a-445e-85fd-3a057b86b223 service nova] [instance: 8297c264-a213-44ee-be46-5c08eabda9f7] Updated VIF entry in instance network info cache for port d26334d2-8040-4022-b658-69be1cf6a025. {{(pid=75234) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3481}}
Nov 29 03:12:03 user nova-compute[75234]: DEBUG nova.network.neutron [req-a1bf16f1-81d7-45d2-ae85-3ab3104a7a99 req-093530ec-115a-445e-85fd-3a057b86b223 service nova] [instance: 8297c264-a213-44ee-be46-5c08eabda9f7] Updating instance_info_cache with network_info: [{"id": "d26334d2-8040-4022-b658-69be1cf6a025", "address": "fa:16:3e:5f:9d:5a", "network": {"id": "e0366490-9fe2-4429-88c8-6625128e645d", "bridge": "br-int", "label": "tempest-ServerStableDeviceRescueTest-804259918-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.31", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "745bdb3c72794155829dc0a69f35b321", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapd26334d2-80", "ovs_interfaceid": "d26334d2-8040-4022-b658-69be1cf6a025", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:12:03 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-0d805482-05bf-44f8-bb11-e441afd66aac tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Acquiring lock "refresh_cache-a9350a18-14e6-42c6-a150-0a1487f20ad9" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:12:03 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-0d805482-05bf-44f8-bb11-e441afd66aac tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Acquired lock "refresh_cache-a9350a18-14e6-42c6-a150-0a1487f20ad9" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:12:03 user nova-compute[75234]: DEBUG nova.network.neutron [None req-0d805482-05bf-44f8-bb11-e441afd66aac tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] [instance: a9350a18-14e6-42c6-a150-0a1487f20ad9] Building network info cache for instance {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2009}}
Nov 29 03:12:03 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-a1bf16f1-81d7-45d2-ae85-3ab3104a7a99 req-093530ec-115a-445e-85fd-3a057b86b223 service nova] Releasing lock "refresh_cache-8297c264-a213-44ee-be46-5c08eabda9f7" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:12:03 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:12:04 user nova-compute[75234]: DEBUG nova.compute.provider_tree [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Inventory has not changed in ProviderTree for provider: 4d291220-7c37-4839-a0e3-b23c66502b74 {{(pid=75234) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Nov 29 03:12:04 user nova-compute[75234]: DEBUG nova.scheduler.client.report [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Inventory has not changed for provider 4d291220-7c37-4839-a0e3-b23c66502b74 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=75234) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Nov 29 03:12:04 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Compute_service record updated for user:user {{(pid=75234) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:1067}}
Nov 29 03:12:04 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.636s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:12:04 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._run_pending_deletes {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:12:04 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Cleaning up deleted instances {{(pid=75234) _run_pending_deletes /opt/stack/nova/nova/compute/manager.py:11209}}
Nov 29 03:12:04 user nova-compute[75234]: DEBUG nova.network.neutron [None req-0d805482-05bf-44f8-bb11-e441afd66aac tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] [instance: a9350a18-14e6-42c6-a150-0a1487f20ad9] Updating instance_info_cache with network_info: [{"id": "52024efc-8cde-4610-b12c-198f19d75ccf", "address": "fa:16:3e:09:b8:70", "network": {"id": "31c4ca3b-3a2f-47df-87fa-4ad095ee5cf9", "bridge": "br-int", "label": "tempest-ServerActionsTestOtherB-1680745008-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.9", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "ad5ea22c63f04f91a3f2fa665e4fc8a1", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap52024efc-8c", "ovs_interfaceid": "52024efc-8cde-4610-b12c-198f19d75ccf", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:12:04 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-0d805482-05bf-44f8-bb11-e441afd66aac tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Releasing lock "refresh_cache-a9350a18-14e6-42c6-a150-0a1487f20ad9" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:12:04 user nova-compute[75234]: DEBUG os_brick.utils [None req-0d805482-05bf-44f8-bb11-e441afd66aac tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:176}}
Nov 29 03:12:04 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:12:04 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] There are 0 instances to clean {{(pid=75234) _run_pending_deletes /opt/stack/nova/nova/compute/manager.py:11218}}
Nov 29 03:12:04 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.017s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:12:04 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[9cd86e74-bd5b-4461-a810-54f749d67012]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:dd1368c5ba2c\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:12:04 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt -v / -n -o SOURCE {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:12:04 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt -v / -n -o SOURCE" returned: 0 in 0.017s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:12:04 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[44dfb5b0-816c-4a64-8464-2e6c4878c508]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:12:04 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:12:04 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.017s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:12:04 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[894694cf-3de5-47ca-9583-5fe67121c0a2]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:12:04 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[b481c0fb-0c43-4c07-a819-089ca7c00ce5]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:12:04 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-0d805482-05bf-44f8-bb11-e441afd66aac tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Running cmd (subprocess): nvme version {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:12:04 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-0d805482-05bf-44f8-bb11-e441afd66aac tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] 'nvme version' failed. Not Retrying. {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:473}}
Nov 29 03:12:04 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.nvmeof [None req-0d805482-05bf-44f8-bb11-e441afd66aac tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] nvme not present on system {{(pid=75234) nvme_present /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/nvmeof.py:756}}
Nov 29 03:12:04 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:12:04 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:473}}
Nov 29 03:12:04 user nova-compute[75234]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Nov 29 03:12:04 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[46977175-7758-423e-9dea-ce8402e08a9f]: (4, '') {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:12:04 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.lightos [None req-0d805482-05bf-44f8-bb11-e441afd66aac tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=75234) find_dsc /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/lightos.py:98}}
Nov 29 03:12:04 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.lightos [None req-0d805482-05bf-44f8-bb11-e441afd66aac tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=75234) get_connector_properties /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/lightos.py:76}}
Nov 29 03:12:04 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.lightos [None req-0d805482-05bf-44f8-bb11-e441afd66aac tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] LIGHTOS: no hostnqn found. {{(pid=75234) get_connector_properties /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/lightos.py:84}}
Nov 29 03:12:04 user nova-compute[75234]: DEBUG os_brick.utils [None req-0d805482-05bf-44f8-bb11-e441afd66aac tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] <== get_connector_properties: return (140ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:dd1368c5ba2c', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:203}}
Nov 29 03:12:05 user nova-compute[75234]: INFO nova.virt.libvirt.driver [-] [instance: 6c0e0184-ca3a-48ec-a6e3-a0a1172260d6] Instance destroyed successfully.
Nov 29 03:12:05 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-4a795bce-c0be-4707-baff-68c516a2f74c tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-11-29T03:09:52Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-tempest.common.compute-instance-176407456',display_name='tempest-ServerActionsTestOtherA-server-1878420946',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-tempest-common-compute-instance-176407456',id=16,image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBFVgvqBN7/vaAFxNi0MSZxI04yEwk9FFUfhV/l7Fr5QumrLMJI6JbWuh0vhY2F6+RLBsF4qtCGuui27rDdawR2ALy+mo/mA7ZKd/Mr3qfGMSgFbF3GghJD1rMovi6Hk9rQ==',key_name='tempest-keypair-382073294',keypairs=<?>,launch_index=0,launched_at=2023-11-29T03:10:00Z,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={rebuild='server'},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=1,progress=0,project_id='c1cb3d1f12f242a785a688d59dbe3cd6',ramdisk_id='',reservation_id='r-7poirwds',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',image_container_format='bare',image_disk_format='qcow2',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-ServerActionsTestOtherA-1495067208',owner_user_name='tempest-ServerActionsTestOtherA-1495067208-project-member'},tags=<?>,task_state='rebuilding',terminated_at=None,trusted_certs=None,updated_at=2023-11-29T03:11:56Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='d611826f3be947c0947b99c763c26d44',uuid=6c0e0184-ca3a-48ec-a6e3-a0a1172260d6,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "50593025-edf1-44ec-b5f0-c92d6a807c23", "address": "fa:16:3e:d8:a2:e7", "network": {"id": "1f946e69-dbec-429c-83f8-a186bad4f2cc", "bridge": "br-int", "label": "tempest-ServerActionsTestOtherA-486369423-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.146", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "c1cb3d1f12f242a785a688d59dbe3cd6", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap50593025-ed", "ovs_interfaceid": "50593025-edf1-44ec-b5f0-c92d6a807c23", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Nov 29 03:12:05 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-4a795bce-c0be-4707-baff-68c516a2f74c tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Converting VIF {"id": "50593025-edf1-44ec-b5f0-c92d6a807c23", "address": "fa:16:3e:d8:a2:e7", "network": {"id": "1f946e69-dbec-429c-83f8-a186bad4f2cc", "bridge": "br-int", "label": "tempest-ServerActionsTestOtherA-486369423-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.146", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "c1cb3d1f12f242a785a688d59dbe3cd6", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap50593025-ed", "ovs_interfaceid": "50593025-edf1-44ec-b5f0-c92d6a807c23", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:12:05 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-4a795bce-c0be-4707-baff-68c516a2f74c tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:d8:a2:e7,bridge_name='br-int',has_traffic_filtering=True,id=50593025-edf1-44ec-b5f0-c92d6a807c23,network=Network(1f946e69-dbec-429c-83f8-a186bad4f2cc),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap50593025-ed') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:12:05 user nova-compute[75234]: DEBUG os_vif [None req-4a795bce-c0be-4707-baff-68c516a2f74c tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Unplugging vif VIFOpenVSwitch(active=True,address=fa:16:3e:d8:a2:e7,bridge_name='br-int',has_traffic_filtering=True,id=50593025-edf1-44ec-b5f0-c92d6a807c23,network=Network(1f946e69-dbec-429c-83f8-a186bad4f2cc),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap50593025-ed') {{(pid=75234) unplug /opt/stack/data/venv/lib/python3.10/site-packages/os_vif/__init__.py:109}}
Nov 29 03:12:05 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:12:05 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tap50593025-ed, bridge=br-int, if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:12:05 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:12:05 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:12:05 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:12:05 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbDestroyCommand(_result=None, table=QoS, record=7ad029b0-ed26-4b2c-a8e9-5b9037b94574) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:12:05 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:12:05 user nova-compute[75234]: INFO os_vif [None req-4a795bce-c0be-4707-baff-68c516a2f74c tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Successfully unplugged vif VIFOpenVSwitch(active=True,address=fa:16:3e:d8:a2:e7,bridge_name='br-int',has_traffic_filtering=True,id=50593025-edf1-44ec-b5f0-c92d6a807c23,network=Network(1f946e69-dbec-429c-83f8-a186bad4f2cc),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap50593025-ed')
Nov 29 03:12:05 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-4a795bce-c0be-4707-baff-68c516a2f74c tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] [instance: 6c0e0184-ca3a-48ec-a6e3-a0a1172260d6] calling os-brick to detach iSCSI Volume {{(pid=75234) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Nov 29 03:12:05 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-4a795bce-c0be-4707-baff-68c516a2f74c tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f8a5050c9a0>, {'target_iqn': 'iqn.2010-10.org.openstack:56237f4c-4138-4f50-aafb-9c0f9d4bb1cf', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'hbYaegTr', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdb'}, None), 'kwargs': True}}" {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:176}}
Nov 29 03:12:05 user nova-compute[75234]: WARNING os_brick.initiator.connectors.base [None req-4a795bce-c0be-4707-baff-68c516a2f74c tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release: NotImplementedError
Nov 29 03:12:05 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-4a795bce-c0be-4707-baff-68c516a2f74c tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:68}}
Nov 29 03:12:05 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-4a795bce-c0be-4707-baff-68c516a2f74c tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.002s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:73}}
Nov 29 03:12:05 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:12:05 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.018s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:12:05 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[6cab7177-bb0e-4858-bf83-ad5059dc04f4]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:a7ef35bf-017a-4fd9-9182-79c1b2ab4f98\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:12:05 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-4a795bce-c0be-4707-baff-68c516a2f74c tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS:
Nov 29 03:12:05 user nova-compute[75234]: No targets found.
Nov 29 03:12:05 user nova-compute[75234]: iSNS:
Nov 29 03:12:05 user nova-compute[75234]: No targets found.
Nov 29 03:12:05 user nova-compute[75234]: STATIC:
Nov 29 03:12:05 user nova-compute[75234]: Target: iqn.2010-10.org.openstack:a7ef35bf-017a-4fd9-9182-79c1b2ab4f98
Nov 29 03:12:05 user nova-compute[75234]:         Portal: 172.16.0.220:3260,-1
Nov 29 03:12:05 user nova-compute[75234]:                 Iface Name: default
Nov 29 03:12:05 user nova-compute[75234]: Target: iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672
Nov 29 03:12:05 user nova-compute[75234]:         Portal: 172.16.0.220:3260,-1
Nov 29 03:12:05 user nova-compute[75234]:                 Iface Name: default
Nov 29 03:12:05 user nova-compute[75234]: FIRMWARE:
Nov 29 03:12:05 user nova-compute[75234]: No targets found.
Nov 29 03:12:05 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:12:05 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-4a795bce-c0be-4707-baff-68c516a2f74c tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Regex to get portals from discoverydb: ^SENDTARGETS:
Nov 29 03:12:05 user nova-compute[75234]: .*?^DiscoveryAddress: 172.16.0.220,3260.*?
Nov 29 03:12:05 user nova-compute[75234]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=75234) _get_discoverydb_portals /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:382}}
Nov 29 03:12:05 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-4a795bce-c0be-4707-baff-68c516a2f74c tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:56237f4c-4138-4f50-aafb-9c0f9d4bb1cf', 0)] {{(pid=75234) _get_connection_devices /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:830}}
Nov 29 03:12:05 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:12:05 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._sync_scheduler_instance_info {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:12:05 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:12:05 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:12:05 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._sync_power_states {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:12:05 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.022s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:12:05 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[c9c01747-6b85-4b3f-9203-82c6e8479d8b]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:a7ef35bf-017a-4fd9-9182-79c1b2ab4f98\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:12:05 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:12:05 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.022s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:12:05 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[3de0b844-b254-4698-b5b8-e7b096bbd109]: (4, ('tcp: [15] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a7ef35bf-017a-4fd9-9182-79c1b2ab4f98 (non-flash)\ntcp: [16] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672 (non-flash)\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:12:05 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-4a795bce-c0be-4707-baff-68c516a2f74c tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [15] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a7ef35bf-017a-4fd9-9182-79c1b2ab4f98 (non-flash)
Nov 29 03:12:05 user nova-compute[75234]: tcp: [16] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672 (non-flash)
Nov 29 03:12:05 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:12:05 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-4a795bce-c0be-4707-baff-68c516a2f74c tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] iscsi session list stdout=tcp: [15] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a7ef35bf-017a-4fd9-9182-79c1b2ab4f98 (non-flash)
Nov 29 03:12:05 user nova-compute[75234]: tcp: [16] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672 (non-flash)
Nov 29 03:12:05 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsi_session /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Nov 29 03:12:05 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-4a795bce-c0be-4707-baff-68c516a2f74c tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7f8a50624430>, {}) {{(pid=75234) _get_connection_devices /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:863}}
Nov 29 03:12:05 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-4a795bce-c0be-4707-baff-68c516a2f74c tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Disconnecting from: [] {{(pid=75234) _disconnect_connection /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Nov 29 03:12:05 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-4a795bce-c0be-4707-baff-68c516a2f74c tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.093s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:87}}
Nov 29 03:12:05 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-4a795bce-c0be-4707-baff-68c516a2f74c tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] <== disconnect_volume: return (94ms) None {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:203}}
Nov 29 03:12:05 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-4a795bce-c0be-4707-baff-68c516a2f74c tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] [instance: 6c0e0184-ca3a-48ec-a6e3-a0a1172260d6] Disconnected iSCSI Volume {{(pid=75234) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:79}}
Nov 29 03:12:05 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-4a795bce-c0be-4707-baff-68c516a2f74c tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] [instance: 6c0e0184-ca3a-48ec-a6e3-a0a1172260d6] Deleting instance files /opt/stack/data/nova/instances/6c0e0184-ca3a-48ec-a6e3-a0a1172260d6_del
Nov 29 03:12:05 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-4a795bce-c0be-4707-baff-68c516a2f74c tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] [instance: 6c0e0184-ca3a-48ec-a6e3-a0a1172260d6] Deletion of /opt/stack/data/nova/instances/6c0e0184-ca3a-48ec-a6e3-a0a1172260d6_del complete
Nov 29 03:12:05 user nova-compute[75234]: WARNING nova.compute.manager [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] While synchronizing instance power states, found 11 instances in the database and 10 instances on the hypervisor.
Nov 29 03:12:05 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Triggering sync for uuid 2bf9fe41-63c2-4882-bcdd-8e314e734ffe {{(pid=75234) _sync_power_states /opt/stack/nova/nova/compute/manager.py:10332}}
Nov 29 03:12:05 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Triggering sync for uuid 21e9c6cd-c39f-41c0-8768-5dd64589c3f3 {{(pid=75234) _sync_power_states /opt/stack/nova/nova/compute/manager.py:10332}}
Nov 29 03:12:05 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Triggering sync for uuid f609a7b7-1be2-44b4-9177-7e72a4d4c060 {{(pid=75234) _sync_power_states /opt/stack/nova/nova/compute/manager.py:10332}}
Nov 29 03:12:05 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Triggering sync for uuid a9350a18-14e6-42c6-a150-0a1487f20ad9 {{(pid=75234) _sync_power_states /opt/stack/nova/nova/compute/manager.py:10332}}
Nov 29 03:12:05 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Triggering sync for uuid 8297c264-a213-44ee-be46-5c08eabda9f7 {{(pid=75234) _sync_power_states /opt/stack/nova/nova/compute/manager.py:10332}}
Nov 29 03:12:05 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Triggering sync for uuid 6c0e0184-ca3a-48ec-a6e3-a0a1172260d6 {{(pid=75234) _sync_power_states /opt/stack/nova/nova/compute/manager.py:10332}}
Nov 29 03:12:05 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Triggering sync for uuid d3b90f39-4b91-4152-b820-bd0e263eec89 {{(pid=75234) _sync_power_states /opt/stack/nova/nova/compute/manager.py:10332}}
Nov 29 03:12:05 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Triggering sync for uuid 13e5da64-9adc-4479-81c7-d329e9b5f08f {{(pid=75234) _sync_power_states /opt/stack/nova/nova/compute/manager.py:10332}}
Nov 29 03:12:05 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Triggering sync for uuid a082c96e-e384-40b6-bcc8-fc5e0e104669 {{(pid=75234) _sync_power_states /opt/stack/nova/nova/compute/manager.py:10332}}
Nov 29 03:12:05 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Triggering sync for uuid 3e355ae8-aab9-4ae9-b6e3-6de108ec1af9 {{(pid=75234) _sync_power_states /opt/stack/nova/nova/compute/manager.py:10332}}
Nov 29 03:12:05 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Triggering sync for uuid 1eb9b734-2711-4463-9743-e5fd95c6857b {{(pid=75234) _sync_power_states /opt/stack/nova/nova/compute/manager.py:10332}}
Nov 29 03:12:05 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Acquiring lock "2bf9fe41-63c2-4882-bcdd-8e314e734ffe" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:12:05 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "2bf9fe41-63c2-4882-bcdd-8e314e734ffe" acquired by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:12:05 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Acquiring lock "21e9c6cd-c39f-41c0-8768-5dd64589c3f3" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:12:05 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "21e9c6cd-c39f-41c0-8768-5dd64589c3f3" acquired by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:12:05 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Acquiring lock "f609a7b7-1be2-44b4-9177-7e72a4d4c060" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:12:05 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Acquiring lock "a9350a18-14e6-42c6-a150-0a1487f20ad9" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:12:05 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "a9350a18-14e6-42c6-a150-0a1487f20ad9" acquired by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:12:05 user nova-compute[75234]: INFO nova.compute.manager [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] [instance: a9350a18-14e6-42c6-a150-0a1487f20ad9] During sync_power_state the instance has a pending task (resize_reverting). Skip.
Nov 29 03:12:05 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "a9350a18-14e6-42c6-a150-0a1487f20ad9" "released" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: held 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:12:05 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Acquiring lock "8297c264-a213-44ee-be46-5c08eabda9f7" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:12:05 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "8297c264-a213-44ee-be46-5c08eabda9f7" acquired by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:12:05 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Acquiring lock "6c0e0184-ca3a-48ec-a6e3-a0a1172260d6" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:12:05 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "6c0e0184-ca3a-48ec-a6e3-a0a1172260d6" acquired by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:12:05 user nova-compute[75234]: INFO nova.compute.manager [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] [instance: 6c0e0184-ca3a-48ec-a6e3-a0a1172260d6] During sync_power_state the instance has a pending task (rebuilding). Skip.
Nov 29 03:12:05 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "6c0e0184-ca3a-48ec-a6e3-a0a1172260d6" "released" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: held 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:12:05 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Acquiring lock "d3b90f39-4b91-4152-b820-bd0e263eec89" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:12:05 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "d3b90f39-4b91-4152-b820-bd0e263eec89" acquired by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:12:05 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Acquiring lock "13e5da64-9adc-4479-81c7-d329e9b5f08f" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:12:05 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "13e5da64-9adc-4479-81c7-d329e9b5f08f" acquired by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:12:05 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Acquiring lock "a082c96e-e384-40b6-bcc8-fc5e0e104669" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:12:05 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "a082c96e-e384-40b6-bcc8-fc5e0e104669" acquired by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:12:05 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Acquiring lock "3e355ae8-aab9-4ae9-b6e3-6de108ec1af9" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:12:05 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "3e355ae8-aab9-4ae9-b6e3-6de108ec1af9" acquired by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:12:05 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Acquiring lock "1eb9b734-2711-4463-9743-e5fd95c6857b" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:12:05 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "1eb9b734-2711-4463-9743-e5fd95c6857b" acquired by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:12:05 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "2bf9fe41-63c2-4882-bcdd-8e314e734ffe" "released" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: held 0.096s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:12:05 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "21e9c6cd-c39f-41c0-8768-5dd64589c3f3" "released" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: held 0.095s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:12:05 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "8297c264-a213-44ee-be46-5c08eabda9f7" "released" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: held 0.093s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:12:05 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "a082c96e-e384-40b6-bcc8-fc5e0e104669" "released" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: held 0.092s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:12:05 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "1eb9b734-2711-4463-9743-e5fd95c6857b" "released" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: held 0.096s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:12:05 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "13e5da64-9adc-4479-81c7-d329e9b5f08f" "released" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: held 0.102s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:12:05 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "d3b90f39-4b91-4152-b820-bd0e263eec89" "released" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: held 0.105s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:12:05 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "3e355ae8-aab9-4ae9-b6e3-6de108ec1af9" "released" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: held 0.103s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:12:05 user nova-compute[75234]: INFO nova.virt.block_device [None req-4a795bce-c0be-4707-baff-68c516a2f74c tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] [instance: 6c0e0184-ca3a-48ec-a6e3-a0a1172260d6] Booting with volume 56237f4c-4138-4f50-aafb-9c0f9d4bb1cf at /dev/vdb
Nov 29 03:12:05 user nova-compute[75234]: DEBUG os_brick.utils [None req-4a795bce-c0be-4707-baff-68c516a2f74c tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:176}}
Nov 29 03:12:05 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:12:05 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.015s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:12:05 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[e6071d9d-1520-4a80-a135-7ed3b5c4e587]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:dd1368c5ba2c\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:12:05 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt -v / -n -o SOURCE {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:12:05 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt -v / -n -o SOURCE" returned: 0 in 0.016s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:12:05 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[16b207ab-b929-4963-b735-774dcdc2d3c2]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:12:05 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:12:05 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.017s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:12:05 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[76b2365f-daf9-49da-99b7-774688114650]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:12:05 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[a3dccb55-e43f-4e9d-8a66-8951d1e654e4]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:12:05 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-4a795bce-c0be-4707-baff-68c516a2f74c tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Running cmd (subprocess): nvme version {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:12:05 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-4a795bce-c0be-4707-baff-68c516a2f74c tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] 'nvme version' failed. Not Retrying. {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:473}}
Nov 29 03:12:05 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.nvmeof [None req-4a795bce-c0be-4707-baff-68c516a2f74c tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] nvme not present on system {{(pid=75234) nvme_present /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/nvmeof.py:756}}
Nov 29 03:12:05 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:12:05 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:473}}
Nov 29 03:12:05 user nova-compute[75234]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Nov 29 03:12:05 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[5069a2cd-f0ad-431b-a9f5-c89018500e88]: (4, '') {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:12:05 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.lightos [None req-4a795bce-c0be-4707-baff-68c516a2f74c tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=75234) find_dsc /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/lightos.py:98}}
Nov 29 03:12:05 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.lightos [None req-4a795bce-c0be-4707-baff-68c516a2f74c tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=75234) get_connector_properties /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/lightos.py:76}}
Nov 29 03:12:05 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.lightos [None req-4a795bce-c0be-4707-baff-68c516a2f74c tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] LIGHTOS: no hostnqn found. {{(pid=75234) get_connector_properties /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/lightos.py:84}}
Nov 29 03:12:05 user nova-compute[75234]: DEBUG os_brick.utils [None req-4a795bce-c0be-4707-baff-68c516a2f74c tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] <== get_connector_properties: return (132ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:dd1368c5ba2c', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:203}}
Nov 29 03:12:05 user nova-compute[75234]: DEBUG nova.virt.block_device [None req-4a795bce-c0be-4707-baff-68c516a2f74c tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] [instance: 6c0e0184-ca3a-48ec-a6e3-a0a1172260d6] Updating existing volume attachment record: ff5dab24-7de3-4833-b5f4-53a45c853e9e {{(pid=75234) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Nov 29 03:12:06 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:12:10 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:12:11 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:12:13 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-0d805482-05bf-44f8-bb11-e441afd66aac tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] [instance: a9350a18-14e6-42c6-a150-0a1487f20ad9] Starting finish_revert_migration {{(pid=75234) finish_revert_migration /opt/stack/nova/nova/virt/libvirt/driver.py:11816}}
Nov 29 03:12:13 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-0d805482-05bf-44f8-bb11-e441afd66aac tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] [instance: a9350a18-14e6-42c6-a150-0a1487f20ad9] Start _get_guest_xml network_info=[{"id": "52024efc-8cde-4610-b12c-198f19d75ccf", "address": "fa:16:3e:09:b8:70", "network": {"id": "31c4ca3b-3a2f-47df-87fa-4ad095ee5cf9", "bridge": "br-int", "label": "tempest-ServerActionsTestOtherB-1680745008-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.9", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "ad5ea22c63f04f91a3f2fa665e4fc8a1", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap52024efc-8c", "ovs_interfaceid": "52024efc-8cde-4610-b12c-198f19d75ccf", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, '/dev/vdb': {'bus': 'virtio', 'dev': 'vdb', 'type': 'disk'}}} image_meta=ImageMeta(checksum=<?>,container_format='bare',created_at=<?>,direct_url=<?>,disk_format='qcow2',id=96c30ba7-9b73-4a30-8ba4-7138879573b7,min_disk=1,min_ram=0,name=<?>,owner=<?>,properties=ImageMetaProps,protected=<?>,size=<?>,status=<?>,tags=<?>,updated_at=<?>,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [{'device_name': '/dev/vda', 'encryption_format': None, 'encrypted': False, 'device_type': 'disk', 'size': 0, 'disk_bus': 'virtio', 'boot_index': 0, 'encryption_options': None, 'encryption_secret_uuid': None, 'guest_format': None, 'image_id': '96c30ba7-9b73-4a30-8ba4-7138879573b7'}], 'ephemerals': [], 'block_device_mapping': [{'mount_device': '/dev/vdb', 'attachment_id': 'd99a0926-57aa-46a2-89e7-db535caaedb2', 'disk_bus': 'virtio', 'device_type': 'disk', 'boot_index': None, 'guest_format': None, 'delete_on_termination': False, 'connection_info': {'driver_volume_type': 'iscsi', 'data': {'target_iqn': 'iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'NFCsGdbh', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}, 'status': 'attaching', 'instance': 'a9350a18-14e6-42c6-a150-0a1487f20ad9', 'attached_at': '2023-11-29T03:12:13.000000', 'detached_at': '', 'volume_id': 'f4130d3c-b329-4ce1-8b6d-d9e270a43b0b', 'serial': 'f4130d3c-b329-4ce1-8b6d-d9e270a43b0b'}, 'volume_type': None} {{(pid=75234) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7562}}
Nov 29 03:12:13 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-0d805482-05bf-44f8-bb11-e441afd66aac tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:12:13 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-0d805482-05bf-44f8-bb11-e441afd66aac tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:12:13 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-0d805482-05bf-44f8-bb11-e441afd66aac tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Searching host: 'user' for CPU controller through CGroups V1... {{(pid=75234) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1652}}
Nov 29 03:12:13 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-0d805482-05bf-44f8-bb11-e441afd66aac tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] CPU controller missing on host. {{(pid=75234) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1662}}
Nov 29 03:12:13 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-0d805482-05bf-44f8-bb11-e441afd66aac tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Searching host: 'user' for CPU controller through CGroups V2... {{(pid=75234) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1671}}
Nov 29 03:12:13 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-0d805482-05bf-44f8-bb11-e441afd66aac tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] CPU controller found on host. {{(pid=75234) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1678}}
Nov 29 03:12:13 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-0d805482-05bf-44f8-bb11-e441afd66aac tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=75234) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5400}}
Nov 29 03:12:13 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-0d805482-05bf-44f8-bb11-e441afd66aac tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-11-29T03:02:04Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='1',id=6,is_public=True,memory_mb=512,name='m1.tiny',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum=<?>,container_format='bare',created_at=<?>,direct_url=<?>,disk_format='qcow2',id=96c30ba7-9b73-4a30-8ba4-7138879573b7,min_disk=1,min_ram=0,name=<?>,owner=<?>,properties=ImageMetaProps,protected=<?>,size=<?>,status=<?>,tags=<?>,updated_at=<?>,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:563}}
Nov 29 03:12:13 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-0d805482-05bf-44f8-bb11-e441afd66aac tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Flavor limits 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:348}}
Nov 29 03:12:13 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-0d805482-05bf-44f8-bb11-e441afd66aac tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Image limits 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:352}}
Nov 29 03:12:13 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-0d805482-05bf-44f8-bb11-e441afd66aac tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Flavor pref 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:388}}
Nov 29 03:12:13 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-0d805482-05bf-44f8-bb11-e441afd66aac tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Image pref 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:392}}
Nov 29 03:12:13 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-0d805482-05bf-44f8-bb11-e441afd66aac tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:430}}
Nov 29 03:12:13 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-0d805482-05bf-44f8-bb11-e441afd66aac tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:569}}
Nov 29 03:12:13 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-0d805482-05bf-44f8-bb11-e441afd66aac tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=75234) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:471}}
Nov 29 03:12:13 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-0d805482-05bf-44f8-bb11-e441afd66aac tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Got 1 possible topologies {{(pid=75234) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:501}}
Nov 29 03:12:13 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-0d805482-05bf-44f8-bb11-e441afd66aac tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:575}}
Nov 29 03:12:13 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-0d805482-05bf-44f8-bb11-e441afd66aac tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:577}}
Nov 29 03:12:13 user nova-compute[75234]: DEBUG nova.objects.instance [None req-0d805482-05bf-44f8-bb11-e441afd66aac tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Lazy-loading 'vcpu_model' on Instance uuid a9350a18-14e6-42c6-a150-0a1487f20ad9 {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:12:13 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-0d805482-05bf-44f8-bb11-e441afd66aac tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] [instance: a9350a18-14e6-42c6-a150-0a1487f20ad9] Calling os-brick to attach iSCSI Volume {{(pid=75234) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Nov 29 03:12:13 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-0d805482-05bf-44f8-bb11-e441afd66aac tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f8a5050c9a0>, 'connection_properties': {'target_iqn': 'iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'NFCsGdbh', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:176}}
Nov 29 03:12:13 user nova-compute[75234]: WARNING os_brick.initiator.connectors.base [None req-0d805482-05bf-44f8-bb11-e441afd66aac tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Nov 29 03:12:13 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-0d805482-05bf-44f8-bb11-e441afd66aac tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:68}}
Nov 29 03:12:13 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-0d805482-05bf-44f8-bb11-e441afd66aac tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:73}}
Nov 29 03:12:13 user nova-compute[75234]: WARNING os_brick.initiator.connectors.base [None req-0d805482-05bf-44f8-bb11-e441afd66aac tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Nov 29 03:12:13 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-0d805482-05bf-44f8-bb11-e441afd66aac tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:68}}
Nov 29 03:12:13 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-0d805482-05bf-44f8-bb11-e441afd66aac tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:73}}
Nov 29 03:12:13 user nova-compute[75234]: INFO os_brick.initiator.connectors.iscsi [None req-0d805482-05bf-44f8-bb11-e441afd66aac tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Nov 29 03:12:13 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b -p 172.16.0.220:3260 {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:12:13 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b -p 172.16.0.220:3260" returned: 21 in 0.021s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:12:13 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[7c9bed39-ef38-417c-a0ab-19cda90b42bf]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:12:13 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-0d805482-05bf-44f8-bb11-e441afd66aac tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Nov 29 03:12:13 user nova-compute[75234]:  {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:12:13 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b -p 172.16.0.220:3260 --interface default --op new {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:12:13 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.020s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:12:13 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[98de312a-5edb-444c-8ca4-0d5077535c49]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b] added\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:12:13 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-0d805482-05bf-44f8-bb11-e441afd66aac tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b] added
Nov 29 03:12:13 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:12:13 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:12:13 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.019s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:12:13 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[9d7b604f-e598-4977-88b2-e17d63faca1e]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:12:13 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-0d805482-05bf-44f8-bb11-e441afd66aac tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:12:13 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:12:13 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.033s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:12:13 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[18f7c347-2c0e-475b-a4d1-07bea8d58f31]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:12:13 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-0d805482-05bf-44f8-bb11-e441afd66aac tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:12:13 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b -p 172.16.0.220:3260 --op update -n node.session.auth.username -v NFCsGdbh {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:12:13 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b -p 172.16.0.220:3260 --op update -n node.session.auth.username -v NFCsGdbh" returned: 0 in 0.025s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:12:13 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[32c88027-ad0a-4100-9fcb-5d9a6b6a94fe]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:12:13 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-0d805482-05bf-44f8-bb11-e441afd66aac tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'NFCsGdbh'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:12:13 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:12:13 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.022s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:12:13 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[7408fe76-a31a-48f8-a251-718409b1427e]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:12:13 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-0d805482-05bf-44f8-bb11-e441afd66aac tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:12:13 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:12:14 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.033s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:12:14 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[1a092941-cad1-45fa-9dc6-cb6e51444f7d]: (4, ('tcp: [15] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a7ef35bf-017a-4fd9-9182-79c1b2ab4f98 (non-flash)\ntcp: [16] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672 (non-flash)\ntcp: [19] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:2e520fdf-97a2-4104-b51b-2bb4798c8133 (non-flash)\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:12:14 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-0d805482-05bf-44f8-bb11-e441afd66aac tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [15] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a7ef35bf-017a-4fd9-9182-79c1b2ab4f98 (non-flash)
Nov 29 03:12:14 user nova-compute[75234]: tcp: [16] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672 (non-flash)
Nov 29 03:12:14 user nova-compute[75234]: tcp: [19] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:2e520fdf-97a2-4104-b51b-2bb4798c8133 (non-flash)
Nov 29 03:12:14 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:12:14 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-0d805482-05bf-44f8-bb11-e441afd66aac tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] iscsi session list stdout=tcp: [15] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a7ef35bf-017a-4fd9-9182-79c1b2ab4f98 (non-flash)
Nov 29 03:12:14 user nova-compute[75234]: tcp: [16] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672 (non-flash)
Nov 29 03:12:14 user nova-compute[75234]: tcp: [19] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:2e520fdf-97a2-4104-b51b-2bb4798c8133 (non-flash)
Nov 29 03:12:14 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsi_session /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Nov 29 03:12:14 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b -p 172.16.0.220:3260 --login {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:12:14 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-4a795bce-c0be-4707-baff-68c516a2f74c tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] [instance: 6c0e0184-ca3a-48ec-a6e3-a0a1172260d6] Creating instance directory {{(pid=75234) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4727}}
Nov 29 03:12:14 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-4a795bce-c0be-4707-baff-68c516a2f74c tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] [instance: 6c0e0184-ca3a-48ec-a6e3-a0a1172260d6] Creating image(s)
Nov 29 03:12:14 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-4a795bce-c0be-4707-baff-68c516a2f74c tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Acquiring lock "/opt/stack/data/nova/instances/6c0e0184-ca3a-48ec-a6e3-a0a1172260d6/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:12:14 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-4a795bce-c0be-4707-baff-68c516a2f74c tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Lock "/opt/stack/data/nova/instances/6c0e0184-ca3a-48ec-a6e3-a0a1172260d6/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.002s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:12:14 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-4a795bce-c0be-4707-baff-68c516a2f74c tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Lock "/opt/stack/data/nova/instances/6c0e0184-ca3a-48ec-a6e3-a0a1172260d6/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.002s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:12:14 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-4a795bce-c0be-4707-baff-68c516a2f74c tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:12:14 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b -p 172.16.0.220:3260 --login" returned: 0 in 0.080s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:12:14 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[3ffa3466-d928-40d8-b9e8-109c5da4b0f6]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:12:14 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-0d805482-05bf-44f8-bb11-e441afd66aac tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b, portal: 172.16.0.220,3260]
Nov 29 03:12:14 user nova-compute[75234]: Login to [iface: default, target: iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b, portal: 172.16.0.220,3260] successful.
Nov 29 03:12:14 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:12:14 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:12:14 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.025s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:12:14 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[15d8549e-7799-447f-ad8d-6b9a15e70ba0]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:12:14 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-0d805482-05bf-44f8-bb11-e441afd66aac tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:12:14 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:12:14 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.032s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:12:14 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[0c8f6678-10d5-4098-a3a3-5019010574d5]: (4, ('tcp: [15] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a7ef35bf-017a-4fd9-9182-79c1b2ab4f98 (non-flash)\ntcp: [16] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672 (non-flash)\ntcp: [19] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:2e520fdf-97a2-4104-b51b-2bb4798c8133 (non-flash)\ntcp: [20] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b (non-flash)\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:12:14 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-0d805482-05bf-44f8-bb11-e441afd66aac tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [15] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a7ef35bf-017a-4fd9-9182-79c1b2ab4f98 (non-flash)
Nov 29 03:12:14 user nova-compute[75234]: tcp: [16] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672 (non-flash)
Nov 29 03:12:14 user nova-compute[75234]: tcp: [19] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:2e520fdf-97a2-4104-b51b-2bb4798c8133 (non-flash)
Nov 29 03:12:14 user nova-compute[75234]: tcp: [20] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b (non-flash)
Nov 29 03:12:14 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:12:14 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-0d805482-05bf-44f8-bb11-e441afd66aac tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] iscsi session list stdout=tcp: [15] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a7ef35bf-017a-4fd9-9182-79c1b2ab4f98 (non-flash)
Nov 29 03:12:14 user nova-compute[75234]: tcp: [16] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672 (non-flash)
Nov 29 03:12:14 user nova-compute[75234]: tcp: [19] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:2e520fdf-97a2-4104-b51b-2bb4798c8133 (non-flash)
Nov 29 03:12:14 user nova-compute[75234]: tcp: [20] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b (non-flash)
Nov 29 03:12:14 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsi_session /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Nov 29 03:12:14 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-0d805482-05bf-44f8-bb11-e441afd66aac tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.367s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:87}}
Nov 29 03:12:14 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-0d805482-05bf-44f8-bb11-e441afd66aac tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Connected to 172.16.0.220:3260 {{(pid=75234) _connect_vol /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:644}}
Nov 29 03:12:14 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-0d805482-05bf-44f8-bb11-e441afd66aac tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] HCTL ('34', '-', '-', 0) found on session 20 with lun 0 {{(pid=75234) get_hctl /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:770}}
Nov 29 03:12:14 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-0d805482-05bf-44f8-bb11-e441afd66aac tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Scanning host 34 c: -, t: -, l: 0) {{(pid=75234) scan_iscsi /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:795}}
Nov 29 03:12:14 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host34/scan {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:12:14 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host34/scan" returned: 0 in 0.030s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:12:14 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[7e6dbe2e-2bbf-43d5-9f67-5af0326e278a]: (4, ('- - 0', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:12:14 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-0d805482-05bf-44f8-bb11-e441afd66aac tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Searching for a device in session 20 and hctl ['34', '*', '*', 0] yield: None {{(pid=75234) device_name_by_hctl /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:789}}
Nov 29 03:12:14 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-4a795bce-c0be-4707-baff-68c516a2f74c tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json" returned: 0 in 0.284s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:12:14 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-4a795bce-c0be-4707-baff-68c516a2f74c tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Acquiring lock "80686e29dc9db6600d03e85cb62af298e12d7918" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:12:14 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-4a795bce-c0be-4707-baff-68c516a2f74c tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Lock "80686e29dc9db6600d03e85cb62af298e12d7918" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.002s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:12:14 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-4a795bce-c0be-4707-baff-68c516a2f74c tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:12:14 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-4a795bce-c0be-4707-baff-68c516a2f74c tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json" returned: 0 in 0.190s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:12:14 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-4a795bce-c0be-4707-baff-68c516a2f74c tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918,backing_fmt=raw /opt/stack/data/nova/instances/6c0e0184-ca3a-48ec-a6e3-a0a1172260d6/disk 1073741824 {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:12:14 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-4a795bce-c0be-4707-baff-68c516a2f74c tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918,backing_fmt=raw /opt/stack/data/nova/instances/6c0e0184-ca3a-48ec-a6e3-a0a1172260d6/disk 1073741824" returned: 0 in 0.392s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:12:14 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-4a795bce-c0be-4707-baff-68c516a2f74c tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Lock "80686e29dc9db6600d03e85cb62af298e12d7918" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.594s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:12:14 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-4a795bce-c0be-4707-baff-68c516a2f74c tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:12:15 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:12:15 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-4a795bce-c0be-4707-baff-68c516a2f74c tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json" returned: 0 in 0.223s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:12:15 user nova-compute[75234]: DEBUG nova.virt.disk.api [None req-4a795bce-c0be-4707-baff-68c516a2f74c tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Checking if we can resize image /opt/stack/data/nova/instances/6c0e0184-ca3a-48ec-a6e3-a0a1172260d6/disk. size=1073741824 {{(pid=75234) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:166}}
Nov 29 03:12:15 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-4a795bce-c0be-4707-baff-68c516a2f74c tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6c0e0184-ca3a-48ec-a6e3-a0a1172260d6/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:12:15 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-0d805482-05bf-44f8-bb11-e441afd66aac tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Searching for a device in session 20 and hctl ['34', '*', '*', 0] yield: sdc {{(pid=75234) device_name_by_hctl /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:789}}
Nov 29 03:12:15 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-0d805482-05bf-44f8-bb11-e441afd66aac tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Connected to sdc using {'target_iqn': 'iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'NFCsGdbh', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False} {{(pid=75234) _connect_vol /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:672}}
Nov 29 03:12:15 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-0d805482-05bf-44f8-bb11-e441afd66aac tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.421s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:87}}
Nov 29 03:12:15 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-0d805482-05bf-44f8-bb11-e441afd66aac tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] <== connect_volume: return (1422ms) {'type': 'block', 'scsi_wwn': '26539313062336265', 'path': '/dev/sdc'} {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:203}}
Nov 29 03:12:15 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-0d805482-05bf-44f8-bb11-e441afd66aac tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] [instance: a9350a18-14e6-42c6-a150-0a1487f20ad9] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '26539313062336265', 'path': '/dev/sdc'} {{(pid=75234) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Nov 29 03:12:15 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-0d805482-05bf-44f8-bb11-e441afd66aac tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-11-29T03:06:06Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-ServerActionsTestOtherB-server-372321920',display_name='tempest-ServerActionsTestOtherB-server-372321920',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-serveractionstestotherb-server-372321920',id=5,image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBFOggqgO1QPxMZPb2P6SX24mx6LOsfQfrzx4GSpNnJiixHqqOn61ObLwI9/TG/r6plC8w1huVCtu+OqAyk9QXUlEr0+VsPNunPSN9WPXBAM4+U+3iCOgHXExdr++34pXkw==',key_name='tempest-keypair-1692341699',keypairs=<?>,launch_index=0,launched_at=2023-11-29T03:10:32Z,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=MigrationContext,new_flavor=Flavor(7),node='user',numa_topology=None,old_flavor=Flavor(6),os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=1,progress=0,project_id='ad5ea22c63f04f91a3f2fa665e4fc8a1',ramdisk_id='',reservation_id='r-f5lkid4e',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',old_vm_state='active',owner_project_name='tempest-ServerActionsTestOtherB-70688183',owner_user_name='tempest-ServerActionsTestOtherB-70688183-project-member'},tags=<?>,task_state='resize_reverting',terminated_at=None,trusted_certs=<?>,updated_at=2023-11-29T03:11:57Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='bdddba0ef9c44b6cb43c32a9ab0e29f6',uuid=a9350a18-14e6-42c6-a150-0a1487f20ad9,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='resized') vif={"id": "52024efc-8cde-4610-b12c-198f19d75ccf", "address": "fa:16:3e:09:b8:70", "network": {"id": "31c4ca3b-3a2f-47df-87fa-4ad095ee5cf9", "bridge": "br-int", "label": "tempest-ServerActionsTestOtherB-1680745008-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.9", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "ad5ea22c63f04f91a3f2fa665e4fc8a1", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap52024efc-8c", "ovs_interfaceid": "52024efc-8cde-4610-b12c-198f19d75ccf", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=75234) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Nov 29 03:12:15 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-0d805482-05bf-44f8-bb11-e441afd66aac tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Converting VIF {"id": "52024efc-8cde-4610-b12c-198f19d75ccf", "address": "fa:16:3e:09:b8:70", "network": {"id": "31c4ca3b-3a2f-47df-87fa-4ad095ee5cf9", "bridge": "br-int", "label": "tempest-ServerActionsTestOtherB-1680745008-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.9", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "ad5ea22c63f04f91a3f2fa665e4fc8a1", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap52024efc-8c", "ovs_interfaceid": "52024efc-8cde-4610-b12c-198f19d75ccf", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:12:15 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-0d805482-05bf-44f8-bb11-e441afd66aac tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:09:b8:70,bridge_name='br-int',has_traffic_filtering=True,id=52024efc-8cde-4610-b12c-198f19d75ccf,network=Network(31c4ca3b-3a2f-47df-87fa-4ad095ee5cf9),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap52024efc-8c') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:12:15 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-0d805482-05bf-44f8-bb11-e441afd66aac tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] [instance: a9350a18-14e6-42c6-a150-0a1487f20ad9] End _get_guest_xml xml=<domain type="kvm">
Nov 29 03:12:15 user nova-compute[75234]:   <uuid>a9350a18-14e6-42c6-a150-0a1487f20ad9</uuid>
Nov 29 03:12:15 user nova-compute[75234]:   <name>instance-00000005</name>
Nov 29 03:12:15 user nova-compute[75234]:   <memory>524288</memory>
Nov 29 03:12:15 user nova-compute[75234]:   <vcpu>1</vcpu>
Nov 29 03:12:15 user nova-compute[75234]:   <metadata>
Nov 29 03:12:15 user nova-compute[75234]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Nov 29 03:12:15 user nova-compute[75234]:       <nova:package version="0.0.0"/>
Nov 29 03:12:15 user nova-compute[75234]:       <nova:name>tempest-ServerActionsTestOtherB-server-372321920</nova:name>
Nov 29 03:12:15 user nova-compute[75234]:       <nova:creationTime>2023-11-29 03:12:13</nova:creationTime>
Nov 29 03:12:15 user nova-compute[75234]:       <nova:flavor name="m1.tiny">
Nov 29 03:12:15 user nova-compute[75234]:         <nova:memory>512</nova:memory>
Nov 29 03:12:15 user nova-compute[75234]:         <nova:disk>1</nova:disk>
Nov 29 03:12:15 user nova-compute[75234]:         <nova:swap>0</nova:swap>
Nov 29 03:12:15 user nova-compute[75234]:         <nova:ephemeral>0</nova:ephemeral>
Nov 29 03:12:15 user nova-compute[75234]:         <nova:vcpus>1</nova:vcpus>
Nov 29 03:12:15 user nova-compute[75234]:       </nova:flavor>
Nov 29 03:12:15 user nova-compute[75234]:       <nova:owner>
Nov 29 03:12:15 user nova-compute[75234]:         <nova:user uuid="bdddba0ef9c44b6cb43c32a9ab0e29f6">tempest-ServerActionsTestOtherB-70688183-project-member</nova:user>
Nov 29 03:12:15 user nova-compute[75234]:         <nova:project uuid="ad5ea22c63f04f91a3f2fa665e4fc8a1">tempest-ServerActionsTestOtherB-70688183</nova:project>
Nov 29 03:12:15 user nova-compute[75234]:       </nova:owner>
Nov 29 03:12:15 user nova-compute[75234]:       <nova:root type="image" uuid="96c30ba7-9b73-4a30-8ba4-7138879573b7"/>
Nov 29 03:12:15 user nova-compute[75234]:       <nova:ports>
Nov 29 03:12:15 user nova-compute[75234]:         <nova:port uuid="52024efc-8cde-4610-b12c-198f19d75ccf">
Nov 29 03:12:15 user nova-compute[75234]:           <nova:ip type="fixed" address="10.0.0.5" ipVersion="4"/>
Nov 29 03:12:15 user nova-compute[75234]:         </nova:port>
Nov 29 03:12:15 user nova-compute[75234]:       </nova:ports>
Nov 29 03:12:15 user nova-compute[75234]:     </nova:instance>
Nov 29 03:12:15 user nova-compute[75234]:   </metadata>
Nov 29 03:12:15 user nova-compute[75234]:   <sysinfo type="smbios">
Nov 29 03:12:15 user nova-compute[75234]:     <system>
Nov 29 03:12:15 user nova-compute[75234]:       <entry name="manufacturer">OpenStack Foundation</entry>
Nov 29 03:12:15 user nova-compute[75234]:       <entry name="product">OpenStack Nova</entry>
Nov 29 03:12:15 user nova-compute[75234]:       <entry name="version">0.0.0</entry>
Nov 29 03:12:15 user nova-compute[75234]:       <entry name="serial">a9350a18-14e6-42c6-a150-0a1487f20ad9</entry>
Nov 29 03:12:15 user nova-compute[75234]:       <entry name="uuid">a9350a18-14e6-42c6-a150-0a1487f20ad9</entry>
Nov 29 03:12:15 user nova-compute[75234]:       <entry name="family">Virtual Machine</entry>
Nov 29 03:12:15 user nova-compute[75234]:     </system>
Nov 29 03:12:15 user nova-compute[75234]:   </sysinfo>
Nov 29 03:12:15 user nova-compute[75234]:   <os>
Nov 29 03:12:15 user nova-compute[75234]:     <type arch="x86_64" machine="pc">hvm</type>
Nov 29 03:12:15 user nova-compute[75234]:     <boot dev="hd"/>
Nov 29 03:12:15 user nova-compute[75234]:     <smbios mode="sysinfo"/>
Nov 29 03:12:15 user nova-compute[75234]:   </os>
Nov 29 03:12:15 user nova-compute[75234]:   <features>
Nov 29 03:12:15 user nova-compute[75234]:     <acpi/>
Nov 29 03:12:15 user nova-compute[75234]:     <apic/>
Nov 29 03:12:15 user nova-compute[75234]:     <vmcoreinfo/>
Nov 29 03:12:15 user nova-compute[75234]:   </features>
Nov 29 03:12:15 user nova-compute[75234]:   <clock offset="utc">
Nov 29 03:12:15 user nova-compute[75234]:     <timer name="pit" tickpolicy="delay"/>
Nov 29 03:12:15 user nova-compute[75234]:     <timer name="rtc" tickpolicy="catchup"/>
Nov 29 03:12:15 user nova-compute[75234]:     <timer name="hpet" present="no"/>
Nov 29 03:12:15 user nova-compute[75234]:   </clock>
Nov 29 03:12:15 user nova-compute[75234]:   <cpu mode="custom" match="exact">
Nov 29 03:12:15 user nova-compute[75234]:     <model>Nehalem</model>
Nov 29 03:12:15 user nova-compute[75234]:     <topology sockets="1" cores="1" threads="1"/>
Nov 29 03:12:15 user nova-compute[75234]:   </cpu>
Nov 29 03:12:15 user nova-compute[75234]:   <devices>
Nov 29 03:12:15 user nova-compute[75234]:     <disk type="file" device="disk">
Nov 29 03:12:15 user nova-compute[75234]:       <driver name="qemu" type="qcow2" cache="none"/>
Nov 29 03:12:15 user nova-compute[75234]:       <source file="/opt/stack/data/nova/instances/a9350a18-14e6-42c6-a150-0a1487f20ad9/disk"/>
Nov 29 03:12:15 user nova-compute[75234]:       <target dev="vda" bus="virtio"/>
Nov 29 03:12:15 user nova-compute[75234]:     </disk>
Nov 29 03:12:15 user nova-compute[75234]:     <disk type="block" device="disk">
Nov 29 03:12:15 user nova-compute[75234]:       <driver name="qemu" type="raw" cache="none" io="native"/>
Nov 29 03:12:15 user nova-compute[75234]:       <source dev="/dev/sdc"/>
Nov 29 03:12:15 user nova-compute[75234]:       <target dev="vdb" bus="virtio"/>
Nov 29 03:12:15 user nova-compute[75234]:       <serial>f4130d3c-b329-4ce1-8b6d-d9e270a43b0b</serial>
Nov 29 03:12:15 user nova-compute[75234]:     </disk>
Nov 29 03:12:15 user nova-compute[75234]:     <interface type="ethernet">
Nov 29 03:12:15 user nova-compute[75234]:       <mac address="fa:16:3e:09:b8:70"/>
Nov 29 03:12:15 user nova-compute[75234]:       <model type="virtio"/>
Nov 29 03:12:15 user nova-compute[75234]:       <mtu size="1442"/>
Nov 29 03:12:15 user nova-compute[75234]:       <target dev="tap52024efc-8c"/>
Nov 29 03:12:15 user nova-compute[75234]:     </interface>
Nov 29 03:12:15 user nova-compute[75234]:     <serial type="pty">
Nov 29 03:12:15 user nova-compute[75234]:       <log file="/opt/stack/data/nova/instances/a9350a18-14e6-42c6-a150-0a1487f20ad9/console.log" append="off"/>
Nov 29 03:12:15 user nova-compute[75234]:     </serial>
Nov 29 03:12:15 user nova-compute[75234]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Nov 29 03:12:15 user nova-compute[75234]:     <video>
Nov 29 03:12:15 user nova-compute[75234]:       <model type="virtio"/>
Nov 29 03:12:15 user nova-compute[75234]:     </video>
Nov 29 03:12:15 user nova-compute[75234]:     <rng model="virtio">
Nov 29 03:12:15 user nova-compute[75234]:       <backend model="random">/dev/urandom</backend>
Nov 29 03:12:15 user nova-compute[75234]:     </rng>
Nov 29 03:12:15 user nova-compute[75234]:     <controller type="usb" index="0" model="none"/>
Nov 29 03:12:15 user nova-compute[75234]:     <memballoon model="virtio">
Nov 29 03:12:15 user nova-compute[75234]:       <stats period="10"/>
Nov 29 03:12:15 user nova-compute[75234]:     </memballoon>
Nov 29 03:12:15 user nova-compute[75234]:   </devices>
Nov 29 03:12:15 user nova-compute[75234]: </domain>
Nov 29 03:12:15 user nova-compute[75234]:  {{(pid=75234) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7568}}
Nov 29 03:12:15 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-0d805482-05bf-44f8-bb11-e441afd66aac tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-11-29T03:06:06Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-ServerActionsTestOtherB-server-372321920',display_name='tempest-ServerActionsTestOtherB-server-372321920',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-serveractionstestotherb-server-372321920',id=5,image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBFOggqgO1QPxMZPb2P6SX24mx6LOsfQfrzx4GSpNnJiixHqqOn61ObLwI9/TG/r6plC8w1huVCtu+OqAyk9QXUlEr0+VsPNunPSN9WPXBAM4+U+3iCOgHXExdr++34pXkw==',key_name='tempest-keypair-1692341699',keypairs=<?>,launch_index=0,launched_at=2023-11-29T03:10:32Z,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=MigrationContext,new_flavor=Flavor(7),node='user',numa_topology=None,old_flavor=Flavor(6),os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=1,progress=0,project_id='ad5ea22c63f04f91a3f2fa665e4fc8a1',ramdisk_id='',reservation_id='r-f5lkid4e',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',old_vm_state='active',owner_project_name='tempest-ServerActionsTestOtherB-70688183',owner_user_name='tempest-ServerActionsTestOtherB-70688183-project-member'},tags=<?>,task_state='resize_reverting',terminated_at=None,trusted_certs=<?>,updated_at=2023-11-29T03:11:57Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='bdddba0ef9c44b6cb43c32a9ab0e29f6',uuid=a9350a18-14e6-42c6-a150-0a1487f20ad9,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='resized') vif={"id": "52024efc-8cde-4610-b12c-198f19d75ccf", "address": "fa:16:3e:09:b8:70", "network": {"id": "31c4ca3b-3a2f-47df-87fa-4ad095ee5cf9", "bridge": "br-int", "label": "tempest-ServerActionsTestOtherB-1680745008-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.9", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "ad5ea22c63f04f91a3f2fa665e4fc8a1", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap52024efc-8c", "ovs_interfaceid": "52024efc-8cde-4610-b12c-198f19d75ccf", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Nov 29 03:12:15 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-0d805482-05bf-44f8-bb11-e441afd66aac tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Converting VIF {"id": "52024efc-8cde-4610-b12c-198f19d75ccf", "address": "fa:16:3e:09:b8:70", "network": {"id": "31c4ca3b-3a2f-47df-87fa-4ad095ee5cf9", "bridge": "br-int", "label": "tempest-ServerActionsTestOtherB-1680745008-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.9", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "ad5ea22c63f04f91a3f2fa665e4fc8a1", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap52024efc-8c", "ovs_interfaceid": "52024efc-8cde-4610-b12c-198f19d75ccf", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:12:15 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-0d805482-05bf-44f8-bb11-e441afd66aac tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:09:b8:70,bridge_name='br-int',has_traffic_filtering=True,id=52024efc-8cde-4610-b12c-198f19d75ccf,network=Network(31c4ca3b-3a2f-47df-87fa-4ad095ee5cf9),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap52024efc-8c') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:12:15 user nova-compute[75234]: DEBUG os_vif [None req-0d805482-05bf-44f8-bb11-e441afd66aac tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:09:b8:70,bridge_name='br-int',has_traffic_filtering=True,id=52024efc-8cde-4610-b12c-198f19d75ccf,network=Network(31c4ca3b-3a2f-47df-87fa-4ad095ee5cf9),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap52024efc-8c') {{(pid=75234) plug /opt/stack/data/venv/lib/python3.10/site-packages/os_vif/__init__.py:76}}
Nov 29 03:12:15 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:12:15 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:12:15 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Nov 29 03:12:15 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:12:15 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbCreateCommand(_result=None, table=QoS, columns={'type': 'linux-noop', 'external_ids': {'id': '2ef8c24f-0c97-504b-852e-38bf52f37f86', '_type': 'linux-noop'}}, row=False) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:12:15 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:12:15 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:12:15 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:12:15 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tap52024efc-8c, may_exist=True, interface_attrs={}) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:12:15 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Port, record=tap52024efc-8c, col_values=(('qos', UUID('fb7af2a8-4e89-4c21-be47-28136cc271f1')),), if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:12:15 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DbSetCommand(_result=None, table=Interface, record=tap52024efc-8c, col_values=(('external_ids', {'iface-id': '52024efc-8cde-4610-b12c-198f19d75ccf', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:09:b8:70', 'vm-uuid': 'a9350a18-14e6-42c6-a150-0a1487f20ad9'}),), if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:12:15 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:12:15 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:12:15 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:12:15 user nova-compute[75234]: INFO os_vif [None req-0d805482-05bf-44f8-bb11-e441afd66aac tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:09:b8:70,bridge_name='br-int',has_traffic_filtering=True,id=52024efc-8cde-4610-b12c-198f19d75ccf,network=Network(31c4ca3b-3a2f-47df-87fa-4ad095ee5cf9),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap52024efc-8c')
Nov 29 03:12:15 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-4a795bce-c0be-4707-baff-68c516a2f74c tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6c0e0184-ca3a-48ec-a6e3-a0a1172260d6/disk --force-share --output=json" returned: 0 in 0.241s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:12:15 user nova-compute[75234]: DEBUG nova.virt.disk.api [None req-4a795bce-c0be-4707-baff-68c516a2f74c tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Cannot resize image /opt/stack/data/nova/instances/6c0e0184-ca3a-48ec-a6e3-a0a1172260d6/disk to a smaller size. {{(pid=75234) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:172}}
Nov 29 03:12:15 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-4a795bce-c0be-4707-baff-68c516a2f74c tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] [instance: 6c0e0184-ca3a-48ec-a6e3-a0a1172260d6] Created local disks {{(pid=75234) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4861}}
Nov 29 03:12:15 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-4a795bce-c0be-4707-baff-68c516a2f74c tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] [instance: 6c0e0184-ca3a-48ec-a6e3-a0a1172260d6] Ensure instance console log exists: /opt/stack/data/nova/instances/6c0e0184-ca3a-48ec-a6e3-a0a1172260d6/console.log {{(pid=75234) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4613}}
Nov 29 03:12:15 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-4a795bce-c0be-4707-baff-68c516a2f74c tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:12:15 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-4a795bce-c0be-4707-baff-68c516a2f74c tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:12:15 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-4a795bce-c0be-4707-baff-68c516a2f74c tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:12:15 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-4a795bce-c0be-4707-baff-68c516a2f74c tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] [instance: 6c0e0184-ca3a-48ec-a6e3-a0a1172260d6] Start _get_guest_xml network_info=[{"id": "50593025-edf1-44ec-b5f0-c92d6a807c23", "address": "fa:16:3e:d8:a2:e7", "network": {"id": "1f946e69-dbec-429c-83f8-a186bad4f2cc", "bridge": "br-int", "label": "tempest-ServerActionsTestOtherA-486369423-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.146", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "c1cb3d1f12f242a785a688d59dbe3cd6", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap50593025-ed", "ovs_interfaceid": "50593025-edf1-44ec-b5f0-c92d6a807c23", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, '/dev/vdb': {'bus': 'virtio', 'dev': 'vdb', 'type': 'disk'}}} image_meta=ImageMeta(checksum='c8fc807773e5354afe61636071771906',container_format='bare',created_at=2023-11-29T03:02:15Z,direct_url=<?>,disk_format='qcow2',id=96c30ba7-9b73-4a30-8ba4-7138879573b7,min_disk=0,min_ram=0,name='cirros-0.6.2-x86_64-disk',owner='b18e380921c748b78517b6f1211cd0f1',properties=ImageMetaProps,protected=<?>,size=21430272,status='active',tags=<?>,updated_at=2023-11-29T03:02:17Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [{'device_name': '/dev/vda', 'encryption_format': None, 'encrypted': False, 'device_type': 'disk', 'size': 0, 'disk_bus': 'virtio', 'boot_index': 0, 'encryption_options': None, 'encryption_secret_uuid': None, 'guest_format': None, 'image_id': '96c30ba7-9b73-4a30-8ba4-7138879573b7'}], 'ephemerals': [], 'block_device_mapping': [{'mount_device': '/dev/vdb', 'attachment_id': 'ff5dab24-7de3-4833-b5f4-53a45c853e9e', 'disk_bus': 'virtio', 'device_type': 'disk', 'boot_index': None, 'guest_format': None, 'delete_on_termination': False, 'connection_info': {'driver_volume_type': 'iscsi', 'data': {'target_iqn': 'iqn.2010-10.org.openstack:56237f4c-4138-4f50-aafb-9c0f9d4bb1cf', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'hbYaegTr', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}, 'status': 'reserved', 'instance': '6c0e0184-ca3a-48ec-a6e3-a0a1172260d6', 'attached_at': '', 'detached_at': '', 'volume_id': '56237f4c-4138-4f50-aafb-9c0f9d4bb1cf', 'serial': '56237f4c-4138-4f50-aafb-9c0f9d4bb1cf'}, 'volume_type': None} {{(pid=75234) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7562}}
Nov 29 03:12:15 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-4a795bce-c0be-4707-baff-68c516a2f74c tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.: NotImplementedError
Nov 29 03:12:15 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-4a795bce-c0be-4707-baff-68c516a2f74c tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.: NotImplementedError
Nov 29 03:12:15 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-4a795bce-c0be-4707-baff-68c516a2f74c tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Searching host: 'user' for CPU controller through CGroups V1... {{(pid=75234) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1652}}
Nov 29 03:12:15 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-4a795bce-c0be-4707-baff-68c516a2f74c tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] CPU controller missing on host. {{(pid=75234) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1662}}
Nov 29 03:12:15 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-4a795bce-c0be-4707-baff-68c516a2f74c tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Searching host: 'user' for CPU controller through CGroups V2... {{(pid=75234) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1671}}
Nov 29 03:12:15 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-4a795bce-c0be-4707-baff-68c516a2f74c tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] CPU controller found on host. {{(pid=75234) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1678}}
Nov 29 03:12:15 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-4a795bce-c0be-4707-baff-68c516a2f74c tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=75234) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5400}}
Nov 29 03:12:15 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-4a795bce-c0be-4707-baff-68c516a2f74c tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-11-29T03:02:04Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='1',id=6,is_public=True,memory_mb=512,name='m1.tiny',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='c8fc807773e5354afe61636071771906',container_format='bare',created_at=2023-11-29T03:02:15Z,direct_url=<?>,disk_format='qcow2',id=96c30ba7-9b73-4a30-8ba4-7138879573b7,min_disk=0,min_ram=0,name='cirros-0.6.2-x86_64-disk',owner='b18e380921c748b78517b6f1211cd0f1',properties=ImageMetaProps,protected=<?>,size=21430272,status='active',tags=<?>,updated_at=2023-11-29T03:02:17Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:563}}
Nov 29 03:12:15 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-4a795bce-c0be-4707-baff-68c516a2f74c tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Flavor limits 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:348}}
Nov 29 03:12:15 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-4a795bce-c0be-4707-baff-68c516a2f74c tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Image limits 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:352}}
Nov 29 03:12:15 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-4a795bce-c0be-4707-baff-68c516a2f74c tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Flavor pref 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:388}}
Nov 29 03:12:15 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-4a795bce-c0be-4707-baff-68c516a2f74c tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Image pref 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:392}}
Nov 29 03:12:15 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-4a795bce-c0be-4707-baff-68c516a2f74c tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:430}}
Nov 29 03:12:15 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-4a795bce-c0be-4707-baff-68c516a2f74c tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:569}}
Nov 29 03:12:15 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-4a795bce-c0be-4707-baff-68c516a2f74c tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=75234) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:471}}
Nov 29 03:12:15 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-4a795bce-c0be-4707-baff-68c516a2f74c tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Got 1 possible topologies {{(pid=75234) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:501}}
Nov 29 03:12:15 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-4a795bce-c0be-4707-baff-68c516a2f74c tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:575}}
Nov 29 03:12:15 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-4a795bce-c0be-4707-baff-68c516a2f74c tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:577}}
Nov 29 03:12:15 user nova-compute[75234]: DEBUG nova.objects.instance [None req-4a795bce-c0be-4707-baff-68c516a2f74c tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Lazy-loading 'vcpu_model' on Instance uuid 6c0e0184-ca3a-48ec-a6e3-a0a1172260d6 {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:12:15 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-4a795bce-c0be-4707-baff-68c516a2f74c tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] [instance: 6c0e0184-ca3a-48ec-a6e3-a0a1172260d6] Calling os-brick to attach iSCSI Volume {{(pid=75234) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Nov 29 03:12:15 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-4a795bce-c0be-4707-baff-68c516a2f74c tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f8a5050c9a0>, 'connection_properties': {'target_iqn': 'iqn.2010-10.org.openstack:56237f4c-4138-4f50-aafb-9c0f9d4bb1cf', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'hbYaegTr', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:176}}
Nov 29 03:12:15 user nova-compute[75234]: WARNING os_brick.initiator.connectors.base [None req-4a795bce-c0be-4707-baff-68c516a2f74c tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release: NotImplementedError
Nov 29 03:12:15 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-4a795bce-c0be-4707-baff-68c516a2f74c tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:68}}
Nov 29 03:12:15 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-4a795bce-c0be-4707-baff-68c516a2f74c tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:73}}
Nov 29 03:12:15 user nova-compute[75234]: WARNING os_brick.initiator.connectors.base [None req-4a795bce-c0be-4707-baff-68c516a2f74c tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release: NotImplementedError
Nov 29 03:12:15 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-4a795bce-c0be-4707-baff-68c516a2f74c tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:56237f4c-4138-4f50-aafb-9c0f9d4bb1cf" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:68}}
Nov 29 03:12:15 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-4a795bce-c0be-4707-baff-68c516a2f74c tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:56237f4c-4138-4f50-aafb-9c0f9d4bb1cf" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.009s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:73}}
Nov 29 03:12:15 user nova-compute[75234]: INFO os_brick.initiator.connectors.iscsi [None req-4a795bce-c0be-4707-baff-68c516a2f74c tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Nov 29 03:12:15 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:56237f4c-4138-4f50-aafb-9c0f9d4bb1cf -p 172.16.0.220:3260 {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:12:15 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:56237f4c-4138-4f50-aafb-9c0f9d4bb1cf -p 172.16.0.220:3260" returned: 21 in 0.020s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:12:15 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[169435ed-eb7b-4941-9118-33841ad1a81b]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:12:15 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-4a795bce-c0be-4707-baff-68c516a2f74c tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Nov 29 03:12:15 user nova-compute[75234]:  {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:12:15 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:56237f4c-4138-4f50-aafb-9c0f9d4bb1cf -p 172.16.0.220:3260 --interface default --op new {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:12:15 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:56237f4c-4138-4f50-aafb-9c0f9d4bb1cf -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.026s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:12:15 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[928b177d-8f13-4189-830e-cf0493695b28]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:56237f4c-4138-4f50-aafb-9c0f9d4bb1cf] added\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:12:15 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-4a795bce-c0be-4707-baff-68c516a2f74c tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:56237f4c-4138-4f50-aafb-9c0f9d4bb1cf] added
Nov 29 03:12:15 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:12:15 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:56237f4c-4138-4f50-aafb-9c0f9d4bb1cf -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:12:15 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:56237f4c-4138-4f50-aafb-9c0f9d4bb1cf -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.028s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:12:15 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[f077cac8-01c9-4b09-aa84-cabb53cecf73]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:12:15 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-4a795bce-c0be-4707-baff-68c516a2f74c tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:12:15 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:56237f4c-4138-4f50-aafb-9c0f9d4bb1cf -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:12:15 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:56237f4c-4138-4f50-aafb-9c0f9d4bb1cf -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.020s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:12:15 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[916b93a1-a2bb-4237-a043-6a688fd45395]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:12:15 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-4a795bce-c0be-4707-baff-68c516a2f74c tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:12:15 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:56237f4c-4138-4f50-aafb-9c0f9d4bb1cf -p 172.16.0.220:3260 --op update -n node.session.auth.username -v hbYaegTr {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:12:15 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:56237f4c-4138-4f50-aafb-9c0f9d4bb1cf -p 172.16.0.220:3260 --op update -n node.session.auth.username -v hbYaegTr" returned: 0 in 0.021s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:12:15 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[d78395ac-bc70-41bf-ab4e-cc284e893778]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:12:15 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-4a795bce-c0be-4707-baff-68c516a2f74c tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'hbYaegTr'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:12:15 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:56237f4c-4138-4f50-aafb-9c0f9d4bb1cf -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:12:15 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:56237f4c-4138-4f50-aafb-9c0f9d4bb1cf -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.022s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:12:15 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[5a18eafb-909b-455f-99a1-14dddb1fce50]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:12:15 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-4a795bce-c0be-4707-baff-68c516a2f74c tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:12:15 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:12:15 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.031s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:12:15 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[1d0d5837-1113-4942-9dc5-9e2fe5f212dc]: (4, ('tcp: [15] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a7ef35bf-017a-4fd9-9182-79c1b2ab4f98 (non-flash)\ntcp: [16] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672 (non-flash)\ntcp: [19] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:2e520fdf-97a2-4104-b51b-2bb4798c8133 (non-flash)\ntcp: [20] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b (non-flash)\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:12:15 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-4a795bce-c0be-4707-baff-68c516a2f74c tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [15] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a7ef35bf-017a-4fd9-9182-79c1b2ab4f98 (non-flash)
Nov 29 03:12:15 user nova-compute[75234]: tcp: [16] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672 (non-flash)
Nov 29 03:12:15 user nova-compute[75234]: tcp: [19] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:2e520fdf-97a2-4104-b51b-2bb4798c8133 (non-flash)
Nov 29 03:12:15 user nova-compute[75234]: tcp: [20] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b (non-flash)
Nov 29 03:12:15 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:12:15 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-4a795bce-c0be-4707-baff-68c516a2f74c tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] iscsi session list stdout=tcp: [15] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a7ef35bf-017a-4fd9-9182-79c1b2ab4f98 (non-flash)
Nov 29 03:12:15 user nova-compute[75234]: tcp: [16] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672 (non-flash)
Nov 29 03:12:15 user nova-compute[75234]: tcp: [19] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:2e520fdf-97a2-4104-b51b-2bb4798c8133 (non-flash)
Nov 29 03:12:15 user nova-compute[75234]: tcp: [20] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b (non-flash)
Nov 29 03:12:15 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsi_session /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Nov 29 03:12:15 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:56237f4c-4138-4f50-aafb-9c0f9d4bb1cf -p 172.16.0.220:3260 --login {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:12:15 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:56237f4c-4138-4f50-aafb-9c0f9d4bb1cf -p 172.16.0.220:3260 --login" returned: 0 in 0.100s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:12:15 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[7035694a-3404-4321-b65f-53c4020fd076]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:56237f4c-4138-4f50-aafb-9c0f9d4bb1cf, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:56237f4c-4138-4f50-aafb-9c0f9d4bb1cf, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:12:15 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-4a795bce-c0be-4707-baff-68c516a2f74c tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:56237f4c-4138-4f50-aafb-9c0f9d4bb1cf, portal: 172.16.0.220,3260]
Nov 29 03:12:15 user nova-compute[75234]: Login to [iface: default, target: iqn.2010-10.org.openstack:56237f4c-4138-4f50-aafb-9c0f9d4bb1cf, portal: 172.16.0.220,3260] successful.
Nov 29 03:12:15 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:12:15 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:56237f4c-4138-4f50-aafb-9c0f9d4bb1cf -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:12:15 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:56237f4c-4138-4f50-aafb-9c0f9d4bb1cf -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.028s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:12:15 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[2e0ef451-7e11-412b-a697-dcac6768ddc6]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:12:15 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-4a795bce-c0be-4707-baff-68c516a2f74c tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:12:15 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:12:15 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.031s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:12:15 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[a1191246-63d0-4c63-81e9-9dda372468e4]: (4, ('tcp: [15] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a7ef35bf-017a-4fd9-9182-79c1b2ab4f98 (non-flash)\ntcp: [16] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672 (non-flash)\ntcp: [19] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:2e520fdf-97a2-4104-b51b-2bb4798c8133 (non-flash)\ntcp: [20] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b (non-flash)\ntcp: [21] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:56237f4c-4138-4f50-aafb-9c0f9d4bb1cf (non-flash)\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:12:15 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-4a795bce-c0be-4707-baff-68c516a2f74c tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [15] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a7ef35bf-017a-4fd9-9182-79c1b2ab4f98 (non-flash)
Nov 29 03:12:15 user nova-compute[75234]: tcp: [16] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672 (non-flash)
Nov 29 03:12:15 user nova-compute[75234]: tcp: [19] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:2e520fdf-97a2-4104-b51b-2bb4798c8133 (non-flash)
Nov 29 03:12:15 user nova-compute[75234]: tcp: [20] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b (non-flash)
Nov 29 03:12:15 user nova-compute[75234]: tcp: [21] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:56237f4c-4138-4f50-aafb-9c0f9d4bb1cf (non-flash)
Nov 29 03:12:15 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:12:15 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-4a795bce-c0be-4707-baff-68c516a2f74c tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] iscsi session list stdout=tcp: [15] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a7ef35bf-017a-4fd9-9182-79c1b2ab4f98 (non-flash)
Nov 29 03:12:15 user nova-compute[75234]: tcp: [16] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672 (non-flash)
Nov 29 03:12:15 user nova-compute[75234]: tcp: [19] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:2e520fdf-97a2-4104-b51b-2bb4798c8133 (non-flash)
Nov 29 03:12:15 user nova-compute[75234]: tcp: [20] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b (non-flash)
Nov 29 03:12:15 user nova-compute[75234]: tcp: [21] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:56237f4c-4138-4f50-aafb-9c0f9d4bb1cf (non-flash)
Nov 29 03:12:15 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsi_session /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Nov 29 03:12:15 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-4a795bce-c0be-4707-baff-68c516a2f74c tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:56237f4c-4138-4f50-aafb-9c0f9d4bb1cf" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.402s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:87}}
Nov 29 03:12:15 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-4a795bce-c0be-4707-baff-68c516a2f74c tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Connected to 172.16.0.220:3260 {{(pid=75234) _connect_vol /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:644}}
Nov 29 03:12:15 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-4a795bce-c0be-4707-baff-68c516a2f74c tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] HCTL ('37', '-', '-', 0) found on session 21 with lun 0 {{(pid=75234) get_hctl /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:770}}
Nov 29 03:12:15 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-4a795bce-c0be-4707-baff-68c516a2f74c tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Scanning host 37 c: -, t: -, l: 0) {{(pid=75234) scan_iscsi /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:795}}
Nov 29 03:12:15 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host37/scan {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:12:15 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host37/scan" returned: 0 in 0.028s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:12:15 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[3308440b-fda7-44fb-8e1b-be1338e99251]: (4, ('- - 0', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:12:15 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-4a795bce-c0be-4707-baff-68c516a2f74c tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Searching for a device in session 21 and hctl ['37', '*', '*', 0] yield: sdf {{(pid=75234) device_name_by_hctl /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:789}}
Nov 29 03:12:15 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-4a795bce-c0be-4707-baff-68c516a2f74c tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Connected to sdf using {'target_iqn': 'iqn.2010-10.org.openstack:56237f4c-4138-4f50-aafb-9c0f9d4bb1cf', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'hbYaegTr', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False} {{(pid=75234) _connect_vol /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:672}}
Nov 29 03:12:16 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:12:16 user nova-compute[75234]: DEBUG nova.compute.manager [req-10ef7f32-f14e-4b6b-b57e-7dfb54b8b62c req-b8edadbe-0bca-48d1-996c-698ff341c4be service nova] [instance: d3b90f39-4b91-4152-b820-bd0e263eec89] Received event network-changed-83eb52bb-5be5-49d6-81ec-86a21dde3df2 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:12:16 user nova-compute[75234]: DEBUG nova.compute.manager [req-10ef7f32-f14e-4b6b-b57e-7dfb54b8b62c req-b8edadbe-0bca-48d1-996c-698ff341c4be service nova] [instance: d3b90f39-4b91-4152-b820-bd0e263eec89] Refreshing instance network info cache due to event network-changed-83eb52bb-5be5-49d6-81ec-86a21dde3df2. {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11117}}
Nov 29 03:12:16 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-10ef7f32-f14e-4b6b-b57e-7dfb54b8b62c req-b8edadbe-0bca-48d1-996c-698ff341c4be service nova] Acquiring lock "refresh_cache-d3b90f39-4b91-4152-b820-bd0e263eec89" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:12:16 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-10ef7f32-f14e-4b6b-b57e-7dfb54b8b62c req-b8edadbe-0bca-48d1-996c-698ff341c4be service nova] Acquired lock "refresh_cache-d3b90f39-4b91-4152-b820-bd0e263eec89" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:12:16 user nova-compute[75234]: DEBUG nova.network.neutron [req-10ef7f32-f14e-4b6b-b57e-7dfb54b8b62c req-b8edadbe-0bca-48d1-996c-698ff341c4be service nova] [instance: d3b90f39-4b91-4152-b820-bd0e263eec89] Refreshing network info cache for port 83eb52bb-5be5-49d6-81ec-86a21dde3df2 {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2006}}
Nov 29 03:12:16 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:12:16 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:12:16 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:12:16 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-4a795bce-c0be-4707-baff-68c516a2f74c tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.463s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:87}}
Nov 29 03:12:16 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-4a795bce-c0be-4707-baff-68c516a2f74c tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] <== connect_volume: return (1464ms) {'type': 'block', 'scsi_wwn': '23932316239353532', 'path': '/dev/sdf'} {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:203}}
Nov 29 03:12:16 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-4a795bce-c0be-4707-baff-68c516a2f74c tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] [instance: 6c0e0184-ca3a-48ec-a6e3-a0a1172260d6] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '23932316239353532', 'path': '/dev/sdf'} {{(pid=75234) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Nov 29 03:12:17 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-4a795bce-c0be-4707-baff-68c516a2f74c tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=True,compute_id=1,config_drive='',created_at=2023-11-29T03:09:52Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-tempest.common.compute-instance-176407456',display_name='tempest-ServerActionsTestOtherA-server-1878420946',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-tempest-common-compute-instance-176407456',id=16,image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBFVgvqBN7/vaAFxNi0MSZxI04yEwk9FFUfhV/l7Fr5QumrLMJI6JbWuh0vhY2F6+RLBsF4qtCGuui27rDdawR2ALy+mo/mA7ZKd/Mr3qfGMSgFbF3GghJD1rMovi6Hk9rQ==',key_name='tempest-keypair-382073294',keypairs=<?>,launch_index=0,launched_at=2023-11-29T03:10:00Z,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={rebuild='server'},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=1,progress=0,project_id='c1cb3d1f12f242a785a688d59dbe3cd6',ramdisk_id='',reservation_id='r-7poirwds',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',clean_attempts='1',image_base_image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-ServerActionsTestOtherA-1495067208',owner_user_name='tempest-ServerActionsTestOtherA-1495067208-project-member'},tags=<?>,task_state='rebuild_spawning',terminated_at=None,trusted_certs=None,updated_at=2023-11-29T03:12:05Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='d611826f3be947c0947b99c763c26d44',uuid=6c0e0184-ca3a-48ec-a6e3-a0a1172260d6,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "50593025-edf1-44ec-b5f0-c92d6a807c23", "address": "fa:16:3e:d8:a2:e7", "network": {"id": "1f946e69-dbec-429c-83f8-a186bad4f2cc", "bridge": "br-int", "label": "tempest-ServerActionsTestOtherA-486369423-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.146", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "c1cb3d1f12f242a785a688d59dbe3cd6", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap50593025-ed", "ovs_interfaceid": "50593025-edf1-44ec-b5f0-c92d6a807c23", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=75234) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Nov 29 03:12:17 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-4a795bce-c0be-4707-baff-68c516a2f74c tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Converting VIF {"id": "50593025-edf1-44ec-b5f0-c92d6a807c23", "address": "fa:16:3e:d8:a2:e7", "network": {"id": "1f946e69-dbec-429c-83f8-a186bad4f2cc", "bridge": "br-int", "label": "tempest-ServerActionsTestOtherA-486369423-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.146", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "c1cb3d1f12f242a785a688d59dbe3cd6", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap50593025-ed", "ovs_interfaceid": "50593025-edf1-44ec-b5f0-c92d6a807c23", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:12:17 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-4a795bce-c0be-4707-baff-68c516a2f74c tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:d8:a2:e7,bridge_name='br-int',has_traffic_filtering=True,id=50593025-edf1-44ec-b5f0-c92d6a807c23,network=Network(1f946e69-dbec-429c-83f8-a186bad4f2cc),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap50593025-ed') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:12:17 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-4a795bce-c0be-4707-baff-68c516a2f74c tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] [instance: 6c0e0184-ca3a-48ec-a6e3-a0a1172260d6] End _get_guest_xml xml=<domain type="kvm">
Nov 29 03:12:17 user nova-compute[75234]:   <uuid>6c0e0184-ca3a-48ec-a6e3-a0a1172260d6</uuid>
Nov 29 03:12:17 user nova-compute[75234]:   <name>instance-00000010</name>
Nov 29 03:12:17 user nova-compute[75234]:   <memory>524288</memory>
Nov 29 03:12:17 user nova-compute[75234]:   <vcpu>1</vcpu>
Nov 29 03:12:17 user nova-compute[75234]:   <metadata>
Nov 29 03:12:17 user nova-compute[75234]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Nov 29 03:12:17 user nova-compute[75234]:       <nova:package version="0.0.0"/>
Nov 29 03:12:17 user nova-compute[75234]:       <nova:name>tempest-ServerActionsTestOtherA-server-1878420946</nova:name>
Nov 29 03:12:17 user nova-compute[75234]:       <nova:creationTime>2023-11-29 03:12:15</nova:creationTime>
Nov 29 03:12:17 user nova-compute[75234]:       <nova:flavor name="m1.tiny">
Nov 29 03:12:17 user nova-compute[75234]:         <nova:memory>512</nova:memory>
Nov 29 03:12:17 user nova-compute[75234]:         <nova:disk>1</nova:disk>
Nov 29 03:12:17 user nova-compute[75234]:         <nova:swap>0</nova:swap>
Nov 29 03:12:17 user nova-compute[75234]:         <nova:ephemeral>0</nova:ephemeral>
Nov 29 03:12:17 user nova-compute[75234]:         <nova:vcpus>1</nova:vcpus>
Nov 29 03:12:17 user nova-compute[75234]:       </nova:flavor>
Nov 29 03:12:17 user nova-compute[75234]:       <nova:owner>
Nov 29 03:12:17 user nova-compute[75234]:         <nova:user uuid="d611826f3be947c0947b99c763c26d44">tempest-ServerActionsTestOtherA-1495067208-project-member</nova:user>
Nov 29 03:12:17 user nova-compute[75234]:         <nova:project uuid="c1cb3d1f12f242a785a688d59dbe3cd6">tempest-ServerActionsTestOtherA-1495067208</nova:project>
Nov 29 03:12:17 user nova-compute[75234]:       </nova:owner>
Nov 29 03:12:17 user nova-compute[75234]:       <nova:root type="image" uuid="96c30ba7-9b73-4a30-8ba4-7138879573b7"/>
Nov 29 03:12:17 user nova-compute[75234]:       <nova:ports>
Nov 29 03:12:17 user nova-compute[75234]:         <nova:port uuid="50593025-edf1-44ec-b5f0-c92d6a807c23">
Nov 29 03:12:17 user nova-compute[75234]:           <nova:ip type="fixed" address="10.0.0.14" ipVersion="4"/>
Nov 29 03:12:17 user nova-compute[75234]:         </nova:port>
Nov 29 03:12:17 user nova-compute[75234]:       </nova:ports>
Nov 29 03:12:17 user nova-compute[75234]:     </nova:instance>
Nov 29 03:12:17 user nova-compute[75234]:   </metadata>
Nov 29 03:12:17 user nova-compute[75234]:   <sysinfo type="smbios">
Nov 29 03:12:17 user nova-compute[75234]:     <system>
Nov 29 03:12:17 user nova-compute[75234]:       <entry name="manufacturer">OpenStack Foundation</entry>
Nov 29 03:12:17 user nova-compute[75234]:       <entry name="product">OpenStack Nova</entry>
Nov 29 03:12:17 user nova-compute[75234]:       <entry name="version">0.0.0</entry>
Nov 29 03:12:17 user nova-compute[75234]:       <entry name="serial">6c0e0184-ca3a-48ec-a6e3-a0a1172260d6</entry>
Nov 29 03:12:17 user nova-compute[75234]:       <entry name="uuid">6c0e0184-ca3a-48ec-a6e3-a0a1172260d6</entry>
Nov 29 03:12:17 user nova-compute[75234]:       <entry name="family">Virtual Machine</entry>
Nov 29 03:12:17 user nova-compute[75234]:     </system>
Nov 29 03:12:17 user nova-compute[75234]:   </sysinfo>
Nov 29 03:12:17 user nova-compute[75234]:   <os>
Nov 29 03:12:17 user nova-compute[75234]:     <type arch="x86_64" machine="pc">hvm</type>
Nov 29 03:12:17 user nova-compute[75234]:     <boot dev="hd"/>
Nov 29 03:12:17 user nova-compute[75234]:     <smbios mode="sysinfo"/>
Nov 29 03:12:17 user nova-compute[75234]:   </os>
Nov 29 03:12:17 user nova-compute[75234]:   <features>
Nov 29 03:12:17 user nova-compute[75234]:     <acpi/>
Nov 29 03:12:17 user nova-compute[75234]:     <apic/>
Nov 29 03:12:17 user nova-compute[75234]:     <vmcoreinfo/>
Nov 29 03:12:17 user nova-compute[75234]:   </features>
Nov 29 03:12:17 user nova-compute[75234]:   <clock offset="utc">
Nov 29 03:12:17 user nova-compute[75234]:     <timer name="pit" tickpolicy="delay"/>
Nov 29 03:12:17 user nova-compute[75234]:     <timer name="rtc" tickpolicy="catchup"/>
Nov 29 03:12:17 user nova-compute[75234]:     <timer name="hpet" present="no"/>
Nov 29 03:12:17 user nova-compute[75234]:   </clock>
Nov 29 03:12:17 user nova-compute[75234]:   <cpu mode="custom" match="exact">
Nov 29 03:12:17 user nova-compute[75234]:     <model>Nehalem</model>
Nov 29 03:12:17 user nova-compute[75234]:     <topology sockets="1" cores="1" threads="1"/>
Nov 29 03:12:17 user nova-compute[75234]:   </cpu>
Nov 29 03:12:17 user nova-compute[75234]:   <devices>
Nov 29 03:12:17 user nova-compute[75234]:     <disk type="file" device="disk">
Nov 29 03:12:17 user nova-compute[75234]:       <driver name="qemu" type="qcow2" cache="none"/>
Nov 29 03:12:17 user nova-compute[75234]:       <source file="/opt/stack/data/nova/instances/6c0e0184-ca3a-48ec-a6e3-a0a1172260d6/disk"/>
Nov 29 03:12:17 user nova-compute[75234]:       <target dev="vda" bus="virtio"/>
Nov 29 03:12:17 user nova-compute[75234]:     </disk>
Nov 29 03:12:17 user nova-compute[75234]:     <disk type="block" device="disk">
Nov 29 03:12:17 user nova-compute[75234]:       <driver name="qemu" type="raw" cache="none" io="native"/>
Nov 29 03:12:17 user nova-compute[75234]:       <source dev="/dev/sdf"/>
Nov 29 03:12:17 user nova-compute[75234]:       <target dev="vdb" bus="virtio"/>
Nov 29 03:12:17 user nova-compute[75234]:       <serial>56237f4c-4138-4f50-aafb-9c0f9d4bb1cf</serial>
Nov 29 03:12:17 user nova-compute[75234]:     </disk>
Nov 29 03:12:17 user nova-compute[75234]:     <interface type="ethernet">
Nov 29 03:12:17 user nova-compute[75234]:       <mac address="fa:16:3e:d8:a2:e7"/>
Nov 29 03:12:17 user nova-compute[75234]:       <model type="virtio"/>
Nov 29 03:12:17 user nova-compute[75234]:       <mtu size="1442"/>
Nov 29 03:12:17 user nova-compute[75234]:       <target dev="tap50593025-ed"/>
Nov 29 03:12:17 user nova-compute[75234]:     </interface>
Nov 29 03:12:17 user nova-compute[75234]:     <serial type="pty">
Nov 29 03:12:17 user nova-compute[75234]:       <log file="/opt/stack/data/nova/instances/6c0e0184-ca3a-48ec-a6e3-a0a1172260d6/console.log" append="off"/>
Nov 29 03:12:17 user nova-compute[75234]:     </serial>
Nov 29 03:12:17 user nova-compute[75234]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Nov 29 03:12:17 user nova-compute[75234]:     <video>
Nov 29 03:12:17 user nova-compute[75234]:       <model type="virtio"/>
Nov 29 03:12:17 user nova-compute[75234]:     </video>
Nov 29 03:12:17 user nova-compute[75234]:     <rng model="virtio">
Nov 29 03:12:17 user nova-compute[75234]:       <backend model="random">/dev/urandom</backend>
Nov 29 03:12:17 user nova-compute[75234]:     </rng>
Nov 29 03:12:17 user nova-compute[75234]:     <controller type="usb" index="0" model="none"/>
Nov 29 03:12:17 user nova-compute[75234]:     <memballoon model="virtio">
Nov 29 03:12:17 user nova-compute[75234]:       <stats period="10"/>
Nov 29 03:12:17 user nova-compute[75234]:     </memballoon>
Nov 29 03:12:17 user nova-compute[75234]:   </devices>
Nov 29 03:12:17 user nova-compute[75234]: </domain>
Nov 29 03:12:17 user nova-compute[75234]:  {{(pid=75234) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7568}}
Nov 29 03:12:17 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-4a795bce-c0be-4707-baff-68c516a2f74c tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=True,compute_id=1,config_drive='',created_at=2023-11-29T03:09:52Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-tempest.common.compute-instance-176407456',display_name='tempest-ServerActionsTestOtherA-server-1878420946',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-tempest-common-compute-instance-176407456',id=16,image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBFVgvqBN7/vaAFxNi0MSZxI04yEwk9FFUfhV/l7Fr5QumrLMJI6JbWuh0vhY2F6+RLBsF4qtCGuui27rDdawR2ALy+mo/mA7ZKd/Mr3qfGMSgFbF3GghJD1rMovi6Hk9rQ==',key_name='tempest-keypair-382073294',keypairs=<?>,launch_index=0,launched_at=2023-11-29T03:10:00Z,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={rebuild='server'},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=1,progress=0,project_id='c1cb3d1f12f242a785a688d59dbe3cd6',ramdisk_id='',reservation_id='r-7poirwds',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',clean_attempts='1',image_base_image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-ServerActionsTestOtherA-1495067208',owner_user_name='tempest-ServerActionsTestOtherA-1495067208-project-member'},tags=<?>,task_state='rebuild_spawning',terminated_at=None,trusted_certs=None,updated_at=2023-11-29T03:12:05Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='d611826f3be947c0947b99c763c26d44',uuid=6c0e0184-ca3a-48ec-a6e3-a0a1172260d6,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "50593025-edf1-44ec-b5f0-c92d6a807c23", "address": "fa:16:3e:d8:a2:e7", "network": {"id": "1f946e69-dbec-429c-83f8-a186bad4f2cc", "bridge": "br-int", "label": "tempest-ServerActionsTestOtherA-486369423-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.146", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "c1cb3d1f12f242a785a688d59dbe3cd6", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap50593025-ed", "ovs_interfaceid": "50593025-edf1-44ec-b5f0-c92d6a807c23", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Nov 29 03:12:17 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-4a795bce-c0be-4707-baff-68c516a2f74c tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Converting VIF {"id": "50593025-edf1-44ec-b5f0-c92d6a807c23", "address": "fa:16:3e:d8:a2:e7", "network": {"id": "1f946e69-dbec-429c-83f8-a186bad4f2cc", "bridge": "br-int", "label": "tempest-ServerActionsTestOtherA-486369423-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.146", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "c1cb3d1f12f242a785a688d59dbe3cd6", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap50593025-ed", "ovs_interfaceid": "50593025-edf1-44ec-b5f0-c92d6a807c23", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:12:17 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-4a795bce-c0be-4707-baff-68c516a2f74c tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:d8:a2:e7,bridge_name='br-int',has_traffic_filtering=True,id=50593025-edf1-44ec-b5f0-c92d6a807c23,network=Network(1f946e69-dbec-429c-83f8-a186bad4f2cc),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap50593025-ed') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:12:17 user nova-compute[75234]: DEBUG os_vif [None req-4a795bce-c0be-4707-baff-68c516a2f74c tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Plugging vif VIFOpenVSwitch(active=True,address=fa:16:3e:d8:a2:e7,bridge_name='br-int',has_traffic_filtering=True,id=50593025-edf1-44ec-b5f0-c92d6a807c23,network=Network(1f946e69-dbec-429c-83f8-a186bad4f2cc),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap50593025-ed') {{(pid=75234) plug /opt/stack/data/venv/lib/python3.10/site-packages/os_vif/__init__.py:76}}
Nov 29 03:12:17 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:12:17 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:12:17 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Nov 29 03:12:17 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:12:17 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbCreateCommand(_result=None, table=QoS, columns={'type': 'linux-noop', 'external_ids': {'id': '5f332f4a-3db6-5b3a-9153-840969a09f3a', '_type': 'linux-noop'}}, row=False) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:12:17 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:12:17 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:12:17 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:12:17 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tap50593025-ed, may_exist=True, interface_attrs={}) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:12:17 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Port, record=tap50593025-ed, col_values=(('qos', UUID('33b9a35e-a78f-4f54-bdd7-32c2752b78cb')),), if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:12:17 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DbSetCommand(_result=None, table=Interface, record=tap50593025-ed, col_values=(('external_ids', {'iface-id': '50593025-edf1-44ec-b5f0-c92d6a807c23', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:d8:a2:e7', 'vm-uuid': '6c0e0184-ca3a-48ec-a6e3-a0a1172260d6'}),), if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:12:17 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:12:17 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:12:17 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:12:17 user nova-compute[75234]: INFO os_vif [None req-4a795bce-c0be-4707-baff-68c516a2f74c tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Successfully plugged vif VIFOpenVSwitch(active=True,address=fa:16:3e:d8:a2:e7,bridge_name='br-int',has_traffic_filtering=True,id=50593025-edf1-44ec-b5f0-c92d6a807c23,network=Network(1f946e69-dbec-429c-83f8-a186bad4f2cc),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap50593025-ed')
Nov 29 03:12:17 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-4a795bce-c0be-4707-baff-68c516a2f74c tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] No BDM found with device name vda, not building metadata. {{(pid=75234) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12089}}
Nov 29 03:12:17 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-4a795bce-c0be-4707-baff-68c516a2f74c tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] No BDM found with device name vdb, not building metadata. {{(pid=75234) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12089}}
Nov 29 03:12:17 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-4a795bce-c0be-4707-baff-68c516a2f74c tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] No VIF found with MAC fa:16:3e:d8:a2:e7, not building metadata {{(pid=75234) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Nov 29 03:12:17 user nova-compute[75234]: DEBUG nova.compute.manager [req-8035a266-51a1-46a7-8f76-37259d6764d2 req-cd6267d3-e519-4eda-a878-e6f4071e1b72 service nova] [instance: a9350a18-14e6-42c6-a150-0a1487f20ad9] Received event network-vif-plugged-52024efc-8cde-4610-b12c-198f19d75ccf {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:12:17 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-8035a266-51a1-46a7-8f76-37259d6764d2 req-cd6267d3-e519-4eda-a878-e6f4071e1b72 service nova] Acquiring lock "a9350a18-14e6-42c6-a150-0a1487f20ad9-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:12:17 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-8035a266-51a1-46a7-8f76-37259d6764d2 req-cd6267d3-e519-4eda-a878-e6f4071e1b72 service nova] Lock "a9350a18-14e6-42c6-a150-0a1487f20ad9-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:12:17 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-8035a266-51a1-46a7-8f76-37259d6764d2 req-cd6267d3-e519-4eda-a878-e6f4071e1b72 service nova] Lock "a9350a18-14e6-42c6-a150-0a1487f20ad9-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:12:17 user nova-compute[75234]: DEBUG nova.compute.manager [req-8035a266-51a1-46a7-8f76-37259d6764d2 req-cd6267d3-e519-4eda-a878-e6f4071e1b72 service nova] [instance: a9350a18-14e6-42c6-a150-0a1487f20ad9] No waiting events found dispatching network-vif-plugged-52024efc-8cde-4610-b12c-198f19d75ccf {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:12:17 user nova-compute[75234]: WARNING nova.compute.manager [req-8035a266-51a1-46a7-8f76-37259d6764d2 req-cd6267d3-e519-4eda-a878-e6f4071e1b72 service nova] [instance: a9350a18-14e6-42c6-a150-0a1487f20ad9] Received unexpected event network-vif-plugged-52024efc-8cde-4610-b12c-198f19d75ccf for instance with vm_state resized and task_state resize_reverting.
Nov 29 03:12:18 user nova-compute[75234]: DEBUG nova.network.neutron [req-10ef7f32-f14e-4b6b-b57e-7dfb54b8b62c req-b8edadbe-0bca-48d1-996c-698ff341c4be service nova] [instance: d3b90f39-4b91-4152-b820-bd0e263eec89] Updated VIF entry in instance network info cache for port 83eb52bb-5be5-49d6-81ec-86a21dde3df2. {{(pid=75234) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3481}}
Nov 29 03:12:18 user nova-compute[75234]: DEBUG nova.network.neutron [req-10ef7f32-f14e-4b6b-b57e-7dfb54b8b62c req-b8edadbe-0bca-48d1-996c-698ff341c4be service nova] [instance: d3b90f39-4b91-4152-b820-bd0e263eec89] Updating instance_info_cache with network_info: [{"id": "83eb52bb-5be5-49d6-81ec-86a21dde3df2", "address": "fa:16:3e:9d:70:dd", "network": {"id": "49d62bf6-34ee-4de8-ab85-6ef489d957c8", "bridge": "br-int", "label": "tempest-VolumesAdminNegativeTest-1304144113-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.107", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "5eaa91ebc2e64f2eba987f02bcd1a9f4", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap83eb52bb-5b", "ovs_interfaceid": "83eb52bb-5be5-49d6-81ec-86a21dde3df2", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:12:18 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:12:18 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:12:18 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:12:18 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-10ef7f32-f14e-4b6b-b57e-7dfb54b8b62c req-b8edadbe-0bca-48d1-996c-698ff341c4be service nova] Releasing lock "refresh_cache-d3b90f39-4b91-4152-b820-bd0e263eec89" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:12:18 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-eab52431-6501-43e3-9e20-651dd629a8bc tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Waiting for libvirt event about the detach of device vdb with device alias virtio-disk1 from instance f609a7b7-1be2-44b4-9177-7e72a4d4c060 is timed out.
Nov 29 03:12:18 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-eab52431-6501-43e3-9e20-651dd629a8bc tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Successfully detached device vdb from instance f609a7b7-1be2-44b4-9177-7e72a4d4c060 from the live domain config.
Nov 29 03:12:18 user nova-compute[75234]: INFO nova.compute.manager [None req-23cb5660-78de-4fac-bd94-51e25fe6824e tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] [instance: a082c96e-e384-40b6-bcc8-fc5e0e104669] Rescuing
Nov 29 03:12:18 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-23cb5660-78de-4fac-bd94-51e25fe6824e tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Acquiring lock "refresh_cache-a082c96e-e384-40b6-bcc8-fc5e0e104669" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:12:18 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-23cb5660-78de-4fac-bd94-51e25fe6824e tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Acquired lock "refresh_cache-a082c96e-e384-40b6-bcc8-fc5e0e104669" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:12:18 user nova-compute[75234]: DEBUG nova.network.neutron [None req-23cb5660-78de-4fac-bd94-51e25fe6824e tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] [instance: a082c96e-e384-40b6-bcc8-fc5e0e104669] Building network info cache for instance {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2009}}
Nov 29 03:12:18 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-eab52431-6501-43e3-9e20-651dd629a8bc tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] [instance: f609a7b7-1be2-44b4-9177-7e72a4d4c060] calling os-brick to detach iSCSI Volume {{(pid=75234) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Nov 29 03:12:18 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-eab52431-6501-43e3-9e20-651dd629a8bc tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f8a5050c9a0>, {'target_iqn': 'iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'frKqcBuE', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sde'}, None), 'kwargs': False}}" {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:176}}
Nov 29 03:12:18 user nova-compute[75234]: WARNING os_brick.initiator.connectors.base [None req-eab52431-6501-43e3-9e20-651dd629a8bc tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Nov 29 03:12:18 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-eab52431-6501-43e3-9e20-651dd629a8bc tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:68}}
Nov 29 03:12:18 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-eab52431-6501-43e3-9e20-651dd629a8bc tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.002s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:73}}
Nov 29 03:12:18 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:12:18 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.027s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:12:18 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[f10791df-9d35-4635-9afb-a28c87c4dc59]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:a7ef35bf-017a-4fd9-9182-79c1b2ab4f98\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:2e520fdf-97a2-4104-b51b-2bb4798c8133\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:56237f4c-4138-4f50-aafb-9c0f9d4bb1cf\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:12:18 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-eab52431-6501-43e3-9e20-651dd629a8bc tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS:
Nov 29 03:12:18 user nova-compute[75234]: No targets found.
Nov 29 03:12:18 user nova-compute[75234]: iSNS:
Nov 29 03:12:18 user nova-compute[75234]: No targets found.
Nov 29 03:12:18 user nova-compute[75234]: STATIC:
Nov 29 03:12:18 user nova-compute[75234]: Target: iqn.2010-10.org.openstack:a7ef35bf-017a-4fd9-9182-79c1b2ab4f98
Nov 29 03:12:18 user nova-compute[75234]:         Portal: 172.16.0.220:3260,-1
Nov 29 03:12:18 user nova-compute[75234]:                 Iface Name: default
Nov 29 03:12:18 user nova-compute[75234]: Target: iqn.2010-10.org.openstack:2e520fdf-97a2-4104-b51b-2bb4798c8133
Nov 29 03:12:18 user nova-compute[75234]:         Portal: 172.16.0.220:3260,-1
Nov 29 03:12:18 user nova-compute[75234]:                 Iface Name: default
Nov 29 03:12:18 user nova-compute[75234]: Target: iqn.2010-10.org.openstack:56237f4c-4138-4f50-aafb-9c0f9d4bb1cf
Nov 29 03:12:18 user nova-compute[75234]:         Portal: 172.16.0.220:3260,-1
Nov 29 03:12:18 user nova-compute[75234]:                 Iface Name: default
Nov 29 03:12:18 user nova-compute[75234]: Target: iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672
Nov 29 03:12:18 user nova-compute[75234]:         Portal: 172.16.0.220:3260,-1
Nov 29 03:12:18 user nova-compute[75234]:                 Iface Name: default
Nov 29 03:12:18 user nova-compute[75234]: Target: iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b
Nov 29 03:12:18 user nova-compute[75234]:         Portal: 172.16.0.220:3260,-1
Nov 29 03:12:18 user nova-compute[75234]:                 Iface Name: default
Nov 29 03:12:18 user nova-compute[75234]: FIRMWARE:
Nov 29 03:12:18 user nova-compute[75234]: No targets found.
Nov 29 03:12:18 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:12:18 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-eab52431-6501-43e3-9e20-651dd629a8bc tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Regex to get portals from discoverydb: ^SENDTARGETS:
Nov 29 03:12:18 user nova-compute[75234]: .*?^DiscoveryAddress: 172.16.0.220,3260.*?
Nov 29 03:12:18 user nova-compute[75234]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=75234) _get_discoverydb_portals /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:382}}
Nov 29 03:12:18 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-eab52431-6501-43e3-9e20-651dd629a8bc tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672', 0)] {{(pid=75234) _get_connection_devices /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:830}}
Nov 29 03:12:18 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:12:18 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.058s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:12:18 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[fcf02687-efba-4a2e-bcfb-1186f3e4237c]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:2e520fdf-97a2-4104-b51b-2bb4798c8133\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:56237f4c-4138-4f50-aafb-9c0f9d4bb1cf\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:a7ef35bf-017a-4fd9-9182-79c1b2ab4f98\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:12:18 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:12:18 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.044s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:12:18 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[f6f84865-41ae-4ef8-b577-bd7aa526d16b]: (4, ('tcp: [15] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a7ef35bf-017a-4fd9-9182-79c1b2ab4f98 (non-flash)\ntcp: [16] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672 (non-flash)\ntcp: [19] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:2e520fdf-97a2-4104-b51b-2bb4798c8133 (non-flash)\ntcp: [20] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b (non-flash)\ntcp: [21] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:56237f4c-4138-4f50-aafb-9c0f9d4bb1cf (non-flash)\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:12:18 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-eab52431-6501-43e3-9e20-651dd629a8bc tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [15] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a7ef35bf-017a-4fd9-9182-79c1b2ab4f98 (non-flash)
Nov 29 03:12:18 user nova-compute[75234]: tcp: [16] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672 (non-flash)
Nov 29 03:12:18 user nova-compute[75234]: tcp: [19] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:2e520fdf-97a2-4104-b51b-2bb4798c8133 (non-flash)
Nov 29 03:12:18 user nova-compute[75234]: tcp: [20] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b (non-flash)
Nov 29 03:12:18 user nova-compute[75234]: tcp: [21] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:56237f4c-4138-4f50-aafb-9c0f9d4bb1cf (non-flash)
Nov 29 03:12:18 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:12:18 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-eab52431-6501-43e3-9e20-651dd629a8bc tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] iscsi session list stdout=tcp: [15] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a7ef35bf-017a-4fd9-9182-79c1b2ab4f98 (non-flash)
Nov 29 03:12:18 user nova-compute[75234]: tcp: [16] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672 (non-flash)
Nov 29 03:12:18 user nova-compute[75234]: tcp: [19] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:2e520fdf-97a2-4104-b51b-2bb4798c8133 (non-flash)
Nov 29 03:12:18 user nova-compute[75234]: tcp: [20] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b (non-flash)
Nov 29 03:12:18 user nova-compute[75234]: tcp: [21] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:56237f4c-4138-4f50-aafb-9c0f9d4bb1cf (non-flash)
Nov 29 03:12:18 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsi_session /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Nov 29 03:12:18 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-eab52431-6501-43e3-9e20-651dd629a8bc tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7f8a50627370>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672'): ({'sde'}, set())}) {{(pid=75234) _get_connection_devices /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:863}}
Nov 29 03:12:18 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-eab52431-6501-43e3-9e20-651dd629a8bc tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Removing single pathed devices sde {{(pid=75234) remove_connection /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:374}}
Nov 29 03:12:18 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:12:19 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.133s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:12:19 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[9e7e0517-2e2f-457a-a204-100d8e20c38a]: (4, ('path checker states:\nup                  5\n\npaths: 0\nbusy: False\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:12:19 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sde {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:12:19 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sde" returned: 0 in 0.077s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:12:19 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[962e3e86-31ba-4e99-9c6b-af7eb8f1c9b5]: (4, ('ok\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:12:19 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-eab52431-6501-43e3-9e20-651dd629a8bc tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Flushing IO for device /dev/sde {{(pid=75234) flush_device_io /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:434}}
Nov 29 03:12:19 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sde {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:12:19 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sde" returned: 0 in 0.047s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:12:19 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[b9a8d6a5-7d5d-4cb0-9cc0-b3402ac88745]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:12:19 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-eab52431-6501-43e3-9e20-651dd629a8bc tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Remove SCSI device /dev/sde with /sys/block/sde/device/delete {{(pid=75234) remove_scsi_device /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:148}}
Nov 29 03:12:19 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sde/device/delete {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:12:19 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sde/device/delete" returned: 0 in 0.088s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:12:19 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[4acac494-e1f3-45c7-b455-23fe84e09d24]: (4, ('1', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:12:19 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-eab52431-6501-43e3-9e20-651dd629a8bc tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Checking to see if SCSI volumes sde have been removed. {{(pid=75234) wait_for_volumes_removal /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:156}}
Nov 29 03:12:19 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-eab52431-6501-43e3-9e20-651dd629a8bc tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] SCSI volumes sde have been removed. {{(pid=75234) wait_for_volumes_removal /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:166}}
Nov 29 03:12:19 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-eab52431-6501-43e3-9e20-651dd629a8bc tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672')] {{(pid=75234) _disconnect_connection /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Nov 29 03:12:19 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672 -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:12:19 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672 -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.029s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:12:19 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[425b2332-3293-4c44-b6a8-7106893b078a]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:12:19 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-eab52431-6501-43e3-9e20-651dd629a8bc tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:12:19 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672 -p 172.16.0.220:3260 --logout {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:12:19 user nova-compute[75234]: DEBUG nova.compute.manager [req-840486d0-7728-4a2d-a82a-2882527d07b8 req-fbafe64b-c49b-4f5d-9198-7a1732fc3a20 service nova] [instance: a9350a18-14e6-42c6-a150-0a1487f20ad9] Received event network-vif-plugged-52024efc-8cde-4610-b12c-198f19d75ccf {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:12:19 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-840486d0-7728-4a2d-a82a-2882527d07b8 req-fbafe64b-c49b-4f5d-9198-7a1732fc3a20 service nova] Acquiring lock "a9350a18-14e6-42c6-a150-0a1487f20ad9-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:12:19 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-840486d0-7728-4a2d-a82a-2882527d07b8 req-fbafe64b-c49b-4f5d-9198-7a1732fc3a20 service nova] Lock "a9350a18-14e6-42c6-a150-0a1487f20ad9-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:12:19 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-840486d0-7728-4a2d-a82a-2882527d07b8 req-fbafe64b-c49b-4f5d-9198-7a1732fc3a20 service nova] Lock "a9350a18-14e6-42c6-a150-0a1487f20ad9-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:12:19 user nova-compute[75234]: DEBUG nova.compute.manager [req-840486d0-7728-4a2d-a82a-2882527d07b8 req-fbafe64b-c49b-4f5d-9198-7a1732fc3a20 service nova] [instance: a9350a18-14e6-42c6-a150-0a1487f20ad9] No waiting events found dispatching network-vif-plugged-52024efc-8cde-4610-b12c-198f19d75ccf {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:12:19 user nova-compute[75234]: WARNING nova.compute.manager [req-840486d0-7728-4a2d-a82a-2882527d07b8 req-fbafe64b-c49b-4f5d-9198-7a1732fc3a20 service nova] [instance: a9350a18-14e6-42c6-a150-0a1487f20ad9] Received unexpected event network-vif-plugged-52024efc-8cde-4610-b12c-198f19d75ccf for instance with vm_state resized and task_state resize_reverting.
Nov 29 03:12:19 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672 -p 172.16.0.220:3260 --logout" returned: 0 in 0.150s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:12:19 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[f5653e28-1c09-43ff-ad6d-055075a2238d]: (4, ('Logging out of session [sid: 16, target: iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672, portal: 172.16.0.220,3260]\nLogout of [sid: 16, target: iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:12:19 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-eab52431-6501-43e3-9e20-651dd629a8bc tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 16, target: iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672, portal: 172.16.0.220,3260]
Nov 29 03:12:19 user nova-compute[75234]: Logout of [sid: 16, target: iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672, portal: 172.16.0.220,3260] successful.
Nov 29 03:12:19 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:12:19 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672 -p 172.16.0.220:3260 --op delete {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:12:19 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:33601b89-9deb-4aa4-bd5b-1654f1237672 -p 172.16.0.220:3260 --op delete" returned: 0 in 0.039s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:12:19 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[b166a9ac-e57f-4cce-9ef6-ad83bb60c4a5]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:12:19 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-eab52431-6501-43e3-9e20-651dd629a8bc tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:12:19 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-eab52431-6501-43e3-9e20-651dd629a8bc tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.884s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:87}}
Nov 29 03:12:19 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-eab52431-6501-43e3-9e20-651dd629a8bc tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] <== disconnect_volume: return (886ms) None {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:203}}
Nov 29 03:12:19 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-eab52431-6501-43e3-9e20-651dd629a8bc tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] [instance: f609a7b7-1be2-44b4-9177-7e72a4d4c060] Disconnected iSCSI Volume {{(pid=75234) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:79}}
Nov 29 03:12:19 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:12:19 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:12:19 user nova-compute[75234]: DEBUG nova.network.neutron [None req-23cb5660-78de-4fac-bd94-51e25fe6824e tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] [instance: a082c96e-e384-40b6-bcc8-fc5e0e104669] Updating instance_info_cache with network_info: [{"id": "0d131138-51b6-49f9-9a11-6b64e32e5796", "address": "fa:16:3e:ee:2e:82", "network": {"id": "2b3c3140-f3b6-45da-8202-bb4d7f0dd130", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-2043420778-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "f8a7884867c944e1ba1024270e4902f3", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap0d131138-51", "ovs_interfaceid": "0d131138-51b6-49f9-9a11-6b64e32e5796", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:12:19 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:12:19 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-23cb5660-78de-4fac-bd94-51e25fe6824e tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Releasing lock "refresh_cache-a082c96e-e384-40b6-bcc8-fc5e0e104669" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:12:20 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:12:20 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:12:20 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:12:20 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:12:20 user nova-compute[75234]: DEBUG nova.compute.manager [req-c2eec774-9b55-4c13-900c-0d5e0ca2136b req-b661099b-5a6c-400b-84ce-d1e60fe280fd service nova] [instance: 6c0e0184-ca3a-48ec-a6e3-a0a1172260d6] Received event network-vif-plugged-50593025-edf1-44ec-b5f0-c92d6a807c23 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:12:20 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-c2eec774-9b55-4c13-900c-0d5e0ca2136b req-b661099b-5a6c-400b-84ce-d1e60fe280fd service nova] Acquiring lock "6c0e0184-ca3a-48ec-a6e3-a0a1172260d6-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:12:20 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-c2eec774-9b55-4c13-900c-0d5e0ca2136b req-b661099b-5a6c-400b-84ce-d1e60fe280fd service nova] Lock "6c0e0184-ca3a-48ec-a6e3-a0a1172260d6-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:12:20 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-c2eec774-9b55-4c13-900c-0d5e0ca2136b req-b661099b-5a6c-400b-84ce-d1e60fe280fd service nova] Lock "6c0e0184-ca3a-48ec-a6e3-a0a1172260d6-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:12:20 user nova-compute[75234]: DEBUG nova.compute.manager [req-c2eec774-9b55-4c13-900c-0d5e0ca2136b req-b661099b-5a6c-400b-84ce-d1e60fe280fd service nova] [instance: 6c0e0184-ca3a-48ec-a6e3-a0a1172260d6] No waiting events found dispatching network-vif-plugged-50593025-edf1-44ec-b5f0-c92d6a807c23 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:12:20 user nova-compute[75234]: WARNING nova.compute.manager [req-c2eec774-9b55-4c13-900c-0d5e0ca2136b req-b661099b-5a6c-400b-84ce-d1e60fe280fd service nova] [instance: 6c0e0184-ca3a-48ec-a6e3-a0a1172260d6] Received unexpected event network-vif-plugged-50593025-edf1-44ec-b5f0-c92d6a807c23 for instance with vm_state active and task_state rebuild_spawning.
Nov 29 03:12:20 user nova-compute[75234]: DEBUG nova.compute.manager [req-273a5a04-111b-4ebd-8f18-2ed5a23643c1 req-4fa8b2ac-9e59-4cb6-8456-5249d44e9cfc service nova] [instance: a082c96e-e384-40b6-bcc8-fc5e0e104669] Received event network-vif-unplugged-0d131138-51b6-49f9-9a11-6b64e32e5796 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:12:20 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-273a5a04-111b-4ebd-8f18-2ed5a23643c1 req-4fa8b2ac-9e59-4cb6-8456-5249d44e9cfc service nova] Acquiring lock "a082c96e-e384-40b6-bcc8-fc5e0e104669-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:12:20 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-273a5a04-111b-4ebd-8f18-2ed5a23643c1 req-4fa8b2ac-9e59-4cb6-8456-5249d44e9cfc service nova] Lock "a082c96e-e384-40b6-bcc8-fc5e0e104669-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:12:20 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-273a5a04-111b-4ebd-8f18-2ed5a23643c1 req-4fa8b2ac-9e59-4cb6-8456-5249d44e9cfc service nova] Lock "a082c96e-e384-40b6-bcc8-fc5e0e104669-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:12:20 user nova-compute[75234]: DEBUG nova.compute.manager [req-273a5a04-111b-4ebd-8f18-2ed5a23643c1 req-4fa8b2ac-9e59-4cb6-8456-5249d44e9cfc service nova] [instance: a082c96e-e384-40b6-bcc8-fc5e0e104669] No waiting events found dispatching network-vif-unplugged-0d131138-51b6-49f9-9a11-6b64e32e5796 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:12:20 user nova-compute[75234]: WARNING nova.compute.manager [req-273a5a04-111b-4ebd-8f18-2ed5a23643c1 req-4fa8b2ac-9e59-4cb6-8456-5249d44e9cfc service nova] [instance: a082c96e-e384-40b6-bcc8-fc5e0e104669] Received unexpected event network-vif-unplugged-0d131138-51b6-49f9-9a11-6b64e32e5796 for instance with vm_state active and task_state rescuing.
Nov 29 03:12:20 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:12:20 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:12:20 user nova-compute[75234]: INFO nova.virt.libvirt.driver [-] [instance: a082c96e-e384-40b6-bcc8-fc5e0e104669] Instance destroyed successfully.
Nov 29 03:12:20 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-23cb5660-78de-4fac-bd94-51e25fe6824e tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] [instance: a082c96e-e384-40b6-bcc8-fc5e0e104669] Attempting rescue
Nov 29 03:12:20 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-23cb5660-78de-4fac-bd94-51e25fe6824e tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] rescue generated disk_info: {'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'disk.rescue': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vdb', 'type': 'disk'}}} {{(pid=75234) rescue /opt/stack/nova/nova/virt/libvirt/driver.py:4318}}
Nov 29 03:12:20 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-23cb5660-78de-4fac-bd94-51e25fe6824e tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] [instance: a082c96e-e384-40b6-bcc8-fc5e0e104669] Instance directory exists: not creating {{(pid=75234) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4723}}
Nov 29 03:12:20 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-23cb5660-78de-4fac-bd94-51e25fe6824e tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] [instance: a082c96e-e384-40b6-bcc8-fc5e0e104669] Creating image(s)
Nov 29 03:12:20 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-23cb5660-78de-4fac-bd94-51e25fe6824e tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Acquiring lock "/opt/stack/data/nova/instances/a082c96e-e384-40b6-bcc8-fc5e0e104669/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:12:20 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-23cb5660-78de-4fac-bd94-51e25fe6824e tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Lock "/opt/stack/data/nova/instances/a082c96e-e384-40b6-bcc8-fc5e0e104669/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:12:20 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-23cb5660-78de-4fac-bd94-51e25fe6824e tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Lock "/opt/stack/data/nova/instances/a082c96e-e384-40b6-bcc8-fc5e0e104669/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.003s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:12:20 user nova-compute[75234]: DEBUG nova.objects.instance [None req-23cb5660-78de-4fac-bd94-51e25fe6824e tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Lazy-loading 'trusted_certs' on Instance uuid a082c96e-e384-40b6-bcc8-fc5e0e104669 {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:12:20 user nova-compute[75234]: DEBUG nova.objects.instance [None req-eab52431-6501-43e3-9e20-651dd629a8bc tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Lazy-loading 'flavor' on Instance uuid f609a7b7-1be2-44b4-9177-7e72a4d4c060 {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:12:20 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-23cb5660-78de-4fac-bd94-51e25fe6824e tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Acquiring lock "80686e29dc9db6600d03e85cb62af298e12d7918" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:12:20 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-23cb5660-78de-4fac-bd94-51e25fe6824e tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Lock "80686e29dc9db6600d03e85cb62af298e12d7918" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:12:20 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-23cb5660-78de-4fac-bd94-51e25fe6824e tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:12:21 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-23cb5660-78de-4fac-bd94-51e25fe6824e tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json" returned: 0 in 0.207s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:12:21 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-23cb5660-78de-4fac-bd94-51e25fe6824e tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918,backing_fmt=raw /opt/stack/data/nova/instances/a082c96e-e384-40b6-bcc8-fc5e0e104669/disk.rescue {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:12:21 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-eee1a7b9-fa96-457b-831f-9ccbd0ccc6aa tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] Acquiring lock "d3b90f39-4b91-4152-b820-bd0e263eec89" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:12:21 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-eee1a7b9-fa96-457b-831f-9ccbd0ccc6aa tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] Lock "d3b90f39-4b91-4152-b820-bd0e263eec89" acquired by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:12:21 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-eab52431-6501-43e3-9e20-651dd629a8bc tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Lock "f609a7b7-1be2-44b4-9177-7e72a4d4c060" "released" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: held 23.445s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:12:21 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "f609a7b7-1be2-44b4-9177-7e72a4d4c060" acquired by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: waited 16.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:12:21 user nova-compute[75234]: DEBUG nova.objects.instance [None req-eee1a7b9-fa96-457b-831f-9ccbd0ccc6aa tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] Lazy-loading 'flavor' on Instance uuid d3b90f39-4b91-4152-b820-bd0e263eec89 {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:12:21 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "f609a7b7-1be2-44b4-9177-7e72a4d4c060" "released" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: held 0.093s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:12:21 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-23cb5660-78de-4fac-bd94-51e25fe6824e tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918,backing_fmt=raw /opt/stack/data/nova/instances/a082c96e-e384-40b6-bcc8-fc5e0e104669/disk.rescue" returned: 0 in 0.217s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:12:21 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-23cb5660-78de-4fac-bd94-51e25fe6824e tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Lock "80686e29dc9db6600d03e85cb62af298e12d7918" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.431s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:12:21 user nova-compute[75234]: DEBUG nova.objects.instance [None req-23cb5660-78de-4fac-bd94-51e25fe6824e tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Lazy-loading 'migration_context' on Instance uuid a082c96e-e384-40b6-bcc8-fc5e0e104669 {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:12:21 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-eee1a7b9-fa96-457b-831f-9ccbd0ccc6aa tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] Lock "d3b90f39-4b91-4152-b820-bd0e263eec89" "released" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: held 0.159s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:12:21 user nova-compute[75234]: DEBUG nova.objects.base [None req-23cb5660-78de-4fac-bd94-51e25fe6824e tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Object Instance<a082c96e-e384-40b6-bcc8-fc5e0e104669> lazy-loaded attributes: trusted_certs,migration_context {{(pid=75234) wrapper /opt/stack/nova/nova/objects/base.py:126}}
Nov 29 03:12:21 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-23cb5660-78de-4fac-bd94-51e25fe6824e tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] [instance: a082c96e-e384-40b6-bcc8-fc5e0e104669] Created local disks {{(pid=75234) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4861}}
Nov 29 03:12:21 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-23cb5660-78de-4fac-bd94-51e25fe6824e tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] [instance: a082c96e-e384-40b6-bcc8-fc5e0e104669] Start _get_guest_xml network_info=[{"id": "0d131138-51b6-49f9-9a11-6b64e32e5796", "address": "fa:16:3e:ee:2e:82", "network": {"id": "2b3c3140-f3b6-45da-8202-bb4d7f0dd130", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-2043420778-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [], "label": "tempest-ServerRescueNegativeTestJSON-2043420778-network", "vif_mac": "fa:16:3e:ee:2e:82"}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "f8a7884867c944e1ba1024270e4902f3", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap0d131138-51", "ovs_interfaceid": "0d131138-51b6-49f9-9a11-6b64e32e5796", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'disk.rescue': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vdb', 'type': 'disk'}}} image_meta=ImageMeta(checksum='c8fc807773e5354afe61636071771906',container_format='bare',created_at=2023-11-29T03:02:15Z,direct_url=<?>,disk_format='qcow2',id=96c30ba7-9b73-4a30-8ba4-7138879573b7,min_disk=0,min_ram=0,name='cirros-0.6.2-x86_64-disk',owner='b18e380921c748b78517b6f1211cd0f1',properties=ImageMetaProps,protected=<?>,size=21430272,status='active',tags=<?>,updated_at=2023-11-29T03:02:17Z,virtual_size=<?>,visibility=<?>) rescue={'image_id': '96c30ba7-9b73-4a30-8ba4-7138879573b7', 'kernel_id': '', 'ramdisk_id': ''} block_device_info=None {{(pid=75234) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7562}}
Nov 29 03:12:21 user nova-compute[75234]: DEBUG nova.objects.instance [None req-23cb5660-78de-4fac-bd94-51e25fe6824e tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Lazy-loading 'resources' on Instance uuid a082c96e-e384-40b6-bcc8-fc5e0e104669 {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:12:21 user nova-compute[75234]: DEBUG nova.objects.base [None req-23cb5660-78de-4fac-bd94-51e25fe6824e tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Object Instance<a082c96e-e384-40b6-bcc8-fc5e0e104669> lazy-loaded attributes: trusted_certs,migration_context,resources {{(pid=75234) wrapper /opt/stack/nova/nova/objects/base.py:126}}
Nov 29 03:12:21 user nova-compute[75234]: DEBUG nova.objects.instance [None req-23cb5660-78de-4fac-bd94-51e25fe6824e tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Lazy-loading 'numa_topology' on Instance uuid a082c96e-e384-40b6-bcc8-fc5e0e104669 {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:12:21 user nova-compute[75234]: DEBUG nova.objects.base [None req-23cb5660-78de-4fac-bd94-51e25fe6824e tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Object Instance<a082c96e-e384-40b6-bcc8-fc5e0e104669> lazy-loaded attributes: trusted_certs,migration_context,resources,numa_topology {{(pid=75234) wrapper /opt/stack/nova/nova/objects/base.py:126}}
Nov 29 03:12:21 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-23cb5660-78de-4fac-bd94-51e25fe6824e tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:12:21 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-23cb5660-78de-4fac-bd94-51e25fe6824e tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:12:21 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:12:21 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-23cb5660-78de-4fac-bd94-51e25fe6824e tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Searching host: 'user' for CPU controller through CGroups V1... {{(pid=75234) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1652}}
Nov 29 03:12:21 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-23cb5660-78de-4fac-bd94-51e25fe6824e tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] CPU controller missing on host. {{(pid=75234) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1662}}
Nov 29 03:12:21 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-23cb5660-78de-4fac-bd94-51e25fe6824e tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Searching host: 'user' for CPU controller through CGroups V2... {{(pid=75234) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1671}}
Nov 29 03:12:21 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-23cb5660-78de-4fac-bd94-51e25fe6824e tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] CPU controller found on host. {{(pid=75234) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1678}}
Nov 29 03:12:21 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-23cb5660-78de-4fac-bd94-51e25fe6824e tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=75234) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5400}}
Nov 29 03:12:21 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-23cb5660-78de-4fac-bd94-51e25fe6824e tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-11-29T03:02:04Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='1',id=6,is_public=True,memory_mb=512,name='m1.tiny',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='c8fc807773e5354afe61636071771906',container_format='bare',created_at=2023-11-29T03:02:15Z,direct_url=<?>,disk_format='qcow2',id=96c30ba7-9b73-4a30-8ba4-7138879573b7,min_disk=0,min_ram=0,name='cirros-0.6.2-x86_64-disk',owner='b18e380921c748b78517b6f1211cd0f1',properties=ImageMetaProps,protected=<?>,size=21430272,status='active',tags=<?>,updated_at=2023-11-29T03:02:17Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:563}}
Nov 29 03:12:21 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-23cb5660-78de-4fac-bd94-51e25fe6824e tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Flavor limits 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:348}}
Nov 29 03:12:21 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-23cb5660-78de-4fac-bd94-51e25fe6824e tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Image limits 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:352}}
Nov 29 03:12:21 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-23cb5660-78de-4fac-bd94-51e25fe6824e tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Flavor pref 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:388}}
Nov 29 03:12:21 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-23cb5660-78de-4fac-bd94-51e25fe6824e tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Image pref 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:392}}
Nov 29 03:12:21 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-23cb5660-78de-4fac-bd94-51e25fe6824e tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:430}}
Nov 29 03:12:21 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-23cb5660-78de-4fac-bd94-51e25fe6824e tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:569}}
Nov 29 03:12:21 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-23cb5660-78de-4fac-bd94-51e25fe6824e tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=75234) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:471}}
Nov 29 03:12:21 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-23cb5660-78de-4fac-bd94-51e25fe6824e tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Got 1 possible topologies {{(pid=75234) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:501}}
Nov 29 03:12:21 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-23cb5660-78de-4fac-bd94-51e25fe6824e tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:575}}
Nov 29 03:12:21 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-23cb5660-78de-4fac-bd94-51e25fe6824e tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:577}}
Nov 29 03:12:21 user nova-compute[75234]: DEBUG nova.objects.instance [None req-23cb5660-78de-4fac-bd94-51e25fe6824e tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Lazy-loading 'vcpu_model' on Instance uuid a082c96e-e384-40b6-bcc8-fc5e0e104669 {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:12:21 user nova-compute[75234]: DEBUG nova.objects.base [None req-23cb5660-78de-4fac-bd94-51e25fe6824e tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Object Instance<a082c96e-e384-40b6-bcc8-fc5e0e104669> lazy-loaded attributes: trusted_certs,migration_context,resources,numa_topology,vcpu_model {{(pid=75234) wrapper /opt/stack/nova/nova/objects/base.py:126}}
Nov 29 03:12:21 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-23cb5660-78de-4fac-bd94-51e25fe6824e tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-11-29T03:10:26Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-ServerRescueNegativeTestJSON-server-2058873758',display_name='tempest-ServerRescueNegativeTestJSON-server-2058873758',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-serverrescuenegativetestjson-server-2058873758',id=19,image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data=None,key_name=None,keypairs=<?>,launch_index=0,launched_at=2023-11-29T03:10:38Z,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='f8a7884867c944e1ba1024270e4902f3',ramdisk_id='',reservation_id='r-58ukk6ag',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-ServerRescueNegativeTestJSON-1182706601',owner_user_name='tempest-ServerRescueNegativeTestJSON-1182706601-project-member'},tags=<?>,task_state='rescuing',terminated_at=None,trusted_certs=None,updated_at=2023-11-29T03:10:39Z,user_data=None,user_id='6792e866677645c598ddb7d2c1723a1a',uuid=a082c96e-e384-40b6-bcc8-fc5e0e104669,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "0d131138-51b6-49f9-9a11-6b64e32e5796", "address": "fa:16:3e:ee:2e:82", "network": {"id": "2b3c3140-f3b6-45da-8202-bb4d7f0dd130", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-2043420778-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [], "label": "tempest-ServerRescueNegativeTestJSON-2043420778-network", "vif_mac": "fa:16:3e:ee:2e:82"}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "f8a7884867c944e1ba1024270e4902f3", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap0d131138-51", "ovs_interfaceid": "0d131138-51b6-49f9-9a11-6b64e32e5796", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=75234) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Nov 29 03:12:21 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-23cb5660-78de-4fac-bd94-51e25fe6824e tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Converting VIF {"id": "0d131138-51b6-49f9-9a11-6b64e32e5796", "address": "fa:16:3e:ee:2e:82", "network": {"id": "2b3c3140-f3b6-45da-8202-bb4d7f0dd130", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-2043420778-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [], "label": "tempest-ServerRescueNegativeTestJSON-2043420778-network", "vif_mac": "fa:16:3e:ee:2e:82"}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "f8a7884867c944e1ba1024270e4902f3", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap0d131138-51", "ovs_interfaceid": "0d131138-51b6-49f9-9a11-6b64e32e5796", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:12:21 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-23cb5660-78de-4fac-bd94-51e25fe6824e tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:ee:2e:82,bridge_name='br-int',has_traffic_filtering=True,id=0d131138-51b6-49f9-9a11-6b64e32e5796,network=Network(2b3c3140-f3b6-45da-8202-bb4d7f0dd130),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap0d131138-51') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:12:21 user nova-compute[75234]: DEBUG nova.objects.instance [None req-23cb5660-78de-4fac-bd94-51e25fe6824e tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Lazy-loading 'pci_devices' on Instance uuid a082c96e-e384-40b6-bcc8-fc5e0e104669 {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:12:21 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-2c12d1f3-c2dc-4d62-b1dd-f4fb3caea12f tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Acquiring lock "f609a7b7-1be2-44b4-9177-7e72a4d4c060" by "nova.compute.manager.ComputeManager.stop_instance.<locals>.do_stop_instance" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:12:21 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-2c12d1f3-c2dc-4d62-b1dd-f4fb3caea12f tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Lock "f609a7b7-1be2-44b4-9177-7e72a4d4c060" acquired by "nova.compute.manager.ComputeManager.stop_instance.<locals>.do_stop_instance" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:12:21 user nova-compute[75234]: DEBUG nova.compute.manager [None req-2c12d1f3-c2dc-4d62-b1dd-f4fb3caea12f tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] [instance: f609a7b7-1be2-44b4-9177-7e72a4d4c060] Checking state {{(pid=75234) _get_power_state /opt/stack/nova/nova/compute/manager.py:1782}}
Nov 29 03:12:21 user nova-compute[75234]: DEBUG nova.objects.base [None req-23cb5660-78de-4fac-bd94-51e25fe6824e tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Object Instance<a082c96e-e384-40b6-bcc8-fc5e0e104669> lazy-loaded attributes: trusted_certs,migration_context,resources,numa_topology,vcpu_model,pci_devices {{(pid=75234) wrapper /opt/stack/nova/nova/objects/base.py:126}}
Nov 29 03:12:21 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-23cb5660-78de-4fac-bd94-51e25fe6824e tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] [instance: a082c96e-e384-40b6-bcc8-fc5e0e104669] End _get_guest_xml xml=<domain type="kvm">
Nov 29 03:12:21 user nova-compute[75234]:   <uuid>a082c96e-e384-40b6-bcc8-fc5e0e104669</uuid>
Nov 29 03:12:21 user nova-compute[75234]:   <name>instance-00000013</name>
Nov 29 03:12:21 user nova-compute[75234]:   <memory>524288</memory>
Nov 29 03:12:21 user nova-compute[75234]:   <vcpu>1</vcpu>
Nov 29 03:12:21 user nova-compute[75234]:   <metadata>
Nov 29 03:12:21 user nova-compute[75234]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Nov 29 03:12:21 user nova-compute[75234]:       <nova:package version="0.0.0"/>
Nov 29 03:12:21 user nova-compute[75234]:       <nova:name>tempest-ServerRescueNegativeTestJSON-server-2058873758</nova:name>
Nov 29 03:12:21 user nova-compute[75234]:       <nova:creationTime>2023-11-29 03:12:21</nova:creationTime>
Nov 29 03:12:21 user nova-compute[75234]:       <nova:flavor name="m1.tiny">
Nov 29 03:12:21 user nova-compute[75234]:         <nova:memory>512</nova:memory>
Nov 29 03:12:21 user nova-compute[75234]:         <nova:disk>1</nova:disk>
Nov 29 03:12:21 user nova-compute[75234]:         <nova:swap>0</nova:swap>
Nov 29 03:12:21 user nova-compute[75234]:         <nova:ephemeral>0</nova:ephemeral>
Nov 29 03:12:21 user nova-compute[75234]:         <nova:vcpus>1</nova:vcpus>
Nov 29 03:12:21 user nova-compute[75234]:       </nova:flavor>
Nov 29 03:12:21 user nova-compute[75234]:       <nova:owner>
Nov 29 03:12:21 user nova-compute[75234]:         <nova:user uuid="6792e866677645c598ddb7d2c1723a1a">tempest-ServerRescueNegativeTestJSON-1182706601-project-member</nova:user>
Nov 29 03:12:21 user nova-compute[75234]:         <nova:project uuid="f8a7884867c944e1ba1024270e4902f3">tempest-ServerRescueNegativeTestJSON-1182706601</nova:project>
Nov 29 03:12:21 user nova-compute[75234]:       </nova:owner>
Nov 29 03:12:21 user nova-compute[75234]:       <nova:root type="image" uuid="96c30ba7-9b73-4a30-8ba4-7138879573b7"/>
Nov 29 03:12:21 user nova-compute[75234]:       <nova:ports>
Nov 29 03:12:21 user nova-compute[75234]:         <nova:port uuid="0d131138-51b6-49f9-9a11-6b64e32e5796">
Nov 29 03:12:21 user nova-compute[75234]:           <nova:ip type="fixed" address="10.0.0.10" ipVersion="4"/>
Nov 29 03:12:21 user nova-compute[75234]:         </nova:port>
Nov 29 03:12:21 user nova-compute[75234]:       </nova:ports>
Nov 29 03:12:21 user nova-compute[75234]:     </nova:instance>
Nov 29 03:12:21 user nova-compute[75234]:   </metadata>
Nov 29 03:12:21 user nova-compute[75234]:   <sysinfo type="smbios">
Nov 29 03:12:21 user nova-compute[75234]:     <system>
Nov 29 03:12:21 user nova-compute[75234]:       <entry name="manufacturer">OpenStack Foundation</entry>
Nov 29 03:12:21 user nova-compute[75234]:       <entry name="product">OpenStack Nova</entry>
Nov 29 03:12:21 user nova-compute[75234]:       <entry name="version">0.0.0</entry>
Nov 29 03:12:21 user nova-compute[75234]:       <entry name="serial">a082c96e-e384-40b6-bcc8-fc5e0e104669</entry>
Nov 29 03:12:21 user nova-compute[75234]:       <entry name="uuid">a082c96e-e384-40b6-bcc8-fc5e0e104669</entry>
Nov 29 03:12:21 user nova-compute[75234]:       <entry name="family">Virtual Machine</entry>
Nov 29 03:12:21 user nova-compute[75234]:     </system>
Nov 29 03:12:21 user nova-compute[75234]:   </sysinfo>
Nov 29 03:12:21 user nova-compute[75234]:   <os>
Nov 29 03:12:21 user nova-compute[75234]:     <type arch="x86_64" machine="pc">hvm</type>
Nov 29 03:12:21 user nova-compute[75234]:     <smbios mode="sysinfo"/>
Nov 29 03:12:21 user nova-compute[75234]:   </os>
Nov 29 03:12:21 user nova-compute[75234]:   <features>
Nov 29 03:12:21 user nova-compute[75234]:     <acpi/>
Nov 29 03:12:21 user nova-compute[75234]:     <apic/>
Nov 29 03:12:21 user nova-compute[75234]:     <vmcoreinfo/>
Nov 29 03:12:21 user nova-compute[75234]:   </features>
Nov 29 03:12:21 user nova-compute[75234]:   <clock offset="utc">
Nov 29 03:12:21 user nova-compute[75234]:     <timer name="pit" tickpolicy="delay"/>
Nov 29 03:12:21 user nova-compute[75234]:     <timer name="rtc" tickpolicy="catchup"/>
Nov 29 03:12:21 user nova-compute[75234]:     <timer name="hpet" present="no"/>
Nov 29 03:12:21 user nova-compute[75234]:   </clock>
Nov 29 03:12:21 user nova-compute[75234]:   <cpu mode="custom" match="exact">
Nov 29 03:12:21 user nova-compute[75234]:     <model>Nehalem</model>
Nov 29 03:12:21 user nova-compute[75234]:     <topology sockets="1" cores="1" threads="1"/>
Nov 29 03:12:21 user nova-compute[75234]:   </cpu>
Nov 29 03:12:21 user nova-compute[75234]:   <devices>
Nov 29 03:12:21 user nova-compute[75234]:     <disk type="file" device="disk">
Nov 29 03:12:21 user nova-compute[75234]:       <driver name="qemu" type="qcow2" cache="none"/>
Nov 29 03:12:21 user nova-compute[75234]:       <source file="/opt/stack/data/nova/instances/a082c96e-e384-40b6-bcc8-fc5e0e104669/disk.rescue"/>
Nov 29 03:12:21 user nova-compute[75234]:       <target dev="vda" bus="virtio"/>
Nov 29 03:12:21 user nova-compute[75234]:     </disk>
Nov 29 03:12:21 user nova-compute[75234]:     <disk type="file" device="disk">
Nov 29 03:12:21 user nova-compute[75234]:       <driver name="qemu" type="qcow2" cache="none"/>
Nov 29 03:12:21 user nova-compute[75234]:       <source file="/opt/stack/data/nova/instances/a082c96e-e384-40b6-bcc8-fc5e0e104669/disk"/>
Nov 29 03:12:21 user nova-compute[75234]:       <target dev="vdb" bus="virtio"/>
Nov 29 03:12:21 user nova-compute[75234]:     </disk>
Nov 29 03:12:21 user nova-compute[75234]:     <interface type="ethernet">
Nov 29 03:12:21 user nova-compute[75234]:       <mac address="fa:16:3e:ee:2e:82"/>
Nov 29 03:12:21 user nova-compute[75234]:       <model type="virtio"/>
Nov 29 03:12:21 user nova-compute[75234]:       <mtu size="1442"/>
Nov 29 03:12:21 user nova-compute[75234]:       <target dev="tap0d131138-51"/>
Nov 29 03:12:21 user nova-compute[75234]:     </interface>
Nov 29 03:12:21 user nova-compute[75234]:     <serial type="pty">
Nov 29 03:12:21 user nova-compute[75234]:       <log file="/opt/stack/data/nova/instances/a082c96e-e384-40b6-bcc8-fc5e0e104669/console.log" append="off"/>
Nov 29 03:12:21 user nova-compute[75234]:     </serial>
Nov 29 03:12:21 user nova-compute[75234]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Nov 29 03:12:21 user nova-compute[75234]:     <video>
Nov 29 03:12:21 user nova-compute[75234]:       <model type="virtio"/>
Nov 29 03:12:21 user nova-compute[75234]:     </video>
Nov 29 03:12:21 user nova-compute[75234]:     <rng model="virtio">
Nov 29 03:12:21 user nova-compute[75234]:       <backend model="random">/dev/urandom</backend>
Nov 29 03:12:21 user nova-compute[75234]:     </rng>
Nov 29 03:12:21 user nova-compute[75234]:     <controller type="usb" index="0" model="none"/>
Nov 29 03:12:21 user nova-compute[75234]:     <memballoon model="virtio">
Nov 29 03:12:21 user nova-compute[75234]:       <stats period="10"/>
Nov 29 03:12:21 user nova-compute[75234]:     </memballoon>
Nov 29 03:12:21 user nova-compute[75234]:   </devices>
Nov 29 03:12:21 user nova-compute[75234]: </domain>
Nov 29 03:12:21 user nova-compute[75234]:  {{(pid=75234) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7568}}
Nov 29 03:12:21 user nova-compute[75234]: DEBUG nova.compute.manager [None req-2c12d1f3-c2dc-4d62-b1dd-f4fb3caea12f tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] [instance: f609a7b7-1be2-44b4-9177-7e72a4d4c060] Stopping instance; current vm_state: active, current task_state: powering-off, current DB power_state: 1, current VM power_state: 1 {{(pid=75234) do_stop_instance /opt/stack/nova/nova/compute/manager.py:3358}}
Nov 29 03:12:21 user nova-compute[75234]: DEBUG nova.objects.instance [None req-2c12d1f3-c2dc-4d62-b1dd-f4fb3caea12f tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Lazy-loading 'flavor' on Instance uuid f609a7b7-1be2-44b4-9177-7e72a4d4c060 {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:12:21 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-eee1a7b9-fa96-457b-831f-9ccbd0ccc6aa tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] Acquiring lock "d3b90f39-4b91-4152-b820-bd0e263eec89" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:12:21 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-eee1a7b9-fa96-457b-831f-9ccbd0ccc6aa tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] Lock "d3b90f39-4b91-4152-b820-bd0e263eec89" acquired by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: waited 0.002s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:12:21 user nova-compute[75234]: INFO nova.compute.manager [None req-eee1a7b9-fa96-457b-831f-9ccbd0ccc6aa tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] [instance: d3b90f39-4b91-4152-b820-bd0e263eec89] Attaching volume 634753d3-60a5-4f55-9485-14b6b488a121 to /dev/vdb
Nov 29 03:12:21 user nova-compute[75234]: INFO nova.virt.libvirt.driver [-] [instance: a082c96e-e384-40b6-bcc8-fc5e0e104669] Instance destroyed successfully.
Nov 29 03:12:21 user nova-compute[75234]: DEBUG os_brick.utils [None req-eee1a7b9-fa96-457b-831f-9ccbd0ccc6aa tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:176}}
Nov 29 03:12:21 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:12:21 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.030s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:12:21 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[d97c955a-6e99-48c1-9ff3-4ea7585889d5]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:dd1368c5ba2c\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:12:21 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt -v / -n -o SOURCE {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:12:22 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt -v / -n -o SOURCE" returned: 0 in 0.019s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:12:22 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[70e0d16b-1203-42c0-a39d-790eb85620cb]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:12:22 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:12:22 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:12:22 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-23cb5660-78de-4fac-bd94-51e25fe6824e tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] No BDM found with device name vda, not building metadata. {{(pid=75234) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12089}}
Nov 29 03:12:22 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-23cb5660-78de-4fac-bd94-51e25fe6824e tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] No BDM found with device name vdb, not building metadata. {{(pid=75234) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12089}}
Nov 29 03:12:22 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-23cb5660-78de-4fac-bd94-51e25fe6824e tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] No VIF found with MAC fa:16:3e:ee:2e:82, not building metadata {{(pid=75234) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Nov 29 03:12:22 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.026s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:12:22 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[9d526646-6cc1-4051-9695-30203ae96b43]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:12:22 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[b0c9cd81-198f-44f2-ad74-bdc0fa058c8e]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:12:22 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:12:22 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-eee1a7b9-fa96-457b-831f-9ccbd0ccc6aa tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] Running cmd (subprocess): nvme version {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:12:22 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-eee1a7b9-fa96-457b-831f-9ccbd0ccc6aa tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] 'nvme version' failed. Not Retrying. {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:473}}
Nov 29 03:12:22 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:12:22 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.nvmeof [None req-eee1a7b9-fa96-457b-831f-9ccbd0ccc6aa tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] nvme not present on system {{(pid=75234) nvme_present /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/nvmeof.py:756}}
Nov 29 03:12:22 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:12:22 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:12:22 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:12:22 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:473}}
Nov 29 03:12:22 user nova-compute[75234]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Nov 29 03:12:22 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[2bbf6034-5299-437b-871c-d8cf02ed3de5]: (4, '') {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:12:22 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.lightos [None req-eee1a7b9-fa96-457b-831f-9ccbd0ccc6aa tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=75234) find_dsc /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/lightos.py:98}}
Nov 29 03:12:22 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.lightos [None req-eee1a7b9-fa96-457b-831f-9ccbd0ccc6aa tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=75234) get_connector_properties /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/lightos.py:76}}
Nov 29 03:12:22 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.lightos [None req-eee1a7b9-fa96-457b-831f-9ccbd0ccc6aa tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] LIGHTOS: no hostnqn found. {{(pid=75234) get_connector_properties /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/lightos.py:84}}
Nov 29 03:12:22 user nova-compute[75234]: DEBUG os_brick.utils [None req-eee1a7b9-fa96-457b-831f-9ccbd0ccc6aa tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] <== get_connector_properties: return (219ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:dd1368c5ba2c', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:203}}
Nov 29 03:12:22 user nova-compute[75234]: DEBUG nova.virt.block_device [None req-eee1a7b9-fa96-457b-831f-9ccbd0ccc6aa tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] [instance: d3b90f39-4b91-4152-b820-bd0e263eec89] Updating existing volume attachment record: 26455adb-3b86-492d-9c31-4c0f84721fc8 {{(pid=75234) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Nov 29 03:12:22 user nova-compute[75234]: DEBUG nova.compute.manager [req-4c83c288-5778-4a3e-96c5-753511935e32 req-09f59958-5e89-4d7e-91e7-0b77f110ba8b service nova] [instance: f609a7b7-1be2-44b4-9177-7e72a4d4c060] Received event network-vif-unplugged-5543f379-2b3a-4dc2-bfe9-52f66a942257 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:12:22 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-4c83c288-5778-4a3e-96c5-753511935e32 req-09f59958-5e89-4d7e-91e7-0b77f110ba8b service nova] Acquiring lock "f609a7b7-1be2-44b4-9177-7e72a4d4c060-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:12:22 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-4c83c288-5778-4a3e-96c5-753511935e32 req-09f59958-5e89-4d7e-91e7-0b77f110ba8b service nova] Lock "f609a7b7-1be2-44b4-9177-7e72a4d4c060-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:12:22 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-4c83c288-5778-4a3e-96c5-753511935e32 req-09f59958-5e89-4d7e-91e7-0b77f110ba8b service nova] Lock "f609a7b7-1be2-44b4-9177-7e72a4d4c060-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:12:22 user nova-compute[75234]: DEBUG nova.compute.manager [req-4c83c288-5778-4a3e-96c5-753511935e32 req-09f59958-5e89-4d7e-91e7-0b77f110ba8b service nova] [instance: f609a7b7-1be2-44b4-9177-7e72a4d4c060] No waiting events found dispatching network-vif-unplugged-5543f379-2b3a-4dc2-bfe9-52f66a942257 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:12:22 user nova-compute[75234]: WARNING nova.compute.manager [req-4c83c288-5778-4a3e-96c5-753511935e32 req-09f59958-5e89-4d7e-91e7-0b77f110ba8b service nova] [instance: f609a7b7-1be2-44b4-9177-7e72a4d4c060] Received unexpected event network-vif-unplugged-5543f379-2b3a-4dc2-bfe9-52f66a942257 for instance with vm_state active and task_state powering-off.
Nov 29 03:12:22 user nova-compute[75234]: DEBUG nova.compute.manager [req-05990c81-6655-484f-9f71-19bf29dd1b32 req-42eb9fb1-5052-491a-978c-e8de603a293e service nova] [instance: 6c0e0184-ca3a-48ec-a6e3-a0a1172260d6] Received event network-vif-plugged-50593025-edf1-44ec-b5f0-c92d6a807c23 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:12:22 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-05990c81-6655-484f-9f71-19bf29dd1b32 req-42eb9fb1-5052-491a-978c-e8de603a293e service nova] Acquiring lock "6c0e0184-ca3a-48ec-a6e3-a0a1172260d6-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:12:22 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-05990c81-6655-484f-9f71-19bf29dd1b32 req-42eb9fb1-5052-491a-978c-e8de603a293e service nova] Lock "6c0e0184-ca3a-48ec-a6e3-a0a1172260d6-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:12:22 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-05990c81-6655-484f-9f71-19bf29dd1b32 req-42eb9fb1-5052-491a-978c-e8de603a293e service nova] Lock "6c0e0184-ca3a-48ec-a6e3-a0a1172260d6-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:12:22 user nova-compute[75234]: DEBUG nova.compute.manager [req-05990c81-6655-484f-9f71-19bf29dd1b32 req-42eb9fb1-5052-491a-978c-e8de603a293e service nova] [instance: 6c0e0184-ca3a-48ec-a6e3-a0a1172260d6] No waiting events found dispatching network-vif-plugged-50593025-edf1-44ec-b5f0-c92d6a807c23 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:12:22 user nova-compute[75234]: WARNING nova.compute.manager [req-05990c81-6655-484f-9f71-19bf29dd1b32 req-42eb9fb1-5052-491a-978c-e8de603a293e service nova] [instance: 6c0e0184-ca3a-48ec-a6e3-a0a1172260d6] Received unexpected event network-vif-plugged-50593025-edf1-44ec-b5f0-c92d6a807c23 for instance with vm_state active and task_state rebuild_spawning.
Nov 29 03:12:22 user nova-compute[75234]: DEBUG nova.compute.manager [req-b8e00106-ffcb-42fd-8b00-9cd58fc2d6d4 req-609d60f2-bc87-484b-8bca-c80f5444dd93 service nova] [instance: a082c96e-e384-40b6-bcc8-fc5e0e104669] Received event network-vif-plugged-0d131138-51b6-49f9-9a11-6b64e32e5796 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:12:22 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-b8e00106-ffcb-42fd-8b00-9cd58fc2d6d4 req-609d60f2-bc87-484b-8bca-c80f5444dd93 service nova] Acquiring lock "a082c96e-e384-40b6-bcc8-fc5e0e104669-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:12:22 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-b8e00106-ffcb-42fd-8b00-9cd58fc2d6d4 req-609d60f2-bc87-484b-8bca-c80f5444dd93 service nova] Lock "a082c96e-e384-40b6-bcc8-fc5e0e104669-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.003s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:12:22 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-b8e00106-ffcb-42fd-8b00-9cd58fc2d6d4 req-609d60f2-bc87-484b-8bca-c80f5444dd93 service nova] Lock "a082c96e-e384-40b6-bcc8-fc5e0e104669-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:12:22 user nova-compute[75234]: DEBUG nova.compute.manager [req-b8e00106-ffcb-42fd-8b00-9cd58fc2d6d4 req-609d60f2-bc87-484b-8bca-c80f5444dd93 service nova] [instance: a082c96e-e384-40b6-bcc8-fc5e0e104669] No waiting events found dispatching network-vif-plugged-0d131138-51b6-49f9-9a11-6b64e32e5796 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:12:22 user nova-compute[75234]: WARNING nova.compute.manager [req-b8e00106-ffcb-42fd-8b00-9cd58fc2d6d4 req-609d60f2-bc87-484b-8bca-c80f5444dd93 service nova] [instance: a082c96e-e384-40b6-bcc8-fc5e0e104669] Received unexpected event network-vif-plugged-0d131138-51b6-49f9-9a11-6b64e32e5796 for instance with vm_state active and task_state rescuing.
Nov 29 03:12:22 user nova-compute[75234]: DEBUG nova.compute.manager [None req-0d805482-05bf-44f8-bb11-e441afd66aac tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] [instance: a9350a18-14e6-42c6-a150-0a1487f20ad9] Instance event wait completed in 0 seconds for  {{(pid=75234) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Nov 29 03:12:22 user nova-compute[75234]: INFO nova.virt.libvirt.driver [-] [instance: a9350a18-14e6-42c6-a150-0a1487f20ad9] Instance running successfully.
Nov 29 03:12:22 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-0d805482-05bf-44f8-bb11-e441afd66aac tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] [instance: a9350a18-14e6-42c6-a150-0a1487f20ad9] finish_revert_migration finished successfully. {{(pid=75234) finish_revert_migration /opt/stack/nova/nova/virt/libvirt/driver.py:11860}}
Nov 29 03:12:22 user nova-compute[75234]: INFO nova.compute.manager [None req-0d805482-05bf-44f8-bb11-e441afd66aac tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] [instance: a9350a18-14e6-42c6-a150-0a1487f20ad9] Updating instance to original state: 'active'
Nov 29 03:12:23 user nova-compute[75234]: INFO nova.virt.libvirt.driver [-] [instance: f609a7b7-1be2-44b4-9177-7e72a4d4c060] Instance destroyed successfully.
Nov 29 03:12:23 user nova-compute[75234]: DEBUG nova.compute.manager [None req-2c12d1f3-c2dc-4d62-b1dd-f4fb3caea12f tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] [instance: f609a7b7-1be2-44b4-9177-7e72a4d4c060] Checking state {{(pid=75234) _get_power_state /opt/stack/nova/nova/compute/manager.py:1782}}
Nov 29 03:12:23 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-2c12d1f3-c2dc-4d62-b1dd-f4fb3caea12f tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Lock "f609a7b7-1be2-44b4-9177-7e72a4d4c060" "released" by "nova.compute.manager.ComputeManager.stop_instance.<locals>.do_stop_instance" :: held 1.494s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:12:23 user nova-compute[75234]: DEBUG nova.compute.manager [None req-4a795bce-c0be-4707-baff-68c516a2f74c tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] [instance: 6c0e0184-ca3a-48ec-a6e3-a0a1172260d6] Instance event wait completed in 0 seconds for  {{(pid=75234) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Nov 29 03:12:23 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-4a795bce-c0be-4707-baff-68c516a2f74c tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] [instance: 6c0e0184-ca3a-48ec-a6e3-a0a1172260d6] Guest created on hypervisor {{(pid=75234) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4421}}
Nov 29 03:12:23 user nova-compute[75234]: INFO nova.virt.libvirt.driver [-] [instance: 6c0e0184-ca3a-48ec-a6e3-a0a1172260d6] Instance spawned successfully.
Nov 29 03:12:23 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-4a795bce-c0be-4707-baff-68c516a2f74c tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] [instance: 6c0e0184-ca3a-48ec-a6e3-a0a1172260d6] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:904}}
Nov 29 03:12:23 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-4a795bce-c0be-4707-baff-68c516a2f74c tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] [instance: 6c0e0184-ca3a-48ec-a6e3-a0a1172260d6] Found default for hw_cdrom_bus of ide {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:12:23 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-4a795bce-c0be-4707-baff-68c516a2f74c tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] [instance: 6c0e0184-ca3a-48ec-a6e3-a0a1172260d6] Found default for hw_disk_bus of virtio {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:12:23 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-4a795bce-c0be-4707-baff-68c516a2f74c tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] [instance: 6c0e0184-ca3a-48ec-a6e3-a0a1172260d6] Found default for hw_input_bus of None {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:12:23 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-4a795bce-c0be-4707-baff-68c516a2f74c tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] [instance: 6c0e0184-ca3a-48ec-a6e3-a0a1172260d6] Found default for hw_pointer_model of None {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:12:23 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-4a795bce-c0be-4707-baff-68c516a2f74c tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] [instance: 6c0e0184-ca3a-48ec-a6e3-a0a1172260d6] Found default for hw_video_model of virtio {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:12:23 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-4a795bce-c0be-4707-baff-68c516a2f74c tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] [instance: 6c0e0184-ca3a-48ec-a6e3-a0a1172260d6] Found default for hw_vif_model of virtio {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:12:23 user nova-compute[75234]: DEBUG nova.compute.manager [None req-4a795bce-c0be-4707-baff-68c516a2f74c tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] [instance: 6c0e0184-ca3a-48ec-a6e3-a0a1172260d6] Checking state {{(pid=75234) _get_power_state /opt/stack/nova/nova/compute/manager.py:1782}}
Nov 29 03:12:23 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-4a795bce-c0be-4707-baff-68c516a2f74c tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.finish_evacuation" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:12:23 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-4a795bce-c0be-4707-baff-68c516a2f74c tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.finish_evacuation" :: waited 0.013s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:12:23 user nova-compute[75234]: DEBUG nova.objects.instance [None req-4a795bce-c0be-4707-baff-68c516a2f74c tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] [instance: 6c0e0184-ca3a-48ec-a6e3-a0a1172260d6] Trying to apply a migration context that does not seem to be set for this instance {{(pid=75234) apply_migration_context /opt/stack/nova/nova/objects/instance.py:1067}}
Nov 29 03:12:24 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-4a795bce-c0be-4707-baff-68c516a2f74c tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.finish_evacuation" :: held 0.380s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:12:24 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:12:24 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:12:24 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:12:24 user nova-compute[75234]: DEBUG nova.compute.manager [req-c9c60ae1-eedc-4672-9c77-9265ce34119f req-e04c7449-c1f2-45b7-90c1-968a8ecff547 service nova] [instance: f609a7b7-1be2-44b4-9177-7e72a4d4c060] Received event network-vif-plugged-5543f379-2b3a-4dc2-bfe9-52f66a942257 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:12:24 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-c9c60ae1-eedc-4672-9c77-9265ce34119f req-e04c7449-c1f2-45b7-90c1-968a8ecff547 service nova] Acquiring lock "f609a7b7-1be2-44b4-9177-7e72a4d4c060-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:12:24 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-c9c60ae1-eedc-4672-9c77-9265ce34119f req-e04c7449-c1f2-45b7-90c1-968a8ecff547 service nova] Lock "f609a7b7-1be2-44b4-9177-7e72a4d4c060-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:12:24 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-c9c60ae1-eedc-4672-9c77-9265ce34119f req-e04c7449-c1f2-45b7-90c1-968a8ecff547 service nova] Lock "f609a7b7-1be2-44b4-9177-7e72a4d4c060-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:12:24 user nova-compute[75234]: DEBUG nova.compute.manager [req-c9c60ae1-eedc-4672-9c77-9265ce34119f req-e04c7449-c1f2-45b7-90c1-968a8ecff547 service nova] [instance: f609a7b7-1be2-44b4-9177-7e72a4d4c060] No waiting events found dispatching network-vif-plugged-5543f379-2b3a-4dc2-bfe9-52f66a942257 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:12:24 user nova-compute[75234]: WARNING nova.compute.manager [req-c9c60ae1-eedc-4672-9c77-9265ce34119f req-e04c7449-c1f2-45b7-90c1-968a8ecff547 service nova] [instance: f609a7b7-1be2-44b4-9177-7e72a4d4c060] Received unexpected event network-vif-plugged-5543f379-2b3a-4dc2-bfe9-52f66a942257 for instance with vm_state stopped and task_state None.
Nov 29 03:12:25 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:12:25 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:12:25 user nova-compute[75234]: DEBUG nova.compute.manager [req-a96641d6-10f3-4d57-8205-bd75e7c2b202 req-06e7b2c6-a4d1-4c56-a2d3-b0657ed548ff service nova] [instance: a082c96e-e384-40b6-bcc8-fc5e0e104669] Received event network-vif-plugged-0d131138-51b6-49f9-9a11-6b64e32e5796 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:12:25 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-a96641d6-10f3-4d57-8205-bd75e7c2b202 req-06e7b2c6-a4d1-4c56-a2d3-b0657ed548ff service nova] Acquiring lock "a082c96e-e384-40b6-bcc8-fc5e0e104669-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:12:25 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-a96641d6-10f3-4d57-8205-bd75e7c2b202 req-06e7b2c6-a4d1-4c56-a2d3-b0657ed548ff service nova] Lock "a082c96e-e384-40b6-bcc8-fc5e0e104669-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:12:25 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-a96641d6-10f3-4d57-8205-bd75e7c2b202 req-06e7b2c6-a4d1-4c56-a2d3-b0657ed548ff service nova] Lock "a082c96e-e384-40b6-bcc8-fc5e0e104669-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:12:25 user nova-compute[75234]: DEBUG nova.compute.manager [req-a96641d6-10f3-4d57-8205-bd75e7c2b202 req-06e7b2c6-a4d1-4c56-a2d3-b0657ed548ff service nova] [instance: a082c96e-e384-40b6-bcc8-fc5e0e104669] No waiting events found dispatching network-vif-plugged-0d131138-51b6-49f9-9a11-6b64e32e5796 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:12:25 user nova-compute[75234]: WARNING nova.compute.manager [req-a96641d6-10f3-4d57-8205-bd75e7c2b202 req-06e7b2c6-a4d1-4c56-a2d3-b0657ed548ff service nova] [instance: a082c96e-e384-40b6-bcc8-fc5e0e104669] Received unexpected event network-vif-plugged-0d131138-51b6-49f9-9a11-6b64e32e5796 for instance with vm_state active and task_state rescuing.
Nov 29 03:12:26 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:12:27 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:12:27 user nova-compute[75234]: DEBUG nova.compute.manager [req-cf022320-5e6f-4972-8861-265572acbf94 req-8ed2391f-67d2-47fe-9f01-1c4cf411ee7a service nova] [instance: a082c96e-e384-40b6-bcc8-fc5e0e104669] Received event network-vif-plugged-0d131138-51b6-49f9-9a11-6b64e32e5796 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:12:27 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-cf022320-5e6f-4972-8861-265572acbf94 req-8ed2391f-67d2-47fe-9f01-1c4cf411ee7a service nova] Acquiring lock "a082c96e-e384-40b6-bcc8-fc5e0e104669-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:12:27 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-cf022320-5e6f-4972-8861-265572acbf94 req-8ed2391f-67d2-47fe-9f01-1c4cf411ee7a service nova] Lock "a082c96e-e384-40b6-bcc8-fc5e0e104669-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:12:27 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-cf022320-5e6f-4972-8861-265572acbf94 req-8ed2391f-67d2-47fe-9f01-1c4cf411ee7a service nova] Lock "a082c96e-e384-40b6-bcc8-fc5e0e104669-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:12:27 user nova-compute[75234]: DEBUG nova.compute.manager [req-cf022320-5e6f-4972-8861-265572acbf94 req-8ed2391f-67d2-47fe-9f01-1c4cf411ee7a service nova] [instance: a082c96e-e384-40b6-bcc8-fc5e0e104669] No waiting events found dispatching network-vif-plugged-0d131138-51b6-49f9-9a11-6b64e32e5796 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:12:27 user nova-compute[75234]: WARNING nova.compute.manager [req-cf022320-5e6f-4972-8861-265572acbf94 req-8ed2391f-67d2-47fe-9f01-1c4cf411ee7a service nova] [instance: a082c96e-e384-40b6-bcc8-fc5e0e104669] Received unexpected event network-vif-plugged-0d131138-51b6-49f9-9a11-6b64e32e5796 for instance with vm_state active and task_state rescuing.
Nov 29 03:12:27 user nova-compute[75234]: DEBUG nova.compute.manager [None req-23cb5660-78de-4fac-bd94-51e25fe6824e tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] [instance: a082c96e-e384-40b6-bcc8-fc5e0e104669] Checking state {{(pid=75234) _get_power_state /opt/stack/nova/nova/compute/manager.py:1782}}
Nov 29 03:12:28 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:12:29 user nova-compute[75234]: ERROR nova.volume.cinder [None req-eee1a7b9-fa96-457b-831f-9ccbd0ccc6aa tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] Update attachment failed for attachment 26455adb-3b86-492d-9c31-4c0f84721fc8. Error: Unable to update attachment.(Bad or unexpected response from the storage volume backend API: Create export for volume failed (Volume driver reported an error: None).). (HTTP 500) (Request-ID: req-7944a0c4-c913-4f1e-9dd3-2c83d6b7b043) Code: 500: cinderclient.exceptions.ClientException: Unable to update attachment.(Bad or unexpected response from the storage volume backend API: Create export for volume failed (Volume driver reported an error: None).). (HTTP 500) (Request-ID: req-7944a0c4-c913-4f1e-9dd3-2c83d6b7b043)
Nov 29 03:12:29 user nova-compute[75234]: ERROR nova.compute.manager [None req-eee1a7b9-fa96-457b-831f-9ccbd0ccc6aa tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] [instance: d3b90f39-4b91-4152-b820-bd0e263eec89] Failed to attach 634753d3-60a5-4f55-9485-14b6b488a121 at /dev/vdb: cinderclient.exceptions.ClientException: Unable to update attachment.(Bad or unexpected response from the storage volume backend API: Create export for volume failed (Volume driver reported an error: None).). (HTTP 500) (Request-ID: req-7944a0c4-c913-4f1e-9dd3-2c83d6b7b043)
Nov 29 03:12:29 user nova-compute[75234]: ERROR nova.compute.manager [instance: d3b90f39-4b91-4152-b820-bd0e263eec89] Traceback (most recent call last):
Nov 29 03:12:29 user nova-compute[75234]: ERROR nova.compute.manager [instance: d3b90f39-4b91-4152-b820-bd0e263eec89]   File "/opt/stack/nova/nova/compute/manager.py", line 7567, in _attach_volume
Nov 29 03:12:29 user nova-compute[75234]: ERROR nova.compute.manager [instance: d3b90f39-4b91-4152-b820-bd0e263eec89]     bdm.attach(context, instance, self.volume_api, self.driver,
Nov 29 03:12:29 user nova-compute[75234]: ERROR nova.compute.manager [instance: d3b90f39-4b91-4152-b820-bd0e263eec89]   File "/opt/stack/nova/nova/virt/block_device.py", line 46, in wrapped
Nov 29 03:12:29 user nova-compute[75234]: ERROR nova.compute.manager [instance: d3b90f39-4b91-4152-b820-bd0e263eec89]     ret_val = method(obj, context, *args, **kwargs)
Nov 29 03:12:29 user nova-compute[75234]: ERROR nova.compute.manager [instance: d3b90f39-4b91-4152-b820-bd0e263eec89]   File "/opt/stack/nova/nova/virt/block_device.py", line 734, in attach
Nov 29 03:12:29 user nova-compute[75234]: ERROR nova.compute.manager [instance: d3b90f39-4b91-4152-b820-bd0e263eec89]     self._do_attach(context, instance, volume, volume_api,
Nov 29 03:12:29 user nova-compute[75234]: ERROR nova.compute.manager [instance: d3b90f39-4b91-4152-b820-bd0e263eec89]   File "/opt/stack/nova/nova/virt/block_device.py", line 719, in _do_attach
Nov 29 03:12:29 user nova-compute[75234]: ERROR nova.compute.manager [instance: d3b90f39-4b91-4152-b820-bd0e263eec89]     self._volume_attach(context, volume, connector, instance,
Nov 29 03:12:29 user nova-compute[75234]: ERROR nova.compute.manager [instance: d3b90f39-4b91-4152-b820-bd0e263eec89]   File "/opt/stack/nova/nova/virt/block_device.py", line 633, in _volume_attach
Nov 29 03:12:29 user nova-compute[75234]: ERROR nova.compute.manager [instance: d3b90f39-4b91-4152-b820-bd0e263eec89]     connection_info = volume_api.attachment_update(
Nov 29 03:12:29 user nova-compute[75234]: ERROR nova.compute.manager [instance: d3b90f39-4b91-4152-b820-bd0e263eec89]   File "/opt/stack/nova/nova/volume/cinder.py", line 397, in wrapper
Nov 29 03:12:29 user nova-compute[75234]: ERROR nova.compute.manager [instance: d3b90f39-4b91-4152-b820-bd0e263eec89]     res = method(self, ctx, *args, **kwargs)
Nov 29 03:12:29 user nova-compute[75234]: ERROR nova.compute.manager [instance: d3b90f39-4b91-4152-b820-bd0e263eec89]   File "/opt/stack/nova/nova/volume/cinder.py", line 451, in wrapper
Nov 29 03:12:29 user nova-compute[75234]: ERROR nova.compute.manager [instance: d3b90f39-4b91-4152-b820-bd0e263eec89]     res = method(self, ctx, attachment_id, *args, **kwargs)
Nov 29 03:12:29 user nova-compute[75234]: ERROR nova.compute.manager [instance: d3b90f39-4b91-4152-b820-bd0e263eec89]   File "/opt/stack/nova/nova/volume/cinder.py", line 915, in attachment_update
Nov 29 03:12:29 user nova-compute[75234]: ERROR nova.compute.manager [instance: d3b90f39-4b91-4152-b820-bd0e263eec89]     with excutils.save_and_reraise_exception():
Nov 29 03:12:29 user nova-compute[75234]: ERROR nova.compute.manager [instance: d3b90f39-4b91-4152-b820-bd0e263eec89]   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__
Nov 29 03:12:29 user nova-compute[75234]: ERROR nova.compute.manager [instance: d3b90f39-4b91-4152-b820-bd0e263eec89]     self.force_reraise()
Nov 29 03:12:29 user nova-compute[75234]: ERROR nova.compute.manager [instance: d3b90f39-4b91-4152-b820-bd0e263eec89]   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise
Nov 29 03:12:29 user nova-compute[75234]: ERROR nova.compute.manager [instance: d3b90f39-4b91-4152-b820-bd0e263eec89]     raise self.value
Nov 29 03:12:29 user nova-compute[75234]: ERROR nova.compute.manager [instance: d3b90f39-4b91-4152-b820-bd0e263eec89]   File "/opt/stack/nova/nova/volume/cinder.py", line 909, in attachment_update
Nov 29 03:12:29 user nova-compute[75234]: ERROR nova.compute.manager [instance: d3b90f39-4b91-4152-b820-bd0e263eec89]     context, '3.44', skip_version_check=True).attachments.update(
Nov 29 03:12:29 user nova-compute[75234]: ERROR nova.compute.manager [instance: d3b90f39-4b91-4152-b820-bd0e263eec89]   File "/opt/stack/data/venv/lib/python3.10/site-packages/cinderclient/api_versions.py", line 421, in substitution
Nov 29 03:12:29 user nova-compute[75234]: ERROR nova.compute.manager [instance: d3b90f39-4b91-4152-b820-bd0e263eec89]     return method.func(obj, *args, **kwargs)
Nov 29 03:12:29 user nova-compute[75234]: ERROR nova.compute.manager [instance: d3b90f39-4b91-4152-b820-bd0e263eec89]   File "/opt/stack/data/venv/lib/python3.10/site-packages/cinderclient/v3/attachments.py", line 75, in update
Nov 29 03:12:29 user nova-compute[75234]: ERROR nova.compute.manager [instance: d3b90f39-4b91-4152-b820-bd0e263eec89]     resp = self._update('/attachments/%s' % id, body)
Nov 29 03:12:29 user nova-compute[75234]: ERROR nova.compute.manager [instance: d3b90f39-4b91-4152-b820-bd0e263eec89]   File "/opt/stack/data/venv/lib/python3.10/site-packages/cinderclient/base.py", line 318, in _update
Nov 29 03:12:29 user nova-compute[75234]: ERROR nova.compute.manager [instance: d3b90f39-4b91-4152-b820-bd0e263eec89]     resp, body = self.api.client.put(url, body=body, **kwargs)
Nov 29 03:12:29 user nova-compute[75234]: ERROR nova.compute.manager [instance: d3b90f39-4b91-4152-b820-bd0e263eec89]   File "/opt/stack/data/venv/lib/python3.10/site-packages/cinderclient/client.py", line 226, in put
Nov 29 03:12:29 user nova-compute[75234]: ERROR nova.compute.manager [instance: d3b90f39-4b91-4152-b820-bd0e263eec89]     return self._cs_request(url, 'PUT', **kwargs)
Nov 29 03:12:29 user nova-compute[75234]: ERROR nova.compute.manager [instance: d3b90f39-4b91-4152-b820-bd0e263eec89]   File "/opt/stack/data/venv/lib/python3.10/site-packages/cinderclient/client.py", line 211, in _cs_request
Nov 29 03:12:29 user nova-compute[75234]: ERROR nova.compute.manager [instance: d3b90f39-4b91-4152-b820-bd0e263eec89]     return self.request(url, method, **kwargs)
Nov 29 03:12:29 user nova-compute[75234]: ERROR nova.compute.manager [instance: d3b90f39-4b91-4152-b820-bd0e263eec89]   File "/opt/stack/data/venv/lib/python3.10/site-packages/cinderclient/client.py", line 197, in request
Nov 29 03:12:29 user nova-compute[75234]: ERROR nova.compute.manager [instance: d3b90f39-4b91-4152-b820-bd0e263eec89]     raise exceptions.from_response(resp, body)
Nov 29 03:12:29 user nova-compute[75234]: ERROR nova.compute.manager [instance: d3b90f39-4b91-4152-b820-bd0e263eec89] cinderclient.exceptions.ClientException: Unable to update attachment.(Bad or unexpected response from the storage volume backend API: Create export for volume failed (Volume driver reported an error: None).). (HTTP 500) (Request-ID: req-7944a0c4-c913-4f1e-9dd3-2c83d6b7b043)
Nov 29 03:12:29 user nova-compute[75234]: ERROR nova.compute.manager [instance: d3b90f39-4b91-4152-b820-bd0e263eec89] 
Nov 29 03:12:30 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-eee1a7b9-fa96-457b-831f-9ccbd0ccc6aa tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] Lock "d3b90f39-4b91-4152-b820-bd0e263eec89" "released" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: held 8.207s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:12:30 user nova-compute[75234]: ERROR oslo_messaging.rpc.server [None req-eee1a7b9-fa96-457b-831f-9ccbd0ccc6aa tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] Exception during message handling: cinderclient.exceptions.ClientException: Unable to update attachment.(Bad or unexpected response from the storage volume backend API: Create export for volume failed (Volume driver reported an error: None).). (HTTP 500) (Request-ID: req-7944a0c4-c913-4f1e-9dd3-2c83d6b7b043)
Nov 29 03:12:30 user nova-compute[75234]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
Nov 29 03:12:30 user nova-compute[75234]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
Nov 29 03:12:30 user nova-compute[75234]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
Nov 29 03:12:30 user nova-compute[75234]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
Nov 29 03:12:30 user nova-compute[75234]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
Nov 29 03:12:30 user nova-compute[75234]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
Nov 29 03:12:30 user nova-compute[75234]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
Nov 29 03:12:30 user nova-compute[75234]: ERROR oslo_messaging.rpc.server   File "/opt/stack/nova/nova/exception_wrapper.py", line 65, in wrapped
Nov 29 03:12:30 user nova-compute[75234]: ERROR oslo_messaging.rpc.server     with excutils.save_and_reraise_exception():
Nov 29 03:12:30 user nova-compute[75234]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__
Nov 29 03:12:30 user nova-compute[75234]: ERROR oslo_messaging.rpc.server     self.force_reraise()
Nov 29 03:12:30 user nova-compute[75234]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise
Nov 29 03:12:30 user nova-compute[75234]: ERROR oslo_messaging.rpc.server     raise self.value
Nov 29 03:12:30 user nova-compute[75234]: ERROR oslo_messaging.rpc.server   File "/opt/stack/nova/nova/exception_wrapper.py", line 63, in wrapped
Nov 29 03:12:30 user nova-compute[75234]: ERROR oslo_messaging.rpc.server     return f(self, context, *args, **kw)
Nov 29 03:12:30 user nova-compute[75234]: ERROR oslo_messaging.rpc.server   File "/opt/stack/nova/nova/compute/utils.py", line 1439, in decorated_function
Nov 29 03:12:30 user nova-compute[75234]: ERROR oslo_messaging.rpc.server     return function(self, context, *args, **kwargs)
Nov 29 03:12:30 user nova-compute[75234]: ERROR oslo_messaging.rpc.server   File "/opt/stack/nova/nova/compute/manager.py", line 213, in decorated_function
Nov 29 03:12:30 user nova-compute[75234]: ERROR oslo_messaging.rpc.server     with excutils.save_and_reraise_exception():
Nov 29 03:12:30 user nova-compute[75234]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__
Nov 29 03:12:30 user nova-compute[75234]: ERROR oslo_messaging.rpc.server     self.force_reraise()
Nov 29 03:12:30 user nova-compute[75234]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise
Nov 29 03:12:30 user nova-compute[75234]: ERROR oslo_messaging.rpc.server     raise self.value
Nov 29 03:12:30 user nova-compute[75234]: ERROR oslo_messaging.rpc.server   File "/opt/stack/nova/nova/compute/manager.py", line 203, in decorated_function
Nov 29 03:12:30 user nova-compute[75234]: ERROR oslo_messaging.rpc.server     return function(self, context, *args, **kwargs)
Nov 29 03:12:30 user nova-compute[75234]: ERROR oslo_messaging.rpc.server   File "/opt/stack/nova/nova/compute/manager.py", line 7553, in attach_volume
Nov 29 03:12:30 user nova-compute[75234]: ERROR oslo_messaging.rpc.server     do_attach_volume(context, instance, driver_bdm)
Nov 29 03:12:30 user nova-compute[75234]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py", line 414, in inner
Nov 29 03:12:30 user nova-compute[75234]: ERROR oslo_messaging.rpc.server     return f(*args, **kwargs)
Nov 29 03:12:30 user nova-compute[75234]: ERROR oslo_messaging.rpc.server   File "/opt/stack/nova/nova/compute/manager.py", line 7550, in do_attach_volume
Nov 29 03:12:30 user nova-compute[75234]: ERROR oslo_messaging.rpc.server     with excutils.save_and_reraise_exception():
Nov 29 03:12:30 user nova-compute[75234]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__
Nov 29 03:12:30 user nova-compute[75234]: ERROR oslo_messaging.rpc.server     self.force_reraise()
Nov 29 03:12:30 user nova-compute[75234]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise
Nov 29 03:12:30 user nova-compute[75234]: ERROR oslo_messaging.rpc.server     raise self.value
Nov 29 03:12:30 user nova-compute[75234]: ERROR oslo_messaging.rpc.server   File "/opt/stack/nova/nova/compute/manager.py", line 7548, in do_attach_volume
Nov 29 03:12:30 user nova-compute[75234]: ERROR oslo_messaging.rpc.server     return self._attach_volume(context, instance, driver_bdm)
Nov 29 03:12:30 user nova-compute[75234]: ERROR oslo_messaging.rpc.server   File "/opt/stack/nova/nova/compute/manager.py", line 7570, in _attach_volume
Nov 29 03:12:30 user nova-compute[75234]: ERROR oslo_messaging.rpc.server     with excutils.save_and_reraise_exception():
Nov 29 03:12:30 user nova-compute[75234]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__
Nov 29 03:12:30 user nova-compute[75234]: ERROR oslo_messaging.rpc.server     self.force_reraise()
Nov 29 03:12:30 user nova-compute[75234]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise
Nov 29 03:12:30 user nova-compute[75234]: ERROR oslo_messaging.rpc.server     raise self.value
Nov 29 03:12:30 user nova-compute[75234]: ERROR oslo_messaging.rpc.server   File "/opt/stack/nova/nova/compute/manager.py", line 7567, in _attach_volume
Nov 29 03:12:30 user nova-compute[75234]: ERROR oslo_messaging.rpc.server     bdm.attach(context, instance, self.volume_api, self.driver,
Nov 29 03:12:30 user nova-compute[75234]: ERROR oslo_messaging.rpc.server   File "/opt/stack/nova/nova/virt/block_device.py", line 46, in wrapped
Nov 29 03:12:30 user nova-compute[75234]: ERROR oslo_messaging.rpc.server     ret_val = method(obj, context, *args, **kwargs)
Nov 29 03:12:30 user nova-compute[75234]: ERROR oslo_messaging.rpc.server   File "/opt/stack/nova/nova/virt/block_device.py", line 734, in attach
Nov 29 03:12:30 user nova-compute[75234]: ERROR oslo_messaging.rpc.server     self._do_attach(context, instance, volume, volume_api,
Nov 29 03:12:30 user nova-compute[75234]: ERROR oslo_messaging.rpc.server   File "/opt/stack/nova/nova/virt/block_device.py", line 719, in _do_attach
Nov 29 03:12:30 user nova-compute[75234]: ERROR oslo_messaging.rpc.server     self._volume_attach(context, volume, connector, instance,
Nov 29 03:12:30 user nova-compute[75234]: ERROR oslo_messaging.rpc.server   File "/opt/stack/nova/nova/virt/block_device.py", line 633, in _volume_attach
Nov 29 03:12:30 user nova-compute[75234]: ERROR oslo_messaging.rpc.server     connection_info = volume_api.attachment_update(
Nov 29 03:12:30 user nova-compute[75234]: ERROR oslo_messaging.rpc.server   File "/opt/stack/nova/nova/volume/cinder.py", line 397, in wrapper
Nov 29 03:12:30 user nova-compute[75234]: ERROR oslo_messaging.rpc.server     res = method(self, ctx, *args, **kwargs)
Nov 29 03:12:30 user nova-compute[75234]: ERROR oslo_messaging.rpc.server   File "/opt/stack/nova/nova/volume/cinder.py", line 451, in wrapper
Nov 29 03:12:30 user nova-compute[75234]: ERROR oslo_messaging.rpc.server     res = method(self, ctx, attachment_id, *args, **kwargs)
Nov 29 03:12:30 user nova-compute[75234]: ERROR oslo_messaging.rpc.server   File "/opt/stack/nova/nova/volume/cinder.py", line 915, in attachment_update
Nov 29 03:12:30 user nova-compute[75234]: ERROR oslo_messaging.rpc.server     with excutils.save_and_reraise_exception():
Nov 29 03:12:30 user nova-compute[75234]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 227, in __exit__
Nov 29 03:12:30 user nova-compute[75234]: ERROR oslo_messaging.rpc.server     self.force_reraise()
Nov 29 03:12:30 user nova-compute[75234]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_utils/excutils.py", line 200, in force_reraise
Nov 29 03:12:30 user nova-compute[75234]: ERROR oslo_messaging.rpc.server     raise self.value
Nov 29 03:12:30 user nova-compute[75234]: ERROR oslo_messaging.rpc.server   File "/opt/stack/nova/nova/volume/cinder.py", line 909, in attachment_update
Nov 29 03:12:30 user nova-compute[75234]: ERROR oslo_messaging.rpc.server     context, '3.44', skip_version_check=True).attachments.update(
Nov 29 03:12:30 user nova-compute[75234]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/cinderclient/api_versions.py", line 421, in substitution
Nov 29 03:12:30 user nova-compute[75234]: ERROR oslo_messaging.rpc.server     return method.func(obj, *args, **kwargs)
Nov 29 03:12:30 user nova-compute[75234]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/cinderclient/v3/attachments.py", line 75, in update
Nov 29 03:12:30 user nova-compute[75234]: ERROR oslo_messaging.rpc.server     resp = self._update('/attachments/%s' % id, body)
Nov 29 03:12:30 user nova-compute[75234]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/cinderclient/base.py", line 318, in _update
Nov 29 03:12:30 user nova-compute[75234]: ERROR oslo_messaging.rpc.server     resp, body = self.api.client.put(url, body=body, **kwargs)
Nov 29 03:12:30 user nova-compute[75234]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/cinderclient/client.py", line 226, in put
Nov 29 03:12:30 user nova-compute[75234]: ERROR oslo_messaging.rpc.server     return self._cs_request(url, 'PUT', **kwargs)
Nov 29 03:12:30 user nova-compute[75234]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/cinderclient/client.py", line 211, in _cs_request
Nov 29 03:12:30 user nova-compute[75234]: ERROR oslo_messaging.rpc.server     return self.request(url, method, **kwargs)
Nov 29 03:12:30 user nova-compute[75234]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/cinderclient/client.py", line 197, in request
Nov 29 03:12:30 user nova-compute[75234]: ERROR oslo_messaging.rpc.server     raise exceptions.from_response(resp, body)
Nov 29 03:12:30 user nova-compute[75234]: ERROR oslo_messaging.rpc.server cinderclient.exceptions.ClientException: Unable to update attachment.(Bad or unexpected response from the storage volume backend API: Create export for volume failed (Volume driver reported an error: None).). (HTTP 500) (Request-ID: req-7944a0c4-c913-4f1e-9dd3-2c83d6b7b043)
Nov 29 03:12:30 user nova-compute[75234]: ERROR oslo_messaging.rpc.server 
Nov 29 03:12:31 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-738e17a6-50e7-42de-8ada-0b2118a0bc5d tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] Acquiring lock "8297c264-a213-44ee-be46-5c08eabda9f7" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:12:31 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-738e17a6-50e7-42de-8ada-0b2118a0bc5d tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] Lock "8297c264-a213-44ee-be46-5c08eabda9f7" acquired by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:12:31 user nova-compute[75234]: INFO nova.compute.manager [None req-738e17a6-50e7-42de-8ada-0b2118a0bc5d tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] [instance: 8297c264-a213-44ee-be46-5c08eabda9f7] Detaching volume a7ef35bf-017a-4fd9-9182-79c1b2ab4f98
Nov 29 03:12:31 user nova-compute[75234]: INFO nova.virt.block_device [None req-738e17a6-50e7-42de-8ada-0b2118a0bc5d tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] [instance: 8297c264-a213-44ee-be46-5c08eabda9f7] Attempting to driver detach volume a7ef35bf-017a-4fd9-9182-79c1b2ab4f98 from mountpoint /dev/vdb
Nov 29 03:12:31 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-738e17a6-50e7-42de-8ada-0b2118a0bc5d tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] Attempting to detach device vdb from instance 8297c264-a213-44ee-be46-5c08eabda9f7 from the persistent domain config. {{(pid=75234) _detach_from_persistent /opt/stack/nova/nova/virt/libvirt/driver.py:2487}}
Nov 29 03:12:31 user nova-compute[75234]: DEBUG nova.virt.libvirt.guest [None req-738e17a6-50e7-42de-8ada-0b2118a0bc5d tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] detach device xml: <disk type="block" device="disk">
Nov 29 03:12:31 user nova-compute[75234]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Nov 29 03:12:31 user nova-compute[75234]:   <source dev="/dev/sdd"/>
Nov 29 03:12:31 user nova-compute[75234]:   <target dev="vdb" bus="virtio"/>
Nov 29 03:12:31 user nova-compute[75234]:   <serial>a7ef35bf-017a-4fd9-9182-79c1b2ab4f98</serial>
Nov 29 03:12:31 user nova-compute[75234]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x07" function="0x0"/>
Nov 29 03:12:31 user nova-compute[75234]: </disk>
Nov 29 03:12:31 user nova-compute[75234]:  {{(pid=75234) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Nov 29 03:12:31 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-738e17a6-50e7-42de-8ada-0b2118a0bc5d tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] Successfully detached device vdb from instance 8297c264-a213-44ee-be46-5c08eabda9f7 from the persistent domain config.
Nov 29 03:12:31 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-738e17a6-50e7-42de-8ada-0b2118a0bc5d tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] (1/8): Attempting to detach device vdb with device alias virtio-disk1 from instance 8297c264-a213-44ee-be46-5c08eabda9f7 from the live domain config. {{(pid=75234) _detach_from_live_with_retry /opt/stack/nova/nova/virt/libvirt/driver.py:2523}}
Nov 29 03:12:31 user nova-compute[75234]: DEBUG nova.virt.libvirt.guest [None req-738e17a6-50e7-42de-8ada-0b2118a0bc5d tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] detach device xml: <disk type="block" device="disk">
Nov 29 03:12:31 user nova-compute[75234]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Nov 29 03:12:31 user nova-compute[75234]:   <source dev="/dev/sdd"/>
Nov 29 03:12:31 user nova-compute[75234]:   <target dev="vdb" bus="virtio"/>
Nov 29 03:12:31 user nova-compute[75234]:   <serial>a7ef35bf-017a-4fd9-9182-79c1b2ab4f98</serial>
Nov 29 03:12:31 user nova-compute[75234]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x07" function="0x0"/>
Nov 29 03:12:31 user nova-compute[75234]: </disk>
Nov 29 03:12:31 user nova-compute[75234]:  {{(pid=75234) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Nov 29 03:12:31 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:12:32 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-738e17a6-50e7-42de-8ada-0b2118a0bc5d tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] Start waiting for the detach event from libvirt for device vdb with device alias virtio-disk1 for instance 8297c264-a213-44ee-be46-5c08eabda9f7 {{(pid=75234) _detach_from_live_and_wait_for_event /opt/stack/nova/nova/virt/libvirt/driver.py:2599}}
Nov 29 03:12:32 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:12:32 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:12:36 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:12:37 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:12:39 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:12:41 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:12:41 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:12:42 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:12:43 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:12:46 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:12:46 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:12:47 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:12:49 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:12:51 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:12:52 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-738e17a6-50e7-42de-8ada-0b2118a0bc5d tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] Waiting for libvirt event about the detach of device vdb with device alias virtio-disk1 from instance 8297c264-a213-44ee-be46-5c08eabda9f7 is timed out.
Nov 29 03:12:52 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-738e17a6-50e7-42de-8ada-0b2118a0bc5d tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] Successfully detached device vdb from instance 8297c264-a213-44ee-be46-5c08eabda9f7 from the live domain config.
Nov 29 03:12:52 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-738e17a6-50e7-42de-8ada-0b2118a0bc5d tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] [instance: 8297c264-a213-44ee-be46-5c08eabda9f7] calling os-brick to detach iSCSI Volume {{(pid=75234) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Nov 29 03:12:52 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-738e17a6-50e7-42de-8ada-0b2118a0bc5d tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f8a5050c9a0>, {'target_iqn': 'iqn.2010-10.org.openstack:a7ef35bf-017a-4fd9-9182-79c1b2ab4f98', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'iSOJqDFN', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdd'}, None), 'kwargs': False}}" {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:176}}
Nov 29 03:12:52 user nova-compute[75234]: WARNING os_brick.initiator.connectors.base [None req-738e17a6-50e7-42de-8ada-0b2118a0bc5d tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Nov 29 03:12:52 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-738e17a6-50e7-42de-8ada-0b2118a0bc5d tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:68}}
Nov 29 03:12:52 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-738e17a6-50e7-42de-8ada-0b2118a0bc5d tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:73}}
Nov 29 03:12:52 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:12:52 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.018s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:12:52 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[2b1716ef-8e62-40c0-beb6-c36545acf8ac]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:a7ef35bf-017a-4fd9-9182-79c1b2ab4f98\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:56237f4c-4138-4f50-aafb-9c0f9d4bb1cf\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:12:52 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-738e17a6-50e7-42de-8ada-0b2118a0bc5d tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS:
Nov 29 03:12:52 user nova-compute[75234]: No targets found.
Nov 29 03:12:52 user nova-compute[75234]: iSNS:
Nov 29 03:12:52 user nova-compute[75234]: No targets found.
Nov 29 03:12:52 user nova-compute[75234]: STATIC:
Nov 29 03:12:52 user nova-compute[75234]: Target: iqn.2010-10.org.openstack:a7ef35bf-017a-4fd9-9182-79c1b2ab4f98
Nov 29 03:12:52 user nova-compute[75234]:         Portal: 172.16.0.220:3260,-1
Nov 29 03:12:52 user nova-compute[75234]:                 Iface Name: default
Nov 29 03:12:52 user nova-compute[75234]: Target: iqn.2010-10.org.openstack:56237f4c-4138-4f50-aafb-9c0f9d4bb1cf
Nov 29 03:12:52 user nova-compute[75234]:         Portal: 172.16.0.220:3260,-1
Nov 29 03:12:52 user nova-compute[75234]:                 Iface Name: default
Nov 29 03:12:52 user nova-compute[75234]: Target: iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b
Nov 29 03:12:52 user nova-compute[75234]:         Portal: 172.16.0.220:3260,-1
Nov 29 03:12:52 user nova-compute[75234]:                 Iface Name: default
Nov 29 03:12:52 user nova-compute[75234]: FIRMWARE:
Nov 29 03:12:52 user nova-compute[75234]: No targets found.
Nov 29 03:12:52 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:12:52 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-738e17a6-50e7-42de-8ada-0b2118a0bc5d tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] Regex to get portals from discoverydb: ^SENDTARGETS:
Nov 29 03:12:52 user nova-compute[75234]: .*?^DiscoveryAddress: 172.16.0.220,3260.*?
Nov 29 03:12:52 user nova-compute[75234]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=75234) _get_discoverydb_portals /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:382}}
Nov 29 03:12:52 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-738e17a6-50e7-42de-8ada-0b2118a0bc5d tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:a7ef35bf-017a-4fd9-9182-79c1b2ab4f98', 0)] {{(pid=75234) _get_connection_devices /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:830}}
Nov 29 03:12:52 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:12:52 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:12:52 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.018s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:12:52 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[18255ff9-1e0d-4fe0-bedf-1b89634c05df]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:56237f4c-4138-4f50-aafb-9c0f9d4bb1cf\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:a7ef35bf-017a-4fd9-9182-79c1b2ab4f98\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:12:52 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:12:52 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.022s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:12:52 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[6a3302e7-96e5-4f80-94c2-228e182ae46d]: (4, ('tcp: [15] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a7ef35bf-017a-4fd9-9182-79c1b2ab4f98 (non-flash)\ntcp: [20] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b (non-flash)\ntcp: [21] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:56237f4c-4138-4f50-aafb-9c0f9d4bb1cf (non-flash)\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:12:52 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-738e17a6-50e7-42de-8ada-0b2118a0bc5d tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [15] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a7ef35bf-017a-4fd9-9182-79c1b2ab4f98 (non-flash)
Nov 29 03:12:52 user nova-compute[75234]: tcp: [20] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b (non-flash)
Nov 29 03:12:52 user nova-compute[75234]: tcp: [21] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:56237f4c-4138-4f50-aafb-9c0f9d4bb1cf (non-flash)
Nov 29 03:12:52 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:12:52 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-738e17a6-50e7-42de-8ada-0b2118a0bc5d tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] iscsi session list stdout=tcp: [15] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a7ef35bf-017a-4fd9-9182-79c1b2ab4f98 (non-flash)
Nov 29 03:12:52 user nova-compute[75234]: tcp: [20] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b (non-flash)
Nov 29 03:12:52 user nova-compute[75234]: tcp: [21] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:56237f4c-4138-4f50-aafb-9c0f9d4bb1cf (non-flash)
Nov 29 03:12:52 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsi_session /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Nov 29 03:12:52 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-738e17a6-50e7-42de-8ada-0b2118a0bc5d tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7f8a6025b010>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:a7ef35bf-017a-4fd9-9182-79c1b2ab4f98'): ({'sdd'}, set())}) {{(pid=75234) _get_connection_devices /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:863}}
Nov 29 03:12:52 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-738e17a6-50e7-42de-8ada-0b2118a0bc5d tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] Removing single pathed devices sdd {{(pid=75234) remove_connection /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:374}}
Nov 29 03:12:52 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:12:52 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.020s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:12:52 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[1779b1f6-45f0-4498-acc0-a090d3446285]: (4, ('path checker states:\nup                  3\n\npaths: 0\nbusy: False\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:12:52 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdd {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:12:52 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdd" returned: 0 in 0.019s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:12:52 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[a1541539-d616-422f-a118-1a07621147ea]: (4, ('ok\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:12:52 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-738e17a6-50e7-42de-8ada-0b2118a0bc5d tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] Flushing IO for device /dev/sdd {{(pid=75234) flush_device_io /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:434}}
Nov 29 03:12:52 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdd {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:12:52 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdd" returned: 0 in 0.013s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:12:52 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[aa601731-a1b3-47dc-84e8-2a645765fea9]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:12:52 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-738e17a6-50e7-42de-8ada-0b2118a0bc5d tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] Remove SCSI device /dev/sdd with /sys/block/sdd/device/delete {{(pid=75234) remove_scsi_device /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:148}}
Nov 29 03:12:52 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdd/device/delete {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:12:52 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdd/device/delete" returned: 0 in 0.049s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:12:52 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[6d039540-d189-41dc-865a-1771f74c2bbd]: (4, ('1', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:12:52 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-738e17a6-50e7-42de-8ada-0b2118a0bc5d tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] Checking to see if SCSI volumes sdd have been removed. {{(pid=75234) wait_for_volumes_removal /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:156}}
Nov 29 03:12:52 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-738e17a6-50e7-42de-8ada-0b2118a0bc5d tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] SCSI volumes sdd have been removed. {{(pid=75234) wait_for_volumes_removal /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:166}}
Nov 29 03:12:52 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-738e17a6-50e7-42de-8ada-0b2118a0bc5d tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:a7ef35bf-017a-4fd9-9182-79c1b2ab4f98')] {{(pid=75234) _disconnect_connection /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Nov 29 03:12:52 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:a7ef35bf-017a-4fd9-9182-79c1b2ab4f98 -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:12:52 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:a7ef35bf-017a-4fd9-9182-79c1b2ab4f98 -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.018s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:12:52 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[ac07d831-74c5-4682-98bc-988141001710]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:12:52 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-738e17a6-50e7-42de-8ada-0b2118a0bc5d tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:12:52 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:a7ef35bf-017a-4fd9-9182-79c1b2ab4f98 -p 172.16.0.220:3260 --logout {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:12:52 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:a7ef35bf-017a-4fd9-9182-79c1b2ab4f98 -p 172.16.0.220:3260 --logout" returned: 0 in 0.065s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:12:52 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[729e6d51-5abc-485c-a5e1-812663385377]: (4, ('Logging out of session [sid: 15, target: iqn.2010-10.org.openstack:a7ef35bf-017a-4fd9-9182-79c1b2ab4f98, portal: 172.16.0.220,3260]\nLogout of [sid: 15, target: iqn.2010-10.org.openstack:a7ef35bf-017a-4fd9-9182-79c1b2ab4f98, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:12:52 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-738e17a6-50e7-42de-8ada-0b2118a0bc5d tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 15, target: iqn.2010-10.org.openstack:a7ef35bf-017a-4fd9-9182-79c1b2ab4f98, portal: 172.16.0.220,3260]
Nov 29 03:12:52 user nova-compute[75234]: Logout of [sid: 15, target: iqn.2010-10.org.openstack:a7ef35bf-017a-4fd9-9182-79c1b2ab4f98, portal: 172.16.0.220,3260] successful.
Nov 29 03:12:52 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:12:52 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:a7ef35bf-017a-4fd9-9182-79c1b2ab4f98 -p 172.16.0.220:3260 --op delete {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:12:52 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:a7ef35bf-017a-4fd9-9182-79c1b2ab4f98 -p 172.16.0.220:3260 --op delete" returned: 0 in 0.026s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:12:52 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[25f08b88-ba2a-4698-962c-8a3e5d57e1cb]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:12:52 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-738e17a6-50e7-42de-8ada-0b2118a0bc5d tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:12:52 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-738e17a6-50e7-42de-8ada-0b2118a0bc5d tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.314s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:87}}
Nov 29 03:12:52 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-738e17a6-50e7-42de-8ada-0b2118a0bc5d tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] <== disconnect_volume: return (316ms) None {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:203}}
Nov 29 03:12:52 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-738e17a6-50e7-42de-8ada-0b2118a0bc5d tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] [instance: 8297c264-a213-44ee-be46-5c08eabda9f7] Disconnected iSCSI Volume {{(pid=75234) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:79}}
Nov 29 03:12:52 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:12:52 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:12:52 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Starting heal instance info cache {{(pid=75234) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9922}}
Nov 29 03:12:52 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Didn't find any instances for network info cache update. {{(pid=75234) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:10008}}
Nov 29 03:12:52 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:12:52 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=75234) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10541}}
Nov 29 03:12:53 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-62d5bf86-a9dc-438e-ab8b-2f7f9591cbde tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Acquiring lock "6894b55f-93c6-43c9-8933-56d5a3c43725" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:12:53 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-62d5bf86-a9dc-438e-ab8b-2f7f9591cbde tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Lock "6894b55f-93c6-43c9-8933-56d5a3c43725" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:12:53 user nova-compute[75234]: DEBUG nova.compute.manager [None req-62d5bf86-a9dc-438e-ab8b-2f7f9591cbde tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] [instance: 6894b55f-93c6-43c9-8933-56d5a3c43725] Starting instance... {{(pid=75234) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2422}}
Nov 29 03:12:53 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-62d5bf86-a9dc-438e-ab8b-2f7f9591cbde tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:12:53 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-62d5bf86-a9dc-438e-ab8b-2f7f9591cbde tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:12:53 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-62d5bf86-a9dc-438e-ab8b-2f7f9591cbde tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=75234) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2366}}
Nov 29 03:12:53 user nova-compute[75234]: INFO nova.compute.claims [None req-62d5bf86-a9dc-438e-ab8b-2f7f9591cbde tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] [instance: 6894b55f-93c6-43c9-8933-56d5a3c43725] Claim successful on node user
Nov 29 03:12:53 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:12:53 user nova-compute[75234]: DEBUG nova.compute.provider_tree [None req-62d5bf86-a9dc-438e-ab8b-2f7f9591cbde tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Inventory has not changed in ProviderTree for provider: 4d291220-7c37-4839-a0e3-b23c66502b74 {{(pid=75234) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Nov 29 03:12:53 user nova-compute[75234]: DEBUG nova.scheduler.client.report [None req-62d5bf86-a9dc-438e-ab8b-2f7f9591cbde tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Inventory has not changed for provider 4d291220-7c37-4839-a0e3-b23c66502b74 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=75234) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Nov 29 03:12:53 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-62d5bf86-a9dc-438e-ab8b-2f7f9591cbde tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.470s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:12:53 user nova-compute[75234]: DEBUG nova.compute.manager [None req-62d5bf86-a9dc-438e-ab8b-2f7f9591cbde tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] [instance: 6894b55f-93c6-43c9-8933-56d5a3c43725] Start building networks asynchronously for instance. {{(pid=75234) _build_resources /opt/stack/nova/nova/compute/manager.py:2819}}
Nov 29 03:12:53 user nova-compute[75234]: DEBUG nova.compute.manager [None req-62d5bf86-a9dc-438e-ab8b-2f7f9591cbde tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] [instance: 6894b55f-93c6-43c9-8933-56d5a3c43725] Allocating IP information in the background. {{(pid=75234) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1972}}
Nov 29 03:12:53 user nova-compute[75234]: DEBUG nova.network.neutron [None req-62d5bf86-a9dc-438e-ab8b-2f7f9591cbde tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] [instance: 6894b55f-93c6-43c9-8933-56d5a3c43725] allocate_for_instance() {{(pid=75234) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1156}}
Nov 29 03:12:53 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-62d5bf86-a9dc-438e-ab8b-2f7f9591cbde tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] [instance: 6894b55f-93c6-43c9-8933-56d5a3c43725] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Nov 29 03:12:53 user nova-compute[75234]: DEBUG nova.compute.manager [None req-62d5bf86-a9dc-438e-ab8b-2f7f9591cbde tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] [instance: 6894b55f-93c6-43c9-8933-56d5a3c43725] Start building block device mappings for instance. {{(pid=75234) _build_resources /opt/stack/nova/nova/compute/manager.py:2854}}
Nov 29 03:12:53 user nova-compute[75234]: DEBUG nova.policy [None req-62d5bf86-a9dc-438e-ab8b-2f7f9591cbde tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': 'dc1de1dad3794b47b0e4bf7445c96997', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c4393bc598a845a9bfb667e5332f14a6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=75234) authorize /opt/stack/nova/nova/policy.py:203}}
Nov 29 03:12:53 user nova-compute[75234]: DEBUG nova.compute.manager [None req-62d5bf86-a9dc-438e-ab8b-2f7f9591cbde tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] [instance: 6894b55f-93c6-43c9-8933-56d5a3c43725] Start spawning the instance on the hypervisor. {{(pid=75234) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2628}}
Nov 29 03:12:53 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-62d5bf86-a9dc-438e-ab8b-2f7f9591cbde tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] [instance: 6894b55f-93c6-43c9-8933-56d5a3c43725] Creating instance directory {{(pid=75234) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4727}}
Nov 29 03:12:53 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-62d5bf86-a9dc-438e-ab8b-2f7f9591cbde tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] [instance: 6894b55f-93c6-43c9-8933-56d5a3c43725] Creating image(s)
Nov 29 03:12:53 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-62d5bf86-a9dc-438e-ab8b-2f7f9591cbde tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Acquiring lock "/opt/stack/data/nova/instances/6894b55f-93c6-43c9-8933-56d5a3c43725/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:12:53 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-62d5bf86-a9dc-438e-ab8b-2f7f9591cbde tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Lock "/opt/stack/data/nova/instances/6894b55f-93c6-43c9-8933-56d5a3c43725/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:12:54 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-62d5bf86-a9dc-438e-ab8b-2f7f9591cbde tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Lock "/opt/stack/data/nova/instances/6894b55f-93c6-43c9-8933-56d5a3c43725/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.017s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:12:54 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-62d5bf86-a9dc-438e-ab8b-2f7f9591cbde tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:12:54 user nova-compute[75234]: DEBUG nova.objects.instance [None req-738e17a6-50e7-42de-8ada-0b2118a0bc5d tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] Lazy-loading 'flavor' on Instance uuid 8297c264-a213-44ee-be46-5c08eabda9f7 {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:12:54 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-62d5bf86-a9dc-438e-ab8b-2f7f9591cbde tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json" returned: 0 in 0.190s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:12:54 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-62d5bf86-a9dc-438e-ab8b-2f7f9591cbde tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Acquiring lock "80686e29dc9db6600d03e85cb62af298e12d7918" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:12:54 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-62d5bf86-a9dc-438e-ab8b-2f7f9591cbde tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Lock "80686e29dc9db6600d03e85cb62af298e12d7918" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:12:54 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-62d5bf86-a9dc-438e-ab8b-2f7f9591cbde tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:12:54 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-738e17a6-50e7-42de-8ada-0b2118a0bc5d tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] Lock "8297c264-a213-44ee-be46-5c08eabda9f7" "released" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: held 22.802s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:12:54 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-62d5bf86-a9dc-438e-ab8b-2f7f9591cbde tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json" returned: 0 in 0.186s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:12:54 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-62d5bf86-a9dc-438e-ab8b-2f7f9591cbde tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918,backing_fmt=raw /opt/stack/data/nova/instances/6894b55f-93c6-43c9-8933-56d5a3c43725/disk 1073741824 {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:12:54 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-62d5bf86-a9dc-438e-ab8b-2f7f9591cbde tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918,backing_fmt=raw /opt/stack/data/nova/instances/6894b55f-93c6-43c9-8933-56d5a3c43725/disk 1073741824" returned: 0 in 0.045s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:12:54 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-62d5bf86-a9dc-438e-ab8b-2f7f9591cbde tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Lock "80686e29dc9db6600d03e85cb62af298e12d7918" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.237s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:12:54 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-62d5bf86-a9dc-438e-ab8b-2f7f9591cbde tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:12:54 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:12:54 user nova-compute[75234]: DEBUG nova.network.neutron [None req-62d5bf86-a9dc-438e-ab8b-2f7f9591cbde tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] [instance: 6894b55f-93c6-43c9-8933-56d5a3c43725] Successfully created port: 9b2cf46e-008c-4ef3-9555-0cbc584dcf8f {{(pid=75234) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:548}}
Nov 29 03:12:54 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-62d5bf86-a9dc-438e-ab8b-2f7f9591cbde tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json" returned: 0 in 0.177s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:12:54 user nova-compute[75234]: DEBUG nova.virt.disk.api [None req-62d5bf86-a9dc-438e-ab8b-2f7f9591cbde tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Checking if we can resize image /opt/stack/data/nova/instances/6894b55f-93c6-43c9-8933-56d5a3c43725/disk. size=1073741824 {{(pid=75234) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:166}}
Nov 29 03:12:54 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-62d5bf86-a9dc-438e-ab8b-2f7f9591cbde tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6894b55f-93c6-43c9-8933-56d5a3c43725/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:12:54 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-62d5bf86-a9dc-438e-ab8b-2f7f9591cbde tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6894b55f-93c6-43c9-8933-56d5a3c43725/disk --force-share --output=json" returned: 0 in 0.166s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:12:54 user nova-compute[75234]: DEBUG nova.virt.disk.api [None req-62d5bf86-a9dc-438e-ab8b-2f7f9591cbde tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Cannot resize image /opt/stack/data/nova/instances/6894b55f-93c6-43c9-8933-56d5a3c43725/disk to a smaller size. {{(pid=75234) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:172}}
Nov 29 03:12:54 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-62d5bf86-a9dc-438e-ab8b-2f7f9591cbde tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] [instance: 6894b55f-93c6-43c9-8933-56d5a3c43725] Created local disks {{(pid=75234) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4861}}
Nov 29 03:12:54 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-62d5bf86-a9dc-438e-ab8b-2f7f9591cbde tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] [instance: 6894b55f-93c6-43c9-8933-56d5a3c43725] Ensure instance console log exists: /opt/stack/data/nova/instances/6894b55f-93c6-43c9-8933-56d5a3c43725/console.log {{(pid=75234) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4613}}
Nov 29 03:12:54 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-62d5bf86-a9dc-438e-ab8b-2f7f9591cbde tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:12:54 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-62d5bf86-a9dc-438e-ab8b-2f7f9591cbde tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:12:54 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-62d5bf86-a9dc-438e-ab8b-2f7f9591cbde tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:12:55 user nova-compute[75234]: DEBUG nova.network.neutron [None req-62d5bf86-a9dc-438e-ab8b-2f7f9591cbde tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] [instance: 6894b55f-93c6-43c9-8933-56d5a3c43725] Successfully updated port: 9b2cf46e-008c-4ef3-9555-0cbc584dcf8f {{(pid=75234) _update_port /opt/stack/nova/nova/network/neutron.py:586}}
Nov 29 03:12:55 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-62d5bf86-a9dc-438e-ab8b-2f7f9591cbde tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Acquiring lock "refresh_cache-6894b55f-93c6-43c9-8933-56d5a3c43725" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:12:55 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-62d5bf86-a9dc-438e-ab8b-2f7f9591cbde tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Acquired lock "refresh_cache-6894b55f-93c6-43c9-8933-56d5a3c43725" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:12:55 user nova-compute[75234]: DEBUG nova.network.neutron [None req-62d5bf86-a9dc-438e-ab8b-2f7f9591cbde tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] [instance: 6894b55f-93c6-43c9-8933-56d5a3c43725] Building network info cache for instance {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2009}}
Nov 29 03:12:55 user nova-compute[75234]: DEBUG nova.compute.manager [req-9af0ca04-e881-4519-803f-ee849398b2cc req-79478b33-0eab-4e95-b700-60ddc83ec078 service nova] [instance: 6894b55f-93c6-43c9-8933-56d5a3c43725] Received event network-changed-9b2cf46e-008c-4ef3-9555-0cbc584dcf8f {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:12:55 user nova-compute[75234]: DEBUG nova.compute.manager [req-9af0ca04-e881-4519-803f-ee849398b2cc req-79478b33-0eab-4e95-b700-60ddc83ec078 service nova] [instance: 6894b55f-93c6-43c9-8933-56d5a3c43725] Refreshing instance network info cache due to event network-changed-9b2cf46e-008c-4ef3-9555-0cbc584dcf8f. {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11117}}
Nov 29 03:12:55 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-9af0ca04-e881-4519-803f-ee849398b2cc req-79478b33-0eab-4e95-b700-60ddc83ec078 service nova] Acquiring lock "refresh_cache-6894b55f-93c6-43c9-8933-56d5a3c43725" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:12:55 user nova-compute[75234]: DEBUG nova.network.neutron [None req-62d5bf86-a9dc-438e-ab8b-2f7f9591cbde tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] [instance: 6894b55f-93c6-43c9-8933-56d5a3c43725] Instance cache missing network info. {{(pid=75234) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3322}}
Nov 29 03:12:55 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:12:55 user nova-compute[75234]: DEBUG nova.network.neutron [None req-62d5bf86-a9dc-438e-ab8b-2f7f9591cbde tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] [instance: 6894b55f-93c6-43c9-8933-56d5a3c43725] Updating instance_info_cache with network_info: [{"id": "9b2cf46e-008c-4ef3-9555-0cbc584dcf8f", "address": "fa:16:3e:d1:0b:00", "network": {"id": "b7a6cef0-4cf1-46e6-99fb-4972879e3913", "bridge": "br-int", "label": "tempest-TestStampPattern-1195861223-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "c4393bc598a845a9bfb667e5332f14a6", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap9b2cf46e-00", "ovs_interfaceid": "9b2cf46e-008c-4ef3-9555-0cbc584dcf8f", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:12:55 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-62d5bf86-a9dc-438e-ab8b-2f7f9591cbde tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Releasing lock "refresh_cache-6894b55f-93c6-43c9-8933-56d5a3c43725" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:12:55 user nova-compute[75234]: DEBUG nova.compute.manager [None req-62d5bf86-a9dc-438e-ab8b-2f7f9591cbde tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] [instance: 6894b55f-93c6-43c9-8933-56d5a3c43725] Instance network_info: |[{"id": "9b2cf46e-008c-4ef3-9555-0cbc584dcf8f", "address": "fa:16:3e:d1:0b:00", "network": {"id": "b7a6cef0-4cf1-46e6-99fb-4972879e3913", "bridge": "br-int", "label": "tempest-TestStampPattern-1195861223-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "c4393bc598a845a9bfb667e5332f14a6", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap9b2cf46e-00", "ovs_interfaceid": "9b2cf46e-008c-4ef3-9555-0cbc584dcf8f", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=75234) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1987}}
Nov 29 03:12:55 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-9af0ca04-e881-4519-803f-ee849398b2cc req-79478b33-0eab-4e95-b700-60ddc83ec078 service nova] Acquired lock "refresh_cache-6894b55f-93c6-43c9-8933-56d5a3c43725" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:12:55 user nova-compute[75234]: DEBUG nova.network.neutron [req-9af0ca04-e881-4519-803f-ee849398b2cc req-79478b33-0eab-4e95-b700-60ddc83ec078 service nova] [instance: 6894b55f-93c6-43c9-8933-56d5a3c43725] Refreshing network info cache for port 9b2cf46e-008c-4ef3-9555-0cbc584dcf8f {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2006}}
Nov 29 03:12:55 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-62d5bf86-a9dc-438e-ab8b-2f7f9591cbde tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] [instance: 6894b55f-93c6-43c9-8933-56d5a3c43725] Start _get_guest_xml network_info=[{"id": "9b2cf46e-008c-4ef3-9555-0cbc584dcf8f", "address": "fa:16:3e:d1:0b:00", "network": {"id": "b7a6cef0-4cf1-46e6-99fb-4972879e3913", "bridge": "br-int", "label": "tempest-TestStampPattern-1195861223-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "c4393bc598a845a9bfb667e5332f14a6", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap9b2cf46e-00", "ovs_interfaceid": "9b2cf46e-008c-4ef3-9555-0cbc584dcf8f", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum='c8fc807773e5354afe61636071771906',container_format='bare',created_at=2023-11-29T03:02:15Z,direct_url=<?>,disk_format='qcow2',id=96c30ba7-9b73-4a30-8ba4-7138879573b7,min_disk=0,min_ram=0,name='cirros-0.6.2-x86_64-disk',owner='b18e380921c748b78517b6f1211cd0f1',properties=ImageMetaProps,protected=<?>,size=21430272,status='active',tags=<?>,updated_at=2023-11-29T03:02:17Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [{'device_name': '/dev/vda', 'encryption_format': None, 'encrypted': False, 'device_type': 'disk', 'size': 0, 'disk_bus': 'virtio', 'boot_index': 0, 'encryption_options': None, 'encryption_secret_uuid': None, 'guest_format': None, 'image_id': '96c30ba7-9b73-4a30-8ba4-7138879573b7'}], 'ephemerals': [], 'block_device_mapping': [], 'swap': None} {{(pid=75234) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7562}}
Nov 29 03:12:55 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-62d5bf86-a9dc-438e-ab8b-2f7f9591cbde tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:12:55 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-62d5bf86-a9dc-438e-ab8b-2f7f9591cbde tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:12:55 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-62d5bf86-a9dc-438e-ab8b-2f7f9591cbde tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Searching host: 'user' for CPU controller through CGroups V1... {{(pid=75234) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1652}}
Nov 29 03:12:55 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-62d5bf86-a9dc-438e-ab8b-2f7f9591cbde tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] CPU controller missing on host. {{(pid=75234) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1662}}
Nov 29 03:12:55 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-62d5bf86-a9dc-438e-ab8b-2f7f9591cbde tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Searching host: 'user' for CPU controller through CGroups V2... {{(pid=75234) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1671}}
Nov 29 03:12:55 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-62d5bf86-a9dc-438e-ab8b-2f7f9591cbde tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] CPU controller found on host. {{(pid=75234) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1678}}
Nov 29 03:12:55 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-62d5bf86-a9dc-438e-ab8b-2f7f9591cbde tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=75234) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5400}}
Nov 29 03:12:55 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-62d5bf86-a9dc-438e-ab8b-2f7f9591cbde tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-11-29T03:02:04Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='1',id=6,is_public=True,memory_mb=512,name='m1.tiny',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='c8fc807773e5354afe61636071771906',container_format='bare',created_at=2023-11-29T03:02:15Z,direct_url=<?>,disk_format='qcow2',id=96c30ba7-9b73-4a30-8ba4-7138879573b7,min_disk=0,min_ram=0,name='cirros-0.6.2-x86_64-disk',owner='b18e380921c748b78517b6f1211cd0f1',properties=ImageMetaProps,protected=<?>,size=21430272,status='active',tags=<?>,updated_at=2023-11-29T03:02:17Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:563}}
Nov 29 03:12:55 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-62d5bf86-a9dc-438e-ab8b-2f7f9591cbde tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Flavor limits 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:348}}
Nov 29 03:12:55 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-62d5bf86-a9dc-438e-ab8b-2f7f9591cbde tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Image limits 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:352}}
Nov 29 03:12:55 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-62d5bf86-a9dc-438e-ab8b-2f7f9591cbde tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Flavor pref 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:388}}
Nov 29 03:12:55 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-62d5bf86-a9dc-438e-ab8b-2f7f9591cbde tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Image pref 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:392}}
Nov 29 03:12:55 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-62d5bf86-a9dc-438e-ab8b-2f7f9591cbde tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:430}}
Nov 29 03:12:55 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-62d5bf86-a9dc-438e-ab8b-2f7f9591cbde tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:569}}
Nov 29 03:12:55 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-62d5bf86-a9dc-438e-ab8b-2f7f9591cbde tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=75234) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:471}}
Nov 29 03:12:55 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-62d5bf86-a9dc-438e-ab8b-2f7f9591cbde tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Got 1 possible topologies {{(pid=75234) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:501}}
Nov 29 03:12:55 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-62d5bf86-a9dc-438e-ab8b-2f7f9591cbde tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:575}}
Nov 29 03:12:55 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-62d5bf86-a9dc-438e-ab8b-2f7f9591cbde tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:577}}
Nov 29 03:12:55 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-62d5bf86-a9dc-438e-ab8b-2f7f9591cbde tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-11-29T03:12:52Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-TestStampPattern-server-472892053',display_name='tempest-TestStampPattern-server-472892053',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-teststamppattern-server-472892053',id=22,image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBMhfiLNgC9PAWcr8xAscyt4FB2hoan+iXcHV+JRry3FLwwpkjQpDEiOvBF2D+f1pgs1A/1GOV9t1zFfztYSi2DW0KkfBHuJTofrSJvEENNcXO0BnR5RakzAYYv+USO8uHQ==',key_name='tempest-TestStampPattern-1617966982',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='c4393bc598a845a9bfb667e5332f14a6',ramdisk_id='',reservation_id='r-b4j3yqsa',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TestStampPattern-448969956',owner_user_name='tempest-TestStampPattern-448969956-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-11-29T03:12:54Z,user_data=None,user_id='dc1de1dad3794b47b0e4bf7445c96997',uuid=6894b55f-93c6-43c9-8933-56d5a3c43725,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "9b2cf46e-008c-4ef3-9555-0cbc584dcf8f", "address": "fa:16:3e:d1:0b:00", "network": {"id": "b7a6cef0-4cf1-46e6-99fb-4972879e3913", "bridge": "br-int", "label": "tempest-TestStampPattern-1195861223-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "c4393bc598a845a9bfb667e5332f14a6", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap9b2cf46e-00", "ovs_interfaceid": "9b2cf46e-008c-4ef3-9555-0cbc584dcf8f", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=75234) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Nov 29 03:12:55 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-62d5bf86-a9dc-438e-ab8b-2f7f9591cbde tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Converting VIF {"id": "9b2cf46e-008c-4ef3-9555-0cbc584dcf8f", "address": "fa:16:3e:d1:0b:00", "network": {"id": "b7a6cef0-4cf1-46e6-99fb-4972879e3913", "bridge": "br-int", "label": "tempest-TestStampPattern-1195861223-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "c4393bc598a845a9bfb667e5332f14a6", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap9b2cf46e-00", "ovs_interfaceid": "9b2cf46e-008c-4ef3-9555-0cbc584dcf8f", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:12:55 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-62d5bf86-a9dc-438e-ab8b-2f7f9591cbde tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:d1:0b:00,bridge_name='br-int',has_traffic_filtering=True,id=9b2cf46e-008c-4ef3-9555-0cbc584dcf8f,network=Network(b7a6cef0-4cf1-46e6-99fb-4972879e3913),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap9b2cf46e-00') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:12:55 user nova-compute[75234]: DEBUG nova.objects.instance [None req-62d5bf86-a9dc-438e-ab8b-2f7f9591cbde tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Lazy-loading 'pci_devices' on Instance uuid 6894b55f-93c6-43c9-8933-56d5a3c43725 {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:12:55 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-62d5bf86-a9dc-438e-ab8b-2f7f9591cbde tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] [instance: 6894b55f-93c6-43c9-8933-56d5a3c43725] End _get_guest_xml xml=<domain type="kvm">
Nov 29 03:12:55 user nova-compute[75234]:   <uuid>6894b55f-93c6-43c9-8933-56d5a3c43725</uuid>
Nov 29 03:12:55 user nova-compute[75234]:   <name>instance-00000016</name>
Nov 29 03:12:55 user nova-compute[75234]:   <memory>524288</memory>
Nov 29 03:12:55 user nova-compute[75234]:   <vcpu>1</vcpu>
Nov 29 03:12:55 user nova-compute[75234]:   <metadata>
Nov 29 03:12:55 user nova-compute[75234]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Nov 29 03:12:55 user nova-compute[75234]:       <nova:package version="0.0.0"/>
Nov 29 03:12:55 user nova-compute[75234]:       <nova:name>tempest-TestStampPattern-server-472892053</nova:name>
Nov 29 03:12:55 user nova-compute[75234]:       <nova:creationTime>2023-11-29 03:12:55</nova:creationTime>
Nov 29 03:12:55 user nova-compute[75234]:       <nova:flavor name="m1.tiny">
Nov 29 03:12:55 user nova-compute[75234]:         <nova:memory>512</nova:memory>
Nov 29 03:12:55 user nova-compute[75234]:         <nova:disk>1</nova:disk>
Nov 29 03:12:55 user nova-compute[75234]:         <nova:swap>0</nova:swap>
Nov 29 03:12:55 user nova-compute[75234]:         <nova:ephemeral>0</nova:ephemeral>
Nov 29 03:12:55 user nova-compute[75234]:         <nova:vcpus>1</nova:vcpus>
Nov 29 03:12:55 user nova-compute[75234]:       </nova:flavor>
Nov 29 03:12:55 user nova-compute[75234]:       <nova:owner>
Nov 29 03:12:55 user nova-compute[75234]:         <nova:user uuid="dc1de1dad3794b47b0e4bf7445c96997">tempest-TestStampPattern-448969956-project-member</nova:user>
Nov 29 03:12:55 user nova-compute[75234]:         <nova:project uuid="c4393bc598a845a9bfb667e5332f14a6">tempest-TestStampPattern-448969956</nova:project>
Nov 29 03:12:55 user nova-compute[75234]:       </nova:owner>
Nov 29 03:12:55 user nova-compute[75234]:       <nova:root type="image" uuid="96c30ba7-9b73-4a30-8ba4-7138879573b7"/>
Nov 29 03:12:55 user nova-compute[75234]:       <nova:ports>
Nov 29 03:12:55 user nova-compute[75234]:         <nova:port uuid="9b2cf46e-008c-4ef3-9555-0cbc584dcf8f">
Nov 29 03:12:55 user nova-compute[75234]:           <nova:ip type="fixed" address="10.0.0.7" ipVersion="4"/>
Nov 29 03:12:55 user nova-compute[75234]:         </nova:port>
Nov 29 03:12:55 user nova-compute[75234]:       </nova:ports>
Nov 29 03:12:55 user nova-compute[75234]:     </nova:instance>
Nov 29 03:12:55 user nova-compute[75234]:   </metadata>
Nov 29 03:12:55 user nova-compute[75234]:   <sysinfo type="smbios">
Nov 29 03:12:55 user nova-compute[75234]:     <system>
Nov 29 03:12:55 user nova-compute[75234]:       <entry name="manufacturer">OpenStack Foundation</entry>
Nov 29 03:12:55 user nova-compute[75234]:       <entry name="product">OpenStack Nova</entry>
Nov 29 03:12:55 user nova-compute[75234]:       <entry name="version">0.0.0</entry>
Nov 29 03:12:55 user nova-compute[75234]:       <entry name="serial">6894b55f-93c6-43c9-8933-56d5a3c43725</entry>
Nov 29 03:12:55 user nova-compute[75234]:       <entry name="uuid">6894b55f-93c6-43c9-8933-56d5a3c43725</entry>
Nov 29 03:12:55 user nova-compute[75234]:       <entry name="family">Virtual Machine</entry>
Nov 29 03:12:55 user nova-compute[75234]:     </system>
Nov 29 03:12:55 user nova-compute[75234]:   </sysinfo>
Nov 29 03:12:55 user nova-compute[75234]:   <os>
Nov 29 03:12:55 user nova-compute[75234]:     <type arch="x86_64" machine="pc">hvm</type>
Nov 29 03:12:55 user nova-compute[75234]:     <boot dev="hd"/>
Nov 29 03:12:55 user nova-compute[75234]:     <smbios mode="sysinfo"/>
Nov 29 03:12:55 user nova-compute[75234]:   </os>
Nov 29 03:12:55 user nova-compute[75234]:   <features>
Nov 29 03:12:55 user nova-compute[75234]:     <acpi/>
Nov 29 03:12:55 user nova-compute[75234]:     <apic/>
Nov 29 03:12:55 user nova-compute[75234]:     <vmcoreinfo/>
Nov 29 03:12:55 user nova-compute[75234]:   </features>
Nov 29 03:12:55 user nova-compute[75234]:   <clock offset="utc">
Nov 29 03:12:55 user nova-compute[75234]:     <timer name="pit" tickpolicy="delay"/>
Nov 29 03:12:55 user nova-compute[75234]:     <timer name="rtc" tickpolicy="catchup"/>
Nov 29 03:12:55 user nova-compute[75234]:     <timer name="hpet" present="no"/>
Nov 29 03:12:55 user nova-compute[75234]:   </clock>
Nov 29 03:12:55 user nova-compute[75234]:   <cpu mode="custom" match="exact">
Nov 29 03:12:55 user nova-compute[75234]:     <model>Nehalem</model>
Nov 29 03:12:55 user nova-compute[75234]:     <topology sockets="1" cores="1" threads="1"/>
Nov 29 03:12:55 user nova-compute[75234]:   </cpu>
Nov 29 03:12:55 user nova-compute[75234]:   <devices>
Nov 29 03:12:55 user nova-compute[75234]:     <disk type="file" device="disk">
Nov 29 03:12:55 user nova-compute[75234]:       <driver name="qemu" type="qcow2" cache="none"/>
Nov 29 03:12:55 user nova-compute[75234]:       <source file="/opt/stack/data/nova/instances/6894b55f-93c6-43c9-8933-56d5a3c43725/disk"/>
Nov 29 03:12:55 user nova-compute[75234]:       <target dev="vda" bus="virtio"/>
Nov 29 03:12:55 user nova-compute[75234]:     </disk>
Nov 29 03:12:55 user nova-compute[75234]:     <interface type="ethernet">
Nov 29 03:12:55 user nova-compute[75234]:       <mac address="fa:16:3e:d1:0b:00"/>
Nov 29 03:12:55 user nova-compute[75234]:       <model type="virtio"/>
Nov 29 03:12:55 user nova-compute[75234]:       <mtu size="1442"/>
Nov 29 03:12:55 user nova-compute[75234]:       <target dev="tap9b2cf46e-00"/>
Nov 29 03:12:55 user nova-compute[75234]:     </interface>
Nov 29 03:12:55 user nova-compute[75234]:     <serial type="pty">
Nov 29 03:12:55 user nova-compute[75234]:       <log file="/opt/stack/data/nova/instances/6894b55f-93c6-43c9-8933-56d5a3c43725/console.log" append="off"/>
Nov 29 03:12:55 user nova-compute[75234]:     </serial>
Nov 29 03:12:55 user nova-compute[75234]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Nov 29 03:12:55 user nova-compute[75234]:     <video>
Nov 29 03:12:55 user nova-compute[75234]:       <model type="virtio"/>
Nov 29 03:12:55 user nova-compute[75234]:     </video>
Nov 29 03:12:55 user nova-compute[75234]:     <rng model="virtio">
Nov 29 03:12:55 user nova-compute[75234]:       <backend model="random">/dev/urandom</backend>
Nov 29 03:12:55 user nova-compute[75234]:     </rng>
Nov 29 03:12:55 user nova-compute[75234]:     <controller type="usb" index="0" model="none"/>
Nov 29 03:12:55 user nova-compute[75234]:     <memballoon model="virtio">
Nov 29 03:12:55 user nova-compute[75234]:       <stats period="10"/>
Nov 29 03:12:55 user nova-compute[75234]:     </memballoon>
Nov 29 03:12:55 user nova-compute[75234]:   </devices>
Nov 29 03:12:55 user nova-compute[75234]: </domain>
Nov 29 03:12:55 user nova-compute[75234]:  {{(pid=75234) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7568}}
Nov 29 03:12:55 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-62d5bf86-a9dc-438e-ab8b-2f7f9591cbde tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-11-29T03:12:52Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-TestStampPattern-server-472892053',display_name='tempest-TestStampPattern-server-472892053',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-teststamppattern-server-472892053',id=22,image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBMhfiLNgC9PAWcr8xAscyt4FB2hoan+iXcHV+JRry3FLwwpkjQpDEiOvBF2D+f1pgs1A/1GOV9t1zFfztYSi2DW0KkfBHuJTofrSJvEENNcXO0BnR5RakzAYYv+USO8uHQ==',key_name='tempest-TestStampPattern-1617966982',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='c4393bc598a845a9bfb667e5332f14a6',ramdisk_id='',reservation_id='r-b4j3yqsa',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TestStampPattern-448969956',owner_user_name='tempest-TestStampPattern-448969956-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-11-29T03:12:54Z,user_data=None,user_id='dc1de1dad3794b47b0e4bf7445c96997',uuid=6894b55f-93c6-43c9-8933-56d5a3c43725,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "9b2cf46e-008c-4ef3-9555-0cbc584dcf8f", "address": "fa:16:3e:d1:0b:00", "network": {"id": "b7a6cef0-4cf1-46e6-99fb-4972879e3913", "bridge": "br-int", "label": "tempest-TestStampPattern-1195861223-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "c4393bc598a845a9bfb667e5332f14a6", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap9b2cf46e-00", "ovs_interfaceid": "9b2cf46e-008c-4ef3-9555-0cbc584dcf8f", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Nov 29 03:12:55 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-62d5bf86-a9dc-438e-ab8b-2f7f9591cbde tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Converting VIF {"id": "9b2cf46e-008c-4ef3-9555-0cbc584dcf8f", "address": "fa:16:3e:d1:0b:00", "network": {"id": "b7a6cef0-4cf1-46e6-99fb-4972879e3913", "bridge": "br-int", "label": "tempest-TestStampPattern-1195861223-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "c4393bc598a845a9bfb667e5332f14a6", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap9b2cf46e-00", "ovs_interfaceid": "9b2cf46e-008c-4ef3-9555-0cbc584dcf8f", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:12:55 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-62d5bf86-a9dc-438e-ab8b-2f7f9591cbde tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:d1:0b:00,bridge_name='br-int',has_traffic_filtering=True,id=9b2cf46e-008c-4ef3-9555-0cbc584dcf8f,network=Network(b7a6cef0-4cf1-46e6-99fb-4972879e3913),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap9b2cf46e-00') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:12:55 user nova-compute[75234]: DEBUG os_vif [None req-62d5bf86-a9dc-438e-ab8b-2f7f9591cbde tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:d1:0b:00,bridge_name='br-int',has_traffic_filtering=True,id=9b2cf46e-008c-4ef3-9555-0cbc584dcf8f,network=Network(b7a6cef0-4cf1-46e6-99fb-4972879e3913),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap9b2cf46e-00') {{(pid=75234) plug /opt/stack/data/venv/lib/python3.10/site-packages/os_vif/__init__.py:76}}
Nov 29 03:12:55 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:12:55 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:12:55 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Nov 29 03:12:55 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:12:55 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbCreateCommand(_result=None, table=QoS, columns={'type': 'linux-noop', 'external_ids': {'id': 'd0017516-49d6-5701-bcac-fd343805628d', '_type': 'linux-noop'}}, row=False) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:12:55 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:12:55 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:12:55 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:12:55 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:12:55 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tap9b2cf46e-00, may_exist=True, interface_attrs={}) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:12:55 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Port, record=tap9b2cf46e-00, col_values=(('qos', UUID('e62b528c-3eeb-433c-97d2-97bfcdc9cbae')),), if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:12:55 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DbSetCommand(_result=None, table=Interface, record=tap9b2cf46e-00, col_values=(('external_ids', {'iface-id': '9b2cf46e-008c-4ef3-9555-0cbc584dcf8f', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:d1:0b:00', 'vm-uuid': '6894b55f-93c6-43c9-8933-56d5a3c43725'}),), if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:12:55 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:12:55 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:12:55 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:12:55 user nova-compute[75234]: INFO os_vif [None req-62d5bf86-a9dc-438e-ab8b-2f7f9591cbde tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:d1:0b:00,bridge_name='br-int',has_traffic_filtering=True,id=9b2cf46e-008c-4ef3-9555-0cbc584dcf8f,network=Network(b7a6cef0-4cf1-46e6-99fb-4972879e3913),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap9b2cf46e-00')
Nov 29 03:12:55 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-62d5bf86-a9dc-438e-ab8b-2f7f9591cbde tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] No BDM found with device name vda, not building metadata. {{(pid=75234) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12089}}
Nov 29 03:12:55 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-62d5bf86-a9dc-438e-ab8b-2f7f9591cbde tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] No VIF found with MAC fa:16:3e:d1:0b:00, not building metadata {{(pid=75234) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Nov 29 03:12:55 user nova-compute[75234]: DEBUG nova.network.neutron [req-9af0ca04-e881-4519-803f-ee849398b2cc req-79478b33-0eab-4e95-b700-60ddc83ec078 service nova] [instance: 6894b55f-93c6-43c9-8933-56d5a3c43725] Updated VIF entry in instance network info cache for port 9b2cf46e-008c-4ef3-9555-0cbc584dcf8f. {{(pid=75234) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3481}}
Nov 29 03:12:55 user nova-compute[75234]: DEBUG nova.network.neutron [req-9af0ca04-e881-4519-803f-ee849398b2cc req-79478b33-0eab-4e95-b700-60ddc83ec078 service nova] [instance: 6894b55f-93c6-43c9-8933-56d5a3c43725] Updating instance_info_cache with network_info: [{"id": "9b2cf46e-008c-4ef3-9555-0cbc584dcf8f", "address": "fa:16:3e:d1:0b:00", "network": {"id": "b7a6cef0-4cf1-46e6-99fb-4972879e3913", "bridge": "br-int", "label": "tempest-TestStampPattern-1195861223-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "c4393bc598a845a9bfb667e5332f14a6", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap9b2cf46e-00", "ovs_interfaceid": "9b2cf46e-008c-4ef3-9555-0cbc584dcf8f", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:12:55 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-9af0ca04-e881-4519-803f-ee849398b2cc req-79478b33-0eab-4e95-b700-60ddc83ec078 service nova] Releasing lock "refresh_cache-6894b55f-93c6-43c9-8933-56d5a3c43725" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:12:56 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager.update_available_resource {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:12:56 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:12:56 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:12:56 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:12:56 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Auditing locally available compute resources for user (node: user) {{(pid=75234) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:907}}
Nov 29 03:12:56 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:12:56 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:12:57 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:12:57 user nova-compute[75234]: DEBUG nova.compute.manager [req-20c7c80d-db2e-4fc6-bd23-064746b48358 req-9e641774-9a55-46b8-ac62-f30804621c1e service nova] [instance: 6894b55f-93c6-43c9-8933-56d5a3c43725] Received event network-vif-plugged-9b2cf46e-008c-4ef3-9555-0cbc584dcf8f {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:12:57 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-20c7c80d-db2e-4fc6-bd23-064746b48358 req-9e641774-9a55-46b8-ac62-f30804621c1e service nova] Acquiring lock "6894b55f-93c6-43c9-8933-56d5a3c43725-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:12:57 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-20c7c80d-db2e-4fc6-bd23-064746b48358 req-9e641774-9a55-46b8-ac62-f30804621c1e service nova] Lock "6894b55f-93c6-43c9-8933-56d5a3c43725-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:12:57 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-20c7c80d-db2e-4fc6-bd23-064746b48358 req-9e641774-9a55-46b8-ac62-f30804621c1e service nova] Lock "6894b55f-93c6-43c9-8933-56d5a3c43725-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:12:57 user nova-compute[75234]: DEBUG nova.compute.manager [req-20c7c80d-db2e-4fc6-bd23-064746b48358 req-9e641774-9a55-46b8-ac62-f30804621c1e service nova] [instance: 6894b55f-93c6-43c9-8933-56d5a3c43725] No waiting events found dispatching network-vif-plugged-9b2cf46e-008c-4ef3-9555-0cbc584dcf8f {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:12:57 user nova-compute[75234]: WARNING nova.compute.manager [req-20c7c80d-db2e-4fc6-bd23-064746b48358 req-9e641774-9a55-46b8-ac62-f30804621c1e service nova] [instance: 6894b55f-93c6-43c9-8933-56d5a3c43725] Received unexpected event network-vif-plugged-9b2cf46e-008c-4ef3-9555-0cbc584dcf8f for instance with vm_state building and task_state spawning.
Nov 29 03:12:57 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/1eb9b734-2711-4463-9743-e5fd95c6857b/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:12:57 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:12:57 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:12:57 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:12:57 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:12:57 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:12:57 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/1eb9b734-2711-4463-9743-e5fd95c6857b/disk --force-share --output=json" returned: 0 in 0.174s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:12:57 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/1eb9b734-2711-4463-9743-e5fd95c6857b/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:12:57 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/1eb9b734-2711-4463-9743-e5fd95c6857b/disk --force-share --output=json" returned: 0 in 0.188s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:12:57 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a9350a18-14e6-42c6-a150-0a1487f20ad9/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:12:58 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a9350a18-14e6-42c6-a150-0a1487f20ad9/disk --force-share --output=json" returned: 0 in 0.195s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:12:58 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a9350a18-14e6-42c6-a150-0a1487f20ad9/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:12:58 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a9350a18-14e6-42c6-a150-0a1487f20ad9/disk --force-share --output=json" returned: 0 in 0.180s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:12:58 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] skipping disk /dev/sdc (vdb) as it is a volume {{(pid=75234) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11213}}
Nov 29 03:12:58 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/8297c264-a213-44ee-be46-5c08eabda9f7/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:12:58 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-8978a107-f9e6-49c8-9019-39540c368d85 tempest-TestVolumeSwap-1089931716 tempest-TestVolumeSwap-1089931716-project-member] Acquiring lock "3e355ae8-aab9-4ae9-b6e3-6de108ec1af9" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:12:58 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-8978a107-f9e6-49c8-9019-39540c368d85 tempest-TestVolumeSwap-1089931716 tempest-TestVolumeSwap-1089931716-project-member] Lock "3e355ae8-aab9-4ae9-b6e3-6de108ec1af9" acquired by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:12:58 user nova-compute[75234]: DEBUG nova.objects.instance [None req-8978a107-f9e6-49c8-9019-39540c368d85 tempest-TestVolumeSwap-1089931716 tempest-TestVolumeSwap-1089931716-project-member] Lazy-loading 'flavor' on Instance uuid 3e355ae8-aab9-4ae9-b6e3-6de108ec1af9 {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:12:58 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-8978a107-f9e6-49c8-9019-39540c368d85 tempest-TestVolumeSwap-1089931716 tempest-TestVolumeSwap-1089931716-project-member] Lock "3e355ae8-aab9-4ae9-b6e3-6de108ec1af9" "released" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: held 0.075s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:12:58 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/8297c264-a213-44ee-be46-5c08eabda9f7/disk --force-share --output=json" returned: 0 in 0.181s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:12:58 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/8297c264-a213-44ee-be46-5c08eabda9f7/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:12:58 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-8978a107-f9e6-49c8-9019-39540c368d85 tempest-TestVolumeSwap-1089931716 tempest-TestVolumeSwap-1089931716-project-member] Acquiring lock "3e355ae8-aab9-4ae9-b6e3-6de108ec1af9" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:12:58 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-8978a107-f9e6-49c8-9019-39540c368d85 tempest-TestVolumeSwap-1089931716 tempest-TestVolumeSwap-1089931716-project-member] Lock "3e355ae8-aab9-4ae9-b6e3-6de108ec1af9" acquired by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:12:58 user nova-compute[75234]: INFO nova.compute.manager [None req-8978a107-f9e6-49c8-9019-39540c368d85 tempest-TestVolumeSwap-1089931716 tempest-TestVolumeSwap-1089931716-project-member] [instance: 3e355ae8-aab9-4ae9-b6e3-6de108ec1af9] Attaching volume 4ed484cc-9de2-42df-8d99-637f3b47d474 to /dev/vdb
Nov 29 03:12:58 user nova-compute[75234]: DEBUG os_brick.utils [None req-8978a107-f9e6-49c8-9019-39540c368d85 tempest-TestVolumeSwap-1089931716 tempest-TestVolumeSwap-1089931716-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:176}}
Nov 29 03:12:58 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:12:58 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/8297c264-a213-44ee-be46-5c08eabda9f7/disk --force-share --output=json" returned: 0 in 0.198s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:12:58 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/3e355ae8-aab9-4ae9-b6e3-6de108ec1af9/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:12:58 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.022s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:12:58 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[0d516267-044d-4173-9921-4918f682c1dd]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:dd1368c5ba2c\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:12:58 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt -v / -n -o SOURCE {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:12:58 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt -v / -n -o SOURCE" returned: 0 in 0.019s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:12:58 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[e919dabe-58b3-46fc-94cd-7fc6fc071683]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:12:58 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:12:58 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.017s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:12:58 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[4f4fab29-7200-4099-a1fa-2c31834aab39]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:12:58 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[4580aece-0665-4ec2-b83f-7fd96e99877c]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:12:58 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-8978a107-f9e6-49c8-9019-39540c368d85 tempest-TestVolumeSwap-1089931716 tempest-TestVolumeSwap-1089931716-project-member] Running cmd (subprocess): nvme version {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:12:58 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-8978a107-f9e6-49c8-9019-39540c368d85 tempest-TestVolumeSwap-1089931716 tempest-TestVolumeSwap-1089931716-project-member] 'nvme version' failed. Not Retrying. {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:473}}
Nov 29 03:12:58 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.nvmeof [None req-8978a107-f9e6-49c8-9019-39540c368d85 tempest-TestVolumeSwap-1089931716 tempest-TestVolumeSwap-1089931716-project-member] nvme not present on system {{(pid=75234) nvme_present /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/nvmeof.py:756}}
Nov 29 03:12:58 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:12:58 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:473}}
Nov 29 03:12:58 user nova-compute[75234]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Nov 29 03:12:58 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[743c8523-f5c1-4608-839e-7c5ec152ebf8]: (4, '') {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:12:58 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.lightos [None req-8978a107-f9e6-49c8-9019-39540c368d85 tempest-TestVolumeSwap-1089931716 tempest-TestVolumeSwap-1089931716-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=75234) find_dsc /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/lightos.py:98}}
Nov 29 03:12:58 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.lightos [None req-8978a107-f9e6-49c8-9019-39540c368d85 tempest-TestVolumeSwap-1089931716 tempest-TestVolumeSwap-1089931716-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=75234) get_connector_properties /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/lightos.py:76}}
Nov 29 03:12:58 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.lightos [None req-8978a107-f9e6-49c8-9019-39540c368d85 tempest-TestVolumeSwap-1089931716 tempest-TestVolumeSwap-1089931716-project-member] LIGHTOS: no hostnqn found. {{(pid=75234) get_connector_properties /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/lightos.py:84}}
Nov 29 03:12:58 user nova-compute[75234]: DEBUG os_brick.utils [None req-8978a107-f9e6-49c8-9019-39540c368d85 tempest-TestVolumeSwap-1089931716 tempest-TestVolumeSwap-1089931716-project-member] <== get_connector_properties: return (162ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:dd1368c5ba2c', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:203}}
Nov 29 03:12:58 user nova-compute[75234]: DEBUG nova.virt.block_device [None req-8978a107-f9e6-49c8-9019-39540c368d85 tempest-TestVolumeSwap-1089931716 tempest-TestVolumeSwap-1089931716-project-member] [instance: 3e355ae8-aab9-4ae9-b6e3-6de108ec1af9] Updating existing volume attachment record: fac071df-257b-43d5-88d4-ab2c31be2bf6 {{(pid=75234) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Nov 29 03:12:58 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/3e355ae8-aab9-4ae9-b6e3-6de108ec1af9/disk --force-share --output=json" returned: 0 in 0.180s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:12:58 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/3e355ae8-aab9-4ae9-b6e3-6de108ec1af9/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:12:59 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/3e355ae8-aab9-4ae9-b6e3-6de108ec1af9/disk --force-share --output=json" returned: 0 in 0.290s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:12:59 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a082c96e-e384-40b6-bcc8-fc5e0e104669/disk.rescue --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:12:59 user nova-compute[75234]: DEBUG nova.compute.manager [None req-62d5bf86-a9dc-438e-ab8b-2f7f9591cbde tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] [instance: 6894b55f-93c6-43c9-8933-56d5a3c43725] Instance event wait completed in 0 seconds for  {{(pid=75234) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Nov 29 03:12:59 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-62d5bf86-a9dc-438e-ab8b-2f7f9591cbde tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] [instance: 6894b55f-93c6-43c9-8933-56d5a3c43725] Guest created on hypervisor {{(pid=75234) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4421}}
Nov 29 03:12:59 user nova-compute[75234]: INFO nova.virt.libvirt.driver [-] [instance: 6894b55f-93c6-43c9-8933-56d5a3c43725] Instance spawned successfully.
Nov 29 03:12:59 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-62d5bf86-a9dc-438e-ab8b-2f7f9591cbde tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] [instance: 6894b55f-93c6-43c9-8933-56d5a3c43725] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:904}}
Nov 29 03:12:59 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-62d5bf86-a9dc-438e-ab8b-2f7f9591cbde tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] [instance: 6894b55f-93c6-43c9-8933-56d5a3c43725] Found default for hw_cdrom_bus of ide {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:12:59 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-62d5bf86-a9dc-438e-ab8b-2f7f9591cbde tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] [instance: 6894b55f-93c6-43c9-8933-56d5a3c43725] Found default for hw_disk_bus of virtio {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:12:59 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-62d5bf86-a9dc-438e-ab8b-2f7f9591cbde tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] [instance: 6894b55f-93c6-43c9-8933-56d5a3c43725] Found default for hw_input_bus of None {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:12:59 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-62d5bf86-a9dc-438e-ab8b-2f7f9591cbde tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] [instance: 6894b55f-93c6-43c9-8933-56d5a3c43725] Found default for hw_pointer_model of None {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:12:59 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-62d5bf86-a9dc-438e-ab8b-2f7f9591cbde tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] [instance: 6894b55f-93c6-43c9-8933-56d5a3c43725] Found default for hw_video_model of virtio {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:12:59 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-62d5bf86-a9dc-438e-ab8b-2f7f9591cbde tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] [instance: 6894b55f-93c6-43c9-8933-56d5a3c43725] Found default for hw_vif_model of virtio {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:12:59 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a082c96e-e384-40b6-bcc8-fc5e0e104669/disk.rescue --force-share --output=json" returned: 0 in 0.200s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:12:59 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a082c96e-e384-40b6-bcc8-fc5e0e104669/disk.rescue --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:12:59 user nova-compute[75234]: DEBUG nova.compute.manager [req-f74d863d-2437-4f18-b78a-76634119e668 req-920ee1b5-3ef2-4074-b783-8895e38899dc service nova] [instance: 6894b55f-93c6-43c9-8933-56d5a3c43725] Received event network-vif-plugged-9b2cf46e-008c-4ef3-9555-0cbc584dcf8f {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:12:59 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-f74d863d-2437-4f18-b78a-76634119e668 req-920ee1b5-3ef2-4074-b783-8895e38899dc service nova] Acquiring lock "6894b55f-93c6-43c9-8933-56d5a3c43725-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:12:59 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-f74d863d-2437-4f18-b78a-76634119e668 req-920ee1b5-3ef2-4074-b783-8895e38899dc service nova] Lock "6894b55f-93c6-43c9-8933-56d5a3c43725-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:12:59 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-f74d863d-2437-4f18-b78a-76634119e668 req-920ee1b5-3ef2-4074-b783-8895e38899dc service nova] Lock "6894b55f-93c6-43c9-8933-56d5a3c43725-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:12:59 user nova-compute[75234]: DEBUG nova.compute.manager [req-f74d863d-2437-4f18-b78a-76634119e668 req-920ee1b5-3ef2-4074-b783-8895e38899dc service nova] [instance: 6894b55f-93c6-43c9-8933-56d5a3c43725] No waiting events found dispatching network-vif-plugged-9b2cf46e-008c-4ef3-9555-0cbc584dcf8f {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:12:59 user nova-compute[75234]: WARNING nova.compute.manager [req-f74d863d-2437-4f18-b78a-76634119e668 req-920ee1b5-3ef2-4074-b783-8895e38899dc service nova] [instance: 6894b55f-93c6-43c9-8933-56d5a3c43725] Received unexpected event network-vif-plugged-9b2cf46e-008c-4ef3-9555-0cbc584dcf8f for instance with vm_state building and task_state spawning.
Nov 29 03:12:59 user nova-compute[75234]: DEBUG nova.compute.manager [req-f74d863d-2437-4f18-b78a-76634119e668 req-920ee1b5-3ef2-4074-b783-8895e38899dc service nova] [instance: 3e355ae8-aab9-4ae9-b6e3-6de108ec1af9] Received event network-changed-b70722c1-452e-4b37-9ff3-7d8a1c7a9ba3 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:12:59 user nova-compute[75234]: DEBUG nova.compute.manager [req-f74d863d-2437-4f18-b78a-76634119e668 req-920ee1b5-3ef2-4074-b783-8895e38899dc service nova] [instance: 3e355ae8-aab9-4ae9-b6e3-6de108ec1af9] Refreshing instance network info cache due to event network-changed-b70722c1-452e-4b37-9ff3-7d8a1c7a9ba3. {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11117}}
Nov 29 03:12:59 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-f74d863d-2437-4f18-b78a-76634119e668 req-920ee1b5-3ef2-4074-b783-8895e38899dc service nova] Acquiring lock "refresh_cache-3e355ae8-aab9-4ae9-b6e3-6de108ec1af9" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:12:59 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-f74d863d-2437-4f18-b78a-76634119e668 req-920ee1b5-3ef2-4074-b783-8895e38899dc service nova] Acquired lock "refresh_cache-3e355ae8-aab9-4ae9-b6e3-6de108ec1af9" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:12:59 user nova-compute[75234]: DEBUG nova.network.neutron [req-f74d863d-2437-4f18-b78a-76634119e668 req-920ee1b5-3ef2-4074-b783-8895e38899dc service nova] [instance: 3e355ae8-aab9-4ae9-b6e3-6de108ec1af9] Refreshing network info cache for port b70722c1-452e-4b37-9ff3-7d8a1c7a9ba3 {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2006}}
Nov 29 03:12:59 user nova-compute[75234]: INFO nova.compute.manager [None req-62d5bf86-a9dc-438e-ab8b-2f7f9591cbde tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] [instance: 6894b55f-93c6-43c9-8933-56d5a3c43725] Took 5.52 seconds to spawn the instance on the hypervisor.
Nov 29 03:12:59 user nova-compute[75234]: DEBUG nova.compute.manager [None req-62d5bf86-a9dc-438e-ab8b-2f7f9591cbde tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] [instance: 6894b55f-93c6-43c9-8933-56d5a3c43725] Checking state {{(pid=75234) _get_power_state /opt/stack/nova/nova/compute/manager.py:1782}}
Nov 29 03:12:59 user nova-compute[75234]: INFO nova.compute.manager [None req-62d5bf86-a9dc-438e-ab8b-2f7f9591cbde tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] [instance: 6894b55f-93c6-43c9-8933-56d5a3c43725] Took 6.34 seconds to build instance.
Nov 29 03:12:59 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a082c96e-e384-40b6-bcc8-fc5e0e104669/disk.rescue --force-share --output=json" returned: 0 in 0.189s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:12:59 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a082c96e-e384-40b6-bcc8-fc5e0e104669/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:12:59 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-62d5bf86-a9dc-438e-ab8b-2f7f9591cbde tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Lock "6894b55f-93c6-43c9-8933-56d5a3c43725" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 6.488s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:12:59 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a082c96e-e384-40b6-bcc8-fc5e0e104669/disk --force-share --output=json" returned: 0 in 0.188s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:12:59 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a082c96e-e384-40b6-bcc8-fc5e0e104669/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:12:59 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a082c96e-e384-40b6-bcc8-fc5e0e104669/disk --force-share --output=json" returned: 0 in 0.191s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:13:00 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2bf9fe41-63c2-4882-bcdd-8e314e734ffe/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:13:00 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2bf9fe41-63c2-4882-bcdd-8e314e734ffe/disk --force-share --output=json" returned: 0 in 0.189s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:13:00 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2bf9fe41-63c2-4882-bcdd-8e314e734ffe/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:13:00 user nova-compute[75234]: DEBUG nova.network.neutron [req-f74d863d-2437-4f18-b78a-76634119e668 req-920ee1b5-3ef2-4074-b783-8895e38899dc service nova] [instance: 3e355ae8-aab9-4ae9-b6e3-6de108ec1af9] Updated VIF entry in instance network info cache for port b70722c1-452e-4b37-9ff3-7d8a1c7a9ba3. {{(pid=75234) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3481}}
Nov 29 03:13:00 user nova-compute[75234]: DEBUG nova.network.neutron [req-f74d863d-2437-4f18-b78a-76634119e668 req-920ee1b5-3ef2-4074-b783-8895e38899dc service nova] [instance: 3e355ae8-aab9-4ae9-b6e3-6de108ec1af9] Updating instance_info_cache with network_info: [{"id": "b70722c1-452e-4b37-9ff3-7d8a1c7a9ba3", "address": "fa:16:3e:db:81:b1", "network": {"id": "67d2e671-98cd-4178-8f5f-1ee06dd66006", "bridge": "br-int", "label": "tempest-TestVolumeSwap-1472119110-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.131", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "cd4a0b5a414044d68524962183d47fca", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapb70722c1-45", "ovs_interfaceid": "b70722c1-452e-4b37-9ff3-7d8a1c7a9ba3", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:13:00 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-f74d863d-2437-4f18-b78a-76634119e668 req-920ee1b5-3ef2-4074-b783-8895e38899dc service nova] Releasing lock "refresh_cache-3e355ae8-aab9-4ae9-b6e3-6de108ec1af9" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:13:00 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2bf9fe41-63c2-4882-bcdd-8e314e734ffe/disk --force-share --output=json" returned: 0 in 0.189s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:13:00 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-2ba9d3c2-d9fa-4fb7-9115-db6c314836c5 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Acquiring lock "062753a6-8f39-4e80-a549-9f0f5a4b877a" by "nova.compute.manager.ComputeManager.unshelve_instance.<locals>.do_unshelve_instance" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:13:00 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-2ba9d3c2-d9fa-4fb7-9115-db6c314836c5 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Lock "062753a6-8f39-4e80-a549-9f0f5a4b877a" acquired by "nova.compute.manager.ComputeManager.unshelve_instance.<locals>.do_unshelve_instance" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:13:00 user nova-compute[75234]: INFO nova.compute.manager [None req-2ba9d3c2-d9fa-4fb7-9115-db6c314836c5 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] [instance: 062753a6-8f39-4e80-a549-9f0f5a4b877a] Unshelving
Nov 29 03:13:00 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f609a7b7-1be2-44b4-9177-7e72a4d4c060/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:13:00 user nova-compute[75234]: INFO nova.virt.block_device [None req-2ba9d3c2-d9fa-4fb7-9115-db6c314836c5 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] [instance: 062753a6-8f39-4e80-a549-9f0f5a4b877a] Booting with volume 6a4d34d3-9974-43bc-afcb-9169c893d9b7 at /dev/vdc
Nov 29 03:13:00 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:13:00 user nova-compute[75234]: DEBUG os_brick.utils [None req-2ba9d3c2-d9fa-4fb7-9115-db6c314836c5 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:176}}
Nov 29 03:13:00 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:13:00 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.015s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:13:00 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[b64d814d-9b91-416a-ba3a-8464bba4b3d9]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:dd1368c5ba2c\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:13:00 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt -v / -n -o SOURCE {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:13:00 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f609a7b7-1be2-44b4-9177-7e72a4d4c060/disk --force-share --output=json" returned: 0 in 0.277s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:13:00 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f609a7b7-1be2-44b4-9177-7e72a4d4c060/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:13:00 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt -v / -n -o SOURCE" returned: 0 in 0.017s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:13:00 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[1bd64b10-c82f-437e-a81f-e5715af747f0]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:13:00 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:13:00 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.021s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:13:00 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[f7def313-6ba5-4810-9bba-2f97e5fd9411]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:13:00 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[4a4780d1-6820-4e53-b8a9-8bd0112af1e8]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:13:00 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-2ba9d3c2-d9fa-4fb7-9115-db6c314836c5 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Running cmd (subprocess): nvme version {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:13:00 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-2ba9d3c2-d9fa-4fb7-9115-db6c314836c5 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] 'nvme version' failed. Not Retrying. {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:473}}
Nov 29 03:13:00 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.nvmeof [None req-2ba9d3c2-d9fa-4fb7-9115-db6c314836c5 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] nvme not present on system {{(pid=75234) nvme_present /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/nvmeof.py:756}}
Nov 29 03:13:00 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:13:00 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:473}}
Nov 29 03:13:00 user nova-compute[75234]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Nov 29 03:13:00 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[f1e64987-d0d7-4af2-87aa-d55e393d5cf3]: (4, '') {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:13:00 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.lightos [None req-2ba9d3c2-d9fa-4fb7-9115-db6c314836c5 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=75234) find_dsc /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/lightos.py:98}}
Nov 29 03:13:00 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.lightos [None req-2ba9d3c2-d9fa-4fb7-9115-db6c314836c5 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=75234) get_connector_properties /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/lightos.py:76}}
Nov 29 03:13:00 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.lightos [None req-2ba9d3c2-d9fa-4fb7-9115-db6c314836c5 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] LIGHTOS: no hostnqn found. {{(pid=75234) get_connector_properties /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/lightos.py:84}}
Nov 29 03:13:00 user nova-compute[75234]: DEBUG os_brick.utils [None req-2ba9d3c2-d9fa-4fb7-9115-db6c314836c5 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] <== get_connector_properties: return (189ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:dd1368c5ba2c', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:203}}
Nov 29 03:13:00 user nova-compute[75234]: DEBUG nova.virt.block_device [None req-2ba9d3c2-d9fa-4fb7-9115-db6c314836c5 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] [instance: 062753a6-8f39-4e80-a549-9f0f5a4b877a] Updating existing volume attachment record: a19e46a6-7567-4073-8a06-c636553e5e46 {{(pid=75234) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Nov 29 03:13:00 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f609a7b7-1be2-44b4-9177-7e72a4d4c060/disk --force-share --output=json" returned: 0 in 0.201s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:13:00 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/21e9c6cd-c39f-41c0-8768-5dd64589c3f3/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:13:01 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/21e9c6cd-c39f-41c0-8768-5dd64589c3f3/disk --force-share --output=json" returned: 0 in 0.182s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:13:01 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/21e9c6cd-c39f-41c0-8768-5dd64589c3f3/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:13:01 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/21e9c6cd-c39f-41c0-8768-5dd64589c3f3/disk --force-share --output=json" returned: 0 in 0.180s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:13:01 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/13e5da64-9adc-4479-81c7-d329e9b5f08f/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:13:01 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/13e5da64-9adc-4479-81c7-d329e9b5f08f/disk --force-share --output=json" returned: 0 in 0.180s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:13:01 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/13e5da64-9adc-4479-81c7-d329e9b5f08f/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:13:01 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:13:01 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/13e5da64-9adc-4479-81c7-d329e9b5f08f/disk --force-share --output=json" returned: 0 in 0.187s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:13:01 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6c0e0184-ca3a-48ec-a6e3-a0a1172260d6/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:13:01 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6c0e0184-ca3a-48ec-a6e3-a0a1172260d6/disk --force-share --output=json" returned: 0 in 0.181s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:13:01 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6c0e0184-ca3a-48ec-a6e3-a0a1172260d6/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:13:02 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6c0e0184-ca3a-48ec-a6e3-a0a1172260d6/disk --force-share --output=json" returned: 0 in 0.198s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:13:02 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] skipping disk /dev/sdf (vdb) as it is a volume {{(pid=75234) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11213}}
Nov 29 03:13:02 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6894b55f-93c6-43c9-8933-56d5a3c43725/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:13:02 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6894b55f-93c6-43c9-8933-56d5a3c43725/disk --force-share --output=json" returned: 0 in 0.189s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:13:02 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6894b55f-93c6-43c9-8933-56d5a3c43725/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:13:02 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6894b55f-93c6-43c9-8933-56d5a3c43725/disk --force-share --output=json" returned: 0 in 0.176s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:13:02 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/d3b90f39-4b91-4152-b820-bd0e263eec89/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:13:02 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/d3b90f39-4b91-4152-b820-bd0e263eec89/disk --force-share --output=json" returned: 0 in 0.171s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:13:02 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/d3b90f39-4b91-4152-b820-bd0e263eec89/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:13:02 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/d3b90f39-4b91-4152-b820-bd0e263eec89/disk --force-share --output=json" returned: 0 in 0.174s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:13:03 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:13:03 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:13:03 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Hypervisor/Node resource view: name=user free_ram=7154MB free_disk=25.287513732910156GB free_vcpus=1 pci_devices=[{"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}] {{(pid=75234) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1106}}
Nov 29 03:13:03 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:13:03 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:13:03 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance 2bf9fe41-63c2-4882-bcdd-8e314e734ffe actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:13:03 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance 21e9c6cd-c39f-41c0-8768-5dd64589c3f3 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:13:03 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance f609a7b7-1be2-44b4-9177-7e72a4d4c060 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:13:03 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance 8297c264-a213-44ee-be46-5c08eabda9f7 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:13:03 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance 6c0e0184-ca3a-48ec-a6e3-a0a1172260d6 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:13:03 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance a9350a18-14e6-42c6-a150-0a1487f20ad9 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 512, 'DISK_GB': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:13:03 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance d3b90f39-4b91-4152-b820-bd0e263eec89 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:13:03 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance 13e5da64-9adc-4479-81c7-d329e9b5f08f actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:13:03 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance a082c96e-e384-40b6-bcc8-fc5e0e104669 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:13:03 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance 3e355ae8-aab9-4ae9-b6e3-6de108ec1af9 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:13:03 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance 1eb9b734-2711-4463-9743-e5fd95c6857b actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:13:03 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance 6894b55f-93c6-43c9-8933-56d5a3c43725 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:13:03 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance 062753a6-8f39-4e80-a549-9f0f5a4b877a has been scheduled to this compute host, the scheduler has made an allocation against this compute node but the instance has yet to start. Skipping heal of allocation: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1761}}
Nov 29 03:13:03 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Total usable vcpus: 12, total allocated vcpus: 12 {{(pid=75234) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1129}}
Nov 29 03:13:03 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Final resource view: name=user phys_ram=16011MB used_ram=6656MB phys_disk=40GB used_disk=12GB total_vcpus=12 used_vcpus=12 pci_stats=[] {{(pid=75234) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1138}}
Nov 29 03:13:03 user nova-compute[75234]: DEBUG nova.compute.provider_tree [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Inventory has not changed in ProviderTree for provider: 4d291220-7c37-4839-a0e3-b23c66502b74 {{(pid=75234) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Nov 29 03:13:03 user nova-compute[75234]: DEBUG nova.scheduler.client.report [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Inventory has not changed for provider 4d291220-7c37-4839-a0e3-b23c66502b74 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=75234) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Nov 29 03:13:03 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Compute_service record updated for user:user {{(pid=75234) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:1067}}
Nov 29 03:13:03 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.689s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:13:04 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:13:04 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:13:05 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:13:06 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:13:10 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:13:14 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-8978a107-f9e6-49c8-9019-39540c368d85 tempest-TestVolumeSwap-1089931716 tempest-TestVolumeSwap-1089931716-project-member] [instance: 3e355ae8-aab9-4ae9-b6e3-6de108ec1af9] Calling os-brick to attach iSCSI Volume {{(pid=75234) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Nov 29 03:13:14 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-8978a107-f9e6-49c8-9019-39540c368d85 tempest-TestVolumeSwap-1089931716 tempest-TestVolumeSwap-1089931716-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f8a5050c9a0>, 'connection_properties': {'target_iqn': 'iqn.2010-10.org.openstack:4ed484cc-9de2-42df-8d99-637f3b47d474', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'mBlevMXQ', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:176}}
Nov 29 03:13:14 user nova-compute[75234]: WARNING os_brick.initiator.connectors.base [None req-8978a107-f9e6-49c8-9019-39540c368d85 tempest-TestVolumeSwap-1089931716 tempest-TestVolumeSwap-1089931716-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Nov 29 03:13:14 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-8978a107-f9e6-49c8-9019-39540c368d85 tempest-TestVolumeSwap-1089931716 tempest-TestVolumeSwap-1089931716-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:68}}
Nov 29 03:13:14 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-8978a107-f9e6-49c8-9019-39540c368d85 tempest-TestVolumeSwap-1089931716 tempest-TestVolumeSwap-1089931716-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:73}}
Nov 29 03:13:14 user nova-compute[75234]: WARNING os_brick.initiator.connectors.base [None req-8978a107-f9e6-49c8-9019-39540c368d85 tempest-TestVolumeSwap-1089931716 tempest-TestVolumeSwap-1089931716-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Nov 29 03:13:14 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-8978a107-f9e6-49c8-9019-39540c368d85 tempest-TestVolumeSwap-1089931716 tempest-TestVolumeSwap-1089931716-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:4ed484cc-9de2-42df-8d99-637f3b47d474" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:68}}
Nov 29 03:13:14 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-8978a107-f9e6-49c8-9019-39540c368d85 tempest-TestVolumeSwap-1089931716 tempest-TestVolumeSwap-1089931716-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:4ed484cc-9de2-42df-8d99-637f3b47d474" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:73}}
Nov 29 03:13:14 user nova-compute[75234]: INFO os_brick.initiator.connectors.iscsi [None req-8978a107-f9e6-49c8-9019-39540c368d85 tempest-TestVolumeSwap-1089931716 tempest-TestVolumeSwap-1089931716-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Nov 29 03:13:14 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:4ed484cc-9de2-42df-8d99-637f3b47d474 -p 172.16.0.220:3260 {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:13:14 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:4ed484cc-9de2-42df-8d99-637f3b47d474 -p 172.16.0.220:3260" returned: 21 in 0.018s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:13:14 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[c3bfcfd6-048b-47f0-b6d3-27988bccb788]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:13:14 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-8978a107-f9e6-49c8-9019-39540c368d85 tempest-TestVolumeSwap-1089931716 tempest-TestVolumeSwap-1089931716-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Nov 29 03:13:14 user nova-compute[75234]:  {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:13:14 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:4ed484cc-9de2-42df-8d99-637f3b47d474 -p 172.16.0.220:3260 --interface default --op new {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:13:14 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:4ed484cc-9de2-42df-8d99-637f3b47d474 -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.018s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:13:14 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[85132f6c-1da1-48c4-b244-bd9ba0eb8b4a]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:4ed484cc-9de2-42df-8d99-637f3b47d474] added\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:13:14 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-8978a107-f9e6-49c8-9019-39540c368d85 tempest-TestVolumeSwap-1089931716 tempest-TestVolumeSwap-1089931716-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:4ed484cc-9de2-42df-8d99-637f3b47d474] added
Nov 29 03:13:14 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:13:14 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:4ed484cc-9de2-42df-8d99-637f3b47d474 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:13:14 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:4ed484cc-9de2-42df-8d99-637f3b47d474 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.017s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:13:14 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[025c22e0-b7d4-4fde-beff-d7971c561618]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:13:14 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-8978a107-f9e6-49c8-9019-39540c368d85 tempest-TestVolumeSwap-1089931716 tempest-TestVolumeSwap-1089931716-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:13:14 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:4ed484cc-9de2-42df-8d99-637f3b47d474 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:13:14 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:4ed484cc-9de2-42df-8d99-637f3b47d474 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.021s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:13:14 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[b42d8956-3b88-4e7c-a1fd-3b9974ff003d]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:13:14 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-8978a107-f9e6-49c8-9019-39540c368d85 tempest-TestVolumeSwap-1089931716 tempest-TestVolumeSwap-1089931716-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:13:14 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:4ed484cc-9de2-42df-8d99-637f3b47d474 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v mBlevMXQ {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:13:14 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:4ed484cc-9de2-42df-8d99-637f3b47d474 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v mBlevMXQ" returned: 0 in 0.019s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:13:14 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[6d5ce88a-9d40-49e4-942b-67f2e50770ac]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:13:14 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-8978a107-f9e6-49c8-9019-39540c368d85 tempest-TestVolumeSwap-1089931716 tempest-TestVolumeSwap-1089931716-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'mBlevMXQ'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:13:14 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:4ed484cc-9de2-42df-8d99-637f3b47d474 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:13:14 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:4ed484cc-9de2-42df-8d99-637f3b47d474 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.018s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:13:14 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[53e9cbaf-aca5-49b9-b7d3-ecc838cb72e0]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:13:14 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-8978a107-f9e6-49c8-9019-39540c368d85 tempest-TestVolumeSwap-1089931716 tempest-TestVolumeSwap-1089931716-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:13:14 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:13:14 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.020s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:13:14 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[df29a9c1-b830-48b7-a77f-cac896e6ff13]: (4, ('tcp: [20] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b (non-flash)\ntcp: [21] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:56237f4c-4138-4f50-aafb-9c0f9d4bb1cf (non-flash)\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:13:14 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-8978a107-f9e6-49c8-9019-39540c368d85 tempest-TestVolumeSwap-1089931716 tempest-TestVolumeSwap-1089931716-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [20] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b (non-flash)
Nov 29 03:13:14 user nova-compute[75234]: tcp: [21] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:56237f4c-4138-4f50-aafb-9c0f9d4bb1cf (non-flash)
Nov 29 03:13:14 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:13:14 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-8978a107-f9e6-49c8-9019-39540c368d85 tempest-TestVolumeSwap-1089931716 tempest-TestVolumeSwap-1089931716-project-member] iscsi session list stdout=tcp: [20] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b (non-flash)
Nov 29 03:13:14 user nova-compute[75234]: tcp: [21] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:56237f4c-4138-4f50-aafb-9c0f9d4bb1cf (non-flash)
Nov 29 03:13:14 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsi_session /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Nov 29 03:13:14 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:4ed484cc-9de2-42df-8d99-637f3b47d474 -p 172.16.0.220:3260 --login {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:13:14 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-2ba9d3c2-d9fa-4fb7-9115-db6c314836c5 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:13:14 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-2ba9d3c2-d9fa-4fb7-9115-db6c314836c5 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:13:14 user nova-compute[75234]: DEBUG nova.objects.instance [None req-2ba9d3c2-d9fa-4fb7-9115-db6c314836c5 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Lazy-loading 'pci_requests' on Instance uuid 062753a6-8f39-4e80-a549-9f0f5a4b877a {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:13:14 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:4ed484cc-9de2-42df-8d99-637f3b47d474 -p 172.16.0.220:3260 --login" returned: 0 in 0.060s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:13:14 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[62e41edc-97c7-4a3b-bb77-71e09b9ab78f]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:4ed484cc-9de2-42df-8d99-637f3b47d474, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:4ed484cc-9de2-42df-8d99-637f3b47d474, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:13:14 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-8978a107-f9e6-49c8-9019-39540c368d85 tempest-TestVolumeSwap-1089931716 tempest-TestVolumeSwap-1089931716-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:4ed484cc-9de2-42df-8d99-637f3b47d474, portal: 172.16.0.220,3260]
Nov 29 03:13:14 user nova-compute[75234]: Login to [iface: default, target: iqn.2010-10.org.openstack:4ed484cc-9de2-42df-8d99-637f3b47d474, portal: 172.16.0.220,3260] successful.
Nov 29 03:13:14 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:13:14 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:4ed484cc-9de2-42df-8d99-637f3b47d474 -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:13:14 user nova-compute[75234]: DEBUG nova.objects.instance [None req-2ba9d3c2-d9fa-4fb7-9115-db6c314836c5 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Lazy-loading 'numa_topology' on Instance uuid 062753a6-8f39-4e80-a549-9f0f5a4b877a {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:13:14 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-2ba9d3c2-d9fa-4fb7-9115-db6c314836c5 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=75234) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2366}}
Nov 29 03:13:14 user nova-compute[75234]: INFO nova.compute.claims [None req-2ba9d3c2-d9fa-4fb7-9115-db6c314836c5 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] [instance: 062753a6-8f39-4e80-a549-9f0f5a4b877a] Claim successful on node user
Nov 29 03:13:14 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:4ed484cc-9de2-42df-8d99-637f3b47d474 -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.023s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:13:14 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[bf14db6d-61a9-4017-aac9-20535a263c4f]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:13:14 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-8978a107-f9e6-49c8-9019-39540c368d85 tempest-TestVolumeSwap-1089931716 tempest-TestVolumeSwap-1089931716-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:13:14 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:13:14 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.024s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:13:14 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[48667c85-3776-4e4c-86fd-3cb33fb4f052]: (4, ('tcp: [20] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b (non-flash)\ntcp: [21] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:56237f4c-4138-4f50-aafb-9c0f9d4bb1cf (non-flash)\ntcp: [22] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4ed484cc-9de2-42df-8d99-637f3b47d474 (non-flash)\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:13:14 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-8978a107-f9e6-49c8-9019-39540c368d85 tempest-TestVolumeSwap-1089931716 tempest-TestVolumeSwap-1089931716-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [20] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b (non-flash)
Nov 29 03:13:14 user nova-compute[75234]: tcp: [21] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:56237f4c-4138-4f50-aafb-9c0f9d4bb1cf (non-flash)
Nov 29 03:13:14 user nova-compute[75234]: tcp: [22] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4ed484cc-9de2-42df-8d99-637f3b47d474 (non-flash)
Nov 29 03:13:14 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:13:14 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-8978a107-f9e6-49c8-9019-39540c368d85 tempest-TestVolumeSwap-1089931716 tempest-TestVolumeSwap-1089931716-project-member] iscsi session list stdout=tcp: [20] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b (non-flash)
Nov 29 03:13:14 user nova-compute[75234]: tcp: [21] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:56237f4c-4138-4f50-aafb-9c0f9d4bb1cf (non-flash)
Nov 29 03:13:14 user nova-compute[75234]: tcp: [22] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4ed484cc-9de2-42df-8d99-637f3b47d474 (non-flash)
Nov 29 03:13:14 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsi_session /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Nov 29 03:13:14 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-8978a107-f9e6-49c8-9019-39540c368d85 tempest-TestVolumeSwap-1089931716 tempest-TestVolumeSwap-1089931716-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:4ed484cc-9de2-42df-8d99-637f3b47d474" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.279s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:87}}
Nov 29 03:13:14 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-8978a107-f9e6-49c8-9019-39540c368d85 tempest-TestVolumeSwap-1089931716 tempest-TestVolumeSwap-1089931716-project-member] Connected to 172.16.0.220:3260 {{(pid=75234) _connect_vol /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:644}}
Nov 29 03:13:14 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-8978a107-f9e6-49c8-9019-39540c368d85 tempest-TestVolumeSwap-1089931716 tempest-TestVolumeSwap-1089931716-project-member] HCTL ('33', '-', '-', 0) found on session 22 with lun 0 {{(pid=75234) get_hctl /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:770}}
Nov 29 03:13:14 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-8978a107-f9e6-49c8-9019-39540c368d85 tempest-TestVolumeSwap-1089931716 tempest-TestVolumeSwap-1089931716-project-member] Scanning host 33 c: -, t: -, l: 0) {{(pid=75234) scan_iscsi /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:795}}
Nov 29 03:13:14 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host33/scan {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:13:14 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host33/scan" returned: 0 in 0.017s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:13:14 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[6a549d99-86f6-4a2b-aabd-ad8c3df74991]: (4, ('- - 0', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:13:14 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-8978a107-f9e6-49c8-9019-39540c368d85 tempest-TestVolumeSwap-1089931716 tempest-TestVolumeSwap-1089931716-project-member] Searching for a device in session 22 and hctl ['33', '*', '*', 0] yield: sdb {{(pid=75234) device_name_by_hctl /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:789}}
Nov 29 03:13:14 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-8978a107-f9e6-49c8-9019-39540c368d85 tempest-TestVolumeSwap-1089931716 tempest-TestVolumeSwap-1089931716-project-member] Connected to sdb using {'target_iqn': 'iqn.2010-10.org.openstack:4ed484cc-9de2-42df-8d99-637f3b47d474', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'mBlevMXQ', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False} {{(pid=75234) _connect_vol /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:672}}
Nov 29 03:13:15 user nova-compute[75234]: DEBUG nova.compute.provider_tree [None req-2ba9d3c2-d9fa-4fb7-9115-db6c314836c5 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Inventory has not changed in ProviderTree for provider: 4d291220-7c37-4839-a0e3-b23c66502b74 {{(pid=75234) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Nov 29 03:13:15 user nova-compute[75234]: DEBUG nova.scheduler.client.report [None req-2ba9d3c2-d9fa-4fb7-9115-db6c314836c5 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Inventory has not changed for provider 4d291220-7c37-4839-a0e3-b23c66502b74 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=75234) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Nov 29 03:13:15 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-2ba9d3c2-d9fa-4fb7-9115-db6c314836c5 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.501s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:13:15 user nova-compute[75234]: INFO nova.network.neutron [None req-2ba9d3c2-d9fa-4fb7-9115-db6c314836c5 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] [instance: 062753a6-8f39-4e80-a549-9f0f5a4b877a] Updating port 8f6751f1-fc65-4842-b77f-1dfe26912afb with attributes {'binding:host_id': 'user', 'device_owner': 'compute:nova'}
Nov 29 03:13:15 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:13:15 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:13:15 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-8978a107-f9e6-49c8-9019-39540c368d85 tempest-TestVolumeSwap-1089931716 tempest-TestVolumeSwap-1089931716-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.315s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:87}}
Nov 29 03:13:15 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-8978a107-f9e6-49c8-9019-39540c368d85 tempest-TestVolumeSwap-1089931716 tempest-TestVolumeSwap-1089931716-project-member] <== connect_volume: return (1317ms) {'type': 'block', 'scsi_wwn': '23338363665343038', 'path': '/dev/sdb'} {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:203}}
Nov 29 03:13:15 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-8978a107-f9e6-49c8-9019-39540c368d85 tempest-TestVolumeSwap-1089931716 tempest-TestVolumeSwap-1089931716-project-member] [instance: 3e355ae8-aab9-4ae9-b6e3-6de108ec1af9] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '23338363665343038', 'path': '/dev/sdb'} {{(pid=75234) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Nov 29 03:13:15 user nova-compute[75234]: DEBUG nova.objects.instance [None req-8978a107-f9e6-49c8-9019-39540c368d85 tempest-TestVolumeSwap-1089931716 tempest-TestVolumeSwap-1089931716-project-member] Lazy-loading 'flavor' on Instance uuid 3e355ae8-aab9-4ae9-b6e3-6de108ec1af9 {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:13:15 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-2ba9d3c2-d9fa-4fb7-9115-db6c314836c5 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Acquiring lock "refresh_cache-062753a6-8f39-4e80-a549-9f0f5a4b877a" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:13:15 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-2ba9d3c2-d9fa-4fb7-9115-db6c314836c5 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Acquired lock "refresh_cache-062753a6-8f39-4e80-a549-9f0f5a4b877a" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:13:15 user nova-compute[75234]: DEBUG nova.network.neutron [None req-2ba9d3c2-d9fa-4fb7-9115-db6c314836c5 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] [instance: 062753a6-8f39-4e80-a549-9f0f5a4b877a] Building network info cache for instance {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2009}}
Nov 29 03:13:15 user nova-compute[75234]: DEBUG nova.virt.libvirt.guest [None req-8978a107-f9e6-49c8-9019-39540c368d85 tempest-TestVolumeSwap-1089931716 tempest-TestVolumeSwap-1089931716-project-member] attach device xml: <disk type="block" device="disk">
Nov 29 03:13:15 user nova-compute[75234]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Nov 29 03:13:15 user nova-compute[75234]:   <source dev="/dev/sdb"/>
Nov 29 03:13:15 user nova-compute[75234]:   <target dev="vdb" bus="virtio"/>
Nov 29 03:13:15 user nova-compute[75234]:   <serial>4ed484cc-9de2-42df-8d99-637f3b47d474</serial>
Nov 29 03:13:15 user nova-compute[75234]: </disk>
Nov 29 03:13:15 user nova-compute[75234]:  {{(pid=75234) attach_device /opt/stack/nova/nova/virt/libvirt/guest.py:339}}
Nov 29 03:13:15 user nova-compute[75234]: DEBUG nova.compute.manager [req-9bf51793-9203-4f11-8ecb-2563c4f6fda0 req-73315d62-17b0-40bc-853a-cee117809fd0 service nova] [instance: 062753a6-8f39-4e80-a549-9f0f5a4b877a] Received event network-changed-8f6751f1-fc65-4842-b77f-1dfe26912afb {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:13:15 user nova-compute[75234]: DEBUG nova.compute.manager [req-9bf51793-9203-4f11-8ecb-2563c4f6fda0 req-73315d62-17b0-40bc-853a-cee117809fd0 service nova] [instance: 062753a6-8f39-4e80-a549-9f0f5a4b877a] Refreshing instance network info cache due to event network-changed-8f6751f1-fc65-4842-b77f-1dfe26912afb. {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11117}}
Nov 29 03:13:15 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-9bf51793-9203-4f11-8ecb-2563c4f6fda0 req-73315d62-17b0-40bc-853a-cee117809fd0 service nova] Acquiring lock "refresh_cache-062753a6-8f39-4e80-a549-9f0f5a4b877a" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:13:16 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-8978a107-f9e6-49c8-9019-39540c368d85 tempest-TestVolumeSwap-1089931716 tempest-TestVolumeSwap-1089931716-project-member] No BDM found with device name vda, not building metadata. {{(pid=75234) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12089}}
Nov 29 03:13:16 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-8978a107-f9e6-49c8-9019-39540c368d85 tempest-TestVolumeSwap-1089931716 tempest-TestVolumeSwap-1089931716-project-member] No BDM found with device name vdb, not building metadata. {{(pid=75234) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12089}}
Nov 29 03:13:16 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-8978a107-f9e6-49c8-9019-39540c368d85 tempest-TestVolumeSwap-1089931716 tempest-TestVolumeSwap-1089931716-project-member] No VIF found with MAC fa:16:3e:db:81:b1, not building metadata {{(pid=75234) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Nov 29 03:13:16 user nova-compute[75234]: DEBUG nova.network.neutron [None req-2ba9d3c2-d9fa-4fb7-9115-db6c314836c5 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] [instance: 062753a6-8f39-4e80-a549-9f0f5a4b877a] Updating instance_info_cache with network_info: [{"id": "8f6751f1-fc65-4842-b77f-1dfe26912afb", "address": "fa:16:3e:62:4a:5a", "network": {"id": "aad085f7-106b-4de6-93f5-f86bb66ec962", "bridge": "br-int", "label": "tempest-AttachVolumeShelveTestJSON-775984268-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.175", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "e17150005a9a449f9373b91e1c00083b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap8f6751f1-fc", "ovs_interfaceid": "8f6751f1-fc65-4842-b77f-1dfe26912afb", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:13:16 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-2ba9d3c2-d9fa-4fb7-9115-db6c314836c5 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Releasing lock "refresh_cache-062753a6-8f39-4e80-a549-9f0f5a4b877a" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:13:16 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-2ba9d3c2-d9fa-4fb7-9115-db6c314836c5 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] [instance: 062753a6-8f39-4e80-a549-9f0f5a4b877a] Creating instance directory {{(pid=75234) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4727}}
Nov 29 03:13:16 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-2ba9d3c2-d9fa-4fb7-9115-db6c314836c5 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] [instance: 062753a6-8f39-4e80-a549-9f0f5a4b877a] Creating image(s)
Nov 29 03:13:16 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-2ba9d3c2-d9fa-4fb7-9115-db6c314836c5 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Acquiring lock "/opt/stack/data/nova/instances/062753a6-8f39-4e80-a549-9f0f5a4b877a/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:13:16 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-2ba9d3c2-d9fa-4fb7-9115-db6c314836c5 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Lock "/opt/stack/data/nova/instances/062753a6-8f39-4e80-a549-9f0f5a4b877a/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:13:16 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-2ba9d3c2-d9fa-4fb7-9115-db6c314836c5 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Lock "/opt/stack/data/nova/instances/062753a6-8f39-4e80-a549-9f0f5a4b877a/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.005s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:13:16 user nova-compute[75234]: DEBUG nova.objects.instance [None req-2ba9d3c2-d9fa-4fb7-9115-db6c314836c5 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Lazy-loading 'trusted_certs' on Instance uuid 062753a6-8f39-4e80-a549-9f0f5a4b877a {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:13:16 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-9bf51793-9203-4f11-8ecb-2563c4f6fda0 req-73315d62-17b0-40bc-853a-cee117809fd0 service nova] Acquired lock "refresh_cache-062753a6-8f39-4e80-a549-9f0f5a4b877a" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:13:16 user nova-compute[75234]: DEBUG nova.network.neutron [req-9bf51793-9203-4f11-8ecb-2563c4f6fda0 req-73315d62-17b0-40bc-853a-cee117809fd0 service nova] [instance: 062753a6-8f39-4e80-a549-9f0f5a4b877a] Refreshing network info cache for port 8f6751f1-fc65-4842-b77f-1dfe26912afb {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2006}}
Nov 29 03:13:16 user nova-compute[75234]: DEBUG nova.objects.base [None req-2ba9d3c2-d9fa-4fb7-9115-db6c314836c5 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Object Instance<062753a6-8f39-4e80-a549-9f0f5a4b877a> lazy-loaded attributes: pci_requests,trusted_certs {{(pid=75234) wrapper /opt/stack/nova/nova/objects/base.py:126}}
Nov 29 03:13:16 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-2ba9d3c2-d9fa-4fb7-9115-db6c314836c5 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Acquiring lock "3d36fa259f782d8fe9840ba00e3e31df800d42d3" by "nova.virt.libvirt.imagebackend.Image.cache.<locals>.fetch_func_sync" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:13:16 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-2ba9d3c2-d9fa-4fb7-9115-db6c314836c5 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Lock "3d36fa259f782d8fe9840ba00e3e31df800d42d3" acquired by "nova.virt.libvirt.imagebackend.Image.cache.<locals>.fetch_func_sync" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:13:16 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-8978a107-f9e6-49c8-9019-39540c368d85 tempest-TestVolumeSwap-1089931716 tempest-TestVolumeSwap-1089931716-project-member] Lock "3e355ae8-aab9-4ae9-b6e3-6de108ec1af9" "released" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: held 17.629s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:13:17 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-2ba9d3c2-d9fa-4fb7-9115-db6c314836c5 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/3d36fa259f782d8fe9840ba00e3e31df800d42d3.part --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:13:17 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:13:17 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-371aaf1d-7bb3-4224-8cba-252fdf4d9e7c tempest-TestVolumeSwap-1761484429 tempest-TestVolumeSwap-1761484429-project-admin] Acquiring lock "3e355ae8-aab9-4ae9-b6e3-6de108ec1af9" by "nova.compute.manager.ComputeManager.swap_volume.<locals>._do_locked_swap_volume" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:13:17 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-371aaf1d-7bb3-4224-8cba-252fdf4d9e7c tempest-TestVolumeSwap-1761484429 tempest-TestVolumeSwap-1761484429-project-admin] Lock "3e355ae8-aab9-4ae9-b6e3-6de108ec1af9" acquired by "nova.compute.manager.ComputeManager.swap_volume.<locals>._do_locked_swap_volume" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:13:17 user nova-compute[75234]: DEBUG os_brick.utils [None req-371aaf1d-7bb3-4224-8cba-252fdf4d9e7c tempest-TestVolumeSwap-1761484429 tempest-TestVolumeSwap-1761484429-project-admin] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:176}}
Nov 29 03:13:17 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:13:17 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-2ba9d3c2-d9fa-4fb7-9115-db6c314836c5 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/3d36fa259f782d8fe9840ba00e3e31df800d42d3.part --force-share --output=json" returned: 0 in 0.184s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:13:17 user nova-compute[75234]: DEBUG nova.virt.images [None req-2ba9d3c2-d9fa-4fb7-9115-db6c314836c5 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] b14da3aa-0b00-47e2-a40c-9736f7ce03a1 was qcow2, converting to raw {{(pid=75234) fetch_to_raw /opt/stack/nova/nova/virt/images.py:165}}
Nov 29 03:13:17 user nova-compute[75234]: DEBUG nova.privsep.utils [None req-2ba9d3c2-d9fa-4fb7-9115-db6c314836c5 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Path '/opt/stack/data/nova/instances' supports direct I/O {{(pid=75234) supports_direct_io /opt/stack/nova/nova/privsep/utils.py:63}}
Nov 29 03:13:17 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-2ba9d3c2-d9fa-4fb7-9115-db6c314836c5 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Running cmd (subprocess): qemu-img convert -t none -O raw -f qcow2 /opt/stack/data/nova/instances/_base/3d36fa259f782d8fe9840ba00e3e31df800d42d3.part /opt/stack/data/nova/instances/_base/3d36fa259f782d8fe9840ba00e3e31df800d42d3.converted {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:13:17 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.016s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:13:17 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[f54ecd6a-6c5c-4368-b36d-71e2eb975672]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:dd1368c5ba2c\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:13:17 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt -v / -n -o SOURCE {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:13:17 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt -v / -n -o SOURCE" returned: 0 in 0.020s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:13:17 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[c1069270-e255-4c03-a40f-7cceb2c5ab99]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:13:17 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:13:17 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.016s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:13:17 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[a62fd7b4-90f5-4cd6-ab23-e7235fdda226]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:13:17 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[86993cbf-e8fc-4d0b-9c39-6921b8c0b34d]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:13:17 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-371aaf1d-7bb3-4224-8cba-252fdf4d9e7c tempest-TestVolumeSwap-1761484429 tempest-TestVolumeSwap-1761484429-project-admin] Running cmd (subprocess): nvme version {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:13:17 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-371aaf1d-7bb3-4224-8cba-252fdf4d9e7c tempest-TestVolumeSwap-1761484429 tempest-TestVolumeSwap-1761484429-project-admin] 'nvme version' failed. Not Retrying. {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:473}}
Nov 29 03:13:17 user nova-compute[75234]: DEBUG nova.network.neutron [req-9bf51793-9203-4f11-8ecb-2563c4f6fda0 req-73315d62-17b0-40bc-853a-cee117809fd0 service nova] [instance: 062753a6-8f39-4e80-a549-9f0f5a4b877a] Updated VIF entry in instance network info cache for port 8f6751f1-fc65-4842-b77f-1dfe26912afb. {{(pid=75234) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3481}}
Nov 29 03:13:17 user nova-compute[75234]: DEBUG nova.network.neutron [req-9bf51793-9203-4f11-8ecb-2563c4f6fda0 req-73315d62-17b0-40bc-853a-cee117809fd0 service nova] [instance: 062753a6-8f39-4e80-a549-9f0f5a4b877a] Updating instance_info_cache with network_info: [{"id": "8f6751f1-fc65-4842-b77f-1dfe26912afb", "address": "fa:16:3e:62:4a:5a", "network": {"id": "aad085f7-106b-4de6-93f5-f86bb66ec962", "bridge": "br-int", "label": "tempest-AttachVolumeShelveTestJSON-775984268-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.175", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "e17150005a9a449f9373b91e1c00083b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap8f6751f1-fc", "ovs_interfaceid": "8f6751f1-fc65-4842-b77f-1dfe26912afb", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:13:17 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.nvmeof [None req-371aaf1d-7bb3-4224-8cba-252fdf4d9e7c tempest-TestVolumeSwap-1761484429 tempest-TestVolumeSwap-1761484429-project-admin] nvme not present on system {{(pid=75234) nvme_present /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/nvmeof.py:756}}
Nov 29 03:13:17 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:13:17 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-9bf51793-9203-4f11-8ecb-2563c4f6fda0 req-73315d62-17b0-40bc-853a-cee117809fd0 service nova] Releasing lock "refresh_cache-062753a6-8f39-4e80-a549-9f0f5a4b877a" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:13:17 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:473}}
Nov 29 03:13:17 user nova-compute[75234]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Nov 29 03:13:17 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[9d5881d2-59b7-4fbc-9ee8-d7269ca09501]: (4, '') {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:13:18 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.lightos [None req-371aaf1d-7bb3-4224-8cba-252fdf4d9e7c tempest-TestVolumeSwap-1761484429 tempest-TestVolumeSwap-1761484429-project-admin] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=75234) find_dsc /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/lightos.py:98}}
Nov 29 03:13:18 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.lightos [None req-371aaf1d-7bb3-4224-8cba-252fdf4d9e7c tempest-TestVolumeSwap-1761484429 tempest-TestVolumeSwap-1761484429-project-admin] LIGHTOS: did not find dsc, continuing anyway. {{(pid=75234) get_connector_properties /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/lightos.py:76}}
Nov 29 03:13:18 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.lightos [None req-371aaf1d-7bb3-4224-8cba-252fdf4d9e7c tempest-TestVolumeSwap-1761484429 tempest-TestVolumeSwap-1761484429-project-admin] LIGHTOS: no hostnqn found. {{(pid=75234) get_connector_properties /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/lightos.py:84}}
Nov 29 03:13:18 user nova-compute[75234]: DEBUG os_brick.utils [None req-371aaf1d-7bb3-4224-8cba-252fdf4d9e7c tempest-TestVolumeSwap-1761484429 tempest-TestVolumeSwap-1761484429-project-admin] <== get_connector_properties: return (356ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:dd1368c5ba2c', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:203}}
Nov 29 03:13:18 user nova-compute[75234]: INFO nova.compute.manager [None req-371aaf1d-7bb3-4224-8cba-252fdf4d9e7c tempest-TestVolumeSwap-1761484429 tempest-TestVolumeSwap-1761484429-project-admin] [instance: 3e355ae8-aab9-4ae9-b6e3-6de108ec1af9] Swapping volume 4ed484cc-9de2-42df-8d99-637f3b47d474 for 94dadf41-624e-4581-81cf-eb5eaaecafb2
Nov 29 03:13:18 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-2ba9d3c2-d9fa-4fb7-9115-db6c314836c5 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] CMD "qemu-img convert -t none -O raw -f qcow2 /opt/stack/data/nova/instances/_base/3d36fa259f782d8fe9840ba00e3e31df800d42d3.part /opt/stack/data/nova/instances/_base/3d36fa259f782d8fe9840ba00e3e31df800d42d3.converted" returned: 0 in 0.929s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:13:18 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-2ba9d3c2-d9fa-4fb7-9115-db6c314836c5 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/3d36fa259f782d8fe9840ba00e3e31df800d42d3.converted --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:13:18 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-2ba9d3c2-d9fa-4fb7-9115-db6c314836c5 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/3d36fa259f782d8fe9840ba00e3e31df800d42d3.converted --force-share --output=json" returned: 0 in 0.183s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:13:18 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-2ba9d3c2-d9fa-4fb7-9115-db6c314836c5 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Lock "3d36fa259f782d8fe9840ba00e3e31df800d42d3" "released" by "nova.virt.libvirt.imagebackend.Image.cache.<locals>.fetch_func_sync" :: held 2.611s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:13:18 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-2ba9d3c2-d9fa-4fb7-9115-db6c314836c5 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/3d36fa259f782d8fe9840ba00e3e31df800d42d3 --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:13:19 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-2ba9d3c2-d9fa-4fb7-9115-db6c314836c5 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/3d36fa259f782d8fe9840ba00e3e31df800d42d3 --force-share --output=json" returned: 0 in 0.192s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:13:19 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-2ba9d3c2-d9fa-4fb7-9115-db6c314836c5 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Acquiring lock "3d36fa259f782d8fe9840ba00e3e31df800d42d3" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:13:19 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-2ba9d3c2-d9fa-4fb7-9115-db6c314836c5 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Lock "3d36fa259f782d8fe9840ba00e3e31df800d42d3" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.002s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:13:19 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-2ba9d3c2-d9fa-4fb7-9115-db6c314836c5 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/3d36fa259f782d8fe9840ba00e3e31df800d42d3 --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:13:19 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-2ba9d3c2-d9fa-4fb7-9115-db6c314836c5 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/3d36fa259f782d8fe9840ba00e3e31df800d42d3 --force-share --output=json" returned: 0 in 0.188s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:13:19 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-2ba9d3c2-d9fa-4fb7-9115-db6c314836c5 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/3d36fa259f782d8fe9840ba00e3e31df800d42d3,backing_fmt=raw /opt/stack/data/nova/instances/062753a6-8f39-4e80-a549-9f0f5a4b877a/disk 1073741824 {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:13:19 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-2ba9d3c2-d9fa-4fb7-9115-db6c314836c5 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/3d36fa259f782d8fe9840ba00e3e31df800d42d3,backing_fmt=raw /opt/stack/data/nova/instances/062753a6-8f39-4e80-a549-9f0f5a4b877a/disk 1073741824" returned: 0 in 0.051s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:13:19 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-2ba9d3c2-d9fa-4fb7-9115-db6c314836c5 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Lock "3d36fa259f782d8fe9840ba00e3e31df800d42d3" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.245s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:13:19 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-2ba9d3c2-d9fa-4fb7-9115-db6c314836c5 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/3d36fa259f782d8fe9840ba00e3e31df800d42d3 --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:13:19 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-2ba9d3c2-d9fa-4fb7-9115-db6c314836c5 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/3d36fa259f782d8fe9840ba00e3e31df800d42d3 --force-share --output=json" returned: 0 in 0.164s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:13:19 user nova-compute[75234]: DEBUG nova.objects.instance [None req-2ba9d3c2-d9fa-4fb7-9115-db6c314836c5 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Lazy-loading 'migration_context' on Instance uuid 062753a6-8f39-4e80-a549-9f0f5a4b877a {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:13:19 user nova-compute[75234]: DEBUG nova.objects.base [None req-2ba9d3c2-d9fa-4fb7-9115-db6c314836c5 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Object Instance<062753a6-8f39-4e80-a549-9f0f5a4b877a> lazy-loaded attributes: pci_requests,trusted_certs,migration_context {{(pid=75234) wrapper /opt/stack/nova/nova/objects/base.py:126}}
Nov 29 03:13:19 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-2ba9d3c2-d9fa-4fb7-9115-db6c314836c5 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] [instance: 062753a6-8f39-4e80-a549-9f0f5a4b877a] Rebasing disk image.
Nov 29 03:13:19 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-2ba9d3c2-d9fa-4fb7-9115-db6c314836c5 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:13:19 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-2ba9d3c2-d9fa-4fb7-9115-db6c314836c5 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json" returned: 0 in 0.174s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:13:19 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-2ba9d3c2-d9fa-4fb7-9115-db6c314836c5 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Running cmd (subprocess): qemu-img rebase -b /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 -F raw /opt/stack/data/nova/instances/062753a6-8f39-4e80-a549-9f0f5a4b877a/disk {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:13:20 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:13:21 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:13:22 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-2ba9d3c2-d9fa-4fb7-9115-db6c314836c5 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] CMD "qemu-img rebase -b /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 -F raw /opt/stack/data/nova/instances/062753a6-8f39-4e80-a549-9f0f5a4b877a/disk" returned: 0 in 2.418s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:13:22 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-2ba9d3c2-d9fa-4fb7-9115-db6c314836c5 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] [instance: 062753a6-8f39-4e80-a549-9f0f5a4b877a] Created local disks {{(pid=75234) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4861}}
Nov 29 03:13:22 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-2ba9d3c2-d9fa-4fb7-9115-db6c314836c5 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] [instance: 062753a6-8f39-4e80-a549-9f0f5a4b877a] Ensure instance console log exists: /opt/stack/data/nova/instances/062753a6-8f39-4e80-a549-9f0f5a4b877a/console.log {{(pid=75234) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4613}}
Nov 29 03:13:22 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-2ba9d3c2-d9fa-4fb7-9115-db6c314836c5 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:13:22 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-2ba9d3c2-d9fa-4fb7-9115-db6c314836c5 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:13:22 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-2ba9d3c2-d9fa-4fb7-9115-db6c314836c5 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:13:22 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-2ba9d3c2-d9fa-4fb7-9115-db6c314836c5 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] [instance: 062753a6-8f39-4e80-a549-9f0f5a4b877a] Start _get_guest_xml network_info=[{"id": "8f6751f1-fc65-4842-b77f-1dfe26912afb", "address": "fa:16:3e:62:4a:5a", "network": {"id": "aad085f7-106b-4de6-93f5-f86bb66ec962", "bridge": "br-int", "label": "tempest-AttachVolumeShelveTestJSON-775984268-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.175", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "e17150005a9a449f9373b91e1c00083b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap8f6751f1-fc", "ovs_interfaceid": "8f6751f1-fc65-4842-b77f-1dfe26912afb", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, '/dev/vdc': {'bus': 'virtio', 'dev': 'vdc', 'type': 'disk'}}} image_meta=ImageMeta(checksum='090a85e8ac66076f59b348337f1e8450',container_format='bare',created_at=2023-11-29T03:11:08Z,direct_url=<?>,disk_format='qcow2',id=b14da3aa-0b00-47e2-a40c-9736f7ce03a1,min_disk=1,min_ram=0,name='tempest-AttachVolumeShelveTestJSON-server-2002337175-shelved',owner='e17150005a9a449f9373b91e1c00083b',properties=ImageMetaProps,protected=<?>,size=52363264,status='active',tags=<?>,updated_at=2023-11-29T03:11:13Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [{'device_name': '/dev/vda', 'encryption_format': None, 'encrypted': False, 'device_type': 'disk', 'size': 0, 'disk_bus': 'virtio', 'boot_index': 0, 'encryption_options': None, 'encryption_secret_uuid': None, 'guest_format': None, 'image_id': '96c30ba7-9b73-4a30-8ba4-7138879573b7'}], 'ephemerals': [], 'block_device_mapping': [{'mount_device': '/dev/vdc', 'attachment_id': 'a19e46a6-7567-4073-8a06-c636553e5e46', 'disk_bus': 'virtio', 'device_type': 'disk', 'boot_index': None, 'guest_format': None, 'delete_on_termination': False, 'connection_info': {'driver_volume_type': 'iscsi', 'data': {'target_iqn': 'iqn.2010-10.org.openstack:6a4d34d3-9974-43bc-afcb-9169c893d9b7', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'OpBhCUuD', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}, 'status': 'attached', 'instance': '062753a6-8f39-4e80-a549-9f0f5a4b877a', 'attached_at': '', 'detached_at': '', 'volume_id': '6a4d34d3-9974-43bc-afcb-9169c893d9b7', 'serial': '6a4d34d3-9974-43bc-afcb-9169c893d9b7'}, 'volume_type': None} {{(pid=75234) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7562}}
Nov 29 03:13:22 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-2ba9d3c2-d9fa-4fb7-9115-db6c314836c5 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:13:22 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-2ba9d3c2-d9fa-4fb7-9115-db6c314836c5 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:13:22 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-2ba9d3c2-d9fa-4fb7-9115-db6c314836c5 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Searching host: 'user' for CPU controller through CGroups V1... {{(pid=75234) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1652}}
Nov 29 03:13:22 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-2ba9d3c2-d9fa-4fb7-9115-db6c314836c5 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] CPU controller missing on host. {{(pid=75234) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1662}}
Nov 29 03:13:22 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-2ba9d3c2-d9fa-4fb7-9115-db6c314836c5 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Searching host: 'user' for CPU controller through CGroups V2... {{(pid=75234) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1671}}
Nov 29 03:13:22 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-2ba9d3c2-d9fa-4fb7-9115-db6c314836c5 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] CPU controller found on host. {{(pid=75234) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1678}}
Nov 29 03:13:22 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-2ba9d3c2-d9fa-4fb7-9115-db6c314836c5 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=75234) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5400}}
Nov 29 03:13:22 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-2ba9d3c2-d9fa-4fb7-9115-db6c314836c5 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-11-29T03:02:04Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='1',id=6,is_public=True,memory_mb=512,name='m1.tiny',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='090a85e8ac66076f59b348337f1e8450',container_format='bare',created_at=2023-11-29T03:11:08Z,direct_url=<?>,disk_format='qcow2',id=b14da3aa-0b00-47e2-a40c-9736f7ce03a1,min_disk=1,min_ram=0,name='tempest-AttachVolumeShelveTestJSON-server-2002337175-shelved',owner='e17150005a9a449f9373b91e1c00083b',properties=ImageMetaProps,protected=<?>,size=52363264,status='active',tags=<?>,updated_at=2023-11-29T03:11:13Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:563}}
Nov 29 03:13:22 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-2ba9d3c2-d9fa-4fb7-9115-db6c314836c5 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Flavor limits 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:348}}
Nov 29 03:13:22 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-2ba9d3c2-d9fa-4fb7-9115-db6c314836c5 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Image limits 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:352}}
Nov 29 03:13:22 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-2ba9d3c2-d9fa-4fb7-9115-db6c314836c5 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Flavor pref 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:388}}
Nov 29 03:13:22 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-2ba9d3c2-d9fa-4fb7-9115-db6c314836c5 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Image pref 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:392}}
Nov 29 03:13:22 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-2ba9d3c2-d9fa-4fb7-9115-db6c314836c5 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:430}}
Nov 29 03:13:22 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-2ba9d3c2-d9fa-4fb7-9115-db6c314836c5 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:569}}
Nov 29 03:13:22 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-2ba9d3c2-d9fa-4fb7-9115-db6c314836c5 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=75234) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:471}}
Nov 29 03:13:22 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-2ba9d3c2-d9fa-4fb7-9115-db6c314836c5 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Got 1 possible topologies {{(pid=75234) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:501}}
Nov 29 03:13:22 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-2ba9d3c2-d9fa-4fb7-9115-db6c314836c5 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:575}}
Nov 29 03:13:22 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-2ba9d3c2-d9fa-4fb7-9115-db6c314836c5 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:577}}
Nov 29 03:13:22 user nova-compute[75234]: DEBUG nova.objects.instance [None req-2ba9d3c2-d9fa-4fb7-9115-db6c314836c5 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Lazy-loading 'vcpu_model' on Instance uuid 062753a6-8f39-4e80-a549-9f0f5a4b877a {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:13:22 user nova-compute[75234]: DEBUG nova.objects.base [None req-2ba9d3c2-d9fa-4fb7-9115-db6c314836c5 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Object Instance<062753a6-8f39-4e80-a549-9f0f5a4b877a> lazy-loaded attributes: pci_requests,trusted_certs,migration_context,vcpu_model {{(pid=75234) wrapper /opt/stack/nova/nova/objects/base.py:126}}
Nov 29 03:13:22 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-2ba9d3c2-d9fa-4fb7-9115-db6c314836c5 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] [instance: 062753a6-8f39-4e80-a549-9f0f5a4b877a] Calling os-brick to attach iSCSI Volume {{(pid=75234) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Nov 29 03:13:22 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-2ba9d3c2-d9fa-4fb7-9115-db6c314836c5 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f8a5050c9a0>, 'connection_properties': {'target_iqn': 'iqn.2010-10.org.openstack:6a4d34d3-9974-43bc-afcb-9169c893d9b7', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'OpBhCUuD', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:176}}
Nov 29 03:13:22 user nova-compute[75234]: WARNING os_brick.initiator.connectors.base [None req-2ba9d3c2-d9fa-4fb7-9115-db6c314836c5 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Nov 29 03:13:22 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-2ba9d3c2-d9fa-4fb7-9115-db6c314836c5 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:68}}
Nov 29 03:13:22 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-2ba9d3c2-d9fa-4fb7-9115-db6c314836c5 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:73}}
Nov 29 03:13:22 user nova-compute[75234]: WARNING os_brick.initiator.connectors.base [None req-2ba9d3c2-d9fa-4fb7-9115-db6c314836c5 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Nov 29 03:13:22 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-2ba9d3c2-d9fa-4fb7-9115-db6c314836c5 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:6a4d34d3-9974-43bc-afcb-9169c893d9b7" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:68}}
Nov 29 03:13:22 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-2ba9d3c2-d9fa-4fb7-9115-db6c314836c5 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:6a4d34d3-9974-43bc-afcb-9169c893d9b7" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:73}}
Nov 29 03:13:22 user nova-compute[75234]: INFO os_brick.initiator.connectors.iscsi [None req-2ba9d3c2-d9fa-4fb7-9115-db6c314836c5 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Nov 29 03:13:22 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:6a4d34d3-9974-43bc-afcb-9169c893d9b7 -p 172.16.0.220:3260 {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:13:22 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-f1ee3ef1-1771-42fa-a7f7-83bad3ad6a3b tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] Acquiring lock "8297c264-a213-44ee-be46-5c08eabda9f7" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:13:22 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-f1ee3ef1-1771-42fa-a7f7-83bad3ad6a3b tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] Lock "8297c264-a213-44ee-be46-5c08eabda9f7" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:13:22 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-f1ee3ef1-1771-42fa-a7f7-83bad3ad6a3b tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] Acquiring lock "8297c264-a213-44ee-be46-5c08eabda9f7-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:13:22 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-f1ee3ef1-1771-42fa-a7f7-83bad3ad6a3b tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] Lock "8297c264-a213-44ee-be46-5c08eabda9f7-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:13:22 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-f1ee3ef1-1771-42fa-a7f7-83bad3ad6a3b tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] Lock "8297c264-a213-44ee-be46-5c08eabda9f7-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:13:22 user nova-compute[75234]: INFO nova.compute.manager [None req-f1ee3ef1-1771-42fa-a7f7-83bad3ad6a3b tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] [instance: 8297c264-a213-44ee-be46-5c08eabda9f7] Terminating instance
Nov 29 03:13:22 user nova-compute[75234]: DEBUG nova.compute.manager [None req-f1ee3ef1-1771-42fa-a7f7-83bad3ad6a3b tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] [instance: 8297c264-a213-44ee-be46-5c08eabda9f7] Start destroying the instance on the hypervisor. {{(pid=75234) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3140}}
Nov 29 03:13:22 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:6a4d34d3-9974-43bc-afcb-9169c893d9b7 -p 172.16.0.220:3260" returned: 21 in 0.039s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:13:22 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[3a713eb2-494f-4ef6-ae22-450bb35514aa]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:13:22 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-2ba9d3c2-d9fa-4fb7-9115-db6c314836c5 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Nov 29 03:13:22 user nova-compute[75234]:  {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:13:22 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:6a4d34d3-9974-43bc-afcb-9169c893d9b7 -p 172.16.0.220:3260 --interface default --op new {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:13:22 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:6a4d34d3-9974-43bc-afcb-9169c893d9b7 -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.020s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:13:22 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[c6b2edda-de9d-44f2-bdb6-192187fa6c3d]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:6a4d34d3-9974-43bc-afcb-9169c893d9b7] added\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:13:22 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-2ba9d3c2-d9fa-4fb7-9115-db6c314836c5 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:6a4d34d3-9974-43bc-afcb-9169c893d9b7] added
Nov 29 03:13:22 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:13:22 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:6a4d34d3-9974-43bc-afcb-9169c893d9b7 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:13:22 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:6a4d34d3-9974-43bc-afcb-9169c893d9b7 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.021s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:13:22 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[c2edffb5-03d8-4e3d-9b42-61a9cb4a10f1]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:13:22 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-2ba9d3c2-d9fa-4fb7-9115-db6c314836c5 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:13:22 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:6a4d34d3-9974-43bc-afcb-9169c893d9b7 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:13:22 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:6a4d34d3-9974-43bc-afcb-9169c893d9b7 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.030s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:13:22 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[ee996d49-1647-472b-84c7-9ab5f9c407f7]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:13:22 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-2ba9d3c2-d9fa-4fb7-9115-db6c314836c5 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:13:22 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:6a4d34d3-9974-43bc-afcb-9169c893d9b7 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v OpBhCUuD {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:13:22 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:13:22 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:6a4d34d3-9974-43bc-afcb-9169c893d9b7 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v OpBhCUuD" returned: 0 in 0.021s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:13:22 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[a67c7bf4-c38a-4758-8dc3-08faa9da4c57]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:13:22 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-2ba9d3c2-d9fa-4fb7-9115-db6c314836c5 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'OpBhCUuD'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:13:22 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:6a4d34d3-9974-43bc-afcb-9169c893d9b7 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:13:22 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:6a4d34d3-9974-43bc-afcb-9169c893d9b7 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.026s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:13:22 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[b3b2daff-0707-44aa-b969-7377f8a48521]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:13:22 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-2ba9d3c2-d9fa-4fb7-9115-db6c314836c5 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:13:22 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:13:22 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:13:22 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:13:22 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.035s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:13:22 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[45305952-dfaf-4b2f-b84b-e8e011de1065]: (4, ('tcp: [20] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b (non-flash)\ntcp: [21] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:56237f4c-4138-4f50-aafb-9c0f9d4bb1cf (non-flash)\ntcp: [22] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4ed484cc-9de2-42df-8d99-637f3b47d474 (non-flash)\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:13:22 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-2ba9d3c2-d9fa-4fb7-9115-db6c314836c5 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [20] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b (non-flash)
Nov 29 03:13:22 user nova-compute[75234]: tcp: [21] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:56237f4c-4138-4f50-aafb-9c0f9d4bb1cf (non-flash)
Nov 29 03:13:22 user nova-compute[75234]: tcp: [22] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4ed484cc-9de2-42df-8d99-637f3b47d474 (non-flash)
Nov 29 03:13:22 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:13:22 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-2ba9d3c2-d9fa-4fb7-9115-db6c314836c5 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] iscsi session list stdout=tcp: [20] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b (non-flash)
Nov 29 03:13:22 user nova-compute[75234]: tcp: [21] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:56237f4c-4138-4f50-aafb-9c0f9d4bb1cf (non-flash)
Nov 29 03:13:22 user nova-compute[75234]: tcp: [22] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4ed484cc-9de2-42df-8d99-637f3b47d474 (non-flash)
Nov 29 03:13:22 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsi_session /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Nov 29 03:13:22 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:6a4d34d3-9974-43bc-afcb-9169c893d9b7 -p 172.16.0.220:3260 --login {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:13:22 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:13:22 user nova-compute[75234]: DEBUG nova.compute.manager [req-6f9e1ca9-11dc-40a9-b96c-ea690cd5ffb2 req-18ca725d-1b65-4331-b440-3a0599e3c4d3 service nova] [instance: 8297c264-a213-44ee-be46-5c08eabda9f7] Received event network-vif-unplugged-d26334d2-8040-4022-b658-69be1cf6a025 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:13:22 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-6f9e1ca9-11dc-40a9-b96c-ea690cd5ffb2 req-18ca725d-1b65-4331-b440-3a0599e3c4d3 service nova] Acquiring lock "8297c264-a213-44ee-be46-5c08eabda9f7-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:13:22 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-6f9e1ca9-11dc-40a9-b96c-ea690cd5ffb2 req-18ca725d-1b65-4331-b440-3a0599e3c4d3 service nova] Lock "8297c264-a213-44ee-be46-5c08eabda9f7-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:13:22 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-6f9e1ca9-11dc-40a9-b96c-ea690cd5ffb2 req-18ca725d-1b65-4331-b440-3a0599e3c4d3 service nova] Lock "8297c264-a213-44ee-be46-5c08eabda9f7-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:13:22 user nova-compute[75234]: DEBUG nova.compute.manager [req-6f9e1ca9-11dc-40a9-b96c-ea690cd5ffb2 req-18ca725d-1b65-4331-b440-3a0599e3c4d3 service nova] [instance: 8297c264-a213-44ee-be46-5c08eabda9f7] No waiting events found dispatching network-vif-unplugged-d26334d2-8040-4022-b658-69be1cf6a025 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:13:22 user nova-compute[75234]: DEBUG nova.compute.manager [req-6f9e1ca9-11dc-40a9-b96c-ea690cd5ffb2 req-18ca725d-1b65-4331-b440-3a0599e3c4d3 service nova] [instance: 8297c264-a213-44ee-be46-5c08eabda9f7] Received event network-vif-unplugged-d26334d2-8040-4022-b658-69be1cf6a025 for instance with task_state deleting. {{(pid=75234) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10890}}
Nov 29 03:13:23 user nova-compute[75234]: INFO nova.virt.libvirt.driver [-] [instance: 8297c264-a213-44ee-be46-5c08eabda9f7] Instance destroyed successfully.
Nov 29 03:13:23 user nova-compute[75234]: DEBUG nova.objects.instance [None req-f1ee3ef1-1771-42fa-a7f7-83bad3ad6a3b tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] Lazy-loading 'resources' on Instance uuid 8297c264-a213-44ee-be46-5c08eabda9f7 {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:13:23 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-f1ee3ef1-1771-42fa-a7f7-83bad3ad6a3b tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-11-29T03:06:07Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-ServerStableDeviceRescueTest-server-428917867',display_name='tempest-ServerStableDeviceRescueTest-server-428917867',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-serverstabledevicerescuetest-server-428917867',id=6,image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBPAGeJQ8OrWhDv/r5lktd6O/obaXRK7/7VEiU/V0MksOr2sSmn5M84mkyhNPvWnFo3k1J68udxxAQOPj9YvQEkVpNgH7z7grcEsUVqlgMVin9R9cWVCI3LAQJjKBdTQkeA==',key_name='tempest-keypair-1167119775',keypairs=<?>,launch_index=0,launched_at=2023-11-29T03:10:17Z,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='745bdb3c72794155829dc0a69f35b321',ramdisk_id='',reservation_id='r-tt096l76',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-ServerStableDeviceRescueTest-1509102897',owner_user_name='tempest-ServerStableDeviceRescueTest-1509102897-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-11-29T03:12:03Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='566e3dc84bbb40d9badcacbcaec9b987',uuid=8297c264-a213-44ee-be46-5c08eabda9f7,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "d26334d2-8040-4022-b658-69be1cf6a025", "address": "fa:16:3e:5f:9d:5a", "network": {"id": "e0366490-9fe2-4429-88c8-6625128e645d", "bridge": "br-int", "label": "tempest-ServerStableDeviceRescueTest-804259918-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.31", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "745bdb3c72794155829dc0a69f35b321", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapd26334d2-80", "ovs_interfaceid": "d26334d2-8040-4022-b658-69be1cf6a025", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Nov 29 03:13:23 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-f1ee3ef1-1771-42fa-a7f7-83bad3ad6a3b tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] Converting VIF {"id": "d26334d2-8040-4022-b658-69be1cf6a025", "address": "fa:16:3e:5f:9d:5a", "network": {"id": "e0366490-9fe2-4429-88c8-6625128e645d", "bridge": "br-int", "label": "tempest-ServerStableDeviceRescueTest-804259918-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.31", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "745bdb3c72794155829dc0a69f35b321", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapd26334d2-80", "ovs_interfaceid": "d26334d2-8040-4022-b658-69be1cf6a025", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:13:23 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-f1ee3ef1-1771-42fa-a7f7-83bad3ad6a3b tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:5f:9d:5a,bridge_name='br-int',has_traffic_filtering=True,id=d26334d2-8040-4022-b658-69be1cf6a025,network=Network(e0366490-9fe2-4429-88c8-6625128e645d),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapd26334d2-80') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:13:23 user nova-compute[75234]: DEBUG os_vif [None req-f1ee3ef1-1771-42fa-a7f7-83bad3ad6a3b tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] Unplugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:5f:9d:5a,bridge_name='br-int',has_traffic_filtering=True,id=d26334d2-8040-4022-b658-69be1cf6a025,network=Network(e0366490-9fe2-4429-88c8-6625128e645d),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapd26334d2-80') {{(pid=75234) unplug /opt/stack/data/venv/lib/python3.10/site-packages/os_vif/__init__.py:109}}
Nov 29 03:13:23 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:13:23 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tapd26334d2-80, bridge=br-int, if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:13:23 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:13:23 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:13:23 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:13:23 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:13:23 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbDestroyCommand(_result=None, table=QoS, record=b05cf5f1-a00b-4a61-bb4b-b25b1545a2a2) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:13:23 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:13:23 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:13:23 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:13:23 user nova-compute[75234]: INFO os_vif [None req-f1ee3ef1-1771-42fa-a7f7-83bad3ad6a3b tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] Successfully unplugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:5f:9d:5a,bridge_name='br-int',has_traffic_filtering=True,id=d26334d2-8040-4022-b658-69be1cf6a025,network=Network(e0366490-9fe2-4429-88c8-6625128e645d),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapd26334d2-80')
Nov 29 03:13:23 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-f1ee3ef1-1771-42fa-a7f7-83bad3ad6a3b tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] [instance: 8297c264-a213-44ee-be46-5c08eabda9f7] Deleting instance files /opt/stack/data/nova/instances/8297c264-a213-44ee-be46-5c08eabda9f7_del
Nov 29 03:13:23 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-f1ee3ef1-1771-42fa-a7f7-83bad3ad6a3b tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] [instance: 8297c264-a213-44ee-be46-5c08eabda9f7] Deletion of /opt/stack/data/nova/instances/8297c264-a213-44ee-be46-5c08eabda9f7_del complete
Nov 29 03:13:23 user nova-compute[75234]: INFO nova.compute.manager [None req-f1ee3ef1-1771-42fa-a7f7-83bad3ad6a3b tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] [instance: 8297c264-a213-44ee-be46-5c08eabda9f7] Took 1.00 seconds to destroy the instance on the hypervisor.
Nov 29 03:13:23 user nova-compute[75234]: DEBUG oslo.service.loopingcall [None req-f1ee3ef1-1771-42fa-a7f7-83bad3ad6a3b tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=75234) func /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/loopingcall.py:435}}
Nov 29 03:13:23 user nova-compute[75234]: DEBUG nova.compute.manager [-] [instance: 8297c264-a213-44ee-be46-5c08eabda9f7] Deallocating network for instance {{(pid=75234) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2279}}
Nov 29 03:13:23 user nova-compute[75234]: DEBUG nova.network.neutron [-] [instance: 8297c264-a213-44ee-be46-5c08eabda9f7] deallocate_for_instance() {{(pid=75234) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1802}}
Nov 29 03:13:23 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:6a4d34d3-9974-43bc-afcb-9169c893d9b7 -p 172.16.0.220:3260 --login" returned: 0 in 1.075s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:13:23 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[8a3e01d9-dd2f-4ff9-b13f-0b50e90f5728]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:6a4d34d3-9974-43bc-afcb-9169c893d9b7, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:6a4d34d3-9974-43bc-afcb-9169c893d9b7, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:13:23 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-2ba9d3c2-d9fa-4fb7-9115-db6c314836c5 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:6a4d34d3-9974-43bc-afcb-9169c893d9b7, portal: 172.16.0.220,3260]
Nov 29 03:13:23 user nova-compute[75234]: Login to [iface: default, target: iqn.2010-10.org.openstack:6a4d34d3-9974-43bc-afcb-9169c893d9b7, portal: 172.16.0.220,3260] successful.
Nov 29 03:13:23 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:13:23 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:6a4d34d3-9974-43bc-afcb-9169c893d9b7 -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:13:23 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:6a4d34d3-9974-43bc-afcb-9169c893d9b7 -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.022s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:13:23 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[097a32b3-03e9-4a14-be17-e480c2eb55c9]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:13:23 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-2ba9d3c2-d9fa-4fb7-9115-db6c314836c5 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:13:23 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:13:23 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.025s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:13:23 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[dcca4111-8275-44fc-960e-5eb72dd38399]: (4, ('tcp: [20] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b (non-flash)\ntcp: [21] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:56237f4c-4138-4f50-aafb-9c0f9d4bb1cf (non-flash)\ntcp: [22] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4ed484cc-9de2-42df-8d99-637f3b47d474 (non-flash)\ntcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6a4d34d3-9974-43bc-afcb-9169c893d9b7 (non-flash)\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:13:23 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-2ba9d3c2-d9fa-4fb7-9115-db6c314836c5 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [20] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b (non-flash)
Nov 29 03:13:23 user nova-compute[75234]: tcp: [21] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:56237f4c-4138-4f50-aafb-9c0f9d4bb1cf (non-flash)
Nov 29 03:13:23 user nova-compute[75234]: tcp: [22] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4ed484cc-9de2-42df-8d99-637f3b47d474 (non-flash)
Nov 29 03:13:23 user nova-compute[75234]: tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6a4d34d3-9974-43bc-afcb-9169c893d9b7 (non-flash)
Nov 29 03:13:23 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:13:23 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-2ba9d3c2-d9fa-4fb7-9115-db6c314836c5 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] iscsi session list stdout=tcp: [20] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b (non-flash)
Nov 29 03:13:23 user nova-compute[75234]: tcp: [21] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:56237f4c-4138-4f50-aafb-9c0f9d4bb1cf (non-flash)
Nov 29 03:13:23 user nova-compute[75234]: tcp: [22] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4ed484cc-9de2-42df-8d99-637f3b47d474 (non-flash)
Nov 29 03:13:23 user nova-compute[75234]: tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6a4d34d3-9974-43bc-afcb-9169c893d9b7 (non-flash)
Nov 29 03:13:23 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsi_session /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Nov 29 03:13:23 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-2ba9d3c2-d9fa-4fb7-9115-db6c314836c5 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:6a4d34d3-9974-43bc-afcb-9169c893d9b7" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 1.366s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:87}}
Nov 29 03:13:23 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-2ba9d3c2-d9fa-4fb7-9115-db6c314836c5 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Connected to 172.16.0.220:3260 {{(pid=75234) _connect_vol /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:644}}
Nov 29 03:13:23 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-2ba9d3c2-d9fa-4fb7-9115-db6c314836c5 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] HCTL ('35', '-', '-', 0) found on session 23 with lun 0 {{(pid=75234) get_hctl /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:770}}
Nov 29 03:13:23 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-2ba9d3c2-d9fa-4fb7-9115-db6c314836c5 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Scanning host 35 c: -, t: -, l: 0) {{(pid=75234) scan_iscsi /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:795}}
Nov 29 03:13:23 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host35/scan {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:13:23 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host35/scan" returned: 0 in 0.029s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:13:23 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[0650a424-8111-4e4e-aef0-cecee910242f]: (4, ('- - 0', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:13:23 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-2ba9d3c2-d9fa-4fb7-9115-db6c314836c5 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Searching for a device in session 23 and hctl ['35', '*', '*', 0] yield: sdd {{(pid=75234) device_name_by_hctl /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:789}}
Nov 29 03:13:23 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-2ba9d3c2-d9fa-4fb7-9115-db6c314836c5 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Connected to sdd using {'target_iqn': 'iqn.2010-10.org.openstack:6a4d34d3-9974-43bc-afcb-9169c893d9b7', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'OpBhCUuD', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False} {{(pid=75234) _connect_vol /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:672}}
Nov 29 03:13:23 user nova-compute[75234]: DEBUG nova.network.neutron [-] [instance: 8297c264-a213-44ee-be46-5c08eabda9f7] Updating instance_info_cache with network_info: [] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:13:24 user nova-compute[75234]: INFO nova.compute.manager [-] [instance: 8297c264-a213-44ee-be46-5c08eabda9f7] Took 0.85 seconds to deallocate network for instance.
Nov 29 03:13:24 user nova-compute[75234]: DEBUG nova.compute.manager [req-0fcb782c-2b6b-481b-bc79-dd0481b5aa8f req-ee0dbe4b-e950-4922-9b59-137475b394a3 service nova] [instance: 8297c264-a213-44ee-be46-5c08eabda9f7] Received event network-vif-deleted-d26334d2-8040-4022-b658-69be1cf6a025 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:13:24 user nova-compute[75234]: INFO nova.compute.manager [req-0fcb782c-2b6b-481b-bc79-dd0481b5aa8f req-ee0dbe4b-e950-4922-9b59-137475b394a3 service nova] [instance: 8297c264-a213-44ee-be46-5c08eabda9f7] Neutron deleted interface d26334d2-8040-4022-b658-69be1cf6a025; detaching it from the instance and deleting it from the info cache
Nov 29 03:13:24 user nova-compute[75234]: DEBUG nova.network.neutron [req-0fcb782c-2b6b-481b-bc79-dd0481b5aa8f req-ee0dbe4b-e950-4922-9b59-137475b394a3 service nova] [instance: 8297c264-a213-44ee-be46-5c08eabda9f7] Updating instance_info_cache with network_info: [] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:13:24 user nova-compute[75234]: DEBUG nova.compute.manager [req-0fcb782c-2b6b-481b-bc79-dd0481b5aa8f req-ee0dbe4b-e950-4922-9b59-137475b394a3 service nova] [instance: 8297c264-a213-44ee-be46-5c08eabda9f7] Detach interface failed, port_id=d26334d2-8040-4022-b658-69be1cf6a025, reason: Instance 8297c264-a213-44ee-be46-5c08eabda9f7 could not be found. {{(pid=75234) _process_instance_vif_deleted_event /opt/stack/nova/nova/compute/manager.py:10946}}
Nov 29 03:13:24 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-f1ee3ef1-1771-42fa-a7f7-83bad3ad6a3b tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:13:24 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-f1ee3ef1-1771-42fa-a7f7-83bad3ad6a3b tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:13:24 user nova-compute[75234]: DEBUG nova.compute.provider_tree [None req-f1ee3ef1-1771-42fa-a7f7-83bad3ad6a3b tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] Inventory has not changed in ProviderTree for provider: 4d291220-7c37-4839-a0e3-b23c66502b74 {{(pid=75234) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Nov 29 03:13:24 user nova-compute[75234]: DEBUG nova.scheduler.client.report [None req-f1ee3ef1-1771-42fa-a7f7-83bad3ad6a3b tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] Inventory has not changed for provider 4d291220-7c37-4839-a0e3-b23c66502b74 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=75234) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Nov 29 03:13:24 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-2ba9d3c2-d9fa-4fb7-9115-db6c314836c5 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 2.410s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:87}}
Nov 29 03:13:24 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-2ba9d3c2-d9fa-4fb7-9115-db6c314836c5 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] <== connect_volume: return (2411ms) {'type': 'block', 'scsi_wwn': '23133336334323534', 'path': '/dev/sdd'} {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:203}}
Nov 29 03:13:24 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-2ba9d3c2-d9fa-4fb7-9115-db6c314836c5 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] [instance: 062753a6-8f39-4e80-a549-9f0f5a4b877a] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '23133336334323534', 'path': '/dev/sdd'} {{(pid=75234) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Nov 29 03:13:24 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-f1ee3ef1-1771-42fa-a7f7-83bad3ad6a3b tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.450s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:13:24 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-2ba9d3c2-d9fa-4fb7-9115-db6c314836c5 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=True,compute_id=1,config_drive='',created_at=2023-11-29T03:09:11Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description=None,display_name='tempest-AttachVolumeShelveTestJSON-server-2002337175',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-attachvolumeshelvetestjson-server-2002337175',id=14,image_ref='b14da3aa-0b00-47e2-a40c-9736f7ce03a1',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data=None,key_name='tempest-keypair-1961974262',keypairs=<?>,launch_index=0,launched_at=2023-11-29T03:09:20Z,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=4,progress=0,project_id='e17150005a9a449f9373b91e1c00083b',ramdisk_id='',reservation_id='r-p3yfidye',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',clean_attempts='1',image_base_image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-AttachVolumeShelveTestJSON-1180535246',owner_user_name='tempest-AttachVolumeShelveTestJSON-1180535246-project-member',shelved_at='2023-11-29T03:11:13.240549',shelved_host='user',shelved_image_id='b14da3aa-0b00-47e2-a40c-9736f7ce03a1'},tags=<?>,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-11-29T03:13:01Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='154f3f4f57514450baf8f22d67768bc9',uuid=062753a6-8f39-4e80-a549-9f0f5a4b877a,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='shelved_offloaded') vif={"id": "8f6751f1-fc65-4842-b77f-1dfe26912afb", "address": "fa:16:3e:62:4a:5a", "network": {"id": "aad085f7-106b-4de6-93f5-f86bb66ec962", "bridge": "br-int", "label": "tempest-AttachVolumeShelveTestJSON-775984268-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.175", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "e17150005a9a449f9373b91e1c00083b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap8f6751f1-fc", "ovs_interfaceid": "8f6751f1-fc65-4842-b77f-1dfe26912afb", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=75234) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Nov 29 03:13:24 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-2ba9d3c2-d9fa-4fb7-9115-db6c314836c5 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Converting VIF {"id": "8f6751f1-fc65-4842-b77f-1dfe26912afb", "address": "fa:16:3e:62:4a:5a", "network": {"id": "aad085f7-106b-4de6-93f5-f86bb66ec962", "bridge": "br-int", "label": "tempest-AttachVolumeShelveTestJSON-775984268-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.175", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "e17150005a9a449f9373b91e1c00083b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap8f6751f1-fc", "ovs_interfaceid": "8f6751f1-fc65-4842-b77f-1dfe26912afb", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:13:24 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-2ba9d3c2-d9fa-4fb7-9115-db6c314836c5 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:62:4a:5a,bridge_name='br-int',has_traffic_filtering=True,id=8f6751f1-fc65-4842-b77f-1dfe26912afb,network=Network(aad085f7-106b-4de6-93f5-f86bb66ec962),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap8f6751f1-fc') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:13:24 user nova-compute[75234]: DEBUG nova.objects.instance [None req-2ba9d3c2-d9fa-4fb7-9115-db6c314836c5 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Lazy-loading 'pci_devices' on Instance uuid 062753a6-8f39-4e80-a549-9f0f5a4b877a {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:13:24 user nova-compute[75234]: DEBUG nova.objects.base [None req-2ba9d3c2-d9fa-4fb7-9115-db6c314836c5 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Object Instance<062753a6-8f39-4e80-a549-9f0f5a4b877a> lazy-loaded attributes: pci_requests,trusted_certs,migration_context,vcpu_model,pci_devices {{(pid=75234) wrapper /opt/stack/nova/nova/objects/base.py:126}}
Nov 29 03:13:24 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-2ba9d3c2-d9fa-4fb7-9115-db6c314836c5 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] [instance: 062753a6-8f39-4e80-a549-9f0f5a4b877a] End _get_guest_xml xml=<domain type="kvm">
Nov 29 03:13:24 user nova-compute[75234]:   <uuid>062753a6-8f39-4e80-a549-9f0f5a4b877a</uuid>
Nov 29 03:13:24 user nova-compute[75234]:   <name>instance-0000000e</name>
Nov 29 03:13:24 user nova-compute[75234]:   <memory>524288</memory>
Nov 29 03:13:24 user nova-compute[75234]:   <vcpu>1</vcpu>
Nov 29 03:13:24 user nova-compute[75234]:   <metadata>
Nov 29 03:13:24 user nova-compute[75234]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Nov 29 03:13:24 user nova-compute[75234]:       <nova:package version="0.0.0"/>
Nov 29 03:13:24 user nova-compute[75234]:       <nova:name>tempest-AttachVolumeShelveTestJSON-server-2002337175</nova:name>
Nov 29 03:13:24 user nova-compute[75234]:       <nova:creationTime>2023-11-29 03:13:22</nova:creationTime>
Nov 29 03:13:24 user nova-compute[75234]:       <nova:flavor name="m1.tiny">
Nov 29 03:13:24 user nova-compute[75234]:         <nova:memory>512</nova:memory>
Nov 29 03:13:24 user nova-compute[75234]:         <nova:disk>1</nova:disk>
Nov 29 03:13:24 user nova-compute[75234]:         <nova:swap>0</nova:swap>
Nov 29 03:13:24 user nova-compute[75234]:         <nova:ephemeral>0</nova:ephemeral>
Nov 29 03:13:24 user nova-compute[75234]:         <nova:vcpus>1</nova:vcpus>
Nov 29 03:13:24 user nova-compute[75234]:       </nova:flavor>
Nov 29 03:13:24 user nova-compute[75234]:       <nova:owner>
Nov 29 03:13:24 user nova-compute[75234]:         <nova:user uuid="154f3f4f57514450baf8f22d67768bc9">tempest-AttachVolumeShelveTestJSON-1180535246-project-member</nova:user>
Nov 29 03:13:24 user nova-compute[75234]:         <nova:project uuid="e17150005a9a449f9373b91e1c00083b">tempest-AttachVolumeShelveTestJSON-1180535246</nova:project>
Nov 29 03:13:24 user nova-compute[75234]:       </nova:owner>
Nov 29 03:13:24 user nova-compute[75234]:       <nova:root type="image" uuid="b14da3aa-0b00-47e2-a40c-9736f7ce03a1"/>
Nov 29 03:13:24 user nova-compute[75234]:       <nova:ports>
Nov 29 03:13:24 user nova-compute[75234]:         <nova:port uuid="8f6751f1-fc65-4842-b77f-1dfe26912afb">
Nov 29 03:13:24 user nova-compute[75234]:           <nova:ip type="fixed" address="10.0.0.3" ipVersion="4"/>
Nov 29 03:13:24 user nova-compute[75234]:         </nova:port>
Nov 29 03:13:24 user nova-compute[75234]:       </nova:ports>
Nov 29 03:13:24 user nova-compute[75234]:     </nova:instance>
Nov 29 03:13:24 user nova-compute[75234]:   </metadata>
Nov 29 03:13:24 user nova-compute[75234]:   <sysinfo type="smbios">
Nov 29 03:13:24 user nova-compute[75234]:     <system>
Nov 29 03:13:24 user nova-compute[75234]:       <entry name="manufacturer">OpenStack Foundation</entry>
Nov 29 03:13:24 user nova-compute[75234]:       <entry name="product">OpenStack Nova</entry>
Nov 29 03:13:24 user nova-compute[75234]:       <entry name="version">0.0.0</entry>
Nov 29 03:13:24 user nova-compute[75234]:       <entry name="serial">062753a6-8f39-4e80-a549-9f0f5a4b877a</entry>
Nov 29 03:13:24 user nova-compute[75234]:       <entry name="uuid">062753a6-8f39-4e80-a549-9f0f5a4b877a</entry>
Nov 29 03:13:24 user nova-compute[75234]:       <entry name="family">Virtual Machine</entry>
Nov 29 03:13:24 user nova-compute[75234]:     </system>
Nov 29 03:13:24 user nova-compute[75234]:   </sysinfo>
Nov 29 03:13:24 user nova-compute[75234]:   <os>
Nov 29 03:13:24 user nova-compute[75234]:     <type arch="x86_64" machine="pc">hvm</type>
Nov 29 03:13:24 user nova-compute[75234]:     <boot dev="hd"/>
Nov 29 03:13:24 user nova-compute[75234]:     <smbios mode="sysinfo"/>
Nov 29 03:13:24 user nova-compute[75234]:   </os>
Nov 29 03:13:24 user nova-compute[75234]:   <features>
Nov 29 03:13:24 user nova-compute[75234]:     <acpi/>
Nov 29 03:13:24 user nova-compute[75234]:     <apic/>
Nov 29 03:13:24 user nova-compute[75234]:     <vmcoreinfo/>
Nov 29 03:13:24 user nova-compute[75234]:   </features>
Nov 29 03:13:24 user nova-compute[75234]:   <clock offset="utc">
Nov 29 03:13:24 user nova-compute[75234]:     <timer name="pit" tickpolicy="delay"/>
Nov 29 03:13:24 user nova-compute[75234]:     <timer name="rtc" tickpolicy="catchup"/>
Nov 29 03:13:24 user nova-compute[75234]:     <timer name="hpet" present="no"/>
Nov 29 03:13:24 user nova-compute[75234]:   </clock>
Nov 29 03:13:24 user nova-compute[75234]:   <cpu mode="custom" match="exact">
Nov 29 03:13:24 user nova-compute[75234]:     <model>Nehalem</model>
Nov 29 03:13:24 user nova-compute[75234]:     <topology sockets="1" cores="1" threads="1"/>
Nov 29 03:13:24 user nova-compute[75234]:   </cpu>
Nov 29 03:13:24 user nova-compute[75234]:   <devices>
Nov 29 03:13:24 user nova-compute[75234]:     <disk type="file" device="disk">
Nov 29 03:13:24 user nova-compute[75234]:       <driver name="qemu" type="qcow2" cache="none"/>
Nov 29 03:13:24 user nova-compute[75234]:       <source file="/opt/stack/data/nova/instances/062753a6-8f39-4e80-a549-9f0f5a4b877a/disk"/>
Nov 29 03:13:24 user nova-compute[75234]:       <target dev="vda" bus="virtio"/>
Nov 29 03:13:24 user nova-compute[75234]:     </disk>
Nov 29 03:13:24 user nova-compute[75234]:     <disk type="block" device="disk">
Nov 29 03:13:24 user nova-compute[75234]:       <driver name="qemu" type="raw" cache="none" io="native"/>
Nov 29 03:13:24 user nova-compute[75234]:       <source dev="/dev/sdd"/>
Nov 29 03:13:24 user nova-compute[75234]:       <target dev="vdc" bus="virtio"/>
Nov 29 03:13:24 user nova-compute[75234]:       <serial>6a4d34d3-9974-43bc-afcb-9169c893d9b7</serial>
Nov 29 03:13:24 user nova-compute[75234]:     </disk>
Nov 29 03:13:24 user nova-compute[75234]:     <interface type="ethernet">
Nov 29 03:13:24 user nova-compute[75234]:       <mac address="fa:16:3e:62:4a:5a"/>
Nov 29 03:13:24 user nova-compute[75234]:       <model type="virtio"/>
Nov 29 03:13:24 user nova-compute[75234]:       <mtu size="1442"/>
Nov 29 03:13:24 user nova-compute[75234]:       <target dev="tap8f6751f1-fc"/>
Nov 29 03:13:24 user nova-compute[75234]:     </interface>
Nov 29 03:13:24 user nova-compute[75234]:     <serial type="pty">
Nov 29 03:13:24 user nova-compute[75234]:       <log file="/opt/stack/data/nova/instances/062753a6-8f39-4e80-a549-9f0f5a4b877a/console.log" append="off"/>
Nov 29 03:13:24 user nova-compute[75234]:     </serial>
Nov 29 03:13:24 user nova-compute[75234]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Nov 29 03:13:24 user nova-compute[75234]:     <video>
Nov 29 03:13:24 user nova-compute[75234]:       <model type="virtio"/>
Nov 29 03:13:24 user nova-compute[75234]:     </video>
Nov 29 03:13:24 user nova-compute[75234]:     <rng model="virtio">
Nov 29 03:13:24 user nova-compute[75234]:       <backend model="random">/dev/urandom</backend>
Nov 29 03:13:24 user nova-compute[75234]:     </rng>
Nov 29 03:13:24 user nova-compute[75234]:     <controller type="usb" index="0" model="none"/>
Nov 29 03:13:24 user nova-compute[75234]:     <memballoon model="virtio">
Nov 29 03:13:24 user nova-compute[75234]:       <stats period="10"/>
Nov 29 03:13:24 user nova-compute[75234]:     </memballoon>
Nov 29 03:13:24 user nova-compute[75234]:   </devices>
Nov 29 03:13:24 user nova-compute[75234]: </domain>
Nov 29 03:13:24 user nova-compute[75234]:  {{(pid=75234) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7568}}
Nov 29 03:13:24 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-2ba9d3c2-d9fa-4fb7-9115-db6c314836c5 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=True,compute_id=1,config_drive='',created_at=2023-11-29T03:09:11Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description=None,display_name='tempest-AttachVolumeShelveTestJSON-server-2002337175',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-attachvolumeshelvetestjson-server-2002337175',id=14,image_ref='b14da3aa-0b00-47e2-a40c-9736f7ce03a1',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data=None,key_name='tempest-keypair-1961974262',keypairs=<?>,launch_index=0,launched_at=2023-11-29T03:09:20Z,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=4,progress=0,project_id='e17150005a9a449f9373b91e1c00083b',ramdisk_id='',reservation_id='r-p3yfidye',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',clean_attempts='1',image_base_image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-AttachVolumeShelveTestJSON-1180535246',owner_user_name='tempest-AttachVolumeShelveTestJSON-1180535246-project-member',shelved_at='2023-11-29T03:11:13.240549',shelved_host='user',shelved_image_id='b14da3aa-0b00-47e2-a40c-9736f7ce03a1'},tags=<?>,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-11-29T03:13:01Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='154f3f4f57514450baf8f22d67768bc9',uuid=062753a6-8f39-4e80-a549-9f0f5a4b877a,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='shelved_offloaded') vif={"id": "8f6751f1-fc65-4842-b77f-1dfe26912afb", "address": "fa:16:3e:62:4a:5a", "network": {"id": "aad085f7-106b-4de6-93f5-f86bb66ec962", "bridge": "br-int", "label": "tempest-AttachVolumeShelveTestJSON-775984268-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.175", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "e17150005a9a449f9373b91e1c00083b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap8f6751f1-fc", "ovs_interfaceid": "8f6751f1-fc65-4842-b77f-1dfe26912afb", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Nov 29 03:13:24 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-2ba9d3c2-d9fa-4fb7-9115-db6c314836c5 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Converting VIF {"id": "8f6751f1-fc65-4842-b77f-1dfe26912afb", "address": "fa:16:3e:62:4a:5a", "network": {"id": "aad085f7-106b-4de6-93f5-f86bb66ec962", "bridge": "br-int", "label": "tempest-AttachVolumeShelveTestJSON-775984268-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.175", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "e17150005a9a449f9373b91e1c00083b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap8f6751f1-fc", "ovs_interfaceid": "8f6751f1-fc65-4842-b77f-1dfe26912afb", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:13:24 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-2ba9d3c2-d9fa-4fb7-9115-db6c314836c5 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:62:4a:5a,bridge_name='br-int',has_traffic_filtering=True,id=8f6751f1-fc65-4842-b77f-1dfe26912afb,network=Network(aad085f7-106b-4de6-93f5-f86bb66ec962),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap8f6751f1-fc') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:13:24 user nova-compute[75234]: DEBUG os_vif [None req-2ba9d3c2-d9fa-4fb7-9115-db6c314836c5 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:62:4a:5a,bridge_name='br-int',has_traffic_filtering=True,id=8f6751f1-fc65-4842-b77f-1dfe26912afb,network=Network(aad085f7-106b-4de6-93f5-f86bb66ec962),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap8f6751f1-fc') {{(pid=75234) plug /opt/stack/data/venv/lib/python3.10/site-packages/os_vif/__init__.py:76}}
Nov 29 03:13:24 user nova-compute[75234]: INFO nova.scheduler.client.report [None req-f1ee3ef1-1771-42fa-a7f7-83bad3ad6a3b tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] Deleted allocations for instance 8297c264-a213-44ee-be46-5c08eabda9f7
Nov 29 03:13:24 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:13:24 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:13:24 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Nov 29 03:13:24 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:13:24 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbCreateCommand(_result=None, table=QoS, columns={'type': 'linux-noop', 'external_ids': {'id': 'ef7de737-0cc6-5031-aedb-46e6cda2e0b1', '_type': 'linux-noop'}}, row=False) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:13:24 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:13:24 user nova-compute[75234]: DEBUG nova.compute.manager [req-da273fde-d2e4-49bb-9bef-e00eed2c0837 req-0f5fc1d1-cced-4043-adde-3b39c95fbabb service nova] [instance: 8297c264-a213-44ee-be46-5c08eabda9f7] Received event network-vif-plugged-d26334d2-8040-4022-b658-69be1cf6a025 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:13:24 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-da273fde-d2e4-49bb-9bef-e00eed2c0837 req-0f5fc1d1-cced-4043-adde-3b39c95fbabb service nova] Acquiring lock "8297c264-a213-44ee-be46-5c08eabda9f7-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:13:24 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-da273fde-d2e4-49bb-9bef-e00eed2c0837 req-0f5fc1d1-cced-4043-adde-3b39c95fbabb service nova] Lock "8297c264-a213-44ee-be46-5c08eabda9f7-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:13:24 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-da273fde-d2e4-49bb-9bef-e00eed2c0837 req-0f5fc1d1-cced-4043-adde-3b39c95fbabb service nova] Lock "8297c264-a213-44ee-be46-5c08eabda9f7-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:13:24 user nova-compute[75234]: DEBUG nova.compute.manager [req-da273fde-d2e4-49bb-9bef-e00eed2c0837 req-0f5fc1d1-cced-4043-adde-3b39c95fbabb service nova] [instance: 8297c264-a213-44ee-be46-5c08eabda9f7] No waiting events found dispatching network-vif-plugged-d26334d2-8040-4022-b658-69be1cf6a025 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:13:24 user nova-compute[75234]: WARNING nova.compute.manager [req-da273fde-d2e4-49bb-9bef-e00eed2c0837 req-0f5fc1d1-cced-4043-adde-3b39c95fbabb service nova] [instance: 8297c264-a213-44ee-be46-5c08eabda9f7] Received unexpected event network-vif-plugged-d26334d2-8040-4022-b658-69be1cf6a025 for instance with vm_state deleted and task_state None.
Nov 29 03:13:24 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:13:24 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:13:24 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tap8f6751f1-fc, may_exist=True, interface_attrs={}) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:13:24 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Port, record=tap8f6751f1-fc, col_values=(('qos', UUID('6433206e-6949-44ca-88eb-706d1ca0c695')),), if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:13:24 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DbSetCommand(_result=None, table=Interface, record=tap8f6751f1-fc, col_values=(('external_ids', {'iface-id': '8f6751f1-fc65-4842-b77f-1dfe26912afb', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:62:4a:5a', 'vm-uuid': '062753a6-8f39-4e80-a549-9f0f5a4b877a'}),), if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:13:24 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:13:24 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:13:24 user nova-compute[75234]: INFO os_vif [None req-2ba9d3c2-d9fa-4fb7-9115-db6c314836c5 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:62:4a:5a,bridge_name='br-int',has_traffic_filtering=True,id=8f6751f1-fc65-4842-b77f-1dfe26912afb,network=Network(aad085f7-106b-4de6-93f5-f86bb66ec962),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap8f6751f1-fc')
Nov 29 03:13:24 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-f1ee3ef1-1771-42fa-a7f7-83bad3ad6a3b tempest-ServerStableDeviceRescueTest-1509102897 tempest-ServerStableDeviceRescueTest-1509102897-project-member] Lock "8297c264-a213-44ee-be46-5c08eabda9f7" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 2.546s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:13:24 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-2ba9d3c2-d9fa-4fb7-9115-db6c314836c5 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] No BDM found with device name vda, not building metadata. {{(pid=75234) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12089}}
Nov 29 03:13:24 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-2ba9d3c2-d9fa-4fb7-9115-db6c314836c5 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] No BDM found with device name vdc, not building metadata. {{(pid=75234) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12089}}
Nov 29 03:13:24 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-2ba9d3c2-d9fa-4fb7-9115-db6c314836c5 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] No VIF found with MAC fa:16:3e:62:4a:5a, not building metadata {{(pid=75234) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Nov 29 03:13:26 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:13:26 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:13:26 user nova-compute[75234]: DEBUG nova.compute.manager [None req-371aaf1d-7bb3-4224-8cba-252fdf4d9e7c tempest-TestVolumeSwap-1761484429 tempest-TestVolumeSwap-1761484429-project-admin] [instance: 3e355ae8-aab9-4ae9-b6e3-6de108ec1af9] swap_volume: Calling driver volume swap with connection infos: new: {'driver_volume_type': 'iscsi', 'data': {'target_iqn': 'iqn.2010-10.org.openstack:94dadf41-624e-4581-81cf-eb5eaaecafb2', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'CIoXTDLA', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}, 'status': 'reserved', 'instance': '3e355ae8-aab9-4ae9-b6e3-6de108ec1af9', 'attached_at': '', 'detached_at': '', 'volume_id': '94dadf41-624e-4581-81cf-eb5eaaecafb2', 'serial': '94dadf41-624e-4581-81cf-eb5eaaecafb2'}; old: {'driver_volume_type': 'iscsi', 'data': {'target_iqn': 'iqn.2010-10.org.openstack:4ed484cc-9de2-42df-8d99-637f3b47d474', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'mBlevMXQ', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdb'}, 'status': 'reserved', 'instance': '3e355ae8-aab9-4ae9-b6e3-6de108ec1af9', 'attached_at': '', 'detached_at': '', 'volume_id': '4ed484cc-9de2-42df-8d99-637f3b47d474', 'serial'} {{(pid=75234) _swap_volume /opt/stack/nova/nova/compute/manager.py:7782}}
Nov 29 03:13:26 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:13:26 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:13:26 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-371aaf1d-7bb3-4224-8cba-252fdf4d9e7c tempest-TestVolumeSwap-1761484429 tempest-TestVolumeSwap-1761484429-project-admin] [instance: 3e355ae8-aab9-4ae9-b6e3-6de108ec1af9] Calling os-brick to attach iSCSI Volume {{(pid=75234) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Nov 29 03:13:26 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-371aaf1d-7bb3-4224-8cba-252fdf4d9e7c tempest-TestVolumeSwap-1761484429 tempest-TestVolumeSwap-1761484429-project-admin] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f8a5050c9a0>, 'connection_properties': {'target_iqn': 'iqn.2010-10.org.openstack:94dadf41-624e-4581-81cf-eb5eaaecafb2', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'CIoXTDLA', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:176}}
Nov 29 03:13:26 user nova-compute[75234]: WARNING os_brick.initiator.connectors.base [None req-371aaf1d-7bb3-4224-8cba-252fdf4d9e7c tempest-TestVolumeSwap-1761484429 tempest-TestVolumeSwap-1761484429-project-admin] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Nov 29 03:13:26 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-371aaf1d-7bb3-4224-8cba-252fdf4d9e7c tempest-TestVolumeSwap-1761484429 tempest-TestVolumeSwap-1761484429-project-admin] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:68}}
Nov 29 03:13:26 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-371aaf1d-7bb3-4224-8cba-252fdf4d9e7c tempest-TestVolumeSwap-1761484429 tempest-TestVolumeSwap-1761484429-project-admin] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:73}}
Nov 29 03:13:26 user nova-compute[75234]: WARNING os_brick.initiator.connectors.base [None req-371aaf1d-7bb3-4224-8cba-252fdf4d9e7c tempest-TestVolumeSwap-1761484429 tempest-TestVolumeSwap-1761484429-project-admin] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Nov 29 03:13:26 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-371aaf1d-7bb3-4224-8cba-252fdf4d9e7c tempest-TestVolumeSwap-1761484429 tempest-TestVolumeSwap-1761484429-project-admin] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:94dadf41-624e-4581-81cf-eb5eaaecafb2" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:68}}
Nov 29 03:13:26 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-371aaf1d-7bb3-4224-8cba-252fdf4d9e7c tempest-TestVolumeSwap-1761484429 tempest-TestVolumeSwap-1761484429-project-admin] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:94dadf41-624e-4581-81cf-eb5eaaecafb2" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:73}}
Nov 29 03:13:26 user nova-compute[75234]: INFO os_brick.initiator.connectors.iscsi [None req-371aaf1d-7bb3-4224-8cba-252fdf4d9e7c tempest-TestVolumeSwap-1761484429 tempest-TestVolumeSwap-1761484429-project-admin] Trying to connect to iSCSI portal 172.16.0.220:3260
Nov 29 03:13:26 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:94dadf41-624e-4581-81cf-eb5eaaecafb2 -p 172.16.0.220:3260 {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:13:26 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:94dadf41-624e-4581-81cf-eb5eaaecafb2 -p 172.16.0.220:3260" returned: 21 in 0.021s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:13:26 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[432761fd-fb44-4e4b-812b-9853464c83b4]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:13:26 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-371aaf1d-7bb3-4224-8cba-252fdf4d9e7c tempest-TestVolumeSwap-1761484429 tempest-TestVolumeSwap-1761484429-project-admin] iscsiadm (): stdout= stderr=iscsiadm: No records found
Nov 29 03:13:26 user nova-compute[75234]:  {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:13:26 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:94dadf41-624e-4581-81cf-eb5eaaecafb2 -p 172.16.0.220:3260 --interface default --op new {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:13:26 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:94dadf41-624e-4581-81cf-eb5eaaecafb2 -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.024s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:13:26 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[4bd361ed-20f9-434f-ad7f-bbb93ff81352]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:94dadf41-624e-4581-81cf-eb5eaaecafb2] added\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:13:26 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-371aaf1d-7bb3-4224-8cba-252fdf4d9e7c tempest-TestVolumeSwap-1761484429 tempest-TestVolumeSwap-1761484429-project-admin] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:94dadf41-624e-4581-81cf-eb5eaaecafb2] added
Nov 29 03:13:26 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:13:26 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:94dadf41-624e-4581-81cf-eb5eaaecafb2 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:13:26 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:94dadf41-624e-4581-81cf-eb5eaaecafb2 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.021s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:13:26 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[74509f88-64f0-4549-867a-2e9e8a19fe66]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:13:26 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-371aaf1d-7bb3-4224-8cba-252fdf4d9e7c tempest-TestVolumeSwap-1761484429 tempest-TestVolumeSwap-1761484429-project-admin] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:13:26 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:94dadf41-624e-4581-81cf-eb5eaaecafb2 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:13:26 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:94dadf41-624e-4581-81cf-eb5eaaecafb2 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.022s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:13:26 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[10d11a4d-5437-4525-966d-66ae539cbf89]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:13:26 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-371aaf1d-7bb3-4224-8cba-252fdf4d9e7c tempest-TestVolumeSwap-1761484429 tempest-TestVolumeSwap-1761484429-project-admin] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:13:26 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:94dadf41-624e-4581-81cf-eb5eaaecafb2 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v CIoXTDLA {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:13:26 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:94dadf41-624e-4581-81cf-eb5eaaecafb2 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v CIoXTDLA" returned: 0 in 0.028s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:13:26 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[97ee09eb-a6eb-4ca0-9ab0-4472abb74165]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:13:26 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-371aaf1d-7bb3-4224-8cba-252fdf4d9e7c tempest-TestVolumeSwap-1761484429 tempest-TestVolumeSwap-1761484429-project-admin] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'CIoXTDLA'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:13:26 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:94dadf41-624e-4581-81cf-eb5eaaecafb2 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:13:26 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:94dadf41-624e-4581-81cf-eb5eaaecafb2 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.029s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:13:26 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[cd629bae-0eb9-4e57-854a-1096982725d0]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:13:26 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-371aaf1d-7bb3-4224-8cba-252fdf4d9e7c tempest-TestVolumeSwap-1761484429 tempest-TestVolumeSwap-1761484429-project-admin] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:13:26 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:13:26 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.036s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:13:26 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[31e17d14-3f96-4e88-ad8f-5d86c5ab22df]: (4, ('tcp: [20] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b (non-flash)\ntcp: [21] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:56237f4c-4138-4f50-aafb-9c0f9d4bb1cf (non-flash)\ntcp: [22] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4ed484cc-9de2-42df-8d99-637f3b47d474 (non-flash)\ntcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6a4d34d3-9974-43bc-afcb-9169c893d9b7 (non-flash)\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:13:26 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-371aaf1d-7bb3-4224-8cba-252fdf4d9e7c tempest-TestVolumeSwap-1761484429 tempest-TestVolumeSwap-1761484429-project-admin] iscsiadm ('-m', 'session'): stdout=tcp: [20] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b (non-flash)
Nov 29 03:13:26 user nova-compute[75234]: tcp: [21] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:56237f4c-4138-4f50-aafb-9c0f9d4bb1cf (non-flash)
Nov 29 03:13:26 user nova-compute[75234]: tcp: [22] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4ed484cc-9de2-42df-8d99-637f3b47d474 (non-flash)
Nov 29 03:13:26 user nova-compute[75234]: tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6a4d34d3-9974-43bc-afcb-9169c893d9b7 (non-flash)
Nov 29 03:13:26 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:13:26 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-371aaf1d-7bb3-4224-8cba-252fdf4d9e7c tempest-TestVolumeSwap-1761484429 tempest-TestVolumeSwap-1761484429-project-admin] iscsi session list stdout=tcp: [20] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b (non-flash)
Nov 29 03:13:26 user nova-compute[75234]: tcp: [21] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:56237f4c-4138-4f50-aafb-9c0f9d4bb1cf (non-flash)
Nov 29 03:13:26 user nova-compute[75234]: tcp: [22] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4ed484cc-9de2-42df-8d99-637f3b47d474 (non-flash)
Nov 29 03:13:26 user nova-compute[75234]: tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6a4d34d3-9974-43bc-afcb-9169c893d9b7 (non-flash)
Nov 29 03:13:26 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsi_session /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Nov 29 03:13:26 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:94dadf41-624e-4581-81cf-eb5eaaecafb2 -p 172.16.0.220:3260 --login {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:13:26 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:94dadf41-624e-4581-81cf-eb5eaaecafb2 -p 172.16.0.220:3260 --login" returned: 0 in 0.078s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:13:26 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[13bb65c5-b7b4-47a9-bcb7-f33411dac81d]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:94dadf41-624e-4581-81cf-eb5eaaecafb2, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:94dadf41-624e-4581-81cf-eb5eaaecafb2, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:13:26 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-371aaf1d-7bb3-4224-8cba-252fdf4d9e7c tempest-TestVolumeSwap-1761484429 tempest-TestVolumeSwap-1761484429-project-admin] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:94dadf41-624e-4581-81cf-eb5eaaecafb2, portal: 172.16.0.220,3260]
Nov 29 03:13:26 user nova-compute[75234]: Login to [iface: default, target: iqn.2010-10.org.openstack:94dadf41-624e-4581-81cf-eb5eaaecafb2, portal: 172.16.0.220,3260] successful.
Nov 29 03:13:26 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:13:26 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:94dadf41-624e-4581-81cf-eb5eaaecafb2 -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:13:26 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:94dadf41-624e-4581-81cf-eb5eaaecafb2 -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.026s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:13:26 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[e1fcf5e5-f638-40bd-93e4-d9046c759e4f]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:13:26 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-371aaf1d-7bb3-4224-8cba-252fdf4d9e7c tempest-TestVolumeSwap-1761484429 tempest-TestVolumeSwap-1761484429-project-admin] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:13:26 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:13:26 user nova-compute[75234]: DEBUG nova.compute.manager [req-3627c813-ce6f-4632-9293-3355af393982 req-0b8aecd6-5d90-4b62-86e8-c441d6d4fb21 service nova] [instance: 062753a6-8f39-4e80-a549-9f0f5a4b877a] Received event network-vif-plugged-8f6751f1-fc65-4842-b77f-1dfe26912afb {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:13:26 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-3627c813-ce6f-4632-9293-3355af393982 req-0b8aecd6-5d90-4b62-86e8-c441d6d4fb21 service nova] Acquiring lock "062753a6-8f39-4e80-a549-9f0f5a4b877a-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:13:26 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-3627c813-ce6f-4632-9293-3355af393982 req-0b8aecd6-5d90-4b62-86e8-c441d6d4fb21 service nova] Lock "062753a6-8f39-4e80-a549-9f0f5a4b877a-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:13:26 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-3627c813-ce6f-4632-9293-3355af393982 req-0b8aecd6-5d90-4b62-86e8-c441d6d4fb21 service nova] Lock "062753a6-8f39-4e80-a549-9f0f5a4b877a-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:13:26 user nova-compute[75234]: DEBUG nova.compute.manager [req-3627c813-ce6f-4632-9293-3355af393982 req-0b8aecd6-5d90-4b62-86e8-c441d6d4fb21 service nova] [instance: 062753a6-8f39-4e80-a549-9f0f5a4b877a] No waiting events found dispatching network-vif-plugged-8f6751f1-fc65-4842-b77f-1dfe26912afb {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:13:26 user nova-compute[75234]: WARNING nova.compute.manager [req-3627c813-ce6f-4632-9293-3355af393982 req-0b8aecd6-5d90-4b62-86e8-c441d6d4fb21 service nova] [instance: 062753a6-8f39-4e80-a549-9f0f5a4b877a] Received unexpected event network-vif-plugged-8f6751f1-fc65-4842-b77f-1dfe26912afb for instance with vm_state shelved_offloaded and task_state spawning.
Nov 29 03:13:26 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.026s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:13:26 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[4f1ff28f-5752-41a4-bdbc-5622e57a5751]: (4, ('tcp: [20] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b (non-flash)\ntcp: [21] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:56237f4c-4138-4f50-aafb-9c0f9d4bb1cf (non-flash)\ntcp: [22] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4ed484cc-9de2-42df-8d99-637f3b47d474 (non-flash)\ntcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6a4d34d3-9974-43bc-afcb-9169c893d9b7 (non-flash)\ntcp: [24] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:94dadf41-624e-4581-81cf-eb5eaaecafb2 (non-flash)\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:13:26 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-371aaf1d-7bb3-4224-8cba-252fdf4d9e7c tempest-TestVolumeSwap-1761484429 tempest-TestVolumeSwap-1761484429-project-admin] iscsiadm ('-m', 'session'): stdout=tcp: [20] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b (non-flash)
Nov 29 03:13:26 user nova-compute[75234]: tcp: [21] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:56237f4c-4138-4f50-aafb-9c0f9d4bb1cf (non-flash)
Nov 29 03:13:26 user nova-compute[75234]: tcp: [22] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4ed484cc-9de2-42df-8d99-637f3b47d474 (non-flash)
Nov 29 03:13:26 user nova-compute[75234]: tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6a4d34d3-9974-43bc-afcb-9169c893d9b7 (non-flash)
Nov 29 03:13:26 user nova-compute[75234]: tcp: [24] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:94dadf41-624e-4581-81cf-eb5eaaecafb2 (non-flash)
Nov 29 03:13:26 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:13:26 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-371aaf1d-7bb3-4224-8cba-252fdf4d9e7c tempest-TestVolumeSwap-1761484429 tempest-TestVolumeSwap-1761484429-project-admin] iscsi session list stdout=tcp: [20] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b (non-flash)
Nov 29 03:13:26 user nova-compute[75234]: tcp: [21] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:56237f4c-4138-4f50-aafb-9c0f9d4bb1cf (non-flash)
Nov 29 03:13:26 user nova-compute[75234]: tcp: [22] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4ed484cc-9de2-42df-8d99-637f3b47d474 (non-flash)
Nov 29 03:13:26 user nova-compute[75234]: tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6a4d34d3-9974-43bc-afcb-9169c893d9b7 (non-flash)
Nov 29 03:13:26 user nova-compute[75234]: tcp: [24] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:94dadf41-624e-4581-81cf-eb5eaaecafb2 (non-flash)
Nov 29 03:13:26 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsi_session /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Nov 29 03:13:26 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-371aaf1d-7bb3-4224-8cba-252fdf4d9e7c tempest-TestVolumeSwap-1761484429 tempest-TestVolumeSwap-1761484429-project-admin] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:94dadf41-624e-4581-81cf-eb5eaaecafb2" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.360s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:87}}
Nov 29 03:13:26 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-371aaf1d-7bb3-4224-8cba-252fdf4d9e7c tempest-TestVolumeSwap-1761484429 tempest-TestVolumeSwap-1761484429-project-admin] Connected to 172.16.0.220:3260 {{(pid=75234) _connect_vol /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:644}}
Nov 29 03:13:26 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-371aaf1d-7bb3-4224-8cba-252fdf4d9e7c tempest-TestVolumeSwap-1761484429 tempest-TestVolumeSwap-1761484429-project-admin] HCTL ('36', '-', '-', 0) found on session 24 with lun 0 {{(pid=75234) get_hctl /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:770}}
Nov 29 03:13:26 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-371aaf1d-7bb3-4224-8cba-252fdf4d9e7c tempest-TestVolumeSwap-1761484429 tempest-TestVolumeSwap-1761484429-project-admin] Scanning host 36 c: -, t: -, l: 0) {{(pid=75234) scan_iscsi /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:795}}
Nov 29 03:13:26 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host36/scan {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:13:26 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host36/scan" returned: 0 in 0.020s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:13:26 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[1d089d9e-6b93-44eb-9003-aa442a64940c]: (4, ('- - 0', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:13:26 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-371aaf1d-7bb3-4224-8cba-252fdf4d9e7c tempest-TestVolumeSwap-1761484429 tempest-TestVolumeSwap-1761484429-project-admin] Searching for a device in session 24 and hctl ['36', '*', '*', 0] yield: sde {{(pid=75234) device_name_by_hctl /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:789}}
Nov 29 03:13:26 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-371aaf1d-7bb3-4224-8cba-252fdf4d9e7c tempest-TestVolumeSwap-1761484429 tempest-TestVolumeSwap-1761484429-project-admin] Connected to sde using {'target_iqn': 'iqn.2010-10.org.openstack:94dadf41-624e-4581-81cf-eb5eaaecafb2', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'CIoXTDLA', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False} {{(pid=75234) _connect_vol /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:672}}
Nov 29 03:13:26 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:13:26 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:13:26 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:13:26 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:13:26 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:13:26 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:13:27 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-371aaf1d-7bb3-4224-8cba-252fdf4d9e7c tempest-TestVolumeSwap-1761484429 tempest-TestVolumeSwap-1761484429-project-admin] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.403s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:87}}
Nov 29 03:13:27 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-371aaf1d-7bb3-4224-8cba-252fdf4d9e7c tempest-TestVolumeSwap-1761484429 tempest-TestVolumeSwap-1761484429-project-admin] <== connect_volume: return (1404ms) {'type': 'block', 'scsi_wwn': '26433653966323430', 'path': '/dev/sde'} {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:203}}
Nov 29 03:13:27 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-371aaf1d-7bb3-4224-8cba-252fdf4d9e7c tempest-TestVolumeSwap-1761484429 tempest-TestVolumeSwap-1761484429-project-admin] [instance: 3e355ae8-aab9-4ae9-b6e3-6de108ec1af9] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '26433653966323430', 'path': '/dev/sde'} {{(pid=75234) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Nov 29 03:13:27 user nova-compute[75234]: DEBUG nova.objects.instance [None req-371aaf1d-7bb3-4224-8cba-252fdf4d9e7c tempest-TestVolumeSwap-1761484429 tempest-TestVolumeSwap-1761484429-project-admin] Lazy-loading 'flavor' on Instance uuid 3e355ae8-aab9-4ae9-b6e3-6de108ec1af9 {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:13:27 user nova-compute[75234]: DEBUG nova.virt.libvirt.guest [None req-371aaf1d-7bb3-4224-8cba-252fdf4d9e7c tempest-TestVolumeSwap-1761484429 tempest-TestVolumeSwap-1761484429-project-admin] COPY block job progress, current cursor: 0 final cursor: 1073741824 {{(pid=75234) is_job_complete /opt/stack/nova/nova/virt/libvirt/guest.py:846}}
Nov 29 03:13:28 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-a4319b7c-65c8-4a50-a342-c9d3e7cfa187 tempest-VolumesActionsTest-1495826017 tempest-VolumesActionsTest-1495826017-project-member] Acquiring lock "1eb9b734-2711-4463-9743-e5fd95c6857b" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:13:28 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-a4319b7c-65c8-4a50-a342-c9d3e7cfa187 tempest-VolumesActionsTest-1495826017 tempest-VolumesActionsTest-1495826017-project-member] Lock "1eb9b734-2711-4463-9743-e5fd95c6857b" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:13:28 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-a4319b7c-65c8-4a50-a342-c9d3e7cfa187 tempest-VolumesActionsTest-1495826017 tempest-VolumesActionsTest-1495826017-project-member] Acquiring lock "1eb9b734-2711-4463-9743-e5fd95c6857b-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:13:28 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-a4319b7c-65c8-4a50-a342-c9d3e7cfa187 tempest-VolumesActionsTest-1495826017 tempest-VolumesActionsTest-1495826017-project-member] Lock "1eb9b734-2711-4463-9743-e5fd95c6857b-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:13:28 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-a4319b7c-65c8-4a50-a342-c9d3e7cfa187 tempest-VolumesActionsTest-1495826017 tempest-VolumesActionsTest-1495826017-project-member] Lock "1eb9b734-2711-4463-9743-e5fd95c6857b-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:13:28 user nova-compute[75234]: INFO nova.compute.manager [None req-a4319b7c-65c8-4a50-a342-c9d3e7cfa187 tempest-VolumesActionsTest-1495826017 tempest-VolumesActionsTest-1495826017-project-member] [instance: 1eb9b734-2711-4463-9743-e5fd95c6857b] Terminating instance
Nov 29 03:13:28 user nova-compute[75234]: DEBUG nova.compute.manager [None req-a4319b7c-65c8-4a50-a342-c9d3e7cfa187 tempest-VolumesActionsTest-1495826017 tempest-VolumesActionsTest-1495826017-project-member] [instance: 1eb9b734-2711-4463-9743-e5fd95c6857b] Start destroying the instance on the hypervisor. {{(pid=75234) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3140}}
Nov 29 03:13:28 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:13:28 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:13:28 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:13:28 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:13:28 user nova-compute[75234]: DEBUG nova.virt.libvirt.guest [None req-371aaf1d-7bb3-4224-8cba-252fdf4d9e7c tempest-TestVolumeSwap-1761484429 tempest-TestVolumeSwap-1761484429-project-admin] COPY block job progress, current cursor: 140509184 final cursor: 1073741824 {{(pid=75234) is_job_complete /opt/stack/nova/nova/virt/libvirt/guest.py:846}}
Nov 29 03:13:28 user nova-compute[75234]: DEBUG nova.compute.manager [req-d0694259-afa2-4885-a815-deb103ecad8a req-1ffed1c2-e12e-4c2c-9590-b6b19fae0b11 service nova] [instance: 062753a6-8f39-4e80-a549-9f0f5a4b877a] Received event network-vif-plugged-8f6751f1-fc65-4842-b77f-1dfe26912afb {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:13:28 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-d0694259-afa2-4885-a815-deb103ecad8a req-1ffed1c2-e12e-4c2c-9590-b6b19fae0b11 service nova] Acquiring lock "062753a6-8f39-4e80-a549-9f0f5a4b877a-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:13:28 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-d0694259-afa2-4885-a815-deb103ecad8a req-1ffed1c2-e12e-4c2c-9590-b6b19fae0b11 service nova] Lock "062753a6-8f39-4e80-a549-9f0f5a4b877a-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:13:28 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-d0694259-afa2-4885-a815-deb103ecad8a req-1ffed1c2-e12e-4c2c-9590-b6b19fae0b11 service nova] Lock "062753a6-8f39-4e80-a549-9f0f5a4b877a-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:13:28 user nova-compute[75234]: DEBUG nova.compute.manager [req-d0694259-afa2-4885-a815-deb103ecad8a req-1ffed1c2-e12e-4c2c-9590-b6b19fae0b11 service nova] [instance: 062753a6-8f39-4e80-a549-9f0f5a4b877a] No waiting events found dispatching network-vif-plugged-8f6751f1-fc65-4842-b77f-1dfe26912afb {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:13:28 user nova-compute[75234]: WARNING nova.compute.manager [req-d0694259-afa2-4885-a815-deb103ecad8a req-1ffed1c2-e12e-4c2c-9590-b6b19fae0b11 service nova] [instance: 062753a6-8f39-4e80-a549-9f0f5a4b877a] Received unexpected event network-vif-plugged-8f6751f1-fc65-4842-b77f-1dfe26912afb for instance with vm_state shelved_offloaded and task_state spawning.
Nov 29 03:13:28 user nova-compute[75234]: DEBUG nova.compute.manager [None req-2ba9d3c2-d9fa-4fb7-9115-db6c314836c5 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] [instance: 062753a6-8f39-4e80-a549-9f0f5a4b877a] Instance event wait completed in 0 seconds for  {{(pid=75234) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Nov 29 03:13:28 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-2ba9d3c2-d9fa-4fb7-9115-db6c314836c5 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] [instance: 062753a6-8f39-4e80-a549-9f0f5a4b877a] Guest created on hypervisor {{(pid=75234) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4421}}
Nov 29 03:13:28 user nova-compute[75234]: INFO nova.virt.libvirt.driver [-] [instance: 062753a6-8f39-4e80-a549-9f0f5a4b877a] Instance spawned successfully.
Nov 29 03:13:29 user nova-compute[75234]: DEBUG nova.virt.libvirt.guest [None req-371aaf1d-7bb3-4224-8cba-252fdf4d9e7c tempest-TestVolumeSwap-1761484429 tempest-TestVolumeSwap-1761484429-project-admin] COPY block job progress, current cursor: 321912832 final cursor: 1073741824 {{(pid=75234) is_job_complete /opt/stack/nova/nova/virt/libvirt/guest.py:846}}
Nov 29 03:13:29 user nova-compute[75234]: INFO nova.virt.libvirt.driver [-] [instance: 1eb9b734-2711-4463-9743-e5fd95c6857b] Instance destroyed successfully.
Nov 29 03:13:29 user nova-compute[75234]: DEBUG nova.objects.instance [None req-a4319b7c-65c8-4a50-a342-c9d3e7cfa187 tempest-VolumesActionsTest-1495826017 tempest-VolumesActionsTest-1495826017-project-member] Lazy-loading 'resources' on Instance uuid 1eb9b734-2711-4463-9743-e5fd95c6857b {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:13:29 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-a4319b7c-65c8-4a50-a342-c9d3e7cfa187 tempest-VolumesActionsTest-1495826017 tempest-VolumesActionsTest-1495826017-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-11-29T03:11:28Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-VolumesActionsTest-instance-2001816366',display_name='tempest-VolumesActionsTest-instance-2001816366',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-volumesactionstest-instance-2001816366',id=21,image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data=None,key_name=None,keypairs=<?>,launch_index=0,launched_at=2023-11-29T03:11:35Z,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='773aac44d1a74a439f4f750dfe8ac142',ramdisk_id='',reservation_id='r-3x0wrwrv',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-VolumesActionsTest-1495826017',owner_user_name='tempest-VolumesActionsTest-1495826017-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-11-29T03:11:36Z,user_data=None,user_id='324ad29d0b8b443086942e61e2be4ceb',uuid=1eb9b734-2711-4463-9743-e5fd95c6857b,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "6d9e1401-d167-4fb3-a59d-ca69b18ed417", "address": "fa:16:3e:24:48:f1", "network": {"id": "cb9b1cd8-2f38-4fe2-b680-380cff4b4d11", "bridge": "br-int", "label": "tempest-VolumesActionsTest-1501188375-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "773aac44d1a74a439f4f750dfe8ac142", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap6d9e1401-d1", "ovs_interfaceid": "6d9e1401-d167-4fb3-a59d-ca69b18ed417", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Nov 29 03:13:29 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-a4319b7c-65c8-4a50-a342-c9d3e7cfa187 tempest-VolumesActionsTest-1495826017 tempest-VolumesActionsTest-1495826017-project-member] Converting VIF {"id": "6d9e1401-d167-4fb3-a59d-ca69b18ed417", "address": "fa:16:3e:24:48:f1", "network": {"id": "cb9b1cd8-2f38-4fe2-b680-380cff4b4d11", "bridge": "br-int", "label": "tempest-VolumesActionsTest-1501188375-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "773aac44d1a74a439f4f750dfe8ac142", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap6d9e1401-d1", "ovs_interfaceid": "6d9e1401-d167-4fb3-a59d-ca69b18ed417", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:13:29 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-a4319b7c-65c8-4a50-a342-c9d3e7cfa187 tempest-VolumesActionsTest-1495826017 tempest-VolumesActionsTest-1495826017-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:24:48:f1,bridge_name='br-int',has_traffic_filtering=True,id=6d9e1401-d167-4fb3-a59d-ca69b18ed417,network=Network(cb9b1cd8-2f38-4fe2-b680-380cff4b4d11),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap6d9e1401-d1') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:13:29 user nova-compute[75234]: DEBUG os_vif [None req-a4319b7c-65c8-4a50-a342-c9d3e7cfa187 tempest-VolumesActionsTest-1495826017 tempest-VolumesActionsTest-1495826017-project-member] Unplugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:24:48:f1,bridge_name='br-int',has_traffic_filtering=True,id=6d9e1401-d167-4fb3-a59d-ca69b18ed417,network=Network(cb9b1cd8-2f38-4fe2-b680-380cff4b4d11),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap6d9e1401-d1') {{(pid=75234) unplug /opt/stack/data/venv/lib/python3.10/site-packages/os_vif/__init__.py:109}}
Nov 29 03:13:29 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:13:29 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tap6d9e1401-d1, bridge=br-int, if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:13:29 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:13:29 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:13:29 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:13:29 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:13:29 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbDestroyCommand(_result=None, table=QoS, record=8bd11dce-012b-44a1-aea9-7d6c1d6bf63c) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:13:29 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:13:29 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:13:29 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:13:29 user nova-compute[75234]: INFO os_vif [None req-a4319b7c-65c8-4a50-a342-c9d3e7cfa187 tempest-VolumesActionsTest-1495826017 tempest-VolumesActionsTest-1495826017-project-member] Successfully unplugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:24:48:f1,bridge_name='br-int',has_traffic_filtering=True,id=6d9e1401-d167-4fb3-a59d-ca69b18ed417,network=Network(cb9b1cd8-2f38-4fe2-b680-380cff4b4d11),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap6d9e1401-d1')
Nov 29 03:13:29 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-a4319b7c-65c8-4a50-a342-c9d3e7cfa187 tempest-VolumesActionsTest-1495826017 tempest-VolumesActionsTest-1495826017-project-member] [instance: 1eb9b734-2711-4463-9743-e5fd95c6857b] Deleting instance files /opt/stack/data/nova/instances/1eb9b734-2711-4463-9743-e5fd95c6857b_del
Nov 29 03:13:29 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-a4319b7c-65c8-4a50-a342-c9d3e7cfa187 tempest-VolumesActionsTest-1495826017 tempest-VolumesActionsTest-1495826017-project-member] [instance: 1eb9b734-2711-4463-9743-e5fd95c6857b] Deletion of /opt/stack/data/nova/instances/1eb9b734-2711-4463-9743-e5fd95c6857b_del complete
Nov 29 03:13:29 user nova-compute[75234]: DEBUG nova.compute.manager [None req-2ba9d3c2-d9fa-4fb7-9115-db6c314836c5 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] [instance: 062753a6-8f39-4e80-a549-9f0f5a4b877a] Checking state {{(pid=75234) _get_power_state /opt/stack/nova/nova/compute/manager.py:1782}}
Nov 29 03:13:29 user nova-compute[75234]: INFO nova.compute.manager [None req-a4319b7c-65c8-4a50-a342-c9d3e7cfa187 tempest-VolumesActionsTest-1495826017 tempest-VolumesActionsTest-1495826017-project-member] [instance: 1eb9b734-2711-4463-9743-e5fd95c6857b] Took 1.12 seconds to destroy the instance on the hypervisor.
Nov 29 03:13:29 user nova-compute[75234]: DEBUG oslo.service.loopingcall [None req-a4319b7c-65c8-4a50-a342-c9d3e7cfa187 tempest-VolumesActionsTest-1495826017 tempest-VolumesActionsTest-1495826017-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=75234) func /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/loopingcall.py:435}}
Nov 29 03:13:29 user nova-compute[75234]: DEBUG nova.compute.manager [-] [instance: 1eb9b734-2711-4463-9743-e5fd95c6857b] Deallocating network for instance {{(pid=75234) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2279}}
Nov 29 03:13:29 user nova-compute[75234]: DEBUG nova.network.neutron [-] [instance: 1eb9b734-2711-4463-9743-e5fd95c6857b] deallocate_for_instance() {{(pid=75234) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1802}}
Nov 29 03:13:29 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-2ba9d3c2-d9fa-4fb7-9115-db6c314836c5 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Lock "062753a6-8f39-4e80-a549-9f0f5a4b877a" "released" by "nova.compute.manager.ComputeManager.unshelve_instance.<locals>.do_unshelve_instance" :: held 28.938s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:13:29 user nova-compute[75234]: DEBUG nova.virt.libvirt.guest [None req-371aaf1d-7bb3-4224-8cba-252fdf4d9e7c tempest-TestVolumeSwap-1761484429 tempest-TestVolumeSwap-1761484429-project-admin] COPY block job progress, current cursor: 491782144 final cursor: 1073741824 {{(pid=75234) is_job_complete /opt/stack/nova/nova/virt/libvirt/guest.py:846}}
Nov 29 03:13:29 user nova-compute[75234]: DEBUG nova.network.neutron [-] [instance: 1eb9b734-2711-4463-9743-e5fd95c6857b] Updating instance_info_cache with network_info: [] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:13:29 user nova-compute[75234]: INFO nova.compute.manager [-] [instance: 1eb9b734-2711-4463-9743-e5fd95c6857b] Took 0.63 seconds to deallocate network for instance.
Nov 29 03:13:29 user nova-compute[75234]: DEBUG nova.compute.manager [req-0d95be91-52bc-4e6d-866e-4858dbf421c4 req-a62800e5-9832-4bfe-90b1-3eded14fa7d2 service nova] [instance: 1eb9b734-2711-4463-9743-e5fd95c6857b] Received event network-vif-deleted-6d9e1401-d167-4fb3-a59d-ca69b18ed417 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:13:29 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-a4319b7c-65c8-4a50-a342-c9d3e7cfa187 tempest-VolumesActionsTest-1495826017 tempest-VolumesActionsTest-1495826017-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:13:29 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-a4319b7c-65c8-4a50-a342-c9d3e7cfa187 tempest-VolumesActionsTest-1495826017 tempest-VolumesActionsTest-1495826017-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:13:30 user nova-compute[75234]: DEBUG nova.virt.libvirt.guest [None req-371aaf1d-7bb3-4224-8cba-252fdf4d9e7c tempest-TestVolumeSwap-1761484429 tempest-TestVolumeSwap-1761484429-project-admin] COPY block job progress, current cursor: 653262848 final cursor: 1073741824 {{(pid=75234) is_job_complete /opt/stack/nova/nova/virt/libvirt/guest.py:846}}
Nov 29 03:13:30 user nova-compute[75234]: DEBUG nova.virt.libvirt.guest [None req-371aaf1d-7bb3-4224-8cba-252fdf4d9e7c tempest-TestVolumeSwap-1761484429 tempest-TestVolumeSwap-1761484429-project-admin] COPY block job progress, current cursor: 810549248 final cursor: 1073741824 {{(pid=75234) is_job_complete /opt/stack/nova/nova/virt/libvirt/guest.py:846}}
Nov 29 03:13:30 user nova-compute[75234]: DEBUG nova.compute.provider_tree [None req-a4319b7c-65c8-4a50-a342-c9d3e7cfa187 tempest-VolumesActionsTest-1495826017 tempest-VolumesActionsTest-1495826017-project-member] Inventory has not changed in ProviderTree for provider: 4d291220-7c37-4839-a0e3-b23c66502b74 {{(pid=75234) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Nov 29 03:13:30 user nova-compute[75234]: DEBUG nova.scheduler.client.report [None req-a4319b7c-65c8-4a50-a342-c9d3e7cfa187 tempest-VolumesActionsTest-1495826017 tempest-VolumesActionsTest-1495826017-project-member] Inventory has not changed for provider 4d291220-7c37-4839-a0e3-b23c66502b74 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=75234) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Nov 29 03:13:30 user nova-compute[75234]: DEBUG nova.compute.manager [req-f948b19e-afc6-4799-ac2f-be3972c0d86f req-ac1de1da-df9f-4b55-8235-adb2946d0ec2 service nova] [instance: 1eb9b734-2711-4463-9743-e5fd95c6857b] Received event network-vif-unplugged-6d9e1401-d167-4fb3-a59d-ca69b18ed417 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:13:30 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-f948b19e-afc6-4799-ac2f-be3972c0d86f req-ac1de1da-df9f-4b55-8235-adb2946d0ec2 service nova] Acquiring lock "1eb9b734-2711-4463-9743-e5fd95c6857b-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:13:30 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-f948b19e-afc6-4799-ac2f-be3972c0d86f req-ac1de1da-df9f-4b55-8235-adb2946d0ec2 service nova] Lock "1eb9b734-2711-4463-9743-e5fd95c6857b-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:13:30 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-f948b19e-afc6-4799-ac2f-be3972c0d86f req-ac1de1da-df9f-4b55-8235-adb2946d0ec2 service nova] Lock "1eb9b734-2711-4463-9743-e5fd95c6857b-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:13:30 user nova-compute[75234]: DEBUG nova.compute.manager [req-f948b19e-afc6-4799-ac2f-be3972c0d86f req-ac1de1da-df9f-4b55-8235-adb2946d0ec2 service nova] [instance: 1eb9b734-2711-4463-9743-e5fd95c6857b] No waiting events found dispatching network-vif-unplugged-6d9e1401-d167-4fb3-a59d-ca69b18ed417 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:13:30 user nova-compute[75234]: WARNING nova.compute.manager [req-f948b19e-afc6-4799-ac2f-be3972c0d86f req-ac1de1da-df9f-4b55-8235-adb2946d0ec2 service nova] [instance: 1eb9b734-2711-4463-9743-e5fd95c6857b] Received unexpected event network-vif-unplugged-6d9e1401-d167-4fb3-a59d-ca69b18ed417 for instance with vm_state deleted and task_state None.
Nov 29 03:13:30 user nova-compute[75234]: DEBUG nova.compute.manager [req-f948b19e-afc6-4799-ac2f-be3972c0d86f req-ac1de1da-df9f-4b55-8235-adb2946d0ec2 service nova] [instance: 1eb9b734-2711-4463-9743-e5fd95c6857b] Received event network-vif-plugged-6d9e1401-d167-4fb3-a59d-ca69b18ed417 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:13:30 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-f948b19e-afc6-4799-ac2f-be3972c0d86f req-ac1de1da-df9f-4b55-8235-adb2946d0ec2 service nova] Acquiring lock "1eb9b734-2711-4463-9743-e5fd95c6857b-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:13:30 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-f948b19e-afc6-4799-ac2f-be3972c0d86f req-ac1de1da-df9f-4b55-8235-adb2946d0ec2 service nova] Lock "1eb9b734-2711-4463-9743-e5fd95c6857b-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:13:30 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-f948b19e-afc6-4799-ac2f-be3972c0d86f req-ac1de1da-df9f-4b55-8235-adb2946d0ec2 service nova] Lock "1eb9b734-2711-4463-9743-e5fd95c6857b-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:13:30 user nova-compute[75234]: DEBUG nova.compute.manager [req-f948b19e-afc6-4799-ac2f-be3972c0d86f req-ac1de1da-df9f-4b55-8235-adb2946d0ec2 service nova] [instance: 1eb9b734-2711-4463-9743-e5fd95c6857b] No waiting events found dispatching network-vif-plugged-6d9e1401-d167-4fb3-a59d-ca69b18ed417 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:13:30 user nova-compute[75234]: WARNING nova.compute.manager [req-f948b19e-afc6-4799-ac2f-be3972c0d86f req-ac1de1da-df9f-4b55-8235-adb2946d0ec2 service nova] [instance: 1eb9b734-2711-4463-9743-e5fd95c6857b] Received unexpected event network-vif-plugged-6d9e1401-d167-4fb3-a59d-ca69b18ed417 for instance with vm_state deleted and task_state None.
Nov 29 03:13:30 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-a4319b7c-65c8-4a50-a342-c9d3e7cfa187 tempest-VolumesActionsTest-1495826017 tempest-VolumesActionsTest-1495826017-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.735s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:13:30 user nova-compute[75234]: INFO nova.scheduler.client.report [None req-a4319b7c-65c8-4a50-a342-c9d3e7cfa187 tempest-VolumesActionsTest-1495826017 tempest-VolumesActionsTest-1495826017-project-member] Deleted allocations for instance 1eb9b734-2711-4463-9743-e5fd95c6857b
Nov 29 03:13:30 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-a4319b7c-65c8-4a50-a342-c9d3e7cfa187 tempest-VolumesActionsTest-1495826017 tempest-VolumesActionsTest-1495826017-project-member] Lock "1eb9b734-2711-4463-9743-e5fd95c6857b" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 2.805s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:13:31 user nova-compute[75234]: DEBUG nova.virt.libvirt.guest [None req-371aaf1d-7bb3-4224-8cba-252fdf4d9e7c tempest-TestVolumeSwap-1761484429 tempest-TestVolumeSwap-1761484429-project-admin] COPY block job progress, current cursor: 972029952 final cursor: 1073741824 {{(pid=75234) is_job_complete /opt/stack/nova/nova/virt/libvirt/guest.py:846}}
Nov 29 03:13:31 user nova-compute[75234]: DEBUG nova.virt.libvirt.guest [None req-371aaf1d-7bb3-4224-8cba-252fdf4d9e7c tempest-TestVolumeSwap-1761484429 tempest-TestVolumeSwap-1761484429-project-admin] COPY block job progress, current cursor: 1073741824 final cursor: 1073741824 {{(pid=75234) is_job_complete /opt/stack/nova/nova/virt/libvirt/guest.py:846}}
Nov 29 03:13:31 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:13:31 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-371aaf1d-7bb3-4224-8cba-252fdf4d9e7c tempest-TestVolumeSwap-1761484429 tempest-TestVolumeSwap-1761484429-project-admin] [instance: 3e355ae8-aab9-4ae9-b6e3-6de108ec1af9] calling os-brick to detach iSCSI Volume {{(pid=75234) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Nov 29 03:13:31 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-371aaf1d-7bb3-4224-8cba-252fdf4d9e7c tempest-TestVolumeSwap-1761484429 tempest-TestVolumeSwap-1761484429-project-admin] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f8a5050c9a0>, {'target_iqn': 'iqn.2010-10.org.openstack:4ed484cc-9de2-42df-8d99-637f3b47d474', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'mBlevMXQ', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdb'}, None), 'kwargs': False}}" {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:176}}
Nov 29 03:13:31 user nova-compute[75234]: WARNING os_brick.initiator.connectors.base [None req-371aaf1d-7bb3-4224-8cba-252fdf4d9e7c tempest-TestVolumeSwap-1761484429 tempest-TestVolumeSwap-1761484429-project-admin] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Nov 29 03:13:31 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-371aaf1d-7bb3-4224-8cba-252fdf4d9e7c tempest-TestVolumeSwap-1761484429 tempest-TestVolumeSwap-1761484429-project-admin] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:68}}
Nov 29 03:13:31 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-371aaf1d-7bb3-4224-8cba-252fdf4d9e7c tempest-TestVolumeSwap-1761484429 tempest-TestVolumeSwap-1761484429-project-admin] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:73}}
Nov 29 03:13:31 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:13:31 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.021s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:13:31 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[27abe016-54a3-4364-947b-34df4aace8b6]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:4ed484cc-9de2-42df-8d99-637f3b47d474\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:56237f4c-4138-4f50-aafb-9c0f9d4bb1cf\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:6a4d34d3-9974-43bc-afcb-9169c893d9b7\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:94dadf41-624e-4581-81cf-eb5eaaecafb2\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:13:31 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-371aaf1d-7bb3-4224-8cba-252fdf4d9e7c tempest-TestVolumeSwap-1761484429 tempest-TestVolumeSwap-1761484429-project-admin] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS:
Nov 29 03:13:31 user nova-compute[75234]: No targets found.
Nov 29 03:13:31 user nova-compute[75234]: iSNS:
Nov 29 03:13:31 user nova-compute[75234]: No targets found.
Nov 29 03:13:31 user nova-compute[75234]: STATIC:
Nov 29 03:13:31 user nova-compute[75234]: Target: iqn.2010-10.org.openstack:4ed484cc-9de2-42df-8d99-637f3b47d474
Nov 29 03:13:31 user nova-compute[75234]:         Portal: 172.16.0.220:3260,-1
Nov 29 03:13:31 user nova-compute[75234]:                 Iface Name: default
Nov 29 03:13:31 user nova-compute[75234]: Target: iqn.2010-10.org.openstack:56237f4c-4138-4f50-aafb-9c0f9d4bb1cf
Nov 29 03:13:31 user nova-compute[75234]:         Portal: 172.16.0.220:3260,-1
Nov 29 03:13:31 user nova-compute[75234]:                 Iface Name: default
Nov 29 03:13:31 user nova-compute[75234]: Target: iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b
Nov 29 03:13:31 user nova-compute[75234]:         Portal: 172.16.0.220:3260,-1
Nov 29 03:13:31 user nova-compute[75234]:                 Iface Name: default
Nov 29 03:13:31 user nova-compute[75234]: Target: iqn.2010-10.org.openstack:6a4d34d3-9974-43bc-afcb-9169c893d9b7
Nov 29 03:13:31 user nova-compute[75234]:         Portal: 172.16.0.220:3260,-1
Nov 29 03:13:31 user nova-compute[75234]:                 Iface Name: default
Nov 29 03:13:31 user nova-compute[75234]: Target: iqn.2010-10.org.openstack:94dadf41-624e-4581-81cf-eb5eaaecafb2
Nov 29 03:13:31 user nova-compute[75234]:         Portal: 172.16.0.220:3260,-1
Nov 29 03:13:31 user nova-compute[75234]:                 Iface Name: default
Nov 29 03:13:31 user nova-compute[75234]: FIRMWARE:
Nov 29 03:13:31 user nova-compute[75234]: No targets found.
Nov 29 03:13:31 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:13:31 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-371aaf1d-7bb3-4224-8cba-252fdf4d9e7c tempest-TestVolumeSwap-1761484429 tempest-TestVolumeSwap-1761484429-project-admin] Regex to get portals from discoverydb: ^SENDTARGETS:
Nov 29 03:13:31 user nova-compute[75234]: .*?^DiscoveryAddress: 172.16.0.220,3260.*?
Nov 29 03:13:31 user nova-compute[75234]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=75234) _get_discoverydb_portals /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:382}}
Nov 29 03:13:31 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-371aaf1d-7bb3-4224-8cba-252fdf4d9e7c tempest-TestVolumeSwap-1761484429 tempest-TestVolumeSwap-1761484429-project-admin] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:4ed484cc-9de2-42df-8d99-637f3b47d474', 0)] {{(pid=75234) _get_connection_devices /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:830}}
Nov 29 03:13:31 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:13:31 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.017s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:13:31 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[ce17c931-db9a-4fbd-b215-172570386b0d]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:4ed484cc-9de2-42df-8d99-637f3b47d474\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:56237f4c-4138-4f50-aafb-9c0f9d4bb1cf\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:6a4d34d3-9974-43bc-afcb-9169c893d9b7\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:94dadf41-624e-4581-81cf-eb5eaaecafb2\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:13:31 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:13:31 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.034s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:13:31 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[484e31a8-1392-448a-8b35-50e30c8b60aa]: (4, ('tcp: [20] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b (non-flash)\ntcp: [21] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:56237f4c-4138-4f50-aafb-9c0f9d4bb1cf (non-flash)\ntcp: [22] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4ed484cc-9de2-42df-8d99-637f3b47d474 (non-flash)\ntcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6a4d34d3-9974-43bc-afcb-9169c893d9b7 (non-flash)\ntcp: [24] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:94dadf41-624e-4581-81cf-eb5eaaecafb2 (non-flash)\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:13:31 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-371aaf1d-7bb3-4224-8cba-252fdf4d9e7c tempest-TestVolumeSwap-1761484429 tempest-TestVolumeSwap-1761484429-project-admin] iscsiadm ('-m', 'session'): stdout=tcp: [20] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b (non-flash)
Nov 29 03:13:31 user nova-compute[75234]: tcp: [21] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:56237f4c-4138-4f50-aafb-9c0f9d4bb1cf (non-flash)
Nov 29 03:13:31 user nova-compute[75234]: tcp: [22] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4ed484cc-9de2-42df-8d99-637f3b47d474 (non-flash)
Nov 29 03:13:31 user nova-compute[75234]: tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6a4d34d3-9974-43bc-afcb-9169c893d9b7 (non-flash)
Nov 29 03:13:31 user nova-compute[75234]: tcp: [24] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:94dadf41-624e-4581-81cf-eb5eaaecafb2 (non-flash)
Nov 29 03:13:31 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:13:31 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-371aaf1d-7bb3-4224-8cba-252fdf4d9e7c tempest-TestVolumeSwap-1761484429 tempest-TestVolumeSwap-1761484429-project-admin] iscsi session list stdout=tcp: [20] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b (non-flash)
Nov 29 03:13:31 user nova-compute[75234]: tcp: [21] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:56237f4c-4138-4f50-aafb-9c0f9d4bb1cf (non-flash)
Nov 29 03:13:31 user nova-compute[75234]: tcp: [22] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4ed484cc-9de2-42df-8d99-637f3b47d474 (non-flash)
Nov 29 03:13:31 user nova-compute[75234]: tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6a4d34d3-9974-43bc-afcb-9169c893d9b7 (non-flash)
Nov 29 03:13:31 user nova-compute[75234]: tcp: [24] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:94dadf41-624e-4581-81cf-eb5eaaecafb2 (non-flash)
Nov 29 03:13:31 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsi_session /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Nov 29 03:13:31 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-371aaf1d-7bb3-4224-8cba-252fdf4d9e7c tempest-TestVolumeSwap-1761484429 tempest-TestVolumeSwap-1761484429-project-admin] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7f8a50626320>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:4ed484cc-9de2-42df-8d99-637f3b47d474'): ({'sdb'}, set())}) {{(pid=75234) _get_connection_devices /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:863}}
Nov 29 03:13:31 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-371aaf1d-7bb3-4224-8cba-252fdf4d9e7c tempest-TestVolumeSwap-1761484429 tempest-TestVolumeSwap-1761484429-project-admin] Removing single pathed devices sdb {{(pid=75234) remove_connection /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:374}}
Nov 29 03:13:31 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:13:31 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.019s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:13:31 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[c475fa51-d056-4918-a3a1-49eee6645066]: (4, ('path checker states:\nup                  5\n\npaths: 0\nbusy: False\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:13:31 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdb {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:13:31 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdb" returned: 0 in 0.022s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:13:31 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[2ecf4481-5878-4ab8-9446-3f83131ef64e]: (4, ('ok\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:13:31 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-371aaf1d-7bb3-4224-8cba-252fdf4d9e7c tempest-TestVolumeSwap-1761484429 tempest-TestVolumeSwap-1761484429-project-admin] Flushing IO for device /dev/sdb {{(pid=75234) flush_device_io /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:434}}
Nov 29 03:13:31 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdb {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:13:31 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdb" returned: 0 in 0.014s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:13:31 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[ecc98242-04a6-498f-aa8f-76d4102d8188]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:13:31 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-371aaf1d-7bb3-4224-8cba-252fdf4d9e7c tempest-TestVolumeSwap-1761484429 tempest-TestVolumeSwap-1761484429-project-admin] Remove SCSI device /dev/sdb with /sys/block/sdb/device/delete {{(pid=75234) remove_scsi_device /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:148}}
Nov 29 03:13:31 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdb/device/delete {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:13:31 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdb/device/delete" returned: 0 in 0.033s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:13:31 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[6d3e635a-f354-4fb3-8499-efa08d74d206]: (4, ('1', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:13:31 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-371aaf1d-7bb3-4224-8cba-252fdf4d9e7c tempest-TestVolumeSwap-1761484429 tempest-TestVolumeSwap-1761484429-project-admin] Checking to see if SCSI volumes sdb have been removed. {{(pid=75234) wait_for_volumes_removal /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:156}}
Nov 29 03:13:31 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-371aaf1d-7bb3-4224-8cba-252fdf4d9e7c tempest-TestVolumeSwap-1761484429 tempest-TestVolumeSwap-1761484429-project-admin] SCSI volumes sdb have been removed. {{(pid=75234) wait_for_volumes_removal /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:166}}
Nov 29 03:13:31 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-371aaf1d-7bb3-4224-8cba-252fdf4d9e7c tempest-TestVolumeSwap-1761484429 tempest-TestVolumeSwap-1761484429-project-admin] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:4ed484cc-9de2-42df-8d99-637f3b47d474')] {{(pid=75234) _disconnect_connection /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Nov 29 03:13:31 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:4ed484cc-9de2-42df-8d99-637f3b47d474 -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:13:31 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:4ed484cc-9de2-42df-8d99-637f3b47d474 -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.018s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:13:31 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[d9a40212-a103-4b37-9d1d-20e2b0ab2bc5]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:13:31 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-371aaf1d-7bb3-4224-8cba-252fdf4d9e7c tempest-TestVolumeSwap-1761484429 tempest-TestVolumeSwap-1761484429-project-admin] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:13:31 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:4ed484cc-9de2-42df-8d99-637f3b47d474 -p 172.16.0.220:3260 --logout {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:13:31 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:4ed484cc-9de2-42df-8d99-637f3b47d474 -p 172.16.0.220:3260 --logout" returned: 0 in 0.064s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:13:31 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[54d3dd28-ac7e-420f-909d-862d99c8c2ce]: (4, ('Logging out of session [sid: 22, target: iqn.2010-10.org.openstack:4ed484cc-9de2-42df-8d99-637f3b47d474, portal: 172.16.0.220,3260]\nLogout of [sid: 22, target: iqn.2010-10.org.openstack:4ed484cc-9de2-42df-8d99-637f3b47d474, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:13:31 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-371aaf1d-7bb3-4224-8cba-252fdf4d9e7c tempest-TestVolumeSwap-1761484429 tempest-TestVolumeSwap-1761484429-project-admin] iscsiadm ('--logout',): stdout=Logging out of session [sid: 22, target: iqn.2010-10.org.openstack:4ed484cc-9de2-42df-8d99-637f3b47d474, portal: 172.16.0.220,3260]
Nov 29 03:13:31 user nova-compute[75234]: Logout of [sid: 22, target: iqn.2010-10.org.openstack:4ed484cc-9de2-42df-8d99-637f3b47d474, portal: 172.16.0.220,3260] successful.
Nov 29 03:13:31 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:13:31 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:4ed484cc-9de2-42df-8d99-637f3b47d474 -p 172.16.0.220:3260 --op delete {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:13:31 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:4ed484cc-9de2-42df-8d99-637f3b47d474 -p 172.16.0.220:3260 --op delete" returned: 0 in 0.026s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:13:31 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[87f36dcf-1cfd-4c15-a56b-3af4fa43201a]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:13:31 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-371aaf1d-7bb3-4224-8cba-252fdf4d9e7c tempest-TestVolumeSwap-1761484429 tempest-TestVolumeSwap-1761484429-project-admin] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:13:31 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-371aaf1d-7bb3-4224-8cba-252fdf4d9e7c tempest-TestVolumeSwap-1761484429 tempest-TestVolumeSwap-1761484429-project-admin] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.316s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:87}}
Nov 29 03:13:31 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-371aaf1d-7bb3-4224-8cba-252fdf4d9e7c tempest-TestVolumeSwap-1761484429 tempest-TestVolumeSwap-1761484429-project-admin] <== disconnect_volume: return (317ms) None {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:203}}
Nov 29 03:13:31 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-371aaf1d-7bb3-4224-8cba-252fdf4d9e7c tempest-TestVolumeSwap-1761484429 tempest-TestVolumeSwap-1761484429-project-admin] [instance: 3e355ae8-aab9-4ae9-b6e3-6de108ec1af9] Disconnected iSCSI Volume {{(pid=75234) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:79}}
Nov 29 03:13:32 user nova-compute[75234]: DEBUG nova.compute.manager [None req-371aaf1d-7bb3-4224-8cba-252fdf4d9e7c tempest-TestVolumeSwap-1761484429 tempest-TestVolumeSwap-1761484429-project-admin] swap_volume: Driver volume swap returned, new connection_info is now : {'driver_volume_type': 'iscsi', 'data': {'target_iqn': 'iqn.2010-10.org.openstack:94dadf41-624e-4581-81cf-eb5eaaecafb2', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'CIoXTDLA', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sde'}, 'status': 'reserved', 'instance': '3e355ae8-aab9-4ae9-b6e3-6de108ec1af9', 'attached_at': '', 'detached_at': '', 'volume_id': '94dadf41-624e-4581-81cf-eb5eaaecafb2', 'serial'} {{(pid=75234) _swap_volume /opt/stack/nova/nova/compute/manager.py:7791}}
Nov 29 03:13:32 user nova-compute[75234]: DEBUG nova.compute.manager [None req-371aaf1d-7bb3-4224-8cba-252fdf4d9e7c tempest-TestVolumeSwap-1761484429 tempest-TestVolumeSwap-1761484429-project-admin] [instance: 3e355ae8-aab9-4ae9-b6e3-6de108ec1af9] swap_volume: removing Cinder connection for volume 4ed484cc-9de2-42df-8d99-637f3b47d474 {{(pid=75234) _swap_volume /opt/stack/nova/nova/compute/manager.py:7836}}
Nov 29 03:13:32 user nova-compute[75234]: DEBUG nova.compute.manager [None req-371aaf1d-7bb3-4224-8cba-252fdf4d9e7c tempest-TestVolumeSwap-1761484429 tempest-TestVolumeSwap-1761484429-project-admin] [instance: 3e355ae8-aab9-4ae9-b6e3-6de108ec1af9] swap_volume: Updating volume 4ed484cc-9de2-42df-8d99-637f3b47d474 BDM record with {'connection_info': '{"driver_volume_type": "iscsi", "data": {"target_iqn": "iqn.2010-10.org.openstack:94dadf41-624e-4581-81cf-eb5eaaecafb2", "target_portal": "172.16.0.220:3260", "target_discovered": false, "auth_method": "CHAP", "auth_username": "CIoXTDLA", "auth_password": "dJf5OjoI73XFQs", "target_lun": 0, "qos_specs": null, "access_mode": "rw", "encrypted": false, "cacheable": false, "device_path": "/dev/sde", "volume_id": "94dadf41-624e-4581-81cf-eb5eaaecafb2"}, "status": "reserved", "instance": "3e355ae8-aab9-4ae9-b6e3-6de108ec1af9", "attached_at": "", "detached_at": "", "volume_id": "94dadf41-624e-4581-81cf-eb5eaaecafb2", "serial": "94dadf41-624e-4581-81cf-eb5eaaecafb2"}', 'source_type': 'volume', 'destination_type': 'volume', 'snapshot_id': None, 'volume_id': '94dadf41-624e-4581-81cf-eb5eaaecafb2', 'no_device': None, 'attachment_id': '5b7ac024-adaa-4049-bd30-3cf9419a3a6f'} {{(pid=75234) _do_swap_volume /opt/stack/nova/nova/compute/manager.py:7984}}
Nov 29 03:13:32 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-371aaf1d-7bb3-4224-8cba-252fdf4d9e7c tempest-TestVolumeSwap-1761484429 tempest-TestVolumeSwap-1761484429-project-admin] Lock "3e355ae8-aab9-4ae9-b6e3-6de108ec1af9" "released" by "nova.compute.manager.ComputeManager.swap_volume.<locals>._do_locked_swap_volume" :: held 15.289s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:13:33 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-17033895-5689-4ec0-83b3-2d4335860c97 tempest-TestVolumeSwap-1761484429 tempest-TestVolumeSwap-1761484429-project-admin] Acquiring lock "3e355ae8-aab9-4ae9-b6e3-6de108ec1af9" by "nova.compute.manager.ComputeManager.swap_volume.<locals>._do_locked_swap_volume" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:13:33 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-17033895-5689-4ec0-83b3-2d4335860c97 tempest-TestVolumeSwap-1761484429 tempest-TestVolumeSwap-1761484429-project-admin] Lock "3e355ae8-aab9-4ae9-b6e3-6de108ec1af9" acquired by "nova.compute.manager.ComputeManager.swap_volume.<locals>._do_locked_swap_volume" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:13:33 user nova-compute[75234]: DEBUG os_brick.utils [None req-17033895-5689-4ec0-83b3-2d4335860c97 tempest-TestVolumeSwap-1761484429 tempest-TestVolumeSwap-1761484429-project-admin] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:176}}
Nov 29 03:13:33 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:13:33 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.016s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:13:33 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[b321a75a-4182-40d4-9bc4-836f933f6d34]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:dd1368c5ba2c\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:13:33 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt -v / -n -o SOURCE {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:13:33 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt -v / -n -o SOURCE" returned: 0 in 0.031s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:13:33 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[ffeae1da-724c-4b4f-bc69-824f32b4e828]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:13:33 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:13:33 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.020s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:13:33 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[bafec39d-5de7-4283-aa0d-df1427b3b699]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:13:33 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[5f866b40-27f0-479f-86c1-a418c64b377d]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:13:33 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-17033895-5689-4ec0-83b3-2d4335860c97 tempest-TestVolumeSwap-1761484429 tempest-TestVolumeSwap-1761484429-project-admin] Running cmd (subprocess): nvme version {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:13:33 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-17033895-5689-4ec0-83b3-2d4335860c97 tempest-TestVolumeSwap-1761484429 tempest-TestVolumeSwap-1761484429-project-admin] 'nvme version' failed. Not Retrying. {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:473}}
Nov 29 03:13:33 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.nvmeof [None req-17033895-5689-4ec0-83b3-2d4335860c97 tempest-TestVolumeSwap-1761484429 tempest-TestVolumeSwap-1761484429-project-admin] nvme not present on system {{(pid=75234) nvme_present /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/nvmeof.py:756}}
Nov 29 03:13:33 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:13:33 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:473}}
Nov 29 03:13:33 user nova-compute[75234]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Nov 29 03:13:33 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[b3050609-2d5f-4647-852e-df4da5f484be]: (4, '') {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:13:33 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.lightos [None req-17033895-5689-4ec0-83b3-2d4335860c97 tempest-TestVolumeSwap-1761484429 tempest-TestVolumeSwap-1761484429-project-admin] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=75234) find_dsc /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/lightos.py:98}}
Nov 29 03:13:33 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.lightos [None req-17033895-5689-4ec0-83b3-2d4335860c97 tempest-TestVolumeSwap-1761484429 tempest-TestVolumeSwap-1761484429-project-admin] LIGHTOS: did not find dsc, continuing anyway. {{(pid=75234) get_connector_properties /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/lightos.py:76}}
Nov 29 03:13:33 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.lightos [None req-17033895-5689-4ec0-83b3-2d4335860c97 tempest-TestVolumeSwap-1761484429 tempest-TestVolumeSwap-1761484429-project-admin] LIGHTOS: no hostnqn found. {{(pid=75234) get_connector_properties /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/lightos.py:84}}
Nov 29 03:13:33 user nova-compute[75234]: DEBUG os_brick.utils [None req-17033895-5689-4ec0-83b3-2d4335860c97 tempest-TestVolumeSwap-1761484429 tempest-TestVolumeSwap-1761484429-project-admin] <== get_connector_properties: return (162ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:dd1368c5ba2c', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:203}}
Nov 29 03:13:34 user nova-compute[75234]: INFO nova.compute.manager [None req-17033895-5689-4ec0-83b3-2d4335860c97 tempest-TestVolumeSwap-1761484429 tempest-TestVolumeSwap-1761484429-project-admin] [instance: 3e355ae8-aab9-4ae9-b6e3-6de108ec1af9] Swapping volume 94dadf41-624e-4581-81cf-eb5eaaecafb2 for 4ed484cc-9de2-42df-8d99-637f3b47d474
Nov 29 03:13:34 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:13:39 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:13:42 user nova-compute[75234]: DEBUG nova.compute.manager [None req-17033895-5689-4ec0-83b3-2d4335860c97 tempest-TestVolumeSwap-1761484429 tempest-TestVolumeSwap-1761484429-project-admin] [instance: 3e355ae8-aab9-4ae9-b6e3-6de108ec1af9] swap_volume: Calling driver volume swap with connection infos: new: {'driver_volume_type': 'iscsi', 'data': {'target_iqn': 'iqn.2010-10.org.openstack:4ed484cc-9de2-42df-8d99-637f3b47d474', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'mBlevMXQ', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}, 'status': 'reserved', 'instance': '3e355ae8-aab9-4ae9-b6e3-6de108ec1af9', 'attached_at': '', 'detached_at': '', 'volume_id': '4ed484cc-9de2-42df-8d99-637f3b47d474', 'serial': '4ed484cc-9de2-42df-8d99-637f3b47d474'}; old: {'driver_volume_type': 'iscsi', 'data': {'target_iqn': 'iqn.2010-10.org.openstack:94dadf41-624e-4581-81cf-eb5eaaecafb2', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'CIoXTDLA', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sde', 'volume_id': '94dadf41-624e-4581-81cf-eb5eaaecafb2'}, 'status': 'reserved', 'instance': '3e355ae8-aab9-4ae9-b6e3-6de108ec1af9', 'attached_at': '', 'detached_at': '', 'volume_id': '94dadf41-624e-4581-81cf-eb5eaaecafb2', 'serial'} {{(pid=75234) _swap_volume /opt/stack/nova/nova/compute/manager.py:7782}}
Nov 29 03:13:42 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-17033895-5689-4ec0-83b3-2d4335860c97 tempest-TestVolumeSwap-1761484429 tempest-TestVolumeSwap-1761484429-project-admin] [instance: 3e355ae8-aab9-4ae9-b6e3-6de108ec1af9] Calling os-brick to attach iSCSI Volume {{(pid=75234) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Nov 29 03:13:42 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-17033895-5689-4ec0-83b3-2d4335860c97 tempest-TestVolumeSwap-1761484429 tempest-TestVolumeSwap-1761484429-project-admin] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f8a5050c9a0>, 'connection_properties': {'target_iqn': 'iqn.2010-10.org.openstack:4ed484cc-9de2-42df-8d99-637f3b47d474', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'mBlevMXQ', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:176}}
Nov 29 03:13:42 user nova-compute[75234]: WARNING os_brick.initiator.connectors.base [None req-17033895-5689-4ec0-83b3-2d4335860c97 tempest-TestVolumeSwap-1761484429 tempest-TestVolumeSwap-1761484429-project-admin] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Nov 29 03:13:42 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-17033895-5689-4ec0-83b3-2d4335860c97 tempest-TestVolumeSwap-1761484429 tempest-TestVolumeSwap-1761484429-project-admin] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:68}}
Nov 29 03:13:42 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-17033895-5689-4ec0-83b3-2d4335860c97 tempest-TestVolumeSwap-1761484429 tempest-TestVolumeSwap-1761484429-project-admin] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.002s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:73}}
Nov 29 03:13:42 user nova-compute[75234]: WARNING os_brick.initiator.connectors.base [None req-17033895-5689-4ec0-83b3-2d4335860c97 tempest-TestVolumeSwap-1761484429 tempest-TestVolumeSwap-1761484429-project-admin] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Nov 29 03:13:42 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-17033895-5689-4ec0-83b3-2d4335860c97 tempest-TestVolumeSwap-1761484429 tempest-TestVolumeSwap-1761484429-project-admin] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:4ed484cc-9de2-42df-8d99-637f3b47d474" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:68}}
Nov 29 03:13:42 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-17033895-5689-4ec0-83b3-2d4335860c97 tempest-TestVolumeSwap-1761484429 tempest-TestVolumeSwap-1761484429-project-admin] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:4ed484cc-9de2-42df-8d99-637f3b47d474" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:73}}
Nov 29 03:13:42 user nova-compute[75234]: INFO os_brick.initiator.connectors.iscsi [None req-17033895-5689-4ec0-83b3-2d4335860c97 tempest-TestVolumeSwap-1761484429 tempest-TestVolumeSwap-1761484429-project-admin] Trying to connect to iSCSI portal 172.16.0.220:3260
Nov 29 03:13:42 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:4ed484cc-9de2-42df-8d99-637f3b47d474 -p 172.16.0.220:3260 {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:13:42 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:4ed484cc-9de2-42df-8d99-637f3b47d474 -p 172.16.0.220:3260" returned: 21 in 0.017s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:13:42 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[436deea6-be02-4cd1-9fb3-e0dac1b24bd4]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:13:42 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-17033895-5689-4ec0-83b3-2d4335860c97 tempest-TestVolumeSwap-1761484429 tempest-TestVolumeSwap-1761484429-project-admin] iscsiadm (): stdout= stderr=iscsiadm: No records found
Nov 29 03:13:42 user nova-compute[75234]:  {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:13:42 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:4ed484cc-9de2-42df-8d99-637f3b47d474 -p 172.16.0.220:3260 --interface default --op new {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:13:42 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:4ed484cc-9de2-42df-8d99-637f3b47d474 -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.018s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:13:42 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[1a994520-c9f3-4aba-92a5-d13f64f91604]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:4ed484cc-9de2-42df-8d99-637f3b47d474] added\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:13:42 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-17033895-5689-4ec0-83b3-2d4335860c97 tempest-TestVolumeSwap-1761484429 tempest-TestVolumeSwap-1761484429-project-admin] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:4ed484cc-9de2-42df-8d99-637f3b47d474] added
Nov 29 03:13:42 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:13:42 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:4ed484cc-9de2-42df-8d99-637f3b47d474 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:13:42 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:4ed484cc-9de2-42df-8d99-637f3b47d474 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.017s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:13:42 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[561e9b8f-1e93-4ec4-b931-b6c8401fef77]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:13:42 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-17033895-5689-4ec0-83b3-2d4335860c97 tempest-TestVolumeSwap-1761484429 tempest-TestVolumeSwap-1761484429-project-admin] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:13:42 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:4ed484cc-9de2-42df-8d99-637f3b47d474 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:13:42 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:4ed484cc-9de2-42df-8d99-637f3b47d474 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.018s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:13:42 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[34b652da-d30f-452d-9175-7417683c14f8]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:13:42 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-17033895-5689-4ec0-83b3-2d4335860c97 tempest-TestVolumeSwap-1761484429 tempest-TestVolumeSwap-1761484429-project-admin] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:13:42 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:4ed484cc-9de2-42df-8d99-637f3b47d474 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v mBlevMXQ {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:13:42 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:4ed484cc-9de2-42df-8d99-637f3b47d474 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v mBlevMXQ" returned: 0 in 0.024s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:13:42 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[d54da7cd-80e0-4ef7-acb4-d813a8026563]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:13:42 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-17033895-5689-4ec0-83b3-2d4335860c97 tempest-TestVolumeSwap-1761484429 tempest-TestVolumeSwap-1761484429-project-admin] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'mBlevMXQ'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:13:42 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:4ed484cc-9de2-42df-8d99-637f3b47d474 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:13:42 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:4ed484cc-9de2-42df-8d99-637f3b47d474 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.018s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:13:42 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[2246df83-762c-4598-b182-a984dcf5e71f]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:13:42 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-17033895-5689-4ec0-83b3-2d4335860c97 tempest-TestVolumeSwap-1761484429 tempest-TestVolumeSwap-1761484429-project-admin] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:13:42 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:13:42 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.023s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:13:42 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[2334e667-4a08-4d74-9a79-6df1a014cd0d]: (4, ('tcp: [20] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b (non-flash)\ntcp: [21] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:56237f4c-4138-4f50-aafb-9c0f9d4bb1cf (non-flash)\ntcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6a4d34d3-9974-43bc-afcb-9169c893d9b7 (non-flash)\ntcp: [24] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:94dadf41-624e-4581-81cf-eb5eaaecafb2 (non-flash)\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:13:42 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-17033895-5689-4ec0-83b3-2d4335860c97 tempest-TestVolumeSwap-1761484429 tempest-TestVolumeSwap-1761484429-project-admin] iscsiadm ('-m', 'session'): stdout=tcp: [20] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b (non-flash)
Nov 29 03:13:42 user nova-compute[75234]: tcp: [21] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:56237f4c-4138-4f50-aafb-9c0f9d4bb1cf (non-flash)
Nov 29 03:13:42 user nova-compute[75234]: tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6a4d34d3-9974-43bc-afcb-9169c893d9b7 (non-flash)
Nov 29 03:13:42 user nova-compute[75234]: tcp: [24] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:94dadf41-624e-4581-81cf-eb5eaaecafb2 (non-flash)
Nov 29 03:13:42 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:13:42 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-17033895-5689-4ec0-83b3-2d4335860c97 tempest-TestVolumeSwap-1761484429 tempest-TestVolumeSwap-1761484429-project-admin] iscsi session list stdout=tcp: [20] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b (non-flash)
Nov 29 03:13:42 user nova-compute[75234]: tcp: [21] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:56237f4c-4138-4f50-aafb-9c0f9d4bb1cf (non-flash)
Nov 29 03:13:42 user nova-compute[75234]: tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6a4d34d3-9974-43bc-afcb-9169c893d9b7 (non-flash)
Nov 29 03:13:42 user nova-compute[75234]: tcp: [24] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:94dadf41-624e-4581-81cf-eb5eaaecafb2 (non-flash)
Nov 29 03:13:42 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsi_session /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Nov 29 03:13:42 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:4ed484cc-9de2-42df-8d99-637f3b47d474 -p 172.16.0.220:3260 --login {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:13:42 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:4ed484cc-9de2-42df-8d99-637f3b47d474 -p 172.16.0.220:3260 --login" returned: 0 in 0.067s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:13:42 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[d682c190-b37c-4765-9346-25b7c825eb00]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:4ed484cc-9de2-42df-8d99-637f3b47d474, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:4ed484cc-9de2-42df-8d99-637f3b47d474, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:13:42 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-17033895-5689-4ec0-83b3-2d4335860c97 tempest-TestVolumeSwap-1761484429 tempest-TestVolumeSwap-1761484429-project-admin] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:4ed484cc-9de2-42df-8d99-637f3b47d474, portal: 172.16.0.220,3260]
Nov 29 03:13:42 user nova-compute[75234]: Login to [iface: default, target: iqn.2010-10.org.openstack:4ed484cc-9de2-42df-8d99-637f3b47d474, portal: 172.16.0.220,3260] successful.
Nov 29 03:13:42 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:13:42 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:4ed484cc-9de2-42df-8d99-637f3b47d474 -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:13:42 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:4ed484cc-9de2-42df-8d99-637f3b47d474 -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.018s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:13:42 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[e99a2e18-cac6-45d5-8f33-5769486660e9]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:13:42 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-17033895-5689-4ec0-83b3-2d4335860c97 tempest-TestVolumeSwap-1761484429 tempest-TestVolumeSwap-1761484429-project-admin] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:13:42 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:13:42 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.028s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:13:42 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[e4e12285-5014-4a4b-8e95-00eaaa0b89c3]: (4, ('tcp: [20] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b (non-flash)\ntcp: [21] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:56237f4c-4138-4f50-aafb-9c0f9d4bb1cf (non-flash)\ntcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6a4d34d3-9974-43bc-afcb-9169c893d9b7 (non-flash)\ntcp: [24] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:94dadf41-624e-4581-81cf-eb5eaaecafb2 (non-flash)\ntcp: [25] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4ed484cc-9de2-42df-8d99-637f3b47d474 (non-flash)\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:13:42 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-17033895-5689-4ec0-83b3-2d4335860c97 tempest-TestVolumeSwap-1761484429 tempest-TestVolumeSwap-1761484429-project-admin] iscsiadm ('-m', 'session'): stdout=tcp: [20] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b (non-flash)
Nov 29 03:13:42 user nova-compute[75234]: tcp: [21] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:56237f4c-4138-4f50-aafb-9c0f9d4bb1cf (non-flash)
Nov 29 03:13:42 user nova-compute[75234]: tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6a4d34d3-9974-43bc-afcb-9169c893d9b7 (non-flash)
Nov 29 03:13:42 user nova-compute[75234]: tcp: [24] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:94dadf41-624e-4581-81cf-eb5eaaecafb2 (non-flash)
Nov 29 03:13:42 user nova-compute[75234]: tcp: [25] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4ed484cc-9de2-42df-8d99-637f3b47d474 (non-flash)
Nov 29 03:13:42 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:13:42 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-17033895-5689-4ec0-83b3-2d4335860c97 tempest-TestVolumeSwap-1761484429 tempest-TestVolumeSwap-1761484429-project-admin] iscsi session list stdout=tcp: [20] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b (non-flash)
Nov 29 03:13:42 user nova-compute[75234]: tcp: [21] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:56237f4c-4138-4f50-aafb-9c0f9d4bb1cf (non-flash)
Nov 29 03:13:42 user nova-compute[75234]: tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6a4d34d3-9974-43bc-afcb-9169c893d9b7 (non-flash)
Nov 29 03:13:42 user nova-compute[75234]: tcp: [24] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:94dadf41-624e-4581-81cf-eb5eaaecafb2 (non-flash)
Nov 29 03:13:42 user nova-compute[75234]: tcp: [25] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4ed484cc-9de2-42df-8d99-637f3b47d474 (non-flash)
Nov 29 03:13:42 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsi_session /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Nov 29 03:13:42 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-17033895-5689-4ec0-83b3-2d4335860c97 tempest-TestVolumeSwap-1761484429 tempest-TestVolumeSwap-1761484429-project-admin] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:4ed484cc-9de2-42df-8d99-637f3b47d474" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.288s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:87}}
Nov 29 03:13:42 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-17033895-5689-4ec0-83b3-2d4335860c97 tempest-TestVolumeSwap-1761484429 tempest-TestVolumeSwap-1761484429-project-admin] Connected to 172.16.0.220:3260 {{(pid=75234) _connect_vol /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:644}}
Nov 29 03:13:42 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-17033895-5689-4ec0-83b3-2d4335860c97 tempest-TestVolumeSwap-1761484429 tempest-TestVolumeSwap-1761484429-project-admin] HCTL ('33', '-', '-', 0) found on session 25 with lun 0 {{(pid=75234) get_hctl /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:770}}
Nov 29 03:13:42 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-17033895-5689-4ec0-83b3-2d4335860c97 tempest-TestVolumeSwap-1761484429 tempest-TestVolumeSwap-1761484429-project-admin] Scanning host 33 c: -, t: -, l: 0) {{(pid=75234) scan_iscsi /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:795}}
Nov 29 03:13:42 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host33/scan {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:13:42 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host33/scan" returned: 0 in 0.017s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:13:42 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[61e654ed-97a7-4ac8-9a1b-57ad5bb1ac66]: (4, ('- - 0', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:13:42 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-17033895-5689-4ec0-83b3-2d4335860c97 tempest-TestVolumeSwap-1761484429 tempest-TestVolumeSwap-1761484429-project-admin] Searching for a device in session 25 and hctl ['33', '*', '*', 0] yield: None {{(pid=75234) device_name_by_hctl /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:789}}
Nov 29 03:13:43 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-17033895-5689-4ec0-83b3-2d4335860c97 tempest-TestVolumeSwap-1761484429 tempest-TestVolumeSwap-1761484429-project-admin] Searching for a device in session 25 and hctl ['33', '*', '*', 0] yield: sdb {{(pid=75234) device_name_by_hctl /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:789}}
Nov 29 03:13:43 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-17033895-5689-4ec0-83b3-2d4335860c97 tempest-TestVolumeSwap-1761484429 tempest-TestVolumeSwap-1761484429-project-admin] Connected to sdb using {'target_iqn': 'iqn.2010-10.org.openstack:4ed484cc-9de2-42df-8d99-637f3b47d474', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'mBlevMXQ', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False} {{(pid=75234) _connect_vol /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:672}}
Nov 29 03:13:43 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-17033895-5689-4ec0-83b3-2d4335860c97 tempest-TestVolumeSwap-1761484429 tempest-TestVolumeSwap-1761484429-project-admin] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.321s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:87}}
Nov 29 03:13:43 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-17033895-5689-4ec0-83b3-2d4335860c97 tempest-TestVolumeSwap-1761484429 tempest-TestVolumeSwap-1761484429-project-admin] <== connect_volume: return (1324ms) {'type': 'block', 'scsi_wwn': '23338363665343038', 'path': '/dev/sdb'} {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:203}}
Nov 29 03:13:43 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-17033895-5689-4ec0-83b3-2d4335860c97 tempest-TestVolumeSwap-1761484429 tempest-TestVolumeSwap-1761484429-project-admin] [instance: 3e355ae8-aab9-4ae9-b6e3-6de108ec1af9] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '23338363665343038', 'path': '/dev/sdb'} {{(pid=75234) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Nov 29 03:13:43 user nova-compute[75234]: DEBUG nova.objects.instance [None req-17033895-5689-4ec0-83b3-2d4335860c97 tempest-TestVolumeSwap-1761484429 tempest-TestVolumeSwap-1761484429-project-admin] Lazy-loading 'flavor' on Instance uuid 3e355ae8-aab9-4ae9-b6e3-6de108ec1af9 {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:13:43 user nova-compute[75234]: DEBUG nova.virt.libvirt.guest [None req-17033895-5689-4ec0-83b3-2d4335860c97 tempest-TestVolumeSwap-1761484429 tempest-TestVolumeSwap-1761484429-project-admin] COPY block job progress, current cursor: 8388608 final cursor: 1073741824 {{(pid=75234) is_job_complete /opt/stack/nova/nova/virt/libvirt/guest.py:846}}
Nov 29 03:13:44 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:13:44 user nova-compute[75234]: DEBUG nova.virt.libvirt.guest [None req-17033895-5689-4ec0-83b3-2d4335860c97 tempest-TestVolumeSwap-1761484429 tempest-TestVolumeSwap-1761484429-project-admin] COPY block job progress, current cursor: 180355072 final cursor: 1073741824 {{(pid=75234) is_job_complete /opt/stack/nova/nova/virt/libvirt/guest.py:846}}
Nov 29 03:13:44 user nova-compute[75234]: DEBUG nova.virt.libvirt.guest [None req-17033895-5689-4ec0-83b3-2d4335860c97 tempest-TestVolumeSwap-1761484429 tempest-TestVolumeSwap-1761484429-project-admin] COPY block job progress, current cursor: 346030080 final cursor: 1073741824 {{(pid=75234) is_job_complete /opt/stack/nova/nova/virt/libvirt/guest.py:846}}
Nov 29 03:13:45 user nova-compute[75234]: DEBUG nova.virt.libvirt.guest [None req-17033895-5689-4ec0-83b3-2d4335860c97 tempest-TestVolumeSwap-1761484429 tempest-TestVolumeSwap-1761484429-project-admin] COPY block job progress, current cursor: 521142272 final cursor: 1073741824 {{(pid=75234) is_job_complete /opt/stack/nova/nova/virt/libvirt/guest.py:846}}
Nov 29 03:13:45 user nova-compute[75234]: DEBUG nova.virt.libvirt.guest [None req-17033895-5689-4ec0-83b3-2d4335860c97 tempest-TestVolumeSwap-1761484429 tempest-TestVolumeSwap-1761484429-project-admin] COPY block job progress, current cursor: 692060160 final cursor: 1073741824 {{(pid=75234) is_job_complete /opt/stack/nova/nova/virt/libvirt/guest.py:846}}
Nov 29 03:13:46 user nova-compute[75234]: DEBUG nova.virt.libvirt.guest [None req-17033895-5689-4ec0-83b3-2d4335860c97 tempest-TestVolumeSwap-1761484429 tempest-TestVolumeSwap-1761484429-project-admin] COPY block job progress, current cursor: 874512384 final cursor: 1073741824 {{(pid=75234) is_job_complete /opt/stack/nova/nova/virt/libvirt/guest.py:846}}
Nov 29 03:13:46 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:13:46 user nova-compute[75234]: DEBUG nova.virt.libvirt.guest [None req-17033895-5689-4ec0-83b3-2d4335860c97 tempest-TestVolumeSwap-1761484429 tempest-TestVolumeSwap-1761484429-project-admin] COPY block job progress, current cursor: 1037041664 final cursor: 1073741824 {{(pid=75234) is_job_complete /opt/stack/nova/nova/virt/libvirt/guest.py:846}}
Nov 29 03:13:47 user nova-compute[75234]: DEBUG nova.virt.libvirt.guest [None req-17033895-5689-4ec0-83b3-2d4335860c97 tempest-TestVolumeSwap-1761484429 tempest-TestVolumeSwap-1761484429-project-admin] COPY block job progress, current cursor: 1073741824 final cursor: 1073741824 {{(pid=75234) is_job_complete /opt/stack/nova/nova/virt/libvirt/guest.py:846}}
Nov 29 03:13:47 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-17033895-5689-4ec0-83b3-2d4335860c97 tempest-TestVolumeSwap-1761484429 tempest-TestVolumeSwap-1761484429-project-admin] [instance: 3e355ae8-aab9-4ae9-b6e3-6de108ec1af9] calling os-brick to detach iSCSI Volume {{(pid=75234) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Nov 29 03:13:47 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-17033895-5689-4ec0-83b3-2d4335860c97 tempest-TestVolumeSwap-1761484429 tempest-TestVolumeSwap-1761484429-project-admin] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f8a5050c9a0>, {'target_iqn': 'iqn.2010-10.org.openstack:94dadf41-624e-4581-81cf-eb5eaaecafb2', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'CIoXTDLA', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sde', 'volume_id': '94dadf41-624e-4581-81cf-eb5eaaecafb2'}, None), 'kwargs': False}}" {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:176}}
Nov 29 03:13:47 user nova-compute[75234]: WARNING os_brick.initiator.connectors.base [None req-17033895-5689-4ec0-83b3-2d4335860c97 tempest-TestVolumeSwap-1761484429 tempest-TestVolumeSwap-1761484429-project-admin] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Nov 29 03:13:47 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-17033895-5689-4ec0-83b3-2d4335860c97 tempest-TestVolumeSwap-1761484429 tempest-TestVolumeSwap-1761484429-project-admin] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:68}}
Nov 29 03:13:47 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-17033895-5689-4ec0-83b3-2d4335860c97 tempest-TestVolumeSwap-1761484429 tempest-TestVolumeSwap-1761484429-project-admin] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:73}}
Nov 29 03:13:47 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:13:47 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.020s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:13:47 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[1ed4dd0e-c381-4119-9af3-bb0b0dcc943e]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:4ed484cc-9de2-42df-8d99-637f3b47d474\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:56237f4c-4138-4f50-aafb-9c0f9d4bb1cf\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:6a4d34d3-9974-43bc-afcb-9169c893d9b7\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:94dadf41-624e-4581-81cf-eb5eaaecafb2\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:13:47 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-17033895-5689-4ec0-83b3-2d4335860c97 tempest-TestVolumeSwap-1761484429 tempest-TestVolumeSwap-1761484429-project-admin] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS:
Nov 29 03:13:47 user nova-compute[75234]: No targets found.
Nov 29 03:13:47 user nova-compute[75234]: iSNS:
Nov 29 03:13:47 user nova-compute[75234]: No targets found.
Nov 29 03:13:47 user nova-compute[75234]: STATIC:
Nov 29 03:13:47 user nova-compute[75234]: Target: iqn.2010-10.org.openstack:4ed484cc-9de2-42df-8d99-637f3b47d474
Nov 29 03:13:47 user nova-compute[75234]:         Portal: 172.16.0.220:3260,-1
Nov 29 03:13:47 user nova-compute[75234]:                 Iface Name: default
Nov 29 03:13:47 user nova-compute[75234]: Target: iqn.2010-10.org.openstack:56237f4c-4138-4f50-aafb-9c0f9d4bb1cf
Nov 29 03:13:47 user nova-compute[75234]:         Portal: 172.16.0.220:3260,-1
Nov 29 03:13:47 user nova-compute[75234]:                 Iface Name: default
Nov 29 03:13:47 user nova-compute[75234]: Target: iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b
Nov 29 03:13:47 user nova-compute[75234]:         Portal: 172.16.0.220:3260,-1
Nov 29 03:13:47 user nova-compute[75234]:                 Iface Name: default
Nov 29 03:13:47 user nova-compute[75234]: Target: iqn.2010-10.org.openstack:6a4d34d3-9974-43bc-afcb-9169c893d9b7
Nov 29 03:13:47 user nova-compute[75234]:         Portal: 172.16.0.220:3260,-1
Nov 29 03:13:47 user nova-compute[75234]:                 Iface Name: default
Nov 29 03:13:47 user nova-compute[75234]: Target: iqn.2010-10.org.openstack:94dadf41-624e-4581-81cf-eb5eaaecafb2
Nov 29 03:13:47 user nova-compute[75234]:         Portal: 172.16.0.220:3260,-1
Nov 29 03:13:47 user nova-compute[75234]:                 Iface Name: default
Nov 29 03:13:47 user nova-compute[75234]: FIRMWARE:
Nov 29 03:13:47 user nova-compute[75234]: No targets found.
Nov 29 03:13:47 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:13:47 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-17033895-5689-4ec0-83b3-2d4335860c97 tempest-TestVolumeSwap-1761484429 tempest-TestVolumeSwap-1761484429-project-admin] Regex to get portals from discoverydb: ^SENDTARGETS:
Nov 29 03:13:47 user nova-compute[75234]: .*?^DiscoveryAddress: 172.16.0.220,3260.*?
Nov 29 03:13:47 user nova-compute[75234]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=75234) _get_discoverydb_portals /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:382}}
Nov 29 03:13:47 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-17033895-5689-4ec0-83b3-2d4335860c97 tempest-TestVolumeSwap-1761484429 tempest-TestVolumeSwap-1761484429-project-admin] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:94dadf41-624e-4581-81cf-eb5eaaecafb2', 0)] {{(pid=75234) _get_connection_devices /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:830}}
Nov 29 03:13:47 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:13:47 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.019s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:13:47 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[c97f111c-3e22-4e2c-afbb-1555bee4ee92]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:4ed484cc-9de2-42df-8d99-637f3b47d474\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:56237f4c-4138-4f50-aafb-9c0f9d4bb1cf\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:6a4d34d3-9974-43bc-afcb-9169c893d9b7\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:94dadf41-624e-4581-81cf-eb5eaaecafb2\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:13:47 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:13:47 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.026s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:13:47 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[e4f6836e-3f5f-43dd-ac25-09085c3c1ae0]: (4, ('tcp: [20] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b (non-flash)\ntcp: [21] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:56237f4c-4138-4f50-aafb-9c0f9d4bb1cf (non-flash)\ntcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6a4d34d3-9974-43bc-afcb-9169c893d9b7 (non-flash)\ntcp: [24] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:94dadf41-624e-4581-81cf-eb5eaaecafb2 (non-flash)\ntcp: [25] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4ed484cc-9de2-42df-8d99-637f3b47d474 (non-flash)\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:13:47 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-17033895-5689-4ec0-83b3-2d4335860c97 tempest-TestVolumeSwap-1761484429 tempest-TestVolumeSwap-1761484429-project-admin] iscsiadm ('-m', 'session'): stdout=tcp: [20] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b (non-flash)
Nov 29 03:13:47 user nova-compute[75234]: tcp: [21] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:56237f4c-4138-4f50-aafb-9c0f9d4bb1cf (non-flash)
Nov 29 03:13:47 user nova-compute[75234]: tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6a4d34d3-9974-43bc-afcb-9169c893d9b7 (non-flash)
Nov 29 03:13:47 user nova-compute[75234]: tcp: [24] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:94dadf41-624e-4581-81cf-eb5eaaecafb2 (non-flash)
Nov 29 03:13:47 user nova-compute[75234]: tcp: [25] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4ed484cc-9de2-42df-8d99-637f3b47d474 (non-flash)
Nov 29 03:13:47 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:13:47 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-17033895-5689-4ec0-83b3-2d4335860c97 tempest-TestVolumeSwap-1761484429 tempest-TestVolumeSwap-1761484429-project-admin] iscsi session list stdout=tcp: [20] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b (non-flash)
Nov 29 03:13:47 user nova-compute[75234]: tcp: [21] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:56237f4c-4138-4f50-aafb-9c0f9d4bb1cf (non-flash)
Nov 29 03:13:47 user nova-compute[75234]: tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6a4d34d3-9974-43bc-afcb-9169c893d9b7 (non-flash)
Nov 29 03:13:47 user nova-compute[75234]: tcp: [24] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:94dadf41-624e-4581-81cf-eb5eaaecafb2 (non-flash)
Nov 29 03:13:47 user nova-compute[75234]: tcp: [25] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4ed484cc-9de2-42df-8d99-637f3b47d474 (non-flash)
Nov 29 03:13:47 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsi_session /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Nov 29 03:13:47 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-17033895-5689-4ec0-83b3-2d4335860c97 tempest-TestVolumeSwap-1761484429 tempest-TestVolumeSwap-1761484429-project-admin] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7f8a50374b80>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:94dadf41-624e-4581-81cf-eb5eaaecafb2'): ({'sde'}, set())}) {{(pid=75234) _get_connection_devices /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:863}}
Nov 29 03:13:47 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-17033895-5689-4ec0-83b3-2d4335860c97 tempest-TestVolumeSwap-1761484429 tempest-TestVolumeSwap-1761484429-project-admin] Removing single pathed devices sde {{(pid=75234) remove_connection /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:374}}
Nov 29 03:13:47 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:13:47 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.019s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:13:47 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[fe732777-82c7-4d89-b495-dc3ce3319d5c]: (4, ('path checker states:\nup                  5\n\npaths: 0\nbusy: False\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:13:47 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sde {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:13:47 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sde" returned: 0 in 0.021s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:13:47 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[f94efe51-e5d2-4d94-9013-9ab00919be37]: (4, ('ok\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:13:47 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-17033895-5689-4ec0-83b3-2d4335860c97 tempest-TestVolumeSwap-1761484429 tempest-TestVolumeSwap-1761484429-project-admin] Flushing IO for device /dev/sde {{(pid=75234) flush_device_io /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:434}}
Nov 29 03:13:47 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sde {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:13:47 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sde" returned: 0 in 0.014s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:13:47 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[ac5e1357-b0bd-4b1b-8d44-d30833076a3f]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:13:47 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-17033895-5689-4ec0-83b3-2d4335860c97 tempest-TestVolumeSwap-1761484429 tempest-TestVolumeSwap-1761484429-project-admin] Remove SCSI device /dev/sde with /sys/block/sde/device/delete {{(pid=75234) remove_scsi_device /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:148}}
Nov 29 03:13:47 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sde/device/delete {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:13:47 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sde/device/delete" returned: 0 in 0.031s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:13:47 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[762b2806-70eb-4c03-b154-50b6bf2a633f]: (4, ('1', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:13:47 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-17033895-5689-4ec0-83b3-2d4335860c97 tempest-TestVolumeSwap-1761484429 tempest-TestVolumeSwap-1761484429-project-admin] Checking to see if SCSI volumes sde have been removed. {{(pid=75234) wait_for_volumes_removal /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:156}}
Nov 29 03:13:47 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-17033895-5689-4ec0-83b3-2d4335860c97 tempest-TestVolumeSwap-1761484429 tempest-TestVolumeSwap-1761484429-project-admin] SCSI volumes sde have been removed. {{(pid=75234) wait_for_volumes_removal /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:166}}
Nov 29 03:13:47 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-17033895-5689-4ec0-83b3-2d4335860c97 tempest-TestVolumeSwap-1761484429 tempest-TestVolumeSwap-1761484429-project-admin] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:94dadf41-624e-4581-81cf-eb5eaaecafb2')] {{(pid=75234) _disconnect_connection /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Nov 29 03:13:47 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:94dadf41-624e-4581-81cf-eb5eaaecafb2 -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:13:47 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:94dadf41-624e-4581-81cf-eb5eaaecafb2 -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.018s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:13:47 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[db48eca4-6b1b-479a-bf5d-a096fb484cd8]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:13:47 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-17033895-5689-4ec0-83b3-2d4335860c97 tempest-TestVolumeSwap-1761484429 tempest-TestVolumeSwap-1761484429-project-admin] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:13:47 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:94dadf41-624e-4581-81cf-eb5eaaecafb2 -p 172.16.0.220:3260 --logout {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:13:47 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:94dadf41-624e-4581-81cf-eb5eaaecafb2 -p 172.16.0.220:3260 --logout" returned: 0 in 0.082s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:13:47 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[9cd8064b-97c5-43d3-978a-ff0f183c41db]: (4, ('Logging out of session [sid: 24, target: iqn.2010-10.org.openstack:94dadf41-624e-4581-81cf-eb5eaaecafb2, portal: 172.16.0.220,3260]\nLogout of [sid: 24, target: iqn.2010-10.org.openstack:94dadf41-624e-4581-81cf-eb5eaaecafb2, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:13:47 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-17033895-5689-4ec0-83b3-2d4335860c97 tempest-TestVolumeSwap-1761484429 tempest-TestVolumeSwap-1761484429-project-admin] iscsiadm ('--logout',): stdout=Logging out of session [sid: 24, target: iqn.2010-10.org.openstack:94dadf41-624e-4581-81cf-eb5eaaecafb2, portal: 172.16.0.220,3260]
Nov 29 03:13:47 user nova-compute[75234]: Logout of [sid: 24, target: iqn.2010-10.org.openstack:94dadf41-624e-4581-81cf-eb5eaaecafb2, portal: 172.16.0.220,3260] successful.
Nov 29 03:13:47 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:13:47 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:94dadf41-624e-4581-81cf-eb5eaaecafb2 -p 172.16.0.220:3260 --op delete {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:13:47 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:94dadf41-624e-4581-81cf-eb5eaaecafb2 -p 172.16.0.220:3260 --op delete" returned: 0 in 0.029s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:13:47 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[076d4ed7-2226-40be-a03c-13acdf33d1fc]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:13:47 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-17033895-5689-4ec0-83b3-2d4335860c97 tempest-TestVolumeSwap-1761484429 tempest-TestVolumeSwap-1761484429-project-admin] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:13:47 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-17033895-5689-4ec0-83b3-2d4335860c97 tempest-TestVolumeSwap-1761484429 tempest-TestVolumeSwap-1761484429-project-admin] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.326s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:87}}
Nov 29 03:13:47 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-17033895-5689-4ec0-83b3-2d4335860c97 tempest-TestVolumeSwap-1761484429 tempest-TestVolumeSwap-1761484429-project-admin] <== disconnect_volume: return (327ms) None {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:203}}
Nov 29 03:13:47 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-17033895-5689-4ec0-83b3-2d4335860c97 tempest-TestVolumeSwap-1761484429 tempest-TestVolumeSwap-1761484429-project-admin] [instance: 3e355ae8-aab9-4ae9-b6e3-6de108ec1af9] Disconnected iSCSI Volume {{(pid=75234) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:79}}
Nov 29 03:13:47 user nova-compute[75234]: DEBUG nova.compute.manager [None req-17033895-5689-4ec0-83b3-2d4335860c97 tempest-TestVolumeSwap-1761484429 tempest-TestVolumeSwap-1761484429-project-admin] swap_volume: Driver volume swap returned, new connection_info is now : {'driver_volume_type': 'iscsi', 'data': {'target_iqn': 'iqn.2010-10.org.openstack:4ed484cc-9de2-42df-8d99-637f3b47d474', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'mBlevMXQ', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdb'}, 'status': 'reserved', 'instance': '3e355ae8-aab9-4ae9-b6e3-6de108ec1af9', 'attached_at': '', 'detached_at': '', 'volume_id': '4ed484cc-9de2-42df-8d99-637f3b47d474', 'serial'} {{(pid=75234) _swap_volume /opt/stack/nova/nova/compute/manager.py:7791}}
Nov 29 03:13:47 user nova-compute[75234]: DEBUG nova.compute.manager [None req-17033895-5689-4ec0-83b3-2d4335860c97 tempest-TestVolumeSwap-1761484429 tempest-TestVolumeSwap-1761484429-project-admin] [instance: 3e355ae8-aab9-4ae9-b6e3-6de108ec1af9] swap_volume: removing Cinder connection for volume 94dadf41-624e-4581-81cf-eb5eaaecafb2 {{(pid=75234) _swap_volume /opt/stack/nova/nova/compute/manager.py:7836}}
Nov 29 03:13:48 user nova-compute[75234]: DEBUG nova.compute.manager [None req-17033895-5689-4ec0-83b3-2d4335860c97 tempest-TestVolumeSwap-1761484429 tempest-TestVolumeSwap-1761484429-project-admin] [instance: 3e355ae8-aab9-4ae9-b6e3-6de108ec1af9] swap_volume: Updating volume 94dadf41-624e-4581-81cf-eb5eaaecafb2 BDM record with {'connection_info': '{"driver_volume_type": "iscsi", "data": {"target_iqn": "iqn.2010-10.org.openstack:4ed484cc-9de2-42df-8d99-637f3b47d474", "target_portal": "172.16.0.220:3260", "target_discovered": false, "auth_method": "CHAP", "auth_username": "mBlevMXQ", "auth_password": "sS7Km2uOJX9HxC", "target_lun": 0, "qos_specs": null, "access_mode": "rw", "encrypted": false, "cacheable": false, "device_path": "/dev/sdb", "volume_id": "4ed484cc-9de2-42df-8d99-637f3b47d474"}, "status": "reserved", "instance": "3e355ae8-aab9-4ae9-b6e3-6de108ec1af9", "attached_at": "", "detached_at": "", "volume_id": "4ed484cc-9de2-42df-8d99-637f3b47d474", "serial": "4ed484cc-9de2-42df-8d99-637f3b47d474"}', 'source_type': 'volume', 'destination_type': 'volume', 'snapshot_id': None, 'volume_id': '4ed484cc-9de2-42df-8d99-637f3b47d474', 'no_device': None, 'attachment_id': 'd3252fa4-b840-41ea-93ad-969f11ef6254'} {{(pid=75234) _do_swap_volume /opt/stack/nova/nova/compute/manager.py:7984}}
Nov 29 03:13:48 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-17033895-5689-4ec0-83b3-2d4335860c97 tempest-TestVolumeSwap-1761484429 tempest-TestVolumeSwap-1761484429-project-admin] Lock "3e355ae8-aab9-4ae9-b6e3-6de108ec1af9" "released" by "nova.compute.manager.ComputeManager.swap_volume.<locals>._do_locked_swap_volume" :: held 14.945s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:13:49 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:13:49 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-bcdfd1f1-cd54-405e-9a1d-a059126b1602 tempest-TestVolumeSwap-1089931716 tempest-TestVolumeSwap-1089931716-project-member] Acquiring lock "3e355ae8-aab9-4ae9-b6e3-6de108ec1af9" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:13:49 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-bcdfd1f1-cd54-405e-9a1d-a059126b1602 tempest-TestVolumeSwap-1089931716 tempest-TestVolumeSwap-1089931716-project-member] Lock "3e355ae8-aab9-4ae9-b6e3-6de108ec1af9" acquired by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:13:49 user nova-compute[75234]: INFO nova.compute.manager [None req-bcdfd1f1-cd54-405e-9a1d-a059126b1602 tempest-TestVolumeSwap-1089931716 tempest-TestVolumeSwap-1089931716-project-member] [instance: 3e355ae8-aab9-4ae9-b6e3-6de108ec1af9] Detaching volume 4ed484cc-9de2-42df-8d99-637f3b47d474
Nov 29 03:13:49 user nova-compute[75234]: INFO nova.virt.block_device [None req-bcdfd1f1-cd54-405e-9a1d-a059126b1602 tempest-TestVolumeSwap-1089931716 tempest-TestVolumeSwap-1089931716-project-member] [instance: 3e355ae8-aab9-4ae9-b6e3-6de108ec1af9] Attempting to driver detach volume 4ed484cc-9de2-42df-8d99-637f3b47d474 from mountpoint /dev/vdb
Nov 29 03:13:49 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-bcdfd1f1-cd54-405e-9a1d-a059126b1602 tempest-TestVolumeSwap-1089931716 tempest-TestVolumeSwap-1089931716-project-member] Attempting to detach device vdb from instance 3e355ae8-aab9-4ae9-b6e3-6de108ec1af9 from the persistent domain config. {{(pid=75234) _detach_from_persistent /opt/stack/nova/nova/virt/libvirt/driver.py:2487}}
Nov 29 03:13:49 user nova-compute[75234]: DEBUG nova.virt.libvirt.guest [None req-bcdfd1f1-cd54-405e-9a1d-a059126b1602 tempest-TestVolumeSwap-1089931716 tempest-TestVolumeSwap-1089931716-project-member] detach device xml: <disk type="block" device="disk">
Nov 29 03:13:49 user nova-compute[75234]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Nov 29 03:13:49 user nova-compute[75234]:   <source dev="/dev/sdb"/>
Nov 29 03:13:49 user nova-compute[75234]:   <target dev="vdb" bus="virtio"/>
Nov 29 03:13:49 user nova-compute[75234]:   <serial>4ed484cc-9de2-42df-8d99-637f3b47d474</serial>
Nov 29 03:13:49 user nova-compute[75234]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x07" function="0x0"/>
Nov 29 03:13:49 user nova-compute[75234]: </disk>
Nov 29 03:13:49 user nova-compute[75234]:  {{(pid=75234) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Nov 29 03:13:49 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-bcdfd1f1-cd54-405e-9a1d-a059126b1602 tempest-TestVolumeSwap-1089931716 tempest-TestVolumeSwap-1089931716-project-member] Successfully detached device vdb from instance 3e355ae8-aab9-4ae9-b6e3-6de108ec1af9 from the persistent domain config.
Nov 29 03:13:49 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-bcdfd1f1-cd54-405e-9a1d-a059126b1602 tempest-TestVolumeSwap-1089931716 tempest-TestVolumeSwap-1089931716-project-member] (1/8): Attempting to detach device vdb with device alias virtio-disk1 from instance 3e355ae8-aab9-4ae9-b6e3-6de108ec1af9 from the live domain config. {{(pid=75234) _detach_from_live_with_retry /opt/stack/nova/nova/virt/libvirt/driver.py:2523}}
Nov 29 03:13:49 user nova-compute[75234]: DEBUG nova.virt.libvirt.guest [None req-bcdfd1f1-cd54-405e-9a1d-a059126b1602 tempest-TestVolumeSwap-1089931716 tempest-TestVolumeSwap-1089931716-project-member] detach device xml: <disk type="block" device="disk">
Nov 29 03:13:49 user nova-compute[75234]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Nov 29 03:13:49 user nova-compute[75234]:   <source dev="/dev/sdb"/>
Nov 29 03:13:49 user nova-compute[75234]:   <target dev="vdb" bus="virtio"/>
Nov 29 03:13:49 user nova-compute[75234]:   <serial>4ed484cc-9de2-42df-8d99-637f3b47d474</serial>
Nov 29 03:13:49 user nova-compute[75234]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x07" function="0x0"/>
Nov 29 03:13:49 user nova-compute[75234]: </disk>
Nov 29 03:13:49 user nova-compute[75234]:  {{(pid=75234) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Nov 29 03:13:49 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-bcdfd1f1-cd54-405e-9a1d-a059126b1602 tempest-TestVolumeSwap-1089931716 tempest-TestVolumeSwap-1089931716-project-member] Start waiting for the detach event from libvirt for device vdb with device alias virtio-disk1 for instance 3e355ae8-aab9-4ae9-b6e3-6de108ec1af9 {{(pid=75234) _detach_from_live_and_wait_for_event /opt/stack/nova/nova/virt/libvirt/driver.py:2599}}
Nov 29 03:13:51 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:13:53 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:13:54 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:13:54 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:13:54 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Starting heal instance info cache {{(pid=75234) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9922}}
Nov 29 03:13:54 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Rebuilding the list of instances to heal {{(pid=75234) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9926}}
Nov 29 03:13:54 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Acquiring lock "refresh_cache-2bf9fe41-63c2-4882-bcdd-8e314e734ffe" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:13:54 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Acquired lock "refresh_cache-2bf9fe41-63c2-4882-bcdd-8e314e734ffe" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:13:54 user nova-compute[75234]: DEBUG nova.network.neutron [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] [instance: 2bf9fe41-63c2-4882-bcdd-8e314e734ffe] Forcefully refreshing network info cache for instance {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2003}}
Nov 29 03:13:54 user nova-compute[75234]: DEBUG nova.objects.instance [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lazy-loading 'info_cache' on Instance uuid 2bf9fe41-63c2-4882-bcdd-8e314e734ffe {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:13:55 user nova-compute[75234]: DEBUG nova.network.neutron [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] [instance: 2bf9fe41-63c2-4882-bcdd-8e314e734ffe] Updating instance_info_cache with network_info: [{"id": "c8bb20fd-5c85-4e89-8c61-13a4f17dd4cc", "address": "fa:16:3e:2c:c9:1d", "network": {"id": "b27ac31e-3601-4055-be79-34b0946b0913", "bridge": "br-int", "label": "tempest-ServersNegativeTestJSON-1491620118-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "b877668e02eb441c85dc2a7003a94c7a", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc8bb20fd-5c", "ovs_interfaceid": "c8bb20fd-5c85-4e89-8c61-13a4f17dd4cc", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:13:55 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Releasing lock "refresh_cache-2bf9fe41-63c2-4882-bcdd-8e314e734ffe" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:13:55 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] [instance: 2bf9fe41-63c2-4882-bcdd-8e314e734ffe] Updated the network info_cache for instance {{(pid=75234) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9993}}
Nov 29 03:13:55 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:13:55 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=75234) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10541}}
Nov 29 03:13:55 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:13:55 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:13:56 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:13:56 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager.update_available_resource {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:13:56 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:13:56 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:13:56 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:13:56 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Auditing locally available compute resources for user (node: user) {{(pid=75234) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:907}}
Nov 29 03:13:56 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a9350a18-14e6-42c6-a150-0a1487f20ad9/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:13:56 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:13:56 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a9350a18-14e6-42c6-a150-0a1487f20ad9/disk --force-share --output=json" returned: 0 in 0.202s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:13:56 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a9350a18-14e6-42c6-a150-0a1487f20ad9/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:13:57 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a9350a18-14e6-42c6-a150-0a1487f20ad9/disk --force-share --output=json" returned: 0 in 0.172s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:13:57 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] skipping disk /dev/sdc (vdb) as it is a volume {{(pid=75234) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11213}}
Nov 29 03:13:57 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/3e355ae8-aab9-4ae9-b6e3-6de108ec1af9/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:13:57 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/3e355ae8-aab9-4ae9-b6e3-6de108ec1af9/disk --force-share --output=json" returned: 0 in 0.165s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:13:57 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/3e355ae8-aab9-4ae9-b6e3-6de108ec1af9/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:13:57 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/3e355ae8-aab9-4ae9-b6e3-6de108ec1af9/disk --force-share --output=json" returned: 0 in 0.165s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:13:57 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a082c96e-e384-40b6-bcc8-fc5e0e104669/disk.rescue --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:13:57 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a082c96e-e384-40b6-bcc8-fc5e0e104669/disk.rescue --force-share --output=json" returned: 0 in 0.163s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:13:57 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a082c96e-e384-40b6-bcc8-fc5e0e104669/disk.rescue --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:13:57 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a082c96e-e384-40b6-bcc8-fc5e0e104669/disk.rescue --force-share --output=json" returned: 0 in 0.170s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:13:57 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a082c96e-e384-40b6-bcc8-fc5e0e104669/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:13:57 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a082c96e-e384-40b6-bcc8-fc5e0e104669/disk --force-share --output=json" returned: 0 in 0.168s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:13:57 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a082c96e-e384-40b6-bcc8-fc5e0e104669/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:13:58 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a082c96e-e384-40b6-bcc8-fc5e0e104669/disk --force-share --output=json" returned: 0 in 0.177s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:13:58 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2bf9fe41-63c2-4882-bcdd-8e314e734ffe/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:13:58 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2bf9fe41-63c2-4882-bcdd-8e314e734ffe/disk --force-share --output=json" returned: 0 in 0.174s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:13:58 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2bf9fe41-63c2-4882-bcdd-8e314e734ffe/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:13:58 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/2bf9fe41-63c2-4882-bcdd-8e314e734ffe/disk --force-share --output=json" returned: 0 in 0.163s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:13:58 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f609a7b7-1be2-44b4-9177-7e72a4d4c060/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:13:58 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f609a7b7-1be2-44b4-9177-7e72a4d4c060/disk --force-share --output=json" returned: 0 in 0.172s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:13:58 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f609a7b7-1be2-44b4-9177-7e72a4d4c060/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:13:58 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f609a7b7-1be2-44b4-9177-7e72a4d4c060/disk --force-share --output=json" returned: 0 in 0.175s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:13:58 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/21e9c6cd-c39f-41c0-8768-5dd64589c3f3/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:13:58 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/21e9c6cd-c39f-41c0-8768-5dd64589c3f3/disk --force-share --output=json" returned: 0 in 0.173s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:13:58 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/21e9c6cd-c39f-41c0-8768-5dd64589c3f3/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:13:59 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/21e9c6cd-c39f-41c0-8768-5dd64589c3f3/disk --force-share --output=json" returned: 0 in 0.167s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:13:59 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:13:59 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/13e5da64-9adc-4479-81c7-d329e9b5f08f/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:13:59 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/13e5da64-9adc-4479-81c7-d329e9b5f08f/disk --force-share --output=json" returned: 0 in 0.181s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:13:59 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/13e5da64-9adc-4479-81c7-d329e9b5f08f/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:13:59 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/13e5da64-9adc-4479-81c7-d329e9b5f08f/disk --force-share --output=json" returned: 0 in 0.172s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:13:59 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6c0e0184-ca3a-48ec-a6e3-a0a1172260d6/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:13:59 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6c0e0184-ca3a-48ec-a6e3-a0a1172260d6/disk --force-share --output=json" returned: 0 in 0.183s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:13:59 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6c0e0184-ca3a-48ec-a6e3-a0a1172260d6/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:13:59 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6c0e0184-ca3a-48ec-a6e3-a0a1172260d6/disk --force-share --output=json" returned: 0 in 0.167s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:13:59 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] skipping disk /dev/sdf (vdb) as it is a volume {{(pid=75234) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11213}}
Nov 29 03:13:59 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6894b55f-93c6-43c9-8933-56d5a3c43725/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:14:00 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6894b55f-93c6-43c9-8933-56d5a3c43725/disk --force-share --output=json" returned: 0 in 0.196s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:14:00 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6894b55f-93c6-43c9-8933-56d5a3c43725/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:14:00 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6894b55f-93c6-43c9-8933-56d5a3c43725/disk --force-share --output=json" returned: 0 in 0.165s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:14:00 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/062753a6-8f39-4e80-a549-9f0f5a4b877a/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:14:00 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/062753a6-8f39-4e80-a549-9f0f5a4b877a/disk --force-share --output=json" returned: 0 in 0.169s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:14:00 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/062753a6-8f39-4e80-a549-9f0f5a4b877a/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:14:00 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/062753a6-8f39-4e80-a549-9f0f5a4b877a/disk --force-share --output=json" returned: 0 in 0.176s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:14:00 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] skipping disk /dev/sdd (vdc) as it is a volume {{(pid=75234) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11213}}
Nov 29 03:14:00 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/d3b90f39-4b91-4152-b820-bd0e263eec89/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:14:00 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/d3b90f39-4b91-4152-b820-bd0e263eec89/disk --force-share --output=json" returned: 0 in 0.187s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:14:00 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/d3b90f39-4b91-4152-b820-bd0e263eec89/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:14:00 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/d3b90f39-4b91-4152-b820-bd0e263eec89/disk --force-share --output=json" returned: 0 in 0.177s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:14:01 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:14:01 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:14:01 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Hypervisor/Node resource view: name=user free_ram=7400MB free_disk=25.37488555908203GB free_vcpus=2 pci_devices=[{"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}] {{(pid=75234) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1106}}
Nov 29 03:14:01 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:14:01 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:14:01 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance 2bf9fe41-63c2-4882-bcdd-8e314e734ffe actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:14:01 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance 21e9c6cd-c39f-41c0-8768-5dd64589c3f3 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:14:01 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance f609a7b7-1be2-44b4-9177-7e72a4d4c060 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:14:01 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance 6c0e0184-ca3a-48ec-a6e3-a0a1172260d6 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:14:01 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance a9350a18-14e6-42c6-a150-0a1487f20ad9 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 512, 'DISK_GB': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:14:01 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance d3b90f39-4b91-4152-b820-bd0e263eec89 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:14:01 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance 13e5da64-9adc-4479-81c7-d329e9b5f08f actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:14:01 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance a082c96e-e384-40b6-bcc8-fc5e0e104669 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:14:01 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance 3e355ae8-aab9-4ae9-b6e3-6de108ec1af9 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:14:01 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance 6894b55f-93c6-43c9-8933-56d5a3c43725 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:14:01 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance 062753a6-8f39-4e80-a549-9f0f5a4b877a actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:14:01 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Total usable vcpus: 12, total allocated vcpus: 11 {{(pid=75234) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1129}}
Nov 29 03:14:01 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Final resource view: name=user phys_ram=16011MB used_ram=6144MB phys_disk=40GB used_disk=11GB total_vcpus=12 used_vcpus=11 pci_stats=[] {{(pid=75234) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1138}}
Nov 29 03:14:01 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:14:01 user nova-compute[75234]: DEBUG nova.compute.provider_tree [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Inventory has not changed in ProviderTree for provider: 4d291220-7c37-4839-a0e3-b23c66502b74 {{(pid=75234) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Nov 29 03:14:01 user nova-compute[75234]: DEBUG nova.scheduler.client.report [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Inventory has not changed for provider 4d291220-7c37-4839-a0e3-b23c66502b74 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=75234) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Nov 29 03:14:01 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Compute_service record updated for user:user {{(pid=75234) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:1067}}
Nov 29 03:14:01 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.535s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:14:02 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._sync_scheduler_instance_info {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:14:02 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:14:02 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:14:04 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:14:09 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:14:09 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-bcdfd1f1-cd54-405e-9a1d-a059126b1602 tempest-TestVolumeSwap-1089931716 tempest-TestVolumeSwap-1089931716-project-member] Waiting for libvirt event about the detach of device vdb with device alias virtio-disk1 from instance 3e355ae8-aab9-4ae9-b6e3-6de108ec1af9 is timed out.
Nov 29 03:14:09 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-bcdfd1f1-cd54-405e-9a1d-a059126b1602 tempest-TestVolumeSwap-1089931716 tempest-TestVolumeSwap-1089931716-project-member] Successfully detached device vdb from instance 3e355ae8-aab9-4ae9-b6e3-6de108ec1af9 from the live domain config.
Nov 29 03:14:09 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-bcdfd1f1-cd54-405e-9a1d-a059126b1602 tempest-TestVolumeSwap-1089931716 tempest-TestVolumeSwap-1089931716-project-member] [instance: 3e355ae8-aab9-4ae9-b6e3-6de108ec1af9] calling os-brick to detach iSCSI Volume {{(pid=75234) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Nov 29 03:14:09 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-bcdfd1f1-cd54-405e-9a1d-a059126b1602 tempest-TestVolumeSwap-1089931716 tempest-TestVolumeSwap-1089931716-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f8a5050c9a0>, {'target_iqn': 'iqn.2010-10.org.openstack:4ed484cc-9de2-42df-8d99-637f3b47d474', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'mBlevMXQ', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdb', 'volume_id': '4ed484cc-9de2-42df-8d99-637f3b47d474'}, None), 'kwargs': False}}" {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:176}}
Nov 29 03:14:09 user nova-compute[75234]: WARNING os_brick.initiator.connectors.base [None req-bcdfd1f1-cd54-405e-9a1d-a059126b1602 tempest-TestVolumeSwap-1089931716 tempest-TestVolumeSwap-1089931716-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Nov 29 03:14:09 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-bcdfd1f1-cd54-405e-9a1d-a059126b1602 tempest-TestVolumeSwap-1089931716 tempest-TestVolumeSwap-1089931716-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:68}}
Nov 29 03:14:09 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-bcdfd1f1-cd54-405e-9a1d-a059126b1602 tempest-TestVolumeSwap-1089931716 tempest-TestVolumeSwap-1089931716-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.002s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:73}}
Nov 29 03:14:09 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:14:09 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.020s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:14:09 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[19860352-207f-4502-b498-ff85654a2ae6]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:4ed484cc-9de2-42df-8d99-637f3b47d474\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:56237f4c-4138-4f50-aafb-9c0f9d4bb1cf\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:6a4d34d3-9974-43bc-afcb-9169c893d9b7\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:14:09 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-bcdfd1f1-cd54-405e-9a1d-a059126b1602 tempest-TestVolumeSwap-1089931716 tempest-TestVolumeSwap-1089931716-project-member] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS:
Nov 29 03:14:09 user nova-compute[75234]: No targets found.
Nov 29 03:14:09 user nova-compute[75234]: iSNS:
Nov 29 03:14:09 user nova-compute[75234]: No targets found.
Nov 29 03:14:09 user nova-compute[75234]: STATIC:
Nov 29 03:14:09 user nova-compute[75234]: Target: iqn.2010-10.org.openstack:4ed484cc-9de2-42df-8d99-637f3b47d474
Nov 29 03:14:09 user nova-compute[75234]:         Portal: 172.16.0.220:3260,-1
Nov 29 03:14:09 user nova-compute[75234]:                 Iface Name: default
Nov 29 03:14:09 user nova-compute[75234]: Target: iqn.2010-10.org.openstack:56237f4c-4138-4f50-aafb-9c0f9d4bb1cf
Nov 29 03:14:09 user nova-compute[75234]:         Portal: 172.16.0.220:3260,-1
Nov 29 03:14:09 user nova-compute[75234]:                 Iface Name: default
Nov 29 03:14:09 user nova-compute[75234]: Target: iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b
Nov 29 03:14:09 user nova-compute[75234]:         Portal: 172.16.0.220:3260,-1
Nov 29 03:14:09 user nova-compute[75234]:                 Iface Name: default
Nov 29 03:14:09 user nova-compute[75234]: Target: iqn.2010-10.org.openstack:6a4d34d3-9974-43bc-afcb-9169c893d9b7
Nov 29 03:14:09 user nova-compute[75234]:         Portal: 172.16.0.220:3260,-1
Nov 29 03:14:09 user nova-compute[75234]:                 Iface Name: default
Nov 29 03:14:09 user nova-compute[75234]: FIRMWARE:
Nov 29 03:14:09 user nova-compute[75234]: No targets found.
Nov 29 03:14:09 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:14:09 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-bcdfd1f1-cd54-405e-9a1d-a059126b1602 tempest-TestVolumeSwap-1089931716 tempest-TestVolumeSwap-1089931716-project-member] Regex to get portals from discoverydb: ^SENDTARGETS:
Nov 29 03:14:09 user nova-compute[75234]: .*?^DiscoveryAddress: 172.16.0.220,3260.*?
Nov 29 03:14:09 user nova-compute[75234]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=75234) _get_discoverydb_portals /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:382}}
Nov 29 03:14:09 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-bcdfd1f1-cd54-405e-9a1d-a059126b1602 tempest-TestVolumeSwap-1089931716 tempest-TestVolumeSwap-1089931716-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:4ed484cc-9de2-42df-8d99-637f3b47d474', 0)] {{(pid=75234) _get_connection_devices /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:830}}
Nov 29 03:14:09 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:14:09 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.018s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:14:09 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[47b7d5d1-2e61-4f1b-a1b0-2e99f0771747]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:4ed484cc-9de2-42df-8d99-637f3b47d474\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:56237f4c-4138-4f50-aafb-9c0f9d4bb1cf\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:6a4d34d3-9974-43bc-afcb-9169c893d9b7\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:14:09 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:14:10 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.022s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:14:10 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[35084a27-4803-4b91-92ae-1d1868818f45]: (4, ('tcp: [20] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b (non-flash)\ntcp: [21] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:56237f4c-4138-4f50-aafb-9c0f9d4bb1cf (non-flash)\ntcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6a4d34d3-9974-43bc-afcb-9169c893d9b7 (non-flash)\ntcp: [25] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4ed484cc-9de2-42df-8d99-637f3b47d474 (non-flash)\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:14:10 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-bcdfd1f1-cd54-405e-9a1d-a059126b1602 tempest-TestVolumeSwap-1089931716 tempest-TestVolumeSwap-1089931716-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [20] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b (non-flash)
Nov 29 03:14:10 user nova-compute[75234]: tcp: [21] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:56237f4c-4138-4f50-aafb-9c0f9d4bb1cf (non-flash)
Nov 29 03:14:10 user nova-compute[75234]: tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6a4d34d3-9974-43bc-afcb-9169c893d9b7 (non-flash)
Nov 29 03:14:10 user nova-compute[75234]: tcp: [25] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4ed484cc-9de2-42df-8d99-637f3b47d474 (non-flash)
Nov 29 03:14:10 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:14:10 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-bcdfd1f1-cd54-405e-9a1d-a059126b1602 tempest-TestVolumeSwap-1089931716 tempest-TestVolumeSwap-1089931716-project-member] iscsi session list stdout=tcp: [20] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b (non-flash)
Nov 29 03:14:10 user nova-compute[75234]: tcp: [21] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:56237f4c-4138-4f50-aafb-9c0f9d4bb1cf (non-flash)
Nov 29 03:14:10 user nova-compute[75234]: tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6a4d34d3-9974-43bc-afcb-9169c893d9b7 (non-flash)
Nov 29 03:14:10 user nova-compute[75234]: tcp: [25] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:4ed484cc-9de2-42df-8d99-637f3b47d474 (non-flash)
Nov 29 03:14:10 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsi_session /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Nov 29 03:14:10 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-bcdfd1f1-cd54-405e-9a1d-a059126b1602 tempest-TestVolumeSwap-1089931716 tempest-TestVolumeSwap-1089931716-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7f8a50112950>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:4ed484cc-9de2-42df-8d99-637f3b47d474'): ({'sdb'}, set())}) {{(pid=75234) _get_connection_devices /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:863}}
Nov 29 03:14:10 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-bcdfd1f1-cd54-405e-9a1d-a059126b1602 tempest-TestVolumeSwap-1089931716 tempest-TestVolumeSwap-1089931716-project-member] Removing single pathed devices sdb {{(pid=75234) remove_connection /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:374}}
Nov 29 03:14:10 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:14:10 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.018s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:14:10 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[d233e61a-9867-4180-b1c7-ab99dc8bb794]: (4, ('path checker states:\nup                  4\n\npaths: 0\nbusy: False\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:14:10 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdb {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:14:10 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdb" returned: 0 in 0.018s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:14:10 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[c335b28a-a89f-4167-80ac-d1e7235275f1]: (4, ('ok\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:14:10 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-bcdfd1f1-cd54-405e-9a1d-a059126b1602 tempest-TestVolumeSwap-1089931716 tempest-TestVolumeSwap-1089931716-project-member] Flushing IO for device /dev/sdb {{(pid=75234) flush_device_io /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:434}}
Nov 29 03:14:10 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdb {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:14:10 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdb" returned: 0 in 0.011s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:14:10 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[8712dbe2-21c3-4db1-b8df-877a429edd64]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:14:10 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-bcdfd1f1-cd54-405e-9a1d-a059126b1602 tempest-TestVolumeSwap-1089931716 tempest-TestVolumeSwap-1089931716-project-member] Remove SCSI device /dev/sdb with /sys/block/sdb/device/delete {{(pid=75234) remove_scsi_device /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:148}}
Nov 29 03:14:10 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdb/device/delete {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:14:10 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdb/device/delete" returned: 0 in 0.036s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:14:10 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[3b8f4b49-3d61-4c9a-b575-6cd1868e1060]: (4, ('1', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:14:10 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-bcdfd1f1-cd54-405e-9a1d-a059126b1602 tempest-TestVolumeSwap-1089931716 tempest-TestVolumeSwap-1089931716-project-member] Checking to see if SCSI volumes sdb have been removed. {{(pid=75234) wait_for_volumes_removal /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:156}}
Nov 29 03:14:10 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-bcdfd1f1-cd54-405e-9a1d-a059126b1602 tempest-TestVolumeSwap-1089931716 tempest-TestVolumeSwap-1089931716-project-member] SCSI volumes sdb have been removed. {{(pid=75234) wait_for_volumes_removal /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:166}}
Nov 29 03:14:10 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-bcdfd1f1-cd54-405e-9a1d-a059126b1602 tempest-TestVolumeSwap-1089931716 tempest-TestVolumeSwap-1089931716-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:4ed484cc-9de2-42df-8d99-637f3b47d474')] {{(pid=75234) _disconnect_connection /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Nov 29 03:14:10 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:4ed484cc-9de2-42df-8d99-637f3b47d474 -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:14:10 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:4ed484cc-9de2-42df-8d99-637f3b47d474 -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.019s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:14:10 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[f8efbb8c-b825-487e-a9d4-16ec5c8dd734]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:14:10 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-bcdfd1f1-cd54-405e-9a1d-a059126b1602 tempest-TestVolumeSwap-1089931716 tempest-TestVolumeSwap-1089931716-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:14:10 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:4ed484cc-9de2-42df-8d99-637f3b47d474 -p 172.16.0.220:3260 --logout {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:14:10 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:4ed484cc-9de2-42df-8d99-637f3b47d474 -p 172.16.0.220:3260 --logout" returned: 0 in 0.082s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:14:10 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[363d808a-f584-4992-85a1-0d1d06a0716d]: (4, ('Logging out of session [sid: 25, target: iqn.2010-10.org.openstack:4ed484cc-9de2-42df-8d99-637f3b47d474, portal: 172.16.0.220,3260]\nLogout of [sid: 25, target: iqn.2010-10.org.openstack:4ed484cc-9de2-42df-8d99-637f3b47d474, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:14:10 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-bcdfd1f1-cd54-405e-9a1d-a059126b1602 tempest-TestVolumeSwap-1089931716 tempest-TestVolumeSwap-1089931716-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 25, target: iqn.2010-10.org.openstack:4ed484cc-9de2-42df-8d99-637f3b47d474, portal: 172.16.0.220,3260]
Nov 29 03:14:10 user nova-compute[75234]: Logout of [sid: 25, target: iqn.2010-10.org.openstack:4ed484cc-9de2-42df-8d99-637f3b47d474, portal: 172.16.0.220,3260] successful.
Nov 29 03:14:10 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:14:10 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:4ed484cc-9de2-42df-8d99-637f3b47d474 -p 172.16.0.220:3260 --op delete {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:14:10 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:4ed484cc-9de2-42df-8d99-637f3b47d474 -p 172.16.0.220:3260 --op delete" returned: 0 in 0.023s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:14:10 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[741d1e02-c0b1-4b97-95dd-3ea732576612]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:14:10 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-bcdfd1f1-cd54-405e-9a1d-a059126b1602 tempest-TestVolumeSwap-1089931716 tempest-TestVolumeSwap-1089931716-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:14:10 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-bcdfd1f1-cd54-405e-9a1d-a059126b1602 tempest-TestVolumeSwap-1089931716 tempest-TestVolumeSwap-1089931716-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.310s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:87}}
Nov 29 03:14:10 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-bcdfd1f1-cd54-405e-9a1d-a059126b1602 tempest-TestVolumeSwap-1089931716 tempest-TestVolumeSwap-1089931716-project-member] <== disconnect_volume: return (311ms) None {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:203}}
Nov 29 03:14:10 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-bcdfd1f1-cd54-405e-9a1d-a059126b1602 tempest-TestVolumeSwap-1089931716 tempest-TestVolumeSwap-1089931716-project-member] [instance: 3e355ae8-aab9-4ae9-b6e3-6de108ec1af9] Disconnected iSCSI Volume {{(pid=75234) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:79}}
Nov 29 03:14:11 user nova-compute[75234]: DEBUG nova.objects.instance [None req-bcdfd1f1-cd54-405e-9a1d-a059126b1602 tempest-TestVolumeSwap-1089931716 tempest-TestVolumeSwap-1089931716-project-member] Lazy-loading 'flavor' on Instance uuid 3e355ae8-aab9-4ae9-b6e3-6de108ec1af9 {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:14:11 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-bcdfd1f1-cd54-405e-9a1d-a059126b1602 tempest-TestVolumeSwap-1089931716 tempest-TestVolumeSwap-1089931716-project-member] Lock "3e355ae8-aab9-4ae9-b6e3-6de108ec1af9" "released" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: held 21.877s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:14:12 user nova-compute[75234]: DEBUG nova.objects.instance [None req-33ffaa40-d676-428f-859e-d86cab3bfa75 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Lazy-loading 'flavor' on Instance uuid f609a7b7-1be2-44b4-9177-7e72a4d4c060 {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:14:12 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-33ffaa40-d676-428f-859e-d86cab3bfa75 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Acquiring lock "refresh_cache-f609a7b7-1be2-44b4-9177-7e72a4d4c060" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:14:12 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-33ffaa40-d676-428f-859e-d86cab3bfa75 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Acquired lock "refresh_cache-f609a7b7-1be2-44b4-9177-7e72a4d4c060" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:14:12 user nova-compute[75234]: DEBUG nova.network.neutron [None req-33ffaa40-d676-428f-859e-d86cab3bfa75 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] [instance: f609a7b7-1be2-44b4-9177-7e72a4d4c060] Building network info cache for instance {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2009}}
Nov 29 03:14:12 user nova-compute[75234]: DEBUG nova.objects.instance [None req-33ffaa40-d676-428f-859e-d86cab3bfa75 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Lazy-loading 'info_cache' on Instance uuid f609a7b7-1be2-44b4-9177-7e72a4d4c060 {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:14:12 user nova-compute[75234]: DEBUG nova.objects.base [None req-33ffaa40-d676-428f-859e-d86cab3bfa75 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Object Instance<f609a7b7-1be2-44b4-9177-7e72a4d4c060> lazy-loaded attributes: flavor,info_cache {{(pid=75234) wrapper /opt/stack/nova/nova/objects/base.py:126}}
Nov 29 03:14:13 user nova-compute[75234]: DEBUG nova.network.neutron [None req-33ffaa40-d676-428f-859e-d86cab3bfa75 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] [instance: f609a7b7-1be2-44b4-9177-7e72a4d4c060] Updating instance_info_cache with network_info: [{"id": "5543f379-2b3a-4dc2-bfe9-52f66a942257", "address": "fa:16:3e:04:ae:43", "network": {"id": "294c3aac-8c9b-45f0-8c0a-344e2d7d8b7c", "bridge": "br-int", "label": "tempest-AttachVolumeTestJSON-700098935-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.14", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "e393d23e783b4a40986674d782606c69", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap5543f379-2b", "ovs_interfaceid": "5543f379-2b3a-4dc2-bfe9-52f66a942257", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:14:13 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-33ffaa40-d676-428f-859e-d86cab3bfa75 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Releasing lock "refresh_cache-f609a7b7-1be2-44b4-9177-7e72a4d4c060" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:14:13 user nova-compute[75234]: INFO nova.virt.libvirt.driver [-] [instance: f609a7b7-1be2-44b4-9177-7e72a4d4c060] Instance destroyed successfully.
Nov 29 03:14:13 user nova-compute[75234]: DEBUG nova.objects.instance [None req-33ffaa40-d676-428f-859e-d86cab3bfa75 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Lazy-loading 'resources' on Instance uuid f609a7b7-1be2-44b4-9177-7e72a4d4c060 {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:14:13 user nova-compute[75234]: DEBUG nova.objects.base [None req-33ffaa40-d676-428f-859e-d86cab3bfa75 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Object Instance<f609a7b7-1be2-44b4-9177-7e72a4d4c060> lazy-loaded attributes: flavor,info_cache,resources {{(pid=75234) wrapper /opt/stack/nova/nova/objects/base.py:126}}
Nov 29 03:14:13 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-33ffaa40-d676-428f-859e-d86cab3bfa75 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-11-29T03:06:06Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-AttachVolumeTestJSON-server-394712256',display_name='tempest-AttachVolumeTestJSON-server-394712256',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-attachvolumetestjson-server-394712256',id=4,image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBGphX+MWMhY0YZ/wlOvEQFKFHTIX4g9KMhxkTe+ErHeVHCk7d+KpOlTDVK9edUWFFxjgJKboch3pp1xrAYGa/rIEWyUV4YOfCGPFoAeneAuweb6ekmwg4qgLxfML0aXASQ==',key_name='tempest-keypair-213174705',keypairs=<?>,launch_index=0,launched_at=2023-11-29T03:06:23Z,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=<?>,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=4,progress=0,project_id='e393d23e783b4a40986674d782606c69',ramdisk_id='',reservation_id='r-9o6ggnbe',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=<?>,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-AttachVolumeTestJSON-1911423306',owner_user_name='tempest-AttachVolumeTestJSON-1911423306-project-member'},tags=<?>,task_state='powering-on',terminated_at=None,trusted_certs=<?>,updated_at=2023-11-29T03:12:23Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='bb1126c998ee430baac2e2494f30c0aa',uuid=f609a7b7-1be2-44b4-9177-7e72a4d4c060,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='stopped') vif={"id": "5543f379-2b3a-4dc2-bfe9-52f66a942257", "address": "fa:16:3e:04:ae:43", "network": {"id": "294c3aac-8c9b-45f0-8c0a-344e2d7d8b7c", "bridge": "br-int", "label": "tempest-AttachVolumeTestJSON-700098935-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.14", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "e393d23e783b4a40986674d782606c69", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap5543f379-2b", "ovs_interfaceid": "5543f379-2b3a-4dc2-bfe9-52f66a942257", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Nov 29 03:14:13 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-33ffaa40-d676-428f-859e-d86cab3bfa75 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Converting VIF {"id": "5543f379-2b3a-4dc2-bfe9-52f66a942257", "address": "fa:16:3e:04:ae:43", "network": {"id": "294c3aac-8c9b-45f0-8c0a-344e2d7d8b7c", "bridge": "br-int", "label": "tempest-AttachVolumeTestJSON-700098935-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.14", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "e393d23e783b4a40986674d782606c69", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap5543f379-2b", "ovs_interfaceid": "5543f379-2b3a-4dc2-bfe9-52f66a942257", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:14:13 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-33ffaa40-d676-428f-859e-d86cab3bfa75 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:04:ae:43,bridge_name='br-int',has_traffic_filtering=True,id=5543f379-2b3a-4dc2-bfe9-52f66a942257,network=Network(294c3aac-8c9b-45f0-8c0a-344e2d7d8b7c),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap5543f379-2b') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:14:13 user nova-compute[75234]: DEBUG os_vif [None req-33ffaa40-d676-428f-859e-d86cab3bfa75 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Unplugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:04:ae:43,bridge_name='br-int',has_traffic_filtering=True,id=5543f379-2b3a-4dc2-bfe9-52f66a942257,network=Network(294c3aac-8c9b-45f0-8c0a-344e2d7d8b7c),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap5543f379-2b') {{(pid=75234) unplug /opt/stack/data/venv/lib/python3.10/site-packages/os_vif/__init__.py:109}}
Nov 29 03:14:13 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:14:13 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tap5543f379-2b, bridge=br-int, if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:14:13 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:14:13 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:14:13 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:14:13 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbDestroyCommand(_result=None, table=QoS, record=de5d95e4-4f1c-4481-b7b3-c4df9ebda4e9) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:14:13 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:14:13 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:14:13 user nova-compute[75234]: INFO os_vif [None req-33ffaa40-d676-428f-859e-d86cab3bfa75 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Successfully unplugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:04:ae:43,bridge_name='br-int',has_traffic_filtering=True,id=5543f379-2b3a-4dc2-bfe9-52f66a942257,network=Network(294c3aac-8c9b-45f0-8c0a-344e2d7d8b7c),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap5543f379-2b')
Nov 29 03:14:13 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-33ffaa40-d676-428f-859e-d86cab3bfa75 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] [instance: f609a7b7-1be2-44b4-9177-7e72a4d4c060] Start _get_guest_xml network_info=[{"id": "5543f379-2b3a-4dc2-bfe9-52f66a942257", "address": "fa:16:3e:04:ae:43", "network": {"id": "294c3aac-8c9b-45f0-8c0a-344e2d7d8b7c", "bridge": "br-int", "label": "tempest-AttachVolumeTestJSON-700098935-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.14", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "e393d23e783b4a40986674d782606c69", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap5543f379-2b", "ovs_interfaceid": "5543f379-2b3a-4dc2-bfe9-52f66a942257", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum=<?>,container_format='bare',created_at=<?>,direct_url=<?>,disk_format='qcow2',id=96c30ba7-9b73-4a30-8ba4-7138879573b7,min_disk=1,min_ram=0,name=<?>,owner=<?>,properties=ImageMetaProps,protected=<?>,size=<?>,status=<?>,tags=<?>,updated_at=<?>,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [{'device_name': '/dev/vda', 'encryption_format': None, 'encrypted': False, 'device_type': 'disk', 'size': 0, 'disk_bus': 'virtio', 'boot_index': 0, 'encryption_options': None, 'encryption_secret_uuid': None, 'guest_format': None, 'image_id': '96c30ba7-9b73-4a30-8ba4-7138879573b7'}], 'ephemerals': [], 'block_device_mapping': [], 'swap': None} {{(pid=75234) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7562}}
Nov 29 03:14:13 user nova-compute[75234]: DEBUG nova.objects.instance [None req-33ffaa40-d676-428f-859e-d86cab3bfa75 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Lazy-loading 'numa_topology' on Instance uuid f609a7b7-1be2-44b4-9177-7e72a4d4c060 {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:14:13 user nova-compute[75234]: DEBUG nova.objects.base [None req-33ffaa40-d676-428f-859e-d86cab3bfa75 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Object Instance<f609a7b7-1be2-44b4-9177-7e72a4d4c060> lazy-loaded attributes: flavor,info_cache,resources,numa_topology {{(pid=75234) wrapper /opt/stack/nova/nova/objects/base.py:126}}
Nov 29 03:14:13 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-33ffaa40-d676-428f-859e-d86cab3bfa75 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:14:13 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-33ffaa40-d676-428f-859e-d86cab3bfa75 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:14:13 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-33ffaa40-d676-428f-859e-d86cab3bfa75 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Searching host: 'user' for CPU controller through CGroups V1... {{(pid=75234) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1652}}
Nov 29 03:14:13 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-33ffaa40-d676-428f-859e-d86cab3bfa75 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] CPU controller missing on host. {{(pid=75234) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1662}}
Nov 29 03:14:13 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-33ffaa40-d676-428f-859e-d86cab3bfa75 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Searching host: 'user' for CPU controller through CGroups V2... {{(pid=75234) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1671}}
Nov 29 03:14:13 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-33ffaa40-d676-428f-859e-d86cab3bfa75 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] CPU controller found on host. {{(pid=75234) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1678}}
Nov 29 03:14:13 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-33ffaa40-d676-428f-859e-d86cab3bfa75 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=75234) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5400}}
Nov 29 03:14:13 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-33ffaa40-d676-428f-859e-d86cab3bfa75 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-11-29T03:02:04Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='1',id=6,is_public=True,memory_mb=512,name='m1.tiny',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum=<?>,container_format='bare',created_at=<?>,direct_url=<?>,disk_format='qcow2',id=96c30ba7-9b73-4a30-8ba4-7138879573b7,min_disk=1,min_ram=0,name=<?>,owner=<?>,properties=ImageMetaProps,protected=<?>,size=<?>,status=<?>,tags=<?>,updated_at=<?>,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:563}}
Nov 29 03:14:13 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-33ffaa40-d676-428f-859e-d86cab3bfa75 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Flavor limits 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:348}}
Nov 29 03:14:13 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-33ffaa40-d676-428f-859e-d86cab3bfa75 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Image limits 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:352}}
Nov 29 03:14:13 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-33ffaa40-d676-428f-859e-d86cab3bfa75 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Flavor pref 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:388}}
Nov 29 03:14:13 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-33ffaa40-d676-428f-859e-d86cab3bfa75 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Image pref 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:392}}
Nov 29 03:14:13 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-33ffaa40-d676-428f-859e-d86cab3bfa75 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:430}}
Nov 29 03:14:13 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-33ffaa40-d676-428f-859e-d86cab3bfa75 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:569}}
Nov 29 03:14:13 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-33ffaa40-d676-428f-859e-d86cab3bfa75 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=75234) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:471}}
Nov 29 03:14:13 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-33ffaa40-d676-428f-859e-d86cab3bfa75 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Got 1 possible topologies {{(pid=75234) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:501}}
Nov 29 03:14:13 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-33ffaa40-d676-428f-859e-d86cab3bfa75 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:575}}
Nov 29 03:14:13 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-33ffaa40-d676-428f-859e-d86cab3bfa75 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:577}}
Nov 29 03:14:13 user nova-compute[75234]: DEBUG nova.objects.instance [None req-33ffaa40-d676-428f-859e-d86cab3bfa75 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Lazy-loading 'vcpu_model' on Instance uuid f609a7b7-1be2-44b4-9177-7e72a4d4c060 {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:14:13 user nova-compute[75234]: DEBUG nova.objects.base [None req-33ffaa40-d676-428f-859e-d86cab3bfa75 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Object Instance<f609a7b7-1be2-44b4-9177-7e72a4d4c060> lazy-loaded attributes: flavor,info_cache,resources,numa_topology,vcpu_model {{(pid=75234) wrapper /opt/stack/nova/nova/objects/base.py:126}}
Nov 29 03:14:13 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-33ffaa40-d676-428f-859e-d86cab3bfa75 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-11-29T03:06:06Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-AttachVolumeTestJSON-server-394712256',display_name='tempest-AttachVolumeTestJSON-server-394712256',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-attachvolumetestjson-server-394712256',id=4,image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBGphX+MWMhY0YZ/wlOvEQFKFHTIX4g9KMhxkTe+ErHeVHCk7d+KpOlTDVK9edUWFFxjgJKboch3pp1xrAYGa/rIEWyUV4YOfCGPFoAeneAuweb6ekmwg4qgLxfML0aXASQ==',key_name='tempest-keypair-213174705',keypairs=<?>,launch_index=0,launched_at=2023-11-29T03:06:23Z,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=4,progress=0,project_id='e393d23e783b4a40986674d782606c69',ramdisk_id='',reservation_id='r-9o6ggnbe',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=<?>,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-AttachVolumeTestJSON-1911423306',owner_user_name='tempest-AttachVolumeTestJSON-1911423306-project-member'},tags=<?>,task_state='powering-on',terminated_at=None,trusted_certs=<?>,updated_at=2023-11-29T03:12:23Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='bb1126c998ee430baac2e2494f30c0aa',uuid=f609a7b7-1be2-44b4-9177-7e72a4d4c060,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='stopped') vif={"id": "5543f379-2b3a-4dc2-bfe9-52f66a942257", "address": "fa:16:3e:04:ae:43", "network": {"id": "294c3aac-8c9b-45f0-8c0a-344e2d7d8b7c", "bridge": "br-int", "label": "tempest-AttachVolumeTestJSON-700098935-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.14", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "e393d23e783b4a40986674d782606c69", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap5543f379-2b", "ovs_interfaceid": "5543f379-2b3a-4dc2-bfe9-52f66a942257", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=75234) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Nov 29 03:14:13 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-33ffaa40-d676-428f-859e-d86cab3bfa75 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Converting VIF {"id": "5543f379-2b3a-4dc2-bfe9-52f66a942257", "address": "fa:16:3e:04:ae:43", "network": {"id": "294c3aac-8c9b-45f0-8c0a-344e2d7d8b7c", "bridge": "br-int", "label": "tempest-AttachVolumeTestJSON-700098935-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.14", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "e393d23e783b4a40986674d782606c69", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap5543f379-2b", "ovs_interfaceid": "5543f379-2b3a-4dc2-bfe9-52f66a942257", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:14:13 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-33ffaa40-d676-428f-859e-d86cab3bfa75 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:04:ae:43,bridge_name='br-int',has_traffic_filtering=True,id=5543f379-2b3a-4dc2-bfe9-52f66a942257,network=Network(294c3aac-8c9b-45f0-8c0a-344e2d7d8b7c),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap5543f379-2b') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:14:13 user nova-compute[75234]: DEBUG nova.objects.instance [None req-33ffaa40-d676-428f-859e-d86cab3bfa75 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Lazy-loading 'pci_devices' on Instance uuid f609a7b7-1be2-44b4-9177-7e72a4d4c060 {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:14:13 user nova-compute[75234]: DEBUG nova.objects.base [None req-33ffaa40-d676-428f-859e-d86cab3bfa75 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Object Instance<f609a7b7-1be2-44b4-9177-7e72a4d4c060> lazy-loaded attributes: flavor,info_cache,resources,numa_topology,vcpu_model,pci_devices {{(pid=75234) wrapper /opt/stack/nova/nova/objects/base.py:126}}
Nov 29 03:14:13 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-33ffaa40-d676-428f-859e-d86cab3bfa75 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] [instance: f609a7b7-1be2-44b4-9177-7e72a4d4c060] End _get_guest_xml xml=<domain type="kvm">
Nov 29 03:14:13 user nova-compute[75234]:   <uuid>f609a7b7-1be2-44b4-9177-7e72a4d4c060</uuid>
Nov 29 03:14:13 user nova-compute[75234]:   <name>instance-00000004</name>
Nov 29 03:14:13 user nova-compute[75234]:   <memory>524288</memory>
Nov 29 03:14:13 user nova-compute[75234]:   <vcpu>1</vcpu>
Nov 29 03:14:13 user nova-compute[75234]:   <metadata>
Nov 29 03:14:13 user nova-compute[75234]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Nov 29 03:14:13 user nova-compute[75234]:       <nova:package version="0.0.0"/>
Nov 29 03:14:13 user nova-compute[75234]:       <nova:name>tempest-AttachVolumeTestJSON-server-394712256</nova:name>
Nov 29 03:14:13 user nova-compute[75234]:       <nova:creationTime>2023-11-29 03:14:13</nova:creationTime>
Nov 29 03:14:13 user nova-compute[75234]:       <nova:flavor name="m1.tiny">
Nov 29 03:14:13 user nova-compute[75234]:         <nova:memory>512</nova:memory>
Nov 29 03:14:13 user nova-compute[75234]:         <nova:disk>1</nova:disk>
Nov 29 03:14:13 user nova-compute[75234]:         <nova:swap>0</nova:swap>
Nov 29 03:14:13 user nova-compute[75234]:         <nova:ephemeral>0</nova:ephemeral>
Nov 29 03:14:13 user nova-compute[75234]:         <nova:vcpus>1</nova:vcpus>
Nov 29 03:14:13 user nova-compute[75234]:       </nova:flavor>
Nov 29 03:14:13 user nova-compute[75234]:       <nova:owner>
Nov 29 03:14:13 user nova-compute[75234]:         <nova:user uuid="bb1126c998ee430baac2e2494f30c0aa">tempest-AttachVolumeTestJSON-1911423306-project-member</nova:user>
Nov 29 03:14:13 user nova-compute[75234]:         <nova:project uuid="e393d23e783b4a40986674d782606c69">tempest-AttachVolumeTestJSON-1911423306</nova:project>
Nov 29 03:14:13 user nova-compute[75234]:       </nova:owner>
Nov 29 03:14:13 user nova-compute[75234]:       <nova:root type="image" uuid="96c30ba7-9b73-4a30-8ba4-7138879573b7"/>
Nov 29 03:14:13 user nova-compute[75234]:       <nova:ports>
Nov 29 03:14:13 user nova-compute[75234]:         <nova:port uuid="5543f379-2b3a-4dc2-bfe9-52f66a942257">
Nov 29 03:14:13 user nova-compute[75234]:           <nova:ip type="fixed" address="10.0.0.3" ipVersion="4"/>
Nov 29 03:14:13 user nova-compute[75234]:         </nova:port>
Nov 29 03:14:13 user nova-compute[75234]:       </nova:ports>
Nov 29 03:14:13 user nova-compute[75234]:     </nova:instance>
Nov 29 03:14:13 user nova-compute[75234]:   </metadata>
Nov 29 03:14:13 user nova-compute[75234]:   <sysinfo type="smbios">
Nov 29 03:14:13 user nova-compute[75234]:     <system>
Nov 29 03:14:13 user nova-compute[75234]:       <entry name="manufacturer">OpenStack Foundation</entry>
Nov 29 03:14:13 user nova-compute[75234]:       <entry name="product">OpenStack Nova</entry>
Nov 29 03:14:13 user nova-compute[75234]:       <entry name="version">0.0.0</entry>
Nov 29 03:14:13 user nova-compute[75234]:       <entry name="serial">f609a7b7-1be2-44b4-9177-7e72a4d4c060</entry>
Nov 29 03:14:13 user nova-compute[75234]:       <entry name="uuid">f609a7b7-1be2-44b4-9177-7e72a4d4c060</entry>
Nov 29 03:14:13 user nova-compute[75234]:       <entry name="family">Virtual Machine</entry>
Nov 29 03:14:13 user nova-compute[75234]:     </system>
Nov 29 03:14:13 user nova-compute[75234]:   </sysinfo>
Nov 29 03:14:13 user nova-compute[75234]:   <os>
Nov 29 03:14:13 user nova-compute[75234]:     <type arch="x86_64" machine="pc">hvm</type>
Nov 29 03:14:13 user nova-compute[75234]:     <boot dev="hd"/>
Nov 29 03:14:13 user nova-compute[75234]:     <smbios mode="sysinfo"/>
Nov 29 03:14:13 user nova-compute[75234]:   </os>
Nov 29 03:14:13 user nova-compute[75234]:   <features>
Nov 29 03:14:13 user nova-compute[75234]:     <acpi/>
Nov 29 03:14:13 user nova-compute[75234]:     <apic/>
Nov 29 03:14:13 user nova-compute[75234]:     <vmcoreinfo/>
Nov 29 03:14:13 user nova-compute[75234]:   </features>
Nov 29 03:14:13 user nova-compute[75234]:   <clock offset="utc">
Nov 29 03:14:13 user nova-compute[75234]:     <timer name="pit" tickpolicy="delay"/>
Nov 29 03:14:13 user nova-compute[75234]:     <timer name="rtc" tickpolicy="catchup"/>
Nov 29 03:14:13 user nova-compute[75234]:     <timer name="hpet" present="no"/>
Nov 29 03:14:13 user nova-compute[75234]:   </clock>
Nov 29 03:14:13 user nova-compute[75234]:   <cpu mode="custom" match="exact">
Nov 29 03:14:13 user nova-compute[75234]:     <model>Nehalem</model>
Nov 29 03:14:13 user nova-compute[75234]:     <topology sockets="1" cores="1" threads="1"/>
Nov 29 03:14:13 user nova-compute[75234]:   </cpu>
Nov 29 03:14:13 user nova-compute[75234]:   <devices>
Nov 29 03:14:13 user nova-compute[75234]:     <disk type="file" device="disk">
Nov 29 03:14:13 user nova-compute[75234]:       <driver name="qemu" type="qcow2" cache="none"/>
Nov 29 03:14:13 user nova-compute[75234]:       <source file="/opt/stack/data/nova/instances/f609a7b7-1be2-44b4-9177-7e72a4d4c060/disk"/>
Nov 29 03:14:13 user nova-compute[75234]:       <target dev="vda" bus="virtio"/>
Nov 29 03:14:13 user nova-compute[75234]:     </disk>
Nov 29 03:14:13 user nova-compute[75234]:     <interface type="ethernet">
Nov 29 03:14:13 user nova-compute[75234]:       <mac address="fa:16:3e:04:ae:43"/>
Nov 29 03:14:13 user nova-compute[75234]:       <model type="virtio"/>
Nov 29 03:14:13 user nova-compute[75234]:       <mtu size="1442"/>
Nov 29 03:14:13 user nova-compute[75234]:       <target dev="tap5543f379-2b"/>
Nov 29 03:14:13 user nova-compute[75234]:     </interface>
Nov 29 03:14:13 user nova-compute[75234]:     <serial type="pty">
Nov 29 03:14:13 user nova-compute[75234]:       <log file="/opt/stack/data/nova/instances/f609a7b7-1be2-44b4-9177-7e72a4d4c060/console.log" append="off"/>
Nov 29 03:14:13 user nova-compute[75234]:     </serial>
Nov 29 03:14:13 user nova-compute[75234]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Nov 29 03:14:13 user nova-compute[75234]:     <video>
Nov 29 03:14:13 user nova-compute[75234]:       <model type="virtio"/>
Nov 29 03:14:13 user nova-compute[75234]:     </video>
Nov 29 03:14:13 user nova-compute[75234]:     <rng model="virtio">
Nov 29 03:14:13 user nova-compute[75234]:       <backend model="random">/dev/urandom</backend>
Nov 29 03:14:13 user nova-compute[75234]:     </rng>
Nov 29 03:14:13 user nova-compute[75234]:     <controller type="usb" index="0" model="none"/>
Nov 29 03:14:13 user nova-compute[75234]:     <memballoon model="virtio">
Nov 29 03:14:13 user nova-compute[75234]:       <stats period="10"/>
Nov 29 03:14:13 user nova-compute[75234]:     </memballoon>
Nov 29 03:14:13 user nova-compute[75234]:   </devices>
Nov 29 03:14:13 user nova-compute[75234]: </domain>
Nov 29 03:14:13 user nova-compute[75234]:  {{(pid=75234) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7568}}
Nov 29 03:14:13 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-33ffaa40-d676-428f-859e-d86cab3bfa75 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f609a7b7-1be2-44b4-9177-7e72a4d4c060/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:14:13 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-33ffaa40-d676-428f-859e-d86cab3bfa75 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f609a7b7-1be2-44b4-9177-7e72a4d4c060/disk --force-share --output=json" returned: 0 in 0.163s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:14:13 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-33ffaa40-d676-428f-859e-d86cab3bfa75 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f609a7b7-1be2-44b4-9177-7e72a4d4c060/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:14:14 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-33ffaa40-d676-428f-859e-d86cab3bfa75 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f609a7b7-1be2-44b4-9177-7e72a4d4c060/disk --force-share --output=json" returned: 0 in 0.173s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:14:14 user nova-compute[75234]: DEBUG nova.objects.instance [None req-33ffaa40-d676-428f-859e-d86cab3bfa75 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Lazy-loading 'trusted_certs' on Instance uuid f609a7b7-1be2-44b4-9177-7e72a4d4c060 {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:14:14 user nova-compute[75234]: DEBUG nova.objects.base [None req-33ffaa40-d676-428f-859e-d86cab3bfa75 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Object Instance<f609a7b7-1be2-44b4-9177-7e72a4d4c060> lazy-loaded attributes: flavor,info_cache,resources,numa_topology,vcpu_model,pci_devices,trusted_certs {{(pid=75234) wrapper /opt/stack/nova/nova/objects/base.py:126}}
Nov 29 03:14:14 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-33ffaa40-d676-428f-859e-d86cab3bfa75 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:14:14 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-33ffaa40-d676-428f-859e-d86cab3bfa75 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json" returned: 0 in 0.166s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:14:14 user nova-compute[75234]: DEBUG nova.virt.disk.api [None req-33ffaa40-d676-428f-859e-d86cab3bfa75 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Checking if we can resize image /opt/stack/data/nova/instances/f609a7b7-1be2-44b4-9177-7e72a4d4c060/disk. size=1073741824 {{(pid=75234) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:166}}
Nov 29 03:14:14 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-33ffaa40-d676-428f-859e-d86cab3bfa75 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f609a7b7-1be2-44b4-9177-7e72a4d4c060/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:14:14 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-33ffaa40-d676-428f-859e-d86cab3bfa75 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f609a7b7-1be2-44b4-9177-7e72a4d4c060/disk --force-share --output=json" returned: 0 in 0.179s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:14:14 user nova-compute[75234]: DEBUG nova.virt.disk.api [None req-33ffaa40-d676-428f-859e-d86cab3bfa75 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Cannot resize image /opt/stack/data/nova/instances/f609a7b7-1be2-44b4-9177-7e72a4d4c060/disk to a smaller size. {{(pid=75234) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:172}}
Nov 29 03:14:14 user nova-compute[75234]: DEBUG nova.objects.instance [None req-33ffaa40-d676-428f-859e-d86cab3bfa75 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Lazy-loading 'migration_context' on Instance uuid f609a7b7-1be2-44b4-9177-7e72a4d4c060 {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:14:14 user nova-compute[75234]: DEBUG nova.objects.base [None req-33ffaa40-d676-428f-859e-d86cab3bfa75 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Object Instance<f609a7b7-1be2-44b4-9177-7e72a4d4c060> lazy-loaded attributes: flavor,info_cache,resources,numa_topology,vcpu_model,pci_devices,trusted_certs,migration_context {{(pid=75234) wrapper /opt/stack/nova/nova/objects/base.py:126}}
Nov 29 03:14:14 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-33ffaa40-d676-428f-859e-d86cab3bfa75 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-11-29T03:06:06Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-AttachVolumeTestJSON-server-394712256',display_name='tempest-AttachVolumeTestJSON-server-394712256',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-attachvolumetestjson-server-394712256',id=4,image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBGphX+MWMhY0YZ/wlOvEQFKFHTIX4g9KMhxkTe+ErHeVHCk7d+KpOlTDVK9edUWFFxjgJKboch3pp1xrAYGa/rIEWyUV4YOfCGPFoAeneAuweb6ekmwg4qgLxfML0aXASQ==',key_name='tempest-keypair-213174705',keypairs=<?>,launch_index=0,launched_at=2023-11-29T03:06:23Z,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=<?>,power_state=4,progress=0,project_id='e393d23e783b4a40986674d782606c69',ramdisk_id='',reservation_id='r-9o6ggnbe',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=<?>,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-AttachVolumeTestJSON-1911423306',owner_user_name='tempest-AttachVolumeTestJSON-1911423306-project-member'},tags=<?>,task_state='powering-on',terminated_at=None,trusted_certs=None,updated_at=2023-11-29T03:12:23Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='bb1126c998ee430baac2e2494f30c0aa',uuid=f609a7b7-1be2-44b4-9177-7e72a4d4c060,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='stopped') vif={"id": "5543f379-2b3a-4dc2-bfe9-52f66a942257", "address": "fa:16:3e:04:ae:43", "network": {"id": "294c3aac-8c9b-45f0-8c0a-344e2d7d8b7c", "bridge": "br-int", "label": "tempest-AttachVolumeTestJSON-700098935-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.14", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "e393d23e783b4a40986674d782606c69", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap5543f379-2b", "ovs_interfaceid": "5543f379-2b3a-4dc2-bfe9-52f66a942257", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Nov 29 03:14:14 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-33ffaa40-d676-428f-859e-d86cab3bfa75 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Converting VIF {"id": "5543f379-2b3a-4dc2-bfe9-52f66a942257", "address": "fa:16:3e:04:ae:43", "network": {"id": "294c3aac-8c9b-45f0-8c0a-344e2d7d8b7c", "bridge": "br-int", "label": "tempest-AttachVolumeTestJSON-700098935-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.14", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "e393d23e783b4a40986674d782606c69", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap5543f379-2b", "ovs_interfaceid": "5543f379-2b3a-4dc2-bfe9-52f66a942257", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:14:14 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-33ffaa40-d676-428f-859e-d86cab3bfa75 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:04:ae:43,bridge_name='br-int',has_traffic_filtering=True,id=5543f379-2b3a-4dc2-bfe9-52f66a942257,network=Network(294c3aac-8c9b-45f0-8c0a-344e2d7d8b7c),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap5543f379-2b') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:14:14 user nova-compute[75234]: DEBUG os_vif [None req-33ffaa40-d676-428f-859e-d86cab3bfa75 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:04:ae:43,bridge_name='br-int',has_traffic_filtering=True,id=5543f379-2b3a-4dc2-bfe9-52f66a942257,network=Network(294c3aac-8c9b-45f0-8c0a-344e2d7d8b7c),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap5543f379-2b') {{(pid=75234) plug /opt/stack/data/venv/lib/python3.10/site-packages/os_vif/__init__.py:76}}
Nov 29 03:14:14 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:14:14 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:14:14 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Nov 29 03:14:14 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:14:14 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbCreateCommand(_result=None, table=QoS, columns={'type': 'linux-noop', 'external_ids': {'id': 'abd56afa-ec17-518b-948a-b5862316fd9e', '_type': 'linux-noop'}}, row=False) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:14:14 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:14:14 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:14:14 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:14:14 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tap5543f379-2b, may_exist=True, interface_attrs={}) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:14:14 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Port, record=tap5543f379-2b, col_values=(('qos', UUID('3d9da991-ce66-4f39-b6e3-d577c41617c5')),), if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:14:14 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DbSetCommand(_result=None, table=Interface, record=tap5543f379-2b, col_values=(('external_ids', {'iface-id': '5543f379-2b3a-4dc2-bfe9-52f66a942257', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:04:ae:43', 'vm-uuid': 'f609a7b7-1be2-44b4-9177-7e72a4d4c060'}),), if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:14:14 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:14:14 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:14:14 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:14:14 user nova-compute[75234]: INFO os_vif [None req-33ffaa40-d676-428f-859e-d86cab3bfa75 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:04:ae:43,bridge_name='br-int',has_traffic_filtering=True,id=5543f379-2b3a-4dc2-bfe9-52f66a942257,network=Network(294c3aac-8c9b-45f0-8c0a-344e2d7d8b7c),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap5543f379-2b')
Nov 29 03:14:15 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:14:15 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:14:15 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:14:15 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:14:15 user nova-compute[75234]: DEBUG nova.compute.manager [req-90a9be22-b2a3-44cb-a26b-eeab1ec10f9f req-716c7230-0aa6-402c-aa9b-e34ae7f0a73e service nova] [instance: f609a7b7-1be2-44b4-9177-7e72a4d4c060] Received event network-vif-plugged-5543f379-2b3a-4dc2-bfe9-52f66a942257 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:14:15 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-90a9be22-b2a3-44cb-a26b-eeab1ec10f9f req-716c7230-0aa6-402c-aa9b-e34ae7f0a73e service nova] Acquiring lock "f609a7b7-1be2-44b4-9177-7e72a4d4c060-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:14:15 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-90a9be22-b2a3-44cb-a26b-eeab1ec10f9f req-716c7230-0aa6-402c-aa9b-e34ae7f0a73e service nova] Lock "f609a7b7-1be2-44b4-9177-7e72a4d4c060-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:14:15 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-90a9be22-b2a3-44cb-a26b-eeab1ec10f9f req-716c7230-0aa6-402c-aa9b-e34ae7f0a73e service nova] Lock "f609a7b7-1be2-44b4-9177-7e72a4d4c060-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:14:15 user nova-compute[75234]: DEBUG nova.compute.manager [req-90a9be22-b2a3-44cb-a26b-eeab1ec10f9f req-716c7230-0aa6-402c-aa9b-e34ae7f0a73e service nova] [instance: f609a7b7-1be2-44b4-9177-7e72a4d4c060] No waiting events found dispatching network-vif-plugged-5543f379-2b3a-4dc2-bfe9-52f66a942257 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:14:15 user nova-compute[75234]: WARNING nova.compute.manager [req-90a9be22-b2a3-44cb-a26b-eeab1ec10f9f req-716c7230-0aa6-402c-aa9b-e34ae7f0a73e service nova] [instance: f609a7b7-1be2-44b4-9177-7e72a4d4c060] Received unexpected event network-vif-plugged-5543f379-2b3a-4dc2-bfe9-52f66a942257 for instance with vm_state stopped and task_state powering-on.
Nov 29 03:14:16 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:14:16 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:14:16 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:14:17 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:14:17 user nova-compute[75234]: DEBUG nova.compute.manager [None req-33ffaa40-d676-428f-859e-d86cab3bfa75 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] [instance: f609a7b7-1be2-44b4-9177-7e72a4d4c060] Instance event wait completed in 0 seconds for  {{(pid=75234) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Nov 29 03:14:17 user nova-compute[75234]: INFO nova.virt.libvirt.driver [-] [instance: f609a7b7-1be2-44b4-9177-7e72a4d4c060] Instance rebooted successfully.
Nov 29 03:14:17 user nova-compute[75234]: DEBUG nova.compute.manager [None req-33ffaa40-d676-428f-859e-d86cab3bfa75 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] [instance: f609a7b7-1be2-44b4-9177-7e72a4d4c060] Checking state {{(pid=75234) _get_power_state /opt/stack/nova/nova/compute/manager.py:1782}}
Nov 29 03:14:18 user nova-compute[75234]: DEBUG nova.compute.manager [req-38e98dcc-6bd2-47d7-b521-be962fab498f req-0adc3d34-1251-4bf5-bb05-5c69c9d92d83 service nova] [instance: f609a7b7-1be2-44b4-9177-7e72a4d4c060] Received event network-vif-plugged-5543f379-2b3a-4dc2-bfe9-52f66a942257 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:14:18 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-38e98dcc-6bd2-47d7-b521-be962fab498f req-0adc3d34-1251-4bf5-bb05-5c69c9d92d83 service nova] Acquiring lock "f609a7b7-1be2-44b4-9177-7e72a4d4c060-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:14:18 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-38e98dcc-6bd2-47d7-b521-be962fab498f req-0adc3d34-1251-4bf5-bb05-5c69c9d92d83 service nova] Lock "f609a7b7-1be2-44b4-9177-7e72a4d4c060-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:14:18 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-38e98dcc-6bd2-47d7-b521-be962fab498f req-0adc3d34-1251-4bf5-bb05-5c69c9d92d83 service nova] Lock "f609a7b7-1be2-44b4-9177-7e72a4d4c060-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:14:18 user nova-compute[75234]: DEBUG nova.compute.manager [req-38e98dcc-6bd2-47d7-b521-be962fab498f req-0adc3d34-1251-4bf5-bb05-5c69c9d92d83 service nova] [instance: f609a7b7-1be2-44b4-9177-7e72a4d4c060] No waiting events found dispatching network-vif-plugged-5543f379-2b3a-4dc2-bfe9-52f66a942257 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:14:18 user nova-compute[75234]: WARNING nova.compute.manager [req-38e98dcc-6bd2-47d7-b521-be962fab498f req-0adc3d34-1251-4bf5-bb05-5c69c9d92d83 service nova] [instance: f609a7b7-1be2-44b4-9177-7e72a4d4c060] Received unexpected event network-vif-plugged-5543f379-2b3a-4dc2-bfe9-52f66a942257 for instance with vm_state active and task_state None.
Nov 29 03:14:19 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:14:19 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:14:21 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:14:24 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:14:29 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:14:29 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:14:29 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5002 ms, sending inactivity probe {{(pid=75234) run /opt/stack/data/venv/lib/python3.10/site-packages/ovs/reconnect.py:117}}
Nov 29 03:14:29 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=75234) _transition /opt/stack/data/venv/lib/python3.10/site-packages/ovs/reconnect.py:519}}
Nov 29 03:14:29 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=75234) _transition /opt/stack/data/venv/lib/python3.10/site-packages/ovs/reconnect.py:519}}
Nov 29 03:14:29 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:14:34 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:14:34 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:14:34 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5002 ms, sending inactivity probe {{(pid=75234) run /opt/stack/data/venv/lib/python3.10/site-packages/ovs/reconnect.py:117}}
Nov 29 03:14:34 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=75234) _transition /opt/stack/data/venv/lib/python3.10/site-packages/ovs/reconnect.py:519}}
Nov 29 03:14:34 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=75234) _transition /opt/stack/data/venv/lib/python3.10/site-packages/ovs/reconnect.py:519}}
Nov 29 03:14:34 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:14:36 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:14:39 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:14:39 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-d2f74453-4585-4b1a-bd85-2dd6d9c54aa0 tempest-TestVolumeSwap-1089931716 tempest-TestVolumeSwap-1089931716-project-member] Acquiring lock "3e355ae8-aab9-4ae9-b6e3-6de108ec1af9" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:14:39 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-d2f74453-4585-4b1a-bd85-2dd6d9c54aa0 tempest-TestVolumeSwap-1089931716 tempest-TestVolumeSwap-1089931716-project-member] Lock "3e355ae8-aab9-4ae9-b6e3-6de108ec1af9" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:14:39 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-d2f74453-4585-4b1a-bd85-2dd6d9c54aa0 tempest-TestVolumeSwap-1089931716 tempest-TestVolumeSwap-1089931716-project-member] Acquiring lock "3e355ae8-aab9-4ae9-b6e3-6de108ec1af9-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:14:39 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-d2f74453-4585-4b1a-bd85-2dd6d9c54aa0 tempest-TestVolumeSwap-1089931716 tempest-TestVolumeSwap-1089931716-project-member] Lock "3e355ae8-aab9-4ae9-b6e3-6de108ec1af9-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:14:39 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-d2f74453-4585-4b1a-bd85-2dd6d9c54aa0 tempest-TestVolumeSwap-1089931716 tempest-TestVolumeSwap-1089931716-project-member] Lock "3e355ae8-aab9-4ae9-b6e3-6de108ec1af9-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:14:39 user nova-compute[75234]: INFO nova.compute.manager [None req-d2f74453-4585-4b1a-bd85-2dd6d9c54aa0 tempest-TestVolumeSwap-1089931716 tempest-TestVolumeSwap-1089931716-project-member] [instance: 3e355ae8-aab9-4ae9-b6e3-6de108ec1af9] Terminating instance
Nov 29 03:14:39 user nova-compute[75234]: DEBUG nova.compute.manager [None req-d2f74453-4585-4b1a-bd85-2dd6d9c54aa0 tempest-TestVolumeSwap-1089931716 tempest-TestVolumeSwap-1089931716-project-member] [instance: 3e355ae8-aab9-4ae9-b6e3-6de108ec1af9] Start destroying the instance on the hypervisor. {{(pid=75234) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3140}}
Nov 29 03:14:39 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:14:39 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:14:39 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:14:39 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:14:39 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:14:40 user nova-compute[75234]: DEBUG nova.compute.manager [req-89b83c54-393c-4a6f-ae25-69891e3932ea req-7b2452e9-e002-4673-8e5a-1ef11a157f81 service nova] [instance: 3e355ae8-aab9-4ae9-b6e3-6de108ec1af9] Received event network-vif-unplugged-b70722c1-452e-4b37-9ff3-7d8a1c7a9ba3 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:14:40 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-89b83c54-393c-4a6f-ae25-69891e3932ea req-7b2452e9-e002-4673-8e5a-1ef11a157f81 service nova] Acquiring lock "3e355ae8-aab9-4ae9-b6e3-6de108ec1af9-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:14:40 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-89b83c54-393c-4a6f-ae25-69891e3932ea req-7b2452e9-e002-4673-8e5a-1ef11a157f81 service nova] Lock "3e355ae8-aab9-4ae9-b6e3-6de108ec1af9-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:14:40 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-89b83c54-393c-4a6f-ae25-69891e3932ea req-7b2452e9-e002-4673-8e5a-1ef11a157f81 service nova] Lock "3e355ae8-aab9-4ae9-b6e3-6de108ec1af9-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:14:40 user nova-compute[75234]: DEBUG nova.compute.manager [req-89b83c54-393c-4a6f-ae25-69891e3932ea req-7b2452e9-e002-4673-8e5a-1ef11a157f81 service nova] [instance: 3e355ae8-aab9-4ae9-b6e3-6de108ec1af9] No waiting events found dispatching network-vif-unplugged-b70722c1-452e-4b37-9ff3-7d8a1c7a9ba3 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:14:40 user nova-compute[75234]: DEBUG nova.compute.manager [req-89b83c54-393c-4a6f-ae25-69891e3932ea req-7b2452e9-e002-4673-8e5a-1ef11a157f81 service nova] [instance: 3e355ae8-aab9-4ae9-b6e3-6de108ec1af9] Received event network-vif-unplugged-b70722c1-452e-4b37-9ff3-7d8a1c7a9ba3 for instance with task_state deleting. {{(pid=75234) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10890}}
Nov 29 03:14:40 user nova-compute[75234]: INFO nova.virt.libvirt.driver [-] [instance: 3e355ae8-aab9-4ae9-b6e3-6de108ec1af9] Instance destroyed successfully.
Nov 29 03:14:40 user nova-compute[75234]: DEBUG nova.objects.instance [None req-d2f74453-4585-4b1a-bd85-2dd6d9c54aa0 tempest-TestVolumeSwap-1089931716 tempest-TestVolumeSwap-1089931716-project-member] Lazy-loading 'resources' on Instance uuid 3e355ae8-aab9-4ae9-b6e3-6de108ec1af9 {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:14:40 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-d2f74453-4585-4b1a-bd85-2dd6d9c54aa0 tempest-TestVolumeSwap-1089931716 tempest-TestVolumeSwap-1089931716-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-11-29T03:11:05Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-TestVolumeSwap-server-1891489941',display_name='tempest-TestVolumeSwap-server-1891489941',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-testvolumeswap-server-1891489941',id=20,image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBErO5gpIWBBUnin4dWgri2kRPv0Oc5PIA/2FQHxsJbVhAZPCTRN7Vo3YWPw19dRA90B59GrDhvW0wQP5nRsc6TH8eYdjlY58N/MAEOrS3Qhl7HjnnODcwnB/z6MrUdlMZA==',key_name='tempest-keypair-238289792',keypairs=<?>,launch_index=0,launched_at=2023-11-29T03:11:13Z,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='cd4a0b5a414044d68524962183d47fca',ramdisk_id='',reservation_id='r-k7dv1o4o',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-TestVolumeSwap-1089931716',owner_user_name='tempest-TestVolumeSwap-1089931716-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-11-29T03:11:14Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='5366feb605674f9a8c7f547ba2beb65c',uuid=3e355ae8-aab9-4ae9-b6e3-6de108ec1af9,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "b70722c1-452e-4b37-9ff3-7d8a1c7a9ba3", "address": "fa:16:3e:db:81:b1", "network": {"id": "67d2e671-98cd-4178-8f5f-1ee06dd66006", "bridge": "br-int", "label": "tempest-TestVolumeSwap-1472119110-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.131", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "cd4a0b5a414044d68524962183d47fca", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapb70722c1-45", "ovs_interfaceid": "b70722c1-452e-4b37-9ff3-7d8a1c7a9ba3", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Nov 29 03:14:40 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-d2f74453-4585-4b1a-bd85-2dd6d9c54aa0 tempest-TestVolumeSwap-1089931716 tempest-TestVolumeSwap-1089931716-project-member] Converting VIF {"id": "b70722c1-452e-4b37-9ff3-7d8a1c7a9ba3", "address": "fa:16:3e:db:81:b1", "network": {"id": "67d2e671-98cd-4178-8f5f-1ee06dd66006", "bridge": "br-int", "label": "tempest-TestVolumeSwap-1472119110-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.131", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "cd4a0b5a414044d68524962183d47fca", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapb70722c1-45", "ovs_interfaceid": "b70722c1-452e-4b37-9ff3-7d8a1c7a9ba3", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:14:40 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-d2f74453-4585-4b1a-bd85-2dd6d9c54aa0 tempest-TestVolumeSwap-1089931716 tempest-TestVolumeSwap-1089931716-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:db:81:b1,bridge_name='br-int',has_traffic_filtering=True,id=b70722c1-452e-4b37-9ff3-7d8a1c7a9ba3,network=Network(67d2e671-98cd-4178-8f5f-1ee06dd66006),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapb70722c1-45') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:14:40 user nova-compute[75234]: DEBUG os_vif [None req-d2f74453-4585-4b1a-bd85-2dd6d9c54aa0 tempest-TestVolumeSwap-1089931716 tempest-TestVolumeSwap-1089931716-project-member] Unplugging vif VIFOpenVSwitch(active=True,address=fa:16:3e:db:81:b1,bridge_name='br-int',has_traffic_filtering=True,id=b70722c1-452e-4b37-9ff3-7d8a1c7a9ba3,network=Network(67d2e671-98cd-4178-8f5f-1ee06dd66006),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapb70722c1-45') {{(pid=75234) unplug /opt/stack/data/venv/lib/python3.10/site-packages/os_vif/__init__.py:109}}
Nov 29 03:14:40 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:14:40 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tapb70722c1-45, bridge=br-int, if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:14:40 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:14:40 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:14:40 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:14:40 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbDestroyCommand(_result=None, table=QoS, record=7aaeed91-7b6e-4ea2-a1f7-fc9e7bd8d604) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:14:40 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:14:40 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:14:40 user nova-compute[75234]: INFO os_vif [None req-d2f74453-4585-4b1a-bd85-2dd6d9c54aa0 tempest-TestVolumeSwap-1089931716 tempest-TestVolumeSwap-1089931716-project-member] Successfully unplugged vif VIFOpenVSwitch(active=True,address=fa:16:3e:db:81:b1,bridge_name='br-int',has_traffic_filtering=True,id=b70722c1-452e-4b37-9ff3-7d8a1c7a9ba3,network=Network(67d2e671-98cd-4178-8f5f-1ee06dd66006),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapb70722c1-45')
Nov 29 03:14:40 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-d2f74453-4585-4b1a-bd85-2dd6d9c54aa0 tempest-TestVolumeSwap-1089931716 tempest-TestVolumeSwap-1089931716-project-member] [instance: 3e355ae8-aab9-4ae9-b6e3-6de108ec1af9] Deleting instance files /opt/stack/data/nova/instances/3e355ae8-aab9-4ae9-b6e3-6de108ec1af9_del
Nov 29 03:14:40 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-d2f74453-4585-4b1a-bd85-2dd6d9c54aa0 tempest-TestVolumeSwap-1089931716 tempest-TestVolumeSwap-1089931716-project-member] [instance: 3e355ae8-aab9-4ae9-b6e3-6de108ec1af9] Deletion of /opt/stack/data/nova/instances/3e355ae8-aab9-4ae9-b6e3-6de108ec1af9_del complete
Nov 29 03:14:40 user nova-compute[75234]: INFO nova.compute.manager [None req-d2f74453-4585-4b1a-bd85-2dd6d9c54aa0 tempest-TestVolumeSwap-1089931716 tempest-TestVolumeSwap-1089931716-project-member] [instance: 3e355ae8-aab9-4ae9-b6e3-6de108ec1af9] Took 0.89 seconds to destroy the instance on the hypervisor.
Nov 29 03:14:40 user nova-compute[75234]: DEBUG oslo.service.loopingcall [None req-d2f74453-4585-4b1a-bd85-2dd6d9c54aa0 tempest-TestVolumeSwap-1089931716 tempest-TestVolumeSwap-1089931716-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=75234) func /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/loopingcall.py:435}}
Nov 29 03:14:40 user nova-compute[75234]: DEBUG nova.compute.manager [-] [instance: 3e355ae8-aab9-4ae9-b6e3-6de108ec1af9] Deallocating network for instance {{(pid=75234) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2279}}
Nov 29 03:14:40 user nova-compute[75234]: DEBUG nova.network.neutron [-] [instance: 3e355ae8-aab9-4ae9-b6e3-6de108ec1af9] deallocate_for_instance() {{(pid=75234) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1802}}
Nov 29 03:14:41 user nova-compute[75234]: DEBUG nova.network.neutron [-] [instance: 3e355ae8-aab9-4ae9-b6e3-6de108ec1af9] Updating instance_info_cache with network_info: [] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:14:41 user nova-compute[75234]: INFO nova.compute.manager [-] [instance: 3e355ae8-aab9-4ae9-b6e3-6de108ec1af9] Took 0.72 seconds to deallocate network for instance.
Nov 29 03:14:41 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-d2f74453-4585-4b1a-bd85-2dd6d9c54aa0 tempest-TestVolumeSwap-1089931716 tempest-TestVolumeSwap-1089931716-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:14:41 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-d2f74453-4585-4b1a-bd85-2dd6d9c54aa0 tempest-TestVolumeSwap-1089931716 tempest-TestVolumeSwap-1089931716-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:14:41 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:14:41 user nova-compute[75234]: DEBUG nova.compute.provider_tree [None req-d2f74453-4585-4b1a-bd85-2dd6d9c54aa0 tempest-TestVolumeSwap-1089931716 tempest-TestVolumeSwap-1089931716-project-member] Inventory has not changed in ProviderTree for provider: 4d291220-7c37-4839-a0e3-b23c66502b74 {{(pid=75234) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Nov 29 03:14:41 user nova-compute[75234]: DEBUG nova.scheduler.client.report [None req-d2f74453-4585-4b1a-bd85-2dd6d9c54aa0 tempest-TestVolumeSwap-1089931716 tempest-TestVolumeSwap-1089931716-project-member] Inventory has not changed for provider 4d291220-7c37-4839-a0e3-b23c66502b74 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=75234) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Nov 29 03:14:41 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-d2f74453-4585-4b1a-bd85-2dd6d9c54aa0 tempest-TestVolumeSwap-1089931716 tempest-TestVolumeSwap-1089931716-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.396s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:14:41 user nova-compute[75234]: INFO nova.scheduler.client.report [None req-d2f74453-4585-4b1a-bd85-2dd6d9c54aa0 tempest-TestVolumeSwap-1089931716 tempest-TestVolumeSwap-1089931716-project-member] Deleted allocations for instance 3e355ae8-aab9-4ae9-b6e3-6de108ec1af9
Nov 29 03:14:41 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-d2f74453-4585-4b1a-bd85-2dd6d9c54aa0 tempest-TestVolumeSwap-1089931716 tempest-TestVolumeSwap-1089931716-project-member] Lock "3e355ae8-aab9-4ae9-b6e3-6de108ec1af9" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 2.227s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:14:42 user nova-compute[75234]: DEBUG nova.compute.manager [req-44bd295a-8ec0-4593-8c9b-36027189ec76 req-cfe1455d-7454-42e2-9319-70bd574a84be service nova] [instance: 3e355ae8-aab9-4ae9-b6e3-6de108ec1af9] Received event network-vif-plugged-b70722c1-452e-4b37-9ff3-7d8a1c7a9ba3 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:14:42 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-44bd295a-8ec0-4593-8c9b-36027189ec76 req-cfe1455d-7454-42e2-9319-70bd574a84be service nova] Acquiring lock "3e355ae8-aab9-4ae9-b6e3-6de108ec1af9-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:14:42 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-44bd295a-8ec0-4593-8c9b-36027189ec76 req-cfe1455d-7454-42e2-9319-70bd574a84be service nova] Lock "3e355ae8-aab9-4ae9-b6e3-6de108ec1af9-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:14:42 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-44bd295a-8ec0-4593-8c9b-36027189ec76 req-cfe1455d-7454-42e2-9319-70bd574a84be service nova] Lock "3e355ae8-aab9-4ae9-b6e3-6de108ec1af9-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:14:42 user nova-compute[75234]: DEBUG nova.compute.manager [req-44bd295a-8ec0-4593-8c9b-36027189ec76 req-cfe1455d-7454-42e2-9319-70bd574a84be service nova] [instance: 3e355ae8-aab9-4ae9-b6e3-6de108ec1af9] No waiting events found dispatching network-vif-plugged-b70722c1-452e-4b37-9ff3-7d8a1c7a9ba3 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:14:42 user nova-compute[75234]: WARNING nova.compute.manager [req-44bd295a-8ec0-4593-8c9b-36027189ec76 req-cfe1455d-7454-42e2-9319-70bd574a84be service nova] [instance: 3e355ae8-aab9-4ae9-b6e3-6de108ec1af9] Received unexpected event network-vif-plugged-b70722c1-452e-4b37-9ff3-7d8a1c7a9ba3 for instance with vm_state deleted and task_state None.
Nov 29 03:14:42 user nova-compute[75234]: DEBUG nova.compute.manager [req-44bd295a-8ec0-4593-8c9b-36027189ec76 req-cfe1455d-7454-42e2-9319-70bd574a84be service nova] [instance: 3e355ae8-aab9-4ae9-b6e3-6de108ec1af9] Received event network-vif-deleted-b70722c1-452e-4b37-9ff3-7d8a1c7a9ba3 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:14:45 user nova-compute[75234]: DEBUG nova.compute.manager [req-8ec65124-2c68-4243-afab-97cc82bd5c50 req-4bb5b662-963d-4619-a99a-2cb454cc172a service nova] [instance: 6894b55f-93c6-43c9-8933-56d5a3c43725] Received event network-changed-9b2cf46e-008c-4ef3-9555-0cbc584dcf8f {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:14:45 user nova-compute[75234]: DEBUG nova.compute.manager [req-8ec65124-2c68-4243-afab-97cc82bd5c50 req-4bb5b662-963d-4619-a99a-2cb454cc172a service nova] [instance: 6894b55f-93c6-43c9-8933-56d5a3c43725] Refreshing instance network info cache due to event network-changed-9b2cf46e-008c-4ef3-9555-0cbc584dcf8f. {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11117}}
Nov 29 03:14:45 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-8ec65124-2c68-4243-afab-97cc82bd5c50 req-4bb5b662-963d-4619-a99a-2cb454cc172a service nova] Acquiring lock "refresh_cache-6894b55f-93c6-43c9-8933-56d5a3c43725" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:14:45 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-8ec65124-2c68-4243-afab-97cc82bd5c50 req-4bb5b662-963d-4619-a99a-2cb454cc172a service nova] Acquired lock "refresh_cache-6894b55f-93c6-43c9-8933-56d5a3c43725" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:14:45 user nova-compute[75234]: DEBUG nova.network.neutron [req-8ec65124-2c68-4243-afab-97cc82bd5c50 req-4bb5b662-963d-4619-a99a-2cb454cc172a service nova] [instance: 6894b55f-93c6-43c9-8933-56d5a3c43725] Refreshing network info cache for port 9b2cf46e-008c-4ef3-9555-0cbc584dcf8f {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2006}}
Nov 29 03:14:45 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:14:45 user nova-compute[75234]: DEBUG nova.network.neutron [req-8ec65124-2c68-4243-afab-97cc82bd5c50 req-4bb5b662-963d-4619-a99a-2cb454cc172a service nova] [instance: 6894b55f-93c6-43c9-8933-56d5a3c43725] Updated VIF entry in instance network info cache for port 9b2cf46e-008c-4ef3-9555-0cbc584dcf8f. {{(pid=75234) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3481}}
Nov 29 03:14:45 user nova-compute[75234]: DEBUG nova.network.neutron [req-8ec65124-2c68-4243-afab-97cc82bd5c50 req-4bb5b662-963d-4619-a99a-2cb454cc172a service nova] [instance: 6894b55f-93c6-43c9-8933-56d5a3c43725] Updating instance_info_cache with network_info: [{"id": "9b2cf46e-008c-4ef3-9555-0cbc584dcf8f", "address": "fa:16:3e:d1:0b:00", "network": {"id": "b7a6cef0-4cf1-46e6-99fb-4972879e3913", "bridge": "br-int", "label": "tempest-TestStampPattern-1195861223-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.73", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "c4393bc598a845a9bfb667e5332f14a6", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap9b2cf46e-00", "ovs_interfaceid": "9b2cf46e-008c-4ef3-9555-0cbc584dcf8f", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:14:45 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-8ec65124-2c68-4243-afab-97cc82bd5c50 req-4bb5b662-963d-4619-a99a-2cb454cc172a service nova] Releasing lock "refresh_cache-6894b55f-93c6-43c9-8933-56d5a3c43725" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:14:45 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-5e94a590-65a3-40ab-b081-3bc7d55c8204 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Acquiring lock "6894b55f-93c6-43c9-8933-56d5a3c43725" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:14:45 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-5e94a590-65a3-40ab-b081-3bc7d55c8204 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Lock "6894b55f-93c6-43c9-8933-56d5a3c43725" acquired by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:14:45 user nova-compute[75234]: DEBUG nova.objects.instance [None req-5e94a590-65a3-40ab-b081-3bc7d55c8204 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Lazy-loading 'flavor' on Instance uuid 6894b55f-93c6-43c9-8933-56d5a3c43725 {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:14:46 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-5e94a590-65a3-40ab-b081-3bc7d55c8204 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Lock "6894b55f-93c6-43c9-8933-56d5a3c43725" "released" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: held 0.060s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:14:46 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-5e94a590-65a3-40ab-b081-3bc7d55c8204 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Acquiring lock "6894b55f-93c6-43c9-8933-56d5a3c43725" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:14:46 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-5e94a590-65a3-40ab-b081-3bc7d55c8204 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Lock "6894b55f-93c6-43c9-8933-56d5a3c43725" acquired by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:14:46 user nova-compute[75234]: INFO nova.compute.manager [None req-5e94a590-65a3-40ab-b081-3bc7d55c8204 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] [instance: 6894b55f-93c6-43c9-8933-56d5a3c43725] Attaching volume 104bcac8-98ce-42f8-a85f-82e29050a44c to /dev/vdb
Nov 29 03:14:46 user nova-compute[75234]: DEBUG os_brick.utils [None req-5e94a590-65a3-40ab-b081-3bc7d55c8204 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:176}}
Nov 29 03:14:46 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:14:46 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.014s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:14:46 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[75da9a02-d8e2-44b3-b338-1b7c5e1ec50b]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:dd1368c5ba2c\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:14:46 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt -v / -n -o SOURCE {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:14:46 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt -v / -n -o SOURCE" returned: 0 in 0.016s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:14:46 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[7ba1e484-31d6-4b37-9688-299334c2b58d]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:14:46 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:14:46 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.025s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:14:46 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[acf5b4cc-5d8b-4c82-92ec-8bbf52860dc9]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:14:46 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[e7358538-c5ca-4ced-a2c7-ab2f463081ec]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:14:46 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-5e94a590-65a3-40ab-b081-3bc7d55c8204 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Running cmd (subprocess): nvme version {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:14:46 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-5e94a590-65a3-40ab-b081-3bc7d55c8204 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] 'nvme version' failed. Not Retrying. {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:473}}
Nov 29 03:14:46 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.nvmeof [None req-5e94a590-65a3-40ab-b081-3bc7d55c8204 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] nvme not present on system {{(pid=75234) nvme_present /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/nvmeof.py:756}}
Nov 29 03:14:46 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:14:46 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:473}}
Nov 29 03:14:46 user nova-compute[75234]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Nov 29 03:14:46 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[dc9ae69e-e76e-4060-b349-aee7233aad3e]: (4, '') {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:14:46 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.lightos [None req-5e94a590-65a3-40ab-b081-3bc7d55c8204 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=75234) find_dsc /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/lightos.py:98}}
Nov 29 03:14:46 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.lightos [None req-5e94a590-65a3-40ab-b081-3bc7d55c8204 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=75234) get_connector_properties /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/lightos.py:76}}
Nov 29 03:14:46 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.lightos [None req-5e94a590-65a3-40ab-b081-3bc7d55c8204 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] LIGHTOS: no hostnqn found. {{(pid=75234) get_connector_properties /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/lightos.py:84}}
Nov 29 03:14:46 user nova-compute[75234]: DEBUG os_brick.utils [None req-5e94a590-65a3-40ab-b081-3bc7d55c8204 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] <== get_connector_properties: return (129ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:dd1368c5ba2c', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:203}}
Nov 29 03:14:46 user nova-compute[75234]: DEBUG nova.virt.block_device [None req-5e94a590-65a3-40ab-b081-3bc7d55c8204 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] [instance: 6894b55f-93c6-43c9-8933-56d5a3c43725] Updating existing volume attachment record: af355cde-e829-4dd0-81c6-3a91b5e1b2e3 {{(pid=75234) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Nov 29 03:14:46 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:14:47 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-78fd111a-b14e-4026-965f-36b5c8becb30 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Acquiring lock "6c0e0184-ca3a-48ec-a6e3-a0a1172260d6" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:14:47 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-78fd111a-b14e-4026-965f-36b5c8becb30 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Lock "6c0e0184-ca3a-48ec-a6e3-a0a1172260d6" acquired by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: waited 0.002s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:14:47 user nova-compute[75234]: INFO nova.compute.manager [None req-78fd111a-b14e-4026-965f-36b5c8becb30 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] [instance: 6c0e0184-ca3a-48ec-a6e3-a0a1172260d6] Detaching volume 56237f4c-4138-4f50-aafb-9c0f9d4bb1cf
Nov 29 03:14:47 user nova-compute[75234]: INFO nova.virt.block_device [None req-78fd111a-b14e-4026-965f-36b5c8becb30 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] [instance: 6c0e0184-ca3a-48ec-a6e3-a0a1172260d6] Attempting to driver detach volume 56237f4c-4138-4f50-aafb-9c0f9d4bb1cf from mountpoint /dev/vdb
Nov 29 03:14:47 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-78fd111a-b14e-4026-965f-36b5c8becb30 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Attempting to detach device vdb from instance 6c0e0184-ca3a-48ec-a6e3-a0a1172260d6 from the persistent domain config. {{(pid=75234) _detach_from_persistent /opt/stack/nova/nova/virt/libvirt/driver.py:2487}}
Nov 29 03:14:47 user nova-compute[75234]: DEBUG nova.virt.libvirt.guest [None req-78fd111a-b14e-4026-965f-36b5c8becb30 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] detach device xml: <disk type="block" device="disk">
Nov 29 03:14:47 user nova-compute[75234]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Nov 29 03:14:47 user nova-compute[75234]:   <source dev="/dev/sdf"/>
Nov 29 03:14:47 user nova-compute[75234]:   <target dev="vdb" bus="virtio"/>
Nov 29 03:14:47 user nova-compute[75234]:   <serial>56237f4c-4138-4f50-aafb-9c0f9d4bb1cf</serial>
Nov 29 03:14:47 user nova-compute[75234]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x05" function="0x0"/>
Nov 29 03:14:47 user nova-compute[75234]: </disk>
Nov 29 03:14:47 user nova-compute[75234]:  {{(pid=75234) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Nov 29 03:14:47 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-78fd111a-b14e-4026-965f-36b5c8becb30 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Successfully detached device vdb from instance 6c0e0184-ca3a-48ec-a6e3-a0a1172260d6 from the persistent domain config.
Nov 29 03:14:47 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-78fd111a-b14e-4026-965f-36b5c8becb30 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] (1/8): Attempting to detach device vdb with device alias virtio-disk1 from instance 6c0e0184-ca3a-48ec-a6e3-a0a1172260d6 from the live domain config. {{(pid=75234) _detach_from_live_with_retry /opt/stack/nova/nova/virt/libvirt/driver.py:2523}}
Nov 29 03:14:47 user nova-compute[75234]: DEBUG nova.virt.libvirt.guest [None req-78fd111a-b14e-4026-965f-36b5c8becb30 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] detach device xml: <disk type="block" device="disk">
Nov 29 03:14:47 user nova-compute[75234]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Nov 29 03:14:47 user nova-compute[75234]:   <source dev="/dev/sdf"/>
Nov 29 03:14:47 user nova-compute[75234]:   <target dev="vdb" bus="virtio"/>
Nov 29 03:14:47 user nova-compute[75234]:   <serial>56237f4c-4138-4f50-aafb-9c0f9d4bb1cf</serial>
Nov 29 03:14:47 user nova-compute[75234]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x05" function="0x0"/>
Nov 29 03:14:47 user nova-compute[75234]: </disk>
Nov 29 03:14:47 user nova-compute[75234]:  {{(pid=75234) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Nov 29 03:14:47 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-78fd111a-b14e-4026-965f-36b5c8becb30 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Start waiting for the detach event from libvirt for device vdb with device alias virtio-disk1 for instance 6c0e0184-ca3a-48ec-a6e3-a0a1172260d6 {{(pid=75234) _detach_from_live_and_wait_for_event /opt/stack/nova/nova/virt/libvirt/driver.py:2599}}
Nov 29 03:14:47 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-91b74874-e949-461c-a452-0a0347214f04 tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] Acquiring lock "2bf9fe41-63c2-4882-bcdd-8e314e734ffe" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:14:47 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-91b74874-e949-461c-a452-0a0347214f04 tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] Lock "2bf9fe41-63c2-4882-bcdd-8e314e734ffe" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:14:47 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-91b74874-e949-461c-a452-0a0347214f04 tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] Acquiring lock "2bf9fe41-63c2-4882-bcdd-8e314e734ffe-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:14:47 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-91b74874-e949-461c-a452-0a0347214f04 tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] Lock "2bf9fe41-63c2-4882-bcdd-8e314e734ffe-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:14:47 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-91b74874-e949-461c-a452-0a0347214f04 tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] Lock "2bf9fe41-63c2-4882-bcdd-8e314e734ffe-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:14:47 user nova-compute[75234]: INFO nova.compute.manager [None req-91b74874-e949-461c-a452-0a0347214f04 tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] [instance: 2bf9fe41-63c2-4882-bcdd-8e314e734ffe] Terminating instance
Nov 29 03:14:47 user nova-compute[75234]: DEBUG nova.compute.manager [None req-91b74874-e949-461c-a452-0a0347214f04 tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] [instance: 2bf9fe41-63c2-4882-bcdd-8e314e734ffe] Start destroying the instance on the hypervisor. {{(pid=75234) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3140}}
Nov 29 03:14:48 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:14:48 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:14:48 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:14:48 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:14:48 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:14:48 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-29b9361b-3bae-4aea-a02b-bbee9ce1102a tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Acquiring lock "a9350a18-14e6-42c6-a150-0a1487f20ad9" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:14:48 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-29b9361b-3bae-4aea-a02b-bbee9ce1102a tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Lock "a9350a18-14e6-42c6-a150-0a1487f20ad9" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:14:48 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-29b9361b-3bae-4aea-a02b-bbee9ce1102a tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Acquiring lock "a9350a18-14e6-42c6-a150-0a1487f20ad9-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:14:48 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-29b9361b-3bae-4aea-a02b-bbee9ce1102a tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Lock "a9350a18-14e6-42c6-a150-0a1487f20ad9-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:14:48 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-29b9361b-3bae-4aea-a02b-bbee9ce1102a tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Lock "a9350a18-14e6-42c6-a150-0a1487f20ad9-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:14:48 user nova-compute[75234]: INFO nova.compute.manager [None req-29b9361b-3bae-4aea-a02b-bbee9ce1102a tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] [instance: a9350a18-14e6-42c6-a150-0a1487f20ad9] Terminating instance
Nov 29 03:14:48 user nova-compute[75234]: DEBUG nova.compute.manager [None req-29b9361b-3bae-4aea-a02b-bbee9ce1102a tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] [instance: a9350a18-14e6-42c6-a150-0a1487f20ad9] Start destroying the instance on the hypervisor. {{(pid=75234) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3140}}
Nov 29 03:14:48 user nova-compute[75234]: DEBUG nova.compute.manager [req-a68e9ef5-292f-4ada-89d9-dad236a443f5 req-81d9cd0e-81cf-41ad-99d3-f36b1727a44e service nova] [instance: 2bf9fe41-63c2-4882-bcdd-8e314e734ffe] Received event network-vif-unplugged-c8bb20fd-5c85-4e89-8c61-13a4f17dd4cc {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:14:48 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-a68e9ef5-292f-4ada-89d9-dad236a443f5 req-81d9cd0e-81cf-41ad-99d3-f36b1727a44e service nova] Acquiring lock "2bf9fe41-63c2-4882-bcdd-8e314e734ffe-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:14:48 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-a68e9ef5-292f-4ada-89d9-dad236a443f5 req-81d9cd0e-81cf-41ad-99d3-f36b1727a44e service nova] Lock "2bf9fe41-63c2-4882-bcdd-8e314e734ffe-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:14:48 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-a68e9ef5-292f-4ada-89d9-dad236a443f5 req-81d9cd0e-81cf-41ad-99d3-f36b1727a44e service nova] Lock "2bf9fe41-63c2-4882-bcdd-8e314e734ffe-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:14:48 user nova-compute[75234]: DEBUG nova.compute.manager [req-a68e9ef5-292f-4ada-89d9-dad236a443f5 req-81d9cd0e-81cf-41ad-99d3-f36b1727a44e service nova] [instance: 2bf9fe41-63c2-4882-bcdd-8e314e734ffe] No waiting events found dispatching network-vif-unplugged-c8bb20fd-5c85-4e89-8c61-13a4f17dd4cc {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:14:48 user nova-compute[75234]: DEBUG nova.compute.manager [req-a68e9ef5-292f-4ada-89d9-dad236a443f5 req-81d9cd0e-81cf-41ad-99d3-f36b1727a44e service nova] [instance: 2bf9fe41-63c2-4882-bcdd-8e314e734ffe] Received event network-vif-unplugged-c8bb20fd-5c85-4e89-8c61-13a4f17dd4cc for instance with task_state deleting. {{(pid=75234) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10890}}
Nov 29 03:14:48 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:14:48 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:14:48 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:14:48 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:14:48 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:14:48 user nova-compute[75234]: DEBUG nova.compute.manager [req-6715308b-96fe-4037-ba6a-7229ba46a098 req-d9433cf1-ac4f-485e-ba1a-59cdfbe9e832 service nova] [instance: a9350a18-14e6-42c6-a150-0a1487f20ad9] Received event network-vif-unplugged-52024efc-8cde-4610-b12c-198f19d75ccf {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:14:48 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-6715308b-96fe-4037-ba6a-7229ba46a098 req-d9433cf1-ac4f-485e-ba1a-59cdfbe9e832 service nova] Acquiring lock "a9350a18-14e6-42c6-a150-0a1487f20ad9-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:14:48 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-6715308b-96fe-4037-ba6a-7229ba46a098 req-d9433cf1-ac4f-485e-ba1a-59cdfbe9e832 service nova] Lock "a9350a18-14e6-42c6-a150-0a1487f20ad9-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:14:48 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-6715308b-96fe-4037-ba6a-7229ba46a098 req-d9433cf1-ac4f-485e-ba1a-59cdfbe9e832 service nova] Lock "a9350a18-14e6-42c6-a150-0a1487f20ad9-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:14:48 user nova-compute[75234]: DEBUG nova.compute.manager [req-6715308b-96fe-4037-ba6a-7229ba46a098 req-d9433cf1-ac4f-485e-ba1a-59cdfbe9e832 service nova] [instance: a9350a18-14e6-42c6-a150-0a1487f20ad9] No waiting events found dispatching network-vif-unplugged-52024efc-8cde-4610-b12c-198f19d75ccf {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:14:48 user nova-compute[75234]: DEBUG nova.compute.manager [req-6715308b-96fe-4037-ba6a-7229ba46a098 req-d9433cf1-ac4f-485e-ba1a-59cdfbe9e832 service nova] [instance: a9350a18-14e6-42c6-a150-0a1487f20ad9] Received event network-vif-unplugged-52024efc-8cde-4610-b12c-198f19d75ccf for instance with task_state deleting. {{(pid=75234) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10890}}
Nov 29 03:14:49 user nova-compute[75234]: INFO nova.virt.libvirt.driver [-] [instance: 2bf9fe41-63c2-4882-bcdd-8e314e734ffe] Instance destroyed successfully.
Nov 29 03:14:49 user nova-compute[75234]: DEBUG nova.objects.instance [None req-91b74874-e949-461c-a452-0a0347214f04 tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] Lazy-loading 'resources' on Instance uuid 2bf9fe41-63c2-4882-bcdd-8e314e734ffe {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:14:49 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-91b74874-e949-461c-a452-0a0347214f04 tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-11-29T03:05:59Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-ServersNegativeTestJSON-server-618399672',display_name='tempest-ServersNegativeTestJSON-server-618399672',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-serversnegativetestjson-server-618399672',id=2,image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data=None,key_name=None,keypairs=<?>,launch_index=0,launched_at=2023-11-29T03:06:12Z,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='b877668e02eb441c85dc2a7003a94c7a',ramdisk_id='',reservation_id='r-poo1d0eh',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-ServersNegativeTestJSON-20765276',owner_user_name='tempest-ServersNegativeTestJSON-20765276-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-11-29T03:06:13Z,user_data=None,user_id='178bfe0ea37e45d8abd2d6b45bd5660c',uuid=2bf9fe41-63c2-4882-bcdd-8e314e734ffe,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "c8bb20fd-5c85-4e89-8c61-13a4f17dd4cc", "address": "fa:16:3e:2c:c9:1d", "network": {"id": "b27ac31e-3601-4055-be79-34b0946b0913", "bridge": "br-int", "label": "tempest-ServersNegativeTestJSON-1491620118-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "b877668e02eb441c85dc2a7003a94c7a", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc8bb20fd-5c", "ovs_interfaceid": "c8bb20fd-5c85-4e89-8c61-13a4f17dd4cc", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Nov 29 03:14:49 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-91b74874-e949-461c-a452-0a0347214f04 tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] Converting VIF {"id": "c8bb20fd-5c85-4e89-8c61-13a4f17dd4cc", "address": "fa:16:3e:2c:c9:1d", "network": {"id": "b27ac31e-3601-4055-be79-34b0946b0913", "bridge": "br-int", "label": "tempest-ServersNegativeTestJSON-1491620118-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "b877668e02eb441c85dc2a7003a94c7a", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapc8bb20fd-5c", "ovs_interfaceid": "c8bb20fd-5c85-4e89-8c61-13a4f17dd4cc", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:14:49 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-91b74874-e949-461c-a452-0a0347214f04 tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:2c:c9:1d,bridge_name='br-int',has_traffic_filtering=True,id=c8bb20fd-5c85-4e89-8c61-13a4f17dd4cc,network=Network(b27ac31e-3601-4055-be79-34b0946b0913),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapc8bb20fd-5c') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:14:49 user nova-compute[75234]: DEBUG os_vif [None req-91b74874-e949-461c-a452-0a0347214f04 tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] Unplugging vif VIFOpenVSwitch(active=True,address=fa:16:3e:2c:c9:1d,bridge_name='br-int',has_traffic_filtering=True,id=c8bb20fd-5c85-4e89-8c61-13a4f17dd4cc,network=Network(b27ac31e-3601-4055-be79-34b0946b0913),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapc8bb20fd-5c') {{(pid=75234) unplug /opt/stack/data/venv/lib/python3.10/site-packages/os_vif/__init__.py:109}}
Nov 29 03:14:49 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:14:49 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tapc8bb20fd-5c, bridge=br-int, if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:14:49 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:14:49 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:14:49 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:14:49 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:14:49 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbDestroyCommand(_result=None, table=QoS, record=1cf52a9a-1591-4592-b347-b5ae0527d249) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:14:49 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:14:49 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:14:49 user nova-compute[75234]: INFO nova.virt.libvirt.driver [-] [instance: a9350a18-14e6-42c6-a150-0a1487f20ad9] Instance destroyed successfully.
Nov 29 03:14:49 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:14:49 user nova-compute[75234]: DEBUG nova.objects.instance [None req-29b9361b-3bae-4aea-a02b-bbee9ce1102a tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Lazy-loading 'resources' on Instance uuid a9350a18-14e6-42c6-a150-0a1487f20ad9 {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:14:49 user nova-compute[75234]: INFO os_vif [None req-91b74874-e949-461c-a452-0a0347214f04 tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] Successfully unplugged vif VIFOpenVSwitch(active=True,address=fa:16:3e:2c:c9:1d,bridge_name='br-int',has_traffic_filtering=True,id=c8bb20fd-5c85-4e89-8c61-13a4f17dd4cc,network=Network(b27ac31e-3601-4055-be79-34b0946b0913),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapc8bb20fd-5c')
Nov 29 03:14:49 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-91b74874-e949-461c-a452-0a0347214f04 tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] [instance: 2bf9fe41-63c2-4882-bcdd-8e314e734ffe] Deleting instance files /opt/stack/data/nova/instances/2bf9fe41-63c2-4882-bcdd-8e314e734ffe_del
Nov 29 03:14:49 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-91b74874-e949-461c-a452-0a0347214f04 tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] [instance: 2bf9fe41-63c2-4882-bcdd-8e314e734ffe] Deletion of /opt/stack/data/nova/instances/2bf9fe41-63c2-4882-bcdd-8e314e734ffe_del complete
Nov 29 03:14:49 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-29b9361b-3bae-4aea-a02b-bbee9ce1102a tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-11-29T03:06:06Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-ServerActionsTestOtherB-server-372321920',display_name='tempest-ServerActionsTestOtherB-server-372321920',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-serveractionstestotherb-server-372321920',id=5,image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBFOggqgO1QPxMZPb2P6SX24mx6LOsfQfrzx4GSpNnJiixHqqOn61ObLwI9/TG/r6plC8w1huVCtu+OqAyk9QXUlEr0+VsPNunPSN9WPXBAM4+U+3iCOgHXExdr++34pXkw==',key_name='tempest-keypair-1692341699',keypairs=<?>,launch_index=0,launched_at=2023-11-29T03:12:22Z,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='ad5ea22c63f04f91a3f2fa665e4fc8a1',ramdisk_id='',reservation_id='r-f5lkid4e',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-ServerActionsTestOtherB-70688183',owner_user_name='tempest-ServerActionsTestOtherB-70688183-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-11-29T03:12:23Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='bdddba0ef9c44b6cb43c32a9ab0e29f6',uuid=a9350a18-14e6-42c6-a150-0a1487f20ad9,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "52024efc-8cde-4610-b12c-198f19d75ccf", "address": "fa:16:3e:09:b8:70", "network": {"id": "31c4ca3b-3a2f-47df-87fa-4ad095ee5cf9", "bridge": "br-int", "label": "tempest-ServerActionsTestOtherB-1680745008-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.9", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "ad5ea22c63f04f91a3f2fa665e4fc8a1", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap52024efc-8c", "ovs_interfaceid": "52024efc-8cde-4610-b12c-198f19d75ccf", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Nov 29 03:14:49 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-29b9361b-3bae-4aea-a02b-bbee9ce1102a tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Converting VIF {"id": "52024efc-8cde-4610-b12c-198f19d75ccf", "address": "fa:16:3e:09:b8:70", "network": {"id": "31c4ca3b-3a2f-47df-87fa-4ad095ee5cf9", "bridge": "br-int", "label": "tempest-ServerActionsTestOtherB-1680745008-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.9", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "ad5ea22c63f04f91a3f2fa665e4fc8a1", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap52024efc-8c", "ovs_interfaceid": "52024efc-8cde-4610-b12c-198f19d75ccf", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:14:49 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-29b9361b-3bae-4aea-a02b-bbee9ce1102a tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:09:b8:70,bridge_name='br-int',has_traffic_filtering=True,id=52024efc-8cde-4610-b12c-198f19d75ccf,network=Network(31c4ca3b-3a2f-47df-87fa-4ad095ee5cf9),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap52024efc-8c') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:14:49 user nova-compute[75234]: DEBUG os_vif [None req-29b9361b-3bae-4aea-a02b-bbee9ce1102a tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Unplugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:09:b8:70,bridge_name='br-int',has_traffic_filtering=True,id=52024efc-8cde-4610-b12c-198f19d75ccf,network=Network(31c4ca3b-3a2f-47df-87fa-4ad095ee5cf9),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap52024efc-8c') {{(pid=75234) unplug /opt/stack/data/venv/lib/python3.10/site-packages/os_vif/__init__.py:109}}
Nov 29 03:14:49 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:14:49 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tap52024efc-8c, bridge=br-int, if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:14:49 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:14:49 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:14:49 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:14:49 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbDestroyCommand(_result=None, table=QoS, record=fb7af2a8-4e89-4c21-be47-28136cc271f1) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:14:49 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:14:49 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:14:49 user nova-compute[75234]: INFO os_vif [None req-29b9361b-3bae-4aea-a02b-bbee9ce1102a tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Successfully unplugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:09:b8:70,bridge_name='br-int',has_traffic_filtering=True,id=52024efc-8cde-4610-b12c-198f19d75ccf,network=Network(31c4ca3b-3a2f-47df-87fa-4ad095ee5cf9),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap52024efc-8c')
Nov 29 03:14:49 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-29b9361b-3bae-4aea-a02b-bbee9ce1102a tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] [instance: a9350a18-14e6-42c6-a150-0a1487f20ad9] calling os-brick to detach iSCSI Volume {{(pid=75234) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Nov 29 03:14:49 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-29b9361b-3bae-4aea-a02b-bbee9ce1102a tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f8a5050c9a0>, {'target_iqn': 'iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'NFCsGdbh', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdc'}, None), 'kwargs': True}}" {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:176}}
Nov 29 03:14:49 user nova-compute[75234]: WARNING os_brick.initiator.connectors.base [None req-29b9361b-3bae-4aea-a02b-bbee9ce1102a tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Nov 29 03:14:49 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-29b9361b-3bae-4aea-a02b-bbee9ce1102a tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:68}}
Nov 29 03:14:49 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-29b9361b-3bae-4aea-a02b-bbee9ce1102a tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.002s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:73}}
Nov 29 03:14:49 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:14:49 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.022s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:14:49 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[c466a787-a0ad-4feb-9f84-a197976ce404]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:56237f4c-4138-4f50-aafb-9c0f9d4bb1cf\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:6a4d34d3-9974-43bc-afcb-9169c893d9b7\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:14:49 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-29b9361b-3bae-4aea-a02b-bbee9ce1102a tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS:
Nov 29 03:14:49 user nova-compute[75234]: No targets found.
Nov 29 03:14:49 user nova-compute[75234]: iSNS:
Nov 29 03:14:49 user nova-compute[75234]: No targets found.
Nov 29 03:14:49 user nova-compute[75234]: STATIC:
Nov 29 03:14:49 user nova-compute[75234]: Target: iqn.2010-10.org.openstack:56237f4c-4138-4f50-aafb-9c0f9d4bb1cf
Nov 29 03:14:49 user nova-compute[75234]:         Portal: 172.16.0.220:3260,-1
Nov 29 03:14:49 user nova-compute[75234]:                 Iface Name: default
Nov 29 03:14:49 user nova-compute[75234]: Target: iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b
Nov 29 03:14:49 user nova-compute[75234]:         Portal: 172.16.0.220:3260,-1
Nov 29 03:14:49 user nova-compute[75234]:                 Iface Name: default
Nov 29 03:14:49 user nova-compute[75234]: Target: iqn.2010-10.org.openstack:6a4d34d3-9974-43bc-afcb-9169c893d9b7
Nov 29 03:14:49 user nova-compute[75234]:         Portal: 172.16.0.220:3260,-1
Nov 29 03:14:49 user nova-compute[75234]:                 Iface Name: default
Nov 29 03:14:49 user nova-compute[75234]: FIRMWARE:
Nov 29 03:14:49 user nova-compute[75234]: No targets found.
Nov 29 03:14:49 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:14:49 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-29b9361b-3bae-4aea-a02b-bbee9ce1102a tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Regex to get portals from discoverydb: ^SENDTARGETS:
Nov 29 03:14:49 user nova-compute[75234]: .*?^DiscoveryAddress: 172.16.0.220,3260.*?
Nov 29 03:14:49 user nova-compute[75234]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=75234) _get_discoverydb_portals /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:382}}
Nov 29 03:14:49 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-29b9361b-3bae-4aea-a02b-bbee9ce1102a tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b', 0)] {{(pid=75234) _get_connection_devices /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:830}}
Nov 29 03:14:49 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:14:49 user nova-compute[75234]: INFO nova.compute.manager [None req-91b74874-e949-461c-a452-0a0347214f04 tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] [instance: 2bf9fe41-63c2-4882-bcdd-8e314e734ffe] Took 1.34 seconds to destroy the instance on the hypervisor.
Nov 29 03:14:49 user nova-compute[75234]: DEBUG oslo.service.loopingcall [None req-91b74874-e949-461c-a452-0a0347214f04 tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=75234) func /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/loopingcall.py:435}}
Nov 29 03:14:49 user nova-compute[75234]: DEBUG nova.compute.manager [-] [instance: 2bf9fe41-63c2-4882-bcdd-8e314e734ffe] Deallocating network for instance {{(pid=75234) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2279}}
Nov 29 03:14:49 user nova-compute[75234]: DEBUG nova.network.neutron [-] [instance: 2bf9fe41-63c2-4882-bcdd-8e314e734ffe] deallocate_for_instance() {{(pid=75234) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1802}}
Nov 29 03:14:49 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.020s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:14:49 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[b8743be6-bcbe-4ea9-aeb7-2172bb544aaf]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:56237f4c-4138-4f50-aafb-9c0f9d4bb1cf\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:6a4d34d3-9974-43bc-afcb-9169c893d9b7\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:14:49 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:14:49 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.023s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:14:49 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[42904b3d-e85f-41e8-9a22-e952e7d2a5e4]: (4, ('tcp: [20] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b (non-flash)\ntcp: [21] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:56237f4c-4138-4f50-aafb-9c0f9d4bb1cf (non-flash)\ntcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6a4d34d3-9974-43bc-afcb-9169c893d9b7 (non-flash)\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:14:49 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-29b9361b-3bae-4aea-a02b-bbee9ce1102a tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [20] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b (non-flash)
Nov 29 03:14:49 user nova-compute[75234]: tcp: [21] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:56237f4c-4138-4f50-aafb-9c0f9d4bb1cf (non-flash)
Nov 29 03:14:49 user nova-compute[75234]: tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6a4d34d3-9974-43bc-afcb-9169c893d9b7 (non-flash)
Nov 29 03:14:49 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:14:49 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-29b9361b-3bae-4aea-a02b-bbee9ce1102a tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] iscsi session list stdout=tcp: [20] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b (non-flash)
Nov 29 03:14:49 user nova-compute[75234]: tcp: [21] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:56237f4c-4138-4f50-aafb-9c0f9d4bb1cf (non-flash)
Nov 29 03:14:49 user nova-compute[75234]: tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6a4d34d3-9974-43bc-afcb-9169c893d9b7 (non-flash)
Nov 29 03:14:49 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsi_session /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Nov 29 03:14:49 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-29b9361b-3bae-4aea-a02b-bbee9ce1102a tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7f8a6025b0a0>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b'): ({'sdc'}, set())}) {{(pid=75234) _get_connection_devices /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:863}}
Nov 29 03:14:49 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-29b9361b-3bae-4aea-a02b-bbee9ce1102a tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Removing single pathed devices sdc {{(pid=75234) remove_connection /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:374}}
Nov 29 03:14:49 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:14:49 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.021s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:14:49 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[b40c6cd9-3b04-4ee2-a92d-cc3a887903b9]: (4, ('path checker states:\nup                  3\n\npaths: 0\nbusy: False\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:14:49 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdc {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:14:49 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdc" returned: 0 in 0.021s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:14:49 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[c58d65f1-7e38-4fde-86eb-354cf3daa399]: (4, ('ok\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:14:49 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-29b9361b-3bae-4aea-a02b-bbee9ce1102a tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Flushing IO for device /dev/sdc {{(pid=75234) flush_device_io /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:434}}
Nov 29 03:14:49 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdc {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:14:49 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdc" returned: 0 in 0.021s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:14:49 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[cea7a162-6d82-425e-b7fe-9a287141d01a]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:14:49 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-29b9361b-3bae-4aea-a02b-bbee9ce1102a tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Remove SCSI device /dev/sdc with /sys/block/sdc/device/delete {{(pid=75234) remove_scsi_device /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:148}}
Nov 29 03:14:49 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdc/device/delete {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:14:49 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdc/device/delete" returned: 0 in 0.052s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:14:49 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[ec93f3dd-3a5a-4f74-acee-6d4b2b228b14]: (4, ('1', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:14:49 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-29b9361b-3bae-4aea-a02b-bbee9ce1102a tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Checking to see if SCSI volumes sdc have been removed. {{(pid=75234) wait_for_volumes_removal /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:156}}
Nov 29 03:14:49 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-29b9361b-3bae-4aea-a02b-bbee9ce1102a tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] SCSI volumes sdc have been removed. {{(pid=75234) wait_for_volumes_removal /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:166}}
Nov 29 03:14:49 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-29b9361b-3bae-4aea-a02b-bbee9ce1102a tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b')] {{(pid=75234) _disconnect_connection /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Nov 29 03:14:49 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:14:49 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.019s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:14:49 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[98bdbd3d-577e-433d-aa81-3e7fba07f62f]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:14:49 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-29b9361b-3bae-4aea-a02b-bbee9ce1102a tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:14:49 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b -p 172.16.0.220:3260 --logout {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:14:49 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b -p 172.16.0.220:3260 --logout" returned: 0 in 0.052s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:14:49 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[c45946ce-7bdf-49eb-b86b-df611403257f]: (4, ('Logging out of session [sid: 20, target: iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b, portal: 172.16.0.220,3260]\nLogout of [sid: 20, target: iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:14:49 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-29b9361b-3bae-4aea-a02b-bbee9ce1102a tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 20, target: iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b, portal: 172.16.0.220,3260]
Nov 29 03:14:49 user nova-compute[75234]: Logout of [sid: 20, target: iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b, portal: 172.16.0.220,3260] successful.
Nov 29 03:14:49 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:14:49 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b -p 172.16.0.220:3260 --op delete {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:14:49 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:f4130d3c-b329-4ce1-8b6d-d9e270a43b0b -p 172.16.0.220:3260 --op delete" returned: 0 in 0.022s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:14:49 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[3866c90d-608f-4b15-a981-fa36504fbea2]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:14:49 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-29b9361b-3bae-4aea-a02b-bbee9ce1102a tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:14:49 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-29b9361b-3bae-4aea-a02b-bbee9ce1102a tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.328s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:87}}
Nov 29 03:14:49 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-29b9361b-3bae-4aea-a02b-bbee9ce1102a tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] <== disconnect_volume: return (330ms) None {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:203}}
Nov 29 03:14:49 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-29b9361b-3bae-4aea-a02b-bbee9ce1102a tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] [instance: a9350a18-14e6-42c6-a150-0a1487f20ad9] Disconnected iSCSI Volume {{(pid=75234) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:79}}
Nov 29 03:14:49 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-29b9361b-3bae-4aea-a02b-bbee9ce1102a tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] [instance: a9350a18-14e6-42c6-a150-0a1487f20ad9] Deleting instance files /opt/stack/data/nova/instances/a9350a18-14e6-42c6-a150-0a1487f20ad9_del
Nov 29 03:14:49 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-29b9361b-3bae-4aea-a02b-bbee9ce1102a tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] [instance: a9350a18-14e6-42c6-a150-0a1487f20ad9] Deletion of /opt/stack/data/nova/instances/a9350a18-14e6-42c6-a150-0a1487f20ad9_del complete
Nov 29 03:14:49 user nova-compute[75234]: INFO nova.compute.manager [None req-29b9361b-3bae-4aea-a02b-bbee9ce1102a tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] [instance: a9350a18-14e6-42c6-a150-0a1487f20ad9] Took 1.36 seconds to destroy the instance on the hypervisor.
Nov 29 03:14:49 user nova-compute[75234]: DEBUG oslo.service.loopingcall [None req-29b9361b-3bae-4aea-a02b-bbee9ce1102a tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=75234) func /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/loopingcall.py:435}}
Nov 29 03:14:49 user nova-compute[75234]: DEBUG nova.compute.manager [-] [instance: a9350a18-14e6-42c6-a150-0a1487f20ad9] Deallocating network for instance {{(pid=75234) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2279}}
Nov 29 03:14:49 user nova-compute[75234]: DEBUG nova.network.neutron [-] [instance: a9350a18-14e6-42c6-a150-0a1487f20ad9] deallocate_for_instance() {{(pid=75234) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1802}}
Nov 29 03:14:49 user nova-compute[75234]: DEBUG nova.network.neutron [-] [instance: 2bf9fe41-63c2-4882-bcdd-8e314e734ffe] Updating instance_info_cache with network_info: [] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:14:49 user nova-compute[75234]: INFO nova.compute.manager [-] [instance: 2bf9fe41-63c2-4882-bcdd-8e314e734ffe] Took 0.45 seconds to deallocate network for instance.
Nov 29 03:14:49 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-91b74874-e949-461c-a452-0a0347214f04 tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:14:49 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-91b74874-e949-461c-a452-0a0347214f04 tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:14:50 user nova-compute[75234]: DEBUG nova.compute.provider_tree [None req-91b74874-e949-461c-a452-0a0347214f04 tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] Inventory has not changed in ProviderTree for provider: 4d291220-7c37-4839-a0e3-b23c66502b74 {{(pid=75234) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Nov 29 03:14:50 user nova-compute[75234]: DEBUG nova.scheduler.client.report [None req-91b74874-e949-461c-a452-0a0347214f04 tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] Inventory has not changed for provider 4d291220-7c37-4839-a0e3-b23c66502b74 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=75234) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Nov 29 03:14:50 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-91b74874-e949-461c-a452-0a0347214f04 tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.422s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:14:50 user nova-compute[75234]: INFO nova.scheduler.client.report [None req-91b74874-e949-461c-a452-0a0347214f04 tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] Deleted allocations for instance 2bf9fe41-63c2-4882-bcdd-8e314e734ffe
Nov 29 03:14:50 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-91b74874-e949-461c-a452-0a0347214f04 tempest-ServersNegativeTestJSON-20765276 tempest-ServersNegativeTestJSON-20765276-project-member] Lock "2bf9fe41-63c2-4882-bcdd-8e314e734ffe" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 2.413s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:14:50 user nova-compute[75234]: DEBUG nova.compute.manager [req-debb9c3f-3788-4fb0-b63d-af74ab866541 req-76a99c41-c0a0-4255-815c-b3e8e8904507 service nova] [instance: 2bf9fe41-63c2-4882-bcdd-8e314e734ffe] Received event network-vif-plugged-c8bb20fd-5c85-4e89-8c61-13a4f17dd4cc {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:14:50 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-debb9c3f-3788-4fb0-b63d-af74ab866541 req-76a99c41-c0a0-4255-815c-b3e8e8904507 service nova] Acquiring lock "2bf9fe41-63c2-4882-bcdd-8e314e734ffe-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:14:50 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-debb9c3f-3788-4fb0-b63d-af74ab866541 req-76a99c41-c0a0-4255-815c-b3e8e8904507 service nova] Lock "2bf9fe41-63c2-4882-bcdd-8e314e734ffe-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:14:50 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-debb9c3f-3788-4fb0-b63d-af74ab866541 req-76a99c41-c0a0-4255-815c-b3e8e8904507 service nova] Lock "2bf9fe41-63c2-4882-bcdd-8e314e734ffe-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:14:50 user nova-compute[75234]: DEBUG nova.compute.manager [req-debb9c3f-3788-4fb0-b63d-af74ab866541 req-76a99c41-c0a0-4255-815c-b3e8e8904507 service nova] [instance: 2bf9fe41-63c2-4882-bcdd-8e314e734ffe] No waiting events found dispatching network-vif-plugged-c8bb20fd-5c85-4e89-8c61-13a4f17dd4cc {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:14:50 user nova-compute[75234]: WARNING nova.compute.manager [req-debb9c3f-3788-4fb0-b63d-af74ab866541 req-76a99c41-c0a0-4255-815c-b3e8e8904507 service nova] [instance: 2bf9fe41-63c2-4882-bcdd-8e314e734ffe] Received unexpected event network-vif-plugged-c8bb20fd-5c85-4e89-8c61-13a4f17dd4cc for instance with vm_state deleted and task_state None.
Nov 29 03:14:50 user nova-compute[75234]: DEBUG nova.compute.manager [req-debb9c3f-3788-4fb0-b63d-af74ab866541 req-76a99c41-c0a0-4255-815c-b3e8e8904507 service nova] [instance: 2bf9fe41-63c2-4882-bcdd-8e314e734ffe] Received event network-vif-deleted-c8bb20fd-5c85-4e89-8c61-13a4f17dd4cc {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:14:50 user nova-compute[75234]: DEBUG nova.network.neutron [-] [instance: a9350a18-14e6-42c6-a150-0a1487f20ad9] Updating instance_info_cache with network_info: [] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:14:50 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-28e7bf3b-25fb-424d-87b1-89eec41681db tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Acquiring lock "062753a6-8f39-4e80-a549-9f0f5a4b877a" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:14:50 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-28e7bf3b-25fb-424d-87b1-89eec41681db tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Lock "062753a6-8f39-4e80-a549-9f0f5a4b877a" acquired by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:14:50 user nova-compute[75234]: INFO nova.compute.manager [-] [instance: a9350a18-14e6-42c6-a150-0a1487f20ad9] Took 0.80 seconds to deallocate network for instance.
Nov 29 03:14:50 user nova-compute[75234]: INFO nova.compute.manager [None req-28e7bf3b-25fb-424d-87b1-89eec41681db tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] [instance: 062753a6-8f39-4e80-a549-9f0f5a4b877a] Detaching volume 6a4d34d3-9974-43bc-afcb-9169c893d9b7
Nov 29 03:14:50 user nova-compute[75234]: DEBUG nova.compute.manager [req-5c92fa51-379e-46ed-b795-1c8f5d44c8f3 req-d70edd72-83a2-4f53-b28e-64250b8f62af service nova] [instance: a9350a18-14e6-42c6-a150-0a1487f20ad9] Received event network-vif-plugged-52024efc-8cde-4610-b12c-198f19d75ccf {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:14:50 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-5c92fa51-379e-46ed-b795-1c8f5d44c8f3 req-d70edd72-83a2-4f53-b28e-64250b8f62af service nova] Acquiring lock "a9350a18-14e6-42c6-a150-0a1487f20ad9-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:14:50 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-5c92fa51-379e-46ed-b795-1c8f5d44c8f3 req-d70edd72-83a2-4f53-b28e-64250b8f62af service nova] Lock "a9350a18-14e6-42c6-a150-0a1487f20ad9-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:14:50 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-5c92fa51-379e-46ed-b795-1c8f5d44c8f3 req-d70edd72-83a2-4f53-b28e-64250b8f62af service nova] Lock "a9350a18-14e6-42c6-a150-0a1487f20ad9-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:14:50 user nova-compute[75234]: DEBUG nova.compute.manager [req-5c92fa51-379e-46ed-b795-1c8f5d44c8f3 req-d70edd72-83a2-4f53-b28e-64250b8f62af service nova] [instance: a9350a18-14e6-42c6-a150-0a1487f20ad9] No waiting events found dispatching network-vif-plugged-52024efc-8cde-4610-b12c-198f19d75ccf {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:14:50 user nova-compute[75234]: WARNING nova.compute.manager [req-5c92fa51-379e-46ed-b795-1c8f5d44c8f3 req-d70edd72-83a2-4f53-b28e-64250b8f62af service nova] [instance: a9350a18-14e6-42c6-a150-0a1487f20ad9] Received unexpected event network-vif-plugged-52024efc-8cde-4610-b12c-198f19d75ccf for instance with vm_state active and task_state deleting.
Nov 29 03:14:50 user nova-compute[75234]: DEBUG nova.compute.manager [req-5c92fa51-379e-46ed-b795-1c8f5d44c8f3 req-d70edd72-83a2-4f53-b28e-64250b8f62af service nova] [instance: a9350a18-14e6-42c6-a150-0a1487f20ad9] Received event network-vif-deleted-52024efc-8cde-4610-b12c-198f19d75ccf {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:14:50 user nova-compute[75234]: INFO nova.virt.block_device [None req-28e7bf3b-25fb-424d-87b1-89eec41681db tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] [instance: 062753a6-8f39-4e80-a549-9f0f5a4b877a] Attempting to driver detach volume 6a4d34d3-9974-43bc-afcb-9169c893d9b7 from mountpoint /dev/vdc
Nov 29 03:14:50 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-28e7bf3b-25fb-424d-87b1-89eec41681db tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Attempting to detach device vdc from instance 062753a6-8f39-4e80-a549-9f0f5a4b877a from the persistent domain config. {{(pid=75234) _detach_from_persistent /opt/stack/nova/nova/virt/libvirt/driver.py:2487}}
Nov 29 03:14:50 user nova-compute[75234]: DEBUG nova.virt.libvirt.guest [None req-28e7bf3b-25fb-424d-87b1-89eec41681db tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] detach device xml: <disk type="block" device="disk">
Nov 29 03:14:50 user nova-compute[75234]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Nov 29 03:14:50 user nova-compute[75234]:   <source dev="/dev/sdd"/>
Nov 29 03:14:50 user nova-compute[75234]:   <target dev="vdc" bus="virtio"/>
Nov 29 03:14:50 user nova-compute[75234]:   <serial>6a4d34d3-9974-43bc-afcb-9169c893d9b7</serial>
Nov 29 03:14:50 user nova-compute[75234]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x05" function="0x0"/>
Nov 29 03:14:50 user nova-compute[75234]: </disk>
Nov 29 03:14:50 user nova-compute[75234]:  {{(pid=75234) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Nov 29 03:14:50 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-28e7bf3b-25fb-424d-87b1-89eec41681db tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Successfully detached device vdc from instance 062753a6-8f39-4e80-a549-9f0f5a4b877a from the persistent domain config.
Nov 29 03:14:50 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-28e7bf3b-25fb-424d-87b1-89eec41681db tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] (1/8): Attempting to detach device vdc with device alias virtio-disk2 from instance 062753a6-8f39-4e80-a549-9f0f5a4b877a from the live domain config. {{(pid=75234) _detach_from_live_with_retry /opt/stack/nova/nova/virt/libvirt/driver.py:2523}}
Nov 29 03:14:50 user nova-compute[75234]: DEBUG nova.virt.libvirt.guest [None req-28e7bf3b-25fb-424d-87b1-89eec41681db tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] detach device xml: <disk type="block" device="disk">
Nov 29 03:14:50 user nova-compute[75234]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Nov 29 03:14:50 user nova-compute[75234]:   <source dev="/dev/sdd"/>
Nov 29 03:14:50 user nova-compute[75234]:   <target dev="vdc" bus="virtio"/>
Nov 29 03:14:50 user nova-compute[75234]:   <serial>6a4d34d3-9974-43bc-afcb-9169c893d9b7</serial>
Nov 29 03:14:50 user nova-compute[75234]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x05" function="0x0"/>
Nov 29 03:14:50 user nova-compute[75234]: </disk>
Nov 29 03:14:50 user nova-compute[75234]:  {{(pid=75234) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Nov 29 03:14:51 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-28e7bf3b-25fb-424d-87b1-89eec41681db tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Start waiting for the detach event from libvirt for device vdc with device alias virtio-disk2 for instance 062753a6-8f39-4e80-a549-9f0f5a4b877a {{(pid=75234) _detach_from_live_and_wait_for_event /opt/stack/nova/nova/virt/libvirt/driver.py:2599}}
Nov 29 03:14:51 user nova-compute[75234]: INFO nova.compute.manager [None req-29b9361b-3bae-4aea-a02b-bbee9ce1102a tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] [instance: a9350a18-14e6-42c6-a150-0a1487f20ad9] Took 1.02 seconds to detach 1 volumes for instance.
Nov 29 03:14:51 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-29b9361b-3bae-4aea-a02b-bbee9ce1102a tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:14:51 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-29b9361b-3bae-4aea-a02b-bbee9ce1102a tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:14:51 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:14:51 user nova-compute[75234]: DEBUG nova.compute.provider_tree [None req-29b9361b-3bae-4aea-a02b-bbee9ce1102a tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Inventory has not changed in ProviderTree for provider: 4d291220-7c37-4839-a0e3-b23c66502b74 {{(pid=75234) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Nov 29 03:14:51 user nova-compute[75234]: DEBUG nova.scheduler.client.report [None req-29b9361b-3bae-4aea-a02b-bbee9ce1102a tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Inventory has not changed for provider 4d291220-7c37-4839-a0e3-b23c66502b74 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=75234) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Nov 29 03:14:51 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-29b9361b-3bae-4aea-a02b-bbee9ce1102a tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.341s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:14:51 user nova-compute[75234]: INFO nova.scheduler.client.report [None req-29b9361b-3bae-4aea-a02b-bbee9ce1102a tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Deleted allocations for instance a9350a18-14e6-42c6-a150-0a1487f20ad9
Nov 29 03:14:52 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-29b9361b-3bae-4aea-a02b-bbee9ce1102a tempest-ServerActionsTestOtherB-70688183 tempest-ServerActionsTestOtherB-70688183-project-member] Lock "a9350a18-14e6-42c6-a150-0a1487f20ad9" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 3.718s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:14:54 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:14:54 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-5e94a590-65a3-40ab-b081-3bc7d55c8204 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] [instance: 6894b55f-93c6-43c9-8933-56d5a3c43725] Calling os-brick to attach iSCSI Volume {{(pid=75234) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Nov 29 03:14:54 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-5e94a590-65a3-40ab-b081-3bc7d55c8204 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f8a5050c9a0>, 'connection_properties': {'target_iqn': 'iqn.2010-10.org.openstack:104bcac8-98ce-42f8-a85f-82e29050a44c', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'oxmZWJRF', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:176}}
Nov 29 03:14:54 user nova-compute[75234]: WARNING os_brick.initiator.connectors.base [None req-5e94a590-65a3-40ab-b081-3bc7d55c8204 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Nov 29 03:14:54 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-5e94a590-65a3-40ab-b081-3bc7d55c8204 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:68}}
Nov 29 03:14:54 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-5e94a590-65a3-40ab-b081-3bc7d55c8204 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:73}}
Nov 29 03:14:54 user nova-compute[75234]: WARNING os_brick.initiator.connectors.base [None req-5e94a590-65a3-40ab-b081-3bc7d55c8204 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Nov 29 03:14:54 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-5e94a590-65a3-40ab-b081-3bc7d55c8204 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:104bcac8-98ce-42f8-a85f-82e29050a44c" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:68}}
Nov 29 03:14:54 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-5e94a590-65a3-40ab-b081-3bc7d55c8204 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:104bcac8-98ce-42f8-a85f-82e29050a44c" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:73}}
Nov 29 03:14:54 user nova-compute[75234]: INFO os_brick.initiator.connectors.iscsi [None req-5e94a590-65a3-40ab-b081-3bc7d55c8204 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Nov 29 03:14:54 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:104bcac8-98ce-42f8-a85f-82e29050a44c -p 172.16.0.220:3260 {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:14:54 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:104bcac8-98ce-42f8-a85f-82e29050a44c -p 172.16.0.220:3260" returned: 21 in 0.026s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:14:54 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[a88c1dec-4b7a-499c-a879-07bb08a9a996]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:14:54 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-5e94a590-65a3-40ab-b081-3bc7d55c8204 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Nov 29 03:14:54 user nova-compute[75234]:  {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:14:54 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:104bcac8-98ce-42f8-a85f-82e29050a44c -p 172.16.0.220:3260 --interface default --op new {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:14:54 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:104bcac8-98ce-42f8-a85f-82e29050a44c -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.024s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:14:54 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[a1e99491-fd45-4e8a-8ac6-119d0c74c5bb]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:104bcac8-98ce-42f8-a85f-82e29050a44c] added\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:14:54 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-5e94a590-65a3-40ab-b081-3bc7d55c8204 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:104bcac8-98ce-42f8-a85f-82e29050a44c] added
Nov 29 03:14:54 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:14:54 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:104bcac8-98ce-42f8-a85f-82e29050a44c -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:14:54 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:104bcac8-98ce-42f8-a85f-82e29050a44c -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.030s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:14:54 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[b1bd1ae6-3021-4549-83ef-ef4ecc24f7a9]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:14:54 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-5e94a590-65a3-40ab-b081-3bc7d55c8204 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:14:54 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:104bcac8-98ce-42f8-a85f-82e29050a44c -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:14:54 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:14:54 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:104bcac8-98ce-42f8-a85f-82e29050a44c -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.018s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:14:54 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[badb56dc-e45d-4fee-a38c-b4971d12ee72]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:14:54 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-5e94a590-65a3-40ab-b081-3bc7d55c8204 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:14:54 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:104bcac8-98ce-42f8-a85f-82e29050a44c -p 172.16.0.220:3260 --op update -n node.session.auth.username -v oxmZWJRF {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:14:54 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:104bcac8-98ce-42f8-a85f-82e29050a44c -p 172.16.0.220:3260 --op update -n node.session.auth.username -v oxmZWJRF" returned: 0 in 0.020s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:14:54 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[4f1cb953-a01b-4084-90aa-c25ce80ec1cc]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:14:54 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-5e94a590-65a3-40ab-b081-3bc7d55c8204 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'oxmZWJRF'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:14:54 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:104bcac8-98ce-42f8-a85f-82e29050a44c -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:14:54 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:14:54 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Starting heal instance info cache {{(pid=75234) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9922}}
Nov 29 03:14:54 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:104bcac8-98ce-42f8-a85f-82e29050a44c -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.021s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:14:54 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[7497639b-8dba-468f-a5a4-1d00bc5d4180]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:14:54 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-5e94a590-65a3-40ab-b081-3bc7d55c8204 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:14:54 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:14:54 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.022s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:14:54 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[b05b407d-eb8c-4aa9-9d64-b13d4227fba7]: (4, ('tcp: [21] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:56237f4c-4138-4f50-aafb-9c0f9d4bb1cf (non-flash)\ntcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6a4d34d3-9974-43bc-afcb-9169c893d9b7 (non-flash)\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:14:54 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-5e94a590-65a3-40ab-b081-3bc7d55c8204 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [21] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:56237f4c-4138-4f50-aafb-9c0f9d4bb1cf (non-flash)
Nov 29 03:14:54 user nova-compute[75234]: tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6a4d34d3-9974-43bc-afcb-9169c893d9b7 (non-flash)
Nov 29 03:14:54 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:14:54 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-5e94a590-65a3-40ab-b081-3bc7d55c8204 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] iscsi session list stdout=tcp: [21] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:56237f4c-4138-4f50-aafb-9c0f9d4bb1cf (non-flash)
Nov 29 03:14:54 user nova-compute[75234]: tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6a4d34d3-9974-43bc-afcb-9169c893d9b7 (non-flash)
Nov 29 03:14:54 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsi_session /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Nov 29 03:14:54 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:104bcac8-98ce-42f8-a85f-82e29050a44c -p 172.16.0.220:3260 --login {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:14:54 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:104bcac8-98ce-42f8-a85f-82e29050a44c -p 172.16.0.220:3260 --login" returned: 0 in 0.064s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:14:54 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[cebfc120-a376-4d9c-9bca-2e0dd6f31d24]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:104bcac8-98ce-42f8-a85f-82e29050a44c, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:104bcac8-98ce-42f8-a85f-82e29050a44c, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:14:54 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-5e94a590-65a3-40ab-b081-3bc7d55c8204 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:104bcac8-98ce-42f8-a85f-82e29050a44c, portal: 172.16.0.220,3260]
Nov 29 03:14:54 user nova-compute[75234]: Login to [iface: default, target: iqn.2010-10.org.openstack:104bcac8-98ce-42f8-a85f-82e29050a44c, portal: 172.16.0.220,3260] successful.
Nov 29 03:14:54 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:14:54 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:104bcac8-98ce-42f8-a85f-82e29050a44c -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:14:54 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:104bcac8-98ce-42f8-a85f-82e29050a44c -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.019s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:14:54 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[29213615-287b-49b0-9f9a-6d6f9cda5385]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:14:54 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-5e94a590-65a3-40ab-b081-3bc7d55c8204 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:14:54 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:14:54 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Acquiring lock "refresh_cache-21e9c6cd-c39f-41c0-8768-5dd64589c3f3" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:14:54 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Acquired lock "refresh_cache-21e9c6cd-c39f-41c0-8768-5dd64589c3f3" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:14:54 user nova-compute[75234]: DEBUG nova.network.neutron [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] [instance: 21e9c6cd-c39f-41c0-8768-5dd64589c3f3] Forcefully refreshing network info cache for instance {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2003}}
Nov 29 03:14:54 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.021s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:14:54 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[144f296a-46c7-41c2-9fa5-e7117b40f869]: (4, ('tcp: [21] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:56237f4c-4138-4f50-aafb-9c0f9d4bb1cf (non-flash)\ntcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6a4d34d3-9974-43bc-afcb-9169c893d9b7 (non-flash)\ntcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:104bcac8-98ce-42f8-a85f-82e29050a44c (non-flash)\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:14:54 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-5e94a590-65a3-40ab-b081-3bc7d55c8204 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [21] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:56237f4c-4138-4f50-aafb-9c0f9d4bb1cf (non-flash)
Nov 29 03:14:54 user nova-compute[75234]: tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6a4d34d3-9974-43bc-afcb-9169c893d9b7 (non-flash)
Nov 29 03:14:54 user nova-compute[75234]: tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:104bcac8-98ce-42f8-a85f-82e29050a44c (non-flash)
Nov 29 03:14:54 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:14:54 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-5e94a590-65a3-40ab-b081-3bc7d55c8204 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] iscsi session list stdout=tcp: [21] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:56237f4c-4138-4f50-aafb-9c0f9d4bb1cf (non-flash)
Nov 29 03:14:54 user nova-compute[75234]: tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6a4d34d3-9974-43bc-afcb-9169c893d9b7 (non-flash)
Nov 29 03:14:54 user nova-compute[75234]: tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:104bcac8-98ce-42f8-a85f-82e29050a44c (non-flash)
Nov 29 03:14:54 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsi_session /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Nov 29 03:14:54 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-5e94a590-65a3-40ab-b081-3bc7d55c8204 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:104bcac8-98ce-42f8-a85f-82e29050a44c" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.308s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:87}}
Nov 29 03:14:54 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-5e94a590-65a3-40ab-b081-3bc7d55c8204 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Connected to 172.16.0.220:3260 {{(pid=75234) _connect_vol /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:644}}
Nov 29 03:14:54 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-5e94a590-65a3-40ab-b081-3bc7d55c8204 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] HCTL ('33', '-', '-', 0) found on session 26 with lun 0 {{(pid=75234) get_hctl /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:770}}
Nov 29 03:14:54 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-5e94a590-65a3-40ab-b081-3bc7d55c8204 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Scanning host 33 c: -, t: -, l: 0) {{(pid=75234) scan_iscsi /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:795}}
Nov 29 03:14:54 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host33/scan {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:14:54 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host33/scan" returned: 0 in 0.015s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:14:54 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[89550784-fea9-4fbb-b904-f514b5a7f3c2]: (4, ('- - 0', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:14:54 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-5e94a590-65a3-40ab-b081-3bc7d55c8204 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Searching for a device in session 26 and hctl ['33', '*', '*', 0] yield: None {{(pid=75234) device_name_by_hctl /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:789}}
Nov 29 03:14:55 user nova-compute[75234]: DEBUG nova.network.neutron [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] [instance: 21e9c6cd-c39f-41c0-8768-5dd64589c3f3] Updating instance_info_cache with network_info: [{"id": "87f1e736-6665-4945-bd19-0cf93bff905e", "address": "fa:16:3e:40:14:df", "network": {"id": "1f946e69-dbec-429c-83f8-a186bad4f2cc", "bridge": "br-int", "label": "tempest-ServerActionsTestOtherA-486369423-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.64", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "c1cb3d1f12f242a785a688d59dbe3cd6", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap87f1e736-66", "ovs_interfaceid": "87f1e736-6665-4945-bd19-0cf93bff905e", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:14:55 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Releasing lock "refresh_cache-21e9c6cd-c39f-41c0-8768-5dd64589c3f3" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:14:55 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] [instance: 21e9c6cd-c39f-41c0-8768-5dd64589c3f3] Updated the network info_cache for instance {{(pid=75234) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9993}}
Nov 29 03:14:55 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-5e94a590-65a3-40ab-b081-3bc7d55c8204 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Searching for a device in session 26 and hctl ['33', '*', '*', 0] yield: sdb {{(pid=75234) device_name_by_hctl /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:789}}
Nov 29 03:14:55 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-5e94a590-65a3-40ab-b081-3bc7d55c8204 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Connected to sdb using {'target_iqn': 'iqn.2010-10.org.openstack:104bcac8-98ce-42f8-a85f-82e29050a44c', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'oxmZWJRF', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False} {{(pid=75234) _connect_vol /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:672}}
Nov 29 03:14:55 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-5e94a590-65a3-40ab-b081-3bc7d55c8204 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.335s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:87}}
Nov 29 03:14:55 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-5e94a590-65a3-40ab-b081-3bc7d55c8204 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] <== connect_volume: return (1336ms) {'type': 'block', 'scsi_wwn': '23335363436666630', 'path': '/dev/sdb'} {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:203}}
Nov 29 03:14:55 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-5e94a590-65a3-40ab-b081-3bc7d55c8204 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] [instance: 6894b55f-93c6-43c9-8933-56d5a3c43725] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '23335363436666630', 'path': '/dev/sdb'} {{(pid=75234) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Nov 29 03:14:55 user nova-compute[75234]: DEBUG nova.objects.instance [None req-5e94a590-65a3-40ab-b081-3bc7d55c8204 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Lazy-loading 'flavor' on Instance uuid 6894b55f-93c6-43c9-8933-56d5a3c43725 {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:14:55 user nova-compute[75234]: DEBUG nova.virt.libvirt.guest [None req-5e94a590-65a3-40ab-b081-3bc7d55c8204 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] attach device xml: <disk type="block" device="disk">
Nov 29 03:14:55 user nova-compute[75234]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Nov 29 03:14:55 user nova-compute[75234]:   <source dev="/dev/sdb"/>
Nov 29 03:14:55 user nova-compute[75234]:   <target dev="vdb" bus="virtio"/>
Nov 29 03:14:55 user nova-compute[75234]:   <serial>104bcac8-98ce-42f8-a85f-82e29050a44c</serial>
Nov 29 03:14:55 user nova-compute[75234]: </disk>
Nov 29 03:14:55 user nova-compute[75234]:  {{(pid=75234) attach_device /opt/stack/nova/nova/virt/libvirt/guest.py:339}}
Nov 29 03:14:56 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-5e94a590-65a3-40ab-b081-3bc7d55c8204 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] No BDM found with device name vda, not building metadata. {{(pid=75234) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12089}}
Nov 29 03:14:56 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-5e94a590-65a3-40ab-b081-3bc7d55c8204 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] No BDM found with device name vdb, not building metadata. {{(pid=75234) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12089}}
Nov 29 03:14:56 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-5e94a590-65a3-40ab-b081-3bc7d55c8204 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] No VIF found with MAC fa:16:3e:d1:0b:00, not building metadata {{(pid=75234) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Nov 29 03:14:56 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-5e94a590-65a3-40ab-b081-3bc7d55c8204 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Lock "6894b55f-93c6-43c9-8933-56d5a3c43725" "released" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: held 10.045s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:14:56 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:14:56 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:14:56 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:14:56 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:14:56 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=75234) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10541}}
Nov 29 03:14:56 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager.update_available_resource {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:14:56 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:14:56 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:14:56 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:14:56 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Auditing locally available compute resources for user (node: user) {{(pid=75234) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:907}}
Nov 29 03:14:56 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a082c96e-e384-40b6-bcc8-fc5e0e104669/disk.rescue --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:14:56 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:14:56 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a082c96e-e384-40b6-bcc8-fc5e0e104669/disk.rescue --force-share --output=json" returned: 0 in 0.205s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:14:56 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a082c96e-e384-40b6-bcc8-fc5e0e104669/disk.rescue --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:14:57 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a082c96e-e384-40b6-bcc8-fc5e0e104669/disk.rescue --force-share --output=json" returned: 0 in 0.175s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:14:57 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a082c96e-e384-40b6-bcc8-fc5e0e104669/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:14:57 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a082c96e-e384-40b6-bcc8-fc5e0e104669/disk --force-share --output=json" returned: 0 in 0.187s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:14:57 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a082c96e-e384-40b6-bcc8-fc5e0e104669/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:14:57 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a082c96e-e384-40b6-bcc8-fc5e0e104669/disk --force-share --output=json" returned: 0 in 0.190s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:14:57 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f609a7b7-1be2-44b4-9177-7e72a4d4c060/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:14:57 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f609a7b7-1be2-44b4-9177-7e72a4d4c060/disk --force-share --output=json" returned: 0 in 0.190s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:14:57 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f609a7b7-1be2-44b4-9177-7e72a4d4c060/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:14:57 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:14:57 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f609a7b7-1be2-44b4-9177-7e72a4d4c060/disk --force-share --output=json" returned: 0 in 0.179s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:14:57 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/21e9c6cd-c39f-41c0-8768-5dd64589c3f3/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:14:57 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/21e9c6cd-c39f-41c0-8768-5dd64589c3f3/disk --force-share --output=json" returned: 0 in 0.183s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:14:57 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/21e9c6cd-c39f-41c0-8768-5dd64589c3f3/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:14:58 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/21e9c6cd-c39f-41c0-8768-5dd64589c3f3/disk --force-share --output=json" returned: 0 in 0.200s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:14:58 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/13e5da64-9adc-4479-81c7-d329e9b5f08f/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:14:58 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/13e5da64-9adc-4479-81c7-d329e9b5f08f/disk --force-share --output=json" returned: 0 in 0.170s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:14:58 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/13e5da64-9adc-4479-81c7-d329e9b5f08f/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:14:58 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/13e5da64-9adc-4479-81c7-d329e9b5f08f/disk --force-share --output=json" returned: 0 in 0.170s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:14:58 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6c0e0184-ca3a-48ec-a6e3-a0a1172260d6/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:14:58 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6c0e0184-ca3a-48ec-a6e3-a0a1172260d6/disk --force-share --output=json" returned: 0 in 0.185s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:14:58 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6c0e0184-ca3a-48ec-a6e3-a0a1172260d6/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:14:58 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6c0e0184-ca3a-48ec-a6e3-a0a1172260d6/disk --force-share --output=json" returned: 0 in 0.197s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:14:58 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6894b55f-93c6-43c9-8933-56d5a3c43725/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:14:59 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6894b55f-93c6-43c9-8933-56d5a3c43725/disk --force-share --output=json" returned: 0 in 0.174s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:14:59 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6894b55f-93c6-43c9-8933-56d5a3c43725/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:14:59 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-216f0360-decd-4f80-a056-6c865d01c068 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Acquiring lock "6894b55f-93c6-43c9-8933-56d5a3c43725" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:14:59 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-216f0360-decd-4f80-a056-6c865d01c068 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Lock "6894b55f-93c6-43c9-8933-56d5a3c43725" acquired by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: waited 0.003s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:14:59 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:14:59 user nova-compute[75234]: INFO nova.compute.manager [None req-216f0360-decd-4f80-a056-6c865d01c068 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] [instance: 6894b55f-93c6-43c9-8933-56d5a3c43725] Detaching volume 104bcac8-98ce-42f8-a85f-82e29050a44c
Nov 29 03:14:59 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6894b55f-93c6-43c9-8933-56d5a3c43725/disk --force-share --output=json" returned: 0 in 0.167s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:14:59 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] skipping disk /dev/sdb (vdb) as it is a volume {{(pid=75234) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11213}}
Nov 29 03:14:59 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/062753a6-8f39-4e80-a549-9f0f5a4b877a/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:14:59 user nova-compute[75234]: INFO nova.virt.block_device [None req-216f0360-decd-4f80-a056-6c865d01c068 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] [instance: 6894b55f-93c6-43c9-8933-56d5a3c43725] Attempting to driver detach volume 104bcac8-98ce-42f8-a85f-82e29050a44c from mountpoint /dev/vdb
Nov 29 03:14:59 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-216f0360-decd-4f80-a056-6c865d01c068 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Attempting to detach device vdb from instance 6894b55f-93c6-43c9-8933-56d5a3c43725 from the persistent domain config. {{(pid=75234) _detach_from_persistent /opt/stack/nova/nova/virt/libvirt/driver.py:2487}}
Nov 29 03:14:59 user nova-compute[75234]: DEBUG nova.virt.libvirt.guest [None req-216f0360-decd-4f80-a056-6c865d01c068 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] detach device xml: <disk type="block" device="disk">
Nov 29 03:14:59 user nova-compute[75234]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Nov 29 03:14:59 user nova-compute[75234]:   <source dev="/dev/sdb"/>
Nov 29 03:14:59 user nova-compute[75234]:   <target dev="vdb" bus="virtio"/>
Nov 29 03:14:59 user nova-compute[75234]:   <serial>104bcac8-98ce-42f8-a85f-82e29050a44c</serial>
Nov 29 03:14:59 user nova-compute[75234]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x07" function="0x0"/>
Nov 29 03:14:59 user nova-compute[75234]: </disk>
Nov 29 03:14:59 user nova-compute[75234]:  {{(pid=75234) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Nov 29 03:14:59 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-216f0360-decd-4f80-a056-6c865d01c068 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Successfully detached device vdb from instance 6894b55f-93c6-43c9-8933-56d5a3c43725 from the persistent domain config.
Nov 29 03:14:59 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-216f0360-decd-4f80-a056-6c865d01c068 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] (1/8): Attempting to detach device vdb with device alias virtio-disk1 from instance 6894b55f-93c6-43c9-8933-56d5a3c43725 from the live domain config. {{(pid=75234) _detach_from_live_with_retry /opt/stack/nova/nova/virt/libvirt/driver.py:2523}}
Nov 29 03:14:59 user nova-compute[75234]: DEBUG nova.virt.libvirt.guest [None req-216f0360-decd-4f80-a056-6c865d01c068 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] detach device xml: <disk type="block" device="disk">
Nov 29 03:14:59 user nova-compute[75234]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Nov 29 03:14:59 user nova-compute[75234]:   <source dev="/dev/sdb"/>
Nov 29 03:14:59 user nova-compute[75234]:   <target dev="vdb" bus="virtio"/>
Nov 29 03:14:59 user nova-compute[75234]:   <serial>104bcac8-98ce-42f8-a85f-82e29050a44c</serial>
Nov 29 03:14:59 user nova-compute[75234]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x07" function="0x0"/>
Nov 29 03:14:59 user nova-compute[75234]: </disk>
Nov 29 03:14:59 user nova-compute[75234]:  {{(pid=75234) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Nov 29 03:14:59 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/062753a6-8f39-4e80-a549-9f0f5a4b877a/disk --force-share --output=json" returned: 0 in 0.172s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:14:59 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/062753a6-8f39-4e80-a549-9f0f5a4b877a/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:14:59 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/062753a6-8f39-4e80-a549-9f0f5a4b877a/disk --force-share --output=json" returned: 0 in 0.173s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:14:59 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/d3b90f39-4b91-4152-b820-bd0e263eec89/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:14:59 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-216f0360-decd-4f80-a056-6c865d01c068 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Start waiting for the detach event from libvirt for device vdb with device alias virtio-disk1 for instance 6894b55f-93c6-43c9-8933-56d5a3c43725 {{(pid=75234) _detach_from_live_and_wait_for_event /opt/stack/nova/nova/virt/libvirt/driver.py:2599}}
Nov 29 03:14:59 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/d3b90f39-4b91-4152-b820-bd0e263eec89/disk --force-share --output=json" returned: 0 in 0.192s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:14:59 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/d3b90f39-4b91-4152-b820-bd0e263eec89/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:15:00 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/d3b90f39-4b91-4152-b820-bd0e263eec89/disk --force-share --output=json" returned: 0 in 0.198s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:15:00 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:15:00 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:15:00 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Hypervisor/Node resource view: name=user free_ram=7742MB free_disk=25.445270538330078GB free_vcpus=4 pci_devices=[{"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}] {{(pid=75234) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1106}}
Nov 29 03:15:00 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:15:00 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:15:00 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance 21e9c6cd-c39f-41c0-8768-5dd64589c3f3 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:15:00 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance f609a7b7-1be2-44b4-9177-7e72a4d4c060 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:15:00 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance 6c0e0184-ca3a-48ec-a6e3-a0a1172260d6 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:15:00 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance d3b90f39-4b91-4152-b820-bd0e263eec89 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:15:00 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance 13e5da64-9adc-4479-81c7-d329e9b5f08f actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:15:00 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance a082c96e-e384-40b6-bcc8-fc5e0e104669 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:15:00 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance 6894b55f-93c6-43c9-8933-56d5a3c43725 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:15:00 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance 062753a6-8f39-4e80-a549-9f0f5a4b877a actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:15:00 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Total usable vcpus: 12, total allocated vcpus: 8 {{(pid=75234) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1129}}
Nov 29 03:15:00 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Final resource view: name=user phys_ram=16011MB used_ram=4608MB phys_disk=40GB used_disk=8GB total_vcpus=12 used_vcpus=8 pci_stats=[] {{(pid=75234) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1138}}
Nov 29 03:15:00 user nova-compute[75234]: DEBUG nova.compute.provider_tree [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Inventory has not changed in ProviderTree for provider: 4d291220-7c37-4839-a0e3-b23c66502b74 {{(pid=75234) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Nov 29 03:15:00 user nova-compute[75234]: DEBUG nova.scheduler.client.report [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Inventory has not changed for provider 4d291220-7c37-4839-a0e3-b23c66502b74 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=75234) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Nov 29 03:15:00 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Compute_service record updated for user:user {{(pid=75234) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:1067}}
Nov 29 03:15:00 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.459s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:15:01 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:15:02 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:15:03 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:15:03 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:15:04 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:15:06 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:15:07 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-78fd111a-b14e-4026-965f-36b5c8becb30 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Waiting for libvirt event about the detach of device vdb with device alias virtio-disk1 from instance 6c0e0184-ca3a-48ec-a6e3-a0a1172260d6 is timed out.
Nov 29 03:15:07 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-78fd111a-b14e-4026-965f-36b5c8becb30 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Successfully detached device vdb from instance 6c0e0184-ca3a-48ec-a6e3-a0a1172260d6 from the live domain config.
Nov 29 03:15:07 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-78fd111a-b14e-4026-965f-36b5c8becb30 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] [instance: 6c0e0184-ca3a-48ec-a6e3-a0a1172260d6] calling os-brick to detach iSCSI Volume {{(pid=75234) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Nov 29 03:15:07 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-78fd111a-b14e-4026-965f-36b5c8becb30 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f8a5050c9a0>, {'target_iqn': 'iqn.2010-10.org.openstack:56237f4c-4138-4f50-aafb-9c0f9d4bb1cf', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'hbYaegTr', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdf'}, None), 'kwargs': False}}" {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:176}}
Nov 29 03:15:07 user nova-compute[75234]: WARNING os_brick.initiator.connectors.base [None req-78fd111a-b14e-4026-965f-36b5c8becb30 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Nov 29 03:15:07 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-78fd111a-b14e-4026-965f-36b5c8becb30 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:68}}
Nov 29 03:15:07 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-78fd111a-b14e-4026-965f-36b5c8becb30 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:73}}
Nov 29 03:15:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:15:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.021s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:15:07 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[beb5bc2a-99ef-4339-b857-12599849e7ba]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:104bcac8-98ce-42f8-a85f-82e29050a44c\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:56237f4c-4138-4f50-aafb-9c0f9d4bb1cf\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:6a4d34d3-9974-43bc-afcb-9169c893d9b7\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:15:07 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-78fd111a-b14e-4026-965f-36b5c8becb30 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS:
Nov 29 03:15:07 user nova-compute[75234]: No targets found.
Nov 29 03:15:07 user nova-compute[75234]: iSNS:
Nov 29 03:15:07 user nova-compute[75234]: No targets found.
Nov 29 03:15:07 user nova-compute[75234]: STATIC:
Nov 29 03:15:07 user nova-compute[75234]: Target: iqn.2010-10.org.openstack:104bcac8-98ce-42f8-a85f-82e29050a44c
Nov 29 03:15:07 user nova-compute[75234]:         Portal: 172.16.0.220:3260,-1
Nov 29 03:15:07 user nova-compute[75234]:                 Iface Name: default
Nov 29 03:15:07 user nova-compute[75234]: Target: iqn.2010-10.org.openstack:56237f4c-4138-4f50-aafb-9c0f9d4bb1cf
Nov 29 03:15:07 user nova-compute[75234]:         Portal: 172.16.0.220:3260,-1
Nov 29 03:15:07 user nova-compute[75234]:                 Iface Name: default
Nov 29 03:15:07 user nova-compute[75234]: Target: iqn.2010-10.org.openstack:6a4d34d3-9974-43bc-afcb-9169c893d9b7
Nov 29 03:15:07 user nova-compute[75234]:         Portal: 172.16.0.220:3260,-1
Nov 29 03:15:07 user nova-compute[75234]:                 Iface Name: default
Nov 29 03:15:07 user nova-compute[75234]: FIRMWARE:
Nov 29 03:15:07 user nova-compute[75234]: No targets found.
Nov 29 03:15:07 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:15:07 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-78fd111a-b14e-4026-965f-36b5c8becb30 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Regex to get portals from discoverydb: ^SENDTARGETS:
Nov 29 03:15:07 user nova-compute[75234]: .*?^DiscoveryAddress: 172.16.0.220,3260.*?
Nov 29 03:15:07 user nova-compute[75234]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=75234) _get_discoverydb_portals /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:382}}
Nov 29 03:15:07 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-78fd111a-b14e-4026-965f-36b5c8becb30 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:56237f4c-4138-4f50-aafb-9c0f9d4bb1cf', 0)] {{(pid=75234) _get_connection_devices /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:830}}
Nov 29 03:15:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:15:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.020s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:15:07 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[5028804a-bf6c-4d55-af6c-78ec2c156b37]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:104bcac8-98ce-42f8-a85f-82e29050a44c\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:56237f4c-4138-4f50-aafb-9c0f9d4bb1cf\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:6a4d34d3-9974-43bc-afcb-9169c893d9b7\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:15:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:15:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.032s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:15:07 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[e1b5f996-4437-41f3-bd19-cc3688443af4]: (4, ('tcp: [21] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:56237f4c-4138-4f50-aafb-9c0f9d4bb1cf (non-flash)\ntcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6a4d34d3-9974-43bc-afcb-9169c893d9b7 (non-flash)\ntcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:104bcac8-98ce-42f8-a85f-82e29050a44c (non-flash)\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:15:07 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-78fd111a-b14e-4026-965f-36b5c8becb30 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [21] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:56237f4c-4138-4f50-aafb-9c0f9d4bb1cf (non-flash)
Nov 29 03:15:07 user nova-compute[75234]: tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6a4d34d3-9974-43bc-afcb-9169c893d9b7 (non-flash)
Nov 29 03:15:07 user nova-compute[75234]: tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:104bcac8-98ce-42f8-a85f-82e29050a44c (non-flash)
Nov 29 03:15:07 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:15:07 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-78fd111a-b14e-4026-965f-36b5c8becb30 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] iscsi session list stdout=tcp: [21] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:56237f4c-4138-4f50-aafb-9c0f9d4bb1cf (non-flash)
Nov 29 03:15:07 user nova-compute[75234]: tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6a4d34d3-9974-43bc-afcb-9169c893d9b7 (non-flash)
Nov 29 03:15:07 user nova-compute[75234]: tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:104bcac8-98ce-42f8-a85f-82e29050a44c (non-flash)
Nov 29 03:15:07 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsi_session /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Nov 29 03:15:07 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-78fd111a-b14e-4026-965f-36b5c8becb30 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7f8a501105e0>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:56237f4c-4138-4f50-aafb-9c0f9d4bb1cf'): ({'sdf'}, set())}) {{(pid=75234) _get_connection_devices /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:863}}
Nov 29 03:15:07 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-78fd111a-b14e-4026-965f-36b5c8becb30 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Removing single pathed devices sdf {{(pid=75234) remove_connection /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:374}}
Nov 29 03:15:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:15:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.025s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:15:07 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[4d4020da-979e-4970-a110-8bfb02a4493c]: (4, ('path checker states:\nup                  3\n\npaths: 0\nbusy: False\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:15:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdf {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:15:08 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdf" returned: 0 in 0.027s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:15:08 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[ca5ad241-5fd4-4f19-a2bd-501c26c0543d]: (4, ('ok\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:15:08 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-78fd111a-b14e-4026-965f-36b5c8becb30 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Flushing IO for device /dev/sdf {{(pid=75234) flush_device_io /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:434}}
Nov 29 03:15:08 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdf {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:15:08 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdf" returned: 0 in 0.015s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:15:08 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[f313eddf-9a8b-4d73-a41f-54f1758ebd1c]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:15:08 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-78fd111a-b14e-4026-965f-36b5c8becb30 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Remove SCSI device /dev/sdf with /sys/block/sdf/device/delete {{(pid=75234) remove_scsi_device /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:148}}
Nov 29 03:15:08 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdf/device/delete {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:15:08 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdf/device/delete" returned: 0 in 0.057s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:15:08 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[c64cb620-73df-4616-a66f-c45bd126affe]: (4, ('1', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:15:08 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-78fd111a-b14e-4026-965f-36b5c8becb30 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Checking to see if SCSI volumes sdf have been removed. {{(pid=75234) wait_for_volumes_removal /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:156}}
Nov 29 03:15:08 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-78fd111a-b14e-4026-965f-36b5c8becb30 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] SCSI volumes sdf have been removed. {{(pid=75234) wait_for_volumes_removal /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:166}}
Nov 29 03:15:08 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-78fd111a-b14e-4026-965f-36b5c8becb30 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:56237f4c-4138-4f50-aafb-9c0f9d4bb1cf')] {{(pid=75234) _disconnect_connection /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Nov 29 03:15:08 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:56237f4c-4138-4f50-aafb-9c0f9d4bb1cf -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:15:08 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:56237f4c-4138-4f50-aafb-9c0f9d4bb1cf -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.021s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:15:08 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[ef61685a-e746-442e-be6a-5fe74fc80f84]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:15:08 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-78fd111a-b14e-4026-965f-36b5c8becb30 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:15:08 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:56237f4c-4138-4f50-aafb-9c0f9d4bb1cf -p 172.16.0.220:3260 --logout {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:15:08 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:56237f4c-4138-4f50-aafb-9c0f9d4bb1cf -p 172.16.0.220:3260 --logout" returned: 0 in 0.064s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:15:08 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[cee54936-8d49-45c9-8b02-a11e5721fd75]: (4, ('Logging out of session [sid: 21, target: iqn.2010-10.org.openstack:56237f4c-4138-4f50-aafb-9c0f9d4bb1cf, portal: 172.16.0.220,3260]\nLogout of [sid: 21, target: iqn.2010-10.org.openstack:56237f4c-4138-4f50-aafb-9c0f9d4bb1cf, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:15:08 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-78fd111a-b14e-4026-965f-36b5c8becb30 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 21, target: iqn.2010-10.org.openstack:56237f4c-4138-4f50-aafb-9c0f9d4bb1cf, portal: 172.16.0.220,3260]
Nov 29 03:15:08 user nova-compute[75234]: Logout of [sid: 21, target: iqn.2010-10.org.openstack:56237f4c-4138-4f50-aafb-9c0f9d4bb1cf, portal: 172.16.0.220,3260] successful.
Nov 29 03:15:08 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:15:08 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:56237f4c-4138-4f50-aafb-9c0f9d4bb1cf -p 172.16.0.220:3260 --op delete {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:15:08 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:56237f4c-4138-4f50-aafb-9c0f9d4bb1cf -p 172.16.0.220:3260 --op delete" returned: 0 in 0.028s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:15:08 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[9b6191f7-2a63-4123-accd-55c6cff4516e]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:15:08 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-78fd111a-b14e-4026-965f-36b5c8becb30 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:15:08 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-78fd111a-b14e-4026-965f-36b5c8becb30 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.364s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:87}}
Nov 29 03:15:08 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-78fd111a-b14e-4026-965f-36b5c8becb30 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] <== disconnect_volume: return (366ms) None {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:203}}
Nov 29 03:15:08 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-78fd111a-b14e-4026-965f-36b5c8becb30 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] [instance: 6c0e0184-ca3a-48ec-a6e3-a0a1172260d6] Disconnected iSCSI Volume {{(pid=75234) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:79}}
Nov 29 03:15:08 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:15:09 user nova-compute[75234]: DEBUG nova.objects.instance [None req-78fd111a-b14e-4026-965f-36b5c8becb30 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Lazy-loading 'flavor' on Instance uuid 6c0e0184-ca3a-48ec-a6e3-a0a1172260d6 {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:15:09 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-78fd111a-b14e-4026-965f-36b5c8becb30 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Lock "6c0e0184-ca3a-48ec-a6e3-a0a1172260d6" "released" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: held 21.834s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:15:09 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:15:11 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-28e7bf3b-25fb-424d-87b1-89eec41681db tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Waiting for libvirt event about the detach of device vdc with device alias virtio-disk2 from instance 062753a6-8f39-4e80-a549-9f0f5a4b877a is timed out.
Nov 29 03:15:11 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-28e7bf3b-25fb-424d-87b1-89eec41681db tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Successfully detached device vdc from instance 062753a6-8f39-4e80-a549-9f0f5a4b877a from the live domain config.
Nov 29 03:15:11 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-28e7bf3b-25fb-424d-87b1-89eec41681db tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] [instance: 062753a6-8f39-4e80-a549-9f0f5a4b877a] calling os-brick to detach iSCSI Volume {{(pid=75234) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Nov 29 03:15:11 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-28e7bf3b-25fb-424d-87b1-89eec41681db tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f8a5050c9a0>, {'target_iqn': 'iqn.2010-10.org.openstack:6a4d34d3-9974-43bc-afcb-9169c893d9b7', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'OpBhCUuD', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdd'}, None), 'kwargs': False}}" {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:176}}
Nov 29 03:15:11 user nova-compute[75234]: WARNING os_brick.initiator.connectors.base [None req-28e7bf3b-25fb-424d-87b1-89eec41681db tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Nov 29 03:15:11 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-28e7bf3b-25fb-424d-87b1-89eec41681db tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:68}}
Nov 29 03:15:11 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-28e7bf3b-25fb-424d-87b1-89eec41681db tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.002s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:73}}
Nov 29 03:15:11 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:15:11 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.019s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:15:11 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[ac544937-a320-409f-a5f3-394c08cca3cd]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:104bcac8-98ce-42f8-a85f-82e29050a44c\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:6a4d34d3-9974-43bc-afcb-9169c893d9b7\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:15:11 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-28e7bf3b-25fb-424d-87b1-89eec41681db tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS:
Nov 29 03:15:11 user nova-compute[75234]: No targets found.
Nov 29 03:15:11 user nova-compute[75234]: iSNS:
Nov 29 03:15:11 user nova-compute[75234]: No targets found.
Nov 29 03:15:11 user nova-compute[75234]: STATIC:
Nov 29 03:15:11 user nova-compute[75234]: Target: iqn.2010-10.org.openstack:104bcac8-98ce-42f8-a85f-82e29050a44c
Nov 29 03:15:11 user nova-compute[75234]:         Portal: 172.16.0.220:3260,-1
Nov 29 03:15:11 user nova-compute[75234]:                 Iface Name: default
Nov 29 03:15:11 user nova-compute[75234]: Target: iqn.2010-10.org.openstack:6a4d34d3-9974-43bc-afcb-9169c893d9b7
Nov 29 03:15:11 user nova-compute[75234]:         Portal: 172.16.0.220:3260,-1
Nov 29 03:15:11 user nova-compute[75234]:                 Iface Name: default
Nov 29 03:15:11 user nova-compute[75234]: FIRMWARE:
Nov 29 03:15:11 user nova-compute[75234]: No targets found.
Nov 29 03:15:11 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:15:11 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-28e7bf3b-25fb-424d-87b1-89eec41681db tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Regex to get portals from discoverydb: ^SENDTARGETS:
Nov 29 03:15:11 user nova-compute[75234]: .*?^DiscoveryAddress: 172.16.0.220,3260.*?
Nov 29 03:15:11 user nova-compute[75234]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=75234) _get_discoverydb_portals /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:382}}
Nov 29 03:15:11 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-28e7bf3b-25fb-424d-87b1-89eec41681db tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:6a4d34d3-9974-43bc-afcb-9169c893d9b7', 0)] {{(pid=75234) _get_connection_devices /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:830}}
Nov 29 03:15:11 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:15:11 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.018s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:15:11 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[647d0523-56b9-4b38-8957-8889b229b241]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:104bcac8-98ce-42f8-a85f-82e29050a44c\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:6a4d34d3-9974-43bc-afcb-9169c893d9b7\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:15:11 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:15:11 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.025s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:15:11 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[026dc714-927a-4313-b8c5-5cc1cbfc91c8]: (4, ('tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6a4d34d3-9974-43bc-afcb-9169c893d9b7 (non-flash)\ntcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:104bcac8-98ce-42f8-a85f-82e29050a44c (non-flash)\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:15:11 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-28e7bf3b-25fb-424d-87b1-89eec41681db tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6a4d34d3-9974-43bc-afcb-9169c893d9b7 (non-flash)
Nov 29 03:15:11 user nova-compute[75234]: tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:104bcac8-98ce-42f8-a85f-82e29050a44c (non-flash)
Nov 29 03:15:11 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:15:11 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-28e7bf3b-25fb-424d-87b1-89eec41681db tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] iscsi session list stdout=tcp: [23] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6a4d34d3-9974-43bc-afcb-9169c893d9b7 (non-flash)
Nov 29 03:15:11 user nova-compute[75234]: tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:104bcac8-98ce-42f8-a85f-82e29050a44c (non-flash)
Nov 29 03:15:11 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsi_session /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Nov 29 03:15:11 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-28e7bf3b-25fb-424d-87b1-89eec41681db tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7f8a506da8c0>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:6a4d34d3-9974-43bc-afcb-9169c893d9b7'): ({'sdd'}, set())}) {{(pid=75234) _get_connection_devices /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:863}}
Nov 29 03:15:11 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-28e7bf3b-25fb-424d-87b1-89eec41681db tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Removing single pathed devices sdd {{(pid=75234) remove_connection /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:374}}
Nov 29 03:15:11 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:15:11 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.018s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:15:11 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[b5ffb4a9-9e58-44b7-8fc4-6cb76fb61ea8]: (4, ('path checker states:\nup                  2\n\npaths: 0\nbusy: False\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:15:11 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdd {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:15:11 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdd" returned: 0 in 0.025s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:15:11 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[4b2d6e8f-ec5a-4278-b677-f511363c4e2f]: (4, ('ok\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:15:11 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-28e7bf3b-25fb-424d-87b1-89eec41681db tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Flushing IO for device /dev/sdd {{(pid=75234) flush_device_io /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:434}}
Nov 29 03:15:11 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdd {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:15:11 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdd" returned: 0 in 0.012s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:15:11 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[50d56305-ef0b-43c6-b9ed-69a71b1c6271]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:15:11 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-28e7bf3b-25fb-424d-87b1-89eec41681db tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Remove SCSI device /dev/sdd with /sys/block/sdd/device/delete {{(pid=75234) remove_scsi_device /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:148}}
Nov 29 03:15:11 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdd/device/delete {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:15:11 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdd/device/delete" returned: 0 in 0.052s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:15:11 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[f0acacd3-af8e-44ec-aa45-2b5eb67b2047]: (4, ('1', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:15:11 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-28e7bf3b-25fb-424d-87b1-89eec41681db tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Checking to see if SCSI volumes sdd have been removed. {{(pid=75234) wait_for_volumes_removal /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:156}}
Nov 29 03:15:11 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-28e7bf3b-25fb-424d-87b1-89eec41681db tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] SCSI volumes sdd have been removed. {{(pid=75234) wait_for_volumes_removal /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:166}}
Nov 29 03:15:11 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-28e7bf3b-25fb-424d-87b1-89eec41681db tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:6a4d34d3-9974-43bc-afcb-9169c893d9b7')] {{(pid=75234) _disconnect_connection /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Nov 29 03:15:11 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:6a4d34d3-9974-43bc-afcb-9169c893d9b7 -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:15:11 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:6a4d34d3-9974-43bc-afcb-9169c893d9b7 -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.018s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:15:11 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[949e72f8-c2eb-4eb8-acc7-b5c7dee1ec14]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:15:11 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-28e7bf3b-25fb-424d-87b1-89eec41681db tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:15:11 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:6a4d34d3-9974-43bc-afcb-9169c893d9b7 -p 172.16.0.220:3260 --logout {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:15:11 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:15:11 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:6a4d34d3-9974-43bc-afcb-9169c893d9b7 -p 172.16.0.220:3260 --logout" returned: 0 in 0.062s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:15:11 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[2e00a508-aa07-4ac8-b7aa-3e353c0af1b3]: (4, ('Logging out of session [sid: 23, target: iqn.2010-10.org.openstack:6a4d34d3-9974-43bc-afcb-9169c893d9b7, portal: 172.16.0.220,3260]\nLogout of [sid: 23, target: iqn.2010-10.org.openstack:6a4d34d3-9974-43bc-afcb-9169c893d9b7, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:15:11 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-28e7bf3b-25fb-424d-87b1-89eec41681db tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 23, target: iqn.2010-10.org.openstack:6a4d34d3-9974-43bc-afcb-9169c893d9b7, portal: 172.16.0.220,3260]
Nov 29 03:15:11 user nova-compute[75234]: Logout of [sid: 23, target: iqn.2010-10.org.openstack:6a4d34d3-9974-43bc-afcb-9169c893d9b7, portal: 172.16.0.220,3260] successful.
Nov 29 03:15:11 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:15:11 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:6a4d34d3-9974-43bc-afcb-9169c893d9b7 -p 172.16.0.220:3260 --op delete {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:15:11 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:6a4d34d3-9974-43bc-afcb-9169c893d9b7 -p 172.16.0.220:3260 --op delete" returned: 0 in 0.025s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:15:11 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[a749cab6-91f2-4fdf-b951-abafe175c9c1]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:15:11 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-28e7bf3b-25fb-424d-87b1-89eec41681db tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:15:11 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-28e7bf3b-25fb-424d-87b1-89eec41681db tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.319s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:87}}
Nov 29 03:15:11 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-28e7bf3b-25fb-424d-87b1-89eec41681db tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] <== disconnect_volume: return (320ms) None {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:203}}
Nov 29 03:15:11 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-28e7bf3b-25fb-424d-87b1-89eec41681db tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] [instance: 062753a6-8f39-4e80-a549-9f0f5a4b877a] Disconnected iSCSI Volume {{(pid=75234) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:79}}
Nov 29 03:15:12 user nova-compute[75234]: DEBUG nova.objects.instance [None req-28e7bf3b-25fb-424d-87b1-89eec41681db tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Lazy-loading 'flavor' on Instance uuid 062753a6-8f39-4e80-a549-9f0f5a4b877a {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:15:12 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-28e7bf3b-25fb-424d-87b1-89eec41681db tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Lock "062753a6-8f39-4e80-a549-9f0f5a4b877a" "released" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: held 22.233s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:15:12 user nova-compute[75234]: INFO nova.compute.manager [None req-dcdf1c1e-c861-4abc-a20d-7a95695a845b tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] [instance: 13e5da64-9adc-4479-81c7-d329e9b5f08f] Rescuing
Nov 29 03:15:12 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-dcdf1c1e-c861-4abc-a20d-7a95695a845b tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Acquiring lock "refresh_cache-13e5da64-9adc-4479-81c7-d329e9b5f08f" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:15:12 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-dcdf1c1e-c861-4abc-a20d-7a95695a845b tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Acquired lock "refresh_cache-13e5da64-9adc-4479-81c7-d329e9b5f08f" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:15:12 user nova-compute[75234]: DEBUG nova.network.neutron [None req-dcdf1c1e-c861-4abc-a20d-7a95695a845b tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] [instance: 13e5da64-9adc-4479-81c7-d329e9b5f08f] Building network info cache for instance {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2009}}
Nov 29 03:15:13 user nova-compute[75234]: DEBUG nova.network.neutron [None req-dcdf1c1e-c861-4abc-a20d-7a95695a845b tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] [instance: 13e5da64-9adc-4479-81c7-d329e9b5f08f] Updating instance_info_cache with network_info: [{"id": "2abf9e25-d1d9-4893-899e-912d41b7c38f", "address": "fa:16:3e:59:79:35", "network": {"id": "2b3c3140-f3b6-45da-8202-bb4d7f0dd130", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-2043420778-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "f8a7884867c944e1ba1024270e4902f3", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2abf9e25-d1", "ovs_interfaceid": "2abf9e25-d1d9-4893-899e-912d41b7c38f", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:15:13 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-dcdf1c1e-c861-4abc-a20d-7a95695a845b tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Releasing lock "refresh_cache-13e5da64-9adc-4479-81c7-d329e9b5f08f" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:15:13 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:15:13 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:15:13 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:15:13 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:15:14 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:15:14 user nova-compute[75234]: INFO nova.virt.libvirt.driver [-] [instance: 13e5da64-9adc-4479-81c7-d329e9b5f08f] Instance destroyed successfully.
Nov 29 03:15:14 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-dcdf1c1e-c861-4abc-a20d-7a95695a845b tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] [instance: 13e5da64-9adc-4479-81c7-d329e9b5f08f] Attempting rescue
Nov 29 03:15:14 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-dcdf1c1e-c861-4abc-a20d-7a95695a845b tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] rescue generated disk_info: {'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'disk.rescue': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vdb', 'type': 'disk'}}} {{(pid=75234) rescue /opt/stack/nova/nova/virt/libvirt/driver.py:4318}}
Nov 29 03:15:14 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-dcdf1c1e-c861-4abc-a20d-7a95695a845b tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] [instance: 13e5da64-9adc-4479-81c7-d329e9b5f08f] Instance directory exists: not creating {{(pid=75234) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4723}}
Nov 29 03:15:14 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-dcdf1c1e-c861-4abc-a20d-7a95695a845b tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] [instance: 13e5da64-9adc-4479-81c7-d329e9b5f08f] Creating image(s)
Nov 29 03:15:14 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-dcdf1c1e-c861-4abc-a20d-7a95695a845b tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Acquiring lock "/opt/stack/data/nova/instances/13e5da64-9adc-4479-81c7-d329e9b5f08f/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:15:14 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-dcdf1c1e-c861-4abc-a20d-7a95695a845b tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Lock "/opt/stack/data/nova/instances/13e5da64-9adc-4479-81c7-d329e9b5f08f/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:15:14 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-dcdf1c1e-c861-4abc-a20d-7a95695a845b tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Lock "/opt/stack/data/nova/instances/13e5da64-9adc-4479-81c7-d329e9b5f08f/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.002s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:15:14 user nova-compute[75234]: DEBUG nova.objects.instance [None req-dcdf1c1e-c861-4abc-a20d-7a95695a845b tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Lazy-loading 'trusted_certs' on Instance uuid 13e5da64-9adc-4479-81c7-d329e9b5f08f {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:15:14 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-dcdf1c1e-c861-4abc-a20d-7a95695a845b tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Acquiring lock "80686e29dc9db6600d03e85cb62af298e12d7918" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:15:14 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-dcdf1c1e-c861-4abc-a20d-7a95695a845b tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Lock "80686e29dc9db6600d03e85cb62af298e12d7918" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:15:14 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-dcdf1c1e-c861-4abc-a20d-7a95695a845b tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:15:14 user nova-compute[75234]: DEBUG nova.compute.manager [req-7236bf9c-fda5-4f67-b414-7c75c5922497 req-61feb8a9-c6c4-492f-8608-4369305dc78e service nova] [instance: 13e5da64-9adc-4479-81c7-d329e9b5f08f] Received event network-vif-unplugged-2abf9e25-d1d9-4893-899e-912d41b7c38f {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:15:14 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-7236bf9c-fda5-4f67-b414-7c75c5922497 req-61feb8a9-c6c4-492f-8608-4369305dc78e service nova] Acquiring lock "13e5da64-9adc-4479-81c7-d329e9b5f08f-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:15:14 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-7236bf9c-fda5-4f67-b414-7c75c5922497 req-61feb8a9-c6c4-492f-8608-4369305dc78e service nova] Lock "13e5da64-9adc-4479-81c7-d329e9b5f08f-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:15:14 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-7236bf9c-fda5-4f67-b414-7c75c5922497 req-61feb8a9-c6c4-492f-8608-4369305dc78e service nova] Lock "13e5da64-9adc-4479-81c7-d329e9b5f08f-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:15:14 user nova-compute[75234]: DEBUG nova.compute.manager [req-7236bf9c-fda5-4f67-b414-7c75c5922497 req-61feb8a9-c6c4-492f-8608-4369305dc78e service nova] [instance: 13e5da64-9adc-4479-81c7-d329e9b5f08f] No waiting events found dispatching network-vif-unplugged-2abf9e25-d1d9-4893-899e-912d41b7c38f {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:15:14 user nova-compute[75234]: WARNING nova.compute.manager [req-7236bf9c-fda5-4f67-b414-7c75c5922497 req-61feb8a9-c6c4-492f-8608-4369305dc78e service nova] [instance: 13e5da64-9adc-4479-81c7-d329e9b5f08f] Received unexpected event network-vif-unplugged-2abf9e25-d1d9-4893-899e-912d41b7c38f for instance with vm_state active and task_state rescuing.
Nov 29 03:15:14 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-dcdf1c1e-c861-4abc-a20d-7a95695a845b tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json" returned: 0 in 0.187s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:15:14 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-dcdf1c1e-c861-4abc-a20d-7a95695a845b tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918,backing_fmt=raw /opt/stack/data/nova/instances/13e5da64-9adc-4479-81c7-d329e9b5f08f/disk.rescue {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:15:14 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-dcdf1c1e-c861-4abc-a20d-7a95695a845b tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918,backing_fmt=raw /opt/stack/data/nova/instances/13e5da64-9adc-4479-81c7-d329e9b5f08f/disk.rescue" returned: 0 in 0.054s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:15:14 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-dcdf1c1e-c861-4abc-a20d-7a95695a845b tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Lock "80686e29dc9db6600d03e85cb62af298e12d7918" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.246s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:15:14 user nova-compute[75234]: DEBUG nova.objects.instance [None req-dcdf1c1e-c861-4abc-a20d-7a95695a845b tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Lazy-loading 'migration_context' on Instance uuid 13e5da64-9adc-4479-81c7-d329e9b5f08f {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:15:14 user nova-compute[75234]: DEBUG nova.objects.base [None req-dcdf1c1e-c861-4abc-a20d-7a95695a845b tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Object Instance<13e5da64-9adc-4479-81c7-d329e9b5f08f> lazy-loaded attributes: trusted_certs,migration_context {{(pid=75234) wrapper /opt/stack/nova/nova/objects/base.py:126}}
Nov 29 03:15:14 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-dcdf1c1e-c861-4abc-a20d-7a95695a845b tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] [instance: 13e5da64-9adc-4479-81c7-d329e9b5f08f] Created local disks {{(pid=75234) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4861}}
Nov 29 03:15:14 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-dcdf1c1e-c861-4abc-a20d-7a95695a845b tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] [instance: 13e5da64-9adc-4479-81c7-d329e9b5f08f] Start _get_guest_xml network_info=[{"id": "2abf9e25-d1d9-4893-899e-912d41b7c38f", "address": "fa:16:3e:59:79:35", "network": {"id": "2b3c3140-f3b6-45da-8202-bb4d7f0dd130", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-2043420778-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [], "label": "tempest-ServerRescueNegativeTestJSON-2043420778-network", "vif_mac": "fa:16:3e:59:79:35"}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "f8a7884867c944e1ba1024270e4902f3", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2abf9e25-d1", "ovs_interfaceid": "2abf9e25-d1d9-4893-899e-912d41b7c38f", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'disk.rescue': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vdb', 'type': 'disk'}}} image_meta=ImageMeta(checksum='c8fc807773e5354afe61636071771906',container_format='bare',created_at=2023-11-29T03:02:15Z,direct_url=<?>,disk_format='qcow2',id=96c30ba7-9b73-4a30-8ba4-7138879573b7,min_disk=0,min_ram=0,name='cirros-0.6.2-x86_64-disk',owner='b18e380921c748b78517b6f1211cd0f1',properties=ImageMetaProps,protected=<?>,size=21430272,status='active',tags=<?>,updated_at=2023-11-29T03:02:17Z,virtual_size=<?>,visibility=<?>) rescue={'image_id': '96c30ba7-9b73-4a30-8ba4-7138879573b7', 'kernel_id': '', 'ramdisk_id': ''} block_device_info=None {{(pid=75234) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7562}}
Nov 29 03:15:14 user nova-compute[75234]: DEBUG nova.objects.instance [None req-dcdf1c1e-c861-4abc-a20d-7a95695a845b tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Lazy-loading 'resources' on Instance uuid 13e5da64-9adc-4479-81c7-d329e9b5f08f {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:15:14 user nova-compute[75234]: DEBUG nova.objects.base [None req-dcdf1c1e-c861-4abc-a20d-7a95695a845b tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Object Instance<13e5da64-9adc-4479-81c7-d329e9b5f08f> lazy-loaded attributes: trusted_certs,migration_context,resources {{(pid=75234) wrapper /opt/stack/nova/nova/objects/base.py:126}}
Nov 29 03:15:14 user nova-compute[75234]: DEBUG nova.objects.instance [None req-dcdf1c1e-c861-4abc-a20d-7a95695a845b tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Lazy-loading 'numa_topology' on Instance uuid 13e5da64-9adc-4479-81c7-d329e9b5f08f {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:15:14 user nova-compute[75234]: DEBUG nova.objects.base [None req-dcdf1c1e-c861-4abc-a20d-7a95695a845b tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Object Instance<13e5da64-9adc-4479-81c7-d329e9b5f08f> lazy-loaded attributes: trusted_certs,migration_context,resources,numa_topology {{(pid=75234) wrapper /opt/stack/nova/nova/objects/base.py:126}}
Nov 29 03:15:14 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-dcdf1c1e-c861-4abc-a20d-7a95695a845b tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:15:14 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-dcdf1c1e-c861-4abc-a20d-7a95695a845b tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:15:14 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-dcdf1c1e-c861-4abc-a20d-7a95695a845b tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Searching host: 'user' for CPU controller through CGroups V1... {{(pid=75234) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1652}}
Nov 29 03:15:14 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-dcdf1c1e-c861-4abc-a20d-7a95695a845b tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] CPU controller missing on host. {{(pid=75234) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1662}}
Nov 29 03:15:14 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-dcdf1c1e-c861-4abc-a20d-7a95695a845b tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Searching host: 'user' for CPU controller through CGroups V2... {{(pid=75234) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1671}}
Nov 29 03:15:14 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-dcdf1c1e-c861-4abc-a20d-7a95695a845b tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] CPU controller found on host. {{(pid=75234) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1678}}
Nov 29 03:15:14 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-dcdf1c1e-c861-4abc-a20d-7a95695a845b tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=75234) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5400}}
Nov 29 03:15:14 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-dcdf1c1e-c861-4abc-a20d-7a95695a845b tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-11-29T03:02:04Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='1',id=6,is_public=True,memory_mb=512,name='m1.tiny',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='c8fc807773e5354afe61636071771906',container_format='bare',created_at=2023-11-29T03:02:15Z,direct_url=<?>,disk_format='qcow2',id=96c30ba7-9b73-4a30-8ba4-7138879573b7,min_disk=0,min_ram=0,name='cirros-0.6.2-x86_64-disk',owner='b18e380921c748b78517b6f1211cd0f1',properties=ImageMetaProps,protected=<?>,size=21430272,status='active',tags=<?>,updated_at=2023-11-29T03:02:17Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:563}}
Nov 29 03:15:14 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-dcdf1c1e-c861-4abc-a20d-7a95695a845b tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Flavor limits 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:348}}
Nov 29 03:15:14 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-dcdf1c1e-c861-4abc-a20d-7a95695a845b tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Image limits 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:352}}
Nov 29 03:15:14 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-dcdf1c1e-c861-4abc-a20d-7a95695a845b tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Flavor pref 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:388}}
Nov 29 03:15:14 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-dcdf1c1e-c861-4abc-a20d-7a95695a845b tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Image pref 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:392}}
Nov 29 03:15:14 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-dcdf1c1e-c861-4abc-a20d-7a95695a845b tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:430}}
Nov 29 03:15:14 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-dcdf1c1e-c861-4abc-a20d-7a95695a845b tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:569}}
Nov 29 03:15:14 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-dcdf1c1e-c861-4abc-a20d-7a95695a845b tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=75234) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:471}}
Nov 29 03:15:14 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-dcdf1c1e-c861-4abc-a20d-7a95695a845b tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Got 1 possible topologies {{(pid=75234) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:501}}
Nov 29 03:15:14 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-dcdf1c1e-c861-4abc-a20d-7a95695a845b tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:575}}
Nov 29 03:15:14 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-dcdf1c1e-c861-4abc-a20d-7a95695a845b tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:577}}
Nov 29 03:15:14 user nova-compute[75234]: DEBUG nova.objects.instance [None req-dcdf1c1e-c861-4abc-a20d-7a95695a845b tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Lazy-loading 'vcpu_model' on Instance uuid 13e5da64-9adc-4479-81c7-d329e9b5f08f {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:15:14 user nova-compute[75234]: DEBUG nova.objects.base [None req-dcdf1c1e-c861-4abc-a20d-7a95695a845b tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Object Instance<13e5da64-9adc-4479-81c7-d329e9b5f08f> lazy-loaded attributes: trusted_certs,migration_context,resources,numa_topology,vcpu_model {{(pid=75234) wrapper /opt/stack/nova/nova/objects/base.py:126}}
Nov 29 03:15:14 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-dcdf1c1e-c861-4abc-a20d-7a95695a845b tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-11-29T03:10:25Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-ServerRescueNegativeTestJSON-server-715053465',display_name='tempest-ServerRescueNegativeTestJSON-server-715053465',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-serverrescuenegativetestjson-server-715053465',id=18,image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data=None,key_name=None,keypairs=<?>,launch_index=0,launched_at=2023-11-29T03:10:38Z,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='f8a7884867c944e1ba1024270e4902f3',ramdisk_id='',reservation_id='r-e0thim3u',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-ServerRescueNegativeTestJSON-1182706601',owner_user_name='tempest-ServerRescueNegativeTestJSON-1182706601-project-member'},tags=<?>,task_state='rescuing',terminated_at=None,trusted_certs=None,updated_at=2023-11-29T03:10:39Z,user_data=None,user_id='6792e866677645c598ddb7d2c1723a1a',uuid=13e5da64-9adc-4479-81c7-d329e9b5f08f,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "2abf9e25-d1d9-4893-899e-912d41b7c38f", "address": "fa:16:3e:59:79:35", "network": {"id": "2b3c3140-f3b6-45da-8202-bb4d7f0dd130", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-2043420778-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [], "label": "tempest-ServerRescueNegativeTestJSON-2043420778-network", "vif_mac": "fa:16:3e:59:79:35"}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "f8a7884867c944e1ba1024270e4902f3", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2abf9e25-d1", "ovs_interfaceid": "2abf9e25-d1d9-4893-899e-912d41b7c38f", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=75234) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Nov 29 03:15:14 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-dcdf1c1e-c861-4abc-a20d-7a95695a845b tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Converting VIF {"id": "2abf9e25-d1d9-4893-899e-912d41b7c38f", "address": "fa:16:3e:59:79:35", "network": {"id": "2b3c3140-f3b6-45da-8202-bb4d7f0dd130", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-2043420778-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [], "label": "tempest-ServerRescueNegativeTestJSON-2043420778-network", "vif_mac": "fa:16:3e:59:79:35"}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "f8a7884867c944e1ba1024270e4902f3", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2abf9e25-d1", "ovs_interfaceid": "2abf9e25-d1d9-4893-899e-912d41b7c38f", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:15:14 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-dcdf1c1e-c861-4abc-a20d-7a95695a845b tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:59:79:35,bridge_name='br-int',has_traffic_filtering=True,id=2abf9e25-d1d9-4893-899e-912d41b7c38f,network=Network(2b3c3140-f3b6-45da-8202-bb4d7f0dd130),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap2abf9e25-d1') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:15:14 user nova-compute[75234]: DEBUG nova.objects.instance [None req-dcdf1c1e-c861-4abc-a20d-7a95695a845b tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Lazy-loading 'pci_devices' on Instance uuid 13e5da64-9adc-4479-81c7-d329e9b5f08f {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:15:14 user nova-compute[75234]: DEBUG nova.objects.base [None req-dcdf1c1e-c861-4abc-a20d-7a95695a845b tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Object Instance<13e5da64-9adc-4479-81c7-d329e9b5f08f> lazy-loaded attributes: trusted_certs,migration_context,resources,numa_topology,vcpu_model,pci_devices {{(pid=75234) wrapper /opt/stack/nova/nova/objects/base.py:126}}
Nov 29 03:15:14 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-dcdf1c1e-c861-4abc-a20d-7a95695a845b tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] [instance: 13e5da64-9adc-4479-81c7-d329e9b5f08f] End _get_guest_xml xml=<domain type="kvm">
Nov 29 03:15:14 user nova-compute[75234]:   <uuid>13e5da64-9adc-4479-81c7-d329e9b5f08f</uuid>
Nov 29 03:15:14 user nova-compute[75234]:   <name>instance-00000012</name>
Nov 29 03:15:14 user nova-compute[75234]:   <memory>524288</memory>
Nov 29 03:15:14 user nova-compute[75234]:   <vcpu>1</vcpu>
Nov 29 03:15:14 user nova-compute[75234]:   <metadata>
Nov 29 03:15:14 user nova-compute[75234]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Nov 29 03:15:14 user nova-compute[75234]:       <nova:package version="0.0.0"/>
Nov 29 03:15:14 user nova-compute[75234]:       <nova:name>tempest-ServerRescueNegativeTestJSON-server-715053465</nova:name>
Nov 29 03:15:14 user nova-compute[75234]:       <nova:creationTime>2023-11-29 03:15:14</nova:creationTime>
Nov 29 03:15:14 user nova-compute[75234]:       <nova:flavor name="m1.tiny">
Nov 29 03:15:14 user nova-compute[75234]:         <nova:memory>512</nova:memory>
Nov 29 03:15:14 user nova-compute[75234]:         <nova:disk>1</nova:disk>
Nov 29 03:15:14 user nova-compute[75234]:         <nova:swap>0</nova:swap>
Nov 29 03:15:14 user nova-compute[75234]:         <nova:ephemeral>0</nova:ephemeral>
Nov 29 03:15:14 user nova-compute[75234]:         <nova:vcpus>1</nova:vcpus>
Nov 29 03:15:14 user nova-compute[75234]:       </nova:flavor>
Nov 29 03:15:14 user nova-compute[75234]:       <nova:owner>
Nov 29 03:15:14 user nova-compute[75234]:         <nova:user uuid="6792e866677645c598ddb7d2c1723a1a">tempest-ServerRescueNegativeTestJSON-1182706601-project-member</nova:user>
Nov 29 03:15:14 user nova-compute[75234]:         <nova:project uuid="f8a7884867c944e1ba1024270e4902f3">tempest-ServerRescueNegativeTestJSON-1182706601</nova:project>
Nov 29 03:15:14 user nova-compute[75234]:       </nova:owner>
Nov 29 03:15:14 user nova-compute[75234]:       <nova:root type="image" uuid="96c30ba7-9b73-4a30-8ba4-7138879573b7"/>
Nov 29 03:15:14 user nova-compute[75234]:       <nova:ports>
Nov 29 03:15:14 user nova-compute[75234]:         <nova:port uuid="2abf9e25-d1d9-4893-899e-912d41b7c38f">
Nov 29 03:15:14 user nova-compute[75234]:           <nova:ip type="fixed" address="10.0.0.4" ipVersion="4"/>
Nov 29 03:15:14 user nova-compute[75234]:         </nova:port>
Nov 29 03:15:14 user nova-compute[75234]:       </nova:ports>
Nov 29 03:15:14 user nova-compute[75234]:     </nova:instance>
Nov 29 03:15:14 user nova-compute[75234]:   </metadata>
Nov 29 03:15:14 user nova-compute[75234]:   <sysinfo type="smbios">
Nov 29 03:15:14 user nova-compute[75234]:     <system>
Nov 29 03:15:14 user nova-compute[75234]:       <entry name="manufacturer">OpenStack Foundation</entry>
Nov 29 03:15:14 user nova-compute[75234]:       <entry name="product">OpenStack Nova</entry>
Nov 29 03:15:14 user nova-compute[75234]:       <entry name="version">0.0.0</entry>
Nov 29 03:15:14 user nova-compute[75234]:       <entry name="serial">13e5da64-9adc-4479-81c7-d329e9b5f08f</entry>
Nov 29 03:15:14 user nova-compute[75234]:       <entry name="uuid">13e5da64-9adc-4479-81c7-d329e9b5f08f</entry>
Nov 29 03:15:14 user nova-compute[75234]:       <entry name="family">Virtual Machine</entry>
Nov 29 03:15:14 user nova-compute[75234]:     </system>
Nov 29 03:15:14 user nova-compute[75234]:   </sysinfo>
Nov 29 03:15:14 user nova-compute[75234]:   <os>
Nov 29 03:15:14 user nova-compute[75234]:     <type arch="x86_64" machine="pc">hvm</type>
Nov 29 03:15:14 user nova-compute[75234]:     <smbios mode="sysinfo"/>
Nov 29 03:15:14 user nova-compute[75234]:   </os>
Nov 29 03:15:14 user nova-compute[75234]:   <features>
Nov 29 03:15:14 user nova-compute[75234]:     <acpi/>
Nov 29 03:15:14 user nova-compute[75234]:     <apic/>
Nov 29 03:15:14 user nova-compute[75234]:     <vmcoreinfo/>
Nov 29 03:15:14 user nova-compute[75234]:   </features>
Nov 29 03:15:14 user nova-compute[75234]:   <clock offset="utc">
Nov 29 03:15:14 user nova-compute[75234]:     <timer name="pit" tickpolicy="delay"/>
Nov 29 03:15:14 user nova-compute[75234]:     <timer name="rtc" tickpolicy="catchup"/>
Nov 29 03:15:14 user nova-compute[75234]:     <timer name="hpet" present="no"/>
Nov 29 03:15:14 user nova-compute[75234]:   </clock>
Nov 29 03:15:14 user nova-compute[75234]:   <cpu mode="custom" match="exact">
Nov 29 03:15:14 user nova-compute[75234]:     <model>Nehalem</model>
Nov 29 03:15:14 user nova-compute[75234]:     <topology sockets="1" cores="1" threads="1"/>
Nov 29 03:15:14 user nova-compute[75234]:   </cpu>
Nov 29 03:15:14 user nova-compute[75234]:   <devices>
Nov 29 03:15:14 user nova-compute[75234]:     <disk type="file" device="disk">
Nov 29 03:15:14 user nova-compute[75234]:       <driver name="qemu" type="qcow2" cache="none"/>
Nov 29 03:15:14 user nova-compute[75234]:       <source file="/opt/stack/data/nova/instances/13e5da64-9adc-4479-81c7-d329e9b5f08f/disk.rescue"/>
Nov 29 03:15:14 user nova-compute[75234]:       <target dev="vda" bus="virtio"/>
Nov 29 03:15:14 user nova-compute[75234]:     </disk>
Nov 29 03:15:14 user nova-compute[75234]:     <disk type="file" device="disk">
Nov 29 03:15:14 user nova-compute[75234]:       <driver name="qemu" type="qcow2" cache="none"/>
Nov 29 03:15:14 user nova-compute[75234]:       <source file="/opt/stack/data/nova/instances/13e5da64-9adc-4479-81c7-d329e9b5f08f/disk"/>
Nov 29 03:15:14 user nova-compute[75234]:       <target dev="vdb" bus="virtio"/>
Nov 29 03:15:14 user nova-compute[75234]:     </disk>
Nov 29 03:15:14 user nova-compute[75234]:     <interface type="ethernet">
Nov 29 03:15:14 user nova-compute[75234]:       <mac address="fa:16:3e:59:79:35"/>
Nov 29 03:15:14 user nova-compute[75234]:       <model type="virtio"/>
Nov 29 03:15:14 user nova-compute[75234]:       <mtu size="1442"/>
Nov 29 03:15:14 user nova-compute[75234]:       <target dev="tap2abf9e25-d1"/>
Nov 29 03:15:14 user nova-compute[75234]:     </interface>
Nov 29 03:15:14 user nova-compute[75234]:     <serial type="pty">
Nov 29 03:15:14 user nova-compute[75234]:       <log file="/opt/stack/data/nova/instances/13e5da64-9adc-4479-81c7-d329e9b5f08f/console.log" append="off"/>
Nov 29 03:15:14 user nova-compute[75234]:     </serial>
Nov 29 03:15:14 user nova-compute[75234]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Nov 29 03:15:14 user nova-compute[75234]:     <video>
Nov 29 03:15:14 user nova-compute[75234]:       <model type="virtio"/>
Nov 29 03:15:14 user nova-compute[75234]:     </video>
Nov 29 03:15:14 user nova-compute[75234]:     <rng model="virtio">
Nov 29 03:15:14 user nova-compute[75234]:       <backend model="random">/dev/urandom</backend>
Nov 29 03:15:14 user nova-compute[75234]:     </rng>
Nov 29 03:15:14 user nova-compute[75234]:     <controller type="usb" index="0" model="none"/>
Nov 29 03:15:14 user nova-compute[75234]:     <memballoon model="virtio">
Nov 29 03:15:14 user nova-compute[75234]:       <stats period="10"/>
Nov 29 03:15:14 user nova-compute[75234]:     </memballoon>
Nov 29 03:15:14 user nova-compute[75234]:   </devices>
Nov 29 03:15:14 user nova-compute[75234]: </domain>
Nov 29 03:15:14 user nova-compute[75234]:  {{(pid=75234) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7568}}
Nov 29 03:15:14 user nova-compute[75234]: INFO nova.virt.libvirt.driver [-] [instance: 13e5da64-9adc-4479-81c7-d329e9b5f08f] Instance destroyed successfully.
Nov 29 03:15:14 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-dcdf1c1e-c861-4abc-a20d-7a95695a845b tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] No BDM found with device name vda, not building metadata. {{(pid=75234) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12089}}
Nov 29 03:15:14 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-dcdf1c1e-c861-4abc-a20d-7a95695a845b tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] No BDM found with device name vdb, not building metadata. {{(pid=75234) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12089}}
Nov 29 03:15:14 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-dcdf1c1e-c861-4abc-a20d-7a95695a845b tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] No VIF found with MAC fa:16:3e:59:79:35, not building metadata {{(pid=75234) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Nov 29 03:15:16 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:15:16 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:15:16 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:15:16 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:15:16 user nova-compute[75234]: DEBUG nova.compute.manager [req-d5e29adc-7292-4f70-bcb4-83c3f04c17bc req-8585aaca-211b-47a3-83f3-baf7734dd870 service nova] [instance: 13e5da64-9adc-4479-81c7-d329e9b5f08f] Received event network-vif-plugged-2abf9e25-d1d9-4893-899e-912d41b7c38f {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:15:16 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-d5e29adc-7292-4f70-bcb4-83c3f04c17bc req-8585aaca-211b-47a3-83f3-baf7734dd870 service nova] Acquiring lock "13e5da64-9adc-4479-81c7-d329e9b5f08f-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:15:16 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-d5e29adc-7292-4f70-bcb4-83c3f04c17bc req-8585aaca-211b-47a3-83f3-baf7734dd870 service nova] Lock "13e5da64-9adc-4479-81c7-d329e9b5f08f-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:15:16 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-d5e29adc-7292-4f70-bcb4-83c3f04c17bc req-8585aaca-211b-47a3-83f3-baf7734dd870 service nova] Lock "13e5da64-9adc-4479-81c7-d329e9b5f08f-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:15:16 user nova-compute[75234]: DEBUG nova.compute.manager [req-d5e29adc-7292-4f70-bcb4-83c3f04c17bc req-8585aaca-211b-47a3-83f3-baf7734dd870 service nova] [instance: 13e5da64-9adc-4479-81c7-d329e9b5f08f] No waiting events found dispatching network-vif-plugged-2abf9e25-d1d9-4893-899e-912d41b7c38f {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:15:16 user nova-compute[75234]: WARNING nova.compute.manager [req-d5e29adc-7292-4f70-bcb4-83c3f04c17bc req-8585aaca-211b-47a3-83f3-baf7734dd870 service nova] [instance: 13e5da64-9adc-4479-81c7-d329e9b5f08f] Received unexpected event network-vif-plugged-2abf9e25-d1d9-4893-899e-912d41b7c38f for instance with vm_state active and task_state rescuing.
Nov 29 03:15:16 user nova-compute[75234]: DEBUG nova.compute.manager [req-d5e29adc-7292-4f70-bcb4-83c3f04c17bc req-8585aaca-211b-47a3-83f3-baf7734dd870 service nova] [instance: 13e5da64-9adc-4479-81c7-d329e9b5f08f] Received event network-vif-plugged-2abf9e25-d1d9-4893-899e-912d41b7c38f {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:15:16 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-d5e29adc-7292-4f70-bcb4-83c3f04c17bc req-8585aaca-211b-47a3-83f3-baf7734dd870 service nova] Acquiring lock "13e5da64-9adc-4479-81c7-d329e9b5f08f-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:15:16 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-d5e29adc-7292-4f70-bcb4-83c3f04c17bc req-8585aaca-211b-47a3-83f3-baf7734dd870 service nova] Lock "13e5da64-9adc-4479-81c7-d329e9b5f08f-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:15:16 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-d5e29adc-7292-4f70-bcb4-83c3f04c17bc req-8585aaca-211b-47a3-83f3-baf7734dd870 service nova] Lock "13e5da64-9adc-4479-81c7-d329e9b5f08f-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:15:16 user nova-compute[75234]: DEBUG nova.compute.manager [req-d5e29adc-7292-4f70-bcb4-83c3f04c17bc req-8585aaca-211b-47a3-83f3-baf7734dd870 service nova] [instance: 13e5da64-9adc-4479-81c7-d329e9b5f08f] No waiting events found dispatching network-vif-plugged-2abf9e25-d1d9-4893-899e-912d41b7c38f {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:15:16 user nova-compute[75234]: WARNING nova.compute.manager [req-d5e29adc-7292-4f70-bcb4-83c3f04c17bc req-8585aaca-211b-47a3-83f3-baf7734dd870 service nova] [instance: 13e5da64-9adc-4479-81c7-d329e9b5f08f] Received unexpected event network-vif-plugged-2abf9e25-d1d9-4893-899e-912d41b7c38f for instance with vm_state active and task_state rescuing.
Nov 29 03:15:16 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:15:16 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:15:16 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:15:18 user nova-compute[75234]: DEBUG nova.compute.manager [None req-dcdf1c1e-c861-4abc-a20d-7a95695a845b tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] [instance: 13e5da64-9adc-4479-81c7-d329e9b5f08f] Checking state {{(pid=75234) _get_power_state /opt/stack/nova/nova/compute/manager.py:1782}}
Nov 29 03:15:18 user nova-compute[75234]: DEBUG nova.compute.manager [req-67d89f78-efae-4563-8d49-2cfa930d4d94 req-9b826465-c821-44eb-b6b2-71dd6f0ff309 service nova] [instance: 13e5da64-9adc-4479-81c7-d329e9b5f08f] Received event network-vif-plugged-2abf9e25-d1d9-4893-899e-912d41b7c38f {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:15:18 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-67d89f78-efae-4563-8d49-2cfa930d4d94 req-9b826465-c821-44eb-b6b2-71dd6f0ff309 service nova] Acquiring lock "13e5da64-9adc-4479-81c7-d329e9b5f08f-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:15:18 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-67d89f78-efae-4563-8d49-2cfa930d4d94 req-9b826465-c821-44eb-b6b2-71dd6f0ff309 service nova] Lock "13e5da64-9adc-4479-81c7-d329e9b5f08f-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:15:18 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-67d89f78-efae-4563-8d49-2cfa930d4d94 req-9b826465-c821-44eb-b6b2-71dd6f0ff309 service nova] Lock "13e5da64-9adc-4479-81c7-d329e9b5f08f-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:15:18 user nova-compute[75234]: DEBUG nova.compute.manager [req-67d89f78-efae-4563-8d49-2cfa930d4d94 req-9b826465-c821-44eb-b6b2-71dd6f0ff309 service nova] [instance: 13e5da64-9adc-4479-81c7-d329e9b5f08f] No waiting events found dispatching network-vif-plugged-2abf9e25-d1d9-4893-899e-912d41b7c38f {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:15:18 user nova-compute[75234]: WARNING nova.compute.manager [req-67d89f78-efae-4563-8d49-2cfa930d4d94 req-9b826465-c821-44eb-b6b2-71dd6f0ff309 service nova] [instance: 13e5da64-9adc-4479-81c7-d329e9b5f08f] Received unexpected event network-vif-plugged-2abf9e25-d1d9-4893-899e-912d41b7c38f for instance with vm_state rescued and task_state None.
Nov 29 03:15:19 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:15:19 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-7044ba48-4a47-4ac0-bc84-6d4b709485c4 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Acquiring lock "ec7cf7cd-8e48-4bb3-a969-e14db705dc09" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:15:19 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-7044ba48-4a47-4ac0-bc84-6d4b709485c4 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Lock "ec7cf7cd-8e48-4bb3-a969-e14db705dc09" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:15:19 user nova-compute[75234]: DEBUG nova.compute.manager [None req-7044ba48-4a47-4ac0-bc84-6d4b709485c4 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] [instance: ec7cf7cd-8e48-4bb3-a969-e14db705dc09] Starting instance... {{(pid=75234) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2422}}
Nov 29 03:15:19 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-7044ba48-4a47-4ac0-bc84-6d4b709485c4 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:15:19 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-7044ba48-4a47-4ac0-bc84-6d4b709485c4 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:15:19 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-7044ba48-4a47-4ac0-bc84-6d4b709485c4 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=75234) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2366}}
Nov 29 03:15:19 user nova-compute[75234]: INFO nova.compute.claims [None req-7044ba48-4a47-4ac0-bc84-6d4b709485c4 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] [instance: ec7cf7cd-8e48-4bb3-a969-e14db705dc09] Claim successful on node user
Nov 29 03:15:19 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-216f0360-decd-4f80-a056-6c865d01c068 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Waiting for libvirt event about the detach of device vdb with device alias virtio-disk1 from instance 6894b55f-93c6-43c9-8933-56d5a3c43725 is timed out.
Nov 29 03:15:19 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-216f0360-decd-4f80-a056-6c865d01c068 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Successfully detached device vdb from instance 6894b55f-93c6-43c9-8933-56d5a3c43725 from the live domain config.
Nov 29 03:15:19 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-216f0360-decd-4f80-a056-6c865d01c068 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] [instance: 6894b55f-93c6-43c9-8933-56d5a3c43725] calling os-brick to detach iSCSI Volume {{(pid=75234) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Nov 29 03:15:19 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-216f0360-decd-4f80-a056-6c865d01c068 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f8a5050c9a0>, {'target_iqn': 'iqn.2010-10.org.openstack:104bcac8-98ce-42f8-a85f-82e29050a44c', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'oxmZWJRF', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdb'}, None), 'kwargs': False}}" {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:176}}
Nov 29 03:15:19 user nova-compute[75234]: WARNING os_brick.initiator.connectors.base [None req-216f0360-decd-4f80-a056-6c865d01c068 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Nov 29 03:15:19 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-216f0360-decd-4f80-a056-6c865d01c068 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:68}}
Nov 29 03:15:19 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-216f0360-decd-4f80-a056-6c865d01c068 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:73}}
Nov 29 03:15:19 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:15:19 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.020s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:15:19 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[aee7890a-c7e8-404e-ae6f-965b11bdd2e4]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:104bcac8-98ce-42f8-a85f-82e29050a44c\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:15:19 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-216f0360-decd-4f80-a056-6c865d01c068 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS:
Nov 29 03:15:19 user nova-compute[75234]: No targets found.
Nov 29 03:15:19 user nova-compute[75234]: iSNS:
Nov 29 03:15:19 user nova-compute[75234]: No targets found.
Nov 29 03:15:19 user nova-compute[75234]: STATIC:
Nov 29 03:15:19 user nova-compute[75234]: Target: iqn.2010-10.org.openstack:104bcac8-98ce-42f8-a85f-82e29050a44c
Nov 29 03:15:19 user nova-compute[75234]:         Portal: 172.16.0.220:3260,-1
Nov 29 03:15:19 user nova-compute[75234]:                 Iface Name: default
Nov 29 03:15:19 user nova-compute[75234]: FIRMWARE:
Nov 29 03:15:19 user nova-compute[75234]: No targets found.
Nov 29 03:15:19 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:15:19 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-216f0360-decd-4f80-a056-6c865d01c068 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Regex to get portals from discoverydb: ^SENDTARGETS:
Nov 29 03:15:19 user nova-compute[75234]: .*?^DiscoveryAddress: 172.16.0.220,3260.*?
Nov 29 03:15:19 user nova-compute[75234]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=75234) _get_discoverydb_portals /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:382}}
Nov 29 03:15:19 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-216f0360-decd-4f80-a056-6c865d01c068 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:104bcac8-98ce-42f8-a85f-82e29050a44c', 0)] {{(pid=75234) _get_connection_devices /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:830}}
Nov 29 03:15:19 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:15:19 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.017s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:15:19 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[02261192-9ca2-4ec7-a3cc-155cf3422134]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:104bcac8-98ce-42f8-a85f-82e29050a44c\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:15:19 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:15:19 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.019s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:15:19 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[9c7df052-3389-49be-8ea6-f79fd92ef408]: (4, ('tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:104bcac8-98ce-42f8-a85f-82e29050a44c (non-flash)\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:15:19 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-216f0360-decd-4f80-a056-6c865d01c068 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:104bcac8-98ce-42f8-a85f-82e29050a44c (non-flash)
Nov 29 03:15:19 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:15:19 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-216f0360-decd-4f80-a056-6c865d01c068 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] iscsi session list stdout=tcp: [26] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:104bcac8-98ce-42f8-a85f-82e29050a44c (non-flash)
Nov 29 03:15:19 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsi_session /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Nov 29 03:15:19 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-216f0360-decd-4f80-a056-6c865d01c068 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7f8a50626320>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:104bcac8-98ce-42f8-a85f-82e29050a44c'): ({'sdb'}, set())}) {{(pid=75234) _get_connection_devices /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:863}}
Nov 29 03:15:19 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-216f0360-decd-4f80-a056-6c865d01c068 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Removing single pathed devices sdb {{(pid=75234) remove_connection /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:374}}
Nov 29 03:15:19 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:15:19 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.022s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:15:19 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[25b469f6-9b62-47ac-bbdd-facdcb5c7763]: (4, ('path checker states:\nup                  1\n\npaths: 0\nbusy: False\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:15:19 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdb {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:15:19 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdb" returned: 0 in 0.020s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:15:19 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[3e53136e-fd05-4a63-a869-721a575366cb]: (4, ('ok\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:15:19 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-216f0360-decd-4f80-a056-6c865d01c068 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Flushing IO for device /dev/sdb {{(pid=75234) flush_device_io /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:434}}
Nov 29 03:15:19 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdb {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:15:20 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdb" returned: 0 in 0.021s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:15:20 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[3f82b6f9-dc6e-4138-855f-40a4ea099245]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:15:20 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-216f0360-decd-4f80-a056-6c865d01c068 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Remove SCSI device /dev/sdb with /sys/block/sdb/device/delete {{(pid=75234) remove_scsi_device /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:148}}
Nov 29 03:15:20 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdb/device/delete {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:15:20 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdb/device/delete" returned: 0 in 0.047s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:15:20 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[cb3032ef-de89-4d59-a41d-a10cbc147ebe]: (4, ('1', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:15:20 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-216f0360-decd-4f80-a056-6c865d01c068 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Checking to see if SCSI volumes sdb have been removed. {{(pid=75234) wait_for_volumes_removal /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:156}}
Nov 29 03:15:20 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-216f0360-decd-4f80-a056-6c865d01c068 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] SCSI volumes sdb have been removed. {{(pid=75234) wait_for_volumes_removal /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:166}}
Nov 29 03:15:20 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-216f0360-decd-4f80-a056-6c865d01c068 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:104bcac8-98ce-42f8-a85f-82e29050a44c')] {{(pid=75234) _disconnect_connection /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Nov 29 03:15:20 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:104bcac8-98ce-42f8-a85f-82e29050a44c -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:15:20 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:104bcac8-98ce-42f8-a85f-82e29050a44c -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.027s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:15:20 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[3a5d5bd3-b9d0-4bc6-820e-3c63557e0a85]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:15:20 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-216f0360-decd-4f80-a056-6c865d01c068 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:15:20 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:104bcac8-98ce-42f8-a85f-82e29050a44c -p 172.16.0.220:3260 --logout {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:15:20 user nova-compute[75234]: DEBUG nova.compute.provider_tree [None req-7044ba48-4a47-4ac0-bc84-6d4b709485c4 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Inventory has not changed in ProviderTree for provider: 4d291220-7c37-4839-a0e3-b23c66502b74 {{(pid=75234) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Nov 29 03:15:20 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:104bcac8-98ce-42f8-a85f-82e29050a44c -p 172.16.0.220:3260 --logout" returned: 0 in 0.053s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:15:20 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[0d2ca264-5636-4550-b628-ccb8df60ff40]: (4, ('Logging out of session [sid: 26, target: iqn.2010-10.org.openstack:104bcac8-98ce-42f8-a85f-82e29050a44c, portal: 172.16.0.220,3260]\nLogout of [sid: 26, target: iqn.2010-10.org.openstack:104bcac8-98ce-42f8-a85f-82e29050a44c, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:15:20 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-216f0360-decd-4f80-a056-6c865d01c068 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 26, target: iqn.2010-10.org.openstack:104bcac8-98ce-42f8-a85f-82e29050a44c, portal: 172.16.0.220,3260]
Nov 29 03:15:20 user nova-compute[75234]: Logout of [sid: 26, target: iqn.2010-10.org.openstack:104bcac8-98ce-42f8-a85f-82e29050a44c, portal: 172.16.0.220,3260] successful.
Nov 29 03:15:20 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:15:20 user nova-compute[75234]: DEBUG nova.scheduler.client.report [None req-7044ba48-4a47-4ac0-bc84-6d4b709485c4 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Inventory has not changed for provider 4d291220-7c37-4839-a0e3-b23c66502b74 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=75234) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Nov 29 03:15:20 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:104bcac8-98ce-42f8-a85f-82e29050a44c -p 172.16.0.220:3260 --op delete {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:15:20 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:104bcac8-98ce-42f8-a85f-82e29050a44c -p 172.16.0.220:3260 --op delete" returned: 0 in 0.022s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:15:20 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[b9cd3cc8-0168-406a-916a-b2c8752ea30a]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:15:20 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-216f0360-decd-4f80-a056-6c865d01c068 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:15:20 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-216f0360-decd-4f80-a056-6c865d01c068 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.334s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:87}}
Nov 29 03:15:20 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-216f0360-decd-4f80-a056-6c865d01c068 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] <== disconnect_volume: return (335ms) None {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:203}}
Nov 29 03:15:20 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-216f0360-decd-4f80-a056-6c865d01c068 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] [instance: 6894b55f-93c6-43c9-8933-56d5a3c43725] Disconnected iSCSI Volume {{(pid=75234) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:79}}
Nov 29 03:15:20 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-7044ba48-4a47-4ac0-bc84-6d4b709485c4 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.416s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:15:20 user nova-compute[75234]: DEBUG nova.compute.manager [None req-7044ba48-4a47-4ac0-bc84-6d4b709485c4 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] [instance: ec7cf7cd-8e48-4bb3-a969-e14db705dc09] Start building networks asynchronously for instance. {{(pid=75234) _build_resources /opt/stack/nova/nova/compute/manager.py:2819}}
Nov 29 03:15:20 user nova-compute[75234]: DEBUG nova.compute.manager [None req-7044ba48-4a47-4ac0-bc84-6d4b709485c4 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] [instance: ec7cf7cd-8e48-4bb3-a969-e14db705dc09] Allocating IP information in the background. {{(pid=75234) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1972}}
Nov 29 03:15:20 user nova-compute[75234]: DEBUG nova.network.neutron [None req-7044ba48-4a47-4ac0-bc84-6d4b709485c4 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] [instance: ec7cf7cd-8e48-4bb3-a969-e14db705dc09] allocate_for_instance() {{(pid=75234) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1156}}
Nov 29 03:15:20 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-7044ba48-4a47-4ac0-bc84-6d4b709485c4 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] [instance: ec7cf7cd-8e48-4bb3-a969-e14db705dc09] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Nov 29 03:15:20 user nova-compute[75234]: DEBUG nova.compute.manager [None req-7044ba48-4a47-4ac0-bc84-6d4b709485c4 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] [instance: ec7cf7cd-8e48-4bb3-a969-e14db705dc09] Start building block device mappings for instance. {{(pid=75234) _build_resources /opt/stack/nova/nova/compute/manager.py:2854}}
Nov 29 03:15:20 user nova-compute[75234]: DEBUG nova.policy [None req-7044ba48-4a47-4ac0-bc84-6d4b709485c4 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '899f638d0de34dcba53f4c2ef7f5e068', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e24dd8accea4c1e9598d643ea79c86d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=75234) authorize /opt/stack/nova/nova/policy.py:203}}
Nov 29 03:15:20 user nova-compute[75234]: DEBUG nova.compute.manager [None req-7044ba48-4a47-4ac0-bc84-6d4b709485c4 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] [instance: ec7cf7cd-8e48-4bb3-a969-e14db705dc09] Start spawning the instance on the hypervisor. {{(pid=75234) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2628}}
Nov 29 03:15:20 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-7044ba48-4a47-4ac0-bc84-6d4b709485c4 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] [instance: ec7cf7cd-8e48-4bb3-a969-e14db705dc09] Creating instance directory {{(pid=75234) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4727}}
Nov 29 03:15:20 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-7044ba48-4a47-4ac0-bc84-6d4b709485c4 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] [instance: ec7cf7cd-8e48-4bb3-a969-e14db705dc09] Creating image(s)
Nov 29 03:15:20 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-7044ba48-4a47-4ac0-bc84-6d4b709485c4 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Acquiring lock "/opt/stack/data/nova/instances/ec7cf7cd-8e48-4bb3-a969-e14db705dc09/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:15:20 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-7044ba48-4a47-4ac0-bc84-6d4b709485c4 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Lock "/opt/stack/data/nova/instances/ec7cf7cd-8e48-4bb3-a969-e14db705dc09/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:15:20 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-7044ba48-4a47-4ac0-bc84-6d4b709485c4 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Lock "/opt/stack/data/nova/instances/ec7cf7cd-8e48-4bb3-a969-e14db705dc09/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.002s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:15:20 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-7044ba48-4a47-4ac0-bc84-6d4b709485c4 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:15:20 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-7044ba48-4a47-4ac0-bc84-6d4b709485c4 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json" returned: 0 in 0.223s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:15:20 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-7044ba48-4a47-4ac0-bc84-6d4b709485c4 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Acquiring lock "80686e29dc9db6600d03e85cb62af298e12d7918" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:15:20 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-7044ba48-4a47-4ac0-bc84-6d4b709485c4 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Lock "80686e29dc9db6600d03e85cb62af298e12d7918" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.002s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:15:20 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-7044ba48-4a47-4ac0-bc84-6d4b709485c4 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:15:20 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-7044ba48-4a47-4ac0-bc84-6d4b709485c4 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json" returned: 0 in 0.210s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:15:20 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-7044ba48-4a47-4ac0-bc84-6d4b709485c4 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918,backing_fmt=raw /opt/stack/data/nova/instances/ec7cf7cd-8e48-4bb3-a969-e14db705dc09/disk 1073741824 {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:15:21 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-7044ba48-4a47-4ac0-bc84-6d4b709485c4 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918,backing_fmt=raw /opt/stack/data/nova/instances/ec7cf7cd-8e48-4bb3-a969-e14db705dc09/disk 1073741824" returned: 0 in 0.055s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:15:21 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-7044ba48-4a47-4ac0-bc84-6d4b709485c4 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Lock "80686e29dc9db6600d03e85cb62af298e12d7918" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.276s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:15:21 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-7044ba48-4a47-4ac0-bc84-6d4b709485c4 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:15:21 user nova-compute[75234]: DEBUG nova.objects.instance [None req-216f0360-decd-4f80-a056-6c865d01c068 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Lazy-loading 'flavor' on Instance uuid 6894b55f-93c6-43c9-8933-56d5a3c43725 {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:15:21 user nova-compute[75234]: DEBUG nova.network.neutron [None req-7044ba48-4a47-4ac0-bc84-6d4b709485c4 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] [instance: ec7cf7cd-8e48-4bb3-a969-e14db705dc09] Successfully created port: 9d89f22e-e411-461b-b9b3-85dbcf58af43 {{(pid=75234) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:548}}
Nov 29 03:15:21 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-216f0360-decd-4f80-a056-6c865d01c068 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Lock "6894b55f-93c6-43c9-8933-56d5a3c43725" "released" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: held 21.899s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:15:21 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-7044ba48-4a47-4ac0-bc84-6d4b709485c4 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json" returned: 0 in 0.193s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:15:21 user nova-compute[75234]: DEBUG nova.virt.disk.api [None req-7044ba48-4a47-4ac0-bc84-6d4b709485c4 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Checking if we can resize image /opt/stack/data/nova/instances/ec7cf7cd-8e48-4bb3-a969-e14db705dc09/disk. size=1073741824 {{(pid=75234) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:166}}
Nov 29 03:15:21 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-7044ba48-4a47-4ac0-bc84-6d4b709485c4 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/ec7cf7cd-8e48-4bb3-a969-e14db705dc09/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:15:21 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-7044ba48-4a47-4ac0-bc84-6d4b709485c4 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/ec7cf7cd-8e48-4bb3-a969-e14db705dc09/disk --force-share --output=json" returned: 0 in 0.191s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:15:21 user nova-compute[75234]: DEBUG nova.virt.disk.api [None req-7044ba48-4a47-4ac0-bc84-6d4b709485c4 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Cannot resize image /opt/stack/data/nova/instances/ec7cf7cd-8e48-4bb3-a969-e14db705dc09/disk to a smaller size. {{(pid=75234) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:172}}
Nov 29 03:15:21 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-7044ba48-4a47-4ac0-bc84-6d4b709485c4 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] [instance: ec7cf7cd-8e48-4bb3-a969-e14db705dc09] Created local disks {{(pid=75234) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4861}}
Nov 29 03:15:21 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-7044ba48-4a47-4ac0-bc84-6d4b709485c4 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] [instance: ec7cf7cd-8e48-4bb3-a969-e14db705dc09] Ensure instance console log exists: /opt/stack/data/nova/instances/ec7cf7cd-8e48-4bb3-a969-e14db705dc09/console.log {{(pid=75234) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4613}}
Nov 29 03:15:21 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-7044ba48-4a47-4ac0-bc84-6d4b709485c4 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:15:21 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-7044ba48-4a47-4ac0-bc84-6d4b709485c4 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:15:21 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-7044ba48-4a47-4ac0-bc84-6d4b709485c4 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:15:21 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:15:21 user nova-compute[75234]: DEBUG nova.network.neutron [None req-7044ba48-4a47-4ac0-bc84-6d4b709485c4 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] [instance: ec7cf7cd-8e48-4bb3-a969-e14db705dc09] Successfully updated port: 9d89f22e-e411-461b-b9b3-85dbcf58af43 {{(pid=75234) _update_port /opt/stack/nova/nova/network/neutron.py:586}}
Nov 29 03:15:21 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-7044ba48-4a47-4ac0-bc84-6d4b709485c4 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Acquiring lock "refresh_cache-ec7cf7cd-8e48-4bb3-a969-e14db705dc09" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:15:21 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-7044ba48-4a47-4ac0-bc84-6d4b709485c4 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Acquired lock "refresh_cache-ec7cf7cd-8e48-4bb3-a969-e14db705dc09" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:15:21 user nova-compute[75234]: DEBUG nova.network.neutron [None req-7044ba48-4a47-4ac0-bc84-6d4b709485c4 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] [instance: ec7cf7cd-8e48-4bb3-a969-e14db705dc09] Building network info cache for instance {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2009}}
Nov 29 03:15:21 user nova-compute[75234]: DEBUG nova.compute.manager [req-375f071e-590c-4a0a-b770-cc5a4aacb758 req-4828b42d-c7ac-4582-8369-476ba16b350b service nova] [instance: ec7cf7cd-8e48-4bb3-a969-e14db705dc09] Received event network-changed-9d89f22e-e411-461b-b9b3-85dbcf58af43 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:15:21 user nova-compute[75234]: DEBUG nova.compute.manager [req-375f071e-590c-4a0a-b770-cc5a4aacb758 req-4828b42d-c7ac-4582-8369-476ba16b350b service nova] [instance: ec7cf7cd-8e48-4bb3-a969-e14db705dc09] Refreshing instance network info cache due to event network-changed-9d89f22e-e411-461b-b9b3-85dbcf58af43. {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11117}}
Nov 29 03:15:21 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-375f071e-590c-4a0a-b770-cc5a4aacb758 req-4828b42d-c7ac-4582-8369-476ba16b350b service nova] Acquiring lock "refresh_cache-ec7cf7cd-8e48-4bb3-a969-e14db705dc09" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:15:21 user nova-compute[75234]: DEBUG nova.network.neutron [None req-7044ba48-4a47-4ac0-bc84-6d4b709485c4 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] [instance: ec7cf7cd-8e48-4bb3-a969-e14db705dc09] Instance cache missing network info. {{(pid=75234) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3322}}
Nov 29 03:15:22 user nova-compute[75234]: DEBUG nova.network.neutron [None req-7044ba48-4a47-4ac0-bc84-6d4b709485c4 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] [instance: ec7cf7cd-8e48-4bb3-a969-e14db705dc09] Updating instance_info_cache with network_info: [{"id": "9d89f22e-e411-461b-b9b3-85dbcf58af43", "address": "fa:16:3e:30:11:ff", "network": {"id": "1898605d-00e7-4e81-964e-821e09f5421c", "bridge": "br-int", "label": "tempest-VolumesSnapshotTestJSON-1896954656-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "7e24dd8accea4c1e9598d643ea79c86d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap9d89f22e-e4", "ovs_interfaceid": "9d89f22e-e411-461b-b9b3-85dbcf58af43", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:15:22 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-7044ba48-4a47-4ac0-bc84-6d4b709485c4 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Releasing lock "refresh_cache-ec7cf7cd-8e48-4bb3-a969-e14db705dc09" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:15:22 user nova-compute[75234]: DEBUG nova.compute.manager [None req-7044ba48-4a47-4ac0-bc84-6d4b709485c4 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] [instance: ec7cf7cd-8e48-4bb3-a969-e14db705dc09] Instance network_info: |[{"id": "9d89f22e-e411-461b-b9b3-85dbcf58af43", "address": "fa:16:3e:30:11:ff", "network": {"id": "1898605d-00e7-4e81-964e-821e09f5421c", "bridge": "br-int", "label": "tempest-VolumesSnapshotTestJSON-1896954656-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "7e24dd8accea4c1e9598d643ea79c86d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap9d89f22e-e4", "ovs_interfaceid": "9d89f22e-e411-461b-b9b3-85dbcf58af43", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=75234) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1987}}
Nov 29 03:15:22 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-375f071e-590c-4a0a-b770-cc5a4aacb758 req-4828b42d-c7ac-4582-8369-476ba16b350b service nova] Acquired lock "refresh_cache-ec7cf7cd-8e48-4bb3-a969-e14db705dc09" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:15:22 user nova-compute[75234]: DEBUG nova.network.neutron [req-375f071e-590c-4a0a-b770-cc5a4aacb758 req-4828b42d-c7ac-4582-8369-476ba16b350b service nova] [instance: ec7cf7cd-8e48-4bb3-a969-e14db705dc09] Refreshing network info cache for port 9d89f22e-e411-461b-b9b3-85dbcf58af43 {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2006}}
Nov 29 03:15:22 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-7044ba48-4a47-4ac0-bc84-6d4b709485c4 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] [instance: ec7cf7cd-8e48-4bb3-a969-e14db705dc09] Start _get_guest_xml network_info=[{"id": "9d89f22e-e411-461b-b9b3-85dbcf58af43", "address": "fa:16:3e:30:11:ff", "network": {"id": "1898605d-00e7-4e81-964e-821e09f5421c", "bridge": "br-int", "label": "tempest-VolumesSnapshotTestJSON-1896954656-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "7e24dd8accea4c1e9598d643ea79c86d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap9d89f22e-e4", "ovs_interfaceid": "9d89f22e-e411-461b-b9b3-85dbcf58af43", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum='c8fc807773e5354afe61636071771906',container_format='bare',created_at=2023-11-29T03:02:15Z,direct_url=<?>,disk_format='qcow2',id=96c30ba7-9b73-4a30-8ba4-7138879573b7,min_disk=0,min_ram=0,name='cirros-0.6.2-x86_64-disk',owner='b18e380921c748b78517b6f1211cd0f1',properties=ImageMetaProps,protected=<?>,size=21430272,status='active',tags=<?>,updated_at=2023-11-29T03:02:17Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [{'device_name': '/dev/vda', 'encryption_format': None, 'encrypted': False, 'device_type': 'disk', 'size': 0, 'disk_bus': 'virtio', 'boot_index': 0, 'encryption_options': None, 'encryption_secret_uuid': None, 'guest_format': None, 'image_id': '96c30ba7-9b73-4a30-8ba4-7138879573b7'}], 'ephemerals': [], 'block_device_mapping': [], 'swap': None} {{(pid=75234) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7562}}
Nov 29 03:15:22 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-7044ba48-4a47-4ac0-bc84-6d4b709485c4 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:15:22 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-7044ba48-4a47-4ac0-bc84-6d4b709485c4 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:15:22 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-7044ba48-4a47-4ac0-bc84-6d4b709485c4 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Searching host: 'user' for CPU controller through CGroups V1... {{(pid=75234) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1652}}
Nov 29 03:15:22 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-7044ba48-4a47-4ac0-bc84-6d4b709485c4 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] CPU controller missing on host. {{(pid=75234) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1662}}
Nov 29 03:15:22 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-7044ba48-4a47-4ac0-bc84-6d4b709485c4 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Searching host: 'user' for CPU controller through CGroups V2... {{(pid=75234) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1671}}
Nov 29 03:15:22 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-7044ba48-4a47-4ac0-bc84-6d4b709485c4 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] CPU controller found on host. {{(pid=75234) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1678}}
Nov 29 03:15:22 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-7044ba48-4a47-4ac0-bc84-6d4b709485c4 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=75234) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5400}}
Nov 29 03:15:22 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-7044ba48-4a47-4ac0-bc84-6d4b709485c4 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-11-29T03:02:04Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='1',id=6,is_public=True,memory_mb=512,name='m1.tiny',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='c8fc807773e5354afe61636071771906',container_format='bare',created_at=2023-11-29T03:02:15Z,direct_url=<?>,disk_format='qcow2',id=96c30ba7-9b73-4a30-8ba4-7138879573b7,min_disk=0,min_ram=0,name='cirros-0.6.2-x86_64-disk',owner='b18e380921c748b78517b6f1211cd0f1',properties=ImageMetaProps,protected=<?>,size=21430272,status='active',tags=<?>,updated_at=2023-11-29T03:02:17Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:563}}
Nov 29 03:15:22 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-7044ba48-4a47-4ac0-bc84-6d4b709485c4 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Flavor limits 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:348}}
Nov 29 03:15:22 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-7044ba48-4a47-4ac0-bc84-6d4b709485c4 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Image limits 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:352}}
Nov 29 03:15:22 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-7044ba48-4a47-4ac0-bc84-6d4b709485c4 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Flavor pref 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:388}}
Nov 29 03:15:22 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-7044ba48-4a47-4ac0-bc84-6d4b709485c4 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Image pref 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:392}}
Nov 29 03:15:22 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-7044ba48-4a47-4ac0-bc84-6d4b709485c4 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:430}}
Nov 29 03:15:22 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-7044ba48-4a47-4ac0-bc84-6d4b709485c4 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:569}}
Nov 29 03:15:22 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-7044ba48-4a47-4ac0-bc84-6d4b709485c4 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=75234) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:471}}
Nov 29 03:15:22 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-7044ba48-4a47-4ac0-bc84-6d4b709485c4 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Got 1 possible topologies {{(pid=75234) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:501}}
Nov 29 03:15:22 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-7044ba48-4a47-4ac0-bc84-6d4b709485c4 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:575}}
Nov 29 03:15:22 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-7044ba48-4a47-4ac0-bc84-6d4b709485c4 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:577}}
Nov 29 03:15:22 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-7044ba48-4a47-4ac0-bc84-6d4b709485c4 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-11-29T03:15:19Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-VolumesSnapshotTestJSON-instance-213375111',display_name='tempest-VolumesSnapshotTestJSON-instance-213375111',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-volumessnapshottestjson-instance-213375111',id=23,image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBKfdFQdP9b2CUucLMguc8tjFgkfx70MU+FbCMNYtTpv3yP6N/lf+rZlWO4aVrwdI3Q2J0as5mFJ1RqaTpNerkZvmi2PeYojCi8bJzCIaL5bklATBcDaVb7EKkDOmV8PS5Q==',key_name='tempest-keypair-798189340',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='7e24dd8accea4c1e9598d643ea79c86d',ramdisk_id='',reservation_id='r-mjtjrmkx',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-VolumesSnapshotTestJSON-902320119',owner_user_name='tempest-VolumesSnapshotTestJSON-902320119-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-11-29T03:15:20Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='899f638d0de34dcba53f4c2ef7f5e068',uuid=ec7cf7cd-8e48-4bb3-a969-e14db705dc09,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "9d89f22e-e411-461b-b9b3-85dbcf58af43", "address": "fa:16:3e:30:11:ff", "network": {"id": "1898605d-00e7-4e81-964e-821e09f5421c", "bridge": "br-int", "label": "tempest-VolumesSnapshotTestJSON-1896954656-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "7e24dd8accea4c1e9598d643ea79c86d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap9d89f22e-e4", "ovs_interfaceid": "9d89f22e-e411-461b-b9b3-85dbcf58af43", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=75234) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Nov 29 03:15:22 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-7044ba48-4a47-4ac0-bc84-6d4b709485c4 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Converting VIF {"id": "9d89f22e-e411-461b-b9b3-85dbcf58af43", "address": "fa:16:3e:30:11:ff", "network": {"id": "1898605d-00e7-4e81-964e-821e09f5421c", "bridge": "br-int", "label": "tempest-VolumesSnapshotTestJSON-1896954656-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "7e24dd8accea4c1e9598d643ea79c86d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap9d89f22e-e4", "ovs_interfaceid": "9d89f22e-e411-461b-b9b3-85dbcf58af43", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:15:22 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-7044ba48-4a47-4ac0-bc84-6d4b709485c4 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:30:11:ff,bridge_name='br-int',has_traffic_filtering=True,id=9d89f22e-e411-461b-b9b3-85dbcf58af43,network=Network(1898605d-00e7-4e81-964e-821e09f5421c),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap9d89f22e-e4') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:15:22 user nova-compute[75234]: DEBUG nova.objects.instance [None req-7044ba48-4a47-4ac0-bc84-6d4b709485c4 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Lazy-loading 'pci_devices' on Instance uuid ec7cf7cd-8e48-4bb3-a969-e14db705dc09 {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:15:22 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-7044ba48-4a47-4ac0-bc84-6d4b709485c4 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] [instance: ec7cf7cd-8e48-4bb3-a969-e14db705dc09] End _get_guest_xml xml=<domain type="kvm">
Nov 29 03:15:22 user nova-compute[75234]:   <uuid>ec7cf7cd-8e48-4bb3-a969-e14db705dc09</uuid>
Nov 29 03:15:22 user nova-compute[75234]:   <name>instance-00000017</name>
Nov 29 03:15:22 user nova-compute[75234]:   <memory>524288</memory>
Nov 29 03:15:22 user nova-compute[75234]:   <vcpu>1</vcpu>
Nov 29 03:15:22 user nova-compute[75234]:   <metadata>
Nov 29 03:15:22 user nova-compute[75234]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Nov 29 03:15:22 user nova-compute[75234]:       <nova:package version="0.0.0"/>
Nov 29 03:15:22 user nova-compute[75234]:       <nova:name>tempest-VolumesSnapshotTestJSON-instance-213375111</nova:name>
Nov 29 03:15:22 user nova-compute[75234]:       <nova:creationTime>2023-11-29 03:15:22</nova:creationTime>
Nov 29 03:15:22 user nova-compute[75234]:       <nova:flavor name="m1.tiny">
Nov 29 03:15:22 user nova-compute[75234]:         <nova:memory>512</nova:memory>
Nov 29 03:15:22 user nova-compute[75234]:         <nova:disk>1</nova:disk>
Nov 29 03:15:22 user nova-compute[75234]:         <nova:swap>0</nova:swap>
Nov 29 03:15:22 user nova-compute[75234]:         <nova:ephemeral>0</nova:ephemeral>
Nov 29 03:15:22 user nova-compute[75234]:         <nova:vcpus>1</nova:vcpus>
Nov 29 03:15:22 user nova-compute[75234]:       </nova:flavor>
Nov 29 03:15:22 user nova-compute[75234]:       <nova:owner>
Nov 29 03:15:22 user nova-compute[75234]:         <nova:user uuid="899f638d0de34dcba53f4c2ef7f5e068">tempest-VolumesSnapshotTestJSON-902320119-project-member</nova:user>
Nov 29 03:15:22 user nova-compute[75234]:         <nova:project uuid="7e24dd8accea4c1e9598d643ea79c86d">tempest-VolumesSnapshotTestJSON-902320119</nova:project>
Nov 29 03:15:22 user nova-compute[75234]:       </nova:owner>
Nov 29 03:15:22 user nova-compute[75234]:       <nova:root type="image" uuid="96c30ba7-9b73-4a30-8ba4-7138879573b7"/>
Nov 29 03:15:22 user nova-compute[75234]:       <nova:ports>
Nov 29 03:15:22 user nova-compute[75234]:         <nova:port uuid="9d89f22e-e411-461b-b9b3-85dbcf58af43">
Nov 29 03:15:22 user nova-compute[75234]:           <nova:ip type="fixed" address="10.0.0.4" ipVersion="4"/>
Nov 29 03:15:22 user nova-compute[75234]:         </nova:port>
Nov 29 03:15:22 user nova-compute[75234]:       </nova:ports>
Nov 29 03:15:22 user nova-compute[75234]:     </nova:instance>
Nov 29 03:15:22 user nova-compute[75234]:   </metadata>
Nov 29 03:15:22 user nova-compute[75234]:   <sysinfo type="smbios">
Nov 29 03:15:22 user nova-compute[75234]:     <system>
Nov 29 03:15:22 user nova-compute[75234]:       <entry name="manufacturer">OpenStack Foundation</entry>
Nov 29 03:15:22 user nova-compute[75234]:       <entry name="product">OpenStack Nova</entry>
Nov 29 03:15:22 user nova-compute[75234]:       <entry name="version">0.0.0</entry>
Nov 29 03:15:22 user nova-compute[75234]:       <entry name="serial">ec7cf7cd-8e48-4bb3-a969-e14db705dc09</entry>
Nov 29 03:15:22 user nova-compute[75234]:       <entry name="uuid">ec7cf7cd-8e48-4bb3-a969-e14db705dc09</entry>
Nov 29 03:15:22 user nova-compute[75234]:       <entry name="family">Virtual Machine</entry>
Nov 29 03:15:22 user nova-compute[75234]:     </system>
Nov 29 03:15:22 user nova-compute[75234]:   </sysinfo>
Nov 29 03:15:22 user nova-compute[75234]:   <os>
Nov 29 03:15:22 user nova-compute[75234]:     <type arch="x86_64" machine="pc">hvm</type>
Nov 29 03:15:22 user nova-compute[75234]:     <boot dev="hd"/>
Nov 29 03:15:22 user nova-compute[75234]:     <smbios mode="sysinfo"/>
Nov 29 03:15:22 user nova-compute[75234]:   </os>
Nov 29 03:15:22 user nova-compute[75234]:   <features>
Nov 29 03:15:22 user nova-compute[75234]:     <acpi/>
Nov 29 03:15:22 user nova-compute[75234]:     <apic/>
Nov 29 03:15:22 user nova-compute[75234]:     <vmcoreinfo/>
Nov 29 03:15:22 user nova-compute[75234]:   </features>
Nov 29 03:15:22 user nova-compute[75234]:   <clock offset="utc">
Nov 29 03:15:22 user nova-compute[75234]:     <timer name="pit" tickpolicy="delay"/>
Nov 29 03:15:22 user nova-compute[75234]:     <timer name="rtc" tickpolicy="catchup"/>
Nov 29 03:15:22 user nova-compute[75234]:     <timer name="hpet" present="no"/>
Nov 29 03:15:22 user nova-compute[75234]:   </clock>
Nov 29 03:15:22 user nova-compute[75234]:   <cpu mode="custom" match="exact">
Nov 29 03:15:22 user nova-compute[75234]:     <model>Nehalem</model>
Nov 29 03:15:22 user nova-compute[75234]:     <topology sockets="1" cores="1" threads="1"/>
Nov 29 03:15:22 user nova-compute[75234]:   </cpu>
Nov 29 03:15:22 user nova-compute[75234]:   <devices>
Nov 29 03:15:22 user nova-compute[75234]:     <disk type="file" device="disk">
Nov 29 03:15:22 user nova-compute[75234]:       <driver name="qemu" type="qcow2" cache="none"/>
Nov 29 03:15:22 user nova-compute[75234]:       <source file="/opt/stack/data/nova/instances/ec7cf7cd-8e48-4bb3-a969-e14db705dc09/disk"/>
Nov 29 03:15:22 user nova-compute[75234]:       <target dev="vda" bus="virtio"/>
Nov 29 03:15:22 user nova-compute[75234]:     </disk>
Nov 29 03:15:22 user nova-compute[75234]:     <interface type="ethernet">
Nov 29 03:15:22 user nova-compute[75234]:       <mac address="fa:16:3e:30:11:ff"/>
Nov 29 03:15:22 user nova-compute[75234]:       <model type="virtio"/>
Nov 29 03:15:22 user nova-compute[75234]:       <mtu size="1442"/>
Nov 29 03:15:22 user nova-compute[75234]:       <target dev="tap9d89f22e-e4"/>
Nov 29 03:15:22 user nova-compute[75234]:     </interface>
Nov 29 03:15:22 user nova-compute[75234]:     <serial type="pty">
Nov 29 03:15:22 user nova-compute[75234]:       <log file="/opt/stack/data/nova/instances/ec7cf7cd-8e48-4bb3-a969-e14db705dc09/console.log" append="off"/>
Nov 29 03:15:22 user nova-compute[75234]:     </serial>
Nov 29 03:15:22 user nova-compute[75234]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Nov 29 03:15:22 user nova-compute[75234]:     <video>
Nov 29 03:15:22 user nova-compute[75234]:       <model type="virtio"/>
Nov 29 03:15:22 user nova-compute[75234]:     </video>
Nov 29 03:15:22 user nova-compute[75234]:     <rng model="virtio">
Nov 29 03:15:22 user nova-compute[75234]:       <backend model="random">/dev/urandom</backend>
Nov 29 03:15:22 user nova-compute[75234]:     </rng>
Nov 29 03:15:22 user nova-compute[75234]:     <controller type="usb" index="0" model="none"/>
Nov 29 03:15:22 user nova-compute[75234]:     <memballoon model="virtio">
Nov 29 03:15:22 user nova-compute[75234]:       <stats period="10"/>
Nov 29 03:15:22 user nova-compute[75234]:     </memballoon>
Nov 29 03:15:22 user nova-compute[75234]:   </devices>
Nov 29 03:15:22 user nova-compute[75234]: </domain>
Nov 29 03:15:22 user nova-compute[75234]:  {{(pid=75234) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7568}}
Nov 29 03:15:22 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-7044ba48-4a47-4ac0-bc84-6d4b709485c4 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-11-29T03:15:19Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-VolumesSnapshotTestJSON-instance-213375111',display_name='tempest-VolumesSnapshotTestJSON-instance-213375111',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-volumessnapshottestjson-instance-213375111',id=23,image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBKfdFQdP9b2CUucLMguc8tjFgkfx70MU+FbCMNYtTpv3yP6N/lf+rZlWO4aVrwdI3Q2J0as5mFJ1RqaTpNerkZvmi2PeYojCi8bJzCIaL5bklATBcDaVb7EKkDOmV8PS5Q==',key_name='tempest-keypair-798189340',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='7e24dd8accea4c1e9598d643ea79c86d',ramdisk_id='',reservation_id='r-mjtjrmkx',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-VolumesSnapshotTestJSON-902320119',owner_user_name='tempest-VolumesSnapshotTestJSON-902320119-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-11-29T03:15:20Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='899f638d0de34dcba53f4c2ef7f5e068',uuid=ec7cf7cd-8e48-4bb3-a969-e14db705dc09,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "9d89f22e-e411-461b-b9b3-85dbcf58af43", "address": "fa:16:3e:30:11:ff", "network": {"id": "1898605d-00e7-4e81-964e-821e09f5421c", "bridge": "br-int", "label": "tempest-VolumesSnapshotTestJSON-1896954656-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "7e24dd8accea4c1e9598d643ea79c86d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap9d89f22e-e4", "ovs_interfaceid": "9d89f22e-e411-461b-b9b3-85dbcf58af43", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Nov 29 03:15:22 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-7044ba48-4a47-4ac0-bc84-6d4b709485c4 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Converting VIF {"id": "9d89f22e-e411-461b-b9b3-85dbcf58af43", "address": "fa:16:3e:30:11:ff", "network": {"id": "1898605d-00e7-4e81-964e-821e09f5421c", "bridge": "br-int", "label": "tempest-VolumesSnapshotTestJSON-1896954656-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "7e24dd8accea4c1e9598d643ea79c86d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap9d89f22e-e4", "ovs_interfaceid": "9d89f22e-e411-461b-b9b3-85dbcf58af43", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:15:22 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-7044ba48-4a47-4ac0-bc84-6d4b709485c4 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:30:11:ff,bridge_name='br-int',has_traffic_filtering=True,id=9d89f22e-e411-461b-b9b3-85dbcf58af43,network=Network(1898605d-00e7-4e81-964e-821e09f5421c),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap9d89f22e-e4') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:15:22 user nova-compute[75234]: DEBUG os_vif [None req-7044ba48-4a47-4ac0-bc84-6d4b709485c4 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:30:11:ff,bridge_name='br-int',has_traffic_filtering=True,id=9d89f22e-e411-461b-b9b3-85dbcf58af43,network=Network(1898605d-00e7-4e81-964e-821e09f5421c),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap9d89f22e-e4') {{(pid=75234) plug /opt/stack/data/venv/lib/python3.10/site-packages/os_vif/__init__.py:76}}
Nov 29 03:15:22 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:15:22 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:15:22 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Nov 29 03:15:22 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:15:22 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbCreateCommand(_result=None, table=QoS, columns={'type': 'linux-noop', 'external_ids': {'id': '67ebc4c0-2875-541d-a09b-6039ba8a68dd', '_type': 'linux-noop'}}, row=False) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:15:22 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:15:22 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:15:22 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:15:22 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tap9d89f22e-e4, may_exist=True, interface_attrs={}) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:15:22 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Port, record=tap9d89f22e-e4, col_values=(('qos', UUID('8951153a-de0d-4c29-9db3-748bfdd49e97')),), if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:15:22 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DbSetCommand(_result=None, table=Interface, record=tap9d89f22e-e4, col_values=(('external_ids', {'iface-id': '9d89f22e-e411-461b-b9b3-85dbcf58af43', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:30:11:ff', 'vm-uuid': 'ec7cf7cd-8e48-4bb3-a969-e14db705dc09'}),), if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:15:22 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:15:22 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:15:22 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:15:22 user nova-compute[75234]: INFO os_vif [None req-7044ba48-4a47-4ac0-bc84-6d4b709485c4 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:30:11:ff,bridge_name='br-int',has_traffic_filtering=True,id=9d89f22e-e411-461b-b9b3-85dbcf58af43,network=Network(1898605d-00e7-4e81-964e-821e09f5421c),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap9d89f22e-e4')
Nov 29 03:15:22 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-7044ba48-4a47-4ac0-bc84-6d4b709485c4 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] No BDM found with device name vda, not building metadata. {{(pid=75234) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12089}}
Nov 29 03:15:22 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-7044ba48-4a47-4ac0-bc84-6d4b709485c4 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] No VIF found with MAC fa:16:3e:30:11:ff, not building metadata {{(pid=75234) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Nov 29 03:15:22 user nova-compute[75234]: DEBUG nova.network.neutron [req-375f071e-590c-4a0a-b770-cc5a4aacb758 req-4828b42d-c7ac-4582-8369-476ba16b350b service nova] [instance: ec7cf7cd-8e48-4bb3-a969-e14db705dc09] Updated VIF entry in instance network info cache for port 9d89f22e-e411-461b-b9b3-85dbcf58af43. {{(pid=75234) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3481}}
Nov 29 03:15:22 user nova-compute[75234]: DEBUG nova.network.neutron [req-375f071e-590c-4a0a-b770-cc5a4aacb758 req-4828b42d-c7ac-4582-8369-476ba16b350b service nova] [instance: ec7cf7cd-8e48-4bb3-a969-e14db705dc09] Updating instance_info_cache with network_info: [{"id": "9d89f22e-e411-461b-b9b3-85dbcf58af43", "address": "fa:16:3e:30:11:ff", "network": {"id": "1898605d-00e7-4e81-964e-821e09f5421c", "bridge": "br-int", "label": "tempest-VolumesSnapshotTestJSON-1896954656-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "7e24dd8accea4c1e9598d643ea79c86d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap9d89f22e-e4", "ovs_interfaceid": "9d89f22e-e411-461b-b9b3-85dbcf58af43", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:15:22 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-375f071e-590c-4a0a-b770-cc5a4aacb758 req-4828b42d-c7ac-4582-8369-476ba16b350b service nova] Releasing lock "refresh_cache-ec7cf7cd-8e48-4bb3-a969-e14db705dc09" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:15:23 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:15:23 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:15:23 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:15:23 user nova-compute[75234]: DEBUG nova.compute.manager [req-930345c6-02f4-44a1-8fb7-827efa8d1806 req-045bf736-3c53-42f8-b034-b431394bda00 service nova] [instance: ec7cf7cd-8e48-4bb3-a969-e14db705dc09] Received event network-vif-plugged-9d89f22e-e411-461b-b9b3-85dbcf58af43 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:15:23 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-930345c6-02f4-44a1-8fb7-827efa8d1806 req-045bf736-3c53-42f8-b034-b431394bda00 service nova] Acquiring lock "ec7cf7cd-8e48-4bb3-a969-e14db705dc09-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:15:23 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-930345c6-02f4-44a1-8fb7-827efa8d1806 req-045bf736-3c53-42f8-b034-b431394bda00 service nova] Lock "ec7cf7cd-8e48-4bb3-a969-e14db705dc09-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:15:23 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-930345c6-02f4-44a1-8fb7-827efa8d1806 req-045bf736-3c53-42f8-b034-b431394bda00 service nova] Lock "ec7cf7cd-8e48-4bb3-a969-e14db705dc09-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:15:23 user nova-compute[75234]: DEBUG nova.compute.manager [req-930345c6-02f4-44a1-8fb7-827efa8d1806 req-045bf736-3c53-42f8-b034-b431394bda00 service nova] [instance: ec7cf7cd-8e48-4bb3-a969-e14db705dc09] No waiting events found dispatching network-vif-plugged-9d89f22e-e411-461b-b9b3-85dbcf58af43 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:15:23 user nova-compute[75234]: WARNING nova.compute.manager [req-930345c6-02f4-44a1-8fb7-827efa8d1806 req-045bf736-3c53-42f8-b034-b431394bda00 service nova] [instance: ec7cf7cd-8e48-4bb3-a969-e14db705dc09] Received unexpected event network-vif-plugged-9d89f22e-e411-461b-b9b3-85dbcf58af43 for instance with vm_state building and task_state spawning.
Nov 29 03:15:24 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:15:24 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:15:24 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:15:25 user nova-compute[75234]: DEBUG nova.compute.manager [None req-7044ba48-4a47-4ac0-bc84-6d4b709485c4 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] [instance: ec7cf7cd-8e48-4bb3-a969-e14db705dc09] Instance event wait completed in 0 seconds for  {{(pid=75234) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Nov 29 03:15:25 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-7044ba48-4a47-4ac0-bc84-6d4b709485c4 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] [instance: ec7cf7cd-8e48-4bb3-a969-e14db705dc09] Guest created on hypervisor {{(pid=75234) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4421}}
Nov 29 03:15:25 user nova-compute[75234]: INFO nova.virt.libvirt.driver [-] [instance: ec7cf7cd-8e48-4bb3-a969-e14db705dc09] Instance spawned successfully.
Nov 29 03:15:25 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-7044ba48-4a47-4ac0-bc84-6d4b709485c4 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] [instance: ec7cf7cd-8e48-4bb3-a969-e14db705dc09] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:904}}
Nov 29 03:15:25 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-7044ba48-4a47-4ac0-bc84-6d4b709485c4 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] [instance: ec7cf7cd-8e48-4bb3-a969-e14db705dc09] Found default for hw_cdrom_bus of ide {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:15:25 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-7044ba48-4a47-4ac0-bc84-6d4b709485c4 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] [instance: ec7cf7cd-8e48-4bb3-a969-e14db705dc09] Found default for hw_disk_bus of virtio {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:15:25 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-7044ba48-4a47-4ac0-bc84-6d4b709485c4 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] [instance: ec7cf7cd-8e48-4bb3-a969-e14db705dc09] Found default for hw_input_bus of None {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:15:25 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-7044ba48-4a47-4ac0-bc84-6d4b709485c4 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] [instance: ec7cf7cd-8e48-4bb3-a969-e14db705dc09] Found default for hw_pointer_model of None {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:15:25 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-7044ba48-4a47-4ac0-bc84-6d4b709485c4 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] [instance: ec7cf7cd-8e48-4bb3-a969-e14db705dc09] Found default for hw_video_model of virtio {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:15:25 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-7044ba48-4a47-4ac0-bc84-6d4b709485c4 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] [instance: ec7cf7cd-8e48-4bb3-a969-e14db705dc09] Found default for hw_vif_model of virtio {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:15:25 user nova-compute[75234]: DEBUG nova.compute.manager [req-d157feb0-ba07-4b07-882c-6d6f64a7b40a req-ba29b8c2-4001-490e-866c-ee693949f2ff service nova] [instance: ec7cf7cd-8e48-4bb3-a969-e14db705dc09] Received event network-vif-plugged-9d89f22e-e411-461b-b9b3-85dbcf58af43 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:15:25 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-d157feb0-ba07-4b07-882c-6d6f64a7b40a req-ba29b8c2-4001-490e-866c-ee693949f2ff service nova] Acquiring lock "ec7cf7cd-8e48-4bb3-a969-e14db705dc09-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:15:25 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-d157feb0-ba07-4b07-882c-6d6f64a7b40a req-ba29b8c2-4001-490e-866c-ee693949f2ff service nova] Lock "ec7cf7cd-8e48-4bb3-a969-e14db705dc09-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:15:25 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-d157feb0-ba07-4b07-882c-6d6f64a7b40a req-ba29b8c2-4001-490e-866c-ee693949f2ff service nova] Lock "ec7cf7cd-8e48-4bb3-a969-e14db705dc09-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:15:25 user nova-compute[75234]: DEBUG nova.compute.manager [req-d157feb0-ba07-4b07-882c-6d6f64a7b40a req-ba29b8c2-4001-490e-866c-ee693949f2ff service nova] [instance: ec7cf7cd-8e48-4bb3-a969-e14db705dc09] No waiting events found dispatching network-vif-plugged-9d89f22e-e411-461b-b9b3-85dbcf58af43 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:15:25 user nova-compute[75234]: WARNING nova.compute.manager [req-d157feb0-ba07-4b07-882c-6d6f64a7b40a req-ba29b8c2-4001-490e-866c-ee693949f2ff service nova] [instance: ec7cf7cd-8e48-4bb3-a969-e14db705dc09] Received unexpected event network-vif-plugged-9d89f22e-e411-461b-b9b3-85dbcf58af43 for instance with vm_state building and task_state spawning.
Nov 29 03:15:26 user nova-compute[75234]: INFO nova.compute.manager [None req-7044ba48-4a47-4ac0-bc84-6d4b709485c4 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] [instance: ec7cf7cd-8e48-4bb3-a969-e14db705dc09] Took 5.50 seconds to spawn the instance on the hypervisor.
Nov 29 03:15:26 user nova-compute[75234]: DEBUG nova.compute.manager [None req-7044ba48-4a47-4ac0-bc84-6d4b709485c4 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] [instance: ec7cf7cd-8e48-4bb3-a969-e14db705dc09] Checking state {{(pid=75234) _get_power_state /opt/stack/nova/nova/compute/manager.py:1782}}
Nov 29 03:15:26 user nova-compute[75234]: INFO nova.compute.manager [None req-7044ba48-4a47-4ac0-bc84-6d4b709485c4 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] [instance: ec7cf7cd-8e48-4bb3-a969-e14db705dc09] Took 6.34 seconds to build instance.
Nov 29 03:15:26 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-7044ba48-4a47-4ac0-bc84-6d4b709485c4 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Lock "ec7cf7cd-8e48-4bb3-a969-e14db705dc09" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 6.438s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:15:26 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:15:27 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:15:30 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:15:32 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:15:36 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:15:37 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:15:37 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-438d669e-c97f-4c6f-bc43-886b9afd2261 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Acquiring lock "6c0e0184-ca3a-48ec-a6e3-a0a1172260d6" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:15:37 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-438d669e-c97f-4c6f-bc43-886b9afd2261 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Lock "6c0e0184-ca3a-48ec-a6e3-a0a1172260d6" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:15:37 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-438d669e-c97f-4c6f-bc43-886b9afd2261 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Acquiring lock "6c0e0184-ca3a-48ec-a6e3-a0a1172260d6-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:15:37 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-438d669e-c97f-4c6f-bc43-886b9afd2261 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Lock "6c0e0184-ca3a-48ec-a6e3-a0a1172260d6-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:15:37 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-438d669e-c97f-4c6f-bc43-886b9afd2261 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Lock "6c0e0184-ca3a-48ec-a6e3-a0a1172260d6-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:15:37 user nova-compute[75234]: INFO nova.compute.manager [None req-438d669e-c97f-4c6f-bc43-886b9afd2261 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] [instance: 6c0e0184-ca3a-48ec-a6e3-a0a1172260d6] Terminating instance
Nov 29 03:15:37 user nova-compute[75234]: DEBUG nova.compute.manager [None req-438d669e-c97f-4c6f-bc43-886b9afd2261 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] [instance: 6c0e0184-ca3a-48ec-a6e3-a0a1172260d6] Start destroying the instance on the hypervisor. {{(pid=75234) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3140}}
Nov 29 03:15:37 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:15:37 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:15:38 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:15:38 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:15:38 user nova-compute[75234]: DEBUG nova.compute.manager [req-671a6c86-82f9-4515-a19a-b20739788681 req-f60f9269-a9fe-422d-bb7c-8795a527da8f service nova] [instance: 6c0e0184-ca3a-48ec-a6e3-a0a1172260d6] Received event network-vif-unplugged-50593025-edf1-44ec-b5f0-c92d6a807c23 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:15:38 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-671a6c86-82f9-4515-a19a-b20739788681 req-f60f9269-a9fe-422d-bb7c-8795a527da8f service nova] Acquiring lock "6c0e0184-ca3a-48ec-a6e3-a0a1172260d6-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:15:38 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-671a6c86-82f9-4515-a19a-b20739788681 req-f60f9269-a9fe-422d-bb7c-8795a527da8f service nova] Lock "6c0e0184-ca3a-48ec-a6e3-a0a1172260d6-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:15:38 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-671a6c86-82f9-4515-a19a-b20739788681 req-f60f9269-a9fe-422d-bb7c-8795a527da8f service nova] Lock "6c0e0184-ca3a-48ec-a6e3-a0a1172260d6-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:15:38 user nova-compute[75234]: DEBUG nova.compute.manager [req-671a6c86-82f9-4515-a19a-b20739788681 req-f60f9269-a9fe-422d-bb7c-8795a527da8f service nova] [instance: 6c0e0184-ca3a-48ec-a6e3-a0a1172260d6] No waiting events found dispatching network-vif-unplugged-50593025-edf1-44ec-b5f0-c92d6a807c23 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:15:38 user nova-compute[75234]: DEBUG nova.compute.manager [req-671a6c86-82f9-4515-a19a-b20739788681 req-f60f9269-a9fe-422d-bb7c-8795a527da8f service nova] [instance: 6c0e0184-ca3a-48ec-a6e3-a0a1172260d6] Received event network-vif-unplugged-50593025-edf1-44ec-b5f0-c92d6a807c23 for instance with task_state deleting. {{(pid=75234) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10890}}
Nov 29 03:15:38 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-6f6a7f28-2a0e-4636-bd66-89560bc244ed tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] Acquiring lock "95b6eb15-7930-4aa5-aff8-73809b0a9277" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:15:38 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-6f6a7f28-2a0e-4636-bd66-89560bc244ed tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] Lock "95b6eb15-7930-4aa5-aff8-73809b0a9277" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:15:38 user nova-compute[75234]: DEBUG nova.compute.manager [None req-6f6a7f28-2a0e-4636-bd66-89560bc244ed tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] [instance: 95b6eb15-7930-4aa5-aff8-73809b0a9277] Starting instance... {{(pid=75234) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2422}}
Nov 29 03:15:38 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-6f6a7f28-2a0e-4636-bd66-89560bc244ed tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:15:38 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-6f6a7f28-2a0e-4636-bd66-89560bc244ed tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:15:38 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-6f6a7f28-2a0e-4636-bd66-89560bc244ed tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=75234) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2366}}
Nov 29 03:15:38 user nova-compute[75234]: INFO nova.compute.claims [None req-6f6a7f28-2a0e-4636-bd66-89560bc244ed tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] [instance: 95b6eb15-7930-4aa5-aff8-73809b0a9277] Claim successful on node user
Nov 29 03:15:38 user nova-compute[75234]: INFO nova.virt.libvirt.driver [-] [instance: 6c0e0184-ca3a-48ec-a6e3-a0a1172260d6] Instance destroyed successfully.
Nov 29 03:15:38 user nova-compute[75234]: DEBUG nova.objects.instance [None req-438d669e-c97f-4c6f-bc43-886b9afd2261 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Lazy-loading 'resources' on Instance uuid 6c0e0184-ca3a-48ec-a6e3-a0a1172260d6 {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:15:38 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-438d669e-c97f-4c6f-bc43-886b9afd2261 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=True,compute_id=1,config_drive='',created_at=2023-11-29T03:09:52Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-tempest.common.compute-instance-176407456',display_name='tempest-ServerActionsTestOtherA-server-1878420946',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-tempest-common-compute-instance-176407456',id=16,image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBFVgvqBN7/vaAFxNi0MSZxI04yEwk9FFUfhV/l7Fr5QumrLMJI6JbWuh0vhY2F6+RLBsF4qtCGuui27rDdawR2ALy+mo/mA7ZKd/Mr3qfGMSgFbF3GghJD1rMovi6Hk9rQ==',key_name='tempest-keypair-382073294',keypairs=<?>,launch_index=0,launched_at=2023-11-29T03:12:23Z,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={rebuild='server'},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='c1cb3d1f12f242a785a688d59dbe3cd6',ramdisk_id='',reservation_id='r-7poirwds',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',clean_attempts='1',image_base_image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-ServerActionsTestOtherA-1495067208',owner_user_name='tempest-ServerActionsTestOtherA-1495067208-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-11-29T03:12:24Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='d611826f3be947c0947b99c763c26d44',uuid=6c0e0184-ca3a-48ec-a6e3-a0a1172260d6,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "50593025-edf1-44ec-b5f0-c92d6a807c23", "address": "fa:16:3e:d8:a2:e7", "network": {"id": "1f946e69-dbec-429c-83f8-a186bad4f2cc", "bridge": "br-int", "label": "tempest-ServerActionsTestOtherA-486369423-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.146", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "c1cb3d1f12f242a785a688d59dbe3cd6", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap50593025-ed", "ovs_interfaceid": "50593025-edf1-44ec-b5f0-c92d6a807c23", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Nov 29 03:15:38 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-438d669e-c97f-4c6f-bc43-886b9afd2261 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Converting VIF {"id": "50593025-edf1-44ec-b5f0-c92d6a807c23", "address": "fa:16:3e:d8:a2:e7", "network": {"id": "1f946e69-dbec-429c-83f8-a186bad4f2cc", "bridge": "br-int", "label": "tempest-ServerActionsTestOtherA-486369423-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.146", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "c1cb3d1f12f242a785a688d59dbe3cd6", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap50593025-ed", "ovs_interfaceid": "50593025-edf1-44ec-b5f0-c92d6a807c23", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:15:38 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-438d669e-c97f-4c6f-bc43-886b9afd2261 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:d8:a2:e7,bridge_name='br-int',has_traffic_filtering=True,id=50593025-edf1-44ec-b5f0-c92d6a807c23,network=Network(1f946e69-dbec-429c-83f8-a186bad4f2cc),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap50593025-ed') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:15:38 user nova-compute[75234]: DEBUG os_vif [None req-438d669e-c97f-4c6f-bc43-886b9afd2261 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Unplugging vif VIFOpenVSwitch(active=True,address=fa:16:3e:d8:a2:e7,bridge_name='br-int',has_traffic_filtering=True,id=50593025-edf1-44ec-b5f0-c92d6a807c23,network=Network(1f946e69-dbec-429c-83f8-a186bad4f2cc),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap50593025-ed') {{(pid=75234) unplug /opt/stack/data/venv/lib/python3.10/site-packages/os_vif/__init__.py:109}}
Nov 29 03:15:38 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:15:38 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tap50593025-ed, bridge=br-int, if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:15:38 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:15:38 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:15:38 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:15:38 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbDestroyCommand(_result=None, table=QoS, record=33b9a35e-a78f-4f54-bdd7-32c2752b78cb) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:15:38 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:15:38 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:15:38 user nova-compute[75234]: INFO os_vif [None req-438d669e-c97f-4c6f-bc43-886b9afd2261 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Successfully unplugged vif VIFOpenVSwitch(active=True,address=fa:16:3e:d8:a2:e7,bridge_name='br-int',has_traffic_filtering=True,id=50593025-edf1-44ec-b5f0-c92d6a807c23,network=Network(1f946e69-dbec-429c-83f8-a186bad4f2cc),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap50593025-ed')
Nov 29 03:15:38 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-438d669e-c97f-4c6f-bc43-886b9afd2261 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] [instance: 6c0e0184-ca3a-48ec-a6e3-a0a1172260d6] Deleting instance files /opt/stack/data/nova/instances/6c0e0184-ca3a-48ec-a6e3-a0a1172260d6_del
Nov 29 03:15:38 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-438d669e-c97f-4c6f-bc43-886b9afd2261 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] [instance: 6c0e0184-ca3a-48ec-a6e3-a0a1172260d6] Deletion of /opt/stack/data/nova/instances/6c0e0184-ca3a-48ec-a6e3-a0a1172260d6_del complete
Nov 29 03:15:38 user nova-compute[75234]: INFO nova.compute.manager [None req-438d669e-c97f-4c6f-bc43-886b9afd2261 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] [instance: 6c0e0184-ca3a-48ec-a6e3-a0a1172260d6] Took 0.91 seconds to destroy the instance on the hypervisor.
Nov 29 03:15:38 user nova-compute[75234]: DEBUG oslo.service.loopingcall [None req-438d669e-c97f-4c6f-bc43-886b9afd2261 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=75234) func /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/loopingcall.py:435}}
Nov 29 03:15:38 user nova-compute[75234]: DEBUG nova.compute.manager [-] [instance: 6c0e0184-ca3a-48ec-a6e3-a0a1172260d6] Deallocating network for instance {{(pid=75234) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2279}}
Nov 29 03:15:38 user nova-compute[75234]: DEBUG nova.network.neutron [-] [instance: 6c0e0184-ca3a-48ec-a6e3-a0a1172260d6] deallocate_for_instance() {{(pid=75234) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1802}}
Nov 29 03:15:38 user nova-compute[75234]: DEBUG nova.compute.provider_tree [None req-6f6a7f28-2a0e-4636-bd66-89560bc244ed tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] Inventory has not changed in ProviderTree for provider: 4d291220-7c37-4839-a0e3-b23c66502b74 {{(pid=75234) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Nov 29 03:15:38 user nova-compute[75234]: DEBUG nova.scheduler.client.report [None req-6f6a7f28-2a0e-4636-bd66-89560bc244ed tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] Inventory has not changed for provider 4d291220-7c37-4839-a0e3-b23c66502b74 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=75234) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Nov 29 03:15:38 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-6f6a7f28-2a0e-4636-bd66-89560bc244ed tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.526s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:15:38 user nova-compute[75234]: DEBUG nova.compute.manager [None req-6f6a7f28-2a0e-4636-bd66-89560bc244ed tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] [instance: 95b6eb15-7930-4aa5-aff8-73809b0a9277] Start building networks asynchronously for instance. {{(pid=75234) _build_resources /opt/stack/nova/nova/compute/manager.py:2819}}
Nov 29 03:15:39 user nova-compute[75234]: DEBUG nova.compute.manager [None req-6f6a7f28-2a0e-4636-bd66-89560bc244ed tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] [instance: 95b6eb15-7930-4aa5-aff8-73809b0a9277] Allocating IP information in the background. {{(pid=75234) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1972}}
Nov 29 03:15:39 user nova-compute[75234]: DEBUG nova.network.neutron [None req-6f6a7f28-2a0e-4636-bd66-89560bc244ed tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] [instance: 95b6eb15-7930-4aa5-aff8-73809b0a9277] allocate_for_instance() {{(pid=75234) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1156}}
Nov 29 03:15:39 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-6f6a7f28-2a0e-4636-bd66-89560bc244ed tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] [instance: 95b6eb15-7930-4aa5-aff8-73809b0a9277] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Nov 29 03:15:39 user nova-compute[75234]: DEBUG nova.compute.manager [None req-6f6a7f28-2a0e-4636-bd66-89560bc244ed tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] [instance: 95b6eb15-7930-4aa5-aff8-73809b0a9277] Start building block device mappings for instance. {{(pid=75234) _build_resources /opt/stack/nova/nova/compute/manager.py:2854}}
Nov 29 03:15:39 user nova-compute[75234]: DEBUG nova.policy [None req-6f6a7f28-2a0e-4636-bd66-89560bc244ed tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '460854be8e05408a80790620f64cf8e1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5eaa91ebc2e64f2eba987f02bcd1a9f4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=75234) authorize /opt/stack/nova/nova/policy.py:203}}
Nov 29 03:15:39 user nova-compute[75234]: DEBUG nova.compute.manager [None req-6f6a7f28-2a0e-4636-bd66-89560bc244ed tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] [instance: 95b6eb15-7930-4aa5-aff8-73809b0a9277] Start spawning the instance on the hypervisor. {{(pid=75234) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2628}}
Nov 29 03:15:39 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-6f6a7f28-2a0e-4636-bd66-89560bc244ed tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] [instance: 95b6eb15-7930-4aa5-aff8-73809b0a9277] Creating instance directory {{(pid=75234) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4727}}
Nov 29 03:15:39 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-6f6a7f28-2a0e-4636-bd66-89560bc244ed tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] [instance: 95b6eb15-7930-4aa5-aff8-73809b0a9277] Creating image(s)
Nov 29 03:15:39 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-6f6a7f28-2a0e-4636-bd66-89560bc244ed tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] Acquiring lock "/opt/stack/data/nova/instances/95b6eb15-7930-4aa5-aff8-73809b0a9277/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:15:39 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-6f6a7f28-2a0e-4636-bd66-89560bc244ed tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] Lock "/opt/stack/data/nova/instances/95b6eb15-7930-4aa5-aff8-73809b0a9277/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:15:39 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-6f6a7f28-2a0e-4636-bd66-89560bc244ed tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] Lock "/opt/stack/data/nova/instances/95b6eb15-7930-4aa5-aff8-73809b0a9277/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.002s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:15:39 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-6f6a7f28-2a0e-4636-bd66-89560bc244ed tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:15:39 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-6f6a7f28-2a0e-4636-bd66-89560bc244ed tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json" returned: 0 in 0.193s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:15:39 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-6f6a7f28-2a0e-4636-bd66-89560bc244ed tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] Acquiring lock "80686e29dc9db6600d03e85cb62af298e12d7918" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:15:39 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-6f6a7f28-2a0e-4636-bd66-89560bc244ed tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] Lock "80686e29dc9db6600d03e85cb62af298e12d7918" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.002s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:15:39 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-6f6a7f28-2a0e-4636-bd66-89560bc244ed tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:15:39 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-6f6a7f28-2a0e-4636-bd66-89560bc244ed tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json" returned: 0 in 0.199s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:15:39 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-6f6a7f28-2a0e-4636-bd66-89560bc244ed tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918,backing_fmt=raw /opt/stack/data/nova/instances/95b6eb15-7930-4aa5-aff8-73809b0a9277/disk 1073741824 {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:15:39 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-6f6a7f28-2a0e-4636-bd66-89560bc244ed tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918,backing_fmt=raw /opt/stack/data/nova/instances/95b6eb15-7930-4aa5-aff8-73809b0a9277/disk 1073741824" returned: 0 in 0.066s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:15:39 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-6f6a7f28-2a0e-4636-bd66-89560bc244ed tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] Lock "80686e29dc9db6600d03e85cb62af298e12d7918" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.274s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:15:39 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-6f6a7f28-2a0e-4636-bd66-89560bc244ed tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:15:39 user nova-compute[75234]: DEBUG nova.network.neutron [None req-6f6a7f28-2a0e-4636-bd66-89560bc244ed tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] [instance: 95b6eb15-7930-4aa5-aff8-73809b0a9277] Successfully created port: 64da7883-fc79-44ea-a9d8-65a3f65cfc82 {{(pid=75234) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:548}}
Nov 29 03:15:39 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-6f6a7f28-2a0e-4636-bd66-89560bc244ed tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json" returned: 0 in 0.199s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:15:39 user nova-compute[75234]: DEBUG nova.virt.disk.api [None req-6f6a7f28-2a0e-4636-bd66-89560bc244ed tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] Checking if we can resize image /opt/stack/data/nova/instances/95b6eb15-7930-4aa5-aff8-73809b0a9277/disk. size=1073741824 {{(pid=75234) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:166}}
Nov 29 03:15:39 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-6f6a7f28-2a0e-4636-bd66-89560bc244ed tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/95b6eb15-7930-4aa5-aff8-73809b0a9277/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:15:40 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-6f6a7f28-2a0e-4636-bd66-89560bc244ed tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/95b6eb15-7930-4aa5-aff8-73809b0a9277/disk --force-share --output=json" returned: 0 in 0.206s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:15:40 user nova-compute[75234]: DEBUG nova.virt.disk.api [None req-6f6a7f28-2a0e-4636-bd66-89560bc244ed tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] Cannot resize image /opt/stack/data/nova/instances/95b6eb15-7930-4aa5-aff8-73809b0a9277/disk to a smaller size. {{(pid=75234) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:172}}
Nov 29 03:15:40 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-6f6a7f28-2a0e-4636-bd66-89560bc244ed tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] [instance: 95b6eb15-7930-4aa5-aff8-73809b0a9277] Created local disks {{(pid=75234) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4861}}
Nov 29 03:15:40 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-6f6a7f28-2a0e-4636-bd66-89560bc244ed tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] [instance: 95b6eb15-7930-4aa5-aff8-73809b0a9277] Ensure instance console log exists: /opt/stack/data/nova/instances/95b6eb15-7930-4aa5-aff8-73809b0a9277/console.log {{(pid=75234) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4613}}
Nov 29 03:15:40 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-6f6a7f28-2a0e-4636-bd66-89560bc244ed tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:15:40 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-6f6a7f28-2a0e-4636-bd66-89560bc244ed tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:15:40 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-6f6a7f28-2a0e-4636-bd66-89560bc244ed tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:15:40 user nova-compute[75234]: DEBUG nova.compute.manager [req-b6f30d4a-fdc5-4071-a494-9945a86160e9 req-4985eac8-f6cf-4e2b-92da-ad36da1e756c service nova] [instance: 6c0e0184-ca3a-48ec-a6e3-a0a1172260d6] Received event network-vif-plugged-50593025-edf1-44ec-b5f0-c92d6a807c23 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:15:40 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-b6f30d4a-fdc5-4071-a494-9945a86160e9 req-4985eac8-f6cf-4e2b-92da-ad36da1e756c service nova] Acquiring lock "6c0e0184-ca3a-48ec-a6e3-a0a1172260d6-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:15:40 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-b6f30d4a-fdc5-4071-a494-9945a86160e9 req-4985eac8-f6cf-4e2b-92da-ad36da1e756c service nova] Lock "6c0e0184-ca3a-48ec-a6e3-a0a1172260d6-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:15:40 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-b6f30d4a-fdc5-4071-a494-9945a86160e9 req-4985eac8-f6cf-4e2b-92da-ad36da1e756c service nova] Lock "6c0e0184-ca3a-48ec-a6e3-a0a1172260d6-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:15:40 user nova-compute[75234]: DEBUG nova.compute.manager [req-b6f30d4a-fdc5-4071-a494-9945a86160e9 req-4985eac8-f6cf-4e2b-92da-ad36da1e756c service nova] [instance: 6c0e0184-ca3a-48ec-a6e3-a0a1172260d6] No waiting events found dispatching network-vif-plugged-50593025-edf1-44ec-b5f0-c92d6a807c23 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:15:40 user nova-compute[75234]: WARNING nova.compute.manager [req-b6f30d4a-fdc5-4071-a494-9945a86160e9 req-4985eac8-f6cf-4e2b-92da-ad36da1e756c service nova] [instance: 6c0e0184-ca3a-48ec-a6e3-a0a1172260d6] Received unexpected event network-vif-plugged-50593025-edf1-44ec-b5f0-c92d6a807c23 for instance with vm_state active and task_state deleting.
Nov 29 03:15:40 user nova-compute[75234]: DEBUG nova.network.neutron [-] [instance: 6c0e0184-ca3a-48ec-a6e3-a0a1172260d6] Updating instance_info_cache with network_info: [] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:15:40 user nova-compute[75234]: DEBUG nova.network.neutron [None req-6f6a7f28-2a0e-4636-bd66-89560bc244ed tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] [instance: 95b6eb15-7930-4aa5-aff8-73809b0a9277] Successfully updated port: 64da7883-fc79-44ea-a9d8-65a3f65cfc82 {{(pid=75234) _update_port /opt/stack/nova/nova/network/neutron.py:586}}
Nov 29 03:15:40 user nova-compute[75234]: INFO nova.compute.manager [-] [instance: 6c0e0184-ca3a-48ec-a6e3-a0a1172260d6] Took 1.92 seconds to deallocate network for instance.
Nov 29 03:15:40 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-6f6a7f28-2a0e-4636-bd66-89560bc244ed tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] Acquiring lock "refresh_cache-95b6eb15-7930-4aa5-aff8-73809b0a9277" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:15:40 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-6f6a7f28-2a0e-4636-bd66-89560bc244ed tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] Acquired lock "refresh_cache-95b6eb15-7930-4aa5-aff8-73809b0a9277" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:15:40 user nova-compute[75234]: DEBUG nova.network.neutron [None req-6f6a7f28-2a0e-4636-bd66-89560bc244ed tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] [instance: 95b6eb15-7930-4aa5-aff8-73809b0a9277] Building network info cache for instance {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2009}}
Nov 29 03:15:40 user nova-compute[75234]: DEBUG nova.compute.manager [req-9bcced37-7479-433e-8bf1-193d3d7f6d64 req-eae19d60-d970-4eca-b627-ad74c3022a36 service nova] [instance: 6c0e0184-ca3a-48ec-a6e3-a0a1172260d6] Received event network-vif-deleted-50593025-edf1-44ec-b5f0-c92d6a807c23 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:15:40 user nova-compute[75234]: INFO nova.compute.manager [req-9bcced37-7479-433e-8bf1-193d3d7f6d64 req-eae19d60-d970-4eca-b627-ad74c3022a36 service nova] [instance: 6c0e0184-ca3a-48ec-a6e3-a0a1172260d6] Neutron deleted interface 50593025-edf1-44ec-b5f0-c92d6a807c23; detaching it from the instance and deleting it from the info cache
Nov 29 03:15:40 user nova-compute[75234]: DEBUG nova.network.neutron [req-9bcced37-7479-433e-8bf1-193d3d7f6d64 req-eae19d60-d970-4eca-b627-ad74c3022a36 service nova] [instance: 6c0e0184-ca3a-48ec-a6e3-a0a1172260d6] Updating instance_info_cache with network_info: [] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:15:40 user nova-compute[75234]: DEBUG nova.compute.manager [req-9bcced37-7479-433e-8bf1-193d3d7f6d64 req-eae19d60-d970-4eca-b627-ad74c3022a36 service nova] [instance: 6c0e0184-ca3a-48ec-a6e3-a0a1172260d6] Detach interface failed, port_id=50593025-edf1-44ec-b5f0-c92d6a807c23, reason: Instance 6c0e0184-ca3a-48ec-a6e3-a0a1172260d6 could not be found. {{(pid=75234) _process_instance_vif_deleted_event /opt/stack/nova/nova/compute/manager.py:10946}}
Nov 29 03:15:40 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-438d669e-c97f-4c6f-bc43-886b9afd2261 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:15:40 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-438d669e-c97f-4c6f-bc43-886b9afd2261 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:15:40 user nova-compute[75234]: DEBUG nova.network.neutron [None req-6f6a7f28-2a0e-4636-bd66-89560bc244ed tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] [instance: 95b6eb15-7930-4aa5-aff8-73809b0a9277] Instance cache missing network info. {{(pid=75234) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3322}}
Nov 29 03:15:40 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-6732f0fb-d7c6-4c58-b125-63f3c8fe5cb5 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Acquiring lock "062753a6-8f39-4e80-a549-9f0f5a4b877a" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:15:40 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-6732f0fb-d7c6-4c58-b125-63f3c8fe5cb5 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Lock "062753a6-8f39-4e80-a549-9f0f5a4b877a" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:15:40 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-6732f0fb-d7c6-4c58-b125-63f3c8fe5cb5 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Acquiring lock "062753a6-8f39-4e80-a549-9f0f5a4b877a-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:15:40 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-6732f0fb-d7c6-4c58-b125-63f3c8fe5cb5 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Lock "062753a6-8f39-4e80-a549-9f0f5a4b877a-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:15:40 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-6732f0fb-d7c6-4c58-b125-63f3c8fe5cb5 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Lock "062753a6-8f39-4e80-a549-9f0f5a4b877a-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:15:40 user nova-compute[75234]: INFO nova.compute.manager [None req-6732f0fb-d7c6-4c58-b125-63f3c8fe5cb5 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] [instance: 062753a6-8f39-4e80-a549-9f0f5a4b877a] Terminating instance
Nov 29 03:15:40 user nova-compute[75234]: DEBUG nova.compute.manager [None req-6732f0fb-d7c6-4c58-b125-63f3c8fe5cb5 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] [instance: 062753a6-8f39-4e80-a549-9f0f5a4b877a] Start destroying the instance on the hypervisor. {{(pid=75234) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3140}}
Nov 29 03:15:41 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:15:41 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:15:41 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:15:41 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:15:41 user nova-compute[75234]: DEBUG nova.compute.provider_tree [None req-438d669e-c97f-4c6f-bc43-886b9afd2261 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Inventory has not changed in ProviderTree for provider: 4d291220-7c37-4839-a0e3-b23c66502b74 {{(pid=75234) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Nov 29 03:15:41 user nova-compute[75234]: DEBUG nova.scheduler.client.report [None req-438d669e-c97f-4c6f-bc43-886b9afd2261 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Inventory has not changed for provider 4d291220-7c37-4839-a0e3-b23c66502b74 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=75234) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Nov 29 03:15:41 user nova-compute[75234]: DEBUG nova.network.neutron [None req-6f6a7f28-2a0e-4636-bd66-89560bc244ed tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] [instance: 95b6eb15-7930-4aa5-aff8-73809b0a9277] Updating instance_info_cache with network_info: [{"id": "64da7883-fc79-44ea-a9d8-65a3f65cfc82", "address": "fa:16:3e:23:9e:1e", "network": {"id": "49d62bf6-34ee-4de8-ab85-6ef489d957c8", "bridge": "br-int", "label": "tempest-VolumesAdminNegativeTest-1304144113-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "5eaa91ebc2e64f2eba987f02bcd1a9f4", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap64da7883-fc", "ovs_interfaceid": "64da7883-fc79-44ea-a9d8-65a3f65cfc82", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:15:41 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-438d669e-c97f-4c6f-bc43-886b9afd2261 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.449s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:15:41 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-6f6a7f28-2a0e-4636-bd66-89560bc244ed tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] Releasing lock "refresh_cache-95b6eb15-7930-4aa5-aff8-73809b0a9277" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:15:41 user nova-compute[75234]: DEBUG nova.compute.manager [None req-6f6a7f28-2a0e-4636-bd66-89560bc244ed tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] [instance: 95b6eb15-7930-4aa5-aff8-73809b0a9277] Instance network_info: |[{"id": "64da7883-fc79-44ea-a9d8-65a3f65cfc82", "address": "fa:16:3e:23:9e:1e", "network": {"id": "49d62bf6-34ee-4de8-ab85-6ef489d957c8", "bridge": "br-int", "label": "tempest-VolumesAdminNegativeTest-1304144113-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "5eaa91ebc2e64f2eba987f02bcd1a9f4", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap64da7883-fc", "ovs_interfaceid": "64da7883-fc79-44ea-a9d8-65a3f65cfc82", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=75234) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1987}}
Nov 29 03:15:41 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-6f6a7f28-2a0e-4636-bd66-89560bc244ed tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] [instance: 95b6eb15-7930-4aa5-aff8-73809b0a9277] Start _get_guest_xml network_info=[{"id": "64da7883-fc79-44ea-a9d8-65a3f65cfc82", "address": "fa:16:3e:23:9e:1e", "network": {"id": "49d62bf6-34ee-4de8-ab85-6ef489d957c8", "bridge": "br-int", "label": "tempest-VolumesAdminNegativeTest-1304144113-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "5eaa91ebc2e64f2eba987f02bcd1a9f4", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap64da7883-fc", "ovs_interfaceid": "64da7883-fc79-44ea-a9d8-65a3f65cfc82", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum='c8fc807773e5354afe61636071771906',container_format='bare',created_at=2023-11-29T03:02:15Z,direct_url=<?>,disk_format='qcow2',id=96c30ba7-9b73-4a30-8ba4-7138879573b7,min_disk=0,min_ram=0,name='cirros-0.6.2-x86_64-disk',owner='b18e380921c748b78517b6f1211cd0f1',properties=ImageMetaProps,protected=<?>,size=21430272,status='active',tags=<?>,updated_at=2023-11-29T03:02:17Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [{'device_name': '/dev/vda', 'encryption_format': None, 'encrypted': False, 'device_type': 'disk', 'size': 0, 'disk_bus': 'virtio', 'boot_index': 0, 'encryption_options': None, 'encryption_secret_uuid': None, 'guest_format': None, 'image_id': '96c30ba7-9b73-4a30-8ba4-7138879573b7'}], 'ephemerals': [], 'block_device_mapping': [], 'swap': None} {{(pid=75234) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7562}}
Nov 29 03:15:41 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-6f6a7f28-2a0e-4636-bd66-89560bc244ed tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:15:41 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-6f6a7f28-2a0e-4636-bd66-89560bc244ed tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:15:41 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-6f6a7f28-2a0e-4636-bd66-89560bc244ed tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] Searching host: 'user' for CPU controller through CGroups V1... {{(pid=75234) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1652}}
Nov 29 03:15:41 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-6f6a7f28-2a0e-4636-bd66-89560bc244ed tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] CPU controller missing on host. {{(pid=75234) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1662}}
Nov 29 03:15:41 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-6f6a7f28-2a0e-4636-bd66-89560bc244ed tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] Searching host: 'user' for CPU controller through CGroups V2... {{(pid=75234) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1671}}
Nov 29 03:15:41 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-6f6a7f28-2a0e-4636-bd66-89560bc244ed tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] CPU controller found on host. {{(pid=75234) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1678}}
Nov 29 03:15:41 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-6f6a7f28-2a0e-4636-bd66-89560bc244ed tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=75234) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5400}}
Nov 29 03:15:41 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-6f6a7f28-2a0e-4636-bd66-89560bc244ed tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-11-29T03:02:04Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='1',id=6,is_public=True,memory_mb=512,name='m1.tiny',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='c8fc807773e5354afe61636071771906',container_format='bare',created_at=2023-11-29T03:02:15Z,direct_url=<?>,disk_format='qcow2',id=96c30ba7-9b73-4a30-8ba4-7138879573b7,min_disk=0,min_ram=0,name='cirros-0.6.2-x86_64-disk',owner='b18e380921c748b78517b6f1211cd0f1',properties=ImageMetaProps,protected=<?>,size=21430272,status='active',tags=<?>,updated_at=2023-11-29T03:02:17Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:563}}
Nov 29 03:15:41 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-6f6a7f28-2a0e-4636-bd66-89560bc244ed tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] Flavor limits 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:348}}
Nov 29 03:15:41 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-6f6a7f28-2a0e-4636-bd66-89560bc244ed tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] Image limits 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:352}}
Nov 29 03:15:41 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-6f6a7f28-2a0e-4636-bd66-89560bc244ed tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] Flavor pref 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:388}}
Nov 29 03:15:41 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-6f6a7f28-2a0e-4636-bd66-89560bc244ed tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] Image pref 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:392}}
Nov 29 03:15:41 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-6f6a7f28-2a0e-4636-bd66-89560bc244ed tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:430}}
Nov 29 03:15:41 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-6f6a7f28-2a0e-4636-bd66-89560bc244ed tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:569}}
Nov 29 03:15:41 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-6f6a7f28-2a0e-4636-bd66-89560bc244ed tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=75234) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:471}}
Nov 29 03:15:41 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-6f6a7f28-2a0e-4636-bd66-89560bc244ed tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] Got 1 possible topologies {{(pid=75234) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:501}}
Nov 29 03:15:41 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-6f6a7f28-2a0e-4636-bd66-89560bc244ed tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:575}}
Nov 29 03:15:41 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-6f6a7f28-2a0e-4636-bd66-89560bc244ed tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:577}}
Nov 29 03:15:41 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-6f6a7f28-2a0e-4636-bd66-89560bc244ed tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-11-29T03:15:37Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-VolumesAdminNegativeTest-server-1869912699',display_name='tempest-VolumesAdminNegativeTest-server-1869912699',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-volumesadminnegativetest-server-1869912699',id=24,image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data=None,key_name=None,keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='5eaa91ebc2e64f2eba987f02bcd1a9f4',ramdisk_id='',reservation_id='r-4m5vs2pl',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-VolumesAdminNegativeTest-1445338761',owner_user_name='tempest-VolumesAdminNegativeTest-1445338761-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-11-29T03:15:39Z,user_data=None,user_id='460854be8e05408a80790620f64cf8e1',uuid=95b6eb15-7930-4aa5-aff8-73809b0a9277,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "64da7883-fc79-44ea-a9d8-65a3f65cfc82", "address": "fa:16:3e:23:9e:1e", "network": {"id": "49d62bf6-34ee-4de8-ab85-6ef489d957c8", "bridge": "br-int", "label": "tempest-VolumesAdminNegativeTest-1304144113-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "5eaa91ebc2e64f2eba987f02bcd1a9f4", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap64da7883-fc", "ovs_interfaceid": "64da7883-fc79-44ea-a9d8-65a3f65cfc82", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=75234) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Nov 29 03:15:41 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-6f6a7f28-2a0e-4636-bd66-89560bc244ed tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] Converting VIF {"id": "64da7883-fc79-44ea-a9d8-65a3f65cfc82", "address": "fa:16:3e:23:9e:1e", "network": {"id": "49d62bf6-34ee-4de8-ab85-6ef489d957c8", "bridge": "br-int", "label": "tempest-VolumesAdminNegativeTest-1304144113-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "5eaa91ebc2e64f2eba987f02bcd1a9f4", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap64da7883-fc", "ovs_interfaceid": "64da7883-fc79-44ea-a9d8-65a3f65cfc82", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:15:41 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-6f6a7f28-2a0e-4636-bd66-89560bc244ed tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:23:9e:1e,bridge_name='br-int',has_traffic_filtering=True,id=64da7883-fc79-44ea-a9d8-65a3f65cfc82,network=Network(49d62bf6-34ee-4de8-ab85-6ef489d957c8),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap64da7883-fc') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:15:41 user nova-compute[75234]: DEBUG nova.objects.instance [None req-6f6a7f28-2a0e-4636-bd66-89560bc244ed tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] Lazy-loading 'pci_devices' on Instance uuid 95b6eb15-7930-4aa5-aff8-73809b0a9277 {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:15:41 user nova-compute[75234]: INFO nova.scheduler.client.report [None req-438d669e-c97f-4c6f-bc43-886b9afd2261 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Deleted allocations for instance 6c0e0184-ca3a-48ec-a6e3-a0a1172260d6
Nov 29 03:15:41 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-6f6a7f28-2a0e-4636-bd66-89560bc244ed tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] [instance: 95b6eb15-7930-4aa5-aff8-73809b0a9277] End _get_guest_xml xml=<domain type="kvm">
Nov 29 03:15:41 user nova-compute[75234]:   <uuid>95b6eb15-7930-4aa5-aff8-73809b0a9277</uuid>
Nov 29 03:15:41 user nova-compute[75234]:   <name>instance-00000018</name>
Nov 29 03:15:41 user nova-compute[75234]:   <memory>524288</memory>
Nov 29 03:15:41 user nova-compute[75234]:   <vcpu>1</vcpu>
Nov 29 03:15:41 user nova-compute[75234]:   <metadata>
Nov 29 03:15:41 user nova-compute[75234]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Nov 29 03:15:41 user nova-compute[75234]:       <nova:package version="0.0.0"/>
Nov 29 03:15:41 user nova-compute[75234]:       <nova:name>tempest-VolumesAdminNegativeTest-server-1869912699</nova:name>
Nov 29 03:15:41 user nova-compute[75234]:       <nova:creationTime>2023-11-29 03:15:41</nova:creationTime>
Nov 29 03:15:41 user nova-compute[75234]:       <nova:flavor name="m1.tiny">
Nov 29 03:15:41 user nova-compute[75234]:         <nova:memory>512</nova:memory>
Nov 29 03:15:41 user nova-compute[75234]:         <nova:disk>1</nova:disk>
Nov 29 03:15:41 user nova-compute[75234]:         <nova:swap>0</nova:swap>
Nov 29 03:15:41 user nova-compute[75234]:         <nova:ephemeral>0</nova:ephemeral>
Nov 29 03:15:41 user nova-compute[75234]:         <nova:vcpus>1</nova:vcpus>
Nov 29 03:15:41 user nova-compute[75234]:       </nova:flavor>
Nov 29 03:15:41 user nova-compute[75234]:       <nova:owner>
Nov 29 03:15:41 user nova-compute[75234]:         <nova:user uuid="460854be8e05408a80790620f64cf8e1">tempest-VolumesAdminNegativeTest-1445338761-project-member</nova:user>
Nov 29 03:15:41 user nova-compute[75234]:         <nova:project uuid="5eaa91ebc2e64f2eba987f02bcd1a9f4">tempest-VolumesAdminNegativeTest-1445338761</nova:project>
Nov 29 03:15:41 user nova-compute[75234]:       </nova:owner>
Nov 29 03:15:41 user nova-compute[75234]:       <nova:root type="image" uuid="96c30ba7-9b73-4a30-8ba4-7138879573b7"/>
Nov 29 03:15:41 user nova-compute[75234]:       <nova:ports>
Nov 29 03:15:41 user nova-compute[75234]:         <nova:port uuid="64da7883-fc79-44ea-a9d8-65a3f65cfc82">
Nov 29 03:15:41 user nova-compute[75234]:           <nova:ip type="fixed" address="10.0.0.9" ipVersion="4"/>
Nov 29 03:15:41 user nova-compute[75234]:         </nova:port>
Nov 29 03:15:41 user nova-compute[75234]:       </nova:ports>
Nov 29 03:15:41 user nova-compute[75234]:     </nova:instance>
Nov 29 03:15:41 user nova-compute[75234]:   </metadata>
Nov 29 03:15:41 user nova-compute[75234]:   <sysinfo type="smbios">
Nov 29 03:15:41 user nova-compute[75234]:     <system>
Nov 29 03:15:41 user nova-compute[75234]:       <entry name="manufacturer">OpenStack Foundation</entry>
Nov 29 03:15:41 user nova-compute[75234]:       <entry name="product">OpenStack Nova</entry>
Nov 29 03:15:41 user nova-compute[75234]:       <entry name="version">0.0.0</entry>
Nov 29 03:15:41 user nova-compute[75234]:       <entry name="serial">95b6eb15-7930-4aa5-aff8-73809b0a9277</entry>
Nov 29 03:15:41 user nova-compute[75234]:       <entry name="uuid">95b6eb15-7930-4aa5-aff8-73809b0a9277</entry>
Nov 29 03:15:41 user nova-compute[75234]:       <entry name="family">Virtual Machine</entry>
Nov 29 03:15:41 user nova-compute[75234]:     </system>
Nov 29 03:15:41 user nova-compute[75234]:   </sysinfo>
Nov 29 03:15:41 user nova-compute[75234]:   <os>
Nov 29 03:15:41 user nova-compute[75234]:     <type arch="x86_64" machine="pc">hvm</type>
Nov 29 03:15:41 user nova-compute[75234]:     <boot dev="hd"/>
Nov 29 03:15:41 user nova-compute[75234]:     <smbios mode="sysinfo"/>
Nov 29 03:15:41 user nova-compute[75234]:   </os>
Nov 29 03:15:41 user nova-compute[75234]:   <features>
Nov 29 03:15:41 user nova-compute[75234]:     <acpi/>
Nov 29 03:15:41 user nova-compute[75234]:     <apic/>
Nov 29 03:15:41 user nova-compute[75234]:     <vmcoreinfo/>
Nov 29 03:15:41 user nova-compute[75234]:   </features>
Nov 29 03:15:41 user nova-compute[75234]:   <clock offset="utc">
Nov 29 03:15:41 user nova-compute[75234]:     <timer name="pit" tickpolicy="delay"/>
Nov 29 03:15:41 user nova-compute[75234]:     <timer name="rtc" tickpolicy="catchup"/>
Nov 29 03:15:41 user nova-compute[75234]:     <timer name="hpet" present="no"/>
Nov 29 03:15:41 user nova-compute[75234]:   </clock>
Nov 29 03:15:41 user nova-compute[75234]:   <cpu mode="custom" match="exact">
Nov 29 03:15:41 user nova-compute[75234]:     <model>Nehalem</model>
Nov 29 03:15:41 user nova-compute[75234]:     <topology sockets="1" cores="1" threads="1"/>
Nov 29 03:15:41 user nova-compute[75234]:   </cpu>
Nov 29 03:15:41 user nova-compute[75234]:   <devices>
Nov 29 03:15:41 user nova-compute[75234]:     <disk type="file" device="disk">
Nov 29 03:15:41 user nova-compute[75234]:       <driver name="qemu" type="qcow2" cache="none"/>
Nov 29 03:15:41 user nova-compute[75234]:       <source file="/opt/stack/data/nova/instances/95b6eb15-7930-4aa5-aff8-73809b0a9277/disk"/>
Nov 29 03:15:41 user nova-compute[75234]:       <target dev="vda" bus="virtio"/>
Nov 29 03:15:41 user nova-compute[75234]:     </disk>
Nov 29 03:15:41 user nova-compute[75234]:     <interface type="ethernet">
Nov 29 03:15:41 user nova-compute[75234]:       <mac address="fa:16:3e:23:9e:1e"/>
Nov 29 03:15:41 user nova-compute[75234]:       <model type="virtio"/>
Nov 29 03:15:41 user nova-compute[75234]:       <mtu size="1442"/>
Nov 29 03:15:41 user nova-compute[75234]:       <target dev="tap64da7883-fc"/>
Nov 29 03:15:41 user nova-compute[75234]:     </interface>
Nov 29 03:15:41 user nova-compute[75234]:     <serial type="pty">
Nov 29 03:15:41 user nova-compute[75234]:       <log file="/opt/stack/data/nova/instances/95b6eb15-7930-4aa5-aff8-73809b0a9277/console.log" append="off"/>
Nov 29 03:15:41 user nova-compute[75234]:     </serial>
Nov 29 03:15:41 user nova-compute[75234]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Nov 29 03:15:41 user nova-compute[75234]:     <video>
Nov 29 03:15:41 user nova-compute[75234]:       <model type="virtio"/>
Nov 29 03:15:41 user nova-compute[75234]:     </video>
Nov 29 03:15:41 user nova-compute[75234]:     <rng model="virtio">
Nov 29 03:15:41 user nova-compute[75234]:       <backend model="random">/dev/urandom</backend>
Nov 29 03:15:41 user nova-compute[75234]:     </rng>
Nov 29 03:15:41 user nova-compute[75234]:     <controller type="usb" index="0" model="none"/>
Nov 29 03:15:41 user nova-compute[75234]:     <memballoon model="virtio">
Nov 29 03:15:41 user nova-compute[75234]:       <stats period="10"/>
Nov 29 03:15:41 user nova-compute[75234]:     </memballoon>
Nov 29 03:15:41 user nova-compute[75234]:   </devices>
Nov 29 03:15:41 user nova-compute[75234]: </domain>
Nov 29 03:15:41 user nova-compute[75234]:  {{(pid=75234) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7568}}
Nov 29 03:15:41 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-6f6a7f28-2a0e-4636-bd66-89560bc244ed tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-11-29T03:15:37Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-VolumesAdminNegativeTest-server-1869912699',display_name='tempest-VolumesAdminNegativeTest-server-1869912699',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-volumesadminnegativetest-server-1869912699',id=24,image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data=None,key_name=None,keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='5eaa91ebc2e64f2eba987f02bcd1a9f4',ramdisk_id='',reservation_id='r-4m5vs2pl',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-VolumesAdminNegativeTest-1445338761',owner_user_name='tempest-VolumesAdminNegativeTest-1445338761-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-11-29T03:15:39Z,user_data=None,user_id='460854be8e05408a80790620f64cf8e1',uuid=95b6eb15-7930-4aa5-aff8-73809b0a9277,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "64da7883-fc79-44ea-a9d8-65a3f65cfc82", "address": "fa:16:3e:23:9e:1e", "network": {"id": "49d62bf6-34ee-4de8-ab85-6ef489d957c8", "bridge": "br-int", "label": "tempest-VolumesAdminNegativeTest-1304144113-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "5eaa91ebc2e64f2eba987f02bcd1a9f4", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap64da7883-fc", "ovs_interfaceid": "64da7883-fc79-44ea-a9d8-65a3f65cfc82", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Nov 29 03:15:41 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-6f6a7f28-2a0e-4636-bd66-89560bc244ed tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] Converting VIF {"id": "64da7883-fc79-44ea-a9d8-65a3f65cfc82", "address": "fa:16:3e:23:9e:1e", "network": {"id": "49d62bf6-34ee-4de8-ab85-6ef489d957c8", "bridge": "br-int", "label": "tempest-VolumesAdminNegativeTest-1304144113-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "5eaa91ebc2e64f2eba987f02bcd1a9f4", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap64da7883-fc", "ovs_interfaceid": "64da7883-fc79-44ea-a9d8-65a3f65cfc82", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:15:41 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-6f6a7f28-2a0e-4636-bd66-89560bc244ed tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:23:9e:1e,bridge_name='br-int',has_traffic_filtering=True,id=64da7883-fc79-44ea-a9d8-65a3f65cfc82,network=Network(49d62bf6-34ee-4de8-ab85-6ef489d957c8),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap64da7883-fc') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:15:41 user nova-compute[75234]: DEBUG os_vif [None req-6f6a7f28-2a0e-4636-bd66-89560bc244ed tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:23:9e:1e,bridge_name='br-int',has_traffic_filtering=True,id=64da7883-fc79-44ea-a9d8-65a3f65cfc82,network=Network(49d62bf6-34ee-4de8-ab85-6ef489d957c8),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap64da7883-fc') {{(pid=75234) plug /opt/stack/data/venv/lib/python3.10/site-packages/os_vif/__init__.py:76}}
Nov 29 03:15:41 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:15:41 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:15:41 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Nov 29 03:15:41 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:15:41 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbCreateCommand(_result=None, table=QoS, columns={'type': 'linux-noop', 'external_ids': {'id': '52983701-f532-57c8-b150-61ac2cb1cc62', '_type': 'linux-noop'}}, row=False) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:15:41 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:15:41 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:15:41 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:15:41 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:15:41 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tap64da7883-fc, may_exist=True, interface_attrs={}) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:15:41 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Port, record=tap64da7883-fc, col_values=(('qos', UUID('5d6fb3b1-1647-457f-9091-27df26eb3a7c')),), if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:15:41 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DbSetCommand(_result=None, table=Interface, record=tap64da7883-fc, col_values=(('external_ids', {'iface-id': '64da7883-fc79-44ea-a9d8-65a3f65cfc82', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:23:9e:1e', 'vm-uuid': '95b6eb15-7930-4aa5-aff8-73809b0a9277'}),), if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:15:41 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:15:41 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:15:41 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:15:41 user nova-compute[75234]: INFO os_vif [None req-6f6a7f28-2a0e-4636-bd66-89560bc244ed tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:23:9e:1e,bridge_name='br-int',has_traffic_filtering=True,id=64da7883-fc79-44ea-a9d8-65a3f65cfc82,network=Network(49d62bf6-34ee-4de8-ab85-6ef489d957c8),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap64da7883-fc')
Nov 29 03:15:41 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-438d669e-c97f-4c6f-bc43-886b9afd2261 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Lock "6c0e0184-ca3a-48ec-a6e3-a0a1172260d6" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 3.644s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:15:41 user nova-compute[75234]: DEBUG nova.compute.manager [req-821d17d0-f49f-4434-b35d-a142dc7a9415 req-0ff209b8-90a2-4fed-a051-525111009484 service nova] [instance: 062753a6-8f39-4e80-a549-9f0f5a4b877a] Received event network-vif-unplugged-8f6751f1-fc65-4842-b77f-1dfe26912afb {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:15:41 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-821d17d0-f49f-4434-b35d-a142dc7a9415 req-0ff209b8-90a2-4fed-a051-525111009484 service nova] Acquiring lock "062753a6-8f39-4e80-a549-9f0f5a4b877a-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:15:41 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-821d17d0-f49f-4434-b35d-a142dc7a9415 req-0ff209b8-90a2-4fed-a051-525111009484 service nova] Lock "062753a6-8f39-4e80-a549-9f0f5a4b877a-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:15:41 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-821d17d0-f49f-4434-b35d-a142dc7a9415 req-0ff209b8-90a2-4fed-a051-525111009484 service nova] Lock "062753a6-8f39-4e80-a549-9f0f5a4b877a-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:15:41 user nova-compute[75234]: DEBUG nova.compute.manager [req-821d17d0-f49f-4434-b35d-a142dc7a9415 req-0ff209b8-90a2-4fed-a051-525111009484 service nova] [instance: 062753a6-8f39-4e80-a549-9f0f5a4b877a] No waiting events found dispatching network-vif-unplugged-8f6751f1-fc65-4842-b77f-1dfe26912afb {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:15:41 user nova-compute[75234]: DEBUG nova.compute.manager [req-821d17d0-f49f-4434-b35d-a142dc7a9415 req-0ff209b8-90a2-4fed-a051-525111009484 service nova] [instance: 062753a6-8f39-4e80-a549-9f0f5a4b877a] Received event network-vif-unplugged-8f6751f1-fc65-4842-b77f-1dfe26912afb for instance with task_state deleting. {{(pid=75234) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10890}}
Nov 29 03:15:41 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-6f6a7f28-2a0e-4636-bd66-89560bc244ed tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] No BDM found with device name vda, not building metadata. {{(pid=75234) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12089}}
Nov 29 03:15:41 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-6f6a7f28-2a0e-4636-bd66-89560bc244ed tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] No VIF found with MAC fa:16:3e:23:9e:1e, not building metadata {{(pid=75234) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Nov 29 03:15:41 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:15:42 user nova-compute[75234]: INFO nova.virt.libvirt.driver [-] [instance: 062753a6-8f39-4e80-a549-9f0f5a4b877a] Instance destroyed successfully.
Nov 29 03:15:42 user nova-compute[75234]: DEBUG nova.objects.instance [None req-6732f0fb-d7c6-4c58-b125-63f3c8fe5cb5 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Lazy-loading 'resources' on Instance uuid 062753a6-8f39-4e80-a549-9f0f5a4b877a {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:15:42 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-6732f0fb-d7c6-4c58-b125-63f3c8fe5cb5 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=True,compute_id=1,config_drive='',created_at=2023-11-29T03:09:11Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description=None,display_name='tempest-AttachVolumeShelveTestJSON-server-2002337175',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-attachvolumeshelvetestjson-server-2002337175',id=14,image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBJL1BPeb1pVpN5A24HF0gsy7slhuswFxR+GxX0R0LvMX9QS8GL1hvY7OFzvrLw6S/VLmgqgKQ7HFXhrJDiWyf/i/jTyWtVuorLQtfVudKMIFZjmVfY2JDvM7kwU/YbL6jQ==',key_name='tempest-keypair-1961974262',keypairs=<?>,launch_index=0,launched_at=2023-11-29T03:13:29Z,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='e17150005a9a449f9373b91e1c00083b',ramdisk_id='',reservation_id='r-p3yfidye',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',clean_attempts='1',image_base_image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-AttachVolumeShelveTestJSON-1180535246',owner_user_name='tempest-AttachVolumeShelveTestJSON-1180535246-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-11-29T03:13:29Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='154f3f4f57514450baf8f22d67768bc9',uuid=062753a6-8f39-4e80-a549-9f0f5a4b877a,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "8f6751f1-fc65-4842-b77f-1dfe26912afb", "address": "fa:16:3e:62:4a:5a", "network": {"id": "aad085f7-106b-4de6-93f5-f86bb66ec962", "bridge": "br-int", "label": "tempest-AttachVolumeShelveTestJSON-775984268-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.175", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "e17150005a9a449f9373b91e1c00083b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap8f6751f1-fc", "ovs_interfaceid": "8f6751f1-fc65-4842-b77f-1dfe26912afb", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Nov 29 03:15:42 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-6732f0fb-d7c6-4c58-b125-63f3c8fe5cb5 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Converting VIF {"id": "8f6751f1-fc65-4842-b77f-1dfe26912afb", "address": "fa:16:3e:62:4a:5a", "network": {"id": "aad085f7-106b-4de6-93f5-f86bb66ec962", "bridge": "br-int", "label": "tempest-AttachVolumeShelveTestJSON-775984268-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.175", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "e17150005a9a449f9373b91e1c00083b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap8f6751f1-fc", "ovs_interfaceid": "8f6751f1-fc65-4842-b77f-1dfe26912afb", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:15:42 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-6732f0fb-d7c6-4c58-b125-63f3c8fe5cb5 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:62:4a:5a,bridge_name='br-int',has_traffic_filtering=True,id=8f6751f1-fc65-4842-b77f-1dfe26912afb,network=Network(aad085f7-106b-4de6-93f5-f86bb66ec962),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap8f6751f1-fc') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:15:42 user nova-compute[75234]: DEBUG os_vif [None req-6732f0fb-d7c6-4c58-b125-63f3c8fe5cb5 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Unplugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:62:4a:5a,bridge_name='br-int',has_traffic_filtering=True,id=8f6751f1-fc65-4842-b77f-1dfe26912afb,network=Network(aad085f7-106b-4de6-93f5-f86bb66ec962),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap8f6751f1-fc') {{(pid=75234) unplug /opt/stack/data/venv/lib/python3.10/site-packages/os_vif/__init__.py:109}}
Nov 29 03:15:42 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:15:42 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tap8f6751f1-fc, bridge=br-int, if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:15:42 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:15:42 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:15:42 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:15:42 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:15:42 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbDestroyCommand(_result=None, table=QoS, record=6433206e-6949-44ca-88eb-706d1ca0c695) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:15:42 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:15:42 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:15:42 user nova-compute[75234]: INFO os_vif [None req-6732f0fb-d7c6-4c58-b125-63f3c8fe5cb5 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Successfully unplugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:62:4a:5a,bridge_name='br-int',has_traffic_filtering=True,id=8f6751f1-fc65-4842-b77f-1dfe26912afb,network=Network(aad085f7-106b-4de6-93f5-f86bb66ec962),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap8f6751f1-fc')
Nov 29 03:15:42 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-6732f0fb-d7c6-4c58-b125-63f3c8fe5cb5 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] [instance: 062753a6-8f39-4e80-a549-9f0f5a4b877a] Deleting instance files /opt/stack/data/nova/instances/062753a6-8f39-4e80-a549-9f0f5a4b877a_del
Nov 29 03:15:42 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-6732f0fb-d7c6-4c58-b125-63f3c8fe5cb5 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] [instance: 062753a6-8f39-4e80-a549-9f0f5a4b877a] Deletion of /opt/stack/data/nova/instances/062753a6-8f39-4e80-a549-9f0f5a4b877a_del complete
Nov 29 03:15:42 user nova-compute[75234]: INFO nova.compute.manager [None req-6732f0fb-d7c6-4c58-b125-63f3c8fe5cb5 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] [instance: 062753a6-8f39-4e80-a549-9f0f5a4b877a] Took 1.26 seconds to destroy the instance on the hypervisor.
Nov 29 03:15:42 user nova-compute[75234]: DEBUG oslo.service.loopingcall [None req-6732f0fb-d7c6-4c58-b125-63f3c8fe5cb5 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=75234) func /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/loopingcall.py:435}}
Nov 29 03:15:42 user nova-compute[75234]: DEBUG nova.compute.manager [-] [instance: 062753a6-8f39-4e80-a549-9f0f5a4b877a] Deallocating network for instance {{(pid=75234) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2279}}
Nov 29 03:15:42 user nova-compute[75234]: DEBUG nova.network.neutron [-] [instance: 062753a6-8f39-4e80-a549-9f0f5a4b877a] deallocate_for_instance() {{(pid=75234) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1802}}
Nov 29 03:15:42 user nova-compute[75234]: DEBUG nova.compute.manager [req-a0c335a3-9b9a-451f-994f-67c6ad53d657 req-73ffb728-a172-41cf-b9c8-7549cf63e7af service nova] [instance: 95b6eb15-7930-4aa5-aff8-73809b0a9277] Received event network-changed-64da7883-fc79-44ea-a9d8-65a3f65cfc82 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:15:42 user nova-compute[75234]: DEBUG nova.compute.manager [req-a0c335a3-9b9a-451f-994f-67c6ad53d657 req-73ffb728-a172-41cf-b9c8-7549cf63e7af service nova] [instance: 95b6eb15-7930-4aa5-aff8-73809b0a9277] Refreshing instance network info cache due to event network-changed-64da7883-fc79-44ea-a9d8-65a3f65cfc82. {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11117}}
Nov 29 03:15:42 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-a0c335a3-9b9a-451f-994f-67c6ad53d657 req-73ffb728-a172-41cf-b9c8-7549cf63e7af service nova] Acquiring lock "refresh_cache-95b6eb15-7930-4aa5-aff8-73809b0a9277" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:15:42 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-a0c335a3-9b9a-451f-994f-67c6ad53d657 req-73ffb728-a172-41cf-b9c8-7549cf63e7af service nova] Acquired lock "refresh_cache-95b6eb15-7930-4aa5-aff8-73809b0a9277" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:15:42 user nova-compute[75234]: DEBUG nova.network.neutron [req-a0c335a3-9b9a-451f-994f-67c6ad53d657 req-73ffb728-a172-41cf-b9c8-7549cf63e7af service nova] [instance: 95b6eb15-7930-4aa5-aff8-73809b0a9277] Refreshing network info cache for port 64da7883-fc79-44ea-a9d8-65a3f65cfc82 {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2006}}
Nov 29 03:15:42 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:15:43 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:15:43 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:15:43 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:15:43 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:15:43 user nova-compute[75234]: DEBUG nova.network.neutron [req-a0c335a3-9b9a-451f-994f-67c6ad53d657 req-73ffb728-a172-41cf-b9c8-7549cf63e7af service nova] [instance: 95b6eb15-7930-4aa5-aff8-73809b0a9277] Updated VIF entry in instance network info cache for port 64da7883-fc79-44ea-a9d8-65a3f65cfc82. {{(pid=75234) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3481}}
Nov 29 03:15:43 user nova-compute[75234]: DEBUG nova.network.neutron [req-a0c335a3-9b9a-451f-994f-67c6ad53d657 req-73ffb728-a172-41cf-b9c8-7549cf63e7af service nova] [instance: 95b6eb15-7930-4aa5-aff8-73809b0a9277] Updating instance_info_cache with network_info: [{"id": "64da7883-fc79-44ea-a9d8-65a3f65cfc82", "address": "fa:16:3e:23:9e:1e", "network": {"id": "49d62bf6-34ee-4de8-ab85-6ef489d957c8", "bridge": "br-int", "label": "tempest-VolumesAdminNegativeTest-1304144113-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "5eaa91ebc2e64f2eba987f02bcd1a9f4", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap64da7883-fc", "ovs_interfaceid": "64da7883-fc79-44ea-a9d8-65a3f65cfc82", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:15:43 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-a0c335a3-9b9a-451f-994f-67c6ad53d657 req-73ffb728-a172-41cf-b9c8-7549cf63e7af service nova] Releasing lock "refresh_cache-95b6eb15-7930-4aa5-aff8-73809b0a9277" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:15:43 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:15:43 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:15:43 user nova-compute[75234]: DEBUG nova.compute.manager [req-7bec3023-28d1-4399-addd-15ad44583ae8 req-b90e93ea-e381-467e-931a-0d586c9d3b4b service nova] [instance: 95b6eb15-7930-4aa5-aff8-73809b0a9277] Received event network-vif-plugged-64da7883-fc79-44ea-a9d8-65a3f65cfc82 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:15:43 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-7bec3023-28d1-4399-addd-15ad44583ae8 req-b90e93ea-e381-467e-931a-0d586c9d3b4b service nova] Acquiring lock "95b6eb15-7930-4aa5-aff8-73809b0a9277-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:15:43 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-7bec3023-28d1-4399-addd-15ad44583ae8 req-b90e93ea-e381-467e-931a-0d586c9d3b4b service nova] Lock "95b6eb15-7930-4aa5-aff8-73809b0a9277-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:15:43 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-7bec3023-28d1-4399-addd-15ad44583ae8 req-b90e93ea-e381-467e-931a-0d586c9d3b4b service nova] Lock "95b6eb15-7930-4aa5-aff8-73809b0a9277-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:15:43 user nova-compute[75234]: DEBUG nova.compute.manager [req-7bec3023-28d1-4399-addd-15ad44583ae8 req-b90e93ea-e381-467e-931a-0d586c9d3b4b service nova] [instance: 95b6eb15-7930-4aa5-aff8-73809b0a9277] No waiting events found dispatching network-vif-plugged-64da7883-fc79-44ea-a9d8-65a3f65cfc82 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:15:43 user nova-compute[75234]: WARNING nova.compute.manager [req-7bec3023-28d1-4399-addd-15ad44583ae8 req-b90e93ea-e381-467e-931a-0d586c9d3b4b service nova] [instance: 95b6eb15-7930-4aa5-aff8-73809b0a9277] Received unexpected event network-vif-plugged-64da7883-fc79-44ea-a9d8-65a3f65cfc82 for instance with vm_state building and task_state spawning.
Nov 29 03:15:43 user nova-compute[75234]: DEBUG nova.compute.manager [req-47e7f17b-2df9-47a1-995a-6649204cabcf req-81e41be0-bfa0-4e37-b7f4-7b49a9df55ee service nova] [instance: 062753a6-8f39-4e80-a549-9f0f5a4b877a] Received event network-vif-plugged-8f6751f1-fc65-4842-b77f-1dfe26912afb {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:15:43 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-47e7f17b-2df9-47a1-995a-6649204cabcf req-81e41be0-bfa0-4e37-b7f4-7b49a9df55ee service nova] Acquiring lock "062753a6-8f39-4e80-a549-9f0f5a4b877a-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:15:43 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-47e7f17b-2df9-47a1-995a-6649204cabcf req-81e41be0-bfa0-4e37-b7f4-7b49a9df55ee service nova] Lock "062753a6-8f39-4e80-a549-9f0f5a4b877a-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:15:43 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-47e7f17b-2df9-47a1-995a-6649204cabcf req-81e41be0-bfa0-4e37-b7f4-7b49a9df55ee service nova] Lock "062753a6-8f39-4e80-a549-9f0f5a4b877a-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.002s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:15:43 user nova-compute[75234]: DEBUG nova.compute.manager [req-47e7f17b-2df9-47a1-995a-6649204cabcf req-81e41be0-bfa0-4e37-b7f4-7b49a9df55ee service nova] [instance: 062753a6-8f39-4e80-a549-9f0f5a4b877a] No waiting events found dispatching network-vif-plugged-8f6751f1-fc65-4842-b77f-1dfe26912afb {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:15:43 user nova-compute[75234]: WARNING nova.compute.manager [req-47e7f17b-2df9-47a1-995a-6649204cabcf req-81e41be0-bfa0-4e37-b7f4-7b49a9df55ee service nova] [instance: 062753a6-8f39-4e80-a549-9f0f5a4b877a] Received unexpected event network-vif-plugged-8f6751f1-fc65-4842-b77f-1dfe26912afb for instance with vm_state active and task_state deleting.
Nov 29 03:15:43 user nova-compute[75234]: DEBUG nova.network.neutron [-] [instance: 062753a6-8f39-4e80-a549-9f0f5a4b877a] Updating instance_info_cache with network_info: [] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:15:43 user nova-compute[75234]: INFO nova.compute.manager [-] [instance: 062753a6-8f39-4e80-a549-9f0f5a4b877a] Took 1.59 seconds to deallocate network for instance.
Nov 29 03:15:43 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-6732f0fb-d7c6-4c58-b125-63f3c8fe5cb5 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:15:43 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-6732f0fb-d7c6-4c58-b125-63f3c8fe5cb5 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:15:44 user nova-compute[75234]: DEBUG nova.compute.provider_tree [None req-6732f0fb-d7c6-4c58-b125-63f3c8fe5cb5 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Inventory has not changed in ProviderTree for provider: 4d291220-7c37-4839-a0e3-b23c66502b74 {{(pid=75234) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Nov 29 03:15:44 user nova-compute[75234]: DEBUG nova.scheduler.client.report [None req-6732f0fb-d7c6-4c58-b125-63f3c8fe5cb5 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Inventory has not changed for provider 4d291220-7c37-4839-a0e3-b23c66502b74 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=75234) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Nov 29 03:15:44 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-6732f0fb-d7c6-4c58-b125-63f3c8fe5cb5 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.449s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:15:44 user nova-compute[75234]: INFO nova.scheduler.client.report [None req-6732f0fb-d7c6-4c58-b125-63f3c8fe5cb5 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Deleted allocations for instance 062753a6-8f39-4e80-a549-9f0f5a4b877a
Nov 29 03:15:44 user nova-compute[75234]: DEBUG nova.compute.manager [req-4b344840-224b-4132-9c7c-4631b7b1cbc1 req-a047be9a-933b-4184-a386-92ca684d45f5 service nova] [instance: 062753a6-8f39-4e80-a549-9f0f5a4b877a] Received event network-vif-deleted-8f6751f1-fc65-4842-b77f-1dfe26912afb {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:15:44 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-6732f0fb-d7c6-4c58-b125-63f3c8fe5cb5 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Lock "062753a6-8f39-4e80-a549-9f0f5a4b877a" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 3.600s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:15:45 user nova-compute[75234]: DEBUG nova.compute.manager [None req-6f6a7f28-2a0e-4636-bd66-89560bc244ed tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] [instance: 95b6eb15-7930-4aa5-aff8-73809b0a9277] Instance event wait completed in 0 seconds for  {{(pid=75234) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Nov 29 03:15:45 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-6f6a7f28-2a0e-4636-bd66-89560bc244ed tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] [instance: 95b6eb15-7930-4aa5-aff8-73809b0a9277] Guest created on hypervisor {{(pid=75234) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4421}}
Nov 29 03:15:45 user nova-compute[75234]: INFO nova.virt.libvirt.driver [-] [instance: 95b6eb15-7930-4aa5-aff8-73809b0a9277] Instance spawned successfully.
Nov 29 03:15:45 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-6f6a7f28-2a0e-4636-bd66-89560bc244ed tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] [instance: 95b6eb15-7930-4aa5-aff8-73809b0a9277] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:904}}
Nov 29 03:15:45 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-6f6a7f28-2a0e-4636-bd66-89560bc244ed tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] [instance: 95b6eb15-7930-4aa5-aff8-73809b0a9277] Found default for hw_cdrom_bus of ide {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:15:45 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-6f6a7f28-2a0e-4636-bd66-89560bc244ed tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] [instance: 95b6eb15-7930-4aa5-aff8-73809b0a9277] Found default for hw_disk_bus of virtio {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:15:45 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-6f6a7f28-2a0e-4636-bd66-89560bc244ed tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] [instance: 95b6eb15-7930-4aa5-aff8-73809b0a9277] Found default for hw_input_bus of None {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:15:45 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-6f6a7f28-2a0e-4636-bd66-89560bc244ed tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] [instance: 95b6eb15-7930-4aa5-aff8-73809b0a9277] Found default for hw_pointer_model of None {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:15:45 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-6f6a7f28-2a0e-4636-bd66-89560bc244ed tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] [instance: 95b6eb15-7930-4aa5-aff8-73809b0a9277] Found default for hw_video_model of virtio {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:15:45 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-6f6a7f28-2a0e-4636-bd66-89560bc244ed tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] [instance: 95b6eb15-7930-4aa5-aff8-73809b0a9277] Found default for hw_vif_model of virtio {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:15:45 user nova-compute[75234]: DEBUG nova.compute.manager [req-90f73946-a15e-4feb-be9d-1801fb0bde78 req-ffa960fc-a5a9-45b6-a8af-41d5b3aeff9a service nova] [instance: 95b6eb15-7930-4aa5-aff8-73809b0a9277] Received event network-vif-plugged-64da7883-fc79-44ea-a9d8-65a3f65cfc82 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:15:45 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-90f73946-a15e-4feb-be9d-1801fb0bde78 req-ffa960fc-a5a9-45b6-a8af-41d5b3aeff9a service nova] Acquiring lock "95b6eb15-7930-4aa5-aff8-73809b0a9277-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:15:45 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-90f73946-a15e-4feb-be9d-1801fb0bde78 req-ffa960fc-a5a9-45b6-a8af-41d5b3aeff9a service nova] Lock "95b6eb15-7930-4aa5-aff8-73809b0a9277-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:15:45 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-90f73946-a15e-4feb-be9d-1801fb0bde78 req-ffa960fc-a5a9-45b6-a8af-41d5b3aeff9a service nova] Lock "95b6eb15-7930-4aa5-aff8-73809b0a9277-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:15:45 user nova-compute[75234]: DEBUG nova.compute.manager [req-90f73946-a15e-4feb-be9d-1801fb0bde78 req-ffa960fc-a5a9-45b6-a8af-41d5b3aeff9a service nova] [instance: 95b6eb15-7930-4aa5-aff8-73809b0a9277] No waiting events found dispatching network-vif-plugged-64da7883-fc79-44ea-a9d8-65a3f65cfc82 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:15:45 user nova-compute[75234]: WARNING nova.compute.manager [req-90f73946-a15e-4feb-be9d-1801fb0bde78 req-ffa960fc-a5a9-45b6-a8af-41d5b3aeff9a service nova] [instance: 95b6eb15-7930-4aa5-aff8-73809b0a9277] Received unexpected event network-vif-plugged-64da7883-fc79-44ea-a9d8-65a3f65cfc82 for instance with vm_state building and task_state spawning.
Nov 29 03:15:45 user nova-compute[75234]: INFO nova.compute.manager [None req-6f6a7f28-2a0e-4636-bd66-89560bc244ed tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] [instance: 95b6eb15-7930-4aa5-aff8-73809b0a9277] Took 6.42 seconds to spawn the instance on the hypervisor.
Nov 29 03:15:45 user nova-compute[75234]: DEBUG nova.compute.manager [None req-6f6a7f28-2a0e-4636-bd66-89560bc244ed tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] [instance: 95b6eb15-7930-4aa5-aff8-73809b0a9277] Checking state {{(pid=75234) _get_power_state /opt/stack/nova/nova/compute/manager.py:1782}}
Nov 29 03:15:45 user nova-compute[75234]: INFO nova.compute.manager [None req-6f6a7f28-2a0e-4636-bd66-89560bc244ed tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] [instance: 95b6eb15-7930-4aa5-aff8-73809b0a9277] Took 7.41 seconds to build instance.
Nov 29 03:15:45 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-6f6a7f28-2a0e-4636-bd66-89560bc244ed tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] Lock "95b6eb15-7930-4aa5-aff8-73809b0a9277" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 7.528s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:15:46 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:15:47 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:15:47 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:15:48 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:15:50 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:15:51 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:15:52 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:15:53 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:15:54 user nova-compute[75234]: DEBUG nova.compute.manager [None req-c4ff21a8-705d-4040-8f98-de9bf08a07fe tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] [instance: 6894b55f-93c6-43c9-8933-56d5a3c43725] Checking state {{(pid=75234) _get_power_state /opt/stack/nova/nova/compute/manager.py:1782}}
Nov 29 03:15:54 user nova-compute[75234]: INFO nova.compute.manager [None req-c4ff21a8-705d-4040-8f98-de9bf08a07fe tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] [instance: 6894b55f-93c6-43c9-8933-56d5a3c43725] instance snapshotting
Nov 29 03:15:54 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-c4ff21a8-705d-4040-8f98-de9bf08a07fe tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] [instance: 6894b55f-93c6-43c9-8933-56d5a3c43725] Beginning live snapshot process
Nov 29 03:15:54 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-c4ff21a8-705d-4040-8f98-de9bf08a07fe tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6894b55f-93c6-43c9-8933-56d5a3c43725/disk --force-share --output=json -f qcow2 {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:15:54 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:15:54 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Starting heal instance info cache {{(pid=75234) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9922}}
Nov 29 03:15:54 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Acquiring lock "refresh_cache-f609a7b7-1be2-44b4-9177-7e72a4d4c060" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:15:54 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Acquired lock "refresh_cache-f609a7b7-1be2-44b4-9177-7e72a4d4c060" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:15:54 user nova-compute[75234]: DEBUG nova.network.neutron [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] [instance: f609a7b7-1be2-44b4-9177-7e72a4d4c060] Forcefully refreshing network info cache for instance {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2003}}
Nov 29 03:15:54 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-c4ff21a8-705d-4040-8f98-de9bf08a07fe tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6894b55f-93c6-43c9-8933-56d5a3c43725/disk --force-share --output=json -f qcow2" returned: 0 in 0.223s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:15:54 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-c4ff21a8-705d-4040-8f98-de9bf08a07fe tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6894b55f-93c6-43c9-8933-56d5a3c43725/disk --force-share --output=json -f qcow2 {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:15:54 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-c4ff21a8-705d-4040-8f98-de9bf08a07fe tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6894b55f-93c6-43c9-8933-56d5a3c43725/disk --force-share --output=json -f qcow2" returned: 0 in 0.196s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:15:54 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-c4ff21a8-705d-4040-8f98-de9bf08a07fe tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:15:55 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-c4ff21a8-705d-4040-8f98-de9bf08a07fe tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json" returned: 0 in 0.196s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:15:55 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-c4ff21a8-705d-4040-8f98-de9bf08a07fe tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918,backing_fmt=raw /opt/stack/data/nova/instances/snapshots/tmpdiw7j4gr/3dcf9620a1234222844d3ccaa8fc521e.delta 1073741824 {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:15:55 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-c4ff21a8-705d-4040-8f98-de9bf08a07fe tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918,backing_fmt=raw /opt/stack/data/nova/instances/snapshots/tmpdiw7j4gr/3dcf9620a1234222844d3ccaa8fc521e.delta 1073741824" returned: 0 in 0.123s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:15:55 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-c4ff21a8-705d-4040-8f98-de9bf08a07fe tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] [instance: 6894b55f-93c6-43c9-8933-56d5a3c43725] Quiescing instance not available: QEMU guest agent is not enabled.
Nov 29 03:15:55 user nova-compute[75234]: DEBUG nova.network.neutron [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] [instance: f609a7b7-1be2-44b4-9177-7e72a4d4c060] Updating instance_info_cache with network_info: [{"id": "5543f379-2b3a-4dc2-bfe9-52f66a942257", "address": "fa:16:3e:04:ae:43", "network": {"id": "294c3aac-8c9b-45f0-8c0a-344e2d7d8b7c", "bridge": "br-int", "label": "tempest-AttachVolumeTestJSON-700098935-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.14", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "e393d23e783b4a40986674d782606c69", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap5543f379-2b", "ovs_interfaceid": "5543f379-2b3a-4dc2-bfe9-52f66a942257", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:15:55 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Releasing lock "refresh_cache-f609a7b7-1be2-44b4-9177-7e72a4d4c060" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:15:55 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] [instance: f609a7b7-1be2-44b4-9177-7e72a4d4c060] Updated the network info_cache for instance {{(pid=75234) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9993}}
Nov 29 03:15:55 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:15:55 user nova-compute[75234]: DEBUG nova.virt.libvirt.guest [None req-c4ff21a8-705d-4040-8f98-de9bf08a07fe tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] COPY block job progress, current cursor: 0 final cursor: 75563008 {{(pid=75234) is_job_complete /opt/stack/nova/nova/virt/libvirt/guest.py:846}}
Nov 29 03:15:56 user nova-compute[75234]: DEBUG nova.virt.libvirt.guest [None req-c4ff21a8-705d-4040-8f98-de9bf08a07fe tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] COPY block job progress, current cursor: 25886720 final cursor: 75563008 {{(pid=75234) is_job_complete /opt/stack/nova/nova/virt/libvirt/guest.py:846}}
Nov 29 03:15:56 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:15:56 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:15:56 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-6eb16830-459a-4a21-82ec-0c37c9633f72 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Acquiring lock "75c5d700-6367-4128-9228-80d2cd24ace7" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:15:56 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-6eb16830-459a-4a21-82ec-0c37c9633f72 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Lock "75c5d700-6367-4128-9228-80d2cd24ace7" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:15:56 user nova-compute[75234]: DEBUG nova.compute.manager [None req-6eb16830-459a-4a21-82ec-0c37c9633f72 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] [instance: 75c5d700-6367-4128-9228-80d2cd24ace7] Starting instance... {{(pid=75234) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2422}}
Nov 29 03:15:56 user nova-compute[75234]: DEBUG nova.virt.libvirt.guest [None req-c4ff21a8-705d-4040-8f98-de9bf08a07fe tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] COPY block job progress, current cursor: 75563008 final cursor: 75563008 {{(pid=75234) is_job_complete /opt/stack/nova/nova/virt/libvirt/guest.py:846}}
Nov 29 03:15:56 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-c4ff21a8-705d-4040-8f98-de9bf08a07fe tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] [instance: 6894b55f-93c6-43c9-8933-56d5a3c43725] Skipping quiescing instance: QEMU guest agent is not enabled.
Nov 29 03:15:57 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-6eb16830-459a-4a21-82ec-0c37c9633f72 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:15:57 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-6eb16830-459a-4a21-82ec-0c37c9633f72 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.002s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:15:57 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:15:57 user nova-compute[75234]: DEBUG nova.privsep.utils [None req-c4ff21a8-705d-4040-8f98-de9bf08a07fe tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Path '/opt/stack/data/nova/instances' supports direct I/O {{(pid=75234) supports_direct_io /opt/stack/nova/nova/privsep/utils.py:63}}
Nov 29 03:15:57 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-c4ff21a8-705d-4040-8f98-de9bf08a07fe tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Running cmd (subprocess): qemu-img convert -t none -O qcow2 -f qcow2 /opt/stack/data/nova/instances/snapshots/tmpdiw7j4gr/3dcf9620a1234222844d3ccaa8fc521e.delta /opt/stack/data/nova/instances/snapshots/tmpdiw7j4gr/3dcf9620a1234222844d3ccaa8fc521e {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:15:57 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-6eb16830-459a-4a21-82ec-0c37c9633f72 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=75234) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2366}}
Nov 29 03:15:57 user nova-compute[75234]: INFO nova.compute.claims [None req-6eb16830-459a-4a21-82ec-0c37c9633f72 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] [instance: 75c5d700-6367-4128-9228-80d2cd24ace7] Claim successful on node user
Nov 29 03:15:57 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:15:57 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:15:57 user nova-compute[75234]: DEBUG nova.compute.provider_tree [None req-6eb16830-459a-4a21-82ec-0c37c9633f72 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Inventory has not changed in ProviderTree for provider: 4d291220-7c37-4839-a0e3-b23c66502b74 {{(pid=75234) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Nov 29 03:15:57 user nova-compute[75234]: DEBUG nova.scheduler.client.report [None req-6eb16830-459a-4a21-82ec-0c37c9633f72 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Inventory has not changed for provider 4d291220-7c37-4839-a0e3-b23c66502b74 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=75234) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Nov 29 03:15:57 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-c4ff21a8-705d-4040-8f98-de9bf08a07fe tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] CMD "qemu-img convert -t none -O qcow2 -f qcow2 /opt/stack/data/nova/instances/snapshots/tmpdiw7j4gr/3dcf9620a1234222844d3ccaa8fc521e.delta /opt/stack/data/nova/instances/snapshots/tmpdiw7j4gr/3dcf9620a1234222844d3ccaa8fc521e" returned: 0 in 0.855s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:15:57 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-c4ff21a8-705d-4040-8f98-de9bf08a07fe tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] [instance: 6894b55f-93c6-43c9-8933-56d5a3c43725] Snapshot extracted, beginning image upload
Nov 29 03:15:58 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-6eb16830-459a-4a21-82ec-0c37c9633f72 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.931s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:15:58 user nova-compute[75234]: DEBUG nova.compute.manager [None req-6eb16830-459a-4a21-82ec-0c37c9633f72 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] [instance: 75c5d700-6367-4128-9228-80d2cd24ace7] Start building networks asynchronously for instance. {{(pid=75234) _build_resources /opt/stack/nova/nova/compute/manager.py:2819}}
Nov 29 03:15:58 user nova-compute[75234]: DEBUG nova.compute.manager [None req-6eb16830-459a-4a21-82ec-0c37c9633f72 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] [instance: 75c5d700-6367-4128-9228-80d2cd24ace7] Allocating IP information in the background. {{(pid=75234) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1972}}
Nov 29 03:15:58 user nova-compute[75234]: DEBUG nova.network.neutron [None req-6eb16830-459a-4a21-82ec-0c37c9633f72 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] [instance: 75c5d700-6367-4128-9228-80d2cd24ace7] allocate_for_instance() {{(pid=75234) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1156}}
Nov 29 03:15:58 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-6eb16830-459a-4a21-82ec-0c37c9633f72 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] [instance: 75c5d700-6367-4128-9228-80d2cd24ace7] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Nov 29 03:15:58 user nova-compute[75234]: DEBUG nova.compute.manager [None req-6eb16830-459a-4a21-82ec-0c37c9633f72 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] [instance: 75c5d700-6367-4128-9228-80d2cd24ace7] Start building block device mappings for instance. {{(pid=75234) _build_resources /opt/stack/nova/nova/compute/manager.py:2854}}
Nov 29 03:15:58 user nova-compute[75234]: DEBUG nova.policy [None req-6eb16830-459a-4a21-82ec-0c37c9633f72 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '66686114450f4f31b3a917667a152d8b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '315732b6e3ee44d584b8e510ef199a90', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=75234) authorize /opt/stack/nova/nova/policy.py:203}}
Nov 29 03:15:58 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:15:58 user nova-compute[75234]: DEBUG nova.compute.manager [None req-6eb16830-459a-4a21-82ec-0c37c9633f72 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] [instance: 75c5d700-6367-4128-9228-80d2cd24ace7] Start spawning the instance on the hypervisor. {{(pid=75234) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2628}}
Nov 29 03:15:58 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-6eb16830-459a-4a21-82ec-0c37c9633f72 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] [instance: 75c5d700-6367-4128-9228-80d2cd24ace7] Creating instance directory {{(pid=75234) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4727}}
Nov 29 03:15:58 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-6eb16830-459a-4a21-82ec-0c37c9633f72 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] [instance: 75c5d700-6367-4128-9228-80d2cd24ace7] Creating image(s)
Nov 29 03:15:58 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-6eb16830-459a-4a21-82ec-0c37c9633f72 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Acquiring lock "/opt/stack/data/nova/instances/75c5d700-6367-4128-9228-80d2cd24ace7/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:15:58 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-6eb16830-459a-4a21-82ec-0c37c9633f72 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Lock "/opt/stack/data/nova/instances/75c5d700-6367-4128-9228-80d2cd24ace7/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:15:58 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-6eb16830-459a-4a21-82ec-0c37c9633f72 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Lock "/opt/stack/data/nova/instances/75c5d700-6367-4128-9228-80d2cd24ace7/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:15:58 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-6eb16830-459a-4a21-82ec-0c37c9633f72 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:15:58 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:15:58 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=75234) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10541}}
Nov 29 03:15:58 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager.update_available_resource {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:15:58 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:15:58 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:15:58 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:15:58 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Auditing locally available compute resources for user (node: user) {{(pid=75234) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:907}}
Nov 29 03:15:58 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-6eb16830-459a-4a21-82ec-0c37c9633f72 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json" returned: 0 in 0.194s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:15:58 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-6eb16830-459a-4a21-82ec-0c37c9633f72 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Acquiring lock "80686e29dc9db6600d03e85cb62af298e12d7918" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:15:58 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-6eb16830-459a-4a21-82ec-0c37c9633f72 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Lock "80686e29dc9db6600d03e85cb62af298e12d7918" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:15:58 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-6eb16830-459a-4a21-82ec-0c37c9633f72 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:15:58 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-6eb16830-459a-4a21-82ec-0c37c9633f72 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json" returned: 0 in 0.222s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:15:58 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-6eb16830-459a-4a21-82ec-0c37c9633f72 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918,backing_fmt=raw /opt/stack/data/nova/instances/75c5d700-6367-4128-9228-80d2cd24ace7/disk 1073741824 {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:15:58 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-6eb16830-459a-4a21-82ec-0c37c9633f72 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918,backing_fmt=raw /opt/stack/data/nova/instances/75c5d700-6367-4128-9228-80d2cd24ace7/disk 1073741824" returned: 0 in 0.065s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:15:58 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-6eb16830-459a-4a21-82ec-0c37c9633f72 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Lock "80686e29dc9db6600d03e85cb62af298e12d7918" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.304s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:15:58 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-6eb16830-459a-4a21-82ec-0c37c9633f72 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:15:58 user nova-compute[75234]: DEBUG nova.network.neutron [None req-6eb16830-459a-4a21-82ec-0c37c9633f72 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] [instance: 75c5d700-6367-4128-9228-80d2cd24ace7] Successfully created port: 29c6743e-29e2-40a8-8de9-ec380f1e6f25 {{(pid=75234) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:548}}
Nov 29 03:15:58 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/ec7cf7cd-8e48-4bb3-a969-e14db705dc09/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:15:59 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-6eb16830-459a-4a21-82ec-0c37c9633f72 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json" returned: 0 in 0.212s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:15:59 user nova-compute[75234]: DEBUG nova.virt.disk.api [None req-6eb16830-459a-4a21-82ec-0c37c9633f72 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Checking if we can resize image /opt/stack/data/nova/instances/75c5d700-6367-4128-9228-80d2cd24ace7/disk. size=1073741824 {{(pid=75234) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:166}}
Nov 29 03:15:59 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-6eb16830-459a-4a21-82ec-0c37c9633f72 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/75c5d700-6367-4128-9228-80d2cd24ace7/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:15:59 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/ec7cf7cd-8e48-4bb3-a969-e14db705dc09/disk --force-share --output=json" returned: 0 in 0.201s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:15:59 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/ec7cf7cd-8e48-4bb3-a969-e14db705dc09/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:15:59 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-6eb16830-459a-4a21-82ec-0c37c9633f72 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/75c5d700-6367-4128-9228-80d2cd24ace7/disk --force-share --output=json" returned: 0 in 0.231s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:15:59 user nova-compute[75234]: DEBUG nova.virt.disk.api [None req-6eb16830-459a-4a21-82ec-0c37c9633f72 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Cannot resize image /opt/stack/data/nova/instances/75c5d700-6367-4128-9228-80d2cd24ace7/disk to a smaller size. {{(pid=75234) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:172}}
Nov 29 03:15:59 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-6eb16830-459a-4a21-82ec-0c37c9633f72 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] [instance: 75c5d700-6367-4128-9228-80d2cd24ace7] Created local disks {{(pid=75234) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4861}}
Nov 29 03:15:59 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-6eb16830-459a-4a21-82ec-0c37c9633f72 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] [instance: 75c5d700-6367-4128-9228-80d2cd24ace7] Ensure instance console log exists: /opt/stack/data/nova/instances/75c5d700-6367-4128-9228-80d2cd24ace7/console.log {{(pid=75234) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4613}}
Nov 29 03:15:59 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-6eb16830-459a-4a21-82ec-0c37c9633f72 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:15:59 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-6eb16830-459a-4a21-82ec-0c37c9633f72 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:15:59 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-6eb16830-459a-4a21-82ec-0c37c9633f72 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:15:59 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/ec7cf7cd-8e48-4bb3-a969-e14db705dc09/disk --force-share --output=json" returned: 0 in 0.231s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:15:59 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a082c96e-e384-40b6-bcc8-fc5e0e104669/disk.rescue --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:15:59 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a082c96e-e384-40b6-bcc8-fc5e0e104669/disk.rescue --force-share --output=json" returned: 0 in 0.203s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:15:59 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a082c96e-e384-40b6-bcc8-fc5e0e104669/disk.rescue --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:15:59 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a082c96e-e384-40b6-bcc8-fc5e0e104669/disk.rescue --force-share --output=json" returned: 0 in 0.214s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:15:59 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a082c96e-e384-40b6-bcc8-fc5e0e104669/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:15:59 user nova-compute[75234]: DEBUG nova.network.neutron [None req-6eb16830-459a-4a21-82ec-0c37c9633f72 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] [instance: 75c5d700-6367-4128-9228-80d2cd24ace7] Successfully updated port: 29c6743e-29e2-40a8-8de9-ec380f1e6f25 {{(pid=75234) _update_port /opt/stack/nova/nova/network/neutron.py:586}}
Nov 29 03:15:59 user nova-compute[75234]: DEBUG nova.compute.manager [req-e8265f5d-d5a4-4a25-9d40-40379a110506 req-a75b808d-5578-46d2-8170-992135b2b076 service nova] [instance: 75c5d700-6367-4128-9228-80d2cd24ace7] Received event network-changed-29c6743e-29e2-40a8-8de9-ec380f1e6f25 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:15:59 user nova-compute[75234]: DEBUG nova.compute.manager [req-e8265f5d-d5a4-4a25-9d40-40379a110506 req-a75b808d-5578-46d2-8170-992135b2b076 service nova] [instance: 75c5d700-6367-4128-9228-80d2cd24ace7] Refreshing instance network info cache due to event network-changed-29c6743e-29e2-40a8-8de9-ec380f1e6f25. {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11117}}
Nov 29 03:15:59 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-e8265f5d-d5a4-4a25-9d40-40379a110506 req-a75b808d-5578-46d2-8170-992135b2b076 service nova] Acquiring lock "refresh_cache-75c5d700-6367-4128-9228-80d2cd24ace7" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:15:59 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-e8265f5d-d5a4-4a25-9d40-40379a110506 req-a75b808d-5578-46d2-8170-992135b2b076 service nova] Acquired lock "refresh_cache-75c5d700-6367-4128-9228-80d2cd24ace7" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:15:59 user nova-compute[75234]: DEBUG nova.network.neutron [req-e8265f5d-d5a4-4a25-9d40-40379a110506 req-a75b808d-5578-46d2-8170-992135b2b076 service nova] [instance: 75c5d700-6367-4128-9228-80d2cd24ace7] Refreshing network info cache for port 29c6743e-29e2-40a8-8de9-ec380f1e6f25 {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2006}}
Nov 29 03:15:59 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-6eb16830-459a-4a21-82ec-0c37c9633f72 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Acquiring lock "refresh_cache-75c5d700-6367-4128-9228-80d2cd24ace7" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:16:00 user nova-compute[75234]: DEBUG nova.network.neutron [req-e8265f5d-d5a4-4a25-9d40-40379a110506 req-a75b808d-5578-46d2-8170-992135b2b076 service nova] [instance: 75c5d700-6367-4128-9228-80d2cd24ace7] Instance cache missing network info. {{(pid=75234) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3322}}
Nov 29 03:16:00 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a082c96e-e384-40b6-bcc8-fc5e0e104669/disk --force-share --output=json" returned: 0 in 0.298s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:16:00 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a082c96e-e384-40b6-bcc8-fc5e0e104669/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:16:00 user nova-compute[75234]: DEBUG nova.network.neutron [req-e8265f5d-d5a4-4a25-9d40-40379a110506 req-a75b808d-5578-46d2-8170-992135b2b076 service nova] [instance: 75c5d700-6367-4128-9228-80d2cd24ace7] Updating instance_info_cache with network_info: [] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:16:00 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-e8265f5d-d5a4-4a25-9d40-40379a110506 req-a75b808d-5578-46d2-8170-992135b2b076 service nova] Releasing lock "refresh_cache-75c5d700-6367-4128-9228-80d2cd24ace7" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:16:00 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-6eb16830-459a-4a21-82ec-0c37c9633f72 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Acquired lock "refresh_cache-75c5d700-6367-4128-9228-80d2cd24ace7" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:16:00 user nova-compute[75234]: DEBUG nova.network.neutron [None req-6eb16830-459a-4a21-82ec-0c37c9633f72 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] [instance: 75c5d700-6367-4128-9228-80d2cd24ace7] Building network info cache for instance {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2009}}
Nov 29 03:16:00 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a082c96e-e384-40b6-bcc8-fc5e0e104669/disk --force-share --output=json" returned: 0 in 0.184s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:16:00 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f609a7b7-1be2-44b4-9177-7e72a4d4c060/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:16:00 user nova-compute[75234]: DEBUG nova.network.neutron [None req-6eb16830-459a-4a21-82ec-0c37c9633f72 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] [instance: 75c5d700-6367-4128-9228-80d2cd24ace7] Instance cache missing network info. {{(pid=75234) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3322}}
Nov 29 03:16:00 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f609a7b7-1be2-44b4-9177-7e72a4d4c060/disk --force-share --output=json" returned: 0 in 0.226s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:16:00 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f609a7b7-1be2-44b4-9177-7e72a4d4c060/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:16:00 user nova-compute[75234]: DEBUG nova.network.neutron [None req-6eb16830-459a-4a21-82ec-0c37c9633f72 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] [instance: 75c5d700-6367-4128-9228-80d2cd24ace7] Updating instance_info_cache with network_info: [{"id": "29c6743e-29e2-40a8-8de9-ec380f1e6f25", "address": "fa:16:3e:4e:5d:e9", "network": {"id": "0f289171-3d73-4054-95a0-2515b4f206a0", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-1787960284-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "315732b6e3ee44d584b8e510ef199a90", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap29c6743e-29", "ovs_interfaceid": "29c6743e-29e2-40a8-8de9-ec380f1e6f25", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:16:00 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f609a7b7-1be2-44b4-9177-7e72a4d4c060/disk --force-share --output=json" returned: 0 in 0.314s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:16:00 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/21e9c6cd-c39f-41c0-8768-5dd64589c3f3/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:16:00 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-6eb16830-459a-4a21-82ec-0c37c9633f72 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Releasing lock "refresh_cache-75c5d700-6367-4128-9228-80d2cd24ace7" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:16:00 user nova-compute[75234]: DEBUG nova.compute.manager [None req-6eb16830-459a-4a21-82ec-0c37c9633f72 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] [instance: 75c5d700-6367-4128-9228-80d2cd24ace7] Instance network_info: |[{"id": "29c6743e-29e2-40a8-8de9-ec380f1e6f25", "address": "fa:16:3e:4e:5d:e9", "network": {"id": "0f289171-3d73-4054-95a0-2515b4f206a0", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-1787960284-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "315732b6e3ee44d584b8e510ef199a90", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap29c6743e-29", "ovs_interfaceid": "29c6743e-29e2-40a8-8de9-ec380f1e6f25", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=75234) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1987}}
Nov 29 03:16:00 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-6eb16830-459a-4a21-82ec-0c37c9633f72 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] [instance: 75c5d700-6367-4128-9228-80d2cd24ace7] Start _get_guest_xml network_info=[{"id": "29c6743e-29e2-40a8-8de9-ec380f1e6f25", "address": "fa:16:3e:4e:5d:e9", "network": {"id": "0f289171-3d73-4054-95a0-2515b4f206a0", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-1787960284-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "315732b6e3ee44d584b8e510ef199a90", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap29c6743e-29", "ovs_interfaceid": "29c6743e-29e2-40a8-8de9-ec380f1e6f25", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum='c8fc807773e5354afe61636071771906',container_format='bare',created_at=2023-11-29T03:02:15Z,direct_url=<?>,disk_format='qcow2',id=96c30ba7-9b73-4a30-8ba4-7138879573b7,min_disk=0,min_ram=0,name='cirros-0.6.2-x86_64-disk',owner='b18e380921c748b78517b6f1211cd0f1',properties=ImageMetaProps,protected=<?>,size=21430272,status='active',tags=<?>,updated_at=2023-11-29T03:02:17Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [{'device_name': '/dev/vda', 'encryption_format': None, 'encrypted': False, 'device_type': 'disk', 'size': 0, 'disk_bus': 'virtio', 'boot_index': 0, 'encryption_options': None, 'encryption_secret_uuid': None, 'guest_format': None, 'image_id': '96c30ba7-9b73-4a30-8ba4-7138879573b7'}], 'ephemerals': [], 'block_device_mapping': [], 'swap': None} {{(pid=75234) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7562}}
Nov 29 03:16:00 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-6eb16830-459a-4a21-82ec-0c37c9633f72 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:16:00 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-6eb16830-459a-4a21-82ec-0c37c9633f72 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:16:00 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-6eb16830-459a-4a21-82ec-0c37c9633f72 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Searching host: 'user' for CPU controller through CGroups V1... {{(pid=75234) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1652}}
Nov 29 03:16:00 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-6eb16830-459a-4a21-82ec-0c37c9633f72 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] CPU controller missing on host. {{(pid=75234) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1662}}
Nov 29 03:16:00 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-6eb16830-459a-4a21-82ec-0c37c9633f72 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Searching host: 'user' for CPU controller through CGroups V2... {{(pid=75234) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1671}}
Nov 29 03:16:00 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-6eb16830-459a-4a21-82ec-0c37c9633f72 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] CPU controller found on host. {{(pid=75234) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1678}}
Nov 29 03:16:00 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-6eb16830-459a-4a21-82ec-0c37c9633f72 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=75234) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5400}}
Nov 29 03:16:00 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-6eb16830-459a-4a21-82ec-0c37c9633f72 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-11-29T03:02:04Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='1',id=6,is_public=True,memory_mb=512,name='m1.tiny',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='c8fc807773e5354afe61636071771906',container_format='bare',created_at=2023-11-29T03:02:15Z,direct_url=<?>,disk_format='qcow2',id=96c30ba7-9b73-4a30-8ba4-7138879573b7,min_disk=0,min_ram=0,name='cirros-0.6.2-x86_64-disk',owner='b18e380921c748b78517b6f1211cd0f1',properties=ImageMetaProps,protected=<?>,size=21430272,status='active',tags=<?>,updated_at=2023-11-29T03:02:17Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:563}}
Nov 29 03:16:00 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-6eb16830-459a-4a21-82ec-0c37c9633f72 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Flavor limits 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:348}}
Nov 29 03:16:00 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-6eb16830-459a-4a21-82ec-0c37c9633f72 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Image limits 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:352}}
Nov 29 03:16:00 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-6eb16830-459a-4a21-82ec-0c37c9633f72 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Flavor pref 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:388}}
Nov 29 03:16:00 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-6eb16830-459a-4a21-82ec-0c37c9633f72 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Image pref 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:392}}
Nov 29 03:16:00 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-6eb16830-459a-4a21-82ec-0c37c9633f72 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:430}}
Nov 29 03:16:00 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-6eb16830-459a-4a21-82ec-0c37c9633f72 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:569}}
Nov 29 03:16:00 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-6eb16830-459a-4a21-82ec-0c37c9633f72 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=75234) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:471}}
Nov 29 03:16:00 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-6eb16830-459a-4a21-82ec-0c37c9633f72 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Got 1 possible topologies {{(pid=75234) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:501}}
Nov 29 03:16:00 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-6eb16830-459a-4a21-82ec-0c37c9633f72 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:575}}
Nov 29 03:16:00 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-6eb16830-459a-4a21-82ec-0c37c9633f72 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:577}}
Nov 29 03:16:01 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-6eb16830-459a-4a21-82ec-0c37c9633f72 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-11-29T03:15:56Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-AttachVolumeNegativeTest-server-194972252',display_name='tempest-AttachVolumeNegativeTest-server-194972252',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-attachvolumenegativetest-server-194972252',id=25,image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBI0t7Q4arbYFn+Iy1wTTAnBoIrlZtFSDcoZm3jtuUD26yMW1xetBXI+9OBf1v6Bmt6DJ2jYAew6ivj/euHwu92De75GASLM8tFPoIdZTEGHNPIEquWrxiVONMRE4Uya8eg==',key_name='tempest-keypair-1224772314',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='315732b6e3ee44d584b8e510ef199a90',ramdisk_id='',reservation_id='r-0hbtjx1d',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-AttachVolumeNegativeTest-1582496806',owner_user_name='tempest-AttachVolumeNegativeTest-1582496806-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-11-29T03:15:58Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='66686114450f4f31b3a917667a152d8b',uuid=75c5d700-6367-4128-9228-80d2cd24ace7,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "29c6743e-29e2-40a8-8de9-ec380f1e6f25", "address": "fa:16:3e:4e:5d:e9", "network": {"id": "0f289171-3d73-4054-95a0-2515b4f206a0", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-1787960284-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "315732b6e3ee44d584b8e510ef199a90", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap29c6743e-29", "ovs_interfaceid": "29c6743e-29e2-40a8-8de9-ec380f1e6f25", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=75234) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Nov 29 03:16:01 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-6eb16830-459a-4a21-82ec-0c37c9633f72 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Converting VIF {"id": "29c6743e-29e2-40a8-8de9-ec380f1e6f25", "address": "fa:16:3e:4e:5d:e9", "network": {"id": "0f289171-3d73-4054-95a0-2515b4f206a0", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-1787960284-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "315732b6e3ee44d584b8e510ef199a90", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap29c6743e-29", "ovs_interfaceid": "29c6743e-29e2-40a8-8de9-ec380f1e6f25", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:16:01 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-6eb16830-459a-4a21-82ec-0c37c9633f72 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:4e:5d:e9,bridge_name='br-int',has_traffic_filtering=True,id=29c6743e-29e2-40a8-8de9-ec380f1e6f25,network=Network(0f289171-3d73-4054-95a0-2515b4f206a0),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap29c6743e-29') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:16:01 user nova-compute[75234]: DEBUG nova.objects.instance [None req-6eb16830-459a-4a21-82ec-0c37c9633f72 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Lazy-loading 'pci_devices' on Instance uuid 75c5d700-6367-4128-9228-80d2cd24ace7 {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:16:01 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-6eb16830-459a-4a21-82ec-0c37c9633f72 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] [instance: 75c5d700-6367-4128-9228-80d2cd24ace7] End _get_guest_xml xml=<domain type="kvm">
Nov 29 03:16:01 user nova-compute[75234]:   <uuid>75c5d700-6367-4128-9228-80d2cd24ace7</uuid>
Nov 29 03:16:01 user nova-compute[75234]:   <name>instance-00000019</name>
Nov 29 03:16:01 user nova-compute[75234]:   <memory>524288</memory>
Nov 29 03:16:01 user nova-compute[75234]:   <vcpu>1</vcpu>
Nov 29 03:16:01 user nova-compute[75234]:   <metadata>
Nov 29 03:16:01 user nova-compute[75234]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Nov 29 03:16:01 user nova-compute[75234]:       <nova:package version="0.0.0"/>
Nov 29 03:16:01 user nova-compute[75234]:       <nova:name>tempest-AttachVolumeNegativeTest-server-194972252</nova:name>
Nov 29 03:16:01 user nova-compute[75234]:       <nova:creationTime>2023-11-29 03:16:00</nova:creationTime>
Nov 29 03:16:01 user nova-compute[75234]:       <nova:flavor name="m1.tiny">
Nov 29 03:16:01 user nova-compute[75234]:         <nova:memory>512</nova:memory>
Nov 29 03:16:01 user nova-compute[75234]:         <nova:disk>1</nova:disk>
Nov 29 03:16:01 user nova-compute[75234]:         <nova:swap>0</nova:swap>
Nov 29 03:16:01 user nova-compute[75234]:         <nova:ephemeral>0</nova:ephemeral>
Nov 29 03:16:01 user nova-compute[75234]:         <nova:vcpus>1</nova:vcpus>
Nov 29 03:16:01 user nova-compute[75234]:       </nova:flavor>
Nov 29 03:16:01 user nova-compute[75234]:       <nova:owner>
Nov 29 03:16:01 user nova-compute[75234]:         <nova:user uuid="66686114450f4f31b3a917667a152d8b">tempest-AttachVolumeNegativeTest-1582496806-project-member</nova:user>
Nov 29 03:16:01 user nova-compute[75234]:         <nova:project uuid="315732b6e3ee44d584b8e510ef199a90">tempest-AttachVolumeNegativeTest-1582496806</nova:project>
Nov 29 03:16:01 user nova-compute[75234]:       </nova:owner>
Nov 29 03:16:01 user nova-compute[75234]:       <nova:root type="image" uuid="96c30ba7-9b73-4a30-8ba4-7138879573b7"/>
Nov 29 03:16:01 user nova-compute[75234]:       <nova:ports>
Nov 29 03:16:01 user nova-compute[75234]:         <nova:port uuid="29c6743e-29e2-40a8-8de9-ec380f1e6f25">
Nov 29 03:16:01 user nova-compute[75234]:           <nova:ip type="fixed" address="10.0.0.8" ipVersion="4"/>
Nov 29 03:16:01 user nova-compute[75234]:         </nova:port>
Nov 29 03:16:01 user nova-compute[75234]:       </nova:ports>
Nov 29 03:16:01 user nova-compute[75234]:     </nova:instance>
Nov 29 03:16:01 user nova-compute[75234]:   </metadata>
Nov 29 03:16:01 user nova-compute[75234]:   <sysinfo type="smbios">
Nov 29 03:16:01 user nova-compute[75234]:     <system>
Nov 29 03:16:01 user nova-compute[75234]:       <entry name="manufacturer">OpenStack Foundation</entry>
Nov 29 03:16:01 user nova-compute[75234]:       <entry name="product">OpenStack Nova</entry>
Nov 29 03:16:01 user nova-compute[75234]:       <entry name="version">0.0.0</entry>
Nov 29 03:16:01 user nova-compute[75234]:       <entry name="serial">75c5d700-6367-4128-9228-80d2cd24ace7</entry>
Nov 29 03:16:01 user nova-compute[75234]:       <entry name="uuid">75c5d700-6367-4128-9228-80d2cd24ace7</entry>
Nov 29 03:16:01 user nova-compute[75234]:       <entry name="family">Virtual Machine</entry>
Nov 29 03:16:01 user nova-compute[75234]:     </system>
Nov 29 03:16:01 user nova-compute[75234]:   </sysinfo>
Nov 29 03:16:01 user nova-compute[75234]:   <os>
Nov 29 03:16:01 user nova-compute[75234]:     <type arch="x86_64" machine="pc">hvm</type>
Nov 29 03:16:01 user nova-compute[75234]:     <boot dev="hd"/>
Nov 29 03:16:01 user nova-compute[75234]:     <smbios mode="sysinfo"/>
Nov 29 03:16:01 user nova-compute[75234]:   </os>
Nov 29 03:16:01 user nova-compute[75234]:   <features>
Nov 29 03:16:01 user nova-compute[75234]:     <acpi/>
Nov 29 03:16:01 user nova-compute[75234]:     <apic/>
Nov 29 03:16:01 user nova-compute[75234]:     <vmcoreinfo/>
Nov 29 03:16:01 user nova-compute[75234]:   </features>
Nov 29 03:16:01 user nova-compute[75234]:   <clock offset="utc">
Nov 29 03:16:01 user nova-compute[75234]:     <timer name="pit" tickpolicy="delay"/>
Nov 29 03:16:01 user nova-compute[75234]:     <timer name="rtc" tickpolicy="catchup"/>
Nov 29 03:16:01 user nova-compute[75234]:     <timer name="hpet" present="no"/>
Nov 29 03:16:01 user nova-compute[75234]:   </clock>
Nov 29 03:16:01 user nova-compute[75234]:   <cpu mode="custom" match="exact">
Nov 29 03:16:01 user nova-compute[75234]:     <model>Nehalem</model>
Nov 29 03:16:01 user nova-compute[75234]:     <topology sockets="1" cores="1" threads="1"/>
Nov 29 03:16:01 user nova-compute[75234]:   </cpu>
Nov 29 03:16:01 user nova-compute[75234]:   <devices>
Nov 29 03:16:01 user nova-compute[75234]:     <disk type="file" device="disk">
Nov 29 03:16:01 user nova-compute[75234]:       <driver name="qemu" type="qcow2" cache="none"/>
Nov 29 03:16:01 user nova-compute[75234]:       <source file="/opt/stack/data/nova/instances/75c5d700-6367-4128-9228-80d2cd24ace7/disk"/>
Nov 29 03:16:01 user nova-compute[75234]:       <target dev="vda" bus="virtio"/>
Nov 29 03:16:01 user nova-compute[75234]:     </disk>
Nov 29 03:16:01 user nova-compute[75234]:     <interface type="ethernet">
Nov 29 03:16:01 user nova-compute[75234]:       <mac address="fa:16:3e:4e:5d:e9"/>
Nov 29 03:16:01 user nova-compute[75234]:       <model type="virtio"/>
Nov 29 03:16:01 user nova-compute[75234]:       <mtu size="1442"/>
Nov 29 03:16:01 user nova-compute[75234]:       <target dev="tap29c6743e-29"/>
Nov 29 03:16:01 user nova-compute[75234]:     </interface>
Nov 29 03:16:01 user nova-compute[75234]:     <serial type="pty">
Nov 29 03:16:01 user nova-compute[75234]:       <log file="/opt/stack/data/nova/instances/75c5d700-6367-4128-9228-80d2cd24ace7/console.log" append="off"/>
Nov 29 03:16:01 user nova-compute[75234]:     </serial>
Nov 29 03:16:01 user nova-compute[75234]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Nov 29 03:16:01 user nova-compute[75234]:     <video>
Nov 29 03:16:01 user nova-compute[75234]:       <model type="virtio"/>
Nov 29 03:16:01 user nova-compute[75234]:     </video>
Nov 29 03:16:01 user nova-compute[75234]:     <rng model="virtio">
Nov 29 03:16:01 user nova-compute[75234]:       <backend model="random">/dev/urandom</backend>
Nov 29 03:16:01 user nova-compute[75234]:     </rng>
Nov 29 03:16:01 user nova-compute[75234]:     <controller type="usb" index="0" model="none"/>
Nov 29 03:16:01 user nova-compute[75234]:     <memballoon model="virtio">
Nov 29 03:16:01 user nova-compute[75234]:       <stats period="10"/>
Nov 29 03:16:01 user nova-compute[75234]:     </memballoon>
Nov 29 03:16:01 user nova-compute[75234]:   </devices>
Nov 29 03:16:01 user nova-compute[75234]: </domain>
Nov 29 03:16:01 user nova-compute[75234]:  {{(pid=75234) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7568}}
Nov 29 03:16:01 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-6eb16830-459a-4a21-82ec-0c37c9633f72 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-11-29T03:15:56Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-AttachVolumeNegativeTest-server-194972252',display_name='tempest-AttachVolumeNegativeTest-server-194972252',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-attachvolumenegativetest-server-194972252',id=25,image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBI0t7Q4arbYFn+Iy1wTTAnBoIrlZtFSDcoZm3jtuUD26yMW1xetBXI+9OBf1v6Bmt6DJ2jYAew6ivj/euHwu92De75GASLM8tFPoIdZTEGHNPIEquWrxiVONMRE4Uya8eg==',key_name='tempest-keypair-1224772314',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='315732b6e3ee44d584b8e510ef199a90',ramdisk_id='',reservation_id='r-0hbtjx1d',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-AttachVolumeNegativeTest-1582496806',owner_user_name='tempest-AttachVolumeNegativeTest-1582496806-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-11-29T03:15:58Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='66686114450f4f31b3a917667a152d8b',uuid=75c5d700-6367-4128-9228-80d2cd24ace7,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "29c6743e-29e2-40a8-8de9-ec380f1e6f25", "address": "fa:16:3e:4e:5d:e9", "network": {"id": "0f289171-3d73-4054-95a0-2515b4f206a0", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-1787960284-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "315732b6e3ee44d584b8e510ef199a90", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap29c6743e-29", "ovs_interfaceid": "29c6743e-29e2-40a8-8de9-ec380f1e6f25", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Nov 29 03:16:01 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-6eb16830-459a-4a21-82ec-0c37c9633f72 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Converting VIF {"id": "29c6743e-29e2-40a8-8de9-ec380f1e6f25", "address": "fa:16:3e:4e:5d:e9", "network": {"id": "0f289171-3d73-4054-95a0-2515b4f206a0", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-1787960284-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "315732b6e3ee44d584b8e510ef199a90", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap29c6743e-29", "ovs_interfaceid": "29c6743e-29e2-40a8-8de9-ec380f1e6f25", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:16:01 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-6eb16830-459a-4a21-82ec-0c37c9633f72 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:4e:5d:e9,bridge_name='br-int',has_traffic_filtering=True,id=29c6743e-29e2-40a8-8de9-ec380f1e6f25,network=Network(0f289171-3d73-4054-95a0-2515b4f206a0),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap29c6743e-29') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:16:01 user nova-compute[75234]: DEBUG os_vif [None req-6eb16830-459a-4a21-82ec-0c37c9633f72 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:4e:5d:e9,bridge_name='br-int',has_traffic_filtering=True,id=29c6743e-29e2-40a8-8de9-ec380f1e6f25,network=Network(0f289171-3d73-4054-95a0-2515b4f206a0),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap29c6743e-29') {{(pid=75234) plug /opt/stack/data/venv/lib/python3.10/site-packages/os_vif/__init__.py:76}}
Nov 29 03:16:01 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:16:01 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:16:01 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Nov 29 03:16:01 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:16:01 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbCreateCommand(_result=None, table=QoS, columns={'type': 'linux-noop', 'external_ids': {'id': 'e4a8843b-1eba-5f8d-a4b4-b6081e11a218', '_type': 'linux-noop'}}, row=False) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:16:01 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:16:01 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:16:01 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:16:01 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tap29c6743e-29, may_exist=True, interface_attrs={}) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:16:01 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Port, record=tap29c6743e-29, col_values=(('qos', UUID('6d3995b0-1eb9-4fab-be86-abf2b5eda897')),), if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:16:01 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DbSetCommand(_result=None, table=Interface, record=tap29c6743e-29, col_values=(('external_ids', {'iface-id': '29c6743e-29e2-40a8-8de9-ec380f1e6f25', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:4e:5d:e9', 'vm-uuid': '75c5d700-6367-4128-9228-80d2cd24ace7'}),), if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:16:01 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:16:01 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:16:01 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:16:01 user nova-compute[75234]: INFO os_vif [None req-6eb16830-459a-4a21-82ec-0c37c9633f72 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:4e:5d:e9,bridge_name='br-int',has_traffic_filtering=True,id=29c6743e-29e2-40a8-8de9-ec380f1e6f25,network=Network(0f289171-3d73-4054-95a0-2515b4f206a0),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap29c6743e-29')
Nov 29 03:16:01 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/21e9c6cd-c39f-41c0-8768-5dd64589c3f3/disk --force-share --output=json" returned: 0 in 0.232s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:16:01 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/21e9c6cd-c39f-41c0-8768-5dd64589c3f3/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:16:01 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-6eb16830-459a-4a21-82ec-0c37c9633f72 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] No BDM found with device name vda, not building metadata. {{(pid=75234) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12089}}
Nov 29 03:16:01 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-6eb16830-459a-4a21-82ec-0c37c9633f72 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] No VIF found with MAC fa:16:3e:4e:5d:e9, not building metadata {{(pid=75234) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Nov 29 03:16:01 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/21e9c6cd-c39f-41c0-8768-5dd64589c3f3/disk --force-share --output=json" returned: 0 in 0.192s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:16:01 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/13e5da64-9adc-4479-81c7-d329e9b5f08f/disk.rescue --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:16:01 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-c4ff21a8-705d-4040-8f98-de9bf08a07fe tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] [instance: 6894b55f-93c6-43c9-8933-56d5a3c43725] Snapshot image upload complete
Nov 29 03:16:01 user nova-compute[75234]: INFO nova.compute.manager [None req-c4ff21a8-705d-4040-8f98-de9bf08a07fe tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] [instance: 6894b55f-93c6-43c9-8933-56d5a3c43725] Took 7.27 seconds to snapshot the instance on the hypervisor.
Nov 29 03:16:01 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/13e5da64-9adc-4479-81c7-d329e9b5f08f/disk.rescue --force-share --output=json" returned: 0 in 0.232s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:16:01 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/13e5da64-9adc-4479-81c7-d329e9b5f08f/disk.rescue --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:16:01 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/13e5da64-9adc-4479-81c7-d329e9b5f08f/disk.rescue --force-share --output=json" returned: 0 in 0.186s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:16:01 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/13e5da64-9adc-4479-81c7-d329e9b5f08f/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:16:01 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:16:01 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/13e5da64-9adc-4479-81c7-d329e9b5f08f/disk --force-share --output=json" returned: 0 in 0.249s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:16:01 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/13e5da64-9adc-4479-81c7-d329e9b5f08f/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:16:02 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/13e5da64-9adc-4479-81c7-d329e9b5f08f/disk --force-share --output=json" returned: 0 in 0.187s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:16:02 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/95b6eb15-7930-4aa5-aff8-73809b0a9277/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:16:02 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/95b6eb15-7930-4aa5-aff8-73809b0a9277/disk --force-share --output=json" returned: 0 in 0.224s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:16:02 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/95b6eb15-7930-4aa5-aff8-73809b0a9277/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:16:02 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/95b6eb15-7930-4aa5-aff8-73809b0a9277/disk --force-share --output=json" returned: 0 in 0.196s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:16:02 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:16:02 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6894b55f-93c6-43c9-8933-56d5a3c43725/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:16:02 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:16:02 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6894b55f-93c6-43c9-8933-56d5a3c43725/disk --force-share --output=json" returned: 0 in 0.202s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:16:02 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6894b55f-93c6-43c9-8933-56d5a3c43725/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:16:02 user nova-compute[75234]: DEBUG nova.compute.manager [req-af3e8774-cade-4dab-93fa-9f7cabfdcdf6 req-28868898-506a-4fb7-88b4-6f637669c370 service nova] [instance: 75c5d700-6367-4128-9228-80d2cd24ace7] Received event network-vif-plugged-29c6743e-29e2-40a8-8de9-ec380f1e6f25 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:16:02 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-af3e8774-cade-4dab-93fa-9f7cabfdcdf6 req-28868898-506a-4fb7-88b4-6f637669c370 service nova] Acquiring lock "75c5d700-6367-4128-9228-80d2cd24ace7-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:16:02 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-af3e8774-cade-4dab-93fa-9f7cabfdcdf6 req-28868898-506a-4fb7-88b4-6f637669c370 service nova] Lock "75c5d700-6367-4128-9228-80d2cd24ace7-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:16:02 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-af3e8774-cade-4dab-93fa-9f7cabfdcdf6 req-28868898-506a-4fb7-88b4-6f637669c370 service nova] Lock "75c5d700-6367-4128-9228-80d2cd24ace7-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:16:02 user nova-compute[75234]: DEBUG nova.compute.manager [req-af3e8774-cade-4dab-93fa-9f7cabfdcdf6 req-28868898-506a-4fb7-88b4-6f637669c370 service nova] [instance: 75c5d700-6367-4128-9228-80d2cd24ace7] No waiting events found dispatching network-vif-plugged-29c6743e-29e2-40a8-8de9-ec380f1e6f25 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:16:02 user nova-compute[75234]: WARNING nova.compute.manager [req-af3e8774-cade-4dab-93fa-9f7cabfdcdf6 req-28868898-506a-4fb7-88b4-6f637669c370 service nova] [instance: 75c5d700-6367-4128-9228-80d2cd24ace7] Received unexpected event network-vif-plugged-29c6743e-29e2-40a8-8de9-ec380f1e6f25 for instance with vm_state building and task_state spawning.
Nov 29 03:16:03 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6894b55f-93c6-43c9-8933-56d5a3c43725/disk --force-share --output=json" returned: 0 in 0.216s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:16:03 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/d3b90f39-4b91-4152-b820-bd0e263eec89/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:16:03 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/d3b90f39-4b91-4152-b820-bd0e263eec89/disk --force-share --output=json" returned: 0 in 0.222s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:16:03 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/d3b90f39-4b91-4152-b820-bd0e263eec89/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:16:03 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/d3b90f39-4b91-4152-b820-bd0e263eec89/disk --force-share --output=json" returned: 0 in 0.236s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:16:05 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:16:05 user nova-compute[75234]: DEBUG nova.compute.manager [None req-6eb16830-459a-4a21-82ec-0c37c9633f72 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] [instance: 75c5d700-6367-4128-9228-80d2cd24ace7] Instance event wait completed in 0 seconds for  {{(pid=75234) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Nov 29 03:16:05 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-6eb16830-459a-4a21-82ec-0c37c9633f72 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] [instance: 75c5d700-6367-4128-9228-80d2cd24ace7] Guest created on hypervisor {{(pid=75234) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4421}}
Nov 29 03:16:05 user nova-compute[75234]: DEBUG nova.compute.manager [req-58ff654a-e511-436a-9b87-14cf6c72de7f req-18a2153b-e98d-48cc-b544-42bc4375c21b service nova] [instance: 75c5d700-6367-4128-9228-80d2cd24ace7] Received event network-vif-plugged-29c6743e-29e2-40a8-8de9-ec380f1e6f25 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:16:05 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-58ff654a-e511-436a-9b87-14cf6c72de7f req-18a2153b-e98d-48cc-b544-42bc4375c21b service nova] Acquiring lock "75c5d700-6367-4128-9228-80d2cd24ace7-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:16:05 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-58ff654a-e511-436a-9b87-14cf6c72de7f req-18a2153b-e98d-48cc-b544-42bc4375c21b service nova] Lock "75c5d700-6367-4128-9228-80d2cd24ace7-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:16:05 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-58ff654a-e511-436a-9b87-14cf6c72de7f req-18a2153b-e98d-48cc-b544-42bc4375c21b service nova] Lock "75c5d700-6367-4128-9228-80d2cd24ace7-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:16:05 user nova-compute[75234]: DEBUG nova.compute.manager [req-58ff654a-e511-436a-9b87-14cf6c72de7f req-18a2153b-e98d-48cc-b544-42bc4375c21b service nova] [instance: 75c5d700-6367-4128-9228-80d2cd24ace7] No waiting events found dispatching network-vif-plugged-29c6743e-29e2-40a8-8de9-ec380f1e6f25 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:16:05 user nova-compute[75234]: WARNING nova.compute.manager [req-58ff654a-e511-436a-9b87-14cf6c72de7f req-18a2153b-e98d-48cc-b544-42bc4375c21b service nova] [instance: 75c5d700-6367-4128-9228-80d2cd24ace7] Received unexpected event network-vif-plugged-29c6743e-29e2-40a8-8de9-ec380f1e6f25 for instance with vm_state building and task_state spawning.
Nov 29 03:16:05 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-a20fe69a-cdbf-482a-8a81-186f342fd171 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Acquiring lock "1c8b7559-29ab-4982-8c75-ea2a0f0101cb" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:16:05 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-a20fe69a-cdbf-482a-8a81-186f342fd171 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Lock "1c8b7559-29ab-4982-8c75-ea2a0f0101cb" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:16:05 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:16:05 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:16:05 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Hypervisor/Node resource view: name=user free_ram=7639MB free_disk=25.39777374267578GB free_vcpus=4 pci_devices=[{"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}] {{(pid=75234) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1106}}
Nov 29 03:16:05 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:16:05 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:16:05 user nova-compute[75234]: INFO nova.virt.libvirt.driver [-] [instance: 75c5d700-6367-4128-9228-80d2cd24ace7] Instance spawned successfully.
Nov 29 03:16:05 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-cedcbf9c-14a1-41bf-a77f-6287a7a77e0c tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Acquiring lock "f609a7b7-1be2-44b4-9177-7e72a4d4c060" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:16:05 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-cedcbf9c-14a1-41bf-a77f-6287a7a77e0c tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Lock "f609a7b7-1be2-44b4-9177-7e72a4d4c060" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:16:05 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-cedcbf9c-14a1-41bf-a77f-6287a7a77e0c tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Acquiring lock "f609a7b7-1be2-44b4-9177-7e72a4d4c060-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:16:05 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-cedcbf9c-14a1-41bf-a77f-6287a7a77e0c tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Lock "f609a7b7-1be2-44b4-9177-7e72a4d4c060-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:16:05 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-cedcbf9c-14a1-41bf-a77f-6287a7a77e0c tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Lock "f609a7b7-1be2-44b4-9177-7e72a4d4c060-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:16:05 user nova-compute[75234]: DEBUG nova.compute.manager [None req-a20fe69a-cdbf-482a-8a81-186f342fd171 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] [instance: 1c8b7559-29ab-4982-8c75-ea2a0f0101cb] Starting instance... {{(pid=75234) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2422}}
Nov 29 03:16:05 user nova-compute[75234]: INFO nova.compute.manager [None req-cedcbf9c-14a1-41bf-a77f-6287a7a77e0c tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] [instance: f609a7b7-1be2-44b4-9177-7e72a4d4c060] Terminating instance
Nov 29 03:16:05 user nova-compute[75234]: DEBUG nova.compute.manager [None req-cedcbf9c-14a1-41bf-a77f-6287a7a77e0c tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] [instance: f609a7b7-1be2-44b4-9177-7e72a4d4c060] Start destroying the instance on the hypervisor. {{(pid=75234) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3140}}
Nov 29 03:16:05 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-6eb16830-459a-4a21-82ec-0c37c9633f72 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] [instance: 75c5d700-6367-4128-9228-80d2cd24ace7] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:904}}
Nov 29 03:16:05 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-6eb16830-459a-4a21-82ec-0c37c9633f72 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] [instance: 75c5d700-6367-4128-9228-80d2cd24ace7] Found default for hw_cdrom_bus of ide {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:16:05 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-6eb16830-459a-4a21-82ec-0c37c9633f72 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] [instance: 75c5d700-6367-4128-9228-80d2cd24ace7] Found default for hw_disk_bus of virtio {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:16:05 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-6eb16830-459a-4a21-82ec-0c37c9633f72 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] [instance: 75c5d700-6367-4128-9228-80d2cd24ace7] Found default for hw_input_bus of None {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:16:05 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-6eb16830-459a-4a21-82ec-0c37c9633f72 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] [instance: 75c5d700-6367-4128-9228-80d2cd24ace7] Found default for hw_pointer_model of None {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:16:05 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-6eb16830-459a-4a21-82ec-0c37c9633f72 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] [instance: 75c5d700-6367-4128-9228-80d2cd24ace7] Found default for hw_video_model of virtio {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:16:05 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-6eb16830-459a-4a21-82ec-0c37c9633f72 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] [instance: 75c5d700-6367-4128-9228-80d2cd24ace7] Found default for hw_vif_model of virtio {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:16:05 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:16:05 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance 21e9c6cd-c39f-41c0-8768-5dd64589c3f3 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:16:05 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance f609a7b7-1be2-44b4-9177-7e72a4d4c060 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:16:05 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance d3b90f39-4b91-4152-b820-bd0e263eec89 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:16:05 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance 13e5da64-9adc-4479-81c7-d329e9b5f08f actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:16:05 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance a082c96e-e384-40b6-bcc8-fc5e0e104669 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:16:05 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance 6894b55f-93c6-43c9-8933-56d5a3c43725 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:16:05 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance ec7cf7cd-8e48-4bb3-a969-e14db705dc09 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:16:05 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance 95b6eb15-7930-4aa5-aff8-73809b0a9277 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:16:05 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance 75c5d700-6367-4128-9228-80d2cd24ace7 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:16:05 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:16:05 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:16:05 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-a20fe69a-cdbf-482a-8a81-186f342fd171 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:16:05 user nova-compute[75234]: INFO nova.compute.manager [None req-6eb16830-459a-4a21-82ec-0c37c9633f72 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] [instance: 75c5d700-6367-4128-9228-80d2cd24ace7] Took 7.47 seconds to spawn the instance on the hypervisor.
Nov 29 03:16:05 user nova-compute[75234]: DEBUG nova.compute.manager [None req-6eb16830-459a-4a21-82ec-0c37c9633f72 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] [instance: 75c5d700-6367-4128-9228-80d2cd24ace7] Checking state {{(pid=75234) _get_power_state /opt/stack/nova/nova/compute/manager.py:1782}}
Nov 29 03:16:05 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance 1c8b7559-29ab-4982-8c75-ea2a0f0101cb has been scheduled to this compute host, the scheduler has made an allocation against this compute node but the instance has yet to start. Skipping heal of allocation: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1761}}
Nov 29 03:16:05 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Total usable vcpus: 12, total allocated vcpus: 9 {{(pid=75234) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1129}}
Nov 29 03:16:05 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Final resource view: name=user phys_ram=16011MB used_ram=5120MB phys_disk=40GB used_disk=9GB total_vcpus=12 used_vcpus=9 pci_stats=[] {{(pid=75234) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1138}}
Nov 29 03:16:05 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:16:05 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:16:06 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:16:06 user nova-compute[75234]: INFO nova.compute.manager [None req-6eb16830-459a-4a21-82ec-0c37c9633f72 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] [instance: 75c5d700-6367-4128-9228-80d2cd24ace7] Took 9.31 seconds to build instance.
Nov 29 03:16:06 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-6eb16830-459a-4a21-82ec-0c37c9633f72 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Lock "75c5d700-6367-4128-9228-80d2cd24ace7" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 9.481s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:16:06 user nova-compute[75234]: DEBUG nova.compute.provider_tree [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Inventory has not changed in ProviderTree for provider: 4d291220-7c37-4839-a0e3-b23c66502b74 {{(pid=75234) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Nov 29 03:16:06 user nova-compute[75234]: DEBUG nova.scheduler.client.report [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Inventory has not changed for provider 4d291220-7c37-4839-a0e3-b23c66502b74 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=75234) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Nov 29 03:16:06 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Compute_service record updated for user:user {{(pid=75234) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:1067}}
Nov 29 03:16:06 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 1.008s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:16:06 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-a20fe69a-cdbf-482a-8a81-186f342fd171 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.717s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:16:06 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-a20fe69a-cdbf-482a-8a81-186f342fd171 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=75234) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2366}}
Nov 29 03:16:06 user nova-compute[75234]: INFO nova.compute.claims [None req-a20fe69a-cdbf-482a-8a81-186f342fd171 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] [instance: 1c8b7559-29ab-4982-8c75-ea2a0f0101cb] Claim successful on node user
Nov 29 03:16:06 user nova-compute[75234]: DEBUG nova.compute.manager [req-56a985a2-cb87-4189-b480-c93eda71a997 req-9e3e8b39-03d4-4ec3-96fc-2c64c1242fe4 service nova] [instance: f609a7b7-1be2-44b4-9177-7e72a4d4c060] Received event network-vif-unplugged-5543f379-2b3a-4dc2-bfe9-52f66a942257 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:16:06 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-56a985a2-cb87-4189-b480-c93eda71a997 req-9e3e8b39-03d4-4ec3-96fc-2c64c1242fe4 service nova] Acquiring lock "f609a7b7-1be2-44b4-9177-7e72a4d4c060-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:16:06 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-56a985a2-cb87-4189-b480-c93eda71a997 req-9e3e8b39-03d4-4ec3-96fc-2c64c1242fe4 service nova] Lock "f609a7b7-1be2-44b4-9177-7e72a4d4c060-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:16:06 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-56a985a2-cb87-4189-b480-c93eda71a997 req-9e3e8b39-03d4-4ec3-96fc-2c64c1242fe4 service nova] Lock "f609a7b7-1be2-44b4-9177-7e72a4d4c060-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:16:06 user nova-compute[75234]: DEBUG nova.compute.manager [req-56a985a2-cb87-4189-b480-c93eda71a997 req-9e3e8b39-03d4-4ec3-96fc-2c64c1242fe4 service nova] [instance: f609a7b7-1be2-44b4-9177-7e72a4d4c060] No waiting events found dispatching network-vif-unplugged-5543f379-2b3a-4dc2-bfe9-52f66a942257 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:16:06 user nova-compute[75234]: DEBUG nova.compute.manager [req-56a985a2-cb87-4189-b480-c93eda71a997 req-9e3e8b39-03d4-4ec3-96fc-2c64c1242fe4 service nova] [instance: f609a7b7-1be2-44b4-9177-7e72a4d4c060] Received event network-vif-unplugged-5543f379-2b3a-4dc2-bfe9-52f66a942257 for instance with task_state deleting. {{(pid=75234) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10890}}
Nov 29 03:16:06 user nova-compute[75234]: INFO nova.virt.libvirt.driver [-] [instance: f609a7b7-1be2-44b4-9177-7e72a4d4c060] Instance destroyed successfully.
Nov 29 03:16:06 user nova-compute[75234]: DEBUG nova.objects.instance [None req-cedcbf9c-14a1-41bf-a77f-6287a7a77e0c tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Lazy-loading 'resources' on Instance uuid f609a7b7-1be2-44b4-9177-7e72a4d4c060 {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:16:06 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-cedcbf9c-14a1-41bf-a77f-6287a7a77e0c tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-11-29T03:06:06Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-AttachVolumeTestJSON-server-394712256',display_name='tempest-AttachVolumeTestJSON-server-394712256',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-attachvolumetestjson-server-394712256',id=4,image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBGphX+MWMhY0YZ/wlOvEQFKFHTIX4g9KMhxkTe+ErHeVHCk7d+KpOlTDVK9edUWFFxjgJKboch3pp1xrAYGa/rIEWyUV4YOfCGPFoAeneAuweb6ekmwg4qgLxfML0aXASQ==',key_name='tempest-keypair-213174705',keypairs=<?>,launch_index=0,launched_at=2023-11-29T03:06:23Z,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='e393d23e783b4a40986674d782606c69',ramdisk_id='',reservation_id='r-9o6ggnbe',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-AttachVolumeTestJSON-1911423306',owner_user_name='tempest-AttachVolumeTestJSON-1911423306-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-11-29T03:14:18Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='bb1126c998ee430baac2e2494f30c0aa',uuid=f609a7b7-1be2-44b4-9177-7e72a4d4c060,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "5543f379-2b3a-4dc2-bfe9-52f66a942257", "address": "fa:16:3e:04:ae:43", "network": {"id": "294c3aac-8c9b-45f0-8c0a-344e2d7d8b7c", "bridge": "br-int", "label": "tempest-AttachVolumeTestJSON-700098935-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.14", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "e393d23e783b4a40986674d782606c69", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap5543f379-2b", "ovs_interfaceid": "5543f379-2b3a-4dc2-bfe9-52f66a942257", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Nov 29 03:16:06 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-cedcbf9c-14a1-41bf-a77f-6287a7a77e0c tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Converting VIF {"id": "5543f379-2b3a-4dc2-bfe9-52f66a942257", "address": "fa:16:3e:04:ae:43", "network": {"id": "294c3aac-8c9b-45f0-8c0a-344e2d7d8b7c", "bridge": "br-int", "label": "tempest-AttachVolumeTestJSON-700098935-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.14", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "e393d23e783b4a40986674d782606c69", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap5543f379-2b", "ovs_interfaceid": "5543f379-2b3a-4dc2-bfe9-52f66a942257", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:16:06 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-cedcbf9c-14a1-41bf-a77f-6287a7a77e0c tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:04:ae:43,bridge_name='br-int',has_traffic_filtering=True,id=5543f379-2b3a-4dc2-bfe9-52f66a942257,network=Network(294c3aac-8c9b-45f0-8c0a-344e2d7d8b7c),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap5543f379-2b') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:16:06 user nova-compute[75234]: DEBUG os_vif [None req-cedcbf9c-14a1-41bf-a77f-6287a7a77e0c tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Unplugging vif VIFOpenVSwitch(active=True,address=fa:16:3e:04:ae:43,bridge_name='br-int',has_traffic_filtering=True,id=5543f379-2b3a-4dc2-bfe9-52f66a942257,network=Network(294c3aac-8c9b-45f0-8c0a-344e2d7d8b7c),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap5543f379-2b') {{(pid=75234) unplug /opt/stack/data/venv/lib/python3.10/site-packages/os_vif/__init__.py:109}}
Nov 29 03:16:06 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:16:06 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tap5543f379-2b, bridge=br-int, if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:16:06 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:16:06 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:16:06 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:16:06 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbDestroyCommand(_result=None, table=QoS, record=3d9da991-ce66-4f39-b6e3-d577c41617c5) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:16:06 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:16:06 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:16:06 user nova-compute[75234]: INFO os_vif [None req-cedcbf9c-14a1-41bf-a77f-6287a7a77e0c tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Successfully unplugged vif VIFOpenVSwitch(active=True,address=fa:16:3e:04:ae:43,bridge_name='br-int',has_traffic_filtering=True,id=5543f379-2b3a-4dc2-bfe9-52f66a942257,network=Network(294c3aac-8c9b-45f0-8c0a-344e2d7d8b7c),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap5543f379-2b')
Nov 29 03:16:06 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-cedcbf9c-14a1-41bf-a77f-6287a7a77e0c tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] [instance: f609a7b7-1be2-44b4-9177-7e72a4d4c060] Deleting instance files /opt/stack/data/nova/instances/f609a7b7-1be2-44b4-9177-7e72a4d4c060_del
Nov 29 03:16:06 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-cedcbf9c-14a1-41bf-a77f-6287a7a77e0c tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] [instance: f609a7b7-1be2-44b4-9177-7e72a4d4c060] Deletion of /opt/stack/data/nova/instances/f609a7b7-1be2-44b4-9177-7e72a4d4c060_del complete
Nov 29 03:16:06 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:16:06 user nova-compute[75234]: INFO nova.compute.manager [None req-cedcbf9c-14a1-41bf-a77f-6287a7a77e0c tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] [instance: f609a7b7-1be2-44b4-9177-7e72a4d4c060] Took 1.33 seconds to destroy the instance on the hypervisor.
Nov 29 03:16:06 user nova-compute[75234]: DEBUG oslo.service.loopingcall [None req-cedcbf9c-14a1-41bf-a77f-6287a7a77e0c tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=75234) func /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/loopingcall.py:435}}
Nov 29 03:16:06 user nova-compute[75234]: DEBUG nova.compute.manager [-] [instance: f609a7b7-1be2-44b4-9177-7e72a4d4c060] Deallocating network for instance {{(pid=75234) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2279}}
Nov 29 03:16:06 user nova-compute[75234]: DEBUG nova.network.neutron [-] [instance: f609a7b7-1be2-44b4-9177-7e72a4d4c060] deallocate_for_instance() {{(pid=75234) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1802}}
Nov 29 03:16:06 user nova-compute[75234]: DEBUG nova.compute.provider_tree [None req-a20fe69a-cdbf-482a-8a81-186f342fd171 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Inventory has not changed in ProviderTree for provider: 4d291220-7c37-4839-a0e3-b23c66502b74 {{(pid=75234) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Nov 29 03:16:06 user nova-compute[75234]: DEBUG nova.scheduler.client.report [None req-a20fe69a-cdbf-482a-8a81-186f342fd171 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Inventory has not changed for provider 4d291220-7c37-4839-a0e3-b23c66502b74 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=75234) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Nov 29 03:16:07 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-a20fe69a-cdbf-482a-8a81-186f342fd171 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.545s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:16:07 user nova-compute[75234]: DEBUG nova.compute.manager [None req-a20fe69a-cdbf-482a-8a81-186f342fd171 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] [instance: 1c8b7559-29ab-4982-8c75-ea2a0f0101cb] Start building networks asynchronously for instance. {{(pid=75234) _build_resources /opt/stack/nova/nova/compute/manager.py:2819}}
Nov 29 03:16:07 user nova-compute[75234]: DEBUG nova.compute.manager [None req-a20fe69a-cdbf-482a-8a81-186f342fd171 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] [instance: 1c8b7559-29ab-4982-8c75-ea2a0f0101cb] Allocating IP information in the background. {{(pid=75234) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1972}}
Nov 29 03:16:07 user nova-compute[75234]: DEBUG nova.network.neutron [None req-a20fe69a-cdbf-482a-8a81-186f342fd171 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] [instance: 1c8b7559-29ab-4982-8c75-ea2a0f0101cb] allocate_for_instance() {{(pid=75234) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1156}}
Nov 29 03:16:07 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-a20fe69a-cdbf-482a-8a81-186f342fd171 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] [instance: 1c8b7559-29ab-4982-8c75-ea2a0f0101cb] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Nov 29 03:16:07 user nova-compute[75234]: DEBUG nova.compute.manager [None req-a20fe69a-cdbf-482a-8a81-186f342fd171 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] [instance: 1c8b7559-29ab-4982-8c75-ea2a0f0101cb] Start building block device mappings for instance. {{(pid=75234) _build_resources /opt/stack/nova/nova/compute/manager.py:2854}}
Nov 29 03:16:07 user nova-compute[75234]: DEBUG nova.compute.manager [None req-a20fe69a-cdbf-482a-8a81-186f342fd171 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] [instance: 1c8b7559-29ab-4982-8c75-ea2a0f0101cb] Start spawning the instance on the hypervisor. {{(pid=75234) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2628}}
Nov 29 03:16:07 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-a20fe69a-cdbf-482a-8a81-186f342fd171 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] [instance: 1c8b7559-29ab-4982-8c75-ea2a0f0101cb] Creating instance directory {{(pid=75234) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4727}}
Nov 29 03:16:07 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-a20fe69a-cdbf-482a-8a81-186f342fd171 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] [instance: 1c8b7559-29ab-4982-8c75-ea2a0f0101cb] Creating image(s)
Nov 29 03:16:07 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-a20fe69a-cdbf-482a-8a81-186f342fd171 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Acquiring lock "/opt/stack/data/nova/instances/1c8b7559-29ab-4982-8c75-ea2a0f0101cb/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:16:07 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-a20fe69a-cdbf-482a-8a81-186f342fd171 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Lock "/opt/stack/data/nova/instances/1c8b7559-29ab-4982-8c75-ea2a0f0101cb/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:16:07 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-a20fe69a-cdbf-482a-8a81-186f342fd171 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Lock "/opt/stack/data/nova/instances/1c8b7559-29ab-4982-8c75-ea2a0f0101cb/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.011s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:16:07 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-a20fe69a-cdbf-482a-8a81-186f342fd171 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Acquiring lock "611b721f56c5ee20e0f039f936b83393504a68ef" by "nova.virt.libvirt.imagebackend.Image.cache.<locals>.fetch_func_sync" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:16:07 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-a20fe69a-cdbf-482a-8a81-186f342fd171 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Lock "611b721f56c5ee20e0f039f936b83393504a68ef" acquired by "nova.virt.libvirt.imagebackend.Image.cache.<locals>.fetch_func_sync" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:16:08 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._sync_scheduler_instance_info {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:16:08 user nova-compute[75234]: DEBUG nova.policy [None req-a20fe69a-cdbf-482a-8a81-186f342fd171 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': 'dc1de1dad3794b47b0e4bf7445c96997', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c4393bc598a845a9bfb667e5332f14a6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=75234) authorize /opt/stack/nova/nova/policy.py:203}}
Nov 29 03:16:08 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:16:08 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-a20fe69a-cdbf-482a-8a81-186f342fd171 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/611b721f56c5ee20e0f039f936b83393504a68ef.part --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:16:08 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:16:08 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:16:08 user nova-compute[75234]: DEBUG nova.compute.manager [req-11d5d792-dbbd-49de-a896-b587b80ed567 req-1e3c7ed3-4f38-425d-9069-bc31584b4350 service nova] [instance: f609a7b7-1be2-44b4-9177-7e72a4d4c060] Received event network-vif-plugged-5543f379-2b3a-4dc2-bfe9-52f66a942257 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:16:08 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-11d5d792-dbbd-49de-a896-b587b80ed567 req-1e3c7ed3-4f38-425d-9069-bc31584b4350 service nova] Acquiring lock "f609a7b7-1be2-44b4-9177-7e72a4d4c060-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:16:08 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-11d5d792-dbbd-49de-a896-b587b80ed567 req-1e3c7ed3-4f38-425d-9069-bc31584b4350 service nova] Lock "f609a7b7-1be2-44b4-9177-7e72a4d4c060-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:16:08 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-11d5d792-dbbd-49de-a896-b587b80ed567 req-1e3c7ed3-4f38-425d-9069-bc31584b4350 service nova] Lock "f609a7b7-1be2-44b4-9177-7e72a4d4c060-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:16:08 user nova-compute[75234]: DEBUG nova.compute.manager [req-11d5d792-dbbd-49de-a896-b587b80ed567 req-1e3c7ed3-4f38-425d-9069-bc31584b4350 service nova] [instance: f609a7b7-1be2-44b4-9177-7e72a4d4c060] No waiting events found dispatching network-vif-plugged-5543f379-2b3a-4dc2-bfe9-52f66a942257 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:16:08 user nova-compute[75234]: WARNING nova.compute.manager [req-11d5d792-dbbd-49de-a896-b587b80ed567 req-1e3c7ed3-4f38-425d-9069-bc31584b4350 service nova] [instance: f609a7b7-1be2-44b4-9177-7e72a4d4c060] Received unexpected event network-vif-plugged-5543f379-2b3a-4dc2-bfe9-52f66a942257 for instance with vm_state active and task_state deleting.
Nov 29 03:16:08 user nova-compute[75234]: DEBUG nova.network.neutron [-] [instance: f609a7b7-1be2-44b4-9177-7e72a4d4c060] Updating instance_info_cache with network_info: [] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:16:08 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-a20fe69a-cdbf-482a-8a81-186f342fd171 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/611b721f56c5ee20e0f039f936b83393504a68ef.part --force-share --output=json" returned: 0 in 0.217s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:16:08 user nova-compute[75234]: DEBUG nova.virt.images [None req-a20fe69a-cdbf-482a-8a81-186f342fd171 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] 5e298e24-6bfe-4aab-ac7a-4b54d9baa799 was qcow2, converting to raw {{(pid=75234) fetch_to_raw /opt/stack/nova/nova/virt/images.py:165}}
Nov 29 03:16:08 user nova-compute[75234]: DEBUG nova.privsep.utils [None req-a20fe69a-cdbf-482a-8a81-186f342fd171 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Path '/opt/stack/data/nova/instances' supports direct I/O {{(pid=75234) supports_direct_io /opt/stack/nova/nova/privsep/utils.py:63}}
Nov 29 03:16:08 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-a20fe69a-cdbf-482a-8a81-186f342fd171 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Running cmd (subprocess): qemu-img convert -t none -O raw -f qcow2 /opt/stack/data/nova/instances/_base/611b721f56c5ee20e0f039f936b83393504a68ef.part /opt/stack/data/nova/instances/_base/611b721f56c5ee20e0f039f936b83393504a68ef.converted {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:16:08 user nova-compute[75234]: INFO nova.compute.manager [-] [instance: f609a7b7-1be2-44b4-9177-7e72a4d4c060] Took 2.01 seconds to deallocate network for instance.
Nov 29 03:16:09 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-cedcbf9c-14a1-41bf-a77f-6287a7a77e0c tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:16:09 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-cedcbf9c-14a1-41bf-a77f-6287a7a77e0c tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:16:09 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-a20fe69a-cdbf-482a-8a81-186f342fd171 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] CMD "qemu-img convert -t none -O raw -f qcow2 /opt/stack/data/nova/instances/_base/611b721f56c5ee20e0f039f936b83393504a68ef.part /opt/stack/data/nova/instances/_base/611b721f56c5ee20e0f039f936b83393504a68ef.converted" returned: 0 in 0.805s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:16:09 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-a20fe69a-cdbf-482a-8a81-186f342fd171 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/611b721f56c5ee20e0f039f936b83393504a68ef.converted --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:16:09 user nova-compute[75234]: DEBUG nova.compute.provider_tree [None req-cedcbf9c-14a1-41bf-a77f-6287a7a77e0c tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Inventory has not changed in ProviderTree for provider: 4d291220-7c37-4839-a0e3-b23c66502b74 {{(pid=75234) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Nov 29 03:16:09 user nova-compute[75234]: DEBUG nova.scheduler.client.report [None req-cedcbf9c-14a1-41bf-a77f-6287a7a77e0c tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Inventory has not changed for provider 4d291220-7c37-4839-a0e3-b23c66502b74 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=75234) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Nov 29 03:16:09 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-cedcbf9c-14a1-41bf-a77f-6287a7a77e0c tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.759s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:16:09 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-a61d564c-9cb3-41fe-ba31-a62940d14628 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] Acquiring lock "aa190a9f-ba53-43fb-8bae-2c429d0d1228" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:16:09 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-a61d564c-9cb3-41fe-ba31-a62940d14628 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] Lock "aa190a9f-ba53-43fb-8bae-2c429d0d1228" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:16:09 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-a20fe69a-cdbf-482a-8a81-186f342fd171 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/611b721f56c5ee20e0f039f936b83393504a68ef.converted --force-share --output=json" returned: 0 in 0.206s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:16:09 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-a20fe69a-cdbf-482a-8a81-186f342fd171 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Lock "611b721f56c5ee20e0f039f936b83393504a68ef" "released" by "nova.virt.libvirt.imagebackend.Image.cache.<locals>.fetch_func_sync" :: held 2.523s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:16:09 user nova-compute[75234]: DEBUG nova.compute.manager [None req-a61d564c-9cb3-41fe-ba31-a62940d14628 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] [instance: aa190a9f-ba53-43fb-8bae-2c429d0d1228] Starting instance... {{(pid=75234) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2422}}
Nov 29 03:16:09 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-a20fe69a-cdbf-482a-8a81-186f342fd171 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/611b721f56c5ee20e0f039f936b83393504a68ef --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:16:09 user nova-compute[75234]: DEBUG nova.network.neutron [None req-a20fe69a-cdbf-482a-8a81-186f342fd171 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] [instance: 1c8b7559-29ab-4982-8c75-ea2a0f0101cb] Successfully created port: 22cdbc69-d4f4-49cd-8104-d508aea94101 {{(pid=75234) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:548}}
Nov 29 03:16:09 user nova-compute[75234]: INFO nova.scheduler.client.report [None req-cedcbf9c-14a1-41bf-a77f-6287a7a77e0c tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Deleted allocations for instance f609a7b7-1be2-44b4-9177-7e72a4d4c060
Nov 29 03:16:09 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-a61d564c-9cb3-41fe-ba31-a62940d14628 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:16:09 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-a61d564c-9cb3-41fe-ba31-a62940d14628 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:16:10 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-a61d564c-9cb3-41fe-ba31-a62940d14628 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=75234) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2366}}
Nov 29 03:16:10 user nova-compute[75234]: INFO nova.compute.claims [None req-a61d564c-9cb3-41fe-ba31-a62940d14628 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] [instance: aa190a9f-ba53-43fb-8bae-2c429d0d1228] Claim successful on node user
Nov 29 03:16:10 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-cedcbf9c-14a1-41bf-a77f-6287a7a77e0c tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Lock "f609a7b7-1be2-44b4-9177-7e72a4d4c060" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 4.573s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:16:10 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-a20fe69a-cdbf-482a-8a81-186f342fd171 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/611b721f56c5ee20e0f039f936b83393504a68ef --force-share --output=json" returned: 0 in 0.239s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:16:10 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-a20fe69a-cdbf-482a-8a81-186f342fd171 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Acquiring lock "611b721f56c5ee20e0f039f936b83393504a68ef" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:16:10 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-a20fe69a-cdbf-482a-8a81-186f342fd171 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Lock "611b721f56c5ee20e0f039f936b83393504a68ef" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.002s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:16:10 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-a20fe69a-cdbf-482a-8a81-186f342fd171 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/611b721f56c5ee20e0f039f936b83393504a68ef --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:16:10 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-a20fe69a-cdbf-482a-8a81-186f342fd171 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/611b721f56c5ee20e0f039f936b83393504a68ef --force-share --output=json" returned: 0 in 0.189s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:16:10 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-a20fe69a-cdbf-482a-8a81-186f342fd171 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/611b721f56c5ee20e0f039f936b83393504a68ef,backing_fmt=raw /opt/stack/data/nova/instances/1c8b7559-29ab-4982-8c75-ea2a0f0101cb/disk 1073741824 {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:16:10 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-a20fe69a-cdbf-482a-8a81-186f342fd171 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/611b721f56c5ee20e0f039f936b83393504a68ef,backing_fmt=raw /opt/stack/data/nova/instances/1c8b7559-29ab-4982-8c75-ea2a0f0101cb/disk 1073741824" returned: 0 in 0.074s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:16:10 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-a20fe69a-cdbf-482a-8a81-186f342fd171 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Lock "611b721f56c5ee20e0f039f936b83393504a68ef" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.269s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:16:10 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-a20fe69a-cdbf-482a-8a81-186f342fd171 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/611b721f56c5ee20e0f039f936b83393504a68ef --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:16:10 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-a20fe69a-cdbf-482a-8a81-186f342fd171 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/611b721f56c5ee20e0f039f936b83393504a68ef --force-share --output=json" returned: 0 in 0.207s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:16:10 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-a20fe69a-cdbf-482a-8a81-186f342fd171 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] [instance: 1c8b7559-29ab-4982-8c75-ea2a0f0101cb] Created local disks {{(pid=75234) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4861}}
Nov 29 03:16:10 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-a20fe69a-cdbf-482a-8a81-186f342fd171 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] [instance: 1c8b7559-29ab-4982-8c75-ea2a0f0101cb] Ensure instance console log exists: /opt/stack/data/nova/instances/1c8b7559-29ab-4982-8c75-ea2a0f0101cb/console.log {{(pid=75234) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4613}}
Nov 29 03:16:10 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-a20fe69a-cdbf-482a-8a81-186f342fd171 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:16:10 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-a20fe69a-cdbf-482a-8a81-186f342fd171 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:16:10 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-a20fe69a-cdbf-482a-8a81-186f342fd171 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:16:10 user nova-compute[75234]: DEBUG nova.compute.provider_tree [None req-a61d564c-9cb3-41fe-ba31-a62940d14628 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] Inventory has not changed in ProviderTree for provider: 4d291220-7c37-4839-a0e3-b23c66502b74 {{(pid=75234) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Nov 29 03:16:10 user nova-compute[75234]: DEBUG nova.network.neutron [None req-a20fe69a-cdbf-482a-8a81-186f342fd171 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] [instance: 1c8b7559-29ab-4982-8c75-ea2a0f0101cb] Successfully updated port: 22cdbc69-d4f4-49cd-8104-d508aea94101 {{(pid=75234) _update_port /opt/stack/nova/nova/network/neutron.py:586}}
Nov 29 03:16:10 user nova-compute[75234]: DEBUG nova.scheduler.client.report [None req-a61d564c-9cb3-41fe-ba31-a62940d14628 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] Inventory has not changed for provider 4d291220-7c37-4839-a0e3-b23c66502b74 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=75234) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Nov 29 03:16:10 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-a20fe69a-cdbf-482a-8a81-186f342fd171 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Acquiring lock "refresh_cache-1c8b7559-29ab-4982-8c75-ea2a0f0101cb" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:16:10 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-a20fe69a-cdbf-482a-8a81-186f342fd171 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Acquired lock "refresh_cache-1c8b7559-29ab-4982-8c75-ea2a0f0101cb" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:16:10 user nova-compute[75234]: DEBUG nova.network.neutron [None req-a20fe69a-cdbf-482a-8a81-186f342fd171 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] [instance: 1c8b7559-29ab-4982-8c75-ea2a0f0101cb] Building network info cache for instance {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2009}}
Nov 29 03:16:10 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-a61d564c-9cb3-41fe-ba31-a62940d14628 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.709s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:16:10 user nova-compute[75234]: DEBUG nova.compute.manager [None req-a61d564c-9cb3-41fe-ba31-a62940d14628 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] [instance: aa190a9f-ba53-43fb-8bae-2c429d0d1228] Start building networks asynchronously for instance. {{(pid=75234) _build_resources /opt/stack/nova/nova/compute/manager.py:2819}}
Nov 29 03:16:10 user nova-compute[75234]: DEBUG nova.compute.manager [req-a60034ab-3a19-4988-9408-f640a1ba2070 req-264073a0-4a3e-40e7-907b-48f95a7d6042 service nova] [instance: f609a7b7-1be2-44b4-9177-7e72a4d4c060] Received event network-vif-deleted-5543f379-2b3a-4dc2-bfe9-52f66a942257 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:16:10 user nova-compute[75234]: DEBUG nova.network.neutron [None req-a20fe69a-cdbf-482a-8a81-186f342fd171 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] [instance: 1c8b7559-29ab-4982-8c75-ea2a0f0101cb] Instance cache missing network info. {{(pid=75234) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3322}}
Nov 29 03:16:10 user nova-compute[75234]: DEBUG nova.compute.manager [None req-a61d564c-9cb3-41fe-ba31-a62940d14628 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] [instance: aa190a9f-ba53-43fb-8bae-2c429d0d1228] Allocating IP information in the background. {{(pid=75234) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1972}}
Nov 29 03:16:10 user nova-compute[75234]: DEBUG nova.network.neutron [None req-a61d564c-9cb3-41fe-ba31-a62940d14628 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] [instance: aa190a9f-ba53-43fb-8bae-2c429d0d1228] allocate_for_instance() {{(pid=75234) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1156}}
Nov 29 03:16:10 user nova-compute[75234]: DEBUG nova.compute.manager [req-7ca9fbce-b146-4300-9e49-2b8c34423c46 req-581ab0eb-4b46-4d9b-a346-5e9d40746050 service nova] [instance: 1c8b7559-29ab-4982-8c75-ea2a0f0101cb] Received event network-changed-22cdbc69-d4f4-49cd-8104-d508aea94101 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:16:10 user nova-compute[75234]: DEBUG nova.compute.manager [req-7ca9fbce-b146-4300-9e49-2b8c34423c46 req-581ab0eb-4b46-4d9b-a346-5e9d40746050 service nova] [instance: 1c8b7559-29ab-4982-8c75-ea2a0f0101cb] Refreshing instance network info cache due to event network-changed-22cdbc69-d4f4-49cd-8104-d508aea94101. {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11117}}
Nov 29 03:16:10 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-7ca9fbce-b146-4300-9e49-2b8c34423c46 req-581ab0eb-4b46-4d9b-a346-5e9d40746050 service nova] Acquiring lock "refresh_cache-1c8b7559-29ab-4982-8c75-ea2a0f0101cb" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:16:10 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-a61d564c-9cb3-41fe-ba31-a62940d14628 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] [instance: aa190a9f-ba53-43fb-8bae-2c429d0d1228] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Nov 29 03:16:10 user nova-compute[75234]: DEBUG nova.compute.manager [None req-a61d564c-9cb3-41fe-ba31-a62940d14628 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] [instance: aa190a9f-ba53-43fb-8bae-2c429d0d1228] Start building block device mappings for instance. {{(pid=75234) _build_resources /opt/stack/nova/nova/compute/manager.py:2854}}
Nov 29 03:16:10 user nova-compute[75234]: INFO nova.virt.block_device [None req-a61d564c-9cb3-41fe-ba31-a62940d14628 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] [instance: aa190a9f-ba53-43fb-8bae-2c429d0d1228] Booting with volume 07f83efc-272d-4e10-9a2b-48d967db613c at /dev/vda
Nov 29 03:16:10 user nova-compute[75234]: DEBUG nova.policy [None req-a61d564c-9cb3-41fe-ba31-a62940d14628 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': 'd11ce50c391d4e3bbe92d6c463d14746', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '581a0b62fee744e1b9a767a0d011c4e0', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=75234) authorize /opt/stack/nova/nova/policy.py:203}}
Nov 29 03:16:11 user nova-compute[75234]: DEBUG os_brick.utils [None req-a61d564c-9cb3-41fe-ba31-a62940d14628 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:176}}
Nov 29 03:16:11 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:16:11 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.015s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:16:11 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[e8564d60-72b4-43fb-948c-6b54f17cfde7]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:dd1368c5ba2c\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:16:11 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt -v / -n -o SOURCE {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:16:11 user nova-compute[75234]: DEBUG nova.network.neutron [None req-a20fe69a-cdbf-482a-8a81-186f342fd171 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] [instance: 1c8b7559-29ab-4982-8c75-ea2a0f0101cb] Updating instance_info_cache with network_info: [{"id": "22cdbc69-d4f4-49cd-8104-d508aea94101", "address": "fa:16:3e:79:d1:ca", "network": {"id": "b7a6cef0-4cf1-46e6-99fb-4972879e3913", "bridge": "br-int", "label": "tempest-TestStampPattern-1195861223-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "c4393bc598a845a9bfb667e5332f14a6", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap22cdbc69-d4", "ovs_interfaceid": "22cdbc69-d4f4-49cd-8104-d508aea94101", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:16:11 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt -v / -n -o SOURCE" returned: 0 in 0.023s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:16:11 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[1723c013-583a-471b-83c4-9342d2c4a279]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:16:11 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:16:11 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-a20fe69a-cdbf-482a-8a81-186f342fd171 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Releasing lock "refresh_cache-1c8b7559-29ab-4982-8c75-ea2a0f0101cb" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:16:11 user nova-compute[75234]: DEBUG nova.compute.manager [None req-a20fe69a-cdbf-482a-8a81-186f342fd171 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] [instance: 1c8b7559-29ab-4982-8c75-ea2a0f0101cb] Instance network_info: |[{"id": "22cdbc69-d4f4-49cd-8104-d508aea94101", "address": "fa:16:3e:79:d1:ca", "network": {"id": "b7a6cef0-4cf1-46e6-99fb-4972879e3913", "bridge": "br-int", "label": "tempest-TestStampPattern-1195861223-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "c4393bc598a845a9bfb667e5332f14a6", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap22cdbc69-d4", "ovs_interfaceid": "22cdbc69-d4f4-49cd-8104-d508aea94101", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=75234) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1987}}
Nov 29 03:16:11 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-7ca9fbce-b146-4300-9e49-2b8c34423c46 req-581ab0eb-4b46-4d9b-a346-5e9d40746050 service nova] Acquired lock "refresh_cache-1c8b7559-29ab-4982-8c75-ea2a0f0101cb" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:16:11 user nova-compute[75234]: DEBUG nova.network.neutron [req-7ca9fbce-b146-4300-9e49-2b8c34423c46 req-581ab0eb-4b46-4d9b-a346-5e9d40746050 service nova] [instance: 1c8b7559-29ab-4982-8c75-ea2a0f0101cb] Refreshing network info cache for port 22cdbc69-d4f4-49cd-8104-d508aea94101 {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2006}}
Nov 29 03:16:11 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-a20fe69a-cdbf-482a-8a81-186f342fd171 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] [instance: 1c8b7559-29ab-4982-8c75-ea2a0f0101cb] Start _get_guest_xml network_info=[{"id": "22cdbc69-d4f4-49cd-8104-d508aea94101", "address": "fa:16:3e:79:d1:ca", "network": {"id": "b7a6cef0-4cf1-46e6-99fb-4972879e3913", "bridge": "br-int", "label": "tempest-TestStampPattern-1195861223-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "c4393bc598a845a9bfb667e5332f14a6", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap22cdbc69-d4", "ovs_interfaceid": "22cdbc69-d4f4-49cd-8104-d508aea94101", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum='2d07e5bfb8e8bcf4ad20b7319f4bc1de',container_format='bare',created_at=2023-11-29T03:15:54Z,direct_url=<?>,disk_format='qcow2',id=5e298e24-6bfe-4aab-ac7a-4b54d9baa799,min_disk=1,min_ram=0,name='tempest-TestStampPatternsnapshot-1691770650',owner='c4393bc598a845a9bfb667e5332f14a6',properties=ImageMetaProps,protected=<?>,size=52363264,status='active',tags=<?>,updated_at=2023-11-29T03:16:01Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [{'device_name': '/dev/vda', 'encryption_format': None, 'encrypted': False, 'device_type': 'disk', 'size': 0, 'disk_bus': 'virtio', 'boot_index': 0, 'encryption_options': None, 'encryption_secret_uuid': None, 'guest_format': None, 'image_id': '5e298e24-6bfe-4aab-ac7a-4b54d9baa799'}], 'ephemerals': [], 'block_device_mapping': [], 'swap': None} {{(pid=75234) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7562}}
Nov 29 03:16:11 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.019s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:16:11 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[c8e40b3c-18e0-492c-9560-3a8270979283]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:16:11 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[e3689620-7166-473f-bfe3-a6d03c8c7d0b]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:16:11 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-a61d564c-9cb3-41fe-ba31-a62940d14628 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] Running cmd (subprocess): nvme version {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:16:11 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-a61d564c-9cb3-41fe-ba31-a62940d14628 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] 'nvme version' failed. Not Retrying. {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:473}}
Nov 29 03:16:11 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.nvmeof [None req-a61d564c-9cb3-41fe-ba31-a62940d14628 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] nvme not present on system {{(pid=75234) nvme_present /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/nvmeof.py:756}}
Nov 29 03:16:11 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:16:11 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-a20fe69a-cdbf-482a-8a81-186f342fd171 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:16:11 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-a20fe69a-cdbf-482a-8a81-186f342fd171 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:16:11 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:473}}
Nov 29 03:16:11 user nova-compute[75234]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Nov 29 03:16:11 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[d643e84a-5c02-453b-8615-b223b55b0612]: (4, '') {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:16:11 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.lightos [None req-a61d564c-9cb3-41fe-ba31-a62940d14628 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=75234) find_dsc /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/lightos.py:98}}
Nov 29 03:16:11 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.lightos [None req-a61d564c-9cb3-41fe-ba31-a62940d14628 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=75234) get_connector_properties /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/lightos.py:76}}
Nov 29 03:16:11 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.lightos [None req-a61d564c-9cb3-41fe-ba31-a62940d14628 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] LIGHTOS: no hostnqn found. {{(pid=75234) get_connector_properties /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/lightos.py:84}}
Nov 29 03:16:11 user nova-compute[75234]: DEBUG os_brick.utils [None req-a61d564c-9cb3-41fe-ba31-a62940d14628 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] <== get_connector_properties: return (196ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:dd1368c5ba2c', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:203}}
Nov 29 03:16:11 user nova-compute[75234]: DEBUG nova.virt.block_device [None req-a61d564c-9cb3-41fe-ba31-a62940d14628 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] [instance: aa190a9f-ba53-43fb-8bae-2c429d0d1228] Updating existing volume attachment record: adf00bdf-5ffc-418d-b9a1-701ce0a95383 {{(pid=75234) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Nov 29 03:16:11 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-a20fe69a-cdbf-482a-8a81-186f342fd171 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Searching host: 'user' for CPU controller through CGroups V1... {{(pid=75234) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1652}}
Nov 29 03:16:11 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-a20fe69a-cdbf-482a-8a81-186f342fd171 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] CPU controller missing on host. {{(pid=75234) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1662}}
Nov 29 03:16:11 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-a20fe69a-cdbf-482a-8a81-186f342fd171 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Searching host: 'user' for CPU controller through CGroups V2... {{(pid=75234) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1671}}
Nov 29 03:16:11 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-a20fe69a-cdbf-482a-8a81-186f342fd171 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] CPU controller found on host. {{(pid=75234) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1678}}
Nov 29 03:16:11 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-a20fe69a-cdbf-482a-8a81-186f342fd171 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=75234) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5400}}
Nov 29 03:16:11 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-a20fe69a-cdbf-482a-8a81-186f342fd171 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-11-29T03:02:04Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='1',id=6,is_public=True,memory_mb=512,name='m1.tiny',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='2d07e5bfb8e8bcf4ad20b7319f4bc1de',container_format='bare',created_at=2023-11-29T03:15:54Z,direct_url=<?>,disk_format='qcow2',id=5e298e24-6bfe-4aab-ac7a-4b54d9baa799,min_disk=1,min_ram=0,name='tempest-TestStampPatternsnapshot-1691770650',owner='c4393bc598a845a9bfb667e5332f14a6',properties=ImageMetaProps,protected=<?>,size=52363264,status='active',tags=<?>,updated_at=2023-11-29T03:16:01Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:563}}
Nov 29 03:16:11 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-a20fe69a-cdbf-482a-8a81-186f342fd171 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Flavor limits 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:348}}
Nov 29 03:16:11 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-a20fe69a-cdbf-482a-8a81-186f342fd171 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Image limits 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:352}}
Nov 29 03:16:11 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-a20fe69a-cdbf-482a-8a81-186f342fd171 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Flavor pref 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:388}}
Nov 29 03:16:11 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-a20fe69a-cdbf-482a-8a81-186f342fd171 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Image pref 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:392}}
Nov 29 03:16:11 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-a20fe69a-cdbf-482a-8a81-186f342fd171 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:430}}
Nov 29 03:16:11 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-a20fe69a-cdbf-482a-8a81-186f342fd171 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:569}}
Nov 29 03:16:11 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-a20fe69a-cdbf-482a-8a81-186f342fd171 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=75234) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:471}}
Nov 29 03:16:11 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-a20fe69a-cdbf-482a-8a81-186f342fd171 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Got 1 possible topologies {{(pid=75234) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:501}}
Nov 29 03:16:11 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-a20fe69a-cdbf-482a-8a81-186f342fd171 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:575}}
Nov 29 03:16:11 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-a20fe69a-cdbf-482a-8a81-186f342fd171 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:577}}
Nov 29 03:16:11 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-a20fe69a-cdbf-482a-8a81-186f342fd171 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-11-29T03:16:03Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-TestStampPattern-server-785913193',display_name='tempest-TestStampPattern-server-785913193',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-teststamppattern-server-785913193',id=26,image_ref='5e298e24-6bfe-4aab-ac7a-4b54d9baa799',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBMhfiLNgC9PAWcr8xAscyt4FB2hoan+iXcHV+JRry3FLwwpkjQpDEiOvBF2D+f1pgs1A/1GOV9t1zFfztYSi2DW0KkfBHuJTofrSJvEENNcXO0BnR5RakzAYYv+USO8uHQ==',key_name='tempest-TestStampPattern-1617966982',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='c4393bc598a845a9bfb667e5332f14a6',ramdisk_id='',reservation_id='r-7smjv19e',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',image_boot_roles='reader,member',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_image_location='snapshot',image_image_state='available',image_image_type='snapshot',image_instance_uuid='6894b55f-93c6-43c9-8933-56d5a3c43725',image_min_disk='1',image_min_ram='0',image_owner_id='c4393bc598a845a9bfb667e5332f14a6',image_owner_project_name='tempest-TestStampPattern-448969956',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',image_owner_user_name='tempest-TestStampPattern-448969956-project-member',image_user_id='dc1de1dad3794b47b0e4bf7445c96997',network_allocated='True',owner_project_name='tempest-TestStampPattern-448969956',owner_user_name='tempest-TestStampPattern-448969956-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-11-29T03:16:07Z,user_data=None,user_id='dc1de1dad3794b47b0e4bf7445c96997',uuid=1c8b7559-29ab-4982-8c75-ea2a0f0101cb,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "22cdbc69-d4f4-49cd-8104-d508aea94101", "address": "fa:16:3e:79:d1:ca", "network": {"id": "b7a6cef0-4cf1-46e6-99fb-4972879e3913", "bridge": "br-int", "label": "tempest-TestStampPattern-1195861223-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "c4393bc598a845a9bfb667e5332f14a6", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap22cdbc69-d4", "ovs_interfaceid": "22cdbc69-d4f4-49cd-8104-d508aea94101", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=75234) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Nov 29 03:16:11 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-a20fe69a-cdbf-482a-8a81-186f342fd171 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Converting VIF {"id": "22cdbc69-d4f4-49cd-8104-d508aea94101", "address": "fa:16:3e:79:d1:ca", "network": {"id": "b7a6cef0-4cf1-46e6-99fb-4972879e3913", "bridge": "br-int", "label": "tempest-TestStampPattern-1195861223-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "c4393bc598a845a9bfb667e5332f14a6", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap22cdbc69-d4", "ovs_interfaceid": "22cdbc69-d4f4-49cd-8104-d508aea94101", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:16:11 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-a20fe69a-cdbf-482a-8a81-186f342fd171 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:79:d1:ca,bridge_name='br-int',has_traffic_filtering=True,id=22cdbc69-d4f4-49cd-8104-d508aea94101,network=Network(b7a6cef0-4cf1-46e6-99fb-4972879e3913),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap22cdbc69-d4') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:16:11 user nova-compute[75234]: DEBUG nova.objects.instance [None req-a20fe69a-cdbf-482a-8a81-186f342fd171 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Lazy-loading 'pci_devices' on Instance uuid 1c8b7559-29ab-4982-8c75-ea2a0f0101cb {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:16:11 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-a20fe69a-cdbf-482a-8a81-186f342fd171 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] [instance: 1c8b7559-29ab-4982-8c75-ea2a0f0101cb] End _get_guest_xml xml=<domain type="kvm">
Nov 29 03:16:11 user nova-compute[75234]:   <uuid>1c8b7559-29ab-4982-8c75-ea2a0f0101cb</uuid>
Nov 29 03:16:11 user nova-compute[75234]:   <name>instance-0000001a</name>
Nov 29 03:16:11 user nova-compute[75234]:   <memory>524288</memory>
Nov 29 03:16:11 user nova-compute[75234]:   <vcpu>1</vcpu>
Nov 29 03:16:11 user nova-compute[75234]:   <metadata>
Nov 29 03:16:11 user nova-compute[75234]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Nov 29 03:16:11 user nova-compute[75234]:       <nova:package version="0.0.0"/>
Nov 29 03:16:11 user nova-compute[75234]:       <nova:name>tempest-TestStampPattern-server-785913193</nova:name>
Nov 29 03:16:11 user nova-compute[75234]:       <nova:creationTime>2023-11-29 03:16:11</nova:creationTime>
Nov 29 03:16:11 user nova-compute[75234]:       <nova:flavor name="m1.tiny">
Nov 29 03:16:11 user nova-compute[75234]:         <nova:memory>512</nova:memory>
Nov 29 03:16:11 user nova-compute[75234]:         <nova:disk>1</nova:disk>
Nov 29 03:16:11 user nova-compute[75234]:         <nova:swap>0</nova:swap>
Nov 29 03:16:11 user nova-compute[75234]:         <nova:ephemeral>0</nova:ephemeral>
Nov 29 03:16:11 user nova-compute[75234]:         <nova:vcpus>1</nova:vcpus>
Nov 29 03:16:11 user nova-compute[75234]:       </nova:flavor>
Nov 29 03:16:11 user nova-compute[75234]:       <nova:owner>
Nov 29 03:16:11 user nova-compute[75234]:         <nova:user uuid="dc1de1dad3794b47b0e4bf7445c96997">tempest-TestStampPattern-448969956-project-member</nova:user>
Nov 29 03:16:11 user nova-compute[75234]:         <nova:project uuid="c4393bc598a845a9bfb667e5332f14a6">tempest-TestStampPattern-448969956</nova:project>
Nov 29 03:16:11 user nova-compute[75234]:       </nova:owner>
Nov 29 03:16:11 user nova-compute[75234]:       <nova:root type="image" uuid="5e298e24-6bfe-4aab-ac7a-4b54d9baa799"/>
Nov 29 03:16:11 user nova-compute[75234]:       <nova:ports>
Nov 29 03:16:11 user nova-compute[75234]:         <nova:port uuid="22cdbc69-d4f4-49cd-8104-d508aea94101">
Nov 29 03:16:11 user nova-compute[75234]:           <nova:ip type="fixed" address="10.0.0.13" ipVersion="4"/>
Nov 29 03:16:11 user nova-compute[75234]:         </nova:port>
Nov 29 03:16:11 user nova-compute[75234]:       </nova:ports>
Nov 29 03:16:11 user nova-compute[75234]:     </nova:instance>
Nov 29 03:16:11 user nova-compute[75234]:   </metadata>
Nov 29 03:16:11 user nova-compute[75234]:   <sysinfo type="smbios">
Nov 29 03:16:11 user nova-compute[75234]:     <system>
Nov 29 03:16:11 user nova-compute[75234]:       <entry name="manufacturer">OpenStack Foundation</entry>
Nov 29 03:16:11 user nova-compute[75234]:       <entry name="product">OpenStack Nova</entry>
Nov 29 03:16:11 user nova-compute[75234]:       <entry name="version">0.0.0</entry>
Nov 29 03:16:11 user nova-compute[75234]:       <entry name="serial">1c8b7559-29ab-4982-8c75-ea2a0f0101cb</entry>
Nov 29 03:16:11 user nova-compute[75234]:       <entry name="uuid">1c8b7559-29ab-4982-8c75-ea2a0f0101cb</entry>
Nov 29 03:16:11 user nova-compute[75234]:       <entry name="family">Virtual Machine</entry>
Nov 29 03:16:11 user nova-compute[75234]:     </system>
Nov 29 03:16:11 user nova-compute[75234]:   </sysinfo>
Nov 29 03:16:11 user nova-compute[75234]:   <os>
Nov 29 03:16:11 user nova-compute[75234]:     <type arch="x86_64" machine="pc">hvm</type>
Nov 29 03:16:11 user nova-compute[75234]:     <boot dev="hd"/>
Nov 29 03:16:11 user nova-compute[75234]:     <smbios mode="sysinfo"/>
Nov 29 03:16:11 user nova-compute[75234]:   </os>
Nov 29 03:16:11 user nova-compute[75234]:   <features>
Nov 29 03:16:11 user nova-compute[75234]:     <acpi/>
Nov 29 03:16:11 user nova-compute[75234]:     <apic/>
Nov 29 03:16:11 user nova-compute[75234]:     <vmcoreinfo/>
Nov 29 03:16:11 user nova-compute[75234]:   </features>
Nov 29 03:16:11 user nova-compute[75234]:   <clock offset="utc">
Nov 29 03:16:11 user nova-compute[75234]:     <timer name="pit" tickpolicy="delay"/>
Nov 29 03:16:11 user nova-compute[75234]:     <timer name="rtc" tickpolicy="catchup"/>
Nov 29 03:16:11 user nova-compute[75234]:     <timer name="hpet" present="no"/>
Nov 29 03:16:11 user nova-compute[75234]:   </clock>
Nov 29 03:16:11 user nova-compute[75234]:   <cpu mode="custom" match="exact">
Nov 29 03:16:11 user nova-compute[75234]:     <model>Nehalem</model>
Nov 29 03:16:11 user nova-compute[75234]:     <topology sockets="1" cores="1" threads="1"/>
Nov 29 03:16:11 user nova-compute[75234]:   </cpu>
Nov 29 03:16:11 user nova-compute[75234]:   <devices>
Nov 29 03:16:11 user nova-compute[75234]:     <disk type="file" device="disk">
Nov 29 03:16:11 user nova-compute[75234]:       <driver name="qemu" type="qcow2" cache="none"/>
Nov 29 03:16:11 user nova-compute[75234]:       <source file="/opt/stack/data/nova/instances/1c8b7559-29ab-4982-8c75-ea2a0f0101cb/disk"/>
Nov 29 03:16:11 user nova-compute[75234]:       <target dev="vda" bus="virtio"/>
Nov 29 03:16:11 user nova-compute[75234]:     </disk>
Nov 29 03:16:11 user nova-compute[75234]:     <interface type="ethernet">
Nov 29 03:16:11 user nova-compute[75234]:       <mac address="fa:16:3e:79:d1:ca"/>
Nov 29 03:16:11 user nova-compute[75234]:       <model type="virtio"/>
Nov 29 03:16:11 user nova-compute[75234]:       <mtu size="1442"/>
Nov 29 03:16:11 user nova-compute[75234]:       <target dev="tap22cdbc69-d4"/>
Nov 29 03:16:11 user nova-compute[75234]:     </interface>
Nov 29 03:16:11 user nova-compute[75234]:     <serial type="pty">
Nov 29 03:16:11 user nova-compute[75234]:       <log file="/opt/stack/data/nova/instances/1c8b7559-29ab-4982-8c75-ea2a0f0101cb/console.log" append="off"/>
Nov 29 03:16:11 user nova-compute[75234]:     </serial>
Nov 29 03:16:11 user nova-compute[75234]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Nov 29 03:16:11 user nova-compute[75234]:     <video>
Nov 29 03:16:11 user nova-compute[75234]:       <model type="virtio"/>
Nov 29 03:16:11 user nova-compute[75234]:     </video>
Nov 29 03:16:11 user nova-compute[75234]:     <rng model="virtio">
Nov 29 03:16:11 user nova-compute[75234]:       <backend model="random">/dev/urandom</backend>
Nov 29 03:16:11 user nova-compute[75234]:     </rng>
Nov 29 03:16:11 user nova-compute[75234]:     <controller type="usb" index="0" model="none"/>
Nov 29 03:16:11 user nova-compute[75234]:     <memballoon model="virtio">
Nov 29 03:16:11 user nova-compute[75234]:       <stats period="10"/>
Nov 29 03:16:11 user nova-compute[75234]:     </memballoon>
Nov 29 03:16:11 user nova-compute[75234]:   </devices>
Nov 29 03:16:11 user nova-compute[75234]: </domain>
Nov 29 03:16:11 user nova-compute[75234]:  {{(pid=75234) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7568}}
Nov 29 03:16:11 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-a20fe69a-cdbf-482a-8a81-186f342fd171 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-11-29T03:16:03Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-TestStampPattern-server-785913193',display_name='tempest-TestStampPattern-server-785913193',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-teststamppattern-server-785913193',id=26,image_ref='5e298e24-6bfe-4aab-ac7a-4b54d9baa799',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBMhfiLNgC9PAWcr8xAscyt4FB2hoan+iXcHV+JRry3FLwwpkjQpDEiOvBF2D+f1pgs1A/1GOV9t1zFfztYSi2DW0KkfBHuJTofrSJvEENNcXO0BnR5RakzAYYv+USO8uHQ==',key_name='tempest-TestStampPattern-1617966982',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='c4393bc598a845a9bfb667e5332f14a6',ramdisk_id='',reservation_id='r-7smjv19e',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',image_boot_roles='reader,member',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_image_location='snapshot',image_image_state='available',image_image_type='snapshot',image_instance_uuid='6894b55f-93c6-43c9-8933-56d5a3c43725',image_min_disk='1',image_min_ram='0',image_owner_id='c4393bc598a845a9bfb667e5332f14a6',image_owner_project_name='tempest-TestStampPattern-448969956',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',image_owner_user_name='tempest-TestStampPattern-448969956-project-member',image_user_id='dc1de1dad3794b47b0e4bf7445c96997',network_allocated='True',owner_project_name='tempest-TestStampPattern-448969956',owner_user_name='tempest-TestStampPattern-448969956-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-11-29T03:16:07Z,user_data=None,user_id='dc1de1dad3794b47b0e4bf7445c96997',uuid=1c8b7559-29ab-4982-8c75-ea2a0f0101cb,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "22cdbc69-d4f4-49cd-8104-d508aea94101", "address": "fa:16:3e:79:d1:ca", "network": {"id": "b7a6cef0-4cf1-46e6-99fb-4972879e3913", "bridge": "br-int", "label": "tempest-TestStampPattern-1195861223-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "c4393bc598a845a9bfb667e5332f14a6", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap22cdbc69-d4", "ovs_interfaceid": "22cdbc69-d4f4-49cd-8104-d508aea94101", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Nov 29 03:16:11 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-a20fe69a-cdbf-482a-8a81-186f342fd171 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Converting VIF {"id": "22cdbc69-d4f4-49cd-8104-d508aea94101", "address": "fa:16:3e:79:d1:ca", "network": {"id": "b7a6cef0-4cf1-46e6-99fb-4972879e3913", "bridge": "br-int", "label": "tempest-TestStampPattern-1195861223-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "c4393bc598a845a9bfb667e5332f14a6", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap22cdbc69-d4", "ovs_interfaceid": "22cdbc69-d4f4-49cd-8104-d508aea94101", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:16:11 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-a20fe69a-cdbf-482a-8a81-186f342fd171 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:79:d1:ca,bridge_name='br-int',has_traffic_filtering=True,id=22cdbc69-d4f4-49cd-8104-d508aea94101,network=Network(b7a6cef0-4cf1-46e6-99fb-4972879e3913),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap22cdbc69-d4') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:16:11 user nova-compute[75234]: DEBUG os_vif [None req-a20fe69a-cdbf-482a-8a81-186f342fd171 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:79:d1:ca,bridge_name='br-int',has_traffic_filtering=True,id=22cdbc69-d4f4-49cd-8104-d508aea94101,network=Network(b7a6cef0-4cf1-46e6-99fb-4972879e3913),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap22cdbc69-d4') {{(pid=75234) plug /opt/stack/data/venv/lib/python3.10/site-packages/os_vif/__init__.py:76}}
Nov 29 03:16:11 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:16:11 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:16:11 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Nov 29 03:16:11 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:16:11 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbCreateCommand(_result=None, table=QoS, columns={'type': 'linux-noop', 'external_ids': {'id': 'a2f08d19-2836-577a-a61f-540297678dc2', '_type': 'linux-noop'}}, row=False) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:16:11 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:16:11 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:16:11 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:16:11 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tap22cdbc69-d4, may_exist=True, interface_attrs={}) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:16:11 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Port, record=tap22cdbc69-d4, col_values=(('qos', UUID('19072a08-5435-46a0-a950-c5e482f57315')),), if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:16:11 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DbSetCommand(_result=None, table=Interface, record=tap22cdbc69-d4, col_values=(('external_ids', {'iface-id': '22cdbc69-d4f4-49cd-8104-d508aea94101', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:79:d1:ca', 'vm-uuid': '1c8b7559-29ab-4982-8c75-ea2a0f0101cb'}),), if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:16:11 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:16:11 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:16:11 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:16:11 user nova-compute[75234]: INFO os_vif [None req-a20fe69a-cdbf-482a-8a81-186f342fd171 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:79:d1:ca,bridge_name='br-int',has_traffic_filtering=True,id=22cdbc69-d4f4-49cd-8104-d508aea94101,network=Network(b7a6cef0-4cf1-46e6-99fb-4972879e3913),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap22cdbc69-d4')
Nov 29 03:16:11 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-a20fe69a-cdbf-482a-8a81-186f342fd171 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] No BDM found with device name vda, not building metadata. {{(pid=75234) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12089}}
Nov 29 03:16:11 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-a20fe69a-cdbf-482a-8a81-186f342fd171 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] No VIF found with MAC fa:16:3e:79:d1:ca, not building metadata {{(pid=75234) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Nov 29 03:16:11 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:16:11 user nova-compute[75234]: DEBUG nova.network.neutron [req-7ca9fbce-b146-4300-9e49-2b8c34423c46 req-581ab0eb-4b46-4d9b-a346-5e9d40746050 service nova] [instance: 1c8b7559-29ab-4982-8c75-ea2a0f0101cb] Updated VIF entry in instance network info cache for port 22cdbc69-d4f4-49cd-8104-d508aea94101. {{(pid=75234) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3481}}
Nov 29 03:16:11 user nova-compute[75234]: DEBUG nova.network.neutron [req-7ca9fbce-b146-4300-9e49-2b8c34423c46 req-581ab0eb-4b46-4d9b-a346-5e9d40746050 service nova] [instance: 1c8b7559-29ab-4982-8c75-ea2a0f0101cb] Updating instance_info_cache with network_info: [{"id": "22cdbc69-d4f4-49cd-8104-d508aea94101", "address": "fa:16:3e:79:d1:ca", "network": {"id": "b7a6cef0-4cf1-46e6-99fb-4972879e3913", "bridge": "br-int", "label": "tempest-TestStampPattern-1195861223-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "c4393bc598a845a9bfb667e5332f14a6", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap22cdbc69-d4", "ovs_interfaceid": "22cdbc69-d4f4-49cd-8104-d508aea94101", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:16:11 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-7ca9fbce-b146-4300-9e49-2b8c34423c46 req-581ab0eb-4b46-4d9b-a346-5e9d40746050 service nova] Releasing lock "refresh_cache-1c8b7559-29ab-4982-8c75-ea2a0f0101cb" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:16:12 user nova-compute[75234]: DEBUG nova.network.neutron [None req-a61d564c-9cb3-41fe-ba31-a62940d14628 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] [instance: aa190a9f-ba53-43fb-8bae-2c429d0d1228] Successfully created port: 2cc37e6a-5c9f-4bb8-98fa-f4f2b94d0c35 {{(pid=75234) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:548}}
Nov 29 03:16:12 user nova-compute[75234]: DEBUG nova.network.neutron [None req-a61d564c-9cb3-41fe-ba31-a62940d14628 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] [instance: aa190a9f-ba53-43fb-8bae-2c429d0d1228] Successfully updated port: 2cc37e6a-5c9f-4bb8-98fa-f4f2b94d0c35 {{(pid=75234) _update_port /opt/stack/nova/nova/network/neutron.py:586}}
Nov 29 03:16:12 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-a61d564c-9cb3-41fe-ba31-a62940d14628 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] Acquiring lock "refresh_cache-aa190a9f-ba53-43fb-8bae-2c429d0d1228" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:16:12 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-a61d564c-9cb3-41fe-ba31-a62940d14628 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] Acquired lock "refresh_cache-aa190a9f-ba53-43fb-8bae-2c429d0d1228" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:16:12 user nova-compute[75234]: DEBUG nova.network.neutron [None req-a61d564c-9cb3-41fe-ba31-a62940d14628 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] [instance: aa190a9f-ba53-43fb-8bae-2c429d0d1228] Building network info cache for instance {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2009}}
Nov 29 03:16:12 user nova-compute[75234]: DEBUG nova.compute.manager [req-d37d2f55-7ec0-46fb-a085-57684b2afbb8 req-eaea9a61-870a-490c-915c-606ec1116ddc service nova] [instance: aa190a9f-ba53-43fb-8bae-2c429d0d1228] Received event network-changed-2cc37e6a-5c9f-4bb8-98fa-f4f2b94d0c35 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:16:12 user nova-compute[75234]: DEBUG nova.compute.manager [req-d37d2f55-7ec0-46fb-a085-57684b2afbb8 req-eaea9a61-870a-490c-915c-606ec1116ddc service nova] [instance: aa190a9f-ba53-43fb-8bae-2c429d0d1228] Refreshing instance network info cache due to event network-changed-2cc37e6a-5c9f-4bb8-98fa-f4f2b94d0c35. {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11117}}
Nov 29 03:16:12 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-d37d2f55-7ec0-46fb-a085-57684b2afbb8 req-eaea9a61-870a-490c-915c-606ec1116ddc service nova] Acquiring lock "refresh_cache-aa190a9f-ba53-43fb-8bae-2c429d0d1228" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:16:12 user nova-compute[75234]: DEBUG nova.network.neutron [None req-a61d564c-9cb3-41fe-ba31-a62940d14628 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] [instance: aa190a9f-ba53-43fb-8bae-2c429d0d1228] Instance cache missing network info. {{(pid=75234) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3322}}
Nov 29 03:16:13 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:16:13 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:16:13 user nova-compute[75234]: DEBUG nova.network.neutron [None req-a61d564c-9cb3-41fe-ba31-a62940d14628 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] [instance: aa190a9f-ba53-43fb-8bae-2c429d0d1228] Updating instance_info_cache with network_info: [{"id": "2cc37e6a-5c9f-4bb8-98fa-f4f2b94d0c35", "address": "fa:16:3e:fb:40:18", "network": {"id": "38517dcf-64c7-4d26-9083-5ea5f6c87c2f", "bridge": "br-int", "label": "tempest-ServerActionsV293TestJSON-266920726-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "581a0b62fee744e1b9a767a0d011c4e0", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2cc37e6a-5c", "ovs_interfaceid": "2cc37e6a-5c9f-4bb8-98fa-f4f2b94d0c35", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:16:13 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-a61d564c-9cb3-41fe-ba31-a62940d14628 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] Releasing lock "refresh_cache-aa190a9f-ba53-43fb-8bae-2c429d0d1228" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:16:13 user nova-compute[75234]: DEBUG nova.compute.manager [None req-a61d564c-9cb3-41fe-ba31-a62940d14628 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] [instance: aa190a9f-ba53-43fb-8bae-2c429d0d1228] Instance network_info: |[{"id": "2cc37e6a-5c9f-4bb8-98fa-f4f2b94d0c35", "address": "fa:16:3e:fb:40:18", "network": {"id": "38517dcf-64c7-4d26-9083-5ea5f6c87c2f", "bridge": "br-int", "label": "tempest-ServerActionsV293TestJSON-266920726-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "581a0b62fee744e1b9a767a0d011c4e0", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2cc37e6a-5c", "ovs_interfaceid": "2cc37e6a-5c9f-4bb8-98fa-f4f2b94d0c35", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=75234) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1987}}
Nov 29 03:16:13 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-d37d2f55-7ec0-46fb-a085-57684b2afbb8 req-eaea9a61-870a-490c-915c-606ec1116ddc service nova] Acquired lock "refresh_cache-aa190a9f-ba53-43fb-8bae-2c429d0d1228" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:16:13 user nova-compute[75234]: DEBUG nova.network.neutron [req-d37d2f55-7ec0-46fb-a085-57684b2afbb8 req-eaea9a61-870a-490c-915c-606ec1116ddc service nova] [instance: aa190a9f-ba53-43fb-8bae-2c429d0d1228] Refreshing network info cache for port 2cc37e6a-5c9f-4bb8-98fa-f4f2b94d0c35 {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2006}}
Nov 29 03:16:13 user nova-compute[75234]: DEBUG nova.compute.manager [req-52adc6f5-e6d0-4521-a878-967f25bd60f2 req-5e6d2095-0f5e-4e75-adb5-4af4cc529a4c service nova] [instance: 1c8b7559-29ab-4982-8c75-ea2a0f0101cb] Received event network-vif-plugged-22cdbc69-d4f4-49cd-8104-d508aea94101 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:16:13 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-52adc6f5-e6d0-4521-a878-967f25bd60f2 req-5e6d2095-0f5e-4e75-adb5-4af4cc529a4c service nova] Acquiring lock "1c8b7559-29ab-4982-8c75-ea2a0f0101cb-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:16:13 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-52adc6f5-e6d0-4521-a878-967f25bd60f2 req-5e6d2095-0f5e-4e75-adb5-4af4cc529a4c service nova] Lock "1c8b7559-29ab-4982-8c75-ea2a0f0101cb-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:16:13 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-52adc6f5-e6d0-4521-a878-967f25bd60f2 req-5e6d2095-0f5e-4e75-adb5-4af4cc529a4c service nova] Lock "1c8b7559-29ab-4982-8c75-ea2a0f0101cb-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:16:13 user nova-compute[75234]: DEBUG nova.compute.manager [req-52adc6f5-e6d0-4521-a878-967f25bd60f2 req-5e6d2095-0f5e-4e75-adb5-4af4cc529a4c service nova] [instance: 1c8b7559-29ab-4982-8c75-ea2a0f0101cb] No waiting events found dispatching network-vif-plugged-22cdbc69-d4f4-49cd-8104-d508aea94101 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:16:13 user nova-compute[75234]: WARNING nova.compute.manager [req-52adc6f5-e6d0-4521-a878-967f25bd60f2 req-5e6d2095-0f5e-4e75-adb5-4af4cc529a4c service nova] [instance: 1c8b7559-29ab-4982-8c75-ea2a0f0101cb] Received unexpected event network-vif-plugged-22cdbc69-d4f4-49cd-8104-d508aea94101 for instance with vm_state building and task_state spawning.
Nov 29 03:16:13 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:16:13 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:16:13 user nova-compute[75234]: DEBUG nova.network.neutron [req-d37d2f55-7ec0-46fb-a085-57684b2afbb8 req-eaea9a61-870a-490c-915c-606ec1116ddc service nova] [instance: aa190a9f-ba53-43fb-8bae-2c429d0d1228] Updated VIF entry in instance network info cache for port 2cc37e6a-5c9f-4bb8-98fa-f4f2b94d0c35. {{(pid=75234) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3481}}
Nov 29 03:16:13 user nova-compute[75234]: DEBUG nova.network.neutron [req-d37d2f55-7ec0-46fb-a085-57684b2afbb8 req-eaea9a61-870a-490c-915c-606ec1116ddc service nova] [instance: aa190a9f-ba53-43fb-8bae-2c429d0d1228] Updating instance_info_cache with network_info: [{"id": "2cc37e6a-5c9f-4bb8-98fa-f4f2b94d0c35", "address": "fa:16:3e:fb:40:18", "network": {"id": "38517dcf-64c7-4d26-9083-5ea5f6c87c2f", "bridge": "br-int", "label": "tempest-ServerActionsV293TestJSON-266920726-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "581a0b62fee744e1b9a767a0d011c4e0", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2cc37e6a-5c", "ovs_interfaceid": "2cc37e6a-5c9f-4bb8-98fa-f4f2b94d0c35", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:16:13 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-d37d2f55-7ec0-46fb-a085-57684b2afbb8 req-eaea9a61-870a-490c-915c-606ec1116ddc service nova] Releasing lock "refresh_cache-aa190a9f-ba53-43fb-8bae-2c429d0d1228" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:16:15 user nova-compute[75234]: DEBUG nova.compute.manager [req-b5bd8702-2f0c-4a84-9af4-ae07404c9de4 req-ac43fdf9-4cf1-4bac-9b69-fa3aa3cc2beb service nova] [instance: 1c8b7559-29ab-4982-8c75-ea2a0f0101cb] Received event network-vif-plugged-22cdbc69-d4f4-49cd-8104-d508aea94101 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:16:15 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-b5bd8702-2f0c-4a84-9af4-ae07404c9de4 req-ac43fdf9-4cf1-4bac-9b69-fa3aa3cc2beb service nova] Acquiring lock "1c8b7559-29ab-4982-8c75-ea2a0f0101cb-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:16:15 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-b5bd8702-2f0c-4a84-9af4-ae07404c9de4 req-ac43fdf9-4cf1-4bac-9b69-fa3aa3cc2beb service nova] Lock "1c8b7559-29ab-4982-8c75-ea2a0f0101cb-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:16:15 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-b5bd8702-2f0c-4a84-9af4-ae07404c9de4 req-ac43fdf9-4cf1-4bac-9b69-fa3aa3cc2beb service nova] Lock "1c8b7559-29ab-4982-8c75-ea2a0f0101cb-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:16:15 user nova-compute[75234]: DEBUG nova.compute.manager [req-b5bd8702-2f0c-4a84-9af4-ae07404c9de4 req-ac43fdf9-4cf1-4bac-9b69-fa3aa3cc2beb service nova] [instance: 1c8b7559-29ab-4982-8c75-ea2a0f0101cb] No waiting events found dispatching network-vif-plugged-22cdbc69-d4f4-49cd-8104-d508aea94101 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:16:15 user nova-compute[75234]: WARNING nova.compute.manager [req-b5bd8702-2f0c-4a84-9af4-ae07404c9de4 req-ac43fdf9-4cf1-4bac-9b69-fa3aa3cc2beb service nova] [instance: 1c8b7559-29ab-4982-8c75-ea2a0f0101cb] Received unexpected event network-vif-plugged-22cdbc69-d4f4-49cd-8104-d508aea94101 for instance with vm_state building and task_state spawning.
Nov 29 03:16:15 user nova-compute[75234]: DEBUG nova.compute.manager [None req-a20fe69a-cdbf-482a-8a81-186f342fd171 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] [instance: 1c8b7559-29ab-4982-8c75-ea2a0f0101cb] Instance event wait completed in 0 seconds for  {{(pid=75234) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Nov 29 03:16:15 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-a20fe69a-cdbf-482a-8a81-186f342fd171 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] [instance: 1c8b7559-29ab-4982-8c75-ea2a0f0101cb] Guest created on hypervisor {{(pid=75234) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4421}}
Nov 29 03:16:15 user nova-compute[75234]: INFO nova.virt.libvirt.driver [-] [instance: 1c8b7559-29ab-4982-8c75-ea2a0f0101cb] Instance spawned successfully.
Nov 29 03:16:15 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-a20fe69a-cdbf-482a-8a81-186f342fd171 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] [instance: 1c8b7559-29ab-4982-8c75-ea2a0f0101cb] Attempting to register defaults for the following image properties: ['hw_input_bus', 'hw_pointer_model'] {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:904}}
Nov 29 03:16:15 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-a20fe69a-cdbf-482a-8a81-186f342fd171 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] [instance: 1c8b7559-29ab-4982-8c75-ea2a0f0101cb] Found default for hw_input_bus of None {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:16:15 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-a20fe69a-cdbf-482a-8a81-186f342fd171 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] [instance: 1c8b7559-29ab-4982-8c75-ea2a0f0101cb] Found default for hw_pointer_model of None {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:16:15 user nova-compute[75234]: INFO nova.compute.manager [None req-a20fe69a-cdbf-482a-8a81-186f342fd171 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] [instance: 1c8b7559-29ab-4982-8c75-ea2a0f0101cb] Took 8.33 seconds to spawn the instance on the hypervisor.
Nov 29 03:16:15 user nova-compute[75234]: DEBUG nova.compute.manager [None req-a20fe69a-cdbf-482a-8a81-186f342fd171 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] [instance: 1c8b7559-29ab-4982-8c75-ea2a0f0101cb] Checking state {{(pid=75234) _get_power_state /opt/stack/nova/nova/compute/manager.py:1782}}
Nov 29 03:16:15 user nova-compute[75234]: INFO nova.compute.manager [None req-a20fe69a-cdbf-482a-8a81-186f342fd171 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] [instance: 1c8b7559-29ab-4982-8c75-ea2a0f0101cb] Took 10.05 seconds to build instance.
Nov 29 03:16:15 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-a20fe69a-cdbf-482a-8a81-186f342fd171 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Lock "1c8b7559-29ab-4982-8c75-ea2a0f0101cb" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 10.340s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:16:16 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:16:16 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:16:19 user nova-compute[75234]: DEBUG nova.compute.manager [None req-a61d564c-9cb3-41fe-ba31-a62940d14628 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] [instance: aa190a9f-ba53-43fb-8bae-2c429d0d1228] Start spawning the instance on the hypervisor. {{(pid=75234) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2628}}
Nov 29 03:16:19 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-a61d564c-9cb3-41fe-ba31-a62940d14628 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] [instance: aa190a9f-ba53-43fb-8bae-2c429d0d1228] Creating instance directory {{(pid=75234) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4727}}
Nov 29 03:16:19 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-a61d564c-9cb3-41fe-ba31-a62940d14628 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] [instance: aa190a9f-ba53-43fb-8bae-2c429d0d1228] Creating image(s)
Nov 29 03:16:19 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-a61d564c-9cb3-41fe-ba31-a62940d14628 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] [instance: aa190a9f-ba53-43fb-8bae-2c429d0d1228] Did not create local disks {{(pid=75234) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4863}}
Nov 29 03:16:19 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-a61d564c-9cb3-41fe-ba31-a62940d14628 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] [instance: aa190a9f-ba53-43fb-8bae-2c429d0d1228] Ensure instance console log exists: /opt/stack/data/nova/instances/aa190a9f-ba53-43fb-8bae-2c429d0d1228/console.log {{(pid=75234) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4613}}
Nov 29 03:16:19 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-a61d564c-9cb3-41fe-ba31-a62940d14628 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:16:19 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-a61d564c-9cb3-41fe-ba31-a62940d14628 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:16:19 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-a61d564c-9cb3-41fe-ba31-a62940d14628 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:16:19 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-a61d564c-9cb3-41fe-ba31-a62940d14628 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] [instance: aa190a9f-ba53-43fb-8bae-2c429d0d1228] Start _get_guest_xml network_info=[{"id": "2cc37e6a-5c9f-4bb8-98fa-f4f2b94d0c35", "address": "fa:16:3e:fb:40:18", "network": {"id": "38517dcf-64c7-4d26-9083-5ea5f6c87c2f", "bridge": "br-int", "label": "tempest-ServerActionsV293TestJSON-266920726-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "581a0b62fee744e1b9a767a0d011c4e0", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2cc37e6a-5c", "ovs_interfaceid": "2cc37e6a-5c9f-4bb8-98fa-f4f2b94d0c35", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, '/dev/vda': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum=<?>,container_format=<?>,created_at=<?>,direct_url=<?>,disk_format=<?>,id=<?>,min_disk=0,min_ram=0,name=<?>,owner=<?>,properties=ImageMetaProps,protected=<?>,size=1073741824,status='active',tags=<?>,updated_at=<?>,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [], 'ephemerals': [], 'block_device_mapping': [{'mount_device': '/dev/vda', 'attachment_id': 'adf00bdf-5ffc-418d-b9a1-701ce0a95383', 'disk_bus': 'virtio', 'device_type': 'disk', 'boot_index': 0, 'guest_format': None, 'delete_on_termination': True, 'connection_info': {'driver_volume_type': 'iscsi', 'data': {'target_iqn': 'iqn.2010-10.org.openstack:07f83efc-272d-4e10-9a2b-48d967db613c', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'kRLjYCVO', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}, 'status': 'reserved', 'instance': 'aa190a9f-ba53-43fb-8bae-2c429d0d1228', 'attached_at': '', 'detached_at': '', 'volume_id': '07f83efc-272d-4e10-9a2b-48d967db613c', 'serial': '07f83efc-272d-4e10-9a2b-48d967db613c'}, 'volume_type': None} {{(pid=75234) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7562}}
Nov 29 03:16:19 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-a61d564c-9cb3-41fe-ba31-a62940d14628 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:16:19 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-a61d564c-9cb3-41fe-ba31-a62940d14628 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:16:19 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-a61d564c-9cb3-41fe-ba31-a62940d14628 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] Searching host: 'user' for CPU controller through CGroups V1... {{(pid=75234) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1652}}
Nov 29 03:16:19 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-a61d564c-9cb3-41fe-ba31-a62940d14628 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] CPU controller missing on host. {{(pid=75234) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1662}}
Nov 29 03:16:19 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-a61d564c-9cb3-41fe-ba31-a62940d14628 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] Searching host: 'user' for CPU controller through CGroups V2... {{(pid=75234) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1671}}
Nov 29 03:16:19 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-a61d564c-9cb3-41fe-ba31-a62940d14628 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] CPU controller found on host. {{(pid=75234) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1678}}
Nov 29 03:16:19 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-a61d564c-9cb3-41fe-ba31-a62940d14628 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=75234) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5400}}
Nov 29 03:16:19 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-a61d564c-9cb3-41fe-ba31-a62940d14628 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-11-29T03:02:04Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='1',id=6,is_public=True,memory_mb=512,name='m1.tiny',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum=<?>,container_format=<?>,created_at=<?>,direct_url=<?>,disk_format=<?>,id=<?>,min_disk=0,min_ram=0,name=<?>,owner=<?>,properties=ImageMetaProps,protected=<?>,size=1073741824,status='active',tags=<?>,updated_at=<?>,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:563}}
Nov 29 03:16:19 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-a61d564c-9cb3-41fe-ba31-a62940d14628 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] Flavor limits 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:348}}
Nov 29 03:16:19 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-a61d564c-9cb3-41fe-ba31-a62940d14628 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] Image limits 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:352}}
Nov 29 03:16:19 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-a61d564c-9cb3-41fe-ba31-a62940d14628 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] Flavor pref 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:388}}
Nov 29 03:16:19 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-a61d564c-9cb3-41fe-ba31-a62940d14628 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] Image pref 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:392}}
Nov 29 03:16:19 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-a61d564c-9cb3-41fe-ba31-a62940d14628 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:430}}
Nov 29 03:16:19 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-a61d564c-9cb3-41fe-ba31-a62940d14628 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:569}}
Nov 29 03:16:19 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-a61d564c-9cb3-41fe-ba31-a62940d14628 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=75234) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:471}}
Nov 29 03:16:19 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-a61d564c-9cb3-41fe-ba31-a62940d14628 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] Got 1 possible topologies {{(pid=75234) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:501}}
Nov 29 03:16:19 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-a61d564c-9cb3-41fe-ba31-a62940d14628 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:575}}
Nov 29 03:16:19 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-a61d564c-9cb3-41fe-ba31-a62940d14628 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:577}}
Nov 29 03:16:19 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-a61d564c-9cb3-41fe-ba31-a62940d14628 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] [instance: aa190a9f-ba53-43fb-8bae-2c429d0d1228] Calling os-brick to attach iSCSI Volume {{(pid=75234) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Nov 29 03:16:19 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-a61d564c-9cb3-41fe-ba31-a62940d14628 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f8a5050c9a0>, 'connection_properties': {'target_iqn': 'iqn.2010-10.org.openstack:07f83efc-272d-4e10-9a2b-48d967db613c', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'kRLjYCVO', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:176}}
Nov 29 03:16:19 user nova-compute[75234]: WARNING os_brick.initiator.connectors.base [None req-a61d564c-9cb3-41fe-ba31-a62940d14628 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Nov 29 03:16:19 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-a61d564c-9cb3-41fe-ba31-a62940d14628 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:68}}
Nov 29 03:16:19 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-a61d564c-9cb3-41fe-ba31-a62940d14628 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.002s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:73}}
Nov 29 03:16:19 user nova-compute[75234]: WARNING os_brick.initiator.connectors.base [None req-a61d564c-9cb3-41fe-ba31-a62940d14628 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Nov 29 03:16:19 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-a61d564c-9cb3-41fe-ba31-a62940d14628 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:07f83efc-272d-4e10-9a2b-48d967db613c" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:68}}
Nov 29 03:16:19 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-a61d564c-9cb3-41fe-ba31-a62940d14628 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:07f83efc-272d-4e10-9a2b-48d967db613c" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:73}}
Nov 29 03:16:19 user nova-compute[75234]: INFO os_brick.initiator.connectors.iscsi [None req-a61d564c-9cb3-41fe-ba31-a62940d14628 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Nov 29 03:16:19 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:07f83efc-272d-4e10-9a2b-48d967db613c -p 172.16.0.220:3260 {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:16:19 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:07f83efc-272d-4e10-9a2b-48d967db613c -p 172.16.0.220:3260" returned: 21 in 0.019s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:16:19 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[27ad3545-9734-4904-8e77-969d70a37f6c]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:16:19 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-a61d564c-9cb3-41fe-ba31-a62940d14628 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Nov 29 03:16:19 user nova-compute[75234]:  {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:16:19 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:07f83efc-272d-4e10-9a2b-48d967db613c -p 172.16.0.220:3260 --interface default --op new {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:16:19 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:07f83efc-272d-4e10-9a2b-48d967db613c -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.020s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:16:19 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[f7c9b171-481e-47e1-90a9-96f52b2ccbd1]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:07f83efc-272d-4e10-9a2b-48d967db613c] added\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:16:19 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-a61d564c-9cb3-41fe-ba31-a62940d14628 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:07f83efc-272d-4e10-9a2b-48d967db613c] added
Nov 29 03:16:19 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:16:19 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:07f83efc-272d-4e10-9a2b-48d967db613c -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:16:19 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:07f83efc-272d-4e10-9a2b-48d967db613c -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.020s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:16:19 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[b5480d6f-bd80-4023-9521-40bb2fb8cdc3]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:16:19 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-a61d564c-9cb3-41fe-ba31-a62940d14628 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:16:19 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:07f83efc-272d-4e10-9a2b-48d967db613c -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:16:19 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:07f83efc-272d-4e10-9a2b-48d967db613c -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.030s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:16:19 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[fe1e6ccd-e2bc-4913-bb02-4fd12bc46f1a]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:16:19 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-a61d564c-9cb3-41fe-ba31-a62940d14628 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:16:19 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:07f83efc-272d-4e10-9a2b-48d967db613c -p 172.16.0.220:3260 --op update -n node.session.auth.username -v kRLjYCVO {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:16:19 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:07f83efc-272d-4e10-9a2b-48d967db613c -p 172.16.0.220:3260 --op update -n node.session.auth.username -v kRLjYCVO" returned: 0 in 0.022s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:16:19 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[6c2fe960-6bbb-4f44-aaa4-934c869013f2]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:16:19 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-a61d564c-9cb3-41fe-ba31-a62940d14628 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'kRLjYCVO'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:16:19 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:07f83efc-272d-4e10-9a2b-48d967db613c -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:16:19 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:07f83efc-272d-4e10-9a2b-48d967db613c -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.024s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:16:19 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[0a6bb444-4857-4120-8828-94f4afeacd88]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:16:19 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-a61d564c-9cb3-41fe-ba31-a62940d14628 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:16:19 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:16:19 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 21 in 0.020s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:16:19 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[24784165-2765-47a6-8eb3-742607823b77]: (4, ('', 'iscsiadm: No active sessions.\n')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:16:19 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-a61d564c-9cb3-41fe-ba31-a62940d14628 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] iscsiadm ('-m', 'session'): stdout= stderr=iscsiadm: No active sessions.
Nov 29 03:16:19 user nova-compute[75234]:  {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:16:19 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-a61d564c-9cb3-41fe-ba31-a62940d14628 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] iscsi session list stdout= stderr=iscsiadm: No active sessions.
Nov 29 03:16:19 user nova-compute[75234]:  {{(pid=75234) _run_iscsi_session /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Nov 29 03:16:19 user nova-compute[75234]: WARNING os_brick.initiator.connectors.iscsi [None req-a61d564c-9cb3-41fe-ba31-a62940d14628 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] iscsiadm stderr output when getting sessions: iscsiadm: No active sessions.
Nov 29 03:16:19 user nova-compute[75234]: 
Nov 29 03:16:19 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:07f83efc-272d-4e10-9a2b-48d967db613c -p 172.16.0.220:3260 --login {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:16:19 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:07f83efc-272d-4e10-9a2b-48d967db613c -p 172.16.0.220:3260 --login" returned: 0 in 0.057s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:16:19 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[785fc087-1361-4df6-8e64-96468499e7bc]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:07f83efc-272d-4e10-9a2b-48d967db613c, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:07f83efc-272d-4e10-9a2b-48d967db613c, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:16:19 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-a61d564c-9cb3-41fe-ba31-a62940d14628 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:07f83efc-272d-4e10-9a2b-48d967db613c, portal: 172.16.0.220,3260]
Nov 29 03:16:19 user nova-compute[75234]: Login to [iface: default, target: iqn.2010-10.org.openstack:07f83efc-272d-4e10-9a2b-48d967db613c, portal: 172.16.0.220,3260] successful.
Nov 29 03:16:19 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:16:19 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:07f83efc-272d-4e10-9a2b-48d967db613c -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:16:19 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:07f83efc-272d-4e10-9a2b-48d967db613c -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.027s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:16:19 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[eae760dd-418f-4c6c-9252-a891830eabf7]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:16:19 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-a61d564c-9cb3-41fe-ba31-a62940d14628 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:16:19 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:16:19 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.027s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:16:19 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[4f42b7e8-b626-426a-ba2b-ba5052bd6787]: (4, ('tcp: [27] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:07f83efc-272d-4e10-9a2b-48d967db613c (non-flash)\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:16:19 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-a61d564c-9cb3-41fe-ba31-a62940d14628 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [27] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:07f83efc-272d-4e10-9a2b-48d967db613c (non-flash)
Nov 29 03:16:19 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:16:19 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-a61d564c-9cb3-41fe-ba31-a62940d14628 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] iscsi session list stdout=tcp: [27] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:07f83efc-272d-4e10-9a2b-48d967db613c (non-flash)
Nov 29 03:16:19 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsi_session /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Nov 29 03:16:19 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-a61d564c-9cb3-41fe-ba31-a62940d14628 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:07f83efc-272d-4e10-9a2b-48d967db613c" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.324s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:87}}
Nov 29 03:16:19 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-a61d564c-9cb3-41fe-ba31-a62940d14628 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] Connected to 172.16.0.220:3260 {{(pid=75234) _connect_vol /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:644}}
Nov 29 03:16:19 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-a61d564c-9cb3-41fe-ba31-a62940d14628 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] HCTL ('33', '-', '-', 0) found on session 27 with lun 0 {{(pid=75234) get_hctl /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:770}}
Nov 29 03:16:19 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-a61d564c-9cb3-41fe-ba31-a62940d14628 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] Scanning host 33 c: -, t: -, l: 0) {{(pid=75234) scan_iscsi /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:795}}
Nov 29 03:16:19 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host33/scan {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:16:19 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host33/scan" returned: 0 in 0.023s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:16:19 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[f5febf2b-c70c-4628-b2e3-4d3894861411]: (4, ('- - 0', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:16:19 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-a61d564c-9cb3-41fe-ba31-a62940d14628 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] Searching for a device in session 27 and hctl ['33', '*', '*', 0] yield: sdb {{(pid=75234) device_name_by_hctl /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:789}}
Nov 29 03:16:19 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-a61d564c-9cb3-41fe-ba31-a62940d14628 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] Connected to sdb using {'target_iqn': 'iqn.2010-10.org.openstack:07f83efc-272d-4e10-9a2b-48d967db613c', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'kRLjYCVO', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False} {{(pid=75234) _connect_vol /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:672}}
Nov 29 03:16:20 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-a61d564c-9cb3-41fe-ba31-a62940d14628 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.369s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:87}}
Nov 29 03:16:20 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-a61d564c-9cb3-41fe-ba31-a62940d14628 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] <== connect_volume: return (1371ms) {'type': 'block', 'scsi_wwn': '23066646661346337', 'path': '/dev/sdb'} {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:203}}
Nov 29 03:16:20 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-a61d564c-9cb3-41fe-ba31-a62940d14628 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] [instance: aa190a9f-ba53-43fb-8bae-2c429d0d1228] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '23066646661346337', 'path': '/dev/sdb'} {{(pid=75234) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Nov 29 03:16:20 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-a61d564c-9cb3-41fe-ba31-a62940d14628 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-11-29T03:16:08Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description=None,display_name='tempest-ServerActionsV293TestJSON-server-381391911',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-serveractionsv293testjson-server-381391911',id=27,image_ref='',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBKnl09Ywqr6/srgll0CbbEeIXPcY1dWQkS0gn6SNfJJJd/3IKDSo4FDOq1mKkootcIJYqYPVMtXdp0HTuNblTn7mhW9LnhxK8ZA0M4lAvd/3jzC0f6c+1EiWTGrTU8NBjw==',key_name='tempest-keypair-1391635144',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='581a0b62fee744e1b9a767a0d011c4e0',ramdisk_id='',reservation_id='r-eq00dfx0',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-ServerActionsV293TestJSON-155733519',owner_user_name='tempest-ServerActionsV293TestJSON-155733519-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-11-29T03:16:11Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='d11ce50c391d4e3bbe92d6c463d14746',uuid=aa190a9f-ba53-43fb-8bae-2c429d0d1228,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "2cc37e6a-5c9f-4bb8-98fa-f4f2b94d0c35", "address": "fa:16:3e:fb:40:18", "network": {"id": "38517dcf-64c7-4d26-9083-5ea5f6c87c2f", "bridge": "br-int", "label": "tempest-ServerActionsV293TestJSON-266920726-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "581a0b62fee744e1b9a767a0d011c4e0", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2cc37e6a-5c", "ovs_interfaceid": "2cc37e6a-5c9f-4bb8-98fa-f4f2b94d0c35", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=75234) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Nov 29 03:16:20 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-a61d564c-9cb3-41fe-ba31-a62940d14628 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] Converting VIF {"id": "2cc37e6a-5c9f-4bb8-98fa-f4f2b94d0c35", "address": "fa:16:3e:fb:40:18", "network": {"id": "38517dcf-64c7-4d26-9083-5ea5f6c87c2f", "bridge": "br-int", "label": "tempest-ServerActionsV293TestJSON-266920726-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "581a0b62fee744e1b9a767a0d011c4e0", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2cc37e6a-5c", "ovs_interfaceid": "2cc37e6a-5c9f-4bb8-98fa-f4f2b94d0c35", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:16:20 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-a61d564c-9cb3-41fe-ba31-a62940d14628 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:fb:40:18,bridge_name='br-int',has_traffic_filtering=True,id=2cc37e6a-5c9f-4bb8-98fa-f4f2b94d0c35,network=Network(38517dcf-64c7-4d26-9083-5ea5f6c87c2f),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap2cc37e6a-5c') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:16:20 user nova-compute[75234]: DEBUG nova.objects.instance [None req-a61d564c-9cb3-41fe-ba31-a62940d14628 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] Lazy-loading 'pci_devices' on Instance uuid aa190a9f-ba53-43fb-8bae-2c429d0d1228 {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:16:20 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-a61d564c-9cb3-41fe-ba31-a62940d14628 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] [instance: aa190a9f-ba53-43fb-8bae-2c429d0d1228] End _get_guest_xml xml=<domain type="kvm">
Nov 29 03:16:20 user nova-compute[75234]:   <uuid>aa190a9f-ba53-43fb-8bae-2c429d0d1228</uuid>
Nov 29 03:16:20 user nova-compute[75234]:   <name>instance-0000001b</name>
Nov 29 03:16:20 user nova-compute[75234]:   <memory>524288</memory>
Nov 29 03:16:20 user nova-compute[75234]:   <vcpu>1</vcpu>
Nov 29 03:16:20 user nova-compute[75234]:   <metadata>
Nov 29 03:16:20 user nova-compute[75234]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Nov 29 03:16:20 user nova-compute[75234]:       <nova:package version="0.0.0"/>
Nov 29 03:16:20 user nova-compute[75234]:       <nova:name>tempest-ServerActionsV293TestJSON-server-381391911</nova:name>
Nov 29 03:16:20 user nova-compute[75234]:       <nova:creationTime>2023-11-29 03:16:19</nova:creationTime>
Nov 29 03:16:20 user nova-compute[75234]:       <nova:flavor name="m1.tiny">
Nov 29 03:16:20 user nova-compute[75234]:         <nova:memory>512</nova:memory>
Nov 29 03:16:20 user nova-compute[75234]:         <nova:disk>1</nova:disk>
Nov 29 03:16:20 user nova-compute[75234]:         <nova:swap>0</nova:swap>
Nov 29 03:16:20 user nova-compute[75234]:         <nova:ephemeral>0</nova:ephemeral>
Nov 29 03:16:20 user nova-compute[75234]:         <nova:vcpus>1</nova:vcpus>
Nov 29 03:16:20 user nova-compute[75234]:       </nova:flavor>
Nov 29 03:16:20 user nova-compute[75234]:       <nova:owner>
Nov 29 03:16:20 user nova-compute[75234]:         <nova:user uuid="d11ce50c391d4e3bbe92d6c463d14746">tempest-ServerActionsV293TestJSON-155733519-project-member</nova:user>
Nov 29 03:16:20 user nova-compute[75234]:         <nova:project uuid="581a0b62fee744e1b9a767a0d011c4e0">tempest-ServerActionsV293TestJSON-155733519</nova:project>
Nov 29 03:16:20 user nova-compute[75234]:       </nova:owner>
Nov 29 03:16:20 user nova-compute[75234]:       <nova:ports>
Nov 29 03:16:20 user nova-compute[75234]:         <nova:port uuid="2cc37e6a-5c9f-4bb8-98fa-f4f2b94d0c35">
Nov 29 03:16:20 user nova-compute[75234]:           <nova:ip type="fixed" address="10.0.0.6" ipVersion="4"/>
Nov 29 03:16:20 user nova-compute[75234]:         </nova:port>
Nov 29 03:16:20 user nova-compute[75234]:       </nova:ports>
Nov 29 03:16:20 user nova-compute[75234]:     </nova:instance>
Nov 29 03:16:20 user nova-compute[75234]:   </metadata>
Nov 29 03:16:20 user nova-compute[75234]:   <sysinfo type="smbios">
Nov 29 03:16:20 user nova-compute[75234]:     <system>
Nov 29 03:16:20 user nova-compute[75234]:       <entry name="manufacturer">OpenStack Foundation</entry>
Nov 29 03:16:20 user nova-compute[75234]:       <entry name="product">OpenStack Nova</entry>
Nov 29 03:16:20 user nova-compute[75234]:       <entry name="version">0.0.0</entry>
Nov 29 03:16:20 user nova-compute[75234]:       <entry name="serial">aa190a9f-ba53-43fb-8bae-2c429d0d1228</entry>
Nov 29 03:16:20 user nova-compute[75234]:       <entry name="uuid">aa190a9f-ba53-43fb-8bae-2c429d0d1228</entry>
Nov 29 03:16:20 user nova-compute[75234]:       <entry name="family">Virtual Machine</entry>
Nov 29 03:16:20 user nova-compute[75234]:     </system>
Nov 29 03:16:20 user nova-compute[75234]:   </sysinfo>
Nov 29 03:16:20 user nova-compute[75234]:   <os>
Nov 29 03:16:20 user nova-compute[75234]:     <type arch="x86_64" machine="pc">hvm</type>
Nov 29 03:16:20 user nova-compute[75234]:     <boot dev="hd"/>
Nov 29 03:16:20 user nova-compute[75234]:     <smbios mode="sysinfo"/>
Nov 29 03:16:20 user nova-compute[75234]:   </os>
Nov 29 03:16:20 user nova-compute[75234]:   <features>
Nov 29 03:16:20 user nova-compute[75234]:     <acpi/>
Nov 29 03:16:20 user nova-compute[75234]:     <apic/>
Nov 29 03:16:20 user nova-compute[75234]:     <vmcoreinfo/>
Nov 29 03:16:20 user nova-compute[75234]:   </features>
Nov 29 03:16:20 user nova-compute[75234]:   <clock offset="utc">
Nov 29 03:16:20 user nova-compute[75234]:     <timer name="pit" tickpolicy="delay"/>
Nov 29 03:16:20 user nova-compute[75234]:     <timer name="rtc" tickpolicy="catchup"/>
Nov 29 03:16:20 user nova-compute[75234]:     <timer name="hpet" present="no"/>
Nov 29 03:16:20 user nova-compute[75234]:   </clock>
Nov 29 03:16:20 user nova-compute[75234]:   <cpu mode="custom" match="exact">
Nov 29 03:16:20 user nova-compute[75234]:     <model>Nehalem</model>
Nov 29 03:16:20 user nova-compute[75234]:     <topology sockets="1" cores="1" threads="1"/>
Nov 29 03:16:20 user nova-compute[75234]:   </cpu>
Nov 29 03:16:20 user nova-compute[75234]:   <devices>
Nov 29 03:16:20 user nova-compute[75234]:     <disk type="block" device="disk">
Nov 29 03:16:20 user nova-compute[75234]:       <driver name="qemu" type="raw" cache="none" io="native"/>
Nov 29 03:16:20 user nova-compute[75234]:       <source dev="/dev/sdb"/>
Nov 29 03:16:20 user nova-compute[75234]:       <target dev="vda" bus="virtio"/>
Nov 29 03:16:20 user nova-compute[75234]:       <serial>07f83efc-272d-4e10-9a2b-48d967db613c</serial>
Nov 29 03:16:20 user nova-compute[75234]:     </disk>
Nov 29 03:16:20 user nova-compute[75234]:     <interface type="ethernet">
Nov 29 03:16:20 user nova-compute[75234]:       <mac address="fa:16:3e:fb:40:18"/>
Nov 29 03:16:20 user nova-compute[75234]:       <model type="virtio"/>
Nov 29 03:16:20 user nova-compute[75234]:       <mtu size="1442"/>
Nov 29 03:16:20 user nova-compute[75234]:       <target dev="tap2cc37e6a-5c"/>
Nov 29 03:16:20 user nova-compute[75234]:     </interface>
Nov 29 03:16:20 user nova-compute[75234]:     <serial type="pty">
Nov 29 03:16:20 user nova-compute[75234]:       <log file="/opt/stack/data/nova/instances/aa190a9f-ba53-43fb-8bae-2c429d0d1228/console.log" append="off"/>
Nov 29 03:16:20 user nova-compute[75234]:     </serial>
Nov 29 03:16:20 user nova-compute[75234]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Nov 29 03:16:20 user nova-compute[75234]:     <video>
Nov 29 03:16:20 user nova-compute[75234]:       <model type="virtio"/>
Nov 29 03:16:20 user nova-compute[75234]:     </video>
Nov 29 03:16:20 user nova-compute[75234]:     <rng model="virtio">
Nov 29 03:16:20 user nova-compute[75234]:       <backend model="random">/dev/urandom</backend>
Nov 29 03:16:20 user nova-compute[75234]:     </rng>
Nov 29 03:16:20 user nova-compute[75234]:     <controller type="usb" index="0" model="none"/>
Nov 29 03:16:20 user nova-compute[75234]:     <memballoon model="virtio">
Nov 29 03:16:20 user nova-compute[75234]:       <stats period="10"/>
Nov 29 03:16:20 user nova-compute[75234]:     </memballoon>
Nov 29 03:16:20 user nova-compute[75234]:   </devices>
Nov 29 03:16:20 user nova-compute[75234]: </domain>
Nov 29 03:16:20 user nova-compute[75234]:  {{(pid=75234) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7568}}
Nov 29 03:16:20 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-a61d564c-9cb3-41fe-ba31-a62940d14628 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-11-29T03:16:08Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description=None,display_name='tempest-ServerActionsV293TestJSON-server-381391911',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-serveractionsv293testjson-server-381391911',id=27,image_ref='',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBKnl09Ywqr6/srgll0CbbEeIXPcY1dWQkS0gn6SNfJJJd/3IKDSo4FDOq1mKkootcIJYqYPVMtXdp0HTuNblTn7mhW9LnhxK8ZA0M4lAvd/3jzC0f6c+1EiWTGrTU8NBjw==',key_name='tempest-keypair-1391635144',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='581a0b62fee744e1b9a767a0d011c4e0',ramdisk_id='',reservation_id='r-eq00dfx0',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-ServerActionsV293TestJSON-155733519',owner_user_name='tempest-ServerActionsV293TestJSON-155733519-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-11-29T03:16:11Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='d11ce50c391d4e3bbe92d6c463d14746',uuid=aa190a9f-ba53-43fb-8bae-2c429d0d1228,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "2cc37e6a-5c9f-4bb8-98fa-f4f2b94d0c35", "address": "fa:16:3e:fb:40:18", "network": {"id": "38517dcf-64c7-4d26-9083-5ea5f6c87c2f", "bridge": "br-int", "label": "tempest-ServerActionsV293TestJSON-266920726-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "581a0b62fee744e1b9a767a0d011c4e0", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2cc37e6a-5c", "ovs_interfaceid": "2cc37e6a-5c9f-4bb8-98fa-f4f2b94d0c35", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Nov 29 03:16:20 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-a61d564c-9cb3-41fe-ba31-a62940d14628 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] Converting VIF {"id": "2cc37e6a-5c9f-4bb8-98fa-f4f2b94d0c35", "address": "fa:16:3e:fb:40:18", "network": {"id": "38517dcf-64c7-4d26-9083-5ea5f6c87c2f", "bridge": "br-int", "label": "tempest-ServerActionsV293TestJSON-266920726-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "581a0b62fee744e1b9a767a0d011c4e0", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2cc37e6a-5c", "ovs_interfaceid": "2cc37e6a-5c9f-4bb8-98fa-f4f2b94d0c35", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:16:20 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-a61d564c-9cb3-41fe-ba31-a62940d14628 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:fb:40:18,bridge_name='br-int',has_traffic_filtering=True,id=2cc37e6a-5c9f-4bb8-98fa-f4f2b94d0c35,network=Network(38517dcf-64c7-4d26-9083-5ea5f6c87c2f),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap2cc37e6a-5c') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:16:20 user nova-compute[75234]: DEBUG os_vif [None req-a61d564c-9cb3-41fe-ba31-a62940d14628 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:fb:40:18,bridge_name='br-int',has_traffic_filtering=True,id=2cc37e6a-5c9f-4bb8-98fa-f4f2b94d0c35,network=Network(38517dcf-64c7-4d26-9083-5ea5f6c87c2f),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap2cc37e6a-5c') {{(pid=75234) plug /opt/stack/data/venv/lib/python3.10/site-packages/os_vif/__init__.py:76}}
Nov 29 03:16:20 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:16:20 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:16:20 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Nov 29 03:16:20 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:16:20 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbCreateCommand(_result=None, table=QoS, columns={'type': 'linux-noop', 'external_ids': {'id': 'de15799a-23e9-5641-8ca0-ba2dd0e0ce5f', '_type': 'linux-noop'}}, row=False) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:16:20 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:16:20 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:16:20 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:16:20 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tap2cc37e6a-5c, may_exist=True, interface_attrs={}) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:16:20 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Port, record=tap2cc37e6a-5c, col_values=(('qos', UUID('667824d2-2ccc-4f1b-8574-9d0ff0febfd6')),), if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:16:20 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DbSetCommand(_result=None, table=Interface, record=tap2cc37e6a-5c, col_values=(('external_ids', {'iface-id': '2cc37e6a-5c9f-4bb8-98fa-f4f2b94d0c35', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:fb:40:18', 'vm-uuid': 'aa190a9f-ba53-43fb-8bae-2c429d0d1228'}),), if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:16:20 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:16:20 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:16:20 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:16:20 user nova-compute[75234]: INFO os_vif [None req-a61d564c-9cb3-41fe-ba31-a62940d14628 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:fb:40:18,bridge_name='br-int',has_traffic_filtering=True,id=2cc37e6a-5c9f-4bb8-98fa-f4f2b94d0c35,network=Network(38517dcf-64c7-4d26-9083-5ea5f6c87c2f),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap2cc37e6a-5c')
Nov 29 03:16:20 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-a61d564c-9cb3-41fe-ba31-a62940d14628 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] No BDM found with device name vda, not building metadata. {{(pid=75234) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12089}}
Nov 29 03:16:20 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-a61d564c-9cb3-41fe-ba31-a62940d14628 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] No VIF found with MAC fa:16:3e:fb:40:18, not building metadata {{(pid=75234) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Nov 29 03:16:21 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:16:22 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:16:22 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:16:22 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:16:22 user nova-compute[75234]: DEBUG nova.compute.manager [req-21cdd0cf-56f5-4a36-a78c-0a6a85a6e50d req-dce0678e-9941-4fac-9ead-d3523a35a09e service nova] [instance: aa190a9f-ba53-43fb-8bae-2c429d0d1228] Received event network-vif-plugged-2cc37e6a-5c9f-4bb8-98fa-f4f2b94d0c35 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:16:22 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-21cdd0cf-56f5-4a36-a78c-0a6a85a6e50d req-dce0678e-9941-4fac-9ead-d3523a35a09e service nova] Acquiring lock "aa190a9f-ba53-43fb-8bae-2c429d0d1228-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:16:22 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-21cdd0cf-56f5-4a36-a78c-0a6a85a6e50d req-dce0678e-9941-4fac-9ead-d3523a35a09e service nova] Lock "aa190a9f-ba53-43fb-8bae-2c429d0d1228-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:16:22 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-21cdd0cf-56f5-4a36-a78c-0a6a85a6e50d req-dce0678e-9941-4fac-9ead-d3523a35a09e service nova] Lock "aa190a9f-ba53-43fb-8bae-2c429d0d1228-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:16:22 user nova-compute[75234]: DEBUG nova.compute.manager [req-21cdd0cf-56f5-4a36-a78c-0a6a85a6e50d req-dce0678e-9941-4fac-9ead-d3523a35a09e service nova] [instance: aa190a9f-ba53-43fb-8bae-2c429d0d1228] No waiting events found dispatching network-vif-plugged-2cc37e6a-5c9f-4bb8-98fa-f4f2b94d0c35 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:16:22 user nova-compute[75234]: WARNING nova.compute.manager [req-21cdd0cf-56f5-4a36-a78c-0a6a85a6e50d req-dce0678e-9941-4fac-9ead-d3523a35a09e service nova] [instance: aa190a9f-ba53-43fb-8bae-2c429d0d1228] Received unexpected event network-vif-plugged-2cc37e6a-5c9f-4bb8-98fa-f4f2b94d0c35 for instance with vm_state building and task_state spawning.
Nov 29 03:16:23 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:16:23 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:16:24 user nova-compute[75234]: DEBUG nova.compute.manager [req-63e9741d-17b5-4a92-ad84-021ba133f4a9 req-c3ae91d9-28db-460f-969d-8ed9c0290171 service nova] [instance: aa190a9f-ba53-43fb-8bae-2c429d0d1228] Received event network-vif-plugged-2cc37e6a-5c9f-4bb8-98fa-f4f2b94d0c35 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:16:24 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-63e9741d-17b5-4a92-ad84-021ba133f4a9 req-c3ae91d9-28db-460f-969d-8ed9c0290171 service nova] Acquiring lock "aa190a9f-ba53-43fb-8bae-2c429d0d1228-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:16:24 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-63e9741d-17b5-4a92-ad84-021ba133f4a9 req-c3ae91d9-28db-460f-969d-8ed9c0290171 service nova] Lock "aa190a9f-ba53-43fb-8bae-2c429d0d1228-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:16:24 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-63e9741d-17b5-4a92-ad84-021ba133f4a9 req-c3ae91d9-28db-460f-969d-8ed9c0290171 service nova] Lock "aa190a9f-ba53-43fb-8bae-2c429d0d1228-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:16:24 user nova-compute[75234]: DEBUG nova.compute.manager [req-63e9741d-17b5-4a92-ad84-021ba133f4a9 req-c3ae91d9-28db-460f-969d-8ed9c0290171 service nova] [instance: aa190a9f-ba53-43fb-8bae-2c429d0d1228] No waiting events found dispatching network-vif-plugged-2cc37e6a-5c9f-4bb8-98fa-f4f2b94d0c35 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:16:24 user nova-compute[75234]: WARNING nova.compute.manager [req-63e9741d-17b5-4a92-ad84-021ba133f4a9 req-c3ae91d9-28db-460f-969d-8ed9c0290171 service nova] [instance: aa190a9f-ba53-43fb-8bae-2c429d0d1228] Received unexpected event network-vif-plugged-2cc37e6a-5c9f-4bb8-98fa-f4f2b94d0c35 for instance with vm_state building and task_state spawning.
Nov 29 03:16:24 user nova-compute[75234]: DEBUG nova.compute.manager [None req-a61d564c-9cb3-41fe-ba31-a62940d14628 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] [instance: aa190a9f-ba53-43fb-8bae-2c429d0d1228] Instance event wait completed in 0 seconds for  {{(pid=75234) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Nov 29 03:16:24 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-a61d564c-9cb3-41fe-ba31-a62940d14628 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] [instance: aa190a9f-ba53-43fb-8bae-2c429d0d1228] Guest created on hypervisor {{(pid=75234) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4421}}
Nov 29 03:16:24 user nova-compute[75234]: INFO nova.virt.libvirt.driver [-] [instance: aa190a9f-ba53-43fb-8bae-2c429d0d1228] Instance spawned successfully.
Nov 29 03:16:24 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-a61d564c-9cb3-41fe-ba31-a62940d14628 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] [instance: aa190a9f-ba53-43fb-8bae-2c429d0d1228] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:904}}
Nov 29 03:16:24 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-a61d564c-9cb3-41fe-ba31-a62940d14628 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] [instance: aa190a9f-ba53-43fb-8bae-2c429d0d1228] Found default for hw_cdrom_bus of ide {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:16:24 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-a61d564c-9cb3-41fe-ba31-a62940d14628 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] [instance: aa190a9f-ba53-43fb-8bae-2c429d0d1228] Found default for hw_disk_bus of virtio {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:16:24 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-a61d564c-9cb3-41fe-ba31-a62940d14628 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] [instance: aa190a9f-ba53-43fb-8bae-2c429d0d1228] Found default for hw_input_bus of None {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:16:24 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-a61d564c-9cb3-41fe-ba31-a62940d14628 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] [instance: aa190a9f-ba53-43fb-8bae-2c429d0d1228] Found default for hw_pointer_model of None {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:16:24 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-a61d564c-9cb3-41fe-ba31-a62940d14628 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] [instance: aa190a9f-ba53-43fb-8bae-2c429d0d1228] Found default for hw_video_model of virtio {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:16:24 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-a61d564c-9cb3-41fe-ba31-a62940d14628 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] [instance: aa190a9f-ba53-43fb-8bae-2c429d0d1228] Found default for hw_vif_model of virtio {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:16:24 user nova-compute[75234]: INFO nova.compute.manager [None req-a61d564c-9cb3-41fe-ba31-a62940d14628 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] [instance: aa190a9f-ba53-43fb-8bae-2c429d0d1228] Took 5.57 seconds to spawn the instance on the hypervisor.
Nov 29 03:16:24 user nova-compute[75234]: DEBUG nova.compute.manager [None req-a61d564c-9cb3-41fe-ba31-a62940d14628 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] [instance: aa190a9f-ba53-43fb-8bae-2c429d0d1228] Checking state {{(pid=75234) _get_power_state /opt/stack/nova/nova/compute/manager.py:1782}}
Nov 29 03:16:25 user nova-compute[75234]: INFO nova.compute.manager [None req-a61d564c-9cb3-41fe-ba31-a62940d14628 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] [instance: aa190a9f-ba53-43fb-8bae-2c429d0d1228] Took 15.08 seconds to build instance.
Nov 29 03:16:25 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-a61d564c-9cb3-41fe-ba31-a62940d14628 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] Lock "aa190a9f-ba53-43fb-8bae-2c429d0d1228" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 15.220s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:16:25 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:16:26 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:16:30 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:16:31 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:16:32 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:16:35 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-f3b0ea6e-b40c-4205-a67d-a35d8dd70426 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Acquiring lock "99d6f85f-1b77-4390-8b61-908c98f584c0" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:16:35 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-f3b0ea6e-b40c-4205-a67d-a35d8dd70426 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Lock "99d6f85f-1b77-4390-8b61-908c98f584c0" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:16:35 user nova-compute[75234]: DEBUG nova.compute.manager [None req-f3b0ea6e-b40c-4205-a67d-a35d8dd70426 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] [instance: 99d6f85f-1b77-4390-8b61-908c98f584c0] Starting instance... {{(pid=75234) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2422}}
Nov 29 03:16:35 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-f3b0ea6e-b40c-4205-a67d-a35d8dd70426 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:16:35 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-f3b0ea6e-b40c-4205-a67d-a35d8dd70426 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:16:35 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-f3b0ea6e-b40c-4205-a67d-a35d8dd70426 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=75234) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2366}}
Nov 29 03:16:35 user nova-compute[75234]: INFO nova.compute.claims [None req-f3b0ea6e-b40c-4205-a67d-a35d8dd70426 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] [instance: 99d6f85f-1b77-4390-8b61-908c98f584c0] Claim successful on node user
Nov 29 03:16:35 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:16:36 user nova-compute[75234]: DEBUG nova.compute.provider_tree [None req-f3b0ea6e-b40c-4205-a67d-a35d8dd70426 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Inventory has not changed in ProviderTree for provider: 4d291220-7c37-4839-a0e3-b23c66502b74 {{(pid=75234) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Nov 29 03:16:36 user nova-compute[75234]: DEBUG nova.scheduler.client.report [None req-f3b0ea6e-b40c-4205-a67d-a35d8dd70426 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Inventory has not changed for provider 4d291220-7c37-4839-a0e3-b23c66502b74 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=75234) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Nov 29 03:16:36 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-f3b0ea6e-b40c-4205-a67d-a35d8dd70426 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.408s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:16:36 user nova-compute[75234]: DEBUG nova.compute.manager [None req-f3b0ea6e-b40c-4205-a67d-a35d8dd70426 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] [instance: 99d6f85f-1b77-4390-8b61-908c98f584c0] Start building networks asynchronously for instance. {{(pid=75234) _build_resources /opt/stack/nova/nova/compute/manager.py:2819}}
Nov 29 03:16:36 user nova-compute[75234]: DEBUG nova.compute.manager [None req-f3b0ea6e-b40c-4205-a67d-a35d8dd70426 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] [instance: 99d6f85f-1b77-4390-8b61-908c98f584c0] Allocating IP information in the background. {{(pid=75234) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1972}}
Nov 29 03:16:36 user nova-compute[75234]: DEBUG nova.network.neutron [None req-f3b0ea6e-b40c-4205-a67d-a35d8dd70426 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] [instance: 99d6f85f-1b77-4390-8b61-908c98f584c0] allocate_for_instance() {{(pid=75234) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1156}}
Nov 29 03:16:36 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-f3b0ea6e-b40c-4205-a67d-a35d8dd70426 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] [instance: 99d6f85f-1b77-4390-8b61-908c98f584c0] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Nov 29 03:16:36 user nova-compute[75234]: DEBUG nova.compute.manager [None req-f3b0ea6e-b40c-4205-a67d-a35d8dd70426 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] [instance: 99d6f85f-1b77-4390-8b61-908c98f584c0] Start building block device mappings for instance. {{(pid=75234) _build_resources /opt/stack/nova/nova/compute/manager.py:2854}}
Nov 29 03:16:36 user nova-compute[75234]: DEBUG nova.policy [None req-f3b0ea6e-b40c-4205-a67d-a35d8dd70426 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '154f3f4f57514450baf8f22d67768bc9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e17150005a9a449f9373b91e1c00083b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=75234) authorize /opt/stack/nova/nova/policy.py:203}}
Nov 29 03:16:36 user nova-compute[75234]: DEBUG nova.compute.manager [None req-f3b0ea6e-b40c-4205-a67d-a35d8dd70426 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] [instance: 99d6f85f-1b77-4390-8b61-908c98f584c0] Start spawning the instance on the hypervisor. {{(pid=75234) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2628}}
Nov 29 03:16:36 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-f3b0ea6e-b40c-4205-a67d-a35d8dd70426 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] [instance: 99d6f85f-1b77-4390-8b61-908c98f584c0] Creating instance directory {{(pid=75234) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4727}}
Nov 29 03:16:36 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-f3b0ea6e-b40c-4205-a67d-a35d8dd70426 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] [instance: 99d6f85f-1b77-4390-8b61-908c98f584c0] Creating image(s)
Nov 29 03:16:36 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-f3b0ea6e-b40c-4205-a67d-a35d8dd70426 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Acquiring lock "/opt/stack/data/nova/instances/99d6f85f-1b77-4390-8b61-908c98f584c0/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:16:36 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-f3b0ea6e-b40c-4205-a67d-a35d8dd70426 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Lock "/opt/stack/data/nova/instances/99d6f85f-1b77-4390-8b61-908c98f584c0/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:16:36 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-f3b0ea6e-b40c-4205-a67d-a35d8dd70426 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Lock "/opt/stack/data/nova/instances/99d6f85f-1b77-4390-8b61-908c98f584c0/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.013s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:16:36 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-f3b0ea6e-b40c-4205-a67d-a35d8dd70426 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:16:36 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-f3b0ea6e-b40c-4205-a67d-a35d8dd70426 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json" returned: 0 in 0.188s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:16:36 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-f3b0ea6e-b40c-4205-a67d-a35d8dd70426 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Acquiring lock "80686e29dc9db6600d03e85cb62af298e12d7918" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:16:36 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-f3b0ea6e-b40c-4205-a67d-a35d8dd70426 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Lock "80686e29dc9db6600d03e85cb62af298e12d7918" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.002s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:16:36 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-f3b0ea6e-b40c-4205-a67d-a35d8dd70426 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:16:36 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:16:36 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-f3b0ea6e-b40c-4205-a67d-a35d8dd70426 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json" returned: 0 in 0.190s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:16:36 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-f3b0ea6e-b40c-4205-a67d-a35d8dd70426 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918,backing_fmt=raw /opt/stack/data/nova/instances/99d6f85f-1b77-4390-8b61-908c98f584c0/disk 1073741824 {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:16:37 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-f3b0ea6e-b40c-4205-a67d-a35d8dd70426 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918,backing_fmt=raw /opt/stack/data/nova/instances/99d6f85f-1b77-4390-8b61-908c98f584c0/disk 1073741824" returned: 0 in 0.071s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:16:37 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-f3b0ea6e-b40c-4205-a67d-a35d8dd70426 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Lock "80686e29dc9db6600d03e85cb62af298e12d7918" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.270s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:16:37 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-f3b0ea6e-b40c-4205-a67d-a35d8dd70426 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:16:37 user nova-compute[75234]: DEBUG nova.network.neutron [None req-f3b0ea6e-b40c-4205-a67d-a35d8dd70426 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] [instance: 99d6f85f-1b77-4390-8b61-908c98f584c0] Successfully created port: b68d722f-9020-4da4-8cdb-e2312ef06df9 {{(pid=75234) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:548}}
Nov 29 03:16:37 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-f3b0ea6e-b40c-4205-a67d-a35d8dd70426 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json" returned: 0 in 0.190s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:16:37 user nova-compute[75234]: DEBUG nova.virt.disk.api [None req-f3b0ea6e-b40c-4205-a67d-a35d8dd70426 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Checking if we can resize image /opt/stack/data/nova/instances/99d6f85f-1b77-4390-8b61-908c98f584c0/disk. size=1073741824 {{(pid=75234) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:166}}
Nov 29 03:16:37 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-f3b0ea6e-b40c-4205-a67d-a35d8dd70426 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/99d6f85f-1b77-4390-8b61-908c98f584c0/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:16:37 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-f3b0ea6e-b40c-4205-a67d-a35d8dd70426 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/99d6f85f-1b77-4390-8b61-908c98f584c0/disk --force-share --output=json" returned: 0 in 0.204s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:16:37 user nova-compute[75234]: DEBUG nova.virt.disk.api [None req-f3b0ea6e-b40c-4205-a67d-a35d8dd70426 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Cannot resize image /opt/stack/data/nova/instances/99d6f85f-1b77-4390-8b61-908c98f584c0/disk to a smaller size. {{(pid=75234) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:172}}
Nov 29 03:16:37 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-f3b0ea6e-b40c-4205-a67d-a35d8dd70426 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] [instance: 99d6f85f-1b77-4390-8b61-908c98f584c0] Created local disks {{(pid=75234) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4861}}
Nov 29 03:16:37 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-f3b0ea6e-b40c-4205-a67d-a35d8dd70426 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] [instance: 99d6f85f-1b77-4390-8b61-908c98f584c0] Ensure instance console log exists: /opt/stack/data/nova/instances/99d6f85f-1b77-4390-8b61-908c98f584c0/console.log {{(pid=75234) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4613}}
Nov 29 03:16:37 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-f3b0ea6e-b40c-4205-a67d-a35d8dd70426 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:16:37 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-f3b0ea6e-b40c-4205-a67d-a35d8dd70426 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:16:37 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-f3b0ea6e-b40c-4205-a67d-a35d8dd70426 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:16:37 user nova-compute[75234]: DEBUG nova.network.neutron [None req-f3b0ea6e-b40c-4205-a67d-a35d8dd70426 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] [instance: 99d6f85f-1b77-4390-8b61-908c98f584c0] Successfully updated port: b68d722f-9020-4da4-8cdb-e2312ef06df9 {{(pid=75234) _update_port /opt/stack/nova/nova/network/neutron.py:586}}
Nov 29 03:16:37 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-f3b0ea6e-b40c-4205-a67d-a35d8dd70426 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Acquiring lock "refresh_cache-99d6f85f-1b77-4390-8b61-908c98f584c0" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:16:37 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-f3b0ea6e-b40c-4205-a67d-a35d8dd70426 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Acquired lock "refresh_cache-99d6f85f-1b77-4390-8b61-908c98f584c0" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:16:37 user nova-compute[75234]: DEBUG nova.network.neutron [None req-f3b0ea6e-b40c-4205-a67d-a35d8dd70426 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] [instance: 99d6f85f-1b77-4390-8b61-908c98f584c0] Building network info cache for instance {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2009}}
Nov 29 03:16:37 user nova-compute[75234]: DEBUG nova.compute.manager [req-f589c615-7f8b-42cf-ab2d-d6da0b6f6fac req-a0710fcf-94c6-4f7a-a2eb-2318371cebaf service nova] [instance: 99d6f85f-1b77-4390-8b61-908c98f584c0] Received event network-changed-b68d722f-9020-4da4-8cdb-e2312ef06df9 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:16:37 user nova-compute[75234]: DEBUG nova.compute.manager [req-f589c615-7f8b-42cf-ab2d-d6da0b6f6fac req-a0710fcf-94c6-4f7a-a2eb-2318371cebaf service nova] [instance: 99d6f85f-1b77-4390-8b61-908c98f584c0] Refreshing instance network info cache due to event network-changed-b68d722f-9020-4da4-8cdb-e2312ef06df9. {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11117}}
Nov 29 03:16:37 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-f589c615-7f8b-42cf-ab2d-d6da0b6f6fac req-a0710fcf-94c6-4f7a-a2eb-2318371cebaf service nova] Acquiring lock "refresh_cache-99d6f85f-1b77-4390-8b61-908c98f584c0" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:16:38 user nova-compute[75234]: DEBUG nova.network.neutron [None req-f3b0ea6e-b40c-4205-a67d-a35d8dd70426 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] [instance: 99d6f85f-1b77-4390-8b61-908c98f584c0] Instance cache missing network info. {{(pid=75234) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3322}}
Nov 29 03:16:38 user nova-compute[75234]: DEBUG nova.network.neutron [None req-f3b0ea6e-b40c-4205-a67d-a35d8dd70426 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] [instance: 99d6f85f-1b77-4390-8b61-908c98f584c0] Updating instance_info_cache with network_info: [{"id": "b68d722f-9020-4da4-8cdb-e2312ef06df9", "address": "fa:16:3e:1e:90:83", "network": {"id": "aad085f7-106b-4de6-93f5-f86bb66ec962", "bridge": "br-int", "label": "tempest-AttachVolumeShelveTestJSON-775984268-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "e17150005a9a449f9373b91e1c00083b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapb68d722f-90", "ovs_interfaceid": "b68d722f-9020-4da4-8cdb-e2312ef06df9", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:16:38 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-f3b0ea6e-b40c-4205-a67d-a35d8dd70426 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Releasing lock "refresh_cache-99d6f85f-1b77-4390-8b61-908c98f584c0" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:16:38 user nova-compute[75234]: DEBUG nova.compute.manager [None req-f3b0ea6e-b40c-4205-a67d-a35d8dd70426 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] [instance: 99d6f85f-1b77-4390-8b61-908c98f584c0] Instance network_info: |[{"id": "b68d722f-9020-4da4-8cdb-e2312ef06df9", "address": "fa:16:3e:1e:90:83", "network": {"id": "aad085f7-106b-4de6-93f5-f86bb66ec962", "bridge": "br-int", "label": "tempest-AttachVolumeShelveTestJSON-775984268-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "e17150005a9a449f9373b91e1c00083b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapb68d722f-90", "ovs_interfaceid": "b68d722f-9020-4da4-8cdb-e2312ef06df9", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=75234) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1987}}
Nov 29 03:16:38 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-f589c615-7f8b-42cf-ab2d-d6da0b6f6fac req-a0710fcf-94c6-4f7a-a2eb-2318371cebaf service nova] Acquired lock "refresh_cache-99d6f85f-1b77-4390-8b61-908c98f584c0" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:16:38 user nova-compute[75234]: DEBUG nova.network.neutron [req-f589c615-7f8b-42cf-ab2d-d6da0b6f6fac req-a0710fcf-94c6-4f7a-a2eb-2318371cebaf service nova] [instance: 99d6f85f-1b77-4390-8b61-908c98f584c0] Refreshing network info cache for port b68d722f-9020-4da4-8cdb-e2312ef06df9 {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2006}}
Nov 29 03:16:38 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-f3b0ea6e-b40c-4205-a67d-a35d8dd70426 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] [instance: 99d6f85f-1b77-4390-8b61-908c98f584c0] Start _get_guest_xml network_info=[{"id": "b68d722f-9020-4da4-8cdb-e2312ef06df9", "address": "fa:16:3e:1e:90:83", "network": {"id": "aad085f7-106b-4de6-93f5-f86bb66ec962", "bridge": "br-int", "label": "tempest-AttachVolumeShelveTestJSON-775984268-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "e17150005a9a449f9373b91e1c00083b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapb68d722f-90", "ovs_interfaceid": "b68d722f-9020-4da4-8cdb-e2312ef06df9", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum='c8fc807773e5354afe61636071771906',container_format='bare',created_at=2023-11-29T03:02:15Z,direct_url=<?>,disk_format='qcow2',id=96c30ba7-9b73-4a30-8ba4-7138879573b7,min_disk=0,min_ram=0,name='cirros-0.6.2-x86_64-disk',owner='b18e380921c748b78517b6f1211cd0f1',properties=ImageMetaProps,protected=<?>,size=21430272,status='active',tags=<?>,updated_at=2023-11-29T03:02:17Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [{'device_name': '/dev/vda', 'encryption_format': None, 'encrypted': False, 'device_type': 'disk', 'size': 0, 'disk_bus': 'virtio', 'boot_index': 0, 'encryption_options': None, 'encryption_secret_uuid': None, 'guest_format': None, 'image_id': '96c30ba7-9b73-4a30-8ba4-7138879573b7'}], 'ephemerals': [], 'block_device_mapping': [], 'swap': None} {{(pid=75234) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7562}}
Nov 29 03:16:38 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-f3b0ea6e-b40c-4205-a67d-a35d8dd70426 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:16:38 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-f3b0ea6e-b40c-4205-a67d-a35d8dd70426 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:16:38 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-f3b0ea6e-b40c-4205-a67d-a35d8dd70426 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Searching host: 'user' for CPU controller through CGroups V1... {{(pid=75234) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1652}}
Nov 29 03:16:38 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-f3b0ea6e-b40c-4205-a67d-a35d8dd70426 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] CPU controller missing on host. {{(pid=75234) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1662}}
Nov 29 03:16:38 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-f3b0ea6e-b40c-4205-a67d-a35d8dd70426 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Searching host: 'user' for CPU controller through CGroups V2... {{(pid=75234) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1671}}
Nov 29 03:16:38 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-f3b0ea6e-b40c-4205-a67d-a35d8dd70426 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] CPU controller found on host. {{(pid=75234) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1678}}
Nov 29 03:16:38 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-f3b0ea6e-b40c-4205-a67d-a35d8dd70426 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=75234) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5400}}
Nov 29 03:16:38 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-f3b0ea6e-b40c-4205-a67d-a35d8dd70426 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-11-29T03:02:04Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='1',id=6,is_public=True,memory_mb=512,name='m1.tiny',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='c8fc807773e5354afe61636071771906',container_format='bare',created_at=2023-11-29T03:02:15Z,direct_url=<?>,disk_format='qcow2',id=96c30ba7-9b73-4a30-8ba4-7138879573b7,min_disk=0,min_ram=0,name='cirros-0.6.2-x86_64-disk',owner='b18e380921c748b78517b6f1211cd0f1',properties=ImageMetaProps,protected=<?>,size=21430272,status='active',tags=<?>,updated_at=2023-11-29T03:02:17Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:563}}
Nov 29 03:16:38 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-f3b0ea6e-b40c-4205-a67d-a35d8dd70426 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Flavor limits 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:348}}
Nov 29 03:16:38 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-f3b0ea6e-b40c-4205-a67d-a35d8dd70426 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Image limits 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:352}}
Nov 29 03:16:38 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-f3b0ea6e-b40c-4205-a67d-a35d8dd70426 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Flavor pref 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:388}}
Nov 29 03:16:38 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-f3b0ea6e-b40c-4205-a67d-a35d8dd70426 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Image pref 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:392}}
Nov 29 03:16:38 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-f3b0ea6e-b40c-4205-a67d-a35d8dd70426 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:430}}
Nov 29 03:16:38 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-f3b0ea6e-b40c-4205-a67d-a35d8dd70426 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:569}}
Nov 29 03:16:38 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-f3b0ea6e-b40c-4205-a67d-a35d8dd70426 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=75234) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:471}}
Nov 29 03:16:38 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-f3b0ea6e-b40c-4205-a67d-a35d8dd70426 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Got 1 possible topologies {{(pid=75234) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:501}}
Nov 29 03:16:38 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-f3b0ea6e-b40c-4205-a67d-a35d8dd70426 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:575}}
Nov 29 03:16:38 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-f3b0ea6e-b40c-4205-a67d-a35d8dd70426 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:577}}
Nov 29 03:16:38 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-f3b0ea6e-b40c-4205-a67d-a35d8dd70426 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-11-29T03:16:35Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description=None,display_name='tempest-AttachVolumeShelveTestJSON-server-491132081',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-attachvolumeshelvetestjson-server-491132081',id=28,image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBICHY8pkIfnX3FeZlRYJkWHQpVAhCpz3AE8GcQPc0ndKAzNj2za38XOUjeohx2bMy5FXksYmprL/THiMHoZRTInmzdW4YHaLMkQCBMth1H/mEdsilbqzkRPZKyCYZLp9Bw==',key_name='tempest-keypair-1848667272',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='e17150005a9a449f9373b91e1c00083b',ramdisk_id='',reservation_id='r-79uedf60',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-AttachVolumeShelveTestJSON-1180535246',owner_user_name='tempest-AttachVolumeShelveTestJSON-1180535246-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-11-29T03:16:36Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='154f3f4f57514450baf8f22d67768bc9',uuid=99d6f85f-1b77-4390-8b61-908c98f584c0,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "b68d722f-9020-4da4-8cdb-e2312ef06df9", "address": "fa:16:3e:1e:90:83", "network": {"id": "aad085f7-106b-4de6-93f5-f86bb66ec962", "bridge": "br-int", "label": "tempest-AttachVolumeShelveTestJSON-775984268-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "e17150005a9a449f9373b91e1c00083b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapb68d722f-90", "ovs_interfaceid": "b68d722f-9020-4da4-8cdb-e2312ef06df9", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=75234) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Nov 29 03:16:38 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-f3b0ea6e-b40c-4205-a67d-a35d8dd70426 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Converting VIF {"id": "b68d722f-9020-4da4-8cdb-e2312ef06df9", "address": "fa:16:3e:1e:90:83", "network": {"id": "aad085f7-106b-4de6-93f5-f86bb66ec962", "bridge": "br-int", "label": "tempest-AttachVolumeShelveTestJSON-775984268-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "e17150005a9a449f9373b91e1c00083b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapb68d722f-90", "ovs_interfaceid": "b68d722f-9020-4da4-8cdb-e2312ef06df9", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:16:38 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-f3b0ea6e-b40c-4205-a67d-a35d8dd70426 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:1e:90:83,bridge_name='br-int',has_traffic_filtering=True,id=b68d722f-9020-4da4-8cdb-e2312ef06df9,network=Network(aad085f7-106b-4de6-93f5-f86bb66ec962),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapb68d722f-90') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:16:38 user nova-compute[75234]: DEBUG nova.objects.instance [None req-f3b0ea6e-b40c-4205-a67d-a35d8dd70426 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Lazy-loading 'pci_devices' on Instance uuid 99d6f85f-1b77-4390-8b61-908c98f584c0 {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:16:38 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-f3b0ea6e-b40c-4205-a67d-a35d8dd70426 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] [instance: 99d6f85f-1b77-4390-8b61-908c98f584c0] End _get_guest_xml xml=<domain type="kvm">
Nov 29 03:16:38 user nova-compute[75234]:   <uuid>99d6f85f-1b77-4390-8b61-908c98f584c0</uuid>
Nov 29 03:16:38 user nova-compute[75234]:   <name>instance-0000001c</name>
Nov 29 03:16:38 user nova-compute[75234]:   <memory>524288</memory>
Nov 29 03:16:38 user nova-compute[75234]:   <vcpu>1</vcpu>
Nov 29 03:16:38 user nova-compute[75234]:   <metadata>
Nov 29 03:16:38 user nova-compute[75234]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Nov 29 03:16:38 user nova-compute[75234]:       <nova:package version="0.0.0"/>
Nov 29 03:16:38 user nova-compute[75234]:       <nova:name>tempest-AttachVolumeShelveTestJSON-server-491132081</nova:name>
Nov 29 03:16:38 user nova-compute[75234]:       <nova:creationTime>2023-11-29 03:16:38</nova:creationTime>
Nov 29 03:16:38 user nova-compute[75234]:       <nova:flavor name="m1.tiny">
Nov 29 03:16:38 user nova-compute[75234]:         <nova:memory>512</nova:memory>
Nov 29 03:16:38 user nova-compute[75234]:         <nova:disk>1</nova:disk>
Nov 29 03:16:38 user nova-compute[75234]:         <nova:swap>0</nova:swap>
Nov 29 03:16:38 user nova-compute[75234]:         <nova:ephemeral>0</nova:ephemeral>
Nov 29 03:16:38 user nova-compute[75234]:         <nova:vcpus>1</nova:vcpus>
Nov 29 03:16:38 user nova-compute[75234]:       </nova:flavor>
Nov 29 03:16:38 user nova-compute[75234]:       <nova:owner>
Nov 29 03:16:38 user nova-compute[75234]:         <nova:user uuid="154f3f4f57514450baf8f22d67768bc9">tempest-AttachVolumeShelveTestJSON-1180535246-project-member</nova:user>
Nov 29 03:16:38 user nova-compute[75234]:         <nova:project uuid="e17150005a9a449f9373b91e1c00083b">tempest-AttachVolumeShelveTestJSON-1180535246</nova:project>
Nov 29 03:16:38 user nova-compute[75234]:       </nova:owner>
Nov 29 03:16:38 user nova-compute[75234]:       <nova:root type="image" uuid="96c30ba7-9b73-4a30-8ba4-7138879573b7"/>
Nov 29 03:16:38 user nova-compute[75234]:       <nova:ports>
Nov 29 03:16:38 user nova-compute[75234]:         <nova:port uuid="b68d722f-9020-4da4-8cdb-e2312ef06df9">
Nov 29 03:16:38 user nova-compute[75234]:           <nova:ip type="fixed" address="10.0.0.3" ipVersion="4"/>
Nov 29 03:16:38 user nova-compute[75234]:         </nova:port>
Nov 29 03:16:38 user nova-compute[75234]:       </nova:ports>
Nov 29 03:16:38 user nova-compute[75234]:     </nova:instance>
Nov 29 03:16:38 user nova-compute[75234]:   </metadata>
Nov 29 03:16:38 user nova-compute[75234]:   <sysinfo type="smbios">
Nov 29 03:16:38 user nova-compute[75234]:     <system>
Nov 29 03:16:38 user nova-compute[75234]:       <entry name="manufacturer">OpenStack Foundation</entry>
Nov 29 03:16:38 user nova-compute[75234]:       <entry name="product">OpenStack Nova</entry>
Nov 29 03:16:38 user nova-compute[75234]:       <entry name="version">0.0.0</entry>
Nov 29 03:16:38 user nova-compute[75234]:       <entry name="serial">99d6f85f-1b77-4390-8b61-908c98f584c0</entry>
Nov 29 03:16:38 user nova-compute[75234]:       <entry name="uuid">99d6f85f-1b77-4390-8b61-908c98f584c0</entry>
Nov 29 03:16:38 user nova-compute[75234]:       <entry name="family">Virtual Machine</entry>
Nov 29 03:16:38 user nova-compute[75234]:     </system>
Nov 29 03:16:38 user nova-compute[75234]:   </sysinfo>
Nov 29 03:16:38 user nova-compute[75234]:   <os>
Nov 29 03:16:38 user nova-compute[75234]:     <type arch="x86_64" machine="pc">hvm</type>
Nov 29 03:16:38 user nova-compute[75234]:     <boot dev="hd"/>
Nov 29 03:16:38 user nova-compute[75234]:     <smbios mode="sysinfo"/>
Nov 29 03:16:38 user nova-compute[75234]:   </os>
Nov 29 03:16:38 user nova-compute[75234]:   <features>
Nov 29 03:16:38 user nova-compute[75234]:     <acpi/>
Nov 29 03:16:38 user nova-compute[75234]:     <apic/>
Nov 29 03:16:38 user nova-compute[75234]:     <vmcoreinfo/>
Nov 29 03:16:38 user nova-compute[75234]:   </features>
Nov 29 03:16:38 user nova-compute[75234]:   <clock offset="utc">
Nov 29 03:16:38 user nova-compute[75234]:     <timer name="pit" tickpolicy="delay"/>
Nov 29 03:16:38 user nova-compute[75234]:     <timer name="rtc" tickpolicy="catchup"/>
Nov 29 03:16:38 user nova-compute[75234]:     <timer name="hpet" present="no"/>
Nov 29 03:16:38 user nova-compute[75234]:   </clock>
Nov 29 03:16:38 user nova-compute[75234]:   <cpu mode="custom" match="exact">
Nov 29 03:16:38 user nova-compute[75234]:     <model>Nehalem</model>
Nov 29 03:16:38 user nova-compute[75234]:     <topology sockets="1" cores="1" threads="1"/>
Nov 29 03:16:38 user nova-compute[75234]:   </cpu>
Nov 29 03:16:38 user nova-compute[75234]:   <devices>
Nov 29 03:16:38 user nova-compute[75234]:     <disk type="file" device="disk">
Nov 29 03:16:38 user nova-compute[75234]:       <driver name="qemu" type="qcow2" cache="none"/>
Nov 29 03:16:38 user nova-compute[75234]:       <source file="/opt/stack/data/nova/instances/99d6f85f-1b77-4390-8b61-908c98f584c0/disk"/>
Nov 29 03:16:38 user nova-compute[75234]:       <target dev="vda" bus="virtio"/>
Nov 29 03:16:38 user nova-compute[75234]:     </disk>
Nov 29 03:16:38 user nova-compute[75234]:     <interface type="ethernet">
Nov 29 03:16:38 user nova-compute[75234]:       <mac address="fa:16:3e:1e:90:83"/>
Nov 29 03:16:38 user nova-compute[75234]:       <model type="virtio"/>
Nov 29 03:16:38 user nova-compute[75234]:       <mtu size="1442"/>
Nov 29 03:16:38 user nova-compute[75234]:       <target dev="tapb68d722f-90"/>
Nov 29 03:16:38 user nova-compute[75234]:     </interface>
Nov 29 03:16:38 user nova-compute[75234]:     <serial type="pty">
Nov 29 03:16:38 user nova-compute[75234]:       <log file="/opt/stack/data/nova/instances/99d6f85f-1b77-4390-8b61-908c98f584c0/console.log" append="off"/>
Nov 29 03:16:38 user nova-compute[75234]:     </serial>
Nov 29 03:16:38 user nova-compute[75234]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Nov 29 03:16:38 user nova-compute[75234]:     <video>
Nov 29 03:16:38 user nova-compute[75234]:       <model type="virtio"/>
Nov 29 03:16:38 user nova-compute[75234]:     </video>
Nov 29 03:16:38 user nova-compute[75234]:     <rng model="virtio">
Nov 29 03:16:38 user nova-compute[75234]:       <backend model="random">/dev/urandom</backend>
Nov 29 03:16:38 user nova-compute[75234]:     </rng>
Nov 29 03:16:38 user nova-compute[75234]:     <controller type="usb" index="0" model="none"/>
Nov 29 03:16:38 user nova-compute[75234]:     <memballoon model="virtio">
Nov 29 03:16:38 user nova-compute[75234]:       <stats period="10"/>
Nov 29 03:16:38 user nova-compute[75234]:     </memballoon>
Nov 29 03:16:38 user nova-compute[75234]:   </devices>
Nov 29 03:16:38 user nova-compute[75234]: </domain>
Nov 29 03:16:38 user nova-compute[75234]:  {{(pid=75234) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7568}}
Nov 29 03:16:38 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-f3b0ea6e-b40c-4205-a67d-a35d8dd70426 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-11-29T03:16:35Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description=None,display_name='tempest-AttachVolumeShelveTestJSON-server-491132081',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-attachvolumeshelvetestjson-server-491132081',id=28,image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBICHY8pkIfnX3FeZlRYJkWHQpVAhCpz3AE8GcQPc0ndKAzNj2za38XOUjeohx2bMy5FXksYmprL/THiMHoZRTInmzdW4YHaLMkQCBMth1H/mEdsilbqzkRPZKyCYZLp9Bw==',key_name='tempest-keypair-1848667272',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='e17150005a9a449f9373b91e1c00083b',ramdisk_id='',reservation_id='r-79uedf60',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-AttachVolumeShelveTestJSON-1180535246',owner_user_name='tempest-AttachVolumeShelveTestJSON-1180535246-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-11-29T03:16:36Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='154f3f4f57514450baf8f22d67768bc9',uuid=99d6f85f-1b77-4390-8b61-908c98f584c0,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "b68d722f-9020-4da4-8cdb-e2312ef06df9", "address": "fa:16:3e:1e:90:83", "network": {"id": "aad085f7-106b-4de6-93f5-f86bb66ec962", "bridge": "br-int", "label": "tempest-AttachVolumeShelveTestJSON-775984268-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "e17150005a9a449f9373b91e1c00083b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapb68d722f-90", "ovs_interfaceid": "b68d722f-9020-4da4-8cdb-e2312ef06df9", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Nov 29 03:16:38 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-f3b0ea6e-b40c-4205-a67d-a35d8dd70426 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Converting VIF {"id": "b68d722f-9020-4da4-8cdb-e2312ef06df9", "address": "fa:16:3e:1e:90:83", "network": {"id": "aad085f7-106b-4de6-93f5-f86bb66ec962", "bridge": "br-int", "label": "tempest-AttachVolumeShelveTestJSON-775984268-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "e17150005a9a449f9373b91e1c00083b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapb68d722f-90", "ovs_interfaceid": "b68d722f-9020-4da4-8cdb-e2312ef06df9", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:16:38 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-f3b0ea6e-b40c-4205-a67d-a35d8dd70426 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:1e:90:83,bridge_name='br-int',has_traffic_filtering=True,id=b68d722f-9020-4da4-8cdb-e2312ef06df9,network=Network(aad085f7-106b-4de6-93f5-f86bb66ec962),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapb68d722f-90') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:16:38 user nova-compute[75234]: DEBUG os_vif [None req-f3b0ea6e-b40c-4205-a67d-a35d8dd70426 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:1e:90:83,bridge_name='br-int',has_traffic_filtering=True,id=b68d722f-9020-4da4-8cdb-e2312ef06df9,network=Network(aad085f7-106b-4de6-93f5-f86bb66ec962),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapb68d722f-90') {{(pid=75234) plug /opt/stack/data/venv/lib/python3.10/site-packages/os_vif/__init__.py:76}}
Nov 29 03:16:38 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:16:38 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:16:38 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Nov 29 03:16:38 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:16:38 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbCreateCommand(_result=None, table=QoS, columns={'type': 'linux-noop', 'external_ids': {'id': '1d5ec683-25aa-5708-bd43-73c9dffea702', '_type': 'linux-noop'}}, row=False) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:16:38 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:16:38 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:16:38 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:16:38 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tapb68d722f-90, may_exist=True, interface_attrs={}) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:16:38 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Port, record=tapb68d722f-90, col_values=(('qos', UUID('93a08874-f41d-4fa3-bdfb-d88561a14a6f')),), if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:16:38 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DbSetCommand(_result=None, table=Interface, record=tapb68d722f-90, col_values=(('external_ids', {'iface-id': 'b68d722f-9020-4da4-8cdb-e2312ef06df9', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:1e:90:83', 'vm-uuid': '99d6f85f-1b77-4390-8b61-908c98f584c0'}),), if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:16:38 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:16:38 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:16:38 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:16:38 user nova-compute[75234]: INFO os_vif [None req-f3b0ea6e-b40c-4205-a67d-a35d8dd70426 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:1e:90:83,bridge_name='br-int',has_traffic_filtering=True,id=b68d722f-9020-4da4-8cdb-e2312ef06df9,network=Network(aad085f7-106b-4de6-93f5-f86bb66ec962),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapb68d722f-90')
Nov 29 03:16:38 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-f3b0ea6e-b40c-4205-a67d-a35d8dd70426 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] No BDM found with device name vda, not building metadata. {{(pid=75234) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12089}}
Nov 29 03:16:38 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-f3b0ea6e-b40c-4205-a67d-a35d8dd70426 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] No VIF found with MAC fa:16:3e:1e:90:83, not building metadata {{(pid=75234) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Nov 29 03:16:38 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:16:38 user nova-compute[75234]: DEBUG nova.network.neutron [req-f589c615-7f8b-42cf-ab2d-d6da0b6f6fac req-a0710fcf-94c6-4f7a-a2eb-2318371cebaf service nova] [instance: 99d6f85f-1b77-4390-8b61-908c98f584c0] Updated VIF entry in instance network info cache for port b68d722f-9020-4da4-8cdb-e2312ef06df9. {{(pid=75234) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3481}}
Nov 29 03:16:38 user nova-compute[75234]: DEBUG nova.network.neutron [req-f589c615-7f8b-42cf-ab2d-d6da0b6f6fac req-a0710fcf-94c6-4f7a-a2eb-2318371cebaf service nova] [instance: 99d6f85f-1b77-4390-8b61-908c98f584c0] Updating instance_info_cache with network_info: [{"id": "b68d722f-9020-4da4-8cdb-e2312ef06df9", "address": "fa:16:3e:1e:90:83", "network": {"id": "aad085f7-106b-4de6-93f5-f86bb66ec962", "bridge": "br-int", "label": "tempest-AttachVolumeShelveTestJSON-775984268-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "e17150005a9a449f9373b91e1c00083b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapb68d722f-90", "ovs_interfaceid": "b68d722f-9020-4da4-8cdb-e2312ef06df9", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:16:38 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-f589c615-7f8b-42cf-ab2d-d6da0b6f6fac req-a0710fcf-94c6-4f7a-a2eb-2318371cebaf service nova] Releasing lock "refresh_cache-99d6f85f-1b77-4390-8b61-908c98f584c0" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:16:39 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:16:39 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:16:39 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:16:40 user nova-compute[75234]: DEBUG nova.compute.manager [req-f6503026-e689-4792-a8b0-b8a5c815afa7 req-431c4601-4b0f-436d-902e-4e108276324f service nova] [instance: 99d6f85f-1b77-4390-8b61-908c98f584c0] Received event network-vif-plugged-b68d722f-9020-4da4-8cdb-e2312ef06df9 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:16:40 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-f6503026-e689-4792-a8b0-b8a5c815afa7 req-431c4601-4b0f-436d-902e-4e108276324f service nova] Acquiring lock "99d6f85f-1b77-4390-8b61-908c98f584c0-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:16:40 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-f6503026-e689-4792-a8b0-b8a5c815afa7 req-431c4601-4b0f-436d-902e-4e108276324f service nova] Lock "99d6f85f-1b77-4390-8b61-908c98f584c0-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:16:40 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-f6503026-e689-4792-a8b0-b8a5c815afa7 req-431c4601-4b0f-436d-902e-4e108276324f service nova] Lock "99d6f85f-1b77-4390-8b61-908c98f584c0-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:16:40 user nova-compute[75234]: DEBUG nova.compute.manager [req-f6503026-e689-4792-a8b0-b8a5c815afa7 req-431c4601-4b0f-436d-902e-4e108276324f service nova] [instance: 99d6f85f-1b77-4390-8b61-908c98f584c0] No waiting events found dispatching network-vif-plugged-b68d722f-9020-4da4-8cdb-e2312ef06df9 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:16:40 user nova-compute[75234]: WARNING nova.compute.manager [req-f6503026-e689-4792-a8b0-b8a5c815afa7 req-431c4601-4b0f-436d-902e-4e108276324f service nova] [instance: 99d6f85f-1b77-4390-8b61-908c98f584c0] Received unexpected event network-vif-plugged-b68d722f-9020-4da4-8cdb-e2312ef06df9 for instance with vm_state building and task_state spawning.
Nov 29 03:16:40 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:16:40 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:16:40 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:16:40 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:16:41 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:16:41 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:16:42 user nova-compute[75234]: DEBUG nova.compute.manager [None req-f3b0ea6e-b40c-4205-a67d-a35d8dd70426 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] [instance: 99d6f85f-1b77-4390-8b61-908c98f584c0] Instance event wait completed in 0 seconds for  {{(pid=75234) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Nov 29 03:16:42 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-f3b0ea6e-b40c-4205-a67d-a35d8dd70426 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] [instance: 99d6f85f-1b77-4390-8b61-908c98f584c0] Guest created on hypervisor {{(pid=75234) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4421}}
Nov 29 03:16:42 user nova-compute[75234]: INFO nova.virt.libvirt.driver [-] [instance: 99d6f85f-1b77-4390-8b61-908c98f584c0] Instance spawned successfully.
Nov 29 03:16:42 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-f3b0ea6e-b40c-4205-a67d-a35d8dd70426 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] [instance: 99d6f85f-1b77-4390-8b61-908c98f584c0] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:904}}
Nov 29 03:16:42 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-f3b0ea6e-b40c-4205-a67d-a35d8dd70426 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] [instance: 99d6f85f-1b77-4390-8b61-908c98f584c0] Found default for hw_cdrom_bus of ide {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:16:42 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-f3b0ea6e-b40c-4205-a67d-a35d8dd70426 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] [instance: 99d6f85f-1b77-4390-8b61-908c98f584c0] Found default for hw_disk_bus of virtio {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:16:42 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-f3b0ea6e-b40c-4205-a67d-a35d8dd70426 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] [instance: 99d6f85f-1b77-4390-8b61-908c98f584c0] Found default for hw_input_bus of None {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:16:42 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-f3b0ea6e-b40c-4205-a67d-a35d8dd70426 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] [instance: 99d6f85f-1b77-4390-8b61-908c98f584c0] Found default for hw_pointer_model of None {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:16:42 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-f3b0ea6e-b40c-4205-a67d-a35d8dd70426 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] [instance: 99d6f85f-1b77-4390-8b61-908c98f584c0] Found default for hw_video_model of virtio {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:16:42 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-f3b0ea6e-b40c-4205-a67d-a35d8dd70426 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] [instance: 99d6f85f-1b77-4390-8b61-908c98f584c0] Found default for hw_vif_model of virtio {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:16:42 user nova-compute[75234]: INFO nova.compute.manager [None req-f3b0ea6e-b40c-4205-a67d-a35d8dd70426 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] [instance: 99d6f85f-1b77-4390-8b61-908c98f584c0] Took 5.70 seconds to spawn the instance on the hypervisor.
Nov 29 03:16:42 user nova-compute[75234]: DEBUG nova.compute.manager [None req-f3b0ea6e-b40c-4205-a67d-a35d8dd70426 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] [instance: 99d6f85f-1b77-4390-8b61-908c98f584c0] Checking state {{(pid=75234) _get_power_state /opt/stack/nova/nova/compute/manager.py:1782}}
Nov 29 03:16:42 user nova-compute[75234]: INFO nova.compute.manager [None req-f3b0ea6e-b40c-4205-a67d-a35d8dd70426 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] [instance: 99d6f85f-1b77-4390-8b61-908c98f584c0] Took 6.58 seconds to build instance.
Nov 29 03:16:42 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-f3b0ea6e-b40c-4205-a67d-a35d8dd70426 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Lock "99d6f85f-1b77-4390-8b61-908c98f584c0" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 6.689s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:16:42 user nova-compute[75234]: DEBUG nova.compute.manager [req-ddee47f0-7198-43f2-9831-ae799830ef5d req-88ac624e-2282-4804-a824-2c43b0f339ec service nova] [instance: 99d6f85f-1b77-4390-8b61-908c98f584c0] Received event network-vif-plugged-b68d722f-9020-4da4-8cdb-e2312ef06df9 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:16:42 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-ddee47f0-7198-43f2-9831-ae799830ef5d req-88ac624e-2282-4804-a824-2c43b0f339ec service nova] Acquiring lock "99d6f85f-1b77-4390-8b61-908c98f584c0-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:16:42 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-ddee47f0-7198-43f2-9831-ae799830ef5d req-88ac624e-2282-4804-a824-2c43b0f339ec service nova] Lock "99d6f85f-1b77-4390-8b61-908c98f584c0-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:16:42 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-ddee47f0-7198-43f2-9831-ae799830ef5d req-88ac624e-2282-4804-a824-2c43b0f339ec service nova] Lock "99d6f85f-1b77-4390-8b61-908c98f584c0-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:16:42 user nova-compute[75234]: DEBUG nova.compute.manager [req-ddee47f0-7198-43f2-9831-ae799830ef5d req-88ac624e-2282-4804-a824-2c43b0f339ec service nova] [instance: 99d6f85f-1b77-4390-8b61-908c98f584c0] No waiting events found dispatching network-vif-plugged-b68d722f-9020-4da4-8cdb-e2312ef06df9 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:16:42 user nova-compute[75234]: WARNING nova.compute.manager [req-ddee47f0-7198-43f2-9831-ae799830ef5d req-88ac624e-2282-4804-a824-2c43b0f339ec service nova] [instance: 99d6f85f-1b77-4390-8b61-908c98f584c0] Received unexpected event network-vif-plugged-b68d722f-9020-4da4-8cdb-e2312ef06df9 for instance with vm_state active and task_state None.
Nov 29 03:16:43 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:16:45 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:16:46 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:16:47 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:16:48 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:16:49 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._cleanup_incomplete_migrations {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:16:49 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Cleaning up deleted instances with incomplete migration  {{(pid=75234) _cleanup_incomplete_migrations /opt/stack/nova/nova/compute/manager.py:11247}}
Nov 29 03:16:50 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-0f63ae7d-b42b-4e6d-a119-dcc8ba491b4d tempest-TestServerVolumeAttachScenarioOldVersion-830274084 tempest-TestServerVolumeAttachScenarioOldVersion-830274084-project-member] Acquiring lock "5e6cca46-c5a7-4303-98c2-1ad483c6d14e" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:16:50 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-0f63ae7d-b42b-4e6d-a119-dcc8ba491b4d tempest-TestServerVolumeAttachScenarioOldVersion-830274084 tempest-TestServerVolumeAttachScenarioOldVersion-830274084-project-member] Lock "5e6cca46-c5a7-4303-98c2-1ad483c6d14e" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:16:50 user nova-compute[75234]: DEBUG nova.compute.manager [None req-0f63ae7d-b42b-4e6d-a119-dcc8ba491b4d tempest-TestServerVolumeAttachScenarioOldVersion-830274084 tempest-TestServerVolumeAttachScenarioOldVersion-830274084-project-member] [instance: 5e6cca46-c5a7-4303-98c2-1ad483c6d14e] Starting instance... {{(pid=75234) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2422}}
Nov 29 03:16:50 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-0f63ae7d-b42b-4e6d-a119-dcc8ba491b4d tempest-TestServerVolumeAttachScenarioOldVersion-830274084 tempest-TestServerVolumeAttachScenarioOldVersion-830274084-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:16:50 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-0f63ae7d-b42b-4e6d-a119-dcc8ba491b4d tempest-TestServerVolumeAttachScenarioOldVersion-830274084 tempest-TestServerVolumeAttachScenarioOldVersion-830274084-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:16:50 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-0f63ae7d-b42b-4e6d-a119-dcc8ba491b4d tempest-TestServerVolumeAttachScenarioOldVersion-830274084 tempest-TestServerVolumeAttachScenarioOldVersion-830274084-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=75234) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2366}}
Nov 29 03:16:50 user nova-compute[75234]: INFO nova.compute.claims [None req-0f63ae7d-b42b-4e6d-a119-dcc8ba491b4d tempest-TestServerVolumeAttachScenarioOldVersion-830274084 tempest-TestServerVolumeAttachScenarioOldVersion-830274084-project-member] [instance: 5e6cca46-c5a7-4303-98c2-1ad483c6d14e] Claim successful on node user
Nov 29 03:16:51 user nova-compute[75234]: DEBUG nova.compute.provider_tree [None req-0f63ae7d-b42b-4e6d-a119-dcc8ba491b4d tempest-TestServerVolumeAttachScenarioOldVersion-830274084 tempest-TestServerVolumeAttachScenarioOldVersion-830274084-project-member] Inventory has not changed in ProviderTree for provider: 4d291220-7c37-4839-a0e3-b23c66502b74 {{(pid=75234) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Nov 29 03:16:51 user nova-compute[75234]: DEBUG nova.scheduler.client.report [None req-0f63ae7d-b42b-4e6d-a119-dcc8ba491b4d tempest-TestServerVolumeAttachScenarioOldVersion-830274084 tempest-TestServerVolumeAttachScenarioOldVersion-830274084-project-member] Inventory has not changed for provider 4d291220-7c37-4839-a0e3-b23c66502b74 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=75234) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Nov 29 03:16:51 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-0f63ae7d-b42b-4e6d-a119-dcc8ba491b4d tempest-TestServerVolumeAttachScenarioOldVersion-830274084 tempest-TestServerVolumeAttachScenarioOldVersion-830274084-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.472s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:16:51 user nova-compute[75234]: DEBUG nova.compute.manager [None req-0f63ae7d-b42b-4e6d-a119-dcc8ba491b4d tempest-TestServerVolumeAttachScenarioOldVersion-830274084 tempest-TestServerVolumeAttachScenarioOldVersion-830274084-project-member] [instance: 5e6cca46-c5a7-4303-98c2-1ad483c6d14e] Start building networks asynchronously for instance. {{(pid=75234) _build_resources /opt/stack/nova/nova/compute/manager.py:2819}}
Nov 29 03:16:51 user nova-compute[75234]: DEBUG nova.compute.manager [None req-0f63ae7d-b42b-4e6d-a119-dcc8ba491b4d tempest-TestServerVolumeAttachScenarioOldVersion-830274084 tempest-TestServerVolumeAttachScenarioOldVersion-830274084-project-member] [instance: 5e6cca46-c5a7-4303-98c2-1ad483c6d14e] Allocating IP information in the background. {{(pid=75234) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1972}}
Nov 29 03:16:51 user nova-compute[75234]: DEBUG nova.network.neutron [None req-0f63ae7d-b42b-4e6d-a119-dcc8ba491b4d tempest-TestServerVolumeAttachScenarioOldVersion-830274084 tempest-TestServerVolumeAttachScenarioOldVersion-830274084-project-member] [instance: 5e6cca46-c5a7-4303-98c2-1ad483c6d14e] allocate_for_instance() {{(pid=75234) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1156}}
Nov 29 03:16:51 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-0f63ae7d-b42b-4e6d-a119-dcc8ba491b4d tempest-TestServerVolumeAttachScenarioOldVersion-830274084 tempest-TestServerVolumeAttachScenarioOldVersion-830274084-project-member] [instance: 5e6cca46-c5a7-4303-98c2-1ad483c6d14e] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Nov 29 03:16:51 user nova-compute[75234]: DEBUG nova.compute.manager [None req-0f63ae7d-b42b-4e6d-a119-dcc8ba491b4d tempest-TestServerVolumeAttachScenarioOldVersion-830274084 tempest-TestServerVolumeAttachScenarioOldVersion-830274084-project-member] [instance: 5e6cca46-c5a7-4303-98c2-1ad483c6d14e] Start building block device mappings for instance. {{(pid=75234) _build_resources /opt/stack/nova/nova/compute/manager.py:2854}}
Nov 29 03:16:51 user nova-compute[75234]: DEBUG nova.policy [None req-0f63ae7d-b42b-4e6d-a119-dcc8ba491b4d tempest-TestServerVolumeAttachScenarioOldVersion-830274084 tempest-TestServerVolumeAttachScenarioOldVersion-830274084-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '68a6c98565d348fb8afabaddcd0b3f9d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'd6f51212dd1c4632b36cfdbd7f901a89', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=75234) authorize /opt/stack/nova/nova/policy.py:203}}
Nov 29 03:16:51 user nova-compute[75234]: DEBUG nova.compute.manager [None req-0f63ae7d-b42b-4e6d-a119-dcc8ba491b4d tempest-TestServerVolumeAttachScenarioOldVersion-830274084 tempest-TestServerVolumeAttachScenarioOldVersion-830274084-project-member] [instance: 5e6cca46-c5a7-4303-98c2-1ad483c6d14e] Start spawning the instance on the hypervisor. {{(pid=75234) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2628}}
Nov 29 03:16:51 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-0f63ae7d-b42b-4e6d-a119-dcc8ba491b4d tempest-TestServerVolumeAttachScenarioOldVersion-830274084 tempest-TestServerVolumeAttachScenarioOldVersion-830274084-project-member] [instance: 5e6cca46-c5a7-4303-98c2-1ad483c6d14e] Creating instance directory {{(pid=75234) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4727}}
Nov 29 03:16:51 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-0f63ae7d-b42b-4e6d-a119-dcc8ba491b4d tempest-TestServerVolumeAttachScenarioOldVersion-830274084 tempest-TestServerVolumeAttachScenarioOldVersion-830274084-project-member] [instance: 5e6cca46-c5a7-4303-98c2-1ad483c6d14e] Creating image(s)
Nov 29 03:16:51 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-0f63ae7d-b42b-4e6d-a119-dcc8ba491b4d tempest-TestServerVolumeAttachScenarioOldVersion-830274084 tempest-TestServerVolumeAttachScenarioOldVersion-830274084-project-member] Acquiring lock "/opt/stack/data/nova/instances/5e6cca46-c5a7-4303-98c2-1ad483c6d14e/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:16:51 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-0f63ae7d-b42b-4e6d-a119-dcc8ba491b4d tempest-TestServerVolumeAttachScenarioOldVersion-830274084 tempest-TestServerVolumeAttachScenarioOldVersion-830274084-project-member] Lock "/opt/stack/data/nova/instances/5e6cca46-c5a7-4303-98c2-1ad483c6d14e/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:16:51 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-0f63ae7d-b42b-4e6d-a119-dcc8ba491b4d tempest-TestServerVolumeAttachScenarioOldVersion-830274084 tempest-TestServerVolumeAttachScenarioOldVersion-830274084-project-member] Lock "/opt/stack/data/nova/instances/5e6cca46-c5a7-4303-98c2-1ad483c6d14e/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.007s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:16:51 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-0f63ae7d-b42b-4e6d-a119-dcc8ba491b4d tempest-TestServerVolumeAttachScenarioOldVersion-830274084 tempest-TestServerVolumeAttachScenarioOldVersion-830274084-project-member] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:16:51 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-0f63ae7d-b42b-4e6d-a119-dcc8ba491b4d tempest-TestServerVolumeAttachScenarioOldVersion-830274084 tempest-TestServerVolumeAttachScenarioOldVersion-830274084-project-member] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json" returned: 0 in 0.173s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:16:51 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-0f63ae7d-b42b-4e6d-a119-dcc8ba491b4d tempest-TestServerVolumeAttachScenarioOldVersion-830274084 tempest-TestServerVolumeAttachScenarioOldVersion-830274084-project-member] Acquiring lock "80686e29dc9db6600d03e85cb62af298e12d7918" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:16:51 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-0f63ae7d-b42b-4e6d-a119-dcc8ba491b4d tempest-TestServerVolumeAttachScenarioOldVersion-830274084 tempest-TestServerVolumeAttachScenarioOldVersion-830274084-project-member] Lock "80686e29dc9db6600d03e85cb62af298e12d7918" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.002s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:16:51 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-0f63ae7d-b42b-4e6d-a119-dcc8ba491b4d tempest-TestServerVolumeAttachScenarioOldVersion-830274084 tempest-TestServerVolumeAttachScenarioOldVersion-830274084-project-member] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:16:51 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:16:51 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-0f63ae7d-b42b-4e6d-a119-dcc8ba491b4d tempest-TestServerVolumeAttachScenarioOldVersion-830274084 tempest-TestServerVolumeAttachScenarioOldVersion-830274084-project-member] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json" returned: 0 in 0.169s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:16:51 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-0f63ae7d-b42b-4e6d-a119-dcc8ba491b4d tempest-TestServerVolumeAttachScenarioOldVersion-830274084 tempest-TestServerVolumeAttachScenarioOldVersion-830274084-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918,backing_fmt=raw /opt/stack/data/nova/instances/5e6cca46-c5a7-4303-98c2-1ad483c6d14e/disk 1073741824 {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:16:51 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-0f63ae7d-b42b-4e6d-a119-dcc8ba491b4d tempest-TestServerVolumeAttachScenarioOldVersion-830274084 tempest-TestServerVolumeAttachScenarioOldVersion-830274084-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918,backing_fmt=raw /opt/stack/data/nova/instances/5e6cca46-c5a7-4303-98c2-1ad483c6d14e/disk 1073741824" returned: 0 in 0.048s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:16:51 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-0f63ae7d-b42b-4e6d-a119-dcc8ba491b4d tempest-TestServerVolumeAttachScenarioOldVersion-830274084 tempest-TestServerVolumeAttachScenarioOldVersion-830274084-project-member] Lock "80686e29dc9db6600d03e85cb62af298e12d7918" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.222s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:16:51 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-0f63ae7d-b42b-4e6d-a119-dcc8ba491b4d tempest-TestServerVolumeAttachScenarioOldVersion-830274084 tempest-TestServerVolumeAttachScenarioOldVersion-830274084-project-member] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:16:52 user nova-compute[75234]: DEBUG nova.network.neutron [None req-0f63ae7d-b42b-4e6d-a119-dcc8ba491b4d tempest-TestServerVolumeAttachScenarioOldVersion-830274084 tempest-TestServerVolumeAttachScenarioOldVersion-830274084-project-member] [instance: 5e6cca46-c5a7-4303-98c2-1ad483c6d14e] Successfully created port: 69d76626-4c79-4557-b501-7c8c92fcdc03 {{(pid=75234) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:548}}
Nov 29 03:16:52 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-0f63ae7d-b42b-4e6d-a119-dcc8ba491b4d tempest-TestServerVolumeAttachScenarioOldVersion-830274084 tempest-TestServerVolumeAttachScenarioOldVersion-830274084-project-member] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json" returned: 0 in 0.181s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:16:52 user nova-compute[75234]: DEBUG nova.virt.disk.api [None req-0f63ae7d-b42b-4e6d-a119-dcc8ba491b4d tempest-TestServerVolumeAttachScenarioOldVersion-830274084 tempest-TestServerVolumeAttachScenarioOldVersion-830274084-project-member] Checking if we can resize image /opt/stack/data/nova/instances/5e6cca46-c5a7-4303-98c2-1ad483c6d14e/disk. size=1073741824 {{(pid=75234) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:166}}
Nov 29 03:16:52 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-0f63ae7d-b42b-4e6d-a119-dcc8ba491b4d tempest-TestServerVolumeAttachScenarioOldVersion-830274084 tempest-TestServerVolumeAttachScenarioOldVersion-830274084-project-member] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/5e6cca46-c5a7-4303-98c2-1ad483c6d14e/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:16:52 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-0f63ae7d-b42b-4e6d-a119-dcc8ba491b4d tempest-TestServerVolumeAttachScenarioOldVersion-830274084 tempest-TestServerVolumeAttachScenarioOldVersion-830274084-project-member] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/5e6cca46-c5a7-4303-98c2-1ad483c6d14e/disk --force-share --output=json" returned: 0 in 0.178s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:16:52 user nova-compute[75234]: DEBUG nova.virt.disk.api [None req-0f63ae7d-b42b-4e6d-a119-dcc8ba491b4d tempest-TestServerVolumeAttachScenarioOldVersion-830274084 tempest-TestServerVolumeAttachScenarioOldVersion-830274084-project-member] Cannot resize image /opt/stack/data/nova/instances/5e6cca46-c5a7-4303-98c2-1ad483c6d14e/disk to a smaller size. {{(pid=75234) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:172}}
Nov 29 03:16:52 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-0f63ae7d-b42b-4e6d-a119-dcc8ba491b4d tempest-TestServerVolumeAttachScenarioOldVersion-830274084 tempest-TestServerVolumeAttachScenarioOldVersion-830274084-project-member] [instance: 5e6cca46-c5a7-4303-98c2-1ad483c6d14e] Created local disks {{(pid=75234) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4861}}
Nov 29 03:16:52 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-0f63ae7d-b42b-4e6d-a119-dcc8ba491b4d tempest-TestServerVolumeAttachScenarioOldVersion-830274084 tempest-TestServerVolumeAttachScenarioOldVersion-830274084-project-member] [instance: 5e6cca46-c5a7-4303-98c2-1ad483c6d14e] Ensure instance console log exists: /opt/stack/data/nova/instances/5e6cca46-c5a7-4303-98c2-1ad483c6d14e/console.log {{(pid=75234) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4613}}
Nov 29 03:16:52 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-0f63ae7d-b42b-4e6d-a119-dcc8ba491b4d tempest-TestServerVolumeAttachScenarioOldVersion-830274084 tempest-TestServerVolumeAttachScenarioOldVersion-830274084-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:16:52 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-0f63ae7d-b42b-4e6d-a119-dcc8ba491b4d tempest-TestServerVolumeAttachScenarioOldVersion-830274084 tempest-TestServerVolumeAttachScenarioOldVersion-830274084-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:16:52 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-0f63ae7d-b42b-4e6d-a119-dcc8ba491b4d tempest-TestServerVolumeAttachScenarioOldVersion-830274084 tempest-TestServerVolumeAttachScenarioOldVersion-830274084-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:16:52 user nova-compute[75234]: DEBUG nova.network.neutron [None req-0f63ae7d-b42b-4e6d-a119-dcc8ba491b4d tempest-TestServerVolumeAttachScenarioOldVersion-830274084 tempest-TestServerVolumeAttachScenarioOldVersion-830274084-project-member] [instance: 5e6cca46-c5a7-4303-98c2-1ad483c6d14e] Successfully updated port: 69d76626-4c79-4557-b501-7c8c92fcdc03 {{(pid=75234) _update_port /opt/stack/nova/nova/network/neutron.py:586}}
Nov 29 03:16:52 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-0f63ae7d-b42b-4e6d-a119-dcc8ba491b4d tempest-TestServerVolumeAttachScenarioOldVersion-830274084 tempest-TestServerVolumeAttachScenarioOldVersion-830274084-project-member] Acquiring lock "refresh_cache-5e6cca46-c5a7-4303-98c2-1ad483c6d14e" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:16:52 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-0f63ae7d-b42b-4e6d-a119-dcc8ba491b4d tempest-TestServerVolumeAttachScenarioOldVersion-830274084 tempest-TestServerVolumeAttachScenarioOldVersion-830274084-project-member] Acquired lock "refresh_cache-5e6cca46-c5a7-4303-98c2-1ad483c6d14e" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:16:52 user nova-compute[75234]: DEBUG nova.network.neutron [None req-0f63ae7d-b42b-4e6d-a119-dcc8ba491b4d tempest-TestServerVolumeAttachScenarioOldVersion-830274084 tempest-TestServerVolumeAttachScenarioOldVersion-830274084-project-member] [instance: 5e6cca46-c5a7-4303-98c2-1ad483c6d14e] Building network info cache for instance {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2009}}
Nov 29 03:16:52 user nova-compute[75234]: DEBUG nova.network.neutron [None req-0f63ae7d-b42b-4e6d-a119-dcc8ba491b4d tempest-TestServerVolumeAttachScenarioOldVersion-830274084 tempest-TestServerVolumeAttachScenarioOldVersion-830274084-project-member] [instance: 5e6cca46-c5a7-4303-98c2-1ad483c6d14e] Instance cache missing network info. {{(pid=75234) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3322}}
Nov 29 03:16:52 user nova-compute[75234]: DEBUG nova.compute.manager [req-ecc9eb85-6ff4-4517-97c5-80a6b3bfcdee req-12353790-6dab-4a09-8af8-75ffa82bcf19 service nova] [instance: 5e6cca46-c5a7-4303-98c2-1ad483c6d14e] Received event network-changed-69d76626-4c79-4557-b501-7c8c92fcdc03 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:16:52 user nova-compute[75234]: DEBUG nova.compute.manager [req-ecc9eb85-6ff4-4517-97c5-80a6b3bfcdee req-12353790-6dab-4a09-8af8-75ffa82bcf19 service nova] [instance: 5e6cca46-c5a7-4303-98c2-1ad483c6d14e] Refreshing instance network info cache due to event network-changed-69d76626-4c79-4557-b501-7c8c92fcdc03. {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11117}}
Nov 29 03:16:52 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-ecc9eb85-6ff4-4517-97c5-80a6b3bfcdee req-12353790-6dab-4a09-8af8-75ffa82bcf19 service nova] Acquiring lock "refresh_cache-5e6cca46-c5a7-4303-98c2-1ad483c6d14e" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:16:53 user nova-compute[75234]: DEBUG nova.network.neutron [None req-0f63ae7d-b42b-4e6d-a119-dcc8ba491b4d tempest-TestServerVolumeAttachScenarioOldVersion-830274084 tempest-TestServerVolumeAttachScenarioOldVersion-830274084-project-member] [instance: 5e6cca46-c5a7-4303-98c2-1ad483c6d14e] Updating instance_info_cache with network_info: [{"id": "69d76626-4c79-4557-b501-7c8c92fcdc03", "address": "fa:16:3e:6e:d5:4a", "network": {"id": "f68acf2f-b01a-4481-845d-46b92646988f", "bridge": "br-int", "label": "tempest-TestServerVolumeAttachScenarioOldVersion-1316398304-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "d6f51212dd1c4632b36cfdbd7f901a89", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap69d76626-4c", "ovs_interfaceid": "69d76626-4c79-4557-b501-7c8c92fcdc03", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:16:53 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-0f63ae7d-b42b-4e6d-a119-dcc8ba491b4d tempest-TestServerVolumeAttachScenarioOldVersion-830274084 tempest-TestServerVolumeAttachScenarioOldVersion-830274084-project-member] Releasing lock "refresh_cache-5e6cca46-c5a7-4303-98c2-1ad483c6d14e" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:16:53 user nova-compute[75234]: DEBUG nova.compute.manager [None req-0f63ae7d-b42b-4e6d-a119-dcc8ba491b4d tempest-TestServerVolumeAttachScenarioOldVersion-830274084 tempest-TestServerVolumeAttachScenarioOldVersion-830274084-project-member] [instance: 5e6cca46-c5a7-4303-98c2-1ad483c6d14e] Instance network_info: |[{"id": "69d76626-4c79-4557-b501-7c8c92fcdc03", "address": "fa:16:3e:6e:d5:4a", "network": {"id": "f68acf2f-b01a-4481-845d-46b92646988f", "bridge": "br-int", "label": "tempest-TestServerVolumeAttachScenarioOldVersion-1316398304-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "d6f51212dd1c4632b36cfdbd7f901a89", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap69d76626-4c", "ovs_interfaceid": "69d76626-4c79-4557-b501-7c8c92fcdc03", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=75234) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1987}}
Nov 29 03:16:53 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-ecc9eb85-6ff4-4517-97c5-80a6b3bfcdee req-12353790-6dab-4a09-8af8-75ffa82bcf19 service nova] Acquired lock "refresh_cache-5e6cca46-c5a7-4303-98c2-1ad483c6d14e" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:16:53 user nova-compute[75234]: DEBUG nova.network.neutron [req-ecc9eb85-6ff4-4517-97c5-80a6b3bfcdee req-12353790-6dab-4a09-8af8-75ffa82bcf19 service nova] [instance: 5e6cca46-c5a7-4303-98c2-1ad483c6d14e] Refreshing network info cache for port 69d76626-4c79-4557-b501-7c8c92fcdc03 {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2006}}
Nov 29 03:16:53 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-0f63ae7d-b42b-4e6d-a119-dcc8ba491b4d tempest-TestServerVolumeAttachScenarioOldVersion-830274084 tempest-TestServerVolumeAttachScenarioOldVersion-830274084-project-member] [instance: 5e6cca46-c5a7-4303-98c2-1ad483c6d14e] Start _get_guest_xml network_info=[{"id": "69d76626-4c79-4557-b501-7c8c92fcdc03", "address": "fa:16:3e:6e:d5:4a", "network": {"id": "f68acf2f-b01a-4481-845d-46b92646988f", "bridge": "br-int", "label": "tempest-TestServerVolumeAttachScenarioOldVersion-1316398304-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "d6f51212dd1c4632b36cfdbd7f901a89", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap69d76626-4c", "ovs_interfaceid": "69d76626-4c79-4557-b501-7c8c92fcdc03", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum='c8fc807773e5354afe61636071771906',container_format='bare',created_at=2023-11-29T03:02:15Z,direct_url=<?>,disk_format='qcow2',id=96c30ba7-9b73-4a30-8ba4-7138879573b7,min_disk=0,min_ram=0,name='cirros-0.6.2-x86_64-disk',owner='b18e380921c748b78517b6f1211cd0f1',properties=ImageMetaProps,protected=<?>,size=21430272,status='active',tags=<?>,updated_at=2023-11-29T03:02:17Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [{'device_name': '/dev/vda', 'encryption_format': None, 'encrypted': False, 'device_type': 'disk', 'size': 0, 'disk_bus': 'virtio', 'boot_index': 0, 'encryption_options': None, 'encryption_secret_uuid': None, 'guest_format': None, 'image_id': '96c30ba7-9b73-4a30-8ba4-7138879573b7'}], 'ephemerals': [], 'block_device_mapping': [], 'swap': None} {{(pid=75234) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7562}}
Nov 29 03:16:53 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-0f63ae7d-b42b-4e6d-a119-dcc8ba491b4d tempest-TestServerVolumeAttachScenarioOldVersion-830274084 tempest-TestServerVolumeAttachScenarioOldVersion-830274084-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:16:53 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-0f63ae7d-b42b-4e6d-a119-dcc8ba491b4d tempest-TestServerVolumeAttachScenarioOldVersion-830274084 tempest-TestServerVolumeAttachScenarioOldVersion-830274084-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:16:53 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-0f63ae7d-b42b-4e6d-a119-dcc8ba491b4d tempest-TestServerVolumeAttachScenarioOldVersion-830274084 tempest-TestServerVolumeAttachScenarioOldVersion-830274084-project-member] Searching host: 'user' for CPU controller through CGroups V1... {{(pid=75234) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1652}}
Nov 29 03:16:53 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-0f63ae7d-b42b-4e6d-a119-dcc8ba491b4d tempest-TestServerVolumeAttachScenarioOldVersion-830274084 tempest-TestServerVolumeAttachScenarioOldVersion-830274084-project-member] CPU controller missing on host. {{(pid=75234) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1662}}
Nov 29 03:16:53 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-0f63ae7d-b42b-4e6d-a119-dcc8ba491b4d tempest-TestServerVolumeAttachScenarioOldVersion-830274084 tempest-TestServerVolumeAttachScenarioOldVersion-830274084-project-member] Searching host: 'user' for CPU controller through CGroups V2... {{(pid=75234) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1671}}
Nov 29 03:16:53 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-0f63ae7d-b42b-4e6d-a119-dcc8ba491b4d tempest-TestServerVolumeAttachScenarioOldVersion-830274084 tempest-TestServerVolumeAttachScenarioOldVersion-830274084-project-member] CPU controller found on host. {{(pid=75234) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1678}}
Nov 29 03:16:53 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-0f63ae7d-b42b-4e6d-a119-dcc8ba491b4d tempest-TestServerVolumeAttachScenarioOldVersion-830274084 tempest-TestServerVolumeAttachScenarioOldVersion-830274084-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=75234) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5400}}
Nov 29 03:16:53 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-0f63ae7d-b42b-4e6d-a119-dcc8ba491b4d tempest-TestServerVolumeAttachScenarioOldVersion-830274084 tempest-TestServerVolumeAttachScenarioOldVersion-830274084-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-11-29T03:02:04Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='1',id=6,is_public=True,memory_mb=512,name='m1.tiny',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='c8fc807773e5354afe61636071771906',container_format='bare',created_at=2023-11-29T03:02:15Z,direct_url=<?>,disk_format='qcow2',id=96c30ba7-9b73-4a30-8ba4-7138879573b7,min_disk=0,min_ram=0,name='cirros-0.6.2-x86_64-disk',owner='b18e380921c748b78517b6f1211cd0f1',properties=ImageMetaProps,protected=<?>,size=21430272,status='active',tags=<?>,updated_at=2023-11-29T03:02:17Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:563}}
Nov 29 03:16:53 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-0f63ae7d-b42b-4e6d-a119-dcc8ba491b4d tempest-TestServerVolumeAttachScenarioOldVersion-830274084 tempest-TestServerVolumeAttachScenarioOldVersion-830274084-project-member] Flavor limits 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:348}}
Nov 29 03:16:53 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-0f63ae7d-b42b-4e6d-a119-dcc8ba491b4d tempest-TestServerVolumeAttachScenarioOldVersion-830274084 tempest-TestServerVolumeAttachScenarioOldVersion-830274084-project-member] Image limits 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:352}}
Nov 29 03:16:53 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-0f63ae7d-b42b-4e6d-a119-dcc8ba491b4d tempest-TestServerVolumeAttachScenarioOldVersion-830274084 tempest-TestServerVolumeAttachScenarioOldVersion-830274084-project-member] Flavor pref 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:388}}
Nov 29 03:16:53 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-0f63ae7d-b42b-4e6d-a119-dcc8ba491b4d tempest-TestServerVolumeAttachScenarioOldVersion-830274084 tempest-TestServerVolumeAttachScenarioOldVersion-830274084-project-member] Image pref 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:392}}
Nov 29 03:16:53 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-0f63ae7d-b42b-4e6d-a119-dcc8ba491b4d tempest-TestServerVolumeAttachScenarioOldVersion-830274084 tempest-TestServerVolumeAttachScenarioOldVersion-830274084-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:430}}
Nov 29 03:16:53 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-0f63ae7d-b42b-4e6d-a119-dcc8ba491b4d tempest-TestServerVolumeAttachScenarioOldVersion-830274084 tempest-TestServerVolumeAttachScenarioOldVersion-830274084-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:569}}
Nov 29 03:16:53 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-0f63ae7d-b42b-4e6d-a119-dcc8ba491b4d tempest-TestServerVolumeAttachScenarioOldVersion-830274084 tempest-TestServerVolumeAttachScenarioOldVersion-830274084-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=75234) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:471}}
Nov 29 03:16:53 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-0f63ae7d-b42b-4e6d-a119-dcc8ba491b4d tempest-TestServerVolumeAttachScenarioOldVersion-830274084 tempest-TestServerVolumeAttachScenarioOldVersion-830274084-project-member] Got 1 possible topologies {{(pid=75234) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:501}}
Nov 29 03:16:53 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-0f63ae7d-b42b-4e6d-a119-dcc8ba491b4d tempest-TestServerVolumeAttachScenarioOldVersion-830274084 tempest-TestServerVolumeAttachScenarioOldVersion-830274084-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:575}}
Nov 29 03:16:53 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-0f63ae7d-b42b-4e6d-a119-dcc8ba491b4d tempest-TestServerVolumeAttachScenarioOldVersion-830274084 tempest-TestServerVolumeAttachScenarioOldVersion-830274084-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:577}}
Nov 29 03:16:53 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-0f63ae7d-b42b-4e6d-a119-dcc8ba491b4d tempest-TestServerVolumeAttachScenarioOldVersion-830274084 tempest-TestServerVolumeAttachScenarioOldVersion-830274084-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-11-29T03:16:50Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-TestServerVolumeAttachScenarioOldVersion-server-816676743',display_name='tempest-TestServerVolumeAttachScenarioOldVersion-server-816676743',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-testservervolumeattachscenariooldversion-server-8166767',id=29,image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBKQV6WOnODZRu0S0Q8BjipN+d4SfGF+oFplEizu9978zfVugP8bBS3zBtCT9lFZz+6aJRHATnsP1lSYQtfjqOryzG+7rBhPRnAeY9RFQZR+EOYZl1GaC/iv/bQZS956elA==',key_name='tempest-keypair-526758182',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='d6f51212dd1c4632b36cfdbd7f901a89',ramdisk_id='',reservation_id='r-49dw65dl',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TestServerVolumeAttachScenarioOldVersion-830274084',owner_user_name='tempest-TestServerVolumeAttachScenarioOldVersion-830274084-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-11-29T03:16:51Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='68a6c98565d348fb8afabaddcd0b3f9d',uuid=5e6cca46-c5a7-4303-98c2-1ad483c6d14e,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "69d76626-4c79-4557-b501-7c8c92fcdc03", "address": "fa:16:3e:6e:d5:4a", "network": {"id": "f68acf2f-b01a-4481-845d-46b92646988f", "bridge": "br-int", "label": "tempest-TestServerVolumeAttachScenarioOldVersion-1316398304-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "d6f51212dd1c4632b36cfdbd7f901a89", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap69d76626-4c", "ovs_interfaceid": "69d76626-4c79-4557-b501-7c8c92fcdc03", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=75234) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Nov 29 03:16:53 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-0f63ae7d-b42b-4e6d-a119-dcc8ba491b4d tempest-TestServerVolumeAttachScenarioOldVersion-830274084 tempest-TestServerVolumeAttachScenarioOldVersion-830274084-project-member] Converting VIF {"id": "69d76626-4c79-4557-b501-7c8c92fcdc03", "address": "fa:16:3e:6e:d5:4a", "network": {"id": "f68acf2f-b01a-4481-845d-46b92646988f", "bridge": "br-int", "label": "tempest-TestServerVolumeAttachScenarioOldVersion-1316398304-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "d6f51212dd1c4632b36cfdbd7f901a89", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap69d76626-4c", "ovs_interfaceid": "69d76626-4c79-4557-b501-7c8c92fcdc03", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:16:53 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-0f63ae7d-b42b-4e6d-a119-dcc8ba491b4d tempest-TestServerVolumeAttachScenarioOldVersion-830274084 tempest-TestServerVolumeAttachScenarioOldVersion-830274084-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:6e:d5:4a,bridge_name='br-int',has_traffic_filtering=True,id=69d76626-4c79-4557-b501-7c8c92fcdc03,network=Network(f68acf2f-b01a-4481-845d-46b92646988f),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap69d76626-4c') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:16:53 user nova-compute[75234]: DEBUG nova.objects.instance [None req-0f63ae7d-b42b-4e6d-a119-dcc8ba491b4d tempest-TestServerVolumeAttachScenarioOldVersion-830274084 tempest-TestServerVolumeAttachScenarioOldVersion-830274084-project-member] Lazy-loading 'pci_devices' on Instance uuid 5e6cca46-c5a7-4303-98c2-1ad483c6d14e {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:16:53 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-0f63ae7d-b42b-4e6d-a119-dcc8ba491b4d tempest-TestServerVolumeAttachScenarioOldVersion-830274084 tempest-TestServerVolumeAttachScenarioOldVersion-830274084-project-member] [instance: 5e6cca46-c5a7-4303-98c2-1ad483c6d14e] End _get_guest_xml xml=<domain type="kvm">
Nov 29 03:16:53 user nova-compute[75234]:   <uuid>5e6cca46-c5a7-4303-98c2-1ad483c6d14e</uuid>
Nov 29 03:16:53 user nova-compute[75234]:   <name>instance-0000001d</name>
Nov 29 03:16:53 user nova-compute[75234]:   <memory>524288</memory>
Nov 29 03:16:53 user nova-compute[75234]:   <vcpu>1</vcpu>
Nov 29 03:16:53 user nova-compute[75234]:   <metadata>
Nov 29 03:16:53 user nova-compute[75234]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Nov 29 03:16:53 user nova-compute[75234]:       <nova:package version="0.0.0"/>
Nov 29 03:16:53 user nova-compute[75234]:       <nova:name>tempest-TestServerVolumeAttachScenarioOldVersion-server-816676743</nova:name>
Nov 29 03:16:53 user nova-compute[75234]:       <nova:creationTime>2023-11-29 03:16:53</nova:creationTime>
Nov 29 03:16:53 user nova-compute[75234]:       <nova:flavor name="m1.tiny">
Nov 29 03:16:53 user nova-compute[75234]:         <nova:memory>512</nova:memory>
Nov 29 03:16:53 user nova-compute[75234]:         <nova:disk>1</nova:disk>
Nov 29 03:16:53 user nova-compute[75234]:         <nova:swap>0</nova:swap>
Nov 29 03:16:53 user nova-compute[75234]:         <nova:ephemeral>0</nova:ephemeral>
Nov 29 03:16:53 user nova-compute[75234]:         <nova:vcpus>1</nova:vcpus>
Nov 29 03:16:53 user nova-compute[75234]:       </nova:flavor>
Nov 29 03:16:53 user nova-compute[75234]:       <nova:owner>
Nov 29 03:16:53 user nova-compute[75234]:         <nova:user uuid="68a6c98565d348fb8afabaddcd0b3f9d">tempest-TestServerVolumeAttachScenarioOldVersion-830274084-project-member</nova:user>
Nov 29 03:16:53 user nova-compute[75234]:         <nova:project uuid="d6f51212dd1c4632b36cfdbd7f901a89">tempest-TestServerVolumeAttachScenarioOldVersion-830274084</nova:project>
Nov 29 03:16:53 user nova-compute[75234]:       </nova:owner>
Nov 29 03:16:53 user nova-compute[75234]:       <nova:root type="image" uuid="96c30ba7-9b73-4a30-8ba4-7138879573b7"/>
Nov 29 03:16:53 user nova-compute[75234]:       <nova:ports>
Nov 29 03:16:53 user nova-compute[75234]:         <nova:port uuid="69d76626-4c79-4557-b501-7c8c92fcdc03">
Nov 29 03:16:53 user nova-compute[75234]:           <nova:ip type="fixed" address="10.0.0.14" ipVersion="4"/>
Nov 29 03:16:53 user nova-compute[75234]:         </nova:port>
Nov 29 03:16:53 user nova-compute[75234]:       </nova:ports>
Nov 29 03:16:53 user nova-compute[75234]:     </nova:instance>
Nov 29 03:16:53 user nova-compute[75234]:   </metadata>
Nov 29 03:16:53 user nova-compute[75234]:   <sysinfo type="smbios">
Nov 29 03:16:53 user nova-compute[75234]:     <system>
Nov 29 03:16:53 user nova-compute[75234]:       <entry name="manufacturer">OpenStack Foundation</entry>
Nov 29 03:16:53 user nova-compute[75234]:       <entry name="product">OpenStack Nova</entry>
Nov 29 03:16:53 user nova-compute[75234]:       <entry name="version">0.0.0</entry>
Nov 29 03:16:53 user nova-compute[75234]:       <entry name="serial">5e6cca46-c5a7-4303-98c2-1ad483c6d14e</entry>
Nov 29 03:16:53 user nova-compute[75234]:       <entry name="uuid">5e6cca46-c5a7-4303-98c2-1ad483c6d14e</entry>
Nov 29 03:16:53 user nova-compute[75234]:       <entry name="family">Virtual Machine</entry>
Nov 29 03:16:53 user nova-compute[75234]:     </system>
Nov 29 03:16:53 user nova-compute[75234]:   </sysinfo>
Nov 29 03:16:53 user nova-compute[75234]:   <os>
Nov 29 03:16:53 user nova-compute[75234]:     <type arch="x86_64" machine="pc">hvm</type>
Nov 29 03:16:53 user nova-compute[75234]:     <boot dev="hd"/>
Nov 29 03:16:53 user nova-compute[75234]:     <smbios mode="sysinfo"/>
Nov 29 03:16:53 user nova-compute[75234]:   </os>
Nov 29 03:16:53 user nova-compute[75234]:   <features>
Nov 29 03:16:53 user nova-compute[75234]:     <acpi/>
Nov 29 03:16:53 user nova-compute[75234]:     <apic/>
Nov 29 03:16:53 user nova-compute[75234]:     <vmcoreinfo/>
Nov 29 03:16:53 user nova-compute[75234]:   </features>
Nov 29 03:16:53 user nova-compute[75234]:   <clock offset="utc">
Nov 29 03:16:53 user nova-compute[75234]:     <timer name="pit" tickpolicy="delay"/>
Nov 29 03:16:53 user nova-compute[75234]:     <timer name="rtc" tickpolicy="catchup"/>
Nov 29 03:16:53 user nova-compute[75234]:     <timer name="hpet" present="no"/>
Nov 29 03:16:53 user nova-compute[75234]:   </clock>
Nov 29 03:16:53 user nova-compute[75234]:   <cpu mode="custom" match="exact">
Nov 29 03:16:53 user nova-compute[75234]:     <model>Nehalem</model>
Nov 29 03:16:53 user nova-compute[75234]:     <topology sockets="1" cores="1" threads="1"/>
Nov 29 03:16:53 user nova-compute[75234]:   </cpu>
Nov 29 03:16:53 user nova-compute[75234]:   <devices>
Nov 29 03:16:53 user nova-compute[75234]:     <disk type="file" device="disk">
Nov 29 03:16:53 user nova-compute[75234]:       <driver name="qemu" type="qcow2" cache="none"/>
Nov 29 03:16:53 user nova-compute[75234]:       <source file="/opt/stack/data/nova/instances/5e6cca46-c5a7-4303-98c2-1ad483c6d14e/disk"/>
Nov 29 03:16:53 user nova-compute[75234]:       <target dev="vda" bus="virtio"/>
Nov 29 03:16:53 user nova-compute[75234]:     </disk>
Nov 29 03:16:53 user nova-compute[75234]:     <interface type="ethernet">
Nov 29 03:16:53 user nova-compute[75234]:       <mac address="fa:16:3e:6e:d5:4a"/>
Nov 29 03:16:53 user nova-compute[75234]:       <model type="virtio"/>
Nov 29 03:16:53 user nova-compute[75234]:       <mtu size="1442"/>
Nov 29 03:16:53 user nova-compute[75234]:       <target dev="tap69d76626-4c"/>
Nov 29 03:16:53 user nova-compute[75234]:     </interface>
Nov 29 03:16:53 user nova-compute[75234]:     <serial type="pty">
Nov 29 03:16:53 user nova-compute[75234]:       <log file="/opt/stack/data/nova/instances/5e6cca46-c5a7-4303-98c2-1ad483c6d14e/console.log" append="off"/>
Nov 29 03:16:53 user nova-compute[75234]:     </serial>
Nov 29 03:16:53 user nova-compute[75234]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Nov 29 03:16:53 user nova-compute[75234]:     <video>
Nov 29 03:16:53 user nova-compute[75234]:       <model type="virtio"/>
Nov 29 03:16:53 user nova-compute[75234]:     </video>
Nov 29 03:16:53 user nova-compute[75234]:     <rng model="virtio">
Nov 29 03:16:53 user nova-compute[75234]:       <backend model="random">/dev/urandom</backend>
Nov 29 03:16:53 user nova-compute[75234]:     </rng>
Nov 29 03:16:53 user nova-compute[75234]:     <controller type="usb" index="0" model="none"/>
Nov 29 03:16:53 user nova-compute[75234]:     <memballoon model="virtio">
Nov 29 03:16:53 user nova-compute[75234]:       <stats period="10"/>
Nov 29 03:16:53 user nova-compute[75234]:     </memballoon>
Nov 29 03:16:53 user nova-compute[75234]:   </devices>
Nov 29 03:16:53 user nova-compute[75234]: </domain>
Nov 29 03:16:53 user nova-compute[75234]:  {{(pid=75234) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7568}}
Nov 29 03:16:53 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-0f63ae7d-b42b-4e6d-a119-dcc8ba491b4d tempest-TestServerVolumeAttachScenarioOldVersion-830274084 tempest-TestServerVolumeAttachScenarioOldVersion-830274084-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-11-29T03:16:50Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-TestServerVolumeAttachScenarioOldVersion-server-816676743',display_name='tempest-TestServerVolumeAttachScenarioOldVersion-server-816676743',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-testservervolumeattachscenariooldversion-server-8166767',id=29,image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBKQV6WOnODZRu0S0Q8BjipN+d4SfGF+oFplEizu9978zfVugP8bBS3zBtCT9lFZz+6aJRHATnsP1lSYQtfjqOryzG+7rBhPRnAeY9RFQZR+EOYZl1GaC/iv/bQZS956elA==',key_name='tempest-keypair-526758182',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='d6f51212dd1c4632b36cfdbd7f901a89',ramdisk_id='',reservation_id='r-49dw65dl',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TestServerVolumeAttachScenarioOldVersion-830274084',owner_user_name='tempest-TestServerVolumeAttachScenarioOldVersion-830274084-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-11-29T03:16:51Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='68a6c98565d348fb8afabaddcd0b3f9d',uuid=5e6cca46-c5a7-4303-98c2-1ad483c6d14e,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "69d76626-4c79-4557-b501-7c8c92fcdc03", "address": "fa:16:3e:6e:d5:4a", "network": {"id": "f68acf2f-b01a-4481-845d-46b92646988f", "bridge": "br-int", "label": "tempest-TestServerVolumeAttachScenarioOldVersion-1316398304-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "d6f51212dd1c4632b36cfdbd7f901a89", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap69d76626-4c", "ovs_interfaceid": "69d76626-4c79-4557-b501-7c8c92fcdc03", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Nov 29 03:16:53 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-0f63ae7d-b42b-4e6d-a119-dcc8ba491b4d tempest-TestServerVolumeAttachScenarioOldVersion-830274084 tempest-TestServerVolumeAttachScenarioOldVersion-830274084-project-member] Converting VIF {"id": "69d76626-4c79-4557-b501-7c8c92fcdc03", "address": "fa:16:3e:6e:d5:4a", "network": {"id": "f68acf2f-b01a-4481-845d-46b92646988f", "bridge": "br-int", "label": "tempest-TestServerVolumeAttachScenarioOldVersion-1316398304-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "d6f51212dd1c4632b36cfdbd7f901a89", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap69d76626-4c", "ovs_interfaceid": "69d76626-4c79-4557-b501-7c8c92fcdc03", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:16:53 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-0f63ae7d-b42b-4e6d-a119-dcc8ba491b4d tempest-TestServerVolumeAttachScenarioOldVersion-830274084 tempest-TestServerVolumeAttachScenarioOldVersion-830274084-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:6e:d5:4a,bridge_name='br-int',has_traffic_filtering=True,id=69d76626-4c79-4557-b501-7c8c92fcdc03,network=Network(f68acf2f-b01a-4481-845d-46b92646988f),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap69d76626-4c') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:16:53 user nova-compute[75234]: DEBUG os_vif [None req-0f63ae7d-b42b-4e6d-a119-dcc8ba491b4d tempest-TestServerVolumeAttachScenarioOldVersion-830274084 tempest-TestServerVolumeAttachScenarioOldVersion-830274084-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:6e:d5:4a,bridge_name='br-int',has_traffic_filtering=True,id=69d76626-4c79-4557-b501-7c8c92fcdc03,network=Network(f68acf2f-b01a-4481-845d-46b92646988f),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap69d76626-4c') {{(pid=75234) plug /opt/stack/data/venv/lib/python3.10/site-packages/os_vif/__init__.py:76}}
Nov 29 03:16:53 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:16:53 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:16:53 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Nov 29 03:16:53 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:16:53 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbCreateCommand(_result=None, table=QoS, columns={'type': 'linux-noop', 'external_ids': {'id': '58eef977-05e2-5a07-a24b-90dfdf77d1df', '_type': 'linux-noop'}}, row=False) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:16:53 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:16:53 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:16:53 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:16:53 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tap69d76626-4c, may_exist=True, interface_attrs={}) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:16:53 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Port, record=tap69d76626-4c, col_values=(('qos', UUID('e3b80cda-4b28-4260-8427-60c589c9cd9b')),), if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:16:53 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DbSetCommand(_result=None, table=Interface, record=tap69d76626-4c, col_values=(('external_ids', {'iface-id': '69d76626-4c79-4557-b501-7c8c92fcdc03', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:6e:d5:4a', 'vm-uuid': '5e6cca46-c5a7-4303-98c2-1ad483c6d14e'}),), if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:16:53 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:16:53 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:16:53 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:16:53 user nova-compute[75234]: INFO os_vif [None req-0f63ae7d-b42b-4e6d-a119-dcc8ba491b4d tempest-TestServerVolumeAttachScenarioOldVersion-830274084 tempest-TestServerVolumeAttachScenarioOldVersion-830274084-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:6e:d5:4a,bridge_name='br-int',has_traffic_filtering=True,id=69d76626-4c79-4557-b501-7c8c92fcdc03,network=Network(f68acf2f-b01a-4481-845d-46b92646988f),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap69d76626-4c')
Nov 29 03:16:53 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-0f63ae7d-b42b-4e6d-a119-dcc8ba491b4d tempest-TestServerVolumeAttachScenarioOldVersion-830274084 tempest-TestServerVolumeAttachScenarioOldVersion-830274084-project-member] No BDM found with device name vda, not building metadata. {{(pid=75234) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12089}}
Nov 29 03:16:53 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-0f63ae7d-b42b-4e6d-a119-dcc8ba491b4d tempest-TestServerVolumeAttachScenarioOldVersion-830274084 tempest-TestServerVolumeAttachScenarioOldVersion-830274084-project-member] No VIF found with MAC fa:16:3e:6e:d5:4a, not building metadata {{(pid=75234) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Nov 29 03:16:53 user nova-compute[75234]: DEBUG nova.network.neutron [req-ecc9eb85-6ff4-4517-97c5-80a6b3bfcdee req-12353790-6dab-4a09-8af8-75ffa82bcf19 service nova] [instance: 5e6cca46-c5a7-4303-98c2-1ad483c6d14e] Updated VIF entry in instance network info cache for port 69d76626-4c79-4557-b501-7c8c92fcdc03. {{(pid=75234) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3481}}
Nov 29 03:16:53 user nova-compute[75234]: DEBUG nova.network.neutron [req-ecc9eb85-6ff4-4517-97c5-80a6b3bfcdee req-12353790-6dab-4a09-8af8-75ffa82bcf19 service nova] [instance: 5e6cca46-c5a7-4303-98c2-1ad483c6d14e] Updating instance_info_cache with network_info: [{"id": "69d76626-4c79-4557-b501-7c8c92fcdc03", "address": "fa:16:3e:6e:d5:4a", "network": {"id": "f68acf2f-b01a-4481-845d-46b92646988f", "bridge": "br-int", "label": "tempest-TestServerVolumeAttachScenarioOldVersion-1316398304-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "d6f51212dd1c4632b36cfdbd7f901a89", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap69d76626-4c", "ovs_interfaceid": "69d76626-4c79-4557-b501-7c8c92fcdc03", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:16:53 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-ecc9eb85-6ff4-4517-97c5-80a6b3bfcdee req-12353790-6dab-4a09-8af8-75ffa82bcf19 service nova] Releasing lock "refresh_cache-5e6cca46-c5a7-4303-98c2-1ad483c6d14e" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:16:54 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:16:54 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:16:54 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:16:54 user nova-compute[75234]: DEBUG nova.compute.manager [req-87dc2dd9-0ef3-4d32-9b57-73f15e12ce0f req-419c6707-03d7-4a29-9afb-3f34e06bfb07 service nova] [instance: 5e6cca46-c5a7-4303-98c2-1ad483c6d14e] Received event network-vif-plugged-69d76626-4c79-4557-b501-7c8c92fcdc03 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:16:54 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-87dc2dd9-0ef3-4d32-9b57-73f15e12ce0f req-419c6707-03d7-4a29-9afb-3f34e06bfb07 service nova] Acquiring lock "5e6cca46-c5a7-4303-98c2-1ad483c6d14e-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:16:55 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-87dc2dd9-0ef3-4d32-9b57-73f15e12ce0f req-419c6707-03d7-4a29-9afb-3f34e06bfb07 service nova] Lock "5e6cca46-c5a7-4303-98c2-1ad483c6d14e-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:16:55 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-87dc2dd9-0ef3-4d32-9b57-73f15e12ce0f req-419c6707-03d7-4a29-9afb-3f34e06bfb07 service nova] Lock "5e6cca46-c5a7-4303-98c2-1ad483c6d14e-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:16:55 user nova-compute[75234]: DEBUG nova.compute.manager [req-87dc2dd9-0ef3-4d32-9b57-73f15e12ce0f req-419c6707-03d7-4a29-9afb-3f34e06bfb07 service nova] [instance: 5e6cca46-c5a7-4303-98c2-1ad483c6d14e] No waiting events found dispatching network-vif-plugged-69d76626-4c79-4557-b501-7c8c92fcdc03 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:16:55 user nova-compute[75234]: WARNING nova.compute.manager [req-87dc2dd9-0ef3-4d32-9b57-73f15e12ce0f req-419c6707-03d7-4a29-9afb-3f34e06bfb07 service nova] [instance: 5e6cca46-c5a7-4303-98c2-1ad483c6d14e] Received unexpected event network-vif-plugged-69d76626-4c79-4557-b501-7c8c92fcdc03 for instance with vm_state building and task_state spawning.
Nov 29 03:16:55 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:16:55 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:16:56 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:16:56 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Starting heal instance info cache {{(pid=75234) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9922}}
Nov 29 03:16:56 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Acquiring lock "refresh_cache-d3b90f39-4b91-4152-b820-bd0e263eec89" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:16:56 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Acquired lock "refresh_cache-d3b90f39-4b91-4152-b820-bd0e263eec89" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:16:56 user nova-compute[75234]: DEBUG nova.network.neutron [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] [instance: d3b90f39-4b91-4152-b820-bd0e263eec89] Forcefully refreshing network info cache for instance {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2003}}
Nov 29 03:16:56 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:16:57 user nova-compute[75234]: DEBUG nova.compute.manager [req-7bf3c293-59d1-480d-900f-6f3a860a554e req-e86621d6-81a2-4ec6-917c-4e89cba54a52 service nova] [instance: 5e6cca46-c5a7-4303-98c2-1ad483c6d14e] Received event network-vif-plugged-69d76626-4c79-4557-b501-7c8c92fcdc03 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:16:57 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-7bf3c293-59d1-480d-900f-6f3a860a554e req-e86621d6-81a2-4ec6-917c-4e89cba54a52 service nova] Acquiring lock "5e6cca46-c5a7-4303-98c2-1ad483c6d14e-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:16:57 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-7bf3c293-59d1-480d-900f-6f3a860a554e req-e86621d6-81a2-4ec6-917c-4e89cba54a52 service nova] Lock "5e6cca46-c5a7-4303-98c2-1ad483c6d14e-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:16:57 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-7bf3c293-59d1-480d-900f-6f3a860a554e req-e86621d6-81a2-4ec6-917c-4e89cba54a52 service nova] Lock "5e6cca46-c5a7-4303-98c2-1ad483c6d14e-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:16:57 user nova-compute[75234]: DEBUG nova.compute.manager [req-7bf3c293-59d1-480d-900f-6f3a860a554e req-e86621d6-81a2-4ec6-917c-4e89cba54a52 service nova] [instance: 5e6cca46-c5a7-4303-98c2-1ad483c6d14e] No waiting events found dispatching network-vif-plugged-69d76626-4c79-4557-b501-7c8c92fcdc03 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:16:57 user nova-compute[75234]: WARNING nova.compute.manager [req-7bf3c293-59d1-480d-900f-6f3a860a554e req-e86621d6-81a2-4ec6-917c-4e89cba54a52 service nova] [instance: 5e6cca46-c5a7-4303-98c2-1ad483c6d14e] Received unexpected event network-vif-plugged-69d76626-4c79-4557-b501-7c8c92fcdc03 for instance with vm_state building and task_state spawning.
Nov 29 03:16:57 user nova-compute[75234]: DEBUG nova.compute.manager [None req-0f63ae7d-b42b-4e6d-a119-dcc8ba491b4d tempest-TestServerVolumeAttachScenarioOldVersion-830274084 tempest-TestServerVolumeAttachScenarioOldVersion-830274084-project-member] [instance: 5e6cca46-c5a7-4303-98c2-1ad483c6d14e] Instance event wait completed in 0 seconds for  {{(pid=75234) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Nov 29 03:16:57 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-0f63ae7d-b42b-4e6d-a119-dcc8ba491b4d tempest-TestServerVolumeAttachScenarioOldVersion-830274084 tempest-TestServerVolumeAttachScenarioOldVersion-830274084-project-member] [instance: 5e6cca46-c5a7-4303-98c2-1ad483c6d14e] Guest created on hypervisor {{(pid=75234) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4421}}
Nov 29 03:16:57 user nova-compute[75234]: INFO nova.virt.libvirt.driver [-] [instance: 5e6cca46-c5a7-4303-98c2-1ad483c6d14e] Instance spawned successfully.
Nov 29 03:16:57 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-0f63ae7d-b42b-4e6d-a119-dcc8ba491b4d tempest-TestServerVolumeAttachScenarioOldVersion-830274084 tempest-TestServerVolumeAttachScenarioOldVersion-830274084-project-member] [instance: 5e6cca46-c5a7-4303-98c2-1ad483c6d14e] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:904}}
Nov 29 03:16:57 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-0f63ae7d-b42b-4e6d-a119-dcc8ba491b4d tempest-TestServerVolumeAttachScenarioOldVersion-830274084 tempest-TestServerVolumeAttachScenarioOldVersion-830274084-project-member] [instance: 5e6cca46-c5a7-4303-98c2-1ad483c6d14e] Found default for hw_cdrom_bus of ide {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:16:57 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-0f63ae7d-b42b-4e6d-a119-dcc8ba491b4d tempest-TestServerVolumeAttachScenarioOldVersion-830274084 tempest-TestServerVolumeAttachScenarioOldVersion-830274084-project-member] [instance: 5e6cca46-c5a7-4303-98c2-1ad483c6d14e] Found default for hw_disk_bus of virtio {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:16:57 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-0f63ae7d-b42b-4e6d-a119-dcc8ba491b4d tempest-TestServerVolumeAttachScenarioOldVersion-830274084 tempest-TestServerVolumeAttachScenarioOldVersion-830274084-project-member] [instance: 5e6cca46-c5a7-4303-98c2-1ad483c6d14e] Found default for hw_input_bus of None {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:16:57 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-0f63ae7d-b42b-4e6d-a119-dcc8ba491b4d tempest-TestServerVolumeAttachScenarioOldVersion-830274084 tempest-TestServerVolumeAttachScenarioOldVersion-830274084-project-member] [instance: 5e6cca46-c5a7-4303-98c2-1ad483c6d14e] Found default for hw_pointer_model of None {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:16:57 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-0f63ae7d-b42b-4e6d-a119-dcc8ba491b4d tempest-TestServerVolumeAttachScenarioOldVersion-830274084 tempest-TestServerVolumeAttachScenarioOldVersion-830274084-project-member] [instance: 5e6cca46-c5a7-4303-98c2-1ad483c6d14e] Found default for hw_video_model of virtio {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:16:57 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-0f63ae7d-b42b-4e6d-a119-dcc8ba491b4d tempest-TestServerVolumeAttachScenarioOldVersion-830274084 tempest-TestServerVolumeAttachScenarioOldVersion-830274084-project-member] [instance: 5e6cca46-c5a7-4303-98c2-1ad483c6d14e] Found default for hw_vif_model of virtio {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:16:57 user nova-compute[75234]: INFO nova.compute.manager [None req-0f63ae7d-b42b-4e6d-a119-dcc8ba491b4d tempest-TestServerVolumeAttachScenarioOldVersion-830274084 tempest-TestServerVolumeAttachScenarioOldVersion-830274084-project-member] [instance: 5e6cca46-c5a7-4303-98c2-1ad483c6d14e] Took 5.68 seconds to spawn the instance on the hypervisor.
Nov 29 03:16:57 user nova-compute[75234]: DEBUG nova.compute.manager [None req-0f63ae7d-b42b-4e6d-a119-dcc8ba491b4d tempest-TestServerVolumeAttachScenarioOldVersion-830274084 tempest-TestServerVolumeAttachScenarioOldVersion-830274084-project-member] [instance: 5e6cca46-c5a7-4303-98c2-1ad483c6d14e] Checking state {{(pid=75234) _get_power_state /opt/stack/nova/nova/compute/manager.py:1782}}
Nov 29 03:16:57 user nova-compute[75234]: INFO nova.compute.manager [None req-0f63ae7d-b42b-4e6d-a119-dcc8ba491b4d tempest-TestServerVolumeAttachScenarioOldVersion-830274084 tempest-TestServerVolumeAttachScenarioOldVersion-830274084-project-member] [instance: 5e6cca46-c5a7-4303-98c2-1ad483c6d14e] Took 6.56 seconds to build instance.
Nov 29 03:16:57 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-0f63ae7d-b42b-4e6d-a119-dcc8ba491b4d tempest-TestServerVolumeAttachScenarioOldVersion-830274084 tempest-TestServerVolumeAttachScenarioOldVersion-830274084-project-member] Lock "5e6cca46-c5a7-4303-98c2-1ad483c6d14e" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 6.658s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:16:57 user nova-compute[75234]: DEBUG nova.network.neutron [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] [instance: d3b90f39-4b91-4152-b820-bd0e263eec89] Updating instance_info_cache with network_info: [{"id": "83eb52bb-5be5-49d6-81ec-86a21dde3df2", "address": "fa:16:3e:9d:70:dd", "network": {"id": "49d62bf6-34ee-4de8-ab85-6ef489d957c8", "bridge": "br-int", "label": "tempest-VolumesAdminNegativeTest-1304144113-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.107", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "5eaa91ebc2e64f2eba987f02bcd1a9f4", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap83eb52bb-5b", "ovs_interfaceid": "83eb52bb-5be5-49d6-81ec-86a21dde3df2", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:16:57 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Releasing lock "refresh_cache-d3b90f39-4b91-4152-b820-bd0e263eec89" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:16:57 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] [instance: d3b90f39-4b91-4152-b820-bd0e263eec89] Updated the network info_cache for instance {{(pid=75234) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9993}}
Nov 29 03:16:57 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:16:57 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:16:57 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:16:58 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:16:58 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:16:58 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=75234) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10541}}
Nov 29 03:16:58 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-52b1f7b3-04b2-4e15-b257-a3eb9a6f4ce9 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Acquiring lock "aaf1c3e3-b73e-42ad-87bd-c5255ec09b46" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:16:58 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-52b1f7b3-04b2-4e15-b257-a3eb9a6f4ce9 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Lock "aaf1c3e3-b73e-42ad-87bd-c5255ec09b46" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:16:58 user nova-compute[75234]: DEBUG nova.compute.manager [None req-52b1f7b3-04b2-4e15-b257-a3eb9a6f4ce9 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] [instance: aaf1c3e3-b73e-42ad-87bd-c5255ec09b46] Starting instance... {{(pid=75234) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2422}}
Nov 29 03:16:59 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-52b1f7b3-04b2-4e15-b257-a3eb9a6f4ce9 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:16:59 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-52b1f7b3-04b2-4e15-b257-a3eb9a6f4ce9 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:16:59 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-52b1f7b3-04b2-4e15-b257-a3eb9a6f4ce9 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=75234) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2366}}
Nov 29 03:16:59 user nova-compute[75234]: INFO nova.compute.claims [None req-52b1f7b3-04b2-4e15-b257-a3eb9a6f4ce9 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] [instance: aaf1c3e3-b73e-42ad-87bd-c5255ec09b46] Claim successful on node user
Nov 29 03:16:59 user nova-compute[75234]: DEBUG nova.compute.provider_tree [None req-52b1f7b3-04b2-4e15-b257-a3eb9a6f4ce9 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Inventory has not changed in ProviderTree for provider: 4d291220-7c37-4839-a0e3-b23c66502b74 {{(pid=75234) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Nov 29 03:16:59 user nova-compute[75234]: DEBUG nova.scheduler.client.report [None req-52b1f7b3-04b2-4e15-b257-a3eb9a6f4ce9 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Inventory has not changed for provider 4d291220-7c37-4839-a0e3-b23c66502b74 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=75234) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Nov 29 03:16:59 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-52b1f7b3-04b2-4e15-b257-a3eb9a6f4ce9 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.510s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:16:59 user nova-compute[75234]: DEBUG nova.compute.manager [None req-52b1f7b3-04b2-4e15-b257-a3eb9a6f4ce9 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] [instance: aaf1c3e3-b73e-42ad-87bd-c5255ec09b46] Start building networks asynchronously for instance. {{(pid=75234) _build_resources /opt/stack/nova/nova/compute/manager.py:2819}}
Nov 29 03:16:59 user nova-compute[75234]: DEBUG nova.compute.manager [None req-52b1f7b3-04b2-4e15-b257-a3eb9a6f4ce9 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] [instance: aaf1c3e3-b73e-42ad-87bd-c5255ec09b46] Allocating IP information in the background. {{(pid=75234) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1972}}
Nov 29 03:16:59 user nova-compute[75234]: DEBUG nova.network.neutron [None req-52b1f7b3-04b2-4e15-b257-a3eb9a6f4ce9 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] [instance: aaf1c3e3-b73e-42ad-87bd-c5255ec09b46] allocate_for_instance() {{(pid=75234) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1156}}
Nov 29 03:16:59 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-52b1f7b3-04b2-4e15-b257-a3eb9a6f4ce9 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] [instance: aaf1c3e3-b73e-42ad-87bd-c5255ec09b46] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Nov 29 03:16:59 user nova-compute[75234]: DEBUG nova.compute.manager [None req-52b1f7b3-04b2-4e15-b257-a3eb9a6f4ce9 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] [instance: aaf1c3e3-b73e-42ad-87bd-c5255ec09b46] Start building block device mappings for instance. {{(pid=75234) _build_resources /opt/stack/nova/nova/compute/manager.py:2854}}
Nov 29 03:16:59 user nova-compute[75234]: DEBUG nova.policy [None req-52b1f7b3-04b2-4e15-b257-a3eb9a6f4ce9 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': 'bb1126c998ee430baac2e2494f30c0aa', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e393d23e783b4a40986674d782606c69', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=75234) authorize /opt/stack/nova/nova/policy.py:203}}
Nov 29 03:16:59 user nova-compute[75234]: DEBUG nova.compute.manager [None req-52b1f7b3-04b2-4e15-b257-a3eb9a6f4ce9 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] [instance: aaf1c3e3-b73e-42ad-87bd-c5255ec09b46] Start spawning the instance on the hypervisor. {{(pid=75234) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2628}}
Nov 29 03:16:59 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-52b1f7b3-04b2-4e15-b257-a3eb9a6f4ce9 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] [instance: aaf1c3e3-b73e-42ad-87bd-c5255ec09b46] Creating instance directory {{(pid=75234) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4727}}
Nov 29 03:16:59 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-52b1f7b3-04b2-4e15-b257-a3eb9a6f4ce9 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] [instance: aaf1c3e3-b73e-42ad-87bd-c5255ec09b46] Creating image(s)
Nov 29 03:16:59 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-52b1f7b3-04b2-4e15-b257-a3eb9a6f4ce9 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Acquiring lock "/opt/stack/data/nova/instances/aaf1c3e3-b73e-42ad-87bd-c5255ec09b46/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:16:59 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-52b1f7b3-04b2-4e15-b257-a3eb9a6f4ce9 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Lock "/opt/stack/data/nova/instances/aaf1c3e3-b73e-42ad-87bd-c5255ec09b46/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:16:59 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-52b1f7b3-04b2-4e15-b257-a3eb9a6f4ce9 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Lock "/opt/stack/data/nova/instances/aaf1c3e3-b73e-42ad-87bd-c5255ec09b46/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:16:59 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-52b1f7b3-04b2-4e15-b257-a3eb9a6f4ce9 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:16:59 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-52b1f7b3-04b2-4e15-b257-a3eb9a6f4ce9 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json" returned: 0 in 0.184s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:16:59 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-52b1f7b3-04b2-4e15-b257-a3eb9a6f4ce9 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Acquiring lock "80686e29dc9db6600d03e85cb62af298e12d7918" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:16:59 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-52b1f7b3-04b2-4e15-b257-a3eb9a6f4ce9 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Lock "80686e29dc9db6600d03e85cb62af298e12d7918" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.002s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:16:59 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-52b1f7b3-04b2-4e15-b257-a3eb9a6f4ce9 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:17:00 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-52b1f7b3-04b2-4e15-b257-a3eb9a6f4ce9 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json" returned: 0 in 0.195s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:17:00 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-52b1f7b3-04b2-4e15-b257-a3eb9a6f4ce9 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918,backing_fmt=raw /opt/stack/data/nova/instances/aaf1c3e3-b73e-42ad-87bd-c5255ec09b46/disk 1073741824 {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:17:00 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-52b1f7b3-04b2-4e15-b257-a3eb9a6f4ce9 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918,backing_fmt=raw /opt/stack/data/nova/instances/aaf1c3e3-b73e-42ad-87bd-c5255ec09b46/disk 1073741824" returned: 0 in 0.062s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:17:00 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-52b1f7b3-04b2-4e15-b257-a3eb9a6f4ce9 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Lock "80686e29dc9db6600d03e85cb62af298e12d7918" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.266s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:17:00 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-52b1f7b3-04b2-4e15-b257-a3eb9a6f4ce9 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:17:00 user nova-compute[75234]: DEBUG nova.network.neutron [None req-52b1f7b3-04b2-4e15-b257-a3eb9a6f4ce9 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] [instance: aaf1c3e3-b73e-42ad-87bd-c5255ec09b46] Successfully created port: 0013653c-9540-4049-91a5-43bfba2d8fa3 {{(pid=75234) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:548}}
Nov 29 03:17:00 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-52b1f7b3-04b2-4e15-b257-a3eb9a6f4ce9 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json" returned: 0 in 0.224s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:17:00 user nova-compute[75234]: DEBUG nova.virt.disk.api [None req-52b1f7b3-04b2-4e15-b257-a3eb9a6f4ce9 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Checking if we can resize image /opt/stack/data/nova/instances/aaf1c3e3-b73e-42ad-87bd-c5255ec09b46/disk. size=1073741824 {{(pid=75234) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:166}}
Nov 29 03:17:00 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-52b1f7b3-04b2-4e15-b257-a3eb9a6f4ce9 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/aaf1c3e3-b73e-42ad-87bd-c5255ec09b46/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:17:00 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:17:00 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager.update_available_resource {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:17:00 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:17:00 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:17:00 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:17:00 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Auditing locally available compute resources for user (node: user) {{(pid=75234) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:907}}
Nov 29 03:17:00 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-52b1f7b3-04b2-4e15-b257-a3eb9a6f4ce9 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/aaf1c3e3-b73e-42ad-87bd-c5255ec09b46/disk --force-share --output=json" returned: 0 in 0.209s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:17:00 user nova-compute[75234]: DEBUG nova.virt.disk.api [None req-52b1f7b3-04b2-4e15-b257-a3eb9a6f4ce9 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Cannot resize image /opt/stack/data/nova/instances/aaf1c3e3-b73e-42ad-87bd-c5255ec09b46/disk to a smaller size. {{(pid=75234) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:172}}
Nov 29 03:17:00 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-52b1f7b3-04b2-4e15-b257-a3eb9a6f4ce9 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] [instance: aaf1c3e3-b73e-42ad-87bd-c5255ec09b46] Created local disks {{(pid=75234) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4861}}
Nov 29 03:17:00 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-52b1f7b3-04b2-4e15-b257-a3eb9a6f4ce9 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] [instance: aaf1c3e3-b73e-42ad-87bd-c5255ec09b46] Ensure instance console log exists: /opt/stack/data/nova/instances/aaf1c3e3-b73e-42ad-87bd-c5255ec09b46/console.log {{(pid=75234) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4613}}
Nov 29 03:17:00 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-52b1f7b3-04b2-4e15-b257-a3eb9a6f4ce9 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:17:00 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-52b1f7b3-04b2-4e15-b257-a3eb9a6f4ce9 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:17:00 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-52b1f7b3-04b2-4e15-b257-a3eb9a6f4ce9 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:17:00 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/ec7cf7cd-8e48-4bb3-a969-e14db705dc09/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:17:01 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/ec7cf7cd-8e48-4bb3-a969-e14db705dc09/disk --force-share --output=json" returned: 0 in 0.234s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:17:01 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/ec7cf7cd-8e48-4bb3-a969-e14db705dc09/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:17:01 user nova-compute[75234]: DEBUG nova.network.neutron [None req-52b1f7b3-04b2-4e15-b257-a3eb9a6f4ce9 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] [instance: aaf1c3e3-b73e-42ad-87bd-c5255ec09b46] Successfully updated port: 0013653c-9540-4049-91a5-43bfba2d8fa3 {{(pid=75234) _update_port /opt/stack/nova/nova/network/neutron.py:586}}
Nov 29 03:17:01 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-52b1f7b3-04b2-4e15-b257-a3eb9a6f4ce9 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Acquiring lock "refresh_cache-aaf1c3e3-b73e-42ad-87bd-c5255ec09b46" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:17:01 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-52b1f7b3-04b2-4e15-b257-a3eb9a6f4ce9 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Acquired lock "refresh_cache-aaf1c3e3-b73e-42ad-87bd-c5255ec09b46" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:17:01 user nova-compute[75234]: DEBUG nova.network.neutron [None req-52b1f7b3-04b2-4e15-b257-a3eb9a6f4ce9 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] [instance: aaf1c3e3-b73e-42ad-87bd-c5255ec09b46] Building network info cache for instance {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2009}}
Nov 29 03:17:01 user nova-compute[75234]: DEBUG nova.compute.manager [req-10bfad7a-9cfa-4389-886c-28710edaefc7 req-1b2f7807-4a14-4cc6-aaae-67ff8a4c6a41 service nova] [instance: aaf1c3e3-b73e-42ad-87bd-c5255ec09b46] Received event network-changed-0013653c-9540-4049-91a5-43bfba2d8fa3 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:17:01 user nova-compute[75234]: DEBUG nova.compute.manager [req-10bfad7a-9cfa-4389-886c-28710edaefc7 req-1b2f7807-4a14-4cc6-aaae-67ff8a4c6a41 service nova] [instance: aaf1c3e3-b73e-42ad-87bd-c5255ec09b46] Refreshing instance network info cache due to event network-changed-0013653c-9540-4049-91a5-43bfba2d8fa3. {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11117}}
Nov 29 03:17:01 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-10bfad7a-9cfa-4389-886c-28710edaefc7 req-1b2f7807-4a14-4cc6-aaae-67ff8a4c6a41 service nova] Acquiring lock "refresh_cache-aaf1c3e3-b73e-42ad-87bd-c5255ec09b46" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:17:01 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/ec7cf7cd-8e48-4bb3-a969-e14db705dc09/disk --force-share --output=json" returned: 0 in 0.195s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:17:01 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/75c5d700-6367-4128-9228-80d2cd24ace7/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:17:01 user nova-compute[75234]: DEBUG nova.network.neutron [None req-52b1f7b3-04b2-4e15-b257-a3eb9a6f4ce9 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] [instance: aaf1c3e3-b73e-42ad-87bd-c5255ec09b46] Instance cache missing network info. {{(pid=75234) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3322}}
Nov 29 03:17:01 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/75c5d700-6367-4128-9228-80d2cd24ace7/disk --force-share --output=json" returned: 0 in 0.175s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:17:01 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/75c5d700-6367-4128-9228-80d2cd24ace7/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:17:01 user nova-compute[75234]: DEBUG nova.network.neutron [None req-52b1f7b3-04b2-4e15-b257-a3eb9a6f4ce9 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] [instance: aaf1c3e3-b73e-42ad-87bd-c5255ec09b46] Updating instance_info_cache with network_info: [{"id": "0013653c-9540-4049-91a5-43bfba2d8fa3", "address": "fa:16:3e:55:a5:13", "network": {"id": "294c3aac-8c9b-45f0-8c0a-344e2d7d8b7c", "bridge": "br-int", "label": "tempest-AttachVolumeTestJSON-700098935-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "e393d23e783b4a40986674d782606c69", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap0013653c-95", "ovs_interfaceid": "0013653c-9540-4049-91a5-43bfba2d8fa3", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:17:01 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-52b1f7b3-04b2-4e15-b257-a3eb9a6f4ce9 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Releasing lock "refresh_cache-aaf1c3e3-b73e-42ad-87bd-c5255ec09b46" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:17:01 user nova-compute[75234]: DEBUG nova.compute.manager [None req-52b1f7b3-04b2-4e15-b257-a3eb9a6f4ce9 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] [instance: aaf1c3e3-b73e-42ad-87bd-c5255ec09b46] Instance network_info: |[{"id": "0013653c-9540-4049-91a5-43bfba2d8fa3", "address": "fa:16:3e:55:a5:13", "network": {"id": "294c3aac-8c9b-45f0-8c0a-344e2d7d8b7c", "bridge": "br-int", "label": "tempest-AttachVolumeTestJSON-700098935-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "e393d23e783b4a40986674d782606c69", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap0013653c-95", "ovs_interfaceid": "0013653c-9540-4049-91a5-43bfba2d8fa3", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=75234) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1987}}
Nov 29 03:17:01 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-10bfad7a-9cfa-4389-886c-28710edaefc7 req-1b2f7807-4a14-4cc6-aaae-67ff8a4c6a41 service nova] Acquired lock "refresh_cache-aaf1c3e3-b73e-42ad-87bd-c5255ec09b46" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:17:01 user nova-compute[75234]: DEBUG nova.network.neutron [req-10bfad7a-9cfa-4389-886c-28710edaefc7 req-1b2f7807-4a14-4cc6-aaae-67ff8a4c6a41 service nova] [instance: aaf1c3e3-b73e-42ad-87bd-c5255ec09b46] Refreshing network info cache for port 0013653c-9540-4049-91a5-43bfba2d8fa3 {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2006}}
Nov 29 03:17:01 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-52b1f7b3-04b2-4e15-b257-a3eb9a6f4ce9 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] [instance: aaf1c3e3-b73e-42ad-87bd-c5255ec09b46] Start _get_guest_xml network_info=[{"id": "0013653c-9540-4049-91a5-43bfba2d8fa3", "address": "fa:16:3e:55:a5:13", "network": {"id": "294c3aac-8c9b-45f0-8c0a-344e2d7d8b7c", "bridge": "br-int", "label": "tempest-AttachVolumeTestJSON-700098935-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "e393d23e783b4a40986674d782606c69", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap0013653c-95", "ovs_interfaceid": "0013653c-9540-4049-91a5-43bfba2d8fa3", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum='c8fc807773e5354afe61636071771906',container_format='bare',created_at=2023-11-29T03:02:15Z,direct_url=<?>,disk_format='qcow2',id=96c30ba7-9b73-4a30-8ba4-7138879573b7,min_disk=0,min_ram=0,name='cirros-0.6.2-x86_64-disk',owner='b18e380921c748b78517b6f1211cd0f1',properties=ImageMetaProps,protected=<?>,size=21430272,status='active',tags=<?>,updated_at=2023-11-29T03:02:17Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [{'device_name': '/dev/vda', 'encryption_format': None, 'encrypted': False, 'device_type': 'disk', 'size': 0, 'disk_bus': 'virtio', 'boot_index': 0, 'encryption_options': None, 'encryption_secret_uuid': None, 'guest_format': None, 'image_id': '96c30ba7-9b73-4a30-8ba4-7138879573b7'}], 'ephemerals': [], 'block_device_mapping': [], 'swap': None} {{(pid=75234) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7562}}
Nov 29 03:17:01 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/75c5d700-6367-4128-9228-80d2cd24ace7/disk --force-share --output=json" returned: 0 in 0.199s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:17:01 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] skipping disk /dev/sdb (vda) as it is a volume {{(pid=75234) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11213}}
Nov 29 03:17:01 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-52b1f7b3-04b2-4e15-b257-a3eb9a6f4ce9 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:17:01 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-52b1f7b3-04b2-4e15-b257-a3eb9a6f4ce9 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:17:01 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a082c96e-e384-40b6-bcc8-fc5e0e104669/disk.rescue --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:17:01 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-52b1f7b3-04b2-4e15-b257-a3eb9a6f4ce9 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Searching host: 'user' for CPU controller through CGroups V1... {{(pid=75234) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1652}}
Nov 29 03:17:01 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-52b1f7b3-04b2-4e15-b257-a3eb9a6f4ce9 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] CPU controller missing on host. {{(pid=75234) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1662}}
Nov 29 03:17:01 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-52b1f7b3-04b2-4e15-b257-a3eb9a6f4ce9 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Searching host: 'user' for CPU controller through CGroups V2... {{(pid=75234) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1671}}
Nov 29 03:17:01 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-52b1f7b3-04b2-4e15-b257-a3eb9a6f4ce9 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] CPU controller found on host. {{(pid=75234) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1678}}
Nov 29 03:17:01 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-52b1f7b3-04b2-4e15-b257-a3eb9a6f4ce9 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=75234) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5400}}
Nov 29 03:17:01 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-52b1f7b3-04b2-4e15-b257-a3eb9a6f4ce9 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-11-29T03:02:04Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='1',id=6,is_public=True,memory_mb=512,name='m1.tiny',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='c8fc807773e5354afe61636071771906',container_format='bare',created_at=2023-11-29T03:02:15Z,direct_url=<?>,disk_format='qcow2',id=96c30ba7-9b73-4a30-8ba4-7138879573b7,min_disk=0,min_ram=0,name='cirros-0.6.2-x86_64-disk',owner='b18e380921c748b78517b6f1211cd0f1',properties=ImageMetaProps,protected=<?>,size=21430272,status='active',tags=<?>,updated_at=2023-11-29T03:02:17Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:563}}
Nov 29 03:17:01 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-52b1f7b3-04b2-4e15-b257-a3eb9a6f4ce9 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Flavor limits 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:348}}
Nov 29 03:17:01 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-52b1f7b3-04b2-4e15-b257-a3eb9a6f4ce9 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Image limits 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:352}}
Nov 29 03:17:01 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-52b1f7b3-04b2-4e15-b257-a3eb9a6f4ce9 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Flavor pref 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:388}}
Nov 29 03:17:01 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-52b1f7b3-04b2-4e15-b257-a3eb9a6f4ce9 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Image pref 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:392}}
Nov 29 03:17:01 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-52b1f7b3-04b2-4e15-b257-a3eb9a6f4ce9 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:430}}
Nov 29 03:17:01 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-52b1f7b3-04b2-4e15-b257-a3eb9a6f4ce9 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:569}}
Nov 29 03:17:01 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-52b1f7b3-04b2-4e15-b257-a3eb9a6f4ce9 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=75234) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:471}}
Nov 29 03:17:01 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-52b1f7b3-04b2-4e15-b257-a3eb9a6f4ce9 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Got 1 possible topologies {{(pid=75234) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:501}}
Nov 29 03:17:01 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-52b1f7b3-04b2-4e15-b257-a3eb9a6f4ce9 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:575}}
Nov 29 03:17:01 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-52b1f7b3-04b2-4e15-b257-a3eb9a6f4ce9 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:577}}
Nov 29 03:17:01 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-52b1f7b3-04b2-4e15-b257-a3eb9a6f4ce9 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-11-29T03:16:58Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-AttachVolumeTestJSON-server-1923356075',display_name='tempest-AttachVolumeTestJSON-server-1923356075',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-attachvolumetestjson-server-1923356075',id=30,image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBOVttVI5swLnmUGLH6ytI05vQQ9gCXIHGezkcU9BBKe3iu2kjwDJGQh46ZdqIMONVOydyYiZFs3183QkjyTIDlELaxy8iT14txRIwVua8qDH79nvvJMG4vqm5zaxFrPXxA==',key_name='tempest-keypair-2054417748',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='e393d23e783b4a40986674d782606c69',ramdisk_id='',reservation_id='r-pfek0fzf',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-AttachVolumeTestJSON-1911423306',owner_user_name='tempest-AttachVolumeTestJSON-1911423306-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-11-29T03:17:00Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='bb1126c998ee430baac2e2494f30c0aa',uuid=aaf1c3e3-b73e-42ad-87bd-c5255ec09b46,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "0013653c-9540-4049-91a5-43bfba2d8fa3", "address": "fa:16:3e:55:a5:13", "network": {"id": "294c3aac-8c9b-45f0-8c0a-344e2d7d8b7c", "bridge": "br-int", "label": "tempest-AttachVolumeTestJSON-700098935-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "e393d23e783b4a40986674d782606c69", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap0013653c-95", "ovs_interfaceid": "0013653c-9540-4049-91a5-43bfba2d8fa3", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=75234) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Nov 29 03:17:01 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-52b1f7b3-04b2-4e15-b257-a3eb9a6f4ce9 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Converting VIF {"id": "0013653c-9540-4049-91a5-43bfba2d8fa3", "address": "fa:16:3e:55:a5:13", "network": {"id": "294c3aac-8c9b-45f0-8c0a-344e2d7d8b7c", "bridge": "br-int", "label": "tempest-AttachVolumeTestJSON-700098935-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "e393d23e783b4a40986674d782606c69", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap0013653c-95", "ovs_interfaceid": "0013653c-9540-4049-91a5-43bfba2d8fa3", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:17:01 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-52b1f7b3-04b2-4e15-b257-a3eb9a6f4ce9 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:55:a5:13,bridge_name='br-int',has_traffic_filtering=True,id=0013653c-9540-4049-91a5-43bfba2d8fa3,network=Network(294c3aac-8c9b-45f0-8c0a-344e2d7d8b7c),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap0013653c-95') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:17:01 user nova-compute[75234]: DEBUG nova.objects.instance [None req-52b1f7b3-04b2-4e15-b257-a3eb9a6f4ce9 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Lazy-loading 'pci_devices' on Instance uuid aaf1c3e3-b73e-42ad-87bd-c5255ec09b46 {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:17:01 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-52b1f7b3-04b2-4e15-b257-a3eb9a6f4ce9 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] [instance: aaf1c3e3-b73e-42ad-87bd-c5255ec09b46] End _get_guest_xml xml=<domain type="kvm">
Nov 29 03:17:01 user nova-compute[75234]:   <uuid>aaf1c3e3-b73e-42ad-87bd-c5255ec09b46</uuid>
Nov 29 03:17:01 user nova-compute[75234]:   <name>instance-0000001e</name>
Nov 29 03:17:01 user nova-compute[75234]:   <memory>524288</memory>
Nov 29 03:17:01 user nova-compute[75234]:   <vcpu>1</vcpu>
Nov 29 03:17:01 user nova-compute[75234]:   <metadata>
Nov 29 03:17:01 user nova-compute[75234]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Nov 29 03:17:01 user nova-compute[75234]:       <nova:package version="0.0.0"/>
Nov 29 03:17:01 user nova-compute[75234]:       <nova:name>tempest-AttachVolumeTestJSON-server-1923356075</nova:name>
Nov 29 03:17:01 user nova-compute[75234]:       <nova:creationTime>2023-11-29 03:17:01</nova:creationTime>
Nov 29 03:17:01 user nova-compute[75234]:       <nova:flavor name="m1.tiny">
Nov 29 03:17:01 user nova-compute[75234]:         <nova:memory>512</nova:memory>
Nov 29 03:17:01 user nova-compute[75234]:         <nova:disk>1</nova:disk>
Nov 29 03:17:01 user nova-compute[75234]:         <nova:swap>0</nova:swap>
Nov 29 03:17:01 user nova-compute[75234]:         <nova:ephemeral>0</nova:ephemeral>
Nov 29 03:17:01 user nova-compute[75234]:         <nova:vcpus>1</nova:vcpus>
Nov 29 03:17:01 user nova-compute[75234]:       </nova:flavor>
Nov 29 03:17:01 user nova-compute[75234]:       <nova:owner>
Nov 29 03:17:01 user nova-compute[75234]:         <nova:user uuid="bb1126c998ee430baac2e2494f30c0aa">tempest-AttachVolumeTestJSON-1911423306-project-member</nova:user>
Nov 29 03:17:01 user nova-compute[75234]:         <nova:project uuid="e393d23e783b4a40986674d782606c69">tempest-AttachVolumeTestJSON-1911423306</nova:project>
Nov 29 03:17:01 user nova-compute[75234]:       </nova:owner>
Nov 29 03:17:01 user nova-compute[75234]:       <nova:root type="image" uuid="96c30ba7-9b73-4a30-8ba4-7138879573b7"/>
Nov 29 03:17:01 user nova-compute[75234]:       <nova:ports>
Nov 29 03:17:01 user nova-compute[75234]:         <nova:port uuid="0013653c-9540-4049-91a5-43bfba2d8fa3">
Nov 29 03:17:01 user nova-compute[75234]:           <nova:ip type="fixed" address="10.0.0.6" ipVersion="4"/>
Nov 29 03:17:01 user nova-compute[75234]:         </nova:port>
Nov 29 03:17:01 user nova-compute[75234]:       </nova:ports>
Nov 29 03:17:01 user nova-compute[75234]:     </nova:instance>
Nov 29 03:17:01 user nova-compute[75234]:   </metadata>
Nov 29 03:17:01 user nova-compute[75234]:   <sysinfo type="smbios">
Nov 29 03:17:01 user nova-compute[75234]:     <system>
Nov 29 03:17:01 user nova-compute[75234]:       <entry name="manufacturer">OpenStack Foundation</entry>
Nov 29 03:17:01 user nova-compute[75234]:       <entry name="product">OpenStack Nova</entry>
Nov 29 03:17:01 user nova-compute[75234]:       <entry name="version">0.0.0</entry>
Nov 29 03:17:01 user nova-compute[75234]:       <entry name="serial">aaf1c3e3-b73e-42ad-87bd-c5255ec09b46</entry>
Nov 29 03:17:01 user nova-compute[75234]:       <entry name="uuid">aaf1c3e3-b73e-42ad-87bd-c5255ec09b46</entry>
Nov 29 03:17:01 user nova-compute[75234]:       <entry name="family">Virtual Machine</entry>
Nov 29 03:17:01 user nova-compute[75234]:     </system>
Nov 29 03:17:01 user nova-compute[75234]:   </sysinfo>
Nov 29 03:17:01 user nova-compute[75234]:   <os>
Nov 29 03:17:01 user nova-compute[75234]:     <type arch="x86_64" machine="pc">hvm</type>
Nov 29 03:17:01 user nova-compute[75234]:     <boot dev="hd"/>
Nov 29 03:17:01 user nova-compute[75234]:     <smbios mode="sysinfo"/>
Nov 29 03:17:01 user nova-compute[75234]:   </os>
Nov 29 03:17:01 user nova-compute[75234]:   <features>
Nov 29 03:17:01 user nova-compute[75234]:     <acpi/>
Nov 29 03:17:01 user nova-compute[75234]:     <apic/>
Nov 29 03:17:01 user nova-compute[75234]:     <vmcoreinfo/>
Nov 29 03:17:01 user nova-compute[75234]:   </features>
Nov 29 03:17:01 user nova-compute[75234]:   <clock offset="utc">
Nov 29 03:17:01 user nova-compute[75234]:     <timer name="pit" tickpolicy="delay"/>
Nov 29 03:17:01 user nova-compute[75234]:     <timer name="rtc" tickpolicy="catchup"/>
Nov 29 03:17:01 user nova-compute[75234]:     <timer name="hpet" present="no"/>
Nov 29 03:17:01 user nova-compute[75234]:   </clock>
Nov 29 03:17:01 user nova-compute[75234]:   <cpu mode="custom" match="exact">
Nov 29 03:17:01 user nova-compute[75234]:     <model>Nehalem</model>
Nov 29 03:17:01 user nova-compute[75234]:     <topology sockets="1" cores="1" threads="1"/>
Nov 29 03:17:01 user nova-compute[75234]:   </cpu>
Nov 29 03:17:01 user nova-compute[75234]:   <devices>
Nov 29 03:17:01 user nova-compute[75234]:     <disk type="file" device="disk">
Nov 29 03:17:01 user nova-compute[75234]:       <driver name="qemu" type="qcow2" cache="none"/>
Nov 29 03:17:01 user nova-compute[75234]:       <source file="/opt/stack/data/nova/instances/aaf1c3e3-b73e-42ad-87bd-c5255ec09b46/disk"/>
Nov 29 03:17:01 user nova-compute[75234]:       <target dev="vda" bus="virtio"/>
Nov 29 03:17:01 user nova-compute[75234]:     </disk>
Nov 29 03:17:01 user nova-compute[75234]:     <interface type="ethernet">
Nov 29 03:17:01 user nova-compute[75234]:       <mac address="fa:16:3e:55:a5:13"/>
Nov 29 03:17:01 user nova-compute[75234]:       <model type="virtio"/>
Nov 29 03:17:01 user nova-compute[75234]:       <mtu size="1442"/>
Nov 29 03:17:01 user nova-compute[75234]:       <target dev="tap0013653c-95"/>
Nov 29 03:17:01 user nova-compute[75234]:     </interface>
Nov 29 03:17:01 user nova-compute[75234]:     <serial type="pty">
Nov 29 03:17:01 user nova-compute[75234]:       <log file="/opt/stack/data/nova/instances/aaf1c3e3-b73e-42ad-87bd-c5255ec09b46/console.log" append="off"/>
Nov 29 03:17:01 user nova-compute[75234]:     </serial>
Nov 29 03:17:01 user nova-compute[75234]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Nov 29 03:17:01 user nova-compute[75234]:     <video>
Nov 29 03:17:01 user nova-compute[75234]:       <model type="virtio"/>
Nov 29 03:17:01 user nova-compute[75234]:     </video>
Nov 29 03:17:01 user nova-compute[75234]:     <rng model="virtio">
Nov 29 03:17:01 user nova-compute[75234]:       <backend model="random">/dev/urandom</backend>
Nov 29 03:17:01 user nova-compute[75234]:     </rng>
Nov 29 03:17:01 user nova-compute[75234]:     <controller type="usb" index="0" model="none"/>
Nov 29 03:17:01 user nova-compute[75234]:     <memballoon model="virtio">
Nov 29 03:17:01 user nova-compute[75234]:       <stats period="10"/>
Nov 29 03:17:01 user nova-compute[75234]:     </memballoon>
Nov 29 03:17:01 user nova-compute[75234]:   </devices>
Nov 29 03:17:01 user nova-compute[75234]: </domain>
Nov 29 03:17:01 user nova-compute[75234]:  {{(pid=75234) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7568}}
Nov 29 03:17:01 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-52b1f7b3-04b2-4e15-b257-a3eb9a6f4ce9 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-11-29T03:16:58Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-AttachVolumeTestJSON-server-1923356075',display_name='tempest-AttachVolumeTestJSON-server-1923356075',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-attachvolumetestjson-server-1923356075',id=30,image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBOVttVI5swLnmUGLH6ytI05vQQ9gCXIHGezkcU9BBKe3iu2kjwDJGQh46ZdqIMONVOydyYiZFs3183QkjyTIDlELaxy8iT14txRIwVua8qDH79nvvJMG4vqm5zaxFrPXxA==',key_name='tempest-keypair-2054417748',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='e393d23e783b4a40986674d782606c69',ramdisk_id='',reservation_id='r-pfek0fzf',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-AttachVolumeTestJSON-1911423306',owner_user_name='tempest-AttachVolumeTestJSON-1911423306-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-11-29T03:17:00Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='bb1126c998ee430baac2e2494f30c0aa',uuid=aaf1c3e3-b73e-42ad-87bd-c5255ec09b46,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "0013653c-9540-4049-91a5-43bfba2d8fa3", "address": "fa:16:3e:55:a5:13", "network": {"id": "294c3aac-8c9b-45f0-8c0a-344e2d7d8b7c", "bridge": "br-int", "label": "tempest-AttachVolumeTestJSON-700098935-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "e393d23e783b4a40986674d782606c69", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap0013653c-95", "ovs_interfaceid": "0013653c-9540-4049-91a5-43bfba2d8fa3", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Nov 29 03:17:01 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-52b1f7b3-04b2-4e15-b257-a3eb9a6f4ce9 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Converting VIF {"id": "0013653c-9540-4049-91a5-43bfba2d8fa3", "address": "fa:16:3e:55:a5:13", "network": {"id": "294c3aac-8c9b-45f0-8c0a-344e2d7d8b7c", "bridge": "br-int", "label": "tempest-AttachVolumeTestJSON-700098935-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "e393d23e783b4a40986674d782606c69", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap0013653c-95", "ovs_interfaceid": "0013653c-9540-4049-91a5-43bfba2d8fa3", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:17:01 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-52b1f7b3-04b2-4e15-b257-a3eb9a6f4ce9 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:55:a5:13,bridge_name='br-int',has_traffic_filtering=True,id=0013653c-9540-4049-91a5-43bfba2d8fa3,network=Network(294c3aac-8c9b-45f0-8c0a-344e2d7d8b7c),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap0013653c-95') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:17:01 user nova-compute[75234]: DEBUG os_vif [None req-52b1f7b3-04b2-4e15-b257-a3eb9a6f4ce9 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:55:a5:13,bridge_name='br-int',has_traffic_filtering=True,id=0013653c-9540-4049-91a5-43bfba2d8fa3,network=Network(294c3aac-8c9b-45f0-8c0a-344e2d7d8b7c),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap0013653c-95') {{(pid=75234) plug /opt/stack/data/venv/lib/python3.10/site-packages/os_vif/__init__.py:76}}
Nov 29 03:17:01 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:17:01 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:17:01 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Nov 29 03:17:01 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:17:01 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbCreateCommand(_result=None, table=QoS, columns={'type': 'linux-noop', 'external_ids': {'id': 'f44f6fff-d6e4-5346-8544-46c9c2e2b587', '_type': 'linux-noop'}}, row=False) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:17:01 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:17:01 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:17:01 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:17:01 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tap0013653c-95, may_exist=True, interface_attrs={}) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:17:01 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Port, record=tap0013653c-95, col_values=(('qos', UUID('f10fccc7-c33a-422a-a9b5-1c23592c935c')),), if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:17:01 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DbSetCommand(_result=None, table=Interface, record=tap0013653c-95, col_values=(('external_ids', {'iface-id': '0013653c-9540-4049-91a5-43bfba2d8fa3', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:55:a5:13', 'vm-uuid': 'aaf1c3e3-b73e-42ad-87bd-c5255ec09b46'}),), if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:17:01 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:17:01 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:17:01 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:17:01 user nova-compute[75234]: INFO os_vif [None req-52b1f7b3-04b2-4e15-b257-a3eb9a6f4ce9 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:55:a5:13,bridge_name='br-int',has_traffic_filtering=True,id=0013653c-9540-4049-91a5-43bfba2d8fa3,network=Network(294c3aac-8c9b-45f0-8c0a-344e2d7d8b7c),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap0013653c-95')
Nov 29 03:17:01 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:17:01 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a082c96e-e384-40b6-bcc8-fc5e0e104669/disk.rescue --force-share --output=json" returned: 0 in 0.210s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:17:01 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a082c96e-e384-40b6-bcc8-fc5e0e104669/disk.rescue --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:17:01 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-52b1f7b3-04b2-4e15-b257-a3eb9a6f4ce9 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] No BDM found with device name vda, not building metadata. {{(pid=75234) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12089}}
Nov 29 03:17:01 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-52b1f7b3-04b2-4e15-b257-a3eb9a6f4ce9 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] No VIF found with MAC fa:16:3e:55:a5:13, not building metadata {{(pid=75234) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Nov 29 03:17:02 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a082c96e-e384-40b6-bcc8-fc5e0e104669/disk.rescue --force-share --output=json" returned: 0 in 0.198s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:17:02 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a082c96e-e384-40b6-bcc8-fc5e0e104669/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:17:02 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a082c96e-e384-40b6-bcc8-fc5e0e104669/disk --force-share --output=json" returned: 0 in 0.186s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:17:02 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a082c96e-e384-40b6-bcc8-fc5e0e104669/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:17:02 user nova-compute[75234]: DEBUG nova.network.neutron [req-10bfad7a-9cfa-4389-886c-28710edaefc7 req-1b2f7807-4a14-4cc6-aaae-67ff8a4c6a41 service nova] [instance: aaf1c3e3-b73e-42ad-87bd-c5255ec09b46] Updated VIF entry in instance network info cache for port 0013653c-9540-4049-91a5-43bfba2d8fa3. {{(pid=75234) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3481}}
Nov 29 03:17:02 user nova-compute[75234]: DEBUG nova.network.neutron [req-10bfad7a-9cfa-4389-886c-28710edaefc7 req-1b2f7807-4a14-4cc6-aaae-67ff8a4c6a41 service nova] [instance: aaf1c3e3-b73e-42ad-87bd-c5255ec09b46] Updating instance_info_cache with network_info: [{"id": "0013653c-9540-4049-91a5-43bfba2d8fa3", "address": "fa:16:3e:55:a5:13", "network": {"id": "294c3aac-8c9b-45f0-8c0a-344e2d7d8b7c", "bridge": "br-int", "label": "tempest-AttachVolumeTestJSON-700098935-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "e393d23e783b4a40986674d782606c69", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap0013653c-95", "ovs_interfaceid": "0013653c-9540-4049-91a5-43bfba2d8fa3", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:17:02 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a082c96e-e384-40b6-bcc8-fc5e0e104669/disk --force-share --output=json" returned: 0 in 0.190s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:17:02 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/1c8b7559-29ab-4982-8c75-ea2a0f0101cb/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:17:02 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-10bfad7a-9cfa-4389-886c-28710edaefc7 req-1b2f7807-4a14-4cc6-aaae-67ff8a4c6a41 service nova] Releasing lock "refresh_cache-aaf1c3e3-b73e-42ad-87bd-c5255ec09b46" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:17:02 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/1c8b7559-29ab-4982-8c75-ea2a0f0101cb/disk --force-share --output=json" returned: 0 in 0.210s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:17:02 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/1c8b7559-29ab-4982-8c75-ea2a0f0101cb/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:17:02 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/1c8b7559-29ab-4982-8c75-ea2a0f0101cb/disk --force-share --output=json" returned: 0 in 0.233s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:17:02 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/21e9c6cd-c39f-41c0-8768-5dd64589c3f3/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:17:03 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/21e9c6cd-c39f-41c0-8768-5dd64589c3f3/disk --force-share --output=json" returned: 0 in 0.199s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:17:03 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/21e9c6cd-c39f-41c0-8768-5dd64589c3f3/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:17:03 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/21e9c6cd-c39f-41c0-8768-5dd64589c3f3/disk --force-share --output=json" returned: 0 in 0.190s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:17:03 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/13e5da64-9adc-4479-81c7-d329e9b5f08f/disk.rescue --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:17:03 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:17:03 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:17:03 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:17:03 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/13e5da64-9adc-4479-81c7-d329e9b5f08f/disk.rescue --force-share --output=json" returned: 0 in 0.212s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:17:03 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/13e5da64-9adc-4479-81c7-d329e9b5f08f/disk.rescue --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:17:03 user nova-compute[75234]: DEBUG nova.compute.manager [req-b3f4c2fc-0c90-4aba-9508-5a19c64653c3 req-ed9f38f5-a99e-43b1-b4a5-ad29ea7e4192 service nova] [instance: aaf1c3e3-b73e-42ad-87bd-c5255ec09b46] Received event network-vif-plugged-0013653c-9540-4049-91a5-43bfba2d8fa3 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:17:03 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-b3f4c2fc-0c90-4aba-9508-5a19c64653c3 req-ed9f38f5-a99e-43b1-b4a5-ad29ea7e4192 service nova] Acquiring lock "aaf1c3e3-b73e-42ad-87bd-c5255ec09b46-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:17:03 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-b3f4c2fc-0c90-4aba-9508-5a19c64653c3 req-ed9f38f5-a99e-43b1-b4a5-ad29ea7e4192 service nova] Lock "aaf1c3e3-b73e-42ad-87bd-c5255ec09b46-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:17:03 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-b3f4c2fc-0c90-4aba-9508-5a19c64653c3 req-ed9f38f5-a99e-43b1-b4a5-ad29ea7e4192 service nova] Lock "aaf1c3e3-b73e-42ad-87bd-c5255ec09b46-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:17:03 user nova-compute[75234]: DEBUG nova.compute.manager [req-b3f4c2fc-0c90-4aba-9508-5a19c64653c3 req-ed9f38f5-a99e-43b1-b4a5-ad29ea7e4192 service nova] [instance: aaf1c3e3-b73e-42ad-87bd-c5255ec09b46] No waiting events found dispatching network-vif-plugged-0013653c-9540-4049-91a5-43bfba2d8fa3 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:17:03 user nova-compute[75234]: WARNING nova.compute.manager [req-b3f4c2fc-0c90-4aba-9508-5a19c64653c3 req-ed9f38f5-a99e-43b1-b4a5-ad29ea7e4192 service nova] [instance: aaf1c3e3-b73e-42ad-87bd-c5255ec09b46] Received unexpected event network-vif-plugged-0013653c-9540-4049-91a5-43bfba2d8fa3 for instance with vm_state building and task_state spawning.
Nov 29 03:17:03 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/13e5da64-9adc-4479-81c7-d329e9b5f08f/disk.rescue --force-share --output=json" returned: 0 in 0.260s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:17:03 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/13e5da64-9adc-4479-81c7-d329e9b5f08f/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:17:03 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/13e5da64-9adc-4479-81c7-d329e9b5f08f/disk --force-share --output=json" returned: 0 in 0.192s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:17:03 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/13e5da64-9adc-4479-81c7-d329e9b5f08f/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:17:04 user nova-compute[75234]: INFO nova.compute.manager [None req-0f740e3e-103b-4bcd-b81c-07aaf56317a5 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] [instance: 13e5da64-9adc-4479-81c7-d329e9b5f08f] Unrescuing
Nov 29 03:17:04 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-0f740e3e-103b-4bcd-b81c-07aaf56317a5 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Acquiring lock "refresh_cache-13e5da64-9adc-4479-81c7-d329e9b5f08f" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:17:04 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-0f740e3e-103b-4bcd-b81c-07aaf56317a5 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Acquired lock "refresh_cache-13e5da64-9adc-4479-81c7-d329e9b5f08f" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:17:04 user nova-compute[75234]: DEBUG nova.network.neutron [None req-0f740e3e-103b-4bcd-b81c-07aaf56317a5 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] [instance: 13e5da64-9adc-4479-81c7-d329e9b5f08f] Building network info cache for instance {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2009}}
Nov 29 03:17:04 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:17:04 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:17:04 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:17:04 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/13e5da64-9adc-4479-81c7-d329e9b5f08f/disk --force-share --output=json" returned: 0 in 0.181s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:17:04 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/95b6eb15-7930-4aa5-aff8-73809b0a9277/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:17:04 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/95b6eb15-7930-4aa5-aff8-73809b0a9277/disk --force-share --output=json" returned: 0 in 0.175s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:17:04 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/95b6eb15-7930-4aa5-aff8-73809b0a9277/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:17:04 user nova-compute[75234]: DEBUG nova.network.neutron [None req-0f740e3e-103b-4bcd-b81c-07aaf56317a5 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] [instance: 13e5da64-9adc-4479-81c7-d329e9b5f08f] Updating instance_info_cache with network_info: [{"id": "2abf9e25-d1d9-4893-899e-912d41b7c38f", "address": "fa:16:3e:59:79:35", "network": {"id": "2b3c3140-f3b6-45da-8202-bb4d7f0dd130", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-2043420778-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "f8a7884867c944e1ba1024270e4902f3", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2abf9e25-d1", "ovs_interfaceid": "2abf9e25-d1d9-4893-899e-912d41b7c38f", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:17:04 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-0f740e3e-103b-4bcd-b81c-07aaf56317a5 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Releasing lock "refresh_cache-13e5da64-9adc-4479-81c7-d329e9b5f08f" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:17:04 user nova-compute[75234]: DEBUG nova.objects.instance [None req-0f740e3e-103b-4bcd-b81c-07aaf56317a5 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Lazy-loading 'flavor' on Instance uuid 13e5da64-9adc-4479-81c7-d329e9b5f08f {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:17:04 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/95b6eb15-7930-4aa5-aff8-73809b0a9277/disk --force-share --output=json" returned: 0 in 0.184s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:17:04 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6894b55f-93c6-43c9-8933-56d5a3c43725/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:17:04 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:17:04 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:17:04 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6894b55f-93c6-43c9-8933-56d5a3c43725/disk --force-share --output=json" returned: 0 in 0.191s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:17:04 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6894b55f-93c6-43c9-8933-56d5a3c43725/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:17:04 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:17:04 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:17:04 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6894b55f-93c6-43c9-8933-56d5a3c43725/disk --force-share --output=json" returned: 0 in 0.203s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:17:04 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/99d6f85f-1b77-4390-8b61-908c98f584c0/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:17:05 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/99d6f85f-1b77-4390-8b61-908c98f584c0/disk --force-share --output=json" returned: 0 in 0.168s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:17:05 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/99d6f85f-1b77-4390-8b61-908c98f584c0/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:17:05 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/99d6f85f-1b77-4390-8b61-908c98f584c0/disk --force-share --output=json" returned: 0 in 0.166s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:17:05 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/d3b90f39-4b91-4152-b820-bd0e263eec89/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:17:05 user nova-compute[75234]: INFO nova.virt.libvirt.driver [-] [instance: 13e5da64-9adc-4479-81c7-d329e9b5f08f] Instance destroyed successfully.
Nov 29 03:17:05 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/d3b90f39-4b91-4152-b820-bd0e263eec89/disk --force-share --output=json" returned: 0 in 0.185s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:17:05 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/d3b90f39-4b91-4152-b820-bd0e263eec89/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:17:05 user nova-compute[75234]: DEBUG nova.compute.manager [req-d1bf5db7-abb3-4ff6-9b23-430e01fb4bc8 req-9b05e3a9-e944-4204-b763-275515cf9a0c service nova] [instance: aaf1c3e3-b73e-42ad-87bd-c5255ec09b46] Received event network-vif-plugged-0013653c-9540-4049-91a5-43bfba2d8fa3 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:17:05 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-d1bf5db7-abb3-4ff6-9b23-430e01fb4bc8 req-9b05e3a9-e944-4204-b763-275515cf9a0c service nova] Acquiring lock "aaf1c3e3-b73e-42ad-87bd-c5255ec09b46-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:17:05 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-d1bf5db7-abb3-4ff6-9b23-430e01fb4bc8 req-9b05e3a9-e944-4204-b763-275515cf9a0c service nova] Lock "aaf1c3e3-b73e-42ad-87bd-c5255ec09b46-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:17:05 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-d1bf5db7-abb3-4ff6-9b23-430e01fb4bc8 req-9b05e3a9-e944-4204-b763-275515cf9a0c service nova] Lock "aaf1c3e3-b73e-42ad-87bd-c5255ec09b46-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:17:05 user nova-compute[75234]: DEBUG nova.compute.manager [req-d1bf5db7-abb3-4ff6-9b23-430e01fb4bc8 req-9b05e3a9-e944-4204-b763-275515cf9a0c service nova] [instance: aaf1c3e3-b73e-42ad-87bd-c5255ec09b46] No waiting events found dispatching network-vif-plugged-0013653c-9540-4049-91a5-43bfba2d8fa3 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:17:05 user nova-compute[75234]: WARNING nova.compute.manager [req-d1bf5db7-abb3-4ff6-9b23-430e01fb4bc8 req-9b05e3a9-e944-4204-b763-275515cf9a0c service nova] [instance: aaf1c3e3-b73e-42ad-87bd-c5255ec09b46] Received unexpected event network-vif-plugged-0013653c-9540-4049-91a5-43bfba2d8fa3 for instance with vm_state building and task_state spawning.
Nov 29 03:17:05 user nova-compute[75234]: DEBUG nova.compute.manager [req-d1bf5db7-abb3-4ff6-9b23-430e01fb4bc8 req-9b05e3a9-e944-4204-b763-275515cf9a0c service nova] [instance: 13e5da64-9adc-4479-81c7-d329e9b5f08f] Received event network-vif-unplugged-2abf9e25-d1d9-4893-899e-912d41b7c38f {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:17:05 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-d1bf5db7-abb3-4ff6-9b23-430e01fb4bc8 req-9b05e3a9-e944-4204-b763-275515cf9a0c service nova] Acquiring lock "13e5da64-9adc-4479-81c7-d329e9b5f08f-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:17:05 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-d1bf5db7-abb3-4ff6-9b23-430e01fb4bc8 req-9b05e3a9-e944-4204-b763-275515cf9a0c service nova] Lock "13e5da64-9adc-4479-81c7-d329e9b5f08f-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:17:05 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-d1bf5db7-abb3-4ff6-9b23-430e01fb4bc8 req-9b05e3a9-e944-4204-b763-275515cf9a0c service nova] Lock "13e5da64-9adc-4479-81c7-d329e9b5f08f-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:17:05 user nova-compute[75234]: DEBUG nova.compute.manager [req-d1bf5db7-abb3-4ff6-9b23-430e01fb4bc8 req-9b05e3a9-e944-4204-b763-275515cf9a0c service nova] [instance: 13e5da64-9adc-4479-81c7-d329e9b5f08f] No waiting events found dispatching network-vif-unplugged-2abf9e25-d1d9-4893-899e-912d41b7c38f {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:17:05 user nova-compute[75234]: WARNING nova.compute.manager [req-d1bf5db7-abb3-4ff6-9b23-430e01fb4bc8 req-9b05e3a9-e944-4204-b763-275515cf9a0c service nova] [instance: 13e5da64-9adc-4479-81c7-d329e9b5f08f] Received unexpected event network-vif-unplugged-2abf9e25-d1d9-4893-899e-912d41b7c38f for instance with vm_state rescued and task_state unrescuing.
Nov 29 03:17:05 user nova-compute[75234]: DEBUG nova.compute.manager [req-d1bf5db7-abb3-4ff6-9b23-430e01fb4bc8 req-9b05e3a9-e944-4204-b763-275515cf9a0c service nova] [instance: 13e5da64-9adc-4479-81c7-d329e9b5f08f] Received event network-vif-plugged-2abf9e25-d1d9-4893-899e-912d41b7c38f {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:17:05 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-d1bf5db7-abb3-4ff6-9b23-430e01fb4bc8 req-9b05e3a9-e944-4204-b763-275515cf9a0c service nova] Acquiring lock "13e5da64-9adc-4479-81c7-d329e9b5f08f-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:17:05 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-d1bf5db7-abb3-4ff6-9b23-430e01fb4bc8 req-9b05e3a9-e944-4204-b763-275515cf9a0c service nova] Lock "13e5da64-9adc-4479-81c7-d329e9b5f08f-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:17:05 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-d1bf5db7-abb3-4ff6-9b23-430e01fb4bc8 req-9b05e3a9-e944-4204-b763-275515cf9a0c service nova] Lock "13e5da64-9adc-4479-81c7-d329e9b5f08f-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:17:05 user nova-compute[75234]: DEBUG nova.compute.manager [req-d1bf5db7-abb3-4ff6-9b23-430e01fb4bc8 req-9b05e3a9-e944-4204-b763-275515cf9a0c service nova] [instance: 13e5da64-9adc-4479-81c7-d329e9b5f08f] No waiting events found dispatching network-vif-plugged-2abf9e25-d1d9-4893-899e-912d41b7c38f {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:17:05 user nova-compute[75234]: WARNING nova.compute.manager [req-d1bf5db7-abb3-4ff6-9b23-430e01fb4bc8 req-9b05e3a9-e944-4204-b763-275515cf9a0c service nova] [instance: 13e5da64-9adc-4479-81c7-d329e9b5f08f] Received unexpected event network-vif-plugged-2abf9e25-d1d9-4893-899e-912d41b7c38f for instance with vm_state rescued and task_state unrescuing.
Nov 29 03:17:05 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/d3b90f39-4b91-4152-b820-bd0e263eec89/disk --force-share --output=json" returned: 0 in 0.216s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:17:05 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/5e6cca46-c5a7-4303-98c2-1ad483c6d14e/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:17:05 user nova-compute[75234]: DEBUG nova.compute.manager [None req-52b1f7b3-04b2-4e15-b257-a3eb9a6f4ce9 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] [instance: aaf1c3e3-b73e-42ad-87bd-c5255ec09b46] Instance event wait completed in 0 seconds for  {{(pid=75234) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Nov 29 03:17:05 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-52b1f7b3-04b2-4e15-b257-a3eb9a6f4ce9 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] [instance: aaf1c3e3-b73e-42ad-87bd-c5255ec09b46] Guest created on hypervisor {{(pid=75234) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4421}}
Nov 29 03:17:05 user nova-compute[75234]: INFO nova.virt.libvirt.driver [-] [instance: aaf1c3e3-b73e-42ad-87bd-c5255ec09b46] Instance spawned successfully.
Nov 29 03:17:05 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-52b1f7b3-04b2-4e15-b257-a3eb9a6f4ce9 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] [instance: aaf1c3e3-b73e-42ad-87bd-c5255ec09b46] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:904}}
Nov 29 03:17:05 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-52b1f7b3-04b2-4e15-b257-a3eb9a6f4ce9 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] [instance: aaf1c3e3-b73e-42ad-87bd-c5255ec09b46] Found default for hw_cdrom_bus of ide {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:17:05 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-52b1f7b3-04b2-4e15-b257-a3eb9a6f4ce9 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] [instance: aaf1c3e3-b73e-42ad-87bd-c5255ec09b46] Found default for hw_disk_bus of virtio {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:17:05 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-52b1f7b3-04b2-4e15-b257-a3eb9a6f4ce9 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] [instance: aaf1c3e3-b73e-42ad-87bd-c5255ec09b46] Found default for hw_input_bus of None {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:17:05 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-52b1f7b3-04b2-4e15-b257-a3eb9a6f4ce9 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] [instance: aaf1c3e3-b73e-42ad-87bd-c5255ec09b46] Found default for hw_pointer_model of None {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:17:05 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-52b1f7b3-04b2-4e15-b257-a3eb9a6f4ce9 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] [instance: aaf1c3e3-b73e-42ad-87bd-c5255ec09b46] Found default for hw_video_model of virtio {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:17:05 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-52b1f7b3-04b2-4e15-b257-a3eb9a6f4ce9 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] [instance: aaf1c3e3-b73e-42ad-87bd-c5255ec09b46] Found default for hw_vif_model of virtio {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:17:05 user nova-compute[75234]: INFO nova.compute.manager [None req-52b1f7b3-04b2-4e15-b257-a3eb9a6f4ce9 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] [instance: aaf1c3e3-b73e-42ad-87bd-c5255ec09b46] Took 6.13 seconds to spawn the instance on the hypervisor.
Nov 29 03:17:05 user nova-compute[75234]: DEBUG nova.compute.manager [None req-52b1f7b3-04b2-4e15-b257-a3eb9a6f4ce9 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] [instance: aaf1c3e3-b73e-42ad-87bd-c5255ec09b46] Checking state {{(pid=75234) _get_power_state /opt/stack/nova/nova/compute/manager.py:1782}}
Nov 29 03:17:05 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/5e6cca46-c5a7-4303-98c2-1ad483c6d14e/disk --force-share --output=json" returned: 0 in 0.188s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:17:05 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/5e6cca46-c5a7-4303-98c2-1ad483c6d14e/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:17:06 user nova-compute[75234]: INFO nova.compute.manager [None req-52b1f7b3-04b2-4e15-b257-a3eb9a6f4ce9 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] [instance: aaf1c3e3-b73e-42ad-87bd-c5255ec09b46] Took 7.13 seconds to build instance.
Nov 29 03:17:06 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-52b1f7b3-04b2-4e15-b257-a3eb9a6f4ce9 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Lock "aaf1c3e3-b73e-42ad-87bd-c5255ec09b46" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 7.261s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:17:06 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/5e6cca46-c5a7-4303-98c2-1ad483c6d14e/disk --force-share --output=json" returned: 0 in 0.212s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:17:09 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:17:09 user nova-compute[75234]: DEBUG nova.compute.manager [req-3127360e-6737-498f-b622-4e3f6fc6b812 req-4af0cbd3-7eff-4cbf-b821-4497f5e4c34f service nova] [instance: 13e5da64-9adc-4479-81c7-d329e9b5f08f] Received event network-vif-plugged-2abf9e25-d1d9-4893-899e-912d41b7c38f {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:17:09 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-3127360e-6737-498f-b622-4e3f6fc6b812 req-4af0cbd3-7eff-4cbf-b821-4497f5e4c34f service nova] Acquiring lock "13e5da64-9adc-4479-81c7-d329e9b5f08f-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:17:09 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-3127360e-6737-498f-b622-4e3f6fc6b812 req-4af0cbd3-7eff-4cbf-b821-4497f5e4c34f service nova] Lock "13e5da64-9adc-4479-81c7-d329e9b5f08f-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:17:09 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-3127360e-6737-498f-b622-4e3f6fc6b812 req-4af0cbd3-7eff-4cbf-b821-4497f5e4c34f service nova] Lock "13e5da64-9adc-4479-81c7-d329e9b5f08f-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:17:09 user nova-compute[75234]: DEBUG nova.compute.manager [req-3127360e-6737-498f-b622-4e3f6fc6b812 req-4af0cbd3-7eff-4cbf-b821-4497f5e4c34f service nova] [instance: 13e5da64-9adc-4479-81c7-d329e9b5f08f] No waiting events found dispatching network-vif-plugged-2abf9e25-d1d9-4893-899e-912d41b7c38f {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:17:09 user nova-compute[75234]: WARNING nova.compute.manager [req-3127360e-6737-498f-b622-4e3f6fc6b812 req-4af0cbd3-7eff-4cbf-b821-4497f5e4c34f service nova] [instance: 13e5da64-9adc-4479-81c7-d329e9b5f08f] Received unexpected event network-vif-plugged-2abf9e25-d1d9-4893-899e-912d41b7c38f for instance with vm_state rescued and task_state unrescuing.
Nov 29 03:17:09 user nova-compute[75234]: DEBUG nova.compute.manager [req-3127360e-6737-498f-b622-4e3f6fc6b812 req-4af0cbd3-7eff-4cbf-b821-4497f5e4c34f service nova] [instance: 13e5da64-9adc-4479-81c7-d329e9b5f08f] Received event network-vif-plugged-2abf9e25-d1d9-4893-899e-912d41b7c38f {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:17:09 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-3127360e-6737-498f-b622-4e3f6fc6b812 req-4af0cbd3-7eff-4cbf-b821-4497f5e4c34f service nova] Acquiring lock "13e5da64-9adc-4479-81c7-d329e9b5f08f-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:17:09 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-3127360e-6737-498f-b622-4e3f6fc6b812 req-4af0cbd3-7eff-4cbf-b821-4497f5e4c34f service nova] Lock "13e5da64-9adc-4479-81c7-d329e9b5f08f-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:17:09 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-3127360e-6737-498f-b622-4e3f6fc6b812 req-4af0cbd3-7eff-4cbf-b821-4497f5e4c34f service nova] Lock "13e5da64-9adc-4479-81c7-d329e9b5f08f-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:17:09 user nova-compute[75234]: DEBUG nova.compute.manager [req-3127360e-6737-498f-b622-4e3f6fc6b812 req-4af0cbd3-7eff-4cbf-b821-4497f5e4c34f service nova] [instance: 13e5da64-9adc-4479-81c7-d329e9b5f08f] No waiting events found dispatching network-vif-plugged-2abf9e25-d1d9-4893-899e-912d41b7c38f {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:17:09 user nova-compute[75234]: WARNING nova.compute.manager [req-3127360e-6737-498f-b622-4e3f6fc6b812 req-4af0cbd3-7eff-4cbf-b821-4497f5e4c34f service nova] [instance: 13e5da64-9adc-4479-81c7-d329e9b5f08f] Received unexpected event network-vif-plugged-2abf9e25-d1d9-4893-899e-912d41b7c38f for instance with vm_state rescued and task_state unrescuing.
Nov 29 03:17:09 user nova-compute[75234]: DEBUG nova.compute.manager [None req-0f740e3e-103b-4bcd-b81c-07aaf56317a5 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] [instance: 13e5da64-9adc-4479-81c7-d329e9b5f08f] Checking state {{(pid=75234) _get_power_state /opt/stack/nova/nova/compute/manager.py:1782}}
Nov 29 03:17:09 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:17:09 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:17:09 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:17:09 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Hypervisor/Node resource view: name=user free_ram=7097MB free_disk=25.22948455810547GB free_vcpus=0 pci_devices=[{"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}] {{(pid=75234) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1106}}
Nov 29 03:17:09 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:17:09 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:17:09 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance 21e9c6cd-c39f-41c0-8768-5dd64589c3f3 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:17:09 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance d3b90f39-4b91-4152-b820-bd0e263eec89 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:17:09 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance 13e5da64-9adc-4479-81c7-d329e9b5f08f actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:17:09 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance a082c96e-e384-40b6-bcc8-fc5e0e104669 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:17:09 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance 6894b55f-93c6-43c9-8933-56d5a3c43725 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:17:09 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance ec7cf7cd-8e48-4bb3-a969-e14db705dc09 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:17:09 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance 95b6eb15-7930-4aa5-aff8-73809b0a9277 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:17:09 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance 75c5d700-6367-4128-9228-80d2cd24ace7 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:17:09 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance 1c8b7559-29ab-4982-8c75-ea2a0f0101cb actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:17:09 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance aa190a9f-ba53-43fb-8bae-2c429d0d1228 actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:17:09 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance 99d6f85f-1b77-4390-8b61-908c98f584c0 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:17:09 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance 5e6cca46-c5a7-4303-98c2-1ad483c6d14e actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:17:09 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance aaf1c3e3-b73e-42ad-87bd-c5255ec09b46 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:17:09 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Total usable vcpus: 12, total allocated vcpus: 13 {{(pid=75234) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1129}}
Nov 29 03:17:09 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Final resource view: name=user phys_ram=16011MB used_ram=7168MB phys_disk=40GB used_disk=12GB total_vcpus=12 used_vcpus=13 pci_stats=[] {{(pid=75234) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1138}}
Nov 29 03:17:09 user nova-compute[75234]: DEBUG nova.scheduler.client.report [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Refreshing inventories for resource provider 4d291220-7c37-4839-a0e3-b23c66502b74 {{(pid=75234) _refresh_associations /opt/stack/nova/nova/scheduler/client/report.py:804}}
Nov 29 03:17:09 user nova-compute[75234]: DEBUG nova.scheduler.client.report [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Updating ProviderTree inventory for provider 4d291220-7c37-4839-a0e3-b23c66502b74 from _refresh_and_get_inventory using data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=75234) _refresh_and_get_inventory /opt/stack/nova/nova/scheduler/client/report.py:768}}
Nov 29 03:17:09 user nova-compute[75234]: DEBUG nova.compute.provider_tree [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Updating inventory in ProviderTree for provider 4d291220-7c37-4839-a0e3-b23c66502b74 with inventory: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=75234) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:176}}
Nov 29 03:17:09 user nova-compute[75234]: DEBUG nova.scheduler.client.report [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Refreshing aggregate associations for resource provider 4d291220-7c37-4839-a0e3-b23c66502b74, aggregates: None {{(pid=75234) _refresh_associations /opt/stack/nova/nova/scheduler/client/report.py:813}}
Nov 29 03:17:09 user nova-compute[75234]: DEBUG nova.scheduler.client.report [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Refreshing trait associations for resource provider 4d291220-7c37-4839-a0e3-b23c66502b74, traits: COMPUTE_IMAGE_TYPE_AMI,COMPUTE_GRAPHICS_MODEL_CIRRUS,COMPUTE_NET_VIF_MODEL_VMXNET3,COMPUTE_STORAGE_BUS_VIRTIO,COMPUTE_GRAPHICS_MODEL_NONE,COMPUTE_IMAGE_TYPE_ISO,COMPUTE_NET_VIF_MODEL_RTL8139,HW_CPU_X86_SSE,COMPUTE_IMAGE_TYPE_QCOW2,COMPUTE_SOCKET_PCI_NUMA_AFFINITY,COMPUTE_GRAPHICS_MODEL_BOCHS,COMPUTE_VOLUME_ATTACH_WITH_TAG,COMPUTE_ACCELERATORS,HW_CPU_X86_SSE2,COMPUTE_STORAGE_BUS_SCSI,COMPUTE_GRAPHICS_MODEL_VGA,COMPUTE_NET_ATTACH_INTERFACE_WITH_TAG,COMPUTE_SECURITY_UEFI_SECURE_BOOT,HW_CPU_X86_SSE41,COMPUTE_NET_VIF_MODEL_E1000E,COMPUTE_TRUSTED_CERTS,COMPUTE_STORAGE_BUS_FDC,COMPUTE_VIOMMU_MODEL_INTEL,COMPUTE_GRAPHICS_MODEL_VIRTIO,COMPUTE_VIOMMU_MODEL_AUTO,COMPUTE_IMAGE_TYPE_AKI,COMPUTE_NET_VIF_MODEL_PCNET,COMPUTE_GRAPHICS_MODEL_QXL,COMPUTE_NET_VIF_MODEL_VIRTIO,COMPUTE_GRAPHICS_MODEL_VMVGA,COMPUTE_VOLUME_MULTI_ATTACH,COMPUTE_STORAGE_BUS_SATA,HW_CPU_X86_MMX,COMPUTE_NET_VIF_MODEL_E1000,COMPUTE_IMAGE_TYPE_ARI,COMPUTE_STORAGE_BUS_IDE,COMPUTE_NODE,HW_CPU_X86_SSSE3,COMPUTE_RESCUE_BFV,COMPUTE_NET_ATTACH_INTERFACE,COMPUTE_STORAGE_BUS_USB,COMPUTE_NET_VIF_MODEL_SPAPR_VLAN,COMPUTE_IMAGE_TYPE_RAW,COMPUTE_NET_VIF_MODEL_NE2K_PCI,COMPUTE_VOLUME_EXTEND,COMPUTE_DEVICE_TAGGING,HW_CPU_X86_SSE42 {{(pid=75234) _refresh_associations /opt/stack/nova/nova/scheduler/client/report.py:825}}
Nov 29 03:17:10 user nova-compute[75234]: DEBUG nova.compute.provider_tree [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Inventory has not changed in ProviderTree for provider: 4d291220-7c37-4839-a0e3-b23c66502b74 {{(pid=75234) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Nov 29 03:17:10 user nova-compute[75234]: DEBUG nova.scheduler.client.report [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Inventory has not changed for provider 4d291220-7c37-4839-a0e3-b23c66502b74 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=75234) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Nov 29 03:17:10 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Compute_service record updated for user:user {{(pid=75234) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:1067}}
Nov 29 03:17:10 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.802s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:17:10 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._cleanup_expired_console_auth_tokens {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:17:10 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:17:10 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:17:10 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._run_pending_deletes {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:17:10 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Cleaning up deleted instances {{(pid=75234) _run_pending_deletes /opt/stack/nova/nova/compute/manager.py:11209}}
Nov 29 03:17:10 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] There are 0 instances to clean {{(pid=75234) _run_pending_deletes /opt/stack/nova/nova/compute/manager.py:11218}}
Nov 29 03:17:11 user nova-compute[75234]: DEBUG nova.compute.manager [req-d198d235-41b1-479e-b848-2ce019a093f1 req-3e41242c-3d46-4e0c-85a0-829242160536 service nova] [instance: ec7cf7cd-8e48-4bb3-a969-e14db705dc09] Received event network-changed-9d89f22e-e411-461b-b9b3-85dbcf58af43 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:17:11 user nova-compute[75234]: DEBUG nova.compute.manager [req-d198d235-41b1-479e-b848-2ce019a093f1 req-3e41242c-3d46-4e0c-85a0-829242160536 service nova] [instance: ec7cf7cd-8e48-4bb3-a969-e14db705dc09] Refreshing instance network info cache due to event network-changed-9d89f22e-e411-461b-b9b3-85dbcf58af43. {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11117}}
Nov 29 03:17:11 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-d198d235-41b1-479e-b848-2ce019a093f1 req-3e41242c-3d46-4e0c-85a0-829242160536 service nova] Acquiring lock "refresh_cache-ec7cf7cd-8e48-4bb3-a969-e14db705dc09" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:17:11 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-d198d235-41b1-479e-b848-2ce019a093f1 req-3e41242c-3d46-4e0c-85a0-829242160536 service nova] Acquired lock "refresh_cache-ec7cf7cd-8e48-4bb3-a969-e14db705dc09" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:17:11 user nova-compute[75234]: DEBUG nova.network.neutron [req-d198d235-41b1-479e-b848-2ce019a093f1 req-3e41242c-3d46-4e0c-85a0-829242160536 service nova] [instance: ec7cf7cd-8e48-4bb3-a969-e14db705dc09] Refreshing network info cache for port 9d89f22e-e411-461b-b9b3-85dbcf58af43 {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2006}}
Nov 29 03:17:11 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-46cd809a-a423-41a8-8636-70fcdfe32f2c tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Acquiring lock "ec7cf7cd-8e48-4bb3-a969-e14db705dc09" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:17:11 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-46cd809a-a423-41a8-8636-70fcdfe32f2c tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Lock "ec7cf7cd-8e48-4bb3-a969-e14db705dc09" acquired by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:17:11 user nova-compute[75234]: DEBUG nova.objects.instance [None req-46cd809a-a423-41a8-8636-70fcdfe32f2c tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Lazy-loading 'flavor' on Instance uuid ec7cf7cd-8e48-4bb3-a969-e14db705dc09 {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:17:11 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-46cd809a-a423-41a8-8636-70fcdfe32f2c tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] [instance: ec7cf7cd-8e48-4bb3-a969-e14db705dc09] Ignoring supplied device name: /dev/vdb
Nov 29 03:17:11 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-46cd809a-a423-41a8-8636-70fcdfe32f2c tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Lock "ec7cf7cd-8e48-4bb3-a969-e14db705dc09" "released" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: held 0.067s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:17:11 user nova-compute[75234]: DEBUG nova.network.neutron [req-d198d235-41b1-479e-b848-2ce019a093f1 req-3e41242c-3d46-4e0c-85a0-829242160536 service nova] [instance: ec7cf7cd-8e48-4bb3-a969-e14db705dc09] Updated VIF entry in instance network info cache for port 9d89f22e-e411-461b-b9b3-85dbcf58af43. {{(pid=75234) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3481}}
Nov 29 03:17:11 user nova-compute[75234]: DEBUG nova.network.neutron [req-d198d235-41b1-479e-b848-2ce019a093f1 req-3e41242c-3d46-4e0c-85a0-829242160536 service nova] [instance: ec7cf7cd-8e48-4bb3-a969-e14db705dc09] Updating instance_info_cache with network_info: [{"id": "9d89f22e-e411-461b-b9b3-85dbcf58af43", "address": "fa:16:3e:30:11:ff", "network": {"id": "1898605d-00e7-4e81-964e-821e09f5421c", "bridge": "br-int", "label": "tempest-VolumesSnapshotTestJSON-1896954656-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.176", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "7e24dd8accea4c1e9598d643ea79c86d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap9d89f22e-e4", "ovs_interfaceid": "9d89f22e-e411-461b-b9b3-85dbcf58af43", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:17:11 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:17:11 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-46cd809a-a423-41a8-8636-70fcdfe32f2c tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Acquiring lock "ec7cf7cd-8e48-4bb3-a969-e14db705dc09" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:17:11 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-46cd809a-a423-41a8-8636-70fcdfe32f2c tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Lock "ec7cf7cd-8e48-4bb3-a969-e14db705dc09" acquired by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:17:11 user nova-compute[75234]: INFO nova.compute.manager [None req-46cd809a-a423-41a8-8636-70fcdfe32f2c tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] [instance: ec7cf7cd-8e48-4bb3-a969-e14db705dc09] Attaching volume 6c2e1179-dd3c-41c4-9594-bf4a16752c67 to /dev/vdb
Nov 29 03:17:11 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-d198d235-41b1-479e-b848-2ce019a093f1 req-3e41242c-3d46-4e0c-85a0-829242160536 service nova] Releasing lock "refresh_cache-ec7cf7cd-8e48-4bb3-a969-e14db705dc09" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:17:11 user nova-compute[75234]: DEBUG os_brick.utils [None req-46cd809a-a423-41a8-8636-70fcdfe32f2c tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:176}}
Nov 29 03:17:11 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:17:11 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.014s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:17:11 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[7a40c6c6-3a58-419a-acef-ea0b6300c78c]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:dd1368c5ba2c\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:17:11 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt -v / -n -o SOURCE {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:17:11 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt -v / -n -o SOURCE" returned: 0 in 0.019s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:17:11 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[bcb78280-99ab-48c8-a28c-1a5edb562eac]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:17:11 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:17:11 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.022s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:17:11 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[5c48fddf-a227-47a9-baf2-614087e7f969]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:17:11 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[f3c5d319-0690-48f5-8cdd-f2a3cb34e9df]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:17:11 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-46cd809a-a423-41a8-8636-70fcdfe32f2c tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Running cmd (subprocess): nvme version {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:17:12 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-46cd809a-a423-41a8-8636-70fcdfe32f2c tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] 'nvme version' failed. Not Retrying. {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:473}}
Nov 29 03:17:12 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.nvmeof [None req-46cd809a-a423-41a8-8636-70fcdfe32f2c tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] nvme not present on system {{(pid=75234) nvme_present /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/nvmeof.py:756}}
Nov 29 03:17:12 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:17:12 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:473}}
Nov 29 03:17:12 user nova-compute[75234]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Nov 29 03:17:12 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[d544e1c7-f9a5-4439-b7eb-d5fefbc77e03]: (4, '') {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:17:12 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.lightos [None req-46cd809a-a423-41a8-8636-70fcdfe32f2c tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=75234) find_dsc /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/lightos.py:98}}
Nov 29 03:17:12 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.lightos [None req-46cd809a-a423-41a8-8636-70fcdfe32f2c tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=75234) get_connector_properties /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/lightos.py:76}}
Nov 29 03:17:12 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.lightos [None req-46cd809a-a423-41a8-8636-70fcdfe32f2c tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] LIGHTOS: no hostnqn found. {{(pid=75234) get_connector_properties /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/lightos.py:84}}
Nov 29 03:17:12 user nova-compute[75234]: DEBUG os_brick.utils [None req-46cd809a-a423-41a8-8636-70fcdfe32f2c tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] <== get_connector_properties: return (142ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:dd1368c5ba2c', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:203}}
Nov 29 03:17:12 user nova-compute[75234]: DEBUG nova.virt.block_device [None req-46cd809a-a423-41a8-8636-70fcdfe32f2c tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] [instance: ec7cf7cd-8e48-4bb3-a969-e14db705dc09] Updating existing volume attachment record: bc636743-cf39-428d-9a00-f4befddada03 {{(pid=75234) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Nov 29 03:17:14 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:17:16 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:17:19 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:17:21 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:17:24 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-46cd809a-a423-41a8-8636-70fcdfe32f2c tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] [instance: ec7cf7cd-8e48-4bb3-a969-e14db705dc09] Calling os-brick to attach iSCSI Volume {{(pid=75234) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Nov 29 03:17:24 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-46cd809a-a423-41a8-8636-70fcdfe32f2c tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f8a5050c9a0>, 'connection_properties': {'target_iqn': 'iqn.2010-10.org.openstack:6c2e1179-dd3c-41c4-9594-bf4a16752c67', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'shDupLmi', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:176}}
Nov 29 03:17:24 user nova-compute[75234]: WARNING os_brick.initiator.connectors.base [None req-46cd809a-a423-41a8-8636-70fcdfe32f2c tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Nov 29 03:17:24 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-46cd809a-a423-41a8-8636-70fcdfe32f2c tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:68}}
Nov 29 03:17:24 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-46cd809a-a423-41a8-8636-70fcdfe32f2c tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.002s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:73}}
Nov 29 03:17:24 user nova-compute[75234]: WARNING os_brick.initiator.connectors.base [None req-46cd809a-a423-41a8-8636-70fcdfe32f2c tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Nov 29 03:17:24 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-46cd809a-a423-41a8-8636-70fcdfe32f2c tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:6c2e1179-dd3c-41c4-9594-bf4a16752c67" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:68}}
Nov 29 03:17:24 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-46cd809a-a423-41a8-8636-70fcdfe32f2c tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:6c2e1179-dd3c-41c4-9594-bf4a16752c67" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:73}}
Nov 29 03:17:24 user nova-compute[75234]: INFO os_brick.initiator.connectors.iscsi [None req-46cd809a-a423-41a8-8636-70fcdfe32f2c tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Nov 29 03:17:24 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:6c2e1179-dd3c-41c4-9594-bf4a16752c67 -p 172.16.0.220:3260 {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:17:24 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:6c2e1179-dd3c-41c4-9594-bf4a16752c67 -p 172.16.0.220:3260" returned: 21 in 0.021s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:17:24 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[9cf4a96b-307c-4d64-b256-ea05614dd4f8]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:17:24 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-46cd809a-a423-41a8-8636-70fcdfe32f2c tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Nov 29 03:17:24 user nova-compute[75234]:  {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:17:24 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:6c2e1179-dd3c-41c4-9594-bf4a16752c67 -p 172.16.0.220:3260 --interface default --op new {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:17:24 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:6c2e1179-dd3c-41c4-9594-bf4a16752c67 -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.021s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:17:24 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[f3419434-4ea3-4f7a-92e0-430690e0f7eb]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:6c2e1179-dd3c-41c4-9594-bf4a16752c67] added\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:17:24 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-46cd809a-a423-41a8-8636-70fcdfe32f2c tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:6c2e1179-dd3c-41c4-9594-bf4a16752c67] added
Nov 29 03:17:24 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:17:24 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:6c2e1179-dd3c-41c4-9594-bf4a16752c67 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:17:24 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:6c2e1179-dd3c-41c4-9594-bf4a16752c67 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.017s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:17:24 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[4e24305c-aaa4-4b64-a537-17ecf64eac29]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:17:24 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-46cd809a-a423-41a8-8636-70fcdfe32f2c tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:17:24 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:6c2e1179-dd3c-41c4-9594-bf4a16752c67 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:17:24 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:6c2e1179-dd3c-41c4-9594-bf4a16752c67 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.019s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:17:24 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[0f1d1510-81fa-4e2a-8c0f-9abf5b28aa2c]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:17:24 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-46cd809a-a423-41a8-8636-70fcdfe32f2c tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:17:24 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:6c2e1179-dd3c-41c4-9594-bf4a16752c67 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v shDupLmi {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:17:24 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:6c2e1179-dd3c-41c4-9594-bf4a16752c67 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v shDupLmi" returned: 0 in 0.018s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:17:24 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[eb931312-a264-4aee-a20c-2f80340ad7f2]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:17:24 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-46cd809a-a423-41a8-8636-70fcdfe32f2c tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'shDupLmi'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:17:24 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:6c2e1179-dd3c-41c4-9594-bf4a16752c67 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:17:24 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:6c2e1179-dd3c-41c4-9594-bf4a16752c67 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.019s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:17:24 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[7b4a2991-d673-48cc-9d3c-04c4161e4d5e]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:17:24 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-46cd809a-a423-41a8-8636-70fcdfe32f2c tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:17:24 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:17:24 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.018s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:17:24 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[2b0a0623-5b1f-4fa6-9f84-072799db4b0c]: (4, ('tcp: [27] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:07f83efc-272d-4e10-9a2b-48d967db613c (non-flash)\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:17:24 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-46cd809a-a423-41a8-8636-70fcdfe32f2c tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [27] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:07f83efc-272d-4e10-9a2b-48d967db613c (non-flash)
Nov 29 03:17:24 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:17:24 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-46cd809a-a423-41a8-8636-70fcdfe32f2c tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] iscsi session list stdout=tcp: [27] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:07f83efc-272d-4e10-9a2b-48d967db613c (non-flash)
Nov 29 03:17:24 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsi_session /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Nov 29 03:17:24 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:6c2e1179-dd3c-41c4-9594-bf4a16752c67 -p 172.16.0.220:3260 --login {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:17:24 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:6c2e1179-dd3c-41c4-9594-bf4a16752c67 -p 172.16.0.220:3260 --login" returned: 0 in 0.054s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:17:24 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[667e7103-3cb4-4143-9879-90b87a3951ce]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:6c2e1179-dd3c-41c4-9594-bf4a16752c67, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:6c2e1179-dd3c-41c4-9594-bf4a16752c67, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:17:24 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-46cd809a-a423-41a8-8636-70fcdfe32f2c tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:6c2e1179-dd3c-41c4-9594-bf4a16752c67, portal: 172.16.0.220,3260]
Nov 29 03:17:24 user nova-compute[75234]: Login to [iface: default, target: iqn.2010-10.org.openstack:6c2e1179-dd3c-41c4-9594-bf4a16752c67, portal: 172.16.0.220,3260] successful.
Nov 29 03:17:24 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:17:24 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:6c2e1179-dd3c-41c4-9594-bf4a16752c67 -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:17:24 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:6c2e1179-dd3c-41c4-9594-bf4a16752c67 -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.020s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:17:24 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[9d42cec9-9939-48b9-a6dd-9dd278d76700]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:17:24 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-46cd809a-a423-41a8-8636-70fcdfe32f2c tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:17:24 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:17:24 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.020s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:17:24 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[42a84cfa-587e-471a-8a65-3061ee5027ea]: (4, ('tcp: [27] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:07f83efc-272d-4e10-9a2b-48d967db613c (non-flash)\ntcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6c2e1179-dd3c-41c4-9594-bf4a16752c67 (non-flash)\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:17:24 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-46cd809a-a423-41a8-8636-70fcdfe32f2c tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [27] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:07f83efc-272d-4e10-9a2b-48d967db613c (non-flash)
Nov 29 03:17:24 user nova-compute[75234]: tcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6c2e1179-dd3c-41c4-9594-bf4a16752c67 (non-flash)
Nov 29 03:17:24 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:17:24 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-46cd809a-a423-41a8-8636-70fcdfe32f2c tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] iscsi session list stdout=tcp: [27] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:07f83efc-272d-4e10-9a2b-48d967db613c (non-flash)
Nov 29 03:17:24 user nova-compute[75234]: tcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6c2e1179-dd3c-41c4-9594-bf4a16752c67 (non-flash)
Nov 29 03:17:24 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsi_session /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Nov 29 03:17:24 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-46cd809a-a423-41a8-8636-70fcdfe32f2c tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:6c2e1179-dd3c-41c4-9594-bf4a16752c67" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.271s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:87}}
Nov 29 03:17:24 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-46cd809a-a423-41a8-8636-70fcdfe32f2c tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Connected to 172.16.0.220:3260 {{(pid=75234) _connect_vol /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:644}}
Nov 29 03:17:24 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-46cd809a-a423-41a8-8636-70fcdfe32f2c tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] HCTL ('34', '-', '-', 0) found on session 28 with lun 0 {{(pid=75234) get_hctl /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:770}}
Nov 29 03:17:24 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-46cd809a-a423-41a8-8636-70fcdfe32f2c tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Scanning host 34 c: -, t: -, l: 0) {{(pid=75234) scan_iscsi /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:795}}
Nov 29 03:17:24 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:17:24 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host34/scan {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:17:24 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host34/scan" returned: 0 in 0.016s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:17:24 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[14f8c759-cfcf-4cbc-bf45-9fe5134cdcbd]: (4, ('- - 0', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:17:24 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-46cd809a-a423-41a8-8636-70fcdfe32f2c tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Searching for a device in session 28 and hctl ['34', '*', '*', 0] yield: sdc {{(pid=75234) device_name_by_hctl /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:789}}
Nov 29 03:17:24 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-46cd809a-a423-41a8-8636-70fcdfe32f2c tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Connected to sdc using {'target_iqn': 'iqn.2010-10.org.openstack:6c2e1179-dd3c-41c4-9594-bf4a16752c67', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'shDupLmi', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False} {{(pid=75234) _connect_vol /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:672}}
Nov 29 03:17:25 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-46cd809a-a423-41a8-8636-70fcdfe32f2c tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.312s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:87}}
Nov 29 03:17:25 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-46cd809a-a423-41a8-8636-70fcdfe32f2c tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] <== connect_volume: return (1314ms) {'type': 'block', 'scsi_wwn': '23762313933343336', 'path': '/dev/sdc'} {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:203}}
Nov 29 03:17:25 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-46cd809a-a423-41a8-8636-70fcdfe32f2c tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] [instance: ec7cf7cd-8e48-4bb3-a969-e14db705dc09] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '23762313933343336', 'path': '/dev/sdc'} {{(pid=75234) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Nov 29 03:17:25 user nova-compute[75234]: DEBUG nova.objects.instance [None req-46cd809a-a423-41a8-8636-70fcdfe32f2c tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Lazy-loading 'flavor' on Instance uuid ec7cf7cd-8e48-4bb3-a969-e14db705dc09 {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:17:25 user nova-compute[75234]: DEBUG nova.virt.libvirt.guest [None req-46cd809a-a423-41a8-8636-70fcdfe32f2c tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] attach device xml: <disk type="block" device="disk">
Nov 29 03:17:25 user nova-compute[75234]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Nov 29 03:17:25 user nova-compute[75234]:   <source dev="/dev/sdc"/>
Nov 29 03:17:25 user nova-compute[75234]:   <target dev="vdb" bus="virtio"/>
Nov 29 03:17:25 user nova-compute[75234]:   <serial>6c2e1179-dd3c-41c4-9594-bf4a16752c67</serial>
Nov 29 03:17:25 user nova-compute[75234]: </disk>
Nov 29 03:17:25 user nova-compute[75234]:  {{(pid=75234) attach_device /opt/stack/nova/nova/virt/libvirt/guest.py:339}}
Nov 29 03:17:25 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-46cd809a-a423-41a8-8636-70fcdfe32f2c tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] No BDM found with device name vda, not building metadata. {{(pid=75234) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12089}}
Nov 29 03:17:25 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-46cd809a-a423-41a8-8636-70fcdfe32f2c tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] No BDM found with device name vdb, not building metadata. {{(pid=75234) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12089}}
Nov 29 03:17:25 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-46cd809a-a423-41a8-8636-70fcdfe32f2c tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] No VIF found with MAC fa:16:3e:30:11:ff, not building metadata {{(pid=75234) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Nov 29 03:17:25 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-46cd809a-a423-41a8-8636-70fcdfe32f2c tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Lock "ec7cf7cd-8e48-4bb3-a969-e14db705dc09" "released" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: held 14.109s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:17:26 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:17:29 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:17:31 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:17:34 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:17:36 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:17:39 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:17:41 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:17:44 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:17:46 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:17:47 user nova-compute[75234]: DEBUG nova.compute.manager [req-082f6090-cf77-4d9f-9a88-137801df18fe req-c8cadf4e-4a07-41d6-897d-7ff5479e7b59 service nova] [instance: 75c5d700-6367-4128-9228-80d2cd24ace7] Received event network-changed-29c6743e-29e2-40a8-8de9-ec380f1e6f25 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:17:47 user nova-compute[75234]: DEBUG nova.compute.manager [req-082f6090-cf77-4d9f-9a88-137801df18fe req-c8cadf4e-4a07-41d6-897d-7ff5479e7b59 service nova] [instance: 75c5d700-6367-4128-9228-80d2cd24ace7] Refreshing instance network info cache due to event network-changed-29c6743e-29e2-40a8-8de9-ec380f1e6f25. {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11117}}
Nov 29 03:17:47 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-082f6090-cf77-4d9f-9a88-137801df18fe req-c8cadf4e-4a07-41d6-897d-7ff5479e7b59 service nova] Acquiring lock "refresh_cache-75c5d700-6367-4128-9228-80d2cd24ace7" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:17:47 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-082f6090-cf77-4d9f-9a88-137801df18fe req-c8cadf4e-4a07-41d6-897d-7ff5479e7b59 service nova] Acquired lock "refresh_cache-75c5d700-6367-4128-9228-80d2cd24ace7" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:17:47 user nova-compute[75234]: DEBUG nova.network.neutron [req-082f6090-cf77-4d9f-9a88-137801df18fe req-c8cadf4e-4a07-41d6-897d-7ff5479e7b59 service nova] [instance: 75c5d700-6367-4128-9228-80d2cd24ace7] Refreshing network info cache for port 29c6743e-29e2-40a8-8de9-ec380f1e6f25 {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2006}}
Nov 29 03:17:48 user nova-compute[75234]: DEBUG nova.network.neutron [req-082f6090-cf77-4d9f-9a88-137801df18fe req-c8cadf4e-4a07-41d6-897d-7ff5479e7b59 service nova] [instance: 75c5d700-6367-4128-9228-80d2cd24ace7] Updated VIF entry in instance network info cache for port 29c6743e-29e2-40a8-8de9-ec380f1e6f25. {{(pid=75234) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3481}}
Nov 29 03:17:48 user nova-compute[75234]: DEBUG nova.network.neutron [req-082f6090-cf77-4d9f-9a88-137801df18fe req-c8cadf4e-4a07-41d6-897d-7ff5479e7b59 service nova] [instance: 75c5d700-6367-4128-9228-80d2cd24ace7] Updating instance_info_cache with network_info: [{"id": "29c6743e-29e2-40a8-8de9-ec380f1e6f25", "address": "fa:16:3e:4e:5d:e9", "network": {"id": "0f289171-3d73-4054-95a0-2515b4f206a0", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-1787960284-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.222", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "315732b6e3ee44d584b8e510ef199a90", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap29c6743e-29", "ovs_interfaceid": "29c6743e-29e2-40a8-8de9-ec380f1e6f25", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:17:48 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-082f6090-cf77-4d9f-9a88-137801df18fe req-c8cadf4e-4a07-41d6-897d-7ff5479e7b59 service nova] Releasing lock "refresh_cache-75c5d700-6367-4128-9228-80d2cd24ace7" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:17:49 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:17:51 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:17:51 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-9e5c7588-51ec-49d2-8d1a-cca0dabbdd7b tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Acquiring lock "75c5d700-6367-4128-9228-80d2cd24ace7" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:17:51 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-9e5c7588-51ec-49d2-8d1a-cca0dabbdd7b tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Lock "75c5d700-6367-4128-9228-80d2cd24ace7" acquired by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:17:51 user nova-compute[75234]: DEBUG nova.objects.instance [None req-9e5c7588-51ec-49d2-8d1a-cca0dabbdd7b tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Lazy-loading 'flavor' on Instance uuid 75c5d700-6367-4128-9228-80d2cd24ace7 {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:17:51 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-9e5c7588-51ec-49d2-8d1a-cca0dabbdd7b tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Lock "75c5d700-6367-4128-9228-80d2cd24ace7" "released" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: held 0.102s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:17:51 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-9e5c7588-51ec-49d2-8d1a-cca0dabbdd7b tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Acquiring lock "75c5d700-6367-4128-9228-80d2cd24ace7" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:17:51 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-9e5c7588-51ec-49d2-8d1a-cca0dabbdd7b tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Lock "75c5d700-6367-4128-9228-80d2cd24ace7" acquired by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:17:51 user nova-compute[75234]: INFO nova.compute.manager [None req-9e5c7588-51ec-49d2-8d1a-cca0dabbdd7b tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] [instance: 75c5d700-6367-4128-9228-80d2cd24ace7] Attaching volume bf57b967-7fad-4d76-abb2-319fe88f8ef9 to /dev/vdb
Nov 29 03:17:51 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:17:51 user nova-compute[75234]: DEBUG os_brick.utils [None req-9e5c7588-51ec-49d2-8d1a-cca0dabbdd7b tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:176}}
Nov 29 03:17:51 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:17:51 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.017s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:17:51 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[f28687b7-44dd-4829-8007-e53205741777]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:dd1368c5ba2c\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:17:51 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt -v / -n -o SOURCE {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:17:51 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt -v / -n -o SOURCE" returned: 0 in 0.016s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:17:51 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[7bca33c9-b5c0-4df6-9dbd-2b9f379aca9a]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:17:51 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:17:51 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.017s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:17:51 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[b7b868f1-ebe1-4b42-bf3f-d9d5438d8bea]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:17:51 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[768e7349-196a-4e68-8410-e863d37fde3b]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:17:51 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-9e5c7588-51ec-49d2-8d1a-cca0dabbdd7b tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Running cmd (subprocess): nvme version {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:17:52 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-9e5c7588-51ec-49d2-8d1a-cca0dabbdd7b tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] 'nvme version' failed. Not Retrying. {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:473}}
Nov 29 03:17:52 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.nvmeof [None req-9e5c7588-51ec-49d2-8d1a-cca0dabbdd7b tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] nvme not present on system {{(pid=75234) nvme_present /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/nvmeof.py:756}}
Nov 29 03:17:52 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:17:52 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:473}}
Nov 29 03:17:52 user nova-compute[75234]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Nov 29 03:17:52 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[de4d1bfc-6c47-45a3-959f-cd4fff873984]: (4, '') {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:17:52 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.lightos [None req-9e5c7588-51ec-49d2-8d1a-cca0dabbdd7b tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=75234) find_dsc /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/lightos.py:98}}
Nov 29 03:17:52 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.lightos [None req-9e5c7588-51ec-49d2-8d1a-cca0dabbdd7b tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=75234) get_connector_properties /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/lightos.py:76}}
Nov 29 03:17:52 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.lightos [None req-9e5c7588-51ec-49d2-8d1a-cca0dabbdd7b tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] LIGHTOS: no hostnqn found. {{(pid=75234) get_connector_properties /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/lightos.py:84}}
Nov 29 03:17:52 user nova-compute[75234]: DEBUG os_brick.utils [None req-9e5c7588-51ec-49d2-8d1a-cca0dabbdd7b tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] <== get_connector_properties: return (221ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:dd1368c5ba2c', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:203}}
Nov 29 03:17:52 user nova-compute[75234]: DEBUG nova.virt.block_device [None req-9e5c7588-51ec-49d2-8d1a-cca0dabbdd7b tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] [instance: 75c5d700-6367-4128-9228-80d2cd24ace7] Updating existing volume attachment record: bbb7c142-15d1-419b-85c9-bd6b95a64d97 {{(pid=75234) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Nov 29 03:17:52 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-4f83f151-3776-4aa8-b91c-a6dd77c94abe tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Acquiring lock "af699bc2-9daa-4b5a-9b4a-ad92fe857d70" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:17:52 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-4f83f151-3776-4aa8-b91c-a6dd77c94abe tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Lock "af699bc2-9daa-4b5a-9b4a-ad92fe857d70" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:17:52 user nova-compute[75234]: DEBUG nova.compute.manager [None req-4f83f151-3776-4aa8-b91c-a6dd77c94abe tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] [instance: af699bc2-9daa-4b5a-9b4a-ad92fe857d70] Starting instance... {{(pid=75234) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2422}}
Nov 29 03:17:52 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-4f83f151-3776-4aa8-b91c-a6dd77c94abe tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:17:52 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-4f83f151-3776-4aa8-b91c-a6dd77c94abe tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:17:52 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-4f83f151-3776-4aa8-b91c-a6dd77c94abe tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=75234) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2366}}
Nov 29 03:17:52 user nova-compute[75234]: INFO nova.compute.claims [None req-4f83f151-3776-4aa8-b91c-a6dd77c94abe tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] [instance: af699bc2-9daa-4b5a-9b4a-ad92fe857d70] Claim successful on node user
Nov 29 03:17:52 user nova-compute[75234]: DEBUG nova.compute.provider_tree [None req-4f83f151-3776-4aa8-b91c-a6dd77c94abe tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Inventory has not changed in ProviderTree for provider: 4d291220-7c37-4839-a0e3-b23c66502b74 {{(pid=75234) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Nov 29 03:17:52 user nova-compute[75234]: DEBUG nova.scheduler.client.report [None req-4f83f151-3776-4aa8-b91c-a6dd77c94abe tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Inventory has not changed for provider 4d291220-7c37-4839-a0e3-b23c66502b74 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=75234) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Nov 29 03:17:52 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-4f83f151-3776-4aa8-b91c-a6dd77c94abe tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.530s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:17:52 user nova-compute[75234]: DEBUG nova.compute.manager [None req-4f83f151-3776-4aa8-b91c-a6dd77c94abe tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] [instance: af699bc2-9daa-4b5a-9b4a-ad92fe857d70] Start building networks asynchronously for instance. {{(pid=75234) _build_resources /opt/stack/nova/nova/compute/manager.py:2819}}
Nov 29 03:17:53 user nova-compute[75234]: DEBUG nova.compute.manager [None req-4f83f151-3776-4aa8-b91c-a6dd77c94abe tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] [instance: af699bc2-9daa-4b5a-9b4a-ad92fe857d70] Allocating IP information in the background. {{(pid=75234) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1972}}
Nov 29 03:17:53 user nova-compute[75234]: DEBUG nova.network.neutron [None req-4f83f151-3776-4aa8-b91c-a6dd77c94abe tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] [instance: af699bc2-9daa-4b5a-9b4a-ad92fe857d70] allocate_for_instance() {{(pid=75234) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1156}}
Nov 29 03:17:53 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-4f83f151-3776-4aa8-b91c-a6dd77c94abe tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] [instance: af699bc2-9daa-4b5a-9b4a-ad92fe857d70] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Nov 29 03:17:53 user nova-compute[75234]: DEBUG nova.compute.manager [None req-4f83f151-3776-4aa8-b91c-a6dd77c94abe tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] [instance: af699bc2-9daa-4b5a-9b4a-ad92fe857d70] Start building block device mappings for instance. {{(pid=75234) _build_resources /opt/stack/nova/nova/compute/manager.py:2854}}
Nov 29 03:17:53 user nova-compute[75234]: DEBUG nova.policy [None req-4f83f151-3776-4aa8-b91c-a6dd77c94abe tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': 'd611826f3be947c0947b99c763c26d44', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c1cb3d1f12f242a785a688d59dbe3cd6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=75234) authorize /opt/stack/nova/nova/policy.py:203}}
Nov 29 03:17:53 user nova-compute[75234]: INFO nova.virt.block_device [None req-4f83f151-3776-4aa8-b91c-a6dd77c94abe tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] [instance: af699bc2-9daa-4b5a-9b4a-ad92fe857d70] Booting with volume cf6ab229-533a-424e-bd67-1d54d50ad4be at /dev/vda
Nov 29 03:17:53 user nova-compute[75234]: DEBUG os_brick.utils [None req-4f83f151-3776-4aa8-b91c-a6dd77c94abe tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:176}}
Nov 29 03:17:53 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:17:53 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.012s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:17:53 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[2395a063-f0b1-4043-abbd-01bde3ce4dfc]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:dd1368c5ba2c\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:17:53 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt -v / -n -o SOURCE {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:17:53 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt -v / -n -o SOURCE" returned: 0 in 0.015s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:17:53 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[cd796afb-3de4-4e34-82e4-76fd0e5a6708]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:17:53 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:17:53 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.015s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:17:53 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[fd4c9fe6-21dd-4122-b75c-d55a9b313baf]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:17:53 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[dee4cfcf-e2bd-49b2-bd4a-b368fec864f3]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:17:53 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-4f83f151-3776-4aa8-b91c-a6dd77c94abe tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Running cmd (subprocess): nvme version {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:17:53 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-4f83f151-3776-4aa8-b91c-a6dd77c94abe tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] 'nvme version' failed. Not Retrying. {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:473}}
Nov 29 03:17:53 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.nvmeof [None req-4f83f151-3776-4aa8-b91c-a6dd77c94abe tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] nvme not present on system {{(pid=75234) nvme_present /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/nvmeof.py:756}}
Nov 29 03:17:53 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:17:53 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:473}}
Nov 29 03:17:53 user nova-compute[75234]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Nov 29 03:17:53 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[b91d6a41-7e13-4b41-96fb-5c183ce4c3de]: (4, '') {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:17:53 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.lightos [None req-4f83f151-3776-4aa8-b91c-a6dd77c94abe tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=75234) find_dsc /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/lightos.py:98}}
Nov 29 03:17:53 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.lightos [None req-4f83f151-3776-4aa8-b91c-a6dd77c94abe tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=75234) get_connector_properties /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/lightos.py:76}}
Nov 29 03:17:53 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.lightos [None req-4f83f151-3776-4aa8-b91c-a6dd77c94abe tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] LIGHTOS: no hostnqn found. {{(pid=75234) get_connector_properties /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/lightos.py:84}}
Nov 29 03:17:53 user nova-compute[75234]: DEBUG os_brick.utils [None req-4f83f151-3776-4aa8-b91c-a6dd77c94abe tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] <== get_connector_properties: return (137ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:dd1368c5ba2c', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:203}}
Nov 29 03:17:53 user nova-compute[75234]: DEBUG nova.virt.block_device [None req-4f83f151-3776-4aa8-b91c-a6dd77c94abe tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] [instance: af699bc2-9daa-4b5a-9b4a-ad92fe857d70] Updating existing volume attachment record: 585001f4-3bc2-4401-9995-c1f93cff9cfd {{(pid=75234) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Nov 29 03:17:53 user nova-compute[75234]: DEBUG nova.network.neutron [None req-4f83f151-3776-4aa8-b91c-a6dd77c94abe tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] [instance: af699bc2-9daa-4b5a-9b4a-ad92fe857d70] Successfully created port: 66dcb0f9-3180-42f7-851d-19b0c4f4119c {{(pid=75234) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:548}}
Nov 29 03:17:54 user nova-compute[75234]: DEBUG nova.network.neutron [None req-4f83f151-3776-4aa8-b91c-a6dd77c94abe tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] [instance: af699bc2-9daa-4b5a-9b4a-ad92fe857d70] Successfully updated port: 66dcb0f9-3180-42f7-851d-19b0c4f4119c {{(pid=75234) _update_port /opt/stack/nova/nova/network/neutron.py:586}}
Nov 29 03:17:54 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-4f83f151-3776-4aa8-b91c-a6dd77c94abe tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Acquiring lock "refresh_cache-af699bc2-9daa-4b5a-9b4a-ad92fe857d70" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:17:54 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-4f83f151-3776-4aa8-b91c-a6dd77c94abe tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Acquired lock "refresh_cache-af699bc2-9daa-4b5a-9b4a-ad92fe857d70" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:17:54 user nova-compute[75234]: DEBUG nova.network.neutron [None req-4f83f151-3776-4aa8-b91c-a6dd77c94abe tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] [instance: af699bc2-9daa-4b5a-9b4a-ad92fe857d70] Building network info cache for instance {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2009}}
Nov 29 03:17:54 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:17:54 user nova-compute[75234]: DEBUG nova.compute.manager [req-20b9b646-0ade-4a5a-958f-876a2874c2ff req-2fccab08-3c10-4e1d-9d70-447b9d584bb6 service nova] [instance: af699bc2-9daa-4b5a-9b4a-ad92fe857d70] Received event network-changed-66dcb0f9-3180-42f7-851d-19b0c4f4119c {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:17:54 user nova-compute[75234]: DEBUG nova.compute.manager [req-20b9b646-0ade-4a5a-958f-876a2874c2ff req-2fccab08-3c10-4e1d-9d70-447b9d584bb6 service nova] [instance: af699bc2-9daa-4b5a-9b4a-ad92fe857d70] Refreshing instance network info cache due to event network-changed-66dcb0f9-3180-42f7-851d-19b0c4f4119c. {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11117}}
Nov 29 03:17:54 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-20b9b646-0ade-4a5a-958f-876a2874c2ff req-2fccab08-3c10-4e1d-9d70-447b9d584bb6 service nova] Acquiring lock "refresh_cache-af699bc2-9daa-4b5a-9b4a-ad92fe857d70" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:17:54 user nova-compute[75234]: DEBUG nova.network.neutron [None req-4f83f151-3776-4aa8-b91c-a6dd77c94abe tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] [instance: af699bc2-9daa-4b5a-9b4a-ad92fe857d70] Instance cache missing network info. {{(pid=75234) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3322}}
Nov 29 03:17:54 user nova-compute[75234]: DEBUG nova.network.neutron [None req-4f83f151-3776-4aa8-b91c-a6dd77c94abe tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] [instance: af699bc2-9daa-4b5a-9b4a-ad92fe857d70] Updating instance_info_cache with network_info: [{"id": "66dcb0f9-3180-42f7-851d-19b0c4f4119c", "address": "fa:16:3e:dd:9e:50", "network": {"id": "1f946e69-dbec-429c-83f8-a186bad4f2cc", "bridge": "br-int", "label": "tempest-ServerActionsTestOtherA-486369423-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "c1cb3d1f12f242a785a688d59dbe3cd6", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap66dcb0f9-31", "ovs_interfaceid": "66dcb0f9-3180-42f7-851d-19b0c4f4119c", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:17:54 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-4f83f151-3776-4aa8-b91c-a6dd77c94abe tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Releasing lock "refresh_cache-af699bc2-9daa-4b5a-9b4a-ad92fe857d70" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:17:54 user nova-compute[75234]: DEBUG nova.compute.manager [None req-4f83f151-3776-4aa8-b91c-a6dd77c94abe tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] [instance: af699bc2-9daa-4b5a-9b4a-ad92fe857d70] Instance network_info: |[{"id": "66dcb0f9-3180-42f7-851d-19b0c4f4119c", "address": "fa:16:3e:dd:9e:50", "network": {"id": "1f946e69-dbec-429c-83f8-a186bad4f2cc", "bridge": "br-int", "label": "tempest-ServerActionsTestOtherA-486369423-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "c1cb3d1f12f242a785a688d59dbe3cd6", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap66dcb0f9-31", "ovs_interfaceid": "66dcb0f9-3180-42f7-851d-19b0c4f4119c", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=75234) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1987}}
Nov 29 03:17:54 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-20b9b646-0ade-4a5a-958f-876a2874c2ff req-2fccab08-3c10-4e1d-9d70-447b9d584bb6 service nova] Acquired lock "refresh_cache-af699bc2-9daa-4b5a-9b4a-ad92fe857d70" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:17:54 user nova-compute[75234]: DEBUG nova.network.neutron [req-20b9b646-0ade-4a5a-958f-876a2874c2ff req-2fccab08-3c10-4e1d-9d70-447b9d584bb6 service nova] [instance: af699bc2-9daa-4b5a-9b4a-ad92fe857d70] Refreshing network info cache for port 66dcb0f9-3180-42f7-851d-19b0c4f4119c {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2006}}
Nov 29 03:17:55 user nova-compute[75234]: DEBUG nova.network.neutron [req-20b9b646-0ade-4a5a-958f-876a2874c2ff req-2fccab08-3c10-4e1d-9d70-447b9d584bb6 service nova] [instance: af699bc2-9daa-4b5a-9b4a-ad92fe857d70] Updated VIF entry in instance network info cache for port 66dcb0f9-3180-42f7-851d-19b0c4f4119c. {{(pid=75234) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3481}}
Nov 29 03:17:55 user nova-compute[75234]: DEBUG nova.network.neutron [req-20b9b646-0ade-4a5a-958f-876a2874c2ff req-2fccab08-3c10-4e1d-9d70-447b9d584bb6 service nova] [instance: af699bc2-9daa-4b5a-9b4a-ad92fe857d70] Updating instance_info_cache with network_info: [{"id": "66dcb0f9-3180-42f7-851d-19b0c4f4119c", "address": "fa:16:3e:dd:9e:50", "network": {"id": "1f946e69-dbec-429c-83f8-a186bad4f2cc", "bridge": "br-int", "label": "tempest-ServerActionsTestOtherA-486369423-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "c1cb3d1f12f242a785a688d59dbe3cd6", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap66dcb0f9-31", "ovs_interfaceid": "66dcb0f9-3180-42f7-851d-19b0c4f4119c", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:17:55 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-20b9b646-0ade-4a5a-958f-876a2874c2ff req-2fccab08-3c10-4e1d-9d70-447b9d584bb6 service nova] Releasing lock "refresh_cache-af699bc2-9daa-4b5a-9b4a-ad92fe857d70" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:17:55 user nova-compute[75234]: DEBUG nova.compute.manager [req-39bda395-a6d4-4336-9410-30c484dc14ca req-0e4352b9-3b0f-4905-b232-9ea6d4a59c7b service nova] [instance: 1c8b7559-29ab-4982-8c75-ea2a0f0101cb] Received event network-changed-22cdbc69-d4f4-49cd-8104-d508aea94101 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:17:55 user nova-compute[75234]: DEBUG nova.compute.manager [req-39bda395-a6d4-4336-9410-30c484dc14ca req-0e4352b9-3b0f-4905-b232-9ea6d4a59c7b service nova] [instance: 1c8b7559-29ab-4982-8c75-ea2a0f0101cb] Refreshing instance network info cache due to event network-changed-22cdbc69-d4f4-49cd-8104-d508aea94101. {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11117}}
Nov 29 03:17:55 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-39bda395-a6d4-4336-9410-30c484dc14ca req-0e4352b9-3b0f-4905-b232-9ea6d4a59c7b service nova] Acquiring lock "refresh_cache-1c8b7559-29ab-4982-8c75-ea2a0f0101cb" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:17:55 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-39bda395-a6d4-4336-9410-30c484dc14ca req-0e4352b9-3b0f-4905-b232-9ea6d4a59c7b service nova] Acquired lock "refresh_cache-1c8b7559-29ab-4982-8c75-ea2a0f0101cb" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:17:55 user nova-compute[75234]: DEBUG nova.network.neutron [req-39bda395-a6d4-4336-9410-30c484dc14ca req-0e4352b9-3b0f-4905-b232-9ea6d4a59c7b service nova] [instance: 1c8b7559-29ab-4982-8c75-ea2a0f0101cb] Refreshing network info cache for port 22cdbc69-d4f4-49cd-8104-d508aea94101 {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2006}}
Nov 29 03:17:56 user nova-compute[75234]: DEBUG nova.network.neutron [req-39bda395-a6d4-4336-9410-30c484dc14ca req-0e4352b9-3b0f-4905-b232-9ea6d4a59c7b service nova] [instance: 1c8b7559-29ab-4982-8c75-ea2a0f0101cb] Updated VIF entry in instance network info cache for port 22cdbc69-d4f4-49cd-8104-d508aea94101. {{(pid=75234) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3481}}
Nov 29 03:17:56 user nova-compute[75234]: DEBUG nova.network.neutron [req-39bda395-a6d4-4336-9410-30c484dc14ca req-0e4352b9-3b0f-4905-b232-9ea6d4a59c7b service nova] [instance: 1c8b7559-29ab-4982-8c75-ea2a0f0101cb] Updating instance_info_cache with network_info: [{"id": "22cdbc69-d4f4-49cd-8104-d508aea94101", "address": "fa:16:3e:79:d1:ca", "network": {"id": "b7a6cef0-4cf1-46e6-99fb-4972879e3913", "bridge": "br-int", "label": "tempest-TestStampPattern-1195861223-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.102", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "c4393bc598a845a9bfb667e5332f14a6", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap22cdbc69-d4", "ovs_interfaceid": "22cdbc69-d4f4-49cd-8104-d508aea94101", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:17:56 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-9bd15904-212d-458a-a01a-a6fdf8021a15 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Acquiring lock "1c8b7559-29ab-4982-8c75-ea2a0f0101cb" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:17:56 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-9bd15904-212d-458a-a01a-a6fdf8021a15 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Lock "1c8b7559-29ab-4982-8c75-ea2a0f0101cb" acquired by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:17:56 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-39bda395-a6d4-4336-9410-30c484dc14ca req-0e4352b9-3b0f-4905-b232-9ea6d4a59c7b service nova] Releasing lock "refresh_cache-1c8b7559-29ab-4982-8c75-ea2a0f0101cb" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:17:56 user nova-compute[75234]: DEBUG nova.objects.instance [None req-9bd15904-212d-458a-a01a-a6fdf8021a15 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Lazy-loading 'flavor' on Instance uuid 1c8b7559-29ab-4982-8c75-ea2a0f0101cb {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:17:56 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-9bd15904-212d-458a-a01a-a6fdf8021a15 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Lock "1c8b7559-29ab-4982-8c75-ea2a0f0101cb" "released" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: held 0.063s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:17:56 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:17:56 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-9bd15904-212d-458a-a01a-a6fdf8021a15 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Acquiring lock "1c8b7559-29ab-4982-8c75-ea2a0f0101cb" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:17:56 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-9bd15904-212d-458a-a01a-a6fdf8021a15 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Lock "1c8b7559-29ab-4982-8c75-ea2a0f0101cb" acquired by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:17:56 user nova-compute[75234]: INFO nova.compute.manager [None req-9bd15904-212d-458a-a01a-a6fdf8021a15 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] [instance: 1c8b7559-29ab-4982-8c75-ea2a0f0101cb] Attaching volume a5b104bd-399e-4445-a186-0d4275ae271a to /dev/vdb
Nov 29 03:17:56 user nova-compute[75234]: DEBUG os_brick.utils [None req-9bd15904-212d-458a-a01a-a6fdf8021a15 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:176}}
Nov 29 03:17:56 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:17:56 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.015s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:17:56 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[2b636121-c40d-4538-967c-f627327262b9]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:dd1368c5ba2c\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:17:56 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt -v / -n -o SOURCE {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:17:56 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt -v / -n -o SOURCE" returned: 0 in 0.017s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:17:56 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[b0d2cb17-3f0c-4b27-9925-7ac3e8952540]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:17:56 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:17:56 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.015s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:17:56 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[f7762267-209f-40ed-9b1b-32ce4a2470f5]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:17:56 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[cd52a125-17d3-42ab-a20a-d505967af7e2]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:17:56 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-9bd15904-212d-458a-a01a-a6fdf8021a15 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Running cmd (subprocess): nvme version {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:17:56 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-9bd15904-212d-458a-a01a-a6fdf8021a15 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] 'nvme version' failed. Not Retrying. {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:473}}
Nov 29 03:17:56 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.nvmeof [None req-9bd15904-212d-458a-a01a-a6fdf8021a15 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] nvme not present on system {{(pid=75234) nvme_present /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/nvmeof.py:756}}
Nov 29 03:17:56 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:17:56 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:473}}
Nov 29 03:17:56 user nova-compute[75234]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Nov 29 03:17:56 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[992d8dab-deaa-42cf-8218-652af1aa8892]: (4, '') {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:17:56 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.lightos [None req-9bd15904-212d-458a-a01a-a6fdf8021a15 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=75234) find_dsc /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/lightos.py:98}}
Nov 29 03:17:56 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.lightos [None req-9bd15904-212d-458a-a01a-a6fdf8021a15 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=75234) get_connector_properties /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/lightos.py:76}}
Nov 29 03:17:56 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.lightos [None req-9bd15904-212d-458a-a01a-a6fdf8021a15 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] LIGHTOS: no hostnqn found. {{(pid=75234) get_connector_properties /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/lightos.py:84}}
Nov 29 03:17:56 user nova-compute[75234]: DEBUG os_brick.utils [None req-9bd15904-212d-458a-a01a-a6fdf8021a15 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] <== get_connector_properties: return (129ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:dd1368c5ba2c', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:203}}
Nov 29 03:17:56 user nova-compute[75234]: DEBUG nova.virt.block_device [None req-9bd15904-212d-458a-a01a-a6fdf8021a15 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] [instance: 1c8b7559-29ab-4982-8c75-ea2a0f0101cb] Updating existing volume attachment record: 0a5086ce-0b00-4204-8673-4d7494917fc7 {{(pid=75234) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Nov 29 03:17:56 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-e1d70590-a225-41ba-a336-cbac91a315dc tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] Acquiring lock "95b6eb15-7930-4aa5-aff8-73809b0a9277" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:17:56 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-e1d70590-a225-41ba-a336-cbac91a315dc tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] Lock "95b6eb15-7930-4aa5-aff8-73809b0a9277" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:17:56 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-e1d70590-a225-41ba-a336-cbac91a315dc tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] Acquiring lock "95b6eb15-7930-4aa5-aff8-73809b0a9277-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:17:56 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-e1d70590-a225-41ba-a336-cbac91a315dc tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] Lock "95b6eb15-7930-4aa5-aff8-73809b0a9277-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:17:56 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-e1d70590-a225-41ba-a336-cbac91a315dc tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] Lock "95b6eb15-7930-4aa5-aff8-73809b0a9277-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:17:56 user nova-compute[75234]: INFO nova.compute.manager [None req-e1d70590-a225-41ba-a336-cbac91a315dc tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] [instance: 95b6eb15-7930-4aa5-aff8-73809b0a9277] Terminating instance
Nov 29 03:17:56 user nova-compute[75234]: DEBUG nova.compute.manager [None req-e1d70590-a225-41ba-a336-cbac91a315dc tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] [instance: 95b6eb15-7930-4aa5-aff8-73809b0a9277] Start destroying the instance on the hypervisor. {{(pid=75234) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3140}}
Nov 29 03:17:56 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:17:56 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:17:57 user nova-compute[75234]: DEBUG nova.compute.manager [req-0735b7d2-b411-450d-9ac1-1cdaceec6da8 req-821da944-b94f-4485-b77b-a00d2900b6d3 service nova] [instance: 95b6eb15-7930-4aa5-aff8-73809b0a9277] Received event network-vif-unplugged-64da7883-fc79-44ea-a9d8-65a3f65cfc82 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:17:57 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-0735b7d2-b411-450d-9ac1-1cdaceec6da8 req-821da944-b94f-4485-b77b-a00d2900b6d3 service nova] Acquiring lock "95b6eb15-7930-4aa5-aff8-73809b0a9277-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:17:57 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-0735b7d2-b411-450d-9ac1-1cdaceec6da8 req-821da944-b94f-4485-b77b-a00d2900b6d3 service nova] Lock "95b6eb15-7930-4aa5-aff8-73809b0a9277-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:17:57 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-0735b7d2-b411-450d-9ac1-1cdaceec6da8 req-821da944-b94f-4485-b77b-a00d2900b6d3 service nova] Lock "95b6eb15-7930-4aa5-aff8-73809b0a9277-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:17:57 user nova-compute[75234]: DEBUG nova.compute.manager [req-0735b7d2-b411-450d-9ac1-1cdaceec6da8 req-821da944-b94f-4485-b77b-a00d2900b6d3 service nova] [instance: 95b6eb15-7930-4aa5-aff8-73809b0a9277] No waiting events found dispatching network-vif-unplugged-64da7883-fc79-44ea-a9d8-65a3f65cfc82 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:17:57 user nova-compute[75234]: DEBUG nova.compute.manager [req-0735b7d2-b411-450d-9ac1-1cdaceec6da8 req-821da944-b94f-4485-b77b-a00d2900b6d3 service nova] [instance: 95b6eb15-7930-4aa5-aff8-73809b0a9277] Received event network-vif-unplugged-64da7883-fc79-44ea-a9d8-65a3f65cfc82 for instance with task_state deleting. {{(pid=75234) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10890}}
Nov 29 03:17:57 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:17:57 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:17:57 user nova-compute[75234]: INFO nova.virt.libvirt.driver [-] [instance: 95b6eb15-7930-4aa5-aff8-73809b0a9277] Instance destroyed successfully.
Nov 29 03:17:57 user nova-compute[75234]: DEBUG nova.objects.instance [None req-e1d70590-a225-41ba-a336-cbac91a315dc tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] Lazy-loading 'resources' on Instance uuid 95b6eb15-7930-4aa5-aff8-73809b0a9277 {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:17:57 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-e1d70590-a225-41ba-a336-cbac91a315dc tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-11-29T03:15:37Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-VolumesAdminNegativeTest-server-1869912699',display_name='tempest-VolumesAdminNegativeTest-server-1869912699',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-volumesadminnegativetest-server-1869912699',id=24,image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data=None,key_name=None,keypairs=<?>,launch_index=0,launched_at=2023-11-29T03:15:45Z,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='5eaa91ebc2e64f2eba987f02bcd1a9f4',ramdisk_id='',reservation_id='r-4m5vs2pl',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-VolumesAdminNegativeTest-1445338761',owner_user_name='tempest-VolumesAdminNegativeTest-1445338761-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-11-29T03:15:46Z,user_data=None,user_id='460854be8e05408a80790620f64cf8e1',uuid=95b6eb15-7930-4aa5-aff8-73809b0a9277,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "64da7883-fc79-44ea-a9d8-65a3f65cfc82", "address": "fa:16:3e:23:9e:1e", "network": {"id": "49d62bf6-34ee-4de8-ab85-6ef489d957c8", "bridge": "br-int", "label": "tempest-VolumesAdminNegativeTest-1304144113-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "5eaa91ebc2e64f2eba987f02bcd1a9f4", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap64da7883-fc", "ovs_interfaceid": "64da7883-fc79-44ea-a9d8-65a3f65cfc82", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Nov 29 03:17:57 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-e1d70590-a225-41ba-a336-cbac91a315dc tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] Converting VIF {"id": "64da7883-fc79-44ea-a9d8-65a3f65cfc82", "address": "fa:16:3e:23:9e:1e", "network": {"id": "49d62bf6-34ee-4de8-ab85-6ef489d957c8", "bridge": "br-int", "label": "tempest-VolumesAdminNegativeTest-1304144113-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "5eaa91ebc2e64f2eba987f02bcd1a9f4", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap64da7883-fc", "ovs_interfaceid": "64da7883-fc79-44ea-a9d8-65a3f65cfc82", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:17:57 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-e1d70590-a225-41ba-a336-cbac91a315dc tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:23:9e:1e,bridge_name='br-int',has_traffic_filtering=True,id=64da7883-fc79-44ea-a9d8-65a3f65cfc82,network=Network(49d62bf6-34ee-4de8-ab85-6ef489d957c8),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap64da7883-fc') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:17:57 user nova-compute[75234]: DEBUG os_vif [None req-e1d70590-a225-41ba-a336-cbac91a315dc tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] Unplugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:23:9e:1e,bridge_name='br-int',has_traffic_filtering=True,id=64da7883-fc79-44ea-a9d8-65a3f65cfc82,network=Network(49d62bf6-34ee-4de8-ab85-6ef489d957c8),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap64da7883-fc') {{(pid=75234) unplug /opt/stack/data/venv/lib/python3.10/site-packages/os_vif/__init__.py:109}}
Nov 29 03:17:57 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:17:57 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tap64da7883-fc, bridge=br-int, if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:17:57 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:17:57 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:17:57 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:17:57 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbDestroyCommand(_result=None, table=QoS, record=5d6fb3b1-1647-457f-9091-27df26eb3a7c) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:17:57 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:17:57 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:17:57 user nova-compute[75234]: INFO os_vif [None req-e1d70590-a225-41ba-a336-cbac91a315dc tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] Successfully unplugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:23:9e:1e,bridge_name='br-int',has_traffic_filtering=True,id=64da7883-fc79-44ea-a9d8-65a3f65cfc82,network=Network(49d62bf6-34ee-4de8-ab85-6ef489d957c8),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap64da7883-fc')
Nov 29 03:17:57 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-e1d70590-a225-41ba-a336-cbac91a315dc tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] [instance: 95b6eb15-7930-4aa5-aff8-73809b0a9277] Deleting instance files /opt/stack/data/nova/instances/95b6eb15-7930-4aa5-aff8-73809b0a9277_del
Nov 29 03:17:57 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-e1d70590-a225-41ba-a336-cbac91a315dc tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] [instance: 95b6eb15-7930-4aa5-aff8-73809b0a9277] Deletion of /opt/stack/data/nova/instances/95b6eb15-7930-4aa5-aff8-73809b0a9277_del complete
Nov 29 03:17:57 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:17:57 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Starting heal instance info cache {{(pid=75234) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9922}}
Nov 29 03:17:57 user nova-compute[75234]: INFO nova.compute.manager [None req-e1d70590-a225-41ba-a336-cbac91a315dc tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] [instance: 95b6eb15-7930-4aa5-aff8-73809b0a9277] Took 0.72 seconds to destroy the instance on the hypervisor.
Nov 29 03:17:57 user nova-compute[75234]: DEBUG oslo.service.loopingcall [None req-e1d70590-a225-41ba-a336-cbac91a315dc tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=75234) func /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/loopingcall.py:435}}
Nov 29 03:17:57 user nova-compute[75234]: DEBUG nova.compute.manager [-] [instance: 95b6eb15-7930-4aa5-aff8-73809b0a9277] Deallocating network for instance {{(pid=75234) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2279}}
Nov 29 03:17:57 user nova-compute[75234]: DEBUG nova.network.neutron [-] [instance: 95b6eb15-7930-4aa5-aff8-73809b0a9277] deallocate_for_instance() {{(pid=75234) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1802}}
Nov 29 03:17:57 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Acquiring lock "refresh_cache-13e5da64-9adc-4479-81c7-d329e9b5f08f" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:17:57 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Acquired lock "refresh_cache-13e5da64-9adc-4479-81c7-d329e9b5f08f" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:17:57 user nova-compute[75234]: DEBUG nova.network.neutron [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] [instance: 13e5da64-9adc-4479-81c7-d329e9b5f08f] Forcefully refreshing network info cache for instance {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2003}}
Nov 29 03:17:58 user nova-compute[75234]: DEBUG nova.network.neutron [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] [instance: 13e5da64-9adc-4479-81c7-d329e9b5f08f] Updating instance_info_cache with network_info: [{"id": "2abf9e25-d1d9-4893-899e-912d41b7c38f", "address": "fa:16:3e:59:79:35", "network": {"id": "2b3c3140-f3b6-45da-8202-bb4d7f0dd130", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-2043420778-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "f8a7884867c944e1ba1024270e4902f3", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2abf9e25-d1", "ovs_interfaceid": "2abf9e25-d1d9-4893-899e-912d41b7c38f", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:17:58 user nova-compute[75234]: DEBUG nova.network.neutron [-] [instance: 95b6eb15-7930-4aa5-aff8-73809b0a9277] Updating instance_info_cache with network_info: [] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:17:58 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Releasing lock "refresh_cache-13e5da64-9adc-4479-81c7-d329e9b5f08f" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:17:58 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] [instance: 13e5da64-9adc-4479-81c7-d329e9b5f08f] Updated the network info_cache for instance {{(pid=75234) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9993}}
Nov 29 03:17:58 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:17:58 user nova-compute[75234]: INFO nova.compute.manager [-] [instance: 95b6eb15-7930-4aa5-aff8-73809b0a9277] Took 0.51 seconds to deallocate network for instance.
Nov 29 03:17:58 user nova-compute[75234]: DEBUG nova.compute.manager [req-6b2ce829-9e15-47f1-a8e3-0fb6d70bd6d9 req-a9c4cf02-4b5c-44e4-a1ee-6490f627e60a service nova] [instance: 95b6eb15-7930-4aa5-aff8-73809b0a9277] Received event network-vif-deleted-64da7883-fc79-44ea-a9d8-65a3f65cfc82 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:17:58 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-e1d70590-a225-41ba-a336-cbac91a315dc tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:17:58 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-e1d70590-a225-41ba-a336-cbac91a315dc tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:17:58 user nova-compute[75234]: DEBUG nova.compute.provider_tree [None req-e1d70590-a225-41ba-a336-cbac91a315dc tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] Inventory has not changed in ProviderTree for provider: 4d291220-7c37-4839-a0e3-b23c66502b74 {{(pid=75234) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Nov 29 03:17:58 user nova-compute[75234]: DEBUG nova.scheduler.client.report [None req-e1d70590-a225-41ba-a336-cbac91a315dc tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] Inventory has not changed for provider 4d291220-7c37-4839-a0e3-b23c66502b74 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=75234) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Nov 29 03:17:58 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-e1d70590-a225-41ba-a336-cbac91a315dc tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.491s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:17:58 user nova-compute[75234]: INFO nova.scheduler.client.report [None req-e1d70590-a225-41ba-a336-cbac91a315dc tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] Deleted allocations for instance 95b6eb15-7930-4aa5-aff8-73809b0a9277
Nov 29 03:17:58 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-e1d70590-a225-41ba-a336-cbac91a315dc tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] Lock "95b6eb15-7930-4aa5-aff8-73809b0a9277" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 2.005s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:17:59 user nova-compute[75234]: DEBUG nova.compute.manager [req-ed723785-235b-4626-a8b7-b3fb81be477b req-85510b3d-9c9c-4bf4-a3d0-d0c876dd8e38 service nova] [instance: 95b6eb15-7930-4aa5-aff8-73809b0a9277] Received event network-vif-plugged-64da7883-fc79-44ea-a9d8-65a3f65cfc82 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:17:59 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-ed723785-235b-4626-a8b7-b3fb81be477b req-85510b3d-9c9c-4bf4-a3d0-d0c876dd8e38 service nova] Acquiring lock "95b6eb15-7930-4aa5-aff8-73809b0a9277-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:17:59 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-ed723785-235b-4626-a8b7-b3fb81be477b req-85510b3d-9c9c-4bf4-a3d0-d0c876dd8e38 service nova] Lock "95b6eb15-7930-4aa5-aff8-73809b0a9277-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:17:59 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-ed723785-235b-4626-a8b7-b3fb81be477b req-85510b3d-9c9c-4bf4-a3d0-d0c876dd8e38 service nova] Lock "95b6eb15-7930-4aa5-aff8-73809b0a9277-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:17:59 user nova-compute[75234]: DEBUG nova.compute.manager [req-ed723785-235b-4626-a8b7-b3fb81be477b req-85510b3d-9c9c-4bf4-a3d0-d0c876dd8e38 service nova] [instance: 95b6eb15-7930-4aa5-aff8-73809b0a9277] No waiting events found dispatching network-vif-plugged-64da7883-fc79-44ea-a9d8-65a3f65cfc82 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:17:59 user nova-compute[75234]: WARNING nova.compute.manager [req-ed723785-235b-4626-a8b7-b3fb81be477b req-85510b3d-9c9c-4bf4-a3d0-d0c876dd8e38 service nova] [instance: 95b6eb15-7930-4aa5-aff8-73809b0a9277] Received unexpected event network-vif-plugged-64da7883-fc79-44ea-a9d8-65a3f65cfc82 for instance with vm_state deleted and task_state None.
Nov 29 03:17:59 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:17:59 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:17:59 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=75234) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10541}}
Nov 29 03:17:59 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-9e5c7588-51ec-49d2-8d1a-cca0dabbdd7b tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] [instance: 75c5d700-6367-4128-9228-80d2cd24ace7] Calling os-brick to attach iSCSI Volume {{(pid=75234) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Nov 29 03:17:59 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-9e5c7588-51ec-49d2-8d1a-cca0dabbdd7b tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f8a5050c9a0>, 'connection_properties': {'target_iqn': 'iqn.2010-10.org.openstack:bf57b967-7fad-4d76-abb2-319fe88f8ef9', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'zuQKYWgv', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:176}}
Nov 29 03:17:59 user nova-compute[75234]: WARNING os_brick.initiator.connectors.base [None req-9e5c7588-51ec-49d2-8d1a-cca0dabbdd7b tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Nov 29 03:17:59 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-9e5c7588-51ec-49d2-8d1a-cca0dabbdd7b tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:68}}
Nov 29 03:17:59 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-9e5c7588-51ec-49d2-8d1a-cca0dabbdd7b tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:73}}
Nov 29 03:17:59 user nova-compute[75234]: WARNING os_brick.initiator.connectors.base [None req-9e5c7588-51ec-49d2-8d1a-cca0dabbdd7b tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Nov 29 03:17:59 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-9e5c7588-51ec-49d2-8d1a-cca0dabbdd7b tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:bf57b967-7fad-4d76-abb2-319fe88f8ef9" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:68}}
Nov 29 03:17:59 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-9e5c7588-51ec-49d2-8d1a-cca0dabbdd7b tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:bf57b967-7fad-4d76-abb2-319fe88f8ef9" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:73}}
Nov 29 03:17:59 user nova-compute[75234]: INFO os_brick.initiator.connectors.iscsi [None req-9e5c7588-51ec-49d2-8d1a-cca0dabbdd7b tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Nov 29 03:17:59 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:bf57b967-7fad-4d76-abb2-319fe88f8ef9 -p 172.16.0.220:3260 {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:17:59 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:bf57b967-7fad-4d76-abb2-319fe88f8ef9 -p 172.16.0.220:3260" returned: 21 in 0.019s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:17:59 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[d9300bdc-3f72-4ee1-9016-227c38b848ca]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:17:59 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-9e5c7588-51ec-49d2-8d1a-cca0dabbdd7b tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Nov 29 03:17:59 user nova-compute[75234]:  {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:17:59 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:bf57b967-7fad-4d76-abb2-319fe88f8ef9 -p 172.16.0.220:3260 --interface default --op new {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:17:59 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:bf57b967-7fad-4d76-abb2-319fe88f8ef9 -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.018s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:17:59 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[13b17295-aeec-4c46-a883-66ad1a6dc807]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:bf57b967-7fad-4d76-abb2-319fe88f8ef9] added\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:17:59 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-9e5c7588-51ec-49d2-8d1a-cca0dabbdd7b tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:bf57b967-7fad-4d76-abb2-319fe88f8ef9] added
Nov 29 03:17:59 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:17:59 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:bf57b967-7fad-4d76-abb2-319fe88f8ef9 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:17:59 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:bf57b967-7fad-4d76-abb2-319fe88f8ef9 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.024s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:17:59 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[cda84873-5f87-4255-85ba-e498e78d9df5]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:17:59 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-9e5c7588-51ec-49d2-8d1a-cca0dabbdd7b tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:17:59 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:bf57b967-7fad-4d76-abb2-319fe88f8ef9 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:17:59 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:bf57b967-7fad-4d76-abb2-319fe88f8ef9 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.018s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:17:59 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[14aa4a49-e9fa-4442-aaaf-0600d72f6d2e]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:17:59 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-9e5c7588-51ec-49d2-8d1a-cca0dabbdd7b tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:17:59 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:bf57b967-7fad-4d76-abb2-319fe88f8ef9 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v zuQKYWgv {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:17:59 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:bf57b967-7fad-4d76-abb2-319fe88f8ef9 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v zuQKYWgv" returned: 0 in 0.020s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:17:59 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[6217092a-79e6-4f61-b765-f647cd6e8ea3]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:17:59 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-9e5c7588-51ec-49d2-8d1a-cca0dabbdd7b tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'zuQKYWgv'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:17:59 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:bf57b967-7fad-4d76-abb2-319fe88f8ef9 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:17:59 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:bf57b967-7fad-4d76-abb2-319fe88f8ef9 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.023s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:17:59 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[5540ca4c-4ac2-46c8-accd-a73610a68f5b]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:17:59 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-9e5c7588-51ec-49d2-8d1a-cca0dabbdd7b tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:17:59 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:18:00 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.022s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:18:00 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[6945d187-82cf-468d-9773-78fb4283ab58]: (4, ('tcp: [27] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:07f83efc-272d-4e10-9a2b-48d967db613c (non-flash)\ntcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6c2e1179-dd3c-41c4-9594-bf4a16752c67 (non-flash)\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:18:00 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-9e5c7588-51ec-49d2-8d1a-cca0dabbdd7b tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [27] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:07f83efc-272d-4e10-9a2b-48d967db613c (non-flash)
Nov 29 03:18:00 user nova-compute[75234]: tcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6c2e1179-dd3c-41c4-9594-bf4a16752c67 (non-flash)
Nov 29 03:18:00 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:18:00 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-9e5c7588-51ec-49d2-8d1a-cca0dabbdd7b tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] iscsi session list stdout=tcp: [27] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:07f83efc-272d-4e10-9a2b-48d967db613c (non-flash)
Nov 29 03:18:00 user nova-compute[75234]: tcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6c2e1179-dd3c-41c4-9594-bf4a16752c67 (non-flash)
Nov 29 03:18:00 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsi_session /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Nov 29 03:18:00 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:bf57b967-7fad-4d76-abb2-319fe88f8ef9 -p 172.16.0.220:3260 --login {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:18:00 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:bf57b967-7fad-4d76-abb2-319fe88f8ef9 -p 172.16.0.220:3260 --login" returned: 0 in 0.059s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:18:00 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[d6448c80-859c-4191-848a-fab9af125f6f]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:bf57b967-7fad-4d76-abb2-319fe88f8ef9, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:bf57b967-7fad-4d76-abb2-319fe88f8ef9, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:18:00 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-9e5c7588-51ec-49d2-8d1a-cca0dabbdd7b tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:bf57b967-7fad-4d76-abb2-319fe88f8ef9, portal: 172.16.0.220,3260]
Nov 29 03:18:00 user nova-compute[75234]: Login to [iface: default, target: iqn.2010-10.org.openstack:bf57b967-7fad-4d76-abb2-319fe88f8ef9, portal: 172.16.0.220,3260] successful.
Nov 29 03:18:00 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:18:00 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:bf57b967-7fad-4d76-abb2-319fe88f8ef9 -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:18:00 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:bf57b967-7fad-4d76-abb2-319fe88f8ef9 -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.019s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:18:00 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[9803936f-10eb-4562-8d99-b2a1bceb1289]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:18:00 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-9e5c7588-51ec-49d2-8d1a-cca0dabbdd7b tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:18:00 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:18:00 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.031s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:18:00 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[9c3ea265-3514-4e94-9a8f-1c164d353bd4]: (4, ('tcp: [27] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:07f83efc-272d-4e10-9a2b-48d967db613c (non-flash)\ntcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6c2e1179-dd3c-41c4-9594-bf4a16752c67 (non-flash)\ntcp: [29] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bf57b967-7fad-4d76-abb2-319fe88f8ef9 (non-flash)\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:18:00 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-9e5c7588-51ec-49d2-8d1a-cca0dabbdd7b tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [27] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:07f83efc-272d-4e10-9a2b-48d967db613c (non-flash)
Nov 29 03:18:00 user nova-compute[75234]: tcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6c2e1179-dd3c-41c4-9594-bf4a16752c67 (non-flash)
Nov 29 03:18:00 user nova-compute[75234]: tcp: [29] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bf57b967-7fad-4d76-abb2-319fe88f8ef9 (non-flash)
Nov 29 03:18:00 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:18:00 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-9e5c7588-51ec-49d2-8d1a-cca0dabbdd7b tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] iscsi session list stdout=tcp: [27] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:07f83efc-272d-4e10-9a2b-48d967db613c (non-flash)
Nov 29 03:18:00 user nova-compute[75234]: tcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6c2e1179-dd3c-41c4-9594-bf4a16752c67 (non-flash)
Nov 29 03:18:00 user nova-compute[75234]: tcp: [29] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bf57b967-7fad-4d76-abb2-319fe88f8ef9 (non-flash)
Nov 29 03:18:00 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsi_session /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Nov 29 03:18:00 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-9e5c7588-51ec-49d2-8d1a-cca0dabbdd7b tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:bf57b967-7fad-4d76-abb2-319fe88f8ef9" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.304s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:87}}
Nov 29 03:18:00 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-9e5c7588-51ec-49d2-8d1a-cca0dabbdd7b tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Connected to 172.16.0.220:3260 {{(pid=75234) _connect_vol /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:644}}
Nov 29 03:18:00 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-9e5c7588-51ec-49d2-8d1a-cca0dabbdd7b tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] HCTL ('35', '-', '-', 0) found on session 29 with lun 0 {{(pid=75234) get_hctl /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:770}}
Nov 29 03:18:00 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-9e5c7588-51ec-49d2-8d1a-cca0dabbdd7b tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Scanning host 35 c: -, t: -, l: 0) {{(pid=75234) scan_iscsi /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:795}}
Nov 29 03:18:00 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host35/scan {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:18:00 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host35/scan" returned: 0 in 0.018s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:18:00 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[d893a21c-f667-48d5-80cd-a78a851494e3]: (4, ('- - 0', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:18:00 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-9e5c7588-51ec-49d2-8d1a-cca0dabbdd7b tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Searching for a device in session 29 and hctl ['35', '*', '*', 0] yield: None {{(pid=75234) device_name_by_hctl /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:789}}
Nov 29 03:18:00 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager.update_available_resource {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:18:00 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:18:00 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:18:00 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:18:00 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Auditing locally available compute resources for user (node: user) {{(pid=75234) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:907}}
Nov 29 03:18:00 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/13e5da64-9adc-4479-81c7-d329e9b5f08f/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:18:00 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/13e5da64-9adc-4479-81c7-d329e9b5f08f/disk --force-share --output=json" returned: 0 in 0.211s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:18:00 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/13e5da64-9adc-4479-81c7-d329e9b5f08f/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:18:01 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/13e5da64-9adc-4479-81c7-d329e9b5f08f/disk --force-share --output=json" returned: 0 in 0.183s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:18:01 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/1c8b7559-29ab-4982-8c75-ea2a0f0101cb/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:18:01 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-9e5c7588-51ec-49d2-8d1a-cca0dabbdd7b tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Searching for a device in session 29 and hctl ['35', '*', '*', 0] yield: sdd {{(pid=75234) device_name_by_hctl /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:789}}
Nov 29 03:18:01 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-9e5c7588-51ec-49d2-8d1a-cca0dabbdd7b tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Connected to sdd using {'target_iqn': 'iqn.2010-10.org.openstack:bf57b967-7fad-4d76-abb2-319fe88f8ef9', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'zuQKYWgv', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False} {{(pid=75234) _connect_vol /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:672}}
Nov 29 03:18:01 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-9e5c7588-51ec-49d2-8d1a-cca0dabbdd7b tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.378s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:87}}
Nov 29 03:18:01 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-9e5c7588-51ec-49d2-8d1a-cca0dabbdd7b tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] <== connect_volume: return (1380ms) {'type': 'block', 'scsi_wwn': '23533636563366235', 'path': '/dev/sdd'} {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:203}}
Nov 29 03:18:01 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-9e5c7588-51ec-49d2-8d1a-cca0dabbdd7b tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] [instance: 75c5d700-6367-4128-9228-80d2cd24ace7] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '23533636563366235', 'path': '/dev/sdd'} {{(pid=75234) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Nov 29 03:18:01 user nova-compute[75234]: DEBUG nova.compute.manager [req-3e8fcaaf-bf38-48ae-8a47-2f137d5e43af req-107b3aa3-7b06-4033-a375-8b2fd7f225f6 service nova] [instance: aa190a9f-ba53-43fb-8bae-2c429d0d1228] Received event network-changed-2cc37e6a-5c9f-4bb8-98fa-f4f2b94d0c35 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:18:01 user nova-compute[75234]: DEBUG nova.compute.manager [req-3e8fcaaf-bf38-48ae-8a47-2f137d5e43af req-107b3aa3-7b06-4033-a375-8b2fd7f225f6 service nova] [instance: aa190a9f-ba53-43fb-8bae-2c429d0d1228] Refreshing instance network info cache due to event network-changed-2cc37e6a-5c9f-4bb8-98fa-f4f2b94d0c35. {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11117}}
Nov 29 03:18:01 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-3e8fcaaf-bf38-48ae-8a47-2f137d5e43af req-107b3aa3-7b06-4033-a375-8b2fd7f225f6 service nova] Acquiring lock "refresh_cache-aa190a9f-ba53-43fb-8bae-2c429d0d1228" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:18:01 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-3e8fcaaf-bf38-48ae-8a47-2f137d5e43af req-107b3aa3-7b06-4033-a375-8b2fd7f225f6 service nova] Acquired lock "refresh_cache-aa190a9f-ba53-43fb-8bae-2c429d0d1228" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:18:01 user nova-compute[75234]: DEBUG nova.network.neutron [req-3e8fcaaf-bf38-48ae-8a47-2f137d5e43af req-107b3aa3-7b06-4033-a375-8b2fd7f225f6 service nova] [instance: aa190a9f-ba53-43fb-8bae-2c429d0d1228] Refreshing network info cache for port 2cc37e6a-5c9f-4bb8-98fa-f4f2b94d0c35 {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2006}}
Nov 29 03:18:01 user nova-compute[75234]: DEBUG nova.objects.instance [None req-9e5c7588-51ec-49d2-8d1a-cca0dabbdd7b tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Lazy-loading 'flavor' on Instance uuid 75c5d700-6367-4128-9228-80d2cd24ace7 {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:18:01 user nova-compute[75234]: DEBUG nova.virt.libvirt.guest [None req-9e5c7588-51ec-49d2-8d1a-cca0dabbdd7b tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] attach device xml: <disk type="block" device="disk">
Nov 29 03:18:01 user nova-compute[75234]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Nov 29 03:18:01 user nova-compute[75234]:   <source dev="/dev/sdd"/>
Nov 29 03:18:01 user nova-compute[75234]:   <target dev="vdb" bus="virtio"/>
Nov 29 03:18:01 user nova-compute[75234]:   <serial>bf57b967-7fad-4d76-abb2-319fe88f8ef9</serial>
Nov 29 03:18:01 user nova-compute[75234]: </disk>
Nov 29 03:18:01 user nova-compute[75234]:  {{(pid=75234) attach_device /opt/stack/nova/nova/virt/libvirt/guest.py:339}}
Nov 29 03:18:01 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/1c8b7559-29ab-4982-8c75-ea2a0f0101cb/disk --force-share --output=json" returned: 0 in 0.197s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:18:01 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/1c8b7559-29ab-4982-8c75-ea2a0f0101cb/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:18:01 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/1c8b7559-29ab-4982-8c75-ea2a0f0101cb/disk --force-share --output=json" returned: 0 in 0.251s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:18:01 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/99d6f85f-1b77-4390-8b61-908c98f584c0/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:18:01 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-9e5c7588-51ec-49d2-8d1a-cca0dabbdd7b tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] No BDM found with device name vda, not building metadata. {{(pid=75234) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12089}}
Nov 29 03:18:01 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-9e5c7588-51ec-49d2-8d1a-cca0dabbdd7b tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] No BDM found with device name vdb, not building metadata. {{(pid=75234) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12089}}
Nov 29 03:18:01 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-9e5c7588-51ec-49d2-8d1a-cca0dabbdd7b tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] No VIF found with MAC fa:16:3e:4e:5d:e9, not building metadata {{(pid=75234) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Nov 29 03:18:01 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/99d6f85f-1b77-4390-8b61-908c98f584c0/disk --force-share --output=json" returned: 0 in 0.254s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:18:01 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/99d6f85f-1b77-4390-8b61-908c98f584c0/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:18:01 user nova-compute[75234]: DEBUG nova.network.neutron [req-3e8fcaaf-bf38-48ae-8a47-2f137d5e43af req-107b3aa3-7b06-4033-a375-8b2fd7f225f6 service nova] [instance: aa190a9f-ba53-43fb-8bae-2c429d0d1228] Updated VIF entry in instance network info cache for port 2cc37e6a-5c9f-4bb8-98fa-f4f2b94d0c35. {{(pid=75234) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3481}}
Nov 29 03:18:01 user nova-compute[75234]: DEBUG nova.network.neutron [req-3e8fcaaf-bf38-48ae-8a47-2f137d5e43af req-107b3aa3-7b06-4033-a375-8b2fd7f225f6 service nova] [instance: aa190a9f-ba53-43fb-8bae-2c429d0d1228] Updating instance_info_cache with network_info: [{"id": "2cc37e6a-5c9f-4bb8-98fa-f4f2b94d0c35", "address": "fa:16:3e:fb:40:18", "network": {"id": "38517dcf-64c7-4d26-9083-5ea5f6c87c2f", "bridge": "br-int", "label": "tempest-ServerActionsV293TestJSON-266920726-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.208", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "581a0b62fee744e1b9a767a0d011c4e0", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2cc37e6a-5c", "ovs_interfaceid": "2cc37e6a-5c9f-4bb8-98fa-f4f2b94d0c35", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:18:01 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:18:01 user nova-compute[75234]: INFO nova.compute.manager [None req-e9bfef58-f895-42b7-bc9c-772ebab67bc8 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] [instance: aa190a9f-ba53-43fb-8bae-2c429d0d1228] Rebuilding instance
Nov 29 03:18:01 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-751217d7-e3a0-4877-b867-15cec419ab79 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Acquiring lock "ec7cf7cd-8e48-4bb3-a969-e14db705dc09" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:18:01 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-751217d7-e3a0-4877-b867-15cec419ab79 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Lock "ec7cf7cd-8e48-4bb3-a969-e14db705dc09" acquired by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:18:01 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-3e8fcaaf-bf38-48ae-8a47-2f137d5e43af req-107b3aa3-7b06-4033-a375-8b2fd7f225f6 service nova] Releasing lock "refresh_cache-aa190a9f-ba53-43fb-8bae-2c429d0d1228" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:18:01 user nova-compute[75234]: INFO nova.compute.manager [None req-751217d7-e3a0-4877-b867-15cec419ab79 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] [instance: ec7cf7cd-8e48-4bb3-a969-e14db705dc09] Detaching volume 6c2e1179-dd3c-41c4-9594-bf4a16752c67
Nov 29 03:18:02 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-5c2a15bc-b937-476d-a3cd-d7aad3257309 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Acquiring lock "ec7cf7cd-8e48-4bb3-a969-e14db705dc09" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:18:02 user nova-compute[75234]: DEBUG nova.compute.manager [None req-e9bfef58-f895-42b7-bc9c-772ebab67bc8 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] [instance: aa190a9f-ba53-43fb-8bae-2c429d0d1228] Checking state {{(pid=75234) _get_power_state /opt/stack/nova/nova/compute/manager.py:1782}}
Nov 29 03:18:02 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/99d6f85f-1b77-4390-8b61-908c98f584c0/disk --force-share --output=json" returned: 0 in 0.203s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:18:02 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-9e5c7588-51ec-49d2-8d1a-cca0dabbdd7b tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Lock "75c5d700-6367-4128-9228-80d2cd24ace7" "released" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: held 10.226s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:18:02 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6894b55f-93c6-43c9-8933-56d5a3c43725/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:18:02 user nova-compute[75234]: INFO nova.virt.block_device [None req-751217d7-e3a0-4877-b867-15cec419ab79 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] [instance: ec7cf7cd-8e48-4bb3-a969-e14db705dc09] Attempting to driver detach volume 6c2e1179-dd3c-41c4-9594-bf4a16752c67 from mountpoint /dev/vdb
Nov 29 03:18:02 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-751217d7-e3a0-4877-b867-15cec419ab79 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Attempting to detach device vdb from instance ec7cf7cd-8e48-4bb3-a969-e14db705dc09 from the persistent domain config. {{(pid=75234) _detach_from_persistent /opt/stack/nova/nova/virt/libvirt/driver.py:2487}}
Nov 29 03:18:02 user nova-compute[75234]: DEBUG nova.virt.libvirt.guest [None req-751217d7-e3a0-4877-b867-15cec419ab79 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] detach device xml: <disk type="block" device="disk">
Nov 29 03:18:02 user nova-compute[75234]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Nov 29 03:18:02 user nova-compute[75234]:   <source dev="/dev/sdc"/>
Nov 29 03:18:02 user nova-compute[75234]:   <target dev="vdb" bus="virtio"/>
Nov 29 03:18:02 user nova-compute[75234]:   <serial>6c2e1179-dd3c-41c4-9594-bf4a16752c67</serial>
Nov 29 03:18:02 user nova-compute[75234]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x07" function="0x0"/>
Nov 29 03:18:02 user nova-compute[75234]: </disk>
Nov 29 03:18:02 user nova-compute[75234]:  {{(pid=75234) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Nov 29 03:18:02 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-751217d7-e3a0-4877-b867-15cec419ab79 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Successfully detached device vdb from instance ec7cf7cd-8e48-4bb3-a969-e14db705dc09 from the persistent domain config.
Nov 29 03:18:02 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-751217d7-e3a0-4877-b867-15cec419ab79 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] (1/8): Attempting to detach device vdb with device alias virtio-disk1 from instance ec7cf7cd-8e48-4bb3-a969-e14db705dc09 from the live domain config. {{(pid=75234) _detach_from_live_with_retry /opt/stack/nova/nova/virt/libvirt/driver.py:2523}}
Nov 29 03:18:02 user nova-compute[75234]: DEBUG nova.virt.libvirt.guest [None req-751217d7-e3a0-4877-b867-15cec419ab79 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] detach device xml: <disk type="block" device="disk">
Nov 29 03:18:02 user nova-compute[75234]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Nov 29 03:18:02 user nova-compute[75234]:   <source dev="/dev/sdc"/>
Nov 29 03:18:02 user nova-compute[75234]:   <target dev="vdb" bus="virtio"/>
Nov 29 03:18:02 user nova-compute[75234]:   <serial>6c2e1179-dd3c-41c4-9594-bf4a16752c67</serial>
Nov 29 03:18:02 user nova-compute[75234]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x07" function="0x0"/>
Nov 29 03:18:02 user nova-compute[75234]: </disk>
Nov 29 03:18:02 user nova-compute[75234]:  {{(pid=75234) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Nov 29 03:18:02 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:18:02 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:18:02 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:18:02 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6894b55f-93c6-43c9-8933-56d5a3c43725/disk --force-share --output=json" returned: 0 in 0.234s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:18:02 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6894b55f-93c6-43c9-8933-56d5a3c43725/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:18:02 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:18:02 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:18:02 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6894b55f-93c6-43c9-8933-56d5a3c43725/disk --force-share --output=json" returned: 0 in 0.236s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:18:02 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-751217d7-e3a0-4877-b867-15cec419ab79 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Start waiting for the detach event from libvirt for device vdb with device alias virtio-disk1 for instance ec7cf7cd-8e48-4bb3-a969-e14db705dc09 {{(pid=75234) _detach_from_live_and_wait_for_event /opt/stack/nova/nova/virt/libvirt/driver.py:2599}}
Nov 29 03:18:02 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/ec7cf7cd-8e48-4bb3-a969-e14db705dc09/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:18:02 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/ec7cf7cd-8e48-4bb3-a969-e14db705dc09/disk --force-share --output=json" returned: 0 in 0.197s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:18:02 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/ec7cf7cd-8e48-4bb3-a969-e14db705dc09/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:18:03 user nova-compute[75234]: INFO nova.virt.libvirt.driver [-] [instance: aa190a9f-ba53-43fb-8bae-2c429d0d1228] Instance destroyed successfully.
Nov 29 03:18:03 user nova-compute[75234]: INFO nova.virt.libvirt.driver [-] [instance: aa190a9f-ba53-43fb-8bae-2c429d0d1228] Instance destroyed successfully.
Nov 29 03:18:03 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-e9bfef58-f895-42b7-bc9c-772ebab67bc8 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-11-29T03:16:08Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description=None,display_name='tempest-ServerActionsV293TestJSON-server-1837520572',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-serveractionsv293testjson-server-381391911',id=27,image_ref='',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBKnl09Ywqr6/srgll0CbbEeIXPcY1dWQkS0gn6SNfJJJd/3IKDSo4FDOq1mKkootcIJYqYPVMtXdp0HTuNblTn7mhW9LnhxK8ZA0M4lAvd/3jzC0f6c+1EiWTGrTU8NBjw==',key_name='tempest-keypair-1391635144',keypairs=<?>,launch_index=0,launched_at=2023-11-29T03:16:24Z,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={rebuild='server'},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=1,progress=0,project_id='581a0b62fee744e1b9a767a0d011c4e0',ramdisk_id='',reservation_id='r-eq00dfx0',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',image_container_format='bare',image_disk_format='qcow2',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-ServerActionsV293TestJSON-155733519',owner_user_name='tempest-ServerActionsV293TestJSON-155733519-project-member'},tags=<?>,task_state='rebuilding',terminated_at=None,trusted_certs=None,updated_at=2023-11-29T03:18:02Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='d11ce50c391d4e3bbe92d6c463d14746',uuid=aa190a9f-ba53-43fb-8bae-2c429d0d1228,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "2cc37e6a-5c9f-4bb8-98fa-f4f2b94d0c35", "address": "fa:16:3e:fb:40:18", "network": {"id": "38517dcf-64c7-4d26-9083-5ea5f6c87c2f", "bridge": "br-int", "label": "tempest-ServerActionsV293TestJSON-266920726-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.208", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "581a0b62fee744e1b9a767a0d011c4e0", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2cc37e6a-5c", "ovs_interfaceid": "2cc37e6a-5c9f-4bb8-98fa-f4f2b94d0c35", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Nov 29 03:18:03 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-e9bfef58-f895-42b7-bc9c-772ebab67bc8 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] Converting VIF {"id": "2cc37e6a-5c9f-4bb8-98fa-f4f2b94d0c35", "address": "fa:16:3e:fb:40:18", "network": {"id": "38517dcf-64c7-4d26-9083-5ea5f6c87c2f", "bridge": "br-int", "label": "tempest-ServerActionsV293TestJSON-266920726-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.208", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "581a0b62fee744e1b9a767a0d011c4e0", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2cc37e6a-5c", "ovs_interfaceid": "2cc37e6a-5c9f-4bb8-98fa-f4f2b94d0c35", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:18:03 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-e9bfef58-f895-42b7-bc9c-772ebab67bc8 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:fb:40:18,bridge_name='br-int',has_traffic_filtering=True,id=2cc37e6a-5c9f-4bb8-98fa-f4f2b94d0c35,network=Network(38517dcf-64c7-4d26-9083-5ea5f6c87c2f),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap2cc37e6a-5c') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:18:03 user nova-compute[75234]: DEBUG os_vif [None req-e9bfef58-f895-42b7-bc9c-772ebab67bc8 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] Unplugging vif VIFOpenVSwitch(active=True,address=fa:16:3e:fb:40:18,bridge_name='br-int',has_traffic_filtering=True,id=2cc37e6a-5c9f-4bb8-98fa-f4f2b94d0c35,network=Network(38517dcf-64c7-4d26-9083-5ea5f6c87c2f),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap2cc37e6a-5c') {{(pid=75234) unplug /opt/stack/data/venv/lib/python3.10/site-packages/os_vif/__init__.py:109}}
Nov 29 03:18:03 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:18:03 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tap2cc37e6a-5c, bridge=br-int, if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:18:03 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/ec7cf7cd-8e48-4bb3-a969-e14db705dc09/disk --force-share --output=json" returned: 0 in 0.201s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:18:03 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:18:03 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:18:03 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:18:03 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbDestroyCommand(_result=None, table=QoS, record=667824d2-2ccc-4f1b-8574-9d0ff0febfd6) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:18:03 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:18:03 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:18:03 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/d3b90f39-4b91-4152-b820-bd0e263eec89/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:18:03 user nova-compute[75234]: INFO os_vif [None req-e9bfef58-f895-42b7-bc9c-772ebab67bc8 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] Successfully unplugged vif VIFOpenVSwitch(active=True,address=fa:16:3e:fb:40:18,bridge_name='br-int',has_traffic_filtering=True,id=2cc37e6a-5c9f-4bb8-98fa-f4f2b94d0c35,network=Network(38517dcf-64c7-4d26-9083-5ea5f6c87c2f),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap2cc37e6a-5c')
Nov 29 03:18:03 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-e9bfef58-f895-42b7-bc9c-772ebab67bc8 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] [instance: aa190a9f-ba53-43fb-8bae-2c429d0d1228] calling os-brick to detach iSCSI Volume {{(pid=75234) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Nov 29 03:18:03 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-e9bfef58-f895-42b7-bc9c-772ebab67bc8 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f8a5050c9a0>, {'target_iqn': 'iqn.2010-10.org.openstack:07f83efc-272d-4e10-9a2b-48d967db613c', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'kRLjYCVO', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdb'}, None), 'kwargs': True}}" {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:176}}
Nov 29 03:18:03 user nova-compute[75234]: WARNING os_brick.initiator.connectors.base [None req-e9bfef58-f895-42b7-bc9c-772ebab67bc8 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release: NotImplementedError
Nov 29 03:18:03 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-e9bfef58-f895-42b7-bc9c-772ebab67bc8 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:68}}
Nov 29 03:18:03 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-e9bfef58-f895-42b7-bc9c-772ebab67bc8 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.003s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:73}}
Nov 29 03:18:03 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:18:03 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.023s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:18:03 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[e78e08e0-a3ec-4a4f-aa7e-5a548e339014]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:07f83efc-272d-4e10-9a2b-48d967db613c\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:6c2e1179-dd3c-41c4-9594-bf4a16752c67\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:bf57b967-7fad-4d76-abb2-319fe88f8ef9\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:18:03 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-e9bfef58-f895-42b7-bc9c-772ebab67bc8 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS:
Nov 29 03:18:03 user nova-compute[75234]: No targets found.
Nov 29 03:18:03 user nova-compute[75234]: iSNS:
Nov 29 03:18:03 user nova-compute[75234]: No targets found.
Nov 29 03:18:03 user nova-compute[75234]: STATIC:
Nov 29 03:18:03 user nova-compute[75234]: Target: iqn.2010-10.org.openstack:07f83efc-272d-4e10-9a2b-48d967db613c
Nov 29 03:18:03 user nova-compute[75234]:         Portal: 172.16.0.220:3260,-1
Nov 29 03:18:03 user nova-compute[75234]:                 Iface Name: default
Nov 29 03:18:03 user nova-compute[75234]: Target: iqn.2010-10.org.openstack:6c2e1179-dd3c-41c4-9594-bf4a16752c67
Nov 29 03:18:03 user nova-compute[75234]:         Portal: 172.16.0.220:3260,-1
Nov 29 03:18:03 user nova-compute[75234]:                 Iface Name: default
Nov 29 03:18:03 user nova-compute[75234]: Target: iqn.2010-10.org.openstack:bf57b967-7fad-4d76-abb2-319fe88f8ef9
Nov 29 03:18:03 user nova-compute[75234]:         Portal: 172.16.0.220:3260,-1
Nov 29 03:18:03 user nova-compute[75234]:                 Iface Name: default
Nov 29 03:18:03 user nova-compute[75234]: FIRMWARE:
Nov 29 03:18:03 user nova-compute[75234]: No targets found.
Nov 29 03:18:03 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:18:03 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-e9bfef58-f895-42b7-bc9c-772ebab67bc8 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] Regex to get portals from discoverydb: ^SENDTARGETS:
Nov 29 03:18:03 user nova-compute[75234]: .*?^DiscoveryAddress: 172.16.0.220,3260.*?
Nov 29 03:18:03 user nova-compute[75234]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=75234) _get_discoverydb_portals /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:382}}
Nov 29 03:18:03 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-e9bfef58-f895-42b7-bc9c-772ebab67bc8 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:07f83efc-272d-4e10-9a2b-48d967db613c', 0)] {{(pid=75234) _get_connection_devices /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:830}}
Nov 29 03:18:03 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:18:03 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.018s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:18:03 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[921326a3-a1e3-4aa9-81e1-fbe6cfab0fce]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:07f83efc-272d-4e10-9a2b-48d967db613c\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:6c2e1179-dd3c-41c4-9594-bf4a16752c67\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:bf57b967-7fad-4d76-abb2-319fe88f8ef9\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:18:03 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:18:03 user nova-compute[75234]: DEBUG nova.compute.manager [req-975125c2-cabc-46bf-a10b-1c26ec5554fb req-b968b292-2016-4ef4-9eb7-ec3afa654402 service nova] [instance: aa190a9f-ba53-43fb-8bae-2c429d0d1228] Received event network-vif-unplugged-2cc37e6a-5c9f-4bb8-98fa-f4f2b94d0c35 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:18:03 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-975125c2-cabc-46bf-a10b-1c26ec5554fb req-b968b292-2016-4ef4-9eb7-ec3afa654402 service nova] Acquiring lock "aa190a9f-ba53-43fb-8bae-2c429d0d1228-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:18:03 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-975125c2-cabc-46bf-a10b-1c26ec5554fb req-b968b292-2016-4ef4-9eb7-ec3afa654402 service nova] Lock "aa190a9f-ba53-43fb-8bae-2c429d0d1228-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:18:03 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-975125c2-cabc-46bf-a10b-1c26ec5554fb req-b968b292-2016-4ef4-9eb7-ec3afa654402 service nova] Lock "aa190a9f-ba53-43fb-8bae-2c429d0d1228-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:18:03 user nova-compute[75234]: DEBUG nova.compute.manager [req-975125c2-cabc-46bf-a10b-1c26ec5554fb req-b968b292-2016-4ef4-9eb7-ec3afa654402 service nova] [instance: aa190a9f-ba53-43fb-8bae-2c429d0d1228] No waiting events found dispatching network-vif-unplugged-2cc37e6a-5c9f-4bb8-98fa-f4f2b94d0c35 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:18:03 user nova-compute[75234]: WARNING nova.compute.manager [req-975125c2-cabc-46bf-a10b-1c26ec5554fb req-b968b292-2016-4ef4-9eb7-ec3afa654402 service nova] [instance: aa190a9f-ba53-43fb-8bae-2c429d0d1228] Received unexpected event network-vif-unplugged-2cc37e6a-5c9f-4bb8-98fa-f4f2b94d0c35 for instance with vm_state active and task_state rebuilding.
Nov 29 03:18:03 user nova-compute[75234]: DEBUG nova.compute.manager [req-975125c2-cabc-46bf-a10b-1c26ec5554fb req-b968b292-2016-4ef4-9eb7-ec3afa654402 service nova] [instance: aa190a9f-ba53-43fb-8bae-2c429d0d1228] Received event network-vif-plugged-2cc37e6a-5c9f-4bb8-98fa-f4f2b94d0c35 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:18:03 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-975125c2-cabc-46bf-a10b-1c26ec5554fb req-b968b292-2016-4ef4-9eb7-ec3afa654402 service nova] Acquiring lock "aa190a9f-ba53-43fb-8bae-2c429d0d1228-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:18:03 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-975125c2-cabc-46bf-a10b-1c26ec5554fb req-b968b292-2016-4ef4-9eb7-ec3afa654402 service nova] Lock "aa190a9f-ba53-43fb-8bae-2c429d0d1228-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:18:03 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-975125c2-cabc-46bf-a10b-1c26ec5554fb req-b968b292-2016-4ef4-9eb7-ec3afa654402 service nova] Lock "aa190a9f-ba53-43fb-8bae-2c429d0d1228-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:18:03 user nova-compute[75234]: DEBUG nova.compute.manager [req-975125c2-cabc-46bf-a10b-1c26ec5554fb req-b968b292-2016-4ef4-9eb7-ec3afa654402 service nova] [instance: aa190a9f-ba53-43fb-8bae-2c429d0d1228] No waiting events found dispatching network-vif-plugged-2cc37e6a-5c9f-4bb8-98fa-f4f2b94d0c35 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:18:03 user nova-compute[75234]: WARNING nova.compute.manager [req-975125c2-cabc-46bf-a10b-1c26ec5554fb req-b968b292-2016-4ef4-9eb7-ec3afa654402 service nova] [instance: aa190a9f-ba53-43fb-8bae-2c429d0d1228] Received unexpected event network-vif-plugged-2cc37e6a-5c9f-4bb8-98fa-f4f2b94d0c35 for instance with vm_state active and task_state rebuilding.
Nov 29 03:18:03 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.024s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:18:03 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[1adef720-df21-4540-97ff-9ada2aafd6ee]: (4, ('tcp: [27] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:07f83efc-272d-4e10-9a2b-48d967db613c (non-flash)\ntcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6c2e1179-dd3c-41c4-9594-bf4a16752c67 (non-flash)\ntcp: [29] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bf57b967-7fad-4d76-abb2-319fe88f8ef9 (non-flash)\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:18:03 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-e9bfef58-f895-42b7-bc9c-772ebab67bc8 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [27] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:07f83efc-272d-4e10-9a2b-48d967db613c (non-flash)
Nov 29 03:18:03 user nova-compute[75234]: tcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6c2e1179-dd3c-41c4-9594-bf4a16752c67 (non-flash)
Nov 29 03:18:03 user nova-compute[75234]: tcp: [29] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bf57b967-7fad-4d76-abb2-319fe88f8ef9 (non-flash)
Nov 29 03:18:03 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:18:03 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-e9bfef58-f895-42b7-bc9c-772ebab67bc8 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] iscsi session list stdout=tcp: [27] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:07f83efc-272d-4e10-9a2b-48d967db613c (non-flash)
Nov 29 03:18:03 user nova-compute[75234]: tcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6c2e1179-dd3c-41c4-9594-bf4a16752c67 (non-flash)
Nov 29 03:18:03 user nova-compute[75234]: tcp: [29] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bf57b967-7fad-4d76-abb2-319fe88f8ef9 (non-flash)
Nov 29 03:18:03 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsi_session /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Nov 29 03:18:03 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-e9bfef58-f895-42b7-bc9c-772ebab67bc8 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7f8a6025b0a0>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:07f83efc-272d-4e10-9a2b-48d967db613c'): ({'sdb'}, set())}) {{(pid=75234) _get_connection_devices /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:863}}
Nov 29 03:18:03 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-e9bfef58-f895-42b7-bc9c-772ebab67bc8 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] Removing single pathed devices sdb {{(pid=75234) remove_connection /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:374}}
Nov 29 03:18:03 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:18:03 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/d3b90f39-4b91-4152-b820-bd0e263eec89/disk --force-share --output=json" returned: 0 in 0.203s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:18:03 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/d3b90f39-4b91-4152-b820-bd0e263eec89/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:18:03 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.027s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:18:03 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[d09500ac-a965-4de0-9602-1125d21fb01d]: (4, ('path checker states:\nup                  3\n\npaths: 0\nbusy: False\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:18:03 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdb {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:18:03 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdb" returned: 0 in 0.022s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:18:03 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[66faea89-e533-4581-b15f-ed3e101712b3]: (4, ('ok\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:18:03 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-e9bfef58-f895-42b7-bc9c-772ebab67bc8 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] Flushing IO for device /dev/sdb {{(pid=75234) flush_device_io /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:434}}
Nov 29 03:18:03 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdb {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:18:03 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdb" returned: 0 in 0.022s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:18:03 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[dc86d571-cab4-4a36-92a1-ac4f0cbdb64e]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:18:03 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-e9bfef58-f895-42b7-bc9c-772ebab67bc8 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] Remove SCSI device /dev/sdb with /sys/block/sdb/device/delete {{(pid=75234) remove_scsi_device /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:148}}
Nov 29 03:18:03 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdb/device/delete {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:18:03 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdb/device/delete" returned: 0 in 0.054s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:18:03 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[0dba30f9-2590-40c5-b2a3-d6f064bdc132]: (4, ('1', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:18:03 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-e9bfef58-f895-42b7-bc9c-772ebab67bc8 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] Checking to see if SCSI volumes sdb have been removed. {{(pid=75234) wait_for_volumes_removal /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:156}}
Nov 29 03:18:03 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-e9bfef58-f895-42b7-bc9c-772ebab67bc8 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] SCSI volumes sdb have been removed. {{(pid=75234) wait_for_volumes_removal /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:166}}
Nov 29 03:18:03 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-e9bfef58-f895-42b7-bc9c-772ebab67bc8 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:07f83efc-272d-4e10-9a2b-48d967db613c')] {{(pid=75234) _disconnect_connection /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Nov 29 03:18:03 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:07f83efc-272d-4e10-9a2b-48d967db613c -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:18:03 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/d3b90f39-4b91-4152-b820-bd0e263eec89/disk --force-share --output=json" returned: 0 in 0.183s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:18:03 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:07f83efc-272d-4e10-9a2b-48d967db613c -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.021s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:18:03 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[8bc084e0-7d11-47a3-b556-aa826c900b70]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:18:03 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-e9bfef58-f895-42b7-bc9c-772ebab67bc8 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:18:03 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:07f83efc-272d-4e10-9a2b-48d967db613c -p 172.16.0.220:3260 --logout {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:18:03 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/aaf1c3e3-b73e-42ad-87bd-c5255ec09b46/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:18:03 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:07f83efc-272d-4e10-9a2b-48d967db613c -p 172.16.0.220:3260 --logout" returned: 0 in 0.057s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:18:03 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[1efbb0f6-fbc4-4ea2-b24e-37ca147e7283]: (4, ('Logging out of session [sid: 27, target: iqn.2010-10.org.openstack:07f83efc-272d-4e10-9a2b-48d967db613c, portal: 172.16.0.220,3260]\nLogout of [sid: 27, target: iqn.2010-10.org.openstack:07f83efc-272d-4e10-9a2b-48d967db613c, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:18:03 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-e9bfef58-f895-42b7-bc9c-772ebab67bc8 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 27, target: iqn.2010-10.org.openstack:07f83efc-272d-4e10-9a2b-48d967db613c, portal: 172.16.0.220,3260]
Nov 29 03:18:03 user nova-compute[75234]: Logout of [sid: 27, target: iqn.2010-10.org.openstack:07f83efc-272d-4e10-9a2b-48d967db613c, portal: 172.16.0.220,3260] successful.
Nov 29 03:18:03 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:18:03 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:07f83efc-272d-4e10-9a2b-48d967db613c -p 172.16.0.220:3260 --op delete {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:18:03 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:07f83efc-272d-4e10-9a2b-48d967db613c -p 172.16.0.220:3260 --op delete" returned: 0 in 0.026s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:18:03 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[7784f1b5-907c-4a69-b037-ff77fe2cda3a]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:18:03 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-e9bfef58-f895-42b7-bc9c-772ebab67bc8 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:18:03 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-e9bfef58-f895-42b7-bc9c-772ebab67bc8 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.440s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:87}}
Nov 29 03:18:03 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-e9bfef58-f895-42b7-bc9c-772ebab67bc8 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] <== disconnect_volume: return (442ms) None {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:203}}
Nov 29 03:18:03 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-e9bfef58-f895-42b7-bc9c-772ebab67bc8 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] [instance: aa190a9f-ba53-43fb-8bae-2c429d0d1228] Disconnected iSCSI Volume {{(pid=75234) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:79}}
Nov 29 03:18:03 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-e9bfef58-f895-42b7-bc9c-772ebab67bc8 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] [instance: aa190a9f-ba53-43fb-8bae-2c429d0d1228] Deleting instance files /opt/stack/data/nova/instances/aa190a9f-ba53-43fb-8bae-2c429d0d1228_del
Nov 29 03:18:03 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-e9bfef58-f895-42b7-bc9c-772ebab67bc8 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] [instance: aa190a9f-ba53-43fb-8bae-2c429d0d1228] Deletion of /opt/stack/data/nova/instances/aa190a9f-ba53-43fb-8bae-2c429d0d1228_del complete
Nov 29 03:18:03 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/aaf1c3e3-b73e-42ad-87bd-c5255ec09b46/disk --force-share --output=json" returned: 0 in 0.170s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:18:03 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/aaf1c3e3-b73e-42ad-87bd-c5255ec09b46/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:18:03 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-e9bfef58-f895-42b7-bc9c-772ebab67bc8 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] [instance: aa190a9f-ba53-43fb-8bae-2c429d0d1228] During detach_volume, instance disappeared.: nova.exception.InstanceNotFound: Instance aa190a9f-ba53-43fb-8bae-2c429d0d1228 could not be found.
Nov 29 03:18:03 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/aaf1c3e3-b73e-42ad-87bd-c5255ec09b46/disk --force-share --output=json" returned: 0 in 0.194s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:18:03 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-e9bfef58-f895-42b7-bc9c-772ebab67bc8 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] [instance: aa190a9f-ba53-43fb-8bae-2c429d0d1228] calling os-brick to detach iSCSI Volume {{(pid=75234) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Nov 29 03:18:03 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-e9bfef58-f895-42b7-bc9c-772ebab67bc8 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f8a5050c9a0>, {'target_iqn': 'iqn.2010-10.org.openstack:07f83efc-272d-4e10-9a2b-48d967db613c', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'kRLjYCVO', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdb'}, None), 'kwargs': False}}" {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:176}}
Nov 29 03:18:03 user nova-compute[75234]: WARNING os_brick.initiator.connectors.base [None req-e9bfef58-f895-42b7-bc9c-772ebab67bc8 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release: NotImplementedError
Nov 29 03:18:03 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-e9bfef58-f895-42b7-bc9c-772ebab67bc8 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:68}}
Nov 29 03:18:03 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-e9bfef58-f895-42b7-bc9c-772ebab67bc8 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.002s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:73}}
Nov 29 03:18:03 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:18:03 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/5e6cca46-c5a7-4303-98c2-1ad483c6d14e/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:18:03 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.020s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:18:03 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[d5c7d8a4-1614-4071-9ea6-f6b64cd3c751]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:6c2e1179-dd3c-41c4-9594-bf4a16752c67\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:bf57b967-7fad-4d76-abb2-319fe88f8ef9\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:18:03 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-e9bfef58-f895-42b7-bc9c-772ebab67bc8 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS:
Nov 29 03:18:03 user nova-compute[75234]: No targets found.
Nov 29 03:18:03 user nova-compute[75234]: iSNS:
Nov 29 03:18:03 user nova-compute[75234]: No targets found.
Nov 29 03:18:03 user nova-compute[75234]: STATIC:
Nov 29 03:18:03 user nova-compute[75234]: Target: iqn.2010-10.org.openstack:6c2e1179-dd3c-41c4-9594-bf4a16752c67
Nov 29 03:18:03 user nova-compute[75234]:         Portal: 172.16.0.220:3260,-1
Nov 29 03:18:03 user nova-compute[75234]:                 Iface Name: default
Nov 29 03:18:03 user nova-compute[75234]: Target: iqn.2010-10.org.openstack:bf57b967-7fad-4d76-abb2-319fe88f8ef9
Nov 29 03:18:03 user nova-compute[75234]:         Portal: 172.16.0.220:3260,-1
Nov 29 03:18:03 user nova-compute[75234]:                 Iface Name: default
Nov 29 03:18:03 user nova-compute[75234]: FIRMWARE:
Nov 29 03:18:03 user nova-compute[75234]: No targets found.
Nov 29 03:18:03 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:18:03 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-e9bfef58-f895-42b7-bc9c-772ebab67bc8 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] Regex to get portals from discoverydb: ^SENDTARGETS:
Nov 29 03:18:03 user nova-compute[75234]: .*?^DiscoveryAddress: 172.16.0.220,3260.*?
Nov 29 03:18:03 user nova-compute[75234]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=75234) _get_discoverydb_portals /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:382}}
Nov 29 03:18:03 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-e9bfef58-f895-42b7-bc9c-772ebab67bc8 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:07f83efc-272d-4e10-9a2b-48d967db613c', 0)] {{(pid=75234) _get_connection_devices /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:830}}
Nov 29 03:18:03 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:18:03 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.018s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:18:03 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[517b51f0-c61e-457c-86c1-c5812fe5cb4d]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:6c2e1179-dd3c-41c4-9594-bf4a16752c67\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:bf57b967-7fad-4d76-abb2-319fe88f8ef9\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:18:03 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:18:03 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.021s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:18:03 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[cd694f85-16ec-44d9-a7c6-8b50024b463d]: (4, ('tcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6c2e1179-dd3c-41c4-9594-bf4a16752c67 (non-flash)\ntcp: [29] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bf57b967-7fad-4d76-abb2-319fe88f8ef9 (non-flash)\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:18:03 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-e9bfef58-f895-42b7-bc9c-772ebab67bc8 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6c2e1179-dd3c-41c4-9594-bf4a16752c67 (non-flash)
Nov 29 03:18:03 user nova-compute[75234]: tcp: [29] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bf57b967-7fad-4d76-abb2-319fe88f8ef9 (non-flash)
Nov 29 03:18:03 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:18:03 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-e9bfef58-f895-42b7-bc9c-772ebab67bc8 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] iscsi session list stdout=tcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6c2e1179-dd3c-41c4-9594-bf4a16752c67 (non-flash)
Nov 29 03:18:03 user nova-compute[75234]: tcp: [29] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bf57b967-7fad-4d76-abb2-319fe88f8ef9 (non-flash)
Nov 29 03:18:03 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsi_session /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Nov 29 03:18:03 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-e9bfef58-f895-42b7-bc9c-772ebab67bc8 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7f8a60135cf0>, {}) {{(pid=75234) _get_connection_devices /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:863}}
Nov 29 03:18:03 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-e9bfef58-f895-42b7-bc9c-772ebab67bc8 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] Disconnecting from: [] {{(pid=75234) _disconnect_connection /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Nov 29 03:18:03 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-e9bfef58-f895-42b7-bc9c-772ebab67bc8 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.097s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:87}}
Nov 29 03:18:03 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-e9bfef58-f895-42b7-bc9c-772ebab67bc8 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] <== disconnect_volume: return (100ms) None {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:203}}
Nov 29 03:18:03 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-e9bfef58-f895-42b7-bc9c-772ebab67bc8 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] [instance: aa190a9f-ba53-43fb-8bae-2c429d0d1228] Disconnected iSCSI Volume {{(pid=75234) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:79}}
Nov 29 03:18:04 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/5e6cca46-c5a7-4303-98c2-1ad483c6d14e/disk --force-share --output=json" returned: 0 in 0.205s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:18:04 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/5e6cca46-c5a7-4303-98c2-1ad483c6d14e/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:18:04 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/5e6cca46-c5a7-4303-98c2-1ad483c6d14e/disk --force-share --output=json" returned: 0 in 0.182s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:18:04 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a082c96e-e384-40b6-bcc8-fc5e0e104669/disk.rescue --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:18:04 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a082c96e-e384-40b6-bcc8-fc5e0e104669/disk.rescue --force-share --output=json" returned: 0 in 0.169s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:18:04 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a082c96e-e384-40b6-bcc8-fc5e0e104669/disk.rescue --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:18:04 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a082c96e-e384-40b6-bcc8-fc5e0e104669/disk.rescue --force-share --output=json" returned: 0 in 0.177s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:18:04 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a082c96e-e384-40b6-bcc8-fc5e0e104669/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:18:04 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a082c96e-e384-40b6-bcc8-fc5e0e104669/disk --force-share --output=json" returned: 0 in 0.165s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:18:04 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a082c96e-e384-40b6-bcc8-fc5e0e104669/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:18:04 user nova-compute[75234]: DEBUG nova.compute.manager [None req-e9bfef58-f895-42b7-bc9c-772ebab67bc8 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] [instance: aa190a9f-ba53-43fb-8bae-2c429d0d1228] Preparing to wait for external event volume-reimaged-07f83efc-272d-4e10-9a2b-48d967db613c {{(pid=75234) prepare_for_instance_event /opt/stack/nova/nova/compute/manager.py:283}}
Nov 29 03:18:04 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-e9bfef58-f895-42b7-bc9c-772ebab67bc8 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] Acquiring lock "aa190a9f-ba53-43fb-8bae-2c429d0d1228-events" by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.<locals>._create_or_get_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:18:04 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-e9bfef58-f895-42b7-bc9c-772ebab67bc8 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] Lock "aa190a9f-ba53-43fb-8bae-2c429d0d1228-events" acquired by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.<locals>._create_or_get_event" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:18:04 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-e9bfef58-f895-42b7-bc9c-772ebab67bc8 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] Lock "aa190a9f-ba53-43fb-8bae-2c429d0d1228-events" "released" by "nova.compute.manager.InstanceEvents.prepare_for_instance_event.<locals>._create_or_get_event" :: held 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:18:05 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a082c96e-e384-40b6-bcc8-fc5e0e104669/disk --force-share --output=json" returned: 0 in 0.195s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:18:05 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/75c5d700-6367-4128-9228-80d2cd24ace7/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:18:05 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/75c5d700-6367-4128-9228-80d2cd24ace7/disk --force-share --output=json" returned: 0 in 0.204s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:18:05 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/75c5d700-6367-4128-9228-80d2cd24ace7/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:18:05 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-fd65259f-eb0e-4ebe-a465-65fbe77d1d6a tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Acquiring lock "0aff446c-f417-477e-a1c5-b51b1b6214e4" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:18:05 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-fd65259f-eb0e-4ebe-a465-65fbe77d1d6a tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Lock "0aff446c-f417-477e-a1c5-b51b1b6214e4" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.002s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:18:05 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/75c5d700-6367-4128-9228-80d2cd24ace7/disk --force-share --output=json" returned: 0 in 0.197s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:18:05 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] skipping disk /dev/sdd (vdb) as it is a volume {{(pid=75234) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11213}}
Nov 29 03:18:05 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Error from libvirt while getting description of instance-0000001b: [Error Code 42] Domain not found: no domain with matching uuid 'aa190a9f-ba53-43fb-8bae-2c429d0d1228' (instance-0000001b): libvirt.libvirtError: Domain not found: no domain with matching uuid 'aa190a9f-ba53-43fb-8bae-2c429d0d1228' (instance-0000001b)
Nov 29 03:18:05 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/21e9c6cd-c39f-41c0-8768-5dd64589c3f3/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:18:05 user nova-compute[75234]: DEBUG nova.compute.manager [None req-fd65259f-eb0e-4ebe-a465-65fbe77d1d6a tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] [instance: 0aff446c-f417-477e-a1c5-b51b1b6214e4] Starting instance... {{(pid=75234) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2422}}
Nov 29 03:18:05 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-fd65259f-eb0e-4ebe-a465-65fbe77d1d6a tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:18:05 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-fd65259f-eb0e-4ebe-a465-65fbe77d1d6a tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:18:05 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-fd65259f-eb0e-4ebe-a465-65fbe77d1d6a tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=75234) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2366}}
Nov 29 03:18:05 user nova-compute[75234]: INFO nova.compute.claims [None req-fd65259f-eb0e-4ebe-a465-65fbe77d1d6a tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] [instance: 0aff446c-f417-477e-a1c5-b51b1b6214e4] Claim successful on node user
Nov 29 03:18:05 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/21e9c6cd-c39f-41c0-8768-5dd64589c3f3/disk --force-share --output=json" returned: 0 in 0.175s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:18:05 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/21e9c6cd-c39f-41c0-8768-5dd64589c3f3/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:18:05 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/21e9c6cd-c39f-41c0-8768-5dd64589c3f3/disk --force-share --output=json" returned: 0 in 0.176s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:18:06 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:18:06 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:18:06 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Hypervisor/Node resource view: name=user free_ram=6994MB free_disk=25.1868896484375GB free_vcpus=0 pci_devices=[{"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}] {{(pid=75234) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1106}}
Nov 29 03:18:06 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:18:06 user nova-compute[75234]: DEBUG nova.compute.provider_tree [None req-fd65259f-eb0e-4ebe-a465-65fbe77d1d6a tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Inventory has not changed in ProviderTree for provider: 4d291220-7c37-4839-a0e3-b23c66502b74 {{(pid=75234) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Nov 29 03:18:06 user nova-compute[75234]: DEBUG nova.scheduler.client.report [None req-fd65259f-eb0e-4ebe-a465-65fbe77d1d6a tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Inventory has not changed for provider 4d291220-7c37-4839-a0e3-b23c66502b74 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=75234) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Nov 29 03:18:06 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-fd65259f-eb0e-4ebe-a465-65fbe77d1d6a tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.965s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:18:06 user nova-compute[75234]: DEBUG nova.compute.manager [None req-fd65259f-eb0e-4ebe-a465-65fbe77d1d6a tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] [instance: 0aff446c-f417-477e-a1c5-b51b1b6214e4] Start building networks asynchronously for instance. {{(pid=75234) _build_resources /opt/stack/nova/nova/compute/manager.py:2819}}
Nov 29 03:18:06 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.288s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:18:06 user nova-compute[75234]: DEBUG nova.compute.manager [None req-fd65259f-eb0e-4ebe-a465-65fbe77d1d6a tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] [instance: 0aff446c-f417-477e-a1c5-b51b1b6214e4] Allocating IP information in the background. {{(pid=75234) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1972}}
Nov 29 03:18:06 user nova-compute[75234]: DEBUG nova.network.neutron [None req-fd65259f-eb0e-4ebe-a465-65fbe77d1d6a tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] [instance: 0aff446c-f417-477e-a1c5-b51b1b6214e4] allocate_for_instance() {{(pid=75234) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1156}}
Nov 29 03:18:06 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-fd65259f-eb0e-4ebe-a465-65fbe77d1d6a tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] [instance: 0aff446c-f417-477e-a1c5-b51b1b6214e4] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Nov 29 03:18:06 user nova-compute[75234]: DEBUG nova.compute.manager [None req-fd65259f-eb0e-4ebe-a465-65fbe77d1d6a tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] [instance: 0aff446c-f417-477e-a1c5-b51b1b6214e4] Start building block device mappings for instance. {{(pid=75234) _build_resources /opt/stack/nova/nova/compute/manager.py:2854}}
Nov 29 03:18:06 user nova-compute[75234]: DEBUG nova.policy [None req-fd65259f-eb0e-4ebe-a465-65fbe77d1d6a tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '66686114450f4f31b3a917667a152d8b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '315732b6e3ee44d584b8e510ef199a90', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=75234) authorize /opt/stack/nova/nova/policy.py:203}}
Nov 29 03:18:06 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance 21e9c6cd-c39f-41c0-8768-5dd64589c3f3 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:18:06 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance d3b90f39-4b91-4152-b820-bd0e263eec89 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:18:06 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance 13e5da64-9adc-4479-81c7-d329e9b5f08f actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:18:06 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance a082c96e-e384-40b6-bcc8-fc5e0e104669 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:18:06 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance 6894b55f-93c6-43c9-8933-56d5a3c43725 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:18:06 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance ec7cf7cd-8e48-4bb3-a969-e14db705dc09 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:18:06 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance 75c5d700-6367-4128-9228-80d2cd24ace7 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:18:06 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance 1c8b7559-29ab-4982-8c75-ea2a0f0101cb actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:18:06 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance aa190a9f-ba53-43fb-8bae-2c429d0d1228 actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:18:06 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance 99d6f85f-1b77-4390-8b61-908c98f584c0 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:18:06 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance 5e6cca46-c5a7-4303-98c2-1ad483c6d14e actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:18:06 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance aaf1c3e3-b73e-42ad-87bd-c5255ec09b46 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:18:06 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance af699bc2-9daa-4b5a-9b4a-ad92fe857d70 actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:18:06 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance 0aff446c-f417-477e-a1c5-b51b1b6214e4 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:18:06 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Total usable vcpus: 12, total allocated vcpus: 14 {{(pid=75234) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1129}}
Nov 29 03:18:06 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Final resource view: name=user phys_ram=16011MB used_ram=7680MB phys_disk=40GB used_disk=12GB total_vcpus=12 used_vcpus=14 pci_stats=[] {{(pid=75234) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1138}}
Nov 29 03:18:06 user nova-compute[75234]: DEBUG nova.compute.manager [None req-fd65259f-eb0e-4ebe-a465-65fbe77d1d6a tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] [instance: 0aff446c-f417-477e-a1c5-b51b1b6214e4] Start spawning the instance on the hypervisor. {{(pid=75234) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2628}}
Nov 29 03:18:06 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-fd65259f-eb0e-4ebe-a465-65fbe77d1d6a tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] [instance: 0aff446c-f417-477e-a1c5-b51b1b6214e4] Creating instance directory {{(pid=75234) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4727}}
Nov 29 03:18:06 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-fd65259f-eb0e-4ebe-a465-65fbe77d1d6a tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] [instance: 0aff446c-f417-477e-a1c5-b51b1b6214e4] Creating image(s)
Nov 29 03:18:06 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-fd65259f-eb0e-4ebe-a465-65fbe77d1d6a tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Acquiring lock "/opt/stack/data/nova/instances/0aff446c-f417-477e-a1c5-b51b1b6214e4/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:18:06 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-fd65259f-eb0e-4ebe-a465-65fbe77d1d6a tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Lock "/opt/stack/data/nova/instances/0aff446c-f417-477e-a1c5-b51b1b6214e4/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:18:06 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-fd65259f-eb0e-4ebe-a465-65fbe77d1d6a tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Lock "/opt/stack/data/nova/instances/0aff446c-f417-477e-a1c5-b51b1b6214e4/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.009s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:18:06 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-fd65259f-eb0e-4ebe-a465-65fbe77d1d6a tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:18:06 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:18:06 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-fd65259f-eb0e-4ebe-a465-65fbe77d1d6a tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json" returned: 0 in 0.181s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:18:06 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-fd65259f-eb0e-4ebe-a465-65fbe77d1d6a tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Acquiring lock "80686e29dc9db6600d03e85cb62af298e12d7918" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:18:06 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-fd65259f-eb0e-4ebe-a465-65fbe77d1d6a tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Lock "80686e29dc9db6600d03e85cb62af298e12d7918" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.002s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:18:06 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-fd65259f-eb0e-4ebe-a465-65fbe77d1d6a tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:18:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-fd65259f-eb0e-4ebe-a465-65fbe77d1d6a tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json" returned: 0 in 0.206s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:18:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-fd65259f-eb0e-4ebe-a465-65fbe77d1d6a tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918,backing_fmt=raw /opt/stack/data/nova/instances/0aff446c-f417-477e-a1c5-b51b1b6214e4/disk 1073741824 {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:18:07 user nova-compute[75234]: DEBUG nova.compute.provider_tree [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Inventory has not changed in ProviderTree for provider: 4d291220-7c37-4839-a0e3-b23c66502b74 {{(pid=75234) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Nov 29 03:18:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-fd65259f-eb0e-4ebe-a465-65fbe77d1d6a tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918,backing_fmt=raw /opt/stack/data/nova/instances/0aff446c-f417-477e-a1c5-b51b1b6214e4/disk 1073741824" returned: 0 in 0.052s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:18:07 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-fd65259f-eb0e-4ebe-a465-65fbe77d1d6a tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Lock "80686e29dc9db6600d03e85cb62af298e12d7918" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.270s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:18:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-fd65259f-eb0e-4ebe-a465-65fbe77d1d6a tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:18:07 user nova-compute[75234]: DEBUG nova.scheduler.client.report [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Inventory has not changed for provider 4d291220-7c37-4839-a0e3-b23c66502b74 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=75234) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Nov 29 03:18:07 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Compute_service record updated for user:user {{(pid=75234) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:1067}}
Nov 29 03:18:07 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.809s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:18:07 user nova-compute[75234]: DEBUG nova.network.neutron [None req-fd65259f-eb0e-4ebe-a465-65fbe77d1d6a tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] [instance: 0aff446c-f417-477e-a1c5-b51b1b6214e4] Successfully created port: f9148192-6e14-44d4-a31b-081ea47aa289 {{(pid=75234) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:548}}
Nov 29 03:18:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-fd65259f-eb0e-4ebe-a465-65fbe77d1d6a tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json" returned: 0 in 0.184s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:18:07 user nova-compute[75234]: DEBUG nova.virt.disk.api [None req-fd65259f-eb0e-4ebe-a465-65fbe77d1d6a tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Checking if we can resize image /opt/stack/data/nova/instances/0aff446c-f417-477e-a1c5-b51b1b6214e4/disk. size=1073741824 {{(pid=75234) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:166}}
Nov 29 03:18:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-fd65259f-eb0e-4ebe-a465-65fbe77d1d6a tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/0aff446c-f417-477e-a1c5-b51b1b6214e4/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:18:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-fd65259f-eb0e-4ebe-a465-65fbe77d1d6a tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/0aff446c-f417-477e-a1c5-b51b1b6214e4/disk --force-share --output=json" returned: 0 in 0.193s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:18:07 user nova-compute[75234]: DEBUG nova.virt.disk.api [None req-fd65259f-eb0e-4ebe-a465-65fbe77d1d6a tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Cannot resize image /opt/stack/data/nova/instances/0aff446c-f417-477e-a1c5-b51b1b6214e4/disk to a smaller size. {{(pid=75234) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:172}}
Nov 29 03:18:07 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-fd65259f-eb0e-4ebe-a465-65fbe77d1d6a tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] [instance: 0aff446c-f417-477e-a1c5-b51b1b6214e4] Created local disks {{(pid=75234) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4861}}
Nov 29 03:18:07 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-fd65259f-eb0e-4ebe-a465-65fbe77d1d6a tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] [instance: 0aff446c-f417-477e-a1c5-b51b1b6214e4] Ensure instance console log exists: /opt/stack/data/nova/instances/0aff446c-f417-477e-a1c5-b51b1b6214e4/console.log {{(pid=75234) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4613}}
Nov 29 03:18:07 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-fd65259f-eb0e-4ebe-a465-65fbe77d1d6a tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:18:07 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-fd65259f-eb0e-4ebe-a465-65fbe77d1d6a tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:18:07 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-fd65259f-eb0e-4ebe-a465-65fbe77d1d6a tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:18:08 user nova-compute[75234]: DEBUG nova.network.neutron [None req-fd65259f-eb0e-4ebe-a465-65fbe77d1d6a tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] [instance: 0aff446c-f417-477e-a1c5-b51b1b6214e4] Successfully updated port: f9148192-6e14-44d4-a31b-081ea47aa289 {{(pid=75234) _update_port /opt/stack/nova/nova/network/neutron.py:586}}
Nov 29 03:18:08 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-fd65259f-eb0e-4ebe-a465-65fbe77d1d6a tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Acquiring lock "refresh_cache-0aff446c-f417-477e-a1c5-b51b1b6214e4" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:18:08 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-fd65259f-eb0e-4ebe-a465-65fbe77d1d6a tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Acquired lock "refresh_cache-0aff446c-f417-477e-a1c5-b51b1b6214e4" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:18:08 user nova-compute[75234]: DEBUG nova.network.neutron [None req-fd65259f-eb0e-4ebe-a465-65fbe77d1d6a tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] [instance: 0aff446c-f417-477e-a1c5-b51b1b6214e4] Building network info cache for instance {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2009}}
Nov 29 03:18:08 user nova-compute[75234]: DEBUG nova.compute.manager [req-2e803a22-b8c0-4fb8-9d5f-2b88ba960ae9 req-245fba30-3466-469b-96d2-89537f2988f1 service nova] [instance: 0aff446c-f417-477e-a1c5-b51b1b6214e4] Received event network-changed-f9148192-6e14-44d4-a31b-081ea47aa289 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:18:08 user nova-compute[75234]: DEBUG nova.compute.manager [req-2e803a22-b8c0-4fb8-9d5f-2b88ba960ae9 req-245fba30-3466-469b-96d2-89537f2988f1 service nova] [instance: 0aff446c-f417-477e-a1c5-b51b1b6214e4] Refreshing instance network info cache due to event network-changed-f9148192-6e14-44d4-a31b-081ea47aa289. {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11117}}
Nov 29 03:18:08 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-2e803a22-b8c0-4fb8-9d5f-2b88ba960ae9 req-245fba30-3466-469b-96d2-89537f2988f1 service nova] Acquiring lock "refresh_cache-0aff446c-f417-477e-a1c5-b51b1b6214e4" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:18:08 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:18:08 user nova-compute[75234]: DEBUG nova.network.neutron [None req-fd65259f-eb0e-4ebe-a465-65fbe77d1d6a tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] [instance: 0aff446c-f417-477e-a1c5-b51b1b6214e4] Instance cache missing network info. {{(pid=75234) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3322}}
Nov 29 03:18:08 user nova-compute[75234]: DEBUG nova.network.neutron [None req-fd65259f-eb0e-4ebe-a465-65fbe77d1d6a tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] [instance: 0aff446c-f417-477e-a1c5-b51b1b6214e4] Updating instance_info_cache with network_info: [{"id": "f9148192-6e14-44d4-a31b-081ea47aa289", "address": "fa:16:3e:46:49:9c", "network": {"id": "0f289171-3d73-4054-95a0-2515b4f206a0", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-1787960284-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "315732b6e3ee44d584b8e510ef199a90", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapf9148192-6e", "ovs_interfaceid": "f9148192-6e14-44d4-a31b-081ea47aa289", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:18:08 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._sync_scheduler_instance_info {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:18:08 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:18:08 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:18:08 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-fd65259f-eb0e-4ebe-a465-65fbe77d1d6a tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Releasing lock "refresh_cache-0aff446c-f417-477e-a1c5-b51b1b6214e4" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:18:08 user nova-compute[75234]: DEBUG nova.compute.manager [None req-fd65259f-eb0e-4ebe-a465-65fbe77d1d6a tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] [instance: 0aff446c-f417-477e-a1c5-b51b1b6214e4] Instance network_info: |[{"id": "f9148192-6e14-44d4-a31b-081ea47aa289", "address": "fa:16:3e:46:49:9c", "network": {"id": "0f289171-3d73-4054-95a0-2515b4f206a0", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-1787960284-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "315732b6e3ee44d584b8e510ef199a90", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapf9148192-6e", "ovs_interfaceid": "f9148192-6e14-44d4-a31b-081ea47aa289", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=75234) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1987}}
Nov 29 03:18:08 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:18:08 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-2e803a22-b8c0-4fb8-9d5f-2b88ba960ae9 req-245fba30-3466-469b-96d2-89537f2988f1 service nova] Acquired lock "refresh_cache-0aff446c-f417-477e-a1c5-b51b1b6214e4" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:18:08 user nova-compute[75234]: DEBUG nova.network.neutron [req-2e803a22-b8c0-4fb8-9d5f-2b88ba960ae9 req-245fba30-3466-469b-96d2-89537f2988f1 service nova] [instance: 0aff446c-f417-477e-a1c5-b51b1b6214e4] Refreshing network info cache for port f9148192-6e14-44d4-a31b-081ea47aa289 {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2006}}
Nov 29 03:18:08 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-fd65259f-eb0e-4ebe-a465-65fbe77d1d6a tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] [instance: 0aff446c-f417-477e-a1c5-b51b1b6214e4] Start _get_guest_xml network_info=[{"id": "f9148192-6e14-44d4-a31b-081ea47aa289", "address": "fa:16:3e:46:49:9c", "network": {"id": "0f289171-3d73-4054-95a0-2515b4f206a0", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-1787960284-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "315732b6e3ee44d584b8e510ef199a90", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapf9148192-6e", "ovs_interfaceid": "f9148192-6e14-44d4-a31b-081ea47aa289", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum='c8fc807773e5354afe61636071771906',container_format='bare',created_at=2023-11-29T03:02:15Z,direct_url=<?>,disk_format='qcow2',id=96c30ba7-9b73-4a30-8ba4-7138879573b7,min_disk=0,min_ram=0,name='cirros-0.6.2-x86_64-disk',owner='b18e380921c748b78517b6f1211cd0f1',properties=ImageMetaProps,protected=<?>,size=21430272,status='active',tags=<?>,updated_at=2023-11-29T03:02:17Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [{'device_name': '/dev/vda', 'encryption_format': None, 'encrypted': False, 'device_type': 'disk', 'size': 0, 'disk_bus': 'virtio', 'boot_index': 0, 'encryption_options': None, 'encryption_secret_uuid': None, 'guest_format': None, 'image_id': '96c30ba7-9b73-4a30-8ba4-7138879573b7'}], 'ephemerals': [], 'block_device_mapping': [], 'swap': None} {{(pid=75234) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7562}}
Nov 29 03:18:08 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-fd65259f-eb0e-4ebe-a465-65fbe77d1d6a tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:18:08 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-fd65259f-eb0e-4ebe-a465-65fbe77d1d6a tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:18:08 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-fd65259f-eb0e-4ebe-a465-65fbe77d1d6a tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Searching host: 'user' for CPU controller through CGroups V1... {{(pid=75234) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1652}}
Nov 29 03:18:08 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-fd65259f-eb0e-4ebe-a465-65fbe77d1d6a tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] CPU controller missing on host. {{(pid=75234) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1662}}
Nov 29 03:18:08 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-fd65259f-eb0e-4ebe-a465-65fbe77d1d6a tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Searching host: 'user' for CPU controller through CGroups V2... {{(pid=75234) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1671}}
Nov 29 03:18:08 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-fd65259f-eb0e-4ebe-a465-65fbe77d1d6a tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] CPU controller found on host. {{(pid=75234) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1678}}
Nov 29 03:18:08 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-fd65259f-eb0e-4ebe-a465-65fbe77d1d6a tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=75234) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5400}}
Nov 29 03:18:08 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-fd65259f-eb0e-4ebe-a465-65fbe77d1d6a tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-11-29T03:02:04Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='1',id=6,is_public=True,memory_mb=512,name='m1.tiny',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='c8fc807773e5354afe61636071771906',container_format='bare',created_at=2023-11-29T03:02:15Z,direct_url=<?>,disk_format='qcow2',id=96c30ba7-9b73-4a30-8ba4-7138879573b7,min_disk=0,min_ram=0,name='cirros-0.6.2-x86_64-disk',owner='b18e380921c748b78517b6f1211cd0f1',properties=ImageMetaProps,protected=<?>,size=21430272,status='active',tags=<?>,updated_at=2023-11-29T03:02:17Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:563}}
Nov 29 03:18:08 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-fd65259f-eb0e-4ebe-a465-65fbe77d1d6a tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Flavor limits 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:348}}
Nov 29 03:18:08 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-fd65259f-eb0e-4ebe-a465-65fbe77d1d6a tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Image limits 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:352}}
Nov 29 03:18:08 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-fd65259f-eb0e-4ebe-a465-65fbe77d1d6a tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Flavor pref 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:388}}
Nov 29 03:18:08 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-fd65259f-eb0e-4ebe-a465-65fbe77d1d6a tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Image pref 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:392}}
Nov 29 03:18:08 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-fd65259f-eb0e-4ebe-a465-65fbe77d1d6a tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:430}}
Nov 29 03:18:08 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-fd65259f-eb0e-4ebe-a465-65fbe77d1d6a tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:569}}
Nov 29 03:18:08 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-fd65259f-eb0e-4ebe-a465-65fbe77d1d6a tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=75234) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:471}}
Nov 29 03:18:08 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-fd65259f-eb0e-4ebe-a465-65fbe77d1d6a tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Got 1 possible topologies {{(pid=75234) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:501}}
Nov 29 03:18:08 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-fd65259f-eb0e-4ebe-a465-65fbe77d1d6a tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:575}}
Nov 29 03:18:08 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-fd65259f-eb0e-4ebe-a465-65fbe77d1d6a tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:577}}
Nov 29 03:18:08 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-fd65259f-eb0e-4ebe-a465-65fbe77d1d6a tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-11-29T03:18:04Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-AttachVolumeNegativeTest-server-717472654',display_name='tempest-AttachVolumeNegativeTest-server-717472654',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-attachvolumenegativetest-server-717472654',id=32,image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBGtbFLqgbiAnwNCauVOxBiJsbekFn33Y3bNQ0wRsVUvi8aJ9cY/djuV8LOklgMHTzP1K/jzTQKALfTimFUTXA6U/C/rVArYt3uY6rb/8pJx8N1xNGV/JwqMC7v6GIr/lwg==',key_name='tempest-keypair-2106534641',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='315732b6e3ee44d584b8e510ef199a90',ramdisk_id='',reservation_id='r-l9ya0pnl',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-AttachVolumeNegativeTest-1582496806',owner_user_name='tempest-AttachVolumeNegativeTest-1582496806-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-11-29T03:18:07Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='66686114450f4f31b3a917667a152d8b',uuid=0aff446c-f417-477e-a1c5-b51b1b6214e4,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "f9148192-6e14-44d4-a31b-081ea47aa289", "address": "fa:16:3e:46:49:9c", "network": {"id": "0f289171-3d73-4054-95a0-2515b4f206a0", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-1787960284-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "315732b6e3ee44d584b8e510ef199a90", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapf9148192-6e", "ovs_interfaceid": "f9148192-6e14-44d4-a31b-081ea47aa289", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=75234) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Nov 29 03:18:08 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-fd65259f-eb0e-4ebe-a465-65fbe77d1d6a tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Converting VIF {"id": "f9148192-6e14-44d4-a31b-081ea47aa289", "address": "fa:16:3e:46:49:9c", "network": {"id": "0f289171-3d73-4054-95a0-2515b4f206a0", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-1787960284-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "315732b6e3ee44d584b8e510ef199a90", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapf9148192-6e", "ovs_interfaceid": "f9148192-6e14-44d4-a31b-081ea47aa289", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:18:08 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-fd65259f-eb0e-4ebe-a465-65fbe77d1d6a tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:46:49:9c,bridge_name='br-int',has_traffic_filtering=True,id=f9148192-6e14-44d4-a31b-081ea47aa289,network=Network(0f289171-3d73-4054-95a0-2515b4f206a0),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapf9148192-6e') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:18:08 user nova-compute[75234]: DEBUG nova.objects.instance [None req-fd65259f-eb0e-4ebe-a465-65fbe77d1d6a tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Lazy-loading 'pci_devices' on Instance uuid 0aff446c-f417-477e-a1c5-b51b1b6214e4 {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:18:08 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-fd65259f-eb0e-4ebe-a465-65fbe77d1d6a tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] [instance: 0aff446c-f417-477e-a1c5-b51b1b6214e4] End _get_guest_xml xml=<domain type="kvm">
Nov 29 03:18:08 user nova-compute[75234]:   <uuid>0aff446c-f417-477e-a1c5-b51b1b6214e4</uuid>
Nov 29 03:18:08 user nova-compute[75234]:   <name>instance-00000020</name>
Nov 29 03:18:08 user nova-compute[75234]:   <memory>524288</memory>
Nov 29 03:18:08 user nova-compute[75234]:   <vcpu>1</vcpu>
Nov 29 03:18:08 user nova-compute[75234]:   <metadata>
Nov 29 03:18:08 user nova-compute[75234]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Nov 29 03:18:08 user nova-compute[75234]:       <nova:package version="0.0.0"/>
Nov 29 03:18:08 user nova-compute[75234]:       <nova:name>tempest-AttachVolumeNegativeTest-server-717472654</nova:name>
Nov 29 03:18:08 user nova-compute[75234]:       <nova:creationTime>2023-11-29 03:18:08</nova:creationTime>
Nov 29 03:18:08 user nova-compute[75234]:       <nova:flavor name="m1.tiny">
Nov 29 03:18:08 user nova-compute[75234]:         <nova:memory>512</nova:memory>
Nov 29 03:18:08 user nova-compute[75234]:         <nova:disk>1</nova:disk>
Nov 29 03:18:08 user nova-compute[75234]:         <nova:swap>0</nova:swap>
Nov 29 03:18:08 user nova-compute[75234]:         <nova:ephemeral>0</nova:ephemeral>
Nov 29 03:18:08 user nova-compute[75234]:         <nova:vcpus>1</nova:vcpus>
Nov 29 03:18:08 user nova-compute[75234]:       </nova:flavor>
Nov 29 03:18:08 user nova-compute[75234]:       <nova:owner>
Nov 29 03:18:08 user nova-compute[75234]:         <nova:user uuid="66686114450f4f31b3a917667a152d8b">tempest-AttachVolumeNegativeTest-1582496806-project-member</nova:user>
Nov 29 03:18:08 user nova-compute[75234]:         <nova:project uuid="315732b6e3ee44d584b8e510ef199a90">tempest-AttachVolumeNegativeTest-1582496806</nova:project>
Nov 29 03:18:08 user nova-compute[75234]:       </nova:owner>
Nov 29 03:18:08 user nova-compute[75234]:       <nova:root type="image" uuid="96c30ba7-9b73-4a30-8ba4-7138879573b7"/>
Nov 29 03:18:08 user nova-compute[75234]:       <nova:ports>
Nov 29 03:18:08 user nova-compute[75234]:         <nova:port uuid="f9148192-6e14-44d4-a31b-081ea47aa289">
Nov 29 03:18:08 user nova-compute[75234]:           <nova:ip type="fixed" address="10.0.0.14" ipVersion="4"/>
Nov 29 03:18:08 user nova-compute[75234]:         </nova:port>
Nov 29 03:18:08 user nova-compute[75234]:       </nova:ports>
Nov 29 03:18:08 user nova-compute[75234]:     </nova:instance>
Nov 29 03:18:08 user nova-compute[75234]:   </metadata>
Nov 29 03:18:08 user nova-compute[75234]:   <sysinfo type="smbios">
Nov 29 03:18:08 user nova-compute[75234]:     <system>
Nov 29 03:18:08 user nova-compute[75234]:       <entry name="manufacturer">OpenStack Foundation</entry>
Nov 29 03:18:08 user nova-compute[75234]:       <entry name="product">OpenStack Nova</entry>
Nov 29 03:18:08 user nova-compute[75234]:       <entry name="version">0.0.0</entry>
Nov 29 03:18:08 user nova-compute[75234]:       <entry name="serial">0aff446c-f417-477e-a1c5-b51b1b6214e4</entry>
Nov 29 03:18:08 user nova-compute[75234]:       <entry name="uuid">0aff446c-f417-477e-a1c5-b51b1b6214e4</entry>
Nov 29 03:18:08 user nova-compute[75234]:       <entry name="family">Virtual Machine</entry>
Nov 29 03:18:08 user nova-compute[75234]:     </system>
Nov 29 03:18:08 user nova-compute[75234]:   </sysinfo>
Nov 29 03:18:08 user nova-compute[75234]:   <os>
Nov 29 03:18:08 user nova-compute[75234]:     <type arch="x86_64" machine="pc">hvm</type>
Nov 29 03:18:08 user nova-compute[75234]:     <boot dev="hd"/>
Nov 29 03:18:08 user nova-compute[75234]:     <smbios mode="sysinfo"/>
Nov 29 03:18:08 user nova-compute[75234]:   </os>
Nov 29 03:18:08 user nova-compute[75234]:   <features>
Nov 29 03:18:08 user nova-compute[75234]:     <acpi/>
Nov 29 03:18:08 user nova-compute[75234]:     <apic/>
Nov 29 03:18:08 user nova-compute[75234]:     <vmcoreinfo/>
Nov 29 03:18:08 user nova-compute[75234]:   </features>
Nov 29 03:18:08 user nova-compute[75234]:   <clock offset="utc">
Nov 29 03:18:08 user nova-compute[75234]:     <timer name="pit" tickpolicy="delay"/>
Nov 29 03:18:08 user nova-compute[75234]:     <timer name="rtc" tickpolicy="catchup"/>
Nov 29 03:18:08 user nova-compute[75234]:     <timer name="hpet" present="no"/>
Nov 29 03:18:08 user nova-compute[75234]:   </clock>
Nov 29 03:18:08 user nova-compute[75234]:   <cpu mode="custom" match="exact">
Nov 29 03:18:08 user nova-compute[75234]:     <model>Nehalem</model>
Nov 29 03:18:08 user nova-compute[75234]:     <topology sockets="1" cores="1" threads="1"/>
Nov 29 03:18:08 user nova-compute[75234]:   </cpu>
Nov 29 03:18:08 user nova-compute[75234]:   <devices>
Nov 29 03:18:08 user nova-compute[75234]:     <disk type="file" device="disk">
Nov 29 03:18:08 user nova-compute[75234]:       <driver name="qemu" type="qcow2" cache="none"/>
Nov 29 03:18:08 user nova-compute[75234]:       <source file="/opt/stack/data/nova/instances/0aff446c-f417-477e-a1c5-b51b1b6214e4/disk"/>
Nov 29 03:18:08 user nova-compute[75234]:       <target dev="vda" bus="virtio"/>
Nov 29 03:18:08 user nova-compute[75234]:     </disk>
Nov 29 03:18:08 user nova-compute[75234]:     <interface type="ethernet">
Nov 29 03:18:08 user nova-compute[75234]:       <mac address="fa:16:3e:46:49:9c"/>
Nov 29 03:18:08 user nova-compute[75234]:       <model type="virtio"/>
Nov 29 03:18:08 user nova-compute[75234]:       <mtu size="1442"/>
Nov 29 03:18:08 user nova-compute[75234]:       <target dev="tapf9148192-6e"/>
Nov 29 03:18:08 user nova-compute[75234]:     </interface>
Nov 29 03:18:08 user nova-compute[75234]:     <serial type="pty">
Nov 29 03:18:08 user nova-compute[75234]:       <log file="/opt/stack/data/nova/instances/0aff446c-f417-477e-a1c5-b51b1b6214e4/console.log" append="off"/>
Nov 29 03:18:08 user nova-compute[75234]:     </serial>
Nov 29 03:18:08 user nova-compute[75234]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Nov 29 03:18:08 user nova-compute[75234]:     <video>
Nov 29 03:18:08 user nova-compute[75234]:       <model type="virtio"/>
Nov 29 03:18:08 user nova-compute[75234]:     </video>
Nov 29 03:18:08 user nova-compute[75234]:     <rng model="virtio">
Nov 29 03:18:08 user nova-compute[75234]:       <backend model="random">/dev/urandom</backend>
Nov 29 03:18:08 user nova-compute[75234]:     </rng>
Nov 29 03:18:08 user nova-compute[75234]:     <controller type="usb" index="0" model="none"/>
Nov 29 03:18:08 user nova-compute[75234]:     <memballoon model="virtio">
Nov 29 03:18:08 user nova-compute[75234]:       <stats period="10"/>
Nov 29 03:18:08 user nova-compute[75234]:     </memballoon>
Nov 29 03:18:08 user nova-compute[75234]:   </devices>
Nov 29 03:18:08 user nova-compute[75234]: </domain>
Nov 29 03:18:08 user nova-compute[75234]:  {{(pid=75234) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7568}}
Nov 29 03:18:08 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-fd65259f-eb0e-4ebe-a465-65fbe77d1d6a tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-11-29T03:18:04Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-AttachVolumeNegativeTest-server-717472654',display_name='tempest-AttachVolumeNegativeTest-server-717472654',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-attachvolumenegativetest-server-717472654',id=32,image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBGtbFLqgbiAnwNCauVOxBiJsbekFn33Y3bNQ0wRsVUvi8aJ9cY/djuV8LOklgMHTzP1K/jzTQKALfTimFUTXA6U/C/rVArYt3uY6rb/8pJx8N1xNGV/JwqMC7v6GIr/lwg==',key_name='tempest-keypair-2106534641',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='315732b6e3ee44d584b8e510ef199a90',ramdisk_id='',reservation_id='r-l9ya0pnl',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-AttachVolumeNegativeTest-1582496806',owner_user_name='tempest-AttachVolumeNegativeTest-1582496806-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-11-29T03:18:07Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='66686114450f4f31b3a917667a152d8b',uuid=0aff446c-f417-477e-a1c5-b51b1b6214e4,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "f9148192-6e14-44d4-a31b-081ea47aa289", "address": "fa:16:3e:46:49:9c", "network": {"id": "0f289171-3d73-4054-95a0-2515b4f206a0", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-1787960284-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "315732b6e3ee44d584b8e510ef199a90", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapf9148192-6e", "ovs_interfaceid": "f9148192-6e14-44d4-a31b-081ea47aa289", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Nov 29 03:18:08 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-fd65259f-eb0e-4ebe-a465-65fbe77d1d6a tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Converting VIF {"id": "f9148192-6e14-44d4-a31b-081ea47aa289", "address": "fa:16:3e:46:49:9c", "network": {"id": "0f289171-3d73-4054-95a0-2515b4f206a0", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-1787960284-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "315732b6e3ee44d584b8e510ef199a90", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapf9148192-6e", "ovs_interfaceid": "f9148192-6e14-44d4-a31b-081ea47aa289", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:18:08 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-fd65259f-eb0e-4ebe-a465-65fbe77d1d6a tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:46:49:9c,bridge_name='br-int',has_traffic_filtering=True,id=f9148192-6e14-44d4-a31b-081ea47aa289,network=Network(0f289171-3d73-4054-95a0-2515b4f206a0),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapf9148192-6e') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:18:08 user nova-compute[75234]: DEBUG os_vif [None req-fd65259f-eb0e-4ebe-a465-65fbe77d1d6a tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:46:49:9c,bridge_name='br-int',has_traffic_filtering=True,id=f9148192-6e14-44d4-a31b-081ea47aa289,network=Network(0f289171-3d73-4054-95a0-2515b4f206a0),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapf9148192-6e') {{(pid=75234) plug /opt/stack/data/venv/lib/python3.10/site-packages/os_vif/__init__.py:76}}
Nov 29 03:18:08 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:18:08 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:18:08 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Nov 29 03:18:08 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:18:08 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbCreateCommand(_result=None, table=QoS, columns={'type': 'linux-noop', 'external_ids': {'id': '19ab28b0-1036-56db-983c-fbe3819e2bb3', '_type': 'linux-noop'}}, row=False) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:18:08 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:18:08 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:18:08 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:18:08 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tapf9148192-6e, may_exist=True, interface_attrs={}) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:18:08 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Port, record=tapf9148192-6e, col_values=(('qos', UUID('eb7c227b-1518-42d1-9b49-f7e618eadee8')),), if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:18:08 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DbSetCommand(_result=None, table=Interface, record=tapf9148192-6e, col_values=(('external_ids', {'iface-id': 'f9148192-6e14-44d4-a31b-081ea47aa289', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:46:49:9c', 'vm-uuid': '0aff446c-f417-477e-a1c5-b51b1b6214e4'}),), if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:18:08 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:18:08 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:18:08 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:18:08 user nova-compute[75234]: INFO os_vif [None req-fd65259f-eb0e-4ebe-a465-65fbe77d1d6a tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:46:49:9c,bridge_name='br-int',has_traffic_filtering=True,id=f9148192-6e14-44d4-a31b-081ea47aa289,network=Network(0f289171-3d73-4054-95a0-2515b4f206a0),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapf9148192-6e')
Nov 29 03:18:08 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-fd65259f-eb0e-4ebe-a465-65fbe77d1d6a tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] No BDM found with device name vda, not building metadata. {{(pid=75234) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12089}}
Nov 29 03:18:08 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-fd65259f-eb0e-4ebe-a465-65fbe77d1d6a tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] No VIF found with MAC fa:16:3e:46:49:9c, not building metadata {{(pid=75234) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Nov 29 03:18:08 user nova-compute[75234]: DEBUG nova.network.neutron [req-2e803a22-b8c0-4fb8-9d5f-2b88ba960ae9 req-245fba30-3466-469b-96d2-89537f2988f1 service nova] [instance: 0aff446c-f417-477e-a1c5-b51b1b6214e4] Updated VIF entry in instance network info cache for port f9148192-6e14-44d4-a31b-081ea47aa289. {{(pid=75234) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3481}}
Nov 29 03:18:08 user nova-compute[75234]: DEBUG nova.network.neutron [req-2e803a22-b8c0-4fb8-9d5f-2b88ba960ae9 req-245fba30-3466-469b-96d2-89537f2988f1 service nova] [instance: 0aff446c-f417-477e-a1c5-b51b1b6214e4] Updating instance_info_cache with network_info: [{"id": "f9148192-6e14-44d4-a31b-081ea47aa289", "address": "fa:16:3e:46:49:9c", "network": {"id": "0f289171-3d73-4054-95a0-2515b4f206a0", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-1787960284-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "315732b6e3ee44d584b8e510ef199a90", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapf9148192-6e", "ovs_interfaceid": "f9148192-6e14-44d4-a31b-081ea47aa289", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:18:08 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-2e803a22-b8c0-4fb8-9d5f-2b88ba960ae9 req-245fba30-3466-469b-96d2-89537f2988f1 service nova] Releasing lock "refresh_cache-0aff446c-f417-477e-a1c5-b51b1b6214e4" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:18:09 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:18:09 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:18:09 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:18:10 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:18:10 user nova-compute[75234]: DEBUG nova.compute.manager [req-004f4a21-d0c0-4f37-9c3e-bb64aca2cab8 req-d378fbd0-59f7-49dd-baef-627a1645fdd7 service nova] [instance: 0aff446c-f417-477e-a1c5-b51b1b6214e4] Received event network-vif-plugged-f9148192-6e14-44d4-a31b-081ea47aa289 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:18:10 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-004f4a21-d0c0-4f37-9c3e-bb64aca2cab8 req-d378fbd0-59f7-49dd-baef-627a1645fdd7 service nova] Acquiring lock "0aff446c-f417-477e-a1c5-b51b1b6214e4-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:18:10 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-004f4a21-d0c0-4f37-9c3e-bb64aca2cab8 req-d378fbd0-59f7-49dd-baef-627a1645fdd7 service nova] Lock "0aff446c-f417-477e-a1c5-b51b1b6214e4-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:18:10 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-004f4a21-d0c0-4f37-9c3e-bb64aca2cab8 req-d378fbd0-59f7-49dd-baef-627a1645fdd7 service nova] Lock "0aff446c-f417-477e-a1c5-b51b1b6214e4-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:18:10 user nova-compute[75234]: DEBUG nova.compute.manager [req-004f4a21-d0c0-4f37-9c3e-bb64aca2cab8 req-d378fbd0-59f7-49dd-baef-627a1645fdd7 service nova] [instance: 0aff446c-f417-477e-a1c5-b51b1b6214e4] No waiting events found dispatching network-vif-plugged-f9148192-6e14-44d4-a31b-081ea47aa289 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:18:10 user nova-compute[75234]: WARNING nova.compute.manager [req-004f4a21-d0c0-4f37-9c3e-bb64aca2cab8 req-d378fbd0-59f7-49dd-baef-627a1645fdd7 service nova] [instance: 0aff446c-f417-477e-a1c5-b51b1b6214e4] Received unexpected event network-vif-plugged-f9148192-6e14-44d4-a31b-081ea47aa289 for instance with vm_state building and task_state spawning.
Nov 29 03:18:11 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:18:11 user nova-compute[75234]: DEBUG nova.compute.manager [None req-fd65259f-eb0e-4ebe-a465-65fbe77d1d6a tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] [instance: 0aff446c-f417-477e-a1c5-b51b1b6214e4] Instance event wait completed in 0 seconds for  {{(pid=75234) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Nov 29 03:18:11 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-fd65259f-eb0e-4ebe-a465-65fbe77d1d6a tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] [instance: 0aff446c-f417-477e-a1c5-b51b1b6214e4] Guest created on hypervisor {{(pid=75234) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4421}}
Nov 29 03:18:11 user nova-compute[75234]: INFO nova.virt.libvirt.driver [-] [instance: 0aff446c-f417-477e-a1c5-b51b1b6214e4] Instance spawned successfully.
Nov 29 03:18:11 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-fd65259f-eb0e-4ebe-a465-65fbe77d1d6a tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] [instance: 0aff446c-f417-477e-a1c5-b51b1b6214e4] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:904}}
Nov 29 03:18:11 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-fd65259f-eb0e-4ebe-a465-65fbe77d1d6a tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] [instance: 0aff446c-f417-477e-a1c5-b51b1b6214e4] Found default for hw_cdrom_bus of ide {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:18:11 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-fd65259f-eb0e-4ebe-a465-65fbe77d1d6a tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] [instance: 0aff446c-f417-477e-a1c5-b51b1b6214e4] Found default for hw_disk_bus of virtio {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:18:11 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-fd65259f-eb0e-4ebe-a465-65fbe77d1d6a tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] [instance: 0aff446c-f417-477e-a1c5-b51b1b6214e4] Found default for hw_input_bus of None {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:18:11 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-fd65259f-eb0e-4ebe-a465-65fbe77d1d6a tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] [instance: 0aff446c-f417-477e-a1c5-b51b1b6214e4] Found default for hw_pointer_model of None {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:18:11 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-fd65259f-eb0e-4ebe-a465-65fbe77d1d6a tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] [instance: 0aff446c-f417-477e-a1c5-b51b1b6214e4] Found default for hw_video_model of virtio {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:18:11 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-fd65259f-eb0e-4ebe-a465-65fbe77d1d6a tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] [instance: 0aff446c-f417-477e-a1c5-b51b1b6214e4] Found default for hw_vif_model of virtio {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:18:12 user nova-compute[75234]: INFO nova.compute.manager [None req-fd65259f-eb0e-4ebe-a465-65fbe77d1d6a tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] [instance: 0aff446c-f417-477e-a1c5-b51b1b6214e4] Took 5.28 seconds to spawn the instance on the hypervisor.
Nov 29 03:18:12 user nova-compute[75234]: DEBUG nova.compute.manager [None req-fd65259f-eb0e-4ebe-a465-65fbe77d1d6a tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] [instance: 0aff446c-f417-477e-a1c5-b51b1b6214e4] Checking state {{(pid=75234) _get_power_state /opt/stack/nova/nova/compute/manager.py:1782}}
Nov 29 03:18:12 user nova-compute[75234]: DEBUG nova.compute.manager [req-49625fbc-5e3e-4ab6-8232-954a5753fb79 req-33652ec1-db0a-4e67-9821-10e78bb9159a service nova] [instance: 0aff446c-f417-477e-a1c5-b51b1b6214e4] Received event network-vif-plugged-f9148192-6e14-44d4-a31b-081ea47aa289 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:18:12 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-49625fbc-5e3e-4ab6-8232-954a5753fb79 req-33652ec1-db0a-4e67-9821-10e78bb9159a service nova] Acquiring lock "0aff446c-f417-477e-a1c5-b51b1b6214e4-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:18:12 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-49625fbc-5e3e-4ab6-8232-954a5753fb79 req-33652ec1-db0a-4e67-9821-10e78bb9159a service nova] Lock "0aff446c-f417-477e-a1c5-b51b1b6214e4-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:18:12 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-49625fbc-5e3e-4ab6-8232-954a5753fb79 req-33652ec1-db0a-4e67-9821-10e78bb9159a service nova] Lock "0aff446c-f417-477e-a1c5-b51b1b6214e4-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:18:12 user nova-compute[75234]: DEBUG nova.compute.manager [req-49625fbc-5e3e-4ab6-8232-954a5753fb79 req-33652ec1-db0a-4e67-9821-10e78bb9159a service nova] [instance: 0aff446c-f417-477e-a1c5-b51b1b6214e4] No waiting events found dispatching network-vif-plugged-f9148192-6e14-44d4-a31b-081ea47aa289 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:18:12 user nova-compute[75234]: WARNING nova.compute.manager [req-49625fbc-5e3e-4ab6-8232-954a5753fb79 req-33652ec1-db0a-4e67-9821-10e78bb9159a service nova] [instance: 0aff446c-f417-477e-a1c5-b51b1b6214e4] Received unexpected event network-vif-plugged-f9148192-6e14-44d4-a31b-081ea47aa289 for instance with vm_state building and task_state spawning.
Nov 29 03:18:12 user nova-compute[75234]: INFO nova.compute.manager [None req-fd65259f-eb0e-4ebe-a465-65fbe77d1d6a tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] [instance: 0aff446c-f417-477e-a1c5-b51b1b6214e4] Took 6.62 seconds to build instance.
Nov 29 03:18:12 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-fd65259f-eb0e-4ebe-a465-65fbe77d1d6a tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Lock "0aff446c-f417-477e-a1c5-b51b1b6214e4" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 6.764s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:18:13 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:18:16 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:18:18 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:18:21 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:18:22 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-9bd15904-212d-458a-a01a-a6fdf8021a15 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] [instance: 1c8b7559-29ab-4982-8c75-ea2a0f0101cb] Calling os-brick to attach iSCSI Volume {{(pid=75234) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Nov 29 03:18:22 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-9bd15904-212d-458a-a01a-a6fdf8021a15 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f8a5050c9a0>, 'connection_properties': {'target_iqn': 'iqn.2010-10.org.openstack:a5b104bd-399e-4445-a186-0d4275ae271a', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'XxdoDMRn', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:176}}
Nov 29 03:18:22 user nova-compute[75234]: WARNING os_brick.initiator.connectors.base [None req-9bd15904-212d-458a-a01a-a6fdf8021a15 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Nov 29 03:18:22 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-9bd15904-212d-458a-a01a-a6fdf8021a15 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:68}}
Nov 29 03:18:22 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-9bd15904-212d-458a-a01a-a6fdf8021a15 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:73}}
Nov 29 03:18:22 user nova-compute[75234]: WARNING os_brick.initiator.connectors.base [None req-9bd15904-212d-458a-a01a-a6fdf8021a15 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Nov 29 03:18:22 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-9bd15904-212d-458a-a01a-a6fdf8021a15 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:a5b104bd-399e-4445-a186-0d4275ae271a" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:68}}
Nov 29 03:18:22 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-9bd15904-212d-458a-a01a-a6fdf8021a15 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:a5b104bd-399e-4445-a186-0d4275ae271a" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:73}}
Nov 29 03:18:22 user nova-compute[75234]: INFO os_brick.initiator.connectors.iscsi [None req-9bd15904-212d-458a-a01a-a6fdf8021a15 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Nov 29 03:18:22 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:a5b104bd-399e-4445-a186-0d4275ae271a -p 172.16.0.220:3260 {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:18:22 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:a5b104bd-399e-4445-a186-0d4275ae271a -p 172.16.0.220:3260" returned: 21 in 0.021s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:18:22 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[db4d0faa-a85c-4041-bac5-af556ac927bc]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:18:22 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-9bd15904-212d-458a-a01a-a6fdf8021a15 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Nov 29 03:18:22 user nova-compute[75234]:  {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:18:22 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:a5b104bd-399e-4445-a186-0d4275ae271a -p 172.16.0.220:3260 --interface default --op new {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:18:22 user nova-compute[75234]: DEBUG nova.compute.manager [req-e9bfef58-f895-42b7-bc9c-772ebab67bc8 req-edf35270-38b3-4487-8ebd-693ffd8a0adf service nova] [instance: aa190a9f-ba53-43fb-8bae-2c429d0d1228] Received event volume-reimaged-07f83efc-272d-4e10-9a2b-48d967db613c {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:18:22 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-e9bfef58-f895-42b7-bc9c-772ebab67bc8 req-edf35270-38b3-4487-8ebd-693ffd8a0adf service nova] Acquiring lock "aa190a9f-ba53-43fb-8bae-2c429d0d1228-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:18:22 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-e9bfef58-f895-42b7-bc9c-772ebab67bc8 req-edf35270-38b3-4487-8ebd-693ffd8a0adf service nova] Lock "aa190a9f-ba53-43fb-8bae-2c429d0d1228-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:18:22 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-e9bfef58-f895-42b7-bc9c-772ebab67bc8 req-edf35270-38b3-4487-8ebd-693ffd8a0adf service nova] Lock "aa190a9f-ba53-43fb-8bae-2c429d0d1228-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:18:22 user nova-compute[75234]: DEBUG nova.compute.manager [req-e9bfef58-f895-42b7-bc9c-772ebab67bc8 req-edf35270-38b3-4487-8ebd-693ffd8a0adf service nova] [instance: aa190a9f-ba53-43fb-8bae-2c429d0d1228] Processing event volume-reimaged-07f83efc-272d-4e10-9a2b-48d967db613c {{(pid=75234) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10872}}
Nov 29 03:18:22 user nova-compute[75234]: DEBUG nova.compute.manager [None req-e9bfef58-f895-42b7-bc9c-772ebab67bc8 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] [instance: aa190a9f-ba53-43fb-8bae-2c429d0d1228] Instance event wait completed in 17 seconds for volume-reimaged {{(pid=75234) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Nov 29 03:18:22 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:a5b104bd-399e-4445-a186-0d4275ae271a -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.024s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:18:22 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[820ae46a-072c-4d82-867d-e5fe3cdca647]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:a5b104bd-399e-4445-a186-0d4275ae271a] added\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:18:22 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-9bd15904-212d-458a-a01a-a6fdf8021a15 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:a5b104bd-399e-4445-a186-0d4275ae271a] added
Nov 29 03:18:22 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:18:22 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:a5b104bd-399e-4445-a186-0d4275ae271a -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:18:22 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:a5b104bd-399e-4445-a186-0d4275ae271a -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.026s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:18:22 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[a6bed540-64f5-4239-be56-0803402c9b62]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:18:22 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-9bd15904-212d-458a-a01a-a6fdf8021a15 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:18:22 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:a5b104bd-399e-4445-a186-0d4275ae271a -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:18:22 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:a5b104bd-399e-4445-a186-0d4275ae271a -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.019s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:18:22 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[0a63aee6-0b74-4692-b6ba-1f82943ee6d2]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:18:22 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-9bd15904-212d-458a-a01a-a6fdf8021a15 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:18:22 user nova-compute[75234]: INFO nova.virt.block_device [None req-e9bfef58-f895-42b7-bc9c-772ebab67bc8 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] [instance: aa190a9f-ba53-43fb-8bae-2c429d0d1228] Booting with volume 07f83efc-272d-4e10-9a2b-48d967db613c at /dev/vda
Nov 29 03:18:22 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:a5b104bd-399e-4445-a186-0d4275ae271a -p 172.16.0.220:3260 --op update -n node.session.auth.username -v XxdoDMRn {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:18:22 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:a5b104bd-399e-4445-a186-0d4275ae271a -p 172.16.0.220:3260 --op update -n node.session.auth.username -v XxdoDMRn" returned: 0 in 0.024s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:18:22 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[26d4e75c-88d5-49bb-b0d5-afe417cfbb66]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:18:22 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-9bd15904-212d-458a-a01a-a6fdf8021a15 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'XxdoDMRn'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:18:22 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:a5b104bd-399e-4445-a186-0d4275ae271a -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:18:22 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:a5b104bd-399e-4445-a186-0d4275ae271a -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.032s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:18:22 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[6eac37e5-eff2-4ac2-8c94-c709df044a57]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:18:22 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-9bd15904-212d-458a-a01a-a6fdf8021a15 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:18:22 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:18:22 user nova-compute[75234]: DEBUG os_brick.utils [None req-e9bfef58-f895-42b7-bc9c-772ebab67bc8 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:176}}
Nov 29 03:18:22 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.021s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:18:22 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[e0ce82d3-d586-4eb3-8328-c3873bc211a2]: (4, ('tcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6c2e1179-dd3c-41c4-9594-bf4a16752c67 (non-flash)\ntcp: [29] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bf57b967-7fad-4d76-abb2-319fe88f8ef9 (non-flash)\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:18:22 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-9bd15904-212d-458a-a01a-a6fdf8021a15 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6c2e1179-dd3c-41c4-9594-bf4a16752c67 (non-flash)
Nov 29 03:18:22 user nova-compute[75234]: tcp: [29] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bf57b967-7fad-4d76-abb2-319fe88f8ef9 (non-flash)
Nov 29 03:18:22 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:18:22 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-9bd15904-212d-458a-a01a-a6fdf8021a15 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] iscsi session list stdout=tcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6c2e1179-dd3c-41c4-9594-bf4a16752c67 (non-flash)
Nov 29 03:18:22 user nova-compute[75234]: tcp: [29] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bf57b967-7fad-4d76-abb2-319fe88f8ef9 (non-flash)
Nov 29 03:18:22 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsi_session /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Nov 29 03:18:22 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:18:22 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:a5b104bd-399e-4445-a186-0d4275ae271a -p 172.16.0.220:3260 --login {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:18:22 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.017s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:18:22 user nova-compute[75234]: DEBUG nova.compute.manager [None req-4f83f151-3776-4aa8-b91c-a6dd77c94abe tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] [instance: af699bc2-9daa-4b5a-9b4a-ad92fe857d70] Start spawning the instance on the hypervisor. {{(pid=75234) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2628}}
Nov 29 03:18:22 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-4f83f151-3776-4aa8-b91c-a6dd77c94abe tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] [instance: af699bc2-9daa-4b5a-9b4a-ad92fe857d70] Creating instance directory {{(pid=75234) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4727}}
Nov 29 03:18:22 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-4f83f151-3776-4aa8-b91c-a6dd77c94abe tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] [instance: af699bc2-9daa-4b5a-9b4a-ad92fe857d70] Creating image(s)
Nov 29 03:18:22 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-4f83f151-3776-4aa8-b91c-a6dd77c94abe tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] [instance: af699bc2-9daa-4b5a-9b4a-ad92fe857d70] Did not create local disks {{(pid=75234) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4863}}
Nov 29 03:18:22 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-4f83f151-3776-4aa8-b91c-a6dd77c94abe tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] [instance: af699bc2-9daa-4b5a-9b4a-ad92fe857d70] Ensure instance console log exists: /opt/stack/data/nova/instances/af699bc2-9daa-4b5a-9b4a-ad92fe857d70/console.log {{(pid=75234) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4613}}
Nov 29 03:18:22 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-4f83f151-3776-4aa8-b91c-a6dd77c94abe tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:18:22 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-4f83f151-3776-4aa8-b91c-a6dd77c94abe tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:18:22 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-4f83f151-3776-4aa8-b91c-a6dd77c94abe tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:18:22 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-4f83f151-3776-4aa8-b91c-a6dd77c94abe tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] [instance: af699bc2-9daa-4b5a-9b4a-ad92fe857d70] Start _get_guest_xml network_info=[{"id": "66dcb0f9-3180-42f7-851d-19b0c4f4119c", "address": "fa:16:3e:dd:9e:50", "network": {"id": "1f946e69-dbec-429c-83f8-a186bad4f2cc", "bridge": "br-int", "label": "tempest-ServerActionsTestOtherA-486369423-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "c1cb3d1f12f242a785a688d59dbe3cd6", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap66dcb0f9-31", "ovs_interfaceid": "66dcb0f9-3180-42f7-851d-19b0c4f4119c", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, '/dev/vda': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum=<?>,container_format=<?>,created_at=<?>,direct_url=<?>,disk_format=<?>,id=<?>,min_disk=0,min_ram=0,name=<?>,owner=<?>,properties=ImageMetaProps,protected=<?>,size=1073741824,status='active',tags=<?>,updated_at=<?>,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [], 'ephemerals': [], 'block_device_mapping': [{'mount_device': '/dev/vda', 'attachment_id': '585001f4-3bc2-4401-9995-c1f93cff9cfd', 'disk_bus': 'virtio', 'device_type': 'disk', 'boot_index': 0, 'guest_format': None, 'delete_on_termination': True, 'connection_info': {'driver_volume_type': 'iscsi', 'data': {'target_iqn': 'iqn.2010-10.org.openstack:cf6ab229-533a-424e-bd67-1d54d50ad4be', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'wodfcngk', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}, 'status': 'reserved', 'instance': 'af699bc2-9daa-4b5a-9b4a-ad92fe857d70', 'attached_at': '', 'detached_at': '', 'volume_id': 'cf6ab229-533a-424e-bd67-1d54d50ad4be', 'serial': 'cf6ab229-533a-424e-bd67-1d54d50ad4be'}, 'volume_type': None} {{(pid=75234) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7562}}
Nov 29 03:18:22 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[ea205bbb-efaa-43d7-a31d-594bd792bea9]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:dd1368c5ba2c\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:18:22 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt -v / -n -o SOURCE {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:18:22 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-4f83f151-3776-4aa8-b91c-a6dd77c94abe tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:18:22 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-4f83f151-3776-4aa8-b91c-a6dd77c94abe tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:18:22 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-4f83f151-3776-4aa8-b91c-a6dd77c94abe tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Searching host: 'user' for CPU controller through CGroups V1... {{(pid=75234) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1652}}
Nov 29 03:18:22 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-4f83f151-3776-4aa8-b91c-a6dd77c94abe tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] CPU controller missing on host. {{(pid=75234) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1662}}
Nov 29 03:18:22 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-4f83f151-3776-4aa8-b91c-a6dd77c94abe tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Searching host: 'user' for CPU controller through CGroups V2... {{(pid=75234) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1671}}
Nov 29 03:18:22 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-4f83f151-3776-4aa8-b91c-a6dd77c94abe tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] CPU controller found on host. {{(pid=75234) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1678}}
Nov 29 03:18:22 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-4f83f151-3776-4aa8-b91c-a6dd77c94abe tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=75234) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5400}}
Nov 29 03:18:22 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-4f83f151-3776-4aa8-b91c-a6dd77c94abe tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-11-29T03:02:04Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='1',id=6,is_public=True,memory_mb=512,name='m1.tiny',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum=<?>,container_format=<?>,created_at=<?>,direct_url=<?>,disk_format=<?>,id=<?>,min_disk=0,min_ram=0,name=<?>,owner=<?>,properties=ImageMetaProps,protected=<?>,size=1073741824,status='active',tags=<?>,updated_at=<?>,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:563}}
Nov 29 03:18:22 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-4f83f151-3776-4aa8-b91c-a6dd77c94abe tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Flavor limits 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:348}}
Nov 29 03:18:22 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-4f83f151-3776-4aa8-b91c-a6dd77c94abe tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Image limits 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:352}}
Nov 29 03:18:22 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-4f83f151-3776-4aa8-b91c-a6dd77c94abe tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Flavor pref 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:388}}
Nov 29 03:18:22 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-4f83f151-3776-4aa8-b91c-a6dd77c94abe tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Image pref 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:392}}
Nov 29 03:18:22 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-4f83f151-3776-4aa8-b91c-a6dd77c94abe tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:430}}
Nov 29 03:18:22 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-4f83f151-3776-4aa8-b91c-a6dd77c94abe tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:569}}
Nov 29 03:18:22 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-4f83f151-3776-4aa8-b91c-a6dd77c94abe tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=75234) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:471}}
Nov 29 03:18:22 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-4f83f151-3776-4aa8-b91c-a6dd77c94abe tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Got 1 possible topologies {{(pid=75234) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:501}}
Nov 29 03:18:22 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-4f83f151-3776-4aa8-b91c-a6dd77c94abe tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:575}}
Nov 29 03:18:22 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-4f83f151-3776-4aa8-b91c-a6dd77c94abe tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:577}}
Nov 29 03:18:22 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-4f83f151-3776-4aa8-b91c-a6dd77c94abe tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] [instance: af699bc2-9daa-4b5a-9b4a-ad92fe857d70] Calling os-brick to attach iSCSI Volume {{(pid=75234) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Nov 29 03:18:22 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-4f83f151-3776-4aa8-b91c-a6dd77c94abe tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f8a5050c9a0>, 'connection_properties': {'target_iqn': 'iqn.2010-10.org.openstack:cf6ab229-533a-424e-bd67-1d54d50ad4be', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'wodfcngk', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:176}}
Nov 29 03:18:22 user nova-compute[75234]: WARNING os_brick.initiator.connectors.base [None req-4f83f151-3776-4aa8-b91c-a6dd77c94abe tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Nov 29 03:18:22 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-4f83f151-3776-4aa8-b91c-a6dd77c94abe tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:68}}
Nov 29 03:18:22 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt -v / -n -o SOURCE" returned: 0 in 0.022s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:18:22 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[ecf19533-3729-4b56-9d4d-6c1af07e8104]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:18:22 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:a5b104bd-399e-4445-a186-0d4275ae271a -p 172.16.0.220:3260 --login" returned: 0 in 0.070s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:18:22 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[b40b6f02-06c9-44b7-a41c-9831369e13a4]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:a5b104bd-399e-4445-a186-0d4275ae271a, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:a5b104bd-399e-4445-a186-0d4275ae271a, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:18:22 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-9bd15904-212d-458a-a01a-a6fdf8021a15 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:a5b104bd-399e-4445-a186-0d4275ae271a, portal: 172.16.0.220,3260]
Nov 29 03:18:22 user nova-compute[75234]: Login to [iface: default, target: iqn.2010-10.org.openstack:a5b104bd-399e-4445-a186-0d4275ae271a, portal: 172.16.0.220,3260] successful.
Nov 29 03:18:22 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:18:22 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:18:22 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:a5b104bd-399e-4445-a186-0d4275ae271a -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:18:22 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.020s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:18:22 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[04a6307f-a51a-4df4-be90-0a98891bd71b]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:18:22 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[445889d5-f114-4e15-b3f1-07446f4b0af1]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:18:22 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-e9bfef58-f895-42b7-bc9c-772ebab67bc8 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] Running cmd (subprocess): nvme version {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:18:22 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-e9bfef58-f895-42b7-bc9c-772ebab67bc8 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] 'nvme version' failed. Not Retrying. {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:473}}
Nov 29 03:18:22 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:a5b104bd-399e-4445-a186-0d4275ae271a -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.031s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:18:22 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[7c1f7409-38a7-4f32-8e99-eaf6f4961c84]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:18:22 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.nvmeof [None req-e9bfef58-f895-42b7-bc9c-772ebab67bc8 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] nvme not present on system {{(pid=75234) nvme_present /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/nvmeof.py:756}}
Nov 29 03:18:22 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-9bd15904-212d-458a-a01a-a6fdf8021a15 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:18:22 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:18:22 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:18:22 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:473}}
Nov 29 03:18:22 user nova-compute[75234]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Nov 29 03:18:22 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[1afffbbe-4ae6-4978-8bd6-abf9b9627a0e]: (4, '') {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:18:22 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.lightos [None req-e9bfef58-f895-42b7-bc9c-772ebab67bc8 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=75234) find_dsc /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/lightos.py:98}}
Nov 29 03:18:22 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.lightos [None req-e9bfef58-f895-42b7-bc9c-772ebab67bc8 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=75234) get_connector_properties /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/lightos.py:76}}
Nov 29 03:18:22 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.lightos [None req-e9bfef58-f895-42b7-bc9c-772ebab67bc8 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] LIGHTOS: no hostnqn found. {{(pid=75234) get_connector_properties /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/lightos.py:84}}
Nov 29 03:18:22 user nova-compute[75234]: DEBUG os_brick.utils [None req-e9bfef58-f895-42b7-bc9c-772ebab67bc8 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] <== get_connector_properties: return (180ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:dd1368c5ba2c', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:203}}
Nov 29 03:18:22 user nova-compute[75234]: DEBUG nova.virt.block_device [None req-e9bfef58-f895-42b7-bc9c-772ebab67bc8 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] [instance: aa190a9f-ba53-43fb-8bae-2c429d0d1228] Updating existing volume attachment record: dd549176-ab97-4d41-a3e2-1466a20e8e4d {{(pid=75234) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Nov 29 03:18:22 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.033s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:18:22 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[43fdd855-726e-4c15-9683-cf593cff7d98]: (4, ('tcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6c2e1179-dd3c-41c4-9594-bf4a16752c67 (non-flash)\ntcp: [29] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bf57b967-7fad-4d76-abb2-319fe88f8ef9 (non-flash)\ntcp: [31] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a5b104bd-399e-4445-a186-0d4275ae271a (non-flash)\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:18:22 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-9bd15904-212d-458a-a01a-a6fdf8021a15 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6c2e1179-dd3c-41c4-9594-bf4a16752c67 (non-flash)
Nov 29 03:18:22 user nova-compute[75234]: tcp: [29] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bf57b967-7fad-4d76-abb2-319fe88f8ef9 (non-flash)
Nov 29 03:18:22 user nova-compute[75234]: tcp: [31] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a5b104bd-399e-4445-a186-0d4275ae271a (non-flash)
Nov 29 03:18:22 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:18:22 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-9bd15904-212d-458a-a01a-a6fdf8021a15 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] iscsi session list stdout=tcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6c2e1179-dd3c-41c4-9594-bf4a16752c67 (non-flash)
Nov 29 03:18:22 user nova-compute[75234]: tcp: [29] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bf57b967-7fad-4d76-abb2-319fe88f8ef9 (non-flash)
Nov 29 03:18:22 user nova-compute[75234]: tcp: [31] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a5b104bd-399e-4445-a186-0d4275ae271a (non-flash)
Nov 29 03:18:22 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsi_session /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Nov 29 03:18:22 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-9bd15904-212d-458a-a01a-a6fdf8021a15 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:a5b104bd-399e-4445-a186-0d4275ae271a" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.402s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:87}}
Nov 29 03:18:22 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-9bd15904-212d-458a-a01a-a6fdf8021a15 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Connected to 172.16.0.220:3260 {{(pid=75234) _connect_vol /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:644}}
Nov 29 03:18:22 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-9bd15904-212d-458a-a01a-a6fdf8021a15 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] HCTL ('33', '-', '-', 0) found on session 31 with lun 0 {{(pid=75234) get_hctl /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:770}}
Nov 29 03:18:22 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-9bd15904-212d-458a-a01a-a6fdf8021a15 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Scanning host 33 c: -, t: -, l: 0) {{(pid=75234) scan_iscsi /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:795}}
Nov 29 03:18:22 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host33/scan {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:18:22 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host33/scan" returned: 0 in 0.016s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:18:22 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[b9e74e5a-d83e-4443-8b5b-249285939b3e]: (4, ('- - 0', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:18:22 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-9bd15904-212d-458a-a01a-a6fdf8021a15 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Searching for a device in session 31 and hctl ['33', '*', '*', 0] yield: None {{(pid=75234) device_name_by_hctl /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:789}}
Nov 29 03:18:22 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-751217d7-e3a0-4877-b867-15cec419ab79 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Waiting for libvirt event about the detach of device vdb with device alias virtio-disk1 from instance ec7cf7cd-8e48-4bb3-a969-e14db705dc09 is timed out.
Nov 29 03:18:22 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-751217d7-e3a0-4877-b867-15cec419ab79 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Successfully detached device vdb from instance ec7cf7cd-8e48-4bb3-a969-e14db705dc09 from the live domain config.
Nov 29 03:18:22 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-751217d7-e3a0-4877-b867-15cec419ab79 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] [instance: ec7cf7cd-8e48-4bb3-a969-e14db705dc09] calling os-brick to detach iSCSI Volume {{(pid=75234) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Nov 29 03:18:22 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-751217d7-e3a0-4877-b867-15cec419ab79 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f8a5050c9a0>, {'target_iqn': 'iqn.2010-10.org.openstack:6c2e1179-dd3c-41c4-9594-bf4a16752c67', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'shDupLmi', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdc'}, None), 'kwargs': False}}" {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:176}}
Nov 29 03:18:22 user nova-compute[75234]: WARNING os_brick.initiator.connectors.base [None req-751217d7-e3a0-4877-b867-15cec419ab79 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Nov 29 03:18:22 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-751217d7-e3a0-4877-b867-15cec419ab79 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:68}}
Nov 29 03:18:23 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:18:23 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-9bd15904-212d-458a-a01a-a6fdf8021a15 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Searching for a device in session 31 and hctl ['33', '*', '*', 0] yield: sdb {{(pid=75234) device_name_by_hctl /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:789}}
Nov 29 03:18:23 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-9bd15904-212d-458a-a01a-a6fdf8021a15 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Connected to sdb using {'target_iqn': 'iqn.2010-10.org.openstack:a5b104bd-399e-4445-a186-0d4275ae271a', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'XxdoDMRn', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False} {{(pid=75234) _connect_vol /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:672}}
Nov 29 03:18:23 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-9bd15904-212d-458a-a01a-a6fdf8021a15 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.431s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:87}}
Nov 29 03:18:23 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-9bd15904-212d-458a-a01a-a6fdf8021a15 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] <== connect_volume: return (1433ms) {'type': 'block', 'scsi_wwn': '23236343064356237', 'path': '/dev/sdb'} {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:203}}
Nov 29 03:18:23 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-9bd15904-212d-458a-a01a-a6fdf8021a15 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] [instance: 1c8b7559-29ab-4982-8c75-ea2a0f0101cb] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '23236343064356237', 'path': '/dev/sdb'} {{(pid=75234) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Nov 29 03:18:23 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-4f83f151-3776-4aa8-b91c-a6dd77c94abe tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 1.143s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:73}}
Nov 29 03:18:23 user nova-compute[75234]: WARNING os_brick.initiator.connectors.base [None req-4f83f151-3776-4aa8-b91c-a6dd77c94abe tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Nov 29 03:18:23 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-4f83f151-3776-4aa8-b91c-a6dd77c94abe tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:cf6ab229-533a-424e-bd67-1d54d50ad4be" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:68}}
Nov 29 03:18:23 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-4f83f151-3776-4aa8-b91c-a6dd77c94abe tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:cf6ab229-533a-424e-bd67-1d54d50ad4be" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:73}}
Nov 29 03:18:23 user nova-compute[75234]: INFO os_brick.initiator.connectors.iscsi [None req-4f83f151-3776-4aa8-b91c-a6dd77c94abe tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Nov 29 03:18:23 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:cf6ab229-533a-424e-bd67-1d54d50ad4be -p 172.16.0.220:3260 {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:18:23 user nova-compute[75234]: DEBUG nova.objects.instance [None req-9bd15904-212d-458a-a01a-a6fdf8021a15 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Lazy-loading 'flavor' on Instance uuid 1c8b7559-29ab-4982-8c75-ea2a0f0101cb {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:18:23 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:cf6ab229-533a-424e-bd67-1d54d50ad4be -p 172.16.0.220:3260" returned: 21 in 0.020s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:18:23 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[22cc5a84-193d-4b53-be52-081432dc0e23]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:18:23 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-4f83f151-3776-4aa8-b91c-a6dd77c94abe tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Nov 29 03:18:23 user nova-compute[75234]:  {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:18:23 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:cf6ab229-533a-424e-bd67-1d54d50ad4be -p 172.16.0.220:3260 --interface default --op new {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:18:23 user nova-compute[75234]: DEBUG nova.virt.libvirt.guest [None req-9bd15904-212d-458a-a01a-a6fdf8021a15 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] attach device xml: <disk type="block" device="disk">
Nov 29 03:18:23 user nova-compute[75234]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Nov 29 03:18:23 user nova-compute[75234]:   <source dev="/dev/sdb"/>
Nov 29 03:18:23 user nova-compute[75234]:   <target dev="vdb" bus="virtio"/>
Nov 29 03:18:23 user nova-compute[75234]:   <serial>a5b104bd-399e-4445-a186-0d4275ae271a</serial>
Nov 29 03:18:23 user nova-compute[75234]: </disk>
Nov 29 03:18:23 user nova-compute[75234]:  {{(pid=75234) attach_device /opt/stack/nova/nova/virt/libvirt/guest.py:339}}
Nov 29 03:18:23 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:cf6ab229-533a-424e-bd67-1d54d50ad4be -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.026s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:18:23 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[6f63e857-1864-4775-9d40-054cfb393607]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:cf6ab229-533a-424e-bd67-1d54d50ad4be] added\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:18:23 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-4f83f151-3776-4aa8-b91c-a6dd77c94abe tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:cf6ab229-533a-424e-bd67-1d54d50ad4be] added
Nov 29 03:18:23 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:18:23 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:cf6ab229-533a-424e-bd67-1d54d50ad4be -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:18:23 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:cf6ab229-533a-424e-bd67-1d54d50ad4be -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.022s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:18:23 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[d8aba33a-018d-4934-8499-458c3d7bf760]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:18:23 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-4f83f151-3776-4aa8-b91c-a6dd77c94abe tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:18:23 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:cf6ab229-533a-424e-bd67-1d54d50ad4be -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:18:23 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:cf6ab229-533a-424e-bd67-1d54d50ad4be -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.018s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:18:23 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[e5fe2416-8091-4be6-b1f4-29ea60adfe31]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:18:23 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-4f83f151-3776-4aa8-b91c-a6dd77c94abe tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:18:23 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:cf6ab229-533a-424e-bd67-1d54d50ad4be -p 172.16.0.220:3260 --op update -n node.session.auth.username -v wodfcngk {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:18:23 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:cf6ab229-533a-424e-bd67-1d54d50ad4be -p 172.16.0.220:3260 --op update -n node.session.auth.username -v wodfcngk" returned: 0 in 0.023s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:18:23 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[19741760-9157-41c7-b961-e31a9b492d9d]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:18:23 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-4f83f151-3776-4aa8-b91c-a6dd77c94abe tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'wodfcngk'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:18:23 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:cf6ab229-533a-424e-bd67-1d54d50ad4be -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:18:23 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:cf6ab229-533a-424e-bd67-1d54d50ad4be -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.018s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:18:23 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[32b8543f-ecc9-4a07-bc44-c5a21a8c8157]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:18:23 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-4f83f151-3776-4aa8-b91c-a6dd77c94abe tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:18:23 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:18:23 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.023s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:18:23 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[e2b6af9d-f22f-409c-9e08-73328f3b882c]: (4, ('tcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6c2e1179-dd3c-41c4-9594-bf4a16752c67 (non-flash)\ntcp: [29] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bf57b967-7fad-4d76-abb2-319fe88f8ef9 (non-flash)\ntcp: [31] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a5b104bd-399e-4445-a186-0d4275ae271a (non-flash)\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:18:23 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-4f83f151-3776-4aa8-b91c-a6dd77c94abe tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6c2e1179-dd3c-41c4-9594-bf4a16752c67 (non-flash)
Nov 29 03:18:23 user nova-compute[75234]: tcp: [29] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bf57b967-7fad-4d76-abb2-319fe88f8ef9 (non-flash)
Nov 29 03:18:23 user nova-compute[75234]: tcp: [31] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a5b104bd-399e-4445-a186-0d4275ae271a (non-flash)
Nov 29 03:18:23 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:18:23 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-4f83f151-3776-4aa8-b91c-a6dd77c94abe tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] iscsi session list stdout=tcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6c2e1179-dd3c-41c4-9594-bf4a16752c67 (non-flash)
Nov 29 03:18:23 user nova-compute[75234]: tcp: [29] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bf57b967-7fad-4d76-abb2-319fe88f8ef9 (non-flash)
Nov 29 03:18:23 user nova-compute[75234]: tcp: [31] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a5b104bd-399e-4445-a186-0d4275ae271a (non-flash)
Nov 29 03:18:23 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsi_session /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Nov 29 03:18:23 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:cf6ab229-533a-424e-bd67-1d54d50ad4be -p 172.16.0.220:3260 --login {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:18:23 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:cf6ab229-533a-424e-bd67-1d54d50ad4be -p 172.16.0.220:3260 --login" returned: 0 in 0.084s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:18:23 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[6d5b3d5b-cd6e-4491-8e02-e4edbbce94cc]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:cf6ab229-533a-424e-bd67-1d54d50ad4be, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:cf6ab229-533a-424e-bd67-1d54d50ad4be, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:18:23 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-4f83f151-3776-4aa8-b91c-a6dd77c94abe tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:cf6ab229-533a-424e-bd67-1d54d50ad4be, portal: 172.16.0.220,3260]
Nov 29 03:18:23 user nova-compute[75234]: Login to [iface: default, target: iqn.2010-10.org.openstack:cf6ab229-533a-424e-bd67-1d54d50ad4be, portal: 172.16.0.220,3260] successful.
Nov 29 03:18:23 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:18:23 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:cf6ab229-533a-424e-bd67-1d54d50ad4be -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:18:23 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:cf6ab229-533a-424e-bd67-1d54d50ad4be -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.024s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:18:23 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[bd9849f1-8876-4b09-8c8c-d8d2b0830f7a]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:18:23 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-4f83f151-3776-4aa8-b91c-a6dd77c94abe tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:18:23 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:18:23 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.024s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:18:23 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[1c1be90c-30bb-4418-b2ac-1a1ba093b9b2]: (4, ('tcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6c2e1179-dd3c-41c4-9594-bf4a16752c67 (non-flash)\ntcp: [29] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bf57b967-7fad-4d76-abb2-319fe88f8ef9 (non-flash)\ntcp: [31] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a5b104bd-399e-4445-a186-0d4275ae271a (non-flash)\ntcp: [32] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:cf6ab229-533a-424e-bd67-1d54d50ad4be (non-flash)\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:18:23 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-4f83f151-3776-4aa8-b91c-a6dd77c94abe tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6c2e1179-dd3c-41c4-9594-bf4a16752c67 (non-flash)
Nov 29 03:18:23 user nova-compute[75234]: tcp: [29] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bf57b967-7fad-4d76-abb2-319fe88f8ef9 (non-flash)
Nov 29 03:18:23 user nova-compute[75234]: tcp: [31] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a5b104bd-399e-4445-a186-0d4275ae271a (non-flash)
Nov 29 03:18:23 user nova-compute[75234]: tcp: [32] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:cf6ab229-533a-424e-bd67-1d54d50ad4be (non-flash)
Nov 29 03:18:23 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:18:23 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-4f83f151-3776-4aa8-b91c-a6dd77c94abe tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] iscsi session list stdout=tcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6c2e1179-dd3c-41c4-9594-bf4a16752c67 (non-flash)
Nov 29 03:18:23 user nova-compute[75234]: tcp: [29] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bf57b967-7fad-4d76-abb2-319fe88f8ef9 (non-flash)
Nov 29 03:18:23 user nova-compute[75234]: tcp: [31] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a5b104bd-399e-4445-a186-0d4275ae271a (non-flash)
Nov 29 03:18:23 user nova-compute[75234]: tcp: [32] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:cf6ab229-533a-424e-bd67-1d54d50ad4be (non-flash)
Nov 29 03:18:23 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsi_session /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Nov 29 03:18:23 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-4f83f151-3776-4aa8-b91c-a6dd77c94abe tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:cf6ab229-533a-424e-bd67-1d54d50ad4be" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.326s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:87}}
Nov 29 03:18:23 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-4f83f151-3776-4aa8-b91c-a6dd77c94abe tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Connected to 172.16.0.220:3260 {{(pid=75234) _connect_vol /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:644}}
Nov 29 03:18:23 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-4f83f151-3776-4aa8-b91c-a6dd77c94abe tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] HCTL ('36', '-', '-', 0) found on session 32 with lun 0 {{(pid=75234) get_hctl /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:770}}
Nov 29 03:18:23 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-4f83f151-3776-4aa8-b91c-a6dd77c94abe tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Scanning host 36 c: -, t: -, l: 0) {{(pid=75234) scan_iscsi /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:795}}
Nov 29 03:18:23 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host36/scan {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:18:23 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host36/scan" returned: 0 in 0.015s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:18:23 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[5e771241-9604-46c3-9bd9-720342ea4a07]: (4, ('- - 0', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:18:23 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-4f83f151-3776-4aa8-b91c-a6dd77c94abe tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Searching for a device in session 32 and hctl ['36', '*', '*', 0] yield: None {{(pid=75234) device_name_by_hctl /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:789}}
Nov 29 03:18:23 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-9bd15904-212d-458a-a01a-a6fdf8021a15 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] No BDM found with device name vda, not building metadata. {{(pid=75234) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12089}}
Nov 29 03:18:23 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-9bd15904-212d-458a-a01a-a6fdf8021a15 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] No BDM found with device name vdb, not building metadata. {{(pid=75234) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12089}}
Nov 29 03:18:23 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-9bd15904-212d-458a-a01a-a6fdf8021a15 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] No VIF found with MAC fa:16:3e:79:d1:ca, not building metadata {{(pid=75234) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Nov 29 03:18:24 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-9bd15904-212d-458a-a01a-a6fdf8021a15 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Lock "1c8b7559-29ab-4982-8c75-ea2a0f0101cb" "released" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: held 27.651s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:18:24 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-4f83f151-3776-4aa8-b91c-a6dd77c94abe tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Searching for a device in session 32 and hctl ['36', '*', '*', 0] yield: sde {{(pid=75234) device_name_by_hctl /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:789}}
Nov 29 03:18:24 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-4f83f151-3776-4aa8-b91c-a6dd77c94abe tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Connected to sde using {'target_iqn': 'iqn.2010-10.org.openstack:cf6ab229-533a-424e-bd67-1d54d50ad4be', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'wodfcngk', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False} {{(pid=75234) _connect_vol /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:672}}
Nov 29 03:18:24 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-4f83f151-3776-4aa8-b91c-a6dd77c94abe tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.356s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:87}}
Nov 29 03:18:24 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-4f83f151-3776-4aa8-b91c-a6dd77c94abe tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] <== connect_volume: return (2499ms) {'type': 'block', 'scsi_wwn': '23365386266373437', 'path': '/dev/sde'} {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:203}}
Nov 29 03:18:24 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-4f83f151-3776-4aa8-b91c-a6dd77c94abe tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] [instance: af699bc2-9daa-4b5a-9b4a-ad92fe857d70] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '23365386266373437', 'path': '/dev/sde'} {{(pid=75234) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Nov 29 03:18:24 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-751217d7-e3a0-4877-b867-15cec419ab79 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 2.185s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:73}}
Nov 29 03:18:24 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:18:24 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-4f83f151-3776-4aa8-b91c-a6dd77c94abe tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-11-29T03:17:51Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-ServerActionsTestOtherA-server-1736967407',display_name='tempest-ServerActionsTestOtherA-server-1736967407',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-serveractionstestothera-server-1736967407',id=31,image_ref='',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBGysIZGz8py0LojrFdRCkkIN/DpphiAjtTxjJQY7OVo0ZaKHLKAtSriy5/opnav7HzQyT3MeG6wW+2nQfBNyK0gW9HDU5His6qvZYEbhyEyMiaUzdXksC9FbE2bauudLhQ==',key_name='tempest-keypair-1767034584',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='c1cb3d1f12f242a785a688d59dbe3cd6',ramdisk_id='',reservation_id='r-p258pz1s',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-ServerActionsTestOtherA-1495067208',owner_user_name='tempest-ServerActionsTestOtherA-1495067208-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-11-29T03:17:53Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='d611826f3be947c0947b99c763c26d44',uuid=af699bc2-9daa-4b5a-9b4a-ad92fe857d70,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "66dcb0f9-3180-42f7-851d-19b0c4f4119c", "address": "fa:16:3e:dd:9e:50", "network": {"id": "1f946e69-dbec-429c-83f8-a186bad4f2cc", "bridge": "br-int", "label": "tempest-ServerActionsTestOtherA-486369423-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "c1cb3d1f12f242a785a688d59dbe3cd6", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap66dcb0f9-31", "ovs_interfaceid": "66dcb0f9-3180-42f7-851d-19b0c4f4119c", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=75234) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Nov 29 03:18:24 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-4f83f151-3776-4aa8-b91c-a6dd77c94abe tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Converting VIF {"id": "66dcb0f9-3180-42f7-851d-19b0c4f4119c", "address": "fa:16:3e:dd:9e:50", "network": {"id": "1f946e69-dbec-429c-83f8-a186bad4f2cc", "bridge": "br-int", "label": "tempest-ServerActionsTestOtherA-486369423-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "c1cb3d1f12f242a785a688d59dbe3cd6", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap66dcb0f9-31", "ovs_interfaceid": "66dcb0f9-3180-42f7-851d-19b0c4f4119c", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:18:24 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-4f83f151-3776-4aa8-b91c-a6dd77c94abe tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:dd:9e:50,bridge_name='br-int',has_traffic_filtering=True,id=66dcb0f9-3180-42f7-851d-19b0c4f4119c,network=Network(1f946e69-dbec-429c-83f8-a186bad4f2cc),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap66dcb0f9-31') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:18:24 user nova-compute[75234]: DEBUG nova.objects.instance [None req-4f83f151-3776-4aa8-b91c-a6dd77c94abe tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Lazy-loading 'pci_devices' on Instance uuid af699bc2-9daa-4b5a-9b4a-ad92fe857d70 {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:18:24 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.023s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:18:24 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[55d23a4b-3d05-4df5-b1ec-c1efb3e7134b]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:6c2e1179-dd3c-41c4-9594-bf4a16752c67\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:cf6ab229-533a-424e-bd67-1d54d50ad4be\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:bf57b967-7fad-4d76-abb2-319fe88f8ef9\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:a5b104bd-399e-4445-a186-0d4275ae271a\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:18:24 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-751217d7-e3a0-4877-b867-15cec419ab79 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS:
Nov 29 03:18:24 user nova-compute[75234]: No targets found.
Nov 29 03:18:24 user nova-compute[75234]: iSNS:
Nov 29 03:18:24 user nova-compute[75234]: No targets found.
Nov 29 03:18:24 user nova-compute[75234]: STATIC:
Nov 29 03:18:24 user nova-compute[75234]: Target: iqn.2010-10.org.openstack:6c2e1179-dd3c-41c4-9594-bf4a16752c67
Nov 29 03:18:24 user nova-compute[75234]:         Portal: 172.16.0.220:3260,-1
Nov 29 03:18:24 user nova-compute[75234]:                 Iface Name: default
Nov 29 03:18:24 user nova-compute[75234]: Target: iqn.2010-10.org.openstack:cf6ab229-533a-424e-bd67-1d54d50ad4be
Nov 29 03:18:24 user nova-compute[75234]:         Portal: 172.16.0.220:3260,-1
Nov 29 03:18:24 user nova-compute[75234]:                 Iface Name: default
Nov 29 03:18:24 user nova-compute[75234]: Target: iqn.2010-10.org.openstack:bf57b967-7fad-4d76-abb2-319fe88f8ef9
Nov 29 03:18:24 user nova-compute[75234]:         Portal: 172.16.0.220:3260,-1
Nov 29 03:18:24 user nova-compute[75234]:                 Iface Name: default
Nov 29 03:18:24 user nova-compute[75234]: Target: iqn.2010-10.org.openstack:a5b104bd-399e-4445-a186-0d4275ae271a
Nov 29 03:18:24 user nova-compute[75234]:         Portal: 172.16.0.220:3260,-1
Nov 29 03:18:24 user nova-compute[75234]:                 Iface Name: default
Nov 29 03:18:24 user nova-compute[75234]: FIRMWARE:
Nov 29 03:18:24 user nova-compute[75234]: No targets found.
Nov 29 03:18:24 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:18:24 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-751217d7-e3a0-4877-b867-15cec419ab79 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Regex to get portals from discoverydb: ^SENDTARGETS:
Nov 29 03:18:24 user nova-compute[75234]: .*?^DiscoveryAddress: 172.16.0.220,3260.*?
Nov 29 03:18:24 user nova-compute[75234]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=75234) _get_discoverydb_portals /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:382}}
Nov 29 03:18:24 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-751217d7-e3a0-4877-b867-15cec419ab79 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:6c2e1179-dd3c-41c4-9594-bf4a16752c67', 0)] {{(pid=75234) _get_connection_devices /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:830}}
Nov 29 03:18:24 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:18:24 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-4f83f151-3776-4aa8-b91c-a6dd77c94abe tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] [instance: af699bc2-9daa-4b5a-9b4a-ad92fe857d70] End _get_guest_xml xml=<domain type="kvm">
Nov 29 03:18:24 user nova-compute[75234]:   <uuid>af699bc2-9daa-4b5a-9b4a-ad92fe857d70</uuid>
Nov 29 03:18:24 user nova-compute[75234]:   <name>instance-0000001f</name>
Nov 29 03:18:24 user nova-compute[75234]:   <memory>524288</memory>
Nov 29 03:18:24 user nova-compute[75234]:   <vcpu>1</vcpu>
Nov 29 03:18:24 user nova-compute[75234]:   <metadata>
Nov 29 03:18:24 user nova-compute[75234]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Nov 29 03:18:24 user nova-compute[75234]:       <nova:package version="0.0.0"/>
Nov 29 03:18:24 user nova-compute[75234]:       <nova:name>tempest-ServerActionsTestOtherA-server-1736967407</nova:name>
Nov 29 03:18:24 user nova-compute[75234]:       <nova:creationTime>2023-11-29 03:18:22</nova:creationTime>
Nov 29 03:18:24 user nova-compute[75234]:       <nova:flavor name="m1.tiny">
Nov 29 03:18:24 user nova-compute[75234]:         <nova:memory>512</nova:memory>
Nov 29 03:18:24 user nova-compute[75234]:         <nova:disk>1</nova:disk>
Nov 29 03:18:24 user nova-compute[75234]:         <nova:swap>0</nova:swap>
Nov 29 03:18:24 user nova-compute[75234]:         <nova:ephemeral>0</nova:ephemeral>
Nov 29 03:18:24 user nova-compute[75234]:         <nova:vcpus>1</nova:vcpus>
Nov 29 03:18:24 user nova-compute[75234]:       </nova:flavor>
Nov 29 03:18:24 user nova-compute[75234]:       <nova:owner>
Nov 29 03:18:24 user nova-compute[75234]:         <nova:user uuid="d611826f3be947c0947b99c763c26d44">tempest-ServerActionsTestOtherA-1495067208-project-member</nova:user>
Nov 29 03:18:24 user nova-compute[75234]:         <nova:project uuid="c1cb3d1f12f242a785a688d59dbe3cd6">tempest-ServerActionsTestOtherA-1495067208</nova:project>
Nov 29 03:18:24 user nova-compute[75234]:       </nova:owner>
Nov 29 03:18:24 user nova-compute[75234]:       <nova:ports>
Nov 29 03:18:24 user nova-compute[75234]:         <nova:port uuid="66dcb0f9-3180-42f7-851d-19b0c4f4119c">
Nov 29 03:18:24 user nova-compute[75234]:           <nova:ip type="fixed" address="10.0.0.14" ipVersion="4"/>
Nov 29 03:18:24 user nova-compute[75234]:         </nova:port>
Nov 29 03:18:24 user nova-compute[75234]:       </nova:ports>
Nov 29 03:18:24 user nova-compute[75234]:     </nova:instance>
Nov 29 03:18:24 user nova-compute[75234]:   </metadata>
Nov 29 03:18:24 user nova-compute[75234]:   <sysinfo type="smbios">
Nov 29 03:18:24 user nova-compute[75234]:     <system>
Nov 29 03:18:24 user nova-compute[75234]:       <entry name="manufacturer">OpenStack Foundation</entry>
Nov 29 03:18:24 user nova-compute[75234]:       <entry name="product">OpenStack Nova</entry>
Nov 29 03:18:24 user nova-compute[75234]:       <entry name="version">0.0.0</entry>
Nov 29 03:18:24 user nova-compute[75234]:       <entry name="serial">af699bc2-9daa-4b5a-9b4a-ad92fe857d70</entry>
Nov 29 03:18:24 user nova-compute[75234]:       <entry name="uuid">af699bc2-9daa-4b5a-9b4a-ad92fe857d70</entry>
Nov 29 03:18:24 user nova-compute[75234]:       <entry name="family">Virtual Machine</entry>
Nov 29 03:18:24 user nova-compute[75234]:     </system>
Nov 29 03:18:24 user nova-compute[75234]:   </sysinfo>
Nov 29 03:18:24 user nova-compute[75234]:   <os>
Nov 29 03:18:24 user nova-compute[75234]:     <type arch="x86_64" machine="pc">hvm</type>
Nov 29 03:18:24 user nova-compute[75234]:     <boot dev="hd"/>
Nov 29 03:18:24 user nova-compute[75234]:     <smbios mode="sysinfo"/>
Nov 29 03:18:24 user nova-compute[75234]:   </os>
Nov 29 03:18:24 user nova-compute[75234]:   <features>
Nov 29 03:18:24 user nova-compute[75234]:     <acpi/>
Nov 29 03:18:24 user nova-compute[75234]:     <apic/>
Nov 29 03:18:24 user nova-compute[75234]:     <vmcoreinfo/>
Nov 29 03:18:24 user nova-compute[75234]:   </features>
Nov 29 03:18:24 user nova-compute[75234]:   <clock offset="utc">
Nov 29 03:18:24 user nova-compute[75234]:     <timer name="pit" tickpolicy="delay"/>
Nov 29 03:18:24 user nova-compute[75234]:     <timer name="rtc" tickpolicy="catchup"/>
Nov 29 03:18:24 user nova-compute[75234]:     <timer name="hpet" present="no"/>
Nov 29 03:18:24 user nova-compute[75234]:   </clock>
Nov 29 03:18:24 user nova-compute[75234]:   <cpu mode="custom" match="exact">
Nov 29 03:18:24 user nova-compute[75234]:     <model>Nehalem</model>
Nov 29 03:18:24 user nova-compute[75234]:     <topology sockets="1" cores="1" threads="1"/>
Nov 29 03:18:24 user nova-compute[75234]:   </cpu>
Nov 29 03:18:24 user nova-compute[75234]:   <devices>
Nov 29 03:18:24 user nova-compute[75234]:     <disk type="block" device="disk">
Nov 29 03:18:24 user nova-compute[75234]:       <driver name="qemu" type="raw" cache="none" io="native"/>
Nov 29 03:18:24 user nova-compute[75234]:       <source dev="/dev/sde"/>
Nov 29 03:18:24 user nova-compute[75234]:       <target dev="vda" bus="virtio"/>
Nov 29 03:18:24 user nova-compute[75234]:       <serial>cf6ab229-533a-424e-bd67-1d54d50ad4be</serial>
Nov 29 03:18:24 user nova-compute[75234]:     </disk>
Nov 29 03:18:24 user nova-compute[75234]:     <interface type="ethernet">
Nov 29 03:18:24 user nova-compute[75234]:       <mac address="fa:16:3e:dd:9e:50"/>
Nov 29 03:18:24 user nova-compute[75234]:       <model type="virtio"/>
Nov 29 03:18:24 user nova-compute[75234]:       <mtu size="1442"/>
Nov 29 03:18:24 user nova-compute[75234]:       <target dev="tap66dcb0f9-31"/>
Nov 29 03:18:24 user nova-compute[75234]:     </interface>
Nov 29 03:18:24 user nova-compute[75234]:     <serial type="pty">
Nov 29 03:18:24 user nova-compute[75234]:       <log file="/opt/stack/data/nova/instances/af699bc2-9daa-4b5a-9b4a-ad92fe857d70/console.log" append="off"/>
Nov 29 03:18:24 user nova-compute[75234]:     </serial>
Nov 29 03:18:24 user nova-compute[75234]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Nov 29 03:18:24 user nova-compute[75234]:     <video>
Nov 29 03:18:24 user nova-compute[75234]:       <model type="virtio"/>
Nov 29 03:18:24 user nova-compute[75234]:     </video>
Nov 29 03:18:24 user nova-compute[75234]:     <rng model="virtio">
Nov 29 03:18:24 user nova-compute[75234]:       <backend model="random">/dev/urandom</backend>
Nov 29 03:18:24 user nova-compute[75234]:     </rng>
Nov 29 03:18:24 user nova-compute[75234]:     <controller type="usb" index="0" model="none"/>
Nov 29 03:18:24 user nova-compute[75234]:     <memballoon model="virtio">
Nov 29 03:18:24 user nova-compute[75234]:       <stats period="10"/>
Nov 29 03:18:24 user nova-compute[75234]:     </memballoon>
Nov 29 03:18:24 user nova-compute[75234]:   </devices>
Nov 29 03:18:24 user nova-compute[75234]: </domain>
Nov 29 03:18:24 user nova-compute[75234]:  {{(pid=75234) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7568}}
Nov 29 03:18:24 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-4f83f151-3776-4aa8-b91c-a6dd77c94abe tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-11-29T03:17:51Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-ServerActionsTestOtherA-server-1736967407',display_name='tempest-ServerActionsTestOtherA-server-1736967407',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-serveractionstestothera-server-1736967407',id=31,image_ref='',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBGysIZGz8py0LojrFdRCkkIN/DpphiAjtTxjJQY7OVo0ZaKHLKAtSriy5/opnav7HzQyT3MeG6wW+2nQfBNyK0gW9HDU5His6qvZYEbhyEyMiaUzdXksC9FbE2bauudLhQ==',key_name='tempest-keypair-1767034584',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='c1cb3d1f12f242a785a688d59dbe3cd6',ramdisk_id='',reservation_id='r-p258pz1s',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-ServerActionsTestOtherA-1495067208',owner_user_name='tempest-ServerActionsTestOtherA-1495067208-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-11-29T03:17:53Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='d611826f3be947c0947b99c763c26d44',uuid=af699bc2-9daa-4b5a-9b4a-ad92fe857d70,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "66dcb0f9-3180-42f7-851d-19b0c4f4119c", "address": "fa:16:3e:dd:9e:50", "network": {"id": "1f946e69-dbec-429c-83f8-a186bad4f2cc", "bridge": "br-int", "label": "tempest-ServerActionsTestOtherA-486369423-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "c1cb3d1f12f242a785a688d59dbe3cd6", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap66dcb0f9-31", "ovs_interfaceid": "66dcb0f9-3180-42f7-851d-19b0c4f4119c", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Nov 29 03:18:24 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-4f83f151-3776-4aa8-b91c-a6dd77c94abe tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Converting VIF {"id": "66dcb0f9-3180-42f7-851d-19b0c4f4119c", "address": "fa:16:3e:dd:9e:50", "network": {"id": "1f946e69-dbec-429c-83f8-a186bad4f2cc", "bridge": "br-int", "label": "tempest-ServerActionsTestOtherA-486369423-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "c1cb3d1f12f242a785a688d59dbe3cd6", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap66dcb0f9-31", "ovs_interfaceid": "66dcb0f9-3180-42f7-851d-19b0c4f4119c", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:18:24 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-4f83f151-3776-4aa8-b91c-a6dd77c94abe tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:dd:9e:50,bridge_name='br-int',has_traffic_filtering=True,id=66dcb0f9-3180-42f7-851d-19b0c4f4119c,network=Network(1f946e69-dbec-429c-83f8-a186bad4f2cc),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap66dcb0f9-31') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:18:24 user nova-compute[75234]: DEBUG os_vif [None req-4f83f151-3776-4aa8-b91c-a6dd77c94abe tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:dd:9e:50,bridge_name='br-int',has_traffic_filtering=True,id=66dcb0f9-3180-42f7-851d-19b0c4f4119c,network=Network(1f946e69-dbec-429c-83f8-a186bad4f2cc),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap66dcb0f9-31') {{(pid=75234) plug /opt/stack/data/venv/lib/python3.10/site-packages/os_vif/__init__.py:76}}
Nov 29 03:18:24 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.023s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:18:24 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[41486295-0640-4c7c-9e89-f23ef2af38f5]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:6c2e1179-dd3c-41c4-9594-bf4a16752c67\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:a5b104bd-399e-4445-a186-0d4275ae271a\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:bf57b967-7fad-4d76-abb2-319fe88f8ef9\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:cf6ab229-533a-424e-bd67-1d54d50ad4be\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:18:24 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:18:24 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:18:24 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Nov 29 03:18:24 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:18:24 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbCreateCommand(_result=None, table=QoS, columns={'type': 'linux-noop', 'external_ids': {'id': '7989ed51-8d0a-5fad-bb2f-ea6f94ff7e01', '_type': 'linux-noop'}}, row=False) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:18:24 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:18:24 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:18:24 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:18:24 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:18:24 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tap66dcb0f9-31, may_exist=True, interface_attrs={}) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:18:24 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Port, record=tap66dcb0f9-31, col_values=(('qos', UUID('8822804c-faad-4793-b442-c527514dab9c')),), if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:18:24 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DbSetCommand(_result=None, table=Interface, record=tap66dcb0f9-31, col_values=(('external_ids', {'iface-id': '66dcb0f9-3180-42f7-851d-19b0c4f4119c', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:dd:9e:50', 'vm-uuid': 'af699bc2-9daa-4b5a-9b4a-ad92fe857d70'}),), if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:18:24 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:18:24 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:18:24 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:18:24 user nova-compute[75234]: INFO os_vif [None req-4f83f151-3776-4aa8-b91c-a6dd77c94abe tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:dd:9e:50,bridge_name='br-int',has_traffic_filtering=True,id=66dcb0f9-3180-42f7-851d-19b0c4f4119c,network=Network(1f946e69-dbec-429c-83f8-a186bad4f2cc),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap66dcb0f9-31')
Nov 29 03:18:24 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.028s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:18:24 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[a97fca76-ccae-45a3-975d-79fd2d00bdb7]: (4, ('tcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6c2e1179-dd3c-41c4-9594-bf4a16752c67 (non-flash)\ntcp: [29] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bf57b967-7fad-4d76-abb2-319fe88f8ef9 (non-flash)\ntcp: [31] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a5b104bd-399e-4445-a186-0d4275ae271a (non-flash)\ntcp: [32] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:cf6ab229-533a-424e-bd67-1d54d50ad4be (non-flash)\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:18:24 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-751217d7-e3a0-4877-b867-15cec419ab79 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6c2e1179-dd3c-41c4-9594-bf4a16752c67 (non-flash)
Nov 29 03:18:24 user nova-compute[75234]: tcp: [29] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bf57b967-7fad-4d76-abb2-319fe88f8ef9 (non-flash)
Nov 29 03:18:24 user nova-compute[75234]: tcp: [31] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a5b104bd-399e-4445-a186-0d4275ae271a (non-flash)
Nov 29 03:18:24 user nova-compute[75234]: tcp: [32] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:cf6ab229-533a-424e-bd67-1d54d50ad4be (non-flash)
Nov 29 03:18:24 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:18:24 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-751217d7-e3a0-4877-b867-15cec419ab79 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] iscsi session list stdout=tcp: [28] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6c2e1179-dd3c-41c4-9594-bf4a16752c67 (non-flash)
Nov 29 03:18:24 user nova-compute[75234]: tcp: [29] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bf57b967-7fad-4d76-abb2-319fe88f8ef9 (non-flash)
Nov 29 03:18:24 user nova-compute[75234]: tcp: [31] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a5b104bd-399e-4445-a186-0d4275ae271a (non-flash)
Nov 29 03:18:24 user nova-compute[75234]: tcp: [32] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:cf6ab229-533a-424e-bd67-1d54d50ad4be (non-flash)
Nov 29 03:18:24 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsi_session /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Nov 29 03:18:24 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-751217d7-e3a0-4877-b867-15cec419ab79 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7f8a506da560>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:6c2e1179-dd3c-41c4-9594-bf4a16752c67'): ({'sdc'}, set())}) {{(pid=75234) _get_connection_devices /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:863}}
Nov 29 03:18:24 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-751217d7-e3a0-4877-b867-15cec419ab79 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Removing single pathed devices sdc {{(pid=75234) remove_connection /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:374}}
Nov 29 03:18:24 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:18:25 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.021s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:18:25 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[aaaea1b3-3c36-479c-a89a-507552c09c71]: (4, ('path checker states:\nup                  4\n\npaths: 0\nbusy: False\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:18:25 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdc {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:18:25 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdc" returned: 0 in 0.021s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:18:25 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[93f565bb-844d-4458-8c15-fba18d95d56e]: (4, ('ok\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:18:25 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-751217d7-e3a0-4877-b867-15cec419ab79 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Flushing IO for device /dev/sdc {{(pid=75234) flush_device_io /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:434}}
Nov 29 03:18:25 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdc {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:18:25 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-4f83f151-3776-4aa8-b91c-a6dd77c94abe tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] No BDM found with device name vda, not building metadata. {{(pid=75234) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12089}}
Nov 29 03:18:25 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-4f83f151-3776-4aa8-b91c-a6dd77c94abe tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] No VIF found with MAC fa:16:3e:dd:9e:50, not building metadata {{(pid=75234) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Nov 29 03:18:25 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdc" returned: 0 in 0.018s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:18:25 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[23575814-5e09-4c56-b19f-b6e5484ef268]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:18:25 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-751217d7-e3a0-4877-b867-15cec419ab79 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Remove SCSI device /dev/sdc with /sys/block/sdc/device/delete {{(pid=75234) remove_scsi_device /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:148}}
Nov 29 03:18:25 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdc/device/delete {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:18:25 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdc/device/delete" returned: 0 in 0.050s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:18:25 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[b6752c59-1acc-457b-af5c-0d3b2a901638]: (4, ('1', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:18:25 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-751217d7-e3a0-4877-b867-15cec419ab79 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Checking to see if SCSI volumes sdc have been removed. {{(pid=75234) wait_for_volumes_removal /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:156}}
Nov 29 03:18:25 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-751217d7-e3a0-4877-b867-15cec419ab79 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] SCSI volumes sdc have been removed. {{(pid=75234) wait_for_volumes_removal /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:166}}
Nov 29 03:18:25 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-751217d7-e3a0-4877-b867-15cec419ab79 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:6c2e1179-dd3c-41c4-9594-bf4a16752c67')] {{(pid=75234) _disconnect_connection /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Nov 29 03:18:25 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:6c2e1179-dd3c-41c4-9594-bf4a16752c67 -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:18:25 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:6c2e1179-dd3c-41c4-9594-bf4a16752c67 -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.018s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:18:25 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[308f6f6e-50b2-4940-b039-c35c0f3d65a1]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:18:25 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-751217d7-e3a0-4877-b867-15cec419ab79 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:18:25 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:6c2e1179-dd3c-41c4-9594-bf4a16752c67 -p 172.16.0.220:3260 --logout {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:18:25 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:6c2e1179-dd3c-41c4-9594-bf4a16752c67 -p 172.16.0.220:3260 --logout" returned: 0 in 0.068s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:18:25 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[4134e6c3-0fa4-4c90-9a74-6581b6a4fa56]: (4, ('Logging out of session [sid: 28, target: iqn.2010-10.org.openstack:6c2e1179-dd3c-41c4-9594-bf4a16752c67, portal: 172.16.0.220,3260]\nLogout of [sid: 28, target: iqn.2010-10.org.openstack:6c2e1179-dd3c-41c4-9594-bf4a16752c67, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:18:25 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-751217d7-e3a0-4877-b867-15cec419ab79 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 28, target: iqn.2010-10.org.openstack:6c2e1179-dd3c-41c4-9594-bf4a16752c67, portal: 172.16.0.220,3260]
Nov 29 03:18:25 user nova-compute[75234]: Logout of [sid: 28, target: iqn.2010-10.org.openstack:6c2e1179-dd3c-41c4-9594-bf4a16752c67, portal: 172.16.0.220,3260] successful.
Nov 29 03:18:25 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:18:25 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:6c2e1179-dd3c-41c4-9594-bf4a16752c67 -p 172.16.0.220:3260 --op delete {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:18:25 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:6c2e1179-dd3c-41c4-9594-bf4a16752c67 -p 172.16.0.220:3260 --op delete" returned: 0 in 0.023s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:18:25 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[fe1fddda-ab80-4586-be11-8cc0c6d899ce]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:18:25 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-751217d7-e3a0-4877-b867-15cec419ab79 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:18:25 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-751217d7-e3a0-4877-b867-15cec419ab79 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.366s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:87}}
Nov 29 03:18:25 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-751217d7-e3a0-4877-b867-15cec419ab79 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] <== disconnect_volume: return (2550ms) None {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:203}}
Nov 29 03:18:25 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-751217d7-e3a0-4877-b867-15cec419ab79 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] [instance: ec7cf7cd-8e48-4bb3-a969-e14db705dc09] Disconnected iSCSI Volume {{(pid=75234) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:79}}
Nov 29 03:18:26 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:18:26 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:18:26 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:18:26 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:18:26 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:18:26 user nova-compute[75234]: DEBUG nova.compute.manager [req-61b0297c-6d10-40b9-8424-57cc960d7cd0 req-213c7807-4797-482f-b758-c70933ac8b66 service nova] [instance: af699bc2-9daa-4b5a-9b4a-ad92fe857d70] Received event network-vif-plugged-66dcb0f9-3180-42f7-851d-19b0c4f4119c {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:18:26 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-61b0297c-6d10-40b9-8424-57cc960d7cd0 req-213c7807-4797-482f-b758-c70933ac8b66 service nova] Acquiring lock "af699bc2-9daa-4b5a-9b4a-ad92fe857d70-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:18:26 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-61b0297c-6d10-40b9-8424-57cc960d7cd0 req-213c7807-4797-482f-b758-c70933ac8b66 service nova] Lock "af699bc2-9daa-4b5a-9b4a-ad92fe857d70-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:18:26 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-61b0297c-6d10-40b9-8424-57cc960d7cd0 req-213c7807-4797-482f-b758-c70933ac8b66 service nova] Lock "af699bc2-9daa-4b5a-9b4a-ad92fe857d70-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:18:26 user nova-compute[75234]: DEBUG nova.compute.manager [req-61b0297c-6d10-40b9-8424-57cc960d7cd0 req-213c7807-4797-482f-b758-c70933ac8b66 service nova] [instance: af699bc2-9daa-4b5a-9b4a-ad92fe857d70] No waiting events found dispatching network-vif-plugged-66dcb0f9-3180-42f7-851d-19b0c4f4119c {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:18:26 user nova-compute[75234]: WARNING nova.compute.manager [req-61b0297c-6d10-40b9-8424-57cc960d7cd0 req-213c7807-4797-482f-b758-c70933ac8b66 service nova] [instance: af699bc2-9daa-4b5a-9b4a-ad92fe857d70] Received unexpected event network-vif-plugged-66dcb0f9-3180-42f7-851d-19b0c4f4119c for instance with vm_state building and task_state spawning.
Nov 29 03:18:26 user nova-compute[75234]: DEBUG nova.compute.manager [req-f6135348-e29c-4f91-8ff5-75aadf7951ec req-71363fc9-e3e1-4a23-ab41-7fd926656e3c service nova] [instance: 99d6f85f-1b77-4390-8b61-908c98f584c0] Received event network-changed-b68d722f-9020-4da4-8cdb-e2312ef06df9 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:18:26 user nova-compute[75234]: DEBUG nova.compute.manager [req-f6135348-e29c-4f91-8ff5-75aadf7951ec req-71363fc9-e3e1-4a23-ab41-7fd926656e3c service nova] [instance: 99d6f85f-1b77-4390-8b61-908c98f584c0] Refreshing instance network info cache due to event network-changed-b68d722f-9020-4da4-8cdb-e2312ef06df9. {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11117}}
Nov 29 03:18:26 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-f6135348-e29c-4f91-8ff5-75aadf7951ec req-71363fc9-e3e1-4a23-ab41-7fd926656e3c service nova] Acquiring lock "refresh_cache-99d6f85f-1b77-4390-8b61-908c98f584c0" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:18:26 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-f6135348-e29c-4f91-8ff5-75aadf7951ec req-71363fc9-e3e1-4a23-ab41-7fd926656e3c service nova] Acquired lock "refresh_cache-99d6f85f-1b77-4390-8b61-908c98f584c0" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:18:26 user nova-compute[75234]: DEBUG nova.network.neutron [req-f6135348-e29c-4f91-8ff5-75aadf7951ec req-71363fc9-e3e1-4a23-ab41-7fd926656e3c service nova] [instance: 99d6f85f-1b77-4390-8b61-908c98f584c0] Refreshing network info cache for port b68d722f-9020-4da4-8cdb-e2312ef06df9 {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2006}}
Nov 29 03:18:26 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:18:27 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-1c44c857-3624-41cb-8724-98ef31810a22 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Acquiring lock "1c8b7559-29ab-4982-8c75-ea2a0f0101cb" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:18:27 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-1c44c857-3624-41cb-8724-98ef31810a22 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Lock "1c8b7559-29ab-4982-8c75-ea2a0f0101cb" acquired by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:18:27 user nova-compute[75234]: INFO nova.compute.manager [None req-1c44c857-3624-41cb-8724-98ef31810a22 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] [instance: 1c8b7559-29ab-4982-8c75-ea2a0f0101cb] Detaching volume a5b104bd-399e-4445-a186-0d4275ae271a
Nov 29 03:18:27 user nova-compute[75234]: INFO nova.virt.block_device [None req-1c44c857-3624-41cb-8724-98ef31810a22 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] [instance: 1c8b7559-29ab-4982-8c75-ea2a0f0101cb] Attempting to driver detach volume a5b104bd-399e-4445-a186-0d4275ae271a from mountpoint /dev/vdb
Nov 29 03:18:27 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-1c44c857-3624-41cb-8724-98ef31810a22 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Attempting to detach device vdb from instance 1c8b7559-29ab-4982-8c75-ea2a0f0101cb from the persistent domain config. {{(pid=75234) _detach_from_persistent /opt/stack/nova/nova/virt/libvirt/driver.py:2487}}
Nov 29 03:18:27 user nova-compute[75234]: DEBUG nova.virt.libvirt.guest [None req-1c44c857-3624-41cb-8724-98ef31810a22 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] detach device xml: <disk type="block" device="disk">
Nov 29 03:18:27 user nova-compute[75234]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Nov 29 03:18:27 user nova-compute[75234]:   <source dev="/dev/sdb"/>
Nov 29 03:18:27 user nova-compute[75234]:   <target dev="vdb" bus="virtio"/>
Nov 29 03:18:27 user nova-compute[75234]:   <serial>a5b104bd-399e-4445-a186-0d4275ae271a</serial>
Nov 29 03:18:27 user nova-compute[75234]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x07" function="0x0"/>
Nov 29 03:18:27 user nova-compute[75234]: </disk>
Nov 29 03:18:27 user nova-compute[75234]:  {{(pid=75234) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Nov 29 03:18:27 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-1c44c857-3624-41cb-8724-98ef31810a22 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Successfully detached device vdb from instance 1c8b7559-29ab-4982-8c75-ea2a0f0101cb from the persistent domain config.
Nov 29 03:18:27 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-1c44c857-3624-41cb-8724-98ef31810a22 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] (1/8): Attempting to detach device vdb with device alias virtio-disk1 from instance 1c8b7559-29ab-4982-8c75-ea2a0f0101cb from the live domain config. {{(pid=75234) _detach_from_live_with_retry /opt/stack/nova/nova/virt/libvirt/driver.py:2523}}
Nov 29 03:18:27 user nova-compute[75234]: DEBUG nova.virt.libvirt.guest [None req-1c44c857-3624-41cb-8724-98ef31810a22 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] detach device xml: <disk type="block" device="disk">
Nov 29 03:18:27 user nova-compute[75234]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Nov 29 03:18:27 user nova-compute[75234]:   <source dev="/dev/sdb"/>
Nov 29 03:18:27 user nova-compute[75234]:   <target dev="vdb" bus="virtio"/>
Nov 29 03:18:27 user nova-compute[75234]:   <serial>a5b104bd-399e-4445-a186-0d4275ae271a</serial>
Nov 29 03:18:27 user nova-compute[75234]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x07" function="0x0"/>
Nov 29 03:18:27 user nova-compute[75234]: </disk>
Nov 29 03:18:27 user nova-compute[75234]:  {{(pid=75234) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Nov 29 03:18:27 user nova-compute[75234]: DEBUG nova.network.neutron [req-f6135348-e29c-4f91-8ff5-75aadf7951ec req-71363fc9-e3e1-4a23-ab41-7fd926656e3c service nova] [instance: 99d6f85f-1b77-4390-8b61-908c98f584c0] Updated VIF entry in instance network info cache for port b68d722f-9020-4da4-8cdb-e2312ef06df9. {{(pid=75234) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3481}}
Nov 29 03:18:27 user nova-compute[75234]: DEBUG nova.network.neutron [req-f6135348-e29c-4f91-8ff5-75aadf7951ec req-71363fc9-e3e1-4a23-ab41-7fd926656e3c service nova] [instance: 99d6f85f-1b77-4390-8b61-908c98f584c0] Updating instance_info_cache with network_info: [{"id": "b68d722f-9020-4da4-8cdb-e2312ef06df9", "address": "fa:16:3e:1e:90:83", "network": {"id": "aad085f7-106b-4de6-93f5-f86bb66ec962", "bridge": "br-int", "label": "tempest-AttachVolumeShelveTestJSON-775984268-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.193", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "e17150005a9a449f9373b91e1c00083b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapb68d722f-90", "ovs_interfaceid": "b68d722f-9020-4da4-8cdb-e2312ef06df9", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:18:27 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-f6135348-e29c-4f91-8ff5-75aadf7951ec req-71363fc9-e3e1-4a23-ab41-7fd926656e3c service nova] Releasing lock "refresh_cache-99d6f85f-1b77-4390-8b61-908c98f584c0" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:18:27 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-1c44c857-3624-41cb-8724-98ef31810a22 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Start waiting for the detach event from libvirt for device vdb with device alias virtio-disk1 for instance 1c8b7559-29ab-4982-8c75-ea2a0f0101cb {{(pid=75234) _detach_from_live_and_wait_for_event /opt/stack/nova/nova/virt/libvirt/driver.py:2599}}
Nov 29 03:18:28 user nova-compute[75234]: DEBUG nova.compute.manager [None req-4f83f151-3776-4aa8-b91c-a6dd77c94abe tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] [instance: af699bc2-9daa-4b5a-9b4a-ad92fe857d70] Instance event wait completed in 0 seconds for  {{(pid=75234) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Nov 29 03:18:28 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-4f83f151-3776-4aa8-b91c-a6dd77c94abe tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] [instance: af699bc2-9daa-4b5a-9b4a-ad92fe857d70] Guest created on hypervisor {{(pid=75234) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4421}}
Nov 29 03:18:28 user nova-compute[75234]: INFO nova.virt.libvirt.driver [-] [instance: af699bc2-9daa-4b5a-9b4a-ad92fe857d70] Instance spawned successfully.
Nov 29 03:18:28 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-4f83f151-3776-4aa8-b91c-a6dd77c94abe tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] [instance: af699bc2-9daa-4b5a-9b4a-ad92fe857d70] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:904}}
Nov 29 03:18:28 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-4f83f151-3776-4aa8-b91c-a6dd77c94abe tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] [instance: af699bc2-9daa-4b5a-9b4a-ad92fe857d70] Found default for hw_cdrom_bus of ide {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:18:28 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-4f83f151-3776-4aa8-b91c-a6dd77c94abe tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] [instance: af699bc2-9daa-4b5a-9b4a-ad92fe857d70] Found default for hw_disk_bus of virtio {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:18:28 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-4f83f151-3776-4aa8-b91c-a6dd77c94abe tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] [instance: af699bc2-9daa-4b5a-9b4a-ad92fe857d70] Found default for hw_input_bus of None {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:18:28 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-4f83f151-3776-4aa8-b91c-a6dd77c94abe tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] [instance: af699bc2-9daa-4b5a-9b4a-ad92fe857d70] Found default for hw_pointer_model of None {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:18:28 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-4f83f151-3776-4aa8-b91c-a6dd77c94abe tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] [instance: af699bc2-9daa-4b5a-9b4a-ad92fe857d70] Found default for hw_video_model of virtio {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:18:28 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-4f83f151-3776-4aa8-b91c-a6dd77c94abe tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] [instance: af699bc2-9daa-4b5a-9b4a-ad92fe857d70] Found default for hw_vif_model of virtio {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:18:28 user nova-compute[75234]: DEBUG nova.compute.manager [req-b5ef440c-09f6-42a8-a7a9-ac31031fa0ef req-70fabefa-c4ba-4b11-bea4-77e77344d160 service nova] [instance: af699bc2-9daa-4b5a-9b4a-ad92fe857d70] Received event network-vif-plugged-66dcb0f9-3180-42f7-851d-19b0c4f4119c {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:18:28 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-b5ef440c-09f6-42a8-a7a9-ac31031fa0ef req-70fabefa-c4ba-4b11-bea4-77e77344d160 service nova] Acquiring lock "af699bc2-9daa-4b5a-9b4a-ad92fe857d70-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:18:28 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-b5ef440c-09f6-42a8-a7a9-ac31031fa0ef req-70fabefa-c4ba-4b11-bea4-77e77344d160 service nova] Lock "af699bc2-9daa-4b5a-9b4a-ad92fe857d70-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.002s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:18:28 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-b5ef440c-09f6-42a8-a7a9-ac31031fa0ef req-70fabefa-c4ba-4b11-bea4-77e77344d160 service nova] Lock "af699bc2-9daa-4b5a-9b4a-ad92fe857d70-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:18:28 user nova-compute[75234]: DEBUG nova.compute.manager [req-b5ef440c-09f6-42a8-a7a9-ac31031fa0ef req-70fabefa-c4ba-4b11-bea4-77e77344d160 service nova] [instance: af699bc2-9daa-4b5a-9b4a-ad92fe857d70] No waiting events found dispatching network-vif-plugged-66dcb0f9-3180-42f7-851d-19b0c4f4119c {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:18:28 user nova-compute[75234]: WARNING nova.compute.manager [req-b5ef440c-09f6-42a8-a7a9-ac31031fa0ef req-70fabefa-c4ba-4b11-bea4-77e77344d160 service nova] [instance: af699bc2-9daa-4b5a-9b4a-ad92fe857d70] Received unexpected event network-vif-plugged-66dcb0f9-3180-42f7-851d-19b0c4f4119c for instance with vm_state building and task_state spawning.
Nov 29 03:18:28 user nova-compute[75234]: INFO nova.compute.manager [None req-4f83f151-3776-4aa8-b91c-a6dd77c94abe tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] [instance: af699bc2-9daa-4b5a-9b4a-ad92fe857d70] Took 6.33 seconds to spawn the instance on the hypervisor.
Nov 29 03:18:28 user nova-compute[75234]: DEBUG nova.compute.manager [None req-4f83f151-3776-4aa8-b91c-a6dd77c94abe tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] [instance: af699bc2-9daa-4b5a-9b4a-ad92fe857d70] Checking state {{(pid=75234) _get_power_state /opt/stack/nova/nova/compute/manager.py:1782}}
Nov 29 03:18:28 user nova-compute[75234]: INFO nova.compute.manager [None req-4f83f151-3776-4aa8-b91c-a6dd77c94abe tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] [instance: af699bc2-9daa-4b5a-9b4a-ad92fe857d70] Took 36.37 seconds to build instance.
Nov 29 03:18:28 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-4f83f151-3776-4aa8-b91c-a6dd77c94abe tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Lock "af699bc2-9daa-4b5a-9b4a-ad92fe857d70" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 36.480s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:18:29 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:18:30 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-13301644-94d0-4a00-b5fd-4cb958706f6d tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Acquiring lock "99d6f85f-1b77-4390-8b61-908c98f584c0" by "nova.compute.manager.ComputeManager.shelve_instance.<locals>.do_shelve_instance" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:18:30 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-13301644-94d0-4a00-b5fd-4cb958706f6d tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Lock "99d6f85f-1b77-4390-8b61-908c98f584c0" acquired by "nova.compute.manager.ComputeManager.shelve_instance.<locals>.do_shelve_instance" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:18:30 user nova-compute[75234]: INFO nova.compute.manager [None req-13301644-94d0-4a00-b5fd-4cb958706f6d tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] [instance: 99d6f85f-1b77-4390-8b61-908c98f584c0] Shelving
Nov 29 03:18:30 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:18:31 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:18:31 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:18:31 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:18:31 user nova-compute[75234]: DEBUG nova.compute.manager [req-ee532a63-b7db-46a0-8557-437522547c47 req-a61517c1-9c46-42c7-9106-2ce9c7f34034 service nova] [instance: 99d6f85f-1b77-4390-8b61-908c98f584c0] Received event network-vif-unplugged-b68d722f-9020-4da4-8cdb-e2312ef06df9 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:18:31 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-ee532a63-b7db-46a0-8557-437522547c47 req-a61517c1-9c46-42c7-9106-2ce9c7f34034 service nova] Acquiring lock "99d6f85f-1b77-4390-8b61-908c98f584c0-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:18:31 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-ee532a63-b7db-46a0-8557-437522547c47 req-a61517c1-9c46-42c7-9106-2ce9c7f34034 service nova] Lock "99d6f85f-1b77-4390-8b61-908c98f584c0-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:18:31 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-ee532a63-b7db-46a0-8557-437522547c47 req-a61517c1-9c46-42c7-9106-2ce9c7f34034 service nova] Lock "99d6f85f-1b77-4390-8b61-908c98f584c0-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:18:31 user nova-compute[75234]: DEBUG nova.compute.manager [req-ee532a63-b7db-46a0-8557-437522547c47 req-a61517c1-9c46-42c7-9106-2ce9c7f34034 service nova] [instance: 99d6f85f-1b77-4390-8b61-908c98f584c0] No waiting events found dispatching network-vif-unplugged-b68d722f-9020-4da4-8cdb-e2312ef06df9 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:18:31 user nova-compute[75234]: WARNING nova.compute.manager [req-ee532a63-b7db-46a0-8557-437522547c47 req-a61517c1-9c46-42c7-9106-2ce9c7f34034 service nova] [instance: 99d6f85f-1b77-4390-8b61-908c98f584c0] Received unexpected event network-vif-unplugged-b68d722f-9020-4da4-8cdb-e2312ef06df9 for instance with vm_state active and task_state shelving.
Nov 29 03:18:31 user nova-compute[75234]: INFO nova.virt.libvirt.driver [-] [instance: 99d6f85f-1b77-4390-8b61-908c98f584c0] Instance destroyed successfully.
Nov 29 03:18:31 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-13301644-94d0-4a00-b5fd-4cb958706f6d tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] [instance: 99d6f85f-1b77-4390-8b61-908c98f584c0] Beginning cold snapshot process
Nov 29 03:18:31 user nova-compute[75234]: DEBUG nova.privsep.utils [None req-13301644-94d0-4a00-b5fd-4cb958706f6d tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Path '/opt/stack/data/nova/instances' supports direct I/O {{(pid=75234) supports_direct_io /opt/stack/nova/nova/privsep/utils.py:63}}
Nov 29 03:18:31 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-13301644-94d0-4a00-b5fd-4cb958706f6d tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Running cmd (subprocess): qemu-img convert -t none -O qcow2 -f qcow2 /opt/stack/data/nova/instances/99d6f85f-1b77-4390-8b61-908c98f584c0/disk /opt/stack/data/nova/instances/snapshots/tmpbfgy6oq9/7826e0db5ddb4e76b9d74e8e129e5e7b {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:18:31 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:18:32 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-13301644-94d0-4a00-b5fd-4cb958706f6d tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] CMD "qemu-img convert -t none -O qcow2 -f qcow2 /opt/stack/data/nova/instances/99d6f85f-1b77-4390-8b61-908c98f584c0/disk /opt/stack/data/nova/instances/snapshots/tmpbfgy6oq9/7826e0db5ddb4e76b9d74e8e129e5e7b" returned: 0 in 1.033s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:18:32 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-13301644-94d0-4a00-b5fd-4cb958706f6d tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] [instance: 99d6f85f-1b77-4390-8b61-908c98f584c0] Snapshot extracted, beginning image upload
Nov 29 03:18:33 user nova-compute[75234]: DEBUG nova.compute.manager [req-db4cc352-48e0-49fd-b6fc-947bac7d1471 req-f9533cf8-556a-4412-98a9-461765a824e1 service nova] [instance: 99d6f85f-1b77-4390-8b61-908c98f584c0] Received event network-vif-plugged-b68d722f-9020-4da4-8cdb-e2312ef06df9 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:18:33 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-db4cc352-48e0-49fd-b6fc-947bac7d1471 req-f9533cf8-556a-4412-98a9-461765a824e1 service nova] Acquiring lock "99d6f85f-1b77-4390-8b61-908c98f584c0-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:18:33 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-db4cc352-48e0-49fd-b6fc-947bac7d1471 req-f9533cf8-556a-4412-98a9-461765a824e1 service nova] Lock "99d6f85f-1b77-4390-8b61-908c98f584c0-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:18:33 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-db4cc352-48e0-49fd-b6fc-947bac7d1471 req-f9533cf8-556a-4412-98a9-461765a824e1 service nova] Lock "99d6f85f-1b77-4390-8b61-908c98f584c0-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:18:33 user nova-compute[75234]: DEBUG nova.compute.manager [req-db4cc352-48e0-49fd-b6fc-947bac7d1471 req-f9533cf8-556a-4412-98a9-461765a824e1 service nova] [instance: 99d6f85f-1b77-4390-8b61-908c98f584c0] No waiting events found dispatching network-vif-plugged-b68d722f-9020-4da4-8cdb-e2312ef06df9 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:18:33 user nova-compute[75234]: WARNING nova.compute.manager [req-db4cc352-48e0-49fd-b6fc-947bac7d1471 req-f9533cf8-556a-4412-98a9-461765a824e1 service nova] [instance: 99d6f85f-1b77-4390-8b61-908c98f584c0] Received unexpected event network-vif-plugged-b68d722f-9020-4da4-8cdb-e2312ef06df9 for instance with vm_state active and task_state shelving_image_uploading.
Nov 29 03:18:34 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:18:35 user nova-compute[75234]: DEBUG nova.objects.instance [None req-751217d7-e3a0-4877-b867-15cec419ab79 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Lazy-loading 'flavor' on Instance uuid ec7cf7cd-8e48-4bb3-a969-e14db705dc09 {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:18:35 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-751217d7-e3a0-4877-b867-15cec419ab79 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Lock "ec7cf7cd-8e48-4bb3-a969-e14db705dc09" "released" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: held 33.432s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:18:35 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-5c2a15bc-b937-476d-a3cd-d7aad3257309 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Lock "ec7cf7cd-8e48-4bb3-a969-e14db705dc09" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 33.338s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:18:35 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-5c2a15bc-b937-476d-a3cd-d7aad3257309 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Acquiring lock "ec7cf7cd-8e48-4bb3-a969-e14db705dc09-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:18:35 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-5c2a15bc-b937-476d-a3cd-d7aad3257309 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Lock "ec7cf7cd-8e48-4bb3-a969-e14db705dc09-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:18:35 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-5c2a15bc-b937-476d-a3cd-d7aad3257309 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Lock "ec7cf7cd-8e48-4bb3-a969-e14db705dc09-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:18:35 user nova-compute[75234]: INFO nova.compute.manager [None req-5c2a15bc-b937-476d-a3cd-d7aad3257309 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] [instance: ec7cf7cd-8e48-4bb3-a969-e14db705dc09] Terminating instance
Nov 29 03:18:35 user nova-compute[75234]: DEBUG nova.compute.manager [None req-5c2a15bc-b937-476d-a3cd-d7aad3257309 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] [instance: ec7cf7cd-8e48-4bb3-a969-e14db705dc09] Start destroying the instance on the hypervisor. {{(pid=75234) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3140}}
Nov 29 03:18:35 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:18:35 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:18:35 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:18:35 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:18:35 user nova-compute[75234]: DEBUG nova.compute.manager [req-262e80ce-2dd0-40d6-b5ce-1a36f12ec426 req-64e31485-4664-48ab-bf71-1b82373444d1 service nova] [instance: ec7cf7cd-8e48-4bb3-a969-e14db705dc09] Received event network-vif-unplugged-9d89f22e-e411-461b-b9b3-85dbcf58af43 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:18:35 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-262e80ce-2dd0-40d6-b5ce-1a36f12ec426 req-64e31485-4664-48ab-bf71-1b82373444d1 service nova] Acquiring lock "ec7cf7cd-8e48-4bb3-a969-e14db705dc09-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:18:35 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-262e80ce-2dd0-40d6-b5ce-1a36f12ec426 req-64e31485-4664-48ab-bf71-1b82373444d1 service nova] Lock "ec7cf7cd-8e48-4bb3-a969-e14db705dc09-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:18:35 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-262e80ce-2dd0-40d6-b5ce-1a36f12ec426 req-64e31485-4664-48ab-bf71-1b82373444d1 service nova] Lock "ec7cf7cd-8e48-4bb3-a969-e14db705dc09-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:18:35 user nova-compute[75234]: DEBUG nova.compute.manager [req-262e80ce-2dd0-40d6-b5ce-1a36f12ec426 req-64e31485-4664-48ab-bf71-1b82373444d1 service nova] [instance: ec7cf7cd-8e48-4bb3-a969-e14db705dc09] No waiting events found dispatching network-vif-unplugged-9d89f22e-e411-461b-b9b3-85dbcf58af43 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:18:35 user nova-compute[75234]: DEBUG nova.compute.manager [req-262e80ce-2dd0-40d6-b5ce-1a36f12ec426 req-64e31485-4664-48ab-bf71-1b82373444d1 service nova] [instance: ec7cf7cd-8e48-4bb3-a969-e14db705dc09] Received event network-vif-unplugged-9d89f22e-e411-461b-b9b3-85dbcf58af43 for instance with task_state deleting. {{(pid=75234) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10890}}
Nov 29 03:18:36 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-13301644-94d0-4a00-b5fd-4cb958706f6d tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] [instance: 99d6f85f-1b77-4390-8b61-908c98f584c0] Snapshot image upload complete
Nov 29 03:18:36 user nova-compute[75234]: DEBUG nova.compute.manager [None req-13301644-94d0-4a00-b5fd-4cb958706f6d tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] [instance: 99d6f85f-1b77-4390-8b61-908c98f584c0] Checking state {{(pid=75234) _get_power_state /opt/stack/nova/nova/compute/manager.py:1782}}
Nov 29 03:18:36 user nova-compute[75234]: INFO nova.compute.manager [None req-13301644-94d0-4a00-b5fd-4cb958706f6d tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] [instance: 99d6f85f-1b77-4390-8b61-908c98f584c0] Shelve offloading
Nov 29 03:18:36 user nova-compute[75234]: INFO nova.virt.libvirt.driver [-] [instance: ec7cf7cd-8e48-4bb3-a969-e14db705dc09] Instance destroyed successfully.
Nov 29 03:18:36 user nova-compute[75234]: DEBUG nova.objects.instance [None req-5c2a15bc-b937-476d-a3cd-d7aad3257309 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Lazy-loading 'resources' on Instance uuid ec7cf7cd-8e48-4bb3-a969-e14db705dc09 {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:18:36 user nova-compute[75234]: INFO nova.virt.libvirt.driver [-] [instance: 99d6f85f-1b77-4390-8b61-908c98f584c0] Instance destroyed successfully.
Nov 29 03:18:36 user nova-compute[75234]: DEBUG nova.compute.manager [None req-13301644-94d0-4a00-b5fd-4cb958706f6d tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] [instance: 99d6f85f-1b77-4390-8b61-908c98f584c0] Checking state {{(pid=75234) _get_power_state /opt/stack/nova/nova/compute/manager.py:1782}}
Nov 29 03:18:36 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-5c2a15bc-b937-476d-a3cd-d7aad3257309 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-11-29T03:15:19Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-VolumesSnapshotTestJSON-instance-213375111',display_name='tempest-VolumesSnapshotTestJSON-instance-213375111',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-volumessnapshottestjson-instance-213375111',id=23,image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBKfdFQdP9b2CUucLMguc8tjFgkfx70MU+FbCMNYtTpv3yP6N/lf+rZlWO4aVrwdI3Q2J0as5mFJ1RqaTpNerkZvmi2PeYojCi8bJzCIaL5bklATBcDaVb7EKkDOmV8PS5Q==',key_name='tempest-keypair-798189340',keypairs=<?>,launch_index=0,launched_at=2023-11-29T03:15:26Z,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='7e24dd8accea4c1e9598d643ea79c86d',ramdisk_id='',reservation_id='r-mjtjrmkx',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-VolumesSnapshotTestJSON-902320119',owner_user_name='tempest-VolumesSnapshotTestJSON-902320119-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-11-29T03:15:26Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='899f638d0de34dcba53f4c2ef7f5e068',uuid=ec7cf7cd-8e48-4bb3-a969-e14db705dc09,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "9d89f22e-e411-461b-b9b3-85dbcf58af43", "address": "fa:16:3e:30:11:ff", "network": {"id": "1898605d-00e7-4e81-964e-821e09f5421c", "bridge": "br-int", "label": "tempest-VolumesSnapshotTestJSON-1896954656-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.176", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "7e24dd8accea4c1e9598d643ea79c86d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap9d89f22e-e4", "ovs_interfaceid": "9d89f22e-e411-461b-b9b3-85dbcf58af43", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Nov 29 03:18:36 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-5c2a15bc-b937-476d-a3cd-d7aad3257309 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Converting VIF {"id": "9d89f22e-e411-461b-b9b3-85dbcf58af43", "address": "fa:16:3e:30:11:ff", "network": {"id": "1898605d-00e7-4e81-964e-821e09f5421c", "bridge": "br-int", "label": "tempest-VolumesSnapshotTestJSON-1896954656-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.176", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "7e24dd8accea4c1e9598d643ea79c86d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap9d89f22e-e4", "ovs_interfaceid": "9d89f22e-e411-461b-b9b3-85dbcf58af43", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:18:36 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-5c2a15bc-b937-476d-a3cd-d7aad3257309 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:30:11:ff,bridge_name='br-int',has_traffic_filtering=True,id=9d89f22e-e411-461b-b9b3-85dbcf58af43,network=Network(1898605d-00e7-4e81-964e-821e09f5421c),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap9d89f22e-e4') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:18:36 user nova-compute[75234]: DEBUG os_vif [None req-5c2a15bc-b937-476d-a3cd-d7aad3257309 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Unplugging vif VIFOpenVSwitch(active=True,address=fa:16:3e:30:11:ff,bridge_name='br-int',has_traffic_filtering=True,id=9d89f22e-e411-461b-b9b3-85dbcf58af43,network=Network(1898605d-00e7-4e81-964e-821e09f5421c),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap9d89f22e-e4') {{(pid=75234) unplug /opt/stack/data/venv/lib/python3.10/site-packages/os_vif/__init__.py:109}}
Nov 29 03:18:36 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:18:36 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tap9d89f22e-e4, bridge=br-int, if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:18:36 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:18:36 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-13301644-94d0-4a00-b5fd-4cb958706f6d tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Acquiring lock "refresh_cache-99d6f85f-1b77-4390-8b61-908c98f584c0" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:18:36 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-13301644-94d0-4a00-b5fd-4cb958706f6d tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Acquired lock "refresh_cache-99d6f85f-1b77-4390-8b61-908c98f584c0" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:18:36 user nova-compute[75234]: DEBUG nova.network.neutron [None req-13301644-94d0-4a00-b5fd-4cb958706f6d tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] [instance: 99d6f85f-1b77-4390-8b61-908c98f584c0] Building network info cache for instance {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2009}}
Nov 29 03:18:36 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:18:36 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:18:36 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:18:36 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbDestroyCommand(_result=None, table=QoS, record=8951153a-de0d-4c29-9db3-748bfdd49e97) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:18:36 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:18:36 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:18:36 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:18:36 user nova-compute[75234]: INFO os_vif [None req-5c2a15bc-b937-476d-a3cd-d7aad3257309 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Successfully unplugged vif VIFOpenVSwitch(active=True,address=fa:16:3e:30:11:ff,bridge_name='br-int',has_traffic_filtering=True,id=9d89f22e-e411-461b-b9b3-85dbcf58af43,network=Network(1898605d-00e7-4e81-964e-821e09f5421c),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap9d89f22e-e4')
Nov 29 03:18:36 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-5c2a15bc-b937-476d-a3cd-d7aad3257309 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] [instance: ec7cf7cd-8e48-4bb3-a969-e14db705dc09] calling os-brick to detach iSCSI Volume {{(pid=75234) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Nov 29 03:18:36 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-5c2a15bc-b937-476d-a3cd-d7aad3257309 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f8a5050c9a0>, {'target_iqn': 'iqn.2010-10.org.openstack:6c2e1179-dd3c-41c4-9594-bf4a16752c67', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'shDupLmi', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdc'}, None), 'kwargs': True}}" {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:176}}
Nov 29 03:18:36 user nova-compute[75234]: WARNING os_brick.initiator.connectors.base [None req-5c2a15bc-b937-476d-a3cd-d7aad3257309 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Nov 29 03:18:36 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-5c2a15bc-b937-476d-a3cd-d7aad3257309 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:68}}
Nov 29 03:18:36 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-5c2a15bc-b937-476d-a3cd-d7aad3257309 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.002s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:73}}
Nov 29 03:18:36 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:18:36 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.023s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:18:36 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[344b0e54-e8b9-42ac-b4de-47327ae3f2ee]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:cf6ab229-533a-424e-bd67-1d54d50ad4be\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:bf57b967-7fad-4d76-abb2-319fe88f8ef9\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:a5b104bd-399e-4445-a186-0d4275ae271a\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:18:36 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-5c2a15bc-b937-476d-a3cd-d7aad3257309 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS:
Nov 29 03:18:36 user nova-compute[75234]: No targets found.
Nov 29 03:18:36 user nova-compute[75234]: iSNS:
Nov 29 03:18:36 user nova-compute[75234]: No targets found.
Nov 29 03:18:36 user nova-compute[75234]: STATIC:
Nov 29 03:18:36 user nova-compute[75234]: Target: iqn.2010-10.org.openstack:cf6ab229-533a-424e-bd67-1d54d50ad4be
Nov 29 03:18:36 user nova-compute[75234]:         Portal: 172.16.0.220:3260,-1
Nov 29 03:18:36 user nova-compute[75234]:                 Iface Name: default
Nov 29 03:18:36 user nova-compute[75234]: Target: iqn.2010-10.org.openstack:bf57b967-7fad-4d76-abb2-319fe88f8ef9
Nov 29 03:18:36 user nova-compute[75234]:         Portal: 172.16.0.220:3260,-1
Nov 29 03:18:36 user nova-compute[75234]:                 Iface Name: default
Nov 29 03:18:36 user nova-compute[75234]: Target: iqn.2010-10.org.openstack:a5b104bd-399e-4445-a186-0d4275ae271a
Nov 29 03:18:36 user nova-compute[75234]:         Portal: 172.16.0.220:3260,-1
Nov 29 03:18:36 user nova-compute[75234]:                 Iface Name: default
Nov 29 03:18:36 user nova-compute[75234]: FIRMWARE:
Nov 29 03:18:36 user nova-compute[75234]: No targets found.
Nov 29 03:18:36 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:18:36 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-5c2a15bc-b937-476d-a3cd-d7aad3257309 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Regex to get portals from discoverydb: ^SENDTARGETS:
Nov 29 03:18:36 user nova-compute[75234]: .*?^DiscoveryAddress: 172.16.0.220,3260.*?
Nov 29 03:18:36 user nova-compute[75234]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=75234) _get_discoverydb_portals /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:382}}
Nov 29 03:18:36 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-5c2a15bc-b937-476d-a3cd-d7aad3257309 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:6c2e1179-dd3c-41c4-9594-bf4a16752c67', 0)] {{(pid=75234) _get_connection_devices /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:830}}
Nov 29 03:18:36 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:18:36 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.019s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:18:36 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[4e33d0cf-a245-41f7-965f-961cd3e2a42c]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:a5b104bd-399e-4445-a186-0d4275ae271a\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:bf57b967-7fad-4d76-abb2-319fe88f8ef9\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:cf6ab229-533a-424e-bd67-1d54d50ad4be\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:18:36 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:18:36 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.024s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:18:36 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[01c0fbe6-eac4-405f-9c41-a4fa8b72e06e]: (4, ('tcp: [29] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bf57b967-7fad-4d76-abb2-319fe88f8ef9 (non-flash)\ntcp: [31] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a5b104bd-399e-4445-a186-0d4275ae271a (non-flash)\ntcp: [32] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:cf6ab229-533a-424e-bd67-1d54d50ad4be (non-flash)\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:18:36 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-5c2a15bc-b937-476d-a3cd-d7aad3257309 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [29] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bf57b967-7fad-4d76-abb2-319fe88f8ef9 (non-flash)
Nov 29 03:18:36 user nova-compute[75234]: tcp: [31] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a5b104bd-399e-4445-a186-0d4275ae271a (non-flash)
Nov 29 03:18:36 user nova-compute[75234]: tcp: [32] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:cf6ab229-533a-424e-bd67-1d54d50ad4be (non-flash)
Nov 29 03:18:36 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:18:36 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-5c2a15bc-b937-476d-a3cd-d7aad3257309 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] iscsi session list stdout=tcp: [29] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bf57b967-7fad-4d76-abb2-319fe88f8ef9 (non-flash)
Nov 29 03:18:36 user nova-compute[75234]: tcp: [31] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a5b104bd-399e-4445-a186-0d4275ae271a (non-flash)
Nov 29 03:18:36 user nova-compute[75234]: tcp: [32] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:cf6ab229-533a-424e-bd67-1d54d50ad4be (non-flash)
Nov 29 03:18:36 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsi_session /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Nov 29 03:18:36 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-5c2a15bc-b937-476d-a3cd-d7aad3257309 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7f8a601341f0>, {}) {{(pid=75234) _get_connection_devices /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:863}}
Nov 29 03:18:36 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-5c2a15bc-b937-476d-a3cd-d7aad3257309 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Disconnecting from: [] {{(pid=75234) _disconnect_connection /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Nov 29 03:18:36 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-5c2a15bc-b937-476d-a3cd-d7aad3257309 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.083s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:87}}
Nov 29 03:18:36 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-5c2a15bc-b937-476d-a3cd-d7aad3257309 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] <== disconnect_volume: return (85ms) None {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:203}}
Nov 29 03:18:36 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-5c2a15bc-b937-476d-a3cd-d7aad3257309 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] [instance: ec7cf7cd-8e48-4bb3-a969-e14db705dc09] Disconnected iSCSI Volume {{(pid=75234) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:79}}
Nov 29 03:18:36 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-5c2a15bc-b937-476d-a3cd-d7aad3257309 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] [instance: ec7cf7cd-8e48-4bb3-a969-e14db705dc09] Deleting instance files /opt/stack/data/nova/instances/ec7cf7cd-8e48-4bb3-a969-e14db705dc09_del
Nov 29 03:18:36 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-5c2a15bc-b937-476d-a3cd-d7aad3257309 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] [instance: ec7cf7cd-8e48-4bb3-a969-e14db705dc09] Deletion of /opt/stack/data/nova/instances/ec7cf7cd-8e48-4bb3-a969-e14db705dc09_del complete
Nov 29 03:18:36 user nova-compute[75234]: INFO nova.compute.manager [None req-5c2a15bc-b937-476d-a3cd-d7aad3257309 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] [instance: ec7cf7cd-8e48-4bb3-a969-e14db705dc09] Took 1.05 seconds to destroy the instance on the hypervisor.
Nov 29 03:18:36 user nova-compute[75234]: DEBUG oslo.service.loopingcall [None req-5c2a15bc-b937-476d-a3cd-d7aad3257309 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=75234) func /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/loopingcall.py:435}}
Nov 29 03:18:36 user nova-compute[75234]: DEBUG nova.compute.manager [-] [instance: ec7cf7cd-8e48-4bb3-a969-e14db705dc09] Deallocating network for instance {{(pid=75234) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2279}}
Nov 29 03:18:36 user nova-compute[75234]: DEBUG nova.network.neutron [-] [instance: ec7cf7cd-8e48-4bb3-a969-e14db705dc09] deallocate_for_instance() {{(pid=75234) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1802}}
Nov 29 03:18:36 user nova-compute[75234]: DEBUG nova.network.neutron [None req-13301644-94d0-4a00-b5fd-4cb958706f6d tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] [instance: 99d6f85f-1b77-4390-8b61-908c98f584c0] Updating instance_info_cache with network_info: [{"id": "b68d722f-9020-4da4-8cdb-e2312ef06df9", "address": "fa:16:3e:1e:90:83", "network": {"id": "aad085f7-106b-4de6-93f5-f86bb66ec962", "bridge": "br-int", "label": "tempest-AttachVolumeShelveTestJSON-775984268-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.193", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "e17150005a9a449f9373b91e1c00083b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapb68d722f-90", "ovs_interfaceid": "b68d722f-9020-4da4-8cdb-e2312ef06df9", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:18:36 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-13301644-94d0-4a00-b5fd-4cb958706f6d tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Releasing lock "refresh_cache-99d6f85f-1b77-4390-8b61-908c98f584c0" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:18:36 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:18:37 user nova-compute[75234]: INFO nova.virt.libvirt.driver [-] [instance: 99d6f85f-1b77-4390-8b61-908c98f584c0] Instance destroyed successfully.
Nov 29 03:18:37 user nova-compute[75234]: DEBUG nova.objects.instance [None req-13301644-94d0-4a00-b5fd-4cb958706f6d tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Lazy-loading 'resources' on Instance uuid 99d6f85f-1b77-4390-8b61-908c98f584c0 {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:18:37 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-13301644-94d0-4a00-b5fd-4cb958706f6d tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-11-29T03:16:35Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description=None,display_name='tempest-AttachVolumeShelveTestJSON-server-491132081',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-attachvolumeshelvetestjson-server-491132081',id=28,image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBICHY8pkIfnX3FeZlRYJkWHQpVAhCpz3AE8GcQPc0ndKAzNj2za38XOUjeohx2bMy5FXksYmprL/THiMHoZRTInmzdW4YHaLMkQCBMth1H/mEdsilbqzkRPZKyCYZLp9Bw==',key_name='tempest-keypair-1848667272',keypairs=<?>,launch_index=0,launched_at=2023-11-29T03:16:42Z,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=<?>,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=4,progress=0,project_id='e17150005a9a449f9373b91e1c00083b',ramdisk_id='',reservation_id='r-79uedf60',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-AttachVolumeShelveTestJSON-1180535246',owner_user_name='tempest-AttachVolumeShelveTestJSON-1180535246-project-member',shelved_at='2023-11-29T03:18:36.093023',shelved_host='user',shelved_image_id='d7de6e9a-26aa-4b83-99e9-7aee3378b12d'},tags=<?>,task_state='shelving_offloading',terminated_at=None,trusted_certs=<?>,updated_at=2023-11-29T03:18:33Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='154f3f4f57514450baf8f22d67768bc9',uuid=99d6f85f-1b77-4390-8b61-908c98f584c0,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='shelved') vif={"id": "b68d722f-9020-4da4-8cdb-e2312ef06df9", "address": "fa:16:3e:1e:90:83", "network": {"id": "aad085f7-106b-4de6-93f5-f86bb66ec962", "bridge": "br-int", "label": "tempest-AttachVolumeShelveTestJSON-775984268-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.193", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "e17150005a9a449f9373b91e1c00083b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapb68d722f-90", "ovs_interfaceid": "b68d722f-9020-4da4-8cdb-e2312ef06df9", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Nov 29 03:18:37 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-13301644-94d0-4a00-b5fd-4cb958706f6d tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Converting VIF {"id": "b68d722f-9020-4da4-8cdb-e2312ef06df9", "address": "fa:16:3e:1e:90:83", "network": {"id": "aad085f7-106b-4de6-93f5-f86bb66ec962", "bridge": "br-int", "label": "tempest-AttachVolumeShelveTestJSON-775984268-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.193", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "e17150005a9a449f9373b91e1c00083b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapb68d722f-90", "ovs_interfaceid": "b68d722f-9020-4da4-8cdb-e2312ef06df9", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:18:37 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-13301644-94d0-4a00-b5fd-4cb958706f6d tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:1e:90:83,bridge_name='br-int',has_traffic_filtering=True,id=b68d722f-9020-4da4-8cdb-e2312ef06df9,network=Network(aad085f7-106b-4de6-93f5-f86bb66ec962),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapb68d722f-90') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:18:37 user nova-compute[75234]: DEBUG os_vif [None req-13301644-94d0-4a00-b5fd-4cb958706f6d tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Unplugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:1e:90:83,bridge_name='br-int',has_traffic_filtering=True,id=b68d722f-9020-4da4-8cdb-e2312ef06df9,network=Network(aad085f7-106b-4de6-93f5-f86bb66ec962),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapb68d722f-90') {{(pid=75234) unplug /opt/stack/data/venv/lib/python3.10/site-packages/os_vif/__init__.py:109}}
Nov 29 03:18:37 user nova-compute[75234]: DEBUG nova.network.neutron [-] [instance: ec7cf7cd-8e48-4bb3-a969-e14db705dc09] Updating instance_info_cache with network_info: [] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:18:37 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:18:37 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tapb68d722f-90, bridge=br-int, if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:18:37 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:18:37 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:18:37 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:18:37 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbDestroyCommand(_result=None, table=QoS, record=93a08874-f41d-4fa3-bdfb-d88561a14a6f) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:18:37 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:18:37 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:18:37 user nova-compute[75234]: INFO os_vif [None req-13301644-94d0-4a00-b5fd-4cb958706f6d tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Successfully unplugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:1e:90:83,bridge_name='br-int',has_traffic_filtering=True,id=b68d722f-9020-4da4-8cdb-e2312ef06df9,network=Network(aad085f7-106b-4de6-93f5-f86bb66ec962),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapb68d722f-90')
Nov 29 03:18:37 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-13301644-94d0-4a00-b5fd-4cb958706f6d tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] [instance: 99d6f85f-1b77-4390-8b61-908c98f584c0] Deleting instance files /opt/stack/data/nova/instances/99d6f85f-1b77-4390-8b61-908c98f584c0_del
Nov 29 03:18:37 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-13301644-94d0-4a00-b5fd-4cb958706f6d tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] [instance: 99d6f85f-1b77-4390-8b61-908c98f584c0] Deletion of /opt/stack/data/nova/instances/99d6f85f-1b77-4390-8b61-908c98f584c0_del complete
Nov 29 03:18:37 user nova-compute[75234]: INFO nova.compute.manager [-] [instance: ec7cf7cd-8e48-4bb3-a969-e14db705dc09] Took 0.93 seconds to deallocate network for instance.
Nov 29 03:18:37 user nova-compute[75234]: DEBUG nova.compute.manager [req-ea95fd81-1335-450f-9333-94fa66c6c444 req-633544b4-e76e-430b-94f1-ba5214122a0f service nova] [instance: 99d6f85f-1b77-4390-8b61-908c98f584c0] Received event network-changed-b68d722f-9020-4da4-8cdb-e2312ef06df9 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:18:37 user nova-compute[75234]: DEBUG nova.compute.manager [req-ea95fd81-1335-450f-9333-94fa66c6c444 req-633544b4-e76e-430b-94f1-ba5214122a0f service nova] [instance: 99d6f85f-1b77-4390-8b61-908c98f584c0] Refreshing instance network info cache due to event network-changed-b68d722f-9020-4da4-8cdb-e2312ef06df9. {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11117}}
Nov 29 03:18:37 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-ea95fd81-1335-450f-9333-94fa66c6c444 req-633544b4-e76e-430b-94f1-ba5214122a0f service nova] Acquiring lock "refresh_cache-99d6f85f-1b77-4390-8b61-908c98f584c0" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:18:37 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-ea95fd81-1335-450f-9333-94fa66c6c444 req-633544b4-e76e-430b-94f1-ba5214122a0f service nova] Acquired lock "refresh_cache-99d6f85f-1b77-4390-8b61-908c98f584c0" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:18:37 user nova-compute[75234]: DEBUG nova.network.neutron [req-ea95fd81-1335-450f-9333-94fa66c6c444 req-633544b4-e76e-430b-94f1-ba5214122a0f service nova] [instance: 99d6f85f-1b77-4390-8b61-908c98f584c0] Refreshing network info cache for port b68d722f-9020-4da4-8cdb-e2312ef06df9 {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2006}}
Nov 29 03:18:37 user nova-compute[75234]: DEBUG nova.compute.manager [req-509304b5-4b1e-4ef3-a767-77c217506df6 req-252defa1-6b21-44ed-bb2f-5ca4e68f0ef4 service nova] [instance: ec7cf7cd-8e48-4bb3-a969-e14db705dc09] Received event network-vif-deleted-9d89f22e-e411-461b-b9b3-85dbcf58af43 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:18:37 user nova-compute[75234]: INFO nova.compute.manager [req-509304b5-4b1e-4ef3-a767-77c217506df6 req-252defa1-6b21-44ed-bb2f-5ca4e68f0ef4 service nova] [instance: ec7cf7cd-8e48-4bb3-a969-e14db705dc09] Neutron deleted interface 9d89f22e-e411-461b-b9b3-85dbcf58af43; detaching it from the instance and deleting it from the info cache
Nov 29 03:18:37 user nova-compute[75234]: DEBUG nova.network.neutron [req-509304b5-4b1e-4ef3-a767-77c217506df6 req-252defa1-6b21-44ed-bb2f-5ca4e68f0ef4 service nova] [instance: ec7cf7cd-8e48-4bb3-a969-e14db705dc09] Updating instance_info_cache with network_info: [] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:18:37 user nova-compute[75234]: DEBUG nova.compute.manager [req-509304b5-4b1e-4ef3-a767-77c217506df6 req-252defa1-6b21-44ed-bb2f-5ca4e68f0ef4 service nova] [instance: ec7cf7cd-8e48-4bb3-a969-e14db705dc09] Detach interface failed, port_id=9d89f22e-e411-461b-b9b3-85dbcf58af43, reason: Instance ec7cf7cd-8e48-4bb3-a969-e14db705dc09 could not be found. {{(pid=75234) _process_instance_vif_deleted_event /opt/stack/nova/nova/compute/manager.py:10946}}
Nov 29 03:18:37 user nova-compute[75234]: WARNING nova.volume.cinder [None req-5c2a15bc-b937-476d-a3cd-d7aad3257309 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Attachment bc636743-cf39-428d-9a00-f4befddada03 does not exist. Ignoring.: cinderclient.exceptions.NotFound: Volume attachment could not be found with filter: attachment_id = bc636743-cf39-428d-9a00-f4befddada03. (HTTP 404) (Request-ID: req-afb7d325-6255-499d-be97-905b81328794)
Nov 29 03:18:37 user nova-compute[75234]: INFO nova.compute.manager [None req-5c2a15bc-b937-476d-a3cd-d7aad3257309 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] [instance: ec7cf7cd-8e48-4bb3-a969-e14db705dc09] Took 0.04 seconds to detach 1 volumes for instance.
Nov 29 03:18:37 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-5c2a15bc-b937-476d-a3cd-d7aad3257309 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:18:37 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-5c2a15bc-b937-476d-a3cd-d7aad3257309 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:18:37 user nova-compute[75234]: INFO nova.scheduler.client.report [None req-13301644-94d0-4a00-b5fd-4cb958706f6d tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Deleted allocations for instance 99d6f85f-1b77-4390-8b61-908c98f584c0
Nov 29 03:18:37 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-13301644-94d0-4a00-b5fd-4cb958706f6d tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:18:37 user nova-compute[75234]: DEBUG nova.compute.manager [req-4c30b2dd-0ebf-44c8-9de6-a05235c6d95c req-b458a09c-423d-4a5c-9f44-58ea6e7f7e68 service nova] [instance: ec7cf7cd-8e48-4bb3-a969-e14db705dc09] Received event network-vif-plugged-9d89f22e-e411-461b-b9b3-85dbcf58af43 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:18:37 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-4c30b2dd-0ebf-44c8-9de6-a05235c6d95c req-b458a09c-423d-4a5c-9f44-58ea6e7f7e68 service nova] Acquiring lock "ec7cf7cd-8e48-4bb3-a969-e14db705dc09-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:18:37 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-4c30b2dd-0ebf-44c8-9de6-a05235c6d95c req-b458a09c-423d-4a5c-9f44-58ea6e7f7e68 service nova] Lock "ec7cf7cd-8e48-4bb3-a969-e14db705dc09-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:18:37 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-4c30b2dd-0ebf-44c8-9de6-a05235c6d95c req-b458a09c-423d-4a5c-9f44-58ea6e7f7e68 service nova] Lock "ec7cf7cd-8e48-4bb3-a969-e14db705dc09-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:18:37 user nova-compute[75234]: DEBUG nova.compute.manager [req-4c30b2dd-0ebf-44c8-9de6-a05235c6d95c req-b458a09c-423d-4a5c-9f44-58ea6e7f7e68 service nova] [instance: ec7cf7cd-8e48-4bb3-a969-e14db705dc09] No waiting events found dispatching network-vif-plugged-9d89f22e-e411-461b-b9b3-85dbcf58af43 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:18:37 user nova-compute[75234]: WARNING nova.compute.manager [req-4c30b2dd-0ebf-44c8-9de6-a05235c6d95c req-b458a09c-423d-4a5c-9f44-58ea6e7f7e68 service nova] [instance: ec7cf7cd-8e48-4bb3-a969-e14db705dc09] Received unexpected event network-vif-plugged-9d89f22e-e411-461b-b9b3-85dbcf58af43 for instance with vm_state deleted and task_state None.
Nov 29 03:18:37 user nova-compute[75234]: DEBUG nova.network.neutron [req-ea95fd81-1335-450f-9333-94fa66c6c444 req-633544b4-e76e-430b-94f1-ba5214122a0f service nova] [instance: 99d6f85f-1b77-4390-8b61-908c98f584c0] Updated VIF entry in instance network info cache for port b68d722f-9020-4da4-8cdb-e2312ef06df9. {{(pid=75234) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3481}}
Nov 29 03:18:37 user nova-compute[75234]: DEBUG nova.network.neutron [req-ea95fd81-1335-450f-9333-94fa66c6c444 req-633544b4-e76e-430b-94f1-ba5214122a0f service nova] [instance: 99d6f85f-1b77-4390-8b61-908c98f584c0] Updating instance_info_cache with network_info: [{"id": "b68d722f-9020-4da4-8cdb-e2312ef06df9", "address": "fa:16:3e:1e:90:83", "network": {"id": "aad085f7-106b-4de6-93f5-f86bb66ec962", "bridge": null, "label": "tempest-AttachVolumeShelveTestJSON-775984268-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.193", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "e17150005a9a449f9373b91e1c00083b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "unbound", "details": {}, "devname": "tapb68d722f-90", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:18:37 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-ea95fd81-1335-450f-9333-94fa66c6c444 req-633544b4-e76e-430b-94f1-ba5214122a0f service nova] Releasing lock "refresh_cache-99d6f85f-1b77-4390-8b61-908c98f584c0" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:18:38 user nova-compute[75234]: DEBUG nova.compute.provider_tree [None req-5c2a15bc-b937-476d-a3cd-d7aad3257309 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Inventory has not changed in ProviderTree for provider: 4d291220-7c37-4839-a0e3-b23c66502b74 {{(pid=75234) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Nov 29 03:18:38 user nova-compute[75234]: DEBUG nova.scheduler.client.report [None req-5c2a15bc-b937-476d-a3cd-d7aad3257309 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Inventory has not changed for provider 4d291220-7c37-4839-a0e3-b23c66502b74 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=75234) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Nov 29 03:18:38 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-5c2a15bc-b937-476d-a3cd-d7aad3257309 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.489s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:18:38 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-13301644-94d0-4a00-b5fd-4cb958706f6d tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.294s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:18:38 user nova-compute[75234]: DEBUG nova.objects.instance [None req-13301644-94d0-4a00-b5fd-4cb958706f6d tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Lazy-loading 'numa_topology' on Instance uuid 99d6f85f-1b77-4390-8b61-908c98f584c0 {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:18:38 user nova-compute[75234]: DEBUG nova.objects.base [None req-13301644-94d0-4a00-b5fd-4cb958706f6d tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Object Instance<99d6f85f-1b77-4390-8b61-908c98f584c0> lazy-loaded attributes: resources,numa_topology {{(pid=75234) wrapper /opt/stack/nova/nova/objects/base.py:126}}
Nov 29 03:18:38 user nova-compute[75234]: INFO nova.scheduler.client.report [None req-5c2a15bc-b937-476d-a3cd-d7aad3257309 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Deleted allocations for instance ec7cf7cd-8e48-4bb3-a969-e14db705dc09
Nov 29 03:18:38 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-5c2a15bc-b937-476d-a3cd-d7aad3257309 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Lock "ec7cf7cd-8e48-4bb3-a969-e14db705dc09" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 2.803s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:18:38 user nova-compute[75234]: DEBUG nova.compute.provider_tree [None req-13301644-94d0-4a00-b5fd-4cb958706f6d tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Inventory has not changed in ProviderTree for provider: 4d291220-7c37-4839-a0e3-b23c66502b74 {{(pid=75234) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Nov 29 03:18:38 user nova-compute[75234]: DEBUG nova.scheduler.client.report [None req-13301644-94d0-4a00-b5fd-4cb958706f6d tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Inventory has not changed for provider 4d291220-7c37-4839-a0e3-b23c66502b74 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=75234) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Nov 29 03:18:38 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-13301644-94d0-4a00-b5fd-4cb958706f6d tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.384s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:18:38 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-13301644-94d0-4a00-b5fd-4cb958706f6d tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Lock "99d6f85f-1b77-4390-8b61-908c98f584c0" "released" by "nova.compute.manager.ComputeManager.shelve_instance.<locals>.do_shelve_instance" :: held 7.687s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:18:38 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-e9bfef58-f895-42b7-bc9c-772ebab67bc8 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] [instance: aa190a9f-ba53-43fb-8bae-2c429d0d1228] Creating instance directory {{(pid=75234) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4727}}
Nov 29 03:18:38 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-e9bfef58-f895-42b7-bc9c-772ebab67bc8 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] [instance: aa190a9f-ba53-43fb-8bae-2c429d0d1228] Creating image(s)
Nov 29 03:18:38 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-e9bfef58-f895-42b7-bc9c-772ebab67bc8 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] [instance: aa190a9f-ba53-43fb-8bae-2c429d0d1228] Did not create local disks {{(pid=75234) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4863}}
Nov 29 03:18:38 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-e9bfef58-f895-42b7-bc9c-772ebab67bc8 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] [instance: aa190a9f-ba53-43fb-8bae-2c429d0d1228] Ensure instance console log exists: /opt/stack/data/nova/instances/aa190a9f-ba53-43fb-8bae-2c429d0d1228/console.log {{(pid=75234) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4613}}
Nov 29 03:18:38 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-e9bfef58-f895-42b7-bc9c-772ebab67bc8 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:18:38 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-e9bfef58-f895-42b7-bc9c-772ebab67bc8 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:18:38 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-e9bfef58-f895-42b7-bc9c-772ebab67bc8 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:18:38 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-e9bfef58-f895-42b7-bc9c-772ebab67bc8 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] [instance: aa190a9f-ba53-43fb-8bae-2c429d0d1228] Start _get_guest_xml network_info=[{"id": "2cc37e6a-5c9f-4bb8-98fa-f4f2b94d0c35", "address": "fa:16:3e:fb:40:18", "network": {"id": "38517dcf-64c7-4d26-9083-5ea5f6c87c2f", "bridge": "br-int", "label": "tempest-ServerActionsV293TestJSON-266920726-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.208", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "581a0b62fee744e1b9a767a0d011c4e0", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2cc37e6a-5c", "ovs_interfaceid": "2cc37e6a-5c9f-4bb8-98fa-f4f2b94d0c35", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, '/dev/vda': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum='c8fc807773e5354afe61636071771906',container_format='bare',created_at=2023-11-29T03:02:15Z,direct_url=<?>,disk_format='qcow2',id=96c30ba7-9b73-4a30-8ba4-7138879573b7,min_disk=0,min_ram=0,name='cirros-0.6.2-x86_64-disk',owner='b18e380921c748b78517b6f1211cd0f1',properties=ImageMetaProps,protected=<?>,size=21430272,status='active',tags=<?>,updated_at=2023-11-29T03:02:17Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [], 'ephemerals': [], 'block_device_mapping': [{'mount_device': '/dev/vda', 'attachment_id': 'dd549176-ab97-4d41-a3e2-1466a20e8e4d', 'disk_bus': 'virtio', 'device_type': 'disk', 'boot_index': 0, 'guest_format': None, 'delete_on_termination': True, 'connection_info': {'driver_volume_type': 'iscsi', 'data': {'target_iqn': 'iqn.2010-10.org.openstack:07f83efc-272d-4e10-9a2b-48d967db613c', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'kRLjYCVO', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}, 'status': 'reserved', 'instance': 'aa190a9f-ba53-43fb-8bae-2c429d0d1228', 'attached_at': '', 'detached_at': '', 'volume_id': '07f83efc-272d-4e10-9a2b-48d967db613c', 'serial': '07f83efc-272d-4e10-9a2b-48d967db613c'}, 'volume_type': None} {{(pid=75234) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7562}}
Nov 29 03:18:38 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-e9bfef58-f895-42b7-bc9c-772ebab67bc8 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.: NotImplementedError
Nov 29 03:18:38 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-e9bfef58-f895-42b7-bc9c-772ebab67bc8 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.: NotImplementedError
Nov 29 03:18:38 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-e9bfef58-f895-42b7-bc9c-772ebab67bc8 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] Searching host: 'user' for CPU controller through CGroups V1... {{(pid=75234) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1652}}
Nov 29 03:18:38 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-e9bfef58-f895-42b7-bc9c-772ebab67bc8 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] CPU controller missing on host. {{(pid=75234) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1662}}
Nov 29 03:18:38 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-e9bfef58-f895-42b7-bc9c-772ebab67bc8 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] Searching host: 'user' for CPU controller through CGroups V2... {{(pid=75234) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1671}}
Nov 29 03:18:38 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-e9bfef58-f895-42b7-bc9c-772ebab67bc8 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] CPU controller found on host. {{(pid=75234) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1678}}
Nov 29 03:18:38 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-e9bfef58-f895-42b7-bc9c-772ebab67bc8 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=75234) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5400}}
Nov 29 03:18:38 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-e9bfef58-f895-42b7-bc9c-772ebab67bc8 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-11-29T03:02:04Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='1',id=6,is_public=True,memory_mb=512,name='m1.tiny',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='c8fc807773e5354afe61636071771906',container_format='bare',created_at=2023-11-29T03:02:15Z,direct_url=<?>,disk_format='qcow2',id=96c30ba7-9b73-4a30-8ba4-7138879573b7,min_disk=0,min_ram=0,name='cirros-0.6.2-x86_64-disk',owner='b18e380921c748b78517b6f1211cd0f1',properties=ImageMetaProps,protected=<?>,size=21430272,status='active',tags=<?>,updated_at=2023-11-29T03:02:17Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:563}}
Nov 29 03:18:38 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-e9bfef58-f895-42b7-bc9c-772ebab67bc8 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] Flavor limits 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:348}}
Nov 29 03:18:38 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-e9bfef58-f895-42b7-bc9c-772ebab67bc8 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] Image limits 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:352}}
Nov 29 03:18:38 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-e9bfef58-f895-42b7-bc9c-772ebab67bc8 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] Flavor pref 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:388}}
Nov 29 03:18:38 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-e9bfef58-f895-42b7-bc9c-772ebab67bc8 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] Image pref 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:392}}
Nov 29 03:18:38 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-e9bfef58-f895-42b7-bc9c-772ebab67bc8 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:430}}
Nov 29 03:18:38 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-e9bfef58-f895-42b7-bc9c-772ebab67bc8 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:569}}
Nov 29 03:18:38 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-e9bfef58-f895-42b7-bc9c-772ebab67bc8 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=75234) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:471}}
Nov 29 03:18:38 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-e9bfef58-f895-42b7-bc9c-772ebab67bc8 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] Got 1 possible topologies {{(pid=75234) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:501}}
Nov 29 03:18:38 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-e9bfef58-f895-42b7-bc9c-772ebab67bc8 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:575}}
Nov 29 03:18:38 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-e9bfef58-f895-42b7-bc9c-772ebab67bc8 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:577}}
Nov 29 03:18:38 user nova-compute[75234]: DEBUG nova.objects.instance [None req-e9bfef58-f895-42b7-bc9c-772ebab67bc8 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] Lazy-loading 'vcpu_model' on Instance uuid aa190a9f-ba53-43fb-8bae-2c429d0d1228 {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:18:39 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-e9bfef58-f895-42b7-bc9c-772ebab67bc8 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] [instance: aa190a9f-ba53-43fb-8bae-2c429d0d1228] Calling os-brick to attach iSCSI Volume {{(pid=75234) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Nov 29 03:18:39 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-e9bfef58-f895-42b7-bc9c-772ebab67bc8 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f8a5050c9a0>, 'connection_properties': {'target_iqn': 'iqn.2010-10.org.openstack:07f83efc-272d-4e10-9a2b-48d967db613c', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'kRLjYCVO', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:176}}
Nov 29 03:18:39 user nova-compute[75234]: WARNING os_brick.initiator.connectors.base [None req-e9bfef58-f895-42b7-bc9c-772ebab67bc8 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release: NotImplementedError
Nov 29 03:18:39 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-e9bfef58-f895-42b7-bc9c-772ebab67bc8 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:68}}
Nov 29 03:18:39 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-e9bfef58-f895-42b7-bc9c-772ebab67bc8 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:73}}
Nov 29 03:18:39 user nova-compute[75234]: WARNING os_brick.initiator.connectors.base [None req-e9bfef58-f895-42b7-bc9c-772ebab67bc8 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release: NotImplementedError
Nov 29 03:18:39 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-e9bfef58-f895-42b7-bc9c-772ebab67bc8 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:07f83efc-272d-4e10-9a2b-48d967db613c" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:68}}
Nov 29 03:18:39 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-e9bfef58-f895-42b7-bc9c-772ebab67bc8 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:07f83efc-272d-4e10-9a2b-48d967db613c" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:73}}
Nov 29 03:18:39 user nova-compute[75234]: INFO os_brick.initiator.connectors.iscsi [None req-e9bfef58-f895-42b7-bc9c-772ebab67bc8 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Nov 29 03:18:39 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:07f83efc-272d-4e10-9a2b-48d967db613c -p 172.16.0.220:3260 {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:18:39 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:07f83efc-272d-4e10-9a2b-48d967db613c -p 172.16.0.220:3260" returned: 21 in 0.021s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:18:39 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[e3f1a002-a20f-47fc-b779-2bdcf1a81ead]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:18:39 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-e9bfef58-f895-42b7-bc9c-772ebab67bc8 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Nov 29 03:18:39 user nova-compute[75234]:  {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:18:39 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:07f83efc-272d-4e10-9a2b-48d967db613c -p 172.16.0.220:3260 --interface default --op new {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:18:39 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:07f83efc-272d-4e10-9a2b-48d967db613c -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.022s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:18:39 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[784aefa6-694b-4e79-ae7c-ddac6df43501]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:07f83efc-272d-4e10-9a2b-48d967db613c] added\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:18:39 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-e9bfef58-f895-42b7-bc9c-772ebab67bc8 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:07f83efc-272d-4e10-9a2b-48d967db613c] added
Nov 29 03:18:39 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:18:39 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:07f83efc-272d-4e10-9a2b-48d967db613c -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:18:39 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:07f83efc-272d-4e10-9a2b-48d967db613c -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.039s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:18:39 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[b2e51d6a-f777-4a23-afa7-b53cc07a0b1c]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:18:39 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-e9bfef58-f895-42b7-bc9c-772ebab67bc8 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:18:39 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:07f83efc-272d-4e10-9a2b-48d967db613c -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:18:39 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:07f83efc-272d-4e10-9a2b-48d967db613c -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.018s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:18:39 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[ab30f603-cbbd-44cf-999a-f54edd8d796b]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:18:39 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-e9bfef58-f895-42b7-bc9c-772ebab67bc8 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:18:39 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:07f83efc-272d-4e10-9a2b-48d967db613c -p 172.16.0.220:3260 --op update -n node.session.auth.username -v kRLjYCVO {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:18:39 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:07f83efc-272d-4e10-9a2b-48d967db613c -p 172.16.0.220:3260 --op update -n node.session.auth.username -v kRLjYCVO" returned: 0 in 0.019s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:18:39 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[fe31cf0a-9555-48b9-b24d-edc84e5c44b8]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:18:39 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-e9bfef58-f895-42b7-bc9c-772ebab67bc8 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'kRLjYCVO'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:18:39 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:07f83efc-272d-4e10-9a2b-48d967db613c -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:18:39 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:07f83efc-272d-4e10-9a2b-48d967db613c -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.020s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:18:39 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[b5cc075d-3ef4-45fc-94ed-67f25a42baa3]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:18:39 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-e9bfef58-f895-42b7-bc9c-772ebab67bc8 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:18:39 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:18:39 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.026s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:18:39 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[43d17899-d9f9-4578-81d9-1ce6b4b3f973]: (4, ('tcp: [29] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bf57b967-7fad-4d76-abb2-319fe88f8ef9 (non-flash)\ntcp: [31] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a5b104bd-399e-4445-a186-0d4275ae271a (non-flash)\ntcp: [32] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:cf6ab229-533a-424e-bd67-1d54d50ad4be (non-flash)\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:18:39 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-e9bfef58-f895-42b7-bc9c-772ebab67bc8 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [29] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bf57b967-7fad-4d76-abb2-319fe88f8ef9 (non-flash)
Nov 29 03:18:39 user nova-compute[75234]: tcp: [31] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a5b104bd-399e-4445-a186-0d4275ae271a (non-flash)
Nov 29 03:18:39 user nova-compute[75234]: tcp: [32] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:cf6ab229-533a-424e-bd67-1d54d50ad4be (non-flash)
Nov 29 03:18:39 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:18:39 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-e9bfef58-f895-42b7-bc9c-772ebab67bc8 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] iscsi session list stdout=tcp: [29] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bf57b967-7fad-4d76-abb2-319fe88f8ef9 (non-flash)
Nov 29 03:18:39 user nova-compute[75234]: tcp: [31] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a5b104bd-399e-4445-a186-0d4275ae271a (non-flash)
Nov 29 03:18:39 user nova-compute[75234]: tcp: [32] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:cf6ab229-533a-424e-bd67-1d54d50ad4be (non-flash)
Nov 29 03:18:39 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsi_session /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Nov 29 03:18:39 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:07f83efc-272d-4e10-9a2b-48d967db613c -p 172.16.0.220:3260 --login {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:18:39 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:07f83efc-272d-4e10-9a2b-48d967db613c -p 172.16.0.220:3260 --login" returned: 0 in 0.066s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:18:39 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[f227e53e-b3af-42f2-95cf-863b960d65d7]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:07f83efc-272d-4e10-9a2b-48d967db613c, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:07f83efc-272d-4e10-9a2b-48d967db613c, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:18:39 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-e9bfef58-f895-42b7-bc9c-772ebab67bc8 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:07f83efc-272d-4e10-9a2b-48d967db613c, portal: 172.16.0.220,3260]
Nov 29 03:18:39 user nova-compute[75234]: Login to [iface: default, target: iqn.2010-10.org.openstack:07f83efc-272d-4e10-9a2b-48d967db613c, portal: 172.16.0.220,3260] successful.
Nov 29 03:18:39 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:18:39 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:07f83efc-272d-4e10-9a2b-48d967db613c -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:18:39 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:07f83efc-272d-4e10-9a2b-48d967db613c -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.019s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:18:39 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[d22cb2e5-51da-447c-9e63-99a75057c034]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:18:39 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-e9bfef58-f895-42b7-bc9c-772ebab67bc8 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:18:39 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:18:39 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.026s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:18:39 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[819daa7f-0397-41db-95d6-304b7a2fff6b]: (4, ('tcp: [29] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bf57b967-7fad-4d76-abb2-319fe88f8ef9 (non-flash)\ntcp: [31] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a5b104bd-399e-4445-a186-0d4275ae271a (non-flash)\ntcp: [32] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:cf6ab229-533a-424e-bd67-1d54d50ad4be (non-flash)\ntcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:07f83efc-272d-4e10-9a2b-48d967db613c (non-flash)\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:18:39 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-e9bfef58-f895-42b7-bc9c-772ebab67bc8 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [29] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bf57b967-7fad-4d76-abb2-319fe88f8ef9 (non-flash)
Nov 29 03:18:39 user nova-compute[75234]: tcp: [31] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a5b104bd-399e-4445-a186-0d4275ae271a (non-flash)
Nov 29 03:18:39 user nova-compute[75234]: tcp: [32] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:cf6ab229-533a-424e-bd67-1d54d50ad4be (non-flash)
Nov 29 03:18:39 user nova-compute[75234]: tcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:07f83efc-272d-4e10-9a2b-48d967db613c (non-flash)
Nov 29 03:18:39 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:18:39 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-e9bfef58-f895-42b7-bc9c-772ebab67bc8 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] iscsi session list stdout=tcp: [29] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bf57b967-7fad-4d76-abb2-319fe88f8ef9 (non-flash)
Nov 29 03:18:39 user nova-compute[75234]: tcp: [31] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a5b104bd-399e-4445-a186-0d4275ae271a (non-flash)
Nov 29 03:18:39 user nova-compute[75234]: tcp: [32] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:cf6ab229-533a-424e-bd67-1d54d50ad4be (non-flash)
Nov 29 03:18:39 user nova-compute[75234]: tcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:07f83efc-272d-4e10-9a2b-48d967db613c (non-flash)
Nov 29 03:18:39 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsi_session /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Nov 29 03:18:39 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-e9bfef58-f895-42b7-bc9c-772ebab67bc8 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:07f83efc-272d-4e10-9a2b-48d967db613c" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.316s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:87}}
Nov 29 03:18:39 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-e9bfef58-f895-42b7-bc9c-772ebab67bc8 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] Connected to 172.16.0.220:3260 {{(pid=75234) _connect_vol /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:644}}
Nov 29 03:18:39 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-e9bfef58-f895-42b7-bc9c-772ebab67bc8 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] HCTL ('34', '-', '-', 0) found on session 33 with lun 0 {{(pid=75234) get_hctl /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:770}}
Nov 29 03:18:39 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-e9bfef58-f895-42b7-bc9c-772ebab67bc8 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] Scanning host 34 c: -, t: -, l: 0) {{(pid=75234) scan_iscsi /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:795}}
Nov 29 03:18:39 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host34/scan {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:18:39 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host34/scan" returned: 0 in 0.017s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:18:39 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[29fe2030-e0d5-4287-9531-6404c2ca66e2]: (4, ('- - 0', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:18:39 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-e9bfef58-f895-42b7-bc9c-772ebab67bc8 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] Searching for a device in session 33 and hctl ['34', '*', '*', 0] yield: sdc {{(pid=75234) device_name_by_hctl /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:789}}
Nov 29 03:18:39 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-e9bfef58-f895-42b7-bc9c-772ebab67bc8 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] Connected to sdc using {'target_iqn': 'iqn.2010-10.org.openstack:07f83efc-272d-4e10-9a2b-48d967db613c', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'kRLjYCVO', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False} {{(pid=75234) _connect_vol /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:672}}
Nov 29 03:18:40 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-e9bfef58-f895-42b7-bc9c-772ebab67bc8 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.349s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:87}}
Nov 29 03:18:40 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-e9bfef58-f895-42b7-bc9c-772ebab67bc8 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] <== connect_volume: return (1351ms) {'type': 'block', 'scsi_wwn': '23066646661346337', 'path': '/dev/sdc'} {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:203}}
Nov 29 03:18:40 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-e9bfef58-f895-42b7-bc9c-772ebab67bc8 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] [instance: aa190a9f-ba53-43fb-8bae-2c429d0d1228] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '23066646661346337', 'path': '/dev/sdc'} {{(pid=75234) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Nov 29 03:18:40 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-e9bfef58-f895-42b7-bc9c-772ebab67bc8 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=True,compute_id=1,config_drive='',created_at=2023-11-29T03:16:08Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description=None,display_name='tempest-ServerActionsV293TestJSON-server-1837520572',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-serveractionsv293testjson-server-381391911',id=27,image_ref='',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBKnl09Ywqr6/srgll0CbbEeIXPcY1dWQkS0gn6SNfJJJd/3IKDSo4FDOq1mKkootcIJYqYPVMtXdp0HTuNblTn7mhW9LnhxK8ZA0M4lAvd/3jzC0f6c+1EiWTGrTU8NBjw==',key_name='tempest-keypair-1391635144',keypairs=<?>,launch_index=0,launched_at=2023-11-29T03:16:24Z,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={rebuild='server'},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=1,progress=0,project_id='581a0b62fee744e1b9a767a0d011c4e0',ramdisk_id='',reservation_id='r-eq00dfx0',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',clean_attempts='1',image_base_image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-ServerActionsV293TestJSON-155733519',owner_user_name='tempest-ServerActionsV293TestJSON-155733519-project-member'},tags=<?>,task_state='rebuild_spawning',terminated_at=None,trusted_certs=None,updated_at=2023-11-29T03:18:22Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='d11ce50c391d4e3bbe92d6c463d14746',uuid=aa190a9f-ba53-43fb-8bae-2c429d0d1228,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "2cc37e6a-5c9f-4bb8-98fa-f4f2b94d0c35", "address": "fa:16:3e:fb:40:18", "network": {"id": "38517dcf-64c7-4d26-9083-5ea5f6c87c2f", "bridge": "br-int", "label": "tempest-ServerActionsV293TestJSON-266920726-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.208", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "581a0b62fee744e1b9a767a0d011c4e0", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2cc37e6a-5c", "ovs_interfaceid": "2cc37e6a-5c9f-4bb8-98fa-f4f2b94d0c35", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=75234) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Nov 29 03:18:40 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-e9bfef58-f895-42b7-bc9c-772ebab67bc8 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] Converting VIF {"id": "2cc37e6a-5c9f-4bb8-98fa-f4f2b94d0c35", "address": "fa:16:3e:fb:40:18", "network": {"id": "38517dcf-64c7-4d26-9083-5ea5f6c87c2f", "bridge": "br-int", "label": "tempest-ServerActionsV293TestJSON-266920726-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.208", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "581a0b62fee744e1b9a767a0d011c4e0", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2cc37e6a-5c", "ovs_interfaceid": "2cc37e6a-5c9f-4bb8-98fa-f4f2b94d0c35", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:18:40 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-e9bfef58-f895-42b7-bc9c-772ebab67bc8 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:fb:40:18,bridge_name='br-int',has_traffic_filtering=True,id=2cc37e6a-5c9f-4bb8-98fa-f4f2b94d0c35,network=Network(38517dcf-64c7-4d26-9083-5ea5f6c87c2f),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap2cc37e6a-5c') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:18:40 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-e9bfef58-f895-42b7-bc9c-772ebab67bc8 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] [instance: aa190a9f-ba53-43fb-8bae-2c429d0d1228] End _get_guest_xml xml=<domain type="kvm">
Nov 29 03:18:40 user nova-compute[75234]:   <uuid>aa190a9f-ba53-43fb-8bae-2c429d0d1228</uuid>
Nov 29 03:18:40 user nova-compute[75234]:   <name>instance-0000001b</name>
Nov 29 03:18:40 user nova-compute[75234]:   <memory>524288</memory>
Nov 29 03:18:40 user nova-compute[75234]:   <vcpu>1</vcpu>
Nov 29 03:18:40 user nova-compute[75234]:   <metadata>
Nov 29 03:18:40 user nova-compute[75234]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Nov 29 03:18:40 user nova-compute[75234]:       <nova:package version="0.0.0"/>
Nov 29 03:18:40 user nova-compute[75234]:       <nova:name>tempest-ServerActionsV293TestJSON-server-1837520572</nova:name>
Nov 29 03:18:40 user nova-compute[75234]:       <nova:creationTime>2023-11-29 03:18:38</nova:creationTime>
Nov 29 03:18:40 user nova-compute[75234]:       <nova:flavor name="m1.tiny">
Nov 29 03:18:40 user nova-compute[75234]:         <nova:memory>512</nova:memory>
Nov 29 03:18:40 user nova-compute[75234]:         <nova:disk>1</nova:disk>
Nov 29 03:18:40 user nova-compute[75234]:         <nova:swap>0</nova:swap>
Nov 29 03:18:40 user nova-compute[75234]:         <nova:ephemeral>0</nova:ephemeral>
Nov 29 03:18:40 user nova-compute[75234]:         <nova:vcpus>1</nova:vcpus>
Nov 29 03:18:40 user nova-compute[75234]:       </nova:flavor>
Nov 29 03:18:40 user nova-compute[75234]:       <nova:owner>
Nov 29 03:18:40 user nova-compute[75234]:         <nova:user uuid="d11ce50c391d4e3bbe92d6c463d14746">tempest-ServerActionsV293TestJSON-155733519-project-member</nova:user>
Nov 29 03:18:40 user nova-compute[75234]:         <nova:project uuid="581a0b62fee744e1b9a767a0d011c4e0">tempest-ServerActionsV293TestJSON-155733519</nova:project>
Nov 29 03:18:40 user nova-compute[75234]:       </nova:owner>
Nov 29 03:18:40 user nova-compute[75234]:       <nova:ports>
Nov 29 03:18:40 user nova-compute[75234]:         <nova:port uuid="2cc37e6a-5c9f-4bb8-98fa-f4f2b94d0c35">
Nov 29 03:18:40 user nova-compute[75234]:           <nova:ip type="fixed" address="10.0.0.6" ipVersion="4"/>
Nov 29 03:18:40 user nova-compute[75234]:         </nova:port>
Nov 29 03:18:40 user nova-compute[75234]:       </nova:ports>
Nov 29 03:18:40 user nova-compute[75234]:     </nova:instance>
Nov 29 03:18:40 user nova-compute[75234]:   </metadata>
Nov 29 03:18:40 user nova-compute[75234]:   <sysinfo type="smbios">
Nov 29 03:18:40 user nova-compute[75234]:     <system>
Nov 29 03:18:40 user nova-compute[75234]:       <entry name="manufacturer">OpenStack Foundation</entry>
Nov 29 03:18:40 user nova-compute[75234]:       <entry name="product">OpenStack Nova</entry>
Nov 29 03:18:40 user nova-compute[75234]:       <entry name="version">0.0.0</entry>
Nov 29 03:18:40 user nova-compute[75234]:       <entry name="serial">aa190a9f-ba53-43fb-8bae-2c429d0d1228</entry>
Nov 29 03:18:40 user nova-compute[75234]:       <entry name="uuid">aa190a9f-ba53-43fb-8bae-2c429d0d1228</entry>
Nov 29 03:18:40 user nova-compute[75234]:       <entry name="family">Virtual Machine</entry>
Nov 29 03:18:40 user nova-compute[75234]:     </system>
Nov 29 03:18:40 user nova-compute[75234]:   </sysinfo>
Nov 29 03:18:40 user nova-compute[75234]:   <os>
Nov 29 03:18:40 user nova-compute[75234]:     <type arch="x86_64" machine="pc">hvm</type>
Nov 29 03:18:40 user nova-compute[75234]:     <boot dev="hd"/>
Nov 29 03:18:40 user nova-compute[75234]:     <smbios mode="sysinfo"/>
Nov 29 03:18:40 user nova-compute[75234]:   </os>
Nov 29 03:18:40 user nova-compute[75234]:   <features>
Nov 29 03:18:40 user nova-compute[75234]:     <acpi/>
Nov 29 03:18:40 user nova-compute[75234]:     <apic/>
Nov 29 03:18:40 user nova-compute[75234]:     <vmcoreinfo/>
Nov 29 03:18:40 user nova-compute[75234]:   </features>
Nov 29 03:18:40 user nova-compute[75234]:   <clock offset="utc">
Nov 29 03:18:40 user nova-compute[75234]:     <timer name="pit" tickpolicy="delay"/>
Nov 29 03:18:40 user nova-compute[75234]:     <timer name="rtc" tickpolicy="catchup"/>
Nov 29 03:18:40 user nova-compute[75234]:     <timer name="hpet" present="no"/>
Nov 29 03:18:40 user nova-compute[75234]:   </clock>
Nov 29 03:18:40 user nova-compute[75234]:   <cpu mode="custom" match="exact">
Nov 29 03:18:40 user nova-compute[75234]:     <model>Nehalem</model>
Nov 29 03:18:40 user nova-compute[75234]:     <topology sockets="1" cores="1" threads="1"/>
Nov 29 03:18:40 user nova-compute[75234]:   </cpu>
Nov 29 03:18:40 user nova-compute[75234]:   <devices>
Nov 29 03:18:40 user nova-compute[75234]:     <disk type="block" device="disk">
Nov 29 03:18:40 user nova-compute[75234]:       <driver name="qemu" type="raw" cache="none" io="native"/>
Nov 29 03:18:40 user nova-compute[75234]:       <source dev="/dev/sdc"/>
Nov 29 03:18:40 user nova-compute[75234]:       <target dev="vda" bus="virtio"/>
Nov 29 03:18:40 user nova-compute[75234]:       <serial>07f83efc-272d-4e10-9a2b-48d967db613c</serial>
Nov 29 03:18:40 user nova-compute[75234]:     </disk>
Nov 29 03:18:40 user nova-compute[75234]:     <interface type="ethernet">
Nov 29 03:18:40 user nova-compute[75234]:       <mac address="fa:16:3e:fb:40:18"/>
Nov 29 03:18:40 user nova-compute[75234]:       <model type="virtio"/>
Nov 29 03:18:40 user nova-compute[75234]:       <mtu size="1442"/>
Nov 29 03:18:40 user nova-compute[75234]:       <target dev="tap2cc37e6a-5c"/>
Nov 29 03:18:40 user nova-compute[75234]:     </interface>
Nov 29 03:18:40 user nova-compute[75234]:     <serial type="pty">
Nov 29 03:18:40 user nova-compute[75234]:       <log file="/opt/stack/data/nova/instances/aa190a9f-ba53-43fb-8bae-2c429d0d1228/console.log" append="off"/>
Nov 29 03:18:40 user nova-compute[75234]:     </serial>
Nov 29 03:18:40 user nova-compute[75234]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Nov 29 03:18:40 user nova-compute[75234]:     <video>
Nov 29 03:18:40 user nova-compute[75234]:       <model type="virtio"/>
Nov 29 03:18:40 user nova-compute[75234]:     </video>
Nov 29 03:18:40 user nova-compute[75234]:     <rng model="virtio">
Nov 29 03:18:40 user nova-compute[75234]:       <backend model="random">/dev/urandom</backend>
Nov 29 03:18:40 user nova-compute[75234]:     </rng>
Nov 29 03:18:40 user nova-compute[75234]:     <controller type="usb" index="0" model="none"/>
Nov 29 03:18:40 user nova-compute[75234]:     <memballoon model="virtio">
Nov 29 03:18:40 user nova-compute[75234]:       <stats period="10"/>
Nov 29 03:18:40 user nova-compute[75234]:     </memballoon>
Nov 29 03:18:40 user nova-compute[75234]:   </devices>
Nov 29 03:18:40 user nova-compute[75234]: </domain>
Nov 29 03:18:40 user nova-compute[75234]:  {{(pid=75234) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7568}}
Nov 29 03:18:40 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-e9bfef58-f895-42b7-bc9c-772ebab67bc8 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=True,compute_id=1,config_drive='',created_at=2023-11-29T03:16:08Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description=None,display_name='tempest-ServerActionsV293TestJSON-server-1837520572',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-serveractionsv293testjson-server-381391911',id=27,image_ref='',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBKnl09Ywqr6/srgll0CbbEeIXPcY1dWQkS0gn6SNfJJJd/3IKDSo4FDOq1mKkootcIJYqYPVMtXdp0HTuNblTn7mhW9LnhxK8ZA0M4lAvd/3jzC0f6c+1EiWTGrTU8NBjw==',key_name='tempest-keypair-1391635144',keypairs=<?>,launch_index=0,launched_at=2023-11-29T03:16:24Z,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={rebuild='server'},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=1,progress=0,project_id='581a0b62fee744e1b9a767a0d011c4e0',ramdisk_id='',reservation_id='r-eq00dfx0',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',clean_attempts='1',image_base_image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-ServerActionsV293TestJSON-155733519',owner_user_name='tempest-ServerActionsV293TestJSON-155733519-project-member'},tags=<?>,task_state='rebuild_spawning',terminated_at=None,trusted_certs=None,updated_at=2023-11-29T03:18:22Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='d11ce50c391d4e3bbe92d6c463d14746',uuid=aa190a9f-ba53-43fb-8bae-2c429d0d1228,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "2cc37e6a-5c9f-4bb8-98fa-f4f2b94d0c35", "address": "fa:16:3e:fb:40:18", "network": {"id": "38517dcf-64c7-4d26-9083-5ea5f6c87c2f", "bridge": "br-int", "label": "tempest-ServerActionsV293TestJSON-266920726-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.208", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "581a0b62fee744e1b9a767a0d011c4e0", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2cc37e6a-5c", "ovs_interfaceid": "2cc37e6a-5c9f-4bb8-98fa-f4f2b94d0c35", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Nov 29 03:18:40 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-e9bfef58-f895-42b7-bc9c-772ebab67bc8 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] Converting VIF {"id": "2cc37e6a-5c9f-4bb8-98fa-f4f2b94d0c35", "address": "fa:16:3e:fb:40:18", "network": {"id": "38517dcf-64c7-4d26-9083-5ea5f6c87c2f", "bridge": "br-int", "label": "tempest-ServerActionsV293TestJSON-266920726-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.208", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "581a0b62fee744e1b9a767a0d011c4e0", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2cc37e6a-5c", "ovs_interfaceid": "2cc37e6a-5c9f-4bb8-98fa-f4f2b94d0c35", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:18:40 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-e9bfef58-f895-42b7-bc9c-772ebab67bc8 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:fb:40:18,bridge_name='br-int',has_traffic_filtering=True,id=2cc37e6a-5c9f-4bb8-98fa-f4f2b94d0c35,network=Network(38517dcf-64c7-4d26-9083-5ea5f6c87c2f),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap2cc37e6a-5c') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:18:40 user nova-compute[75234]: DEBUG os_vif [None req-e9bfef58-f895-42b7-bc9c-772ebab67bc8 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] Plugging vif VIFOpenVSwitch(active=True,address=fa:16:3e:fb:40:18,bridge_name='br-int',has_traffic_filtering=True,id=2cc37e6a-5c9f-4bb8-98fa-f4f2b94d0c35,network=Network(38517dcf-64c7-4d26-9083-5ea5f6c87c2f),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap2cc37e6a-5c') {{(pid=75234) plug /opt/stack/data/venv/lib/python3.10/site-packages/os_vif/__init__.py:76}}
Nov 29 03:18:40 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:18:40 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:18:40 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Nov 29 03:18:40 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:18:40 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbCreateCommand(_result=None, table=QoS, columns={'type': 'linux-noop', 'external_ids': {'id': 'de15799a-23e9-5641-8ca0-ba2dd0e0ce5f', '_type': 'linux-noop'}}, row=False) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:18:40 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:18:40 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:18:40 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:18:40 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tap2cc37e6a-5c, may_exist=True, interface_attrs={}) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:18:40 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Port, record=tap2cc37e6a-5c, col_values=(('qos', UUID('c6b334b0-0f89-425a-a9b9-59b6dfe53338')),), if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:18:40 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DbSetCommand(_result=None, table=Interface, record=tap2cc37e6a-5c, col_values=(('external_ids', {'iface-id': '2cc37e6a-5c9f-4bb8-98fa-f4f2b94d0c35', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:fb:40:18', 'vm-uuid': 'aa190a9f-ba53-43fb-8bae-2c429d0d1228'}),), if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:18:40 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:18:40 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:18:40 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:18:40 user nova-compute[75234]: INFO os_vif [None req-e9bfef58-f895-42b7-bc9c-772ebab67bc8 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] Successfully plugged vif VIFOpenVSwitch(active=True,address=fa:16:3e:fb:40:18,bridge_name='br-int',has_traffic_filtering=True,id=2cc37e6a-5c9f-4bb8-98fa-f4f2b94d0c35,network=Network(38517dcf-64c7-4d26-9083-5ea5f6c87c2f),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap2cc37e6a-5c')
Nov 29 03:18:40 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-e9bfef58-f895-42b7-bc9c-772ebab67bc8 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] No BDM found with device name vda, not building metadata. {{(pid=75234) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12089}}
Nov 29 03:18:40 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-e9bfef58-f895-42b7-bc9c-772ebab67bc8 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] No VIF found with MAC fa:16:3e:fb:40:18, not building metadata {{(pid=75234) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Nov 29 03:18:41 user nova-compute[75234]: DEBUG nova.compute.manager [req-b2fc882a-1d09-4da8-8c24-2c49dc0c433f req-7aad488f-dab6-4bb8-8d52-01fea1e3fffe service nova] [instance: 5e6cca46-c5a7-4303-98c2-1ad483c6d14e] Received event network-changed-69d76626-4c79-4557-b501-7c8c92fcdc03 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:18:41 user nova-compute[75234]: DEBUG nova.compute.manager [req-b2fc882a-1d09-4da8-8c24-2c49dc0c433f req-7aad488f-dab6-4bb8-8d52-01fea1e3fffe service nova] [instance: 5e6cca46-c5a7-4303-98c2-1ad483c6d14e] Refreshing instance network info cache due to event network-changed-69d76626-4c79-4557-b501-7c8c92fcdc03. {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11117}}
Nov 29 03:18:41 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-b2fc882a-1d09-4da8-8c24-2c49dc0c433f req-7aad488f-dab6-4bb8-8d52-01fea1e3fffe service nova] Acquiring lock "refresh_cache-5e6cca46-c5a7-4303-98c2-1ad483c6d14e" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:18:41 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-b2fc882a-1d09-4da8-8c24-2c49dc0c433f req-7aad488f-dab6-4bb8-8d52-01fea1e3fffe service nova] Acquired lock "refresh_cache-5e6cca46-c5a7-4303-98c2-1ad483c6d14e" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:18:41 user nova-compute[75234]: DEBUG nova.network.neutron [req-b2fc882a-1d09-4da8-8c24-2c49dc0c433f req-7aad488f-dab6-4bb8-8d52-01fea1e3fffe service nova] [instance: 5e6cca46-c5a7-4303-98c2-1ad483c6d14e] Refreshing network info cache for port 69d76626-4c79-4557-b501-7c8c92fcdc03 {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2006}}
Nov 29 03:18:41 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:18:41 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:18:41 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:18:41 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:18:42 user nova-compute[75234]: DEBUG nova.network.neutron [req-b2fc882a-1d09-4da8-8c24-2c49dc0c433f req-7aad488f-dab6-4bb8-8d52-01fea1e3fffe service nova] [instance: 5e6cca46-c5a7-4303-98c2-1ad483c6d14e] Updated VIF entry in instance network info cache for port 69d76626-4c79-4557-b501-7c8c92fcdc03. {{(pid=75234) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3481}}
Nov 29 03:18:42 user nova-compute[75234]: DEBUG nova.network.neutron [req-b2fc882a-1d09-4da8-8c24-2c49dc0c433f req-7aad488f-dab6-4bb8-8d52-01fea1e3fffe service nova] [instance: 5e6cca46-c5a7-4303-98c2-1ad483c6d14e] Updating instance_info_cache with network_info: [{"id": "69d76626-4c79-4557-b501-7c8c92fcdc03", "address": "fa:16:3e:6e:d5:4a", "network": {"id": "f68acf2f-b01a-4481-845d-46b92646988f", "bridge": "br-int", "label": "tempest-TestServerVolumeAttachScenarioOldVersion-1316398304-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.144", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "d6f51212dd1c4632b36cfdbd7f901a89", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap69d76626-4c", "ovs_interfaceid": "69d76626-4c79-4557-b501-7c8c92fcdc03", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:18:42 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-b2fc882a-1d09-4da8-8c24-2c49dc0c433f req-7aad488f-dab6-4bb8-8d52-01fea1e3fffe service nova] Releasing lock "refresh_cache-5e6cca46-c5a7-4303-98c2-1ad483c6d14e" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:18:42 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:18:42 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:18:43 user nova-compute[75234]: DEBUG nova.compute.manager [req-e72f49f0-613d-4248-8fae-d5fdcab84df4 req-f8b41f39-95b8-4bce-9078-c8541ac9b6c4 service nova] [instance: aa190a9f-ba53-43fb-8bae-2c429d0d1228] Received event network-vif-plugged-2cc37e6a-5c9f-4bb8-98fa-f4f2b94d0c35 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:18:43 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-e72f49f0-613d-4248-8fae-d5fdcab84df4 req-f8b41f39-95b8-4bce-9078-c8541ac9b6c4 service nova] Acquiring lock "aa190a9f-ba53-43fb-8bae-2c429d0d1228-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:18:43 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-e72f49f0-613d-4248-8fae-d5fdcab84df4 req-f8b41f39-95b8-4bce-9078-c8541ac9b6c4 service nova] Lock "aa190a9f-ba53-43fb-8bae-2c429d0d1228-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:18:43 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-e72f49f0-613d-4248-8fae-d5fdcab84df4 req-f8b41f39-95b8-4bce-9078-c8541ac9b6c4 service nova] Lock "aa190a9f-ba53-43fb-8bae-2c429d0d1228-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:18:43 user nova-compute[75234]: DEBUG nova.compute.manager [req-e72f49f0-613d-4248-8fae-d5fdcab84df4 req-f8b41f39-95b8-4bce-9078-c8541ac9b6c4 service nova] [instance: aa190a9f-ba53-43fb-8bae-2c429d0d1228] No waiting events found dispatching network-vif-plugged-2cc37e6a-5c9f-4bb8-98fa-f4f2b94d0c35 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:18:43 user nova-compute[75234]: WARNING nova.compute.manager [req-e72f49f0-613d-4248-8fae-d5fdcab84df4 req-f8b41f39-95b8-4bce-9078-c8541ac9b6c4 service nova] [instance: aa190a9f-ba53-43fb-8bae-2c429d0d1228] Received unexpected event network-vif-plugged-2cc37e6a-5c9f-4bb8-98fa-f4f2b94d0c35 for instance with vm_state active and task_state rebuild_spawning.
Nov 29 03:18:43 user nova-compute[75234]: DEBUG nova.compute.manager [req-e72f49f0-613d-4248-8fae-d5fdcab84df4 req-f8b41f39-95b8-4bce-9078-c8541ac9b6c4 service nova] [instance: aa190a9f-ba53-43fb-8bae-2c429d0d1228] Received event network-vif-plugged-2cc37e6a-5c9f-4bb8-98fa-f4f2b94d0c35 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:18:43 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-e72f49f0-613d-4248-8fae-d5fdcab84df4 req-f8b41f39-95b8-4bce-9078-c8541ac9b6c4 service nova] Acquiring lock "aa190a9f-ba53-43fb-8bae-2c429d0d1228-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:18:43 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-e72f49f0-613d-4248-8fae-d5fdcab84df4 req-f8b41f39-95b8-4bce-9078-c8541ac9b6c4 service nova] Lock "aa190a9f-ba53-43fb-8bae-2c429d0d1228-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:18:43 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-e72f49f0-613d-4248-8fae-d5fdcab84df4 req-f8b41f39-95b8-4bce-9078-c8541ac9b6c4 service nova] Lock "aa190a9f-ba53-43fb-8bae-2c429d0d1228-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:18:43 user nova-compute[75234]: DEBUG nova.compute.manager [req-e72f49f0-613d-4248-8fae-d5fdcab84df4 req-f8b41f39-95b8-4bce-9078-c8541ac9b6c4 service nova] [instance: aa190a9f-ba53-43fb-8bae-2c429d0d1228] No waiting events found dispatching network-vif-plugged-2cc37e6a-5c9f-4bb8-98fa-f4f2b94d0c35 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:18:43 user nova-compute[75234]: WARNING nova.compute.manager [req-e72f49f0-613d-4248-8fae-d5fdcab84df4 req-f8b41f39-95b8-4bce-9078-c8541ac9b6c4 service nova] [instance: aa190a9f-ba53-43fb-8bae-2c429d0d1228] Received unexpected event network-vif-plugged-2cc37e6a-5c9f-4bb8-98fa-f4f2b94d0c35 for instance with vm_state active and task_state rebuild_spawning.
Nov 29 03:18:44 user nova-compute[75234]: DEBUG nova.compute.manager [None req-e9bfef58-f895-42b7-bc9c-772ebab67bc8 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] [instance: aa190a9f-ba53-43fb-8bae-2c429d0d1228] Instance event wait completed in 0 seconds for  {{(pid=75234) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Nov 29 03:18:44 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-e9bfef58-f895-42b7-bc9c-772ebab67bc8 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] [instance: aa190a9f-ba53-43fb-8bae-2c429d0d1228] Guest created on hypervisor {{(pid=75234) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4421}}
Nov 29 03:18:44 user nova-compute[75234]: INFO nova.virt.libvirt.driver [-] [instance: aa190a9f-ba53-43fb-8bae-2c429d0d1228] Instance spawned successfully.
Nov 29 03:18:44 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-e9bfef58-f895-42b7-bc9c-772ebab67bc8 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] [instance: aa190a9f-ba53-43fb-8bae-2c429d0d1228] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:904}}
Nov 29 03:18:44 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-e9bfef58-f895-42b7-bc9c-772ebab67bc8 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] [instance: aa190a9f-ba53-43fb-8bae-2c429d0d1228] Found default for hw_cdrom_bus of ide {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:18:44 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-e9bfef58-f895-42b7-bc9c-772ebab67bc8 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] [instance: aa190a9f-ba53-43fb-8bae-2c429d0d1228] Found default for hw_disk_bus of virtio {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:18:44 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-e9bfef58-f895-42b7-bc9c-772ebab67bc8 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] [instance: aa190a9f-ba53-43fb-8bae-2c429d0d1228] Found default for hw_input_bus of None {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:18:44 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-e9bfef58-f895-42b7-bc9c-772ebab67bc8 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] [instance: aa190a9f-ba53-43fb-8bae-2c429d0d1228] Found default for hw_pointer_model of None {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:18:44 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-e9bfef58-f895-42b7-bc9c-772ebab67bc8 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] [instance: aa190a9f-ba53-43fb-8bae-2c429d0d1228] Found default for hw_video_model of virtio {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:18:44 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-e9bfef58-f895-42b7-bc9c-772ebab67bc8 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] [instance: aa190a9f-ba53-43fb-8bae-2c429d0d1228] Found default for hw_vif_model of virtio {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:18:44 user nova-compute[75234]: DEBUG nova.compute.manager [None req-e9bfef58-f895-42b7-bc9c-772ebab67bc8 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] [instance: aa190a9f-ba53-43fb-8bae-2c429d0d1228] Checking state {{(pid=75234) _get_power_state /opt/stack/nova/nova/compute/manager.py:1782}}
Nov 29 03:18:44 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-e9bfef58-f895-42b7-bc9c-772ebab67bc8 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.finish_evacuation" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:18:44 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-e9bfef58-f895-42b7-bc9c-772ebab67bc8 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.finish_evacuation" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:18:44 user nova-compute[75234]: DEBUG nova.objects.instance [None req-e9bfef58-f895-42b7-bc9c-772ebab67bc8 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] [instance: aa190a9f-ba53-43fb-8bae-2c429d0d1228] Trying to apply a migration context that does not seem to be set for this instance {{(pid=75234) apply_migration_context /opt/stack/nova/nova/objects/instance.py:1067}}
Nov 29 03:18:44 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-e9bfef58-f895-42b7-bc9c-772ebab67bc8 tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.finish_evacuation" :: held 0.113s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:18:45 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-4eb5b990-3c16-4fae-9864-ad176b8e7a22 tempest-TestServerVolumeAttachScenarioOldVersion-830274084 tempest-TestServerVolumeAttachScenarioOldVersion-830274084-project-member] Acquiring lock "5e6cca46-c5a7-4303-98c2-1ad483c6d14e" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:18:45 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-4eb5b990-3c16-4fae-9864-ad176b8e7a22 tempest-TestServerVolumeAttachScenarioOldVersion-830274084 tempest-TestServerVolumeAttachScenarioOldVersion-830274084-project-member] Lock "5e6cca46-c5a7-4303-98c2-1ad483c6d14e" acquired by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:18:45 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:18:45 user nova-compute[75234]: DEBUG nova.objects.instance [None req-4eb5b990-3c16-4fae-9864-ad176b8e7a22 tempest-TestServerVolumeAttachScenarioOldVersion-830274084 tempest-TestServerVolumeAttachScenarioOldVersion-830274084-project-member] Lazy-loading 'flavor' on Instance uuid 5e6cca46-c5a7-4303-98c2-1ad483c6d14e {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:18:45 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-4eb5b990-3c16-4fae-9864-ad176b8e7a22 tempest-TestServerVolumeAttachScenarioOldVersion-830274084 tempest-TestServerVolumeAttachScenarioOldVersion-830274084-project-member] Lock "5e6cca46-c5a7-4303-98c2-1ad483c6d14e" "released" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: held 0.676s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:18:46 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-4eb5b990-3c16-4fae-9864-ad176b8e7a22 tempest-TestServerVolumeAttachScenarioOldVersion-830274084 tempest-TestServerVolumeAttachScenarioOldVersion-830274084-project-member] Acquiring lock "5e6cca46-c5a7-4303-98c2-1ad483c6d14e" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:18:46 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-4eb5b990-3c16-4fae-9864-ad176b8e7a22 tempest-TestServerVolumeAttachScenarioOldVersion-830274084 tempest-TestServerVolumeAttachScenarioOldVersion-830274084-project-member] Lock "5e6cca46-c5a7-4303-98c2-1ad483c6d14e" acquired by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:18:46 user nova-compute[75234]: INFO nova.compute.manager [None req-4eb5b990-3c16-4fae-9864-ad176b8e7a22 tempest-TestServerVolumeAttachScenarioOldVersion-830274084 tempest-TestServerVolumeAttachScenarioOldVersion-830274084-project-member] [instance: 5e6cca46-c5a7-4303-98c2-1ad483c6d14e] Attaching volume a06ab71b-02cb-4c8a-8e7b-26d8176cddf6 to /dev/vdb
Nov 29 03:18:46 user nova-compute[75234]: DEBUG os_brick.utils [None req-4eb5b990-3c16-4fae-9864-ad176b8e7a22 tempest-TestServerVolumeAttachScenarioOldVersion-830274084 tempest-TestServerVolumeAttachScenarioOldVersion-830274084-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:176}}
Nov 29 03:18:46 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:18:46 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.013s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:18:46 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[f865125e-00ae-4670-bccf-577603760076]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:dd1368c5ba2c\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:18:46 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt -v / -n -o SOURCE {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:18:46 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt -v / -n -o SOURCE" returned: 0 in 0.015s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:18:46 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[19da6f17-275e-4bfe-b2cf-73dd72f1d0eb]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:18:46 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:18:46 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.016s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:18:46 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[0355b652-fd87-40a4-b087-e3208d6e2bdf]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:18:46 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[f25e3d83-6c78-48f0-be56-d123207d4809]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:18:46 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-4eb5b990-3c16-4fae-9864-ad176b8e7a22 tempest-TestServerVolumeAttachScenarioOldVersion-830274084 tempest-TestServerVolumeAttachScenarioOldVersion-830274084-project-member] Running cmd (subprocess): nvme version {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:18:46 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-4eb5b990-3c16-4fae-9864-ad176b8e7a22 tempest-TestServerVolumeAttachScenarioOldVersion-830274084 tempest-TestServerVolumeAttachScenarioOldVersion-830274084-project-member] 'nvme version' failed. Not Retrying. {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:473}}
Nov 29 03:18:46 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.nvmeof [None req-4eb5b990-3c16-4fae-9864-ad176b8e7a22 tempest-TestServerVolumeAttachScenarioOldVersion-830274084 tempest-TestServerVolumeAttachScenarioOldVersion-830274084-project-member] nvme not present on system {{(pid=75234) nvme_present /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/nvmeof.py:756}}
Nov 29 03:18:46 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:18:46 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:473}}
Nov 29 03:18:46 user nova-compute[75234]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Nov 29 03:18:46 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[2b4ccb32-3d6f-4b07-9c33-2171385ec6e1]: (4, '') {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:18:46 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.lightos [None req-4eb5b990-3c16-4fae-9864-ad176b8e7a22 tempest-TestServerVolumeAttachScenarioOldVersion-830274084 tempest-TestServerVolumeAttachScenarioOldVersion-830274084-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=75234) find_dsc /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/lightos.py:98}}
Nov 29 03:18:46 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.lightos [None req-4eb5b990-3c16-4fae-9864-ad176b8e7a22 tempest-TestServerVolumeAttachScenarioOldVersion-830274084 tempest-TestServerVolumeAttachScenarioOldVersion-830274084-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=75234) get_connector_properties /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/lightos.py:76}}
Nov 29 03:18:46 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.lightos [None req-4eb5b990-3c16-4fae-9864-ad176b8e7a22 tempest-TestServerVolumeAttachScenarioOldVersion-830274084 tempest-TestServerVolumeAttachScenarioOldVersion-830274084-project-member] LIGHTOS: no hostnqn found. {{(pid=75234) get_connector_properties /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/lightos.py:84}}
Nov 29 03:18:46 user nova-compute[75234]: DEBUG os_brick.utils [None req-4eb5b990-3c16-4fae-9864-ad176b8e7a22 tempest-TestServerVolumeAttachScenarioOldVersion-830274084 tempest-TestServerVolumeAttachScenarioOldVersion-830274084-project-member] <== get_connector_properties: return (132ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:dd1368c5ba2c', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:203}}
Nov 29 03:18:46 user nova-compute[75234]: DEBUG nova.virt.block_device [None req-4eb5b990-3c16-4fae-9864-ad176b8e7a22 tempest-TestServerVolumeAttachScenarioOldVersion-830274084 tempest-TestServerVolumeAttachScenarioOldVersion-830274084-project-member] [instance: 5e6cca46-c5a7-4303-98c2-1ad483c6d14e] Updating existing volume attachment record: 31714965-89cd-4b9e-990c-7031e38964fc {{(pid=75234) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Nov 29 03:18:46 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:18:47 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-1c44c857-3624-41cb-8724-98ef31810a22 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Waiting for libvirt event about the detach of device vdb with device alias virtio-disk1 from instance 1c8b7559-29ab-4982-8c75-ea2a0f0101cb is timed out.
Nov 29 03:18:47 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-1c44c857-3624-41cb-8724-98ef31810a22 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Successfully detached device vdb from instance 1c8b7559-29ab-4982-8c75-ea2a0f0101cb from the live domain config.
Nov 29 03:18:47 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-1c44c857-3624-41cb-8724-98ef31810a22 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] [instance: 1c8b7559-29ab-4982-8c75-ea2a0f0101cb] calling os-brick to detach iSCSI Volume {{(pid=75234) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Nov 29 03:18:47 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-1c44c857-3624-41cb-8724-98ef31810a22 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f8a5050c9a0>, {'target_iqn': 'iqn.2010-10.org.openstack:a5b104bd-399e-4445-a186-0d4275ae271a', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'XxdoDMRn', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdb'}, None), 'kwargs': False}}" {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:176}}
Nov 29 03:18:47 user nova-compute[75234]: WARNING os_brick.initiator.connectors.base [None req-1c44c857-3624-41cb-8724-98ef31810a22 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Nov 29 03:18:47 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-1c44c857-3624-41cb-8724-98ef31810a22 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:68}}
Nov 29 03:18:47 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-1c44c857-3624-41cb-8724-98ef31810a22 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.003s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:73}}
Nov 29 03:18:47 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:18:47 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.021s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:18:47 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[7085b8ca-01f7-4841-a01e-96a6b6ef51f3]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:07f83efc-272d-4e10-9a2b-48d967db613c\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:cf6ab229-533a-424e-bd67-1d54d50ad4be\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:bf57b967-7fad-4d76-abb2-319fe88f8ef9\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:a5b104bd-399e-4445-a186-0d4275ae271a\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:18:47 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-1c44c857-3624-41cb-8724-98ef31810a22 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS:
Nov 29 03:18:47 user nova-compute[75234]: No targets found.
Nov 29 03:18:47 user nova-compute[75234]: iSNS:
Nov 29 03:18:47 user nova-compute[75234]: No targets found.
Nov 29 03:18:47 user nova-compute[75234]: STATIC:
Nov 29 03:18:47 user nova-compute[75234]: Target: iqn.2010-10.org.openstack:07f83efc-272d-4e10-9a2b-48d967db613c
Nov 29 03:18:47 user nova-compute[75234]:         Portal: 172.16.0.220:3260,-1
Nov 29 03:18:47 user nova-compute[75234]:                 Iface Name: default
Nov 29 03:18:47 user nova-compute[75234]: Target: iqn.2010-10.org.openstack:cf6ab229-533a-424e-bd67-1d54d50ad4be
Nov 29 03:18:47 user nova-compute[75234]:         Portal: 172.16.0.220:3260,-1
Nov 29 03:18:47 user nova-compute[75234]:                 Iface Name: default
Nov 29 03:18:47 user nova-compute[75234]: Target: iqn.2010-10.org.openstack:bf57b967-7fad-4d76-abb2-319fe88f8ef9
Nov 29 03:18:47 user nova-compute[75234]:         Portal: 172.16.0.220:3260,-1
Nov 29 03:18:47 user nova-compute[75234]:                 Iface Name: default
Nov 29 03:18:47 user nova-compute[75234]: Target: iqn.2010-10.org.openstack:a5b104bd-399e-4445-a186-0d4275ae271a
Nov 29 03:18:47 user nova-compute[75234]:         Portal: 172.16.0.220:3260,-1
Nov 29 03:18:47 user nova-compute[75234]:                 Iface Name: default
Nov 29 03:18:47 user nova-compute[75234]: FIRMWARE:
Nov 29 03:18:47 user nova-compute[75234]: No targets found.
Nov 29 03:18:47 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:18:47 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-1c44c857-3624-41cb-8724-98ef31810a22 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Regex to get portals from discoverydb: ^SENDTARGETS:
Nov 29 03:18:47 user nova-compute[75234]: .*?^DiscoveryAddress: 172.16.0.220,3260.*?
Nov 29 03:18:47 user nova-compute[75234]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=75234) _get_discoverydb_portals /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:382}}
Nov 29 03:18:47 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-1c44c857-3624-41cb-8724-98ef31810a22 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:a5b104bd-399e-4445-a186-0d4275ae271a', 0)] {{(pid=75234) _get_connection_devices /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:830}}
Nov 29 03:18:47 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:18:47 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.024s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:18:47 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[c6d7403f-d84b-489e-8362-501a5bebd2bf]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:07f83efc-272d-4e10-9a2b-48d967db613c\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:a5b104bd-399e-4445-a186-0d4275ae271a\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:bf57b967-7fad-4d76-abb2-319fe88f8ef9\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:cf6ab229-533a-424e-bd67-1d54d50ad4be\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:18:47 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:18:47 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.029s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:18:47 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[3c178413-06f9-4232-a61d-17ea401d6b8c]: (4, ('tcp: [29] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bf57b967-7fad-4d76-abb2-319fe88f8ef9 (non-flash)\ntcp: [31] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a5b104bd-399e-4445-a186-0d4275ae271a (non-flash)\ntcp: [32] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:cf6ab229-533a-424e-bd67-1d54d50ad4be (non-flash)\ntcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:07f83efc-272d-4e10-9a2b-48d967db613c (non-flash)\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:18:47 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-1c44c857-3624-41cb-8724-98ef31810a22 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [29] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bf57b967-7fad-4d76-abb2-319fe88f8ef9 (non-flash)
Nov 29 03:18:47 user nova-compute[75234]: tcp: [31] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a5b104bd-399e-4445-a186-0d4275ae271a (non-flash)
Nov 29 03:18:47 user nova-compute[75234]: tcp: [32] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:cf6ab229-533a-424e-bd67-1d54d50ad4be (non-flash)
Nov 29 03:18:47 user nova-compute[75234]: tcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:07f83efc-272d-4e10-9a2b-48d967db613c (non-flash)
Nov 29 03:18:47 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:18:47 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-1c44c857-3624-41cb-8724-98ef31810a22 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] iscsi session list stdout=tcp: [29] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bf57b967-7fad-4d76-abb2-319fe88f8ef9 (non-flash)
Nov 29 03:18:47 user nova-compute[75234]: tcp: [31] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a5b104bd-399e-4445-a186-0d4275ae271a (non-flash)
Nov 29 03:18:47 user nova-compute[75234]: tcp: [32] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:cf6ab229-533a-424e-bd67-1d54d50ad4be (non-flash)
Nov 29 03:18:47 user nova-compute[75234]: tcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:07f83efc-272d-4e10-9a2b-48d967db613c (non-flash)
Nov 29 03:18:47 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsi_session /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Nov 29 03:18:47 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-1c44c857-3624-41cb-8724-98ef31810a22 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7f8a6037e050>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:a5b104bd-399e-4445-a186-0d4275ae271a'): ({'sdb'}, set())}) {{(pid=75234) _get_connection_devices /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:863}}
Nov 29 03:18:47 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-1c44c857-3624-41cb-8724-98ef31810a22 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Removing single pathed devices sdb {{(pid=75234) remove_connection /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:374}}
Nov 29 03:18:47 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:18:47 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.026s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:18:47 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[d0b51463-5e6b-4f92-9f89-8142a0893184]: (4, ('path checker states:\nup                  4\n\npaths: 0\nbusy: False\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:18:47 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdb {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:18:47 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdb" returned: 0 in 0.021s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:18:47 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[dc893436-16ec-47da-9127-1d46ef38c6c9]: (4, ('ok\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:18:47 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-1c44c857-3624-41cb-8724-98ef31810a22 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Flushing IO for device /dev/sdb {{(pid=75234) flush_device_io /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:434}}
Nov 29 03:18:47 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdb {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:18:47 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdb" returned: 0 in 0.018s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:18:47 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[843e2d50-8be5-477f-a7b5-1ca2234ad049]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:18:47 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-1c44c857-3624-41cb-8724-98ef31810a22 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Remove SCSI device /dev/sdb with /sys/block/sdb/device/delete {{(pid=75234) remove_scsi_device /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:148}}
Nov 29 03:18:47 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdb/device/delete {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:18:47 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdb/device/delete" returned: 0 in 0.041s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:18:47 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[a8f80f4c-6ced-4f81-9ac8-0f030e0b427c]: (4, ('1', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:18:47 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-1c44c857-3624-41cb-8724-98ef31810a22 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Checking to see if SCSI volumes sdb have been removed. {{(pid=75234) wait_for_volumes_removal /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:156}}
Nov 29 03:18:47 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-1c44c857-3624-41cb-8724-98ef31810a22 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] SCSI volumes sdb have been removed. {{(pid=75234) wait_for_volumes_removal /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:166}}
Nov 29 03:18:47 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-1c44c857-3624-41cb-8724-98ef31810a22 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:a5b104bd-399e-4445-a186-0d4275ae271a')] {{(pid=75234) _disconnect_connection /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Nov 29 03:18:47 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:a5b104bd-399e-4445-a186-0d4275ae271a -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:18:47 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:a5b104bd-399e-4445-a186-0d4275ae271a -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.025s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:18:47 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[5767bef3-7166-4450-a2ba-e16b1b77ef33]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:18:47 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-1c44c857-3624-41cb-8724-98ef31810a22 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:18:47 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:a5b104bd-399e-4445-a186-0d4275ae271a -p 172.16.0.220:3260 --logout {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:18:47 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:a5b104bd-399e-4445-a186-0d4275ae271a -p 172.16.0.220:3260 --logout" returned: 0 in 0.080s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:18:47 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[b2dc6149-de9d-4539-9224-cfc5750bbf89]: (4, ('Logging out of session [sid: 31, target: iqn.2010-10.org.openstack:a5b104bd-399e-4445-a186-0d4275ae271a, portal: 172.16.0.220,3260]\nLogout of [sid: 31, target: iqn.2010-10.org.openstack:a5b104bd-399e-4445-a186-0d4275ae271a, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:18:47 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-1c44c857-3624-41cb-8724-98ef31810a22 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 31, target: iqn.2010-10.org.openstack:a5b104bd-399e-4445-a186-0d4275ae271a, portal: 172.16.0.220,3260]
Nov 29 03:18:47 user nova-compute[75234]: Logout of [sid: 31, target: iqn.2010-10.org.openstack:a5b104bd-399e-4445-a186-0d4275ae271a, portal: 172.16.0.220,3260] successful.
Nov 29 03:18:47 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:18:47 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:a5b104bd-399e-4445-a186-0d4275ae271a -p 172.16.0.220:3260 --op delete {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:18:47 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:a5b104bd-399e-4445-a186-0d4275ae271a -p 172.16.0.220:3260 --op delete" returned: 0 in 0.031s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:18:47 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[4361af9c-f27c-4d54-97fe-2e3a241dfa83]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:18:47 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-1c44c857-3624-41cb-8724-98ef31810a22 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:18:47 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-1c44c857-3624-41cb-8724-98ef31810a22 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.376s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:87}}
Nov 29 03:18:47 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-1c44c857-3624-41cb-8724-98ef31810a22 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] <== disconnect_volume: return (378ms) None {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:203}}
Nov 29 03:18:47 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-1c44c857-3624-41cb-8724-98ef31810a22 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] [instance: 1c8b7559-29ab-4982-8c75-ea2a0f0101cb] Disconnected iSCSI Volume {{(pid=75234) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:79}}
Nov 29 03:18:50 user nova-compute[75234]: DEBUG nova.compute.manager [req-b872f006-7f02-4a4a-85bf-fc6481cede72 req-c7ae5e3d-8ab2-43c8-b683-8463ac0f0db4 service nova] [instance: aaf1c3e3-b73e-42ad-87bd-c5255ec09b46] Received event network-changed-0013653c-9540-4049-91a5-43bfba2d8fa3 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:18:50 user nova-compute[75234]: DEBUG nova.compute.manager [req-b872f006-7f02-4a4a-85bf-fc6481cede72 req-c7ae5e3d-8ab2-43c8-b683-8463ac0f0db4 service nova] [instance: aaf1c3e3-b73e-42ad-87bd-c5255ec09b46] Refreshing instance network info cache due to event network-changed-0013653c-9540-4049-91a5-43bfba2d8fa3. {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11117}}
Nov 29 03:18:50 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-b872f006-7f02-4a4a-85bf-fc6481cede72 req-c7ae5e3d-8ab2-43c8-b683-8463ac0f0db4 service nova] Acquiring lock "refresh_cache-aaf1c3e3-b73e-42ad-87bd-c5255ec09b46" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:18:50 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-b872f006-7f02-4a4a-85bf-fc6481cede72 req-c7ae5e3d-8ab2-43c8-b683-8463ac0f0db4 service nova] Acquired lock "refresh_cache-aaf1c3e3-b73e-42ad-87bd-c5255ec09b46" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:18:50 user nova-compute[75234]: DEBUG nova.network.neutron [req-b872f006-7f02-4a4a-85bf-fc6481cede72 req-c7ae5e3d-8ab2-43c8-b683-8463ac0f0db4 service nova] [instance: aaf1c3e3-b73e-42ad-87bd-c5255ec09b46] Refreshing network info cache for port 0013653c-9540-4049-91a5-43bfba2d8fa3 {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2006}}
Nov 29 03:18:50 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:18:50 user nova-compute[75234]: DEBUG nova.network.neutron [req-b872f006-7f02-4a4a-85bf-fc6481cede72 req-c7ae5e3d-8ab2-43c8-b683-8463ac0f0db4 service nova] [instance: aaf1c3e3-b73e-42ad-87bd-c5255ec09b46] Updated VIF entry in instance network info cache for port 0013653c-9540-4049-91a5-43bfba2d8fa3. {{(pid=75234) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3481}}
Nov 29 03:18:50 user nova-compute[75234]: DEBUG nova.network.neutron [req-b872f006-7f02-4a4a-85bf-fc6481cede72 req-c7ae5e3d-8ab2-43c8-b683-8463ac0f0db4 service nova] [instance: aaf1c3e3-b73e-42ad-87bd-c5255ec09b46] Updating instance_info_cache with network_info: [{"id": "0013653c-9540-4049-91a5-43bfba2d8fa3", "address": "fa:16:3e:55:a5:13", "network": {"id": "294c3aac-8c9b-45f0-8c0a-344e2d7d8b7c", "bridge": "br-int", "label": "tempest-AttachVolumeTestJSON-700098935-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.209", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "e393d23e783b4a40986674d782606c69", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap0013653c-95", "ovs_interfaceid": "0013653c-9540-4049-91a5-43bfba2d8fa3", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:18:50 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-b872f006-7f02-4a4a-85bf-fc6481cede72 req-c7ae5e3d-8ab2-43c8-b683-8463ac0f0db4 service nova] Releasing lock "refresh_cache-aaf1c3e3-b73e-42ad-87bd-c5255ec09b46" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:18:51 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-105ba571-0ae2-40f0-9809-503c98c59bdb tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] Acquiring lock "d3b90f39-4b91-4152-b820-bd0e263eec89" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:18:51 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-105ba571-0ae2-40f0-9809-503c98c59bdb tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] Lock "d3b90f39-4b91-4152-b820-bd0e263eec89" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:18:51 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-105ba571-0ae2-40f0-9809-503c98c59bdb tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] Acquiring lock "d3b90f39-4b91-4152-b820-bd0e263eec89-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:18:51 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-105ba571-0ae2-40f0-9809-503c98c59bdb tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] Lock "d3b90f39-4b91-4152-b820-bd0e263eec89-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:18:51 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-105ba571-0ae2-40f0-9809-503c98c59bdb tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] Lock "d3b90f39-4b91-4152-b820-bd0e263eec89-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:18:51 user nova-compute[75234]: INFO nova.compute.manager [None req-105ba571-0ae2-40f0-9809-503c98c59bdb tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] [instance: d3b90f39-4b91-4152-b820-bd0e263eec89] Terminating instance
Nov 29 03:18:51 user nova-compute[75234]: DEBUG nova.compute.manager [None req-105ba571-0ae2-40f0-9809-503c98c59bdb tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] [instance: d3b90f39-4b91-4152-b820-bd0e263eec89] Start destroying the instance on the hypervisor. {{(pid=75234) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3140}}
Nov 29 03:18:51 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:18:51 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:18:51 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:18:51 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:18:52 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:18:52 user nova-compute[75234]: DEBUG nova.compute.manager [req-9e526326-f93e-47cc-a491-2c58b272f40b req-abcc430b-774a-4591-8a0b-3b3edbee26a1 service nova] [instance: d3b90f39-4b91-4152-b820-bd0e263eec89] Received event network-vif-unplugged-83eb52bb-5be5-49d6-81ec-86a21dde3df2 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:18:52 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-9e526326-f93e-47cc-a491-2c58b272f40b req-abcc430b-774a-4591-8a0b-3b3edbee26a1 service nova] Acquiring lock "d3b90f39-4b91-4152-b820-bd0e263eec89-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:18:52 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-9e526326-f93e-47cc-a491-2c58b272f40b req-abcc430b-774a-4591-8a0b-3b3edbee26a1 service nova] Lock "d3b90f39-4b91-4152-b820-bd0e263eec89-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:18:52 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-9e526326-f93e-47cc-a491-2c58b272f40b req-abcc430b-774a-4591-8a0b-3b3edbee26a1 service nova] Lock "d3b90f39-4b91-4152-b820-bd0e263eec89-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:18:52 user nova-compute[75234]: DEBUG nova.compute.manager [req-9e526326-f93e-47cc-a491-2c58b272f40b req-abcc430b-774a-4591-8a0b-3b3edbee26a1 service nova] [instance: d3b90f39-4b91-4152-b820-bd0e263eec89] No waiting events found dispatching network-vif-unplugged-83eb52bb-5be5-49d6-81ec-86a21dde3df2 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:18:52 user nova-compute[75234]: DEBUG nova.compute.manager [req-9e526326-f93e-47cc-a491-2c58b272f40b req-abcc430b-774a-4591-8a0b-3b3edbee26a1 service nova] [instance: d3b90f39-4b91-4152-b820-bd0e263eec89] Received event network-vif-unplugged-83eb52bb-5be5-49d6-81ec-86a21dde3df2 for instance with task_state deleting. {{(pid=75234) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10890}}
Nov 29 03:18:52 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:18:52 user nova-compute[75234]: INFO nova.virt.libvirt.driver [-] [instance: d3b90f39-4b91-4152-b820-bd0e263eec89] Instance destroyed successfully.
Nov 29 03:18:52 user nova-compute[75234]: DEBUG nova.objects.instance [None req-105ba571-0ae2-40f0-9809-503c98c59bdb tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] Lazy-loading 'resources' on Instance uuid d3b90f39-4b91-4152-b820-bd0e263eec89 {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:18:52 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-105ba571-0ae2-40f0-9809-503c98c59bdb tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-11-29T03:10:24Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-VolumesAdminNegativeTest-server-846124845',display_name='tempest-VolumesAdminNegativeTest-server-846124845',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-volumesadminnegativetest-server-846124845',id=17,image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBOhY8TGmPVW2eahQ/7NwxuWatHHhSisv4kKa49hvwZ8KZoBzAoVyERO5tPD8b1ELtbNUxo/5KbqYKeCg6v3iRzJlbCyhkM5yEZuV+qUhHx8TePthXUgsgJEbpNvQr1IWCw==',key_name='tempest-keypair-534984900',keypairs=<?>,launch_index=0,launched_at=2023-11-29T03:10:37Z,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='5eaa91ebc2e64f2eba987f02bcd1a9f4',ramdisk_id='',reservation_id='r-2l9d0jow',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-VolumesAdminNegativeTest-1445338761',owner_user_name='tempest-VolumesAdminNegativeTest-1445338761-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-11-29T03:10:37Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='460854be8e05408a80790620f64cf8e1',uuid=d3b90f39-4b91-4152-b820-bd0e263eec89,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "83eb52bb-5be5-49d6-81ec-86a21dde3df2", "address": "fa:16:3e:9d:70:dd", "network": {"id": "49d62bf6-34ee-4de8-ab85-6ef489d957c8", "bridge": "br-int", "label": "tempest-VolumesAdminNegativeTest-1304144113-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.107", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "5eaa91ebc2e64f2eba987f02bcd1a9f4", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap83eb52bb-5b", "ovs_interfaceid": "83eb52bb-5be5-49d6-81ec-86a21dde3df2", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Nov 29 03:18:52 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-105ba571-0ae2-40f0-9809-503c98c59bdb tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] Converting VIF {"id": "83eb52bb-5be5-49d6-81ec-86a21dde3df2", "address": "fa:16:3e:9d:70:dd", "network": {"id": "49d62bf6-34ee-4de8-ab85-6ef489d957c8", "bridge": "br-int", "label": "tempest-VolumesAdminNegativeTest-1304144113-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.12", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.107", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "5eaa91ebc2e64f2eba987f02bcd1a9f4", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap83eb52bb-5b", "ovs_interfaceid": "83eb52bb-5be5-49d6-81ec-86a21dde3df2", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:18:52 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-105ba571-0ae2-40f0-9809-503c98c59bdb tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:9d:70:dd,bridge_name='br-int',has_traffic_filtering=True,id=83eb52bb-5be5-49d6-81ec-86a21dde3df2,network=Network(49d62bf6-34ee-4de8-ab85-6ef489d957c8),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap83eb52bb-5b') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:18:52 user nova-compute[75234]: DEBUG os_vif [None req-105ba571-0ae2-40f0-9809-503c98c59bdb tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] Unplugging vif VIFOpenVSwitch(active=True,address=fa:16:3e:9d:70:dd,bridge_name='br-int',has_traffic_filtering=True,id=83eb52bb-5be5-49d6-81ec-86a21dde3df2,network=Network(49d62bf6-34ee-4de8-ab85-6ef489d957c8),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap83eb52bb-5b') {{(pid=75234) unplug /opt/stack/data/venv/lib/python3.10/site-packages/os_vif/__init__.py:109}}
Nov 29 03:18:52 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:18:52 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tap83eb52bb-5b, bridge=br-int, if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:18:52 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:18:52 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:18:52 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:18:52 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbDestroyCommand(_result=None, table=QoS, record=c5f89983-4f48-4826-b11b-d1c063cdaea7) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:18:52 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:18:52 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:18:52 user nova-compute[75234]: INFO os_vif [None req-105ba571-0ae2-40f0-9809-503c98c59bdb tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] Successfully unplugged vif VIFOpenVSwitch(active=True,address=fa:16:3e:9d:70:dd,bridge_name='br-int',has_traffic_filtering=True,id=83eb52bb-5be5-49d6-81ec-86a21dde3df2,network=Network(49d62bf6-34ee-4de8-ab85-6ef489d957c8),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap83eb52bb-5b')
Nov 29 03:18:52 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-105ba571-0ae2-40f0-9809-503c98c59bdb tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] [instance: d3b90f39-4b91-4152-b820-bd0e263eec89] Deleting instance files /opt/stack/data/nova/instances/d3b90f39-4b91-4152-b820-bd0e263eec89_del
Nov 29 03:18:52 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-105ba571-0ae2-40f0-9809-503c98c59bdb tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] [instance: d3b90f39-4b91-4152-b820-bd0e263eec89] Deletion of /opt/stack/data/nova/instances/d3b90f39-4b91-4152-b820-bd0e263eec89_del complete
Nov 29 03:18:52 user nova-compute[75234]: INFO nova.compute.manager [None req-105ba571-0ae2-40f0-9809-503c98c59bdb tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] [instance: d3b90f39-4b91-4152-b820-bd0e263eec89] Took 0.99 seconds to destroy the instance on the hypervisor.
Nov 29 03:18:52 user nova-compute[75234]: DEBUG oslo.service.loopingcall [None req-105ba571-0ae2-40f0-9809-503c98c59bdb tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=75234) func /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/loopingcall.py:435}}
Nov 29 03:18:52 user nova-compute[75234]: DEBUG nova.compute.manager [-] [instance: d3b90f39-4b91-4152-b820-bd0e263eec89] Deallocating network for instance {{(pid=75234) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2279}}
Nov 29 03:18:52 user nova-compute[75234]: DEBUG nova.network.neutron [-] [instance: d3b90f39-4b91-4152-b820-bd0e263eec89] deallocate_for_instance() {{(pid=75234) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1802}}
Nov 29 03:18:53 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-0f1135d4-0472-46b4-8d7c-46d1e239a06d tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Acquiring lock "aaf1c3e3-b73e-42ad-87bd-c5255ec09b46" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:18:53 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-0f1135d4-0472-46b4-8d7c-46d1e239a06d tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Lock "aaf1c3e3-b73e-42ad-87bd-c5255ec09b46" acquired by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:18:53 user nova-compute[75234]: DEBUG nova.objects.instance [None req-0f1135d4-0472-46b4-8d7c-46d1e239a06d tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Lazy-loading 'flavor' on Instance uuid aaf1c3e3-b73e-42ad-87bd-c5255ec09b46 {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:18:53 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-0f1135d4-0472-46b4-8d7c-46d1e239a06d tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Lock "aaf1c3e3-b73e-42ad-87bd-c5255ec09b46" "released" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: held 0.085s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:18:53 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-0f1135d4-0472-46b4-8d7c-46d1e239a06d tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Acquiring lock "aaf1c3e3-b73e-42ad-87bd-c5255ec09b46" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:18:53 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-0f1135d4-0472-46b4-8d7c-46d1e239a06d tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Lock "aaf1c3e3-b73e-42ad-87bd-c5255ec09b46" acquired by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:18:53 user nova-compute[75234]: INFO nova.compute.manager [None req-0f1135d4-0472-46b4-8d7c-46d1e239a06d tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] [instance: aaf1c3e3-b73e-42ad-87bd-c5255ec09b46] Attaching volume 7a1a1247-abe3-4a99-a7cb-f168fb2cb0b0 to /dev/vdb
Nov 29 03:18:53 user nova-compute[75234]: DEBUG nova.network.neutron [-] [instance: d3b90f39-4b91-4152-b820-bd0e263eec89] Updating instance_info_cache with network_info: [] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:18:53 user nova-compute[75234]: INFO nova.compute.manager [-] [instance: d3b90f39-4b91-4152-b820-bd0e263eec89] Took 1.03 seconds to deallocate network for instance.
Nov 29 03:18:53 user nova-compute[75234]: DEBUG nova.compute.manager [req-f34d5917-7df8-4995-afba-5d9e858a8419 req-58046817-17ef-4cb2-b730-1cea1eca59f5 service nova] [instance: d3b90f39-4b91-4152-b820-bd0e263eec89] Received event network-vif-deleted-83eb52bb-5be5-49d6-81ec-86a21dde3df2 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:18:53 user nova-compute[75234]: DEBUG os_brick.utils [None req-0f1135d4-0472-46b4-8d7c-46d1e239a06d tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:176}}
Nov 29 03:18:53 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-105ba571-0ae2-40f0-9809-503c98c59bdb tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:18:53 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-105ba571-0ae2-40f0-9809-503c98c59bdb tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:18:53 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:18:53 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.018s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:18:53 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[945b7595-fc52-46cf-a6dc-4f0066559ebc]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:dd1368c5ba2c\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:18:53 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt -v / -n -o SOURCE {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:18:53 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt -v / -n -o SOURCE" returned: 0 in 0.021s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:18:53 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[611c32db-df14-4e01-8c1e-10f369953565]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:18:53 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:18:53 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.019s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:18:53 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[8346f6dd-422a-467a-8379-eb363fcca8f5]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:18:53 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[9837da22-a0d8-4f54-b5e6-9c957ef64831]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:18:53 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-0f1135d4-0472-46b4-8d7c-46d1e239a06d tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Running cmd (subprocess): nvme version {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:18:54 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-0f1135d4-0472-46b4-8d7c-46d1e239a06d tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] 'nvme version' failed. Not Retrying. {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:473}}
Nov 29 03:18:54 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.nvmeof [None req-0f1135d4-0472-46b4-8d7c-46d1e239a06d tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] nvme not present on system {{(pid=75234) nvme_present /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/nvmeof.py:756}}
Nov 29 03:18:54 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:18:54 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:473}}
Nov 29 03:18:54 user nova-compute[75234]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Nov 29 03:18:54 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[24f061f0-e584-492c-816e-c0757df0198d]: (4, '') {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:18:54 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.lightos [None req-0f1135d4-0472-46b4-8d7c-46d1e239a06d tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=75234) find_dsc /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/lightos.py:98}}
Nov 29 03:18:54 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.lightos [None req-0f1135d4-0472-46b4-8d7c-46d1e239a06d tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=75234) get_connector_properties /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/lightos.py:76}}
Nov 29 03:18:54 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.lightos [None req-0f1135d4-0472-46b4-8d7c-46d1e239a06d tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] LIGHTOS: no hostnqn found. {{(pid=75234) get_connector_properties /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/lightos.py:84}}
Nov 29 03:18:54 user nova-compute[75234]: DEBUG os_brick.utils [None req-0f1135d4-0472-46b4-8d7c-46d1e239a06d tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] <== get_connector_properties: return (156ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:dd1368c5ba2c', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:203}}
Nov 29 03:18:54 user nova-compute[75234]: DEBUG nova.virt.block_device [None req-0f1135d4-0472-46b4-8d7c-46d1e239a06d tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] [instance: aaf1c3e3-b73e-42ad-87bd-c5255ec09b46] Updating existing volume attachment record: 61da819c-fe78-4a5e-b443-0ec2f5f405a2 {{(pid=75234) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Nov 29 03:18:54 user nova-compute[75234]: DEBUG nova.compute.manager [req-99e48ee4-6ad4-42ac-ac36-0705271ec5ec req-a2cddeda-592f-4cf5-a898-e4d90fcec462 service nova] [instance: d3b90f39-4b91-4152-b820-bd0e263eec89] Received event network-vif-plugged-83eb52bb-5be5-49d6-81ec-86a21dde3df2 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:18:54 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-99e48ee4-6ad4-42ac-ac36-0705271ec5ec req-a2cddeda-592f-4cf5-a898-e4d90fcec462 service nova] Acquiring lock "d3b90f39-4b91-4152-b820-bd0e263eec89-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:18:54 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-99e48ee4-6ad4-42ac-ac36-0705271ec5ec req-a2cddeda-592f-4cf5-a898-e4d90fcec462 service nova] Lock "d3b90f39-4b91-4152-b820-bd0e263eec89-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:18:54 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-99e48ee4-6ad4-42ac-ac36-0705271ec5ec req-a2cddeda-592f-4cf5-a898-e4d90fcec462 service nova] Lock "d3b90f39-4b91-4152-b820-bd0e263eec89-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:18:54 user nova-compute[75234]: DEBUG nova.compute.manager [req-99e48ee4-6ad4-42ac-ac36-0705271ec5ec req-a2cddeda-592f-4cf5-a898-e4d90fcec462 service nova] [instance: d3b90f39-4b91-4152-b820-bd0e263eec89] No waiting events found dispatching network-vif-plugged-83eb52bb-5be5-49d6-81ec-86a21dde3df2 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:18:54 user nova-compute[75234]: WARNING nova.compute.manager [req-99e48ee4-6ad4-42ac-ac36-0705271ec5ec req-a2cddeda-592f-4cf5-a898-e4d90fcec462 service nova] [instance: d3b90f39-4b91-4152-b820-bd0e263eec89] Received unexpected event network-vif-plugged-83eb52bb-5be5-49d6-81ec-86a21dde3df2 for instance with vm_state deleted and task_state None.
Nov 29 03:18:54 user nova-compute[75234]: DEBUG nova.compute.provider_tree [None req-105ba571-0ae2-40f0-9809-503c98c59bdb tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] Inventory has not changed in ProviderTree for provider: 4d291220-7c37-4839-a0e3-b23c66502b74 {{(pid=75234) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Nov 29 03:18:54 user nova-compute[75234]: DEBUG nova.scheduler.client.report [None req-105ba571-0ae2-40f0-9809-503c98c59bdb tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] Inventory has not changed for provider 4d291220-7c37-4839-a0e3-b23c66502b74 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=75234) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Nov 29 03:18:54 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-105ba571-0ae2-40f0-9809-503c98c59bdb tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.552s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:18:54 user nova-compute[75234]: INFO nova.scheduler.client.report [None req-105ba571-0ae2-40f0-9809-503c98c59bdb tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] Deleted allocations for instance d3b90f39-4b91-4152-b820-bd0e263eec89
Nov 29 03:18:54 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-105ba571-0ae2-40f0-9809-503c98c59bdb tempest-VolumesAdminNegativeTest-1445338761 tempest-VolumesAdminNegativeTest-1445338761-project-member] Lock "d3b90f39-4b91-4152-b820-bd0e263eec89" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 2.980s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:18:56 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:18:57 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:18:57 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:18:59 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:18:59 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Starting heal instance info cache {{(pid=75234) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9922}}
Nov 29 03:18:59 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Acquiring lock "refresh_cache-a082c96e-e384-40b6-bcc8-fc5e0e104669" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:18:59 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Acquired lock "refresh_cache-a082c96e-e384-40b6-bcc8-fc5e0e104669" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:18:59 user nova-compute[75234]: DEBUG nova.network.neutron [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] [instance: a082c96e-e384-40b6-bcc8-fc5e0e104669] Forcefully refreshing network info cache for instance {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2003}}
Nov 29 03:18:59 user nova-compute[75234]: DEBUG nova.network.neutron [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] [instance: a082c96e-e384-40b6-bcc8-fc5e0e104669] Updating instance_info_cache with network_info: [{"id": "0d131138-51b6-49f9-9a11-6b64e32e5796", "address": "fa:16:3e:ee:2e:82", "network": {"id": "2b3c3140-f3b6-45da-8202-bb4d7f0dd130", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-2043420778-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "f8a7884867c944e1ba1024270e4902f3", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap0d131138-51", "ovs_interfaceid": "0d131138-51b6-49f9-9a11-6b64e32e5796", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:18:59 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Releasing lock "refresh_cache-a082c96e-e384-40b6-bcc8-fc5e0e104669" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:18:59 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] [instance: a082c96e-e384-40b6-bcc8-fc5e0e104669] Updated the network info_cache for instance {{(pid=75234) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9993}}
Nov 29 03:18:59 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:19:00 user nova-compute[75234]: DEBUG nova.objects.instance [None req-1c44c857-3624-41cb-8724-98ef31810a22 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Lazy-loading 'flavor' on Instance uuid 1c8b7559-29ab-4982-8c75-ea2a0f0101cb {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:19:00 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-1c44c857-3624-41cb-8724-98ef31810a22 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Lock "1c8b7559-29ab-4982-8c75-ea2a0f0101cb" "released" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: held 33.937s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:19:00 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-4eb5b990-3c16-4fae-9864-ad176b8e7a22 tempest-TestServerVolumeAttachScenarioOldVersion-830274084 tempest-TestServerVolumeAttachScenarioOldVersion-830274084-project-member] [instance: 5e6cca46-c5a7-4303-98c2-1ad483c6d14e] Calling os-brick to attach iSCSI Volume {{(pid=75234) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Nov 29 03:19:00 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-4eb5b990-3c16-4fae-9864-ad176b8e7a22 tempest-TestServerVolumeAttachScenarioOldVersion-830274084 tempest-TestServerVolumeAttachScenarioOldVersion-830274084-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f8a5050c9a0>, 'connection_properties': {'target_iqn': 'iqn.2010-10.org.openstack:a06ab71b-02cb-4c8a-8e7b-26d8176cddf6', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'hanOIiWP', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:176}}
Nov 29 03:19:00 user nova-compute[75234]: WARNING os_brick.initiator.connectors.base [None req-4eb5b990-3c16-4fae-9864-ad176b8e7a22 tempest-TestServerVolumeAttachScenarioOldVersion-830274084 tempest-TestServerVolumeAttachScenarioOldVersion-830274084-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Nov 29 03:19:00 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-4eb5b990-3c16-4fae-9864-ad176b8e7a22 tempest-TestServerVolumeAttachScenarioOldVersion-830274084 tempest-TestServerVolumeAttachScenarioOldVersion-830274084-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:68}}
Nov 29 03:19:00 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-4eb5b990-3c16-4fae-9864-ad176b8e7a22 tempest-TestServerVolumeAttachScenarioOldVersion-830274084 tempest-TestServerVolumeAttachScenarioOldVersion-830274084-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:73}}
Nov 29 03:19:00 user nova-compute[75234]: WARNING os_brick.initiator.connectors.base [None req-4eb5b990-3c16-4fae-9864-ad176b8e7a22 tempest-TestServerVolumeAttachScenarioOldVersion-830274084 tempest-TestServerVolumeAttachScenarioOldVersion-830274084-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Nov 29 03:19:00 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-4eb5b990-3c16-4fae-9864-ad176b8e7a22 tempest-TestServerVolumeAttachScenarioOldVersion-830274084 tempest-TestServerVolumeAttachScenarioOldVersion-830274084-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:a06ab71b-02cb-4c8a-8e7b-26d8176cddf6" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:68}}
Nov 29 03:19:00 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-4eb5b990-3c16-4fae-9864-ad176b8e7a22 tempest-TestServerVolumeAttachScenarioOldVersion-830274084 tempest-TestServerVolumeAttachScenarioOldVersion-830274084-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:a06ab71b-02cb-4c8a-8e7b-26d8176cddf6" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:73}}
Nov 29 03:19:00 user nova-compute[75234]: INFO os_brick.initiator.connectors.iscsi [None req-4eb5b990-3c16-4fae-9864-ad176b8e7a22 tempest-TestServerVolumeAttachScenarioOldVersion-830274084 tempest-TestServerVolumeAttachScenarioOldVersion-830274084-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Nov 29 03:19:00 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:a06ab71b-02cb-4c8a-8e7b-26d8176cddf6 -p 172.16.0.220:3260 {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:19:01 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:a06ab71b-02cb-4c8a-8e7b-26d8176cddf6 -p 172.16.0.220:3260" returned: 21 in 0.020s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:19:01 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[fca55130-0855-4936-be90-8f7e2990e645]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:19:01 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-4eb5b990-3c16-4fae-9864-ad176b8e7a22 tempest-TestServerVolumeAttachScenarioOldVersion-830274084 tempest-TestServerVolumeAttachScenarioOldVersion-830274084-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Nov 29 03:19:01 user nova-compute[75234]:  {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:19:01 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:a06ab71b-02cb-4c8a-8e7b-26d8176cddf6 -p 172.16.0.220:3260 --interface default --op new {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:19:01 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:a06ab71b-02cb-4c8a-8e7b-26d8176cddf6 -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.020s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:19:01 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[97330fcb-7bfd-476e-86e3-891de20e0f1e]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:a06ab71b-02cb-4c8a-8e7b-26d8176cddf6] added\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:19:01 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-4eb5b990-3c16-4fae-9864-ad176b8e7a22 tempest-TestServerVolumeAttachScenarioOldVersion-830274084 tempest-TestServerVolumeAttachScenarioOldVersion-830274084-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:a06ab71b-02cb-4c8a-8e7b-26d8176cddf6] added
Nov 29 03:19:01 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:19:01 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:a06ab71b-02cb-4c8a-8e7b-26d8176cddf6 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:19:01 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:a06ab71b-02cb-4c8a-8e7b-26d8176cddf6 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.020s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:19:01 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[8c316455-a6e6-4ecc-9130-c43c5a01acb8]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:19:01 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-4eb5b990-3c16-4fae-9864-ad176b8e7a22 tempest-TestServerVolumeAttachScenarioOldVersion-830274084 tempest-TestServerVolumeAttachScenarioOldVersion-830274084-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:19:01 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:a06ab71b-02cb-4c8a-8e7b-26d8176cddf6 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:19:01 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:a06ab71b-02cb-4c8a-8e7b-26d8176cddf6 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.018s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:19:01 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[6bd0d80c-4343-424f-a95e-5159fc3cdee8]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:19:01 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-4eb5b990-3c16-4fae-9864-ad176b8e7a22 tempest-TestServerVolumeAttachScenarioOldVersion-830274084 tempest-TestServerVolumeAttachScenarioOldVersion-830274084-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:19:01 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:a06ab71b-02cb-4c8a-8e7b-26d8176cddf6 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v hanOIiWP {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:19:01 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:a06ab71b-02cb-4c8a-8e7b-26d8176cddf6 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v hanOIiWP" returned: 0 in 0.021s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:19:01 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[26187829-f9f3-42e7-a8c9-84dc4bc9223c]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:19:01 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-4eb5b990-3c16-4fae-9864-ad176b8e7a22 tempest-TestServerVolumeAttachScenarioOldVersion-830274084 tempest-TestServerVolumeAttachScenarioOldVersion-830274084-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'hanOIiWP'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:19:01 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:a06ab71b-02cb-4c8a-8e7b-26d8176cddf6 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:19:01 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:a06ab71b-02cb-4c8a-8e7b-26d8176cddf6 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.018s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:19:01 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[5d4482fc-9685-46ae-9f5b-dd1b5e35d894]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:19:01 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-4eb5b990-3c16-4fae-9864-ad176b8e7a22 tempest-TestServerVolumeAttachScenarioOldVersion-830274084 tempest-TestServerVolumeAttachScenarioOldVersion-830274084-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:19:01 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:19:01 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.024s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:19:01 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[440ec45f-5be6-4295-bab4-07e43dfb804d]: (4, ('tcp: [29] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bf57b967-7fad-4d76-abb2-319fe88f8ef9 (non-flash)\ntcp: [32] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:cf6ab229-533a-424e-bd67-1d54d50ad4be (non-flash)\ntcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:07f83efc-272d-4e10-9a2b-48d967db613c (non-flash)\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:19:01 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-4eb5b990-3c16-4fae-9864-ad176b8e7a22 tempest-TestServerVolumeAttachScenarioOldVersion-830274084 tempest-TestServerVolumeAttachScenarioOldVersion-830274084-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [29] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bf57b967-7fad-4d76-abb2-319fe88f8ef9 (non-flash)
Nov 29 03:19:01 user nova-compute[75234]: tcp: [32] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:cf6ab229-533a-424e-bd67-1d54d50ad4be (non-flash)
Nov 29 03:19:01 user nova-compute[75234]: tcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:07f83efc-272d-4e10-9a2b-48d967db613c (non-flash)
Nov 29 03:19:01 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:19:01 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-4eb5b990-3c16-4fae-9864-ad176b8e7a22 tempest-TestServerVolumeAttachScenarioOldVersion-830274084 tempest-TestServerVolumeAttachScenarioOldVersion-830274084-project-member] iscsi session list stdout=tcp: [29] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bf57b967-7fad-4d76-abb2-319fe88f8ef9 (non-flash)
Nov 29 03:19:01 user nova-compute[75234]: tcp: [32] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:cf6ab229-533a-424e-bd67-1d54d50ad4be (non-flash)
Nov 29 03:19:01 user nova-compute[75234]: tcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:07f83efc-272d-4e10-9a2b-48d967db613c (non-flash)
Nov 29 03:19:01 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsi_session /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Nov 29 03:19:01 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:a06ab71b-02cb-4c8a-8e7b-26d8176cddf6 -p 172.16.0.220:3260 --login {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:19:01 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:a06ab71b-02cb-4c8a-8e7b-26d8176cddf6 -p 172.16.0.220:3260 --login" returned: 0 in 0.166s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:19:01 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[d54e8e34-a9a7-4f9f-a5c8-37f337241060]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:a06ab71b-02cb-4c8a-8e7b-26d8176cddf6, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:a06ab71b-02cb-4c8a-8e7b-26d8176cddf6, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:19:01 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-4eb5b990-3c16-4fae-9864-ad176b8e7a22 tempest-TestServerVolumeAttachScenarioOldVersion-830274084 tempest-TestServerVolumeAttachScenarioOldVersion-830274084-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:a06ab71b-02cb-4c8a-8e7b-26d8176cddf6, portal: 172.16.0.220,3260]
Nov 29 03:19:01 user nova-compute[75234]: Login to [iface: default, target: iqn.2010-10.org.openstack:a06ab71b-02cb-4c8a-8e7b-26d8176cddf6, portal: 172.16.0.220,3260] successful.
Nov 29 03:19:01 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:19:01 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:a06ab71b-02cb-4c8a-8e7b-26d8176cddf6 -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:19:01 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:a06ab71b-02cb-4c8a-8e7b-26d8176cddf6 -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.022s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:19:01 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[95b2e2fd-5ba6-4c43-ba4d-112b4a208756]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:19:01 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-4eb5b990-3c16-4fae-9864-ad176b8e7a22 tempest-TestServerVolumeAttachScenarioOldVersion-830274084 tempest-TestServerVolumeAttachScenarioOldVersion-830274084-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:19:01 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:19:01 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:19:01 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:19:01 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:19:01 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=75234) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10541}}
Nov 29 03:19:01 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager.update_available_resource {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:19:01 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.245s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:19:01 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[0145e61f-6844-4c0f-bb21-2aaccbd1aec6]: (4, ('tcp: [29] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bf57b967-7fad-4d76-abb2-319fe88f8ef9 (non-flash)\ntcp: [32] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:cf6ab229-533a-424e-bd67-1d54d50ad4be (non-flash)\ntcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:07f83efc-272d-4e10-9a2b-48d967db613c (non-flash)\ntcp: [34] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a06ab71b-02cb-4c8a-8e7b-26d8176cddf6 (non-flash)\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:19:01 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-4eb5b990-3c16-4fae-9864-ad176b8e7a22 tempest-TestServerVolumeAttachScenarioOldVersion-830274084 tempest-TestServerVolumeAttachScenarioOldVersion-830274084-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [29] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bf57b967-7fad-4d76-abb2-319fe88f8ef9 (non-flash)
Nov 29 03:19:01 user nova-compute[75234]: tcp: [32] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:cf6ab229-533a-424e-bd67-1d54d50ad4be (non-flash)
Nov 29 03:19:01 user nova-compute[75234]: tcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:07f83efc-272d-4e10-9a2b-48d967db613c (non-flash)
Nov 29 03:19:01 user nova-compute[75234]: tcp: [34] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a06ab71b-02cb-4c8a-8e7b-26d8176cddf6 (non-flash)
Nov 29 03:19:01 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:19:01 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-4eb5b990-3c16-4fae-9864-ad176b8e7a22 tempest-TestServerVolumeAttachScenarioOldVersion-830274084 tempest-TestServerVolumeAttachScenarioOldVersion-830274084-project-member] iscsi session list stdout=tcp: [29] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bf57b967-7fad-4d76-abb2-319fe88f8ef9 (non-flash)
Nov 29 03:19:01 user nova-compute[75234]: tcp: [32] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:cf6ab229-533a-424e-bd67-1d54d50ad4be (non-flash)
Nov 29 03:19:01 user nova-compute[75234]: tcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:07f83efc-272d-4e10-9a2b-48d967db613c (non-flash)
Nov 29 03:19:01 user nova-compute[75234]: tcp: [34] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a06ab71b-02cb-4c8a-8e7b-26d8176cddf6 (non-flash)
Nov 29 03:19:01 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsi_session /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Nov 29 03:19:01 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-4eb5b990-3c16-4fae-9864-ad176b8e7a22 tempest-TestServerVolumeAttachScenarioOldVersion-830274084 tempest-TestServerVolumeAttachScenarioOldVersion-830274084-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:a06ab71b-02cb-4c8a-8e7b-26d8176cddf6" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.634s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:87}}
Nov 29 03:19:01 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-4eb5b990-3c16-4fae-9864-ad176b8e7a22 tempest-TestServerVolumeAttachScenarioOldVersion-830274084 tempest-TestServerVolumeAttachScenarioOldVersion-830274084-project-member] Connected to 172.16.0.220:3260 {{(pid=75234) _connect_vol /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:644}}
Nov 29 03:19:01 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-4eb5b990-3c16-4fae-9864-ad176b8e7a22 tempest-TestServerVolumeAttachScenarioOldVersion-830274084 tempest-TestServerVolumeAttachScenarioOldVersion-830274084-project-member] HCTL ('33', '-', '-', 0) found on session 34 with lun 0 {{(pid=75234) get_hctl /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:770}}
Nov 29 03:19:01 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-4eb5b990-3c16-4fae-9864-ad176b8e7a22 tempest-TestServerVolumeAttachScenarioOldVersion-830274084 tempest-TestServerVolumeAttachScenarioOldVersion-830274084-project-member] Scanning host 33 c: -, t: -, l: 0) {{(pid=75234) scan_iscsi /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:795}}
Nov 29 03:19:01 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host33/scan {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:19:01 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host33/scan" returned: 0 in 0.026s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:19:01 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[191e9379-d4e0-4bcc-a7d7-390b611a5ae1]: (4, ('- - 0', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:19:01 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-4eb5b990-3c16-4fae-9864-ad176b8e7a22 tempest-TestServerVolumeAttachScenarioOldVersion-830274084 tempest-TestServerVolumeAttachScenarioOldVersion-830274084-project-member] Searching for a device in session 34 and hctl ['33', '*', '*', 0] yield: sdb {{(pid=75234) device_name_by_hctl /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:789}}
Nov 29 03:19:01 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-4eb5b990-3c16-4fae-9864-ad176b8e7a22 tempest-TestServerVolumeAttachScenarioOldVersion-830274084 tempest-TestServerVolumeAttachScenarioOldVersion-830274084-project-member] Connected to sdb using {'target_iqn': 'iqn.2010-10.org.openstack:a06ab71b-02cb-4c8a-8e7b-26d8176cddf6', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'hanOIiWP', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False} {{(pid=75234) _connect_vol /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:672}}
Nov 29 03:19:01 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:19:01 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:19:01 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:19:01 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Auditing locally available compute resources for user (node: user) {{(pid=75234) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:907}}
Nov 29 03:19:01 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] skipping disk /dev/sde (vda) as it is a volume {{(pid=75234) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11213}}
Nov 29 03:19:01 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/13e5da64-9adc-4479-81c7-d329e9b5f08f/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:19:02 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:19:02 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/13e5da64-9adc-4479-81c7-d329e9b5f08f/disk --force-share --output=json" returned: 0 in 0.184s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:19:02 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/13e5da64-9adc-4479-81c7-d329e9b5f08f/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:19:02 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/13e5da64-9adc-4479-81c7-d329e9b5f08f/disk --force-share --output=json" returned: 0 in 0.201s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:19:02 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/1c8b7559-29ab-4982-8c75-ea2a0f0101cb/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:19:02 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/1c8b7559-29ab-4982-8c75-ea2a0f0101cb/disk --force-share --output=json" returned: 0 in 0.207s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:19:02 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/1c8b7559-29ab-4982-8c75-ea2a0f0101cb/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:19:02 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-4eb5b990-3c16-4fae-9864-ad176b8e7a22 tempest-TestServerVolumeAttachScenarioOldVersion-830274084 tempest-TestServerVolumeAttachScenarioOldVersion-830274084-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.684s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:87}}
Nov 29 03:19:02 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-4eb5b990-3c16-4fae-9864-ad176b8e7a22 tempest-TestServerVolumeAttachScenarioOldVersion-830274084 tempest-TestServerVolumeAttachScenarioOldVersion-830274084-project-member] <== connect_volume: return (1688ms) {'type': 'block', 'scsi_wwn': '26337363864646435', 'path': '/dev/sdb'} {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:203}}
Nov 29 03:19:02 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-4eb5b990-3c16-4fae-9864-ad176b8e7a22 tempest-TestServerVolumeAttachScenarioOldVersion-830274084 tempest-TestServerVolumeAttachScenarioOldVersion-830274084-project-member] [instance: 5e6cca46-c5a7-4303-98c2-1ad483c6d14e] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '26337363864646435', 'path': '/dev/sdb'} {{(pid=75234) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Nov 29 03:19:02 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:19:02 user nova-compute[75234]: DEBUG nova.objects.instance [None req-4eb5b990-3c16-4fae-9864-ad176b8e7a22 tempest-TestServerVolumeAttachScenarioOldVersion-830274084 tempest-TestServerVolumeAttachScenarioOldVersion-830274084-project-member] Lazy-loading 'flavor' on Instance uuid 5e6cca46-c5a7-4303-98c2-1ad483c6d14e {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:19:02 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/1c8b7559-29ab-4982-8c75-ea2a0f0101cb/disk --force-share --output=json" returned: 0 in 0.193s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:19:02 user nova-compute[75234]: DEBUG nova.virt.libvirt.guest [None req-4eb5b990-3c16-4fae-9864-ad176b8e7a22 tempest-TestServerVolumeAttachScenarioOldVersion-830274084 tempest-TestServerVolumeAttachScenarioOldVersion-830274084-project-member] attach device xml: <disk type="block" device="disk">
Nov 29 03:19:02 user nova-compute[75234]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Nov 29 03:19:02 user nova-compute[75234]:   <source dev="/dev/sdb"/>
Nov 29 03:19:02 user nova-compute[75234]:   <target dev="vdb" bus="virtio"/>
Nov 29 03:19:02 user nova-compute[75234]:   <serial>a06ab71b-02cb-4c8a-8e7b-26d8176cddf6</serial>
Nov 29 03:19:02 user nova-compute[75234]: </disk>
Nov 29 03:19:02 user nova-compute[75234]:  {{(pid=75234) attach_device /opt/stack/nova/nova/virt/libvirt/guest.py:339}}
Nov 29 03:19:02 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6894b55f-93c6-43c9-8933-56d5a3c43725/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:19:02 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6894b55f-93c6-43c9-8933-56d5a3c43725/disk --force-share --output=json" returned: 0 in 0.193s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:19:02 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6894b55f-93c6-43c9-8933-56d5a3c43725/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:19:03 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6894b55f-93c6-43c9-8933-56d5a3c43725/disk --force-share --output=json" returned: 0 in 0.181s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:19:03 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/aaf1c3e3-b73e-42ad-87bd-c5255ec09b46/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:19:03 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-0f1135d4-0472-46b4-8d7c-46d1e239a06d tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] [instance: aaf1c3e3-b73e-42ad-87bd-c5255ec09b46] Calling os-brick to attach iSCSI Volume {{(pid=75234) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Nov 29 03:19:03 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-0f1135d4-0472-46b4-8d7c-46d1e239a06d tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f8a5050c9a0>, 'connection_properties': {'target_iqn': 'iqn.2010-10.org.openstack:7a1a1247-abe3-4a99-a7cb-f168fb2cb0b0', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'lgpvtzQE', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:176}}
Nov 29 03:19:03 user nova-compute[75234]: WARNING os_brick.initiator.connectors.base [None req-0f1135d4-0472-46b4-8d7c-46d1e239a06d tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Nov 29 03:19:03 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-0f1135d4-0472-46b4-8d7c-46d1e239a06d tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:68}}
Nov 29 03:19:03 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-0f1135d4-0472-46b4-8d7c-46d1e239a06d tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.002s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:73}}
Nov 29 03:19:03 user nova-compute[75234]: WARNING os_brick.initiator.connectors.base [None req-0f1135d4-0472-46b4-8d7c-46d1e239a06d tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Nov 29 03:19:03 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-0f1135d4-0472-46b4-8d7c-46d1e239a06d tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:7a1a1247-abe3-4a99-a7cb-f168fb2cb0b0" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:68}}
Nov 29 03:19:03 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-0f1135d4-0472-46b4-8d7c-46d1e239a06d tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:7a1a1247-abe3-4a99-a7cb-f168fb2cb0b0" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.002s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:73}}
Nov 29 03:19:03 user nova-compute[75234]: INFO os_brick.initiator.connectors.iscsi [None req-0f1135d4-0472-46b4-8d7c-46d1e239a06d tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Nov 29 03:19:03 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:7a1a1247-abe3-4a99-a7cb-f168fb2cb0b0 -p 172.16.0.220:3260 {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:19:03 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-4eb5b990-3c16-4fae-9864-ad176b8e7a22 tempest-TestServerVolumeAttachScenarioOldVersion-830274084 tempest-TestServerVolumeAttachScenarioOldVersion-830274084-project-member] No BDM found with device name vda, not building metadata. {{(pid=75234) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12089}}
Nov 29 03:19:03 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-4eb5b990-3c16-4fae-9864-ad176b8e7a22 tempest-TestServerVolumeAttachScenarioOldVersion-830274084 tempest-TestServerVolumeAttachScenarioOldVersion-830274084-project-member] No BDM found with device name vdb, not building metadata. {{(pid=75234) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12089}}
Nov 29 03:19:03 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-4eb5b990-3c16-4fae-9864-ad176b8e7a22 tempest-TestServerVolumeAttachScenarioOldVersion-830274084 tempest-TestServerVolumeAttachScenarioOldVersion-830274084-project-member] No VIF found with MAC fa:16:3e:6e:d5:4a, not building metadata {{(pid=75234) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Nov 29 03:19:03 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:7a1a1247-abe3-4a99-a7cb-f168fb2cb0b0 -p 172.16.0.220:3260" returned: 21 in 0.019s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:19:03 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[29e55b51-f9ec-4275-bcd2-6ec7d73e888c]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:19:03 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-0f1135d4-0472-46b4-8d7c-46d1e239a06d tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Nov 29 03:19:03 user nova-compute[75234]:  {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:19:03 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:7a1a1247-abe3-4a99-a7cb-f168fb2cb0b0 -p 172.16.0.220:3260 --interface default --op new {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:19:03 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:7a1a1247-abe3-4a99-a7cb-f168fb2cb0b0 -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.027s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:19:03 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[edd14ab2-a922-4998-a574-413ed1fd0220]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:7a1a1247-abe3-4a99-a7cb-f168fb2cb0b0] added\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:19:03 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-0f1135d4-0472-46b4-8d7c-46d1e239a06d tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:7a1a1247-abe3-4a99-a7cb-f168fb2cb0b0] added
Nov 29 03:19:03 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:19:03 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:7a1a1247-abe3-4a99-a7cb-f168fb2cb0b0 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:19:03 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:7a1a1247-abe3-4a99-a7cb-f168fb2cb0b0 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.030s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:19:03 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[125c8440-2129-42b8-a2a1-a78bb366d4d7]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:19:03 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-0f1135d4-0472-46b4-8d7c-46d1e239a06d tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:19:03 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:7a1a1247-abe3-4a99-a7cb-f168fb2cb0b0 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:19:03 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:7a1a1247-abe3-4a99-a7cb-f168fb2cb0b0 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.018s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:19:03 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[e2abe8f7-a783-41ed-aac8-4e7bd0c5cdf7]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:19:03 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-0f1135d4-0472-46b4-8d7c-46d1e239a06d tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:19:03 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:7a1a1247-abe3-4a99-a7cb-f168fb2cb0b0 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v lgpvtzQE {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:19:03 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/aaf1c3e3-b73e-42ad-87bd-c5255ec09b46/disk --force-share --output=json" returned: 0 in 0.199s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:19:03 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/aaf1c3e3-b73e-42ad-87bd-c5255ec09b46/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:19:03 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:7a1a1247-abe3-4a99-a7cb-f168fb2cb0b0 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v lgpvtzQE" returned: 0 in 0.019s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:19:03 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[5df0a20c-cbed-402b-9ec2-91ef22d022f5]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:19:03 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-0f1135d4-0472-46b4-8d7c-46d1e239a06d tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'lgpvtzQE'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:19:03 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:7a1a1247-abe3-4a99-a7cb-f168fb2cb0b0 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:19:03 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-4eb5b990-3c16-4fae-9864-ad176b8e7a22 tempest-TestServerVolumeAttachScenarioOldVersion-830274084 tempest-TestServerVolumeAttachScenarioOldVersion-830274084-project-member] Lock "5e6cca46-c5a7-4303-98c2-1ad483c6d14e" "released" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: held 17.258s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:19:03 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:7a1a1247-abe3-4a99-a7cb-f168fb2cb0b0 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.023s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:19:03 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[f358a62a-089c-4252-84e2-e3116f99bba6]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:19:03 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-0f1135d4-0472-46b4-8d7c-46d1e239a06d tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:19:03 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:19:03 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.027s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:19:03 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[d3d6edb0-eab9-4b86-adf9-3b0e7981ed40]: (4, ('tcp: [29] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bf57b967-7fad-4d76-abb2-319fe88f8ef9 (non-flash)\ntcp: [32] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:cf6ab229-533a-424e-bd67-1d54d50ad4be (non-flash)\ntcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:07f83efc-272d-4e10-9a2b-48d967db613c (non-flash)\ntcp: [34] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a06ab71b-02cb-4c8a-8e7b-26d8176cddf6 (non-flash)\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:19:03 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-0f1135d4-0472-46b4-8d7c-46d1e239a06d tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [29] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bf57b967-7fad-4d76-abb2-319fe88f8ef9 (non-flash)
Nov 29 03:19:03 user nova-compute[75234]: tcp: [32] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:cf6ab229-533a-424e-bd67-1d54d50ad4be (non-flash)
Nov 29 03:19:03 user nova-compute[75234]: tcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:07f83efc-272d-4e10-9a2b-48d967db613c (non-flash)
Nov 29 03:19:03 user nova-compute[75234]: tcp: [34] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a06ab71b-02cb-4c8a-8e7b-26d8176cddf6 (non-flash)
Nov 29 03:19:03 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:19:03 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-0f1135d4-0472-46b4-8d7c-46d1e239a06d tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] iscsi session list stdout=tcp: [29] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bf57b967-7fad-4d76-abb2-319fe88f8ef9 (non-flash)
Nov 29 03:19:03 user nova-compute[75234]: tcp: [32] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:cf6ab229-533a-424e-bd67-1d54d50ad4be (non-flash)
Nov 29 03:19:03 user nova-compute[75234]: tcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:07f83efc-272d-4e10-9a2b-48d967db613c (non-flash)
Nov 29 03:19:03 user nova-compute[75234]: tcp: [34] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a06ab71b-02cb-4c8a-8e7b-26d8176cddf6 (non-flash)
Nov 29 03:19:03 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsi_session /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Nov 29 03:19:03 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:7a1a1247-abe3-4a99-a7cb-f168fb2cb0b0 -p 172.16.0.220:3260 --login {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:19:03 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:7a1a1247-abe3-4a99-a7cb-f168fb2cb0b0 -p 172.16.0.220:3260 --login" returned: 0 in 0.073s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:19:03 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[eb923d4e-9ab1-4906-92b6-eeac49e9b2da]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:7a1a1247-abe3-4a99-a7cb-f168fb2cb0b0, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:7a1a1247-abe3-4a99-a7cb-f168fb2cb0b0, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:19:03 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-0f1135d4-0472-46b4-8d7c-46d1e239a06d tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:7a1a1247-abe3-4a99-a7cb-f168fb2cb0b0, portal: 172.16.0.220,3260]
Nov 29 03:19:03 user nova-compute[75234]: Login to [iface: default, target: iqn.2010-10.org.openstack:7a1a1247-abe3-4a99-a7cb-f168fb2cb0b0, portal: 172.16.0.220,3260] successful.
Nov 29 03:19:03 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:19:03 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/aaf1c3e3-b73e-42ad-87bd-c5255ec09b46/disk --force-share --output=json" returned: 0 in 0.189s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:19:03 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:7a1a1247-abe3-4a99-a7cb-f168fb2cb0b0 -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:19:03 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/0aff446c-f417-477e-a1c5-b51b1b6214e4/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:19:03 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:7a1a1247-abe3-4a99-a7cb-f168fb2cb0b0 -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.020s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:19:03 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[ba01d336-95c7-4ee9-9c39-8b0128924e70]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:19:03 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-0f1135d4-0472-46b4-8d7c-46d1e239a06d tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:19:03 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:19:03 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.027s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:19:03 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[ebe16cbf-764b-45ba-945a-45ece6673033]: (4, ('tcp: [29] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bf57b967-7fad-4d76-abb2-319fe88f8ef9 (non-flash)\ntcp: [32] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:cf6ab229-533a-424e-bd67-1d54d50ad4be (non-flash)\ntcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:07f83efc-272d-4e10-9a2b-48d967db613c (non-flash)\ntcp: [34] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a06ab71b-02cb-4c8a-8e7b-26d8176cddf6 (non-flash)\ntcp: [35] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7a1a1247-abe3-4a99-a7cb-f168fb2cb0b0 (non-flash)\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:19:03 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-0f1135d4-0472-46b4-8d7c-46d1e239a06d tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [29] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bf57b967-7fad-4d76-abb2-319fe88f8ef9 (non-flash)
Nov 29 03:19:03 user nova-compute[75234]: tcp: [32] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:cf6ab229-533a-424e-bd67-1d54d50ad4be (non-flash)
Nov 29 03:19:03 user nova-compute[75234]: tcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:07f83efc-272d-4e10-9a2b-48d967db613c (non-flash)
Nov 29 03:19:03 user nova-compute[75234]: tcp: [34] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a06ab71b-02cb-4c8a-8e7b-26d8176cddf6 (non-flash)
Nov 29 03:19:03 user nova-compute[75234]: tcp: [35] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7a1a1247-abe3-4a99-a7cb-f168fb2cb0b0 (non-flash)
Nov 29 03:19:03 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:19:03 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-0f1135d4-0472-46b4-8d7c-46d1e239a06d tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] iscsi session list stdout=tcp: [29] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bf57b967-7fad-4d76-abb2-319fe88f8ef9 (non-flash)
Nov 29 03:19:03 user nova-compute[75234]: tcp: [32] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:cf6ab229-533a-424e-bd67-1d54d50ad4be (non-flash)
Nov 29 03:19:03 user nova-compute[75234]: tcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:07f83efc-272d-4e10-9a2b-48d967db613c (non-flash)
Nov 29 03:19:03 user nova-compute[75234]: tcp: [34] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a06ab71b-02cb-4c8a-8e7b-26d8176cddf6 (non-flash)
Nov 29 03:19:03 user nova-compute[75234]: tcp: [35] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7a1a1247-abe3-4a99-a7cb-f168fb2cb0b0 (non-flash)
Nov 29 03:19:03 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsi_session /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Nov 29 03:19:03 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-0f1135d4-0472-46b4-8d7c-46d1e239a06d tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:7a1a1247-abe3-4a99-a7cb-f168fb2cb0b0" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.387s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:87}}
Nov 29 03:19:03 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-0f1135d4-0472-46b4-8d7c-46d1e239a06d tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Connected to 172.16.0.220:3260 {{(pid=75234) _connect_vol /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:644}}
Nov 29 03:19:03 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-0f1135d4-0472-46b4-8d7c-46d1e239a06d tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] HCTL ('37', '-', '-', 0) found on session 35 with lun 0 {{(pid=75234) get_hctl /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:770}}
Nov 29 03:19:03 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-0f1135d4-0472-46b4-8d7c-46d1e239a06d tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Scanning host 37 c: -, t: -, l: 0) {{(pid=75234) scan_iscsi /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:795}}
Nov 29 03:19:03 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host37/scan {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:19:03 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host37/scan" returned: 0 in 0.017s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:19:03 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[69735d34-e69d-4cdd-8599-4cab796d1cd7]: (4, ('- - 0', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:19:03 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-0f1135d4-0472-46b4-8d7c-46d1e239a06d tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Searching for a device in session 35 and hctl ['37', '*', '*', 0] yield: None {{(pid=75234) device_name_by_hctl /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:789}}
Nov 29 03:19:03 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/0aff446c-f417-477e-a1c5-b51b1b6214e4/disk --force-share --output=json" returned: 0 in 0.192s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:19:03 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/0aff446c-f417-477e-a1c5-b51b1b6214e4/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:19:03 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/0aff446c-f417-477e-a1c5-b51b1b6214e4/disk --force-share --output=json" returned: 0 in 0.172s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:19:03 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/5e6cca46-c5a7-4303-98c2-1ad483c6d14e/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:19:04 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/5e6cca46-c5a7-4303-98c2-1ad483c6d14e/disk --force-share --output=json" returned: 0 in 0.183s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:19:04 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/5e6cca46-c5a7-4303-98c2-1ad483c6d14e/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:19:04 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/5e6cca46-c5a7-4303-98c2-1ad483c6d14e/disk --force-share --output=json" returned: 0 in 0.183s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:19:04 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] skipping disk /dev/sdb (vdb) as it is a volume {{(pid=75234) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11213}}
Nov 29 03:19:04 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a082c96e-e384-40b6-bcc8-fc5e0e104669/disk.rescue --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:19:04 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a082c96e-e384-40b6-bcc8-fc5e0e104669/disk.rescue --force-share --output=json" returned: 0 in 0.175s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:19:04 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a082c96e-e384-40b6-bcc8-fc5e0e104669/disk.rescue --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:19:04 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-0f1135d4-0472-46b4-8d7c-46d1e239a06d tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Searching for a device in session 35 and hctl ['37', '*', '*', 0] yield: sdf {{(pid=75234) device_name_by_hctl /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:789}}
Nov 29 03:19:04 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-0f1135d4-0472-46b4-8d7c-46d1e239a06d tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Connected to sdf using {'target_iqn': 'iqn.2010-10.org.openstack:7a1a1247-abe3-4a99-a7cb-f168fb2cb0b0', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'lgpvtzQE', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False} {{(pid=75234) _connect_vol /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:672}}
Nov 29 03:19:04 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-0f1135d4-0472-46b4-8d7c-46d1e239a06d tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.425s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:87}}
Nov 29 03:19:04 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-0f1135d4-0472-46b4-8d7c-46d1e239a06d tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] <== connect_volume: return (1427ms) {'type': 'block', 'scsi_wwn': '23431653234356331', 'path': '/dev/sdf'} {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:203}}
Nov 29 03:19:04 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-0f1135d4-0472-46b4-8d7c-46d1e239a06d tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] [instance: aaf1c3e3-b73e-42ad-87bd-c5255ec09b46] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '23431653234356331', 'path': '/dev/sdf'} {{(pid=75234) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Nov 29 03:19:04 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a082c96e-e384-40b6-bcc8-fc5e0e104669/disk.rescue --force-share --output=json" returned: 0 in 0.169s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:19:04 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a082c96e-e384-40b6-bcc8-fc5e0e104669/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:19:04 user nova-compute[75234]: DEBUG nova.objects.instance [None req-0f1135d4-0472-46b4-8d7c-46d1e239a06d tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Lazy-loading 'flavor' on Instance uuid aaf1c3e3-b73e-42ad-87bd-c5255ec09b46 {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:19:04 user nova-compute[75234]: DEBUG nova.virt.libvirt.guest [None req-0f1135d4-0472-46b4-8d7c-46d1e239a06d tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] attach device xml: <disk type="block" device="disk">
Nov 29 03:19:04 user nova-compute[75234]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Nov 29 03:19:04 user nova-compute[75234]:   <source dev="/dev/sdf"/>
Nov 29 03:19:04 user nova-compute[75234]:   <target dev="vdb" bus="virtio"/>
Nov 29 03:19:04 user nova-compute[75234]:   <serial>7a1a1247-abe3-4a99-a7cb-f168fb2cb0b0</serial>
Nov 29 03:19:04 user nova-compute[75234]: </disk>
Nov 29 03:19:04 user nova-compute[75234]:  {{(pid=75234) attach_device /opt/stack/nova/nova/virt/libvirt/guest.py:339}}
Nov 29 03:19:04 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a082c96e-e384-40b6-bcc8-fc5e0e104669/disk --force-share --output=json" returned: 0 in 0.175s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:19:04 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a082c96e-e384-40b6-bcc8-fc5e0e104669/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:19:04 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a082c96e-e384-40b6-bcc8-fc5e0e104669/disk --force-share --output=json" returned: 0 in 0.188s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:19:05 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/75c5d700-6367-4128-9228-80d2cd24ace7/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:19:05 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/75c5d700-6367-4128-9228-80d2cd24ace7/disk --force-share --output=json" returned: 0 in 0.173s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:19:05 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/75c5d700-6367-4128-9228-80d2cd24ace7/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:19:05 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-0f1135d4-0472-46b4-8d7c-46d1e239a06d tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] No BDM found with device name vda, not building metadata. {{(pid=75234) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12089}}
Nov 29 03:19:05 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-0f1135d4-0472-46b4-8d7c-46d1e239a06d tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] No BDM found with device name vdb, not building metadata. {{(pid=75234) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12089}}
Nov 29 03:19:05 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-0f1135d4-0472-46b4-8d7c-46d1e239a06d tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] No VIF found with MAC fa:16:3e:55:a5:13, not building metadata {{(pid=75234) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Nov 29 03:19:05 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/75c5d700-6367-4128-9228-80d2cd24ace7/disk --force-share --output=json" returned: 0 in 0.169s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:19:05 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] skipping disk /dev/sdd (vdb) as it is a volume {{(pid=75234) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11213}}
Nov 29 03:19:05 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] skipping disk /dev/sdc (vda) as it is a volume {{(pid=75234) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11213}}
Nov 29 03:19:05 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/21e9c6cd-c39f-41c0-8768-5dd64589c3f3/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:19:05 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-0f1135d4-0472-46b4-8d7c-46d1e239a06d tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Lock "aaf1c3e3-b73e-42ad-87bd-c5255ec09b46" "released" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: held 11.819s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:19:05 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-0b7fb330-8408-410c-b9e8-3270e7e2d64f tempest-TestServerVolumeAttachScenarioOldVersion-830274084 tempest-TestServerVolumeAttachScenarioOldVersion-830274084-project-member] Acquiring lock "5e6cca46-c5a7-4303-98c2-1ad483c6d14e" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:19:05 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-0b7fb330-8408-410c-b9e8-3270e7e2d64f tempest-TestServerVolumeAttachScenarioOldVersion-830274084 tempest-TestServerVolumeAttachScenarioOldVersion-830274084-project-member] Lock "5e6cca46-c5a7-4303-98c2-1ad483c6d14e" acquired by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:19:05 user nova-compute[75234]: INFO nova.compute.manager [None req-0b7fb330-8408-410c-b9e8-3270e7e2d64f tempest-TestServerVolumeAttachScenarioOldVersion-830274084 tempest-TestServerVolumeAttachScenarioOldVersion-830274084-project-member] [instance: 5e6cca46-c5a7-4303-98c2-1ad483c6d14e] Detaching volume a06ab71b-02cb-4c8a-8e7b-26d8176cddf6
Nov 29 03:19:05 user nova-compute[75234]: INFO nova.virt.block_device [None req-0b7fb330-8408-410c-b9e8-3270e7e2d64f tempest-TestServerVolumeAttachScenarioOldVersion-830274084 tempest-TestServerVolumeAttachScenarioOldVersion-830274084-project-member] [instance: 5e6cca46-c5a7-4303-98c2-1ad483c6d14e] Attempting to driver detach volume a06ab71b-02cb-4c8a-8e7b-26d8176cddf6 from mountpoint /dev/vdb
Nov 29 03:19:05 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-0b7fb330-8408-410c-b9e8-3270e7e2d64f tempest-TestServerVolumeAttachScenarioOldVersion-830274084 tempest-TestServerVolumeAttachScenarioOldVersion-830274084-project-member] Attempting to detach device vdb from instance 5e6cca46-c5a7-4303-98c2-1ad483c6d14e from the persistent domain config. {{(pid=75234) _detach_from_persistent /opt/stack/nova/nova/virt/libvirt/driver.py:2487}}
Nov 29 03:19:05 user nova-compute[75234]: DEBUG nova.virt.libvirt.guest [None req-0b7fb330-8408-410c-b9e8-3270e7e2d64f tempest-TestServerVolumeAttachScenarioOldVersion-830274084 tempest-TestServerVolumeAttachScenarioOldVersion-830274084-project-member] detach device xml: <disk type="block" device="disk">
Nov 29 03:19:05 user nova-compute[75234]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Nov 29 03:19:05 user nova-compute[75234]:   <source dev="/dev/sdb"/>
Nov 29 03:19:05 user nova-compute[75234]:   <target dev="vdb" bus="virtio"/>
Nov 29 03:19:05 user nova-compute[75234]:   <serial>a06ab71b-02cb-4c8a-8e7b-26d8176cddf6</serial>
Nov 29 03:19:05 user nova-compute[75234]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x07" function="0x0"/>
Nov 29 03:19:05 user nova-compute[75234]: </disk>
Nov 29 03:19:05 user nova-compute[75234]:  {{(pid=75234) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Nov 29 03:19:05 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/21e9c6cd-c39f-41c0-8768-5dd64589c3f3/disk --force-share --output=json" returned: 0 in 0.190s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:19:05 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/21e9c6cd-c39f-41c0-8768-5dd64589c3f3/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:19:05 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-0b7fb330-8408-410c-b9e8-3270e7e2d64f tempest-TestServerVolumeAttachScenarioOldVersion-830274084 tempest-TestServerVolumeAttachScenarioOldVersion-830274084-project-member] Successfully detached device vdb from instance 5e6cca46-c5a7-4303-98c2-1ad483c6d14e from the persistent domain config.
Nov 29 03:19:05 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-0b7fb330-8408-410c-b9e8-3270e7e2d64f tempest-TestServerVolumeAttachScenarioOldVersion-830274084 tempest-TestServerVolumeAttachScenarioOldVersion-830274084-project-member] (1/8): Attempting to detach device vdb with device alias virtio-disk1 from instance 5e6cca46-c5a7-4303-98c2-1ad483c6d14e from the live domain config. {{(pid=75234) _detach_from_live_with_retry /opt/stack/nova/nova/virt/libvirt/driver.py:2523}}
Nov 29 03:19:05 user nova-compute[75234]: DEBUG nova.virt.libvirt.guest [None req-0b7fb330-8408-410c-b9e8-3270e7e2d64f tempest-TestServerVolumeAttachScenarioOldVersion-830274084 tempest-TestServerVolumeAttachScenarioOldVersion-830274084-project-member] detach device xml: <disk type="block" device="disk">
Nov 29 03:19:05 user nova-compute[75234]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Nov 29 03:19:05 user nova-compute[75234]:   <source dev="/dev/sdb"/>
Nov 29 03:19:05 user nova-compute[75234]:   <target dev="vdb" bus="virtio"/>
Nov 29 03:19:05 user nova-compute[75234]:   <serial>a06ab71b-02cb-4c8a-8e7b-26d8176cddf6</serial>
Nov 29 03:19:05 user nova-compute[75234]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x07" function="0x0"/>
Nov 29 03:19:05 user nova-compute[75234]: </disk>
Nov 29 03:19:05 user nova-compute[75234]:  {{(pid=75234) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Nov 29 03:19:05 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/21e9c6cd-c39f-41c0-8768-5dd64589c3f3/disk --force-share --output=json" returned: 0 in 0.173s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:19:06 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-0b7fb330-8408-410c-b9e8-3270e7e2d64f tempest-TestServerVolumeAttachScenarioOldVersion-830274084 tempest-TestServerVolumeAttachScenarioOldVersion-830274084-project-member] Start waiting for the detach event from libvirt for device vdb with device alias virtio-disk1 for instance 5e6cca46-c5a7-4303-98c2-1ad483c6d14e {{(pid=75234) _detach_from_live_and_wait_for_event /opt/stack/nova/nova/virt/libvirt/driver.py:2599}}
Nov 29 03:19:06 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:19:06 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:19:06 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Hypervisor/Node resource view: name=user free_ram=7218MB free_disk=25.179546356201172GB free_vcpus=1 pci_devices=[{"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}] {{(pid=75234) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1106}}
Nov 29 03:19:06 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:19:06 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:19:06 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance 21e9c6cd-c39f-41c0-8768-5dd64589c3f3 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:19:06 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance 13e5da64-9adc-4479-81c7-d329e9b5f08f actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:19:06 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance a082c96e-e384-40b6-bcc8-fc5e0e104669 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:19:06 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance 6894b55f-93c6-43c9-8933-56d5a3c43725 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:19:06 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance 75c5d700-6367-4128-9228-80d2cd24ace7 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:19:06 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance 1c8b7559-29ab-4982-8c75-ea2a0f0101cb actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:19:06 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance aa190a9f-ba53-43fb-8bae-2c429d0d1228 actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:19:06 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance 5e6cca46-c5a7-4303-98c2-1ad483c6d14e actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:19:06 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance aaf1c3e3-b73e-42ad-87bd-c5255ec09b46 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:19:06 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance af699bc2-9daa-4b5a-9b4a-ad92fe857d70 actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:19:06 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance 0aff446c-f417-477e-a1c5-b51b1b6214e4 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:19:06 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Total usable vcpus: 12, total allocated vcpus: 11 {{(pid=75234) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1129}}
Nov 29 03:19:06 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Final resource view: name=user phys_ram=16011MB used_ram=6144MB phys_disk=40GB used_disk=9GB total_vcpus=12 used_vcpus=11 pci_stats=[] {{(pid=75234) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1138}}
Nov 29 03:19:06 user nova-compute[75234]: DEBUG nova.compute.provider_tree [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Inventory has not changed in ProviderTree for provider: 4d291220-7c37-4839-a0e3-b23c66502b74 {{(pid=75234) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Nov 29 03:19:06 user nova-compute[75234]: DEBUG nova.scheduler.client.report [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Inventory has not changed for provider 4d291220-7c37-4839-a0e3-b23c66502b74 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=75234) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Nov 29 03:19:06 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Compute_service record updated for user:user {{(pid=75234) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:1067}}
Nov 29 03:19:06 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.506s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:19:07 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:19:07 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:19:10 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-be1e6180-8347-498b-b8a9-92cf51694217 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Acquiring lock "1df140f1-fbd9-4b5d-b356-d08b7e908f64" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:19:10 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-be1e6180-8347-498b-b8a9-92cf51694217 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Lock "1df140f1-fbd9-4b5d-b356-d08b7e908f64" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:19:10 user nova-compute[75234]: DEBUG nova.compute.manager [None req-be1e6180-8347-498b-b8a9-92cf51694217 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] [instance: 1df140f1-fbd9-4b5d-b356-d08b7e908f64] Starting instance... {{(pid=75234) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2422}}
Nov 29 03:19:11 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-be1e6180-8347-498b-b8a9-92cf51694217 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:19:11 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-be1e6180-8347-498b-b8a9-92cf51694217 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:19:11 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-be1e6180-8347-498b-b8a9-92cf51694217 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=75234) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2366}}
Nov 29 03:19:11 user nova-compute[75234]: INFO nova.compute.claims [None req-be1e6180-8347-498b-b8a9-92cf51694217 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] [instance: 1df140f1-fbd9-4b5d-b356-d08b7e908f64] Claim successful on node user
Nov 29 03:19:11 user nova-compute[75234]: DEBUG nova.compute.provider_tree [None req-be1e6180-8347-498b-b8a9-92cf51694217 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Inventory has not changed in ProviderTree for provider: 4d291220-7c37-4839-a0e3-b23c66502b74 {{(pid=75234) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Nov 29 03:19:11 user nova-compute[75234]: DEBUG nova.scheduler.client.report [None req-be1e6180-8347-498b-b8a9-92cf51694217 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Inventory has not changed for provider 4d291220-7c37-4839-a0e3-b23c66502b74 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=75234) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Nov 29 03:19:11 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-be1e6180-8347-498b-b8a9-92cf51694217 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.498s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:19:11 user nova-compute[75234]: DEBUG nova.compute.manager [None req-be1e6180-8347-498b-b8a9-92cf51694217 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] [instance: 1df140f1-fbd9-4b5d-b356-d08b7e908f64] Start building networks asynchronously for instance. {{(pid=75234) _build_resources /opt/stack/nova/nova/compute/manager.py:2819}}
Nov 29 03:19:11 user nova-compute[75234]: DEBUG nova.compute.manager [None req-be1e6180-8347-498b-b8a9-92cf51694217 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] [instance: 1df140f1-fbd9-4b5d-b356-d08b7e908f64] Allocating IP information in the background. {{(pid=75234) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1972}}
Nov 29 03:19:11 user nova-compute[75234]: DEBUG nova.network.neutron [None req-be1e6180-8347-498b-b8a9-92cf51694217 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] [instance: 1df140f1-fbd9-4b5d-b356-d08b7e908f64] allocate_for_instance() {{(pid=75234) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1156}}
Nov 29 03:19:11 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-be1e6180-8347-498b-b8a9-92cf51694217 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] [instance: 1df140f1-fbd9-4b5d-b356-d08b7e908f64] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Nov 29 03:19:11 user nova-compute[75234]: DEBUG nova.compute.manager [None req-be1e6180-8347-498b-b8a9-92cf51694217 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] [instance: 1df140f1-fbd9-4b5d-b356-d08b7e908f64] Start building block device mappings for instance. {{(pid=75234) _build_resources /opt/stack/nova/nova/compute/manager.py:2854}}
Nov 29 03:19:11 user nova-compute[75234]: DEBUG nova.policy [None req-be1e6180-8347-498b-b8a9-92cf51694217 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '6792e866677645c598ddb7d2c1723a1a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f8a7884867c944e1ba1024270e4902f3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=75234) authorize /opt/stack/nova/nova/policy.py:203}}
Nov 29 03:19:11 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:19:11 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:19:11 user nova-compute[75234]: DEBUG nova.compute.manager [None req-be1e6180-8347-498b-b8a9-92cf51694217 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] [instance: 1df140f1-fbd9-4b5d-b356-d08b7e908f64] Start spawning the instance on the hypervisor. {{(pid=75234) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2628}}
Nov 29 03:19:11 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-be1e6180-8347-498b-b8a9-92cf51694217 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] [instance: 1df140f1-fbd9-4b5d-b356-d08b7e908f64] Creating instance directory {{(pid=75234) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4727}}
Nov 29 03:19:11 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-be1e6180-8347-498b-b8a9-92cf51694217 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] [instance: 1df140f1-fbd9-4b5d-b356-d08b7e908f64] Creating image(s)
Nov 29 03:19:11 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-be1e6180-8347-498b-b8a9-92cf51694217 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Acquiring lock "/opt/stack/data/nova/instances/1df140f1-fbd9-4b5d-b356-d08b7e908f64/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:19:11 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-be1e6180-8347-498b-b8a9-92cf51694217 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Lock "/opt/stack/data/nova/instances/1df140f1-fbd9-4b5d-b356-d08b7e908f64/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:19:11 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-be1e6180-8347-498b-b8a9-92cf51694217 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Lock "/opt/stack/data/nova/instances/1df140f1-fbd9-4b5d-b356-d08b7e908f64/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.002s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:19:11 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-be1e6180-8347-498b-b8a9-92cf51694217 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:19:11 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-be1e6180-8347-498b-b8a9-92cf51694217 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json" returned: 0 in 0.172s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:19:11 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-be1e6180-8347-498b-b8a9-92cf51694217 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Acquiring lock "80686e29dc9db6600d03e85cb62af298e12d7918" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:19:11 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-be1e6180-8347-498b-b8a9-92cf51694217 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Lock "80686e29dc9db6600d03e85cb62af298e12d7918" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:19:11 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-be1e6180-8347-498b-b8a9-92cf51694217 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:19:12 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:19:12 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-be1e6180-8347-498b-b8a9-92cf51694217 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json" returned: 0 in 0.168s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:19:12 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-be1e6180-8347-498b-b8a9-92cf51694217 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918,backing_fmt=raw /opt/stack/data/nova/instances/1df140f1-fbd9-4b5d-b356-d08b7e908f64/disk 1073741824 {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:19:12 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-be1e6180-8347-498b-b8a9-92cf51694217 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918,backing_fmt=raw /opt/stack/data/nova/instances/1df140f1-fbd9-4b5d-b356-d08b7e908f64/disk 1073741824" returned: 0 in 0.069s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:19:12 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-be1e6180-8347-498b-b8a9-92cf51694217 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Lock "80686e29dc9db6600d03e85cb62af298e12d7918" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.243s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:19:12 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-be1e6180-8347-498b-b8a9-92cf51694217 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:19:12 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-1006b7b9-8d9e-489e-90e9-503d7a3fd306 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Acquiring lock "aaf1c3e3-b73e-42ad-87bd-c5255ec09b46" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:19:12 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-1006b7b9-8d9e-489e-90e9-503d7a3fd306 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Lock "aaf1c3e3-b73e-42ad-87bd-c5255ec09b46" acquired by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: waited 0.002s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:19:12 user nova-compute[75234]: DEBUG nova.objects.instance [None req-1006b7b9-8d9e-489e-90e9-503d7a3fd306 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Lazy-loading 'flavor' on Instance uuid aaf1c3e3-b73e-42ad-87bd-c5255ec09b46 {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:19:12 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-be1e6180-8347-498b-b8a9-92cf51694217 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json" returned: 0 in 0.188s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:19:12 user nova-compute[75234]: DEBUG nova.virt.disk.api [None req-be1e6180-8347-498b-b8a9-92cf51694217 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Checking if we can resize image /opt/stack/data/nova/instances/1df140f1-fbd9-4b5d-b356-d08b7e908f64/disk. size=1073741824 {{(pid=75234) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:166}}
Nov 29 03:19:12 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-be1e6180-8347-498b-b8a9-92cf51694217 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/1df140f1-fbd9-4b5d-b356-d08b7e908f64/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:19:12 user nova-compute[75234]: DEBUG nova.network.neutron [None req-be1e6180-8347-498b-b8a9-92cf51694217 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] [instance: 1df140f1-fbd9-4b5d-b356-d08b7e908f64] Successfully created port: ea9a3586-da52-49b7-8ced-5d27806885ad {{(pid=75234) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:548}}
Nov 29 03:19:12 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-1006b7b9-8d9e-489e-90e9-503d7a3fd306 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Lock "aaf1c3e3-b73e-42ad-87bd-c5255ec09b46" "released" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: held 0.101s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:19:12 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-be1e6180-8347-498b-b8a9-92cf51694217 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/1df140f1-fbd9-4b5d-b356-d08b7e908f64/disk --force-share --output=json" returned: 0 in 0.196s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:19:12 user nova-compute[75234]: DEBUG nova.virt.disk.api [None req-be1e6180-8347-498b-b8a9-92cf51694217 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Cannot resize image /opt/stack/data/nova/instances/1df140f1-fbd9-4b5d-b356-d08b7e908f64/disk to a smaller size. {{(pid=75234) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:172}}
Nov 29 03:19:12 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-be1e6180-8347-498b-b8a9-92cf51694217 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] [instance: 1df140f1-fbd9-4b5d-b356-d08b7e908f64] Created local disks {{(pid=75234) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4861}}
Nov 29 03:19:12 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-be1e6180-8347-498b-b8a9-92cf51694217 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] [instance: 1df140f1-fbd9-4b5d-b356-d08b7e908f64] Ensure instance console log exists: /opt/stack/data/nova/instances/1df140f1-fbd9-4b5d-b356-d08b7e908f64/console.log {{(pid=75234) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4613}}
Nov 29 03:19:12 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-be1e6180-8347-498b-b8a9-92cf51694217 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:19:12 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-be1e6180-8347-498b-b8a9-92cf51694217 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:19:12 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-be1e6180-8347-498b-b8a9-92cf51694217 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:19:12 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:19:12 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-1006b7b9-8d9e-489e-90e9-503d7a3fd306 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Acquiring lock "aaf1c3e3-b73e-42ad-87bd-c5255ec09b46" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:19:12 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-1006b7b9-8d9e-489e-90e9-503d7a3fd306 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Lock "aaf1c3e3-b73e-42ad-87bd-c5255ec09b46" acquired by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:19:12 user nova-compute[75234]: INFO nova.compute.manager [None req-1006b7b9-8d9e-489e-90e9-503d7a3fd306 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] [instance: aaf1c3e3-b73e-42ad-87bd-c5255ec09b46] Attaching volume 9d734633-75e6-4a97-a3c5-bd9e45d70cca to /dev/vdc
Nov 29 03:19:12 user nova-compute[75234]: DEBUG os_brick.utils [None req-1006b7b9-8d9e-489e-90e9-503d7a3fd306 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:176}}
Nov 29 03:19:12 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:19:12 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.014s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:19:12 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[72637625-7219-464f-ac31-f5448c5ba8ff]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:dd1368c5ba2c\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:19:12 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt -v / -n -o SOURCE {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:19:12 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt -v / -n -o SOURCE" returned: 0 in 0.015s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:19:12 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[16ee1195-cdb0-4c7a-9c89-9e32704c4c1f]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:19:12 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:19:12 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.017s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:19:12 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[35bfbef3-45c9-4f3d-b12c-95bd6aa365f4]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:19:12 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[a1e63552-8500-4f12-9350-f209d687c08b]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:19:12 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-1006b7b9-8d9e-489e-90e9-503d7a3fd306 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Running cmd (subprocess): nvme version {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:19:12 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-1006b7b9-8d9e-489e-90e9-503d7a3fd306 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] 'nvme version' failed. Not Retrying. {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:473}}
Nov 29 03:19:12 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.nvmeof [None req-1006b7b9-8d9e-489e-90e9-503d7a3fd306 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] nvme not present on system {{(pid=75234) nvme_present /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/nvmeof.py:756}}
Nov 29 03:19:12 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:19:12 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:473}}
Nov 29 03:19:12 user nova-compute[75234]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Nov 29 03:19:12 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[d3fc5d3b-4990-4370-98bf-78a2cd7f388f]: (4, '') {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:19:12 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.lightos [None req-1006b7b9-8d9e-489e-90e9-503d7a3fd306 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=75234) find_dsc /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/lightos.py:98}}
Nov 29 03:19:12 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.lightos [None req-1006b7b9-8d9e-489e-90e9-503d7a3fd306 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=75234) get_connector_properties /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/lightos.py:76}}
Nov 29 03:19:12 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.lightos [None req-1006b7b9-8d9e-489e-90e9-503d7a3fd306 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] LIGHTOS: no hostnqn found. {{(pid=75234) get_connector_properties /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/lightos.py:84}}
Nov 29 03:19:12 user nova-compute[75234]: DEBUG os_brick.utils [None req-1006b7b9-8d9e-489e-90e9-503d7a3fd306 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] <== get_connector_properties: return (123ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:dd1368c5ba2c', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:203}}
Nov 29 03:19:12 user nova-compute[75234]: DEBUG nova.virt.block_device [None req-1006b7b9-8d9e-489e-90e9-503d7a3fd306 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] [instance: aaf1c3e3-b73e-42ad-87bd-c5255ec09b46] Updating existing volume attachment record: 1b34d318-b9ef-42ca-868e-eb4eff804000 {{(pid=75234) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Nov 29 03:19:13 user nova-compute[75234]: DEBUG nova.network.neutron [None req-be1e6180-8347-498b-b8a9-92cf51694217 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] [instance: 1df140f1-fbd9-4b5d-b356-d08b7e908f64] Successfully updated port: ea9a3586-da52-49b7-8ced-5d27806885ad {{(pid=75234) _update_port /opt/stack/nova/nova/network/neutron.py:586}}
Nov 29 03:19:13 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-be1e6180-8347-498b-b8a9-92cf51694217 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Acquiring lock "refresh_cache-1df140f1-fbd9-4b5d-b356-d08b7e908f64" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:19:13 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-be1e6180-8347-498b-b8a9-92cf51694217 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Acquired lock "refresh_cache-1df140f1-fbd9-4b5d-b356-d08b7e908f64" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:19:13 user nova-compute[75234]: DEBUG nova.network.neutron [None req-be1e6180-8347-498b-b8a9-92cf51694217 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] [instance: 1df140f1-fbd9-4b5d-b356-d08b7e908f64] Building network info cache for instance {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2009}}
Nov 29 03:19:13 user nova-compute[75234]: DEBUG nova.compute.manager [req-df00f97e-ac3e-4778-9e88-63b5e5c48c50 req-c230fbae-4486-47ce-96c7-201060781c69 service nova] [instance: 1df140f1-fbd9-4b5d-b356-d08b7e908f64] Received event network-changed-ea9a3586-da52-49b7-8ced-5d27806885ad {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:19:13 user nova-compute[75234]: DEBUG nova.compute.manager [req-df00f97e-ac3e-4778-9e88-63b5e5c48c50 req-c230fbae-4486-47ce-96c7-201060781c69 service nova] [instance: 1df140f1-fbd9-4b5d-b356-d08b7e908f64] Refreshing instance network info cache due to event network-changed-ea9a3586-da52-49b7-8ced-5d27806885ad. {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11117}}
Nov 29 03:19:13 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-df00f97e-ac3e-4778-9e88-63b5e5c48c50 req-c230fbae-4486-47ce-96c7-201060781c69 service nova] Acquiring lock "refresh_cache-1df140f1-fbd9-4b5d-b356-d08b7e908f64" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:19:13 user nova-compute[75234]: DEBUG nova.network.neutron [None req-be1e6180-8347-498b-b8a9-92cf51694217 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] [instance: 1df140f1-fbd9-4b5d-b356-d08b7e908f64] Instance cache missing network info. {{(pid=75234) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3322}}
Nov 29 03:19:13 user nova-compute[75234]: DEBUG nova.network.neutron [None req-be1e6180-8347-498b-b8a9-92cf51694217 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] [instance: 1df140f1-fbd9-4b5d-b356-d08b7e908f64] Updating instance_info_cache with network_info: [{"id": "ea9a3586-da52-49b7-8ced-5d27806885ad", "address": "fa:16:3e:80:25:b8", "network": {"id": "2b3c3140-f3b6-45da-8202-bb4d7f0dd130", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-2043420778-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "f8a7884867c944e1ba1024270e4902f3", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapea9a3586-da", "ovs_interfaceid": "ea9a3586-da52-49b7-8ced-5d27806885ad", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:19:13 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-be1e6180-8347-498b-b8a9-92cf51694217 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Releasing lock "refresh_cache-1df140f1-fbd9-4b5d-b356-d08b7e908f64" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:19:13 user nova-compute[75234]: DEBUG nova.compute.manager [None req-be1e6180-8347-498b-b8a9-92cf51694217 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] [instance: 1df140f1-fbd9-4b5d-b356-d08b7e908f64] Instance network_info: |[{"id": "ea9a3586-da52-49b7-8ced-5d27806885ad", "address": "fa:16:3e:80:25:b8", "network": {"id": "2b3c3140-f3b6-45da-8202-bb4d7f0dd130", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-2043420778-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "f8a7884867c944e1ba1024270e4902f3", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapea9a3586-da", "ovs_interfaceid": "ea9a3586-da52-49b7-8ced-5d27806885ad", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=75234) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1987}}
Nov 29 03:19:13 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-df00f97e-ac3e-4778-9e88-63b5e5c48c50 req-c230fbae-4486-47ce-96c7-201060781c69 service nova] Acquired lock "refresh_cache-1df140f1-fbd9-4b5d-b356-d08b7e908f64" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:19:13 user nova-compute[75234]: DEBUG nova.network.neutron [req-df00f97e-ac3e-4778-9e88-63b5e5c48c50 req-c230fbae-4486-47ce-96c7-201060781c69 service nova] [instance: 1df140f1-fbd9-4b5d-b356-d08b7e908f64] Refreshing network info cache for port ea9a3586-da52-49b7-8ced-5d27806885ad {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2006}}
Nov 29 03:19:13 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-be1e6180-8347-498b-b8a9-92cf51694217 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] [instance: 1df140f1-fbd9-4b5d-b356-d08b7e908f64] Start _get_guest_xml network_info=[{"id": "ea9a3586-da52-49b7-8ced-5d27806885ad", "address": "fa:16:3e:80:25:b8", "network": {"id": "2b3c3140-f3b6-45da-8202-bb4d7f0dd130", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-2043420778-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "f8a7884867c944e1ba1024270e4902f3", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapea9a3586-da", "ovs_interfaceid": "ea9a3586-da52-49b7-8ced-5d27806885ad", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum='c8fc807773e5354afe61636071771906',container_format='bare',created_at=2023-11-29T03:02:15Z,direct_url=<?>,disk_format='qcow2',id=96c30ba7-9b73-4a30-8ba4-7138879573b7,min_disk=0,min_ram=0,name='cirros-0.6.2-x86_64-disk',owner='b18e380921c748b78517b6f1211cd0f1',properties=ImageMetaProps,protected=<?>,size=21430272,status='active',tags=<?>,updated_at=2023-11-29T03:02:17Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [{'device_name': '/dev/vda', 'encryption_format': None, 'encrypted': False, 'device_type': 'disk', 'size': 0, 'disk_bus': 'virtio', 'boot_index': 0, 'encryption_options': None, 'encryption_secret_uuid': None, 'guest_format': None, 'image_id': '96c30ba7-9b73-4a30-8ba4-7138879573b7'}], 'ephemerals': [], 'block_device_mapping': [], 'swap': None} {{(pid=75234) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7562}}
Nov 29 03:19:13 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-be1e6180-8347-498b-b8a9-92cf51694217 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:19:13 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-be1e6180-8347-498b-b8a9-92cf51694217 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:19:13 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-be1e6180-8347-498b-b8a9-92cf51694217 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Searching host: 'user' for CPU controller through CGroups V1... {{(pid=75234) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1652}}
Nov 29 03:19:13 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-be1e6180-8347-498b-b8a9-92cf51694217 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] CPU controller missing on host. {{(pid=75234) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1662}}
Nov 29 03:19:13 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-be1e6180-8347-498b-b8a9-92cf51694217 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Searching host: 'user' for CPU controller through CGroups V2... {{(pid=75234) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1671}}
Nov 29 03:19:13 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-be1e6180-8347-498b-b8a9-92cf51694217 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] CPU controller found on host. {{(pid=75234) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1678}}
Nov 29 03:19:13 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-be1e6180-8347-498b-b8a9-92cf51694217 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=75234) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5400}}
Nov 29 03:19:13 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-be1e6180-8347-498b-b8a9-92cf51694217 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-11-29T03:02:04Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='1',id=6,is_public=True,memory_mb=512,name='m1.tiny',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='c8fc807773e5354afe61636071771906',container_format='bare',created_at=2023-11-29T03:02:15Z,direct_url=<?>,disk_format='qcow2',id=96c30ba7-9b73-4a30-8ba4-7138879573b7,min_disk=0,min_ram=0,name='cirros-0.6.2-x86_64-disk',owner='b18e380921c748b78517b6f1211cd0f1',properties=ImageMetaProps,protected=<?>,size=21430272,status='active',tags=<?>,updated_at=2023-11-29T03:02:17Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:563}}
Nov 29 03:19:13 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-be1e6180-8347-498b-b8a9-92cf51694217 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Flavor limits 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:348}}
Nov 29 03:19:13 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-be1e6180-8347-498b-b8a9-92cf51694217 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Image limits 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:352}}
Nov 29 03:19:13 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-be1e6180-8347-498b-b8a9-92cf51694217 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Flavor pref 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:388}}
Nov 29 03:19:13 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-be1e6180-8347-498b-b8a9-92cf51694217 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Image pref 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:392}}
Nov 29 03:19:13 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-be1e6180-8347-498b-b8a9-92cf51694217 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:430}}
Nov 29 03:19:13 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-be1e6180-8347-498b-b8a9-92cf51694217 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:569}}
Nov 29 03:19:13 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-be1e6180-8347-498b-b8a9-92cf51694217 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=75234) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:471}}
Nov 29 03:19:13 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-be1e6180-8347-498b-b8a9-92cf51694217 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Got 1 possible topologies {{(pid=75234) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:501}}
Nov 29 03:19:13 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-be1e6180-8347-498b-b8a9-92cf51694217 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:575}}
Nov 29 03:19:13 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-be1e6180-8347-498b-b8a9-92cf51694217 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:577}}
Nov 29 03:19:13 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-be1e6180-8347-498b-b8a9-92cf51694217 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-11-29T03:19:10Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-ServerRescueNegativeTestJSON-server-278539854',display_name='tempest-ServerRescueNegativeTestJSON-server-278539854',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-serverrescuenegativetestjson-server-278539854',id=33,image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBKySS+rrPWcijvoGGhKEPiK1xSDIj74zuelaF50GYBXUvR1Yet4BFb/2sYsZA/5NzUIJ43mYGg7Ig5cf2ANImb2nTP65fl4DOr9JzbUOlYxAqRtjhkf+S+OjV52cB6+jDw==',key_name='tempest-keypair-605296868',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='f8a7884867c944e1ba1024270e4902f3',ramdisk_id='',reservation_id='r-s6t6psv1',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-ServerRescueNegativeTestJSON-1182706601',owner_user_name='tempest-ServerRescueNegativeTestJSON-1182706601-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-11-29T03:19:12Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='6792e866677645c598ddb7d2c1723a1a',uuid=1df140f1-fbd9-4b5d-b356-d08b7e908f64,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "ea9a3586-da52-49b7-8ced-5d27806885ad", "address": "fa:16:3e:80:25:b8", "network": {"id": "2b3c3140-f3b6-45da-8202-bb4d7f0dd130", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-2043420778-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "f8a7884867c944e1ba1024270e4902f3", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapea9a3586-da", "ovs_interfaceid": "ea9a3586-da52-49b7-8ced-5d27806885ad", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=75234) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Nov 29 03:19:13 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-be1e6180-8347-498b-b8a9-92cf51694217 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Converting VIF {"id": "ea9a3586-da52-49b7-8ced-5d27806885ad", "address": "fa:16:3e:80:25:b8", "network": {"id": "2b3c3140-f3b6-45da-8202-bb4d7f0dd130", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-2043420778-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "f8a7884867c944e1ba1024270e4902f3", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapea9a3586-da", "ovs_interfaceid": "ea9a3586-da52-49b7-8ced-5d27806885ad", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:19:13 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-be1e6180-8347-498b-b8a9-92cf51694217 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:80:25:b8,bridge_name='br-int',has_traffic_filtering=True,id=ea9a3586-da52-49b7-8ced-5d27806885ad,network=Network(2b3c3140-f3b6-45da-8202-bb4d7f0dd130),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapea9a3586-da') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:19:13 user nova-compute[75234]: DEBUG nova.objects.instance [None req-be1e6180-8347-498b-b8a9-92cf51694217 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Lazy-loading 'pci_devices' on Instance uuid 1df140f1-fbd9-4b5d-b356-d08b7e908f64 {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:19:14 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-be1e6180-8347-498b-b8a9-92cf51694217 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] [instance: 1df140f1-fbd9-4b5d-b356-d08b7e908f64] End _get_guest_xml xml=<domain type="kvm">
Nov 29 03:19:14 user nova-compute[75234]:   <uuid>1df140f1-fbd9-4b5d-b356-d08b7e908f64</uuid>
Nov 29 03:19:14 user nova-compute[75234]:   <name>instance-00000021</name>
Nov 29 03:19:14 user nova-compute[75234]:   <memory>524288</memory>
Nov 29 03:19:14 user nova-compute[75234]:   <vcpu>1</vcpu>
Nov 29 03:19:14 user nova-compute[75234]:   <metadata>
Nov 29 03:19:14 user nova-compute[75234]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Nov 29 03:19:14 user nova-compute[75234]:       <nova:package version="0.0.0"/>
Nov 29 03:19:14 user nova-compute[75234]:       <nova:name>tempest-ServerRescueNegativeTestJSON-server-278539854</nova:name>
Nov 29 03:19:14 user nova-compute[75234]:       <nova:creationTime>2023-11-29 03:19:13</nova:creationTime>
Nov 29 03:19:14 user nova-compute[75234]:       <nova:flavor name="m1.tiny">
Nov 29 03:19:14 user nova-compute[75234]:         <nova:memory>512</nova:memory>
Nov 29 03:19:14 user nova-compute[75234]:         <nova:disk>1</nova:disk>
Nov 29 03:19:14 user nova-compute[75234]:         <nova:swap>0</nova:swap>
Nov 29 03:19:14 user nova-compute[75234]:         <nova:ephemeral>0</nova:ephemeral>
Nov 29 03:19:14 user nova-compute[75234]:         <nova:vcpus>1</nova:vcpus>
Nov 29 03:19:14 user nova-compute[75234]:       </nova:flavor>
Nov 29 03:19:14 user nova-compute[75234]:       <nova:owner>
Nov 29 03:19:14 user nova-compute[75234]:         <nova:user uuid="6792e866677645c598ddb7d2c1723a1a">tempest-ServerRescueNegativeTestJSON-1182706601-project-member</nova:user>
Nov 29 03:19:14 user nova-compute[75234]:         <nova:project uuid="f8a7884867c944e1ba1024270e4902f3">tempest-ServerRescueNegativeTestJSON-1182706601</nova:project>
Nov 29 03:19:14 user nova-compute[75234]:       </nova:owner>
Nov 29 03:19:14 user nova-compute[75234]:       <nova:root type="image" uuid="96c30ba7-9b73-4a30-8ba4-7138879573b7"/>
Nov 29 03:19:14 user nova-compute[75234]:       <nova:ports>
Nov 29 03:19:14 user nova-compute[75234]:         <nova:port uuid="ea9a3586-da52-49b7-8ced-5d27806885ad">
Nov 29 03:19:14 user nova-compute[75234]:           <nova:ip type="fixed" address="10.0.0.3" ipVersion="4"/>
Nov 29 03:19:14 user nova-compute[75234]:         </nova:port>
Nov 29 03:19:14 user nova-compute[75234]:       </nova:ports>
Nov 29 03:19:14 user nova-compute[75234]:     </nova:instance>
Nov 29 03:19:14 user nova-compute[75234]:   </metadata>
Nov 29 03:19:14 user nova-compute[75234]:   <sysinfo type="smbios">
Nov 29 03:19:14 user nova-compute[75234]:     <system>
Nov 29 03:19:14 user nova-compute[75234]:       <entry name="manufacturer">OpenStack Foundation</entry>
Nov 29 03:19:14 user nova-compute[75234]:       <entry name="product">OpenStack Nova</entry>
Nov 29 03:19:14 user nova-compute[75234]:       <entry name="version">0.0.0</entry>
Nov 29 03:19:14 user nova-compute[75234]:       <entry name="serial">1df140f1-fbd9-4b5d-b356-d08b7e908f64</entry>
Nov 29 03:19:14 user nova-compute[75234]:       <entry name="uuid">1df140f1-fbd9-4b5d-b356-d08b7e908f64</entry>
Nov 29 03:19:14 user nova-compute[75234]:       <entry name="family">Virtual Machine</entry>
Nov 29 03:19:14 user nova-compute[75234]:     </system>
Nov 29 03:19:14 user nova-compute[75234]:   </sysinfo>
Nov 29 03:19:14 user nova-compute[75234]:   <os>
Nov 29 03:19:14 user nova-compute[75234]:     <type arch="x86_64" machine="pc">hvm</type>
Nov 29 03:19:14 user nova-compute[75234]:     <boot dev="hd"/>
Nov 29 03:19:14 user nova-compute[75234]:     <smbios mode="sysinfo"/>
Nov 29 03:19:14 user nova-compute[75234]:   </os>
Nov 29 03:19:14 user nova-compute[75234]:   <features>
Nov 29 03:19:14 user nova-compute[75234]:     <acpi/>
Nov 29 03:19:14 user nova-compute[75234]:     <apic/>
Nov 29 03:19:14 user nova-compute[75234]:     <vmcoreinfo/>
Nov 29 03:19:14 user nova-compute[75234]:   </features>
Nov 29 03:19:14 user nova-compute[75234]:   <clock offset="utc">
Nov 29 03:19:14 user nova-compute[75234]:     <timer name="pit" tickpolicy="delay"/>
Nov 29 03:19:14 user nova-compute[75234]:     <timer name="rtc" tickpolicy="catchup"/>
Nov 29 03:19:14 user nova-compute[75234]:     <timer name="hpet" present="no"/>
Nov 29 03:19:14 user nova-compute[75234]:   </clock>
Nov 29 03:19:14 user nova-compute[75234]:   <cpu mode="custom" match="exact">
Nov 29 03:19:14 user nova-compute[75234]:     <model>Nehalem</model>
Nov 29 03:19:14 user nova-compute[75234]:     <topology sockets="1" cores="1" threads="1"/>
Nov 29 03:19:14 user nova-compute[75234]:   </cpu>
Nov 29 03:19:14 user nova-compute[75234]:   <devices>
Nov 29 03:19:14 user nova-compute[75234]:     <disk type="file" device="disk">
Nov 29 03:19:14 user nova-compute[75234]:       <driver name="qemu" type="qcow2" cache="none"/>
Nov 29 03:19:14 user nova-compute[75234]:       <source file="/opt/stack/data/nova/instances/1df140f1-fbd9-4b5d-b356-d08b7e908f64/disk"/>
Nov 29 03:19:14 user nova-compute[75234]:       <target dev="vda" bus="virtio"/>
Nov 29 03:19:14 user nova-compute[75234]:     </disk>
Nov 29 03:19:14 user nova-compute[75234]:     <interface type="ethernet">
Nov 29 03:19:14 user nova-compute[75234]:       <mac address="fa:16:3e:80:25:b8"/>
Nov 29 03:19:14 user nova-compute[75234]:       <model type="virtio"/>
Nov 29 03:19:14 user nova-compute[75234]:       <mtu size="1442"/>
Nov 29 03:19:14 user nova-compute[75234]:       <target dev="tapea9a3586-da"/>
Nov 29 03:19:14 user nova-compute[75234]:     </interface>
Nov 29 03:19:14 user nova-compute[75234]:     <serial type="pty">
Nov 29 03:19:14 user nova-compute[75234]:       <log file="/opt/stack/data/nova/instances/1df140f1-fbd9-4b5d-b356-d08b7e908f64/console.log" append="off"/>
Nov 29 03:19:14 user nova-compute[75234]:     </serial>
Nov 29 03:19:14 user nova-compute[75234]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Nov 29 03:19:14 user nova-compute[75234]:     <video>
Nov 29 03:19:14 user nova-compute[75234]:       <model type="virtio"/>
Nov 29 03:19:14 user nova-compute[75234]:     </video>
Nov 29 03:19:14 user nova-compute[75234]:     <rng model="virtio">
Nov 29 03:19:14 user nova-compute[75234]:       <backend model="random">/dev/urandom</backend>
Nov 29 03:19:14 user nova-compute[75234]:     </rng>
Nov 29 03:19:14 user nova-compute[75234]:     <controller type="usb" index="0" model="none"/>
Nov 29 03:19:14 user nova-compute[75234]:     <memballoon model="virtio">
Nov 29 03:19:14 user nova-compute[75234]:       <stats period="10"/>
Nov 29 03:19:14 user nova-compute[75234]:     </memballoon>
Nov 29 03:19:14 user nova-compute[75234]:   </devices>
Nov 29 03:19:14 user nova-compute[75234]: </domain>
Nov 29 03:19:14 user nova-compute[75234]:  {{(pid=75234) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7568}}
Nov 29 03:19:14 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-be1e6180-8347-498b-b8a9-92cf51694217 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-11-29T03:19:10Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-ServerRescueNegativeTestJSON-server-278539854',display_name='tempest-ServerRescueNegativeTestJSON-server-278539854',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-serverrescuenegativetestjson-server-278539854',id=33,image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBKySS+rrPWcijvoGGhKEPiK1xSDIj74zuelaF50GYBXUvR1Yet4BFb/2sYsZA/5NzUIJ43mYGg7Ig5cf2ANImb2nTP65fl4DOr9JzbUOlYxAqRtjhkf+S+OjV52cB6+jDw==',key_name='tempest-keypair-605296868',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='f8a7884867c944e1ba1024270e4902f3',ramdisk_id='',reservation_id='r-s6t6psv1',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-ServerRescueNegativeTestJSON-1182706601',owner_user_name='tempest-ServerRescueNegativeTestJSON-1182706601-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-11-29T03:19:12Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='6792e866677645c598ddb7d2c1723a1a',uuid=1df140f1-fbd9-4b5d-b356-d08b7e908f64,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "ea9a3586-da52-49b7-8ced-5d27806885ad", "address": "fa:16:3e:80:25:b8", "network": {"id": "2b3c3140-f3b6-45da-8202-bb4d7f0dd130", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-2043420778-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "f8a7884867c944e1ba1024270e4902f3", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapea9a3586-da", "ovs_interfaceid": "ea9a3586-da52-49b7-8ced-5d27806885ad", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Nov 29 03:19:14 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-be1e6180-8347-498b-b8a9-92cf51694217 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Converting VIF {"id": "ea9a3586-da52-49b7-8ced-5d27806885ad", "address": "fa:16:3e:80:25:b8", "network": {"id": "2b3c3140-f3b6-45da-8202-bb4d7f0dd130", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-2043420778-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "f8a7884867c944e1ba1024270e4902f3", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapea9a3586-da", "ovs_interfaceid": "ea9a3586-da52-49b7-8ced-5d27806885ad", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:19:14 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-be1e6180-8347-498b-b8a9-92cf51694217 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:80:25:b8,bridge_name='br-int',has_traffic_filtering=True,id=ea9a3586-da52-49b7-8ced-5d27806885ad,network=Network(2b3c3140-f3b6-45da-8202-bb4d7f0dd130),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapea9a3586-da') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:19:14 user nova-compute[75234]: DEBUG os_vif [None req-be1e6180-8347-498b-b8a9-92cf51694217 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:80:25:b8,bridge_name='br-int',has_traffic_filtering=True,id=ea9a3586-da52-49b7-8ced-5d27806885ad,network=Network(2b3c3140-f3b6-45da-8202-bb4d7f0dd130),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapea9a3586-da') {{(pid=75234) plug /opt/stack/data/venv/lib/python3.10/site-packages/os_vif/__init__.py:76}}
Nov 29 03:19:14 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:19:14 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:19:14 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Nov 29 03:19:14 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:19:14 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbCreateCommand(_result=None, table=QoS, columns={'type': 'linux-noop', 'external_ids': {'id': '9f726120-16ce-59cf-af7e-6e9471defcf0', '_type': 'linux-noop'}}, row=False) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:19:14 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:19:14 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:19:14 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:19:14 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tapea9a3586-da, may_exist=True, interface_attrs={}) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:19:14 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Port, record=tapea9a3586-da, col_values=(('qos', UUID('67201c1a-050b-493c-9462-e0f47e8035fd')),), if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:19:14 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DbSetCommand(_result=None, table=Interface, record=tapea9a3586-da, col_values=(('external_ids', {'iface-id': 'ea9a3586-da52-49b7-8ced-5d27806885ad', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:80:25:b8', 'vm-uuid': '1df140f1-fbd9-4b5d-b356-d08b7e908f64'}),), if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:19:14 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:19:14 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:19:14 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:19:14 user nova-compute[75234]: INFO os_vif [None req-be1e6180-8347-498b-b8a9-92cf51694217 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:80:25:b8,bridge_name='br-int',has_traffic_filtering=True,id=ea9a3586-da52-49b7-8ced-5d27806885ad,network=Network(2b3c3140-f3b6-45da-8202-bb4d7f0dd130),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapea9a3586-da')
Nov 29 03:19:14 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-be1e6180-8347-498b-b8a9-92cf51694217 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] No BDM found with device name vda, not building metadata. {{(pid=75234) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12089}}
Nov 29 03:19:14 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-be1e6180-8347-498b-b8a9-92cf51694217 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] No VIF found with MAC fa:16:3e:80:25:b8, not building metadata {{(pid=75234) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Nov 29 03:19:14 user nova-compute[75234]: DEBUG nova.network.neutron [req-df00f97e-ac3e-4778-9e88-63b5e5c48c50 req-c230fbae-4486-47ce-96c7-201060781c69 service nova] [instance: 1df140f1-fbd9-4b5d-b356-d08b7e908f64] Updated VIF entry in instance network info cache for port ea9a3586-da52-49b7-8ced-5d27806885ad. {{(pid=75234) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3481}}
Nov 29 03:19:14 user nova-compute[75234]: DEBUG nova.network.neutron [req-df00f97e-ac3e-4778-9e88-63b5e5c48c50 req-c230fbae-4486-47ce-96c7-201060781c69 service nova] [instance: 1df140f1-fbd9-4b5d-b356-d08b7e908f64] Updating instance_info_cache with network_info: [{"id": "ea9a3586-da52-49b7-8ced-5d27806885ad", "address": "fa:16:3e:80:25:b8", "network": {"id": "2b3c3140-f3b6-45da-8202-bb4d7f0dd130", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-2043420778-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "f8a7884867c944e1ba1024270e4902f3", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapea9a3586-da", "ovs_interfaceid": "ea9a3586-da52-49b7-8ced-5d27806885ad", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:19:14 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-df00f97e-ac3e-4778-9e88-63b5e5c48c50 req-c230fbae-4486-47ce-96c7-201060781c69 service nova] Releasing lock "refresh_cache-1df140f1-fbd9-4b5d-b356-d08b7e908f64" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:19:15 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:19:15 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:19:15 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:19:15 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:19:15 user nova-compute[75234]: DEBUG nova.compute.manager [req-65cb8b17-4d7a-4498-8836-e08fb3b3448b req-2e127ba3-b112-4cc1-8749-8554a010d1ef service nova] [instance: 1df140f1-fbd9-4b5d-b356-d08b7e908f64] Received event network-vif-plugged-ea9a3586-da52-49b7-8ced-5d27806885ad {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:19:15 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-65cb8b17-4d7a-4498-8836-e08fb3b3448b req-2e127ba3-b112-4cc1-8749-8554a010d1ef service nova] Acquiring lock "1df140f1-fbd9-4b5d-b356-d08b7e908f64-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:19:15 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-65cb8b17-4d7a-4498-8836-e08fb3b3448b req-2e127ba3-b112-4cc1-8749-8554a010d1ef service nova] Lock "1df140f1-fbd9-4b5d-b356-d08b7e908f64-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:19:15 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-65cb8b17-4d7a-4498-8836-e08fb3b3448b req-2e127ba3-b112-4cc1-8749-8554a010d1ef service nova] Lock "1df140f1-fbd9-4b5d-b356-d08b7e908f64-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:19:15 user nova-compute[75234]: DEBUG nova.compute.manager [req-65cb8b17-4d7a-4498-8836-e08fb3b3448b req-2e127ba3-b112-4cc1-8749-8554a010d1ef service nova] [instance: 1df140f1-fbd9-4b5d-b356-d08b7e908f64] No waiting events found dispatching network-vif-plugged-ea9a3586-da52-49b7-8ced-5d27806885ad {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:19:15 user nova-compute[75234]: WARNING nova.compute.manager [req-65cb8b17-4d7a-4498-8836-e08fb3b3448b req-2e127ba3-b112-4cc1-8749-8554a010d1ef service nova] [instance: 1df140f1-fbd9-4b5d-b356-d08b7e908f64] Received unexpected event network-vif-plugged-ea9a3586-da52-49b7-8ced-5d27806885ad for instance with vm_state building and task_state spawning.
Nov 29 03:19:17 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:19:17 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ba301a44-9f9b-4af1-ae20-14afd8363114 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Acquiring lock "72ebd454-0275-4c3d-95d2-227ffb045a0c" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:19:17 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ba301a44-9f9b-4af1-ae20-14afd8363114 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Lock "72ebd454-0275-4c3d-95d2-227ffb045a0c" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:19:17 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ba301a44-9f9b-4af1-ae20-14afd8363114 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] [instance: 72ebd454-0275-4c3d-95d2-227ffb045a0c] Starting instance... {{(pid=75234) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2422}}
Nov 29 03:19:17 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ba301a44-9f9b-4af1-ae20-14afd8363114 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:19:17 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ba301a44-9f9b-4af1-ae20-14afd8363114 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:19:17 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-ba301a44-9f9b-4af1-ae20-14afd8363114 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=75234) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2366}}
Nov 29 03:19:17 user nova-compute[75234]: INFO nova.compute.claims [None req-ba301a44-9f9b-4af1-ae20-14afd8363114 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] [instance: 72ebd454-0275-4c3d-95d2-227ffb045a0c] Claim successful on node user
Nov 29 03:19:17 user nova-compute[75234]: DEBUG nova.compute.manager [None req-be1e6180-8347-498b-b8a9-92cf51694217 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] [instance: 1df140f1-fbd9-4b5d-b356-d08b7e908f64] Instance event wait completed in 0 seconds for  {{(pid=75234) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Nov 29 03:19:17 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-be1e6180-8347-498b-b8a9-92cf51694217 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] [instance: 1df140f1-fbd9-4b5d-b356-d08b7e908f64] Guest created on hypervisor {{(pid=75234) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4421}}
Nov 29 03:19:17 user nova-compute[75234]: INFO nova.virt.libvirt.driver [-] [instance: 1df140f1-fbd9-4b5d-b356-d08b7e908f64] Instance spawned successfully.
Nov 29 03:19:17 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-be1e6180-8347-498b-b8a9-92cf51694217 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] [instance: 1df140f1-fbd9-4b5d-b356-d08b7e908f64] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:904}}
Nov 29 03:19:17 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-be1e6180-8347-498b-b8a9-92cf51694217 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] [instance: 1df140f1-fbd9-4b5d-b356-d08b7e908f64] Found default for hw_cdrom_bus of ide {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:19:17 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-be1e6180-8347-498b-b8a9-92cf51694217 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] [instance: 1df140f1-fbd9-4b5d-b356-d08b7e908f64] Found default for hw_disk_bus of virtio {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:19:17 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-be1e6180-8347-498b-b8a9-92cf51694217 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] [instance: 1df140f1-fbd9-4b5d-b356-d08b7e908f64] Found default for hw_input_bus of None {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:19:17 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-be1e6180-8347-498b-b8a9-92cf51694217 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] [instance: 1df140f1-fbd9-4b5d-b356-d08b7e908f64] Found default for hw_pointer_model of None {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:19:17 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-be1e6180-8347-498b-b8a9-92cf51694217 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] [instance: 1df140f1-fbd9-4b5d-b356-d08b7e908f64] Found default for hw_video_model of virtio {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:19:17 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-be1e6180-8347-498b-b8a9-92cf51694217 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] [instance: 1df140f1-fbd9-4b5d-b356-d08b7e908f64] Found default for hw_vif_model of virtio {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:19:17 user nova-compute[75234]: INFO nova.compute.manager [None req-be1e6180-8347-498b-b8a9-92cf51694217 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] [instance: 1df140f1-fbd9-4b5d-b356-d08b7e908f64] Took 6.02 seconds to spawn the instance on the hypervisor.
Nov 29 03:19:17 user nova-compute[75234]: DEBUG nova.compute.manager [None req-be1e6180-8347-498b-b8a9-92cf51694217 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] [instance: 1df140f1-fbd9-4b5d-b356-d08b7e908f64] Checking state {{(pid=75234) _get_power_state /opt/stack/nova/nova/compute/manager.py:1782}}
Nov 29 03:19:17 user nova-compute[75234]: DEBUG nova.compute.manager [req-159b523d-82f6-4a45-9536-95d43b1886f0 req-9325f016-3c96-40a9-ae38-662a2e8bf1c0 service nova] [instance: 1df140f1-fbd9-4b5d-b356-d08b7e908f64] Received event network-vif-plugged-ea9a3586-da52-49b7-8ced-5d27806885ad {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:19:17 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-159b523d-82f6-4a45-9536-95d43b1886f0 req-9325f016-3c96-40a9-ae38-662a2e8bf1c0 service nova] Acquiring lock "1df140f1-fbd9-4b5d-b356-d08b7e908f64-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:19:17 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-159b523d-82f6-4a45-9536-95d43b1886f0 req-9325f016-3c96-40a9-ae38-662a2e8bf1c0 service nova] Lock "1df140f1-fbd9-4b5d-b356-d08b7e908f64-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:19:17 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-159b523d-82f6-4a45-9536-95d43b1886f0 req-9325f016-3c96-40a9-ae38-662a2e8bf1c0 service nova] Lock "1df140f1-fbd9-4b5d-b356-d08b7e908f64-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:19:17 user nova-compute[75234]: DEBUG nova.compute.manager [req-159b523d-82f6-4a45-9536-95d43b1886f0 req-9325f016-3c96-40a9-ae38-662a2e8bf1c0 service nova] [instance: 1df140f1-fbd9-4b5d-b356-d08b7e908f64] No waiting events found dispatching network-vif-plugged-ea9a3586-da52-49b7-8ced-5d27806885ad {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:19:17 user nova-compute[75234]: WARNING nova.compute.manager [req-159b523d-82f6-4a45-9536-95d43b1886f0 req-9325f016-3c96-40a9-ae38-662a2e8bf1c0 service nova] [instance: 1df140f1-fbd9-4b5d-b356-d08b7e908f64] Received unexpected event network-vif-plugged-ea9a3586-da52-49b7-8ced-5d27806885ad for instance with vm_state building and task_state spawning.
Nov 29 03:19:17 user nova-compute[75234]: INFO nova.compute.manager [None req-be1e6180-8347-498b-b8a9-92cf51694217 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] [instance: 1df140f1-fbd9-4b5d-b356-d08b7e908f64] Took 6.86 seconds to build instance.
Nov 29 03:19:17 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-be1e6180-8347-498b-b8a9-92cf51694217 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Lock "1df140f1-fbd9-4b5d-b356-d08b7e908f64" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 6.954s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:19:17 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ec0d94a4-4e08-4b8e-97dd-ecb8d660b350 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Acquiring lock "1c8b7559-29ab-4982-8c75-ea2a0f0101cb" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:19:17 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ec0d94a4-4e08-4b8e-97dd-ecb8d660b350 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Lock "1c8b7559-29ab-4982-8c75-ea2a0f0101cb" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:19:17 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ec0d94a4-4e08-4b8e-97dd-ecb8d660b350 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Acquiring lock "1c8b7559-29ab-4982-8c75-ea2a0f0101cb-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:19:17 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ec0d94a4-4e08-4b8e-97dd-ecb8d660b350 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Lock "1c8b7559-29ab-4982-8c75-ea2a0f0101cb-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:19:17 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ec0d94a4-4e08-4b8e-97dd-ecb8d660b350 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Lock "1c8b7559-29ab-4982-8c75-ea2a0f0101cb-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:19:17 user nova-compute[75234]: INFO nova.compute.manager [None req-ec0d94a4-4e08-4b8e-97dd-ecb8d660b350 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] [instance: 1c8b7559-29ab-4982-8c75-ea2a0f0101cb] Terminating instance
Nov 29 03:19:17 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ec0d94a4-4e08-4b8e-97dd-ecb8d660b350 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] [instance: 1c8b7559-29ab-4982-8c75-ea2a0f0101cb] Start destroying the instance on the hypervisor. {{(pid=75234) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3140}}
Nov 29 03:19:18 user nova-compute[75234]: DEBUG nova.compute.provider_tree [None req-ba301a44-9f9b-4af1-ae20-14afd8363114 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Inventory has not changed in ProviderTree for provider: 4d291220-7c37-4839-a0e3-b23c66502b74 {{(pid=75234) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Nov 29 03:19:18 user nova-compute[75234]: DEBUG nova.scheduler.client.report [None req-ba301a44-9f9b-4af1-ae20-14afd8363114 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Inventory has not changed for provider 4d291220-7c37-4839-a0e3-b23c66502b74 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=75234) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Nov 29 03:19:18 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ba301a44-9f9b-4af1-ae20-14afd8363114 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.698s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:19:18 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ba301a44-9f9b-4af1-ae20-14afd8363114 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] [instance: 72ebd454-0275-4c3d-95d2-227ffb045a0c] Start building networks asynchronously for instance. {{(pid=75234) _build_resources /opt/stack/nova/nova/compute/manager.py:2819}}
Nov 29 03:19:18 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:19:18 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ba301a44-9f9b-4af1-ae20-14afd8363114 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] [instance: 72ebd454-0275-4c3d-95d2-227ffb045a0c] Allocating IP information in the background. {{(pid=75234) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1972}}
Nov 29 03:19:18 user nova-compute[75234]: DEBUG nova.network.neutron [None req-ba301a44-9f9b-4af1-ae20-14afd8363114 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] [instance: 72ebd454-0275-4c3d-95d2-227ffb045a0c] allocate_for_instance() {{(pid=75234) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1156}}
Nov 29 03:19:18 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:19:18 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-ba301a44-9f9b-4af1-ae20-14afd8363114 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] [instance: 72ebd454-0275-4c3d-95d2-227ffb045a0c] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Nov 29 03:19:18 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ba301a44-9f9b-4af1-ae20-14afd8363114 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] [instance: 72ebd454-0275-4c3d-95d2-227ffb045a0c] Start building block device mappings for instance. {{(pid=75234) _build_resources /opt/stack/nova/nova/compute/manager.py:2854}}
Nov 29 03:19:18 user nova-compute[75234]: DEBUG nova.policy [None req-ba301a44-9f9b-4af1-ae20-14afd8363114 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '899f638d0de34dcba53f4c2ef7f5e068', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7e24dd8accea4c1e9598d643ea79c86d', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=75234) authorize /opt/stack/nova/nova/policy.py:203}}
Nov 29 03:19:18 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ba301a44-9f9b-4af1-ae20-14afd8363114 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] [instance: 72ebd454-0275-4c3d-95d2-227ffb045a0c] Start spawning the instance on the hypervisor. {{(pid=75234) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2628}}
Nov 29 03:19:18 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-ba301a44-9f9b-4af1-ae20-14afd8363114 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] [instance: 72ebd454-0275-4c3d-95d2-227ffb045a0c] Creating instance directory {{(pid=75234) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4727}}
Nov 29 03:19:18 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-ba301a44-9f9b-4af1-ae20-14afd8363114 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] [instance: 72ebd454-0275-4c3d-95d2-227ffb045a0c] Creating image(s)
Nov 29 03:19:18 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ba301a44-9f9b-4af1-ae20-14afd8363114 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Acquiring lock "/opt/stack/data/nova/instances/72ebd454-0275-4c3d-95d2-227ffb045a0c/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:19:18 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ba301a44-9f9b-4af1-ae20-14afd8363114 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Lock "/opt/stack/data/nova/instances/72ebd454-0275-4c3d-95d2-227ffb045a0c/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:19:18 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ba301a44-9f9b-4af1-ae20-14afd8363114 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Lock "/opt/stack/data/nova/instances/72ebd454-0275-4c3d-95d2-227ffb045a0c/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:19:18 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ba301a44-9f9b-4af1-ae20-14afd8363114 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:19:18 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:19:18 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:19:18 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ba301a44-9f9b-4af1-ae20-14afd8363114 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json" returned: 0 in 0.220s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:19:18 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ba301a44-9f9b-4af1-ae20-14afd8363114 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Acquiring lock "80686e29dc9db6600d03e85cb62af298e12d7918" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:19:18 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ba301a44-9f9b-4af1-ae20-14afd8363114 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Lock "80686e29dc9db6600d03e85cb62af298e12d7918" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:19:18 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ba301a44-9f9b-4af1-ae20-14afd8363114 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:19:18 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ba301a44-9f9b-4af1-ae20-14afd8363114 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json" returned: 0 in 0.189s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:19:18 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ba301a44-9f9b-4af1-ae20-14afd8363114 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918,backing_fmt=raw /opt/stack/data/nova/instances/72ebd454-0275-4c3d-95d2-227ffb045a0c/disk 1073741824 {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:19:18 user nova-compute[75234]: INFO nova.virt.libvirt.driver [-] [instance: 1c8b7559-29ab-4982-8c75-ea2a0f0101cb] Instance destroyed successfully.
Nov 29 03:19:18 user nova-compute[75234]: DEBUG nova.objects.instance [None req-ec0d94a4-4e08-4b8e-97dd-ecb8d660b350 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Lazy-loading 'resources' on Instance uuid 1c8b7559-29ab-4982-8c75-ea2a0f0101cb {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:19:18 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ba301a44-9f9b-4af1-ae20-14afd8363114 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918,backing_fmt=raw /opt/stack/data/nova/instances/72ebd454-0275-4c3d-95d2-227ffb045a0c/disk 1073741824" returned: 0 in 0.063s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:19:18 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ba301a44-9f9b-4af1-ae20-14afd8363114 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Lock "80686e29dc9db6600d03e85cb62af298e12d7918" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.258s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:19:18 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ba301a44-9f9b-4af1-ae20-14afd8363114 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:19:18 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-ec0d94a4-4e08-4b8e-97dd-ecb8d660b350 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-11-29T03:16:03Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-TestStampPattern-server-785913193',display_name='tempest-TestStampPattern-server-785913193',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-teststamppattern-server-785913193',id=26,image_ref='5e298e24-6bfe-4aab-ac7a-4b54d9baa799',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBMhfiLNgC9PAWcr8xAscyt4FB2hoan+iXcHV+JRry3FLwwpkjQpDEiOvBF2D+f1pgs1A/1GOV9t1zFfztYSi2DW0KkfBHuJTofrSJvEENNcXO0BnR5RakzAYYv+USO8uHQ==',key_name='tempest-TestStampPattern-1617966982',keypairs=<?>,launch_index=0,launched_at=2023-11-29T03:16:15Z,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='c4393bc598a845a9bfb667e5332f14a6',ramdisk_id='',reservation_id='r-7smjv19e',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',image_boot_roles='reader,member',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_image_location='snapshot',image_image_state='available',image_image_type='snapshot',image_instance_uuid='6894b55f-93c6-43c9-8933-56d5a3c43725',image_min_disk='1',image_min_ram='0',image_owner_id='c4393bc598a845a9bfb667e5332f14a6',image_owner_project_name='tempest-TestStampPattern-448969956',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',image_owner_user_name='tempest-TestStampPattern-448969956-project-member',image_user_id='dc1de1dad3794b47b0e4bf7445c96997',owner_project_name='tempest-TestStampPattern-448969956',owner_user_name='tempest-TestStampPattern-448969956-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-11-29T03:16:16Z,user_data=None,user_id='dc1de1dad3794b47b0e4bf7445c96997',uuid=1c8b7559-29ab-4982-8c75-ea2a0f0101cb,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "22cdbc69-d4f4-49cd-8104-d508aea94101", "address": "fa:16:3e:79:d1:ca", "network": {"id": "b7a6cef0-4cf1-46e6-99fb-4972879e3913", "bridge": "br-int", "label": "tempest-TestStampPattern-1195861223-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.102", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "c4393bc598a845a9bfb667e5332f14a6", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap22cdbc69-d4", "ovs_interfaceid": "22cdbc69-d4f4-49cd-8104-d508aea94101", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Nov 29 03:19:18 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-ec0d94a4-4e08-4b8e-97dd-ecb8d660b350 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Converting VIF {"id": "22cdbc69-d4f4-49cd-8104-d508aea94101", "address": "fa:16:3e:79:d1:ca", "network": {"id": "b7a6cef0-4cf1-46e6-99fb-4972879e3913", "bridge": "br-int", "label": "tempest-TestStampPattern-1195861223-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.102", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "c4393bc598a845a9bfb667e5332f14a6", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap22cdbc69-d4", "ovs_interfaceid": "22cdbc69-d4f4-49cd-8104-d508aea94101", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:19:18 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-ec0d94a4-4e08-4b8e-97dd-ecb8d660b350 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:79:d1:ca,bridge_name='br-int',has_traffic_filtering=True,id=22cdbc69-d4f4-49cd-8104-d508aea94101,network=Network(b7a6cef0-4cf1-46e6-99fb-4972879e3913),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap22cdbc69-d4') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:19:18 user nova-compute[75234]: DEBUG os_vif [None req-ec0d94a4-4e08-4b8e-97dd-ecb8d660b350 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Unplugging vif VIFOpenVSwitch(active=True,address=fa:16:3e:79:d1:ca,bridge_name='br-int',has_traffic_filtering=True,id=22cdbc69-d4f4-49cd-8104-d508aea94101,network=Network(b7a6cef0-4cf1-46e6-99fb-4972879e3913),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap22cdbc69-d4') {{(pid=75234) unplug /opt/stack/data/venv/lib/python3.10/site-packages/os_vif/__init__.py:109}}
Nov 29 03:19:19 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:19:19 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tap22cdbc69-d4, bridge=br-int, if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:19:19 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:19:19 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:19:19 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:19:19 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbDestroyCommand(_result=None, table=QoS, record=19072a08-5435-46a0-a950-c5e482f57315) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:19:19 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:19:19 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:19:19 user nova-compute[75234]: INFO os_vif [None req-ec0d94a4-4e08-4b8e-97dd-ecb8d660b350 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Successfully unplugged vif VIFOpenVSwitch(active=True,address=fa:16:3e:79:d1:ca,bridge_name='br-int',has_traffic_filtering=True,id=22cdbc69-d4f4-49cd-8104-d508aea94101,network=Network(b7a6cef0-4cf1-46e6-99fb-4972879e3913),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap22cdbc69-d4')
Nov 29 03:19:19 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-ec0d94a4-4e08-4b8e-97dd-ecb8d660b350 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] [instance: 1c8b7559-29ab-4982-8c75-ea2a0f0101cb] Deleting instance files /opt/stack/data/nova/instances/1c8b7559-29ab-4982-8c75-ea2a0f0101cb_del
Nov 29 03:19:19 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-ec0d94a4-4e08-4b8e-97dd-ecb8d660b350 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] [instance: 1c8b7559-29ab-4982-8c75-ea2a0f0101cb] Deletion of /opt/stack/data/nova/instances/1c8b7559-29ab-4982-8c75-ea2a0f0101cb_del complete
Nov 29 03:19:19 user nova-compute[75234]: DEBUG nova.network.neutron [None req-ba301a44-9f9b-4af1-ae20-14afd8363114 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] [instance: 72ebd454-0275-4c3d-95d2-227ffb045a0c] Successfully created port: 7b066f7f-bc63-4a5f-8b1b-20e4611cf387 {{(pid=75234) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:548}}
Nov 29 03:19:19 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ba301a44-9f9b-4af1-ae20-14afd8363114 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json" returned: 0 in 0.184s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:19:19 user nova-compute[75234]: DEBUG nova.virt.disk.api [None req-ba301a44-9f9b-4af1-ae20-14afd8363114 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Checking if we can resize image /opt/stack/data/nova/instances/72ebd454-0275-4c3d-95d2-227ffb045a0c/disk. size=1073741824 {{(pid=75234) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:166}}
Nov 29 03:19:19 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ba301a44-9f9b-4af1-ae20-14afd8363114 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/72ebd454-0275-4c3d-95d2-227ffb045a0c/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:19:19 user nova-compute[75234]: INFO nova.compute.manager [None req-ec0d94a4-4e08-4b8e-97dd-ecb8d660b350 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] [instance: 1c8b7559-29ab-4982-8c75-ea2a0f0101cb] Took 1.23 seconds to destroy the instance on the hypervisor.
Nov 29 03:19:19 user nova-compute[75234]: DEBUG oslo.service.loopingcall [None req-ec0d94a4-4e08-4b8e-97dd-ecb8d660b350 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=75234) func /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/loopingcall.py:435}}
Nov 29 03:19:19 user nova-compute[75234]: DEBUG nova.compute.manager [-] [instance: 1c8b7559-29ab-4982-8c75-ea2a0f0101cb] Deallocating network for instance {{(pid=75234) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2279}}
Nov 29 03:19:19 user nova-compute[75234]: DEBUG nova.network.neutron [-] [instance: 1c8b7559-29ab-4982-8c75-ea2a0f0101cb] deallocate_for_instance() {{(pid=75234) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1802}}
Nov 29 03:19:19 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ba301a44-9f9b-4af1-ae20-14afd8363114 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/72ebd454-0275-4c3d-95d2-227ffb045a0c/disk --force-share --output=json" returned: 0 in 0.235s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:19:19 user nova-compute[75234]: DEBUG nova.virt.disk.api [None req-ba301a44-9f9b-4af1-ae20-14afd8363114 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Cannot resize image /opt/stack/data/nova/instances/72ebd454-0275-4c3d-95d2-227ffb045a0c/disk to a smaller size. {{(pid=75234) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:172}}
Nov 29 03:19:19 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-ba301a44-9f9b-4af1-ae20-14afd8363114 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] [instance: 72ebd454-0275-4c3d-95d2-227ffb045a0c] Created local disks {{(pid=75234) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4861}}
Nov 29 03:19:19 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-ba301a44-9f9b-4af1-ae20-14afd8363114 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] [instance: 72ebd454-0275-4c3d-95d2-227ffb045a0c] Ensure instance console log exists: /opt/stack/data/nova/instances/72ebd454-0275-4c3d-95d2-227ffb045a0c/console.log {{(pid=75234) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4613}}
Nov 29 03:19:19 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ba301a44-9f9b-4af1-ae20-14afd8363114 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:19:19 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ba301a44-9f9b-4af1-ae20-14afd8363114 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:19:19 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ba301a44-9f9b-4af1-ae20-14afd8363114 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:19:19 user nova-compute[75234]: DEBUG nova.network.neutron [-] [instance: 1c8b7559-29ab-4982-8c75-ea2a0f0101cb] Updating instance_info_cache with network_info: [] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:19:19 user nova-compute[75234]: INFO nova.compute.manager [-] [instance: 1c8b7559-29ab-4982-8c75-ea2a0f0101cb] Took 0.61 seconds to deallocate network for instance.
Nov 29 03:19:19 user nova-compute[75234]: DEBUG nova.compute.manager [req-13209ed3-c8a7-41ea-aba4-61c0b4a9a4fa req-f99ec0aa-5e68-4354-95bb-a40a4e7456aa service nova] [instance: 1c8b7559-29ab-4982-8c75-ea2a0f0101cb] Received event network-vif-deleted-22cdbc69-d4f4-49cd-8104-d508aea94101 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:19:19 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ec0d94a4-4e08-4b8e-97dd-ecb8d660b350 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:19:19 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ec0d94a4-4e08-4b8e-97dd-ecb8d660b350 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:19:19 user nova-compute[75234]: DEBUG nova.compute.manager [req-54891cee-31e8-4c0d-8f90-6dca8e5263e4 req-9c800873-1be7-40bc-bda3-308e50d0808e service nova] [instance: 1c8b7559-29ab-4982-8c75-ea2a0f0101cb] Received event network-changed-22cdbc69-d4f4-49cd-8104-d508aea94101 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:19:19 user nova-compute[75234]: DEBUG nova.compute.manager [req-54891cee-31e8-4c0d-8f90-6dca8e5263e4 req-9c800873-1be7-40bc-bda3-308e50d0808e service nova] [instance: 1c8b7559-29ab-4982-8c75-ea2a0f0101cb] Refreshing instance network info cache due to event network-changed-22cdbc69-d4f4-49cd-8104-d508aea94101. {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11117}}
Nov 29 03:19:19 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-54891cee-31e8-4c0d-8f90-6dca8e5263e4 req-9c800873-1be7-40bc-bda3-308e50d0808e service nova] Acquiring lock "refresh_cache-1c8b7559-29ab-4982-8c75-ea2a0f0101cb" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:19:19 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-54891cee-31e8-4c0d-8f90-6dca8e5263e4 req-9c800873-1be7-40bc-bda3-308e50d0808e service nova] Acquired lock "refresh_cache-1c8b7559-29ab-4982-8c75-ea2a0f0101cb" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:19:19 user nova-compute[75234]: DEBUG nova.network.neutron [req-54891cee-31e8-4c0d-8f90-6dca8e5263e4 req-9c800873-1be7-40bc-bda3-308e50d0808e service nova] [instance: 1c8b7559-29ab-4982-8c75-ea2a0f0101cb] Refreshing network info cache for port 22cdbc69-d4f4-49cd-8104-d508aea94101 {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2006}}
Nov 29 03:19:19 user nova-compute[75234]: DEBUG nova.network.neutron [None req-ba301a44-9f9b-4af1-ae20-14afd8363114 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] [instance: 72ebd454-0275-4c3d-95d2-227ffb045a0c] Successfully updated port: 7b066f7f-bc63-4a5f-8b1b-20e4611cf387 {{(pid=75234) _update_port /opt/stack/nova/nova/network/neutron.py:586}}
Nov 29 03:19:19 user nova-compute[75234]: DEBUG nova.network.neutron [req-54891cee-31e8-4c0d-8f90-6dca8e5263e4 req-9c800873-1be7-40bc-bda3-308e50d0808e service nova] [instance: 1c8b7559-29ab-4982-8c75-ea2a0f0101cb] Instance cache missing network info. {{(pid=75234) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3322}}
Nov 29 03:19:19 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ba301a44-9f9b-4af1-ae20-14afd8363114 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Acquiring lock "refresh_cache-72ebd454-0275-4c3d-95d2-227ffb045a0c" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:19:19 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ba301a44-9f9b-4af1-ae20-14afd8363114 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Acquired lock "refresh_cache-72ebd454-0275-4c3d-95d2-227ffb045a0c" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:19:19 user nova-compute[75234]: DEBUG nova.network.neutron [None req-ba301a44-9f9b-4af1-ae20-14afd8363114 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] [instance: 72ebd454-0275-4c3d-95d2-227ffb045a0c] Building network info cache for instance {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2009}}
Nov 29 03:19:20 user nova-compute[75234]: DEBUG nova.compute.manager [req-190522d9-1bd7-440b-8c14-62454e55d1bd req-1deab735-aa06-4339-a980-b7edad79895d service nova] [instance: 72ebd454-0275-4c3d-95d2-227ffb045a0c] Received event network-changed-7b066f7f-bc63-4a5f-8b1b-20e4611cf387 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:19:20 user nova-compute[75234]: DEBUG nova.compute.manager [req-190522d9-1bd7-440b-8c14-62454e55d1bd req-1deab735-aa06-4339-a980-b7edad79895d service nova] [instance: 72ebd454-0275-4c3d-95d2-227ffb045a0c] Refreshing instance network info cache due to event network-changed-7b066f7f-bc63-4a5f-8b1b-20e4611cf387. {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11117}}
Nov 29 03:19:20 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-190522d9-1bd7-440b-8c14-62454e55d1bd req-1deab735-aa06-4339-a980-b7edad79895d service nova] Acquiring lock "refresh_cache-72ebd454-0275-4c3d-95d2-227ffb045a0c" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:19:20 user nova-compute[75234]: DEBUG nova.network.neutron [None req-ba301a44-9f9b-4af1-ae20-14afd8363114 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] [instance: 72ebd454-0275-4c3d-95d2-227ffb045a0c] Instance cache missing network info. {{(pid=75234) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3322}}
Nov 29 03:19:20 user nova-compute[75234]: DEBUG nova.network.neutron [req-54891cee-31e8-4c0d-8f90-6dca8e5263e4 req-9c800873-1be7-40bc-bda3-308e50d0808e service nova] [instance: 1c8b7559-29ab-4982-8c75-ea2a0f0101cb] Updating instance_info_cache with network_info: [] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:19:20 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-54891cee-31e8-4c0d-8f90-6dca8e5263e4 req-9c800873-1be7-40bc-bda3-308e50d0808e service nova] Releasing lock "refresh_cache-1c8b7559-29ab-4982-8c75-ea2a0f0101cb" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:19:20 user nova-compute[75234]: DEBUG nova.compute.manager [req-54891cee-31e8-4c0d-8f90-6dca8e5263e4 req-9c800873-1be7-40bc-bda3-308e50d0808e service nova] [instance: 1c8b7559-29ab-4982-8c75-ea2a0f0101cb] Received event network-vif-unplugged-22cdbc69-d4f4-49cd-8104-d508aea94101 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:19:20 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-54891cee-31e8-4c0d-8f90-6dca8e5263e4 req-9c800873-1be7-40bc-bda3-308e50d0808e service nova] Acquiring lock "1c8b7559-29ab-4982-8c75-ea2a0f0101cb-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:19:20 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-54891cee-31e8-4c0d-8f90-6dca8e5263e4 req-9c800873-1be7-40bc-bda3-308e50d0808e service nova] Lock "1c8b7559-29ab-4982-8c75-ea2a0f0101cb-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:19:20 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-54891cee-31e8-4c0d-8f90-6dca8e5263e4 req-9c800873-1be7-40bc-bda3-308e50d0808e service nova] Lock "1c8b7559-29ab-4982-8c75-ea2a0f0101cb-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:19:20 user nova-compute[75234]: DEBUG nova.compute.manager [req-54891cee-31e8-4c0d-8f90-6dca8e5263e4 req-9c800873-1be7-40bc-bda3-308e50d0808e service nova] [instance: 1c8b7559-29ab-4982-8c75-ea2a0f0101cb] No waiting events found dispatching network-vif-unplugged-22cdbc69-d4f4-49cd-8104-d508aea94101 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:19:20 user nova-compute[75234]: WARNING nova.compute.manager [req-54891cee-31e8-4c0d-8f90-6dca8e5263e4 req-9c800873-1be7-40bc-bda3-308e50d0808e service nova] [instance: 1c8b7559-29ab-4982-8c75-ea2a0f0101cb] Received unexpected event network-vif-unplugged-22cdbc69-d4f4-49cd-8104-d508aea94101 for instance with vm_state deleted and task_state None.
Nov 29 03:19:20 user nova-compute[75234]: DEBUG nova.compute.manager [req-54891cee-31e8-4c0d-8f90-6dca8e5263e4 req-9c800873-1be7-40bc-bda3-308e50d0808e service nova] [instance: 1c8b7559-29ab-4982-8c75-ea2a0f0101cb] Received event network-vif-plugged-22cdbc69-d4f4-49cd-8104-d508aea94101 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:19:20 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-54891cee-31e8-4c0d-8f90-6dca8e5263e4 req-9c800873-1be7-40bc-bda3-308e50d0808e service nova] Acquiring lock "1c8b7559-29ab-4982-8c75-ea2a0f0101cb-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:19:20 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-54891cee-31e8-4c0d-8f90-6dca8e5263e4 req-9c800873-1be7-40bc-bda3-308e50d0808e service nova] Lock "1c8b7559-29ab-4982-8c75-ea2a0f0101cb-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:19:20 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-54891cee-31e8-4c0d-8f90-6dca8e5263e4 req-9c800873-1be7-40bc-bda3-308e50d0808e service nova] Lock "1c8b7559-29ab-4982-8c75-ea2a0f0101cb-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:19:20 user nova-compute[75234]: DEBUG nova.compute.manager [req-54891cee-31e8-4c0d-8f90-6dca8e5263e4 req-9c800873-1be7-40bc-bda3-308e50d0808e service nova] [instance: 1c8b7559-29ab-4982-8c75-ea2a0f0101cb] No waiting events found dispatching network-vif-plugged-22cdbc69-d4f4-49cd-8104-d508aea94101 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:19:20 user nova-compute[75234]: WARNING nova.compute.manager [req-54891cee-31e8-4c0d-8f90-6dca8e5263e4 req-9c800873-1be7-40bc-bda3-308e50d0808e service nova] [instance: 1c8b7559-29ab-4982-8c75-ea2a0f0101cb] Received unexpected event network-vif-plugged-22cdbc69-d4f4-49cd-8104-d508aea94101 for instance with vm_state deleted and task_state None.
Nov 29 03:19:20 user nova-compute[75234]: DEBUG nova.compute.provider_tree [None req-ec0d94a4-4e08-4b8e-97dd-ecb8d660b350 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Inventory has not changed in ProviderTree for provider: 4d291220-7c37-4839-a0e3-b23c66502b74 {{(pid=75234) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Nov 29 03:19:20 user nova-compute[75234]: DEBUG nova.network.neutron [None req-ba301a44-9f9b-4af1-ae20-14afd8363114 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] [instance: 72ebd454-0275-4c3d-95d2-227ffb045a0c] Updating instance_info_cache with network_info: [{"id": "7b066f7f-bc63-4a5f-8b1b-20e4611cf387", "address": "fa:16:3e:52:a7:f8", "network": {"id": "1898605d-00e7-4e81-964e-821e09f5421c", "bridge": "br-int", "label": "tempest-VolumesSnapshotTestJSON-1896954656-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "7e24dd8accea4c1e9598d643ea79c86d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap7b066f7f-bc", "ovs_interfaceid": "7b066f7f-bc63-4a5f-8b1b-20e4611cf387", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:19:20 user nova-compute[75234]: DEBUG nova.scheduler.client.report [None req-ec0d94a4-4e08-4b8e-97dd-ecb8d660b350 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Inventory has not changed for provider 4d291220-7c37-4839-a0e3-b23c66502b74 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=75234) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Nov 29 03:19:20 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ba301a44-9f9b-4af1-ae20-14afd8363114 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Releasing lock "refresh_cache-72ebd454-0275-4c3d-95d2-227ffb045a0c" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:19:20 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ba301a44-9f9b-4af1-ae20-14afd8363114 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] [instance: 72ebd454-0275-4c3d-95d2-227ffb045a0c] Instance network_info: |[{"id": "7b066f7f-bc63-4a5f-8b1b-20e4611cf387", "address": "fa:16:3e:52:a7:f8", "network": {"id": "1898605d-00e7-4e81-964e-821e09f5421c", "bridge": "br-int", "label": "tempest-VolumesSnapshotTestJSON-1896954656-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "7e24dd8accea4c1e9598d643ea79c86d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap7b066f7f-bc", "ovs_interfaceid": "7b066f7f-bc63-4a5f-8b1b-20e4611cf387", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=75234) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1987}}
Nov 29 03:19:20 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-190522d9-1bd7-440b-8c14-62454e55d1bd req-1deab735-aa06-4339-a980-b7edad79895d service nova] Acquired lock "refresh_cache-72ebd454-0275-4c3d-95d2-227ffb045a0c" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:19:20 user nova-compute[75234]: DEBUG nova.network.neutron [req-190522d9-1bd7-440b-8c14-62454e55d1bd req-1deab735-aa06-4339-a980-b7edad79895d service nova] [instance: 72ebd454-0275-4c3d-95d2-227ffb045a0c] Refreshing network info cache for port 7b066f7f-bc63-4a5f-8b1b-20e4611cf387 {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2006}}
Nov 29 03:19:20 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-ba301a44-9f9b-4af1-ae20-14afd8363114 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] [instance: 72ebd454-0275-4c3d-95d2-227ffb045a0c] Start _get_guest_xml network_info=[{"id": "7b066f7f-bc63-4a5f-8b1b-20e4611cf387", "address": "fa:16:3e:52:a7:f8", "network": {"id": "1898605d-00e7-4e81-964e-821e09f5421c", "bridge": "br-int", "label": "tempest-VolumesSnapshotTestJSON-1896954656-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "7e24dd8accea4c1e9598d643ea79c86d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap7b066f7f-bc", "ovs_interfaceid": "7b066f7f-bc63-4a5f-8b1b-20e4611cf387", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum='c8fc807773e5354afe61636071771906',container_format='bare',created_at=2023-11-29T03:02:15Z,direct_url=<?>,disk_format='qcow2',id=96c30ba7-9b73-4a30-8ba4-7138879573b7,min_disk=0,min_ram=0,name='cirros-0.6.2-x86_64-disk',owner='b18e380921c748b78517b6f1211cd0f1',properties=ImageMetaProps,protected=<?>,size=21430272,status='active',tags=<?>,updated_at=2023-11-29T03:02:17Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [{'device_name': '/dev/vda', 'encryption_format': None, 'encrypted': False, 'device_type': 'disk', 'size': 0, 'disk_bus': 'virtio', 'boot_index': 0, 'encryption_options': None, 'encryption_secret_uuid': None, 'guest_format': None, 'image_id': '96c30ba7-9b73-4a30-8ba4-7138879573b7'}], 'ephemerals': [], 'block_device_mapping': [], 'swap': None} {{(pid=75234) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7562}}
Nov 29 03:19:20 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-ba301a44-9f9b-4af1-ae20-14afd8363114 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:19:20 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-ba301a44-9f9b-4af1-ae20-14afd8363114 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:19:20 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-ba301a44-9f9b-4af1-ae20-14afd8363114 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Searching host: 'user' for CPU controller through CGroups V1... {{(pid=75234) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1652}}
Nov 29 03:19:20 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-ba301a44-9f9b-4af1-ae20-14afd8363114 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] CPU controller missing on host. {{(pid=75234) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1662}}
Nov 29 03:19:20 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-ba301a44-9f9b-4af1-ae20-14afd8363114 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Searching host: 'user' for CPU controller through CGroups V2... {{(pid=75234) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1671}}
Nov 29 03:19:20 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-ba301a44-9f9b-4af1-ae20-14afd8363114 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] CPU controller found on host. {{(pid=75234) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1678}}
Nov 29 03:19:20 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-ba301a44-9f9b-4af1-ae20-14afd8363114 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=75234) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5400}}
Nov 29 03:19:20 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-ba301a44-9f9b-4af1-ae20-14afd8363114 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-11-29T03:02:04Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='1',id=6,is_public=True,memory_mb=512,name='m1.tiny',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='c8fc807773e5354afe61636071771906',container_format='bare',created_at=2023-11-29T03:02:15Z,direct_url=<?>,disk_format='qcow2',id=96c30ba7-9b73-4a30-8ba4-7138879573b7,min_disk=0,min_ram=0,name='cirros-0.6.2-x86_64-disk',owner='b18e380921c748b78517b6f1211cd0f1',properties=ImageMetaProps,protected=<?>,size=21430272,status='active',tags=<?>,updated_at=2023-11-29T03:02:17Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:563}}
Nov 29 03:19:20 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-ba301a44-9f9b-4af1-ae20-14afd8363114 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Flavor limits 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:348}}
Nov 29 03:19:20 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-ba301a44-9f9b-4af1-ae20-14afd8363114 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Image limits 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:352}}
Nov 29 03:19:20 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-ba301a44-9f9b-4af1-ae20-14afd8363114 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Flavor pref 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:388}}
Nov 29 03:19:20 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-ba301a44-9f9b-4af1-ae20-14afd8363114 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Image pref 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:392}}
Nov 29 03:19:20 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-ba301a44-9f9b-4af1-ae20-14afd8363114 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:430}}
Nov 29 03:19:20 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-ba301a44-9f9b-4af1-ae20-14afd8363114 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:569}}
Nov 29 03:19:20 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-ba301a44-9f9b-4af1-ae20-14afd8363114 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=75234) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:471}}
Nov 29 03:19:20 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-ba301a44-9f9b-4af1-ae20-14afd8363114 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Got 1 possible topologies {{(pid=75234) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:501}}
Nov 29 03:19:20 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-ba301a44-9f9b-4af1-ae20-14afd8363114 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:575}}
Nov 29 03:19:20 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-ba301a44-9f9b-4af1-ae20-14afd8363114 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:577}}
Nov 29 03:19:20 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-ba301a44-9f9b-4af1-ae20-14afd8363114 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-11-29T03:19:16Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-VolumesSnapshotTestJSON-instance-1321793627',display_name='tempest-VolumesSnapshotTestJSON-instance-1321793627',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-volumessnapshottestjson-instance-1321793627',id=34,image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBMhLPqL+EpbVwJ3QgH6H+iq2siyx+OzEETODSHhyLDZVvMNvMkCCiM9MM+DdJnAGXLmWBUOWlS++Zcp6L8MZ4+GmIYRjpSS0hgQLzUmSsnG7tldFPkINDXoEPZUO6xjMqw==',key_name='tempest-keypair-1810678573',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='7e24dd8accea4c1e9598d643ea79c86d',ramdisk_id='',reservation_id='r-me7x9yk0',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-VolumesSnapshotTestJSON-902320119',owner_user_name='tempest-VolumesSnapshotTestJSON-902320119-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-11-29T03:19:18Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='899f638d0de34dcba53f4c2ef7f5e068',uuid=72ebd454-0275-4c3d-95d2-227ffb045a0c,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "7b066f7f-bc63-4a5f-8b1b-20e4611cf387", "address": "fa:16:3e:52:a7:f8", "network": {"id": "1898605d-00e7-4e81-964e-821e09f5421c", "bridge": "br-int", "label": "tempest-VolumesSnapshotTestJSON-1896954656-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "7e24dd8accea4c1e9598d643ea79c86d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap7b066f7f-bc", "ovs_interfaceid": "7b066f7f-bc63-4a5f-8b1b-20e4611cf387", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=75234) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Nov 29 03:19:20 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-ba301a44-9f9b-4af1-ae20-14afd8363114 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Converting VIF {"id": "7b066f7f-bc63-4a5f-8b1b-20e4611cf387", "address": "fa:16:3e:52:a7:f8", "network": {"id": "1898605d-00e7-4e81-964e-821e09f5421c", "bridge": "br-int", "label": "tempest-VolumesSnapshotTestJSON-1896954656-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "7e24dd8accea4c1e9598d643ea79c86d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap7b066f7f-bc", "ovs_interfaceid": "7b066f7f-bc63-4a5f-8b1b-20e4611cf387", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:19:20 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-ba301a44-9f9b-4af1-ae20-14afd8363114 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:52:a7:f8,bridge_name='br-int',has_traffic_filtering=True,id=7b066f7f-bc63-4a5f-8b1b-20e4611cf387,network=Network(1898605d-00e7-4e81-964e-821e09f5421c),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap7b066f7f-bc') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:19:20 user nova-compute[75234]: DEBUG nova.objects.instance [None req-ba301a44-9f9b-4af1-ae20-14afd8363114 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Lazy-loading 'pci_devices' on Instance uuid 72ebd454-0275-4c3d-95d2-227ffb045a0c {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:19:20 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ec0d94a4-4e08-4b8e-97dd-ecb8d660b350 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.625s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:19:20 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-ba301a44-9f9b-4af1-ae20-14afd8363114 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] [instance: 72ebd454-0275-4c3d-95d2-227ffb045a0c] End _get_guest_xml xml=<domain type="kvm">
Nov 29 03:19:20 user nova-compute[75234]:   <uuid>72ebd454-0275-4c3d-95d2-227ffb045a0c</uuid>
Nov 29 03:19:20 user nova-compute[75234]:   <name>instance-00000022</name>
Nov 29 03:19:20 user nova-compute[75234]:   <memory>524288</memory>
Nov 29 03:19:20 user nova-compute[75234]:   <vcpu>1</vcpu>
Nov 29 03:19:20 user nova-compute[75234]:   <metadata>
Nov 29 03:19:20 user nova-compute[75234]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Nov 29 03:19:20 user nova-compute[75234]:       <nova:package version="0.0.0"/>
Nov 29 03:19:20 user nova-compute[75234]:       <nova:name>tempest-VolumesSnapshotTestJSON-instance-1321793627</nova:name>
Nov 29 03:19:20 user nova-compute[75234]:       <nova:creationTime>2023-11-29 03:19:20</nova:creationTime>
Nov 29 03:19:20 user nova-compute[75234]:       <nova:flavor name="m1.tiny">
Nov 29 03:19:20 user nova-compute[75234]:         <nova:memory>512</nova:memory>
Nov 29 03:19:20 user nova-compute[75234]:         <nova:disk>1</nova:disk>
Nov 29 03:19:20 user nova-compute[75234]:         <nova:swap>0</nova:swap>
Nov 29 03:19:20 user nova-compute[75234]:         <nova:ephemeral>0</nova:ephemeral>
Nov 29 03:19:20 user nova-compute[75234]:         <nova:vcpus>1</nova:vcpus>
Nov 29 03:19:20 user nova-compute[75234]:       </nova:flavor>
Nov 29 03:19:20 user nova-compute[75234]:       <nova:owner>
Nov 29 03:19:20 user nova-compute[75234]:         <nova:user uuid="899f638d0de34dcba53f4c2ef7f5e068">tempest-VolumesSnapshotTestJSON-902320119-project-member</nova:user>
Nov 29 03:19:20 user nova-compute[75234]:         <nova:project uuid="7e24dd8accea4c1e9598d643ea79c86d">tempest-VolumesSnapshotTestJSON-902320119</nova:project>
Nov 29 03:19:20 user nova-compute[75234]:       </nova:owner>
Nov 29 03:19:20 user nova-compute[75234]:       <nova:root type="image" uuid="96c30ba7-9b73-4a30-8ba4-7138879573b7"/>
Nov 29 03:19:20 user nova-compute[75234]:       <nova:ports>
Nov 29 03:19:20 user nova-compute[75234]:         <nova:port uuid="7b066f7f-bc63-4a5f-8b1b-20e4611cf387">
Nov 29 03:19:20 user nova-compute[75234]:           <nova:ip type="fixed" address="10.0.0.5" ipVersion="4"/>
Nov 29 03:19:20 user nova-compute[75234]:         </nova:port>
Nov 29 03:19:20 user nova-compute[75234]:       </nova:ports>
Nov 29 03:19:20 user nova-compute[75234]:     </nova:instance>
Nov 29 03:19:20 user nova-compute[75234]:   </metadata>
Nov 29 03:19:20 user nova-compute[75234]:   <sysinfo type="smbios">
Nov 29 03:19:20 user nova-compute[75234]:     <system>
Nov 29 03:19:20 user nova-compute[75234]:       <entry name="manufacturer">OpenStack Foundation</entry>
Nov 29 03:19:20 user nova-compute[75234]:       <entry name="product">OpenStack Nova</entry>
Nov 29 03:19:20 user nova-compute[75234]:       <entry name="version">0.0.0</entry>
Nov 29 03:19:20 user nova-compute[75234]:       <entry name="serial">72ebd454-0275-4c3d-95d2-227ffb045a0c</entry>
Nov 29 03:19:20 user nova-compute[75234]:       <entry name="uuid">72ebd454-0275-4c3d-95d2-227ffb045a0c</entry>
Nov 29 03:19:20 user nova-compute[75234]:       <entry name="family">Virtual Machine</entry>
Nov 29 03:19:20 user nova-compute[75234]:     </system>
Nov 29 03:19:20 user nova-compute[75234]:   </sysinfo>
Nov 29 03:19:20 user nova-compute[75234]:   <os>
Nov 29 03:19:20 user nova-compute[75234]:     <type arch="x86_64" machine="pc">hvm</type>
Nov 29 03:19:20 user nova-compute[75234]:     <boot dev="hd"/>
Nov 29 03:19:20 user nova-compute[75234]:     <smbios mode="sysinfo"/>
Nov 29 03:19:20 user nova-compute[75234]:   </os>
Nov 29 03:19:20 user nova-compute[75234]:   <features>
Nov 29 03:19:20 user nova-compute[75234]:     <acpi/>
Nov 29 03:19:20 user nova-compute[75234]:     <apic/>
Nov 29 03:19:20 user nova-compute[75234]:     <vmcoreinfo/>
Nov 29 03:19:20 user nova-compute[75234]:   </features>
Nov 29 03:19:20 user nova-compute[75234]:   <clock offset="utc">
Nov 29 03:19:20 user nova-compute[75234]:     <timer name="pit" tickpolicy="delay"/>
Nov 29 03:19:20 user nova-compute[75234]:     <timer name="rtc" tickpolicy="catchup"/>
Nov 29 03:19:20 user nova-compute[75234]:     <timer name="hpet" present="no"/>
Nov 29 03:19:20 user nova-compute[75234]:   </clock>
Nov 29 03:19:20 user nova-compute[75234]:   <cpu mode="custom" match="exact">
Nov 29 03:19:20 user nova-compute[75234]:     <model>Nehalem</model>
Nov 29 03:19:20 user nova-compute[75234]:     <topology sockets="1" cores="1" threads="1"/>
Nov 29 03:19:20 user nova-compute[75234]:   </cpu>
Nov 29 03:19:20 user nova-compute[75234]:   <devices>
Nov 29 03:19:20 user nova-compute[75234]:     <disk type="file" device="disk">
Nov 29 03:19:20 user nova-compute[75234]:       <driver name="qemu" type="qcow2" cache="none"/>
Nov 29 03:19:20 user nova-compute[75234]:       <source file="/opt/stack/data/nova/instances/72ebd454-0275-4c3d-95d2-227ffb045a0c/disk"/>
Nov 29 03:19:20 user nova-compute[75234]:       <target dev="vda" bus="virtio"/>
Nov 29 03:19:20 user nova-compute[75234]:     </disk>
Nov 29 03:19:20 user nova-compute[75234]:     <interface type="ethernet">
Nov 29 03:19:20 user nova-compute[75234]:       <mac address="fa:16:3e:52:a7:f8"/>
Nov 29 03:19:20 user nova-compute[75234]:       <model type="virtio"/>
Nov 29 03:19:20 user nova-compute[75234]:       <mtu size="1442"/>
Nov 29 03:19:20 user nova-compute[75234]:       <target dev="tap7b066f7f-bc"/>
Nov 29 03:19:20 user nova-compute[75234]:     </interface>
Nov 29 03:19:20 user nova-compute[75234]:     <serial type="pty">
Nov 29 03:19:20 user nova-compute[75234]:       <log file="/opt/stack/data/nova/instances/72ebd454-0275-4c3d-95d2-227ffb045a0c/console.log" append="off"/>
Nov 29 03:19:20 user nova-compute[75234]:     </serial>
Nov 29 03:19:20 user nova-compute[75234]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Nov 29 03:19:20 user nova-compute[75234]:     <video>
Nov 29 03:19:20 user nova-compute[75234]:       <model type="virtio"/>
Nov 29 03:19:20 user nova-compute[75234]:     </video>
Nov 29 03:19:20 user nova-compute[75234]:     <rng model="virtio">
Nov 29 03:19:20 user nova-compute[75234]:       <backend model="random">/dev/urandom</backend>
Nov 29 03:19:20 user nova-compute[75234]:     </rng>
Nov 29 03:19:20 user nova-compute[75234]:     <controller type="usb" index="0" model="none"/>
Nov 29 03:19:20 user nova-compute[75234]:     <memballoon model="virtio">
Nov 29 03:19:20 user nova-compute[75234]:       <stats period="10"/>
Nov 29 03:19:20 user nova-compute[75234]:     </memballoon>
Nov 29 03:19:20 user nova-compute[75234]:   </devices>
Nov 29 03:19:20 user nova-compute[75234]: </domain>
Nov 29 03:19:20 user nova-compute[75234]:  {{(pid=75234) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7568}}
Nov 29 03:19:20 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-ba301a44-9f9b-4af1-ae20-14afd8363114 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-11-29T03:19:16Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-VolumesSnapshotTestJSON-instance-1321793627',display_name='tempest-VolumesSnapshotTestJSON-instance-1321793627',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-volumessnapshottestjson-instance-1321793627',id=34,image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBMhLPqL+EpbVwJ3QgH6H+iq2siyx+OzEETODSHhyLDZVvMNvMkCCiM9MM+DdJnAGXLmWBUOWlS++Zcp6L8MZ4+GmIYRjpSS0hgQLzUmSsnG7tldFPkINDXoEPZUO6xjMqw==',key_name='tempest-keypair-1810678573',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='7e24dd8accea4c1e9598d643ea79c86d',ramdisk_id='',reservation_id='r-me7x9yk0',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-VolumesSnapshotTestJSON-902320119',owner_user_name='tempest-VolumesSnapshotTestJSON-902320119-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-11-29T03:19:18Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='899f638d0de34dcba53f4c2ef7f5e068',uuid=72ebd454-0275-4c3d-95d2-227ffb045a0c,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "7b066f7f-bc63-4a5f-8b1b-20e4611cf387", "address": "fa:16:3e:52:a7:f8", "network": {"id": "1898605d-00e7-4e81-964e-821e09f5421c", "bridge": "br-int", "label": "tempest-VolumesSnapshotTestJSON-1896954656-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "7e24dd8accea4c1e9598d643ea79c86d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap7b066f7f-bc", "ovs_interfaceid": "7b066f7f-bc63-4a5f-8b1b-20e4611cf387", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Nov 29 03:19:20 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-ba301a44-9f9b-4af1-ae20-14afd8363114 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Converting VIF {"id": "7b066f7f-bc63-4a5f-8b1b-20e4611cf387", "address": "fa:16:3e:52:a7:f8", "network": {"id": "1898605d-00e7-4e81-964e-821e09f5421c", "bridge": "br-int", "label": "tempest-VolumesSnapshotTestJSON-1896954656-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "7e24dd8accea4c1e9598d643ea79c86d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap7b066f7f-bc", "ovs_interfaceid": "7b066f7f-bc63-4a5f-8b1b-20e4611cf387", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:19:20 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-ba301a44-9f9b-4af1-ae20-14afd8363114 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:52:a7:f8,bridge_name='br-int',has_traffic_filtering=True,id=7b066f7f-bc63-4a5f-8b1b-20e4611cf387,network=Network(1898605d-00e7-4e81-964e-821e09f5421c),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap7b066f7f-bc') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:19:20 user nova-compute[75234]: DEBUG os_vif [None req-ba301a44-9f9b-4af1-ae20-14afd8363114 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:52:a7:f8,bridge_name='br-int',has_traffic_filtering=True,id=7b066f7f-bc63-4a5f-8b1b-20e4611cf387,network=Network(1898605d-00e7-4e81-964e-821e09f5421c),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap7b066f7f-bc') {{(pid=75234) plug /opt/stack/data/venv/lib/python3.10/site-packages/os_vif/__init__.py:76}}
Nov 29 03:19:20 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:19:20 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:19:20 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Nov 29 03:19:20 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:19:20 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbCreateCommand(_result=None, table=QoS, columns={'type': 'linux-noop', 'external_ids': {'id': 'b8a946f4-c884-59bf-b61b-f1c683689bf4', '_type': 'linux-noop'}}, row=False) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:19:20 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:19:20 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:19:20 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:19:20 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tap7b066f7f-bc, may_exist=True, interface_attrs={}) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:19:20 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Port, record=tap7b066f7f-bc, col_values=(('qos', UUID('d2acd126-5481-4982-8788-302bf299aab8')),), if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:19:20 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DbSetCommand(_result=None, table=Interface, record=tap7b066f7f-bc, col_values=(('external_ids', {'iface-id': '7b066f7f-bc63-4a5f-8b1b-20e4611cf387', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:52:a7:f8', 'vm-uuid': '72ebd454-0275-4c3d-95d2-227ffb045a0c'}),), if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:19:20 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:19:20 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:19:20 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:19:20 user nova-compute[75234]: INFO os_vif [None req-ba301a44-9f9b-4af1-ae20-14afd8363114 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:52:a7:f8,bridge_name='br-int',has_traffic_filtering=True,id=7b066f7f-bc63-4a5f-8b1b-20e4611cf387,network=Network(1898605d-00e7-4e81-964e-821e09f5421c),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap7b066f7f-bc')
Nov 29 03:19:20 user nova-compute[75234]: INFO nova.scheduler.client.report [None req-ec0d94a4-4e08-4b8e-97dd-ecb8d660b350 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Deleted allocations for instance 1c8b7559-29ab-4982-8c75-ea2a0f0101cb
Nov 29 03:19:20 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-ba301a44-9f9b-4af1-ae20-14afd8363114 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] No BDM found with device name vda, not building metadata. {{(pid=75234) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12089}}
Nov 29 03:19:20 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-ba301a44-9f9b-4af1-ae20-14afd8363114 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] No VIF found with MAC fa:16:3e:52:a7:f8, not building metadata {{(pid=75234) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Nov 29 03:19:20 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ec0d94a4-4e08-4b8e-97dd-ecb8d660b350 tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Lock "1c8b7559-29ab-4982-8c75-ea2a0f0101cb" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 2.701s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:19:20 user nova-compute[75234]: DEBUG nova.network.neutron [req-190522d9-1bd7-440b-8c14-62454e55d1bd req-1deab735-aa06-4339-a980-b7edad79895d service nova] [instance: 72ebd454-0275-4c3d-95d2-227ffb045a0c] Updated VIF entry in instance network info cache for port 7b066f7f-bc63-4a5f-8b1b-20e4611cf387. {{(pid=75234) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3481}}
Nov 29 03:19:20 user nova-compute[75234]: DEBUG nova.network.neutron [req-190522d9-1bd7-440b-8c14-62454e55d1bd req-1deab735-aa06-4339-a980-b7edad79895d service nova] [instance: 72ebd454-0275-4c3d-95d2-227ffb045a0c] Updating instance_info_cache with network_info: [{"id": "7b066f7f-bc63-4a5f-8b1b-20e4611cf387", "address": "fa:16:3e:52:a7:f8", "network": {"id": "1898605d-00e7-4e81-964e-821e09f5421c", "bridge": "br-int", "label": "tempest-VolumesSnapshotTestJSON-1896954656-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "7e24dd8accea4c1e9598d643ea79c86d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap7b066f7f-bc", "ovs_interfaceid": "7b066f7f-bc63-4a5f-8b1b-20e4611cf387", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:19:20 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-190522d9-1bd7-440b-8c14-62454e55d1bd req-1deab735-aa06-4339-a980-b7edad79895d service nova] Releasing lock "refresh_cache-72ebd454-0275-4c3d-95d2-227ffb045a0c" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:19:21 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:19:21 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:19:22 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:19:22 user nova-compute[75234]: DEBUG nova.compute.manager [req-b6827c1b-43eb-4444-a32e-bded0efe8e80 req-c9d9441b-3ba9-4a17-b9e1-02b953f36346 service nova] [instance: 72ebd454-0275-4c3d-95d2-227ffb045a0c] Received event network-vif-plugged-7b066f7f-bc63-4a5f-8b1b-20e4611cf387 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:19:22 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-b6827c1b-43eb-4444-a32e-bded0efe8e80 req-c9d9441b-3ba9-4a17-b9e1-02b953f36346 service nova] Acquiring lock "72ebd454-0275-4c3d-95d2-227ffb045a0c-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:19:22 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-b6827c1b-43eb-4444-a32e-bded0efe8e80 req-c9d9441b-3ba9-4a17-b9e1-02b953f36346 service nova] Lock "72ebd454-0275-4c3d-95d2-227ffb045a0c-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:19:22 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-b6827c1b-43eb-4444-a32e-bded0efe8e80 req-c9d9441b-3ba9-4a17-b9e1-02b953f36346 service nova] Lock "72ebd454-0275-4c3d-95d2-227ffb045a0c-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:19:22 user nova-compute[75234]: DEBUG nova.compute.manager [req-b6827c1b-43eb-4444-a32e-bded0efe8e80 req-c9d9441b-3ba9-4a17-b9e1-02b953f36346 service nova] [instance: 72ebd454-0275-4c3d-95d2-227ffb045a0c] No waiting events found dispatching network-vif-plugged-7b066f7f-bc63-4a5f-8b1b-20e4611cf387 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:19:22 user nova-compute[75234]: WARNING nova.compute.manager [req-b6827c1b-43eb-4444-a32e-bded0efe8e80 req-c9d9441b-3ba9-4a17-b9e1-02b953f36346 service nova] [instance: 72ebd454-0275-4c3d-95d2-227ffb045a0c] Received unexpected event network-vif-plugged-7b066f7f-bc63-4a5f-8b1b-20e4611cf387 for instance with vm_state building and task_state spawning.
Nov 29 03:19:22 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-1006b7b9-8d9e-489e-90e9-503d7a3fd306 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] [instance: aaf1c3e3-b73e-42ad-87bd-c5255ec09b46] Calling os-brick to attach iSCSI Volume {{(pid=75234) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Nov 29 03:19:22 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-1006b7b9-8d9e-489e-90e9-503d7a3fd306 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f8a5050c9a0>, 'connection_properties': {'target_iqn': 'iqn.2010-10.org.openstack:9d734633-75e6-4a97-a3c5-bd9e45d70cca', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'nBvENxIP', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:176}}
Nov 29 03:19:22 user nova-compute[75234]: WARNING os_brick.initiator.connectors.base [None req-1006b7b9-8d9e-489e-90e9-503d7a3fd306 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Nov 29 03:19:22 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-1006b7b9-8d9e-489e-90e9-503d7a3fd306 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:68}}
Nov 29 03:19:22 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-1006b7b9-8d9e-489e-90e9-503d7a3fd306 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.002s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:73}}
Nov 29 03:19:22 user nova-compute[75234]: WARNING os_brick.initiator.connectors.base [None req-1006b7b9-8d9e-489e-90e9-503d7a3fd306 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Nov 29 03:19:22 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-1006b7b9-8d9e-489e-90e9-503d7a3fd306 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:9d734633-75e6-4a97-a3c5-bd9e45d70cca" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:68}}
Nov 29 03:19:22 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-1006b7b9-8d9e-489e-90e9-503d7a3fd306 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:9d734633-75e6-4a97-a3c5-bd9e45d70cca" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:73}}
Nov 29 03:19:22 user nova-compute[75234]: INFO os_brick.initiator.connectors.iscsi [None req-1006b7b9-8d9e-489e-90e9-503d7a3fd306 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Nov 29 03:19:22 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:9d734633-75e6-4a97-a3c5-bd9e45d70cca -p 172.16.0.220:3260 {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:19:22 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:19:22 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:19:22 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:9d734633-75e6-4a97-a3c5-bd9e45d70cca -p 172.16.0.220:3260" returned: 21 in 0.017s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:19:22 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[ca40aeee-f89f-490f-9974-c9372499b023]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:19:22 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-1006b7b9-8d9e-489e-90e9-503d7a3fd306 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Nov 29 03:19:22 user nova-compute[75234]:  {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:19:22 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:9d734633-75e6-4a97-a3c5-bd9e45d70cca -p 172.16.0.220:3260 --interface default --op new {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:19:22 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:19:22 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:9d734633-75e6-4a97-a3c5-bd9e45d70cca -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.023s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:19:22 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[7e781e7e-497b-4fba-abda-9020575717bb]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:9d734633-75e6-4a97-a3c5-bd9e45d70cca] added\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:19:22 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-1006b7b9-8d9e-489e-90e9-503d7a3fd306 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:9d734633-75e6-4a97-a3c5-bd9e45d70cca] added
Nov 29 03:19:22 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:19:22 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:9d734633-75e6-4a97-a3c5-bd9e45d70cca -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:19:22 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:19:22 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:9d734633-75e6-4a97-a3c5-bd9e45d70cca -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.028s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:19:22 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[9f5c3396-fa52-4aff-95fe-bfe4fbda988e]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:19:22 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-1006b7b9-8d9e-489e-90e9-503d7a3fd306 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:19:22 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:9d734633-75e6-4a97-a3c5-bd9e45d70cca -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:19:22 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:9d734633-75e6-4a97-a3c5-bd9e45d70cca -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.019s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:19:22 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[07325f97-32ec-4f3c-bb72-51f5b07c1503]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:19:22 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-1006b7b9-8d9e-489e-90e9-503d7a3fd306 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:19:22 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:9d734633-75e6-4a97-a3c5-bd9e45d70cca -p 172.16.0.220:3260 --op update -n node.session.auth.username -v nBvENxIP {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:19:22 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:9d734633-75e6-4a97-a3c5-bd9e45d70cca -p 172.16.0.220:3260 --op update -n node.session.auth.username -v nBvENxIP" returned: 0 in 0.019s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:19:22 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[75fddc7c-f3a1-424f-9447-f6f1a507f2ea]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:19:22 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-1006b7b9-8d9e-489e-90e9-503d7a3fd306 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'nBvENxIP'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:19:22 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:9d734633-75e6-4a97-a3c5-bd9e45d70cca -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:19:22 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:9d734633-75e6-4a97-a3c5-bd9e45d70cca -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.020s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:19:22 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[c6e079ae-ee95-4a4b-820d-a4536abbd9a3]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:19:22 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-1006b7b9-8d9e-489e-90e9-503d7a3fd306 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:19:22 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:19:22 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.025s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:19:22 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[55b49738-881a-4482-8d10-58f09520677e]: (4, ('tcp: [29] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bf57b967-7fad-4d76-abb2-319fe88f8ef9 (non-flash)\ntcp: [32] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:cf6ab229-533a-424e-bd67-1d54d50ad4be (non-flash)\ntcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:07f83efc-272d-4e10-9a2b-48d967db613c (non-flash)\ntcp: [34] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a06ab71b-02cb-4c8a-8e7b-26d8176cddf6 (non-flash)\ntcp: [35] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7a1a1247-abe3-4a99-a7cb-f168fb2cb0b0 (non-flash)\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:19:22 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-1006b7b9-8d9e-489e-90e9-503d7a3fd306 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [29] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bf57b967-7fad-4d76-abb2-319fe88f8ef9 (non-flash)
Nov 29 03:19:22 user nova-compute[75234]: tcp: [32] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:cf6ab229-533a-424e-bd67-1d54d50ad4be (non-flash)
Nov 29 03:19:22 user nova-compute[75234]: tcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:07f83efc-272d-4e10-9a2b-48d967db613c (non-flash)
Nov 29 03:19:22 user nova-compute[75234]: tcp: [34] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a06ab71b-02cb-4c8a-8e7b-26d8176cddf6 (non-flash)
Nov 29 03:19:22 user nova-compute[75234]: tcp: [35] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7a1a1247-abe3-4a99-a7cb-f168fb2cb0b0 (non-flash)
Nov 29 03:19:22 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:19:22 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-1006b7b9-8d9e-489e-90e9-503d7a3fd306 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] iscsi session list stdout=tcp: [29] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bf57b967-7fad-4d76-abb2-319fe88f8ef9 (non-flash)
Nov 29 03:19:22 user nova-compute[75234]: tcp: [32] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:cf6ab229-533a-424e-bd67-1d54d50ad4be (non-flash)
Nov 29 03:19:22 user nova-compute[75234]: tcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:07f83efc-272d-4e10-9a2b-48d967db613c (non-flash)
Nov 29 03:19:22 user nova-compute[75234]: tcp: [34] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a06ab71b-02cb-4c8a-8e7b-26d8176cddf6 (non-flash)
Nov 29 03:19:22 user nova-compute[75234]: tcp: [35] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7a1a1247-abe3-4a99-a7cb-f168fb2cb0b0 (non-flash)
Nov 29 03:19:22 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsi_session /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Nov 29 03:19:22 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:9d734633-75e6-4a97-a3c5-bd9e45d70cca -p 172.16.0.220:3260 --login {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:19:22 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:9d734633-75e6-4a97-a3c5-bd9e45d70cca -p 172.16.0.220:3260 --login" returned: 0 in 0.073s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:19:22 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[f9d04730-7a13-48b3-907f-d631d824482a]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:9d734633-75e6-4a97-a3c5-bd9e45d70cca, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:9d734633-75e6-4a97-a3c5-bd9e45d70cca, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:19:22 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-1006b7b9-8d9e-489e-90e9-503d7a3fd306 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:9d734633-75e6-4a97-a3c5-bd9e45d70cca, portal: 172.16.0.220,3260]
Nov 29 03:19:22 user nova-compute[75234]: Login to [iface: default, target: iqn.2010-10.org.openstack:9d734633-75e6-4a97-a3c5-bd9e45d70cca, portal: 172.16.0.220,3260] successful.
Nov 29 03:19:22 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:19:22 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:9d734633-75e6-4a97-a3c5-bd9e45d70cca -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:19:22 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:9d734633-75e6-4a97-a3c5-bd9e45d70cca -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.019s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:19:22 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[6ee16acc-f23f-425c-8e40-2750bac594fa]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:19:22 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-1006b7b9-8d9e-489e-90e9-503d7a3fd306 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:19:22 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:19:22 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.028s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:19:22 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[e8f8b6c3-11c8-403b-86cb-15a527a3359c]: (4, ('tcp: [29] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bf57b967-7fad-4d76-abb2-319fe88f8ef9 (non-flash)\ntcp: [32] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:cf6ab229-533a-424e-bd67-1d54d50ad4be (non-flash)\ntcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:07f83efc-272d-4e10-9a2b-48d967db613c (non-flash)\ntcp: [34] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a06ab71b-02cb-4c8a-8e7b-26d8176cddf6 (non-flash)\ntcp: [35] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7a1a1247-abe3-4a99-a7cb-f168fb2cb0b0 (non-flash)\ntcp: [36] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9d734633-75e6-4a97-a3c5-bd9e45d70cca (non-flash)\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:19:22 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-1006b7b9-8d9e-489e-90e9-503d7a3fd306 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [29] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bf57b967-7fad-4d76-abb2-319fe88f8ef9 (non-flash)
Nov 29 03:19:22 user nova-compute[75234]: tcp: [32] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:cf6ab229-533a-424e-bd67-1d54d50ad4be (non-flash)
Nov 29 03:19:22 user nova-compute[75234]: tcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:07f83efc-272d-4e10-9a2b-48d967db613c (non-flash)
Nov 29 03:19:22 user nova-compute[75234]: tcp: [34] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a06ab71b-02cb-4c8a-8e7b-26d8176cddf6 (non-flash)
Nov 29 03:19:22 user nova-compute[75234]: tcp: [35] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7a1a1247-abe3-4a99-a7cb-f168fb2cb0b0 (non-flash)
Nov 29 03:19:22 user nova-compute[75234]: tcp: [36] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9d734633-75e6-4a97-a3c5-bd9e45d70cca (non-flash)
Nov 29 03:19:22 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:19:22 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-1006b7b9-8d9e-489e-90e9-503d7a3fd306 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] iscsi session list stdout=tcp: [29] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bf57b967-7fad-4d76-abb2-319fe88f8ef9 (non-flash)
Nov 29 03:19:22 user nova-compute[75234]: tcp: [32] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:cf6ab229-533a-424e-bd67-1d54d50ad4be (non-flash)
Nov 29 03:19:22 user nova-compute[75234]: tcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:07f83efc-272d-4e10-9a2b-48d967db613c (non-flash)
Nov 29 03:19:22 user nova-compute[75234]: tcp: [34] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a06ab71b-02cb-4c8a-8e7b-26d8176cddf6 (non-flash)
Nov 29 03:19:22 user nova-compute[75234]: tcp: [35] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7a1a1247-abe3-4a99-a7cb-f168fb2cb0b0 (non-flash)
Nov 29 03:19:22 user nova-compute[75234]: tcp: [36] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9d734633-75e6-4a97-a3c5-bd9e45d70cca (non-flash)
Nov 29 03:19:22 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsi_session /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Nov 29 03:19:22 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-1006b7b9-8d9e-489e-90e9-503d7a3fd306 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:9d734633-75e6-4a97-a3c5-bd9e45d70cca" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.310s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:87}}
Nov 29 03:19:22 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-1006b7b9-8d9e-489e-90e9-503d7a3fd306 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Connected to 172.16.0.220:3260 {{(pid=75234) _connect_vol /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:644}}
Nov 29 03:19:22 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-1006b7b9-8d9e-489e-90e9-503d7a3fd306 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] HCTL ('38', '-', '-', 0) found on session 36 with lun 0 {{(pid=75234) get_hctl /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:770}}
Nov 29 03:19:22 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-1006b7b9-8d9e-489e-90e9-503d7a3fd306 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Scanning host 38 c: -, t: -, l: 0) {{(pid=75234) scan_iscsi /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:795}}
Nov 29 03:19:22 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host38/scan {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:19:23 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host38/scan" returned: 0 in 0.016s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:19:23 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[131c90d0-abf9-451a-8f31-6a91a44a8a87]: (4, ('- - 0', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:19:23 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-1006b7b9-8d9e-489e-90e9-503d7a3fd306 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Searching for a device in session 36 and hctl ['38', '*', '*', 0] yield: None {{(pid=75234) device_name_by_hctl /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:789}}
Nov 29 03:19:24 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-1006b7b9-8d9e-489e-90e9-503d7a3fd306 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Searching for a device in session 36 and hctl ['38', '*', '*', 0] yield: sdg {{(pid=75234) device_name_by_hctl /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:789}}
Nov 29 03:19:24 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-1006b7b9-8d9e-489e-90e9-503d7a3fd306 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Connected to sdg using {'target_iqn': 'iqn.2010-10.org.openstack:9d734633-75e6-4a97-a3c5-bd9e45d70cca', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'nBvENxIP', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False} {{(pid=75234) _connect_vol /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:672}}
Nov 29 03:19:24 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-1006b7b9-8d9e-489e-90e9-503d7a3fd306 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.340s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:87}}
Nov 29 03:19:24 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-1006b7b9-8d9e-489e-90e9-503d7a3fd306 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] <== connect_volume: return (1342ms) {'type': 'block', 'scsi_wwn': '23761616662393831', 'path': '/dev/sdg'} {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:203}}
Nov 29 03:19:24 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-1006b7b9-8d9e-489e-90e9-503d7a3fd306 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] [instance: aaf1c3e3-b73e-42ad-87bd-c5255ec09b46] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '23761616662393831', 'path': '/dev/sdg'} {{(pid=75234) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Nov 29 03:19:24 user nova-compute[75234]: DEBUG nova.objects.instance [None req-1006b7b9-8d9e-489e-90e9-503d7a3fd306 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Lazy-loading 'flavor' on Instance uuid aaf1c3e3-b73e-42ad-87bd-c5255ec09b46 {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:19:24 user nova-compute[75234]: DEBUG nova.virt.libvirt.guest [None req-1006b7b9-8d9e-489e-90e9-503d7a3fd306 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] attach device xml: <disk type="block" device="disk">
Nov 29 03:19:24 user nova-compute[75234]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Nov 29 03:19:24 user nova-compute[75234]:   <source dev="/dev/sdg"/>
Nov 29 03:19:24 user nova-compute[75234]:   <target dev="vdc" bus="virtio"/>
Nov 29 03:19:24 user nova-compute[75234]:   <serial>9d734633-75e6-4a97-a3c5-bd9e45d70cca</serial>
Nov 29 03:19:24 user nova-compute[75234]: </disk>
Nov 29 03:19:24 user nova-compute[75234]:  {{(pid=75234) attach_device /opt/stack/nova/nova/virt/libvirt/guest.py:339}}
Nov 29 03:19:24 user nova-compute[75234]: DEBUG nova.compute.manager [req-73571a3f-80c5-4473-8610-88f1be057a34 req-b543108e-c933-492b-9e9c-f2aef832003f service nova] [instance: 72ebd454-0275-4c3d-95d2-227ffb045a0c] Received event network-vif-plugged-7b066f7f-bc63-4a5f-8b1b-20e4611cf387 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:19:24 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-73571a3f-80c5-4473-8610-88f1be057a34 req-b543108e-c933-492b-9e9c-f2aef832003f service nova] Acquiring lock "72ebd454-0275-4c3d-95d2-227ffb045a0c-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:19:24 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-73571a3f-80c5-4473-8610-88f1be057a34 req-b543108e-c933-492b-9e9c-f2aef832003f service nova] Lock "72ebd454-0275-4c3d-95d2-227ffb045a0c-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:19:24 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-73571a3f-80c5-4473-8610-88f1be057a34 req-b543108e-c933-492b-9e9c-f2aef832003f service nova] Lock "72ebd454-0275-4c3d-95d2-227ffb045a0c-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:19:24 user nova-compute[75234]: DEBUG nova.compute.manager [req-73571a3f-80c5-4473-8610-88f1be057a34 req-b543108e-c933-492b-9e9c-f2aef832003f service nova] [instance: 72ebd454-0275-4c3d-95d2-227ffb045a0c] No waiting events found dispatching network-vif-plugged-7b066f7f-bc63-4a5f-8b1b-20e4611cf387 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:19:24 user nova-compute[75234]: WARNING nova.compute.manager [req-73571a3f-80c5-4473-8610-88f1be057a34 req-b543108e-c933-492b-9e9c-f2aef832003f service nova] [instance: 72ebd454-0275-4c3d-95d2-227ffb045a0c] Received unexpected event network-vif-plugged-7b066f7f-bc63-4a5f-8b1b-20e4611cf387 for instance with vm_state building and task_state spawning.
Nov 29 03:19:24 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ba301a44-9f9b-4af1-ae20-14afd8363114 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] [instance: 72ebd454-0275-4c3d-95d2-227ffb045a0c] Instance event wait completed in 0 seconds for  {{(pid=75234) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Nov 29 03:19:24 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-ba301a44-9f9b-4af1-ae20-14afd8363114 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] [instance: 72ebd454-0275-4c3d-95d2-227ffb045a0c] Guest created on hypervisor {{(pid=75234) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4421}}
Nov 29 03:19:24 user nova-compute[75234]: INFO nova.virt.libvirt.driver [-] [instance: 72ebd454-0275-4c3d-95d2-227ffb045a0c] Instance spawned successfully.
Nov 29 03:19:24 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-ba301a44-9f9b-4af1-ae20-14afd8363114 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] [instance: 72ebd454-0275-4c3d-95d2-227ffb045a0c] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:904}}
Nov 29 03:19:24 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-ba301a44-9f9b-4af1-ae20-14afd8363114 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] [instance: 72ebd454-0275-4c3d-95d2-227ffb045a0c] Found default for hw_cdrom_bus of ide {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:19:24 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-ba301a44-9f9b-4af1-ae20-14afd8363114 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] [instance: 72ebd454-0275-4c3d-95d2-227ffb045a0c] Found default for hw_disk_bus of virtio {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:19:24 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-ba301a44-9f9b-4af1-ae20-14afd8363114 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] [instance: 72ebd454-0275-4c3d-95d2-227ffb045a0c] Found default for hw_input_bus of None {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:19:24 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-ba301a44-9f9b-4af1-ae20-14afd8363114 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] [instance: 72ebd454-0275-4c3d-95d2-227ffb045a0c] Found default for hw_pointer_model of None {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:19:24 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-ba301a44-9f9b-4af1-ae20-14afd8363114 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] [instance: 72ebd454-0275-4c3d-95d2-227ffb045a0c] Found default for hw_video_model of virtio {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:19:24 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-ba301a44-9f9b-4af1-ae20-14afd8363114 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] [instance: 72ebd454-0275-4c3d-95d2-227ffb045a0c] Found default for hw_vif_model of virtio {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:19:24 user nova-compute[75234]: INFO nova.compute.manager [None req-ba301a44-9f9b-4af1-ae20-14afd8363114 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] [instance: 72ebd454-0275-4c3d-95d2-227ffb045a0c] Took 6.01 seconds to spawn the instance on the hypervisor.
Nov 29 03:19:24 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ba301a44-9f9b-4af1-ae20-14afd8363114 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] [instance: 72ebd454-0275-4c3d-95d2-227ffb045a0c] Checking state {{(pid=75234) _get_power_state /opt/stack/nova/nova/compute/manager.py:1782}}
Nov 29 03:19:24 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-1006b7b9-8d9e-489e-90e9-503d7a3fd306 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] No BDM found with device name vda, not building metadata. {{(pid=75234) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12089}}
Nov 29 03:19:24 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-1006b7b9-8d9e-489e-90e9-503d7a3fd306 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] No BDM found with device name vdb, not building metadata. {{(pid=75234) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12089}}
Nov 29 03:19:24 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-1006b7b9-8d9e-489e-90e9-503d7a3fd306 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] No BDM found with device name vdc, not building metadata. {{(pid=75234) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12089}}
Nov 29 03:19:24 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-1006b7b9-8d9e-489e-90e9-503d7a3fd306 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] No VIF found with MAC fa:16:3e:55:a5:13, not building metadata {{(pid=75234) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Nov 29 03:19:24 user nova-compute[75234]: INFO nova.compute.manager [None req-ba301a44-9f9b-4af1-ae20-14afd8363114 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] [instance: 72ebd454-0275-4c3d-95d2-227ffb045a0c] Took 7.08 seconds to build instance.
Nov 29 03:19:24 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ba301a44-9f9b-4af1-ae20-14afd8363114 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Lock "72ebd454-0275-4c3d-95d2-227ffb045a0c" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 7.172s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:19:24 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-1006b7b9-8d9e-489e-90e9-503d7a3fd306 tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Lock "aaf1c3e3-b73e-42ad-87bd-c5255ec09b46" "released" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: held 11.919s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:19:25 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:19:26 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-3fad31e7-f6ab-4b09-aae9-831ef75d71ed tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Acquiring lock "aaf1c3e3-b73e-42ad-87bd-c5255ec09b46" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:19:26 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-3fad31e7-f6ab-4b09-aae9-831ef75d71ed tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Lock "aaf1c3e3-b73e-42ad-87bd-c5255ec09b46" acquired by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:19:26 user nova-compute[75234]: INFO nova.compute.manager [None req-3fad31e7-f6ab-4b09-aae9-831ef75d71ed tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] [instance: aaf1c3e3-b73e-42ad-87bd-c5255ec09b46] Detaching volume 7a1a1247-abe3-4a99-a7cb-f168fb2cb0b0
Nov 29 03:19:26 user nova-compute[75234]: INFO nova.virt.block_device [None req-3fad31e7-f6ab-4b09-aae9-831ef75d71ed tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] [instance: aaf1c3e3-b73e-42ad-87bd-c5255ec09b46] Attempting to driver detach volume 7a1a1247-abe3-4a99-a7cb-f168fb2cb0b0 from mountpoint /dev/vdb
Nov 29 03:19:26 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-3fad31e7-f6ab-4b09-aae9-831ef75d71ed tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Attempting to detach device vdb from instance aaf1c3e3-b73e-42ad-87bd-c5255ec09b46 from the persistent domain config. {{(pid=75234) _detach_from_persistent /opt/stack/nova/nova/virt/libvirt/driver.py:2487}}
Nov 29 03:19:26 user nova-compute[75234]: DEBUG nova.virt.libvirt.guest [None req-3fad31e7-f6ab-4b09-aae9-831ef75d71ed tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] detach device xml: <disk type="block" device="disk">
Nov 29 03:19:26 user nova-compute[75234]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Nov 29 03:19:26 user nova-compute[75234]:   <source dev="/dev/sdf"/>
Nov 29 03:19:26 user nova-compute[75234]:   <target dev="vdb" bus="virtio"/>
Nov 29 03:19:26 user nova-compute[75234]:   <serial>7a1a1247-abe3-4a99-a7cb-f168fb2cb0b0</serial>
Nov 29 03:19:26 user nova-compute[75234]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x07" function="0x0"/>
Nov 29 03:19:26 user nova-compute[75234]: </disk>
Nov 29 03:19:26 user nova-compute[75234]:  {{(pid=75234) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Nov 29 03:19:26 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-3fad31e7-f6ab-4b09-aae9-831ef75d71ed tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Successfully detached device vdb from instance aaf1c3e3-b73e-42ad-87bd-c5255ec09b46 from the persistent domain config.
Nov 29 03:19:26 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-3fad31e7-f6ab-4b09-aae9-831ef75d71ed tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] (1/8): Attempting to detach device vdb with device alias virtio-disk1 from instance aaf1c3e3-b73e-42ad-87bd-c5255ec09b46 from the live domain config. {{(pid=75234) _detach_from_live_with_retry /opt/stack/nova/nova/virt/libvirt/driver.py:2523}}
Nov 29 03:19:26 user nova-compute[75234]: DEBUG nova.virt.libvirt.guest [None req-3fad31e7-f6ab-4b09-aae9-831ef75d71ed tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] detach device xml: <disk type="block" device="disk">
Nov 29 03:19:26 user nova-compute[75234]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Nov 29 03:19:26 user nova-compute[75234]:   <source dev="/dev/sdf"/>
Nov 29 03:19:26 user nova-compute[75234]:   <target dev="vdb" bus="virtio"/>
Nov 29 03:19:26 user nova-compute[75234]:   <serial>7a1a1247-abe3-4a99-a7cb-f168fb2cb0b0</serial>
Nov 29 03:19:26 user nova-compute[75234]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x07" function="0x0"/>
Nov 29 03:19:26 user nova-compute[75234]: </disk>
Nov 29 03:19:26 user nova-compute[75234]:  {{(pid=75234) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Nov 29 03:19:26 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-0b7fb330-8408-410c-b9e8-3270e7e2d64f tempest-TestServerVolumeAttachScenarioOldVersion-830274084 tempest-TestServerVolumeAttachScenarioOldVersion-830274084-project-member] Waiting for libvirt event about the detach of device vdb with device alias virtio-disk1 from instance 5e6cca46-c5a7-4303-98c2-1ad483c6d14e is timed out.
Nov 29 03:19:26 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-0b7fb330-8408-410c-b9e8-3270e7e2d64f tempest-TestServerVolumeAttachScenarioOldVersion-830274084 tempest-TestServerVolumeAttachScenarioOldVersion-830274084-project-member] Successfully detached device vdb from instance 5e6cca46-c5a7-4303-98c2-1ad483c6d14e from the live domain config.
Nov 29 03:19:26 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-0b7fb330-8408-410c-b9e8-3270e7e2d64f tempest-TestServerVolumeAttachScenarioOldVersion-830274084 tempest-TestServerVolumeAttachScenarioOldVersion-830274084-project-member] [instance: 5e6cca46-c5a7-4303-98c2-1ad483c6d14e] calling os-brick to detach iSCSI Volume {{(pid=75234) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Nov 29 03:19:26 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-0b7fb330-8408-410c-b9e8-3270e7e2d64f tempest-TestServerVolumeAttachScenarioOldVersion-830274084 tempest-TestServerVolumeAttachScenarioOldVersion-830274084-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f8a5050c9a0>, {'target_iqn': 'iqn.2010-10.org.openstack:a06ab71b-02cb-4c8a-8e7b-26d8176cddf6', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'hanOIiWP', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdb'}, None), 'kwargs': False}}" {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:176}}
Nov 29 03:19:26 user nova-compute[75234]: WARNING os_brick.initiator.connectors.base [None req-0b7fb330-8408-410c-b9e8-3270e7e2d64f tempest-TestServerVolumeAttachScenarioOldVersion-830274084 tempest-TestServerVolumeAttachScenarioOldVersion-830274084-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Nov 29 03:19:26 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-0b7fb330-8408-410c-b9e8-3270e7e2d64f tempest-TestServerVolumeAttachScenarioOldVersion-830274084 tempest-TestServerVolumeAttachScenarioOldVersion-830274084-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:68}}
Nov 29 03:19:26 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-0b7fb330-8408-410c-b9e8-3270e7e2d64f tempest-TestServerVolumeAttachScenarioOldVersion-830274084 tempest-TestServerVolumeAttachScenarioOldVersion-830274084-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:73}}
Nov 29 03:19:26 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:19:26 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.032s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:19:26 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[42c5d905-b680-4dba-98f6-8294b353c6be]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:9d734633-75e6-4a97-a3c5-bd9e45d70cca\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:07f83efc-272d-4e10-9a2b-48d967db613c\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:cf6ab229-533a-424e-bd67-1d54d50ad4be\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:bf57b967-7fad-4d76-abb2-319fe88f8ef9\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:7a1a1247-abe3-4a99-a7cb-f168fb2cb0b0\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:a06ab71b-02cb-4c8a-8e7b-26d8176cddf6\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:19:26 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-0b7fb330-8408-410c-b9e8-3270e7e2d64f tempest-TestServerVolumeAttachScenarioOldVersion-830274084 tempest-TestServerVolumeAttachScenarioOldVersion-830274084-project-member] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS:
Nov 29 03:19:26 user nova-compute[75234]: No targets found.
Nov 29 03:19:26 user nova-compute[75234]: iSNS:
Nov 29 03:19:26 user nova-compute[75234]: No targets found.
Nov 29 03:19:26 user nova-compute[75234]: STATIC:
Nov 29 03:19:26 user nova-compute[75234]: Target: iqn.2010-10.org.openstack:9d734633-75e6-4a97-a3c5-bd9e45d70cca
Nov 29 03:19:26 user nova-compute[75234]:         Portal: 172.16.0.220:3260,-1
Nov 29 03:19:26 user nova-compute[75234]:                 Iface Name: default
Nov 29 03:19:26 user nova-compute[75234]: Target: iqn.2010-10.org.openstack:07f83efc-272d-4e10-9a2b-48d967db613c
Nov 29 03:19:26 user nova-compute[75234]:         Portal: 172.16.0.220:3260,-1
Nov 29 03:19:26 user nova-compute[75234]:                 Iface Name: default
Nov 29 03:19:26 user nova-compute[75234]: Target: iqn.2010-10.org.openstack:cf6ab229-533a-424e-bd67-1d54d50ad4be
Nov 29 03:19:26 user nova-compute[75234]:         Portal: 172.16.0.220:3260,-1
Nov 29 03:19:26 user nova-compute[75234]:                 Iface Name: default
Nov 29 03:19:26 user nova-compute[75234]: Target: iqn.2010-10.org.openstack:bf57b967-7fad-4d76-abb2-319fe88f8ef9
Nov 29 03:19:26 user nova-compute[75234]:         Portal: 172.16.0.220:3260,-1
Nov 29 03:19:26 user nova-compute[75234]:                 Iface Name: default
Nov 29 03:19:26 user nova-compute[75234]: Target: iqn.2010-10.org.openstack:7a1a1247-abe3-4a99-a7cb-f168fb2cb0b0
Nov 29 03:19:26 user nova-compute[75234]:         Portal: 172.16.0.220:3260,-1
Nov 29 03:19:26 user nova-compute[75234]:                 Iface Name: default
Nov 29 03:19:26 user nova-compute[75234]: Target: iqn.2010-10.org.openstack:a06ab71b-02cb-4c8a-8e7b-26d8176cddf6
Nov 29 03:19:26 user nova-compute[75234]:         Portal: 172.16.0.220:3260,-1
Nov 29 03:19:26 user nova-compute[75234]:                 Iface Name: default
Nov 29 03:19:26 user nova-compute[75234]: FIRMWARE:
Nov 29 03:19:26 user nova-compute[75234]: No targets found.
Nov 29 03:19:26 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:19:26 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-0b7fb330-8408-410c-b9e8-3270e7e2d64f tempest-TestServerVolumeAttachScenarioOldVersion-830274084 tempest-TestServerVolumeAttachScenarioOldVersion-830274084-project-member] Regex to get portals from discoverydb: ^SENDTARGETS:
Nov 29 03:19:26 user nova-compute[75234]: .*?^DiscoveryAddress: 172.16.0.220,3260.*?
Nov 29 03:19:26 user nova-compute[75234]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=75234) _get_discoverydb_portals /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:382}}
Nov 29 03:19:26 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-0b7fb330-8408-410c-b9e8-3270e7e2d64f tempest-TestServerVolumeAttachScenarioOldVersion-830274084 tempest-TestServerVolumeAttachScenarioOldVersion-830274084-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:a06ab71b-02cb-4c8a-8e7b-26d8176cddf6', 0)] {{(pid=75234) _get_connection_devices /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:830}}
Nov 29 03:19:26 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:19:26 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.018s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:19:26 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[0f5d0124-5c3b-482b-9fbb-e8b22f41cf03]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:07f83efc-272d-4e10-9a2b-48d967db613c\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:7a1a1247-abe3-4a99-a7cb-f168fb2cb0b0\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:9d734633-75e6-4a97-a3c5-bd9e45d70cca\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:a06ab71b-02cb-4c8a-8e7b-26d8176cddf6\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:bf57b967-7fad-4d76-abb2-319fe88f8ef9\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:cf6ab229-533a-424e-bd67-1d54d50ad4be\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:19:26 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:19:26 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.027s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:19:26 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[2e755629-10bf-4068-be50-6b83e141faee]: (4, ('tcp: [29] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bf57b967-7fad-4d76-abb2-319fe88f8ef9 (non-flash)\ntcp: [32] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:cf6ab229-533a-424e-bd67-1d54d50ad4be (non-flash)\ntcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:07f83efc-272d-4e10-9a2b-48d967db613c (non-flash)\ntcp: [34] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a06ab71b-02cb-4c8a-8e7b-26d8176cddf6 (non-flash)\ntcp: [35] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7a1a1247-abe3-4a99-a7cb-f168fb2cb0b0 (non-flash)\ntcp: [36] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9d734633-75e6-4a97-a3c5-bd9e45d70cca (non-flash)\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:19:26 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-0b7fb330-8408-410c-b9e8-3270e7e2d64f tempest-TestServerVolumeAttachScenarioOldVersion-830274084 tempest-TestServerVolumeAttachScenarioOldVersion-830274084-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [29] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bf57b967-7fad-4d76-abb2-319fe88f8ef9 (non-flash)
Nov 29 03:19:26 user nova-compute[75234]: tcp: [32] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:cf6ab229-533a-424e-bd67-1d54d50ad4be (non-flash)
Nov 29 03:19:26 user nova-compute[75234]: tcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:07f83efc-272d-4e10-9a2b-48d967db613c (non-flash)
Nov 29 03:19:26 user nova-compute[75234]: tcp: [34] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a06ab71b-02cb-4c8a-8e7b-26d8176cddf6 (non-flash)
Nov 29 03:19:26 user nova-compute[75234]: tcp: [35] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7a1a1247-abe3-4a99-a7cb-f168fb2cb0b0 (non-flash)
Nov 29 03:19:26 user nova-compute[75234]: tcp: [36] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9d734633-75e6-4a97-a3c5-bd9e45d70cca (non-flash)
Nov 29 03:19:26 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:19:26 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-0b7fb330-8408-410c-b9e8-3270e7e2d64f tempest-TestServerVolumeAttachScenarioOldVersion-830274084 tempest-TestServerVolumeAttachScenarioOldVersion-830274084-project-member] iscsi session list stdout=tcp: [29] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bf57b967-7fad-4d76-abb2-319fe88f8ef9 (non-flash)
Nov 29 03:19:26 user nova-compute[75234]: tcp: [32] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:cf6ab229-533a-424e-bd67-1d54d50ad4be (non-flash)
Nov 29 03:19:26 user nova-compute[75234]: tcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:07f83efc-272d-4e10-9a2b-48d967db613c (non-flash)
Nov 29 03:19:26 user nova-compute[75234]: tcp: [34] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a06ab71b-02cb-4c8a-8e7b-26d8176cddf6 (non-flash)
Nov 29 03:19:26 user nova-compute[75234]: tcp: [35] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7a1a1247-abe3-4a99-a7cb-f168fb2cb0b0 (non-flash)
Nov 29 03:19:26 user nova-compute[75234]: tcp: [36] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9d734633-75e6-4a97-a3c5-bd9e45d70cca (non-flash)
Nov 29 03:19:26 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsi_session /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Nov 29 03:19:26 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-0b7fb330-8408-410c-b9e8-3270e7e2d64f tempest-TestServerVolumeAttachScenarioOldVersion-830274084 tempest-TestServerVolumeAttachScenarioOldVersion-830274084-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7f8a6035e7a0>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:a06ab71b-02cb-4c8a-8e7b-26d8176cddf6'): ({'sdb'}, set())}) {{(pid=75234) _get_connection_devices /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:863}}
Nov 29 03:19:26 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-0b7fb330-8408-410c-b9e8-3270e7e2d64f tempest-TestServerVolumeAttachScenarioOldVersion-830274084 tempest-TestServerVolumeAttachScenarioOldVersion-830274084-project-member] Removing single pathed devices sdb {{(pid=75234) remove_connection /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:374}}
Nov 29 03:19:26 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:19:26 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.020s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:19:26 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[9daf3ee8-0b88-4bdf-b021-7c9e2fd6a8b5]: (4, ('path checker states:\nup                  6\n\npaths: 0\nbusy: False\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:19:26 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdb {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:19:26 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdb" returned: 0 in 0.018s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:19:26 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[c015562e-8de7-4d65-bc33-a166a4b54cad]: (4, ('ok\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:19:26 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-0b7fb330-8408-410c-b9e8-3270e7e2d64f tempest-TestServerVolumeAttachScenarioOldVersion-830274084 tempest-TestServerVolumeAttachScenarioOldVersion-830274084-project-member] Flushing IO for device /dev/sdb {{(pid=75234) flush_device_io /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:434}}
Nov 29 03:19:26 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdb {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:19:26 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdb" returned: 0 in 0.013s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:19:26 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[1e73f583-7dbd-4a5c-b62f-51d7530412f5]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:19:26 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-0b7fb330-8408-410c-b9e8-3270e7e2d64f tempest-TestServerVolumeAttachScenarioOldVersion-830274084 tempest-TestServerVolumeAttachScenarioOldVersion-830274084-project-member] Remove SCSI device /dev/sdb with /sys/block/sdb/device/delete {{(pid=75234) remove_scsi_device /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:148}}
Nov 29 03:19:26 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdb/device/delete {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:19:26 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdb/device/delete" returned: 0 in 0.041s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:19:26 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[04dd41fa-e52d-4b18-bc81-59cc14ff3e92]: (4, ('1', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:19:26 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-0b7fb330-8408-410c-b9e8-3270e7e2d64f tempest-TestServerVolumeAttachScenarioOldVersion-830274084 tempest-TestServerVolumeAttachScenarioOldVersion-830274084-project-member] Checking to see if SCSI volumes sdb have been removed. {{(pid=75234) wait_for_volumes_removal /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:156}}
Nov 29 03:19:26 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-0b7fb330-8408-410c-b9e8-3270e7e2d64f tempest-TestServerVolumeAttachScenarioOldVersion-830274084 tempest-TestServerVolumeAttachScenarioOldVersion-830274084-project-member] SCSI volumes sdb have been removed. {{(pid=75234) wait_for_volumes_removal /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:166}}
Nov 29 03:19:26 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-0b7fb330-8408-410c-b9e8-3270e7e2d64f tempest-TestServerVolumeAttachScenarioOldVersion-830274084 tempest-TestServerVolumeAttachScenarioOldVersion-830274084-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:a06ab71b-02cb-4c8a-8e7b-26d8176cddf6')] {{(pid=75234) _disconnect_connection /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Nov 29 03:19:26 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:a06ab71b-02cb-4c8a-8e7b-26d8176cddf6 -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:19:26 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:a06ab71b-02cb-4c8a-8e7b-26d8176cddf6 -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.023s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:19:26 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[9aa6cf2b-0fe2-45eb-aef3-3afbf49f2f89]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:19:26 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-0b7fb330-8408-410c-b9e8-3270e7e2d64f tempest-TestServerVolumeAttachScenarioOldVersion-830274084 tempest-TestServerVolumeAttachScenarioOldVersion-830274084-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:19:26 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:a06ab71b-02cb-4c8a-8e7b-26d8176cddf6 -p 172.16.0.220:3260 --logout {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:19:26 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:a06ab71b-02cb-4c8a-8e7b-26d8176cddf6 -p 172.16.0.220:3260 --logout" returned: 0 in 0.064s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:19:26 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[1df057ae-cfa5-4c3e-a420-207512b8b6c6]: (4, ('Logging out of session [sid: 34, target: iqn.2010-10.org.openstack:a06ab71b-02cb-4c8a-8e7b-26d8176cddf6, portal: 172.16.0.220,3260]\nLogout of [sid: 34, target: iqn.2010-10.org.openstack:a06ab71b-02cb-4c8a-8e7b-26d8176cddf6, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:19:26 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-0b7fb330-8408-410c-b9e8-3270e7e2d64f tempest-TestServerVolumeAttachScenarioOldVersion-830274084 tempest-TestServerVolumeAttachScenarioOldVersion-830274084-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 34, target: iqn.2010-10.org.openstack:a06ab71b-02cb-4c8a-8e7b-26d8176cddf6, portal: 172.16.0.220,3260]
Nov 29 03:19:26 user nova-compute[75234]: Logout of [sid: 34, target: iqn.2010-10.org.openstack:a06ab71b-02cb-4c8a-8e7b-26d8176cddf6, portal: 172.16.0.220,3260] successful.
Nov 29 03:19:26 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:19:26 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:a06ab71b-02cb-4c8a-8e7b-26d8176cddf6 -p 172.16.0.220:3260 --op delete {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:19:26 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:a06ab71b-02cb-4c8a-8e7b-26d8176cddf6 -p 172.16.0.220:3260 --op delete" returned: 0 in 0.031s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:19:26 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[6e6d49ff-ae97-4764-b789-3e5f118eec6a]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:19:26 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-0b7fb330-8408-410c-b9e8-3270e7e2d64f tempest-TestServerVolumeAttachScenarioOldVersion-830274084 tempest-TestServerVolumeAttachScenarioOldVersion-830274084-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:19:26 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-0b7fb330-8408-410c-b9e8-3270e7e2d64f tempest-TestServerVolumeAttachScenarioOldVersion-830274084 tempest-TestServerVolumeAttachScenarioOldVersion-830274084-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.342s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:87}}
Nov 29 03:19:26 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-0b7fb330-8408-410c-b9e8-3270e7e2d64f tempest-TestServerVolumeAttachScenarioOldVersion-830274084 tempest-TestServerVolumeAttachScenarioOldVersion-830274084-project-member] <== disconnect_volume: return (343ms) None {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:203}}
Nov 29 03:19:26 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-0b7fb330-8408-410c-b9e8-3270e7e2d64f tempest-TestServerVolumeAttachScenarioOldVersion-830274084 tempest-TestServerVolumeAttachScenarioOldVersion-830274084-project-member] [instance: 5e6cca46-c5a7-4303-98c2-1ad483c6d14e] Disconnected iSCSI Volume {{(pid=75234) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:79}}
Nov 29 03:19:26 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-3fad31e7-f6ab-4b09-aae9-831ef75d71ed tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Start waiting for the detach event from libvirt for device vdb with device alias virtio-disk1 for instance aaf1c3e3-b73e-42ad-87bd-c5255ec09b46 {{(pid=75234) _detach_from_live_and_wait_for_event /opt/stack/nova/nova/virt/libvirt/driver.py:2599}}
Nov 29 03:19:27 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:19:27 user nova-compute[75234]: DEBUG nova.objects.instance [None req-0b7fb330-8408-410c-b9e8-3270e7e2d64f tempest-TestServerVolumeAttachScenarioOldVersion-830274084 tempest-TestServerVolumeAttachScenarioOldVersion-830274084-project-member] Lazy-loading 'flavor' on Instance uuid 5e6cca46-c5a7-4303-98c2-1ad483c6d14e {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:19:27 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-0b7fb330-8408-410c-b9e8-3270e7e2d64f tempest-TestServerVolumeAttachScenarioOldVersion-830274084 tempest-TestServerVolumeAttachScenarioOldVersion-830274084-project-member] Lock "5e6cca46-c5a7-4303-98c2-1ad483c6d14e" "released" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: held 22.185s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:19:30 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:19:32 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:19:35 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:19:37 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:19:40 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:19:42 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:19:43 user nova-compute[75234]: DEBUG nova.compute.manager [req-b505fd68-fd0f-43ac-9851-920eda96f092 req-247d1d1b-2fea-4004-8afc-c44d79c80464 service nova] [instance: 21e9c6cd-c39f-41c0-8768-5dd64589c3f3] Received event network-changed-87f1e736-6665-4945-bd19-0cf93bff905e {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:19:43 user nova-compute[75234]: DEBUG nova.compute.manager [req-b505fd68-fd0f-43ac-9851-920eda96f092 req-247d1d1b-2fea-4004-8afc-c44d79c80464 service nova] [instance: 21e9c6cd-c39f-41c0-8768-5dd64589c3f3] Refreshing instance network info cache due to event network-changed-87f1e736-6665-4945-bd19-0cf93bff905e. {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11117}}
Nov 29 03:19:43 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-b505fd68-fd0f-43ac-9851-920eda96f092 req-247d1d1b-2fea-4004-8afc-c44d79c80464 service nova] Acquiring lock "refresh_cache-21e9c6cd-c39f-41c0-8768-5dd64589c3f3" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:19:43 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-b505fd68-fd0f-43ac-9851-920eda96f092 req-247d1d1b-2fea-4004-8afc-c44d79c80464 service nova] Acquired lock "refresh_cache-21e9c6cd-c39f-41c0-8768-5dd64589c3f3" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:19:43 user nova-compute[75234]: DEBUG nova.network.neutron [req-b505fd68-fd0f-43ac-9851-920eda96f092 req-247d1d1b-2fea-4004-8afc-c44d79c80464 service nova] [instance: 21e9c6cd-c39f-41c0-8768-5dd64589c3f3] Refreshing network info cache for port 87f1e736-6665-4945-bd19-0cf93bff905e {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2006}}
Nov 29 03:19:44 user nova-compute[75234]: DEBUG nova.network.neutron [req-b505fd68-fd0f-43ac-9851-920eda96f092 req-247d1d1b-2fea-4004-8afc-c44d79c80464 service nova] [instance: 21e9c6cd-c39f-41c0-8768-5dd64589c3f3] Updated VIF entry in instance network info cache for port 87f1e736-6665-4945-bd19-0cf93bff905e. {{(pid=75234) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3481}}
Nov 29 03:19:44 user nova-compute[75234]: DEBUG nova.network.neutron [req-b505fd68-fd0f-43ac-9851-920eda96f092 req-247d1d1b-2fea-4004-8afc-c44d79c80464 service nova] [instance: 21e9c6cd-c39f-41c0-8768-5dd64589c3f3] Updating instance_info_cache with network_info: [{"id": "87f1e736-6665-4945-bd19-0cf93bff905e", "address": "fa:16:3e:40:14:df", "network": {"id": "1f946e69-dbec-429c-83f8-a186bad4f2cc", "bridge": "br-int", "label": "tempest-ServerActionsTestOtherA-486369423-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "c1cb3d1f12f242a785a688d59dbe3cd6", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap87f1e736-66", "ovs_interfaceid": "87f1e736-6665-4945-bd19-0cf93bff905e", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:19:44 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-b505fd68-fd0f-43ac-9851-920eda96f092 req-247d1d1b-2fea-4004-8afc-c44d79c80464 service nova] Releasing lock "refresh_cache-21e9c6cd-c39f-41c0-8768-5dd64589c3f3" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:19:45 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:19:45 user nova-compute[75234]: DEBUG nova.compute.manager [None req-7b720460-3d17-493e-a134-1ef47ad6fba2 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] [instance: af699bc2-9daa-4b5a-9b4a-ad92fe857d70] Stashing vm_state: active {{(pid=75234) _prep_resize /opt/stack/nova/nova/compute/manager.py:5615}}
Nov 29 03:19:45 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:19:45 user nova-compute[75234]: DEBUG nova.compute.manager [req-d99f59d1-27b9-465b-8e99-5ad2ba2ad40d req-b1ff634c-9f8e-4d46-9eb0-90a698f18647 service nova] [instance: af699bc2-9daa-4b5a-9b4a-ad92fe857d70] Received event network-changed-66dcb0f9-3180-42f7-851d-19b0c4f4119c {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:19:45 user nova-compute[75234]: DEBUG nova.compute.manager [req-d99f59d1-27b9-465b-8e99-5ad2ba2ad40d req-b1ff634c-9f8e-4d46-9eb0-90a698f18647 service nova] [instance: af699bc2-9daa-4b5a-9b4a-ad92fe857d70] Refreshing instance network info cache due to event network-changed-66dcb0f9-3180-42f7-851d-19b0c4f4119c. {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11117}}
Nov 29 03:19:45 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-d99f59d1-27b9-465b-8e99-5ad2ba2ad40d req-b1ff634c-9f8e-4d46-9eb0-90a698f18647 service nova] Acquiring lock "refresh_cache-af699bc2-9daa-4b5a-9b4a-ad92fe857d70" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:19:45 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-d99f59d1-27b9-465b-8e99-5ad2ba2ad40d req-b1ff634c-9f8e-4d46-9eb0-90a698f18647 service nova] Acquired lock "refresh_cache-af699bc2-9daa-4b5a-9b4a-ad92fe857d70" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:19:45 user nova-compute[75234]: DEBUG nova.network.neutron [req-d99f59d1-27b9-465b-8e99-5ad2ba2ad40d req-b1ff634c-9f8e-4d46-9eb0-90a698f18647 service nova] [instance: af699bc2-9daa-4b5a-9b4a-ad92fe857d70] Refreshing network info cache for port 66dcb0f9-3180-42f7-851d-19b0c4f4119c {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2006}}
Nov 29 03:19:45 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-7b720460-3d17-493e-a134-1ef47ad6fba2 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.resize_claim" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:19:45 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-7b720460-3d17-493e-a134-1ef47ad6fba2 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.resize_claim" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:19:45 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-7b720460-3d17-493e-a134-1ef47ad6fba2 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=75234) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2366}}
Nov 29 03:19:45 user nova-compute[75234]: INFO nova.compute.claims [None req-7b720460-3d17-493e-a134-1ef47ad6fba2 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] [instance: af699bc2-9daa-4b5a-9b4a-ad92fe857d70] Claim successful on node user
Nov 29 03:19:45 user nova-compute[75234]: INFO nova.compute.resource_tracker [None req-7b720460-3d17-493e-a134-1ef47ad6fba2 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] [instance: af699bc2-9daa-4b5a-9b4a-ad92fe857d70] Updating resource usage from migration 9297d372-3baa-4214-a4fd-ae62460f818a
Nov 29 03:19:46 user nova-compute[75234]: DEBUG nova.network.neutron [req-d99f59d1-27b9-465b-8e99-5ad2ba2ad40d req-b1ff634c-9f8e-4d46-9eb0-90a698f18647 service nova] [instance: af699bc2-9daa-4b5a-9b4a-ad92fe857d70] Updated VIF entry in instance network info cache for port 66dcb0f9-3180-42f7-851d-19b0c4f4119c. {{(pid=75234) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3481}}
Nov 29 03:19:46 user nova-compute[75234]: DEBUG nova.network.neutron [req-d99f59d1-27b9-465b-8e99-5ad2ba2ad40d req-b1ff634c-9f8e-4d46-9eb0-90a698f18647 service nova] [instance: af699bc2-9daa-4b5a-9b4a-ad92fe857d70] Updating instance_info_cache with network_info: [{"id": "66dcb0f9-3180-42f7-851d-19b0c4f4119c", "address": "fa:16:3e:dd:9e:50", "network": {"id": "1f946e69-dbec-429c-83f8-a186bad4f2cc", "bridge": "br-int", "label": "tempest-ServerActionsTestOtherA-486369423-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.64", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "c1cb3d1f12f242a785a688d59dbe3cd6", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap66dcb0f9-31", "ovs_interfaceid": "66dcb0f9-3180-42f7-851d-19b0c4f4119c", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:19:46 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-d99f59d1-27b9-465b-8e99-5ad2ba2ad40d req-b1ff634c-9f8e-4d46-9eb0-90a698f18647 service nova] Releasing lock "refresh_cache-af699bc2-9daa-4b5a-9b4a-ad92fe857d70" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:19:46 user nova-compute[75234]: DEBUG nova.compute.provider_tree [None req-7b720460-3d17-493e-a134-1ef47ad6fba2 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Inventory has not changed in ProviderTree for provider: 4d291220-7c37-4839-a0e3-b23c66502b74 {{(pid=75234) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Nov 29 03:19:46 user nova-compute[75234]: DEBUG nova.scheduler.client.report [None req-7b720460-3d17-493e-a134-1ef47ad6fba2 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Inventory has not changed for provider 4d291220-7c37-4839-a0e3-b23c66502b74 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=75234) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Nov 29 03:19:46 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-7b720460-3d17-493e-a134-1ef47ad6fba2 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.resize_claim" :: held 0.486s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:19:46 user nova-compute[75234]: INFO nova.compute.manager [None req-7b720460-3d17-493e-a134-1ef47ad6fba2 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] [instance: af699bc2-9daa-4b5a-9b4a-ad92fe857d70] Migrating
Nov 29 03:19:46 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-7b720460-3d17-493e-a134-1ef47ad6fba2 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Acquiring lock "refresh_cache-af699bc2-9daa-4b5a-9b4a-ad92fe857d70" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:19:46 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-7b720460-3d17-493e-a134-1ef47ad6fba2 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Acquired lock "refresh_cache-af699bc2-9daa-4b5a-9b4a-ad92fe857d70" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:19:46 user nova-compute[75234]: DEBUG nova.network.neutron [None req-7b720460-3d17-493e-a134-1ef47ad6fba2 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] [instance: af699bc2-9daa-4b5a-9b4a-ad92fe857d70] Building network info cache for instance {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2009}}
Nov 29 03:19:46 user nova-compute[75234]: DEBUG nova.network.neutron [None req-7b720460-3d17-493e-a134-1ef47ad6fba2 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] [instance: af699bc2-9daa-4b5a-9b4a-ad92fe857d70] Updating instance_info_cache with network_info: [{"id": "66dcb0f9-3180-42f7-851d-19b0c4f4119c", "address": "fa:16:3e:dd:9e:50", "network": {"id": "1f946e69-dbec-429c-83f8-a186bad4f2cc", "bridge": "br-int", "label": "tempest-ServerActionsTestOtherA-486369423-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.64", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "c1cb3d1f12f242a785a688d59dbe3cd6", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap66dcb0f9-31", "ovs_interfaceid": "66dcb0f9-3180-42f7-851d-19b0c4f4119c", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:19:46 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-7b720460-3d17-493e-a134-1ef47ad6fba2 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Releasing lock "refresh_cache-af699bc2-9daa-4b5a-9b4a-ad92fe857d70" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:19:46 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-3fad31e7-f6ab-4b09-aae9-831ef75d71ed tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Waiting for libvirt event about the detach of device vdb with device alias virtio-disk1 from instance aaf1c3e3-b73e-42ad-87bd-c5255ec09b46 is timed out.
Nov 29 03:19:46 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-3fad31e7-f6ab-4b09-aae9-831ef75d71ed tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Successfully detached device vdb from instance aaf1c3e3-b73e-42ad-87bd-c5255ec09b46 from the live domain config.
Nov 29 03:19:46 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-3fad31e7-f6ab-4b09-aae9-831ef75d71ed tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] [instance: aaf1c3e3-b73e-42ad-87bd-c5255ec09b46] calling os-brick to detach iSCSI Volume {{(pid=75234) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Nov 29 03:19:46 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-3fad31e7-f6ab-4b09-aae9-831ef75d71ed tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f8a5050c9a0>, {'target_iqn': 'iqn.2010-10.org.openstack:7a1a1247-abe3-4a99-a7cb-f168fb2cb0b0', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'lgpvtzQE', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdf'}, None), 'kwargs': False}}" {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:176}}
Nov 29 03:19:46 user nova-compute[75234]: WARNING os_brick.initiator.connectors.base [None req-3fad31e7-f6ab-4b09-aae9-831ef75d71ed tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Nov 29 03:19:46 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-3fad31e7-f6ab-4b09-aae9-831ef75d71ed tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:68}}
Nov 29 03:19:46 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-3fad31e7-f6ab-4b09-aae9-831ef75d71ed tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.002s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:73}}
Nov 29 03:19:46 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:19:46 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.047s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:19:46 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[73067fdf-0c5e-4e19-88d6-6ff4ab8653b0]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:9d734633-75e6-4a97-a3c5-bd9e45d70cca\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:07f83efc-272d-4e10-9a2b-48d967db613c\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:cf6ab229-533a-424e-bd67-1d54d50ad4be\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:bf57b967-7fad-4d76-abb2-319fe88f8ef9\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:7a1a1247-abe3-4a99-a7cb-f168fb2cb0b0\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:19:46 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-3fad31e7-f6ab-4b09-aae9-831ef75d71ed tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS:
Nov 29 03:19:46 user nova-compute[75234]: No targets found.
Nov 29 03:19:46 user nova-compute[75234]: iSNS:
Nov 29 03:19:46 user nova-compute[75234]: No targets found.
Nov 29 03:19:46 user nova-compute[75234]: STATIC:
Nov 29 03:19:46 user nova-compute[75234]: Target: iqn.2010-10.org.openstack:9d734633-75e6-4a97-a3c5-bd9e45d70cca
Nov 29 03:19:46 user nova-compute[75234]:         Portal: 172.16.0.220:3260,-1
Nov 29 03:19:46 user nova-compute[75234]:                 Iface Name: default
Nov 29 03:19:46 user nova-compute[75234]: Target: iqn.2010-10.org.openstack:07f83efc-272d-4e10-9a2b-48d967db613c
Nov 29 03:19:46 user nova-compute[75234]:         Portal: 172.16.0.220:3260,-1
Nov 29 03:19:46 user nova-compute[75234]:                 Iface Name: default
Nov 29 03:19:46 user nova-compute[75234]: Target: iqn.2010-10.org.openstack:cf6ab229-533a-424e-bd67-1d54d50ad4be
Nov 29 03:19:46 user nova-compute[75234]:         Portal: 172.16.0.220:3260,-1
Nov 29 03:19:46 user nova-compute[75234]:                 Iface Name: default
Nov 29 03:19:46 user nova-compute[75234]: Target: iqn.2010-10.org.openstack:bf57b967-7fad-4d76-abb2-319fe88f8ef9
Nov 29 03:19:46 user nova-compute[75234]:         Portal: 172.16.0.220:3260,-1
Nov 29 03:19:46 user nova-compute[75234]:                 Iface Name: default
Nov 29 03:19:46 user nova-compute[75234]: Target: iqn.2010-10.org.openstack:7a1a1247-abe3-4a99-a7cb-f168fb2cb0b0
Nov 29 03:19:46 user nova-compute[75234]:         Portal: 172.16.0.220:3260,-1
Nov 29 03:19:46 user nova-compute[75234]:                 Iface Name: default
Nov 29 03:19:46 user nova-compute[75234]: FIRMWARE:
Nov 29 03:19:46 user nova-compute[75234]: No targets found.
Nov 29 03:19:46 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:19:46 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-3fad31e7-f6ab-4b09-aae9-831ef75d71ed tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Regex to get portals from discoverydb: ^SENDTARGETS:
Nov 29 03:19:46 user nova-compute[75234]: .*?^DiscoveryAddress: 172.16.0.220,3260.*?
Nov 29 03:19:46 user nova-compute[75234]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=75234) _get_discoverydb_portals /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:382}}
Nov 29 03:19:46 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-3fad31e7-f6ab-4b09-aae9-831ef75d71ed tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:7a1a1247-abe3-4a99-a7cb-f168fb2cb0b0', 0)] {{(pid=75234) _get_connection_devices /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:830}}
Nov 29 03:19:46 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:19:46 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.021s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:19:46 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[e0c8ee0e-3b7e-4a93-bacf-74f02c06e237]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:07f83efc-272d-4e10-9a2b-48d967db613c\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:7a1a1247-abe3-4a99-a7cb-f168fb2cb0b0\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:9d734633-75e6-4a97-a3c5-bd9e45d70cca\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:bf57b967-7fad-4d76-abb2-319fe88f8ef9\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:cf6ab229-533a-424e-bd67-1d54d50ad4be\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:19:46 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-7b720460-3d17-493e-a134-1ef47ad6fba2 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] [instance: af699bc2-9daa-4b5a-9b4a-ad92fe857d70] Starting migrate_disk_and_power_off {{(pid=75234) migrate_disk_and_power_off /opt/stack/nova/nova/virt/libvirt/driver.py:11484}}
Nov 29 03:19:46 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:19:46 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.029s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:19:46 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[d2e604cc-ab38-466a-8182-bb594bf4b02a]: (4, ('tcp: [29] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bf57b967-7fad-4d76-abb2-319fe88f8ef9 (non-flash)\ntcp: [32] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:cf6ab229-533a-424e-bd67-1d54d50ad4be (non-flash)\ntcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:07f83efc-272d-4e10-9a2b-48d967db613c (non-flash)\ntcp: [35] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7a1a1247-abe3-4a99-a7cb-f168fb2cb0b0 (non-flash)\ntcp: [36] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9d734633-75e6-4a97-a3c5-bd9e45d70cca (non-flash)\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:19:46 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-3fad31e7-f6ab-4b09-aae9-831ef75d71ed tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [29] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bf57b967-7fad-4d76-abb2-319fe88f8ef9 (non-flash)
Nov 29 03:19:46 user nova-compute[75234]: tcp: [32] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:cf6ab229-533a-424e-bd67-1d54d50ad4be (non-flash)
Nov 29 03:19:46 user nova-compute[75234]: tcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:07f83efc-272d-4e10-9a2b-48d967db613c (non-flash)
Nov 29 03:19:46 user nova-compute[75234]: tcp: [35] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7a1a1247-abe3-4a99-a7cb-f168fb2cb0b0 (non-flash)
Nov 29 03:19:46 user nova-compute[75234]: tcp: [36] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9d734633-75e6-4a97-a3c5-bd9e45d70cca (non-flash)
Nov 29 03:19:46 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:19:46 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-3fad31e7-f6ab-4b09-aae9-831ef75d71ed tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] iscsi session list stdout=tcp: [29] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bf57b967-7fad-4d76-abb2-319fe88f8ef9 (non-flash)
Nov 29 03:19:46 user nova-compute[75234]: tcp: [32] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:cf6ab229-533a-424e-bd67-1d54d50ad4be (non-flash)
Nov 29 03:19:46 user nova-compute[75234]: tcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:07f83efc-272d-4e10-9a2b-48d967db613c (non-flash)
Nov 29 03:19:46 user nova-compute[75234]: tcp: [35] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:7a1a1247-abe3-4a99-a7cb-f168fb2cb0b0 (non-flash)
Nov 29 03:19:46 user nova-compute[75234]: tcp: [36] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9d734633-75e6-4a97-a3c5-bd9e45d70cca (non-flash)
Nov 29 03:19:46 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsi_session /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Nov 29 03:19:46 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-3fad31e7-f6ab-4b09-aae9-831ef75d71ed tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7f8a50374e50>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:7a1a1247-abe3-4a99-a7cb-f168fb2cb0b0'): ({'sdf'}, set())}) {{(pid=75234) _get_connection_devices /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:863}}
Nov 29 03:19:46 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-3fad31e7-f6ab-4b09-aae9-831ef75d71ed tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Removing single pathed devices sdf {{(pid=75234) remove_connection /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:374}}
Nov 29 03:19:46 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:19:46 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:19:46 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.019s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:19:46 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[88a35cdd-5653-4aa7-aa69-86ca6f5d27f0]: (4, ('path checker states:\nup                  5\n\npaths: 0\nbusy: False\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:19:46 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdf {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:19:46 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdf" returned: 0 in 0.021s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:19:46 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[62d7526e-11e2-4c90-bd9b-a626da2975e4]: (4, ('ok\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:19:46 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-3fad31e7-f6ab-4b09-aae9-831ef75d71ed tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Flushing IO for device /dev/sdf {{(pid=75234) flush_device_io /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:434}}
Nov 29 03:19:46 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdf {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:19:46 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:19:46 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdf" returned: 0 in 0.017s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:19:46 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[fa1e16b5-44c8-4afa-8f1d-4556031d4edd]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:19:46 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-3fad31e7-f6ab-4b09-aae9-831ef75d71ed tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Remove SCSI device /dev/sdf with /sys/block/sdf/device/delete {{(pid=75234) remove_scsi_device /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:148}}
Nov 29 03:19:46 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdf/device/delete {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:19:46 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdf/device/delete" returned: 0 in 0.045s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:19:46 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[1c72de31-3484-4c14-97f6-a51b61644679]: (4, ('1', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:19:46 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-3fad31e7-f6ab-4b09-aae9-831ef75d71ed tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Checking to see if SCSI volumes sdf have been removed. {{(pid=75234) wait_for_volumes_removal /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:156}}
Nov 29 03:19:46 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-3fad31e7-f6ab-4b09-aae9-831ef75d71ed tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] SCSI volumes sdf have been removed. {{(pid=75234) wait_for_volumes_removal /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:166}}
Nov 29 03:19:46 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-3fad31e7-f6ab-4b09-aae9-831ef75d71ed tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:7a1a1247-abe3-4a99-a7cb-f168fb2cb0b0')] {{(pid=75234) _disconnect_connection /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Nov 29 03:19:46 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:7a1a1247-abe3-4a99-a7cb-f168fb2cb0b0 -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:19:47 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:7a1a1247-abe3-4a99-a7cb-f168fb2cb0b0 -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.027s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:19:47 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[f5100511-9330-4464-befc-9eb23a77f6f8]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:19:47 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-3fad31e7-f6ab-4b09-aae9-831ef75d71ed tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:19:47 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:7a1a1247-abe3-4a99-a7cb-f168fb2cb0b0 -p 172.16.0.220:3260 --logout {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:19:47 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:7a1a1247-abe3-4a99-a7cb-f168fb2cb0b0 -p 172.16.0.220:3260 --logout" returned: 0 in 0.082s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:19:47 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[c3821ac5-c2d8-42bf-81e5-59eb6437bc72]: (4, ('Logging out of session [sid: 35, target: iqn.2010-10.org.openstack:7a1a1247-abe3-4a99-a7cb-f168fb2cb0b0, portal: 172.16.0.220,3260]\nLogout of [sid: 35, target: iqn.2010-10.org.openstack:7a1a1247-abe3-4a99-a7cb-f168fb2cb0b0, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:19:47 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-3fad31e7-f6ab-4b09-aae9-831ef75d71ed tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 35, target: iqn.2010-10.org.openstack:7a1a1247-abe3-4a99-a7cb-f168fb2cb0b0, portal: 172.16.0.220,3260]
Nov 29 03:19:47 user nova-compute[75234]: Logout of [sid: 35, target: iqn.2010-10.org.openstack:7a1a1247-abe3-4a99-a7cb-f168fb2cb0b0, portal: 172.16.0.220,3260] successful.
Nov 29 03:19:47 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:19:47 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:7a1a1247-abe3-4a99-a7cb-f168fb2cb0b0 -p 172.16.0.220:3260 --op delete {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:19:47 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:19:47 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:7a1a1247-abe3-4a99-a7cb-f168fb2cb0b0 -p 172.16.0.220:3260 --op delete" returned: 0 in 0.028s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:19:47 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[17d3945a-338e-4eee-891f-edcf8a02b1b1]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:19:47 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-3fad31e7-f6ab-4b09-aae9-831ef75d71ed tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:19:47 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-3fad31e7-f6ab-4b09-aae9-831ef75d71ed tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.402s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:87}}
Nov 29 03:19:47 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-3fad31e7-f6ab-4b09-aae9-831ef75d71ed tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] <== disconnect_volume: return (404ms) None {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:203}}
Nov 29 03:19:47 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-3fad31e7-f6ab-4b09-aae9-831ef75d71ed tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] [instance: aaf1c3e3-b73e-42ad-87bd-c5255ec09b46] Disconnected iSCSI Volume {{(pid=75234) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:79}}
Nov 29 03:19:47 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:19:47 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:19:47 user nova-compute[75234]: DEBUG nova.compute.manager [req-c9616eb9-898e-4e76-98ca-1d61849830fa req-2be08a5e-70b1-4702-bc89-524878d5e1a9 service nova] [instance: af699bc2-9daa-4b5a-9b4a-ad92fe857d70] Received event network-vif-unplugged-66dcb0f9-3180-42f7-851d-19b0c4f4119c {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:19:47 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-c9616eb9-898e-4e76-98ca-1d61849830fa req-2be08a5e-70b1-4702-bc89-524878d5e1a9 service nova] Acquiring lock "af699bc2-9daa-4b5a-9b4a-ad92fe857d70-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:19:47 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-c9616eb9-898e-4e76-98ca-1d61849830fa req-2be08a5e-70b1-4702-bc89-524878d5e1a9 service nova] Lock "af699bc2-9daa-4b5a-9b4a-ad92fe857d70-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:19:47 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-c9616eb9-898e-4e76-98ca-1d61849830fa req-2be08a5e-70b1-4702-bc89-524878d5e1a9 service nova] Lock "af699bc2-9daa-4b5a-9b4a-ad92fe857d70-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:19:47 user nova-compute[75234]: DEBUG nova.compute.manager [req-c9616eb9-898e-4e76-98ca-1d61849830fa req-2be08a5e-70b1-4702-bc89-524878d5e1a9 service nova] [instance: af699bc2-9daa-4b5a-9b4a-ad92fe857d70] No waiting events found dispatching network-vif-unplugged-66dcb0f9-3180-42f7-851d-19b0c4f4119c {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:19:47 user nova-compute[75234]: WARNING nova.compute.manager [req-c9616eb9-898e-4e76-98ca-1d61849830fa req-2be08a5e-70b1-4702-bc89-524878d5e1a9 service nova] [instance: af699bc2-9daa-4b5a-9b4a-ad92fe857d70] Received unexpected event network-vif-unplugged-66dcb0f9-3180-42f7-851d-19b0c4f4119c for instance with vm_state active and task_state resize_migrating.
Nov 29 03:19:47 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:19:47 user nova-compute[75234]: INFO nova.virt.libvirt.driver [-] [instance: af699bc2-9daa-4b5a-9b4a-ad92fe857d70] Instance destroyed successfully.
Nov 29 03:19:47 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-7b720460-3d17-493e-a134-1ef47ad6fba2 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-11-29T03:17:51Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-ServerActionsTestOtherA-server-1736967407',display_name='tempest-ServerActionsTestOtherA-server-1736967407',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-serveractionstestothera-server-1736967407',id=31,image_ref='',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBGysIZGz8py0LojrFdRCkkIN/DpphiAjtTxjJQY7OVo0ZaKHLKAtSriy5/opnav7HzQyT3MeG6wW+2nQfBNyK0gW9HDU5His6qvZYEbhyEyMiaUzdXksC9FbE2bauudLhQ==',key_name='tempest-keypair-1767034584',keypairs=<?>,launch_index=0,launched_at=2023-11-29T03:18:28Z,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=MigrationContext,new_flavor=Flavor(7),node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=1,progress=0,project_id='c1cb3d1f12f242a785a688d59dbe3cd6',ramdisk_id='',reservation_id='r-p258pz1s',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=ServiceList,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',old_vm_state='active',owner_project_name='tempest-ServerActionsTestOtherA-1495067208',owner_user_name='tempest-ServerActionsTestOtherA-1495067208-project-member'},tags=<?>,task_state='resize_migrating',terminated_at=None,trusted_certs=None,updated_at=2023-11-29T03:19:46Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='d611826f3be947c0947b99c763c26d44',uuid=af699bc2-9daa-4b5a-9b4a-ad92fe857d70,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "66dcb0f9-3180-42f7-851d-19b0c4f4119c", "address": "fa:16:3e:dd:9e:50", "network": {"id": "1f946e69-dbec-429c-83f8-a186bad4f2cc", "bridge": "br-int", "label": "tempest-ServerActionsTestOtherA-486369423-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.64", "type": "floating", "version": 4, "meta": {}}], "label": "tempest-ServerActionsTestOtherA-486369423-network", "vif_mac": "fa:16:3e:dd:9e:50"}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "c1cb3d1f12f242a785a688d59dbe3cd6", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap66dcb0f9-31", "ovs_interfaceid": "66dcb0f9-3180-42f7-851d-19b0c4f4119c", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Nov 29 03:19:47 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-7b720460-3d17-493e-a134-1ef47ad6fba2 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Converting VIF {"id": "66dcb0f9-3180-42f7-851d-19b0c4f4119c", "address": "fa:16:3e:dd:9e:50", "network": {"id": "1f946e69-dbec-429c-83f8-a186bad4f2cc", "bridge": "br-int", "label": "tempest-ServerActionsTestOtherA-486369423-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.64", "type": "floating", "version": 4, "meta": {}}], "label": "tempest-ServerActionsTestOtherA-486369423-network", "vif_mac": "fa:16:3e:dd:9e:50"}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "c1cb3d1f12f242a785a688d59dbe3cd6", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap66dcb0f9-31", "ovs_interfaceid": "66dcb0f9-3180-42f7-851d-19b0c4f4119c", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:19:47 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-7b720460-3d17-493e-a134-1ef47ad6fba2 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:dd:9e:50,bridge_name='br-int',has_traffic_filtering=True,id=66dcb0f9-3180-42f7-851d-19b0c4f4119c,network=Network(1f946e69-dbec-429c-83f8-a186bad4f2cc),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap66dcb0f9-31') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:19:47 user nova-compute[75234]: DEBUG os_vif [None req-7b720460-3d17-493e-a134-1ef47ad6fba2 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Unplugging vif VIFOpenVSwitch(active=True,address=fa:16:3e:dd:9e:50,bridge_name='br-int',has_traffic_filtering=True,id=66dcb0f9-3180-42f7-851d-19b0c4f4119c,network=Network(1f946e69-dbec-429c-83f8-a186bad4f2cc),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap66dcb0f9-31') {{(pid=75234) unplug /opt/stack/data/venv/lib/python3.10/site-packages/os_vif/__init__.py:109}}
Nov 29 03:19:47 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:19:47 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tap66dcb0f9-31, bridge=br-int, if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:19:47 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:19:47 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:19:47 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:19:47 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbDestroyCommand(_result=None, table=QoS, record=8822804c-faad-4793-b442-c527514dab9c) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:19:47 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:19:47 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:19:47 user nova-compute[75234]: INFO os_vif [None req-7b720460-3d17-493e-a134-1ef47ad6fba2 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Successfully unplugged vif VIFOpenVSwitch(active=True,address=fa:16:3e:dd:9e:50,bridge_name='br-int',has_traffic_filtering=True,id=66dcb0f9-3180-42f7-851d-19b0c4f4119c,network=Network(1f946e69-dbec-429c-83f8-a186bad4f2cc),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap66dcb0f9-31')
Nov 29 03:19:47 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-7b720460-3d17-493e-a134-1ef47ad6fba2 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] [instance: af699bc2-9daa-4b5a-9b4a-ad92fe857d70] calling os-brick to detach iSCSI Volume {{(pid=75234) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Nov 29 03:19:47 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-7b720460-3d17-493e-a134-1ef47ad6fba2 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f8a5050c9a0>, {'target_iqn': 'iqn.2010-10.org.openstack:cf6ab229-533a-424e-bd67-1d54d50ad4be', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'wodfcngk', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sde'}, None), 'kwargs': False}}" {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:176}}
Nov 29 03:19:47 user nova-compute[75234]: WARNING os_brick.initiator.connectors.base [None req-7b720460-3d17-493e-a134-1ef47ad6fba2 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Nov 29 03:19:47 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-7b720460-3d17-493e-a134-1ef47ad6fba2 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:68}}
Nov 29 03:19:47 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-7b720460-3d17-493e-a134-1ef47ad6fba2 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:73}}
Nov 29 03:19:47 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:19:47 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.022s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:19:47 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[9824606f-8d11-415d-bc0e-5c55411c81d2]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:9d734633-75e6-4a97-a3c5-bd9e45d70cca\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:07f83efc-272d-4e10-9a2b-48d967db613c\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:cf6ab229-533a-424e-bd67-1d54d50ad4be\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:bf57b967-7fad-4d76-abb2-319fe88f8ef9\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:19:47 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-7b720460-3d17-493e-a134-1ef47ad6fba2 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS:
Nov 29 03:19:47 user nova-compute[75234]: No targets found.
Nov 29 03:19:47 user nova-compute[75234]: iSNS:
Nov 29 03:19:47 user nova-compute[75234]: No targets found.
Nov 29 03:19:47 user nova-compute[75234]: STATIC:
Nov 29 03:19:47 user nova-compute[75234]: Target: iqn.2010-10.org.openstack:9d734633-75e6-4a97-a3c5-bd9e45d70cca
Nov 29 03:19:47 user nova-compute[75234]:         Portal: 172.16.0.220:3260,-1
Nov 29 03:19:47 user nova-compute[75234]:                 Iface Name: default
Nov 29 03:19:47 user nova-compute[75234]: Target: iqn.2010-10.org.openstack:07f83efc-272d-4e10-9a2b-48d967db613c
Nov 29 03:19:47 user nova-compute[75234]:         Portal: 172.16.0.220:3260,-1
Nov 29 03:19:47 user nova-compute[75234]:                 Iface Name: default
Nov 29 03:19:47 user nova-compute[75234]: Target: iqn.2010-10.org.openstack:cf6ab229-533a-424e-bd67-1d54d50ad4be
Nov 29 03:19:47 user nova-compute[75234]:         Portal: 172.16.0.220:3260,-1
Nov 29 03:19:47 user nova-compute[75234]:                 Iface Name: default
Nov 29 03:19:47 user nova-compute[75234]: Target: iqn.2010-10.org.openstack:bf57b967-7fad-4d76-abb2-319fe88f8ef9
Nov 29 03:19:47 user nova-compute[75234]:         Portal: 172.16.0.220:3260,-1
Nov 29 03:19:47 user nova-compute[75234]:                 Iface Name: default
Nov 29 03:19:47 user nova-compute[75234]: FIRMWARE:
Nov 29 03:19:47 user nova-compute[75234]: No targets found.
Nov 29 03:19:47 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:19:47 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-7b720460-3d17-493e-a134-1ef47ad6fba2 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Regex to get portals from discoverydb: ^SENDTARGETS:
Nov 29 03:19:47 user nova-compute[75234]: .*?^DiscoveryAddress: 172.16.0.220,3260.*?
Nov 29 03:19:47 user nova-compute[75234]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=75234) _get_discoverydb_portals /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:382}}
Nov 29 03:19:47 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-7b720460-3d17-493e-a134-1ef47ad6fba2 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:cf6ab229-533a-424e-bd67-1d54d50ad4be', 0)] {{(pid=75234) _get_connection_devices /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:830}}
Nov 29 03:19:47 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:19:47 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.023s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:19:47 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[c4942c7d-7c91-4ce6-ad1d-78b8b1f44bc8]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:07f83efc-272d-4e10-9a2b-48d967db613c\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:9d734633-75e6-4a97-a3c5-bd9e45d70cca\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:bf57b967-7fad-4d76-abb2-319fe88f8ef9\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:cf6ab229-533a-424e-bd67-1d54d50ad4be\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:19:47 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:19:47 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.033s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:19:47 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[205194ed-f24a-4d72-8732-808411bc37c5]: (4, ('tcp: [29] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bf57b967-7fad-4d76-abb2-319fe88f8ef9 (non-flash)\ntcp: [32] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:cf6ab229-533a-424e-bd67-1d54d50ad4be (non-flash)\ntcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:07f83efc-272d-4e10-9a2b-48d967db613c (non-flash)\ntcp: [36] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9d734633-75e6-4a97-a3c5-bd9e45d70cca (non-flash)\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:19:47 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-7b720460-3d17-493e-a134-1ef47ad6fba2 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [29] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bf57b967-7fad-4d76-abb2-319fe88f8ef9 (non-flash)
Nov 29 03:19:47 user nova-compute[75234]: tcp: [32] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:cf6ab229-533a-424e-bd67-1d54d50ad4be (non-flash)
Nov 29 03:19:47 user nova-compute[75234]: tcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:07f83efc-272d-4e10-9a2b-48d967db613c (non-flash)
Nov 29 03:19:47 user nova-compute[75234]: tcp: [36] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9d734633-75e6-4a97-a3c5-bd9e45d70cca (non-flash)
Nov 29 03:19:47 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:19:47 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-7b720460-3d17-493e-a134-1ef47ad6fba2 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] iscsi session list stdout=tcp: [29] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bf57b967-7fad-4d76-abb2-319fe88f8ef9 (non-flash)
Nov 29 03:19:47 user nova-compute[75234]: tcp: [32] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:cf6ab229-533a-424e-bd67-1d54d50ad4be (non-flash)
Nov 29 03:19:47 user nova-compute[75234]: tcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:07f83efc-272d-4e10-9a2b-48d967db613c (non-flash)
Nov 29 03:19:47 user nova-compute[75234]: tcp: [36] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9d734633-75e6-4a97-a3c5-bd9e45d70cca (non-flash)
Nov 29 03:19:47 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsi_session /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Nov 29 03:19:47 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-7b720460-3d17-493e-a134-1ef47ad6fba2 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7f8a2c7103a0>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:cf6ab229-533a-424e-bd67-1d54d50ad4be'): ({'sde'}, set())}) {{(pid=75234) _get_connection_devices /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:863}}
Nov 29 03:19:47 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-7b720460-3d17-493e-a134-1ef47ad6fba2 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Removing single pathed devices sde {{(pid=75234) remove_connection /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:374}}
Nov 29 03:19:47 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:19:47 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.019s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:19:47 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[2e38c9ef-0084-4aba-9407-e8c6436e2873]: (4, ('path checker states:\nup                  4\n\npaths: 0\nbusy: False\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:19:47 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sde {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:19:47 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sde" returned: 0 in 0.020s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:19:47 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[3cc0b1cd-dd9a-4c09-8f26-10d0cd9e0354]: (4, ('ok\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:19:47 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-7b720460-3d17-493e-a134-1ef47ad6fba2 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Flushing IO for device /dev/sde {{(pid=75234) flush_device_io /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:434}}
Nov 29 03:19:47 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sde {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:19:47 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sde" returned: 0 in 0.013s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:19:47 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[f92565be-90e0-41ab-9541-3f1a03fbc1bd]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:19:47 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-7b720460-3d17-493e-a134-1ef47ad6fba2 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Remove SCSI device /dev/sde with /sys/block/sde/device/delete {{(pid=75234) remove_scsi_device /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:148}}
Nov 29 03:19:47 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sde/device/delete {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:19:47 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sde/device/delete" returned: 0 in 0.044s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:19:47 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[fea2abd2-0a28-4dab-b39d-603ffc13c38d]: (4, ('1', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:19:47 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-7b720460-3d17-493e-a134-1ef47ad6fba2 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Checking to see if SCSI volumes sde have been removed. {{(pid=75234) wait_for_volumes_removal /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:156}}
Nov 29 03:19:47 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-7b720460-3d17-493e-a134-1ef47ad6fba2 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] SCSI volumes sde have been removed. {{(pid=75234) wait_for_volumes_removal /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:166}}
Nov 29 03:19:47 user nova-compute[75234]: DEBUG os_brick.privileged.rootwrap [-] Unlinking ('/dev/disk/by-id/scsi-SSCST_BIO_Storage_71b33293', '/dev/disk/by-id/scsi-23365386266373437', '/dev/disk/by-id/scsi-1SCST_BIO71b33293-3e8bf747541ac246') {{(pid=86324) unlink_root /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/privileged/rootwrap.py:219}}
Nov 29 03:19:47 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[0a7134cf-4212-4550-88b9-125253d3ae6d]: (4, None) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:19:47 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-7b720460-3d17-493e-a134-1ef47ad6fba2 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:cf6ab229-533a-424e-bd67-1d54d50ad4be')] {{(pid=75234) _disconnect_connection /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Nov 29 03:19:47 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:cf6ab229-533a-424e-bd67-1d54d50ad4be -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:19:47 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:cf6ab229-533a-424e-bd67-1d54d50ad4be -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.020s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:19:47 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[80bbae55-788b-4102-9257-1c6eb5590ac9]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:19:47 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-7b720460-3d17-493e-a134-1ef47ad6fba2 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:19:47 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:cf6ab229-533a-424e-bd67-1d54d50ad4be -p 172.16.0.220:3260 --logout {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:19:47 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:cf6ab229-533a-424e-bd67-1d54d50ad4be -p 172.16.0.220:3260 --logout" returned: 0 in 0.048s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:19:47 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[ab64afc3-bc60-4116-87b5-709084327174]: (4, ('Logging out of session [sid: 32, target: iqn.2010-10.org.openstack:cf6ab229-533a-424e-bd67-1d54d50ad4be, portal: 172.16.0.220,3260]\nLogout of [sid: 32, target: iqn.2010-10.org.openstack:cf6ab229-533a-424e-bd67-1d54d50ad4be, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:19:47 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-7b720460-3d17-493e-a134-1ef47ad6fba2 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 32, target: iqn.2010-10.org.openstack:cf6ab229-533a-424e-bd67-1d54d50ad4be, portal: 172.16.0.220,3260]
Nov 29 03:19:47 user nova-compute[75234]: Logout of [sid: 32, target: iqn.2010-10.org.openstack:cf6ab229-533a-424e-bd67-1d54d50ad4be, portal: 172.16.0.220,3260] successful.
Nov 29 03:19:47 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:19:47 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:cf6ab229-533a-424e-bd67-1d54d50ad4be -p 172.16.0.220:3260 --op delete {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:19:47 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:cf6ab229-533a-424e-bd67-1d54d50ad4be -p 172.16.0.220:3260 --op delete" returned: 0 in 0.028s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:19:47 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[bb7d6721-8938-4f59-9514-95e566fe0201]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:19:47 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-7b720460-3d17-493e-a134-1ef47ad6fba2 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:19:47 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-7b720460-3d17-493e-a134-1ef47ad6fba2 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.323s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:87}}
Nov 29 03:19:47 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-7b720460-3d17-493e-a134-1ef47ad6fba2 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] <== disconnect_volume: return (324ms) None {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:203}}
Nov 29 03:19:47 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-7b720460-3d17-493e-a134-1ef47ad6fba2 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] [instance: af699bc2-9daa-4b5a-9b4a-ad92fe857d70] Disconnected iSCSI Volume {{(pid=75234) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:79}}
Nov 29 03:19:47 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-7b720460-3d17-493e-a134-1ef47ad6fba2 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] skipping disk /dev/sde (vda) as it is a volume {{(pid=75234) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11213}}
Nov 29 03:19:48 user nova-compute[75234]: DEBUG nova.objects.instance [None req-3fad31e7-f6ab-4b09-aae9-831ef75d71ed tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Lazy-loading 'flavor' on Instance uuid aaf1c3e3-b73e-42ad-87bd-c5255ec09b46 {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:19:48 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-3fad31e7-f6ab-4b09-aae9-831ef75d71ed tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Lock "aaf1c3e3-b73e-42ad-87bd-c5255ec09b46" "released" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: held 22.190s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:19:48 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-2d09198a-1bdd-420e-8d57-3ce49058592a tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Acquiring lock "aaf1c3e3-b73e-42ad-87bd-c5255ec09b46" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:19:48 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-2d09198a-1bdd-420e-8d57-3ce49058592a tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Lock "aaf1c3e3-b73e-42ad-87bd-c5255ec09b46" acquired by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:19:49 user nova-compute[75234]: INFO nova.compute.manager [None req-2d09198a-1bdd-420e-8d57-3ce49058592a tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] [instance: aaf1c3e3-b73e-42ad-87bd-c5255ec09b46] Detaching volume 9d734633-75e6-4a97-a3c5-bd9e45d70cca
Nov 29 03:19:49 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:19:49 user nova-compute[75234]: INFO nova.virt.block_device [None req-2d09198a-1bdd-420e-8d57-3ce49058592a tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] [instance: aaf1c3e3-b73e-42ad-87bd-c5255ec09b46] Attempting to driver detach volume 9d734633-75e6-4a97-a3c5-bd9e45d70cca from mountpoint /dev/vdc
Nov 29 03:19:49 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-2d09198a-1bdd-420e-8d57-3ce49058592a tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Attempting to detach device vdc from instance aaf1c3e3-b73e-42ad-87bd-c5255ec09b46 from the persistent domain config. {{(pid=75234) _detach_from_persistent /opt/stack/nova/nova/virt/libvirt/driver.py:2487}}
Nov 29 03:19:49 user nova-compute[75234]: DEBUG nova.virt.libvirt.guest [None req-2d09198a-1bdd-420e-8d57-3ce49058592a tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] detach device xml: <disk type="block" device="disk">
Nov 29 03:19:49 user nova-compute[75234]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Nov 29 03:19:49 user nova-compute[75234]:   <source dev="/dev/sdg"/>
Nov 29 03:19:49 user nova-compute[75234]:   <target dev="vdc" bus="virtio"/>
Nov 29 03:19:49 user nova-compute[75234]:   <serial>9d734633-75e6-4a97-a3c5-bd9e45d70cca</serial>
Nov 29 03:19:49 user nova-compute[75234]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x08" function="0x0"/>
Nov 29 03:19:49 user nova-compute[75234]: </disk>
Nov 29 03:19:49 user nova-compute[75234]:  {{(pid=75234) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Nov 29 03:19:49 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-2d09198a-1bdd-420e-8d57-3ce49058592a tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Successfully detached device vdc from instance aaf1c3e3-b73e-42ad-87bd-c5255ec09b46 from the persistent domain config.
Nov 29 03:19:49 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-2d09198a-1bdd-420e-8d57-3ce49058592a tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] (1/8): Attempting to detach device vdc with device alias virtio-disk2 from instance aaf1c3e3-b73e-42ad-87bd-c5255ec09b46 from the live domain config. {{(pid=75234) _detach_from_live_with_retry /opt/stack/nova/nova/virt/libvirt/driver.py:2523}}
Nov 29 03:19:49 user nova-compute[75234]: DEBUG nova.virt.libvirt.guest [None req-2d09198a-1bdd-420e-8d57-3ce49058592a tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] detach device xml: <disk type="block" device="disk">
Nov 29 03:19:49 user nova-compute[75234]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Nov 29 03:19:49 user nova-compute[75234]:   <source dev="/dev/sdg"/>
Nov 29 03:19:49 user nova-compute[75234]:   <target dev="vdc" bus="virtio"/>
Nov 29 03:19:49 user nova-compute[75234]:   <serial>9d734633-75e6-4a97-a3c5-bd9e45d70cca</serial>
Nov 29 03:19:49 user nova-compute[75234]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x08" function="0x0"/>
Nov 29 03:19:49 user nova-compute[75234]: </disk>
Nov 29 03:19:49 user nova-compute[75234]:  {{(pid=75234) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Nov 29 03:19:49 user nova-compute[75234]: DEBUG nova.network.neutron [None req-7b720460-3d17-493e-a134-1ef47ad6fba2 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] [instance: af699bc2-9daa-4b5a-9b4a-ad92fe857d70] Port 66dcb0f9-3180-42f7-851d-19b0c4f4119c binding to destination host user is already ACTIVE {{(pid=75234) migrate_instance_start /opt/stack/nova/nova/network/neutron.py:3170}}
Nov 29 03:19:49 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-7b720460-3d17-493e-a134-1ef47ad6fba2 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Acquiring lock "af699bc2-9daa-4b5a-9b4a-ad92fe857d70-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:19:49 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-7b720460-3d17-493e-a134-1ef47ad6fba2 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Lock "af699bc2-9daa-4b5a-9b4a-ad92fe857d70-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:19:49 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-7b720460-3d17-493e-a134-1ef47ad6fba2 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Lock "af699bc2-9daa-4b5a-9b4a-ad92fe857d70-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:19:49 user nova-compute[75234]: DEBUG nova.compute.manager [req-ffb2c367-8d3c-4f8b-9021-8ebbade6d299 req-0af0491e-804a-4263-abce-b224192d86c7 service nova] [instance: af699bc2-9daa-4b5a-9b4a-ad92fe857d70] Received event network-vif-plugged-66dcb0f9-3180-42f7-851d-19b0c4f4119c {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:19:49 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-ffb2c367-8d3c-4f8b-9021-8ebbade6d299 req-0af0491e-804a-4263-abce-b224192d86c7 service nova] Acquiring lock "af699bc2-9daa-4b5a-9b4a-ad92fe857d70-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:19:49 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-ffb2c367-8d3c-4f8b-9021-8ebbade6d299 req-0af0491e-804a-4263-abce-b224192d86c7 service nova] Lock "af699bc2-9daa-4b5a-9b4a-ad92fe857d70-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:19:49 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-ffb2c367-8d3c-4f8b-9021-8ebbade6d299 req-0af0491e-804a-4263-abce-b224192d86c7 service nova] Lock "af699bc2-9daa-4b5a-9b4a-ad92fe857d70-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:19:49 user nova-compute[75234]: DEBUG nova.compute.manager [req-ffb2c367-8d3c-4f8b-9021-8ebbade6d299 req-0af0491e-804a-4263-abce-b224192d86c7 service nova] [instance: af699bc2-9daa-4b5a-9b4a-ad92fe857d70] No waiting events found dispatching network-vif-plugged-66dcb0f9-3180-42f7-851d-19b0c4f4119c {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:19:49 user nova-compute[75234]: WARNING nova.compute.manager [req-ffb2c367-8d3c-4f8b-9021-8ebbade6d299 req-0af0491e-804a-4263-abce-b224192d86c7 service nova] [instance: af699bc2-9daa-4b5a-9b4a-ad92fe857d70] Received unexpected event network-vif-plugged-66dcb0f9-3180-42f7-851d-19b0c4f4119c for instance with vm_state active and task_state resize_migrated.
Nov 29 03:19:49 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-7b720460-3d17-493e-a134-1ef47ad6fba2 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Acquiring lock "refresh_cache-af699bc2-9daa-4b5a-9b4a-ad92fe857d70" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:19:49 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-7b720460-3d17-493e-a134-1ef47ad6fba2 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Acquired lock "refresh_cache-af699bc2-9daa-4b5a-9b4a-ad92fe857d70" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:19:49 user nova-compute[75234]: DEBUG nova.network.neutron [None req-7b720460-3d17-493e-a134-1ef47ad6fba2 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] [instance: af699bc2-9daa-4b5a-9b4a-ad92fe857d70] Building network info cache for instance {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2009}}
Nov 29 03:19:49 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-2d09198a-1bdd-420e-8d57-3ce49058592a tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Start waiting for the detach event from libvirt for device vdc with device alias virtio-disk2 for instance aaf1c3e3-b73e-42ad-87bd-c5255ec09b46 {{(pid=75234) _detach_from_live_and_wait_for_event /opt/stack/nova/nova/virt/libvirt/driver.py:2599}}
Nov 29 03:19:49 user nova-compute[75234]: DEBUG nova.network.neutron [None req-7b720460-3d17-493e-a134-1ef47ad6fba2 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] [instance: af699bc2-9daa-4b5a-9b4a-ad92fe857d70] Updating instance_info_cache with network_info: [{"id": "66dcb0f9-3180-42f7-851d-19b0c4f4119c", "address": "fa:16:3e:dd:9e:50", "network": {"id": "1f946e69-dbec-429c-83f8-a186bad4f2cc", "bridge": "br-int", "label": "tempest-ServerActionsTestOtherA-486369423-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.64", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "c1cb3d1f12f242a785a688d59dbe3cd6", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap66dcb0f9-31", "ovs_interfaceid": "66dcb0f9-3180-42f7-851d-19b0c4f4119c", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:19:49 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-7b720460-3d17-493e-a134-1ef47ad6fba2 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Releasing lock "refresh_cache-af699bc2-9daa-4b5a-9b4a-ad92fe857d70" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:19:49 user nova-compute[75234]: DEBUG os_brick.utils [None req-7b720460-3d17-493e-a134-1ef47ad6fba2 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:176}}
Nov 29 03:19:49 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:19:49 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.014s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:19:49 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[8418a73e-e762-46d7-babd-492b7f9618ee]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:dd1368c5ba2c\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:19:49 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt -v / -n -o SOURCE {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:19:49 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt -v / -n -o SOURCE" returned: 0 in 0.017s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:19:49 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[09a0e4e3-044d-47cd-82ec-87447630c0b9]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:19:49 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:19:49 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.015s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:19:49 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[48d7abe7-371a-4a27-a2ac-d85459a105f3]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:19:49 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[088c0a8e-e28b-4a26-a247-76847e666333]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:19:49 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-7b720460-3d17-493e-a134-1ef47ad6fba2 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Running cmd (subprocess): nvme version {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:19:49 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-7b720460-3d17-493e-a134-1ef47ad6fba2 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] 'nvme version' failed. Not Retrying. {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:473}}
Nov 29 03:19:49 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.nvmeof [None req-7b720460-3d17-493e-a134-1ef47ad6fba2 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] nvme not present on system {{(pid=75234) nvme_present /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/nvmeof.py:756}}
Nov 29 03:19:49 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:19:49 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:473}}
Nov 29 03:19:49 user nova-compute[75234]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Nov 29 03:19:49 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[01fc8364-898b-4155-ba1d-d667cd3d9a6e]: (4, '') {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:19:49 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.lightos [None req-7b720460-3d17-493e-a134-1ef47ad6fba2 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=75234) find_dsc /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/lightos.py:98}}
Nov 29 03:19:49 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.lightos [None req-7b720460-3d17-493e-a134-1ef47ad6fba2 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=75234) get_connector_properties /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/lightos.py:76}}
Nov 29 03:19:49 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.lightos [None req-7b720460-3d17-493e-a134-1ef47ad6fba2 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] LIGHTOS: no hostnqn found. {{(pid=75234) get_connector_properties /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/lightos.py:84}}
Nov 29 03:19:49 user nova-compute[75234]: DEBUG os_brick.utils [None req-7b720460-3d17-493e-a134-1ef47ad6fba2 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] <== get_connector_properties: return (121ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:dd1368c5ba2c', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:203}}
Nov 29 03:19:52 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:19:52 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:19:53 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ac7b122c-8440-4607-bea5-d9305f99725b tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] Acquiring lock "aa190a9f-ba53-43fb-8bae-2c429d0d1228" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:19:53 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ac7b122c-8440-4607-bea5-d9305f99725b tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] Lock "aa190a9f-ba53-43fb-8bae-2c429d0d1228" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:19:53 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ac7b122c-8440-4607-bea5-d9305f99725b tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] Acquiring lock "aa190a9f-ba53-43fb-8bae-2c429d0d1228-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:19:53 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ac7b122c-8440-4607-bea5-d9305f99725b tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] Lock "aa190a9f-ba53-43fb-8bae-2c429d0d1228-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:19:53 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ac7b122c-8440-4607-bea5-d9305f99725b tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] Lock "aa190a9f-ba53-43fb-8bae-2c429d0d1228-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:19:53 user nova-compute[75234]: INFO nova.compute.manager [None req-ac7b122c-8440-4607-bea5-d9305f99725b tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] [instance: aa190a9f-ba53-43fb-8bae-2c429d0d1228] Terminating instance
Nov 29 03:19:53 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ac7b122c-8440-4607-bea5-d9305f99725b tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] [instance: aa190a9f-ba53-43fb-8bae-2c429d0d1228] Start destroying the instance on the hypervisor. {{(pid=75234) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3140}}
Nov 29 03:19:53 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:19:53 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:19:53 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:19:53 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:19:53 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:19:54 user nova-compute[75234]: DEBUG nova.compute.manager [req-51e90b72-d015-43e9-9781-8f1965b48a94 req-b8fe606b-ca68-4402-8fb3-a7887b43ba16 service nova] [instance: aa190a9f-ba53-43fb-8bae-2c429d0d1228] Received event network-vif-unplugged-2cc37e6a-5c9f-4bb8-98fa-f4f2b94d0c35 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:19:54 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-51e90b72-d015-43e9-9781-8f1965b48a94 req-b8fe606b-ca68-4402-8fb3-a7887b43ba16 service nova] Acquiring lock "aa190a9f-ba53-43fb-8bae-2c429d0d1228-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:19:54 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-51e90b72-d015-43e9-9781-8f1965b48a94 req-b8fe606b-ca68-4402-8fb3-a7887b43ba16 service nova] Lock "aa190a9f-ba53-43fb-8bae-2c429d0d1228-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:19:54 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-51e90b72-d015-43e9-9781-8f1965b48a94 req-b8fe606b-ca68-4402-8fb3-a7887b43ba16 service nova] Lock "aa190a9f-ba53-43fb-8bae-2c429d0d1228-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:19:54 user nova-compute[75234]: DEBUG nova.compute.manager [req-51e90b72-d015-43e9-9781-8f1965b48a94 req-b8fe606b-ca68-4402-8fb3-a7887b43ba16 service nova] [instance: aa190a9f-ba53-43fb-8bae-2c429d0d1228] No waiting events found dispatching network-vif-unplugged-2cc37e6a-5c9f-4bb8-98fa-f4f2b94d0c35 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:19:54 user nova-compute[75234]: DEBUG nova.compute.manager [req-51e90b72-d015-43e9-9781-8f1965b48a94 req-b8fe606b-ca68-4402-8fb3-a7887b43ba16 service nova] [instance: aa190a9f-ba53-43fb-8bae-2c429d0d1228] Received event network-vif-unplugged-2cc37e6a-5c9f-4bb8-98fa-f4f2b94d0c35 for instance with task_state deleting. {{(pid=75234) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10890}}
Nov 29 03:19:54 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:19:54 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:19:54 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:19:54 user nova-compute[75234]: INFO nova.virt.libvirt.driver [-] [instance: aa190a9f-ba53-43fb-8bae-2c429d0d1228] Instance destroyed successfully.
Nov 29 03:19:54 user nova-compute[75234]: DEBUG nova.objects.instance [None req-ac7b122c-8440-4607-bea5-d9305f99725b tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] Lazy-loading 'resources' on Instance uuid aa190a9f-ba53-43fb-8bae-2c429d0d1228 {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:19:54 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-ac7b122c-8440-4607-bea5-d9305f99725b tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=True,compute_id=1,config_drive='',created_at=2023-11-29T03:16:08Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description=None,display_name='tempest-ServerActionsV293TestJSON-server-1837520572',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-serveractionsv293testjson-server-381391911',id=27,image_ref='',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBKnl09Ywqr6/srgll0CbbEeIXPcY1dWQkS0gn6SNfJJJd/3IKDSo4FDOq1mKkootcIJYqYPVMtXdp0HTuNblTn7mhW9LnhxK8ZA0M4lAvd/3jzC0f6c+1EiWTGrTU8NBjw==',key_name='tempest-keypair-1391635144',keypairs=<?>,launch_index=0,launched_at=2023-11-29T03:18:44Z,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={rebuild='server'},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='581a0b62fee744e1b9a767a0d011c4e0',ramdisk_id='',reservation_id='r-eq00dfx0',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',clean_attempts='1',image_base_image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-ServerActionsV293TestJSON-155733519',owner_user_name='tempest-ServerActionsV293TestJSON-155733519-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-11-29T03:18:45Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='d11ce50c391d4e3bbe92d6c463d14746',uuid=aa190a9f-ba53-43fb-8bae-2c429d0d1228,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "2cc37e6a-5c9f-4bb8-98fa-f4f2b94d0c35", "address": "fa:16:3e:fb:40:18", "network": {"id": "38517dcf-64c7-4d26-9083-5ea5f6c87c2f", "bridge": "br-int", "label": "tempest-ServerActionsV293TestJSON-266920726-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.208", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "581a0b62fee744e1b9a767a0d011c4e0", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2cc37e6a-5c", "ovs_interfaceid": "2cc37e6a-5c9f-4bb8-98fa-f4f2b94d0c35", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Nov 29 03:19:54 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-ac7b122c-8440-4607-bea5-d9305f99725b tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] Converting VIF {"id": "2cc37e6a-5c9f-4bb8-98fa-f4f2b94d0c35", "address": "fa:16:3e:fb:40:18", "network": {"id": "38517dcf-64c7-4d26-9083-5ea5f6c87c2f", "bridge": "br-int", "label": "tempest-ServerActionsV293TestJSON-266920726-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.208", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "581a0b62fee744e1b9a767a0d011c4e0", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2cc37e6a-5c", "ovs_interfaceid": "2cc37e6a-5c9f-4bb8-98fa-f4f2b94d0c35", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:19:54 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-ac7b122c-8440-4607-bea5-d9305f99725b tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:fb:40:18,bridge_name='br-int',has_traffic_filtering=True,id=2cc37e6a-5c9f-4bb8-98fa-f4f2b94d0c35,network=Network(38517dcf-64c7-4d26-9083-5ea5f6c87c2f),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap2cc37e6a-5c') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:19:54 user nova-compute[75234]: DEBUG os_vif [None req-ac7b122c-8440-4607-bea5-d9305f99725b tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] Unplugging vif VIFOpenVSwitch(active=True,address=fa:16:3e:fb:40:18,bridge_name='br-int',has_traffic_filtering=True,id=2cc37e6a-5c9f-4bb8-98fa-f4f2b94d0c35,network=Network(38517dcf-64c7-4d26-9083-5ea5f6c87c2f),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap2cc37e6a-5c') {{(pid=75234) unplug /opt/stack/data/venv/lib/python3.10/site-packages/os_vif/__init__.py:109}}
Nov 29 03:19:54 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:19:54 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tap2cc37e6a-5c, bridge=br-int, if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:19:54 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:19:54 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:19:54 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:19:54 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbDestroyCommand(_result=None, table=QoS, record=c6b334b0-0f89-425a-a9b9-59b6dfe53338) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:19:54 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:19:54 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:19:54 user nova-compute[75234]: INFO os_vif [None req-ac7b122c-8440-4607-bea5-d9305f99725b tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] Successfully unplugged vif VIFOpenVSwitch(active=True,address=fa:16:3e:fb:40:18,bridge_name='br-int',has_traffic_filtering=True,id=2cc37e6a-5c9f-4bb8-98fa-f4f2b94d0c35,network=Network(38517dcf-64c7-4d26-9083-5ea5f6c87c2f),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap2cc37e6a-5c')
Nov 29 03:19:54 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-ac7b122c-8440-4607-bea5-d9305f99725b tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] [instance: aa190a9f-ba53-43fb-8bae-2c429d0d1228] calling os-brick to detach iSCSI Volume {{(pid=75234) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Nov 29 03:19:54 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-ac7b122c-8440-4607-bea5-d9305f99725b tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f8a5050c9a0>, {'target_iqn': 'iqn.2010-10.org.openstack:07f83efc-272d-4e10-9a2b-48d967db613c', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'kRLjYCVO', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdc'}, None), 'kwargs': True}}" {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:176}}
Nov 29 03:19:54 user nova-compute[75234]: WARNING os_brick.initiator.connectors.base [None req-ac7b122c-8440-4607-bea5-d9305f99725b tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Nov 29 03:19:54 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-ac7b122c-8440-4607-bea5-d9305f99725b tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:68}}
Nov 29 03:19:54 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-ac7b122c-8440-4607-bea5-d9305f99725b tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:73}}
Nov 29 03:19:54 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:19:54 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.021s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:19:54 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[508dceb6-8257-440c-a82b-ce952dd6b67e]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:9d734633-75e6-4a97-a3c5-bd9e45d70cca\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:07f83efc-272d-4e10-9a2b-48d967db613c\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:bf57b967-7fad-4d76-abb2-319fe88f8ef9\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:19:54 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-ac7b122c-8440-4607-bea5-d9305f99725b tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS:
Nov 29 03:19:54 user nova-compute[75234]: No targets found.
Nov 29 03:19:54 user nova-compute[75234]: iSNS:
Nov 29 03:19:54 user nova-compute[75234]: No targets found.
Nov 29 03:19:54 user nova-compute[75234]: STATIC:
Nov 29 03:19:54 user nova-compute[75234]: Target: iqn.2010-10.org.openstack:9d734633-75e6-4a97-a3c5-bd9e45d70cca
Nov 29 03:19:54 user nova-compute[75234]:         Portal: 172.16.0.220:3260,-1
Nov 29 03:19:54 user nova-compute[75234]:                 Iface Name: default
Nov 29 03:19:54 user nova-compute[75234]: Target: iqn.2010-10.org.openstack:07f83efc-272d-4e10-9a2b-48d967db613c
Nov 29 03:19:54 user nova-compute[75234]:         Portal: 172.16.0.220:3260,-1
Nov 29 03:19:54 user nova-compute[75234]:                 Iface Name: default
Nov 29 03:19:54 user nova-compute[75234]: Target: iqn.2010-10.org.openstack:bf57b967-7fad-4d76-abb2-319fe88f8ef9
Nov 29 03:19:54 user nova-compute[75234]:         Portal: 172.16.0.220:3260,-1
Nov 29 03:19:54 user nova-compute[75234]:                 Iface Name: default
Nov 29 03:19:54 user nova-compute[75234]: FIRMWARE:
Nov 29 03:19:54 user nova-compute[75234]: No targets found.
Nov 29 03:19:54 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:19:54 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-ac7b122c-8440-4607-bea5-d9305f99725b tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] Regex to get portals from discoverydb: ^SENDTARGETS:
Nov 29 03:19:54 user nova-compute[75234]: .*?^DiscoveryAddress: 172.16.0.220,3260.*?
Nov 29 03:19:54 user nova-compute[75234]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=75234) _get_discoverydb_portals /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:382}}
Nov 29 03:19:54 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-ac7b122c-8440-4607-bea5-d9305f99725b tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:07f83efc-272d-4e10-9a2b-48d967db613c', 0)] {{(pid=75234) _get_connection_devices /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:830}}
Nov 29 03:19:54 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:19:54 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.019s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:19:54 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[6e84b73b-a7ad-427c-b85a-ab7a1394b0b0]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:07f83efc-272d-4e10-9a2b-48d967db613c\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:9d734633-75e6-4a97-a3c5-bd9e45d70cca\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:bf57b967-7fad-4d76-abb2-319fe88f8ef9\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:19:54 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:19:54 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.023s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:19:54 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[957d5749-a4f2-49c4-9b25-54bfade8466c]: (4, ('tcp: [29] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bf57b967-7fad-4d76-abb2-319fe88f8ef9 (non-flash)\ntcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:07f83efc-272d-4e10-9a2b-48d967db613c (non-flash)\ntcp: [36] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9d734633-75e6-4a97-a3c5-bd9e45d70cca (non-flash)\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:19:54 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-ac7b122c-8440-4607-bea5-d9305f99725b tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [29] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bf57b967-7fad-4d76-abb2-319fe88f8ef9 (non-flash)
Nov 29 03:19:54 user nova-compute[75234]: tcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:07f83efc-272d-4e10-9a2b-48d967db613c (non-flash)
Nov 29 03:19:54 user nova-compute[75234]: tcp: [36] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9d734633-75e6-4a97-a3c5-bd9e45d70cca (non-flash)
Nov 29 03:19:54 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:19:54 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-ac7b122c-8440-4607-bea5-d9305f99725b tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] iscsi session list stdout=tcp: [29] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bf57b967-7fad-4d76-abb2-319fe88f8ef9 (non-flash)
Nov 29 03:19:54 user nova-compute[75234]: tcp: [33] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:07f83efc-272d-4e10-9a2b-48d967db613c (non-flash)
Nov 29 03:19:54 user nova-compute[75234]: tcp: [36] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9d734633-75e6-4a97-a3c5-bd9e45d70cca (non-flash)
Nov 29 03:19:54 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsi_session /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Nov 29 03:19:54 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-ac7b122c-8440-4607-bea5-d9305f99725b tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7f8a682fdf30>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:07f83efc-272d-4e10-9a2b-48d967db613c'): ({'sdc'}, set())}) {{(pid=75234) _get_connection_devices /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:863}}
Nov 29 03:19:54 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-ac7b122c-8440-4607-bea5-d9305f99725b tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] Removing single pathed devices sdc {{(pid=75234) remove_connection /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:374}}
Nov 29 03:19:54 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:19:54 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.018s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:19:54 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[ad006157-66c3-44e8-81b8-31d76689466d]: (4, ('path checker states:\nup                  3\n\npaths: 0\nbusy: False\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:19:54 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdc {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:19:54 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdc" returned: 0 in 0.020s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:19:54 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[9a78dbe4-4d8f-4f2e-a67f-0a579836c5fe]: (4, ('ok\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:19:54 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-ac7b122c-8440-4607-bea5-d9305f99725b tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] Flushing IO for device /dev/sdc {{(pid=75234) flush_device_io /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:434}}
Nov 29 03:19:54 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdc {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:19:54 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdc" returned: 0 in 0.012s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:19:54 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[0759cf51-2a0d-4a75-87a7-e40b458684ec]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:19:54 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-ac7b122c-8440-4607-bea5-d9305f99725b tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] Remove SCSI device /dev/sdc with /sys/block/sdc/device/delete {{(pid=75234) remove_scsi_device /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:148}}
Nov 29 03:19:54 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdc/device/delete {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:19:54 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdc/device/delete" returned: 0 in 0.040s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:19:54 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[8da64dce-1548-49cb-b528-5a80d69a5e80]: (4, ('1', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:19:54 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-ac7b122c-8440-4607-bea5-d9305f99725b tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] Checking to see if SCSI volumes sdc have been removed. {{(pid=75234) wait_for_volumes_removal /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:156}}
Nov 29 03:19:54 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-ac7b122c-8440-4607-bea5-d9305f99725b tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] SCSI volumes sdc have been removed. {{(pid=75234) wait_for_volumes_removal /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:166}}
Nov 29 03:19:54 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-ac7b122c-8440-4607-bea5-d9305f99725b tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:07f83efc-272d-4e10-9a2b-48d967db613c')] {{(pid=75234) _disconnect_connection /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Nov 29 03:19:54 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:07f83efc-272d-4e10-9a2b-48d967db613c -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:19:54 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:07f83efc-272d-4e10-9a2b-48d967db613c -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.018s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:19:54 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[8792d9c2-833b-4664-b1db-695ddc6cb44f]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:19:54 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-ac7b122c-8440-4607-bea5-d9305f99725b tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:19:54 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:07f83efc-272d-4e10-9a2b-48d967db613c -p 172.16.0.220:3260 --logout {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:19:54 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:07f83efc-272d-4e10-9a2b-48d967db613c -p 172.16.0.220:3260 --logout" returned: 0 in 0.066s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:19:54 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[1e788533-3554-4fb3-9348-67fd9f091f38]: (4, ('Logging out of session [sid: 33, target: iqn.2010-10.org.openstack:07f83efc-272d-4e10-9a2b-48d967db613c, portal: 172.16.0.220,3260]\nLogout of [sid: 33, target: iqn.2010-10.org.openstack:07f83efc-272d-4e10-9a2b-48d967db613c, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:19:54 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-ac7b122c-8440-4607-bea5-d9305f99725b tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 33, target: iqn.2010-10.org.openstack:07f83efc-272d-4e10-9a2b-48d967db613c, portal: 172.16.0.220,3260]
Nov 29 03:19:54 user nova-compute[75234]: Logout of [sid: 33, target: iqn.2010-10.org.openstack:07f83efc-272d-4e10-9a2b-48d967db613c, portal: 172.16.0.220,3260] successful.
Nov 29 03:19:54 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:19:54 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:07f83efc-272d-4e10-9a2b-48d967db613c -p 172.16.0.220:3260 --op delete {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:19:54 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:07f83efc-272d-4e10-9a2b-48d967db613c -p 172.16.0.220:3260 --op delete" returned: 0 in 0.031s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:19:54 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[08d554de-411d-49b7-b73f-def55569e86d]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:19:54 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-ac7b122c-8440-4607-bea5-d9305f99725b tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:19:54 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-ac7b122c-8440-4607-bea5-d9305f99725b tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.319s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:87}}
Nov 29 03:19:54 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-ac7b122c-8440-4607-bea5-d9305f99725b tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] <== disconnect_volume: return (321ms) None {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:203}}
Nov 29 03:19:54 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-ac7b122c-8440-4607-bea5-d9305f99725b tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] [instance: aa190a9f-ba53-43fb-8bae-2c429d0d1228] Disconnected iSCSI Volume {{(pid=75234) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:79}}
Nov 29 03:19:54 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-ac7b122c-8440-4607-bea5-d9305f99725b tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] [instance: aa190a9f-ba53-43fb-8bae-2c429d0d1228] Deleting instance files /opt/stack/data/nova/instances/aa190a9f-ba53-43fb-8bae-2c429d0d1228_del
Nov 29 03:19:54 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-ac7b122c-8440-4607-bea5-d9305f99725b tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] [instance: aa190a9f-ba53-43fb-8bae-2c429d0d1228] Deletion of /opt/stack/data/nova/instances/aa190a9f-ba53-43fb-8bae-2c429d0d1228_del complete
Nov 29 03:19:54 user nova-compute[75234]: INFO nova.compute.manager [None req-ac7b122c-8440-4607-bea5-d9305f99725b tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] [instance: aa190a9f-ba53-43fb-8bae-2c429d0d1228] Took 1.18 seconds to destroy the instance on the hypervisor.
Nov 29 03:19:54 user nova-compute[75234]: DEBUG oslo.service.loopingcall [None req-ac7b122c-8440-4607-bea5-d9305f99725b tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=75234) func /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/loopingcall.py:435}}
Nov 29 03:19:54 user nova-compute[75234]: DEBUG nova.compute.manager [-] [instance: aa190a9f-ba53-43fb-8bae-2c429d0d1228] Deallocating network for instance {{(pid=75234) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2279}}
Nov 29 03:19:54 user nova-compute[75234]: DEBUG nova.network.neutron [-] [instance: aa190a9f-ba53-43fb-8bae-2c429d0d1228] deallocate_for_instance() {{(pid=75234) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1802}}
Nov 29 03:19:55 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:19:55 user nova-compute[75234]: DEBUG nova.network.neutron [-] [instance: aa190a9f-ba53-43fb-8bae-2c429d0d1228] Updating instance_info_cache with network_info: [] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:19:55 user nova-compute[75234]: INFO nova.compute.manager [-] [instance: aa190a9f-ba53-43fb-8bae-2c429d0d1228] Took 0.81 seconds to deallocate network for instance.
Nov 29 03:19:56 user nova-compute[75234]: DEBUG nova.compute.manager [req-131d5d74-57e3-4c74-b58f-25a6130285cc req-0656da20-3620-4766-ad9b-3223c8e56a75 service nova] [instance: aa190a9f-ba53-43fb-8bae-2c429d0d1228] Received event network-vif-plugged-2cc37e6a-5c9f-4bb8-98fa-f4f2b94d0c35 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:19:56 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-131d5d74-57e3-4c74-b58f-25a6130285cc req-0656da20-3620-4766-ad9b-3223c8e56a75 service nova] Acquiring lock "aa190a9f-ba53-43fb-8bae-2c429d0d1228-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:19:56 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-131d5d74-57e3-4c74-b58f-25a6130285cc req-0656da20-3620-4766-ad9b-3223c8e56a75 service nova] Lock "aa190a9f-ba53-43fb-8bae-2c429d0d1228-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:19:56 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-131d5d74-57e3-4c74-b58f-25a6130285cc req-0656da20-3620-4766-ad9b-3223c8e56a75 service nova] Lock "aa190a9f-ba53-43fb-8bae-2c429d0d1228-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:19:56 user nova-compute[75234]: DEBUG nova.compute.manager [req-131d5d74-57e3-4c74-b58f-25a6130285cc req-0656da20-3620-4766-ad9b-3223c8e56a75 service nova] [instance: aa190a9f-ba53-43fb-8bae-2c429d0d1228] No waiting events found dispatching network-vif-plugged-2cc37e6a-5c9f-4bb8-98fa-f4f2b94d0c35 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:19:56 user nova-compute[75234]: WARNING nova.compute.manager [req-131d5d74-57e3-4c74-b58f-25a6130285cc req-0656da20-3620-4766-ad9b-3223c8e56a75 service nova] [instance: aa190a9f-ba53-43fb-8bae-2c429d0d1228] Received unexpected event network-vif-plugged-2cc37e6a-5c9f-4bb8-98fa-f4f2b94d0c35 for instance with vm_state active and task_state deleting.
Nov 29 03:19:56 user nova-compute[75234]: DEBUG nova.compute.manager [req-131d5d74-57e3-4c74-b58f-25a6130285cc req-0656da20-3620-4766-ad9b-3223c8e56a75 service nova] [instance: aa190a9f-ba53-43fb-8bae-2c429d0d1228] Received event network-vif-deleted-2cc37e6a-5c9f-4bb8-98fa-f4f2b94d0c35 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:19:57 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:19:57 user nova-compute[75234]: DEBUG nova.compute.manager [req-2f89a13f-3986-43f4-a879-7776c02e0244 req-f1936122-f7d6-4901-9c2b-cc480b915cad service nova] [instance: 0aff446c-f417-477e-a1c5-b51b1b6214e4] Received event network-changed-f9148192-6e14-44d4-a31b-081ea47aa289 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:19:57 user nova-compute[75234]: DEBUG nova.compute.manager [req-2f89a13f-3986-43f4-a879-7776c02e0244 req-f1936122-f7d6-4901-9c2b-cc480b915cad service nova] [instance: 0aff446c-f417-477e-a1c5-b51b1b6214e4] Refreshing instance network info cache due to event network-changed-f9148192-6e14-44d4-a31b-081ea47aa289. {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11117}}
Nov 29 03:19:57 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-2f89a13f-3986-43f4-a879-7776c02e0244 req-f1936122-f7d6-4901-9c2b-cc480b915cad service nova] Acquiring lock "refresh_cache-0aff446c-f417-477e-a1c5-b51b1b6214e4" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:19:57 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-2f89a13f-3986-43f4-a879-7776c02e0244 req-f1936122-f7d6-4901-9c2b-cc480b915cad service nova] Acquired lock "refresh_cache-0aff446c-f417-477e-a1c5-b51b1b6214e4" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:19:57 user nova-compute[75234]: DEBUG nova.network.neutron [req-2f89a13f-3986-43f4-a879-7776c02e0244 req-f1936122-f7d6-4901-9c2b-cc480b915cad service nova] [instance: 0aff446c-f417-477e-a1c5-b51b1b6214e4] Refreshing network info cache for port f9148192-6e14-44d4-a31b-081ea47aa289 {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2006}}
Nov 29 03:19:57 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:19:57 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-1fcf07e3-484b-422a-bc7c-c42ac90da2fb tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Acquiring lock "0aff446c-f417-477e-a1c5-b51b1b6214e4" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:19:57 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-1fcf07e3-484b-422a-bc7c-c42ac90da2fb tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Lock "0aff446c-f417-477e-a1c5-b51b1b6214e4" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:19:57 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-1fcf07e3-484b-422a-bc7c-c42ac90da2fb tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Acquiring lock "0aff446c-f417-477e-a1c5-b51b1b6214e4-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:19:57 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-1fcf07e3-484b-422a-bc7c-c42ac90da2fb tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Lock "0aff446c-f417-477e-a1c5-b51b1b6214e4-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:19:57 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-1fcf07e3-484b-422a-bc7c-c42ac90da2fb tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Lock "0aff446c-f417-477e-a1c5-b51b1b6214e4-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:19:57 user nova-compute[75234]: INFO nova.compute.manager [None req-1fcf07e3-484b-422a-bc7c-c42ac90da2fb tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] [instance: 0aff446c-f417-477e-a1c5-b51b1b6214e4] Terminating instance
Nov 29 03:19:57 user nova-compute[75234]: DEBUG nova.compute.manager [None req-1fcf07e3-484b-422a-bc7c-c42ac90da2fb tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] [instance: 0aff446c-f417-477e-a1c5-b51b1b6214e4] Start destroying the instance on the hypervisor. {{(pid=75234) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3140}}
Nov 29 03:19:57 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-7b720460-3d17-493e-a134-1ef47ad6fba2 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] [instance: af699bc2-9daa-4b5a-9b4a-ad92fe857d70] Starting finish_migration {{(pid=75234) finish_migration /opt/stack/nova/nova/virt/libvirt/driver.py:11671}}
Nov 29 03:19:57 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-7b720460-3d17-493e-a134-1ef47ad6fba2 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] [instance: af699bc2-9daa-4b5a-9b4a-ad92fe857d70] Instance directory exists: not creating {{(pid=75234) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4723}}
Nov 29 03:19:57 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-7b720460-3d17-493e-a134-1ef47ad6fba2 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] [instance: af699bc2-9daa-4b5a-9b4a-ad92fe857d70] Creating image(s)
Nov 29 03:19:57 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-7b720460-3d17-493e-a134-1ef47ad6fba2 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] [instance: af699bc2-9daa-4b5a-9b4a-ad92fe857d70] Did not create local disks {{(pid=75234) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4863}}
Nov 29 03:19:57 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-7b720460-3d17-493e-a134-1ef47ad6fba2 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] [instance: af699bc2-9daa-4b5a-9b4a-ad92fe857d70] Ensure instance console log exists: /opt/stack/data/nova/instances/af699bc2-9daa-4b5a-9b4a-ad92fe857d70/console.log {{(pid=75234) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4613}}
Nov 29 03:19:57 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-7b720460-3d17-493e-a134-1ef47ad6fba2 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:19:57 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-7b720460-3d17-493e-a134-1ef47ad6fba2 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:19:57 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-7b720460-3d17-493e-a134-1ef47ad6fba2 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:19:57 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-7b720460-3d17-493e-a134-1ef47ad6fba2 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] [instance: af699bc2-9daa-4b5a-9b4a-ad92fe857d70] Start _get_guest_xml network_info=[{"id": "66dcb0f9-3180-42f7-851d-19b0c4f4119c", "address": "fa:16:3e:dd:9e:50", "network": {"id": "1f946e69-dbec-429c-83f8-a186bad4f2cc", "bridge": "br-int", "label": "tempest-ServerActionsTestOtherA-486369423-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.64", "type": "floating", "version": 4, "meta": {}}], "label": "tempest-ServerActionsTestOtherA-486369423-network", "vif_mac": "fa:16:3e:dd:9e:50"}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "c1cb3d1f12f242a785a688d59dbe3cd6", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap66dcb0f9-31", "ovs_interfaceid": "66dcb0f9-3180-42f7-851d-19b0c4f4119c", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, '/dev/vda': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum=<?>,container_format=<?>,created_at=<?>,direct_url=<?>,disk_format=<?>,id=<?>,min_disk=0,min_ram=0,name=<?>,owner=<?>,properties=ImageMetaProps,protected=<?>,size=1073741824,status='active',tags=<?>,updated_at=<?>,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [], 'ephemerals': [], 'block_device_mapping': [{'mount_device': '/dev/vda', 'attachment_id': '308dacef-92a3-44e0-aa01-4be453ac92ba', 'disk_bus': 'virtio', 'device_type': 'disk', 'boot_index': 0, 'guest_format': None, 'delete_on_termination': True, 'connection_info': {'driver_volume_type': 'iscsi', 'data': {'target_iqn': 'iqn.2010-10.org.openstack:cf6ab229-533a-424e-bd67-1d54d50ad4be', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'wodfcngk', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}, 'status': 'attaching', 'instance': 'af699bc2-9daa-4b5a-9b4a-ad92fe857d70', 'attached_at': '2023-11-29T03:19:58.000000', 'detached_at': '', 'volume_id': 'cf6ab229-533a-424e-bd67-1d54d50ad4be', 'serial': 'cf6ab229-533a-424e-bd67-1d54d50ad4be'}, 'volume_type': None} {{(pid=75234) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7562}}
Nov 29 03:19:57 user nova-compute[75234]: DEBUG nova.network.neutron [req-2f89a13f-3986-43f4-a879-7776c02e0244 req-f1936122-f7d6-4901-9c2b-cc480b915cad service nova] [instance: 0aff446c-f417-477e-a1c5-b51b1b6214e4] Updated VIF entry in instance network info cache for port f9148192-6e14-44d4-a31b-081ea47aa289. {{(pid=75234) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3481}}
Nov 29 03:19:57 user nova-compute[75234]: DEBUG nova.network.neutron [req-2f89a13f-3986-43f4-a879-7776c02e0244 req-f1936122-f7d6-4901-9c2b-cc480b915cad service nova] [instance: 0aff446c-f417-477e-a1c5-b51b1b6214e4] Updating instance_info_cache with network_info: [{"id": "f9148192-6e14-44d4-a31b-081ea47aa289", "address": "fa:16:3e:46:49:9c", "network": {"id": "0f289171-3d73-4054-95a0-2515b4f206a0", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-1787960284-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.218", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "315732b6e3ee44d584b8e510ef199a90", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapf9148192-6e", "ovs_interfaceid": "f9148192-6e14-44d4-a31b-081ea47aa289", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:19:57 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-7b720460-3d17-493e-a134-1ef47ad6fba2 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:19:57 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-7b720460-3d17-493e-a134-1ef47ad6fba2 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:19:57 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:19:57 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-2f89a13f-3986-43f4-a879-7776c02e0244 req-f1936122-f7d6-4901-9c2b-cc480b915cad service nova] Releasing lock "refresh_cache-0aff446c-f417-477e-a1c5-b51b1b6214e4" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:19:57 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-7b720460-3d17-493e-a134-1ef47ad6fba2 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Searching host: 'user' for CPU controller through CGroups V1... {{(pid=75234) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1652}}
Nov 29 03:19:57 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-7b720460-3d17-493e-a134-1ef47ad6fba2 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] CPU controller missing on host. {{(pid=75234) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1662}}
Nov 29 03:19:57 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-7b720460-3d17-493e-a134-1ef47ad6fba2 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Searching host: 'user' for CPU controller through CGroups V2... {{(pid=75234) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1671}}
Nov 29 03:19:57 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-7b720460-3d17-493e-a134-1ef47ad6fba2 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] CPU controller found on host. {{(pid=75234) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1678}}
Nov 29 03:19:57 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-7b720460-3d17-493e-a134-1ef47ad6fba2 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=75234) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5400}}
Nov 29 03:19:57 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-7b720460-3d17-493e-a134-1ef47ad6fba2 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-11-29T03:02:06Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='2',id=7,is_public=True,memory_mb=2048,name='m1.small',projects=<?>,root_gb=20,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum=<?>,container_format=<?>,created_at=<?>,direct_url=<?>,disk_format=<?>,id=<?>,min_disk=0,min_ram=0,name=<?>,owner=<?>,properties=ImageMetaProps,protected=<?>,size=1073741824,status='active',tags=<?>,updated_at=<?>,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:563}}
Nov 29 03:19:57 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-7b720460-3d17-493e-a134-1ef47ad6fba2 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Flavor limits 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:348}}
Nov 29 03:19:57 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-7b720460-3d17-493e-a134-1ef47ad6fba2 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Image limits 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:352}}
Nov 29 03:19:57 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-7b720460-3d17-493e-a134-1ef47ad6fba2 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Flavor pref 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:388}}
Nov 29 03:19:57 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-7b720460-3d17-493e-a134-1ef47ad6fba2 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Image pref 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:392}}
Nov 29 03:19:57 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-7b720460-3d17-493e-a134-1ef47ad6fba2 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:430}}
Nov 29 03:19:57 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-7b720460-3d17-493e-a134-1ef47ad6fba2 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:569}}
Nov 29 03:19:57 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-7b720460-3d17-493e-a134-1ef47ad6fba2 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=75234) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:471}}
Nov 29 03:19:57 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-7b720460-3d17-493e-a134-1ef47ad6fba2 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Got 1 possible topologies {{(pid=75234) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:501}}
Nov 29 03:19:57 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-7b720460-3d17-493e-a134-1ef47ad6fba2 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:575}}
Nov 29 03:19:57 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-7b720460-3d17-493e-a134-1ef47ad6fba2 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:577}}
Nov 29 03:19:57 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-7b720460-3d17-493e-a134-1ef47ad6fba2 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] [instance: af699bc2-9daa-4b5a-9b4a-ad92fe857d70] Calling os-brick to attach iSCSI Volume {{(pid=75234) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Nov 29 03:19:57 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-7b720460-3d17-493e-a134-1ef47ad6fba2 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f8a5050c9a0>, 'connection_properties': {'target_iqn': 'iqn.2010-10.org.openstack:cf6ab229-533a-424e-bd67-1d54d50ad4be', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'wodfcngk', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:176}}
Nov 29 03:19:57 user nova-compute[75234]: WARNING os_brick.initiator.connectors.base [None req-7b720460-3d17-493e-a134-1ef47ad6fba2 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Nov 29 03:19:57 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-7b720460-3d17-493e-a134-1ef47ad6fba2 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:68}}
Nov 29 03:19:57 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-7b720460-3d17-493e-a134-1ef47ad6fba2 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:73}}
Nov 29 03:19:57 user nova-compute[75234]: WARNING os_brick.initiator.connectors.base [None req-7b720460-3d17-493e-a134-1ef47ad6fba2 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Nov 29 03:19:57 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-7b720460-3d17-493e-a134-1ef47ad6fba2 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:cf6ab229-533a-424e-bd67-1d54d50ad4be" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:68}}
Nov 29 03:19:57 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-7b720460-3d17-493e-a134-1ef47ad6fba2 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:cf6ab229-533a-424e-bd67-1d54d50ad4be" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:73}}
Nov 29 03:19:57 user nova-compute[75234]: INFO os_brick.initiator.connectors.iscsi [None req-7b720460-3d17-493e-a134-1ef47ad6fba2 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Nov 29 03:19:57 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:19:57 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:cf6ab229-533a-424e-bd67-1d54d50ad4be -p 172.16.0.220:3260 {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:19:57 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:cf6ab229-533a-424e-bd67-1d54d50ad4be -p 172.16.0.220:3260" returned: 21 in 0.024s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:19:57 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[3b576842-bcd2-413a-957e-360d211cbe30]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:19:57 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-7b720460-3d17-493e-a134-1ef47ad6fba2 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Nov 29 03:19:57 user nova-compute[75234]:  {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:19:57 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:cf6ab229-533a-424e-bd67-1d54d50ad4be -p 172.16.0.220:3260 --interface default --op new {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:19:57 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:cf6ab229-533a-424e-bd67-1d54d50ad4be -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.022s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:19:57 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[0097d966-317d-41d6-ac6d-0ca517364974]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:cf6ab229-533a-424e-bd67-1d54d50ad4be] added\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:19:57 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-7b720460-3d17-493e-a134-1ef47ad6fba2 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:cf6ab229-533a-424e-bd67-1d54d50ad4be] added
Nov 29 03:19:57 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:19:57 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:cf6ab229-533a-424e-bd67-1d54d50ad4be -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:19:57 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:cf6ab229-533a-424e-bd67-1d54d50ad4be -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.017s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:19:57 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[99de04b1-97bf-49f5-8445-a3acfa804eb1]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:19:57 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-7b720460-3d17-493e-a134-1ef47ad6fba2 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:19:57 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:cf6ab229-533a-424e-bd67-1d54d50ad4be -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:19:57 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:cf6ab229-533a-424e-bd67-1d54d50ad4be -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.018s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:19:57 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[38c9e86e-351f-4e8d-8988-7148845cc287]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:19:57 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-7b720460-3d17-493e-a134-1ef47ad6fba2 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:19:57 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:cf6ab229-533a-424e-bd67-1d54d50ad4be -p 172.16.0.220:3260 --op update -n node.session.auth.username -v wodfcngk {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:19:57 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:cf6ab229-533a-424e-bd67-1d54d50ad4be -p 172.16.0.220:3260 --op update -n node.session.auth.username -v wodfcngk" returned: 0 in 0.023s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:19:57 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[e9c7b37d-0d80-46d7-9b7a-7188eb62c886]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:19:57 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-7b720460-3d17-493e-a134-1ef47ad6fba2 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'wodfcngk'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:19:57 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:cf6ab229-533a-424e-bd67-1d54d50ad4be -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:19:57 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:cf6ab229-533a-424e-bd67-1d54d50ad4be -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.021s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:19:57 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[abcf2c7a-b630-4172-9672-dd0b0197f352]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:19:57 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-7b720460-3d17-493e-a134-1ef47ad6fba2 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:19:57 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:19:57 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.029s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:19:57 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[c58b2a4d-7114-4c5d-8b1f-f07f8c852de4]: (4, ('tcp: [29] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bf57b967-7fad-4d76-abb2-319fe88f8ef9 (non-flash)\ntcp: [36] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9d734633-75e6-4a97-a3c5-bd9e45d70cca (non-flash)\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:19:57 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-7b720460-3d17-493e-a134-1ef47ad6fba2 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [29] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bf57b967-7fad-4d76-abb2-319fe88f8ef9 (non-flash)
Nov 29 03:19:57 user nova-compute[75234]: tcp: [36] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9d734633-75e6-4a97-a3c5-bd9e45d70cca (non-flash)
Nov 29 03:19:57 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:19:57 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-7b720460-3d17-493e-a134-1ef47ad6fba2 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] iscsi session list stdout=tcp: [29] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bf57b967-7fad-4d76-abb2-319fe88f8ef9 (non-flash)
Nov 29 03:19:57 user nova-compute[75234]: tcp: [36] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9d734633-75e6-4a97-a3c5-bd9e45d70cca (non-flash)
Nov 29 03:19:57 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsi_session /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Nov 29 03:19:57 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:cf6ab229-533a-424e-bd67-1d54d50ad4be -p 172.16.0.220:3260 --login {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:19:58 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:19:58 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:19:58 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:cf6ab229-533a-424e-bd67-1d54d50ad4be -p 172.16.0.220:3260 --login" returned: 0 in 0.065s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:19:58 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[cd7a6101-8c9c-4777-b19b-39686c91b7d6]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:cf6ab229-533a-424e-bd67-1d54d50ad4be, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:cf6ab229-533a-424e-bd67-1d54d50ad4be, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:19:58 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-7b720460-3d17-493e-a134-1ef47ad6fba2 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:cf6ab229-533a-424e-bd67-1d54d50ad4be, portal: 172.16.0.220,3260]
Nov 29 03:19:58 user nova-compute[75234]: Login to [iface: default, target: iqn.2010-10.org.openstack:cf6ab229-533a-424e-bd67-1d54d50ad4be, portal: 172.16.0.220,3260] successful.
Nov 29 03:19:58 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:19:58 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:cf6ab229-533a-424e-bd67-1d54d50ad4be -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:19:58 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:cf6ab229-533a-424e-bd67-1d54d50ad4be -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.031s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:19:58 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[5b27c165-5d5b-4b04-bfb1-dd45fd765112]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:19:58 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-7b720460-3d17-493e-a134-1ef47ad6fba2 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:19:58 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:19:58 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.023s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:19:58 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[b7b93141-e39b-4833-b93a-ae5dfd428769]: (4, ('tcp: [29] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bf57b967-7fad-4d76-abb2-319fe88f8ef9 (non-flash)\ntcp: [36] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9d734633-75e6-4a97-a3c5-bd9e45d70cca (non-flash)\ntcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:cf6ab229-533a-424e-bd67-1d54d50ad4be (non-flash)\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:19:58 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-7b720460-3d17-493e-a134-1ef47ad6fba2 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [29] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bf57b967-7fad-4d76-abb2-319fe88f8ef9 (non-flash)
Nov 29 03:19:58 user nova-compute[75234]: tcp: [36] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9d734633-75e6-4a97-a3c5-bd9e45d70cca (non-flash)
Nov 29 03:19:58 user nova-compute[75234]: tcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:cf6ab229-533a-424e-bd67-1d54d50ad4be (non-flash)
Nov 29 03:19:58 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:19:58 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-7b720460-3d17-493e-a134-1ef47ad6fba2 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] iscsi session list stdout=tcp: [29] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bf57b967-7fad-4d76-abb2-319fe88f8ef9 (non-flash)
Nov 29 03:19:58 user nova-compute[75234]: tcp: [36] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9d734633-75e6-4a97-a3c5-bd9e45d70cca (non-flash)
Nov 29 03:19:58 user nova-compute[75234]: tcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:cf6ab229-533a-424e-bd67-1d54d50ad4be (non-flash)
Nov 29 03:19:58 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsi_session /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Nov 29 03:19:58 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-7b720460-3d17-493e-a134-1ef47ad6fba2 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:cf6ab229-533a-424e-bd67-1d54d50ad4be" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.316s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:87}}
Nov 29 03:19:58 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-7b720460-3d17-493e-a134-1ef47ad6fba2 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Connected to 172.16.0.220:3260 {{(pid=75234) _connect_vol /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:644}}
Nov 29 03:19:58 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-7b720460-3d17-493e-a134-1ef47ad6fba2 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] HCTL ('33', '-', '-', 0) found on session 37 with lun 0 {{(pid=75234) get_hctl /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:770}}
Nov 29 03:19:58 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-7b720460-3d17-493e-a134-1ef47ad6fba2 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Scanning host 33 c: -, t: -, l: 0) {{(pid=75234) scan_iscsi /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:795}}
Nov 29 03:19:58 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host33/scan {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:19:58 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host33/scan" returned: 0 in 0.024s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:19:58 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[ba39ef85-a902-47e1-9f39-b9e228a1d270]: (4, ('- - 0', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:19:58 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-7b720460-3d17-493e-a134-1ef47ad6fba2 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Searching for a device in session 37 and hctl ['33', '*', '*', 0] yield: sdb {{(pid=75234) device_name_by_hctl /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:789}}
Nov 29 03:19:58 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-7b720460-3d17-493e-a134-1ef47ad6fba2 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Connected to sdb using {'target_iqn': 'iqn.2010-10.org.openstack:cf6ab229-533a-424e-bd67-1d54d50ad4be', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'wodfcngk', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False} {{(pid=75234) _connect_vol /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:672}}
Nov 29 03:19:58 user nova-compute[75234]: DEBUG nova.compute.manager [req-8f80e96e-45c3-42c9-871a-0e3f76db17e2 req-c2cb08a9-a054-40a3-b5f5-56418a6da91c service nova] [instance: 0aff446c-f417-477e-a1c5-b51b1b6214e4] Received event network-vif-unplugged-f9148192-6e14-44d4-a31b-081ea47aa289 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:19:58 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-8f80e96e-45c3-42c9-871a-0e3f76db17e2 req-c2cb08a9-a054-40a3-b5f5-56418a6da91c service nova] Acquiring lock "0aff446c-f417-477e-a1c5-b51b1b6214e4-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:19:58 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-8f80e96e-45c3-42c9-871a-0e3f76db17e2 req-c2cb08a9-a054-40a3-b5f5-56418a6da91c service nova] Lock "0aff446c-f417-477e-a1c5-b51b1b6214e4-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:19:58 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-8f80e96e-45c3-42c9-871a-0e3f76db17e2 req-c2cb08a9-a054-40a3-b5f5-56418a6da91c service nova] Lock "0aff446c-f417-477e-a1c5-b51b1b6214e4-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:19:58 user nova-compute[75234]: DEBUG nova.compute.manager [req-8f80e96e-45c3-42c9-871a-0e3f76db17e2 req-c2cb08a9-a054-40a3-b5f5-56418a6da91c service nova] [instance: 0aff446c-f417-477e-a1c5-b51b1b6214e4] No waiting events found dispatching network-vif-unplugged-f9148192-6e14-44d4-a31b-081ea47aa289 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:19:58 user nova-compute[75234]: DEBUG nova.compute.manager [req-8f80e96e-45c3-42c9-871a-0e3f76db17e2 req-c2cb08a9-a054-40a3-b5f5-56418a6da91c service nova] [instance: 0aff446c-f417-477e-a1c5-b51b1b6214e4] Received event network-vif-unplugged-f9148192-6e14-44d4-a31b-081ea47aa289 for instance with task_state deleting. {{(pid=75234) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10890}}
Nov 29 03:19:58 user nova-compute[75234]: DEBUG nova.compute.manager [req-8f80e96e-45c3-42c9-871a-0e3f76db17e2 req-c2cb08a9-a054-40a3-b5f5-56418a6da91c service nova] [instance: 0aff446c-f417-477e-a1c5-b51b1b6214e4] Received event network-vif-plugged-f9148192-6e14-44d4-a31b-081ea47aa289 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:19:58 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-8f80e96e-45c3-42c9-871a-0e3f76db17e2 req-c2cb08a9-a054-40a3-b5f5-56418a6da91c service nova] Acquiring lock "0aff446c-f417-477e-a1c5-b51b1b6214e4-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:19:58 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-8f80e96e-45c3-42c9-871a-0e3f76db17e2 req-c2cb08a9-a054-40a3-b5f5-56418a6da91c service nova] Lock "0aff446c-f417-477e-a1c5-b51b1b6214e4-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:19:58 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-8f80e96e-45c3-42c9-871a-0e3f76db17e2 req-c2cb08a9-a054-40a3-b5f5-56418a6da91c service nova] Lock "0aff446c-f417-477e-a1c5-b51b1b6214e4-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:19:58 user nova-compute[75234]: DEBUG nova.compute.manager [req-8f80e96e-45c3-42c9-871a-0e3f76db17e2 req-c2cb08a9-a054-40a3-b5f5-56418a6da91c service nova] [instance: 0aff446c-f417-477e-a1c5-b51b1b6214e4] No waiting events found dispatching network-vif-plugged-f9148192-6e14-44d4-a31b-081ea47aa289 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:19:58 user nova-compute[75234]: WARNING nova.compute.manager [req-8f80e96e-45c3-42c9-871a-0e3f76db17e2 req-c2cb08a9-a054-40a3-b5f5-56418a6da91c service nova] [instance: 0aff446c-f417-477e-a1c5-b51b1b6214e4] Received unexpected event network-vif-plugged-f9148192-6e14-44d4-a31b-081ea47aa289 for instance with vm_state active and task_state deleting.
Nov 29 03:19:58 user nova-compute[75234]: INFO nova.virt.libvirt.driver [-] [instance: 0aff446c-f417-477e-a1c5-b51b1b6214e4] Instance destroyed successfully.
Nov 29 03:19:58 user nova-compute[75234]: DEBUG nova.objects.instance [None req-1fcf07e3-484b-422a-bc7c-c42ac90da2fb tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Lazy-loading 'resources' on Instance uuid 0aff446c-f417-477e-a1c5-b51b1b6214e4 {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:19:58 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-1fcf07e3-484b-422a-bc7c-c42ac90da2fb tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-11-29T03:18:04Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-AttachVolumeNegativeTest-server-717472654',display_name='tempest-AttachVolumeNegativeTest-server-717472654',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-attachvolumenegativetest-server-717472654',id=32,image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBGtbFLqgbiAnwNCauVOxBiJsbekFn33Y3bNQ0wRsVUvi8aJ9cY/djuV8LOklgMHTzP1K/jzTQKALfTimFUTXA6U/C/rVArYt3uY6rb/8pJx8N1xNGV/JwqMC7v6GIr/lwg==',key_name='tempest-keypair-2106534641',keypairs=<?>,launch_index=0,launched_at=2023-11-29T03:18:12Z,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='315732b6e3ee44d584b8e510ef199a90',ramdisk_id='',reservation_id='r-l9ya0pnl',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-AttachVolumeNegativeTest-1582496806',owner_user_name='tempest-AttachVolumeNegativeTest-1582496806-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-11-29T03:18:12Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='66686114450f4f31b3a917667a152d8b',uuid=0aff446c-f417-477e-a1c5-b51b1b6214e4,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "f9148192-6e14-44d4-a31b-081ea47aa289", "address": "fa:16:3e:46:49:9c", "network": {"id": "0f289171-3d73-4054-95a0-2515b4f206a0", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-1787960284-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "315732b6e3ee44d584b8e510ef199a90", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapf9148192-6e", "ovs_interfaceid": "f9148192-6e14-44d4-a31b-081ea47aa289", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Nov 29 03:19:58 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-1fcf07e3-484b-422a-bc7c-c42ac90da2fb tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Converting VIF {"id": "f9148192-6e14-44d4-a31b-081ea47aa289", "address": "fa:16:3e:46:49:9c", "network": {"id": "0f289171-3d73-4054-95a0-2515b4f206a0", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-1787960284-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "315732b6e3ee44d584b8e510ef199a90", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapf9148192-6e", "ovs_interfaceid": "f9148192-6e14-44d4-a31b-081ea47aa289", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:19:58 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-1fcf07e3-484b-422a-bc7c-c42ac90da2fb tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:46:49:9c,bridge_name='br-int',has_traffic_filtering=True,id=f9148192-6e14-44d4-a31b-081ea47aa289,network=Network(0f289171-3d73-4054-95a0-2515b4f206a0),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapf9148192-6e') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:19:58 user nova-compute[75234]: DEBUG os_vif [None req-1fcf07e3-484b-422a-bc7c-c42ac90da2fb tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Unplugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:46:49:9c,bridge_name='br-int',has_traffic_filtering=True,id=f9148192-6e14-44d4-a31b-081ea47aa289,network=Network(0f289171-3d73-4054-95a0-2515b4f206a0),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapf9148192-6e') {{(pid=75234) unplug /opt/stack/data/venv/lib/python3.10/site-packages/os_vif/__init__.py:109}}
Nov 29 03:19:58 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:19:58 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tapf9148192-6e, bridge=br-int, if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:19:58 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:19:58 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:19:58 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:19:58 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbDestroyCommand(_result=None, table=QoS, record=eb7c227b-1518-42d1-9b49-f7e618eadee8) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:19:58 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:19:58 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:19:58 user nova-compute[75234]: INFO os_vif [None req-1fcf07e3-484b-422a-bc7c-c42ac90da2fb tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Successfully unplugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:46:49:9c,bridge_name='br-int',has_traffic_filtering=True,id=f9148192-6e14-44d4-a31b-081ea47aa289,network=Network(0f289171-3d73-4054-95a0-2515b4f206a0),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapf9148192-6e')
Nov 29 03:19:58 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-1fcf07e3-484b-422a-bc7c-c42ac90da2fb tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] [instance: 0aff446c-f417-477e-a1c5-b51b1b6214e4] Deleting instance files /opt/stack/data/nova/instances/0aff446c-f417-477e-a1c5-b51b1b6214e4_del
Nov 29 03:19:58 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-1fcf07e3-484b-422a-bc7c-c42ac90da2fb tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] [instance: 0aff446c-f417-477e-a1c5-b51b1b6214e4] Deletion of /opt/stack/data/nova/instances/0aff446c-f417-477e-a1c5-b51b1b6214e4_del complete
Nov 29 03:19:58 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:19:58 user nova-compute[75234]: INFO nova.compute.manager [None req-1fcf07e3-484b-422a-bc7c-c42ac90da2fb tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] [instance: 0aff446c-f417-477e-a1c5-b51b1b6214e4] Took 0.95 seconds to destroy the instance on the hypervisor.
Nov 29 03:19:58 user nova-compute[75234]: DEBUG oslo.service.loopingcall [None req-1fcf07e3-484b-422a-bc7c-c42ac90da2fb tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=75234) func /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/loopingcall.py:435}}
Nov 29 03:19:58 user nova-compute[75234]: DEBUG nova.compute.manager [-] [instance: 0aff446c-f417-477e-a1c5-b51b1b6214e4] Deallocating network for instance {{(pid=75234) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2279}}
Nov 29 03:19:58 user nova-compute[75234]: DEBUG nova.network.neutron [-] [instance: 0aff446c-f417-477e-a1c5-b51b1b6214e4] deallocate_for_instance() {{(pid=75234) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1802}}
Nov 29 03:19:59 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-7b720460-3d17-493e-a134-1ef47ad6fba2 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.363s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:87}}
Nov 29 03:19:59 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-7b720460-3d17-493e-a134-1ef47ad6fba2 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] <== connect_volume: return (1365ms) {'type': 'block', 'scsi_wwn': '23365386266373437', 'path': '/dev/sdb'} {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:203}}
Nov 29 03:19:59 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-7b720460-3d17-493e-a134-1ef47ad6fba2 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] [instance: af699bc2-9daa-4b5a-9b4a-ad92fe857d70] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '23365386266373437', 'path': '/dev/sdb'} {{(pid=75234) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Nov 29 03:19:59 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-7b720460-3d17-493e-a134-1ef47ad6fba2 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-11-29T03:17:51Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-ServerActionsTestOtherA-server-1736967407',display_name='tempest-ServerActionsTestOtherA-server-1736967407',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(7),hidden=False,host='user',hostname='tempest-serveractionstestothera-server-1736967407',id=31,image_ref='',info_cache=InstanceInfoCache,instance_type_id=7,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBGysIZGz8py0LojrFdRCkkIN/DpphiAjtTxjJQY7OVo0ZaKHLKAtSriy5/opnav7HzQyT3MeG6wW+2nQfBNyK0gW9HDU5His6qvZYEbhyEyMiaUzdXksC9FbE2bauudLhQ==',key_name='tempest-keypair-1767034584',keypairs=<?>,launch_index=0,launched_at=2023-11-29T03:18:28Z,launched_on='user',locked=False,locked_by=None,memory_mb=2048,metadata={},migration_context=MigrationContext,new_flavor=Flavor(7),node='user',numa_topology=None,old_flavor=Flavor(6),os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=1,progress=0,project_id='c1cb3d1f12f242a785a688d59dbe3cd6',ramdisk_id='',reservation_id='r-p258pz1s',resources=None,root_device_name='/dev/vda',root_gb=20,security_groups=SecurityGroupList,services=ServiceList,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',old_vm_state='active',owner_project_name='tempest-ServerActionsTestOtherA-1495067208',owner_user_name='tempest-ServerActionsTestOtherA-1495067208-project-member'},tags=<?>,task_state='resize_finish',terminated_at=None,trusted_certs=None,updated_at=2023-11-29T03:19:49Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='d611826f3be947c0947b99c763c26d44',uuid=af699bc2-9daa-4b5a-9b4a-ad92fe857d70,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "66dcb0f9-3180-42f7-851d-19b0c4f4119c", "address": "fa:16:3e:dd:9e:50", "network": {"id": "1f946e69-dbec-429c-83f8-a186bad4f2cc", "bridge": "br-int", "label": "tempest-ServerActionsTestOtherA-486369423-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.64", "type": "floating", "version": 4, "meta": {}}], "label": "tempest-ServerActionsTestOtherA-486369423-network", "vif_mac": "fa:16:3e:dd:9e:50"}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "c1cb3d1f12f242a785a688d59dbe3cd6", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap66dcb0f9-31", "ovs_interfaceid": "66dcb0f9-3180-42f7-851d-19b0c4f4119c", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=75234) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Nov 29 03:19:59 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-7b720460-3d17-493e-a134-1ef47ad6fba2 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Converting VIF {"id": "66dcb0f9-3180-42f7-851d-19b0c4f4119c", "address": "fa:16:3e:dd:9e:50", "network": {"id": "1f946e69-dbec-429c-83f8-a186bad4f2cc", "bridge": "br-int", "label": "tempest-ServerActionsTestOtherA-486369423-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.64", "type": "floating", "version": 4, "meta": {}}], "label": "tempest-ServerActionsTestOtherA-486369423-network", "vif_mac": "fa:16:3e:dd:9e:50"}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "c1cb3d1f12f242a785a688d59dbe3cd6", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap66dcb0f9-31", "ovs_interfaceid": "66dcb0f9-3180-42f7-851d-19b0c4f4119c", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:19:59 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-7b720460-3d17-493e-a134-1ef47ad6fba2 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:dd:9e:50,bridge_name='br-int',has_traffic_filtering=True,id=66dcb0f9-3180-42f7-851d-19b0c4f4119c,network=Network(1f946e69-dbec-429c-83f8-a186bad4f2cc),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap66dcb0f9-31') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:19:59 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-7b720460-3d17-493e-a134-1ef47ad6fba2 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] [instance: af699bc2-9daa-4b5a-9b4a-ad92fe857d70] End _get_guest_xml xml=<domain type="kvm">
Nov 29 03:19:59 user nova-compute[75234]:   <uuid>af699bc2-9daa-4b5a-9b4a-ad92fe857d70</uuid>
Nov 29 03:19:59 user nova-compute[75234]:   <name>instance-0000001f</name>
Nov 29 03:19:59 user nova-compute[75234]:   <memory>2097152</memory>
Nov 29 03:19:59 user nova-compute[75234]:   <vcpu>1</vcpu>
Nov 29 03:19:59 user nova-compute[75234]:   <metadata>
Nov 29 03:19:59 user nova-compute[75234]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Nov 29 03:19:59 user nova-compute[75234]:       <nova:package version="0.0.0"/>
Nov 29 03:19:59 user nova-compute[75234]:       <nova:name>tempest-ServerActionsTestOtherA-server-1736967407</nova:name>
Nov 29 03:19:59 user nova-compute[75234]:       <nova:creationTime>2023-11-29 03:19:57</nova:creationTime>
Nov 29 03:19:59 user nova-compute[75234]:       <nova:flavor name="m1.small">
Nov 29 03:19:59 user nova-compute[75234]:         <nova:memory>2048</nova:memory>
Nov 29 03:19:59 user nova-compute[75234]:         <nova:disk>20</nova:disk>
Nov 29 03:19:59 user nova-compute[75234]:         <nova:swap>0</nova:swap>
Nov 29 03:19:59 user nova-compute[75234]:         <nova:ephemeral>0</nova:ephemeral>
Nov 29 03:19:59 user nova-compute[75234]:         <nova:vcpus>1</nova:vcpus>
Nov 29 03:19:59 user nova-compute[75234]:       </nova:flavor>
Nov 29 03:19:59 user nova-compute[75234]:       <nova:owner>
Nov 29 03:19:59 user nova-compute[75234]:         <nova:user uuid="d611826f3be947c0947b99c763c26d44">tempest-ServerActionsTestOtherA-1495067208-project-member</nova:user>
Nov 29 03:19:59 user nova-compute[75234]:         <nova:project uuid="c1cb3d1f12f242a785a688d59dbe3cd6">tempest-ServerActionsTestOtherA-1495067208</nova:project>
Nov 29 03:19:59 user nova-compute[75234]:       </nova:owner>
Nov 29 03:19:59 user nova-compute[75234]:       <nova:ports>
Nov 29 03:19:59 user nova-compute[75234]:         <nova:port uuid="66dcb0f9-3180-42f7-851d-19b0c4f4119c">
Nov 29 03:19:59 user nova-compute[75234]:           <nova:ip type="fixed" address="10.0.0.14" ipVersion="4"/>
Nov 29 03:19:59 user nova-compute[75234]:         </nova:port>
Nov 29 03:19:59 user nova-compute[75234]:       </nova:ports>
Nov 29 03:19:59 user nova-compute[75234]:     </nova:instance>
Nov 29 03:19:59 user nova-compute[75234]:   </metadata>
Nov 29 03:19:59 user nova-compute[75234]:   <sysinfo type="smbios">
Nov 29 03:19:59 user nova-compute[75234]:     <system>
Nov 29 03:19:59 user nova-compute[75234]:       <entry name="manufacturer">OpenStack Foundation</entry>
Nov 29 03:19:59 user nova-compute[75234]:       <entry name="product">OpenStack Nova</entry>
Nov 29 03:19:59 user nova-compute[75234]:       <entry name="version">0.0.0</entry>
Nov 29 03:19:59 user nova-compute[75234]:       <entry name="serial">af699bc2-9daa-4b5a-9b4a-ad92fe857d70</entry>
Nov 29 03:19:59 user nova-compute[75234]:       <entry name="uuid">af699bc2-9daa-4b5a-9b4a-ad92fe857d70</entry>
Nov 29 03:19:59 user nova-compute[75234]:       <entry name="family">Virtual Machine</entry>
Nov 29 03:19:59 user nova-compute[75234]:     </system>
Nov 29 03:19:59 user nova-compute[75234]:   </sysinfo>
Nov 29 03:19:59 user nova-compute[75234]:   <os>
Nov 29 03:19:59 user nova-compute[75234]:     <type arch="x86_64" machine="pc">hvm</type>
Nov 29 03:19:59 user nova-compute[75234]:     <boot dev="hd"/>
Nov 29 03:19:59 user nova-compute[75234]:     <smbios mode="sysinfo"/>
Nov 29 03:19:59 user nova-compute[75234]:   </os>
Nov 29 03:19:59 user nova-compute[75234]:   <features>
Nov 29 03:19:59 user nova-compute[75234]:     <acpi/>
Nov 29 03:19:59 user nova-compute[75234]:     <apic/>
Nov 29 03:19:59 user nova-compute[75234]:     <vmcoreinfo/>
Nov 29 03:19:59 user nova-compute[75234]:   </features>
Nov 29 03:19:59 user nova-compute[75234]:   <clock offset="utc">
Nov 29 03:19:59 user nova-compute[75234]:     <timer name="pit" tickpolicy="delay"/>
Nov 29 03:19:59 user nova-compute[75234]:     <timer name="rtc" tickpolicy="catchup"/>
Nov 29 03:19:59 user nova-compute[75234]:     <timer name="hpet" present="no"/>
Nov 29 03:19:59 user nova-compute[75234]:   </clock>
Nov 29 03:19:59 user nova-compute[75234]:   <cpu mode="custom" match="exact">
Nov 29 03:19:59 user nova-compute[75234]:     <model>Nehalem</model>
Nov 29 03:19:59 user nova-compute[75234]:     <topology sockets="1" cores="1" threads="1"/>
Nov 29 03:19:59 user nova-compute[75234]:   </cpu>
Nov 29 03:19:59 user nova-compute[75234]:   <devices>
Nov 29 03:19:59 user nova-compute[75234]:     <disk type="block" device="disk">
Nov 29 03:19:59 user nova-compute[75234]:       <driver name="qemu" type="raw" cache="none" io="native"/>
Nov 29 03:19:59 user nova-compute[75234]:       <source dev="/dev/sdb"/>
Nov 29 03:19:59 user nova-compute[75234]:       <target dev="vda" bus="virtio"/>
Nov 29 03:19:59 user nova-compute[75234]:       <serial>cf6ab229-533a-424e-bd67-1d54d50ad4be</serial>
Nov 29 03:19:59 user nova-compute[75234]:     </disk>
Nov 29 03:19:59 user nova-compute[75234]:     <interface type="ethernet">
Nov 29 03:19:59 user nova-compute[75234]:       <mac address="fa:16:3e:dd:9e:50"/>
Nov 29 03:19:59 user nova-compute[75234]:       <model type="virtio"/>
Nov 29 03:19:59 user nova-compute[75234]:       <mtu size="1442"/>
Nov 29 03:19:59 user nova-compute[75234]:       <target dev="tap66dcb0f9-31"/>
Nov 29 03:19:59 user nova-compute[75234]:     </interface>
Nov 29 03:19:59 user nova-compute[75234]:     <serial type="pty">
Nov 29 03:19:59 user nova-compute[75234]:       <log file="/opt/stack/data/nova/instances/af699bc2-9daa-4b5a-9b4a-ad92fe857d70/console.log" append="off"/>
Nov 29 03:19:59 user nova-compute[75234]:     </serial>
Nov 29 03:19:59 user nova-compute[75234]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Nov 29 03:19:59 user nova-compute[75234]:     <video>
Nov 29 03:19:59 user nova-compute[75234]:       <model type="virtio"/>
Nov 29 03:19:59 user nova-compute[75234]:     </video>
Nov 29 03:19:59 user nova-compute[75234]:     <rng model="virtio">
Nov 29 03:19:59 user nova-compute[75234]:       <backend model="random">/dev/urandom</backend>
Nov 29 03:19:59 user nova-compute[75234]:     </rng>
Nov 29 03:19:59 user nova-compute[75234]:     <controller type="usb" index="0" model="none"/>
Nov 29 03:19:59 user nova-compute[75234]:     <memballoon model="virtio">
Nov 29 03:19:59 user nova-compute[75234]:       <stats period="10"/>
Nov 29 03:19:59 user nova-compute[75234]:     </memballoon>
Nov 29 03:19:59 user nova-compute[75234]:   </devices>
Nov 29 03:19:59 user nova-compute[75234]: </domain>
Nov 29 03:19:59 user nova-compute[75234]:  {{(pid=75234) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7568}}
Nov 29 03:19:59 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-7b720460-3d17-493e-a134-1ef47ad6fba2 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-11-29T03:17:51Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-ServerActionsTestOtherA-server-1736967407',display_name='tempest-ServerActionsTestOtherA-server-1736967407',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(7),hidden=False,host='user',hostname='tempest-serveractionstestothera-server-1736967407',id=31,image_ref='',info_cache=InstanceInfoCache,instance_type_id=7,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBGysIZGz8py0LojrFdRCkkIN/DpphiAjtTxjJQY7OVo0ZaKHLKAtSriy5/opnav7HzQyT3MeG6wW+2nQfBNyK0gW9HDU5His6qvZYEbhyEyMiaUzdXksC9FbE2bauudLhQ==',key_name='tempest-keypair-1767034584',keypairs=<?>,launch_index=0,launched_at=2023-11-29T03:18:28Z,launched_on='user',locked=False,locked_by=None,memory_mb=2048,metadata={},migration_context=MigrationContext,new_flavor=Flavor(7),node='user',numa_topology=None,old_flavor=Flavor(6),os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=1,progress=0,project_id='c1cb3d1f12f242a785a688d59dbe3cd6',ramdisk_id='',reservation_id='r-p258pz1s',resources=None,root_device_name='/dev/vda',root_gb=20,security_groups=SecurityGroupList,services=ServiceList,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',old_vm_state='active',owner_project_name='tempest-ServerActionsTestOtherA-1495067208',owner_user_name='tempest-ServerActionsTestOtherA-1495067208-project-member'},tags=<?>,task_state='resize_finish',terminated_at=None,trusted_certs=None,updated_at=2023-11-29T03:19:49Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='d611826f3be947c0947b99c763c26d44',uuid=af699bc2-9daa-4b5a-9b4a-ad92fe857d70,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "66dcb0f9-3180-42f7-851d-19b0c4f4119c", "address": "fa:16:3e:dd:9e:50", "network": {"id": "1f946e69-dbec-429c-83f8-a186bad4f2cc", "bridge": "br-int", "label": "tempest-ServerActionsTestOtherA-486369423-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.64", "type": "floating", "version": 4, "meta": {}}], "label": "tempest-ServerActionsTestOtherA-486369423-network", "vif_mac": "fa:16:3e:dd:9e:50"}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "c1cb3d1f12f242a785a688d59dbe3cd6", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap66dcb0f9-31", "ovs_interfaceid": "66dcb0f9-3180-42f7-851d-19b0c4f4119c", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Nov 29 03:19:59 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-7b720460-3d17-493e-a134-1ef47ad6fba2 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Converting VIF {"id": "66dcb0f9-3180-42f7-851d-19b0c4f4119c", "address": "fa:16:3e:dd:9e:50", "network": {"id": "1f946e69-dbec-429c-83f8-a186bad4f2cc", "bridge": "br-int", "label": "tempest-ServerActionsTestOtherA-486369423-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.64", "type": "floating", "version": 4, "meta": {}}], "label": "tempest-ServerActionsTestOtherA-486369423-network", "vif_mac": "fa:16:3e:dd:9e:50"}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "c1cb3d1f12f242a785a688d59dbe3cd6", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap66dcb0f9-31", "ovs_interfaceid": "66dcb0f9-3180-42f7-851d-19b0c4f4119c", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:19:59 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-7b720460-3d17-493e-a134-1ef47ad6fba2 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:dd:9e:50,bridge_name='br-int',has_traffic_filtering=True,id=66dcb0f9-3180-42f7-851d-19b0c4f4119c,network=Network(1f946e69-dbec-429c-83f8-a186bad4f2cc),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap66dcb0f9-31') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:19:59 user nova-compute[75234]: DEBUG os_vif [None req-7b720460-3d17-493e-a134-1ef47ad6fba2 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:dd:9e:50,bridge_name='br-int',has_traffic_filtering=True,id=66dcb0f9-3180-42f7-851d-19b0c4f4119c,network=Network(1f946e69-dbec-429c-83f8-a186bad4f2cc),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap66dcb0f9-31') {{(pid=75234) plug /opt/stack/data/venv/lib/python3.10/site-packages/os_vif/__init__.py:76}}
Nov 29 03:19:59 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:19:59 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:19:59 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Nov 29 03:19:59 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:19:59 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbCreateCommand(_result=None, table=QoS, columns={'type': 'linux-noop', 'external_ids': {'id': '7989ed51-8d0a-5fad-bb2f-ea6f94ff7e01', '_type': 'linux-noop'}}, row=False) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:19:59 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:19:59 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:19:59 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:19:59 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tap66dcb0f9-31, may_exist=True, interface_attrs={}) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:19:59 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Port, record=tap66dcb0f9-31, col_values=(('qos', UUID('2b57c14a-84ee-47a7-9057-0ca1849af5f4')),), if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:19:59 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DbSetCommand(_result=None, table=Interface, record=tap66dcb0f9-31, col_values=(('external_ids', {'iface-id': '66dcb0f9-3180-42f7-851d-19b0c4f4119c', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:dd:9e:50', 'vm-uuid': 'af699bc2-9daa-4b5a-9b4a-ad92fe857d70'}),), if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:19:59 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:19:59 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:19:59 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:19:59 user nova-compute[75234]: INFO os_vif [None req-7b720460-3d17-493e-a134-1ef47ad6fba2 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:dd:9e:50,bridge_name='br-int',has_traffic_filtering=True,id=66dcb0f9-3180-42f7-851d-19b0c4f4119c,network=Network(1f946e69-dbec-429c-83f8-a186bad4f2cc),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap66dcb0f9-31')
Nov 29 03:19:59 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-7b720460-3d17-493e-a134-1ef47ad6fba2 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] No BDM found with device name vda, not building metadata. {{(pid=75234) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12089}}
Nov 29 03:19:59 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-7b720460-3d17-493e-a134-1ef47ad6fba2 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] No VIF found with MAC fa:16:3e:dd:9e:50, not building metadata {{(pid=75234) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Nov 29 03:19:59 user nova-compute[75234]: DEBUG nova.network.neutron [-] [instance: 0aff446c-f417-477e-a1c5-b51b1b6214e4] Updating instance_info_cache with network_info: [] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:19:59 user nova-compute[75234]: INFO nova.compute.manager [-] [instance: 0aff446c-f417-477e-a1c5-b51b1b6214e4] Took 0.75 seconds to deallocate network for instance.
Nov 29 03:19:59 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-1fcf07e3-484b-422a-bc7c-c42ac90da2fb tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:19:59 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-1fcf07e3-484b-422a-bc7c-c42ac90da2fb tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:19:59 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:19:59 user nova-compute[75234]: DEBUG nova.compute.provider_tree [None req-1fcf07e3-484b-422a-bc7c-c42ac90da2fb tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Inventory has not changed in ProviderTree for provider: 4d291220-7c37-4839-a0e3-b23c66502b74 {{(pid=75234) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Nov 29 03:19:59 user nova-compute[75234]: DEBUG nova.scheduler.client.report [None req-1fcf07e3-484b-422a-bc7c-c42ac90da2fb tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Inventory has not changed for provider 4d291220-7c37-4839-a0e3-b23c66502b74 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=75234) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Nov 29 03:19:59 user nova-compute[75234]: INFO nova.compute.manager [None req-ac7b122c-8440-4607-bea5-d9305f99725b tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] [instance: aa190a9f-ba53-43fb-8bae-2c429d0d1228] Took 4.11 seconds to detach 1 volumes for instance.
Nov 29 03:19:59 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ac7b122c-8440-4607-bea5-d9305f99725b tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] [instance: aa190a9f-ba53-43fb-8bae-2c429d0d1228] Deleting volume: 07f83efc-272d-4e10-9a2b-48d967db613c {{(pid=75234) _cleanup_volumes /opt/stack/nova/nova/compute/manager.py:3237}}
Nov 29 03:19:59 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-1fcf07e3-484b-422a-bc7c-c42ac90da2fb tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.469s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:19:59 user nova-compute[75234]: INFO nova.scheduler.client.report [None req-1fcf07e3-484b-422a-bc7c-c42ac90da2fb tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Deleted allocations for instance 0aff446c-f417-477e-a1c5-b51b1b6214e4
Nov 29 03:20:00 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ac7b122c-8440-4607-bea5-d9305f99725b tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:20:00 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ac7b122c-8440-4607-bea5-d9305f99725b tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:20:00 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-1fcf07e3-484b-422a-bc7c-c42ac90da2fb tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Lock "0aff446c-f417-477e-a1c5-b51b1b6214e4" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 2.431s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:20:00 user nova-compute[75234]: DEBUG nova.compute.manager [req-101edc05-7150-4dd4-b43e-c1a5505b9e01 req-54b45f4b-0e06-45ff-9484-fe80f91a953f service nova] [instance: 0aff446c-f417-477e-a1c5-b51b1b6214e4] Received event network-vif-deleted-f9148192-6e14-44d4-a31b-081ea47aa289 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:20:00 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-10541310-8b0c-44a5-9ba4-cec9ff30ffd5 tempest-TestServerVolumeAttachScenarioOldVersion-830274084 tempest-TestServerVolumeAttachScenarioOldVersion-830274084-project-member] Acquiring lock "5e6cca46-c5a7-4303-98c2-1ad483c6d14e" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:20:00 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-10541310-8b0c-44a5-9ba4-cec9ff30ffd5 tempest-TestServerVolumeAttachScenarioOldVersion-830274084 tempest-TestServerVolumeAttachScenarioOldVersion-830274084-project-member] Lock "5e6cca46-c5a7-4303-98c2-1ad483c6d14e" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:20:00 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-10541310-8b0c-44a5-9ba4-cec9ff30ffd5 tempest-TestServerVolumeAttachScenarioOldVersion-830274084 tempest-TestServerVolumeAttachScenarioOldVersion-830274084-project-member] Acquiring lock "5e6cca46-c5a7-4303-98c2-1ad483c6d14e-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:20:00 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-10541310-8b0c-44a5-9ba4-cec9ff30ffd5 tempest-TestServerVolumeAttachScenarioOldVersion-830274084 tempest-TestServerVolumeAttachScenarioOldVersion-830274084-project-member] Lock "5e6cca46-c5a7-4303-98c2-1ad483c6d14e-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:20:00 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-10541310-8b0c-44a5-9ba4-cec9ff30ffd5 tempest-TestServerVolumeAttachScenarioOldVersion-830274084 tempest-TestServerVolumeAttachScenarioOldVersion-830274084-project-member] Lock "5e6cca46-c5a7-4303-98c2-1ad483c6d14e-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:20:00 user nova-compute[75234]: INFO nova.compute.manager [None req-10541310-8b0c-44a5-9ba4-cec9ff30ffd5 tempest-TestServerVolumeAttachScenarioOldVersion-830274084 tempest-TestServerVolumeAttachScenarioOldVersion-830274084-project-member] [instance: 5e6cca46-c5a7-4303-98c2-1ad483c6d14e] Terminating instance
Nov 29 03:20:00 user nova-compute[75234]: DEBUG nova.compute.manager [None req-10541310-8b0c-44a5-9ba4-cec9ff30ffd5 tempest-TestServerVolumeAttachScenarioOldVersion-830274084 tempest-TestServerVolumeAttachScenarioOldVersion-830274084-project-member] [instance: 5e6cca46-c5a7-4303-98c2-1ad483c6d14e] Start destroying the instance on the hypervisor. {{(pid=75234) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3140}}
Nov 29 03:20:00 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:20:00 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:20:00 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:20:00 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:20:00 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:20:00 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:20:00 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:20:00 user nova-compute[75234]: DEBUG nova.compute.provider_tree [None req-ac7b122c-8440-4607-bea5-d9305f99725b tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] Inventory has not changed in ProviderTree for provider: 4d291220-7c37-4839-a0e3-b23c66502b74 {{(pid=75234) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Nov 29 03:20:00 user nova-compute[75234]: DEBUG nova.scheduler.client.report [None req-ac7b122c-8440-4607-bea5-d9305f99725b tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] Inventory has not changed for provider 4d291220-7c37-4839-a0e3-b23c66502b74 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=75234) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Nov 29 03:20:00 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ac7b122c-8440-4607-bea5-d9305f99725b tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.715s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:20:00 user nova-compute[75234]: INFO nova.scheduler.client.report [None req-ac7b122c-8440-4607-bea5-d9305f99725b tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] Deleted allocations for instance aa190a9f-ba53-43fb-8bae-2c429d0d1228
Nov 29 03:20:00 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ac7b122c-8440-4607-bea5-d9305f99725b tempest-ServerActionsV293TestJSON-155733519 tempest-ServerActionsV293TestJSON-155733519-project-member] Lock "aa190a9f-ba53-43fb-8bae-2c429d0d1228" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 7.106s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:20:01 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:20:01 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:20:01 user nova-compute[75234]: INFO nova.virt.libvirt.driver [-] [instance: 5e6cca46-c5a7-4303-98c2-1ad483c6d14e] Instance destroyed successfully.
Nov 29 03:20:01 user nova-compute[75234]: DEBUG nova.objects.instance [None req-10541310-8b0c-44a5-9ba4-cec9ff30ffd5 tempest-TestServerVolumeAttachScenarioOldVersion-830274084 tempest-TestServerVolumeAttachScenarioOldVersion-830274084-project-member] Lazy-loading 'resources' on Instance uuid 5e6cca46-c5a7-4303-98c2-1ad483c6d14e {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:20:01 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-10541310-8b0c-44a5-9ba4-cec9ff30ffd5 tempest-TestServerVolumeAttachScenarioOldVersion-830274084 tempest-TestServerVolumeAttachScenarioOldVersion-830274084-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-11-29T03:16:50Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-TestServerVolumeAttachScenarioOldVersion-server-816676743',display_name='tempest-TestServerVolumeAttachScenarioOldVersion-server-816676743',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-testservervolumeattachscenariooldversion-server-8166767',id=29,image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBKQV6WOnODZRu0S0Q8BjipN+d4SfGF+oFplEizu9978zfVugP8bBS3zBtCT9lFZz+6aJRHATnsP1lSYQtfjqOryzG+7rBhPRnAeY9RFQZR+EOYZl1GaC/iv/bQZS956elA==',key_name='tempest-keypair-526758182',keypairs=<?>,launch_index=0,launched_at=2023-11-29T03:16:57Z,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='d6f51212dd1c4632b36cfdbd7f901a89',ramdisk_id='',reservation_id='r-49dw65dl',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-TestServerVolumeAttachScenarioOldVersion-830274084',owner_user_name='tempest-TestServerVolumeAttachScenarioOldVersion-830274084-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-11-29T03:16:57Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='68a6c98565d348fb8afabaddcd0b3f9d',uuid=5e6cca46-c5a7-4303-98c2-1ad483c6d14e,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "69d76626-4c79-4557-b501-7c8c92fcdc03", "address": "fa:16:3e:6e:d5:4a", "network": {"id": "f68acf2f-b01a-4481-845d-46b92646988f", "bridge": "br-int", "label": "tempest-TestServerVolumeAttachScenarioOldVersion-1316398304-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.144", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "d6f51212dd1c4632b36cfdbd7f901a89", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap69d76626-4c", "ovs_interfaceid": "69d76626-4c79-4557-b501-7c8c92fcdc03", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Nov 29 03:20:01 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-10541310-8b0c-44a5-9ba4-cec9ff30ffd5 tempest-TestServerVolumeAttachScenarioOldVersion-830274084 tempest-TestServerVolumeAttachScenarioOldVersion-830274084-project-member] Converting VIF {"id": "69d76626-4c79-4557-b501-7c8c92fcdc03", "address": "fa:16:3e:6e:d5:4a", "network": {"id": "f68acf2f-b01a-4481-845d-46b92646988f", "bridge": "br-int", "label": "tempest-TestServerVolumeAttachScenarioOldVersion-1316398304-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.144", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "d6f51212dd1c4632b36cfdbd7f901a89", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap69d76626-4c", "ovs_interfaceid": "69d76626-4c79-4557-b501-7c8c92fcdc03", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:20:01 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-10541310-8b0c-44a5-9ba4-cec9ff30ffd5 tempest-TestServerVolumeAttachScenarioOldVersion-830274084 tempest-TestServerVolumeAttachScenarioOldVersion-830274084-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:6e:d5:4a,bridge_name='br-int',has_traffic_filtering=True,id=69d76626-4c79-4557-b501-7c8c92fcdc03,network=Network(f68acf2f-b01a-4481-845d-46b92646988f),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap69d76626-4c') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:20:01 user nova-compute[75234]: DEBUG os_vif [None req-10541310-8b0c-44a5-9ba4-cec9ff30ffd5 tempest-TestServerVolumeAttachScenarioOldVersion-830274084 tempest-TestServerVolumeAttachScenarioOldVersion-830274084-project-member] Unplugging vif VIFOpenVSwitch(active=True,address=fa:16:3e:6e:d5:4a,bridge_name='br-int',has_traffic_filtering=True,id=69d76626-4c79-4557-b501-7c8c92fcdc03,network=Network(f68acf2f-b01a-4481-845d-46b92646988f),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap69d76626-4c') {{(pid=75234) unplug /opt/stack/data/venv/lib/python3.10/site-packages/os_vif/__init__.py:109}}
Nov 29 03:20:01 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:20:01 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tap69d76626-4c, bridge=br-int, if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:20:01 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:20:01 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:20:01 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:20:01 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbDestroyCommand(_result=None, table=QoS, record=e3b80cda-4b28-4260-8427-60c589c9cd9b) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:20:01 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:20:01 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:20:01 user nova-compute[75234]: INFO os_vif [None req-10541310-8b0c-44a5-9ba4-cec9ff30ffd5 tempest-TestServerVolumeAttachScenarioOldVersion-830274084 tempest-TestServerVolumeAttachScenarioOldVersion-830274084-project-member] Successfully unplugged vif VIFOpenVSwitch(active=True,address=fa:16:3e:6e:d5:4a,bridge_name='br-int',has_traffic_filtering=True,id=69d76626-4c79-4557-b501-7c8c92fcdc03,network=Network(f68acf2f-b01a-4481-845d-46b92646988f),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap69d76626-4c')
Nov 29 03:20:01 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-10541310-8b0c-44a5-9ba4-cec9ff30ffd5 tempest-TestServerVolumeAttachScenarioOldVersion-830274084 tempest-TestServerVolumeAttachScenarioOldVersion-830274084-project-member] [instance: 5e6cca46-c5a7-4303-98c2-1ad483c6d14e] Deleting instance files /opt/stack/data/nova/instances/5e6cca46-c5a7-4303-98c2-1ad483c6d14e_del
Nov 29 03:20:01 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-10541310-8b0c-44a5-9ba4-cec9ff30ffd5 tempest-TestServerVolumeAttachScenarioOldVersion-830274084 tempest-TestServerVolumeAttachScenarioOldVersion-830274084-project-member] [instance: 5e6cca46-c5a7-4303-98c2-1ad483c6d14e] Deletion of /opt/stack/data/nova/instances/5e6cca46-c5a7-4303-98c2-1ad483c6d14e_del complete
Nov 29 03:20:01 user nova-compute[75234]: INFO nova.compute.manager [None req-10541310-8b0c-44a5-9ba4-cec9ff30ffd5 tempest-TestServerVolumeAttachScenarioOldVersion-830274084 tempest-TestServerVolumeAttachScenarioOldVersion-830274084-project-member] [instance: 5e6cca46-c5a7-4303-98c2-1ad483c6d14e] Took 1.12 seconds to destroy the instance on the hypervisor.
Nov 29 03:20:01 user nova-compute[75234]: DEBUG oslo.service.loopingcall [None req-10541310-8b0c-44a5-9ba4-cec9ff30ffd5 tempest-TestServerVolumeAttachScenarioOldVersion-830274084 tempest-TestServerVolumeAttachScenarioOldVersion-830274084-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=75234) func /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/loopingcall.py:435}}
Nov 29 03:20:01 user nova-compute[75234]: DEBUG nova.compute.manager [-] [instance: 5e6cca46-c5a7-4303-98c2-1ad483c6d14e] Deallocating network for instance {{(pid=75234) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2279}}
Nov 29 03:20:01 user nova-compute[75234]: DEBUG nova.network.neutron [-] [instance: 5e6cca46-c5a7-4303-98c2-1ad483c6d14e] deallocate_for_instance() {{(pid=75234) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1802}}
Nov 29 03:20:01 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:20:01 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Starting heal instance info cache {{(pid=75234) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9922}}
Nov 29 03:20:01 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Acquiring lock "refresh_cache-6894b55f-93c6-43c9-8933-56d5a3c43725" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:20:01 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Acquired lock "refresh_cache-6894b55f-93c6-43c9-8933-56d5a3c43725" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:20:01 user nova-compute[75234]: DEBUG nova.network.neutron [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] [instance: 6894b55f-93c6-43c9-8933-56d5a3c43725] Forcefully refreshing network info cache for instance {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2003}}
Nov 29 03:20:02 user nova-compute[75234]: DEBUG nova.network.neutron [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] [instance: 6894b55f-93c6-43c9-8933-56d5a3c43725] Updating instance_info_cache with network_info: [{"id": "9b2cf46e-008c-4ef3-9555-0cbc584dcf8f", "address": "fa:16:3e:d1:0b:00", "network": {"id": "b7a6cef0-4cf1-46e6-99fb-4972879e3913", "bridge": "br-int", "label": "tempest-TestStampPattern-1195861223-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.73", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "c4393bc598a845a9bfb667e5332f14a6", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap9b2cf46e-00", "ovs_interfaceid": "9b2cf46e-008c-4ef3-9555-0cbc584dcf8f", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:20:02 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Releasing lock "refresh_cache-6894b55f-93c6-43c9-8933-56d5a3c43725" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:20:02 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] [instance: 6894b55f-93c6-43c9-8933-56d5a3c43725] Updated the network info_cache for instance {{(pid=75234) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9993}}
Nov 29 03:20:02 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager.update_available_resource {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:20:02 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:20:02 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:20:02 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:20:02 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Auditing locally available compute resources for user (node: user) {{(pid=75234) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:907}}
Nov 29 03:20:02 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:20:02 user nova-compute[75234]: DEBUG nova.compute.manager [req-83631f9c-8ab7-4d2d-a3a1-232eab9de911 req-234b648d-b0e7-4450-9a9c-cc0a0190a212 service nova] [instance: 5e6cca46-c5a7-4303-98c2-1ad483c6d14e] Received event network-vif-unplugged-69d76626-4c79-4557-b501-7c8c92fcdc03 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:20:02 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-83631f9c-8ab7-4d2d-a3a1-232eab9de911 req-234b648d-b0e7-4450-9a9c-cc0a0190a212 service nova] Acquiring lock "5e6cca46-c5a7-4303-98c2-1ad483c6d14e-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:20:02 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-83631f9c-8ab7-4d2d-a3a1-232eab9de911 req-234b648d-b0e7-4450-9a9c-cc0a0190a212 service nova] Lock "5e6cca46-c5a7-4303-98c2-1ad483c6d14e-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:20:02 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-83631f9c-8ab7-4d2d-a3a1-232eab9de911 req-234b648d-b0e7-4450-9a9c-cc0a0190a212 service nova] Lock "5e6cca46-c5a7-4303-98c2-1ad483c6d14e-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:20:02 user nova-compute[75234]: DEBUG nova.compute.manager [req-83631f9c-8ab7-4d2d-a3a1-232eab9de911 req-234b648d-b0e7-4450-9a9c-cc0a0190a212 service nova] [instance: 5e6cca46-c5a7-4303-98c2-1ad483c6d14e] No waiting events found dispatching network-vif-unplugged-69d76626-4c79-4557-b501-7c8c92fcdc03 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:20:02 user nova-compute[75234]: DEBUG nova.compute.manager [req-83631f9c-8ab7-4d2d-a3a1-232eab9de911 req-234b648d-b0e7-4450-9a9c-cc0a0190a212 service nova] [instance: 5e6cca46-c5a7-4303-98c2-1ad483c6d14e] Received event network-vif-unplugged-69d76626-4c79-4557-b501-7c8c92fcdc03 for instance with task_state deleting. {{(pid=75234) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10890}}
Nov 29 03:20:02 user nova-compute[75234]: DEBUG nova.compute.manager [req-83631f9c-8ab7-4d2d-a3a1-232eab9de911 req-234b648d-b0e7-4450-9a9c-cc0a0190a212 service nova] [instance: 5e6cca46-c5a7-4303-98c2-1ad483c6d14e] Received event network-vif-plugged-69d76626-4c79-4557-b501-7c8c92fcdc03 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:20:02 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-83631f9c-8ab7-4d2d-a3a1-232eab9de911 req-234b648d-b0e7-4450-9a9c-cc0a0190a212 service nova] Acquiring lock "5e6cca46-c5a7-4303-98c2-1ad483c6d14e-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:20:02 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-83631f9c-8ab7-4d2d-a3a1-232eab9de911 req-234b648d-b0e7-4450-9a9c-cc0a0190a212 service nova] Lock "5e6cca46-c5a7-4303-98c2-1ad483c6d14e-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:20:02 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-83631f9c-8ab7-4d2d-a3a1-232eab9de911 req-234b648d-b0e7-4450-9a9c-cc0a0190a212 service nova] Lock "5e6cca46-c5a7-4303-98c2-1ad483c6d14e-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:20:02 user nova-compute[75234]: DEBUG nova.compute.manager [req-83631f9c-8ab7-4d2d-a3a1-232eab9de911 req-234b648d-b0e7-4450-9a9c-cc0a0190a212 service nova] [instance: 5e6cca46-c5a7-4303-98c2-1ad483c6d14e] No waiting events found dispatching network-vif-plugged-69d76626-4c79-4557-b501-7c8c92fcdc03 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:20:02 user nova-compute[75234]: WARNING nova.compute.manager [req-83631f9c-8ab7-4d2d-a3a1-232eab9de911 req-234b648d-b0e7-4450-9a9c-cc0a0190a212 service nova] [instance: 5e6cca46-c5a7-4303-98c2-1ad483c6d14e] Received unexpected event network-vif-plugged-69d76626-4c79-4557-b501-7c8c92fcdc03 for instance with vm_state active and task_state deleting.
Nov 29 03:20:02 user nova-compute[75234]: DEBUG nova.network.neutron [-] [instance: 5e6cca46-c5a7-4303-98c2-1ad483c6d14e] Updating instance_info_cache with network_info: [] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:20:03 user nova-compute[75234]: INFO nova.compute.manager [-] [instance: 5e6cca46-c5a7-4303-98c2-1ad483c6d14e] Took 1.56 seconds to deallocate network for instance.
Nov 29 03:20:03 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-10541310-8b0c-44a5-9ba4-cec9ff30ffd5 tempest-TestServerVolumeAttachScenarioOldVersion-830274084 tempest-TestServerVolumeAttachScenarioOldVersion-830274084-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:20:03 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-10541310-8b0c-44a5-9ba4-cec9ff30ffd5 tempest-TestServerVolumeAttachScenarioOldVersion-830274084 tempest-TestServerVolumeAttachScenarioOldVersion-830274084-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:20:03 user nova-compute[75234]: DEBUG nova.compute.manager [None req-7b720460-3d17-493e-a134-1ef47ad6fba2 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] [instance: af699bc2-9daa-4b5a-9b4a-ad92fe857d70] Instance event wait completed in 0 seconds for  {{(pid=75234) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Nov 29 03:20:03 user nova-compute[75234]: INFO nova.virt.libvirt.driver [-] [instance: af699bc2-9daa-4b5a-9b4a-ad92fe857d70] Instance running successfully.
Nov 29 03:20:03 user nova-compute[75234]: DEBUG nova.virt.libvirt.guest [None req-7b720460-3d17-493e-a134-1ef47ad6fba2 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] [instance: af699bc2-9daa-4b5a-9b4a-ad92fe857d70] Failed to set time: agent not configured {{(pid=75234) sync_guest_time /opt/stack/nova/nova/virt/libvirt/guest.py:200}}
Nov 29 03:20:03 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-7b720460-3d17-493e-a134-1ef47ad6fba2 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] [instance: af699bc2-9daa-4b5a-9b4a-ad92fe857d70] finish_migration finished successfully. {{(pid=75234) finish_migration /opt/stack/nova/nova/virt/libvirt/driver.py:11766}}
Nov 29 03:20:03 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] skipping disk /dev/sdb (vda) as it is a volume {{(pid=75234) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11213}}
Nov 29 03:20:03 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/13e5da64-9adc-4479-81c7-d329e9b5f08f/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:20:03 user nova-compute[75234]: DEBUG nova.compute.provider_tree [None req-10541310-8b0c-44a5-9ba4-cec9ff30ffd5 tempest-TestServerVolumeAttachScenarioOldVersion-830274084 tempest-TestServerVolumeAttachScenarioOldVersion-830274084-project-member] Inventory has not changed in ProviderTree for provider: 4d291220-7c37-4839-a0e3-b23c66502b74 {{(pid=75234) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Nov 29 03:20:03 user nova-compute[75234]: DEBUG nova.scheduler.client.report [None req-10541310-8b0c-44a5-9ba4-cec9ff30ffd5 tempest-TestServerVolumeAttachScenarioOldVersion-830274084 tempest-TestServerVolumeAttachScenarioOldVersion-830274084-project-member] Inventory has not changed for provider 4d291220-7c37-4839-a0e3-b23c66502b74 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=75234) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Nov 29 03:20:03 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/13e5da64-9adc-4479-81c7-d329e9b5f08f/disk --force-share --output=json" returned: 0 in 0.209s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:20:03 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/13e5da64-9adc-4479-81c7-d329e9b5f08f/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:20:03 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-10541310-8b0c-44a5-9ba4-cec9ff30ffd5 tempest-TestServerVolumeAttachScenarioOldVersion-830274084 tempest-TestServerVolumeAttachScenarioOldVersion-830274084-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.451s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:20:03 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/13e5da64-9adc-4479-81c7-d329e9b5f08f/disk --force-share --output=json" returned: 0 in 0.184s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:20:03 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/72ebd454-0275-4c3d-95d2-227ffb045a0c/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:20:03 user nova-compute[75234]: INFO nova.scheduler.client.report [None req-10541310-8b0c-44a5-9ba4-cec9ff30ffd5 tempest-TestServerVolumeAttachScenarioOldVersion-830274084 tempest-TestServerVolumeAttachScenarioOldVersion-830274084-project-member] Deleted allocations for instance 5e6cca46-c5a7-4303-98c2-1ad483c6d14e
Nov 29 03:20:03 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-10541310-8b0c-44a5-9ba4-cec9ff30ffd5 tempest-TestServerVolumeAttachScenarioOldVersion-830274084 tempest-TestServerVolumeAttachScenarioOldVersion-830274084-project-member] Lock "5e6cca46-c5a7-4303-98c2-1ad483c6d14e" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 3.566s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:20:03 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/72ebd454-0275-4c3d-95d2-227ffb045a0c/disk --force-share --output=json" returned: 0 in 0.301s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:20:03 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/72ebd454-0275-4c3d-95d2-227ffb045a0c/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:20:04 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:20:04 user nova-compute[75234]: DEBUG nova.compute.manager [req-d3d96010-b698-4c43-b93c-efca02a8508c req-9978bbed-3e82-4c57-9207-75c4bb1b02c6 service nova] [instance: af699bc2-9daa-4b5a-9b4a-ad92fe857d70] Received event network-vif-plugged-66dcb0f9-3180-42f7-851d-19b0c4f4119c {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:20:04 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-d3d96010-b698-4c43-b93c-efca02a8508c req-9978bbed-3e82-4c57-9207-75c4bb1b02c6 service nova] Acquiring lock "af699bc2-9daa-4b5a-9b4a-ad92fe857d70-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:20:04 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-d3d96010-b698-4c43-b93c-efca02a8508c req-9978bbed-3e82-4c57-9207-75c4bb1b02c6 service nova] Lock "af699bc2-9daa-4b5a-9b4a-ad92fe857d70-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:20:04 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-d3d96010-b698-4c43-b93c-efca02a8508c req-9978bbed-3e82-4c57-9207-75c4bb1b02c6 service nova] Lock "af699bc2-9daa-4b5a-9b4a-ad92fe857d70-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:20:04 user nova-compute[75234]: DEBUG nova.compute.manager [req-d3d96010-b698-4c43-b93c-efca02a8508c req-9978bbed-3e82-4c57-9207-75c4bb1b02c6 service nova] [instance: af699bc2-9daa-4b5a-9b4a-ad92fe857d70] No waiting events found dispatching network-vif-plugged-66dcb0f9-3180-42f7-851d-19b0c4f4119c {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:20:04 user nova-compute[75234]: WARNING nova.compute.manager [req-d3d96010-b698-4c43-b93c-efca02a8508c req-9978bbed-3e82-4c57-9207-75c4bb1b02c6 service nova] [instance: af699bc2-9daa-4b5a-9b4a-ad92fe857d70] Received unexpected event network-vif-plugged-66dcb0f9-3180-42f7-851d-19b0c4f4119c for instance with vm_state resized and task_state None.
Nov 29 03:20:04 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/72ebd454-0275-4c3d-95d2-227ffb045a0c/disk --force-share --output=json" returned: 0 in 0.227s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:20:04 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6894b55f-93c6-43c9-8933-56d5a3c43725/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:20:04 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6894b55f-93c6-43c9-8933-56d5a3c43725/disk --force-share --output=json" returned: 0 in 0.201s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:20:04 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6894b55f-93c6-43c9-8933-56d5a3c43725/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:20:04 user nova-compute[75234]: DEBUG nova.compute.manager [req-6f4a244a-b69e-4218-a004-03de906f358d req-e1b9c588-3c63-4d13-8e79-fd1dec4243b2 service nova] [instance: 5e6cca46-c5a7-4303-98c2-1ad483c6d14e] Received event network-vif-deleted-69d76626-4c79-4557-b501-7c8c92fcdc03 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:20:04 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/6894b55f-93c6-43c9-8933-56d5a3c43725/disk --force-share --output=json" returned: 0 in 0.224s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:20:04 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/1df140f1-fbd9-4b5d-b356-d08b7e908f64/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:20:04 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/1df140f1-fbd9-4b5d-b356-d08b7e908f64/disk --force-share --output=json" returned: 0 in 0.181s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:20:04 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/1df140f1-fbd9-4b5d-b356-d08b7e908f64/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:20:05 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/1df140f1-fbd9-4b5d-b356-d08b7e908f64/disk --force-share --output=json" returned: 0 in 0.193s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:20:05 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/aaf1c3e3-b73e-42ad-87bd-c5255ec09b46/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:20:05 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/aaf1c3e3-b73e-42ad-87bd-c5255ec09b46/disk --force-share --output=json" returned: 0 in 0.197s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:20:05 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/aaf1c3e3-b73e-42ad-87bd-c5255ec09b46/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:20:05 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/aaf1c3e3-b73e-42ad-87bd-c5255ec09b46/disk --force-share --output=json" returned: 0 in 0.221s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:20:05 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a082c96e-e384-40b6-bcc8-fc5e0e104669/disk.rescue --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:20:05 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a082c96e-e384-40b6-bcc8-fc5e0e104669/disk.rescue --force-share --output=json" returned: 0 in 0.195s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:20:05 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a082c96e-e384-40b6-bcc8-fc5e0e104669/disk.rescue --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:20:05 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a082c96e-e384-40b6-bcc8-fc5e0e104669/disk.rescue --force-share --output=json" returned: 0 in 0.201s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:20:05 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a082c96e-e384-40b6-bcc8-fc5e0e104669/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:20:06 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a082c96e-e384-40b6-bcc8-fc5e0e104669/disk --force-share --output=json" returned: 0 in 0.197s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:20:06 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a082c96e-e384-40b6-bcc8-fc5e0e104669/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:20:06 user nova-compute[75234]: DEBUG nova.compute.manager [req-2610c048-6434-4ce5-a4fa-3d6b93701d60 req-347d3457-a568-4662-954b-633e72d6b462 service nova] [instance: af699bc2-9daa-4b5a-9b4a-ad92fe857d70] Received event network-vif-plugged-66dcb0f9-3180-42f7-851d-19b0c4f4119c {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:20:06 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-2610c048-6434-4ce5-a4fa-3d6b93701d60 req-347d3457-a568-4662-954b-633e72d6b462 service nova] Acquiring lock "af699bc2-9daa-4b5a-9b4a-ad92fe857d70-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:20:06 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-2610c048-6434-4ce5-a4fa-3d6b93701d60 req-347d3457-a568-4662-954b-633e72d6b462 service nova] Lock "af699bc2-9daa-4b5a-9b4a-ad92fe857d70-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:20:06 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-2610c048-6434-4ce5-a4fa-3d6b93701d60 req-347d3457-a568-4662-954b-633e72d6b462 service nova] Lock "af699bc2-9daa-4b5a-9b4a-ad92fe857d70-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:20:06 user nova-compute[75234]: DEBUG nova.compute.manager [req-2610c048-6434-4ce5-a4fa-3d6b93701d60 req-347d3457-a568-4662-954b-633e72d6b462 service nova] [instance: af699bc2-9daa-4b5a-9b4a-ad92fe857d70] No waiting events found dispatching network-vif-plugged-66dcb0f9-3180-42f7-851d-19b0c4f4119c {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:20:06 user nova-compute[75234]: WARNING nova.compute.manager [req-2610c048-6434-4ce5-a4fa-3d6b93701d60 req-347d3457-a568-4662-954b-633e72d6b462 service nova] [instance: af699bc2-9daa-4b5a-9b4a-ad92fe857d70] Received unexpected event network-vif-plugged-66dcb0f9-3180-42f7-851d-19b0c4f4119c for instance with vm_state resized and task_state None.
Nov 29 03:20:06 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a082c96e-e384-40b6-bcc8-fc5e0e104669/disk --force-share --output=json" returned: 0 in 0.187s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:20:06 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/75c5d700-6367-4128-9228-80d2cd24ace7/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:20:06 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:20:06 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/75c5d700-6367-4128-9228-80d2cd24ace7/disk --force-share --output=json" returned: 0 in 0.187s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:20:06 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/75c5d700-6367-4128-9228-80d2cd24ace7/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:20:06 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:20:06 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/75c5d700-6367-4128-9228-80d2cd24ace7/disk --force-share --output=json" returned: 0 in 0.186s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:20:06 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] skipping disk /dev/sdd (vdb) as it is a volume {{(pid=75234) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11213}}
Nov 29 03:20:06 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/21e9c6cd-c39f-41c0-8768-5dd64589c3f3/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:20:06 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/21e9c6cd-c39f-41c0-8768-5dd64589c3f3/disk --force-share --output=json" returned: 0 in 0.195s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:20:06 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/21e9c6cd-c39f-41c0-8768-5dd64589c3f3/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:20:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/21e9c6cd-c39f-41c0-8768-5dd64589c3f3/disk --force-share --output=json" returned: 0 in 0.189s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:20:07 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:20:07 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:20:07 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:20:07 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Hypervisor/Node resource view: name=user free_ram=7672MB free_disk=25.16641616821289GB free_vcpus=3 pci_devices=[{"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}] {{(pid=75234) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1106}}
Nov 29 03:20:07 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:20:07 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:20:07 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Applying migration context for instance af699bc2-9daa-4b5a-9b4a-ad92fe857d70 as it has an incoming, in-progress migration 9297d372-3baa-4214-a4fd-ae62460f818a. Migration status is finished {{(pid=75234) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:1016}}
Nov 29 03:20:07 user nova-compute[75234]: INFO nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] [instance: af699bc2-9daa-4b5a-9b4a-ad92fe857d70] Updating resource usage from migration 9297d372-3baa-4214-a4fd-ae62460f818a
Nov 29 03:20:07 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance 21e9c6cd-c39f-41c0-8768-5dd64589c3f3 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:20:07 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance 13e5da64-9adc-4479-81c7-d329e9b5f08f actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:20:07 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance a082c96e-e384-40b6-bcc8-fc5e0e104669 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:20:07 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance 6894b55f-93c6-43c9-8933-56d5a3c43725 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:20:07 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance 75c5d700-6367-4128-9228-80d2cd24ace7 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:20:07 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance aaf1c3e3-b73e-42ad-87bd-c5255ec09b46 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:20:07 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance 1df140f1-fbd9-4b5d-b356-d08b7e908f64 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:20:07 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance 72ebd454-0275-4c3d-95d2-227ffb045a0c actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:20:07 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Migration 9297d372-3baa-4214-a4fd-ae62460f818a is active on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 512}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1709}}
Nov 29 03:20:07 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance af699bc2-9daa-4b5a-9b4a-ad92fe857d70 actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 2048, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:20:07 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Total usable vcpus: 12, total allocated vcpus: 10 {{(pid=75234) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1129}}
Nov 29 03:20:07 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Final resource view: name=user phys_ram=16011MB used_ram=7168MB phys_disk=40GB used_disk=8GB total_vcpus=12 used_vcpus=10 pci_stats=[] {{(pid=75234) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1138}}
Nov 29 03:20:08 user nova-compute[75234]: DEBUG nova.compute.provider_tree [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Inventory has not changed in ProviderTree for provider: 4d291220-7c37-4839-a0e3-b23c66502b74 {{(pid=75234) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Nov 29 03:20:08 user nova-compute[75234]: DEBUG nova.scheduler.client.report [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Inventory has not changed for provider 4d291220-7c37-4839-a0e3-b23c66502b74 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=75234) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Nov 29 03:20:08 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Compute_service record updated for user:user {{(pid=75234) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:1067}}
Nov 29 03:20:08 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.471s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:20:08 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:20:08 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._sync_scheduler_instance_info {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:20:08 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:20:08 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:20:08 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:20:08 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:20:08 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=75234) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10541}}
Nov 29 03:20:09 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-2d09198a-1bdd-420e-8d57-3ce49058592a tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Waiting for libvirt event about the detach of device vdc with device alias virtio-disk2 from instance aaf1c3e3-b73e-42ad-87bd-c5255ec09b46 is timed out.
Nov 29 03:20:09 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-2d09198a-1bdd-420e-8d57-3ce49058592a tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Successfully detached device vdc from instance aaf1c3e3-b73e-42ad-87bd-c5255ec09b46 from the live domain config.
Nov 29 03:20:09 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-2d09198a-1bdd-420e-8d57-3ce49058592a tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] [instance: aaf1c3e3-b73e-42ad-87bd-c5255ec09b46] calling os-brick to detach iSCSI Volume {{(pid=75234) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Nov 29 03:20:09 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-2d09198a-1bdd-420e-8d57-3ce49058592a tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f8a5050c9a0>, {'target_iqn': 'iqn.2010-10.org.openstack:9d734633-75e6-4a97-a3c5-bd9e45d70cca', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'nBvENxIP', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdg'}, None), 'kwargs': False}}" {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:176}}
Nov 29 03:20:09 user nova-compute[75234]: WARNING os_brick.initiator.connectors.base [None req-2d09198a-1bdd-420e-8d57-3ce49058592a tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Nov 29 03:20:09 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-2d09198a-1bdd-420e-8d57-3ce49058592a tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:68}}
Nov 29 03:20:09 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-2d09198a-1bdd-420e-8d57-3ce49058592a tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.002s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:73}}
Nov 29 03:20:09 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:20:09 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.021s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:20:09 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[9702a0d1-def9-46fe-81e7-59ef3198c83e]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:9d734633-75e6-4a97-a3c5-bd9e45d70cca\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:cf6ab229-533a-424e-bd67-1d54d50ad4be\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:bf57b967-7fad-4d76-abb2-319fe88f8ef9\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:20:09 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-2d09198a-1bdd-420e-8d57-3ce49058592a tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS:
Nov 29 03:20:09 user nova-compute[75234]: No targets found.
Nov 29 03:20:09 user nova-compute[75234]: iSNS:
Nov 29 03:20:09 user nova-compute[75234]: No targets found.
Nov 29 03:20:09 user nova-compute[75234]: STATIC:
Nov 29 03:20:09 user nova-compute[75234]: Target: iqn.2010-10.org.openstack:9d734633-75e6-4a97-a3c5-bd9e45d70cca
Nov 29 03:20:09 user nova-compute[75234]:         Portal: 172.16.0.220:3260,-1
Nov 29 03:20:09 user nova-compute[75234]:                 Iface Name: default
Nov 29 03:20:09 user nova-compute[75234]: Target: iqn.2010-10.org.openstack:cf6ab229-533a-424e-bd67-1d54d50ad4be
Nov 29 03:20:09 user nova-compute[75234]:         Portal: 172.16.0.220:3260,-1
Nov 29 03:20:09 user nova-compute[75234]:                 Iface Name: default
Nov 29 03:20:09 user nova-compute[75234]: Target: iqn.2010-10.org.openstack:bf57b967-7fad-4d76-abb2-319fe88f8ef9
Nov 29 03:20:09 user nova-compute[75234]:         Portal: 172.16.0.220:3260,-1
Nov 29 03:20:09 user nova-compute[75234]:                 Iface Name: default
Nov 29 03:20:09 user nova-compute[75234]: FIRMWARE:
Nov 29 03:20:09 user nova-compute[75234]: No targets found.
Nov 29 03:20:09 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:20:09 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-2d09198a-1bdd-420e-8d57-3ce49058592a tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Regex to get portals from discoverydb: ^SENDTARGETS:
Nov 29 03:20:09 user nova-compute[75234]: .*?^DiscoveryAddress: 172.16.0.220,3260.*?
Nov 29 03:20:09 user nova-compute[75234]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=75234) _get_discoverydb_portals /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:382}}
Nov 29 03:20:09 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-2d09198a-1bdd-420e-8d57-3ce49058592a tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:9d734633-75e6-4a97-a3c5-bd9e45d70cca', 0)] {{(pid=75234) _get_connection_devices /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:830}}
Nov 29 03:20:09 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:20:09 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.017s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:20:09 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[b0b66bd0-db8e-4f95-8c83-46eb23d3277b]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:9d734633-75e6-4a97-a3c5-bd9e45d70cca\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:bf57b967-7fad-4d76-abb2-319fe88f8ef9\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:cf6ab229-533a-424e-bd67-1d54d50ad4be\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:20:09 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:20:09 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.024s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:20:09 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[c566b62b-6375-40b9-9f92-8af83d37845d]: (4, ('tcp: [29] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bf57b967-7fad-4d76-abb2-319fe88f8ef9 (non-flash)\ntcp: [36] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9d734633-75e6-4a97-a3c5-bd9e45d70cca (non-flash)\ntcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:cf6ab229-533a-424e-bd67-1d54d50ad4be (non-flash)\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:20:09 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-2d09198a-1bdd-420e-8d57-3ce49058592a tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [29] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bf57b967-7fad-4d76-abb2-319fe88f8ef9 (non-flash)
Nov 29 03:20:09 user nova-compute[75234]: tcp: [36] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9d734633-75e6-4a97-a3c5-bd9e45d70cca (non-flash)
Nov 29 03:20:09 user nova-compute[75234]: tcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:cf6ab229-533a-424e-bd67-1d54d50ad4be (non-flash)
Nov 29 03:20:09 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:20:09 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-2d09198a-1bdd-420e-8d57-3ce49058592a tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] iscsi session list stdout=tcp: [29] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bf57b967-7fad-4d76-abb2-319fe88f8ef9 (non-flash)
Nov 29 03:20:09 user nova-compute[75234]: tcp: [36] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:9d734633-75e6-4a97-a3c5-bd9e45d70cca (non-flash)
Nov 29 03:20:09 user nova-compute[75234]: tcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:cf6ab229-533a-424e-bd67-1d54d50ad4be (non-flash)
Nov 29 03:20:09 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsi_session /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Nov 29 03:20:09 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-2d09198a-1bdd-420e-8d57-3ce49058592a tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7f8a682fdcf0>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:9d734633-75e6-4a97-a3c5-bd9e45d70cca'): ({'sdg'}, set())}) {{(pid=75234) _get_connection_devices /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:863}}
Nov 29 03:20:09 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-2d09198a-1bdd-420e-8d57-3ce49058592a tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Removing single pathed devices sdg {{(pid=75234) remove_connection /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:374}}
Nov 29 03:20:09 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:20:09 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.020s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:20:09 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[4040edad-409a-4874-a0ef-d7f0ee6c3428]: (4, ('path checker states:\nup                  3\n\npaths: 0\nbusy: False\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:20:09 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdg {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:20:09 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdg" returned: 0 in 0.019s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:20:09 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[e877c5fc-854a-4b9d-abb0-8b56e6d5229f]: (4, ('ok\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:20:09 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-2d09198a-1bdd-420e-8d57-3ce49058592a tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Flushing IO for device /dev/sdg {{(pid=75234) flush_device_io /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:434}}
Nov 29 03:20:09 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdg {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:20:09 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdg" returned: 0 in 0.017s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:20:09 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[6871f9e9-94a6-4f18-b21f-1299dfe897a2]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:20:09 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-2d09198a-1bdd-420e-8d57-3ce49058592a tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Remove SCSI device /dev/sdg with /sys/block/sdg/device/delete {{(pid=75234) remove_scsi_device /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:148}}
Nov 29 03:20:09 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdg/device/delete {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:20:09 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdg/device/delete" returned: 0 in 0.035s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:20:09 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[c635cbda-899a-4b78-81a4-335da229ddd0]: (4, ('1', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:20:09 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-2d09198a-1bdd-420e-8d57-3ce49058592a tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Checking to see if SCSI volumes sdg have been removed. {{(pid=75234) wait_for_volumes_removal /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:156}}
Nov 29 03:20:09 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-2d09198a-1bdd-420e-8d57-3ce49058592a tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] SCSI volumes sdg have been removed. {{(pid=75234) wait_for_volumes_removal /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:166}}
Nov 29 03:20:09 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-2d09198a-1bdd-420e-8d57-3ce49058592a tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:9d734633-75e6-4a97-a3c5-bd9e45d70cca')] {{(pid=75234) _disconnect_connection /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Nov 29 03:20:09 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:9d734633-75e6-4a97-a3c5-bd9e45d70cca -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:20:09 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:9d734633-75e6-4a97-a3c5-bd9e45d70cca -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.019s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:20:09 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[b858be85-7694-4e80-8913-61a9fdbe66dc]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:20:09 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-2d09198a-1bdd-420e-8d57-3ce49058592a tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:20:09 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:9d734633-75e6-4a97-a3c5-bd9e45d70cca -p 172.16.0.220:3260 --logout {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:20:09 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:9d734633-75e6-4a97-a3c5-bd9e45d70cca -p 172.16.0.220:3260 --logout" returned: 0 in 0.077s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:20:09 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[329e347a-8d11-4a9f-8600-edcded779e3f]: (4, ('Logging out of session [sid: 36, target: iqn.2010-10.org.openstack:9d734633-75e6-4a97-a3c5-bd9e45d70cca, portal: 172.16.0.220,3260]\nLogout of [sid: 36, target: iqn.2010-10.org.openstack:9d734633-75e6-4a97-a3c5-bd9e45d70cca, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:20:09 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-2d09198a-1bdd-420e-8d57-3ce49058592a tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 36, target: iqn.2010-10.org.openstack:9d734633-75e6-4a97-a3c5-bd9e45d70cca, portal: 172.16.0.220,3260]
Nov 29 03:20:09 user nova-compute[75234]: Logout of [sid: 36, target: iqn.2010-10.org.openstack:9d734633-75e6-4a97-a3c5-bd9e45d70cca, portal: 172.16.0.220,3260] successful.
Nov 29 03:20:09 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:20:09 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:9d734633-75e6-4a97-a3c5-bd9e45d70cca -p 172.16.0.220:3260 --op delete {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:20:09 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:9d734633-75e6-4a97-a3c5-bd9e45d70cca -p 172.16.0.220:3260 --op delete" returned: 0 in 0.023s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:20:09 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[136b7bb8-acee-4441-bbb3-c0ceb2855892]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:20:09 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-2d09198a-1bdd-420e-8d57-3ce49058592a tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:20:09 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-2d09198a-1bdd-420e-8d57-3ce49058592a tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.319s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:87}}
Nov 29 03:20:09 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-2d09198a-1bdd-420e-8d57-3ce49058592a tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] <== disconnect_volume: return (321ms) None {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:203}}
Nov 29 03:20:09 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-2d09198a-1bdd-420e-8d57-3ce49058592a tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] [instance: aaf1c3e3-b73e-42ad-87bd-c5255ec09b46] Disconnected iSCSI Volume {{(pid=75234) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:79}}
Nov 29 03:20:10 user nova-compute[75234]: DEBUG nova.objects.instance [None req-2d09198a-1bdd-420e-8d57-3ce49058592a tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Lazy-loading 'flavor' on Instance uuid aaf1c3e3-b73e-42ad-87bd-c5255ec09b46 {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:20:10 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-2d09198a-1bdd-420e-8d57-3ce49058592a tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Lock "aaf1c3e3-b73e-42ad-87bd-c5255ec09b46" "released" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: held 21.997s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:20:11 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:20:12 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:20:12 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-f9f01322-3491-44ae-a0e6-e915722645eb tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Acquiring lock "aaf1c3e3-b73e-42ad-87bd-c5255ec09b46" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:20:12 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-f9f01322-3491-44ae-a0e6-e915722645eb tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Lock "aaf1c3e3-b73e-42ad-87bd-c5255ec09b46" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:20:12 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-f9f01322-3491-44ae-a0e6-e915722645eb tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Acquiring lock "aaf1c3e3-b73e-42ad-87bd-c5255ec09b46-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:20:12 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-f9f01322-3491-44ae-a0e6-e915722645eb tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Lock "aaf1c3e3-b73e-42ad-87bd-c5255ec09b46-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:20:12 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-f9f01322-3491-44ae-a0e6-e915722645eb tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Lock "aaf1c3e3-b73e-42ad-87bd-c5255ec09b46-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:20:12 user nova-compute[75234]: INFO nova.compute.manager [None req-f9f01322-3491-44ae-a0e6-e915722645eb tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] [instance: aaf1c3e3-b73e-42ad-87bd-c5255ec09b46] Terminating instance
Nov 29 03:20:12 user nova-compute[75234]: DEBUG nova.compute.manager [None req-f9f01322-3491-44ae-a0e6-e915722645eb tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] [instance: aaf1c3e3-b73e-42ad-87bd-c5255ec09b46] Start destroying the instance on the hypervisor. {{(pid=75234) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3140}}
Nov 29 03:20:12 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:20:12 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:20:12 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:20:12 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:20:12 user nova-compute[75234]: DEBUG nova.compute.manager [req-b4d3c4cd-0c1e-4f91-9b68-c4bd57a145df req-e3e06c10-e5ea-4c03-ad8e-56f61cc4b15c service nova] [instance: aaf1c3e3-b73e-42ad-87bd-c5255ec09b46] Received event network-vif-unplugged-0013653c-9540-4049-91a5-43bfba2d8fa3 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:20:12 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-b4d3c4cd-0c1e-4f91-9b68-c4bd57a145df req-e3e06c10-e5ea-4c03-ad8e-56f61cc4b15c service nova] Acquiring lock "aaf1c3e3-b73e-42ad-87bd-c5255ec09b46-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:20:12 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-b4d3c4cd-0c1e-4f91-9b68-c4bd57a145df req-e3e06c10-e5ea-4c03-ad8e-56f61cc4b15c service nova] Lock "aaf1c3e3-b73e-42ad-87bd-c5255ec09b46-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:20:12 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-b4d3c4cd-0c1e-4f91-9b68-c4bd57a145df req-e3e06c10-e5ea-4c03-ad8e-56f61cc4b15c service nova] Lock "aaf1c3e3-b73e-42ad-87bd-c5255ec09b46-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:20:12 user nova-compute[75234]: DEBUG nova.compute.manager [req-b4d3c4cd-0c1e-4f91-9b68-c4bd57a145df req-e3e06c10-e5ea-4c03-ad8e-56f61cc4b15c service nova] [instance: aaf1c3e3-b73e-42ad-87bd-c5255ec09b46] No waiting events found dispatching network-vif-unplugged-0013653c-9540-4049-91a5-43bfba2d8fa3 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:20:12 user nova-compute[75234]: DEBUG nova.compute.manager [req-b4d3c4cd-0c1e-4f91-9b68-c4bd57a145df req-e3e06c10-e5ea-4c03-ad8e-56f61cc4b15c service nova] [instance: aaf1c3e3-b73e-42ad-87bd-c5255ec09b46] Received event network-vif-unplugged-0013653c-9540-4049-91a5-43bfba2d8fa3 for instance with task_state deleting. {{(pid=75234) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10890}}
Nov 29 03:20:13 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:20:13 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:20:13 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:20:13 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:20:13 user nova-compute[75234]: INFO nova.virt.libvirt.driver [-] [instance: aaf1c3e3-b73e-42ad-87bd-c5255ec09b46] Instance destroyed successfully.
Nov 29 03:20:13 user nova-compute[75234]: DEBUG nova.objects.instance [None req-f9f01322-3491-44ae-a0e6-e915722645eb tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Lazy-loading 'resources' on Instance uuid aaf1c3e3-b73e-42ad-87bd-c5255ec09b46 {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:20:13 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-f9f01322-3491-44ae-a0e6-e915722645eb tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-11-29T03:16:58Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-AttachVolumeTestJSON-server-1923356075',display_name='tempest-AttachVolumeTestJSON-server-1923356075',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-attachvolumetestjson-server-1923356075',id=30,image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBOVttVI5swLnmUGLH6ytI05vQQ9gCXIHGezkcU9BBKe3iu2kjwDJGQh46ZdqIMONVOydyYiZFs3183QkjyTIDlELaxy8iT14txRIwVua8qDH79nvvJMG4vqm5zaxFrPXxA==',key_name='tempest-keypair-2054417748',keypairs=<?>,launch_index=0,launched_at=2023-11-29T03:17:05Z,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='e393d23e783b4a40986674d782606c69',ramdisk_id='',reservation_id='r-pfek0fzf',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-AttachVolumeTestJSON-1911423306',owner_user_name='tempest-AttachVolumeTestJSON-1911423306-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-11-29T03:17:06Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='bb1126c998ee430baac2e2494f30c0aa',uuid=aaf1c3e3-b73e-42ad-87bd-c5255ec09b46,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "0013653c-9540-4049-91a5-43bfba2d8fa3", "address": "fa:16:3e:55:a5:13", "network": {"id": "294c3aac-8c9b-45f0-8c0a-344e2d7d8b7c", "bridge": "br-int", "label": "tempest-AttachVolumeTestJSON-700098935-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.209", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "e393d23e783b4a40986674d782606c69", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap0013653c-95", "ovs_interfaceid": "0013653c-9540-4049-91a5-43bfba2d8fa3", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Nov 29 03:20:13 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-f9f01322-3491-44ae-a0e6-e915722645eb tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Converting VIF {"id": "0013653c-9540-4049-91a5-43bfba2d8fa3", "address": "fa:16:3e:55:a5:13", "network": {"id": "294c3aac-8c9b-45f0-8c0a-344e2d7d8b7c", "bridge": "br-int", "label": "tempest-AttachVolumeTestJSON-700098935-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.209", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "e393d23e783b4a40986674d782606c69", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap0013653c-95", "ovs_interfaceid": "0013653c-9540-4049-91a5-43bfba2d8fa3", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:20:13 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-f9f01322-3491-44ae-a0e6-e915722645eb tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:55:a5:13,bridge_name='br-int',has_traffic_filtering=True,id=0013653c-9540-4049-91a5-43bfba2d8fa3,network=Network(294c3aac-8c9b-45f0-8c0a-344e2d7d8b7c),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap0013653c-95') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:20:13 user nova-compute[75234]: DEBUG os_vif [None req-f9f01322-3491-44ae-a0e6-e915722645eb tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Unplugging vif VIFOpenVSwitch(active=True,address=fa:16:3e:55:a5:13,bridge_name='br-int',has_traffic_filtering=True,id=0013653c-9540-4049-91a5-43bfba2d8fa3,network=Network(294c3aac-8c9b-45f0-8c0a-344e2d7d8b7c),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap0013653c-95') {{(pid=75234) unplug /opt/stack/data/venv/lib/python3.10/site-packages/os_vif/__init__.py:109}}
Nov 29 03:20:13 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:20:13 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tap0013653c-95, bridge=br-int, if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:20:13 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:20:13 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:20:13 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:20:13 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbDestroyCommand(_result=None, table=QoS, record=f10fccc7-c33a-422a-a9b5-1c23592c935c) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:20:13 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:20:13 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:20:13 user nova-compute[75234]: INFO os_vif [None req-f9f01322-3491-44ae-a0e6-e915722645eb tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Successfully unplugged vif VIFOpenVSwitch(active=True,address=fa:16:3e:55:a5:13,bridge_name='br-int',has_traffic_filtering=True,id=0013653c-9540-4049-91a5-43bfba2d8fa3,network=Network(294c3aac-8c9b-45f0-8c0a-344e2d7d8b7c),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap0013653c-95')
Nov 29 03:20:13 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-f9f01322-3491-44ae-a0e6-e915722645eb tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] [instance: aaf1c3e3-b73e-42ad-87bd-c5255ec09b46] Deleting instance files /opt/stack/data/nova/instances/aaf1c3e3-b73e-42ad-87bd-c5255ec09b46_del
Nov 29 03:20:13 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-f9f01322-3491-44ae-a0e6-e915722645eb tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] [instance: aaf1c3e3-b73e-42ad-87bd-c5255ec09b46] Deletion of /opt/stack/data/nova/instances/aaf1c3e3-b73e-42ad-87bd-c5255ec09b46_del complete
Nov 29 03:20:13 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:20:13 user nova-compute[75234]: INFO nova.compute.manager [None req-f9f01322-3491-44ae-a0e6-e915722645eb tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] [instance: aaf1c3e3-b73e-42ad-87bd-c5255ec09b46] Took 1.02 seconds to destroy the instance on the hypervisor.
Nov 29 03:20:13 user nova-compute[75234]: DEBUG oslo.service.loopingcall [None req-f9f01322-3491-44ae-a0e6-e915722645eb tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=75234) func /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/loopingcall.py:435}}
Nov 29 03:20:13 user nova-compute[75234]: DEBUG nova.compute.manager [-] [instance: aaf1c3e3-b73e-42ad-87bd-c5255ec09b46] Deallocating network for instance {{(pid=75234) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2279}}
Nov 29 03:20:13 user nova-compute[75234]: DEBUG nova.network.neutron [-] [instance: aaf1c3e3-b73e-42ad-87bd-c5255ec09b46] deallocate_for_instance() {{(pid=75234) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1802}}
Nov 29 03:20:13 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:20:13 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:20:13 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:20:13 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:20:14 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:20:14 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:20:14 user nova-compute[75234]: DEBUG nova.network.neutron [-] [instance: aaf1c3e3-b73e-42ad-87bd-c5255ec09b46] Updating instance_info_cache with network_info: [] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:20:14 user nova-compute[75234]: DEBUG nova.compute.manager [req-273f9cb1-5b9b-4cec-8b81-0a51d72c6b1a req-105f21f3-abad-4e32-b134-5ce469e623f6 service nova] [instance: aaf1c3e3-b73e-42ad-87bd-c5255ec09b46] Received event network-vif-deleted-0013653c-9540-4049-91a5-43bfba2d8fa3 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:20:14 user nova-compute[75234]: INFO nova.compute.manager [req-273f9cb1-5b9b-4cec-8b81-0a51d72c6b1a req-105f21f3-abad-4e32-b134-5ce469e623f6 service nova] [instance: aaf1c3e3-b73e-42ad-87bd-c5255ec09b46] Neutron deleted interface 0013653c-9540-4049-91a5-43bfba2d8fa3; detaching it from the instance and deleting it from the info cache
Nov 29 03:20:14 user nova-compute[75234]: DEBUG nova.network.neutron [req-273f9cb1-5b9b-4cec-8b81-0a51d72c6b1a req-105f21f3-abad-4e32-b134-5ce469e623f6 service nova] [instance: aaf1c3e3-b73e-42ad-87bd-c5255ec09b46] Updating instance_info_cache with network_info: [] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:20:14 user nova-compute[75234]: INFO nova.compute.manager [-] [instance: aaf1c3e3-b73e-42ad-87bd-c5255ec09b46] Took 1.00 seconds to deallocate network for instance.
Nov 29 03:20:14 user nova-compute[75234]: DEBUG nova.compute.manager [req-273f9cb1-5b9b-4cec-8b81-0a51d72c6b1a req-105f21f3-abad-4e32-b134-5ce469e623f6 service nova] [instance: aaf1c3e3-b73e-42ad-87bd-c5255ec09b46] Detach interface failed, port_id=0013653c-9540-4049-91a5-43bfba2d8fa3, reason: Instance aaf1c3e3-b73e-42ad-87bd-c5255ec09b46 could not be found. {{(pid=75234) _process_instance_vif_deleted_event /opt/stack/nova/nova/compute/manager.py:10946}}
Nov 29 03:20:14 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-f9f01322-3491-44ae-a0e6-e915722645eb tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:20:14 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-f9f01322-3491-44ae-a0e6-e915722645eb tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:20:14 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-2f5399a7-4261-4e4b-908f-67a229c77d82 tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] Acquiring lock "946b919b-39ae-4d23-aefd-01952f4b40b6" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:20:14 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-2f5399a7-4261-4e4b-908f-67a229c77d82 tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] Lock "946b919b-39ae-4d23-aefd-01952f4b40b6" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:20:14 user nova-compute[75234]: DEBUG nova.compute.provider_tree [None req-f9f01322-3491-44ae-a0e6-e915722645eb tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Inventory has not changed in ProviderTree for provider: 4d291220-7c37-4839-a0e3-b23c66502b74 {{(pid=75234) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Nov 29 03:20:14 user nova-compute[75234]: DEBUG nova.compute.manager [None req-2f5399a7-4261-4e4b-908f-67a229c77d82 tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] [instance: 946b919b-39ae-4d23-aefd-01952f4b40b6] Starting instance... {{(pid=75234) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2422}}
Nov 29 03:20:14 user nova-compute[75234]: DEBUG nova.scheduler.client.report [None req-f9f01322-3491-44ae-a0e6-e915722645eb tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Inventory has not changed for provider 4d291220-7c37-4839-a0e3-b23c66502b74 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=75234) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Nov 29 03:20:14 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-f9f01322-3491-44ae-a0e6-e915722645eb tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.404s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:20:14 user nova-compute[75234]: INFO nova.scheduler.client.report [None req-f9f01322-3491-44ae-a0e6-e915722645eb tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Deleted allocations for instance aaf1c3e3-b73e-42ad-87bd-c5255ec09b46
Nov 29 03:20:14 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-2f5399a7-4261-4e4b-908f-67a229c77d82 tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:20:14 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-2f5399a7-4261-4e4b-908f-67a229c77d82 tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:20:14 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-2f5399a7-4261-4e4b-908f-67a229c77d82 tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=75234) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2366}}
Nov 29 03:20:14 user nova-compute[75234]: INFO nova.compute.claims [None req-2f5399a7-4261-4e4b-908f-67a229c77d82 tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] [instance: 946b919b-39ae-4d23-aefd-01952f4b40b6] Claim successful on node user
Nov 29 03:20:14 user nova-compute[75234]: DEBUG nova.compute.manager [req-c6a77c1e-5e88-4a00-adfb-3bf138a52cae req-fd95c9cc-d241-4f0c-b38f-18af30d7f05f service nova] [instance: aaf1c3e3-b73e-42ad-87bd-c5255ec09b46] Received event network-vif-plugged-0013653c-9540-4049-91a5-43bfba2d8fa3 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:20:14 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-c6a77c1e-5e88-4a00-adfb-3bf138a52cae req-fd95c9cc-d241-4f0c-b38f-18af30d7f05f service nova] Acquiring lock "aaf1c3e3-b73e-42ad-87bd-c5255ec09b46-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:20:14 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-c6a77c1e-5e88-4a00-adfb-3bf138a52cae req-fd95c9cc-d241-4f0c-b38f-18af30d7f05f service nova] Lock "aaf1c3e3-b73e-42ad-87bd-c5255ec09b46-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:20:14 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-c6a77c1e-5e88-4a00-adfb-3bf138a52cae req-fd95c9cc-d241-4f0c-b38f-18af30d7f05f service nova] Lock "aaf1c3e3-b73e-42ad-87bd-c5255ec09b46-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:20:14 user nova-compute[75234]: DEBUG nova.compute.manager [req-c6a77c1e-5e88-4a00-adfb-3bf138a52cae req-fd95c9cc-d241-4f0c-b38f-18af30d7f05f service nova] [instance: aaf1c3e3-b73e-42ad-87bd-c5255ec09b46] No waiting events found dispatching network-vif-plugged-0013653c-9540-4049-91a5-43bfba2d8fa3 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:20:14 user nova-compute[75234]: WARNING nova.compute.manager [req-c6a77c1e-5e88-4a00-adfb-3bf138a52cae req-fd95c9cc-d241-4f0c-b38f-18af30d7f05f service nova] [instance: aaf1c3e3-b73e-42ad-87bd-c5255ec09b46] Received unexpected event network-vif-plugged-0013653c-9540-4049-91a5-43bfba2d8fa3 for instance with vm_state deleted and task_state None.
Nov 29 03:20:14 user nova-compute[75234]: DEBUG nova.compute.manager [req-c6a77c1e-5e88-4a00-adfb-3bf138a52cae req-fd95c9cc-d241-4f0c-b38f-18af30d7f05f service nova] [instance: aaf1c3e3-b73e-42ad-87bd-c5255ec09b46] Received event network-vif-plugged-0013653c-9540-4049-91a5-43bfba2d8fa3 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:20:14 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-c6a77c1e-5e88-4a00-adfb-3bf138a52cae req-fd95c9cc-d241-4f0c-b38f-18af30d7f05f service nova] Acquiring lock "aaf1c3e3-b73e-42ad-87bd-c5255ec09b46-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:20:14 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-c6a77c1e-5e88-4a00-adfb-3bf138a52cae req-fd95c9cc-d241-4f0c-b38f-18af30d7f05f service nova] Lock "aaf1c3e3-b73e-42ad-87bd-c5255ec09b46-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:20:14 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-c6a77c1e-5e88-4a00-adfb-3bf138a52cae req-fd95c9cc-d241-4f0c-b38f-18af30d7f05f service nova] Lock "aaf1c3e3-b73e-42ad-87bd-c5255ec09b46-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:20:14 user nova-compute[75234]: DEBUG nova.compute.manager [req-c6a77c1e-5e88-4a00-adfb-3bf138a52cae req-fd95c9cc-d241-4f0c-b38f-18af30d7f05f service nova] [instance: aaf1c3e3-b73e-42ad-87bd-c5255ec09b46] No waiting events found dispatching network-vif-plugged-0013653c-9540-4049-91a5-43bfba2d8fa3 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:20:14 user nova-compute[75234]: WARNING nova.compute.manager [req-c6a77c1e-5e88-4a00-adfb-3bf138a52cae req-fd95c9cc-d241-4f0c-b38f-18af30d7f05f service nova] [instance: aaf1c3e3-b73e-42ad-87bd-c5255ec09b46] Received unexpected event network-vif-plugged-0013653c-9540-4049-91a5-43bfba2d8fa3 for instance with vm_state deleted and task_state None.
Nov 29 03:20:14 user nova-compute[75234]: DEBUG nova.compute.manager [req-c6a77c1e-5e88-4a00-adfb-3bf138a52cae req-fd95c9cc-d241-4f0c-b38f-18af30d7f05f service nova] [instance: aaf1c3e3-b73e-42ad-87bd-c5255ec09b46] Received event network-vif-plugged-0013653c-9540-4049-91a5-43bfba2d8fa3 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:20:14 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-c6a77c1e-5e88-4a00-adfb-3bf138a52cae req-fd95c9cc-d241-4f0c-b38f-18af30d7f05f service nova] Acquiring lock "aaf1c3e3-b73e-42ad-87bd-c5255ec09b46-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:20:14 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-c6a77c1e-5e88-4a00-adfb-3bf138a52cae req-fd95c9cc-d241-4f0c-b38f-18af30d7f05f service nova] Lock "aaf1c3e3-b73e-42ad-87bd-c5255ec09b46-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:20:14 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-c6a77c1e-5e88-4a00-adfb-3bf138a52cae req-fd95c9cc-d241-4f0c-b38f-18af30d7f05f service nova] Lock "aaf1c3e3-b73e-42ad-87bd-c5255ec09b46-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:20:14 user nova-compute[75234]: DEBUG nova.compute.manager [req-c6a77c1e-5e88-4a00-adfb-3bf138a52cae req-fd95c9cc-d241-4f0c-b38f-18af30d7f05f service nova] [instance: aaf1c3e3-b73e-42ad-87bd-c5255ec09b46] No waiting events found dispatching network-vif-plugged-0013653c-9540-4049-91a5-43bfba2d8fa3 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:20:14 user nova-compute[75234]: WARNING nova.compute.manager [req-c6a77c1e-5e88-4a00-adfb-3bf138a52cae req-fd95c9cc-d241-4f0c-b38f-18af30d7f05f service nova] [instance: aaf1c3e3-b73e-42ad-87bd-c5255ec09b46] Received unexpected event network-vif-plugged-0013653c-9540-4049-91a5-43bfba2d8fa3 for instance with vm_state deleted and task_state None.
Nov 29 03:20:14 user nova-compute[75234]: DEBUG nova.compute.manager [req-c6a77c1e-5e88-4a00-adfb-3bf138a52cae req-fd95c9cc-d241-4f0c-b38f-18af30d7f05f service nova] [instance: aaf1c3e3-b73e-42ad-87bd-c5255ec09b46] Received event network-vif-plugged-0013653c-9540-4049-91a5-43bfba2d8fa3 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:20:14 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-c6a77c1e-5e88-4a00-adfb-3bf138a52cae req-fd95c9cc-d241-4f0c-b38f-18af30d7f05f service nova] Acquiring lock "aaf1c3e3-b73e-42ad-87bd-c5255ec09b46-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:20:14 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-c6a77c1e-5e88-4a00-adfb-3bf138a52cae req-fd95c9cc-d241-4f0c-b38f-18af30d7f05f service nova] Lock "aaf1c3e3-b73e-42ad-87bd-c5255ec09b46-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:20:14 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-c6a77c1e-5e88-4a00-adfb-3bf138a52cae req-fd95c9cc-d241-4f0c-b38f-18af30d7f05f service nova] Lock "aaf1c3e3-b73e-42ad-87bd-c5255ec09b46-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:20:14 user nova-compute[75234]: DEBUG nova.compute.manager [req-c6a77c1e-5e88-4a00-adfb-3bf138a52cae req-fd95c9cc-d241-4f0c-b38f-18af30d7f05f service nova] [instance: aaf1c3e3-b73e-42ad-87bd-c5255ec09b46] No waiting events found dispatching network-vif-plugged-0013653c-9540-4049-91a5-43bfba2d8fa3 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:20:14 user nova-compute[75234]: WARNING nova.compute.manager [req-c6a77c1e-5e88-4a00-adfb-3bf138a52cae req-fd95c9cc-d241-4f0c-b38f-18af30d7f05f service nova] [instance: aaf1c3e3-b73e-42ad-87bd-c5255ec09b46] Received unexpected event network-vif-plugged-0013653c-9540-4049-91a5-43bfba2d8fa3 for instance with vm_state deleted and task_state None.
Nov 29 03:20:14 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-f9f01322-3491-44ae-a0e6-e915722645eb tempest-AttachVolumeTestJSON-1911423306 tempest-AttachVolumeTestJSON-1911423306-project-member] Lock "aaf1c3e3-b73e-42ad-87bd-c5255ec09b46" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 2.652s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:20:15 user nova-compute[75234]: DEBUG nova.compute.provider_tree [None req-2f5399a7-4261-4e4b-908f-67a229c77d82 tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] Inventory has not changed in ProviderTree for provider: 4d291220-7c37-4839-a0e3-b23c66502b74 {{(pid=75234) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Nov 29 03:20:15 user nova-compute[75234]: DEBUG nova.scheduler.client.report [None req-2f5399a7-4261-4e4b-908f-67a229c77d82 tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] Inventory has not changed for provider 4d291220-7c37-4839-a0e3-b23c66502b74 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=75234) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Nov 29 03:20:15 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-2f5399a7-4261-4e4b-908f-67a229c77d82 tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.463s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:20:15 user nova-compute[75234]: DEBUG nova.compute.manager [None req-2f5399a7-4261-4e4b-908f-67a229c77d82 tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] [instance: 946b919b-39ae-4d23-aefd-01952f4b40b6] Start building networks asynchronously for instance. {{(pid=75234) _build_resources /opt/stack/nova/nova/compute/manager.py:2819}}
Nov 29 03:20:15 user nova-compute[75234]: DEBUG nova.compute.manager [None req-2f5399a7-4261-4e4b-908f-67a229c77d82 tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] [instance: 946b919b-39ae-4d23-aefd-01952f4b40b6] Allocating IP information in the background. {{(pid=75234) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1972}}
Nov 29 03:20:15 user nova-compute[75234]: DEBUG nova.network.neutron [None req-2f5399a7-4261-4e4b-908f-67a229c77d82 tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] [instance: 946b919b-39ae-4d23-aefd-01952f4b40b6] allocate_for_instance() {{(pid=75234) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1156}}
Nov 29 03:20:15 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-2f5399a7-4261-4e4b-908f-67a229c77d82 tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] [instance: 946b919b-39ae-4d23-aefd-01952f4b40b6] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Nov 29 03:20:15 user nova-compute[75234]: DEBUG nova.compute.manager [None req-2f5399a7-4261-4e4b-908f-67a229c77d82 tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] [instance: 946b919b-39ae-4d23-aefd-01952f4b40b6] Start building block device mappings for instance. {{(pid=75234) _build_resources /opt/stack/nova/nova/compute/manager.py:2854}}
Nov 29 03:20:15 user nova-compute[75234]: DEBUG nova.policy [None req-2f5399a7-4261-4e4b-908f-67a229c77d82 tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '522c3c107614404fbe3af8c38fdd189e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e28b5baf29b14aa4b2b98cb1b615705f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=75234) authorize /opt/stack/nova/nova/policy.py:203}}
Nov 29 03:20:15 user nova-compute[75234]: DEBUG nova.compute.manager [None req-2f5399a7-4261-4e4b-908f-67a229c77d82 tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] [instance: 946b919b-39ae-4d23-aefd-01952f4b40b6] Start spawning the instance on the hypervisor. {{(pid=75234) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2628}}
Nov 29 03:20:15 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-2f5399a7-4261-4e4b-908f-67a229c77d82 tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] [instance: 946b919b-39ae-4d23-aefd-01952f4b40b6] Creating instance directory {{(pid=75234) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4727}}
Nov 29 03:20:15 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-2f5399a7-4261-4e4b-908f-67a229c77d82 tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] [instance: 946b919b-39ae-4d23-aefd-01952f4b40b6] Creating image(s)
Nov 29 03:20:15 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-2f5399a7-4261-4e4b-908f-67a229c77d82 tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] Acquiring lock "/opt/stack/data/nova/instances/946b919b-39ae-4d23-aefd-01952f4b40b6/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:20:15 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-2f5399a7-4261-4e4b-908f-67a229c77d82 tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] Lock "/opt/stack/data/nova/instances/946b919b-39ae-4d23-aefd-01952f4b40b6/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:20:15 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-2f5399a7-4261-4e4b-908f-67a229c77d82 tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] Lock "/opt/stack/data/nova/instances/946b919b-39ae-4d23-aefd-01952f4b40b6/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.002s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:20:15 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-2f5399a7-4261-4e4b-908f-67a229c77d82 tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:20:15 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-2f5399a7-4261-4e4b-908f-67a229c77d82 tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json" returned: 0 in 0.169s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:20:15 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-2f5399a7-4261-4e4b-908f-67a229c77d82 tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] Acquiring lock "80686e29dc9db6600d03e85cb62af298e12d7918" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:20:15 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-2f5399a7-4261-4e4b-908f-67a229c77d82 tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] Lock "80686e29dc9db6600d03e85cb62af298e12d7918" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.002s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:20:15 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-2f5399a7-4261-4e4b-908f-67a229c77d82 tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:20:15 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-2f5399a7-4261-4e4b-908f-67a229c77d82 tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json" returned: 0 in 0.171s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:20:15 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-2f5399a7-4261-4e4b-908f-67a229c77d82 tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918,backing_fmt=raw /opt/stack/data/nova/instances/946b919b-39ae-4d23-aefd-01952f4b40b6/disk 1073741824 {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:20:15 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-2f5399a7-4261-4e4b-908f-67a229c77d82 tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918,backing_fmt=raw /opt/stack/data/nova/instances/946b919b-39ae-4d23-aefd-01952f4b40b6/disk 1073741824" returned: 0 in 0.046s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:20:15 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-2f5399a7-4261-4e4b-908f-67a229c77d82 tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] Lock "80686e29dc9db6600d03e85cb62af298e12d7918" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.224s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:20:15 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-2f5399a7-4261-4e4b-908f-67a229c77d82 tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:20:16 user nova-compute[75234]: DEBUG nova.network.neutron [None req-2f5399a7-4261-4e4b-908f-67a229c77d82 tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] [instance: 946b919b-39ae-4d23-aefd-01952f4b40b6] Successfully created port: e1f71ce6-58f3-4665-afc5-f92d1b6801c5 {{(pid=75234) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:548}}
Nov 29 03:20:16 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-2f5399a7-4261-4e4b-908f-67a229c77d82 tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json" returned: 0 in 0.193s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:20:16 user nova-compute[75234]: DEBUG nova.virt.disk.api [None req-2f5399a7-4261-4e4b-908f-67a229c77d82 tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] Checking if we can resize image /opt/stack/data/nova/instances/946b919b-39ae-4d23-aefd-01952f4b40b6/disk. size=1073741824 {{(pid=75234) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:166}}
Nov 29 03:20:16 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-2f5399a7-4261-4e4b-908f-67a229c77d82 tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/946b919b-39ae-4d23-aefd-01952f4b40b6/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:20:16 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-2f5399a7-4261-4e4b-908f-67a229c77d82 tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/946b919b-39ae-4d23-aefd-01952f4b40b6/disk --force-share --output=json" returned: 0 in 0.189s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:20:16 user nova-compute[75234]: DEBUG nova.virt.disk.api [None req-2f5399a7-4261-4e4b-908f-67a229c77d82 tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] Cannot resize image /opt/stack/data/nova/instances/946b919b-39ae-4d23-aefd-01952f4b40b6/disk to a smaller size. {{(pid=75234) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:172}}
Nov 29 03:20:16 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-2f5399a7-4261-4e4b-908f-67a229c77d82 tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] [instance: 946b919b-39ae-4d23-aefd-01952f4b40b6] Created local disks {{(pid=75234) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4861}}
Nov 29 03:20:16 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-2f5399a7-4261-4e4b-908f-67a229c77d82 tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] [instance: 946b919b-39ae-4d23-aefd-01952f4b40b6] Ensure instance console log exists: /opt/stack/data/nova/instances/946b919b-39ae-4d23-aefd-01952f4b40b6/console.log {{(pid=75234) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4613}}
Nov 29 03:20:16 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-2f5399a7-4261-4e4b-908f-67a229c77d82 tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:20:16 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-2f5399a7-4261-4e4b-908f-67a229c77d82 tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:20:16 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-2f5399a7-4261-4e4b-908f-67a229c77d82 tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:20:16 user nova-compute[75234]: DEBUG nova.network.neutron [None req-2f5399a7-4261-4e4b-908f-67a229c77d82 tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] [instance: 946b919b-39ae-4d23-aefd-01952f4b40b6] Successfully updated port: e1f71ce6-58f3-4665-afc5-f92d1b6801c5 {{(pid=75234) _update_port /opt/stack/nova/nova/network/neutron.py:586}}
Nov 29 03:20:16 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-2f5399a7-4261-4e4b-908f-67a229c77d82 tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] Acquiring lock "refresh_cache-946b919b-39ae-4d23-aefd-01952f4b40b6" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:20:16 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-2f5399a7-4261-4e4b-908f-67a229c77d82 tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] Acquired lock "refresh_cache-946b919b-39ae-4d23-aefd-01952f4b40b6" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:20:16 user nova-compute[75234]: DEBUG nova.network.neutron [None req-2f5399a7-4261-4e4b-908f-67a229c77d82 tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] [instance: 946b919b-39ae-4d23-aefd-01952f4b40b6] Building network info cache for instance {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2009}}
Nov 29 03:20:16 user nova-compute[75234]: DEBUG nova.compute.manager [req-2089ebfc-6aa1-48d0-a444-86200772d24b req-411c9c17-5ac6-4c93-a0a5-08f665c1c32c service nova] [instance: 946b919b-39ae-4d23-aefd-01952f4b40b6] Received event network-changed-e1f71ce6-58f3-4665-afc5-f92d1b6801c5 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:20:16 user nova-compute[75234]: DEBUG nova.compute.manager [req-2089ebfc-6aa1-48d0-a444-86200772d24b req-411c9c17-5ac6-4c93-a0a5-08f665c1c32c service nova] [instance: 946b919b-39ae-4d23-aefd-01952f4b40b6] Refreshing instance network info cache due to event network-changed-e1f71ce6-58f3-4665-afc5-f92d1b6801c5. {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11117}}
Nov 29 03:20:16 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-2089ebfc-6aa1-48d0-a444-86200772d24b req-411c9c17-5ac6-4c93-a0a5-08f665c1c32c service nova] Acquiring lock "refresh_cache-946b919b-39ae-4d23-aefd-01952f4b40b6" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:20:16 user nova-compute[75234]: DEBUG nova.network.neutron [None req-2f5399a7-4261-4e4b-908f-67a229c77d82 tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] [instance: 946b919b-39ae-4d23-aefd-01952f4b40b6] Instance cache missing network info. {{(pid=75234) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3322}}
Nov 29 03:20:16 user nova-compute[75234]: DEBUG nova.compute.manager [req-4fb5eaa1-d795-436b-9e5e-8e36698e68c6 req-1d33db4e-0b5e-45b0-8111-43f5d40d424c service nova] [instance: 6894b55f-93c6-43c9-8933-56d5a3c43725] Received event network-changed-9b2cf46e-008c-4ef3-9555-0cbc584dcf8f {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:20:16 user nova-compute[75234]: DEBUG nova.compute.manager [req-4fb5eaa1-d795-436b-9e5e-8e36698e68c6 req-1d33db4e-0b5e-45b0-8111-43f5d40d424c service nova] [instance: 6894b55f-93c6-43c9-8933-56d5a3c43725] Refreshing instance network info cache due to event network-changed-9b2cf46e-008c-4ef3-9555-0cbc584dcf8f. {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11117}}
Nov 29 03:20:16 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-4fb5eaa1-d795-436b-9e5e-8e36698e68c6 req-1d33db4e-0b5e-45b0-8111-43f5d40d424c service nova] Acquiring lock "refresh_cache-6894b55f-93c6-43c9-8933-56d5a3c43725" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:20:16 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-4fb5eaa1-d795-436b-9e5e-8e36698e68c6 req-1d33db4e-0b5e-45b0-8111-43f5d40d424c service nova] Acquired lock "refresh_cache-6894b55f-93c6-43c9-8933-56d5a3c43725" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:20:16 user nova-compute[75234]: DEBUG nova.network.neutron [req-4fb5eaa1-d795-436b-9e5e-8e36698e68c6 req-1d33db4e-0b5e-45b0-8111-43f5d40d424c service nova] [instance: 6894b55f-93c6-43c9-8933-56d5a3c43725] Refreshing network info cache for port 9b2cf46e-008c-4ef3-9555-0cbc584dcf8f {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2006}}
Nov 29 03:20:17 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-515241ec-541d-4828-82f7-7ad5f08ca57e tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Acquiring lock "6894b55f-93c6-43c9-8933-56d5a3c43725" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:20:17 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-515241ec-541d-4828-82f7-7ad5f08ca57e tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Lock "6894b55f-93c6-43c9-8933-56d5a3c43725" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:20:17 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-515241ec-541d-4828-82f7-7ad5f08ca57e tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Acquiring lock "6894b55f-93c6-43c9-8933-56d5a3c43725-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:20:17 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-515241ec-541d-4828-82f7-7ad5f08ca57e tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Lock "6894b55f-93c6-43c9-8933-56d5a3c43725-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:20:17 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-515241ec-541d-4828-82f7-7ad5f08ca57e tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Lock "6894b55f-93c6-43c9-8933-56d5a3c43725-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:20:17 user nova-compute[75234]: INFO nova.compute.manager [None req-515241ec-541d-4828-82f7-7ad5f08ca57e tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] [instance: 6894b55f-93c6-43c9-8933-56d5a3c43725] Terminating instance
Nov 29 03:20:17 user nova-compute[75234]: DEBUG nova.compute.manager [None req-515241ec-541d-4828-82f7-7ad5f08ca57e tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] [instance: 6894b55f-93c6-43c9-8933-56d5a3c43725] Start destroying the instance on the hypervisor. {{(pid=75234) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3140}}
Nov 29 03:20:17 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:20:17 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:20:17 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:20:17 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:20:17 user nova-compute[75234]: DEBUG nova.network.neutron [None req-2f5399a7-4261-4e4b-908f-67a229c77d82 tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] [instance: 946b919b-39ae-4d23-aefd-01952f4b40b6] Updating instance_info_cache with network_info: [{"id": "e1f71ce6-58f3-4665-afc5-f92d1b6801c5", "address": "fa:16:3e:81:ca:ab", "network": {"id": "93ca21e4-138b-4f00-8d1d-b30d63dbe7aa", "bridge": "br-int", "label": "tempest-TaggedAttachmentsTest-1844288661-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "e28b5baf29b14aa4b2b98cb1b615705f", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape1f71ce6-58", "ovs_interfaceid": "e1f71ce6-58f3-4665-afc5-f92d1b6801c5", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:20:17 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:20:17 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-2f5399a7-4261-4e4b-908f-67a229c77d82 tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] Releasing lock "refresh_cache-946b919b-39ae-4d23-aefd-01952f4b40b6" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:20:17 user nova-compute[75234]: DEBUG nova.compute.manager [None req-2f5399a7-4261-4e4b-908f-67a229c77d82 tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] [instance: 946b919b-39ae-4d23-aefd-01952f4b40b6] Instance network_info: |[{"id": "e1f71ce6-58f3-4665-afc5-f92d1b6801c5", "address": "fa:16:3e:81:ca:ab", "network": {"id": "93ca21e4-138b-4f00-8d1d-b30d63dbe7aa", "bridge": "br-int", "label": "tempest-TaggedAttachmentsTest-1844288661-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "e28b5baf29b14aa4b2b98cb1b615705f", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape1f71ce6-58", "ovs_interfaceid": "e1f71ce6-58f3-4665-afc5-f92d1b6801c5", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=75234) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1987}}
Nov 29 03:20:17 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-2089ebfc-6aa1-48d0-a444-86200772d24b req-411c9c17-5ac6-4c93-a0a5-08f665c1c32c service nova] Acquired lock "refresh_cache-946b919b-39ae-4d23-aefd-01952f4b40b6" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:20:17 user nova-compute[75234]: DEBUG nova.network.neutron [req-2089ebfc-6aa1-48d0-a444-86200772d24b req-411c9c17-5ac6-4c93-a0a5-08f665c1c32c service nova] [instance: 946b919b-39ae-4d23-aefd-01952f4b40b6] Refreshing network info cache for port e1f71ce6-58f3-4665-afc5-f92d1b6801c5 {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2006}}
Nov 29 03:20:17 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-2f5399a7-4261-4e4b-908f-67a229c77d82 tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] [instance: 946b919b-39ae-4d23-aefd-01952f4b40b6] Start _get_guest_xml network_info=[{"id": "e1f71ce6-58f3-4665-afc5-f92d1b6801c5", "address": "fa:16:3e:81:ca:ab", "network": {"id": "93ca21e4-138b-4f00-8d1d-b30d63dbe7aa", "bridge": "br-int", "label": "tempest-TaggedAttachmentsTest-1844288661-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "e28b5baf29b14aa4b2b98cb1b615705f", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape1f71ce6-58", "ovs_interfaceid": "e1f71ce6-58f3-4665-afc5-f92d1b6801c5", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk.config': {'bus': 'ide', 'dev': 'hda', 'type': 'cdrom'}}} image_meta=ImageMeta(checksum='c8fc807773e5354afe61636071771906',container_format='bare',created_at=2023-11-29T03:02:15Z,direct_url=<?>,disk_format='qcow2',id=96c30ba7-9b73-4a30-8ba4-7138879573b7,min_disk=0,min_ram=0,name='cirros-0.6.2-x86_64-disk',owner='b18e380921c748b78517b6f1211cd0f1',properties=ImageMetaProps,protected=<?>,size=21430272,status='active',tags=<?>,updated_at=2023-11-29T03:02:17Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [{'device_name': '/dev/vda', 'encryption_format': None, 'encrypted': False, 'device_type': 'disk', 'size': 0, 'disk_bus': 'virtio', 'boot_index': 0, 'encryption_options': None, 'encryption_secret_uuid': None, 'guest_format': None, 'image_id': '96c30ba7-9b73-4a30-8ba4-7138879573b7'}], 'ephemerals': [], 'block_device_mapping': [], 'swap': None} {{(pid=75234) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7562}}
Nov 29 03:20:17 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-2f5399a7-4261-4e4b-908f-67a229c77d82 tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:20:17 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-2f5399a7-4261-4e4b-908f-67a229c77d82 tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:20:17 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-2f5399a7-4261-4e4b-908f-67a229c77d82 tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] Searching host: 'user' for CPU controller through CGroups V1... {{(pid=75234) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1652}}
Nov 29 03:20:17 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-2f5399a7-4261-4e4b-908f-67a229c77d82 tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] CPU controller missing on host. {{(pid=75234) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1662}}
Nov 29 03:20:17 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-2f5399a7-4261-4e4b-908f-67a229c77d82 tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] Searching host: 'user' for CPU controller through CGroups V2... {{(pid=75234) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1671}}
Nov 29 03:20:17 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-2f5399a7-4261-4e4b-908f-67a229c77d82 tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] CPU controller found on host. {{(pid=75234) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1678}}
Nov 29 03:20:17 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-2f5399a7-4261-4e4b-908f-67a229c77d82 tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=75234) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5400}}
Nov 29 03:20:17 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-2f5399a7-4261-4e4b-908f-67a229c77d82 tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-11-29T03:02:04Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='1',id=6,is_public=True,memory_mb=512,name='m1.tiny',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='c8fc807773e5354afe61636071771906',container_format='bare',created_at=2023-11-29T03:02:15Z,direct_url=<?>,disk_format='qcow2',id=96c30ba7-9b73-4a30-8ba4-7138879573b7,min_disk=0,min_ram=0,name='cirros-0.6.2-x86_64-disk',owner='b18e380921c748b78517b6f1211cd0f1',properties=ImageMetaProps,protected=<?>,size=21430272,status='active',tags=<?>,updated_at=2023-11-29T03:02:17Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:563}}
Nov 29 03:20:17 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-2f5399a7-4261-4e4b-908f-67a229c77d82 tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] Flavor limits 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:348}}
Nov 29 03:20:17 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-2f5399a7-4261-4e4b-908f-67a229c77d82 tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] Image limits 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:352}}
Nov 29 03:20:17 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-2f5399a7-4261-4e4b-908f-67a229c77d82 tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] Flavor pref 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:388}}
Nov 29 03:20:17 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-2f5399a7-4261-4e4b-908f-67a229c77d82 tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] Image pref 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:392}}
Nov 29 03:20:17 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-2f5399a7-4261-4e4b-908f-67a229c77d82 tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:430}}
Nov 29 03:20:17 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-2f5399a7-4261-4e4b-908f-67a229c77d82 tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:569}}
Nov 29 03:20:17 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-2f5399a7-4261-4e4b-908f-67a229c77d82 tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=75234) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:471}}
Nov 29 03:20:17 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-2f5399a7-4261-4e4b-908f-67a229c77d82 tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] Got 1 possible topologies {{(pid=75234) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:501}}
Nov 29 03:20:17 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-2f5399a7-4261-4e4b-908f-67a229c77d82 tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:575}}
Nov 29 03:20:17 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-2f5399a7-4261-4e4b-908f-67a229c77d82 tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:577}}
Nov 29 03:20:17 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-2f5399a7-4261-4e4b-908f-67a229c77d82 tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='True',created_at=2023-11-29T03:20:14Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-1777937262',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-device-tagging-server-1777937262',id=35,image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBJrVo4ngcPAPF4oP6xidv+ykI7O7g9y5APrTr7iXhruQjDIx7zqy1/omzWf7G2p8xg1QAr8Js8NrVgD6XT2ZMw0fSSCLDKoKJk+X3O2rM5aB1leyuLU4v60YiI/3ZRg9Ew==',key_name='tempest-keypair-1887924020',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='e28b5baf29b14aa4b2b98cb1b615705f',ramdisk_id='',reservation_id='r-bfdfrqjg',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TaggedAttachmentsTest-399278293',owner_user_name='tempest-TaggedAttachmentsTest-399278293-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-11-29T03:20:15Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='522c3c107614404fbe3af8c38fdd189e',uuid=946b919b-39ae-4d23-aefd-01952f4b40b6,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "e1f71ce6-58f3-4665-afc5-f92d1b6801c5", "address": "fa:16:3e:81:ca:ab", "network": {"id": "93ca21e4-138b-4f00-8d1d-b30d63dbe7aa", "bridge": "br-int", "label": "tempest-TaggedAttachmentsTest-1844288661-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "e28b5baf29b14aa4b2b98cb1b615705f", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape1f71ce6-58", "ovs_interfaceid": "e1f71ce6-58f3-4665-afc5-f92d1b6801c5", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=75234) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Nov 29 03:20:17 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-2f5399a7-4261-4e4b-908f-67a229c77d82 tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] Converting VIF {"id": "e1f71ce6-58f3-4665-afc5-f92d1b6801c5", "address": "fa:16:3e:81:ca:ab", "network": {"id": "93ca21e4-138b-4f00-8d1d-b30d63dbe7aa", "bridge": "br-int", "label": "tempest-TaggedAttachmentsTest-1844288661-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "e28b5baf29b14aa4b2b98cb1b615705f", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape1f71ce6-58", "ovs_interfaceid": "e1f71ce6-58f3-4665-afc5-f92d1b6801c5", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:20:17 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-2f5399a7-4261-4e4b-908f-67a229c77d82 tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:81:ca:ab,bridge_name='br-int',has_traffic_filtering=True,id=e1f71ce6-58f3-4665-afc5-f92d1b6801c5,network=Network(93ca21e4-138b-4f00-8d1d-b30d63dbe7aa),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tape1f71ce6-58') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:20:17 user nova-compute[75234]: DEBUG nova.objects.instance [None req-2f5399a7-4261-4e4b-908f-67a229c77d82 tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] Lazy-loading 'pci_devices' on Instance uuid 946b919b-39ae-4d23-aefd-01952f4b40b6 {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:20:17 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-2f5399a7-4261-4e4b-908f-67a229c77d82 tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] [instance: 946b919b-39ae-4d23-aefd-01952f4b40b6] End _get_guest_xml xml=<domain type="kvm">
Nov 29 03:20:17 user nova-compute[75234]:   <uuid>946b919b-39ae-4d23-aefd-01952f4b40b6</uuid>
Nov 29 03:20:17 user nova-compute[75234]:   <name>instance-00000023</name>
Nov 29 03:20:17 user nova-compute[75234]:   <memory>524288</memory>
Nov 29 03:20:17 user nova-compute[75234]:   <vcpu>1</vcpu>
Nov 29 03:20:17 user nova-compute[75234]:   <metadata>
Nov 29 03:20:17 user nova-compute[75234]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Nov 29 03:20:17 user nova-compute[75234]:       <nova:package version="0.0.0"/>
Nov 29 03:20:17 user nova-compute[75234]:       <nova:name>tempest-device-tagging-server-1777937262</nova:name>
Nov 29 03:20:17 user nova-compute[75234]:       <nova:creationTime>2023-11-29 03:20:17</nova:creationTime>
Nov 29 03:20:17 user nova-compute[75234]:       <nova:flavor name="m1.tiny">
Nov 29 03:20:17 user nova-compute[75234]:         <nova:memory>512</nova:memory>
Nov 29 03:20:17 user nova-compute[75234]:         <nova:disk>1</nova:disk>
Nov 29 03:20:17 user nova-compute[75234]:         <nova:swap>0</nova:swap>
Nov 29 03:20:17 user nova-compute[75234]:         <nova:ephemeral>0</nova:ephemeral>
Nov 29 03:20:17 user nova-compute[75234]:         <nova:vcpus>1</nova:vcpus>
Nov 29 03:20:17 user nova-compute[75234]:       </nova:flavor>
Nov 29 03:20:17 user nova-compute[75234]:       <nova:owner>
Nov 29 03:20:17 user nova-compute[75234]:         <nova:user uuid="522c3c107614404fbe3af8c38fdd189e">tempest-TaggedAttachmentsTest-399278293-project-member</nova:user>
Nov 29 03:20:17 user nova-compute[75234]:         <nova:project uuid="e28b5baf29b14aa4b2b98cb1b615705f">tempest-TaggedAttachmentsTest-399278293</nova:project>
Nov 29 03:20:17 user nova-compute[75234]:       </nova:owner>
Nov 29 03:20:17 user nova-compute[75234]:       <nova:root type="image" uuid="96c30ba7-9b73-4a30-8ba4-7138879573b7"/>
Nov 29 03:20:17 user nova-compute[75234]:       <nova:ports>
Nov 29 03:20:17 user nova-compute[75234]:         <nova:port uuid="e1f71ce6-58f3-4665-afc5-f92d1b6801c5">
Nov 29 03:20:17 user nova-compute[75234]:           <nova:ip type="fixed" address="10.0.0.4" ipVersion="4"/>
Nov 29 03:20:17 user nova-compute[75234]:         </nova:port>
Nov 29 03:20:17 user nova-compute[75234]:       </nova:ports>
Nov 29 03:20:17 user nova-compute[75234]:     </nova:instance>
Nov 29 03:20:17 user nova-compute[75234]:   </metadata>
Nov 29 03:20:17 user nova-compute[75234]:   <sysinfo type="smbios">
Nov 29 03:20:17 user nova-compute[75234]:     <system>
Nov 29 03:20:17 user nova-compute[75234]:       <entry name="manufacturer">OpenStack Foundation</entry>
Nov 29 03:20:17 user nova-compute[75234]:       <entry name="product">OpenStack Nova</entry>
Nov 29 03:20:17 user nova-compute[75234]:       <entry name="version">0.0.0</entry>
Nov 29 03:20:17 user nova-compute[75234]:       <entry name="serial">946b919b-39ae-4d23-aefd-01952f4b40b6</entry>
Nov 29 03:20:17 user nova-compute[75234]:       <entry name="uuid">946b919b-39ae-4d23-aefd-01952f4b40b6</entry>
Nov 29 03:20:17 user nova-compute[75234]:       <entry name="family">Virtual Machine</entry>
Nov 29 03:20:17 user nova-compute[75234]:     </system>
Nov 29 03:20:17 user nova-compute[75234]:   </sysinfo>
Nov 29 03:20:17 user nova-compute[75234]:   <os>
Nov 29 03:20:17 user nova-compute[75234]:     <type arch="x86_64" machine="pc">hvm</type>
Nov 29 03:20:17 user nova-compute[75234]:     <boot dev="hd"/>
Nov 29 03:20:17 user nova-compute[75234]:     <smbios mode="sysinfo"/>
Nov 29 03:20:17 user nova-compute[75234]:   </os>
Nov 29 03:20:17 user nova-compute[75234]:   <features>
Nov 29 03:20:17 user nova-compute[75234]:     <acpi/>
Nov 29 03:20:17 user nova-compute[75234]:     <apic/>
Nov 29 03:20:17 user nova-compute[75234]:     <vmcoreinfo/>
Nov 29 03:20:17 user nova-compute[75234]:   </features>
Nov 29 03:20:17 user nova-compute[75234]:   <clock offset="utc">
Nov 29 03:20:17 user nova-compute[75234]:     <timer name="pit" tickpolicy="delay"/>
Nov 29 03:20:17 user nova-compute[75234]:     <timer name="rtc" tickpolicy="catchup"/>
Nov 29 03:20:17 user nova-compute[75234]:     <timer name="hpet" present="no"/>
Nov 29 03:20:17 user nova-compute[75234]:   </clock>
Nov 29 03:20:17 user nova-compute[75234]:   <cpu mode="custom" match="exact">
Nov 29 03:20:17 user nova-compute[75234]:     <model>Nehalem</model>
Nov 29 03:20:17 user nova-compute[75234]:     <topology sockets="1" cores="1" threads="1"/>
Nov 29 03:20:17 user nova-compute[75234]:   </cpu>
Nov 29 03:20:17 user nova-compute[75234]:   <devices>
Nov 29 03:20:17 user nova-compute[75234]:     <disk type="file" device="disk">
Nov 29 03:20:17 user nova-compute[75234]:       <driver name="qemu" type="qcow2" cache="none"/>
Nov 29 03:20:17 user nova-compute[75234]:       <source file="/opt/stack/data/nova/instances/946b919b-39ae-4d23-aefd-01952f4b40b6/disk"/>
Nov 29 03:20:17 user nova-compute[75234]:       <target dev="vda" bus="virtio"/>
Nov 29 03:20:17 user nova-compute[75234]:     </disk>
Nov 29 03:20:17 user nova-compute[75234]:     <disk type="file" device="cdrom">
Nov 29 03:20:17 user nova-compute[75234]:       <driver name="qemu" type="raw" cache="none"/>
Nov 29 03:20:17 user nova-compute[75234]:       <source file="/opt/stack/data/nova/instances/946b919b-39ae-4d23-aefd-01952f4b40b6/disk.config"/>
Nov 29 03:20:17 user nova-compute[75234]:       <target dev="hda" bus="ide"/>
Nov 29 03:20:17 user nova-compute[75234]:     </disk>
Nov 29 03:20:17 user nova-compute[75234]:     <interface type="ethernet">
Nov 29 03:20:17 user nova-compute[75234]:       <mac address="fa:16:3e:81:ca:ab"/>
Nov 29 03:20:17 user nova-compute[75234]:       <model type="virtio"/>
Nov 29 03:20:17 user nova-compute[75234]:       <mtu size="1442"/>
Nov 29 03:20:17 user nova-compute[75234]:       <target dev="tape1f71ce6-58"/>
Nov 29 03:20:17 user nova-compute[75234]:     </interface>
Nov 29 03:20:17 user nova-compute[75234]:     <serial type="pty">
Nov 29 03:20:17 user nova-compute[75234]:       <log file="/opt/stack/data/nova/instances/946b919b-39ae-4d23-aefd-01952f4b40b6/console.log" append="off"/>
Nov 29 03:20:17 user nova-compute[75234]:     </serial>
Nov 29 03:20:17 user nova-compute[75234]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Nov 29 03:20:17 user nova-compute[75234]:     <video>
Nov 29 03:20:17 user nova-compute[75234]:       <model type="virtio"/>
Nov 29 03:20:17 user nova-compute[75234]:     </video>
Nov 29 03:20:17 user nova-compute[75234]:     <rng model="virtio">
Nov 29 03:20:17 user nova-compute[75234]:       <backend model="random">/dev/urandom</backend>
Nov 29 03:20:17 user nova-compute[75234]:     </rng>
Nov 29 03:20:17 user nova-compute[75234]:     <controller type="usb" index="0" model="none"/>
Nov 29 03:20:17 user nova-compute[75234]:     <memballoon model="virtio">
Nov 29 03:20:17 user nova-compute[75234]:       <stats period="10"/>
Nov 29 03:20:17 user nova-compute[75234]:     </memballoon>
Nov 29 03:20:17 user nova-compute[75234]:   </devices>
Nov 29 03:20:17 user nova-compute[75234]: </domain>
Nov 29 03:20:17 user nova-compute[75234]:  {{(pid=75234) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7568}}
Nov 29 03:20:17 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-2f5399a7-4261-4e4b-908f-67a229c77d82 tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='True',created_at=2023-11-29T03:20:14Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-1777937262',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-device-tagging-server-1777937262',id=35,image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBJrVo4ngcPAPF4oP6xidv+ykI7O7g9y5APrTr7iXhruQjDIx7zqy1/omzWf7G2p8xg1QAr8Js8NrVgD6XT2ZMw0fSSCLDKoKJk+X3O2rM5aB1leyuLU4v60YiI/3ZRg9Ew==',key_name='tempest-keypair-1887924020',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='e28b5baf29b14aa4b2b98cb1b615705f',ramdisk_id='',reservation_id='r-bfdfrqjg',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TaggedAttachmentsTest-399278293',owner_user_name='tempest-TaggedAttachmentsTest-399278293-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-11-29T03:20:15Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='522c3c107614404fbe3af8c38fdd189e',uuid=946b919b-39ae-4d23-aefd-01952f4b40b6,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "e1f71ce6-58f3-4665-afc5-f92d1b6801c5", "address": "fa:16:3e:81:ca:ab", "network": {"id": "93ca21e4-138b-4f00-8d1d-b30d63dbe7aa", "bridge": "br-int", "label": "tempest-TaggedAttachmentsTest-1844288661-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "e28b5baf29b14aa4b2b98cb1b615705f", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape1f71ce6-58", "ovs_interfaceid": "e1f71ce6-58f3-4665-afc5-f92d1b6801c5", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Nov 29 03:20:17 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-2f5399a7-4261-4e4b-908f-67a229c77d82 tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] Converting VIF {"id": "e1f71ce6-58f3-4665-afc5-f92d1b6801c5", "address": "fa:16:3e:81:ca:ab", "network": {"id": "93ca21e4-138b-4f00-8d1d-b30d63dbe7aa", "bridge": "br-int", "label": "tempest-TaggedAttachmentsTest-1844288661-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "e28b5baf29b14aa4b2b98cb1b615705f", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape1f71ce6-58", "ovs_interfaceid": "e1f71ce6-58f3-4665-afc5-f92d1b6801c5", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:20:17 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-2f5399a7-4261-4e4b-908f-67a229c77d82 tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:81:ca:ab,bridge_name='br-int',has_traffic_filtering=True,id=e1f71ce6-58f3-4665-afc5-f92d1b6801c5,network=Network(93ca21e4-138b-4f00-8d1d-b30d63dbe7aa),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tape1f71ce6-58') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:20:17 user nova-compute[75234]: DEBUG os_vif [None req-2f5399a7-4261-4e4b-908f-67a229c77d82 tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:81:ca:ab,bridge_name='br-int',has_traffic_filtering=True,id=e1f71ce6-58f3-4665-afc5-f92d1b6801c5,network=Network(93ca21e4-138b-4f00-8d1d-b30d63dbe7aa),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tape1f71ce6-58') {{(pid=75234) plug /opt/stack/data/venv/lib/python3.10/site-packages/os_vif/__init__.py:76}}
Nov 29 03:20:17 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:20:17 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:20:17 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Nov 29 03:20:17 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:20:17 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbCreateCommand(_result=None, table=QoS, columns={'type': 'linux-noop', 'external_ids': {'id': '35e90f2a-6bb5-5995-b40a-3b6828c2efeb', '_type': 'linux-noop'}}, row=False) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:20:17 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:20:17 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:20:17 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:20:17 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:20:17 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tape1f71ce6-58, may_exist=True, interface_attrs={}) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:20:17 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Port, record=tape1f71ce6-58, col_values=(('qos', UUID('2288fbe0-80b5-4935-b658-10771df36056')),), if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:20:17 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DbSetCommand(_result=None, table=Interface, record=tape1f71ce6-58, col_values=(('external_ids', {'iface-id': 'e1f71ce6-58f3-4665-afc5-f92d1b6801c5', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:81:ca:ab', 'vm-uuid': '946b919b-39ae-4d23-aefd-01952f4b40b6'}),), if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:20:17 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:20:17 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:20:17 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:20:17 user nova-compute[75234]: INFO os_vif [None req-2f5399a7-4261-4e4b-908f-67a229c77d82 tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:81:ca:ab,bridge_name='br-int',has_traffic_filtering=True,id=e1f71ce6-58f3-4665-afc5-f92d1b6801c5,network=Network(93ca21e4-138b-4f00-8d1d-b30d63dbe7aa),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tape1f71ce6-58')
Nov 29 03:20:17 user nova-compute[75234]: DEBUG nova.network.neutron [req-4fb5eaa1-d795-436b-9e5e-8e36698e68c6 req-1d33db4e-0b5e-45b0-8111-43f5d40d424c service nova] [instance: 6894b55f-93c6-43c9-8933-56d5a3c43725] Updated VIF entry in instance network info cache for port 9b2cf46e-008c-4ef3-9555-0cbc584dcf8f. {{(pid=75234) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3481}}
Nov 29 03:20:17 user nova-compute[75234]: DEBUG nova.network.neutron [req-4fb5eaa1-d795-436b-9e5e-8e36698e68c6 req-1d33db4e-0b5e-45b0-8111-43f5d40d424c service nova] [instance: 6894b55f-93c6-43c9-8933-56d5a3c43725] Updating instance_info_cache with network_info: [{"id": "9b2cf46e-008c-4ef3-9555-0cbc584dcf8f", "address": "fa:16:3e:d1:0b:00", "network": {"id": "b7a6cef0-4cf1-46e6-99fb-4972879e3913", "bridge": "br-int", "label": "tempest-TestStampPattern-1195861223-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "c4393bc598a845a9bfb667e5332f14a6", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap9b2cf46e-00", "ovs_interfaceid": "9b2cf46e-008c-4ef3-9555-0cbc584dcf8f", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:20:17 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-4fb5eaa1-d795-436b-9e5e-8e36698e68c6 req-1d33db4e-0b5e-45b0-8111-43f5d40d424c service nova] Releasing lock "refresh_cache-6894b55f-93c6-43c9-8933-56d5a3c43725" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:20:17 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-2f5399a7-4261-4e4b-908f-67a229c77d82 tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] No BDM found with device name vda, not building metadata. {{(pid=75234) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12089}}
Nov 29 03:20:17 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-2f5399a7-4261-4e4b-908f-67a229c77d82 tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] No BDM found with device name hda, not building metadata. {{(pid=75234) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12089}}
Nov 29 03:20:17 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-2f5399a7-4261-4e4b-908f-67a229c77d82 tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] No VIF found with MAC fa:16:3e:81:ca:ab, not building metadata {{(pid=75234) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Nov 29 03:20:17 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-2f5399a7-4261-4e4b-908f-67a229c77d82 tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] [instance: 946b919b-39ae-4d23-aefd-01952f4b40b6] Using config drive
Nov 29 03:20:17 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-2f5399a7-4261-4e4b-908f-67a229c77d82 tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] [instance: 946b919b-39ae-4d23-aefd-01952f4b40b6] Creating config drive at /opt/stack/data/nova/instances/946b919b-39ae-4d23-aefd-01952f4b40b6/disk.config
Nov 29 03:20:17 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-2f5399a7-4261-4e4b-908f-67a229c77d82 tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] Running cmd (subprocess): genisoimage -o /opt/stack/data/nova/instances/946b919b-39ae-4d23-aefd-01952f4b40b6/disk.config -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 0.0.0 -quiet -J -r -V config-2 /tmp/tmpqs4j_e_3 {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:20:17 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-2f5399a7-4261-4e4b-908f-67a229c77d82 tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] CMD "genisoimage -o /opt/stack/data/nova/instances/946b919b-39ae-4d23-aefd-01952f4b40b6/disk.config -ldots -allow-lowercase -allow-multidot -l -publisher OpenStack Nova 0.0.0 -quiet -J -r -V config-2 /tmp/tmpqs4j_e_3" returned: 0 in 0.077s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:20:17 user nova-compute[75234]: DEBUG nova.network.neutron [req-2089ebfc-6aa1-48d0-a444-86200772d24b req-411c9c17-5ac6-4c93-a0a5-08f665c1c32c service nova] [instance: 946b919b-39ae-4d23-aefd-01952f4b40b6] Updated VIF entry in instance network info cache for port e1f71ce6-58f3-4665-afc5-f92d1b6801c5. {{(pid=75234) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3481}}
Nov 29 03:20:17 user nova-compute[75234]: DEBUG nova.network.neutron [req-2089ebfc-6aa1-48d0-a444-86200772d24b req-411c9c17-5ac6-4c93-a0a5-08f665c1c32c service nova] [instance: 946b919b-39ae-4d23-aefd-01952f4b40b6] Updating instance_info_cache with network_info: [{"id": "e1f71ce6-58f3-4665-afc5-f92d1b6801c5", "address": "fa:16:3e:81:ca:ab", "network": {"id": "93ca21e4-138b-4f00-8d1d-b30d63dbe7aa", "bridge": "br-int", "label": "tempest-TaggedAttachmentsTest-1844288661-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "e28b5baf29b14aa4b2b98cb1b615705f", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape1f71ce6-58", "ovs_interfaceid": "e1f71ce6-58f3-4665-afc5-f92d1b6801c5", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:20:17 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:20:17 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-2089ebfc-6aa1-48d0-a444-86200772d24b req-411c9c17-5ac6-4c93-a0a5-08f665c1c32c service nova] Releasing lock "refresh_cache-946b919b-39ae-4d23-aefd-01952f4b40b6" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:20:17 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:20:18 user nova-compute[75234]: INFO nova.virt.libvirt.driver [-] [instance: 6894b55f-93c6-43c9-8933-56d5a3c43725] Instance destroyed successfully.
Nov 29 03:20:18 user nova-compute[75234]: DEBUG nova.objects.instance [None req-515241ec-541d-4828-82f7-7ad5f08ca57e tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Lazy-loading 'resources' on Instance uuid 6894b55f-93c6-43c9-8933-56d5a3c43725 {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:20:18 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-515241ec-541d-4828-82f7-7ad5f08ca57e tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-11-29T03:12:52Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-TestStampPattern-server-472892053',display_name='tempest-TestStampPattern-server-472892053',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-teststamppattern-server-472892053',id=22,image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBMhfiLNgC9PAWcr8xAscyt4FB2hoan+iXcHV+JRry3FLwwpkjQpDEiOvBF2D+f1pgs1A/1GOV9t1zFfztYSi2DW0KkfBHuJTofrSJvEENNcXO0BnR5RakzAYYv+USO8uHQ==',key_name='tempest-TestStampPattern-1617966982',keypairs=<?>,launch_index=0,launched_at=2023-11-29T03:12:59Z,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='c4393bc598a845a9bfb667e5332f14a6',ramdisk_id='',reservation_id='r-b4j3yqsa',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-TestStampPattern-448969956',owner_user_name='tempest-TestStampPattern-448969956-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-11-29T03:16:02Z,user_data=None,user_id='dc1de1dad3794b47b0e4bf7445c96997',uuid=6894b55f-93c6-43c9-8933-56d5a3c43725,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "9b2cf46e-008c-4ef3-9555-0cbc584dcf8f", "address": "fa:16:3e:d1:0b:00", "network": {"id": "b7a6cef0-4cf1-46e6-99fb-4972879e3913", "bridge": "br-int", "label": "tempest-TestStampPattern-1195861223-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.73", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "c4393bc598a845a9bfb667e5332f14a6", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap9b2cf46e-00", "ovs_interfaceid": "9b2cf46e-008c-4ef3-9555-0cbc584dcf8f", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Nov 29 03:20:18 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-515241ec-541d-4828-82f7-7ad5f08ca57e tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Converting VIF {"id": "9b2cf46e-008c-4ef3-9555-0cbc584dcf8f", "address": "fa:16:3e:d1:0b:00", "network": {"id": "b7a6cef0-4cf1-46e6-99fb-4972879e3913", "bridge": "br-int", "label": "tempest-TestStampPattern-1195861223-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.7", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.73", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "c4393bc598a845a9bfb667e5332f14a6", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap9b2cf46e-00", "ovs_interfaceid": "9b2cf46e-008c-4ef3-9555-0cbc584dcf8f", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:20:18 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-515241ec-541d-4828-82f7-7ad5f08ca57e tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:d1:0b:00,bridge_name='br-int',has_traffic_filtering=True,id=9b2cf46e-008c-4ef3-9555-0cbc584dcf8f,network=Network(b7a6cef0-4cf1-46e6-99fb-4972879e3913),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap9b2cf46e-00') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:20:18 user nova-compute[75234]: DEBUG os_vif [None req-515241ec-541d-4828-82f7-7ad5f08ca57e tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Unplugging vif VIFOpenVSwitch(active=True,address=fa:16:3e:d1:0b:00,bridge_name='br-int',has_traffic_filtering=True,id=9b2cf46e-008c-4ef3-9555-0cbc584dcf8f,network=Network(b7a6cef0-4cf1-46e6-99fb-4972879e3913),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap9b2cf46e-00') {{(pid=75234) unplug /opt/stack/data/venv/lib/python3.10/site-packages/os_vif/__init__.py:109}}
Nov 29 03:20:18 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:20:18 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tap9b2cf46e-00, bridge=br-int, if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:20:18 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:20:18 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:20:18 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:20:18 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:20:18 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbDestroyCommand(_result=None, table=QoS, record=e62b528c-3eeb-433c-97d2-97bfcdc9cbae) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:20:18 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:20:18 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:20:18 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:20:18 user nova-compute[75234]: INFO os_vif [None req-515241ec-541d-4828-82f7-7ad5f08ca57e tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Successfully unplugged vif VIFOpenVSwitch(active=True,address=fa:16:3e:d1:0b:00,bridge_name='br-int',has_traffic_filtering=True,id=9b2cf46e-008c-4ef3-9555-0cbc584dcf8f,network=Network(b7a6cef0-4cf1-46e6-99fb-4972879e3913),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap9b2cf46e-00')
Nov 29 03:20:18 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-515241ec-541d-4828-82f7-7ad5f08ca57e tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] [instance: 6894b55f-93c6-43c9-8933-56d5a3c43725] Deleting instance files /opt/stack/data/nova/instances/6894b55f-93c6-43c9-8933-56d5a3c43725_del
Nov 29 03:20:18 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-515241ec-541d-4828-82f7-7ad5f08ca57e tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] [instance: 6894b55f-93c6-43c9-8933-56d5a3c43725] Deletion of /opt/stack/data/nova/instances/6894b55f-93c6-43c9-8933-56d5a3c43725_del complete
Nov 29 03:20:18 user nova-compute[75234]: INFO nova.compute.manager [None req-515241ec-541d-4828-82f7-7ad5f08ca57e tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] [instance: 6894b55f-93c6-43c9-8933-56d5a3c43725] Took 1.24 seconds to destroy the instance on the hypervisor.
Nov 29 03:20:18 user nova-compute[75234]: DEBUG oslo.service.loopingcall [None req-515241ec-541d-4828-82f7-7ad5f08ca57e tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=75234) func /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/loopingcall.py:435}}
Nov 29 03:20:18 user nova-compute[75234]: DEBUG nova.compute.manager [-] [instance: 6894b55f-93c6-43c9-8933-56d5a3c43725] Deallocating network for instance {{(pid=75234) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2279}}
Nov 29 03:20:18 user nova-compute[75234]: DEBUG nova.network.neutron [-] [instance: 6894b55f-93c6-43c9-8933-56d5a3c43725] deallocate_for_instance() {{(pid=75234) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1802}}
Nov 29 03:20:18 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:20:18 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:20:18 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:20:18 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:20:18 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:20:18 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:20:18 user nova-compute[75234]: DEBUG nova.compute.manager [req-6eef2ade-d5e6-439a-802a-88f66ada27b1 req-baf44b68-84b1-4a47-a63d-b62056a2baf9 service nova] [instance: 6894b55f-93c6-43c9-8933-56d5a3c43725] Received event network-vif-unplugged-9b2cf46e-008c-4ef3-9555-0cbc584dcf8f {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:20:18 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-6eef2ade-d5e6-439a-802a-88f66ada27b1 req-baf44b68-84b1-4a47-a63d-b62056a2baf9 service nova] Acquiring lock "6894b55f-93c6-43c9-8933-56d5a3c43725-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:20:18 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-6eef2ade-d5e6-439a-802a-88f66ada27b1 req-baf44b68-84b1-4a47-a63d-b62056a2baf9 service nova] Lock "6894b55f-93c6-43c9-8933-56d5a3c43725-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:20:18 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-6eef2ade-d5e6-439a-802a-88f66ada27b1 req-baf44b68-84b1-4a47-a63d-b62056a2baf9 service nova] Lock "6894b55f-93c6-43c9-8933-56d5a3c43725-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:20:18 user nova-compute[75234]: DEBUG nova.compute.manager [req-6eef2ade-d5e6-439a-802a-88f66ada27b1 req-baf44b68-84b1-4a47-a63d-b62056a2baf9 service nova] [instance: 6894b55f-93c6-43c9-8933-56d5a3c43725] No waiting events found dispatching network-vif-unplugged-9b2cf46e-008c-4ef3-9555-0cbc584dcf8f {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:20:18 user nova-compute[75234]: DEBUG nova.compute.manager [req-6eef2ade-d5e6-439a-802a-88f66ada27b1 req-baf44b68-84b1-4a47-a63d-b62056a2baf9 service nova] [instance: 6894b55f-93c6-43c9-8933-56d5a3c43725] Received event network-vif-unplugged-9b2cf46e-008c-4ef3-9555-0cbc584dcf8f for instance with task_state deleting. {{(pid=75234) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10890}}
Nov 29 03:20:18 user nova-compute[75234]: DEBUG nova.compute.manager [req-6eef2ade-d5e6-439a-802a-88f66ada27b1 req-baf44b68-84b1-4a47-a63d-b62056a2baf9 service nova] [instance: 6894b55f-93c6-43c9-8933-56d5a3c43725] Received event network-vif-plugged-9b2cf46e-008c-4ef3-9555-0cbc584dcf8f {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:20:18 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-6eef2ade-d5e6-439a-802a-88f66ada27b1 req-baf44b68-84b1-4a47-a63d-b62056a2baf9 service nova] Acquiring lock "6894b55f-93c6-43c9-8933-56d5a3c43725-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:20:18 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-6eef2ade-d5e6-439a-802a-88f66ada27b1 req-baf44b68-84b1-4a47-a63d-b62056a2baf9 service nova] Lock "6894b55f-93c6-43c9-8933-56d5a3c43725-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:20:18 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-6eef2ade-d5e6-439a-802a-88f66ada27b1 req-baf44b68-84b1-4a47-a63d-b62056a2baf9 service nova] Lock "6894b55f-93c6-43c9-8933-56d5a3c43725-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:20:18 user nova-compute[75234]: DEBUG nova.compute.manager [req-6eef2ade-d5e6-439a-802a-88f66ada27b1 req-baf44b68-84b1-4a47-a63d-b62056a2baf9 service nova] [instance: 6894b55f-93c6-43c9-8933-56d5a3c43725] No waiting events found dispatching network-vif-plugged-9b2cf46e-008c-4ef3-9555-0cbc584dcf8f {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:20:18 user nova-compute[75234]: WARNING nova.compute.manager [req-6eef2ade-d5e6-439a-802a-88f66ada27b1 req-baf44b68-84b1-4a47-a63d-b62056a2baf9 service nova] [instance: 6894b55f-93c6-43c9-8933-56d5a3c43725] Received unexpected event network-vif-plugged-9b2cf46e-008c-4ef3-9555-0cbc584dcf8f for instance with vm_state active and task_state deleting.
Nov 29 03:20:19 user nova-compute[75234]: DEBUG nova.network.neutron [-] [instance: 6894b55f-93c6-43c9-8933-56d5a3c43725] Updating instance_info_cache with network_info: [] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:20:19 user nova-compute[75234]: INFO nova.compute.manager [-] [instance: 6894b55f-93c6-43c9-8933-56d5a3c43725] Took 0.73 seconds to deallocate network for instance.
Nov 29 03:20:19 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-515241ec-541d-4828-82f7-7ad5f08ca57e tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:20:19 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-515241ec-541d-4828-82f7-7ad5f08ca57e tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:20:19 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:20:19 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:20:19 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:20:19 user nova-compute[75234]: DEBUG nova.compute.provider_tree [None req-515241ec-541d-4828-82f7-7ad5f08ca57e tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Inventory has not changed in ProviderTree for provider: 4d291220-7c37-4839-a0e3-b23c66502b74 {{(pid=75234) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Nov 29 03:20:19 user nova-compute[75234]: DEBUG nova.scheduler.client.report [None req-515241ec-541d-4828-82f7-7ad5f08ca57e tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Inventory has not changed for provider 4d291220-7c37-4839-a0e3-b23c66502b74 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=75234) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Nov 29 03:20:19 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-515241ec-541d-4828-82f7-7ad5f08ca57e tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.595s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:20:19 user nova-compute[75234]: INFO nova.scheduler.client.report [None req-515241ec-541d-4828-82f7-7ad5f08ca57e tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Deleted allocations for instance 6894b55f-93c6-43c9-8933-56d5a3c43725
Nov 29 03:20:19 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-515241ec-541d-4828-82f7-7ad5f08ca57e tempest-TestStampPattern-448969956 tempest-TestStampPattern-448969956-project-member] Lock "6894b55f-93c6-43c9-8933-56d5a3c43725" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 2.803s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:20:19 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:20:20 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:20:20 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:20:20 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:20:21 user nova-compute[75234]: DEBUG nova.compute.manager [req-7842bcbb-e072-48a7-9477-d5f3da2666af req-632f41a9-94ee-4b41-becf-4ed4c3bd9d30 service nova] [instance: 6894b55f-93c6-43c9-8933-56d5a3c43725] Received event network-vif-deleted-9b2cf46e-008c-4ef3-9555-0cbc584dcf8f {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:20:21 user nova-compute[75234]: DEBUG nova.compute.manager [req-7842bcbb-e072-48a7-9477-d5f3da2666af req-632f41a9-94ee-4b41-becf-4ed4c3bd9d30 service nova] [instance: 946b919b-39ae-4d23-aefd-01952f4b40b6] Received event network-vif-plugged-e1f71ce6-58f3-4665-afc5-f92d1b6801c5 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:20:21 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-7842bcbb-e072-48a7-9477-d5f3da2666af req-632f41a9-94ee-4b41-becf-4ed4c3bd9d30 service nova] Acquiring lock "946b919b-39ae-4d23-aefd-01952f4b40b6-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:20:21 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-7842bcbb-e072-48a7-9477-d5f3da2666af req-632f41a9-94ee-4b41-becf-4ed4c3bd9d30 service nova] Lock "946b919b-39ae-4d23-aefd-01952f4b40b6-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:20:21 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-7842bcbb-e072-48a7-9477-d5f3da2666af req-632f41a9-94ee-4b41-becf-4ed4c3bd9d30 service nova] Lock "946b919b-39ae-4d23-aefd-01952f4b40b6-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:20:21 user nova-compute[75234]: DEBUG nova.compute.manager [req-7842bcbb-e072-48a7-9477-d5f3da2666af req-632f41a9-94ee-4b41-becf-4ed4c3bd9d30 service nova] [instance: 946b919b-39ae-4d23-aefd-01952f4b40b6] No waiting events found dispatching network-vif-plugged-e1f71ce6-58f3-4665-afc5-f92d1b6801c5 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:20:21 user nova-compute[75234]: WARNING nova.compute.manager [req-7842bcbb-e072-48a7-9477-d5f3da2666af req-632f41a9-94ee-4b41-becf-4ed4c3bd9d30 service nova] [instance: 946b919b-39ae-4d23-aefd-01952f4b40b6] Received unexpected event network-vif-plugged-e1f71ce6-58f3-4665-afc5-f92d1b6801c5 for instance with vm_state building and task_state spawning.
Nov 29 03:20:21 user nova-compute[75234]: DEBUG nova.compute.manager [req-7842bcbb-e072-48a7-9477-d5f3da2666af req-632f41a9-94ee-4b41-becf-4ed4c3bd9d30 service nova] [instance: 946b919b-39ae-4d23-aefd-01952f4b40b6] Received event network-vif-plugged-e1f71ce6-58f3-4665-afc5-f92d1b6801c5 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:20:21 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-7842bcbb-e072-48a7-9477-d5f3da2666af req-632f41a9-94ee-4b41-becf-4ed4c3bd9d30 service nova] Acquiring lock "946b919b-39ae-4d23-aefd-01952f4b40b6-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:20:21 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-7842bcbb-e072-48a7-9477-d5f3da2666af req-632f41a9-94ee-4b41-becf-4ed4c3bd9d30 service nova] Lock "946b919b-39ae-4d23-aefd-01952f4b40b6-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:20:21 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-7842bcbb-e072-48a7-9477-d5f3da2666af req-632f41a9-94ee-4b41-becf-4ed4c3bd9d30 service nova] Lock "946b919b-39ae-4d23-aefd-01952f4b40b6-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:20:21 user nova-compute[75234]: DEBUG nova.compute.manager [req-7842bcbb-e072-48a7-9477-d5f3da2666af req-632f41a9-94ee-4b41-becf-4ed4c3bd9d30 service nova] [instance: 946b919b-39ae-4d23-aefd-01952f4b40b6] No waiting events found dispatching network-vif-plugged-e1f71ce6-58f3-4665-afc5-f92d1b6801c5 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:20:21 user nova-compute[75234]: WARNING nova.compute.manager [req-7842bcbb-e072-48a7-9477-d5f3da2666af req-632f41a9-94ee-4b41-becf-4ed4c3bd9d30 service nova] [instance: 946b919b-39ae-4d23-aefd-01952f4b40b6] Received unexpected event network-vif-plugged-e1f71ce6-58f3-4665-afc5-f92d1b6801c5 for instance with vm_state building and task_state spawning.
Nov 29 03:20:21 user nova-compute[75234]: DEBUG nova.compute.manager [None req-2f5399a7-4261-4e4b-908f-67a229c77d82 tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] [instance: 946b919b-39ae-4d23-aefd-01952f4b40b6] Instance event wait completed in 0 seconds for  {{(pid=75234) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Nov 29 03:20:21 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-2f5399a7-4261-4e4b-908f-67a229c77d82 tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] [instance: 946b919b-39ae-4d23-aefd-01952f4b40b6] Guest created on hypervisor {{(pid=75234) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4421}}
Nov 29 03:20:21 user nova-compute[75234]: INFO nova.virt.libvirt.driver [-] [instance: 946b919b-39ae-4d23-aefd-01952f4b40b6] Instance spawned successfully.
Nov 29 03:20:21 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-2f5399a7-4261-4e4b-908f-67a229c77d82 tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] [instance: 946b919b-39ae-4d23-aefd-01952f4b40b6] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:904}}
Nov 29 03:20:21 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-2f5399a7-4261-4e4b-908f-67a229c77d82 tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] [instance: 946b919b-39ae-4d23-aefd-01952f4b40b6] Found default for hw_cdrom_bus of ide {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:20:21 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-2f5399a7-4261-4e4b-908f-67a229c77d82 tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] [instance: 946b919b-39ae-4d23-aefd-01952f4b40b6] Found default for hw_disk_bus of virtio {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:20:21 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-2f5399a7-4261-4e4b-908f-67a229c77d82 tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] [instance: 946b919b-39ae-4d23-aefd-01952f4b40b6] Found default for hw_input_bus of None {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:20:21 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-2f5399a7-4261-4e4b-908f-67a229c77d82 tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] [instance: 946b919b-39ae-4d23-aefd-01952f4b40b6] Found default for hw_pointer_model of None {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:20:21 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-2f5399a7-4261-4e4b-908f-67a229c77d82 tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] [instance: 946b919b-39ae-4d23-aefd-01952f4b40b6] Found default for hw_video_model of virtio {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:20:21 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-2f5399a7-4261-4e4b-908f-67a229c77d82 tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] [instance: 946b919b-39ae-4d23-aefd-01952f4b40b6] Found default for hw_vif_model of virtio {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:20:21 user nova-compute[75234]: INFO nova.compute.manager [None req-2f5399a7-4261-4e4b-908f-67a229c77d82 tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] [instance: 946b919b-39ae-4d23-aefd-01952f4b40b6] Took 6.09 seconds to spawn the instance on the hypervisor.
Nov 29 03:20:21 user nova-compute[75234]: DEBUG nova.compute.manager [None req-2f5399a7-4261-4e4b-908f-67a229c77d82 tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] [instance: 946b919b-39ae-4d23-aefd-01952f4b40b6] Checking state {{(pid=75234) _get_power_state /opt/stack/nova/nova/compute/manager.py:1782}}
Nov 29 03:20:21 user nova-compute[75234]: INFO nova.compute.manager [None req-2f5399a7-4261-4e4b-908f-67a229c77d82 tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] [instance: 946b919b-39ae-4d23-aefd-01952f4b40b6] Took 6.98 seconds to build instance.
Nov 29 03:20:21 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-2f5399a7-4261-4e4b-908f-67a229c77d82 tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] Lock "946b919b-39ae-4d23-aefd-01952f4b40b6" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 7.100s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:20:22 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:20:23 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:20:24 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-5ecc0c25-6668-4140-a70a-ea341b999b4c tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Acquiring lock "99d6f85f-1b77-4390-8b61-908c98f584c0" by "nova.compute.manager.ComputeManager.unshelve_instance.<locals>.do_unshelve_instance" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:20:24 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-5ecc0c25-6668-4140-a70a-ea341b999b4c tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Lock "99d6f85f-1b77-4390-8b61-908c98f584c0" acquired by "nova.compute.manager.ComputeManager.unshelve_instance.<locals>.do_unshelve_instance" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:20:24 user nova-compute[75234]: INFO nova.compute.manager [None req-5ecc0c25-6668-4140-a70a-ea341b999b4c tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] [instance: 99d6f85f-1b77-4390-8b61-908c98f584c0] Unshelving
Nov 29 03:20:24 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-5ecc0c25-6668-4140-a70a-ea341b999b4c tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:20:24 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-5ecc0c25-6668-4140-a70a-ea341b999b4c tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:20:24 user nova-compute[75234]: DEBUG nova.objects.instance [None req-5ecc0c25-6668-4140-a70a-ea341b999b4c tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Lazy-loading 'pci_requests' on Instance uuid 99d6f85f-1b77-4390-8b61-908c98f584c0 {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:20:24 user nova-compute[75234]: DEBUG nova.objects.instance [None req-5ecc0c25-6668-4140-a70a-ea341b999b4c tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Lazy-loading 'numa_topology' on Instance uuid 99d6f85f-1b77-4390-8b61-908c98f584c0 {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:20:24 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-5ecc0c25-6668-4140-a70a-ea341b999b4c tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=75234) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2366}}
Nov 29 03:20:24 user nova-compute[75234]: INFO nova.compute.claims [None req-5ecc0c25-6668-4140-a70a-ea341b999b4c tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] [instance: 99d6f85f-1b77-4390-8b61-908c98f584c0] Claim successful on node user
Nov 29 03:20:24 user nova-compute[75234]: DEBUG nova.compute.provider_tree [None req-5ecc0c25-6668-4140-a70a-ea341b999b4c tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Inventory has not changed in ProviderTree for provider: 4d291220-7c37-4839-a0e3-b23c66502b74 {{(pid=75234) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Nov 29 03:20:24 user nova-compute[75234]: DEBUG nova.scheduler.client.report [None req-5ecc0c25-6668-4140-a70a-ea341b999b4c tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Inventory has not changed for provider 4d291220-7c37-4839-a0e3-b23c66502b74 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=75234) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Nov 29 03:20:24 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-5ecc0c25-6668-4140-a70a-ea341b999b4c tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.461s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:20:24 user nova-compute[75234]: INFO nova.network.neutron [None req-5ecc0c25-6668-4140-a70a-ea341b999b4c tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] [instance: 99d6f85f-1b77-4390-8b61-908c98f584c0] Updating port b68d722f-9020-4da4-8cdb-e2312ef06df9 with attributes {'binding:host_id': 'user', 'device_owner': 'compute:nova'}
Nov 29 03:20:25 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-5ecc0c25-6668-4140-a70a-ea341b999b4c tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Acquiring lock "refresh_cache-99d6f85f-1b77-4390-8b61-908c98f584c0" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:20:25 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-5ecc0c25-6668-4140-a70a-ea341b999b4c tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Acquired lock "refresh_cache-99d6f85f-1b77-4390-8b61-908c98f584c0" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:20:25 user nova-compute[75234]: DEBUG nova.network.neutron [None req-5ecc0c25-6668-4140-a70a-ea341b999b4c tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] [instance: 99d6f85f-1b77-4390-8b61-908c98f584c0] Building network info cache for instance {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2009}}
Nov 29 03:20:25 user nova-compute[75234]: DEBUG nova.compute.manager [req-a14260c5-7371-4389-bad6-71751a8a808d req-2a8c7c34-d37d-4327-a7fd-0a7fd5749d7b service nova] [instance: 99d6f85f-1b77-4390-8b61-908c98f584c0] Received event network-changed-b68d722f-9020-4da4-8cdb-e2312ef06df9 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:20:25 user nova-compute[75234]: DEBUG nova.compute.manager [req-a14260c5-7371-4389-bad6-71751a8a808d req-2a8c7c34-d37d-4327-a7fd-0a7fd5749d7b service nova] [instance: 99d6f85f-1b77-4390-8b61-908c98f584c0] Refreshing instance network info cache due to event network-changed-b68d722f-9020-4da4-8cdb-e2312ef06df9. {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11117}}
Nov 29 03:20:25 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-a14260c5-7371-4389-bad6-71751a8a808d req-2a8c7c34-d37d-4327-a7fd-0a7fd5749d7b service nova] Acquiring lock "refresh_cache-99d6f85f-1b77-4390-8b61-908c98f584c0" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:20:25 user nova-compute[75234]: DEBUG nova.network.neutron [None req-5ecc0c25-6668-4140-a70a-ea341b999b4c tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] [instance: 99d6f85f-1b77-4390-8b61-908c98f584c0] Updating instance_info_cache with network_info: [{"id": "b68d722f-9020-4da4-8cdb-e2312ef06df9", "address": "fa:16:3e:1e:90:83", "network": {"id": "aad085f7-106b-4de6-93f5-f86bb66ec962", "bridge": "br-int", "label": "tempest-AttachVolumeShelveTestJSON-775984268-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.193", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "e17150005a9a449f9373b91e1c00083b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapb68d722f-90", "ovs_interfaceid": "b68d722f-9020-4da4-8cdb-e2312ef06df9", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:20:25 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-5ecc0c25-6668-4140-a70a-ea341b999b4c tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Releasing lock "refresh_cache-99d6f85f-1b77-4390-8b61-908c98f584c0" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:20:25 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-5ecc0c25-6668-4140-a70a-ea341b999b4c tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] [instance: 99d6f85f-1b77-4390-8b61-908c98f584c0] Creating instance directory {{(pid=75234) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4727}}
Nov 29 03:20:25 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-5ecc0c25-6668-4140-a70a-ea341b999b4c tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] [instance: 99d6f85f-1b77-4390-8b61-908c98f584c0] Creating image(s)
Nov 29 03:20:25 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-5ecc0c25-6668-4140-a70a-ea341b999b4c tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Acquiring lock "/opt/stack/data/nova/instances/99d6f85f-1b77-4390-8b61-908c98f584c0/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:20:25 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-5ecc0c25-6668-4140-a70a-ea341b999b4c tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Lock "/opt/stack/data/nova/instances/99d6f85f-1b77-4390-8b61-908c98f584c0/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:20:25 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-5ecc0c25-6668-4140-a70a-ea341b999b4c tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Lock "/opt/stack/data/nova/instances/99d6f85f-1b77-4390-8b61-908c98f584c0/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:20:25 user nova-compute[75234]: DEBUG nova.objects.instance [None req-5ecc0c25-6668-4140-a70a-ea341b999b4c tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Lazy-loading 'trusted_certs' on Instance uuid 99d6f85f-1b77-4390-8b61-908c98f584c0 {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:20:25 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-a14260c5-7371-4389-bad6-71751a8a808d req-2a8c7c34-d37d-4327-a7fd-0a7fd5749d7b service nova] Acquired lock "refresh_cache-99d6f85f-1b77-4390-8b61-908c98f584c0" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:20:25 user nova-compute[75234]: DEBUG nova.network.neutron [req-a14260c5-7371-4389-bad6-71751a8a808d req-2a8c7c34-d37d-4327-a7fd-0a7fd5749d7b service nova] [instance: 99d6f85f-1b77-4390-8b61-908c98f584c0] Refreshing network info cache for port b68d722f-9020-4da4-8cdb-e2312ef06df9 {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2006}}
Nov 29 03:20:25 user nova-compute[75234]: DEBUG nova.objects.base [None req-5ecc0c25-6668-4140-a70a-ea341b999b4c tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Object Instance<99d6f85f-1b77-4390-8b61-908c98f584c0> lazy-loaded attributes: pci_requests,trusted_certs {{(pid=75234) wrapper /opt/stack/nova/nova/objects/base.py:126}}
Nov 29 03:20:25 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-5ecc0c25-6668-4140-a70a-ea341b999b4c tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Acquiring lock "db2d1f2e1d03090216196960bcd01ca47391c21d" by "nova.virt.libvirt.imagebackend.Image.cache.<locals>.fetch_func_sync" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:20:25 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-5ecc0c25-6668-4140-a70a-ea341b999b4c tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Lock "db2d1f2e1d03090216196960bcd01ca47391c21d" acquired by "nova.virt.libvirt.imagebackend.Image.cache.<locals>.fetch_func_sync" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:20:26 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-5ecc0c25-6668-4140-a70a-ea341b999b4c tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/db2d1f2e1d03090216196960bcd01ca47391c21d.part --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:20:27 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:20:27 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-5ecc0c25-6668-4140-a70a-ea341b999b4c tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/db2d1f2e1d03090216196960bcd01ca47391c21d.part --force-share --output=json" returned: 0 in 0.236s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:20:27 user nova-compute[75234]: DEBUG nova.virt.images [None req-5ecc0c25-6668-4140-a70a-ea341b999b4c tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] d7de6e9a-26aa-4b83-99e9-7aee3378b12d was qcow2, converting to raw {{(pid=75234) fetch_to_raw /opt/stack/nova/nova/virt/images.py:165}}
Nov 29 03:20:27 user nova-compute[75234]: DEBUG nova.privsep.utils [None req-5ecc0c25-6668-4140-a70a-ea341b999b4c tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Path '/opt/stack/data/nova/instances' supports direct I/O {{(pid=75234) supports_direct_io /opt/stack/nova/nova/privsep/utils.py:63}}
Nov 29 03:20:27 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-5ecc0c25-6668-4140-a70a-ea341b999b4c tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Running cmd (subprocess): qemu-img convert -t none -O raw -f qcow2 /opt/stack/data/nova/instances/_base/db2d1f2e1d03090216196960bcd01ca47391c21d.part /opt/stack/data/nova/instances/_base/db2d1f2e1d03090216196960bcd01ca47391c21d.converted {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:20:27 user nova-compute[75234]: DEBUG nova.network.neutron [req-a14260c5-7371-4389-bad6-71751a8a808d req-2a8c7c34-d37d-4327-a7fd-0a7fd5749d7b service nova] [instance: 99d6f85f-1b77-4390-8b61-908c98f584c0] Updated VIF entry in instance network info cache for port b68d722f-9020-4da4-8cdb-e2312ef06df9. {{(pid=75234) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3481}}
Nov 29 03:20:27 user nova-compute[75234]: DEBUG nova.network.neutron [req-a14260c5-7371-4389-bad6-71751a8a808d req-2a8c7c34-d37d-4327-a7fd-0a7fd5749d7b service nova] [instance: 99d6f85f-1b77-4390-8b61-908c98f584c0] Updating instance_info_cache with network_info: [{"id": "b68d722f-9020-4da4-8cdb-e2312ef06df9", "address": "fa:16:3e:1e:90:83", "network": {"id": "aad085f7-106b-4de6-93f5-f86bb66ec962", "bridge": "br-int", "label": "tempest-AttachVolumeShelveTestJSON-775984268-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.193", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "e17150005a9a449f9373b91e1c00083b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapb68d722f-90", "ovs_interfaceid": "b68d722f-9020-4da4-8cdb-e2312ef06df9", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:20:27 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-a14260c5-7371-4389-bad6-71751a8a808d req-2a8c7c34-d37d-4327-a7fd-0a7fd5749d7b service nova] Releasing lock "refresh_cache-99d6f85f-1b77-4390-8b61-908c98f584c0" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:20:27 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-5ecc0c25-6668-4140-a70a-ea341b999b4c tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] CMD "qemu-img convert -t none -O raw -f qcow2 /opt/stack/data/nova/instances/_base/db2d1f2e1d03090216196960bcd01ca47391c21d.part /opt/stack/data/nova/instances/_base/db2d1f2e1d03090216196960bcd01ca47391c21d.converted" returned: 0 in 0.654s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:20:27 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-5ecc0c25-6668-4140-a70a-ea341b999b4c tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/db2d1f2e1d03090216196960bcd01ca47391c21d.converted --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:20:28 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-5ecc0c25-6668-4140-a70a-ea341b999b4c tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/db2d1f2e1d03090216196960bcd01ca47391c21d.converted --force-share --output=json" returned: 0 in 0.169s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:20:28 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-5ecc0c25-6668-4140-a70a-ea341b999b4c tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Lock "db2d1f2e1d03090216196960bcd01ca47391c21d" "released" by "nova.virt.libvirt.imagebackend.Image.cache.<locals>.fetch_func_sync" :: held 2.258s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:20:28 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-5ecc0c25-6668-4140-a70a-ea341b999b4c tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/db2d1f2e1d03090216196960bcd01ca47391c21d --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:20:28 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-5ecc0c25-6668-4140-a70a-ea341b999b4c tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/db2d1f2e1d03090216196960bcd01ca47391c21d --force-share --output=json" returned: 0 in 0.200s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:20:28 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-5ecc0c25-6668-4140-a70a-ea341b999b4c tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Acquiring lock "db2d1f2e1d03090216196960bcd01ca47391c21d" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:20:28 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-5ecc0c25-6668-4140-a70a-ea341b999b4c tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Lock "db2d1f2e1d03090216196960bcd01ca47391c21d" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.002s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:20:28 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-5ecc0c25-6668-4140-a70a-ea341b999b4c tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/db2d1f2e1d03090216196960bcd01ca47391c21d --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:20:28 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:20:28 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-5ecc0c25-6668-4140-a70a-ea341b999b4c tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/db2d1f2e1d03090216196960bcd01ca47391c21d --force-share --output=json" returned: 0 in 0.177s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:20:28 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-5ecc0c25-6668-4140-a70a-ea341b999b4c tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/db2d1f2e1d03090216196960bcd01ca47391c21d,backing_fmt=raw /opt/stack/data/nova/instances/99d6f85f-1b77-4390-8b61-908c98f584c0/disk 1073741824 {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:20:28 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-5ecc0c25-6668-4140-a70a-ea341b999b4c tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/db2d1f2e1d03090216196960bcd01ca47391c21d,backing_fmt=raw /opt/stack/data/nova/instances/99d6f85f-1b77-4390-8b61-908c98f584c0/disk 1073741824" returned: 0 in 0.050s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:20:28 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-5ecc0c25-6668-4140-a70a-ea341b999b4c tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Lock "db2d1f2e1d03090216196960bcd01ca47391c21d" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.235s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:20:28 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-5ecc0c25-6668-4140-a70a-ea341b999b4c tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/db2d1f2e1d03090216196960bcd01ca47391c21d --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:20:28 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-5ecc0c25-6668-4140-a70a-ea341b999b4c tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/db2d1f2e1d03090216196960bcd01ca47391c21d --force-share --output=json" returned: 0 in 0.175s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:20:28 user nova-compute[75234]: DEBUG nova.objects.instance [None req-5ecc0c25-6668-4140-a70a-ea341b999b4c tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Lazy-loading 'migration_context' on Instance uuid 99d6f85f-1b77-4390-8b61-908c98f584c0 {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:20:28 user nova-compute[75234]: DEBUG nova.objects.base [None req-5ecc0c25-6668-4140-a70a-ea341b999b4c tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Object Instance<99d6f85f-1b77-4390-8b61-908c98f584c0> lazy-loaded attributes: pci_requests,trusted_certs,migration_context {{(pid=75234) wrapper /opt/stack/nova/nova/objects/base.py:126}}
Nov 29 03:20:28 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-5ecc0c25-6668-4140-a70a-ea341b999b4c tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] [instance: 99d6f85f-1b77-4390-8b61-908c98f584c0] Rebasing disk image.
Nov 29 03:20:28 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-5ecc0c25-6668-4140-a70a-ea341b999b4c tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:20:28 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-5ecc0c25-6668-4140-a70a-ea341b999b4c tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json" returned: 0 in 0.181s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:20:28 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-5ecc0c25-6668-4140-a70a-ea341b999b4c tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Running cmd (subprocess): qemu-img rebase -b /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 -F raw /opt/stack/data/nova/instances/99d6f85f-1b77-4390-8b61-908c98f584c0/disk {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:20:31 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-5ecc0c25-6668-4140-a70a-ea341b999b4c tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] CMD "qemu-img rebase -b /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 -F raw /opt/stack/data/nova/instances/99d6f85f-1b77-4390-8b61-908c98f584c0/disk" returned: 0 in 2.136s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:20:31 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-5ecc0c25-6668-4140-a70a-ea341b999b4c tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] [instance: 99d6f85f-1b77-4390-8b61-908c98f584c0] Created local disks {{(pid=75234) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4861}}
Nov 29 03:20:31 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-5ecc0c25-6668-4140-a70a-ea341b999b4c tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] [instance: 99d6f85f-1b77-4390-8b61-908c98f584c0] Ensure instance console log exists: /opt/stack/data/nova/instances/99d6f85f-1b77-4390-8b61-908c98f584c0/console.log {{(pid=75234) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4613}}
Nov 29 03:20:31 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-5ecc0c25-6668-4140-a70a-ea341b999b4c tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:20:31 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-5ecc0c25-6668-4140-a70a-ea341b999b4c tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:20:31 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-5ecc0c25-6668-4140-a70a-ea341b999b4c tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:20:31 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-5ecc0c25-6668-4140-a70a-ea341b999b4c tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] [instance: 99d6f85f-1b77-4390-8b61-908c98f584c0] Start _get_guest_xml network_info=[{"id": "b68d722f-9020-4da4-8cdb-e2312ef06df9", "address": "fa:16:3e:1e:90:83", "network": {"id": "aad085f7-106b-4de6-93f5-f86bb66ec962", "bridge": "br-int", "label": "tempest-AttachVolumeShelveTestJSON-775984268-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.193", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "e17150005a9a449f9373b91e1c00083b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapb68d722f-90", "ovs_interfaceid": "b68d722f-9020-4da4-8cdb-e2312ef06df9", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum='5edb86c87c689823595d5f48bdc05aac',container_format='bare',created_at=2023-11-29T03:18:31Z,direct_url=<?>,disk_format='qcow2',id=d7de6e9a-26aa-4b83-99e9-7aee3378b12d,min_disk=1,min_ram=0,name='tempest-AttachVolumeShelveTestJSON-server-491132081-shelved',owner='e17150005a9a449f9373b91e1c00083b',properties=ImageMetaProps,protected=<?>,size=52363264,status='active',tags=<?>,updated_at=2023-11-29T03:18:36Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [{'device_name': '/dev/vda', 'encryption_format': None, 'encrypted': False, 'device_type': 'disk', 'size': 0, 'disk_bus': 'virtio', 'boot_index': 0, 'encryption_options': None, 'encryption_secret_uuid': None, 'guest_format': None, 'image_id': '96c30ba7-9b73-4a30-8ba4-7138879573b7'}], 'ephemerals': [], 'block_device_mapping': [], 'swap': None} {{(pid=75234) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7562}}
Nov 29 03:20:31 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-5ecc0c25-6668-4140-a70a-ea341b999b4c tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:20:31 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-5ecc0c25-6668-4140-a70a-ea341b999b4c tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:20:31 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-5ecc0c25-6668-4140-a70a-ea341b999b4c tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Searching host: 'user' for CPU controller through CGroups V1... {{(pid=75234) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1652}}
Nov 29 03:20:31 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-5ecc0c25-6668-4140-a70a-ea341b999b4c tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] CPU controller missing on host. {{(pid=75234) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1662}}
Nov 29 03:20:31 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-5ecc0c25-6668-4140-a70a-ea341b999b4c tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Searching host: 'user' for CPU controller through CGroups V2... {{(pid=75234) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1671}}
Nov 29 03:20:31 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-5ecc0c25-6668-4140-a70a-ea341b999b4c tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] CPU controller found on host. {{(pid=75234) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1678}}
Nov 29 03:20:31 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-5ecc0c25-6668-4140-a70a-ea341b999b4c tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=75234) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5400}}
Nov 29 03:20:31 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-5ecc0c25-6668-4140-a70a-ea341b999b4c tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-11-29T03:02:04Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='1',id=6,is_public=True,memory_mb=512,name='m1.tiny',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='5edb86c87c689823595d5f48bdc05aac',container_format='bare',created_at=2023-11-29T03:18:31Z,direct_url=<?>,disk_format='qcow2',id=d7de6e9a-26aa-4b83-99e9-7aee3378b12d,min_disk=1,min_ram=0,name='tempest-AttachVolumeShelveTestJSON-server-491132081-shelved',owner='e17150005a9a449f9373b91e1c00083b',properties=ImageMetaProps,protected=<?>,size=52363264,status='active',tags=<?>,updated_at=2023-11-29T03:18:36Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:563}}
Nov 29 03:20:31 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-5ecc0c25-6668-4140-a70a-ea341b999b4c tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Flavor limits 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:348}}
Nov 29 03:20:31 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-5ecc0c25-6668-4140-a70a-ea341b999b4c tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Image limits 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:352}}
Nov 29 03:20:31 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-5ecc0c25-6668-4140-a70a-ea341b999b4c tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Flavor pref 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:388}}
Nov 29 03:20:31 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-5ecc0c25-6668-4140-a70a-ea341b999b4c tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Image pref 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:392}}
Nov 29 03:20:31 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-5ecc0c25-6668-4140-a70a-ea341b999b4c tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:430}}
Nov 29 03:20:31 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-5ecc0c25-6668-4140-a70a-ea341b999b4c tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:569}}
Nov 29 03:20:31 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-5ecc0c25-6668-4140-a70a-ea341b999b4c tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=75234) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:471}}
Nov 29 03:20:31 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-5ecc0c25-6668-4140-a70a-ea341b999b4c tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Got 1 possible topologies {{(pid=75234) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:501}}
Nov 29 03:20:31 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-5ecc0c25-6668-4140-a70a-ea341b999b4c tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:575}}
Nov 29 03:20:31 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-5ecc0c25-6668-4140-a70a-ea341b999b4c tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:577}}
Nov 29 03:20:31 user nova-compute[75234]: DEBUG nova.objects.instance [None req-5ecc0c25-6668-4140-a70a-ea341b999b4c tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Lazy-loading 'vcpu_model' on Instance uuid 99d6f85f-1b77-4390-8b61-908c98f584c0 {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:20:31 user nova-compute[75234]: DEBUG nova.objects.base [None req-5ecc0c25-6668-4140-a70a-ea341b999b4c tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Object Instance<99d6f85f-1b77-4390-8b61-908c98f584c0> lazy-loaded attributes: pci_requests,trusted_certs,migration_context,vcpu_model {{(pid=75234) wrapper /opt/stack/nova/nova/objects/base.py:126}}
Nov 29 03:20:31 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-5ecc0c25-6668-4140-a70a-ea341b999b4c tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=True,compute_id=1,config_drive='',created_at=2023-11-29T03:16:35Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description=None,display_name='tempest-AttachVolumeShelveTestJSON-server-491132081',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-attachvolumeshelvetestjson-server-491132081',id=28,image_ref='d7de6e9a-26aa-4b83-99e9-7aee3378b12d',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data=None,key_name='tempest-keypair-1848667272',keypairs=<?>,launch_index=0,launched_at=2023-11-29T03:16:42Z,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=4,progress=0,project_id='e17150005a9a449f9373b91e1c00083b',ramdisk_id='',reservation_id='r-79uedf60',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',clean_attempts='1',image_base_image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-AttachVolumeShelveTestJSON-1180535246',owner_user_name='tempest-AttachVolumeShelveTestJSON-1180535246-project-member',shelved_at='2023-11-29T03:18:36.093023',shelved_host='user',shelved_image_id='d7de6e9a-26aa-4b83-99e9-7aee3378b12d'},tags=<?>,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-11-29T03:20:24Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='154f3f4f57514450baf8f22d67768bc9',uuid=99d6f85f-1b77-4390-8b61-908c98f584c0,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='shelved_offloaded') vif={"id": "b68d722f-9020-4da4-8cdb-e2312ef06df9", "address": "fa:16:3e:1e:90:83", "network": {"id": "aad085f7-106b-4de6-93f5-f86bb66ec962", "bridge": "br-int", "label": "tempest-AttachVolumeShelveTestJSON-775984268-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.193", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "e17150005a9a449f9373b91e1c00083b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapb68d722f-90", "ovs_interfaceid": "b68d722f-9020-4da4-8cdb-e2312ef06df9", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=75234) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Nov 29 03:20:31 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-5ecc0c25-6668-4140-a70a-ea341b999b4c tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Converting VIF {"id": "b68d722f-9020-4da4-8cdb-e2312ef06df9", "address": "fa:16:3e:1e:90:83", "network": {"id": "aad085f7-106b-4de6-93f5-f86bb66ec962", "bridge": "br-int", "label": "tempest-AttachVolumeShelveTestJSON-775984268-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.193", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "e17150005a9a449f9373b91e1c00083b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapb68d722f-90", "ovs_interfaceid": "b68d722f-9020-4da4-8cdb-e2312ef06df9", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:20:31 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-5ecc0c25-6668-4140-a70a-ea341b999b4c tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:1e:90:83,bridge_name='br-int',has_traffic_filtering=True,id=b68d722f-9020-4da4-8cdb-e2312ef06df9,network=Network(aad085f7-106b-4de6-93f5-f86bb66ec962),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapb68d722f-90') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:20:31 user nova-compute[75234]: DEBUG nova.objects.instance [None req-5ecc0c25-6668-4140-a70a-ea341b999b4c tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Lazy-loading 'pci_devices' on Instance uuid 99d6f85f-1b77-4390-8b61-908c98f584c0 {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:20:31 user nova-compute[75234]: DEBUG nova.objects.base [None req-5ecc0c25-6668-4140-a70a-ea341b999b4c tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Object Instance<99d6f85f-1b77-4390-8b61-908c98f584c0> lazy-loaded attributes: pci_requests,trusted_certs,migration_context,vcpu_model,pci_devices {{(pid=75234) wrapper /opt/stack/nova/nova/objects/base.py:126}}
Nov 29 03:20:31 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-5ecc0c25-6668-4140-a70a-ea341b999b4c tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] [instance: 99d6f85f-1b77-4390-8b61-908c98f584c0] End _get_guest_xml xml=<domain type="kvm">
Nov 29 03:20:31 user nova-compute[75234]:   <uuid>99d6f85f-1b77-4390-8b61-908c98f584c0</uuid>
Nov 29 03:20:31 user nova-compute[75234]:   <name>instance-0000001c</name>
Nov 29 03:20:31 user nova-compute[75234]:   <memory>524288</memory>
Nov 29 03:20:31 user nova-compute[75234]:   <vcpu>1</vcpu>
Nov 29 03:20:31 user nova-compute[75234]:   <metadata>
Nov 29 03:20:31 user nova-compute[75234]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Nov 29 03:20:31 user nova-compute[75234]:       <nova:package version="0.0.0"/>
Nov 29 03:20:31 user nova-compute[75234]:       <nova:name>tempest-AttachVolumeShelveTestJSON-server-491132081</nova:name>
Nov 29 03:20:31 user nova-compute[75234]:       <nova:creationTime>2023-11-29 03:20:31</nova:creationTime>
Nov 29 03:20:31 user nova-compute[75234]:       <nova:flavor name="m1.tiny">
Nov 29 03:20:31 user nova-compute[75234]:         <nova:memory>512</nova:memory>
Nov 29 03:20:31 user nova-compute[75234]:         <nova:disk>1</nova:disk>
Nov 29 03:20:31 user nova-compute[75234]:         <nova:swap>0</nova:swap>
Nov 29 03:20:31 user nova-compute[75234]:         <nova:ephemeral>0</nova:ephemeral>
Nov 29 03:20:31 user nova-compute[75234]:         <nova:vcpus>1</nova:vcpus>
Nov 29 03:20:31 user nova-compute[75234]:       </nova:flavor>
Nov 29 03:20:31 user nova-compute[75234]:       <nova:owner>
Nov 29 03:20:31 user nova-compute[75234]:         <nova:user uuid="154f3f4f57514450baf8f22d67768bc9">tempest-AttachVolumeShelveTestJSON-1180535246-project-member</nova:user>
Nov 29 03:20:31 user nova-compute[75234]:         <nova:project uuid="e17150005a9a449f9373b91e1c00083b">tempest-AttachVolumeShelveTestJSON-1180535246</nova:project>
Nov 29 03:20:31 user nova-compute[75234]:       </nova:owner>
Nov 29 03:20:31 user nova-compute[75234]:       <nova:root type="image" uuid="d7de6e9a-26aa-4b83-99e9-7aee3378b12d"/>
Nov 29 03:20:31 user nova-compute[75234]:       <nova:ports>
Nov 29 03:20:31 user nova-compute[75234]:         <nova:port uuid="b68d722f-9020-4da4-8cdb-e2312ef06df9">
Nov 29 03:20:31 user nova-compute[75234]:           <nova:ip type="fixed" address="10.0.0.3" ipVersion="4"/>
Nov 29 03:20:31 user nova-compute[75234]:         </nova:port>
Nov 29 03:20:31 user nova-compute[75234]:       </nova:ports>
Nov 29 03:20:31 user nova-compute[75234]:     </nova:instance>
Nov 29 03:20:31 user nova-compute[75234]:   </metadata>
Nov 29 03:20:31 user nova-compute[75234]:   <sysinfo type="smbios">
Nov 29 03:20:31 user nova-compute[75234]:     <system>
Nov 29 03:20:31 user nova-compute[75234]:       <entry name="manufacturer">OpenStack Foundation</entry>
Nov 29 03:20:31 user nova-compute[75234]:       <entry name="product">OpenStack Nova</entry>
Nov 29 03:20:31 user nova-compute[75234]:       <entry name="version">0.0.0</entry>
Nov 29 03:20:31 user nova-compute[75234]:       <entry name="serial">99d6f85f-1b77-4390-8b61-908c98f584c0</entry>
Nov 29 03:20:31 user nova-compute[75234]:       <entry name="uuid">99d6f85f-1b77-4390-8b61-908c98f584c0</entry>
Nov 29 03:20:31 user nova-compute[75234]:       <entry name="family">Virtual Machine</entry>
Nov 29 03:20:31 user nova-compute[75234]:     </system>
Nov 29 03:20:31 user nova-compute[75234]:   </sysinfo>
Nov 29 03:20:31 user nova-compute[75234]:   <os>
Nov 29 03:20:31 user nova-compute[75234]:     <type arch="x86_64" machine="pc">hvm</type>
Nov 29 03:20:31 user nova-compute[75234]:     <boot dev="hd"/>
Nov 29 03:20:31 user nova-compute[75234]:     <smbios mode="sysinfo"/>
Nov 29 03:20:31 user nova-compute[75234]:   </os>
Nov 29 03:20:31 user nova-compute[75234]:   <features>
Nov 29 03:20:31 user nova-compute[75234]:     <acpi/>
Nov 29 03:20:31 user nova-compute[75234]:     <apic/>
Nov 29 03:20:31 user nova-compute[75234]:     <vmcoreinfo/>
Nov 29 03:20:31 user nova-compute[75234]:   </features>
Nov 29 03:20:31 user nova-compute[75234]:   <clock offset="utc">
Nov 29 03:20:31 user nova-compute[75234]:     <timer name="pit" tickpolicy="delay"/>
Nov 29 03:20:31 user nova-compute[75234]:     <timer name="rtc" tickpolicy="catchup"/>
Nov 29 03:20:31 user nova-compute[75234]:     <timer name="hpet" present="no"/>
Nov 29 03:20:31 user nova-compute[75234]:   </clock>
Nov 29 03:20:31 user nova-compute[75234]:   <cpu mode="custom" match="exact">
Nov 29 03:20:31 user nova-compute[75234]:     <model>Nehalem</model>
Nov 29 03:20:31 user nova-compute[75234]:     <topology sockets="1" cores="1" threads="1"/>
Nov 29 03:20:31 user nova-compute[75234]:   </cpu>
Nov 29 03:20:31 user nova-compute[75234]:   <devices>
Nov 29 03:20:31 user nova-compute[75234]:     <disk type="file" device="disk">
Nov 29 03:20:31 user nova-compute[75234]:       <driver name="qemu" type="qcow2" cache="none"/>
Nov 29 03:20:31 user nova-compute[75234]:       <source file="/opt/stack/data/nova/instances/99d6f85f-1b77-4390-8b61-908c98f584c0/disk"/>
Nov 29 03:20:31 user nova-compute[75234]:       <target dev="vda" bus="virtio"/>
Nov 29 03:20:31 user nova-compute[75234]:     </disk>
Nov 29 03:20:31 user nova-compute[75234]:     <interface type="ethernet">
Nov 29 03:20:31 user nova-compute[75234]:       <mac address="fa:16:3e:1e:90:83"/>
Nov 29 03:20:31 user nova-compute[75234]:       <model type="virtio"/>
Nov 29 03:20:31 user nova-compute[75234]:       <mtu size="1442"/>
Nov 29 03:20:31 user nova-compute[75234]:       <target dev="tapb68d722f-90"/>
Nov 29 03:20:31 user nova-compute[75234]:     </interface>
Nov 29 03:20:31 user nova-compute[75234]:     <serial type="pty">
Nov 29 03:20:31 user nova-compute[75234]:       <log file="/opt/stack/data/nova/instances/99d6f85f-1b77-4390-8b61-908c98f584c0/console.log" append="off"/>
Nov 29 03:20:31 user nova-compute[75234]:     </serial>
Nov 29 03:20:31 user nova-compute[75234]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Nov 29 03:20:31 user nova-compute[75234]:     <video>
Nov 29 03:20:31 user nova-compute[75234]:       <model type="virtio"/>
Nov 29 03:20:31 user nova-compute[75234]:     </video>
Nov 29 03:20:31 user nova-compute[75234]:     <rng model="virtio">
Nov 29 03:20:31 user nova-compute[75234]:       <backend model="random">/dev/urandom</backend>
Nov 29 03:20:31 user nova-compute[75234]:     </rng>
Nov 29 03:20:31 user nova-compute[75234]:     <controller type="usb" index="0" model="none"/>
Nov 29 03:20:31 user nova-compute[75234]:     <memballoon model="virtio">
Nov 29 03:20:31 user nova-compute[75234]:       <stats period="10"/>
Nov 29 03:20:31 user nova-compute[75234]:     </memballoon>
Nov 29 03:20:31 user nova-compute[75234]:   </devices>
Nov 29 03:20:31 user nova-compute[75234]: </domain>
Nov 29 03:20:31 user nova-compute[75234]:  {{(pid=75234) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7568}}
Nov 29 03:20:31 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-5ecc0c25-6668-4140-a70a-ea341b999b4c tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=True,compute_id=1,config_drive='',created_at=2023-11-29T03:16:35Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description=None,display_name='tempest-AttachVolumeShelveTestJSON-server-491132081',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-attachvolumeshelvetestjson-server-491132081',id=28,image_ref='d7de6e9a-26aa-4b83-99e9-7aee3378b12d',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data=None,key_name='tempest-keypair-1848667272',keypairs=<?>,launch_index=0,launched_at=2023-11-29T03:16:42Z,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=4,progress=0,project_id='e17150005a9a449f9373b91e1c00083b',ramdisk_id='',reservation_id='r-79uedf60',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',clean_attempts='1',image_base_image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-AttachVolumeShelveTestJSON-1180535246',owner_user_name='tempest-AttachVolumeShelveTestJSON-1180535246-project-member',shelved_at='2023-11-29T03:18:36.093023',shelved_host='user',shelved_image_id='d7de6e9a-26aa-4b83-99e9-7aee3378b12d'},tags=<?>,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-11-29T03:20:24Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='154f3f4f57514450baf8f22d67768bc9',uuid=99d6f85f-1b77-4390-8b61-908c98f584c0,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='shelved_offloaded') vif={"id": "b68d722f-9020-4da4-8cdb-e2312ef06df9", "address": "fa:16:3e:1e:90:83", "network": {"id": "aad085f7-106b-4de6-93f5-f86bb66ec962", "bridge": "br-int", "label": "tempest-AttachVolumeShelveTestJSON-775984268-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.193", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "e17150005a9a449f9373b91e1c00083b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapb68d722f-90", "ovs_interfaceid": "b68d722f-9020-4da4-8cdb-e2312ef06df9", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Nov 29 03:20:31 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-5ecc0c25-6668-4140-a70a-ea341b999b4c tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Converting VIF {"id": "b68d722f-9020-4da4-8cdb-e2312ef06df9", "address": "fa:16:3e:1e:90:83", "network": {"id": "aad085f7-106b-4de6-93f5-f86bb66ec962", "bridge": "br-int", "label": "tempest-AttachVolumeShelveTestJSON-775984268-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.193", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "e17150005a9a449f9373b91e1c00083b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapb68d722f-90", "ovs_interfaceid": "b68d722f-9020-4da4-8cdb-e2312ef06df9", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:20:31 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-5ecc0c25-6668-4140-a70a-ea341b999b4c tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:1e:90:83,bridge_name='br-int',has_traffic_filtering=True,id=b68d722f-9020-4da4-8cdb-e2312ef06df9,network=Network(aad085f7-106b-4de6-93f5-f86bb66ec962),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapb68d722f-90') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:20:31 user nova-compute[75234]: DEBUG os_vif [None req-5ecc0c25-6668-4140-a70a-ea341b999b4c tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:1e:90:83,bridge_name='br-int',has_traffic_filtering=True,id=b68d722f-9020-4da4-8cdb-e2312ef06df9,network=Network(aad085f7-106b-4de6-93f5-f86bb66ec962),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapb68d722f-90') {{(pid=75234) plug /opt/stack/data/venv/lib/python3.10/site-packages/os_vif/__init__.py:76}}
Nov 29 03:20:31 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:20:31 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:20:31 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Nov 29 03:20:31 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:20:31 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbCreateCommand(_result=None, table=QoS, columns={'type': 'linux-noop', 'external_ids': {'id': '1d5ec683-25aa-5708-bd43-73c9dffea702', '_type': 'linux-noop'}}, row=False) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:20:31 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:20:31 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:20:31 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:20:31 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tapb68d722f-90, may_exist=True, interface_attrs={}) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:20:31 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Port, record=tapb68d722f-90, col_values=(('qos', UUID('465e9465-20d7-40f4-a194-907ed3dbd63a')),), if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:20:31 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DbSetCommand(_result=None, table=Interface, record=tapb68d722f-90, col_values=(('external_ids', {'iface-id': 'b68d722f-9020-4da4-8cdb-e2312ef06df9', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:1e:90:83', 'vm-uuid': '99d6f85f-1b77-4390-8b61-908c98f584c0'}),), if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:20:31 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:20:31 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:20:31 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:20:31 user nova-compute[75234]: INFO os_vif [None req-5ecc0c25-6668-4140-a70a-ea341b999b4c tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:1e:90:83,bridge_name='br-int',has_traffic_filtering=True,id=b68d722f-9020-4da4-8cdb-e2312ef06df9,network=Network(aad085f7-106b-4de6-93f5-f86bb66ec962),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapb68d722f-90')
Nov 29 03:20:31 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-5ecc0c25-6668-4140-a70a-ea341b999b4c tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] No BDM found with device name vda, not building metadata. {{(pid=75234) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12089}}
Nov 29 03:20:31 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-5ecc0c25-6668-4140-a70a-ea341b999b4c tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] No VIF found with MAC fa:16:3e:1e:90:83, not building metadata {{(pid=75234) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Nov 29 03:20:32 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:20:32 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:20:32 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:20:32 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:20:32 user nova-compute[75234]: DEBUG nova.compute.manager [req-43ed6125-68ae-4a91-9700-5bf13a784680 req-ceecaee7-f5b9-471c-bd15-310d56df3750 service nova] [instance: 99d6f85f-1b77-4390-8b61-908c98f584c0] Received event network-vif-plugged-b68d722f-9020-4da4-8cdb-e2312ef06df9 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:20:32 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-43ed6125-68ae-4a91-9700-5bf13a784680 req-ceecaee7-f5b9-471c-bd15-310d56df3750 service nova] Acquiring lock "99d6f85f-1b77-4390-8b61-908c98f584c0-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:20:32 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-43ed6125-68ae-4a91-9700-5bf13a784680 req-ceecaee7-f5b9-471c-bd15-310d56df3750 service nova] Lock "99d6f85f-1b77-4390-8b61-908c98f584c0-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:20:32 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-43ed6125-68ae-4a91-9700-5bf13a784680 req-ceecaee7-f5b9-471c-bd15-310d56df3750 service nova] Lock "99d6f85f-1b77-4390-8b61-908c98f584c0-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:20:32 user nova-compute[75234]: DEBUG nova.compute.manager [req-43ed6125-68ae-4a91-9700-5bf13a784680 req-ceecaee7-f5b9-471c-bd15-310d56df3750 service nova] [instance: 99d6f85f-1b77-4390-8b61-908c98f584c0] No waiting events found dispatching network-vif-plugged-b68d722f-9020-4da4-8cdb-e2312ef06df9 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:20:32 user nova-compute[75234]: WARNING nova.compute.manager [req-43ed6125-68ae-4a91-9700-5bf13a784680 req-ceecaee7-f5b9-471c-bd15-310d56df3750 service nova] [instance: 99d6f85f-1b77-4390-8b61-908c98f584c0] Received unexpected event network-vif-plugged-b68d722f-9020-4da4-8cdb-e2312ef06df9 for instance with vm_state shelved_offloaded and task_state spawning.
Nov 29 03:20:33 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:20:33 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:20:33 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:20:33 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:20:34 user nova-compute[75234]: DEBUG nova.compute.manager [req-8f159c9c-d0da-48c4-87ee-3fec6b18ee3d req-975b912f-1108-43b0-8e34-e8b8817985a2 service nova] [instance: 99d6f85f-1b77-4390-8b61-908c98f584c0] Received event network-vif-plugged-b68d722f-9020-4da4-8cdb-e2312ef06df9 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:20:34 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-8f159c9c-d0da-48c4-87ee-3fec6b18ee3d req-975b912f-1108-43b0-8e34-e8b8817985a2 service nova] Acquiring lock "99d6f85f-1b77-4390-8b61-908c98f584c0-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:20:34 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-8f159c9c-d0da-48c4-87ee-3fec6b18ee3d req-975b912f-1108-43b0-8e34-e8b8817985a2 service nova] Lock "99d6f85f-1b77-4390-8b61-908c98f584c0-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:20:34 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-8f159c9c-d0da-48c4-87ee-3fec6b18ee3d req-975b912f-1108-43b0-8e34-e8b8817985a2 service nova] Lock "99d6f85f-1b77-4390-8b61-908c98f584c0-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:20:34 user nova-compute[75234]: DEBUG nova.compute.manager [req-8f159c9c-d0da-48c4-87ee-3fec6b18ee3d req-975b912f-1108-43b0-8e34-e8b8817985a2 service nova] [instance: 99d6f85f-1b77-4390-8b61-908c98f584c0] No waiting events found dispatching network-vif-plugged-b68d722f-9020-4da4-8cdb-e2312ef06df9 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:20:34 user nova-compute[75234]: WARNING nova.compute.manager [req-8f159c9c-d0da-48c4-87ee-3fec6b18ee3d req-975b912f-1108-43b0-8e34-e8b8817985a2 service nova] [instance: 99d6f85f-1b77-4390-8b61-908c98f584c0] Received unexpected event network-vif-plugged-b68d722f-9020-4da4-8cdb-e2312ef06df9 for instance with vm_state shelved_offloaded and task_state spawning.
Nov 29 03:20:35 user nova-compute[75234]: DEBUG nova.compute.manager [None req-5ecc0c25-6668-4140-a70a-ea341b999b4c tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] [instance: 99d6f85f-1b77-4390-8b61-908c98f584c0] Instance event wait completed in 0 seconds for  {{(pid=75234) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Nov 29 03:20:35 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-5ecc0c25-6668-4140-a70a-ea341b999b4c tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] [instance: 99d6f85f-1b77-4390-8b61-908c98f584c0] Guest created on hypervisor {{(pid=75234) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4421}}
Nov 29 03:20:35 user nova-compute[75234]: INFO nova.virt.libvirt.driver [-] [instance: 99d6f85f-1b77-4390-8b61-908c98f584c0] Instance spawned successfully.
Nov 29 03:20:35 user nova-compute[75234]: DEBUG nova.compute.manager [None req-5ecc0c25-6668-4140-a70a-ea341b999b4c tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] [instance: 99d6f85f-1b77-4390-8b61-908c98f584c0] Checking state {{(pid=75234) _get_power_state /opt/stack/nova/nova/compute/manager.py:1782}}
Nov 29 03:20:35 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-5ecc0c25-6668-4140-a70a-ea341b999b4c tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Lock "99d6f85f-1b77-4390-8b61-908c98f584c0" "released" by "nova.compute.manager.ComputeManager.unshelve_instance.<locals>.do_unshelve_instance" :: held 11.280s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:20:36 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:20:37 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:20:41 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:20:42 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:20:44 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:20:46 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:20:47 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:20:47 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:20:49 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-226b3276-674a-4a31-873c-36e3f6fb069c tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Acquiring lock "75c5d700-6367-4128-9228-80d2cd24ace7" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:20:49 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-226b3276-674a-4a31-873c-36e3f6fb069c tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Lock "75c5d700-6367-4128-9228-80d2cd24ace7" acquired by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:20:49 user nova-compute[75234]: INFO nova.compute.manager [None req-226b3276-674a-4a31-873c-36e3f6fb069c tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] [instance: 75c5d700-6367-4128-9228-80d2cd24ace7] Detaching volume bf57b967-7fad-4d76-abb2-319fe88f8ef9
Nov 29 03:20:49 user nova-compute[75234]: INFO nova.virt.block_device [None req-226b3276-674a-4a31-873c-36e3f6fb069c tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] [instance: 75c5d700-6367-4128-9228-80d2cd24ace7] Attempting to driver detach volume bf57b967-7fad-4d76-abb2-319fe88f8ef9 from mountpoint /dev/vdb
Nov 29 03:20:49 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-226b3276-674a-4a31-873c-36e3f6fb069c tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Attempting to detach device vdb from instance 75c5d700-6367-4128-9228-80d2cd24ace7 from the persistent domain config. {{(pid=75234) _detach_from_persistent /opt/stack/nova/nova/virt/libvirt/driver.py:2487}}
Nov 29 03:20:49 user nova-compute[75234]: DEBUG nova.virt.libvirt.guest [None req-226b3276-674a-4a31-873c-36e3f6fb069c tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] detach device xml: <disk type="block" device="disk">
Nov 29 03:20:49 user nova-compute[75234]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Nov 29 03:20:49 user nova-compute[75234]:   <source dev="/dev/sdd"/>
Nov 29 03:20:49 user nova-compute[75234]:   <target dev="vdb" bus="virtio"/>
Nov 29 03:20:49 user nova-compute[75234]:   <serial>bf57b967-7fad-4d76-abb2-319fe88f8ef9</serial>
Nov 29 03:20:49 user nova-compute[75234]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x07" function="0x0"/>
Nov 29 03:20:49 user nova-compute[75234]: </disk>
Nov 29 03:20:49 user nova-compute[75234]:  {{(pid=75234) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Nov 29 03:20:49 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-226b3276-674a-4a31-873c-36e3f6fb069c tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Successfully detached device vdb from instance 75c5d700-6367-4128-9228-80d2cd24ace7 from the persistent domain config.
Nov 29 03:20:49 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-226b3276-674a-4a31-873c-36e3f6fb069c tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] (1/8): Attempting to detach device vdb with device alias virtio-disk1 from instance 75c5d700-6367-4128-9228-80d2cd24ace7 from the live domain config. {{(pid=75234) _detach_from_live_with_retry /opt/stack/nova/nova/virt/libvirt/driver.py:2523}}
Nov 29 03:20:49 user nova-compute[75234]: DEBUG nova.virt.libvirt.guest [None req-226b3276-674a-4a31-873c-36e3f6fb069c tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] detach device xml: <disk type="block" device="disk">
Nov 29 03:20:49 user nova-compute[75234]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Nov 29 03:20:49 user nova-compute[75234]:   <source dev="/dev/sdd"/>
Nov 29 03:20:49 user nova-compute[75234]:   <target dev="vdb" bus="virtio"/>
Nov 29 03:20:49 user nova-compute[75234]:   <serial>bf57b967-7fad-4d76-abb2-319fe88f8ef9</serial>
Nov 29 03:20:49 user nova-compute[75234]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x07" function="0x0"/>
Nov 29 03:20:49 user nova-compute[75234]: </disk>
Nov 29 03:20:49 user nova-compute[75234]:  {{(pid=75234) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Nov 29 03:20:49 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-226b3276-674a-4a31-873c-36e3f6fb069c tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Start waiting for the detach event from libvirt for device vdb with device alias virtio-disk1 for instance 75c5d700-6367-4128-9228-80d2cd24ace7 {{(pid=75234) _detach_from_live_and_wait_for_event /opt/stack/nova/nova/virt/libvirt/driver.py:2599}}
Nov 29 03:20:51 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:20:52 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:20:52 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:20:54 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:20:54 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:20:55 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-e48c1f07-127f-4b50-94a2-33e86178ae5d tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Acquiring lock "65d48c93-b6da-4f51-9d5a-0676dda62a60" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:20:55 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-e48c1f07-127f-4b50-94a2-33e86178ae5d tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Lock "65d48c93-b6da-4f51-9d5a-0676dda62a60" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:20:55 user nova-compute[75234]: DEBUG nova.compute.manager [None req-e48c1f07-127f-4b50-94a2-33e86178ae5d tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] [instance: 65d48c93-b6da-4f51-9d5a-0676dda62a60] Starting instance... {{(pid=75234) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2422}}
Nov 29 03:20:55 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-e48c1f07-127f-4b50-94a2-33e86178ae5d tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:20:55 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-e48c1f07-127f-4b50-94a2-33e86178ae5d tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:20:55 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-e48c1f07-127f-4b50-94a2-33e86178ae5d tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=75234) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2366}}
Nov 29 03:20:55 user nova-compute[75234]: INFO nova.compute.claims [None req-e48c1f07-127f-4b50-94a2-33e86178ae5d tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] [instance: 65d48c93-b6da-4f51-9d5a-0676dda62a60] Claim successful on node user
Nov 29 03:20:56 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:20:56 user nova-compute[75234]: DEBUG nova.compute.provider_tree [None req-e48c1f07-127f-4b50-94a2-33e86178ae5d tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Inventory has not changed in ProviderTree for provider: 4d291220-7c37-4839-a0e3-b23c66502b74 {{(pid=75234) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Nov 29 03:20:56 user nova-compute[75234]: DEBUG nova.scheduler.client.report [None req-e48c1f07-127f-4b50-94a2-33e86178ae5d tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Inventory has not changed for provider 4d291220-7c37-4839-a0e3-b23c66502b74 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=75234) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Nov 29 03:20:56 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-e48c1f07-127f-4b50-94a2-33e86178ae5d tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.417s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:20:56 user nova-compute[75234]: DEBUG nova.compute.manager [None req-e48c1f07-127f-4b50-94a2-33e86178ae5d tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] [instance: 65d48c93-b6da-4f51-9d5a-0676dda62a60] Start building networks asynchronously for instance. {{(pid=75234) _build_resources /opt/stack/nova/nova/compute/manager.py:2819}}
Nov 29 03:20:56 user nova-compute[75234]: DEBUG nova.compute.manager [None req-e48c1f07-127f-4b50-94a2-33e86178ae5d tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] [instance: 65d48c93-b6da-4f51-9d5a-0676dda62a60] Allocating IP information in the background. {{(pid=75234) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1972}}
Nov 29 03:20:56 user nova-compute[75234]: DEBUG nova.network.neutron [None req-e48c1f07-127f-4b50-94a2-33e86178ae5d tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] [instance: 65d48c93-b6da-4f51-9d5a-0676dda62a60] allocate_for_instance() {{(pid=75234) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1156}}
Nov 29 03:20:56 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-e48c1f07-127f-4b50-94a2-33e86178ae5d tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] [instance: 65d48c93-b6da-4f51-9d5a-0676dda62a60] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Nov 29 03:20:56 user nova-compute[75234]: DEBUG nova.compute.manager [None req-e48c1f07-127f-4b50-94a2-33e86178ae5d tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] [instance: 65d48c93-b6da-4f51-9d5a-0676dda62a60] Start building block device mappings for instance. {{(pid=75234) _build_resources /opt/stack/nova/nova/compute/manager.py:2854}}
Nov 29 03:20:56 user nova-compute[75234]: DEBUG nova.policy [None req-e48c1f07-127f-4b50-94a2-33e86178ae5d tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '877aa877f13a4f84ac4725278010aea3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'db00bcd0c82c44f1bda739e9857f6e84', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=75234) authorize /opt/stack/nova/nova/policy.py:203}}
Nov 29 03:20:56 user nova-compute[75234]: DEBUG nova.compute.manager [None req-e48c1f07-127f-4b50-94a2-33e86178ae5d tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] [instance: 65d48c93-b6da-4f51-9d5a-0676dda62a60] Start spawning the instance on the hypervisor. {{(pid=75234) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2628}}
Nov 29 03:20:56 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-e48c1f07-127f-4b50-94a2-33e86178ae5d tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] [instance: 65d48c93-b6da-4f51-9d5a-0676dda62a60] Creating instance directory {{(pid=75234) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4727}}
Nov 29 03:20:56 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-e48c1f07-127f-4b50-94a2-33e86178ae5d tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] [instance: 65d48c93-b6da-4f51-9d5a-0676dda62a60] Creating image(s)
Nov 29 03:20:56 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-e48c1f07-127f-4b50-94a2-33e86178ae5d tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Acquiring lock "/opt/stack/data/nova/instances/65d48c93-b6da-4f51-9d5a-0676dda62a60/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:20:56 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-e48c1f07-127f-4b50-94a2-33e86178ae5d tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Lock "/opt/stack/data/nova/instances/65d48c93-b6da-4f51-9d5a-0676dda62a60/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:20:56 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-e48c1f07-127f-4b50-94a2-33e86178ae5d tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Lock "/opt/stack/data/nova/instances/65d48c93-b6da-4f51-9d5a-0676dda62a60/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.014s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:20:56 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-e48c1f07-127f-4b50-94a2-33e86178ae5d tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:20:56 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-e48c1f07-127f-4b50-94a2-33e86178ae5d tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json" returned: 0 in 0.179s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:20:56 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-e48c1f07-127f-4b50-94a2-33e86178ae5d tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Acquiring lock "80686e29dc9db6600d03e85cb62af298e12d7918" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:20:56 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-e48c1f07-127f-4b50-94a2-33e86178ae5d tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Lock "80686e29dc9db6600d03e85cb62af298e12d7918" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.002s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:20:56 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-e48c1f07-127f-4b50-94a2-33e86178ae5d tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:20:57 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-e48c1f07-127f-4b50-94a2-33e86178ae5d tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json" returned: 0 in 0.184s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:20:57 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-e48c1f07-127f-4b50-94a2-33e86178ae5d tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918,backing_fmt=raw /opt/stack/data/nova/instances/65d48c93-b6da-4f51-9d5a-0676dda62a60/disk 1073741824 {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:20:57 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:20:57 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-e48c1f07-127f-4b50-94a2-33e86178ae5d tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918,backing_fmt=raw /opt/stack/data/nova/instances/65d48c93-b6da-4f51-9d5a-0676dda62a60/disk 1073741824" returned: 0 in 0.063s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:20:57 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-e48c1f07-127f-4b50-94a2-33e86178ae5d tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Lock "80686e29dc9db6600d03e85cb62af298e12d7918" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.252s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:20:57 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-e48c1f07-127f-4b50-94a2-33e86178ae5d tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:20:57 user nova-compute[75234]: DEBUG nova.network.neutron [None req-e48c1f07-127f-4b50-94a2-33e86178ae5d tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] [instance: 65d48c93-b6da-4f51-9d5a-0676dda62a60] Successfully created port: 967835a2-4b55-4ae6-b071-91233af412fd {{(pid=75234) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:548}}
Nov 29 03:20:57 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-e48c1f07-127f-4b50-94a2-33e86178ae5d tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json" returned: 0 in 0.175s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:20:57 user nova-compute[75234]: DEBUG nova.virt.disk.api [None req-e48c1f07-127f-4b50-94a2-33e86178ae5d tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Checking if we can resize image /opt/stack/data/nova/instances/65d48c93-b6da-4f51-9d5a-0676dda62a60/disk. size=1073741824 {{(pid=75234) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:166}}
Nov 29 03:20:57 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-e48c1f07-127f-4b50-94a2-33e86178ae5d tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/65d48c93-b6da-4f51-9d5a-0676dda62a60/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:20:57 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:20:57 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-e48c1f07-127f-4b50-94a2-33e86178ae5d tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/65d48c93-b6da-4f51-9d5a-0676dda62a60/disk --force-share --output=json" returned: 0 in 0.174s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:20:57 user nova-compute[75234]: DEBUG nova.virt.disk.api [None req-e48c1f07-127f-4b50-94a2-33e86178ae5d tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Cannot resize image /opt/stack/data/nova/instances/65d48c93-b6da-4f51-9d5a-0676dda62a60/disk to a smaller size. {{(pid=75234) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:172}}
Nov 29 03:20:57 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-e48c1f07-127f-4b50-94a2-33e86178ae5d tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] [instance: 65d48c93-b6da-4f51-9d5a-0676dda62a60] Created local disks {{(pid=75234) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4861}}
Nov 29 03:20:57 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-e48c1f07-127f-4b50-94a2-33e86178ae5d tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] [instance: 65d48c93-b6da-4f51-9d5a-0676dda62a60] Ensure instance console log exists: /opt/stack/data/nova/instances/65d48c93-b6da-4f51-9d5a-0676dda62a60/console.log {{(pid=75234) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4613}}
Nov 29 03:20:57 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-e48c1f07-127f-4b50-94a2-33e86178ae5d tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:20:57 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-e48c1f07-127f-4b50-94a2-33e86178ae5d tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:20:57 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-e48c1f07-127f-4b50-94a2-33e86178ae5d tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:20:57 user nova-compute[75234]: DEBUG nova.network.neutron [None req-e48c1f07-127f-4b50-94a2-33e86178ae5d tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] [instance: 65d48c93-b6da-4f51-9d5a-0676dda62a60] Successfully updated port: 967835a2-4b55-4ae6-b071-91233af412fd {{(pid=75234) _update_port /opt/stack/nova/nova/network/neutron.py:586}}
Nov 29 03:20:57 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-e48c1f07-127f-4b50-94a2-33e86178ae5d tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Acquiring lock "refresh_cache-65d48c93-b6da-4f51-9d5a-0676dda62a60" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:20:57 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-e48c1f07-127f-4b50-94a2-33e86178ae5d tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Acquired lock "refresh_cache-65d48c93-b6da-4f51-9d5a-0676dda62a60" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:20:57 user nova-compute[75234]: DEBUG nova.network.neutron [None req-e48c1f07-127f-4b50-94a2-33e86178ae5d tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] [instance: 65d48c93-b6da-4f51-9d5a-0676dda62a60] Building network info cache for instance {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2009}}
Nov 29 03:20:57 user nova-compute[75234]: DEBUG nova.compute.manager [req-1785f4c9-c13e-4ef9-9c74-aaca6da002f2 req-70232499-bfe2-444d-b7e2-bb4386f974c3 service nova] [instance: 65d48c93-b6da-4f51-9d5a-0676dda62a60] Received event network-changed-967835a2-4b55-4ae6-b071-91233af412fd {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:20:57 user nova-compute[75234]: DEBUG nova.compute.manager [req-1785f4c9-c13e-4ef9-9c74-aaca6da002f2 req-70232499-bfe2-444d-b7e2-bb4386f974c3 service nova] [instance: 65d48c93-b6da-4f51-9d5a-0676dda62a60] Refreshing instance network info cache due to event network-changed-967835a2-4b55-4ae6-b071-91233af412fd. {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11117}}
Nov 29 03:20:57 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-1785f4c9-c13e-4ef9-9c74-aaca6da002f2 req-70232499-bfe2-444d-b7e2-bb4386f974c3 service nova] Acquiring lock "refresh_cache-65d48c93-b6da-4f51-9d5a-0676dda62a60" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:20:57 user nova-compute[75234]: DEBUG nova.network.neutron [None req-e48c1f07-127f-4b50-94a2-33e86178ae5d tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] [instance: 65d48c93-b6da-4f51-9d5a-0676dda62a60] Instance cache missing network info. {{(pid=75234) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3322}}
Nov 29 03:20:58 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:20:58 user nova-compute[75234]: DEBUG nova.network.neutron [None req-e48c1f07-127f-4b50-94a2-33e86178ae5d tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] [instance: 65d48c93-b6da-4f51-9d5a-0676dda62a60] Updating instance_info_cache with network_info: [{"id": "967835a2-4b55-4ae6-b071-91233af412fd", "address": "fa:16:3e:ee:3c:e2", "network": {"id": "4d086e08-7798-4cfe-9365-86be6f0a52d2", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-305624008-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "db00bcd0c82c44f1bda739e9857f6e84", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap967835a2-4b", "ovs_interfaceid": "967835a2-4b55-4ae6-b071-91233af412fd", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:20:58 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-e48c1f07-127f-4b50-94a2-33e86178ae5d tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Releasing lock "refresh_cache-65d48c93-b6da-4f51-9d5a-0676dda62a60" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:20:58 user nova-compute[75234]: DEBUG nova.compute.manager [None req-e48c1f07-127f-4b50-94a2-33e86178ae5d tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] [instance: 65d48c93-b6da-4f51-9d5a-0676dda62a60] Instance network_info: |[{"id": "967835a2-4b55-4ae6-b071-91233af412fd", "address": "fa:16:3e:ee:3c:e2", "network": {"id": "4d086e08-7798-4cfe-9365-86be6f0a52d2", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-305624008-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "db00bcd0c82c44f1bda739e9857f6e84", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap967835a2-4b", "ovs_interfaceid": "967835a2-4b55-4ae6-b071-91233af412fd", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=75234) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1987}}
Nov 29 03:20:58 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-1785f4c9-c13e-4ef9-9c74-aaca6da002f2 req-70232499-bfe2-444d-b7e2-bb4386f974c3 service nova] Acquired lock "refresh_cache-65d48c93-b6da-4f51-9d5a-0676dda62a60" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:20:58 user nova-compute[75234]: DEBUG nova.network.neutron [req-1785f4c9-c13e-4ef9-9c74-aaca6da002f2 req-70232499-bfe2-444d-b7e2-bb4386f974c3 service nova] [instance: 65d48c93-b6da-4f51-9d5a-0676dda62a60] Refreshing network info cache for port 967835a2-4b55-4ae6-b071-91233af412fd {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2006}}
Nov 29 03:20:58 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-e48c1f07-127f-4b50-94a2-33e86178ae5d tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] [instance: 65d48c93-b6da-4f51-9d5a-0676dda62a60] Start _get_guest_xml network_info=[{"id": "967835a2-4b55-4ae6-b071-91233af412fd", "address": "fa:16:3e:ee:3c:e2", "network": {"id": "4d086e08-7798-4cfe-9365-86be6f0a52d2", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-305624008-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "db00bcd0c82c44f1bda739e9857f6e84", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap967835a2-4b", "ovs_interfaceid": "967835a2-4b55-4ae6-b071-91233af412fd", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum='c8fc807773e5354afe61636071771906',container_format='bare',created_at=2023-11-29T03:02:15Z,direct_url=<?>,disk_format='qcow2',id=96c30ba7-9b73-4a30-8ba4-7138879573b7,min_disk=0,min_ram=0,name='cirros-0.6.2-x86_64-disk',owner='b18e380921c748b78517b6f1211cd0f1',properties=ImageMetaProps,protected=<?>,size=21430272,status='active',tags=<?>,updated_at=2023-11-29T03:02:17Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [{'device_name': '/dev/vda', 'encryption_format': None, 'encrypted': False, 'device_type': 'disk', 'size': 0, 'disk_bus': 'virtio', 'boot_index': 0, 'encryption_options': None, 'encryption_secret_uuid': None, 'guest_format': None, 'image_id': '96c30ba7-9b73-4a30-8ba4-7138879573b7'}], 'ephemerals': [], 'block_device_mapping': [], 'swap': None} {{(pid=75234) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7562}}
Nov 29 03:20:58 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-e48c1f07-127f-4b50-94a2-33e86178ae5d tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:20:58 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-e48c1f07-127f-4b50-94a2-33e86178ae5d tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:20:58 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-e48c1f07-127f-4b50-94a2-33e86178ae5d tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Searching host: 'user' for CPU controller through CGroups V1... {{(pid=75234) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1652}}
Nov 29 03:20:58 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-e48c1f07-127f-4b50-94a2-33e86178ae5d tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] CPU controller missing on host. {{(pid=75234) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1662}}
Nov 29 03:20:58 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-e48c1f07-127f-4b50-94a2-33e86178ae5d tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Searching host: 'user' for CPU controller through CGroups V2... {{(pid=75234) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1671}}
Nov 29 03:20:58 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-e48c1f07-127f-4b50-94a2-33e86178ae5d tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] CPU controller found on host. {{(pid=75234) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1678}}
Nov 29 03:20:58 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-e48c1f07-127f-4b50-94a2-33e86178ae5d tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=75234) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5400}}
Nov 29 03:20:58 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-e48c1f07-127f-4b50-94a2-33e86178ae5d tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-11-29T03:02:04Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='1',id=6,is_public=True,memory_mb=512,name='m1.tiny',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='c8fc807773e5354afe61636071771906',container_format='bare',created_at=2023-11-29T03:02:15Z,direct_url=<?>,disk_format='qcow2',id=96c30ba7-9b73-4a30-8ba4-7138879573b7,min_disk=0,min_ram=0,name='cirros-0.6.2-x86_64-disk',owner='b18e380921c748b78517b6f1211cd0f1',properties=ImageMetaProps,protected=<?>,size=21430272,status='active',tags=<?>,updated_at=2023-11-29T03:02:17Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:563}}
Nov 29 03:20:58 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-e48c1f07-127f-4b50-94a2-33e86178ae5d tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Flavor limits 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:348}}
Nov 29 03:20:58 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-e48c1f07-127f-4b50-94a2-33e86178ae5d tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Image limits 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:352}}
Nov 29 03:20:58 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-e48c1f07-127f-4b50-94a2-33e86178ae5d tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Flavor pref 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:388}}
Nov 29 03:20:58 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-e48c1f07-127f-4b50-94a2-33e86178ae5d tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Image pref 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:392}}
Nov 29 03:20:58 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-e48c1f07-127f-4b50-94a2-33e86178ae5d tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:430}}
Nov 29 03:20:58 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-e48c1f07-127f-4b50-94a2-33e86178ae5d tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:569}}
Nov 29 03:20:58 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-e48c1f07-127f-4b50-94a2-33e86178ae5d tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=75234) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:471}}
Nov 29 03:20:58 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-e48c1f07-127f-4b50-94a2-33e86178ae5d tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Got 1 possible topologies {{(pid=75234) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:501}}
Nov 29 03:20:58 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-e48c1f07-127f-4b50-94a2-33e86178ae5d tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:575}}
Nov 29 03:20:58 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-e48c1f07-127f-4b50-94a2-33e86178ae5d tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:577}}
Nov 29 03:20:58 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-e48c1f07-127f-4b50-94a2-33e86178ae5d tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-11-29T03:20:55Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description=None,display_name='tempest-ServerBootFromVolumeStableRescueTest-server-378302862',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-serverbootfromvolumestablerescuetest-server-378302862',id=36,image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data=None,key_name=None,keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='db00bcd0c82c44f1bda739e9857f6e84',ramdisk_id='',reservation_id='r-ihbm4mrc',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-ServerBootFromVolumeStableRescueTest-1601266069',owner_user_name='tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-11-29T03:20:56Z,user_data=None,user_id='877aa877f13a4f84ac4725278010aea3',uuid=65d48c93-b6da-4f51-9d5a-0676dda62a60,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "967835a2-4b55-4ae6-b071-91233af412fd", "address": "fa:16:3e:ee:3c:e2", "network": {"id": "4d086e08-7798-4cfe-9365-86be6f0a52d2", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-305624008-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "db00bcd0c82c44f1bda739e9857f6e84", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap967835a2-4b", "ovs_interfaceid": "967835a2-4b55-4ae6-b071-91233af412fd", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=75234) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Nov 29 03:20:58 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-e48c1f07-127f-4b50-94a2-33e86178ae5d tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Converting VIF {"id": "967835a2-4b55-4ae6-b071-91233af412fd", "address": "fa:16:3e:ee:3c:e2", "network": {"id": "4d086e08-7798-4cfe-9365-86be6f0a52d2", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-305624008-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "db00bcd0c82c44f1bda739e9857f6e84", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap967835a2-4b", "ovs_interfaceid": "967835a2-4b55-4ae6-b071-91233af412fd", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:20:58 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-e48c1f07-127f-4b50-94a2-33e86178ae5d tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:ee:3c:e2,bridge_name='br-int',has_traffic_filtering=True,id=967835a2-4b55-4ae6-b071-91233af412fd,network=Network(4d086e08-7798-4cfe-9365-86be6f0a52d2),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap967835a2-4b') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:20:58 user nova-compute[75234]: DEBUG nova.objects.instance [None req-e48c1f07-127f-4b50-94a2-33e86178ae5d tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Lazy-loading 'pci_devices' on Instance uuid 65d48c93-b6da-4f51-9d5a-0676dda62a60 {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:20:58 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-e48c1f07-127f-4b50-94a2-33e86178ae5d tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] [instance: 65d48c93-b6da-4f51-9d5a-0676dda62a60] End _get_guest_xml xml=<domain type="kvm">
Nov 29 03:20:58 user nova-compute[75234]:   <uuid>65d48c93-b6da-4f51-9d5a-0676dda62a60</uuid>
Nov 29 03:20:58 user nova-compute[75234]:   <name>instance-00000024</name>
Nov 29 03:20:58 user nova-compute[75234]:   <memory>524288</memory>
Nov 29 03:20:58 user nova-compute[75234]:   <vcpu>1</vcpu>
Nov 29 03:20:58 user nova-compute[75234]:   <metadata>
Nov 29 03:20:58 user nova-compute[75234]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Nov 29 03:20:58 user nova-compute[75234]:       <nova:package version="0.0.0"/>
Nov 29 03:20:58 user nova-compute[75234]:       <nova:name>tempest-ServerBootFromVolumeStableRescueTest-server-378302862</nova:name>
Nov 29 03:20:58 user nova-compute[75234]:       <nova:creationTime>2023-11-29 03:20:58</nova:creationTime>
Nov 29 03:20:58 user nova-compute[75234]:       <nova:flavor name="m1.tiny">
Nov 29 03:20:58 user nova-compute[75234]:         <nova:memory>512</nova:memory>
Nov 29 03:20:58 user nova-compute[75234]:         <nova:disk>1</nova:disk>
Nov 29 03:20:58 user nova-compute[75234]:         <nova:swap>0</nova:swap>
Nov 29 03:20:58 user nova-compute[75234]:         <nova:ephemeral>0</nova:ephemeral>
Nov 29 03:20:58 user nova-compute[75234]:         <nova:vcpus>1</nova:vcpus>
Nov 29 03:20:58 user nova-compute[75234]:       </nova:flavor>
Nov 29 03:20:58 user nova-compute[75234]:       <nova:owner>
Nov 29 03:20:58 user nova-compute[75234]:         <nova:user uuid="877aa877f13a4f84ac4725278010aea3">tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member</nova:user>
Nov 29 03:20:58 user nova-compute[75234]:         <nova:project uuid="db00bcd0c82c44f1bda739e9857f6e84">tempest-ServerBootFromVolumeStableRescueTest-1601266069</nova:project>
Nov 29 03:20:58 user nova-compute[75234]:       </nova:owner>
Nov 29 03:20:58 user nova-compute[75234]:       <nova:root type="image" uuid="96c30ba7-9b73-4a30-8ba4-7138879573b7"/>
Nov 29 03:20:58 user nova-compute[75234]:       <nova:ports>
Nov 29 03:20:58 user nova-compute[75234]:         <nova:port uuid="967835a2-4b55-4ae6-b071-91233af412fd">
Nov 29 03:20:58 user nova-compute[75234]:           <nova:ip type="fixed" address="10.0.0.11" ipVersion="4"/>
Nov 29 03:20:58 user nova-compute[75234]:         </nova:port>
Nov 29 03:20:58 user nova-compute[75234]:       </nova:ports>
Nov 29 03:20:58 user nova-compute[75234]:     </nova:instance>
Nov 29 03:20:58 user nova-compute[75234]:   </metadata>
Nov 29 03:20:58 user nova-compute[75234]:   <sysinfo type="smbios">
Nov 29 03:20:58 user nova-compute[75234]:     <system>
Nov 29 03:20:58 user nova-compute[75234]:       <entry name="manufacturer">OpenStack Foundation</entry>
Nov 29 03:20:58 user nova-compute[75234]:       <entry name="product">OpenStack Nova</entry>
Nov 29 03:20:58 user nova-compute[75234]:       <entry name="version">0.0.0</entry>
Nov 29 03:20:58 user nova-compute[75234]:       <entry name="serial">65d48c93-b6da-4f51-9d5a-0676dda62a60</entry>
Nov 29 03:20:58 user nova-compute[75234]:       <entry name="uuid">65d48c93-b6da-4f51-9d5a-0676dda62a60</entry>
Nov 29 03:20:58 user nova-compute[75234]:       <entry name="family">Virtual Machine</entry>
Nov 29 03:20:58 user nova-compute[75234]:     </system>
Nov 29 03:20:58 user nova-compute[75234]:   </sysinfo>
Nov 29 03:20:58 user nova-compute[75234]:   <os>
Nov 29 03:20:58 user nova-compute[75234]:     <type arch="x86_64" machine="pc">hvm</type>
Nov 29 03:20:58 user nova-compute[75234]:     <boot dev="hd"/>
Nov 29 03:20:58 user nova-compute[75234]:     <smbios mode="sysinfo"/>
Nov 29 03:20:58 user nova-compute[75234]:   </os>
Nov 29 03:20:58 user nova-compute[75234]:   <features>
Nov 29 03:20:58 user nova-compute[75234]:     <acpi/>
Nov 29 03:20:58 user nova-compute[75234]:     <apic/>
Nov 29 03:20:58 user nova-compute[75234]:     <vmcoreinfo/>
Nov 29 03:20:58 user nova-compute[75234]:   </features>
Nov 29 03:20:58 user nova-compute[75234]:   <clock offset="utc">
Nov 29 03:20:58 user nova-compute[75234]:     <timer name="pit" tickpolicy="delay"/>
Nov 29 03:20:58 user nova-compute[75234]:     <timer name="rtc" tickpolicy="catchup"/>
Nov 29 03:20:58 user nova-compute[75234]:     <timer name="hpet" present="no"/>
Nov 29 03:20:58 user nova-compute[75234]:   </clock>
Nov 29 03:20:58 user nova-compute[75234]:   <cpu mode="custom" match="exact">
Nov 29 03:20:58 user nova-compute[75234]:     <model>Nehalem</model>
Nov 29 03:20:58 user nova-compute[75234]:     <topology sockets="1" cores="1" threads="1"/>
Nov 29 03:20:58 user nova-compute[75234]:   </cpu>
Nov 29 03:20:58 user nova-compute[75234]:   <devices>
Nov 29 03:20:58 user nova-compute[75234]:     <disk type="file" device="disk">
Nov 29 03:20:58 user nova-compute[75234]:       <driver name="qemu" type="qcow2" cache="none"/>
Nov 29 03:20:58 user nova-compute[75234]:       <source file="/opt/stack/data/nova/instances/65d48c93-b6da-4f51-9d5a-0676dda62a60/disk"/>
Nov 29 03:20:58 user nova-compute[75234]:       <target dev="vda" bus="virtio"/>
Nov 29 03:20:58 user nova-compute[75234]:     </disk>
Nov 29 03:20:58 user nova-compute[75234]:     <interface type="ethernet">
Nov 29 03:20:58 user nova-compute[75234]:       <mac address="fa:16:3e:ee:3c:e2"/>
Nov 29 03:20:58 user nova-compute[75234]:       <model type="virtio"/>
Nov 29 03:20:58 user nova-compute[75234]:       <mtu size="1442"/>
Nov 29 03:20:58 user nova-compute[75234]:       <target dev="tap967835a2-4b"/>
Nov 29 03:20:58 user nova-compute[75234]:     </interface>
Nov 29 03:20:58 user nova-compute[75234]:     <serial type="pty">
Nov 29 03:20:58 user nova-compute[75234]:       <log file="/opt/stack/data/nova/instances/65d48c93-b6da-4f51-9d5a-0676dda62a60/console.log" append="off"/>
Nov 29 03:20:58 user nova-compute[75234]:     </serial>
Nov 29 03:20:58 user nova-compute[75234]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Nov 29 03:20:58 user nova-compute[75234]:     <video>
Nov 29 03:20:58 user nova-compute[75234]:       <model type="virtio"/>
Nov 29 03:20:58 user nova-compute[75234]:     </video>
Nov 29 03:20:58 user nova-compute[75234]:     <rng model="virtio">
Nov 29 03:20:58 user nova-compute[75234]:       <backend model="random">/dev/urandom</backend>
Nov 29 03:20:58 user nova-compute[75234]:     </rng>
Nov 29 03:20:58 user nova-compute[75234]:     <controller type="usb" index="0" model="none"/>
Nov 29 03:20:58 user nova-compute[75234]:     <memballoon model="virtio">
Nov 29 03:20:58 user nova-compute[75234]:       <stats period="10"/>
Nov 29 03:20:58 user nova-compute[75234]:     </memballoon>
Nov 29 03:20:58 user nova-compute[75234]:   </devices>
Nov 29 03:20:58 user nova-compute[75234]: </domain>
Nov 29 03:20:58 user nova-compute[75234]:  {{(pid=75234) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7568}}
Nov 29 03:20:58 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-e48c1f07-127f-4b50-94a2-33e86178ae5d tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-11-29T03:20:55Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description=None,display_name='tempest-ServerBootFromVolumeStableRescueTest-server-378302862',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-serverbootfromvolumestablerescuetest-server-378302862',id=36,image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data=None,key_name=None,keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='db00bcd0c82c44f1bda739e9857f6e84',ramdisk_id='',reservation_id='r-ihbm4mrc',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-ServerBootFromVolumeStableRescueTest-1601266069',owner_user_name='tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-11-29T03:20:56Z,user_data=None,user_id='877aa877f13a4f84ac4725278010aea3',uuid=65d48c93-b6da-4f51-9d5a-0676dda62a60,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "967835a2-4b55-4ae6-b071-91233af412fd", "address": "fa:16:3e:ee:3c:e2", "network": {"id": "4d086e08-7798-4cfe-9365-86be6f0a52d2", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-305624008-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "db00bcd0c82c44f1bda739e9857f6e84", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap967835a2-4b", "ovs_interfaceid": "967835a2-4b55-4ae6-b071-91233af412fd", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Nov 29 03:20:58 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-e48c1f07-127f-4b50-94a2-33e86178ae5d tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Converting VIF {"id": "967835a2-4b55-4ae6-b071-91233af412fd", "address": "fa:16:3e:ee:3c:e2", "network": {"id": "4d086e08-7798-4cfe-9365-86be6f0a52d2", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-305624008-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "db00bcd0c82c44f1bda739e9857f6e84", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap967835a2-4b", "ovs_interfaceid": "967835a2-4b55-4ae6-b071-91233af412fd", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:20:58 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-e48c1f07-127f-4b50-94a2-33e86178ae5d tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:ee:3c:e2,bridge_name='br-int',has_traffic_filtering=True,id=967835a2-4b55-4ae6-b071-91233af412fd,network=Network(4d086e08-7798-4cfe-9365-86be6f0a52d2),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap967835a2-4b') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:20:58 user nova-compute[75234]: DEBUG os_vif [None req-e48c1f07-127f-4b50-94a2-33e86178ae5d tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:ee:3c:e2,bridge_name='br-int',has_traffic_filtering=True,id=967835a2-4b55-4ae6-b071-91233af412fd,network=Network(4d086e08-7798-4cfe-9365-86be6f0a52d2),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap967835a2-4b') {{(pid=75234) plug /opt/stack/data/venv/lib/python3.10/site-packages/os_vif/__init__.py:76}}
Nov 29 03:20:58 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:20:58 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:20:58 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Nov 29 03:20:58 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:20:58 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbCreateCommand(_result=None, table=QoS, columns={'type': 'linux-noop', 'external_ids': {'id': 'c61e83e5-a425-576b-9cea-9c24009ee98a', '_type': 'linux-noop'}}, row=False) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:20:58 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:20:58 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:20:58 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:20:58 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tap967835a2-4b, may_exist=True, interface_attrs={}) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:20:58 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Port, record=tap967835a2-4b, col_values=(('qos', UUID('4b2e8007-c067-47e8-bcdb-22ee4cd31118')),), if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:20:58 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DbSetCommand(_result=None, table=Interface, record=tap967835a2-4b, col_values=(('external_ids', {'iface-id': '967835a2-4b55-4ae6-b071-91233af412fd', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:ee:3c:e2', 'vm-uuid': '65d48c93-b6da-4f51-9d5a-0676dda62a60'}),), if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:20:58 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:20:58 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:20:58 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:20:58 user nova-compute[75234]: INFO os_vif [None req-e48c1f07-127f-4b50-94a2-33e86178ae5d tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:ee:3c:e2,bridge_name='br-int',has_traffic_filtering=True,id=967835a2-4b55-4ae6-b071-91233af412fd,network=Network(4d086e08-7798-4cfe-9365-86be6f0a52d2),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap967835a2-4b')
Nov 29 03:20:58 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-e48c1f07-127f-4b50-94a2-33e86178ae5d tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] No BDM found with device name vda, not building metadata. {{(pid=75234) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12089}}
Nov 29 03:20:58 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-e48c1f07-127f-4b50-94a2-33e86178ae5d tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] No VIF found with MAC fa:16:3e:ee:3c:e2, not building metadata {{(pid=75234) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Nov 29 03:20:58 user nova-compute[75234]: DEBUG nova.network.neutron [req-1785f4c9-c13e-4ef9-9c74-aaca6da002f2 req-70232499-bfe2-444d-b7e2-bb4386f974c3 service nova] [instance: 65d48c93-b6da-4f51-9d5a-0676dda62a60] Updated VIF entry in instance network info cache for port 967835a2-4b55-4ae6-b071-91233af412fd. {{(pid=75234) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3481}}
Nov 29 03:20:58 user nova-compute[75234]: DEBUG nova.network.neutron [req-1785f4c9-c13e-4ef9-9c74-aaca6da002f2 req-70232499-bfe2-444d-b7e2-bb4386f974c3 service nova] [instance: 65d48c93-b6da-4f51-9d5a-0676dda62a60] Updating instance_info_cache with network_info: [{"id": "967835a2-4b55-4ae6-b071-91233af412fd", "address": "fa:16:3e:ee:3c:e2", "network": {"id": "4d086e08-7798-4cfe-9365-86be6f0a52d2", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-305624008-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "db00bcd0c82c44f1bda739e9857f6e84", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap967835a2-4b", "ovs_interfaceid": "967835a2-4b55-4ae6-b071-91233af412fd", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:20:58 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-1785f4c9-c13e-4ef9-9c74-aaca6da002f2 req-70232499-bfe2-444d-b7e2-bb4386f974c3 service nova] Releasing lock "refresh_cache-65d48c93-b6da-4f51-9d5a-0676dda62a60" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:20:59 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:21:00 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:21:00 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:21:00 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:21:00 user nova-compute[75234]: DEBUG nova.compute.manager [req-cc564109-f970-4287-810f-17d085f59bdb req-71d1fc94-533c-411f-ad44-005444ce5b7e service nova] [instance: 65d48c93-b6da-4f51-9d5a-0676dda62a60] Received event network-vif-plugged-967835a2-4b55-4ae6-b071-91233af412fd {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:21:00 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-cc564109-f970-4287-810f-17d085f59bdb req-71d1fc94-533c-411f-ad44-005444ce5b7e service nova] Acquiring lock "65d48c93-b6da-4f51-9d5a-0676dda62a60-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:21:00 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-cc564109-f970-4287-810f-17d085f59bdb req-71d1fc94-533c-411f-ad44-005444ce5b7e service nova] Lock "65d48c93-b6da-4f51-9d5a-0676dda62a60-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:21:00 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-cc564109-f970-4287-810f-17d085f59bdb req-71d1fc94-533c-411f-ad44-005444ce5b7e service nova] Lock "65d48c93-b6da-4f51-9d5a-0676dda62a60-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:21:00 user nova-compute[75234]: DEBUG nova.compute.manager [req-cc564109-f970-4287-810f-17d085f59bdb req-71d1fc94-533c-411f-ad44-005444ce5b7e service nova] [instance: 65d48c93-b6da-4f51-9d5a-0676dda62a60] No waiting events found dispatching network-vif-plugged-967835a2-4b55-4ae6-b071-91233af412fd {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:21:00 user nova-compute[75234]: WARNING nova.compute.manager [req-cc564109-f970-4287-810f-17d085f59bdb req-71d1fc94-533c-411f-ad44-005444ce5b7e service nova] [instance: 65d48c93-b6da-4f51-9d5a-0676dda62a60] Received unexpected event network-vif-plugged-967835a2-4b55-4ae6-b071-91233af412fd for instance with vm_state building and task_state spawning.
Nov 29 03:21:00 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:21:00 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:21:00 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:21:00 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:21:01 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:21:01 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Starting heal instance info cache {{(pid=75234) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9922}}
Nov 29 03:21:01 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Rebuilding the list of instances to heal {{(pid=75234) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9926}}
Nov 29 03:21:01 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] [instance: 65d48c93-b6da-4f51-9d5a-0676dda62a60] Skipping network cache update for instance because it is Building. {{(pid=75234) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9935}}
Nov 29 03:21:01 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Acquiring lock "refresh_cache-21e9c6cd-c39f-41c0-8768-5dd64589c3f3" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:21:01 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Acquired lock "refresh_cache-21e9c6cd-c39f-41c0-8768-5dd64589c3f3" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:21:01 user nova-compute[75234]: DEBUG nova.network.neutron [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] [instance: 21e9c6cd-c39f-41c0-8768-5dd64589c3f3] Forcefully refreshing network info cache for instance {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2003}}
Nov 29 03:21:01 user nova-compute[75234]: DEBUG nova.objects.instance [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lazy-loading 'info_cache' on Instance uuid 21e9c6cd-c39f-41c0-8768-5dd64589c3f3 {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:21:01 user nova-compute[75234]: DEBUG nova.network.neutron [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] [instance: 21e9c6cd-c39f-41c0-8768-5dd64589c3f3] Updating instance_info_cache with network_info: [{"id": "87f1e736-6665-4945-bd19-0cf93bff905e", "address": "fa:16:3e:40:14:df", "network": {"id": "1f946e69-dbec-429c-83f8-a186bad4f2cc", "bridge": "br-int", "label": "tempest-ServerActionsTestOtherA-486369423-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "c1cb3d1f12f242a785a688d59dbe3cd6", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap87f1e736-66", "ovs_interfaceid": "87f1e736-6665-4945-bd19-0cf93bff905e", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:21:01 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Releasing lock "refresh_cache-21e9c6cd-c39f-41c0-8768-5dd64589c3f3" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:21:01 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] [instance: 21e9c6cd-c39f-41c0-8768-5dd64589c3f3] Updated the network info_cache for instance {{(pid=75234) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9993}}
Nov 29 03:21:02 user nova-compute[75234]: DEBUG nova.compute.manager [None req-e48c1f07-127f-4b50-94a2-33e86178ae5d tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] [instance: 65d48c93-b6da-4f51-9d5a-0676dda62a60] Instance event wait completed in 0 seconds for  {{(pid=75234) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Nov 29 03:21:02 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-e48c1f07-127f-4b50-94a2-33e86178ae5d tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] [instance: 65d48c93-b6da-4f51-9d5a-0676dda62a60] Guest created on hypervisor {{(pid=75234) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4421}}
Nov 29 03:21:02 user nova-compute[75234]: INFO nova.virt.libvirt.driver [-] [instance: 65d48c93-b6da-4f51-9d5a-0676dda62a60] Instance spawned successfully.
Nov 29 03:21:02 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-e48c1f07-127f-4b50-94a2-33e86178ae5d tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] [instance: 65d48c93-b6da-4f51-9d5a-0676dda62a60] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:904}}
Nov 29 03:21:02 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-e48c1f07-127f-4b50-94a2-33e86178ae5d tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] [instance: 65d48c93-b6da-4f51-9d5a-0676dda62a60] Found default for hw_cdrom_bus of ide {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:21:02 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-e48c1f07-127f-4b50-94a2-33e86178ae5d tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] [instance: 65d48c93-b6da-4f51-9d5a-0676dda62a60] Found default for hw_disk_bus of virtio {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:21:02 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-e48c1f07-127f-4b50-94a2-33e86178ae5d tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] [instance: 65d48c93-b6da-4f51-9d5a-0676dda62a60] Found default for hw_input_bus of None {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:21:02 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-e48c1f07-127f-4b50-94a2-33e86178ae5d tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] [instance: 65d48c93-b6da-4f51-9d5a-0676dda62a60] Found default for hw_pointer_model of None {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:21:02 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-e48c1f07-127f-4b50-94a2-33e86178ae5d tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] [instance: 65d48c93-b6da-4f51-9d5a-0676dda62a60] Found default for hw_video_model of virtio {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:21:02 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-e48c1f07-127f-4b50-94a2-33e86178ae5d tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] [instance: 65d48c93-b6da-4f51-9d5a-0676dda62a60] Found default for hw_vif_model of virtio {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:21:02 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:21:02 user nova-compute[75234]: DEBUG nova.compute.manager [req-0b21914e-51be-4970-8494-e63d00908db8 req-e1915de3-8a02-48ca-a0cb-9f0024eee380 service nova] [instance: 65d48c93-b6da-4f51-9d5a-0676dda62a60] Received event network-vif-plugged-967835a2-4b55-4ae6-b071-91233af412fd {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:21:02 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-0b21914e-51be-4970-8494-e63d00908db8 req-e1915de3-8a02-48ca-a0cb-9f0024eee380 service nova] Acquiring lock "65d48c93-b6da-4f51-9d5a-0676dda62a60-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:21:02 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-0b21914e-51be-4970-8494-e63d00908db8 req-e1915de3-8a02-48ca-a0cb-9f0024eee380 service nova] Lock "65d48c93-b6da-4f51-9d5a-0676dda62a60-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:21:02 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-0b21914e-51be-4970-8494-e63d00908db8 req-e1915de3-8a02-48ca-a0cb-9f0024eee380 service nova] Lock "65d48c93-b6da-4f51-9d5a-0676dda62a60-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:21:02 user nova-compute[75234]: DEBUG nova.compute.manager [req-0b21914e-51be-4970-8494-e63d00908db8 req-e1915de3-8a02-48ca-a0cb-9f0024eee380 service nova] [instance: 65d48c93-b6da-4f51-9d5a-0676dda62a60] No waiting events found dispatching network-vif-plugged-967835a2-4b55-4ae6-b071-91233af412fd {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:21:02 user nova-compute[75234]: WARNING nova.compute.manager [req-0b21914e-51be-4970-8494-e63d00908db8 req-e1915de3-8a02-48ca-a0cb-9f0024eee380 service nova] [instance: 65d48c93-b6da-4f51-9d5a-0676dda62a60] Received unexpected event network-vif-plugged-967835a2-4b55-4ae6-b071-91233af412fd for instance with vm_state building and task_state spawning.
Nov 29 03:21:02 user nova-compute[75234]: INFO nova.compute.manager [None req-e48c1f07-127f-4b50-94a2-33e86178ae5d tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] [instance: 65d48c93-b6da-4f51-9d5a-0676dda62a60] Took 5.56 seconds to spawn the instance on the hypervisor.
Nov 29 03:21:02 user nova-compute[75234]: DEBUG nova.compute.manager [None req-e48c1f07-127f-4b50-94a2-33e86178ae5d tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] [instance: 65d48c93-b6da-4f51-9d5a-0676dda62a60] Checking state {{(pid=75234) _get_power_state /opt/stack/nova/nova/compute/manager.py:1782}}
Nov 29 03:21:02 user nova-compute[75234]: DEBUG nova.compute.manager [req-d2600b7f-18e0-4b89-ad01-c8ba7bcb0e7b req-47553458-4381-43ac-b9bf-e0fb517ad4ee service nova] [instance: 1df140f1-fbd9-4b5d-b356-d08b7e908f64] Received event network-changed-ea9a3586-da52-49b7-8ced-5d27806885ad {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:21:02 user nova-compute[75234]: DEBUG nova.compute.manager [req-d2600b7f-18e0-4b89-ad01-c8ba7bcb0e7b req-47553458-4381-43ac-b9bf-e0fb517ad4ee service nova] [instance: 1df140f1-fbd9-4b5d-b356-d08b7e908f64] Refreshing instance network info cache due to event network-changed-ea9a3586-da52-49b7-8ced-5d27806885ad. {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11117}}
Nov 29 03:21:02 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-d2600b7f-18e0-4b89-ad01-c8ba7bcb0e7b req-47553458-4381-43ac-b9bf-e0fb517ad4ee service nova] Acquiring lock "refresh_cache-1df140f1-fbd9-4b5d-b356-d08b7e908f64" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:21:02 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-d2600b7f-18e0-4b89-ad01-c8ba7bcb0e7b req-47553458-4381-43ac-b9bf-e0fb517ad4ee service nova] Acquired lock "refresh_cache-1df140f1-fbd9-4b5d-b356-d08b7e908f64" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:21:02 user nova-compute[75234]: DEBUG nova.network.neutron [req-d2600b7f-18e0-4b89-ad01-c8ba7bcb0e7b req-47553458-4381-43ac-b9bf-e0fb517ad4ee service nova] [instance: 1df140f1-fbd9-4b5d-b356-d08b7e908f64] Refreshing network info cache for port ea9a3586-da52-49b7-8ced-5d27806885ad {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2006}}
Nov 29 03:21:02 user nova-compute[75234]: INFO nova.compute.manager [None req-e48c1f07-127f-4b50-94a2-33e86178ae5d tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] [instance: 65d48c93-b6da-4f51-9d5a-0676dda62a60] Took 6.47 seconds to build instance.
Nov 29 03:21:02 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-e48c1f07-127f-4b50-94a2-33e86178ae5d tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Lock "65d48c93-b6da-4f51-9d5a-0676dda62a60" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 6.575s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:21:02 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager.update_available_resource {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:21:02 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:21:02 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:21:02 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:21:02 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Auditing locally available compute resources for user (node: user) {{(pid=75234) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:907}}
Nov 29 03:21:02 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] skipping disk /dev/sdb (vda) as it is a volume {{(pid=75234) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11213}}
Nov 29 03:21:02 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/13e5da64-9adc-4479-81c7-d329e9b5f08f/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:21:02 user nova-compute[75234]: DEBUG nova.network.neutron [req-d2600b7f-18e0-4b89-ad01-c8ba7bcb0e7b req-47553458-4381-43ac-b9bf-e0fb517ad4ee service nova] [instance: 1df140f1-fbd9-4b5d-b356-d08b7e908f64] Updated VIF entry in instance network info cache for port ea9a3586-da52-49b7-8ced-5d27806885ad. {{(pid=75234) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3481}}
Nov 29 03:21:02 user nova-compute[75234]: DEBUG nova.network.neutron [req-d2600b7f-18e0-4b89-ad01-c8ba7bcb0e7b req-47553458-4381-43ac-b9bf-e0fb517ad4ee service nova] [instance: 1df140f1-fbd9-4b5d-b356-d08b7e908f64] Updating instance_info_cache with network_info: [{"id": "ea9a3586-da52-49b7-8ced-5d27806885ad", "address": "fa:16:3e:80:25:b8", "network": {"id": "2b3c3140-f3b6-45da-8202-bb4d7f0dd130", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-2043420778-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.158", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "f8a7884867c944e1ba1024270e4902f3", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapea9a3586-da", "ovs_interfaceid": "ea9a3586-da52-49b7-8ced-5d27806885ad", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:21:02 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-d2600b7f-18e0-4b89-ad01-c8ba7bcb0e7b req-47553458-4381-43ac-b9bf-e0fb517ad4ee service nova] Releasing lock "refresh_cache-1df140f1-fbd9-4b5d-b356-d08b7e908f64" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:21:02 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/13e5da64-9adc-4479-81c7-d329e9b5f08f/disk --force-share --output=json" returned: 0 in 0.211s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:21:02 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/13e5da64-9adc-4479-81c7-d329e9b5f08f/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:21:02 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ca328ca9-8de1-4354-bb6a-6bb3035177e5 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Acquiring lock "1df140f1-fbd9-4b5d-b356-d08b7e908f64" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:21:02 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ca328ca9-8de1-4354-bb6a-6bb3035177e5 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Lock "1df140f1-fbd9-4b5d-b356-d08b7e908f64" acquired by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: waited 0.002s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:21:03 user nova-compute[75234]: DEBUG nova.objects.instance [None req-ca328ca9-8de1-4354-bb6a-6bb3035177e5 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Lazy-loading 'flavor' on Instance uuid 1df140f1-fbd9-4b5d-b356-d08b7e908f64 {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:21:03 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ca328ca9-8de1-4354-bb6a-6bb3035177e5 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Lock "1df140f1-fbd9-4b5d-b356-d08b7e908f64" "released" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: held 0.074s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:21:03 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/13e5da64-9adc-4479-81c7-d329e9b5f08f/disk --force-share --output=json" returned: 0 in 0.190s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:21:03 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/99d6f85f-1b77-4390-8b61-908c98f584c0/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:21:03 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ca328ca9-8de1-4354-bb6a-6bb3035177e5 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Acquiring lock "1df140f1-fbd9-4b5d-b356-d08b7e908f64" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:21:03 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ca328ca9-8de1-4354-bb6a-6bb3035177e5 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Lock "1df140f1-fbd9-4b5d-b356-d08b7e908f64" acquired by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: waited 0.002s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:21:03 user nova-compute[75234]: INFO nova.compute.manager [None req-ca328ca9-8de1-4354-bb6a-6bb3035177e5 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] [instance: 1df140f1-fbd9-4b5d-b356-d08b7e908f64] Attaching volume 41f08854-31dd-4172-a5cf-01b0e695c10f to /dev/vdb
Nov 29 03:21:03 user nova-compute[75234]: DEBUG os_brick.utils [None req-ca328ca9-8de1-4354-bb6a-6bb3035177e5 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:176}}
Nov 29 03:21:03 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:21:03 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/99d6f85f-1b77-4390-8b61-908c98f584c0/disk --force-share --output=json" returned: 0 in 0.200s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:21:03 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/99d6f85f-1b77-4390-8b61-908c98f584c0/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:21:03 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.018s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:21:03 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[18b9030e-0e5a-4722-894e-d627006d2800]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:dd1368c5ba2c\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:21:03 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt -v / -n -o SOURCE {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:21:03 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt -v / -n -o SOURCE" returned: 0 in 0.017s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:21:03 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[1d5855e4-1cda-43bd-bf5e-f7f72e3193a2]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:21:03 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:21:03 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.017s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:21:03 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[f40547aa-4991-4714-a404-d0897a432806]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:21:03 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[c7f87e57-d72e-4f37-8784-378273b2dc71]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:21:03 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ca328ca9-8de1-4354-bb6a-6bb3035177e5 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Running cmd (subprocess): nvme version {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:21:03 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ca328ca9-8de1-4354-bb6a-6bb3035177e5 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] 'nvme version' failed. Not Retrying. {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:473}}
Nov 29 03:21:03 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.nvmeof [None req-ca328ca9-8de1-4354-bb6a-6bb3035177e5 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] nvme not present on system {{(pid=75234) nvme_present /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/nvmeof.py:756}}
Nov 29 03:21:03 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:21:03 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:473}}
Nov 29 03:21:03 user nova-compute[75234]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Nov 29 03:21:03 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[5d9ae31f-2315-4328-881e-821f02f11585]: (4, '') {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:21:03 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.lightos [None req-ca328ca9-8de1-4354-bb6a-6bb3035177e5 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=75234) find_dsc /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/lightos.py:98}}
Nov 29 03:21:03 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.lightos [None req-ca328ca9-8de1-4354-bb6a-6bb3035177e5 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=75234) get_connector_properties /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/lightos.py:76}}
Nov 29 03:21:03 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.lightos [None req-ca328ca9-8de1-4354-bb6a-6bb3035177e5 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] LIGHTOS: no hostnqn found. {{(pid=75234) get_connector_properties /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/lightos.py:84}}
Nov 29 03:21:03 user nova-compute[75234]: DEBUG os_brick.utils [None req-ca328ca9-8de1-4354-bb6a-6bb3035177e5 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] <== get_connector_properties: return (161ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:dd1368c5ba2c', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:203}}
Nov 29 03:21:03 user nova-compute[75234]: DEBUG nova.virt.block_device [None req-ca328ca9-8de1-4354-bb6a-6bb3035177e5 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] [instance: 1df140f1-fbd9-4b5d-b356-d08b7e908f64] Updating existing volume attachment record: 08b61ec9-0ec0-4637-808c-d7a48ac2bb0f {{(pid=75234) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Nov 29 03:21:03 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/99d6f85f-1b77-4390-8b61-908c98f584c0/disk --force-share --output=json" returned: 0 in 0.178s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:21:03 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/72ebd454-0275-4c3d-95d2-227ffb045a0c/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:21:03 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:21:03 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/72ebd454-0275-4c3d-95d2-227ffb045a0c/disk --force-share --output=json" returned: 0 in 0.182s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:21:03 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/72ebd454-0275-4c3d-95d2-227ffb045a0c/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:21:03 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/72ebd454-0275-4c3d-95d2-227ffb045a0c/disk --force-share --output=json" returned: 0 in 0.198s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:21:03 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/1df140f1-fbd9-4b5d-b356-d08b7e908f64/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:21:04 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/1df140f1-fbd9-4b5d-b356-d08b7e908f64/disk --force-share --output=json" returned: 0 in 0.188s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:21:04 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/1df140f1-fbd9-4b5d-b356-d08b7e908f64/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:21:04 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/1df140f1-fbd9-4b5d-b356-d08b7e908f64/disk --force-share --output=json" returned: 0 in 0.177s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:21:04 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/65d48c93-b6da-4f51-9d5a-0676dda62a60/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:21:04 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:21:04 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/65d48c93-b6da-4f51-9d5a-0676dda62a60/disk --force-share --output=json" returned: 0 in 0.265s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:21:04 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/65d48c93-b6da-4f51-9d5a-0676dda62a60/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:21:04 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/65d48c93-b6da-4f51-9d5a-0676dda62a60/disk --force-share --output=json" returned: 0 in 0.195s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:21:04 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/946b919b-39ae-4d23-aefd-01952f4b40b6/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:21:04 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/946b919b-39ae-4d23-aefd-01952f4b40b6/disk --force-share --output=json" returned: 0 in 0.194s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:21:04 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/946b919b-39ae-4d23-aefd-01952f4b40b6/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:21:05 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/946b919b-39ae-4d23-aefd-01952f4b40b6/disk --force-share --output=json" returned: 0 in 0.208s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:21:05 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a082c96e-e384-40b6-bcc8-fc5e0e104669/disk.rescue --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:21:05 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a082c96e-e384-40b6-bcc8-fc5e0e104669/disk.rescue --force-share --output=json" returned: 0 in 0.194s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:21:05 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a082c96e-e384-40b6-bcc8-fc5e0e104669/disk.rescue --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:21:05 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a082c96e-e384-40b6-bcc8-fc5e0e104669/disk.rescue --force-share --output=json" returned: 0 in 0.283s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:21:05 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a082c96e-e384-40b6-bcc8-fc5e0e104669/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:21:05 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a082c96e-e384-40b6-bcc8-fc5e0e104669/disk --force-share --output=json" returned: 0 in 0.195s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:21:05 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a082c96e-e384-40b6-bcc8-fc5e0e104669/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:21:06 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a082c96e-e384-40b6-bcc8-fc5e0e104669/disk --force-share --output=json" returned: 0 in 0.201s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:21:06 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/75c5d700-6367-4128-9228-80d2cd24ace7/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:21:06 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/75c5d700-6367-4128-9228-80d2cd24ace7/disk --force-share --output=json" returned: 0 in 0.219s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:21:06 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/75c5d700-6367-4128-9228-80d2cd24ace7/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:21:06 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/75c5d700-6367-4128-9228-80d2cd24ace7/disk --force-share --output=json" returned: 0 in 0.216s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:21:06 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/21e9c6cd-c39f-41c0-8768-5dd64589c3f3/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:21:06 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:21:06 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/21e9c6cd-c39f-41c0-8768-5dd64589c3f3/disk --force-share --output=json" returned: 0 in 0.273s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:21:06 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/21e9c6cd-c39f-41c0-8768-5dd64589c3f3/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:21:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/21e9c6cd-c39f-41c0-8768-5dd64589c3f3/disk --force-share --output=json" returned: 0 in 0.225s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:21:07 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:21:07 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:21:07 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:21:07 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Hypervisor/Node resource view: name=user free_ram=7449MB free_disk=25.259788513183594GB free_vcpus=2 pci_devices=[{"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}] {{(pid=75234) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1106}}
Nov 29 03:21:07 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:21:07 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:21:07 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Applying migration context for instance af699bc2-9daa-4b5a-9b4a-ad92fe857d70 as it has an incoming, in-progress migration 9297d372-3baa-4214-a4fd-ae62460f818a. Migration status is finished {{(pid=75234) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:1016}}
Nov 29 03:21:07 user nova-compute[75234]: INFO nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] [instance: af699bc2-9daa-4b5a-9b4a-ad92fe857d70] Updating resource usage from migration 9297d372-3baa-4214-a4fd-ae62460f818a
Nov 29 03:21:07 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance 21e9c6cd-c39f-41c0-8768-5dd64589c3f3 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:21:07 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance 13e5da64-9adc-4479-81c7-d329e9b5f08f actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:21:07 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance a082c96e-e384-40b6-bcc8-fc5e0e104669 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:21:07 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance 75c5d700-6367-4128-9228-80d2cd24ace7 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:21:07 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance 1df140f1-fbd9-4b5d-b356-d08b7e908f64 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:21:07 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance 72ebd454-0275-4c3d-95d2-227ffb045a0c actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:21:07 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Migration 9297d372-3baa-4214-a4fd-ae62460f818a is active on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 512}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1709}}
Nov 29 03:21:07 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance af699bc2-9daa-4b5a-9b4a-ad92fe857d70 actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 2048, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:21:07 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance 946b919b-39ae-4d23-aefd-01952f4b40b6 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:21:07 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance 99d6f85f-1b77-4390-8b61-908c98f584c0 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:21:07 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance 65d48c93-b6da-4f51-9d5a-0676dda62a60 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:21:07 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Total usable vcpus: 12, total allocated vcpus: 11 {{(pid=75234) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1129}}
Nov 29 03:21:07 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Final resource view: name=user phys_ram=16011MB used_ram=7680MB phys_disk=40GB used_disk=9GB total_vcpus=12 used_vcpus=11 pci_stats=[] {{(pid=75234) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1138}}
Nov 29 03:21:08 user nova-compute[75234]: DEBUG nova.compute.provider_tree [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Inventory has not changed in ProviderTree for provider: 4d291220-7c37-4839-a0e3-b23c66502b74 {{(pid=75234) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Nov 29 03:21:08 user nova-compute[75234]: DEBUG nova.scheduler.client.report [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Inventory has not changed for provider 4d291220-7c37-4839-a0e3-b23c66502b74 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=75234) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Nov 29 03:21:08 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Compute_service record updated for user:user {{(pid=75234) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:1067}}
Nov 29 03:21:08 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.680s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:21:08 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:21:09 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:21:09 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:21:09 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:21:09 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:21:09 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:21:09 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=75234) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10541}}
Nov 29 03:21:09 user nova-compute[75234]: DEBUG nova.compute.manager [req-51c6f4a4-475f-4844-8b9b-d262e32d2af2 req-6ac99c28-44de-4da0-b3ae-add8db354f82 service nova] [instance: 72ebd454-0275-4c3d-95d2-227ffb045a0c] Received event network-changed-7b066f7f-bc63-4a5f-8b1b-20e4611cf387 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:21:09 user nova-compute[75234]: DEBUG nova.compute.manager [req-51c6f4a4-475f-4844-8b9b-d262e32d2af2 req-6ac99c28-44de-4da0-b3ae-add8db354f82 service nova] [instance: 72ebd454-0275-4c3d-95d2-227ffb045a0c] Refreshing instance network info cache due to event network-changed-7b066f7f-bc63-4a5f-8b1b-20e4611cf387. {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11117}}
Nov 29 03:21:09 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-51c6f4a4-475f-4844-8b9b-d262e32d2af2 req-6ac99c28-44de-4da0-b3ae-add8db354f82 service nova] Acquiring lock "refresh_cache-72ebd454-0275-4c3d-95d2-227ffb045a0c" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:21:09 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-51c6f4a4-475f-4844-8b9b-d262e32d2af2 req-6ac99c28-44de-4da0-b3ae-add8db354f82 service nova] Acquired lock "refresh_cache-72ebd454-0275-4c3d-95d2-227ffb045a0c" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:21:09 user nova-compute[75234]: DEBUG nova.network.neutron [req-51c6f4a4-475f-4844-8b9b-d262e32d2af2 req-6ac99c28-44de-4da0-b3ae-add8db354f82 service nova] [instance: 72ebd454-0275-4c3d-95d2-227ffb045a0c] Refreshing network info cache for port 7b066f7f-bc63-4a5f-8b1b-20e4611cf387 {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2006}}
Nov 29 03:21:09 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-bd252e8d-9d3d-443e-8f47-f07d0eeb5b2e tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Acquiring lock "72ebd454-0275-4c3d-95d2-227ffb045a0c" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:21:09 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-bd252e8d-9d3d-443e-8f47-f07d0eeb5b2e tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Lock "72ebd454-0275-4c3d-95d2-227ffb045a0c" acquired by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:21:09 user nova-compute[75234]: DEBUG nova.objects.instance [None req-bd252e8d-9d3d-443e-8f47-f07d0eeb5b2e tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Lazy-loading 'flavor' on Instance uuid 72ebd454-0275-4c3d-95d2-227ffb045a0c {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:21:09 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-bd252e8d-9d3d-443e-8f47-f07d0eeb5b2e tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] [instance: 72ebd454-0275-4c3d-95d2-227ffb045a0c] Ignoring supplied device name: /dev/vdb
Nov 29 03:21:09 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-bd252e8d-9d3d-443e-8f47-f07d0eeb5b2e tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Lock "72ebd454-0275-4c3d-95d2-227ffb045a0c" "released" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: held 0.077s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:21:09 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-226b3276-674a-4a31-873c-36e3f6fb069c tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Waiting for libvirt event about the detach of device vdb with device alias virtio-disk1 from instance 75c5d700-6367-4128-9228-80d2cd24ace7 is timed out.
Nov 29 03:21:09 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-226b3276-674a-4a31-873c-36e3f6fb069c tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Successfully detached device vdb from instance 75c5d700-6367-4128-9228-80d2cd24ace7 from the live domain config.
Nov 29 03:21:09 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-226b3276-674a-4a31-873c-36e3f6fb069c tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] [instance: 75c5d700-6367-4128-9228-80d2cd24ace7] calling os-brick to detach iSCSI Volume {{(pid=75234) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Nov 29 03:21:09 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-226b3276-674a-4a31-873c-36e3f6fb069c tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f8a5050c9a0>, {'target_iqn': 'iqn.2010-10.org.openstack:bf57b967-7fad-4d76-abb2-319fe88f8ef9', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'zuQKYWgv', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdd'}, None), 'kwargs': False}}" {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:176}}
Nov 29 03:21:09 user nova-compute[75234]: WARNING os_brick.initiator.connectors.base [None req-226b3276-674a-4a31-873c-36e3f6fb069c tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Nov 29 03:21:09 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-226b3276-674a-4a31-873c-36e3f6fb069c tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:68}}
Nov 29 03:21:09 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-226b3276-674a-4a31-873c-36e3f6fb069c tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.002s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:73}}
Nov 29 03:21:09 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:21:09 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.021s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:21:09 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[38853159-0840-4d08-b4c6-bf5f9600edfb]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:cf6ab229-533a-424e-bd67-1d54d50ad4be\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:bf57b967-7fad-4d76-abb2-319fe88f8ef9\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:21:09 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-226b3276-674a-4a31-873c-36e3f6fb069c tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS:
Nov 29 03:21:09 user nova-compute[75234]: No targets found.
Nov 29 03:21:09 user nova-compute[75234]: iSNS:
Nov 29 03:21:09 user nova-compute[75234]: No targets found.
Nov 29 03:21:09 user nova-compute[75234]: STATIC:
Nov 29 03:21:09 user nova-compute[75234]: Target: iqn.2010-10.org.openstack:cf6ab229-533a-424e-bd67-1d54d50ad4be
Nov 29 03:21:09 user nova-compute[75234]:         Portal: 172.16.0.220:3260,-1
Nov 29 03:21:09 user nova-compute[75234]:                 Iface Name: default
Nov 29 03:21:09 user nova-compute[75234]: Target: iqn.2010-10.org.openstack:bf57b967-7fad-4d76-abb2-319fe88f8ef9
Nov 29 03:21:09 user nova-compute[75234]:         Portal: 172.16.0.220:3260,-1
Nov 29 03:21:09 user nova-compute[75234]:                 Iface Name: default
Nov 29 03:21:09 user nova-compute[75234]: FIRMWARE:
Nov 29 03:21:09 user nova-compute[75234]: No targets found.
Nov 29 03:21:09 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:21:09 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-226b3276-674a-4a31-873c-36e3f6fb069c tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Regex to get portals from discoverydb: ^SENDTARGETS:
Nov 29 03:21:09 user nova-compute[75234]: .*?^DiscoveryAddress: 172.16.0.220,3260.*?
Nov 29 03:21:09 user nova-compute[75234]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=75234) _get_discoverydb_portals /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:382}}
Nov 29 03:21:09 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-226b3276-674a-4a31-873c-36e3f6fb069c tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:bf57b967-7fad-4d76-abb2-319fe88f8ef9', 0)] {{(pid=75234) _get_connection_devices /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:830}}
Nov 29 03:21:09 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:21:09 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.022s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:21:09 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[73269e73-c829-405d-bd92-a46f7a8708b9]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:bf57b967-7fad-4d76-abb2-319fe88f8ef9\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:cf6ab229-533a-424e-bd67-1d54d50ad4be\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:21:09 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:21:09 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-bd252e8d-9d3d-443e-8f47-f07d0eeb5b2e tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Acquiring lock "72ebd454-0275-4c3d-95d2-227ffb045a0c" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:21:09 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-bd252e8d-9d3d-443e-8f47-f07d0eeb5b2e tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Lock "72ebd454-0275-4c3d-95d2-227ffb045a0c" acquired by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: waited 0.002s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:21:09 user nova-compute[75234]: INFO nova.compute.manager [None req-bd252e8d-9d3d-443e-8f47-f07d0eeb5b2e tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] [instance: 72ebd454-0275-4c3d-95d2-227ffb045a0c] Attaching volume 0e8591d9-0c7f-4575-9e91-7fd99d87327a to /dev/vdb
Nov 29 03:21:09 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.031s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:21:09 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[08583260-7fbb-4088-b1e4-48c774135f4c]: (4, ('tcp: [29] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bf57b967-7fad-4d76-abb2-319fe88f8ef9 (non-flash)\ntcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:cf6ab229-533a-424e-bd67-1d54d50ad4be (non-flash)\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:21:09 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-226b3276-674a-4a31-873c-36e3f6fb069c tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [29] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bf57b967-7fad-4d76-abb2-319fe88f8ef9 (non-flash)
Nov 29 03:21:09 user nova-compute[75234]: tcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:cf6ab229-533a-424e-bd67-1d54d50ad4be (non-flash)
Nov 29 03:21:09 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:21:09 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-226b3276-674a-4a31-873c-36e3f6fb069c tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] iscsi session list stdout=tcp: [29] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:bf57b967-7fad-4d76-abb2-319fe88f8ef9 (non-flash)
Nov 29 03:21:09 user nova-compute[75234]: tcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:cf6ab229-533a-424e-bd67-1d54d50ad4be (non-flash)
Nov 29 03:21:09 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsi_session /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Nov 29 03:21:09 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-226b3276-674a-4a31-873c-36e3f6fb069c tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7f8a2c65e5f0>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:bf57b967-7fad-4d76-abb2-319fe88f8ef9'): ({'sdd'}, set())}) {{(pid=75234) _get_connection_devices /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:863}}
Nov 29 03:21:09 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-226b3276-674a-4a31-873c-36e3f6fb069c tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Removing single pathed devices sdd {{(pid=75234) remove_connection /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:374}}
Nov 29 03:21:09 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:21:09 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.020s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:21:09 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[2fda79fc-02ca-4a7e-babb-29e82df84795]: (4, ('path checker states:\nup                  2\n\npaths: 0\nbusy: False\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:21:09 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdd {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:21:09 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdd" returned: 0 in 0.027s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:21:09 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[3256897c-35af-49c5-9eaa-23265bf3ef5a]: (4, ('ok\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:21:09 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-226b3276-674a-4a31-873c-36e3f6fb069c tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Flushing IO for device /dev/sdd {{(pid=75234) flush_device_io /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:434}}
Nov 29 03:21:09 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdd {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:21:09 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdd" returned: 0 in 0.015s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:21:09 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[80263271-bb87-4f10-8ec0-9141b81e9184]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:21:09 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-226b3276-674a-4a31-873c-36e3f6fb069c tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Remove SCSI device /dev/sdd with /sys/block/sdd/device/delete {{(pid=75234) remove_scsi_device /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:148}}
Nov 29 03:21:09 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdd/device/delete {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:21:09 user nova-compute[75234]: DEBUG os_brick.utils [None req-bd252e8d-9d3d-443e-8f47-f07d0eeb5b2e tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:176}}
Nov 29 03:21:09 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:21:10 user nova-compute[75234]: DEBUG nova.network.neutron [req-51c6f4a4-475f-4844-8b9b-d262e32d2af2 req-6ac99c28-44de-4da0-b3ae-add8db354f82 service nova] [instance: 72ebd454-0275-4c3d-95d2-227ffb045a0c] Updated VIF entry in instance network info cache for port 7b066f7f-bc63-4a5f-8b1b-20e4611cf387. {{(pid=75234) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3481}}
Nov 29 03:21:10 user nova-compute[75234]: DEBUG nova.network.neutron [req-51c6f4a4-475f-4844-8b9b-d262e32d2af2 req-6ac99c28-44de-4da0-b3ae-add8db354f82 service nova] [instance: 72ebd454-0275-4c3d-95d2-227ffb045a0c] Updating instance_info_cache with network_info: [{"id": "7b066f7f-bc63-4a5f-8b1b-20e4611cf387", "address": "fa:16:3e:52:a7:f8", "network": {"id": "1898605d-00e7-4e81-964e-821e09f5421c", "bridge": "br-int", "label": "tempest-VolumesSnapshotTestJSON-1896954656-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.25", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "7e24dd8accea4c1e9598d643ea79c86d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap7b066f7f-bc", "ovs_interfaceid": "7b066f7f-bc63-4a5f-8b1b-20e4611cf387", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:21:10 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.017s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:21:10 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[9640bd31-efd3-4e70-9ec6-f18eb2f5742b]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:dd1368c5ba2c\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:21:10 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt -v / -n -o SOURCE {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:21:10 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdd/device/delete" returned: 0 in 0.053s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:21:10 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[a0a3e5bd-5219-4bc9-bb21-96d6ba3b0149]: (4, ('1', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:21:10 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-226b3276-674a-4a31-873c-36e3f6fb069c tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Checking to see if SCSI volumes sdd have been removed. {{(pid=75234) wait_for_volumes_removal /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:156}}
Nov 29 03:21:10 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-226b3276-674a-4a31-873c-36e3f6fb069c tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] SCSI volumes sdd have been removed. {{(pid=75234) wait_for_volumes_removal /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:166}}
Nov 29 03:21:10 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-226b3276-674a-4a31-873c-36e3f6fb069c tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:bf57b967-7fad-4d76-abb2-319fe88f8ef9')] {{(pid=75234) _disconnect_connection /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Nov 29 03:21:10 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-51c6f4a4-475f-4844-8b9b-d262e32d2af2 req-6ac99c28-44de-4da0-b3ae-add8db354f82 service nova] Releasing lock "refresh_cache-72ebd454-0275-4c3d-95d2-227ffb045a0c" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:21:10 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:bf57b967-7fad-4d76-abb2-319fe88f8ef9 -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:21:10 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt -v / -n -o SOURCE" returned: 0 in 0.026s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:21:10 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[cab1444b-e92d-4f21-b214-7eaadf531405]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:21:10 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:21:10 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:bf57b967-7fad-4d76-abb2-319fe88f8ef9 -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.025s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:21:10 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[a3719bbd-cf59-48d8-9e9a-9eb9e86c0de6]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:21:10 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-226b3276-674a-4a31-873c-36e3f6fb069c tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:21:10 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:bf57b967-7fad-4d76-abb2-319fe88f8ef9 -p 172.16.0.220:3260 --logout {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:21:10 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.018s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:21:10 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[e72dfe93-c6ee-4b16-9da7-39fe538ecba5]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:21:10 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[d72c5177-6aee-42bf-82ad-a77d013ffcba]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:21:10 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-bd252e8d-9d3d-443e-8f47-f07d0eeb5b2e tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Running cmd (subprocess): nvme version {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:21:10 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-bd252e8d-9d3d-443e-8f47-f07d0eeb5b2e tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] 'nvme version' failed. Not Retrying. {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:473}}
Nov 29 03:21:10 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.nvmeof [None req-bd252e8d-9d3d-443e-8f47-f07d0eeb5b2e tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] nvme not present on system {{(pid=75234) nvme_present /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/nvmeof.py:756}}
Nov 29 03:21:10 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:21:10 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:bf57b967-7fad-4d76-abb2-319fe88f8ef9 -p 172.16.0.220:3260 --logout" returned: 0 in 0.076s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:21:10 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[c85b2df1-9cb9-49e3-bd68-cc593b86bae5]: (4, ('Logging out of session [sid: 29, target: iqn.2010-10.org.openstack:bf57b967-7fad-4d76-abb2-319fe88f8ef9, portal: 172.16.0.220,3260]\nLogout of [sid: 29, target: iqn.2010-10.org.openstack:bf57b967-7fad-4d76-abb2-319fe88f8ef9, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:21:10 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-226b3276-674a-4a31-873c-36e3f6fb069c tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 29, target: iqn.2010-10.org.openstack:bf57b967-7fad-4d76-abb2-319fe88f8ef9, portal: 172.16.0.220,3260]
Nov 29 03:21:10 user nova-compute[75234]: Logout of [sid: 29, target: iqn.2010-10.org.openstack:bf57b967-7fad-4d76-abb2-319fe88f8ef9, portal: 172.16.0.220,3260] successful.
Nov 29 03:21:10 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:21:10 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:473}}
Nov 29 03:21:10 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:bf57b967-7fad-4d76-abb2-319fe88f8ef9 -p 172.16.0.220:3260 --op delete {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:21:10 user nova-compute[75234]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Nov 29 03:21:10 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[8f15cbbf-40ca-4f92-a325-2e94af1c8941]: (4, '') {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:21:10 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.lightos [None req-bd252e8d-9d3d-443e-8f47-f07d0eeb5b2e tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=75234) find_dsc /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/lightos.py:98}}
Nov 29 03:21:10 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.lightos [None req-bd252e8d-9d3d-443e-8f47-f07d0eeb5b2e tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=75234) get_connector_properties /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/lightos.py:76}}
Nov 29 03:21:10 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.lightos [None req-bd252e8d-9d3d-443e-8f47-f07d0eeb5b2e tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] LIGHTOS: no hostnqn found. {{(pid=75234) get_connector_properties /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/lightos.py:84}}
Nov 29 03:21:10 user nova-compute[75234]: DEBUG os_brick.utils [None req-bd252e8d-9d3d-443e-8f47-f07d0eeb5b2e tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] <== get_connector_properties: return (174ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:dd1368c5ba2c', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:203}}
Nov 29 03:21:10 user nova-compute[75234]: DEBUG nova.virt.block_device [None req-bd252e8d-9d3d-443e-8f47-f07d0eeb5b2e tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] [instance: 72ebd454-0275-4c3d-95d2-227ffb045a0c] Updating existing volume attachment record: a9af2839-0f6b-4f49-bee0-e3d296464cba {{(pid=75234) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Nov 29 03:21:10 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:bf57b967-7fad-4d76-abb2-319fe88f8ef9 -p 172.16.0.220:3260 --op delete" returned: 0 in 0.021s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:21:10 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[33973abf-293d-4584-93f6-cd336821f18d]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:21:10 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-226b3276-674a-4a31-873c-36e3f6fb069c tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:21:10 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-226b3276-674a-4a31-873c-36e3f6fb069c tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.406s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:87}}
Nov 29 03:21:10 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-226b3276-674a-4a31-873c-36e3f6fb069c tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] <== disconnect_volume: return (408ms) None {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:203}}
Nov 29 03:21:10 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-226b3276-674a-4a31-873c-36e3f6fb069c tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] [instance: 75c5d700-6367-4128-9228-80d2cd24ace7] Disconnected iSCSI Volume {{(pid=75234) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:79}}
Nov 29 03:21:10 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:21:11 user nova-compute[75234]: DEBUG nova.objects.instance [None req-226b3276-674a-4a31-873c-36e3f6fb069c tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Lazy-loading 'flavor' on Instance uuid 75c5d700-6367-4128-9228-80d2cd24ace7 {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:21:12 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-226b3276-674a-4a31-873c-36e3f6fb069c tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Lock "75c5d700-6367-4128-9228-80d2cd24ace7" "released" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: held 22.775s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:21:12 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:21:13 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:21:13 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:21:17 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:21:17 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-bd252e8d-9d3d-443e-8f47-f07d0eeb5b2e tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] [instance: 72ebd454-0275-4c3d-95d2-227ffb045a0c] Calling os-brick to attach iSCSI Volume {{(pid=75234) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Nov 29 03:21:17 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-bd252e8d-9d3d-443e-8f47-f07d0eeb5b2e tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f8a5050c9a0>, 'connection_properties': {'target_iqn': 'iqn.2010-10.org.openstack:0e8591d9-0c7f-4575-9e91-7fd99d87327a', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'yfqNpnzW', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:176}}
Nov 29 03:21:17 user nova-compute[75234]: WARNING os_brick.initiator.connectors.base [None req-bd252e8d-9d3d-443e-8f47-f07d0eeb5b2e tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Nov 29 03:21:17 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-bd252e8d-9d3d-443e-8f47-f07d0eeb5b2e tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:68}}
Nov 29 03:21:17 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-bd252e8d-9d3d-443e-8f47-f07d0eeb5b2e tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.002s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:73}}
Nov 29 03:21:17 user nova-compute[75234]: WARNING os_brick.initiator.connectors.base [None req-bd252e8d-9d3d-443e-8f47-f07d0eeb5b2e tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Nov 29 03:21:17 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-bd252e8d-9d3d-443e-8f47-f07d0eeb5b2e tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:0e8591d9-0c7f-4575-9e91-7fd99d87327a" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:68}}
Nov 29 03:21:17 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-bd252e8d-9d3d-443e-8f47-f07d0eeb5b2e tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:0e8591d9-0c7f-4575-9e91-7fd99d87327a" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:73}}
Nov 29 03:21:17 user nova-compute[75234]: INFO os_brick.initiator.connectors.iscsi [None req-bd252e8d-9d3d-443e-8f47-f07d0eeb5b2e tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Nov 29 03:21:17 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:0e8591d9-0c7f-4575-9e91-7fd99d87327a -p 172.16.0.220:3260 {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:21:17 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:0e8591d9-0c7f-4575-9e91-7fd99d87327a -p 172.16.0.220:3260" returned: 21 in 0.017s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:21:17 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[dd32facf-c0ec-4abb-9e21-7d8c646fef20]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:21:17 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-bd252e8d-9d3d-443e-8f47-f07d0eeb5b2e tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Nov 29 03:21:17 user nova-compute[75234]:  {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:21:17 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:0e8591d9-0c7f-4575-9e91-7fd99d87327a -p 172.16.0.220:3260 --interface default --op new {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:21:17 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:0e8591d9-0c7f-4575-9e91-7fd99d87327a -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.020s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:21:17 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[305db624-2c96-4227-91fa-827817a3b91b]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:0e8591d9-0c7f-4575-9e91-7fd99d87327a] added\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:21:17 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-bd252e8d-9d3d-443e-8f47-f07d0eeb5b2e tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:0e8591d9-0c7f-4575-9e91-7fd99d87327a] added
Nov 29 03:21:17 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:21:17 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:0e8591d9-0c7f-4575-9e91-7fd99d87327a -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:21:17 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:0e8591d9-0c7f-4575-9e91-7fd99d87327a -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.018s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:21:17 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[acc499b9-565a-41b0-89d6-14c8744df457]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:21:17 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-bd252e8d-9d3d-443e-8f47-f07d0eeb5b2e tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:21:17 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:0e8591d9-0c7f-4575-9e91-7fd99d87327a -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:21:17 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:0e8591d9-0c7f-4575-9e91-7fd99d87327a -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.018s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:21:17 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[556f158f-e925-412c-b180-21bbf04dce3d]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:21:17 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-bd252e8d-9d3d-443e-8f47-f07d0eeb5b2e tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:21:17 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:0e8591d9-0c7f-4575-9e91-7fd99d87327a -p 172.16.0.220:3260 --op update -n node.session.auth.username -v yfqNpnzW {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:21:17 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:0e8591d9-0c7f-4575-9e91-7fd99d87327a -p 172.16.0.220:3260 --op update -n node.session.auth.username -v yfqNpnzW" returned: 0 in 0.018s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:21:17 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[69e3a7f9-bc1f-4156-a45a-cc38cd70b27c]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:21:17 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-bd252e8d-9d3d-443e-8f47-f07d0eeb5b2e tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'yfqNpnzW'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:21:17 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:0e8591d9-0c7f-4575-9e91-7fd99d87327a -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:21:18 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:0e8591d9-0c7f-4575-9e91-7fd99d87327a -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.021s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:21:18 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[8953995c-8d3d-4a68-9f22-14622d0e841b]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:21:18 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-bd252e8d-9d3d-443e-8f47-f07d0eeb5b2e tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:21:18 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:21:18 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.023s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:21:18 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[9980466a-b57c-4202-9df0-786379c49dd1]: (4, ('tcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:cf6ab229-533a-424e-bd67-1d54d50ad4be (non-flash)\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:21:18 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-bd252e8d-9d3d-443e-8f47-f07d0eeb5b2e tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:cf6ab229-533a-424e-bd67-1d54d50ad4be (non-flash)
Nov 29 03:21:18 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:21:18 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-bd252e8d-9d3d-443e-8f47-f07d0eeb5b2e tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] iscsi session list stdout=tcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:cf6ab229-533a-424e-bd67-1d54d50ad4be (non-flash)
Nov 29 03:21:18 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsi_session /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Nov 29 03:21:18 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:0e8591d9-0c7f-4575-9e91-7fd99d87327a -p 172.16.0.220:3260 --login {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:21:18 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:0e8591d9-0c7f-4575-9e91-7fd99d87327a -p 172.16.0.220:3260 --login" returned: 0 in 0.054s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:21:18 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[641ff4ad-ba22-4615-9efc-fc872bab88c3]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:0e8591d9-0c7f-4575-9e91-7fd99d87327a, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:0e8591d9-0c7f-4575-9e91-7fd99d87327a, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:21:18 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-bd252e8d-9d3d-443e-8f47-f07d0eeb5b2e tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:0e8591d9-0c7f-4575-9e91-7fd99d87327a, portal: 172.16.0.220,3260]
Nov 29 03:21:18 user nova-compute[75234]: Login to [iface: default, target: iqn.2010-10.org.openstack:0e8591d9-0c7f-4575-9e91-7fd99d87327a, portal: 172.16.0.220,3260] successful.
Nov 29 03:21:18 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:21:18 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:0e8591d9-0c7f-4575-9e91-7fd99d87327a -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:21:18 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:0e8591d9-0c7f-4575-9e91-7fd99d87327a -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.018s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:21:18 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[8403cc70-248a-47f9-89a5-fb5da025faf6]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:21:18 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-bd252e8d-9d3d-443e-8f47-f07d0eeb5b2e tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:21:18 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:21:18 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.023s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:21:18 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[4c395313-3350-4d28-9532-efa35be23069]: (4, ('tcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:cf6ab229-533a-424e-bd67-1d54d50ad4be (non-flash)\ntcp: [38] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0e8591d9-0c7f-4575-9e91-7fd99d87327a (non-flash)\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:21:18 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-bd252e8d-9d3d-443e-8f47-f07d0eeb5b2e tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:cf6ab229-533a-424e-bd67-1d54d50ad4be (non-flash)
Nov 29 03:21:18 user nova-compute[75234]: tcp: [38] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0e8591d9-0c7f-4575-9e91-7fd99d87327a (non-flash)
Nov 29 03:21:18 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:21:18 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-bd252e8d-9d3d-443e-8f47-f07d0eeb5b2e tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] iscsi session list stdout=tcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:cf6ab229-533a-424e-bd67-1d54d50ad4be (non-flash)
Nov 29 03:21:18 user nova-compute[75234]: tcp: [38] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0e8591d9-0c7f-4575-9e91-7fd99d87327a (non-flash)
Nov 29 03:21:18 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsi_session /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Nov 29 03:21:18 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-bd252e8d-9d3d-443e-8f47-f07d0eeb5b2e tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:0e8591d9-0c7f-4575-9e91-7fd99d87327a" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.272s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:87}}
Nov 29 03:21:18 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-bd252e8d-9d3d-443e-8f47-f07d0eeb5b2e tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Connected to 172.16.0.220:3260 {{(pid=75234) _connect_vol /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:644}}
Nov 29 03:21:18 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-bd252e8d-9d3d-443e-8f47-f07d0eeb5b2e tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] HCTL ('34', '-', '-', 0) found on session 38 with lun 0 {{(pid=75234) get_hctl /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:770}}
Nov 29 03:21:18 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-bd252e8d-9d3d-443e-8f47-f07d0eeb5b2e tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Scanning host 34 c: -, t: -, l: 0) {{(pid=75234) scan_iscsi /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:795}}
Nov 29 03:21:18 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host34/scan {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:21:18 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host34/scan" returned: 0 in 0.017s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:21:18 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[7b682ce9-8d04-484a-8562-ed8cc0e0882d]: (4, ('- - 0', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:21:18 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-bd252e8d-9d3d-443e-8f47-f07d0eeb5b2e tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Searching for a device in session 38 and hctl ['34', '*', '*', 0] yield: sdc {{(pid=75234) device_name_by_hctl /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:789}}
Nov 29 03:21:18 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-bd252e8d-9d3d-443e-8f47-f07d0eeb5b2e tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Connected to sdc using {'target_iqn': 'iqn.2010-10.org.openstack:0e8591d9-0c7f-4575-9e91-7fd99d87327a', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'yfqNpnzW', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False} {{(pid=75234) _connect_vol /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:672}}
Nov 29 03:21:18 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:21:18 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-5643b891-bd60-4956-82ad-f62045205121 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Acquiring lock "f6bc212d-85cd-479c-92ae-112c9f0101bc" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:21:18 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-5643b891-bd60-4956-82ad-f62045205121 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Lock "f6bc212d-85cd-479c-92ae-112c9f0101bc" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:21:18 user nova-compute[75234]: DEBUG nova.compute.manager [None req-5643b891-bd60-4956-82ad-f62045205121 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] [instance: f6bc212d-85cd-479c-92ae-112c9f0101bc] Starting instance... {{(pid=75234) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2422}}
Nov 29 03:21:18 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-ca328ca9-8de1-4354-bb6a-6bb3035177e5 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] [instance: 1df140f1-fbd9-4b5d-b356-d08b7e908f64] Calling os-brick to attach iSCSI Volume {{(pid=75234) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Nov 29 03:21:18 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-ca328ca9-8de1-4354-bb6a-6bb3035177e5 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f8a5050c9a0>, 'connection_properties': {'target_iqn': 'iqn.2010-10.org.openstack:41f08854-31dd-4172-a5cf-01b0e695c10f', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'XswhptcD', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:176}}
Nov 29 03:21:18 user nova-compute[75234]: WARNING os_brick.initiator.connectors.base [None req-ca328ca9-8de1-4354-bb6a-6bb3035177e5 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Nov 29 03:21:18 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-ca328ca9-8de1-4354-bb6a-6bb3035177e5 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:68}}
Nov 29 03:21:18 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-5643b891-bd60-4956-82ad-f62045205121 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:21:18 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-5643b891-bd60-4956-82ad-f62045205121 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:21:18 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-5643b891-bd60-4956-82ad-f62045205121 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=75234) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2366}}
Nov 29 03:21:18 user nova-compute[75234]: INFO nova.compute.claims [None req-5643b891-bd60-4956-82ad-f62045205121 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] [instance: f6bc212d-85cd-479c-92ae-112c9f0101bc] Claim successful on node user
Nov 29 03:21:19 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-bd252e8d-9d3d-443e-8f47-f07d0eeb5b2e tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.307s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:87}}
Nov 29 03:21:19 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-bd252e8d-9d3d-443e-8f47-f07d0eeb5b2e tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] <== connect_volume: return (1308ms) {'type': 'block', 'scsi_wwn': '23764643135303461', 'path': '/dev/sdc'} {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:203}}
Nov 29 03:21:19 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-bd252e8d-9d3d-443e-8f47-f07d0eeb5b2e tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] [instance: 72ebd454-0275-4c3d-95d2-227ffb045a0c] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '23764643135303461', 'path': '/dev/sdc'} {{(pid=75234) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Nov 29 03:21:19 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-ca328ca9-8de1-4354-bb6a-6bb3035177e5 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.297s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:73}}
Nov 29 03:21:19 user nova-compute[75234]: WARNING os_brick.initiator.connectors.base [None req-ca328ca9-8de1-4354-bb6a-6bb3035177e5 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Nov 29 03:21:19 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-ca328ca9-8de1-4354-bb6a-6bb3035177e5 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:41f08854-31dd-4172-a5cf-01b0e695c10f" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:68}}
Nov 29 03:21:19 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-ca328ca9-8de1-4354-bb6a-6bb3035177e5 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:41f08854-31dd-4172-a5cf-01b0e695c10f" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:73}}
Nov 29 03:21:19 user nova-compute[75234]: INFO os_brick.initiator.connectors.iscsi [None req-ca328ca9-8de1-4354-bb6a-6bb3035177e5 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Nov 29 03:21:19 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:41f08854-31dd-4172-a5cf-01b0e695c10f -p 172.16.0.220:3260 {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:21:19 user nova-compute[75234]: DEBUG nova.objects.instance [None req-bd252e8d-9d3d-443e-8f47-f07d0eeb5b2e tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Lazy-loading 'flavor' on Instance uuid 72ebd454-0275-4c3d-95d2-227ffb045a0c {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:21:19 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:41f08854-31dd-4172-a5cf-01b0e695c10f -p 172.16.0.220:3260" returned: 21 in 0.019s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:21:19 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[7b418d5a-292e-4fff-b1c7-66b57496a5fa]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:21:19 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-ca328ca9-8de1-4354-bb6a-6bb3035177e5 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Nov 29 03:21:19 user nova-compute[75234]:  {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:21:19 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:41f08854-31dd-4172-a5cf-01b0e695c10f -p 172.16.0.220:3260 --interface default --op new {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:21:19 user nova-compute[75234]: DEBUG nova.virt.libvirt.guest [None req-bd252e8d-9d3d-443e-8f47-f07d0eeb5b2e tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] attach device xml: <disk type="block" device="disk">
Nov 29 03:21:19 user nova-compute[75234]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Nov 29 03:21:19 user nova-compute[75234]:   <source dev="/dev/sdc"/>
Nov 29 03:21:19 user nova-compute[75234]:   <target dev="vdb" bus="virtio"/>
Nov 29 03:21:19 user nova-compute[75234]:   <serial>0e8591d9-0c7f-4575-9e91-7fd99d87327a</serial>
Nov 29 03:21:19 user nova-compute[75234]: </disk>
Nov 29 03:21:19 user nova-compute[75234]:  {{(pid=75234) attach_device /opt/stack/nova/nova/virt/libvirt/guest.py:339}}
Nov 29 03:21:19 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:41f08854-31dd-4172-a5cf-01b0e695c10f -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.018s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:21:19 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[1f64f58c-50bb-4cea-a60a-8181e6ac4ade]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:41f08854-31dd-4172-a5cf-01b0e695c10f] added\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:21:19 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-ca328ca9-8de1-4354-bb6a-6bb3035177e5 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:41f08854-31dd-4172-a5cf-01b0e695c10f] added
Nov 29 03:21:19 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:21:19 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:41f08854-31dd-4172-a5cf-01b0e695c10f -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:21:19 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:41f08854-31dd-4172-a5cf-01b0e695c10f -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.020s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:21:19 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[5ba70304-9809-46f0-adc5-64f362e048e9]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:21:19 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-ca328ca9-8de1-4354-bb6a-6bb3035177e5 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:21:19 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:41f08854-31dd-4172-a5cf-01b0e695c10f -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:21:19 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:41f08854-31dd-4172-a5cf-01b0e695c10f -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.028s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:21:19 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[4c7f6b69-0690-47c8-ad28-9246b2dee938]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:21:19 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-ca328ca9-8de1-4354-bb6a-6bb3035177e5 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:21:19 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:41f08854-31dd-4172-a5cf-01b0e695c10f -p 172.16.0.220:3260 --op update -n node.session.auth.username -v XswhptcD {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:21:19 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:41f08854-31dd-4172-a5cf-01b0e695c10f -p 172.16.0.220:3260 --op update -n node.session.auth.username -v XswhptcD" returned: 0 in 0.019s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:21:19 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[f645865c-aa5d-422f-badd-a9353c054692]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:21:19 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-ca328ca9-8de1-4354-bb6a-6bb3035177e5 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'XswhptcD'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:21:19 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:41f08854-31dd-4172-a5cf-01b0e695c10f -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:21:19 user nova-compute[75234]: DEBUG nova.compute.provider_tree [None req-5643b891-bd60-4956-82ad-f62045205121 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Inventory has not changed in ProviderTree for provider: 4d291220-7c37-4839-a0e3-b23c66502b74 {{(pid=75234) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Nov 29 03:21:19 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:41f08854-31dd-4172-a5cf-01b0e695c10f -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.018s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:21:19 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[b8c427c9-f730-4805-9a2f-2fea0125ca0a]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:21:19 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-ca328ca9-8de1-4354-bb6a-6bb3035177e5 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:21:19 user nova-compute[75234]: DEBUG nova.scheduler.client.report [None req-5643b891-bd60-4956-82ad-f62045205121 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Inventory has not changed for provider 4d291220-7c37-4839-a0e3-b23c66502b74 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=75234) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Nov 29 03:21:19 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:21:19 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.022s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:21:19 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[2618f817-a7b9-4313-9912-a053d441a91b]: (4, ('tcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:cf6ab229-533a-424e-bd67-1d54d50ad4be (non-flash)\ntcp: [38] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0e8591d9-0c7f-4575-9e91-7fd99d87327a (non-flash)\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:21:19 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-ca328ca9-8de1-4354-bb6a-6bb3035177e5 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:cf6ab229-533a-424e-bd67-1d54d50ad4be (non-flash)
Nov 29 03:21:19 user nova-compute[75234]: tcp: [38] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0e8591d9-0c7f-4575-9e91-7fd99d87327a (non-flash)
Nov 29 03:21:19 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:21:19 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-ca328ca9-8de1-4354-bb6a-6bb3035177e5 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] iscsi session list stdout=tcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:cf6ab229-533a-424e-bd67-1d54d50ad4be (non-flash)
Nov 29 03:21:19 user nova-compute[75234]: tcp: [38] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0e8591d9-0c7f-4575-9e91-7fd99d87327a (non-flash)
Nov 29 03:21:19 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsi_session /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Nov 29 03:21:19 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:41f08854-31dd-4172-a5cf-01b0e695c10f -p 172.16.0.220:3260 --login {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:21:19 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-5643b891-bd60-4956-82ad-f62045205121 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.436s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:21:19 user nova-compute[75234]: DEBUG nova.compute.manager [None req-5643b891-bd60-4956-82ad-f62045205121 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] [instance: f6bc212d-85cd-479c-92ae-112c9f0101bc] Start building networks asynchronously for instance. {{(pid=75234) _build_resources /opt/stack/nova/nova/compute/manager.py:2819}}
Nov 29 03:21:19 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:41f08854-31dd-4172-a5cf-01b0e695c10f -p 172.16.0.220:3260 --login" returned: 0 in 0.062s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:21:19 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[0aae55f8-a122-4143-9cbc-102e442077bb]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:41f08854-31dd-4172-a5cf-01b0e695c10f, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:41f08854-31dd-4172-a5cf-01b0e695c10f, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:21:19 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-ca328ca9-8de1-4354-bb6a-6bb3035177e5 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:41f08854-31dd-4172-a5cf-01b0e695c10f, portal: 172.16.0.220,3260]
Nov 29 03:21:19 user nova-compute[75234]: Login to [iface: default, target: iqn.2010-10.org.openstack:41f08854-31dd-4172-a5cf-01b0e695c10f, portal: 172.16.0.220,3260] successful.
Nov 29 03:21:19 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:21:19 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:41f08854-31dd-4172-a5cf-01b0e695c10f -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:21:19 user nova-compute[75234]: DEBUG nova.compute.manager [None req-5643b891-bd60-4956-82ad-f62045205121 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] [instance: f6bc212d-85cd-479c-92ae-112c9f0101bc] Allocating IP information in the background. {{(pid=75234) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1972}}
Nov 29 03:21:19 user nova-compute[75234]: DEBUG nova.network.neutron [None req-5643b891-bd60-4956-82ad-f62045205121 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] [instance: f6bc212d-85cd-479c-92ae-112c9f0101bc] allocate_for_instance() {{(pid=75234) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1156}}
Nov 29 03:21:19 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:41f08854-31dd-4172-a5cf-01b0e695c10f -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.020s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:21:19 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[89c5fd7a-b298-4db0-84dc-a3771cae98df]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:21:19 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-ca328ca9-8de1-4354-bb6a-6bb3035177e5 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:21:19 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-5643b891-bd60-4956-82ad-f62045205121 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] [instance: f6bc212d-85cd-479c-92ae-112c9f0101bc] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Nov 29 03:21:19 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:21:19 user nova-compute[75234]: DEBUG nova.compute.manager [None req-5643b891-bd60-4956-82ad-f62045205121 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] [instance: f6bc212d-85cd-479c-92ae-112c9f0101bc] Start building block device mappings for instance. {{(pid=75234) _build_resources /opt/stack/nova/nova/compute/manager.py:2854}}
Nov 29 03:21:19 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.024s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:21:19 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[ce59266d-aec1-47e2-880a-c3ce05579355]: (4, ('tcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:cf6ab229-533a-424e-bd67-1d54d50ad4be (non-flash)\ntcp: [38] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0e8591d9-0c7f-4575-9e91-7fd99d87327a (non-flash)\ntcp: [39] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:41f08854-31dd-4172-a5cf-01b0e695c10f (non-flash)\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:21:19 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-ca328ca9-8de1-4354-bb6a-6bb3035177e5 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:cf6ab229-533a-424e-bd67-1d54d50ad4be (non-flash)
Nov 29 03:21:19 user nova-compute[75234]: tcp: [38] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0e8591d9-0c7f-4575-9e91-7fd99d87327a (non-flash)
Nov 29 03:21:19 user nova-compute[75234]: tcp: [39] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:41f08854-31dd-4172-a5cf-01b0e695c10f (non-flash)
Nov 29 03:21:19 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:21:19 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-ca328ca9-8de1-4354-bb6a-6bb3035177e5 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] iscsi session list stdout=tcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:cf6ab229-533a-424e-bd67-1d54d50ad4be (non-flash)
Nov 29 03:21:19 user nova-compute[75234]: tcp: [38] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0e8591d9-0c7f-4575-9e91-7fd99d87327a (non-flash)
Nov 29 03:21:19 user nova-compute[75234]: tcp: [39] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:41f08854-31dd-4172-a5cf-01b0e695c10f (non-flash)
Nov 29 03:21:19 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsi_session /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Nov 29 03:21:19 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-ca328ca9-8de1-4354-bb6a-6bb3035177e5 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:41f08854-31dd-4172-a5cf-01b0e695c10f" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.301s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:87}}
Nov 29 03:21:19 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-ca328ca9-8de1-4354-bb6a-6bb3035177e5 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Connected to 172.16.0.220:3260 {{(pid=75234) _connect_vol /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:644}}
Nov 29 03:21:19 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-ca328ca9-8de1-4354-bb6a-6bb3035177e5 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] HCTL ('35', '-', '-', 0) found on session 39 with lun 0 {{(pid=75234) get_hctl /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:770}}
Nov 29 03:21:19 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-ca328ca9-8de1-4354-bb6a-6bb3035177e5 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Scanning host 35 c: -, t: -, l: 0) {{(pid=75234) scan_iscsi /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:795}}
Nov 29 03:21:19 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host35/scan {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:21:19 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host35/scan" returned: 0 in 0.018s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:21:19 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[16844533-c093-49df-bac2-406758b733aa]: (4, ('- - 0', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:21:19 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-ca328ca9-8de1-4354-bb6a-6bb3035177e5 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Searching for a device in session 39 and hctl ['35', '*', '*', 0] yield: sdd {{(pid=75234) device_name_by_hctl /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:789}}
Nov 29 03:21:19 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-ca328ca9-8de1-4354-bb6a-6bb3035177e5 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Connected to sdd using {'target_iqn': 'iqn.2010-10.org.openstack:41f08854-31dd-4172-a5cf-01b0e695c10f', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'XswhptcD', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False} {{(pid=75234) _connect_vol /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:672}}
Nov 29 03:21:19 user nova-compute[75234]: DEBUG nova.policy [None req-5643b891-bd60-4956-82ad-f62045205121 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': 'e4aed253decd45a4b1f9e42550f7dd81', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '7c3b05b1968540dbbb40817c87846ff6', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=75234) authorize /opt/stack/nova/nova/policy.py:203}}
Nov 29 03:21:19 user nova-compute[75234]: DEBUG nova.compute.manager [None req-5643b891-bd60-4956-82ad-f62045205121 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] [instance: f6bc212d-85cd-479c-92ae-112c9f0101bc] Start spawning the instance on the hypervisor. {{(pid=75234) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2628}}
Nov 29 03:21:19 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-5643b891-bd60-4956-82ad-f62045205121 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] [instance: f6bc212d-85cd-479c-92ae-112c9f0101bc] Creating instance directory {{(pid=75234) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4727}}
Nov 29 03:21:19 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-5643b891-bd60-4956-82ad-f62045205121 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] [instance: f6bc212d-85cd-479c-92ae-112c9f0101bc] Creating image(s)
Nov 29 03:21:19 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-5643b891-bd60-4956-82ad-f62045205121 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Acquiring lock "/opt/stack/data/nova/instances/f6bc212d-85cd-479c-92ae-112c9f0101bc/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:21:19 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-5643b891-bd60-4956-82ad-f62045205121 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Lock "/opt/stack/data/nova/instances/f6bc212d-85cd-479c-92ae-112c9f0101bc/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:21:19 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-5643b891-bd60-4956-82ad-f62045205121 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Lock "/opt/stack/data/nova/instances/f6bc212d-85cd-479c-92ae-112c9f0101bc/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.002s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:21:19 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-5643b891-bd60-4956-82ad-f62045205121 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:21:19 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-bd252e8d-9d3d-443e-8f47-f07d0eeb5b2e tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] No BDM found with device name vda, not building metadata. {{(pid=75234) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12089}}
Nov 29 03:21:19 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-bd252e8d-9d3d-443e-8f47-f07d0eeb5b2e tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] No BDM found with device name vdb, not building metadata. {{(pid=75234) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12089}}
Nov 29 03:21:19 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-bd252e8d-9d3d-443e-8f47-f07d0eeb5b2e tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] No VIF found with MAC fa:16:3e:52:a7:f8, not building metadata {{(pid=75234) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Nov 29 03:21:19 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-5643b891-bd60-4956-82ad-f62045205121 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json" returned: 0 in 0.185s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:21:19 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-5643b891-bd60-4956-82ad-f62045205121 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Acquiring lock "80686e29dc9db6600d03e85cb62af298e12d7918" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:21:19 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-5643b891-bd60-4956-82ad-f62045205121 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Lock "80686e29dc9db6600d03e85cb62af298e12d7918" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:21:19 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-5643b891-bd60-4956-82ad-f62045205121 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:21:19 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-bd252e8d-9d3d-443e-8f47-f07d0eeb5b2e tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Lock "72ebd454-0275-4c3d-95d2-227ffb045a0c" "released" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: held 10.024s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:21:20 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-5643b891-bd60-4956-82ad-f62045205121 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json" returned: 0 in 0.432s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:21:20 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-5643b891-bd60-4956-82ad-f62045205121 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918,backing_fmt=raw /opt/stack/data/nova/instances/f6bc212d-85cd-479c-92ae-112c9f0101bc/disk 1073741824 {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:21:20 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-5643b891-bd60-4956-82ad-f62045205121 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918,backing_fmt=raw /opt/stack/data/nova/instances/f6bc212d-85cd-479c-92ae-112c9f0101bc/disk 1073741824" returned: 0 in 0.050s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:21:20 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-5643b891-bd60-4956-82ad-f62045205121 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Lock "80686e29dc9db6600d03e85cb62af298e12d7918" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.488s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:21:20 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-5643b891-bd60-4956-82ad-f62045205121 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:21:20 user nova-compute[75234]: DEBUG nova.network.neutron [None req-5643b891-bd60-4956-82ad-f62045205121 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] [instance: f6bc212d-85cd-479c-92ae-112c9f0101bc] Successfully created port: e43ebf8d-4145-4a78-8be5-600ea409e775 {{(pid=75234) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:548}}
Nov 29 03:21:20 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-5643b891-bd60-4956-82ad-f62045205121 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json" returned: 0 in 0.172s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:21:20 user nova-compute[75234]: DEBUG nova.virt.disk.api [None req-5643b891-bd60-4956-82ad-f62045205121 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Checking if we can resize image /opt/stack/data/nova/instances/f6bc212d-85cd-479c-92ae-112c9f0101bc/disk. size=1073741824 {{(pid=75234) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:166}}
Nov 29 03:21:20 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-5643b891-bd60-4956-82ad-f62045205121 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f6bc212d-85cd-479c-92ae-112c9f0101bc/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:21:20 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-ca328ca9-8de1-4354-bb6a-6bb3035177e5 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.348s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:87}}
Nov 29 03:21:20 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-ca328ca9-8de1-4354-bb6a-6bb3035177e5 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] <== connect_volume: return (1647ms) {'type': 'block', 'scsi_wwn': '23666336235303666', 'path': '/dev/sdd'} {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:203}}
Nov 29 03:21:20 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-ca328ca9-8de1-4354-bb6a-6bb3035177e5 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] [instance: 1df140f1-fbd9-4b5d-b356-d08b7e908f64] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '23666336235303666', 'path': '/dev/sdd'} {{(pid=75234) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Nov 29 03:21:20 user nova-compute[75234]: DEBUG nova.objects.instance [None req-ca328ca9-8de1-4354-bb6a-6bb3035177e5 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Lazy-loading 'flavor' on Instance uuid 1df140f1-fbd9-4b5d-b356-d08b7e908f64 {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:21:20 user nova-compute[75234]: DEBUG nova.virt.libvirt.guest [None req-ca328ca9-8de1-4354-bb6a-6bb3035177e5 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] attach device xml: <disk type="block" device="disk">
Nov 29 03:21:20 user nova-compute[75234]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Nov 29 03:21:20 user nova-compute[75234]:   <source dev="/dev/sdd"/>
Nov 29 03:21:20 user nova-compute[75234]:   <target dev="vdb" bus="virtio"/>
Nov 29 03:21:20 user nova-compute[75234]:   <serial>41f08854-31dd-4172-a5cf-01b0e695c10f</serial>
Nov 29 03:21:20 user nova-compute[75234]: </disk>
Nov 29 03:21:20 user nova-compute[75234]:  {{(pid=75234) attach_device /opt/stack/nova/nova/virt/libvirt/guest.py:339}}
Nov 29 03:21:20 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-5643b891-bd60-4956-82ad-f62045205121 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f6bc212d-85cd-479c-92ae-112c9f0101bc/disk --force-share --output=json" returned: 0 in 0.213s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:21:20 user nova-compute[75234]: DEBUG nova.virt.disk.api [None req-5643b891-bd60-4956-82ad-f62045205121 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Cannot resize image /opt/stack/data/nova/instances/f6bc212d-85cd-479c-92ae-112c9f0101bc/disk to a smaller size. {{(pid=75234) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:172}}
Nov 29 03:21:20 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-5643b891-bd60-4956-82ad-f62045205121 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] [instance: f6bc212d-85cd-479c-92ae-112c9f0101bc] Created local disks {{(pid=75234) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4861}}
Nov 29 03:21:20 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-5643b891-bd60-4956-82ad-f62045205121 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] [instance: f6bc212d-85cd-479c-92ae-112c9f0101bc] Ensure instance console log exists: /opt/stack/data/nova/instances/f6bc212d-85cd-479c-92ae-112c9f0101bc/console.log {{(pid=75234) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4613}}
Nov 29 03:21:20 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-5643b891-bd60-4956-82ad-f62045205121 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:21:20 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-5643b891-bd60-4956-82ad-f62045205121 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:21:20 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-5643b891-bd60-4956-82ad-f62045205121 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:21:20 user nova-compute[75234]: DEBUG nova.network.neutron [None req-5643b891-bd60-4956-82ad-f62045205121 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] [instance: f6bc212d-85cd-479c-92ae-112c9f0101bc] Successfully updated port: e43ebf8d-4145-4a78-8be5-600ea409e775 {{(pid=75234) _update_port /opt/stack/nova/nova/network/neutron.py:586}}
Nov 29 03:21:20 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-5643b891-bd60-4956-82ad-f62045205121 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Acquiring lock "refresh_cache-f6bc212d-85cd-479c-92ae-112c9f0101bc" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:21:20 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-5643b891-bd60-4956-82ad-f62045205121 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Acquired lock "refresh_cache-f6bc212d-85cd-479c-92ae-112c9f0101bc" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:21:20 user nova-compute[75234]: DEBUG nova.network.neutron [None req-5643b891-bd60-4956-82ad-f62045205121 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] [instance: f6bc212d-85cd-479c-92ae-112c9f0101bc] Building network info cache for instance {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2009}}
Nov 29 03:21:21 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-ca328ca9-8de1-4354-bb6a-6bb3035177e5 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] No BDM found with device name vda, not building metadata. {{(pid=75234) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12089}}
Nov 29 03:21:21 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-ca328ca9-8de1-4354-bb6a-6bb3035177e5 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] No BDM found with device name vdb, not building metadata. {{(pid=75234) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12089}}
Nov 29 03:21:21 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-ca328ca9-8de1-4354-bb6a-6bb3035177e5 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] No VIF found with MAC fa:16:3e:80:25:b8, not building metadata {{(pid=75234) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Nov 29 03:21:21 user nova-compute[75234]: DEBUG nova.network.neutron [None req-5643b891-bd60-4956-82ad-f62045205121 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] [instance: f6bc212d-85cd-479c-92ae-112c9f0101bc] Instance cache missing network info. {{(pid=75234) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3322}}
Nov 29 03:21:21 user nova-compute[75234]: DEBUG nova.compute.manager [req-23947139-8215-4907-b13b-12fd394a5e34 req-42a4ec7b-d725-48ca-8c10-368ccacfa18c service nova] [instance: f6bc212d-85cd-479c-92ae-112c9f0101bc] Received event network-changed-e43ebf8d-4145-4a78-8be5-600ea409e775 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:21:21 user nova-compute[75234]: DEBUG nova.compute.manager [req-23947139-8215-4907-b13b-12fd394a5e34 req-42a4ec7b-d725-48ca-8c10-368ccacfa18c service nova] [instance: f6bc212d-85cd-479c-92ae-112c9f0101bc] Refreshing instance network info cache due to event network-changed-e43ebf8d-4145-4a78-8be5-600ea409e775. {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11117}}
Nov 29 03:21:21 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-23947139-8215-4907-b13b-12fd394a5e34 req-42a4ec7b-d725-48ca-8c10-368ccacfa18c service nova] Acquiring lock "refresh_cache-f6bc212d-85cd-479c-92ae-112c9f0101bc" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:21:21 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ca328ca9-8de1-4354-bb6a-6bb3035177e5 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Lock "1df140f1-fbd9-4b5d-b356-d08b7e908f64" "released" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: held 17.958s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:21:21 user nova-compute[75234]: DEBUG nova.network.neutron [None req-5643b891-bd60-4956-82ad-f62045205121 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] [instance: f6bc212d-85cd-479c-92ae-112c9f0101bc] Updating instance_info_cache with network_info: [{"id": "e43ebf8d-4145-4a78-8be5-600ea409e775", "address": "fa:16:3e:4e:8e:1d", "network": {"id": "9247dc93-6a54-495e-b113-d1ff7c215178", "bridge": "br-int", "label": "tempest-SnapshotDataIntegrityTests-261469187-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "7c3b05b1968540dbbb40817c87846ff6", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape43ebf8d-41", "ovs_interfaceid": "e43ebf8d-4145-4a78-8be5-600ea409e775", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:21:21 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-5643b891-bd60-4956-82ad-f62045205121 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Releasing lock "refresh_cache-f6bc212d-85cd-479c-92ae-112c9f0101bc" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:21:21 user nova-compute[75234]: DEBUG nova.compute.manager [None req-5643b891-bd60-4956-82ad-f62045205121 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] [instance: f6bc212d-85cd-479c-92ae-112c9f0101bc] Instance network_info: |[{"id": "e43ebf8d-4145-4a78-8be5-600ea409e775", "address": "fa:16:3e:4e:8e:1d", "network": {"id": "9247dc93-6a54-495e-b113-d1ff7c215178", "bridge": "br-int", "label": "tempest-SnapshotDataIntegrityTests-261469187-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "7c3b05b1968540dbbb40817c87846ff6", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape43ebf8d-41", "ovs_interfaceid": "e43ebf8d-4145-4a78-8be5-600ea409e775", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=75234) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1987}}
Nov 29 03:21:21 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-23947139-8215-4907-b13b-12fd394a5e34 req-42a4ec7b-d725-48ca-8c10-368ccacfa18c service nova] Acquired lock "refresh_cache-f6bc212d-85cd-479c-92ae-112c9f0101bc" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:21:21 user nova-compute[75234]: DEBUG nova.network.neutron [req-23947139-8215-4907-b13b-12fd394a5e34 req-42a4ec7b-d725-48ca-8c10-368ccacfa18c service nova] [instance: f6bc212d-85cd-479c-92ae-112c9f0101bc] Refreshing network info cache for port e43ebf8d-4145-4a78-8be5-600ea409e775 {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2006}}
Nov 29 03:21:21 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-5643b891-bd60-4956-82ad-f62045205121 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] [instance: f6bc212d-85cd-479c-92ae-112c9f0101bc] Start _get_guest_xml network_info=[{"id": "e43ebf8d-4145-4a78-8be5-600ea409e775", "address": "fa:16:3e:4e:8e:1d", "network": {"id": "9247dc93-6a54-495e-b113-d1ff7c215178", "bridge": "br-int", "label": "tempest-SnapshotDataIntegrityTests-261469187-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "7c3b05b1968540dbbb40817c87846ff6", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape43ebf8d-41", "ovs_interfaceid": "e43ebf8d-4145-4a78-8be5-600ea409e775", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum='c8fc807773e5354afe61636071771906',container_format='bare',created_at=2023-11-29T03:02:15Z,direct_url=<?>,disk_format='qcow2',id=96c30ba7-9b73-4a30-8ba4-7138879573b7,min_disk=0,min_ram=0,name='cirros-0.6.2-x86_64-disk',owner='b18e380921c748b78517b6f1211cd0f1',properties=ImageMetaProps,protected=<?>,size=21430272,status='active',tags=<?>,updated_at=2023-11-29T03:02:17Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [{'device_name': '/dev/vda', 'encryption_format': None, 'encrypted': False, 'device_type': 'disk', 'size': 0, 'disk_bus': 'virtio', 'boot_index': 0, 'encryption_options': None, 'encryption_secret_uuid': None, 'guest_format': None, 'image_id': '96c30ba7-9b73-4a30-8ba4-7138879573b7'}], 'ephemerals': [], 'block_device_mapping': [], 'swap': None} {{(pid=75234) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7562}}
Nov 29 03:21:21 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-5643b891-bd60-4956-82ad-f62045205121 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:21:21 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-5643b891-bd60-4956-82ad-f62045205121 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:21:21 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-5643b891-bd60-4956-82ad-f62045205121 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Searching host: 'user' for CPU controller through CGroups V1... {{(pid=75234) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1652}}
Nov 29 03:21:21 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-5643b891-bd60-4956-82ad-f62045205121 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] CPU controller missing on host. {{(pid=75234) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1662}}
Nov 29 03:21:21 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-5643b891-bd60-4956-82ad-f62045205121 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Searching host: 'user' for CPU controller through CGroups V2... {{(pid=75234) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1671}}
Nov 29 03:21:21 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-5643b891-bd60-4956-82ad-f62045205121 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] CPU controller found on host. {{(pid=75234) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1678}}
Nov 29 03:21:21 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-5643b891-bd60-4956-82ad-f62045205121 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=75234) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5400}}
Nov 29 03:21:21 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-5643b891-bd60-4956-82ad-f62045205121 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-11-29T03:02:04Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='1',id=6,is_public=True,memory_mb=512,name='m1.tiny',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='c8fc807773e5354afe61636071771906',container_format='bare',created_at=2023-11-29T03:02:15Z,direct_url=<?>,disk_format='qcow2',id=96c30ba7-9b73-4a30-8ba4-7138879573b7,min_disk=0,min_ram=0,name='cirros-0.6.2-x86_64-disk',owner='b18e380921c748b78517b6f1211cd0f1',properties=ImageMetaProps,protected=<?>,size=21430272,status='active',tags=<?>,updated_at=2023-11-29T03:02:17Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:563}}
Nov 29 03:21:21 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-5643b891-bd60-4956-82ad-f62045205121 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Flavor limits 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:348}}
Nov 29 03:21:21 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-5643b891-bd60-4956-82ad-f62045205121 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Image limits 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:352}}
Nov 29 03:21:21 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-5643b891-bd60-4956-82ad-f62045205121 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Flavor pref 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:388}}
Nov 29 03:21:21 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-5643b891-bd60-4956-82ad-f62045205121 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Image pref 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:392}}
Nov 29 03:21:21 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-5643b891-bd60-4956-82ad-f62045205121 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:430}}
Nov 29 03:21:21 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-5643b891-bd60-4956-82ad-f62045205121 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:569}}
Nov 29 03:21:21 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-5643b891-bd60-4956-82ad-f62045205121 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=75234) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:471}}
Nov 29 03:21:21 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-5643b891-bd60-4956-82ad-f62045205121 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Got 1 possible topologies {{(pid=75234) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:501}}
Nov 29 03:21:21 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-5643b891-bd60-4956-82ad-f62045205121 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:575}}
Nov 29 03:21:21 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-5643b891-bd60-4956-82ad-f62045205121 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:577}}
Nov 29 03:21:21 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-5643b891-bd60-4956-82ad-f62045205121 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-11-29T03:21:17Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-SnapshotDataIntegrityTests-server-744808219',display_name='tempest-SnapshotDataIntegrityTests-server-744808219',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-snapshotdataintegritytests-server-744808219',id=37,image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBLcl+lLJR5zldffje/+52KkXZkw4CLRUBrN092/lLCyYU8Rz6cZyByDOlFzBvjoKUTlEBjgZpSHtec+jLeJl2h2rkNlWQ+il4II7lulZv0B8LAiMbGsvFeL2w1CWre/RQw==',key_name='tempest-keypair-1458143874',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='7c3b05b1968540dbbb40817c87846ff6',ramdisk_id='',reservation_id='r-vu6047ud',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-SnapshotDataIntegrityTests-1450496038',owner_user_name='tempest-SnapshotDataIntegrityTests-1450496038-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-11-29T03:21:20Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='e4aed253decd45a4b1f9e42550f7dd81',uuid=f6bc212d-85cd-479c-92ae-112c9f0101bc,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "e43ebf8d-4145-4a78-8be5-600ea409e775", "address": "fa:16:3e:4e:8e:1d", "network": {"id": "9247dc93-6a54-495e-b113-d1ff7c215178", "bridge": "br-int", "label": "tempest-SnapshotDataIntegrityTests-261469187-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "7c3b05b1968540dbbb40817c87846ff6", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape43ebf8d-41", "ovs_interfaceid": "e43ebf8d-4145-4a78-8be5-600ea409e775", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=75234) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Nov 29 03:21:21 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-5643b891-bd60-4956-82ad-f62045205121 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Converting VIF {"id": "e43ebf8d-4145-4a78-8be5-600ea409e775", "address": "fa:16:3e:4e:8e:1d", "network": {"id": "9247dc93-6a54-495e-b113-d1ff7c215178", "bridge": "br-int", "label": "tempest-SnapshotDataIntegrityTests-261469187-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "7c3b05b1968540dbbb40817c87846ff6", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape43ebf8d-41", "ovs_interfaceid": "e43ebf8d-4145-4a78-8be5-600ea409e775", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:21:21 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-5643b891-bd60-4956-82ad-f62045205121 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:4e:8e:1d,bridge_name='br-int',has_traffic_filtering=True,id=e43ebf8d-4145-4a78-8be5-600ea409e775,network=Network(9247dc93-6a54-495e-b113-d1ff7c215178),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tape43ebf8d-41') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:21:21 user nova-compute[75234]: DEBUG nova.objects.instance [None req-5643b891-bd60-4956-82ad-f62045205121 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Lazy-loading 'pci_devices' on Instance uuid f6bc212d-85cd-479c-92ae-112c9f0101bc {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:21:21 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-5643b891-bd60-4956-82ad-f62045205121 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] [instance: f6bc212d-85cd-479c-92ae-112c9f0101bc] End _get_guest_xml xml=<domain type="kvm">
Nov 29 03:21:21 user nova-compute[75234]:   <uuid>f6bc212d-85cd-479c-92ae-112c9f0101bc</uuid>
Nov 29 03:21:21 user nova-compute[75234]:   <name>instance-00000025</name>
Nov 29 03:21:21 user nova-compute[75234]:   <memory>524288</memory>
Nov 29 03:21:21 user nova-compute[75234]:   <vcpu>1</vcpu>
Nov 29 03:21:21 user nova-compute[75234]:   <metadata>
Nov 29 03:21:21 user nova-compute[75234]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Nov 29 03:21:21 user nova-compute[75234]:       <nova:package version="0.0.0"/>
Nov 29 03:21:21 user nova-compute[75234]:       <nova:name>tempest-SnapshotDataIntegrityTests-server-744808219</nova:name>
Nov 29 03:21:21 user nova-compute[75234]:       <nova:creationTime>2023-11-29 03:21:21</nova:creationTime>
Nov 29 03:21:21 user nova-compute[75234]:       <nova:flavor name="m1.tiny">
Nov 29 03:21:21 user nova-compute[75234]:         <nova:memory>512</nova:memory>
Nov 29 03:21:21 user nova-compute[75234]:         <nova:disk>1</nova:disk>
Nov 29 03:21:21 user nova-compute[75234]:         <nova:swap>0</nova:swap>
Nov 29 03:21:21 user nova-compute[75234]:         <nova:ephemeral>0</nova:ephemeral>
Nov 29 03:21:21 user nova-compute[75234]:         <nova:vcpus>1</nova:vcpus>
Nov 29 03:21:21 user nova-compute[75234]:       </nova:flavor>
Nov 29 03:21:21 user nova-compute[75234]:       <nova:owner>
Nov 29 03:21:21 user nova-compute[75234]:         <nova:user uuid="e4aed253decd45a4b1f9e42550f7dd81">tempest-SnapshotDataIntegrityTests-1450496038-project-member</nova:user>
Nov 29 03:21:21 user nova-compute[75234]:         <nova:project uuid="7c3b05b1968540dbbb40817c87846ff6">tempest-SnapshotDataIntegrityTests-1450496038</nova:project>
Nov 29 03:21:21 user nova-compute[75234]:       </nova:owner>
Nov 29 03:21:21 user nova-compute[75234]:       <nova:root type="image" uuid="96c30ba7-9b73-4a30-8ba4-7138879573b7"/>
Nov 29 03:21:21 user nova-compute[75234]:       <nova:ports>
Nov 29 03:21:21 user nova-compute[75234]:         <nova:port uuid="e43ebf8d-4145-4a78-8be5-600ea409e775">
Nov 29 03:21:21 user nova-compute[75234]:           <nova:ip type="fixed" address="10.0.0.14" ipVersion="4"/>
Nov 29 03:21:21 user nova-compute[75234]:         </nova:port>
Nov 29 03:21:21 user nova-compute[75234]:       </nova:ports>
Nov 29 03:21:21 user nova-compute[75234]:     </nova:instance>
Nov 29 03:21:21 user nova-compute[75234]:   </metadata>
Nov 29 03:21:21 user nova-compute[75234]:   <sysinfo type="smbios">
Nov 29 03:21:21 user nova-compute[75234]:     <system>
Nov 29 03:21:21 user nova-compute[75234]:       <entry name="manufacturer">OpenStack Foundation</entry>
Nov 29 03:21:21 user nova-compute[75234]:       <entry name="product">OpenStack Nova</entry>
Nov 29 03:21:21 user nova-compute[75234]:       <entry name="version">0.0.0</entry>
Nov 29 03:21:21 user nova-compute[75234]:       <entry name="serial">f6bc212d-85cd-479c-92ae-112c9f0101bc</entry>
Nov 29 03:21:21 user nova-compute[75234]:       <entry name="uuid">f6bc212d-85cd-479c-92ae-112c9f0101bc</entry>
Nov 29 03:21:21 user nova-compute[75234]:       <entry name="family">Virtual Machine</entry>
Nov 29 03:21:21 user nova-compute[75234]:     </system>
Nov 29 03:21:21 user nova-compute[75234]:   </sysinfo>
Nov 29 03:21:21 user nova-compute[75234]:   <os>
Nov 29 03:21:21 user nova-compute[75234]:     <type arch="x86_64" machine="pc">hvm</type>
Nov 29 03:21:21 user nova-compute[75234]:     <boot dev="hd"/>
Nov 29 03:21:21 user nova-compute[75234]:     <smbios mode="sysinfo"/>
Nov 29 03:21:21 user nova-compute[75234]:   </os>
Nov 29 03:21:21 user nova-compute[75234]:   <features>
Nov 29 03:21:21 user nova-compute[75234]:     <acpi/>
Nov 29 03:21:21 user nova-compute[75234]:     <apic/>
Nov 29 03:21:21 user nova-compute[75234]:     <vmcoreinfo/>
Nov 29 03:21:21 user nova-compute[75234]:   </features>
Nov 29 03:21:21 user nova-compute[75234]:   <clock offset="utc">
Nov 29 03:21:21 user nova-compute[75234]:     <timer name="pit" tickpolicy="delay"/>
Nov 29 03:21:21 user nova-compute[75234]:     <timer name="rtc" tickpolicy="catchup"/>
Nov 29 03:21:21 user nova-compute[75234]:     <timer name="hpet" present="no"/>
Nov 29 03:21:21 user nova-compute[75234]:   </clock>
Nov 29 03:21:21 user nova-compute[75234]:   <cpu mode="custom" match="exact">
Nov 29 03:21:21 user nova-compute[75234]:     <model>Nehalem</model>
Nov 29 03:21:21 user nova-compute[75234]:     <topology sockets="1" cores="1" threads="1"/>
Nov 29 03:21:21 user nova-compute[75234]:   </cpu>
Nov 29 03:21:21 user nova-compute[75234]:   <devices>
Nov 29 03:21:21 user nova-compute[75234]:     <disk type="file" device="disk">
Nov 29 03:21:21 user nova-compute[75234]:       <driver name="qemu" type="qcow2" cache="none"/>
Nov 29 03:21:21 user nova-compute[75234]:       <source file="/opt/stack/data/nova/instances/f6bc212d-85cd-479c-92ae-112c9f0101bc/disk"/>
Nov 29 03:21:21 user nova-compute[75234]:       <target dev="vda" bus="virtio"/>
Nov 29 03:21:21 user nova-compute[75234]:     </disk>
Nov 29 03:21:21 user nova-compute[75234]:     <interface type="ethernet">
Nov 29 03:21:21 user nova-compute[75234]:       <mac address="fa:16:3e:4e:8e:1d"/>
Nov 29 03:21:21 user nova-compute[75234]:       <model type="virtio"/>
Nov 29 03:21:21 user nova-compute[75234]:       <mtu size="1442"/>
Nov 29 03:21:21 user nova-compute[75234]:       <target dev="tape43ebf8d-41"/>
Nov 29 03:21:21 user nova-compute[75234]:     </interface>
Nov 29 03:21:21 user nova-compute[75234]:     <serial type="pty">
Nov 29 03:21:21 user nova-compute[75234]:       <log file="/opt/stack/data/nova/instances/f6bc212d-85cd-479c-92ae-112c9f0101bc/console.log" append="off"/>
Nov 29 03:21:21 user nova-compute[75234]:     </serial>
Nov 29 03:21:21 user nova-compute[75234]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Nov 29 03:21:21 user nova-compute[75234]:     <video>
Nov 29 03:21:21 user nova-compute[75234]:       <model type="virtio"/>
Nov 29 03:21:21 user nova-compute[75234]:     </video>
Nov 29 03:21:21 user nova-compute[75234]:     <rng model="virtio">
Nov 29 03:21:21 user nova-compute[75234]:       <backend model="random">/dev/urandom</backend>
Nov 29 03:21:21 user nova-compute[75234]:     </rng>
Nov 29 03:21:21 user nova-compute[75234]:     <controller type="usb" index="0" model="none"/>
Nov 29 03:21:21 user nova-compute[75234]:     <memballoon model="virtio">
Nov 29 03:21:21 user nova-compute[75234]:       <stats period="10"/>
Nov 29 03:21:21 user nova-compute[75234]:     </memballoon>
Nov 29 03:21:21 user nova-compute[75234]:   </devices>
Nov 29 03:21:21 user nova-compute[75234]: </domain>
Nov 29 03:21:21 user nova-compute[75234]:  {{(pid=75234) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7568}}
Nov 29 03:21:21 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-5643b891-bd60-4956-82ad-f62045205121 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-11-29T03:21:17Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-SnapshotDataIntegrityTests-server-744808219',display_name='tempest-SnapshotDataIntegrityTests-server-744808219',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-snapshotdataintegritytests-server-744808219',id=37,image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBLcl+lLJR5zldffje/+52KkXZkw4CLRUBrN092/lLCyYU8Rz6cZyByDOlFzBvjoKUTlEBjgZpSHtec+jLeJl2h2rkNlWQ+il4II7lulZv0B8LAiMbGsvFeL2w1CWre/RQw==',key_name='tempest-keypair-1458143874',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='7c3b05b1968540dbbb40817c87846ff6',ramdisk_id='',reservation_id='r-vu6047ud',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-SnapshotDataIntegrityTests-1450496038',owner_user_name='tempest-SnapshotDataIntegrityTests-1450496038-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-11-29T03:21:20Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='e4aed253decd45a4b1f9e42550f7dd81',uuid=f6bc212d-85cd-479c-92ae-112c9f0101bc,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "e43ebf8d-4145-4a78-8be5-600ea409e775", "address": "fa:16:3e:4e:8e:1d", "network": {"id": "9247dc93-6a54-495e-b113-d1ff7c215178", "bridge": "br-int", "label": "tempest-SnapshotDataIntegrityTests-261469187-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "7c3b05b1968540dbbb40817c87846ff6", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape43ebf8d-41", "ovs_interfaceid": "e43ebf8d-4145-4a78-8be5-600ea409e775", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Nov 29 03:21:21 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-5643b891-bd60-4956-82ad-f62045205121 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Converting VIF {"id": "e43ebf8d-4145-4a78-8be5-600ea409e775", "address": "fa:16:3e:4e:8e:1d", "network": {"id": "9247dc93-6a54-495e-b113-d1ff7c215178", "bridge": "br-int", "label": "tempest-SnapshotDataIntegrityTests-261469187-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "7c3b05b1968540dbbb40817c87846ff6", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape43ebf8d-41", "ovs_interfaceid": "e43ebf8d-4145-4a78-8be5-600ea409e775", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:21:21 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-5643b891-bd60-4956-82ad-f62045205121 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:4e:8e:1d,bridge_name='br-int',has_traffic_filtering=True,id=e43ebf8d-4145-4a78-8be5-600ea409e775,network=Network(9247dc93-6a54-495e-b113-d1ff7c215178),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tape43ebf8d-41') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:21:21 user nova-compute[75234]: DEBUG os_vif [None req-5643b891-bd60-4956-82ad-f62045205121 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:4e:8e:1d,bridge_name='br-int',has_traffic_filtering=True,id=e43ebf8d-4145-4a78-8be5-600ea409e775,network=Network(9247dc93-6a54-495e-b113-d1ff7c215178),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tape43ebf8d-41') {{(pid=75234) plug /opt/stack/data/venv/lib/python3.10/site-packages/os_vif/__init__.py:76}}
Nov 29 03:21:21 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:21:21 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:21:21 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Nov 29 03:21:21 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:21:21 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbCreateCommand(_result=None, table=QoS, columns={'type': 'linux-noop', 'external_ids': {'id': '9ddee55a-8bd7-570a-af2c-d7102df7b1ac', '_type': 'linux-noop'}}, row=False) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:21:21 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:21:21 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:21:21 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:21:21 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tape43ebf8d-41, may_exist=True, interface_attrs={}) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:21:21 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Port, record=tape43ebf8d-41, col_values=(('qos', UUID('b0d5fe14-fca7-4a5d-8797-b3197b45ace5')),), if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:21:21 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DbSetCommand(_result=None, table=Interface, record=tape43ebf8d-41, col_values=(('external_ids', {'iface-id': 'e43ebf8d-4145-4a78-8be5-600ea409e775', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:4e:8e:1d', 'vm-uuid': 'f6bc212d-85cd-479c-92ae-112c9f0101bc'}),), if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:21:21 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:21:21 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:21:21 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:21:21 user nova-compute[75234]: INFO os_vif [None req-5643b891-bd60-4956-82ad-f62045205121 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:4e:8e:1d,bridge_name='br-int',has_traffic_filtering=True,id=e43ebf8d-4145-4a78-8be5-600ea409e775,network=Network(9247dc93-6a54-495e-b113-d1ff7c215178),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tape43ebf8d-41')
Nov 29 03:21:21 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-5643b891-bd60-4956-82ad-f62045205121 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] No BDM found with device name vda, not building metadata. {{(pid=75234) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12089}}
Nov 29 03:21:21 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-5643b891-bd60-4956-82ad-f62045205121 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] No VIF found with MAC fa:16:3e:4e:8e:1d, not building metadata {{(pid=75234) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Nov 29 03:21:22 user nova-compute[75234]: INFO nova.compute.manager [None req-88a3cd8c-0fc7-4ee0-b899-c619a4d9aca9 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] [instance: 1df140f1-fbd9-4b5d-b356-d08b7e908f64] Rescuing
Nov 29 03:21:22 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-88a3cd8c-0fc7-4ee0-b899-c619a4d9aca9 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Acquiring lock "refresh_cache-1df140f1-fbd9-4b5d-b356-d08b7e908f64" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:21:22 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-88a3cd8c-0fc7-4ee0-b899-c619a4d9aca9 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Acquired lock "refresh_cache-1df140f1-fbd9-4b5d-b356-d08b7e908f64" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:21:22 user nova-compute[75234]: DEBUG nova.network.neutron [None req-88a3cd8c-0fc7-4ee0-b899-c619a4d9aca9 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] [instance: 1df140f1-fbd9-4b5d-b356-d08b7e908f64] Building network info cache for instance {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2009}}
Nov 29 03:21:22 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:21:22 user nova-compute[75234]: DEBUG nova.network.neutron [req-23947139-8215-4907-b13b-12fd394a5e34 req-42a4ec7b-d725-48ca-8c10-368ccacfa18c service nova] [instance: f6bc212d-85cd-479c-92ae-112c9f0101bc] Updated VIF entry in instance network info cache for port e43ebf8d-4145-4a78-8be5-600ea409e775. {{(pid=75234) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3481}}
Nov 29 03:21:22 user nova-compute[75234]: DEBUG nova.network.neutron [req-23947139-8215-4907-b13b-12fd394a5e34 req-42a4ec7b-d725-48ca-8c10-368ccacfa18c service nova] [instance: f6bc212d-85cd-479c-92ae-112c9f0101bc] Updating instance_info_cache with network_info: [{"id": "e43ebf8d-4145-4a78-8be5-600ea409e775", "address": "fa:16:3e:4e:8e:1d", "network": {"id": "9247dc93-6a54-495e-b113-d1ff7c215178", "bridge": "br-int", "label": "tempest-SnapshotDataIntegrityTests-261469187-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "7c3b05b1968540dbbb40817c87846ff6", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape43ebf8d-41", "ovs_interfaceid": "e43ebf8d-4145-4a78-8be5-600ea409e775", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:21:22 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-23947139-8215-4907-b13b-12fd394a5e34 req-42a4ec7b-d725-48ca-8c10-368ccacfa18c service nova] Releasing lock "refresh_cache-f6bc212d-85cd-479c-92ae-112c9f0101bc" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:21:22 user nova-compute[75234]: DEBUG nova.network.neutron [None req-88a3cd8c-0fc7-4ee0-b899-c619a4d9aca9 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] [instance: 1df140f1-fbd9-4b5d-b356-d08b7e908f64] Updating instance_info_cache with network_info: [{"id": "ea9a3586-da52-49b7-8ced-5d27806885ad", "address": "fa:16:3e:80:25:b8", "network": {"id": "2b3c3140-f3b6-45da-8202-bb4d7f0dd130", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-2043420778-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.158", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "f8a7884867c944e1ba1024270e4902f3", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapea9a3586-da", "ovs_interfaceid": "ea9a3586-da52-49b7-8ced-5d27806885ad", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:21:22 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-88a3cd8c-0fc7-4ee0-b899-c619a4d9aca9 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Releasing lock "refresh_cache-1df140f1-fbd9-4b5d-b356-d08b7e908f64" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:21:22 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:21:22 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:21:23 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:21:23 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:21:23 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:21:23 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:21:23 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:21:23 user nova-compute[75234]: DEBUG nova.compute.manager [req-4b48c4ef-40ad-4f31-a330-c7fb9d3d4451 req-07ea525f-72c1-4da9-b14b-3073abe36cd1 service nova] [instance: 1df140f1-fbd9-4b5d-b356-d08b7e908f64] Received event network-vif-unplugged-ea9a3586-da52-49b7-8ced-5d27806885ad {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:21:23 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-4b48c4ef-40ad-4f31-a330-c7fb9d3d4451 req-07ea525f-72c1-4da9-b14b-3073abe36cd1 service nova] Acquiring lock "1df140f1-fbd9-4b5d-b356-d08b7e908f64-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:21:23 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-4b48c4ef-40ad-4f31-a330-c7fb9d3d4451 req-07ea525f-72c1-4da9-b14b-3073abe36cd1 service nova] Lock "1df140f1-fbd9-4b5d-b356-d08b7e908f64-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:21:23 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-4b48c4ef-40ad-4f31-a330-c7fb9d3d4451 req-07ea525f-72c1-4da9-b14b-3073abe36cd1 service nova] Lock "1df140f1-fbd9-4b5d-b356-d08b7e908f64-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:21:23 user nova-compute[75234]: DEBUG nova.compute.manager [req-4b48c4ef-40ad-4f31-a330-c7fb9d3d4451 req-07ea525f-72c1-4da9-b14b-3073abe36cd1 service nova] [instance: 1df140f1-fbd9-4b5d-b356-d08b7e908f64] No waiting events found dispatching network-vif-unplugged-ea9a3586-da52-49b7-8ced-5d27806885ad {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:21:23 user nova-compute[75234]: WARNING nova.compute.manager [req-4b48c4ef-40ad-4f31-a330-c7fb9d3d4451 req-07ea525f-72c1-4da9-b14b-3073abe36cd1 service nova] [instance: 1df140f1-fbd9-4b5d-b356-d08b7e908f64] Received unexpected event network-vif-unplugged-ea9a3586-da52-49b7-8ced-5d27806885ad for instance with vm_state active and task_state rescuing.
Nov 29 03:21:23 user nova-compute[75234]: DEBUG nova.compute.manager [req-c6f58509-65ac-487d-b0c9-0050707541e5 req-e0a46062-f5e0-48c5-894c-1f73c205f721 service nova] [instance: f6bc212d-85cd-479c-92ae-112c9f0101bc] Received event network-vif-plugged-e43ebf8d-4145-4a78-8be5-600ea409e775 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:21:23 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-c6f58509-65ac-487d-b0c9-0050707541e5 req-e0a46062-f5e0-48c5-894c-1f73c205f721 service nova] Acquiring lock "f6bc212d-85cd-479c-92ae-112c9f0101bc-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:21:23 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-c6f58509-65ac-487d-b0c9-0050707541e5 req-e0a46062-f5e0-48c5-894c-1f73c205f721 service nova] Lock "f6bc212d-85cd-479c-92ae-112c9f0101bc-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:21:23 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-c6f58509-65ac-487d-b0c9-0050707541e5 req-e0a46062-f5e0-48c5-894c-1f73c205f721 service nova] Lock "f6bc212d-85cd-479c-92ae-112c9f0101bc-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:21:23 user nova-compute[75234]: DEBUG nova.compute.manager [req-c6f58509-65ac-487d-b0c9-0050707541e5 req-e0a46062-f5e0-48c5-894c-1f73c205f721 service nova] [instance: f6bc212d-85cd-479c-92ae-112c9f0101bc] No waiting events found dispatching network-vif-plugged-e43ebf8d-4145-4a78-8be5-600ea409e775 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:21:23 user nova-compute[75234]: WARNING nova.compute.manager [req-c6f58509-65ac-487d-b0c9-0050707541e5 req-e0a46062-f5e0-48c5-894c-1f73c205f721 service nova] [instance: f6bc212d-85cd-479c-92ae-112c9f0101bc] Received unexpected event network-vif-plugged-e43ebf8d-4145-4a78-8be5-600ea409e775 for instance with vm_state building and task_state spawning.
Nov 29 03:21:23 user nova-compute[75234]: INFO nova.virt.libvirt.driver [-] [instance: 1df140f1-fbd9-4b5d-b356-d08b7e908f64] Instance destroyed successfully.
Nov 29 03:21:23 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-88a3cd8c-0fc7-4ee0-b899-c619a4d9aca9 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] [instance: 1df140f1-fbd9-4b5d-b356-d08b7e908f64] Attempting rescue
Nov 29 03:21:23 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-88a3cd8c-0fc7-4ee0-b899-c619a4d9aca9 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] rescue generated disk_info: {'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'disk.rescue': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vdb', 'type': 'disk'}}} {{(pid=75234) rescue /opt/stack/nova/nova/virt/libvirt/driver.py:4318}}
Nov 29 03:21:23 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:21:23 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-88a3cd8c-0fc7-4ee0-b899-c619a4d9aca9 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] [instance: 1df140f1-fbd9-4b5d-b356-d08b7e908f64] Instance directory exists: not creating {{(pid=75234) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4723}}
Nov 29 03:21:23 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-88a3cd8c-0fc7-4ee0-b899-c619a4d9aca9 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] [instance: 1df140f1-fbd9-4b5d-b356-d08b7e908f64] Creating image(s)
Nov 29 03:21:23 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-88a3cd8c-0fc7-4ee0-b899-c619a4d9aca9 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Acquiring lock "/opt/stack/data/nova/instances/1df140f1-fbd9-4b5d-b356-d08b7e908f64/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:21:23 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-88a3cd8c-0fc7-4ee0-b899-c619a4d9aca9 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Lock "/opt/stack/data/nova/instances/1df140f1-fbd9-4b5d-b356-d08b7e908f64/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:21:23 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-88a3cd8c-0fc7-4ee0-b899-c619a4d9aca9 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Lock "/opt/stack/data/nova/instances/1df140f1-fbd9-4b5d-b356-d08b7e908f64/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:21:23 user nova-compute[75234]: DEBUG nova.objects.instance [None req-88a3cd8c-0fc7-4ee0-b899-c619a4d9aca9 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Lazy-loading 'trusted_certs' on Instance uuid 1df140f1-fbd9-4b5d-b356-d08b7e908f64 {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:21:23 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:21:23 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:21:23 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-88a3cd8c-0fc7-4ee0-b899-c619a4d9aca9 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Acquiring lock "80686e29dc9db6600d03e85cb62af298e12d7918" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:21:23 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-88a3cd8c-0fc7-4ee0-b899-c619a4d9aca9 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Lock "80686e29dc9db6600d03e85cb62af298e12d7918" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:21:23 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-88a3cd8c-0fc7-4ee0-b899-c619a4d9aca9 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:21:24 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:21:24 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-88a3cd8c-0fc7-4ee0-b899-c619a4d9aca9 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json" returned: 0 in 0.201s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:21:24 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-88a3cd8c-0fc7-4ee0-b899-c619a4d9aca9 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918,backing_fmt=raw /opt/stack/data/nova/instances/1df140f1-fbd9-4b5d-b356-d08b7e908f64/disk.rescue {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:21:24 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-88a3cd8c-0fc7-4ee0-b899-c619a4d9aca9 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918,backing_fmt=raw /opt/stack/data/nova/instances/1df140f1-fbd9-4b5d-b356-d08b7e908f64/disk.rescue" returned: 0 in 0.061s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:21:24 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-88a3cd8c-0fc7-4ee0-b899-c619a4d9aca9 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Lock "80686e29dc9db6600d03e85cb62af298e12d7918" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.268s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:21:24 user nova-compute[75234]: DEBUG nova.objects.instance [None req-88a3cd8c-0fc7-4ee0-b899-c619a4d9aca9 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Lazy-loading 'migration_context' on Instance uuid 1df140f1-fbd9-4b5d-b356-d08b7e908f64 {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:21:24 user nova-compute[75234]: DEBUG nova.objects.base [None req-88a3cd8c-0fc7-4ee0-b899-c619a4d9aca9 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Object Instance<1df140f1-fbd9-4b5d-b356-d08b7e908f64> lazy-loaded attributes: trusted_certs,migration_context {{(pid=75234) wrapper /opt/stack/nova/nova/objects/base.py:126}}
Nov 29 03:21:24 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-88a3cd8c-0fc7-4ee0-b899-c619a4d9aca9 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] [instance: 1df140f1-fbd9-4b5d-b356-d08b7e908f64] Created local disks {{(pid=75234) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4861}}
Nov 29 03:21:24 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-88a3cd8c-0fc7-4ee0-b899-c619a4d9aca9 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] [instance: 1df140f1-fbd9-4b5d-b356-d08b7e908f64] Start _get_guest_xml network_info=[{"id": "ea9a3586-da52-49b7-8ced-5d27806885ad", "address": "fa:16:3e:80:25:b8", "network": {"id": "2b3c3140-f3b6-45da-8202-bb4d7f0dd130", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-2043420778-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.158", "type": "floating", "version": 4, "meta": {}}], "label": "tempest-ServerRescueNegativeTestJSON-2043420778-network", "vif_mac": "fa:16:3e:80:25:b8"}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "f8a7884867c944e1ba1024270e4902f3", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapea9a3586-da", "ovs_interfaceid": "ea9a3586-da52-49b7-8ced-5d27806885ad", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'disk.rescue': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vdb', 'type': 'disk'}}} image_meta=ImageMeta(checksum='c8fc807773e5354afe61636071771906',container_format='bare',created_at=2023-11-29T03:02:15Z,direct_url=<?>,disk_format='qcow2',id=96c30ba7-9b73-4a30-8ba4-7138879573b7,min_disk=0,min_ram=0,name='cirros-0.6.2-x86_64-disk',owner='b18e380921c748b78517b6f1211cd0f1',properties=ImageMetaProps,protected=<?>,size=21430272,status='active',tags=<?>,updated_at=2023-11-29T03:02:17Z,virtual_size=<?>,visibility=<?>) rescue={'image_id': '96c30ba7-9b73-4a30-8ba4-7138879573b7', 'kernel_id': '', 'ramdisk_id': ''} block_device_info=None {{(pid=75234) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7562}}
Nov 29 03:21:24 user nova-compute[75234]: DEBUG nova.objects.instance [None req-88a3cd8c-0fc7-4ee0-b899-c619a4d9aca9 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Lazy-loading 'resources' on Instance uuid 1df140f1-fbd9-4b5d-b356-d08b7e908f64 {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:21:24 user nova-compute[75234]: DEBUG nova.objects.base [None req-88a3cd8c-0fc7-4ee0-b899-c619a4d9aca9 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Object Instance<1df140f1-fbd9-4b5d-b356-d08b7e908f64> lazy-loaded attributes: trusted_certs,migration_context,resources {{(pid=75234) wrapper /opt/stack/nova/nova/objects/base.py:126}}
Nov 29 03:21:24 user nova-compute[75234]: DEBUG nova.objects.instance [None req-88a3cd8c-0fc7-4ee0-b899-c619a4d9aca9 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Lazy-loading 'numa_topology' on Instance uuid 1df140f1-fbd9-4b5d-b356-d08b7e908f64 {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:21:24 user nova-compute[75234]: DEBUG nova.objects.base [None req-88a3cd8c-0fc7-4ee0-b899-c619a4d9aca9 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Object Instance<1df140f1-fbd9-4b5d-b356-d08b7e908f64> lazy-loaded attributes: trusted_certs,migration_context,resources,numa_topology {{(pid=75234) wrapper /opt/stack/nova/nova/objects/base.py:126}}
Nov 29 03:21:24 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-88a3cd8c-0fc7-4ee0-b899-c619a4d9aca9 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:21:24 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-88a3cd8c-0fc7-4ee0-b899-c619a4d9aca9 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:21:24 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-88a3cd8c-0fc7-4ee0-b899-c619a4d9aca9 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Searching host: 'user' for CPU controller through CGroups V1... {{(pid=75234) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1652}}
Nov 29 03:21:24 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-88a3cd8c-0fc7-4ee0-b899-c619a4d9aca9 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] CPU controller missing on host. {{(pid=75234) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1662}}
Nov 29 03:21:24 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-88a3cd8c-0fc7-4ee0-b899-c619a4d9aca9 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Searching host: 'user' for CPU controller through CGroups V2... {{(pid=75234) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1671}}
Nov 29 03:21:24 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-88a3cd8c-0fc7-4ee0-b899-c619a4d9aca9 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] CPU controller found on host. {{(pid=75234) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1678}}
Nov 29 03:21:24 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-88a3cd8c-0fc7-4ee0-b899-c619a4d9aca9 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=75234) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5400}}
Nov 29 03:21:24 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-88a3cd8c-0fc7-4ee0-b899-c619a4d9aca9 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-11-29T03:02:04Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='1',id=6,is_public=True,memory_mb=512,name='m1.tiny',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='c8fc807773e5354afe61636071771906',container_format='bare',created_at=2023-11-29T03:02:15Z,direct_url=<?>,disk_format='qcow2',id=96c30ba7-9b73-4a30-8ba4-7138879573b7,min_disk=0,min_ram=0,name='cirros-0.6.2-x86_64-disk',owner='b18e380921c748b78517b6f1211cd0f1',properties=ImageMetaProps,protected=<?>,size=21430272,status='active',tags=<?>,updated_at=2023-11-29T03:02:17Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:563}}
Nov 29 03:21:24 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-88a3cd8c-0fc7-4ee0-b899-c619a4d9aca9 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Flavor limits 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:348}}
Nov 29 03:21:24 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-88a3cd8c-0fc7-4ee0-b899-c619a4d9aca9 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Image limits 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:352}}
Nov 29 03:21:24 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-88a3cd8c-0fc7-4ee0-b899-c619a4d9aca9 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Flavor pref 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:388}}
Nov 29 03:21:24 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-88a3cd8c-0fc7-4ee0-b899-c619a4d9aca9 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Image pref 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:392}}
Nov 29 03:21:24 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-88a3cd8c-0fc7-4ee0-b899-c619a4d9aca9 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:430}}
Nov 29 03:21:24 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-88a3cd8c-0fc7-4ee0-b899-c619a4d9aca9 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:569}}
Nov 29 03:21:24 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-88a3cd8c-0fc7-4ee0-b899-c619a4d9aca9 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=75234) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:471}}
Nov 29 03:21:24 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-88a3cd8c-0fc7-4ee0-b899-c619a4d9aca9 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Got 1 possible topologies {{(pid=75234) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:501}}
Nov 29 03:21:24 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-88a3cd8c-0fc7-4ee0-b899-c619a4d9aca9 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:575}}
Nov 29 03:21:24 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-88a3cd8c-0fc7-4ee0-b899-c619a4d9aca9 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:577}}
Nov 29 03:21:24 user nova-compute[75234]: DEBUG nova.objects.instance [None req-88a3cd8c-0fc7-4ee0-b899-c619a4d9aca9 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Lazy-loading 'vcpu_model' on Instance uuid 1df140f1-fbd9-4b5d-b356-d08b7e908f64 {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:21:24 user nova-compute[75234]: DEBUG nova.objects.base [None req-88a3cd8c-0fc7-4ee0-b899-c619a4d9aca9 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Object Instance<1df140f1-fbd9-4b5d-b356-d08b7e908f64> lazy-loaded attributes: trusted_certs,migration_context,resources,numa_topology,vcpu_model {{(pid=75234) wrapper /opt/stack/nova/nova/objects/base.py:126}}
Nov 29 03:21:24 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-88a3cd8c-0fc7-4ee0-b899-c619a4d9aca9 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-11-29T03:19:10Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-ServerRescueNegativeTestJSON-server-278539854',display_name='tempest-ServerRescueNegativeTestJSON-server-278539854',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-serverrescuenegativetestjson-server-278539854',id=33,image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBKySS+rrPWcijvoGGhKEPiK1xSDIj74zuelaF50GYBXUvR1Yet4BFb/2sYsZA/5NzUIJ43mYGg7Ig5cf2ANImb2nTP65fl4DOr9JzbUOlYxAqRtjhkf+S+OjV52cB6+jDw==',key_name='tempest-keypair-605296868',keypairs=<?>,launch_index=0,launched_at=2023-11-29T03:19:17Z,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='f8a7884867c944e1ba1024270e4902f3',ramdisk_id='',reservation_id='r-s6t6psv1',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-ServerRescueNegativeTestJSON-1182706601',owner_user_name='tempest-ServerRescueNegativeTestJSON-1182706601-project-member'},tags=<?>,task_state='rescuing',terminated_at=None,trusted_certs=None,updated_at=2023-11-29T03:19:18Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='6792e866677645c598ddb7d2c1723a1a',uuid=1df140f1-fbd9-4b5d-b356-d08b7e908f64,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "ea9a3586-da52-49b7-8ced-5d27806885ad", "address": "fa:16:3e:80:25:b8", "network": {"id": "2b3c3140-f3b6-45da-8202-bb4d7f0dd130", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-2043420778-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.158", "type": "floating", "version": 4, "meta": {}}], "label": "tempest-ServerRescueNegativeTestJSON-2043420778-network", "vif_mac": "fa:16:3e:80:25:b8"}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "f8a7884867c944e1ba1024270e4902f3", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapea9a3586-da", "ovs_interfaceid": "ea9a3586-da52-49b7-8ced-5d27806885ad", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=75234) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Nov 29 03:21:24 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-88a3cd8c-0fc7-4ee0-b899-c619a4d9aca9 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Converting VIF {"id": "ea9a3586-da52-49b7-8ced-5d27806885ad", "address": "fa:16:3e:80:25:b8", "network": {"id": "2b3c3140-f3b6-45da-8202-bb4d7f0dd130", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-2043420778-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.158", "type": "floating", "version": 4, "meta": {}}], "label": "tempest-ServerRescueNegativeTestJSON-2043420778-network", "vif_mac": "fa:16:3e:80:25:b8"}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "f8a7884867c944e1ba1024270e4902f3", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapea9a3586-da", "ovs_interfaceid": "ea9a3586-da52-49b7-8ced-5d27806885ad", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:21:24 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-88a3cd8c-0fc7-4ee0-b899-c619a4d9aca9 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:80:25:b8,bridge_name='br-int',has_traffic_filtering=True,id=ea9a3586-da52-49b7-8ced-5d27806885ad,network=Network(2b3c3140-f3b6-45da-8202-bb4d7f0dd130),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapea9a3586-da') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:21:24 user nova-compute[75234]: DEBUG nova.objects.instance [None req-88a3cd8c-0fc7-4ee0-b899-c619a4d9aca9 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Lazy-loading 'pci_devices' on Instance uuid 1df140f1-fbd9-4b5d-b356-d08b7e908f64 {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:21:24 user nova-compute[75234]: DEBUG nova.objects.base [None req-88a3cd8c-0fc7-4ee0-b899-c619a4d9aca9 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Object Instance<1df140f1-fbd9-4b5d-b356-d08b7e908f64> lazy-loaded attributes: trusted_certs,migration_context,resources,numa_topology,vcpu_model,pci_devices {{(pid=75234) wrapper /opt/stack/nova/nova/objects/base.py:126}}
Nov 29 03:21:24 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-88a3cd8c-0fc7-4ee0-b899-c619a4d9aca9 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] [instance: 1df140f1-fbd9-4b5d-b356-d08b7e908f64] End _get_guest_xml xml=<domain type="kvm">
Nov 29 03:21:24 user nova-compute[75234]:   <uuid>1df140f1-fbd9-4b5d-b356-d08b7e908f64</uuid>
Nov 29 03:21:24 user nova-compute[75234]:   <name>instance-00000021</name>
Nov 29 03:21:24 user nova-compute[75234]:   <memory>524288</memory>
Nov 29 03:21:24 user nova-compute[75234]:   <vcpu>1</vcpu>
Nov 29 03:21:24 user nova-compute[75234]:   <metadata>
Nov 29 03:21:24 user nova-compute[75234]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Nov 29 03:21:24 user nova-compute[75234]:       <nova:package version="0.0.0"/>
Nov 29 03:21:24 user nova-compute[75234]:       <nova:name>tempest-ServerRescueNegativeTestJSON-server-278539854</nova:name>
Nov 29 03:21:24 user nova-compute[75234]:       <nova:creationTime>2023-11-29 03:21:24</nova:creationTime>
Nov 29 03:21:24 user nova-compute[75234]:       <nova:flavor name="m1.tiny">
Nov 29 03:21:24 user nova-compute[75234]:         <nova:memory>512</nova:memory>
Nov 29 03:21:24 user nova-compute[75234]:         <nova:disk>1</nova:disk>
Nov 29 03:21:24 user nova-compute[75234]:         <nova:swap>0</nova:swap>
Nov 29 03:21:24 user nova-compute[75234]:         <nova:ephemeral>0</nova:ephemeral>
Nov 29 03:21:24 user nova-compute[75234]:         <nova:vcpus>1</nova:vcpus>
Nov 29 03:21:24 user nova-compute[75234]:       </nova:flavor>
Nov 29 03:21:24 user nova-compute[75234]:       <nova:owner>
Nov 29 03:21:24 user nova-compute[75234]:         <nova:user uuid="6792e866677645c598ddb7d2c1723a1a">tempest-ServerRescueNegativeTestJSON-1182706601-project-member</nova:user>
Nov 29 03:21:24 user nova-compute[75234]:         <nova:project uuid="f8a7884867c944e1ba1024270e4902f3">tempest-ServerRescueNegativeTestJSON-1182706601</nova:project>
Nov 29 03:21:24 user nova-compute[75234]:       </nova:owner>
Nov 29 03:21:24 user nova-compute[75234]:       <nova:root type="image" uuid="96c30ba7-9b73-4a30-8ba4-7138879573b7"/>
Nov 29 03:21:24 user nova-compute[75234]:       <nova:ports>
Nov 29 03:21:24 user nova-compute[75234]:         <nova:port uuid="ea9a3586-da52-49b7-8ced-5d27806885ad">
Nov 29 03:21:24 user nova-compute[75234]:           <nova:ip type="fixed" address="10.0.0.3" ipVersion="4"/>
Nov 29 03:21:24 user nova-compute[75234]:         </nova:port>
Nov 29 03:21:24 user nova-compute[75234]:       </nova:ports>
Nov 29 03:21:24 user nova-compute[75234]:     </nova:instance>
Nov 29 03:21:24 user nova-compute[75234]:   </metadata>
Nov 29 03:21:24 user nova-compute[75234]:   <sysinfo type="smbios">
Nov 29 03:21:24 user nova-compute[75234]:     <system>
Nov 29 03:21:24 user nova-compute[75234]:       <entry name="manufacturer">OpenStack Foundation</entry>
Nov 29 03:21:24 user nova-compute[75234]:       <entry name="product">OpenStack Nova</entry>
Nov 29 03:21:24 user nova-compute[75234]:       <entry name="version">0.0.0</entry>
Nov 29 03:21:24 user nova-compute[75234]:       <entry name="serial">1df140f1-fbd9-4b5d-b356-d08b7e908f64</entry>
Nov 29 03:21:24 user nova-compute[75234]:       <entry name="uuid">1df140f1-fbd9-4b5d-b356-d08b7e908f64</entry>
Nov 29 03:21:24 user nova-compute[75234]:       <entry name="family">Virtual Machine</entry>
Nov 29 03:21:24 user nova-compute[75234]:     </system>
Nov 29 03:21:24 user nova-compute[75234]:   </sysinfo>
Nov 29 03:21:24 user nova-compute[75234]:   <os>
Nov 29 03:21:24 user nova-compute[75234]:     <type arch="x86_64" machine="pc">hvm</type>
Nov 29 03:21:24 user nova-compute[75234]:     <smbios mode="sysinfo"/>
Nov 29 03:21:24 user nova-compute[75234]:   </os>
Nov 29 03:21:24 user nova-compute[75234]:   <features>
Nov 29 03:21:24 user nova-compute[75234]:     <acpi/>
Nov 29 03:21:24 user nova-compute[75234]:     <apic/>
Nov 29 03:21:24 user nova-compute[75234]:     <vmcoreinfo/>
Nov 29 03:21:24 user nova-compute[75234]:   </features>
Nov 29 03:21:24 user nova-compute[75234]:   <clock offset="utc">
Nov 29 03:21:24 user nova-compute[75234]:     <timer name="pit" tickpolicy="delay"/>
Nov 29 03:21:24 user nova-compute[75234]:     <timer name="rtc" tickpolicy="catchup"/>
Nov 29 03:21:24 user nova-compute[75234]:     <timer name="hpet" present="no"/>
Nov 29 03:21:24 user nova-compute[75234]:   </clock>
Nov 29 03:21:24 user nova-compute[75234]:   <cpu mode="custom" match="exact">
Nov 29 03:21:24 user nova-compute[75234]:     <model>Nehalem</model>
Nov 29 03:21:24 user nova-compute[75234]:     <topology sockets="1" cores="1" threads="1"/>
Nov 29 03:21:24 user nova-compute[75234]:   </cpu>
Nov 29 03:21:24 user nova-compute[75234]:   <devices>
Nov 29 03:21:24 user nova-compute[75234]:     <disk type="file" device="disk">
Nov 29 03:21:24 user nova-compute[75234]:       <driver name="qemu" type="qcow2" cache="none"/>
Nov 29 03:21:24 user nova-compute[75234]:       <source file="/opt/stack/data/nova/instances/1df140f1-fbd9-4b5d-b356-d08b7e908f64/disk.rescue"/>
Nov 29 03:21:24 user nova-compute[75234]:       <target dev="vda" bus="virtio"/>
Nov 29 03:21:24 user nova-compute[75234]:     </disk>
Nov 29 03:21:24 user nova-compute[75234]:     <disk type="file" device="disk">
Nov 29 03:21:24 user nova-compute[75234]:       <driver name="qemu" type="qcow2" cache="none"/>
Nov 29 03:21:24 user nova-compute[75234]:       <source file="/opt/stack/data/nova/instances/1df140f1-fbd9-4b5d-b356-d08b7e908f64/disk"/>
Nov 29 03:21:24 user nova-compute[75234]:       <target dev="vdb" bus="virtio"/>
Nov 29 03:21:24 user nova-compute[75234]:     </disk>
Nov 29 03:21:24 user nova-compute[75234]:     <interface type="ethernet">
Nov 29 03:21:24 user nova-compute[75234]:       <mac address="fa:16:3e:80:25:b8"/>
Nov 29 03:21:24 user nova-compute[75234]:       <model type="virtio"/>
Nov 29 03:21:24 user nova-compute[75234]:       <mtu size="1442"/>
Nov 29 03:21:24 user nova-compute[75234]:       <target dev="tapea9a3586-da"/>
Nov 29 03:21:24 user nova-compute[75234]:     </interface>
Nov 29 03:21:24 user nova-compute[75234]:     <serial type="pty">
Nov 29 03:21:24 user nova-compute[75234]:       <log file="/opt/stack/data/nova/instances/1df140f1-fbd9-4b5d-b356-d08b7e908f64/console.log" append="off"/>
Nov 29 03:21:24 user nova-compute[75234]:     </serial>
Nov 29 03:21:24 user nova-compute[75234]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Nov 29 03:21:24 user nova-compute[75234]:     <video>
Nov 29 03:21:24 user nova-compute[75234]:       <model type="virtio"/>
Nov 29 03:21:24 user nova-compute[75234]:     </video>
Nov 29 03:21:24 user nova-compute[75234]:     <rng model="virtio">
Nov 29 03:21:24 user nova-compute[75234]:       <backend model="random">/dev/urandom</backend>
Nov 29 03:21:24 user nova-compute[75234]:     </rng>
Nov 29 03:21:24 user nova-compute[75234]:     <controller type="usb" index="0" model="none"/>
Nov 29 03:21:24 user nova-compute[75234]:     <memballoon model="virtio">
Nov 29 03:21:24 user nova-compute[75234]:       <stats period="10"/>
Nov 29 03:21:24 user nova-compute[75234]:     </memballoon>
Nov 29 03:21:24 user nova-compute[75234]:   </devices>
Nov 29 03:21:24 user nova-compute[75234]: </domain>
Nov 29 03:21:24 user nova-compute[75234]:  {{(pid=75234) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7568}}
Nov 29 03:21:24 user nova-compute[75234]: INFO nova.virt.libvirt.driver [-] [instance: 1df140f1-fbd9-4b5d-b356-d08b7e908f64] Instance destroyed successfully.
Nov 29 03:21:24 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:21:24 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-88a3cd8c-0fc7-4ee0-b899-c619a4d9aca9 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] No BDM found with device name vda, not building metadata. {{(pid=75234) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12089}}
Nov 29 03:21:24 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-88a3cd8c-0fc7-4ee0-b899-c619a4d9aca9 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] No BDM found with device name vdb, not building metadata. {{(pid=75234) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12089}}
Nov 29 03:21:24 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-88a3cd8c-0fc7-4ee0-b899-c619a4d9aca9 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] No VIF found with MAC fa:16:3e:80:25:b8, not building metadata {{(pid=75234) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Nov 29 03:21:25 user nova-compute[75234]: DEBUG nova.compute.manager [req-5f4e5acb-c7a1-483f-8469-38f04354e0a9 req-ffa0e493-84e9-4181-b283-f8a96f75ab17 service nova] [instance: 1df140f1-fbd9-4b5d-b356-d08b7e908f64] Received event network-vif-plugged-ea9a3586-da52-49b7-8ced-5d27806885ad {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:21:25 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-5f4e5acb-c7a1-483f-8469-38f04354e0a9 req-ffa0e493-84e9-4181-b283-f8a96f75ab17 service nova] Acquiring lock "1df140f1-fbd9-4b5d-b356-d08b7e908f64-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:21:25 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-5f4e5acb-c7a1-483f-8469-38f04354e0a9 req-ffa0e493-84e9-4181-b283-f8a96f75ab17 service nova] Lock "1df140f1-fbd9-4b5d-b356-d08b7e908f64-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:21:25 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-5f4e5acb-c7a1-483f-8469-38f04354e0a9 req-ffa0e493-84e9-4181-b283-f8a96f75ab17 service nova] Lock "1df140f1-fbd9-4b5d-b356-d08b7e908f64-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:21:25 user nova-compute[75234]: DEBUG nova.compute.manager [req-5f4e5acb-c7a1-483f-8469-38f04354e0a9 req-ffa0e493-84e9-4181-b283-f8a96f75ab17 service nova] [instance: 1df140f1-fbd9-4b5d-b356-d08b7e908f64] No waiting events found dispatching network-vif-plugged-ea9a3586-da52-49b7-8ced-5d27806885ad {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:21:25 user nova-compute[75234]: WARNING nova.compute.manager [req-5f4e5acb-c7a1-483f-8469-38f04354e0a9 req-ffa0e493-84e9-4181-b283-f8a96f75ab17 service nova] [instance: 1df140f1-fbd9-4b5d-b356-d08b7e908f64] Received unexpected event network-vif-plugged-ea9a3586-da52-49b7-8ced-5d27806885ad for instance with vm_state active and task_state rescuing.
Nov 29 03:21:25 user nova-compute[75234]: DEBUG nova.compute.manager [req-200d430f-9903-401b-8177-134ecee895fd req-7e8063b3-5aa5-4924-b890-d44f243e58c5 service nova] [instance: f6bc212d-85cd-479c-92ae-112c9f0101bc] Received event network-vif-plugged-e43ebf8d-4145-4a78-8be5-600ea409e775 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:21:25 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-200d430f-9903-401b-8177-134ecee895fd req-7e8063b3-5aa5-4924-b890-d44f243e58c5 service nova] Acquiring lock "f6bc212d-85cd-479c-92ae-112c9f0101bc-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:21:25 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-200d430f-9903-401b-8177-134ecee895fd req-7e8063b3-5aa5-4924-b890-d44f243e58c5 service nova] Lock "f6bc212d-85cd-479c-92ae-112c9f0101bc-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:21:25 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-200d430f-9903-401b-8177-134ecee895fd req-7e8063b3-5aa5-4924-b890-d44f243e58c5 service nova] Lock "f6bc212d-85cd-479c-92ae-112c9f0101bc-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:21:25 user nova-compute[75234]: DEBUG nova.compute.manager [req-200d430f-9903-401b-8177-134ecee895fd req-7e8063b3-5aa5-4924-b890-d44f243e58c5 service nova] [instance: f6bc212d-85cd-479c-92ae-112c9f0101bc] No waiting events found dispatching network-vif-plugged-e43ebf8d-4145-4a78-8be5-600ea409e775 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:21:25 user nova-compute[75234]: WARNING nova.compute.manager [req-200d430f-9903-401b-8177-134ecee895fd req-7e8063b3-5aa5-4924-b890-d44f243e58c5 service nova] [instance: f6bc212d-85cd-479c-92ae-112c9f0101bc] Received unexpected event network-vif-plugged-e43ebf8d-4145-4a78-8be5-600ea409e775 for instance with vm_state building and task_state spawning.
Nov 29 03:21:25 user nova-compute[75234]: DEBUG nova.compute.manager [None req-5643b891-bd60-4956-82ad-f62045205121 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] [instance: f6bc212d-85cd-479c-92ae-112c9f0101bc] Instance event wait completed in 0 seconds for  {{(pid=75234) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Nov 29 03:21:25 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-5643b891-bd60-4956-82ad-f62045205121 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] [instance: f6bc212d-85cd-479c-92ae-112c9f0101bc] Guest created on hypervisor {{(pid=75234) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4421}}
Nov 29 03:21:25 user nova-compute[75234]: INFO nova.virt.libvirt.driver [-] [instance: f6bc212d-85cd-479c-92ae-112c9f0101bc] Instance spawned successfully.
Nov 29 03:21:25 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-5643b891-bd60-4956-82ad-f62045205121 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] [instance: f6bc212d-85cd-479c-92ae-112c9f0101bc] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:904}}
Nov 29 03:21:25 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-5643b891-bd60-4956-82ad-f62045205121 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] [instance: f6bc212d-85cd-479c-92ae-112c9f0101bc] Found default for hw_cdrom_bus of ide {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:21:25 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-5643b891-bd60-4956-82ad-f62045205121 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] [instance: f6bc212d-85cd-479c-92ae-112c9f0101bc] Found default for hw_disk_bus of virtio {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:21:25 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-5643b891-bd60-4956-82ad-f62045205121 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] [instance: f6bc212d-85cd-479c-92ae-112c9f0101bc] Found default for hw_input_bus of None {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:21:25 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-5643b891-bd60-4956-82ad-f62045205121 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] [instance: f6bc212d-85cd-479c-92ae-112c9f0101bc] Found default for hw_pointer_model of None {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:21:25 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-5643b891-bd60-4956-82ad-f62045205121 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] [instance: f6bc212d-85cd-479c-92ae-112c9f0101bc] Found default for hw_video_model of virtio {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:21:25 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-5643b891-bd60-4956-82ad-f62045205121 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] [instance: f6bc212d-85cd-479c-92ae-112c9f0101bc] Found default for hw_vif_model of virtio {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:21:25 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:21:25 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:21:26 user nova-compute[75234]: INFO nova.compute.manager [None req-5643b891-bd60-4956-82ad-f62045205121 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] [instance: f6bc212d-85cd-479c-92ae-112c9f0101bc] Took 6.39 seconds to spawn the instance on the hypervisor.
Nov 29 03:21:26 user nova-compute[75234]: DEBUG nova.compute.manager [None req-5643b891-bd60-4956-82ad-f62045205121 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] [instance: f6bc212d-85cd-479c-92ae-112c9f0101bc] Checking state {{(pid=75234) _get_power_state /opt/stack/nova/nova/compute/manager.py:1782}}
Nov 29 03:21:26 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:21:26 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:21:26 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:21:26 user nova-compute[75234]: INFO nova.compute.manager [None req-5643b891-bd60-4956-82ad-f62045205121 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] [instance: f6bc212d-85cd-479c-92ae-112c9f0101bc] Took 7.21 seconds to build instance.
Nov 29 03:21:26 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-5643b891-bd60-4956-82ad-f62045205121 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Lock "f6bc212d-85cd-479c-92ae-112c9f0101bc" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 7.319s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:21:26 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:21:26 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:21:26 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:21:26 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:21:27 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:21:27 user nova-compute[75234]: DEBUG nova.compute.manager [req-47859afc-dbc8-4af2-9689-8cd5e254793c req-44b12e2f-a810-446a-92f1-92983f5683bc service nova] [instance: 1df140f1-fbd9-4b5d-b356-d08b7e908f64] Received event network-vif-plugged-ea9a3586-da52-49b7-8ced-5d27806885ad {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:21:27 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-47859afc-dbc8-4af2-9689-8cd5e254793c req-44b12e2f-a810-446a-92f1-92983f5683bc service nova] Acquiring lock "1df140f1-fbd9-4b5d-b356-d08b7e908f64-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:21:27 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-47859afc-dbc8-4af2-9689-8cd5e254793c req-44b12e2f-a810-446a-92f1-92983f5683bc service nova] Lock "1df140f1-fbd9-4b5d-b356-d08b7e908f64-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:21:27 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-47859afc-dbc8-4af2-9689-8cd5e254793c req-44b12e2f-a810-446a-92f1-92983f5683bc service nova] Lock "1df140f1-fbd9-4b5d-b356-d08b7e908f64-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.003s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:21:27 user nova-compute[75234]: DEBUG nova.compute.manager [req-47859afc-dbc8-4af2-9689-8cd5e254793c req-44b12e2f-a810-446a-92f1-92983f5683bc service nova] [instance: 1df140f1-fbd9-4b5d-b356-d08b7e908f64] No waiting events found dispatching network-vif-plugged-ea9a3586-da52-49b7-8ced-5d27806885ad {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:21:27 user nova-compute[75234]: WARNING nova.compute.manager [req-47859afc-dbc8-4af2-9689-8cd5e254793c req-44b12e2f-a810-446a-92f1-92983f5683bc service nova] [instance: 1df140f1-fbd9-4b5d-b356-d08b7e908f64] Received unexpected event network-vif-plugged-ea9a3586-da52-49b7-8ced-5d27806885ad for instance with vm_state active and task_state rescuing.
Nov 29 03:21:27 user nova-compute[75234]: DEBUG nova.compute.manager [req-47859afc-dbc8-4af2-9689-8cd5e254793c req-44b12e2f-a810-446a-92f1-92983f5683bc service nova] [instance: 1df140f1-fbd9-4b5d-b356-d08b7e908f64] Received event network-vif-plugged-ea9a3586-da52-49b7-8ced-5d27806885ad {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:21:27 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-47859afc-dbc8-4af2-9689-8cd5e254793c req-44b12e2f-a810-446a-92f1-92983f5683bc service nova] Acquiring lock "1df140f1-fbd9-4b5d-b356-d08b7e908f64-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:21:27 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-47859afc-dbc8-4af2-9689-8cd5e254793c req-44b12e2f-a810-446a-92f1-92983f5683bc service nova] Lock "1df140f1-fbd9-4b5d-b356-d08b7e908f64-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:21:27 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-47859afc-dbc8-4af2-9689-8cd5e254793c req-44b12e2f-a810-446a-92f1-92983f5683bc service nova] Lock "1df140f1-fbd9-4b5d-b356-d08b7e908f64-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:21:27 user nova-compute[75234]: DEBUG nova.compute.manager [req-47859afc-dbc8-4af2-9689-8cd5e254793c req-44b12e2f-a810-446a-92f1-92983f5683bc service nova] [instance: 1df140f1-fbd9-4b5d-b356-d08b7e908f64] No waiting events found dispatching network-vif-plugged-ea9a3586-da52-49b7-8ced-5d27806885ad {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:21:27 user nova-compute[75234]: WARNING nova.compute.manager [req-47859afc-dbc8-4af2-9689-8cd5e254793c req-44b12e2f-a810-446a-92f1-92983f5683bc service nova] [instance: 1df140f1-fbd9-4b5d-b356-d08b7e908f64] Received unexpected event network-vif-plugged-ea9a3586-da52-49b7-8ced-5d27806885ad for instance with vm_state active and task_state rescuing.
Nov 29 03:21:28 user nova-compute[75234]: DEBUG nova.compute.manager [None req-88a3cd8c-0fc7-4ee0-b899-c619a4d9aca9 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] [instance: 1df140f1-fbd9-4b5d-b356-d08b7e908f64] Checking state {{(pid=75234) _get_power_state /opt/stack/nova/nova/compute/manager.py:1782}}
Nov 29 03:21:31 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:21:32 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:21:34 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ef86907e-1288-4d3a-95aa-9c176972b75c tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Acquiring lock "af699bc2-9daa-4b5a-9b4a-ad92fe857d70" by "nova.compute.manager.ComputeManager.confirm_resize.<locals>.do_confirm_resize" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:21:34 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ef86907e-1288-4d3a-95aa-9c176972b75c tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Lock "af699bc2-9daa-4b5a-9b4a-ad92fe857d70" acquired by "nova.compute.manager.ComputeManager.confirm_resize.<locals>.do_confirm_resize" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:21:34 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ef86907e-1288-4d3a-95aa-9c176972b75c tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] [instance: af699bc2-9daa-4b5a-9b4a-ad92fe857d70] Going to confirm migration 2 {{(pid=75234) do_confirm_resize /opt/stack/nova/nova/compute/manager.py:4774}}
Nov 29 03:21:35 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ef86907e-1288-4d3a-95aa-9c176972b75c tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Acquiring lock "refresh_cache-af699bc2-9daa-4b5a-9b4a-ad92fe857d70" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:21:35 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ef86907e-1288-4d3a-95aa-9c176972b75c tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Acquired lock "refresh_cache-af699bc2-9daa-4b5a-9b4a-ad92fe857d70" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:21:35 user nova-compute[75234]: DEBUG nova.network.neutron [None req-ef86907e-1288-4d3a-95aa-9c176972b75c tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] [instance: af699bc2-9daa-4b5a-9b4a-ad92fe857d70] Building network info cache for instance {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2009}}
Nov 29 03:21:35 user nova-compute[75234]: DEBUG nova.objects.instance [None req-ef86907e-1288-4d3a-95aa-9c176972b75c tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Lazy-loading 'info_cache' on Instance uuid af699bc2-9daa-4b5a-9b4a-ad92fe857d70 {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:21:35 user nova-compute[75234]: DEBUG nova.network.neutron [None req-ef86907e-1288-4d3a-95aa-9c176972b75c tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] [instance: af699bc2-9daa-4b5a-9b4a-ad92fe857d70] Updating instance_info_cache with network_info: [{"id": "66dcb0f9-3180-42f7-851d-19b0c4f4119c", "address": "fa:16:3e:dd:9e:50", "network": {"id": "1f946e69-dbec-429c-83f8-a186bad4f2cc", "bridge": "br-int", "label": "tempest-ServerActionsTestOtherA-486369423-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.64", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "c1cb3d1f12f242a785a688d59dbe3cd6", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap66dcb0f9-31", "ovs_interfaceid": "66dcb0f9-3180-42f7-851d-19b0c4f4119c", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:21:35 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ef86907e-1288-4d3a-95aa-9c176972b75c tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Releasing lock "refresh_cache-af699bc2-9daa-4b5a-9b4a-ad92fe857d70" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:21:35 user nova-compute[75234]: DEBUG nova.objects.instance [None req-ef86907e-1288-4d3a-95aa-9c176972b75c tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Lazy-loading 'migration_context' on Instance uuid af699bc2-9daa-4b5a-9b4a-ad92fe857d70 {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:21:35 user nova-compute[75234]: DEBUG nova.objects.base [None req-ef86907e-1288-4d3a-95aa-9c176972b75c tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Object Instance<af699bc2-9daa-4b5a-9b4a-ad92fe857d70> lazy-loaded attributes: info_cache,migration_context {{(pid=75234) wrapper /opt/stack/nova/nova/objects/base.py:126}}
Nov 29 03:21:35 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ef86907e-1288-4d3a-95aa-9c176972b75c tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.drop_move_claim_at_source" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:21:35 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ef86907e-1288-4d3a-95aa-9c176972b75c tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.drop_move_claim_at_source" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:21:36 user nova-compute[75234]: DEBUG nova.compute.provider_tree [None req-ef86907e-1288-4d3a-95aa-9c176972b75c tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Inventory has not changed in ProviderTree for provider: 4d291220-7c37-4839-a0e3-b23c66502b74 {{(pid=75234) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Nov 29 03:21:36 user nova-compute[75234]: DEBUG nova.scheduler.client.report [None req-ef86907e-1288-4d3a-95aa-9c176972b75c tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Inventory has not changed for provider 4d291220-7c37-4839-a0e3-b23c66502b74 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=75234) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Nov 29 03:21:36 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ef86907e-1288-4d3a-95aa-9c176972b75c tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.drop_move_claim_at_source" :: held 0.520s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:21:36 user nova-compute[75234]: INFO nova.scheduler.client.report [None req-ef86907e-1288-4d3a-95aa-9c176972b75c tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Deleted allocation for migration 9297d372-3baa-4214-a4fd-ae62460f818a
Nov 29 03:21:36 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ef86907e-1288-4d3a-95aa-9c176972b75c tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Lock "af699bc2-9daa-4b5a-9b4a-ad92fe857d70" "released" by "nova.compute.manager.ComputeManager.confirm_resize.<locals>.do_confirm_resize" :: held 1.589s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:21:36 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:21:37 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:21:39 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-80688ce8-5cdc-4381-bb7e-6b03a5509431 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Acquiring lock "75c5d700-6367-4128-9228-80d2cd24ace7" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:21:39 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-80688ce8-5cdc-4381-bb7e-6b03a5509431 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Lock "75c5d700-6367-4128-9228-80d2cd24ace7" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:21:39 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-80688ce8-5cdc-4381-bb7e-6b03a5509431 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Acquiring lock "75c5d700-6367-4128-9228-80d2cd24ace7-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:21:39 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-80688ce8-5cdc-4381-bb7e-6b03a5509431 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Lock "75c5d700-6367-4128-9228-80d2cd24ace7-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:21:39 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-80688ce8-5cdc-4381-bb7e-6b03a5509431 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Lock "75c5d700-6367-4128-9228-80d2cd24ace7-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:21:39 user nova-compute[75234]: INFO nova.compute.manager [None req-80688ce8-5cdc-4381-bb7e-6b03a5509431 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] [instance: 75c5d700-6367-4128-9228-80d2cd24ace7] Terminating instance
Nov 29 03:21:39 user nova-compute[75234]: DEBUG nova.compute.manager [None req-80688ce8-5cdc-4381-bb7e-6b03a5509431 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] [instance: 75c5d700-6367-4128-9228-80d2cd24ace7] Start destroying the instance on the hypervisor. {{(pid=75234) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3140}}
Nov 29 03:21:39 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:21:39 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:21:39 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:21:39 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:21:39 user nova-compute[75234]: DEBUG nova.compute.manager [req-bf0cf58b-841e-414f-a9da-c30690629811 req-9fdaf44c-4e16-453b-918f-ba5b724cdea0 service nova] [instance: 75c5d700-6367-4128-9228-80d2cd24ace7] Received event network-vif-unplugged-29c6743e-29e2-40a8-8de9-ec380f1e6f25 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:21:39 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-bf0cf58b-841e-414f-a9da-c30690629811 req-9fdaf44c-4e16-453b-918f-ba5b724cdea0 service nova] Acquiring lock "75c5d700-6367-4128-9228-80d2cd24ace7-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:21:39 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-bf0cf58b-841e-414f-a9da-c30690629811 req-9fdaf44c-4e16-453b-918f-ba5b724cdea0 service nova] Lock "75c5d700-6367-4128-9228-80d2cd24ace7-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:21:39 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-bf0cf58b-841e-414f-a9da-c30690629811 req-9fdaf44c-4e16-453b-918f-ba5b724cdea0 service nova] Lock "75c5d700-6367-4128-9228-80d2cd24ace7-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:21:39 user nova-compute[75234]: DEBUG nova.compute.manager [req-bf0cf58b-841e-414f-a9da-c30690629811 req-9fdaf44c-4e16-453b-918f-ba5b724cdea0 service nova] [instance: 75c5d700-6367-4128-9228-80d2cd24ace7] No waiting events found dispatching network-vif-unplugged-29c6743e-29e2-40a8-8de9-ec380f1e6f25 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:21:39 user nova-compute[75234]: DEBUG nova.compute.manager [req-bf0cf58b-841e-414f-a9da-c30690629811 req-9fdaf44c-4e16-453b-918f-ba5b724cdea0 service nova] [instance: 75c5d700-6367-4128-9228-80d2cd24ace7] Received event network-vif-unplugged-29c6743e-29e2-40a8-8de9-ec380f1e6f25 for instance with task_state deleting. {{(pid=75234) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10890}}
Nov 29 03:21:40 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:21:40 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:21:40 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:21:40 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:21:40 user nova-compute[75234]: INFO nova.virt.libvirt.driver [-] [instance: 75c5d700-6367-4128-9228-80d2cd24ace7] Instance destroyed successfully.
Nov 29 03:21:40 user nova-compute[75234]: DEBUG nova.objects.instance [None req-80688ce8-5cdc-4381-bb7e-6b03a5509431 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Lazy-loading 'resources' on Instance uuid 75c5d700-6367-4128-9228-80d2cd24ace7 {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:21:40 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-80688ce8-5cdc-4381-bb7e-6b03a5509431 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-11-29T03:15:56Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-AttachVolumeNegativeTest-server-194972252',display_name='tempest-AttachVolumeNegativeTest-server-194972252',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-attachvolumenegativetest-server-194972252',id=25,image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBI0t7Q4arbYFn+Iy1wTTAnBoIrlZtFSDcoZm3jtuUD26yMW1xetBXI+9OBf1v6Bmt6DJ2jYAew6ivj/euHwu92De75GASLM8tFPoIdZTEGHNPIEquWrxiVONMRE4Uya8eg==',key_name='tempest-keypair-1224772314',keypairs=<?>,launch_index=0,launched_at=2023-11-29T03:16:06Z,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='315732b6e3ee44d584b8e510ef199a90',ramdisk_id='',reservation_id='r-0hbtjx1d',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-AttachVolumeNegativeTest-1582496806',owner_user_name='tempest-AttachVolumeNegativeTest-1582496806-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-11-29T03:16:06Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='66686114450f4f31b3a917667a152d8b',uuid=75c5d700-6367-4128-9228-80d2cd24ace7,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "29c6743e-29e2-40a8-8de9-ec380f1e6f25", "address": "fa:16:3e:4e:5d:e9", "network": {"id": "0f289171-3d73-4054-95a0-2515b4f206a0", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-1787960284-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.222", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "315732b6e3ee44d584b8e510ef199a90", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap29c6743e-29", "ovs_interfaceid": "29c6743e-29e2-40a8-8de9-ec380f1e6f25", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Nov 29 03:21:40 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-80688ce8-5cdc-4381-bb7e-6b03a5509431 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Converting VIF {"id": "29c6743e-29e2-40a8-8de9-ec380f1e6f25", "address": "fa:16:3e:4e:5d:e9", "network": {"id": "0f289171-3d73-4054-95a0-2515b4f206a0", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-1787960284-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.222", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "315732b6e3ee44d584b8e510ef199a90", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap29c6743e-29", "ovs_interfaceid": "29c6743e-29e2-40a8-8de9-ec380f1e6f25", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:21:40 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-80688ce8-5cdc-4381-bb7e-6b03a5509431 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:4e:5d:e9,bridge_name='br-int',has_traffic_filtering=True,id=29c6743e-29e2-40a8-8de9-ec380f1e6f25,network=Network(0f289171-3d73-4054-95a0-2515b4f206a0),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap29c6743e-29') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:21:40 user nova-compute[75234]: DEBUG os_vif [None req-80688ce8-5cdc-4381-bb7e-6b03a5509431 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Unplugging vif VIFOpenVSwitch(active=True,address=fa:16:3e:4e:5d:e9,bridge_name='br-int',has_traffic_filtering=True,id=29c6743e-29e2-40a8-8de9-ec380f1e6f25,network=Network(0f289171-3d73-4054-95a0-2515b4f206a0),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap29c6743e-29') {{(pid=75234) unplug /opt/stack/data/venv/lib/python3.10/site-packages/os_vif/__init__.py:109}}
Nov 29 03:21:40 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:21:40 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tap29c6743e-29, bridge=br-int, if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:21:40 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:21:40 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:21:40 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:21:40 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbDestroyCommand(_result=None, table=QoS, record=6d3995b0-1eb9-4fab-be86-abf2b5eda897) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:21:40 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:21:40 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:21:40 user nova-compute[75234]: INFO os_vif [None req-80688ce8-5cdc-4381-bb7e-6b03a5509431 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Successfully unplugged vif VIFOpenVSwitch(active=True,address=fa:16:3e:4e:5d:e9,bridge_name='br-int',has_traffic_filtering=True,id=29c6743e-29e2-40a8-8de9-ec380f1e6f25,network=Network(0f289171-3d73-4054-95a0-2515b4f206a0),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap29c6743e-29')
Nov 29 03:21:40 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-80688ce8-5cdc-4381-bb7e-6b03a5509431 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] [instance: 75c5d700-6367-4128-9228-80d2cd24ace7] Deleting instance files /opt/stack/data/nova/instances/75c5d700-6367-4128-9228-80d2cd24ace7_del
Nov 29 03:21:40 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-80688ce8-5cdc-4381-bb7e-6b03a5509431 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] [instance: 75c5d700-6367-4128-9228-80d2cd24ace7] Deletion of /opt/stack/data/nova/instances/75c5d700-6367-4128-9228-80d2cd24ace7_del complete
Nov 29 03:21:40 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:21:40 user nova-compute[75234]: INFO nova.compute.manager [None req-80688ce8-5cdc-4381-bb7e-6b03a5509431 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] [instance: 75c5d700-6367-4128-9228-80d2cd24ace7] Took 1.03 seconds to destroy the instance on the hypervisor.
Nov 29 03:21:40 user nova-compute[75234]: DEBUG oslo.service.loopingcall [None req-80688ce8-5cdc-4381-bb7e-6b03a5509431 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=75234) func /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/loopingcall.py:435}}
Nov 29 03:21:40 user nova-compute[75234]: DEBUG nova.compute.manager [-] [instance: 75c5d700-6367-4128-9228-80d2cd24ace7] Deallocating network for instance {{(pid=75234) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2279}}
Nov 29 03:21:40 user nova-compute[75234]: DEBUG nova.network.neutron [-] [instance: 75c5d700-6367-4128-9228-80d2cd24ace7] deallocate_for_instance() {{(pid=75234) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1802}}
Nov 29 03:21:41 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:21:41 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:21:41 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:21:41 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:21:42 user nova-compute[75234]: DEBUG nova.compute.manager [req-7b97a1fc-6814-441b-adec-7962b36d60e5 req-8a07ab2d-e9f2-4ee5-9eb8-2b8f6ba56e05 service nova] [instance: 75c5d700-6367-4128-9228-80d2cd24ace7] Received event network-vif-plugged-29c6743e-29e2-40a8-8de9-ec380f1e6f25 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:21:42 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-7b97a1fc-6814-441b-adec-7962b36d60e5 req-8a07ab2d-e9f2-4ee5-9eb8-2b8f6ba56e05 service nova] Acquiring lock "75c5d700-6367-4128-9228-80d2cd24ace7-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:21:42 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-7b97a1fc-6814-441b-adec-7962b36d60e5 req-8a07ab2d-e9f2-4ee5-9eb8-2b8f6ba56e05 service nova] Lock "75c5d700-6367-4128-9228-80d2cd24ace7-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:21:42 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-7b97a1fc-6814-441b-adec-7962b36d60e5 req-8a07ab2d-e9f2-4ee5-9eb8-2b8f6ba56e05 service nova] Lock "75c5d700-6367-4128-9228-80d2cd24ace7-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:21:42 user nova-compute[75234]: DEBUG nova.compute.manager [req-7b97a1fc-6814-441b-adec-7962b36d60e5 req-8a07ab2d-e9f2-4ee5-9eb8-2b8f6ba56e05 service nova] [instance: 75c5d700-6367-4128-9228-80d2cd24ace7] No waiting events found dispatching network-vif-plugged-29c6743e-29e2-40a8-8de9-ec380f1e6f25 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:21:42 user nova-compute[75234]: WARNING nova.compute.manager [req-7b97a1fc-6814-441b-adec-7962b36d60e5 req-8a07ab2d-e9f2-4ee5-9eb8-2b8f6ba56e05 service nova] [instance: 75c5d700-6367-4128-9228-80d2cd24ace7] Received unexpected event network-vif-plugged-29c6743e-29e2-40a8-8de9-ec380f1e6f25 for instance with vm_state active and task_state deleting.
Nov 29 03:21:42 user nova-compute[75234]: DEBUG nova.compute.manager [req-7b97a1fc-6814-441b-adec-7962b36d60e5 req-8a07ab2d-e9f2-4ee5-9eb8-2b8f6ba56e05 service nova] [instance: 75c5d700-6367-4128-9228-80d2cd24ace7] Received event network-vif-plugged-29c6743e-29e2-40a8-8de9-ec380f1e6f25 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:21:42 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-7b97a1fc-6814-441b-adec-7962b36d60e5 req-8a07ab2d-e9f2-4ee5-9eb8-2b8f6ba56e05 service nova] Acquiring lock "75c5d700-6367-4128-9228-80d2cd24ace7-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:21:42 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-7b97a1fc-6814-441b-adec-7962b36d60e5 req-8a07ab2d-e9f2-4ee5-9eb8-2b8f6ba56e05 service nova] Lock "75c5d700-6367-4128-9228-80d2cd24ace7-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:21:42 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-7b97a1fc-6814-441b-adec-7962b36d60e5 req-8a07ab2d-e9f2-4ee5-9eb8-2b8f6ba56e05 service nova] Lock "75c5d700-6367-4128-9228-80d2cd24ace7-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:21:42 user nova-compute[75234]: DEBUG nova.compute.manager [req-7b97a1fc-6814-441b-adec-7962b36d60e5 req-8a07ab2d-e9f2-4ee5-9eb8-2b8f6ba56e05 service nova] [instance: 75c5d700-6367-4128-9228-80d2cd24ace7] No waiting events found dispatching network-vif-plugged-29c6743e-29e2-40a8-8de9-ec380f1e6f25 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:21:42 user nova-compute[75234]: WARNING nova.compute.manager [req-7b97a1fc-6814-441b-adec-7962b36d60e5 req-8a07ab2d-e9f2-4ee5-9eb8-2b8f6ba56e05 service nova] [instance: 75c5d700-6367-4128-9228-80d2cd24ace7] Received unexpected event network-vif-plugged-29c6743e-29e2-40a8-8de9-ec380f1e6f25 for instance with vm_state active and task_state deleting.
Nov 29 03:21:42 user nova-compute[75234]: DEBUG nova.compute.manager [req-7b97a1fc-6814-441b-adec-7962b36d60e5 req-8a07ab2d-e9f2-4ee5-9eb8-2b8f6ba56e05 service nova] [instance: 75c5d700-6367-4128-9228-80d2cd24ace7] Received event network-vif-plugged-29c6743e-29e2-40a8-8de9-ec380f1e6f25 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:21:42 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-7b97a1fc-6814-441b-adec-7962b36d60e5 req-8a07ab2d-e9f2-4ee5-9eb8-2b8f6ba56e05 service nova] Acquiring lock "75c5d700-6367-4128-9228-80d2cd24ace7-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:21:42 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-7b97a1fc-6814-441b-adec-7962b36d60e5 req-8a07ab2d-e9f2-4ee5-9eb8-2b8f6ba56e05 service nova] Lock "75c5d700-6367-4128-9228-80d2cd24ace7-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:21:42 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-7b97a1fc-6814-441b-adec-7962b36d60e5 req-8a07ab2d-e9f2-4ee5-9eb8-2b8f6ba56e05 service nova] Lock "75c5d700-6367-4128-9228-80d2cd24ace7-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:21:42 user nova-compute[75234]: DEBUG nova.compute.manager [req-7b97a1fc-6814-441b-adec-7962b36d60e5 req-8a07ab2d-e9f2-4ee5-9eb8-2b8f6ba56e05 service nova] [instance: 75c5d700-6367-4128-9228-80d2cd24ace7] No waiting events found dispatching network-vif-plugged-29c6743e-29e2-40a8-8de9-ec380f1e6f25 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:21:42 user nova-compute[75234]: WARNING nova.compute.manager [req-7b97a1fc-6814-441b-adec-7962b36d60e5 req-8a07ab2d-e9f2-4ee5-9eb8-2b8f6ba56e05 service nova] [instance: 75c5d700-6367-4128-9228-80d2cd24ace7] Received unexpected event network-vif-plugged-29c6743e-29e2-40a8-8de9-ec380f1e6f25 for instance with vm_state active and task_state deleting.
Nov 29 03:21:42 user nova-compute[75234]: DEBUG nova.compute.manager [req-7b97a1fc-6814-441b-adec-7962b36d60e5 req-8a07ab2d-e9f2-4ee5-9eb8-2b8f6ba56e05 service nova] [instance: 75c5d700-6367-4128-9228-80d2cd24ace7] Received event network-vif-unplugged-29c6743e-29e2-40a8-8de9-ec380f1e6f25 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:21:42 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-7b97a1fc-6814-441b-adec-7962b36d60e5 req-8a07ab2d-e9f2-4ee5-9eb8-2b8f6ba56e05 service nova] Acquiring lock "75c5d700-6367-4128-9228-80d2cd24ace7-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:21:42 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-7b97a1fc-6814-441b-adec-7962b36d60e5 req-8a07ab2d-e9f2-4ee5-9eb8-2b8f6ba56e05 service nova] Lock "75c5d700-6367-4128-9228-80d2cd24ace7-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:21:42 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-7b97a1fc-6814-441b-adec-7962b36d60e5 req-8a07ab2d-e9f2-4ee5-9eb8-2b8f6ba56e05 service nova] Lock "75c5d700-6367-4128-9228-80d2cd24ace7-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:21:42 user nova-compute[75234]: DEBUG nova.compute.manager [req-7b97a1fc-6814-441b-adec-7962b36d60e5 req-8a07ab2d-e9f2-4ee5-9eb8-2b8f6ba56e05 service nova] [instance: 75c5d700-6367-4128-9228-80d2cd24ace7] No waiting events found dispatching network-vif-unplugged-29c6743e-29e2-40a8-8de9-ec380f1e6f25 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:21:42 user nova-compute[75234]: DEBUG nova.compute.manager [req-7b97a1fc-6814-441b-adec-7962b36d60e5 req-8a07ab2d-e9f2-4ee5-9eb8-2b8f6ba56e05 service nova] [instance: 75c5d700-6367-4128-9228-80d2cd24ace7] Received event network-vif-unplugged-29c6743e-29e2-40a8-8de9-ec380f1e6f25 for instance with task_state deleting. {{(pid=75234) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10890}}
Nov 29 03:21:42 user nova-compute[75234]: DEBUG nova.compute.manager [req-7b97a1fc-6814-441b-adec-7962b36d60e5 req-8a07ab2d-e9f2-4ee5-9eb8-2b8f6ba56e05 service nova] [instance: 75c5d700-6367-4128-9228-80d2cd24ace7] Received event network-vif-plugged-29c6743e-29e2-40a8-8de9-ec380f1e6f25 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:21:42 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-7b97a1fc-6814-441b-adec-7962b36d60e5 req-8a07ab2d-e9f2-4ee5-9eb8-2b8f6ba56e05 service nova] Acquiring lock "75c5d700-6367-4128-9228-80d2cd24ace7-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:21:42 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-7b97a1fc-6814-441b-adec-7962b36d60e5 req-8a07ab2d-e9f2-4ee5-9eb8-2b8f6ba56e05 service nova] Lock "75c5d700-6367-4128-9228-80d2cd24ace7-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:21:42 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-7b97a1fc-6814-441b-adec-7962b36d60e5 req-8a07ab2d-e9f2-4ee5-9eb8-2b8f6ba56e05 service nova] Lock "75c5d700-6367-4128-9228-80d2cd24ace7-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:21:42 user nova-compute[75234]: DEBUG nova.compute.manager [req-7b97a1fc-6814-441b-adec-7962b36d60e5 req-8a07ab2d-e9f2-4ee5-9eb8-2b8f6ba56e05 service nova] [instance: 75c5d700-6367-4128-9228-80d2cd24ace7] No waiting events found dispatching network-vif-plugged-29c6743e-29e2-40a8-8de9-ec380f1e6f25 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:21:42 user nova-compute[75234]: WARNING nova.compute.manager [req-7b97a1fc-6814-441b-adec-7962b36d60e5 req-8a07ab2d-e9f2-4ee5-9eb8-2b8f6ba56e05 service nova] [instance: 75c5d700-6367-4128-9228-80d2cd24ace7] Received unexpected event network-vif-plugged-29c6743e-29e2-40a8-8de9-ec380f1e6f25 for instance with vm_state active and task_state deleting.
Nov 29 03:21:42 user nova-compute[75234]: DEBUG nova.network.neutron [-] [instance: 75c5d700-6367-4128-9228-80d2cd24ace7] Updating instance_info_cache with network_info: [] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:21:42 user nova-compute[75234]: INFO nova.compute.manager [-] [instance: 75c5d700-6367-4128-9228-80d2cd24ace7] Took 1.48 seconds to deallocate network for instance.
Nov 29 03:21:42 user nova-compute[75234]: DEBUG nova.compute.manager [req-3d56ea82-08cd-43d6-abf4-fb81969d02f2 req-1fb1da7d-27ac-4e08-a297-d57ff83915ef service nova] [instance: 75c5d700-6367-4128-9228-80d2cd24ace7] Received event network-vif-deleted-29c6743e-29e2-40a8-8de9-ec380f1e6f25 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:21:42 user nova-compute[75234]: INFO nova.compute.manager [req-3d56ea82-08cd-43d6-abf4-fb81969d02f2 req-1fb1da7d-27ac-4e08-a297-d57ff83915ef service nova] [instance: 75c5d700-6367-4128-9228-80d2cd24ace7] Neutron deleted interface 29c6743e-29e2-40a8-8de9-ec380f1e6f25; detaching it from the instance and deleting it from the info cache
Nov 29 03:21:42 user nova-compute[75234]: DEBUG nova.network.neutron [req-3d56ea82-08cd-43d6-abf4-fb81969d02f2 req-1fb1da7d-27ac-4e08-a297-d57ff83915ef service nova] [instance: 75c5d700-6367-4128-9228-80d2cd24ace7] Updating instance_info_cache with network_info: [] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:21:42 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:21:42 user nova-compute[75234]: DEBUG nova.compute.manager [req-3d56ea82-08cd-43d6-abf4-fb81969d02f2 req-1fb1da7d-27ac-4e08-a297-d57ff83915ef service nova] [instance: 75c5d700-6367-4128-9228-80d2cd24ace7] Detach interface failed, port_id=29c6743e-29e2-40a8-8de9-ec380f1e6f25, reason: Instance 75c5d700-6367-4128-9228-80d2cd24ace7 could not be found. {{(pid=75234) _process_instance_vif_deleted_event /opt/stack/nova/nova/compute/manager.py:10946}}
Nov 29 03:21:42 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-80688ce8-5cdc-4381-bb7e-6b03a5509431 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:21:42 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-80688ce8-5cdc-4381-bb7e-6b03a5509431 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:21:42 user nova-compute[75234]: DEBUG nova.compute.provider_tree [None req-80688ce8-5cdc-4381-bb7e-6b03a5509431 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Inventory has not changed in ProviderTree for provider: 4d291220-7c37-4839-a0e3-b23c66502b74 {{(pid=75234) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Nov 29 03:21:42 user nova-compute[75234]: DEBUG nova.scheduler.client.report [None req-80688ce8-5cdc-4381-bb7e-6b03a5509431 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Inventory has not changed for provider 4d291220-7c37-4839-a0e3-b23c66502b74 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=75234) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Nov 29 03:21:42 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-80688ce8-5cdc-4381-bb7e-6b03a5509431 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.576s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:21:42 user nova-compute[75234]: INFO nova.scheduler.client.report [None req-80688ce8-5cdc-4381-bb7e-6b03a5509431 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Deleted allocations for instance 75c5d700-6367-4128-9228-80d2cd24ace7
Nov 29 03:21:42 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-80688ce8-5cdc-4381-bb7e-6b03a5509431 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Lock "75c5d700-6367-4128-9228-80d2cd24ace7" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 3.303s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:21:45 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:21:47 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:21:48 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-d2d0f5e8-4212-4ff8-9eb8-9f307567e392 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Acquiring lock "72ebd454-0275-4c3d-95d2-227ffb045a0c" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:21:48 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-d2d0f5e8-4212-4ff8-9eb8-9f307567e392 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Lock "72ebd454-0275-4c3d-95d2-227ffb045a0c" acquired by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:21:48 user nova-compute[75234]: INFO nova.compute.manager [None req-d2d0f5e8-4212-4ff8-9eb8-9f307567e392 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] [instance: 72ebd454-0275-4c3d-95d2-227ffb045a0c] Detaching volume 0e8591d9-0c7f-4575-9e91-7fd99d87327a
Nov 29 03:21:49 user nova-compute[75234]: INFO nova.virt.block_device [None req-d2d0f5e8-4212-4ff8-9eb8-9f307567e392 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] [instance: 72ebd454-0275-4c3d-95d2-227ffb045a0c] Attempting to driver detach volume 0e8591d9-0c7f-4575-9e91-7fd99d87327a from mountpoint /dev/vdb
Nov 29 03:21:49 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-d2d0f5e8-4212-4ff8-9eb8-9f307567e392 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Attempting to detach device vdb from instance 72ebd454-0275-4c3d-95d2-227ffb045a0c from the persistent domain config. {{(pid=75234) _detach_from_persistent /opt/stack/nova/nova/virt/libvirt/driver.py:2487}}
Nov 29 03:21:49 user nova-compute[75234]: DEBUG nova.virt.libvirt.guest [None req-d2d0f5e8-4212-4ff8-9eb8-9f307567e392 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] detach device xml: <disk type="block" device="disk">
Nov 29 03:21:49 user nova-compute[75234]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Nov 29 03:21:49 user nova-compute[75234]:   <source dev="/dev/sdc"/>
Nov 29 03:21:49 user nova-compute[75234]:   <target dev="vdb" bus="virtio"/>
Nov 29 03:21:49 user nova-compute[75234]:   <serial>0e8591d9-0c7f-4575-9e91-7fd99d87327a</serial>
Nov 29 03:21:49 user nova-compute[75234]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x07" function="0x0"/>
Nov 29 03:21:49 user nova-compute[75234]: </disk>
Nov 29 03:21:49 user nova-compute[75234]:  {{(pid=75234) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Nov 29 03:21:49 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-d2d0f5e8-4212-4ff8-9eb8-9f307567e392 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Successfully detached device vdb from instance 72ebd454-0275-4c3d-95d2-227ffb045a0c from the persistent domain config.
Nov 29 03:21:49 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-d2d0f5e8-4212-4ff8-9eb8-9f307567e392 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] (1/8): Attempting to detach device vdb with device alias virtio-disk1 from instance 72ebd454-0275-4c3d-95d2-227ffb045a0c from the live domain config. {{(pid=75234) _detach_from_live_with_retry /opt/stack/nova/nova/virt/libvirt/driver.py:2523}}
Nov 29 03:21:49 user nova-compute[75234]: DEBUG nova.virt.libvirt.guest [None req-d2d0f5e8-4212-4ff8-9eb8-9f307567e392 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] detach device xml: <disk type="block" device="disk">
Nov 29 03:21:49 user nova-compute[75234]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Nov 29 03:21:49 user nova-compute[75234]:   <source dev="/dev/sdc"/>
Nov 29 03:21:49 user nova-compute[75234]:   <target dev="vdb" bus="virtio"/>
Nov 29 03:21:49 user nova-compute[75234]:   <serial>0e8591d9-0c7f-4575-9e91-7fd99d87327a</serial>
Nov 29 03:21:49 user nova-compute[75234]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x07" function="0x0"/>
Nov 29 03:21:49 user nova-compute[75234]: </disk>
Nov 29 03:21:49 user nova-compute[75234]:  {{(pid=75234) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Nov 29 03:21:49 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-9e4b0d4f-a264-497a-bcbf-17cceda54c9b tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Acquiring lock "72ebd454-0275-4c3d-95d2-227ffb045a0c" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:21:49 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-d2d0f5e8-4212-4ff8-9eb8-9f307567e392 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Start waiting for the detach event from libvirt for device vdb with device alias virtio-disk1 for instance 72ebd454-0275-4c3d-95d2-227ffb045a0c {{(pid=75234) _detach_from_live_and_wait_for_event /opt/stack/nova/nova/virt/libvirt/driver.py:2599}}
Nov 29 03:21:50 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:21:52 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:21:53 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:21:55 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:21:57 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:21:59 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:21:59 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._cleanup_incomplete_migrations {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:21:59 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Cleaning up deleted instances with incomplete migration  {{(pid=75234) _cleanup_incomplete_migrations /opt/stack/nova/nova/compute/manager.py:11247}}
Nov 29 03:22:00 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:22:00 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:22:02 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:22:02 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:22:02 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Starting heal instance info cache {{(pid=75234) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9922}}
Nov 29 03:22:02 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Acquiring lock "refresh_cache-13e5da64-9adc-4479-81c7-d329e9b5f08f" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:22:02 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Acquired lock "refresh_cache-13e5da64-9adc-4479-81c7-d329e9b5f08f" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:22:02 user nova-compute[75234]: DEBUG nova.network.neutron [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] [instance: 13e5da64-9adc-4479-81c7-d329e9b5f08f] Forcefully refreshing network info cache for instance {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2003}}
Nov 29 03:22:02 user nova-compute[75234]: DEBUG nova.network.neutron [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] [instance: 13e5da64-9adc-4479-81c7-d329e9b5f08f] Updating instance_info_cache with network_info: [{"id": "2abf9e25-d1d9-4893-899e-912d41b7c38f", "address": "fa:16:3e:59:79:35", "network": {"id": "2b3c3140-f3b6-45da-8202-bb4d7f0dd130", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-2043420778-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "f8a7884867c944e1ba1024270e4902f3", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2abf9e25-d1", "ovs_interfaceid": "2abf9e25-d1d9-4893-899e-912d41b7c38f", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:22:03 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Releasing lock "refresh_cache-13e5da64-9adc-4479-81c7-d329e9b5f08f" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:22:03 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] [instance: 13e5da64-9adc-4479-81c7-d329e9b5f08f] Updated the network info_cache for instance {{(pid=75234) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9993}}
Nov 29 03:22:03 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager.update_available_resource {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:22:03 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:22:03 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:22:03 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:22:03 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Auditing locally available compute resources for user (node: user) {{(pid=75234) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:907}}
Nov 29 03:22:03 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] skipping disk /dev/sdb (vda) as it is a volume {{(pid=75234) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11213}}
Nov 29 03:22:03 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/13e5da64-9adc-4479-81c7-d329e9b5f08f/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:22:03 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/13e5da64-9adc-4479-81c7-d329e9b5f08f/disk --force-share --output=json" returned: 0 in 0.172s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:22:03 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/13e5da64-9adc-4479-81c7-d329e9b5f08f/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:22:03 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/13e5da64-9adc-4479-81c7-d329e9b5f08f/disk --force-share --output=json" returned: 0 in 0.207s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:22:03 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/99d6f85f-1b77-4390-8b61-908c98f584c0/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:22:03 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/99d6f85f-1b77-4390-8b61-908c98f584c0/disk --force-share --output=json" returned: 0 in 0.170s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:22:03 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/99d6f85f-1b77-4390-8b61-908c98f584c0/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:22:03 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/99d6f85f-1b77-4390-8b61-908c98f584c0/disk --force-share --output=json" returned: 0 in 0.172s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:22:03 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/72ebd454-0275-4c3d-95d2-227ffb045a0c/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:22:04 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/72ebd454-0275-4c3d-95d2-227ffb045a0c/disk --force-share --output=json" returned: 0 in 0.171s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:22:04 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/72ebd454-0275-4c3d-95d2-227ffb045a0c/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:22:04 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/72ebd454-0275-4c3d-95d2-227ffb045a0c/disk --force-share --output=json" returned: 0 in 0.167s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:22:04 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/1df140f1-fbd9-4b5d-b356-d08b7e908f64/disk.rescue --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:22:04 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/1df140f1-fbd9-4b5d-b356-d08b7e908f64/disk.rescue --force-share --output=json" returned: 0 in 0.192s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:22:04 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/1df140f1-fbd9-4b5d-b356-d08b7e908f64/disk.rescue --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:22:04 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/1df140f1-fbd9-4b5d-b356-d08b7e908f64/disk.rescue --force-share --output=json" returned: 0 in 0.178s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:22:04 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] skipping disk /opt/stack/data/nova/instances/1df140f1-fbd9-4b5d-b356-d08b7e908f64/disk (vdb) as it is a volume {{(pid=75234) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11213}}
Nov 29 03:22:04 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f6bc212d-85cd-479c-92ae-112c9f0101bc/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:22:04 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f6bc212d-85cd-479c-92ae-112c9f0101bc/disk --force-share --output=json" returned: 0 in 0.178s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:22:04 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f6bc212d-85cd-479c-92ae-112c9f0101bc/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:22:05 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f6bc212d-85cd-479c-92ae-112c9f0101bc/disk --force-share --output=json" returned: 0 in 0.202s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:22:05 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/65d48c93-b6da-4f51-9d5a-0676dda62a60/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:22:05 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/65d48c93-b6da-4f51-9d5a-0676dda62a60/disk --force-share --output=json" returned: 0 in 0.190s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:22:05 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/65d48c93-b6da-4f51-9d5a-0676dda62a60/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:22:05 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/65d48c93-b6da-4f51-9d5a-0676dda62a60/disk --force-share --output=json" returned: 0 in 0.174s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:22:05 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/946b919b-39ae-4d23-aefd-01952f4b40b6/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:22:05 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:22:05 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/946b919b-39ae-4d23-aefd-01952f4b40b6/disk --force-share --output=json" returned: 0 in 0.196s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:22:05 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/946b919b-39ae-4d23-aefd-01952f4b40b6/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:22:05 user nova-compute[75234]: DEBUG nova.compute.manager [req-b13ed118-229a-4055-83bf-d5e2381e0dbe req-f5893e34-18ba-4b55-b872-2a9f5bb5c604 service nova] [instance: 946b919b-39ae-4d23-aefd-01952f4b40b6] Received event network-changed-e1f71ce6-58f3-4665-afc5-f92d1b6801c5 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:22:05 user nova-compute[75234]: DEBUG nova.compute.manager [req-b13ed118-229a-4055-83bf-d5e2381e0dbe req-f5893e34-18ba-4b55-b872-2a9f5bb5c604 service nova] [instance: 946b919b-39ae-4d23-aefd-01952f4b40b6] Refreshing instance network info cache due to event network-changed-e1f71ce6-58f3-4665-afc5-f92d1b6801c5. {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11117}}
Nov 29 03:22:05 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-b13ed118-229a-4055-83bf-d5e2381e0dbe req-f5893e34-18ba-4b55-b872-2a9f5bb5c604 service nova] Acquiring lock "refresh_cache-946b919b-39ae-4d23-aefd-01952f4b40b6" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:22:05 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-b13ed118-229a-4055-83bf-d5e2381e0dbe req-f5893e34-18ba-4b55-b872-2a9f5bb5c604 service nova] Acquired lock "refresh_cache-946b919b-39ae-4d23-aefd-01952f4b40b6" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:22:05 user nova-compute[75234]: DEBUG nova.network.neutron [req-b13ed118-229a-4055-83bf-d5e2381e0dbe req-f5893e34-18ba-4b55-b872-2a9f5bb5c604 service nova] [instance: 946b919b-39ae-4d23-aefd-01952f4b40b6] Refreshing network info cache for port e1f71ce6-58f3-4665-afc5-f92d1b6801c5 {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2006}}
Nov 29 03:22:05 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/946b919b-39ae-4d23-aefd-01952f4b40b6/disk --force-share --output=json" returned: 0 in 0.196s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:22:05 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a082c96e-e384-40b6-bcc8-fc5e0e104669/disk.rescue --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:22:06 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a082c96e-e384-40b6-bcc8-fc5e0e104669/disk.rescue --force-share --output=json" returned: 0 in 0.198s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:22:06 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a082c96e-e384-40b6-bcc8-fc5e0e104669/disk.rescue --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:22:06 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a082c96e-e384-40b6-bcc8-fc5e0e104669/disk.rescue --force-share --output=json" returned: 0 in 0.195s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:22:06 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a082c96e-e384-40b6-bcc8-fc5e0e104669/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:22:06 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-7860c3b0-ce88-4704-9331-30d017f21c6d tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] Acquiring lock "interface-946b919b-39ae-4d23-aefd-01952f4b40b6-None" by "nova.compute.manager.ComputeManager.attach_interface.<locals>.do_attach_interface" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:22:06 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-7860c3b0-ce88-4704-9331-30d017f21c6d tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] Lock "interface-946b919b-39ae-4d23-aefd-01952f4b40b6-None" acquired by "nova.compute.manager.ComputeManager.attach_interface.<locals>.do_attach_interface" :: waited 0.003s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:22:06 user nova-compute[75234]: DEBUG nova.objects.instance [None req-7860c3b0-ce88-4704-9331-30d017f21c6d tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] Lazy-loading 'flavor' on Instance uuid 946b919b-39ae-4d23-aefd-01952f4b40b6 {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:22:06 user nova-compute[75234]: DEBUG nova.network.neutron [req-b13ed118-229a-4055-83bf-d5e2381e0dbe req-f5893e34-18ba-4b55-b872-2a9f5bb5c604 service nova] [instance: 946b919b-39ae-4d23-aefd-01952f4b40b6] Updated VIF entry in instance network info cache for port e1f71ce6-58f3-4665-afc5-f92d1b6801c5. {{(pid=75234) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3481}}
Nov 29 03:22:06 user nova-compute[75234]: DEBUG nova.network.neutron [req-b13ed118-229a-4055-83bf-d5e2381e0dbe req-f5893e34-18ba-4b55-b872-2a9f5bb5c604 service nova] [instance: 946b919b-39ae-4d23-aefd-01952f4b40b6] Updating instance_info_cache with network_info: [{"id": "e1f71ce6-58f3-4665-afc5-f92d1b6801c5", "address": "fa:16:3e:81:ca:ab", "network": {"id": "93ca21e4-138b-4f00-8d1d-b30d63dbe7aa", "bridge": "br-int", "label": "tempest-TaggedAttachmentsTest-1844288661-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.47", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "e28b5baf29b14aa4b2b98cb1b615705f", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape1f71ce6-58", "ovs_interfaceid": "e1f71ce6-58f3-4665-afc5-f92d1b6801c5", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:22:06 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-b13ed118-229a-4055-83bf-d5e2381e0dbe req-f5893e34-18ba-4b55-b872-2a9f5bb5c604 service nova] Releasing lock "refresh_cache-946b919b-39ae-4d23-aefd-01952f4b40b6" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:22:06 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a082c96e-e384-40b6-bcc8-fc5e0e104669/disk --force-share --output=json" returned: 0 in 0.205s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:22:06 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a082c96e-e384-40b6-bcc8-fc5e0e104669/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:22:06 user nova-compute[75234]: DEBUG nova.objects.instance [None req-7860c3b0-ce88-4704-9331-30d017f21c6d tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] Lazy-loading 'pci_requests' on Instance uuid 946b919b-39ae-4d23-aefd-01952f4b40b6 {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:22:06 user nova-compute[75234]: DEBUG nova.objects.base [None req-7860c3b0-ce88-4704-9331-30d017f21c6d tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] Object Instance<946b919b-39ae-4d23-aefd-01952f4b40b6> lazy-loaded attributes: flavor,pci_requests {{(pid=75234) wrapper /opt/stack/nova/nova/objects/base.py:126}}
Nov 29 03:22:06 user nova-compute[75234]: DEBUG nova.network.neutron [None req-7860c3b0-ce88-4704-9331-30d017f21c6d tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] [instance: 946b919b-39ae-4d23-aefd-01952f4b40b6] allocate_for_instance() {{(pid=75234) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1156}}
Nov 29 03:22:06 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a082c96e-e384-40b6-bcc8-fc5e0e104669/disk --force-share --output=json" returned: 0 in 0.167s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:22:06 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/21e9c6cd-c39f-41c0-8768-5dd64589c3f3/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:22:06 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/21e9c6cd-c39f-41c0-8768-5dd64589c3f3/disk --force-share --output=json" returned: 0 in 0.170s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:22:06 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/21e9c6cd-c39f-41c0-8768-5dd64589c3f3/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:22:06 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/21e9c6cd-c39f-41c0-8768-5dd64589c3f3/disk --force-share --output=json" returned: 0 in 0.185s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:22:07 user nova-compute[75234]: DEBUG nova.policy [None req-7860c3b0-ce88-4704-9331-30d017f21c6d tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '522c3c107614404fbe3af8c38fdd189e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e28b5baf29b14aa4b2b98cb1b615705f', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=75234) authorize /opt/stack/nova/nova/policy.py:203}}
Nov 29 03:22:07 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:22:07 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:22:07 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:22:07 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Hypervisor/Node resource view: name=user free_ram=7335MB free_disk=25.215957641601562GB free_vcpus=2 pci_devices=[{"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}] {{(pid=75234) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1106}}
Nov 29 03:22:07 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:22:07 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:22:07 user nova-compute[75234]: DEBUG nova.network.neutron [None req-7860c3b0-ce88-4704-9331-30d017f21c6d tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] [instance: 946b919b-39ae-4d23-aefd-01952f4b40b6] Successfully created port: 501886e8-de21-4108-86d4-d7a9138f8057 {{(pid=75234) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:548}}
Nov 29 03:22:07 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance 21e9c6cd-c39f-41c0-8768-5dd64589c3f3 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:22:07 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance 13e5da64-9adc-4479-81c7-d329e9b5f08f actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:22:07 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance a082c96e-e384-40b6-bcc8-fc5e0e104669 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:22:07 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance 1df140f1-fbd9-4b5d-b356-d08b7e908f64 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:22:07 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance 72ebd454-0275-4c3d-95d2-227ffb045a0c actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:22:07 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance af699bc2-9daa-4b5a-9b4a-ad92fe857d70 actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 2048, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:22:07 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance 946b919b-39ae-4d23-aefd-01952f4b40b6 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:22:07 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance 99d6f85f-1b77-4390-8b61-908c98f584c0 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:22:07 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance 65d48c93-b6da-4f51-9d5a-0676dda62a60 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:22:07 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance f6bc212d-85cd-479c-92ae-112c9f0101bc actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:22:07 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Total usable vcpus: 12, total allocated vcpus: 10 {{(pid=75234) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1129}}
Nov 29 03:22:07 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Final resource view: name=user phys_ram=16011MB used_ram=7168MB phys_disk=40GB used_disk=9GB total_vcpus=12 used_vcpus=10 pci_stats=[] {{(pid=75234) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1138}}
Nov 29 03:22:08 user nova-compute[75234]: DEBUG nova.compute.provider_tree [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Inventory has not changed in ProviderTree for provider: 4d291220-7c37-4839-a0e3-b23c66502b74 {{(pid=75234) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Nov 29 03:22:08 user nova-compute[75234]: DEBUG nova.scheduler.client.report [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Inventory has not changed for provider 4d291220-7c37-4839-a0e3-b23c66502b74 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=75234) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Nov 29 03:22:08 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Compute_service record updated for user:user {{(pid=75234) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:1067}}
Nov 29 03:22:08 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.692s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:22:08 user nova-compute[75234]: DEBUG nova.network.neutron [None req-7860c3b0-ce88-4704-9331-30d017f21c6d tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] [instance: 946b919b-39ae-4d23-aefd-01952f4b40b6] Successfully updated port: 501886e8-de21-4108-86d4-d7a9138f8057 {{(pid=75234) _update_port /opt/stack/nova/nova/network/neutron.py:586}}
Nov 29 03:22:08 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-7860c3b0-ce88-4704-9331-30d017f21c6d tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] Acquiring lock "refresh_cache-946b919b-39ae-4d23-aefd-01952f4b40b6" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:22:08 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-7860c3b0-ce88-4704-9331-30d017f21c6d tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] Acquired lock "refresh_cache-946b919b-39ae-4d23-aefd-01952f4b40b6" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:22:08 user nova-compute[75234]: DEBUG nova.network.neutron [None req-7860c3b0-ce88-4704-9331-30d017f21c6d tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] [instance: 946b919b-39ae-4d23-aefd-01952f4b40b6] Building network info cache for instance {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2009}}
Nov 29 03:22:08 user nova-compute[75234]: DEBUG nova.compute.manager [req-9fe5f931-56cd-42cf-818b-f22181f931cb req-ae3cae7d-899e-410b-b37d-6786d32ae5de service nova] [instance: 946b919b-39ae-4d23-aefd-01952f4b40b6] Received event network-changed-501886e8-de21-4108-86d4-d7a9138f8057 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:22:08 user nova-compute[75234]: DEBUG nova.compute.manager [req-9fe5f931-56cd-42cf-818b-f22181f931cb req-ae3cae7d-899e-410b-b37d-6786d32ae5de service nova] [instance: 946b919b-39ae-4d23-aefd-01952f4b40b6] Refreshing instance network info cache due to event network-changed-501886e8-de21-4108-86d4-d7a9138f8057. {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11117}}
Nov 29 03:22:08 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-9fe5f931-56cd-42cf-818b-f22181f931cb req-ae3cae7d-899e-410b-b37d-6786d32ae5de service nova] Acquiring lock "refresh_cache-946b919b-39ae-4d23-aefd-01952f4b40b6" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:22:08 user nova-compute[75234]: DEBUG nova.network.neutron [None req-7860c3b0-ce88-4704-9331-30d017f21c6d tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] [instance: 946b919b-39ae-4d23-aefd-01952f4b40b6] Updating instance_info_cache with network_info: [{"id": "e1f71ce6-58f3-4665-afc5-f92d1b6801c5", "address": "fa:16:3e:81:ca:ab", "network": {"id": "93ca21e4-138b-4f00-8d1d-b30d63dbe7aa", "bridge": "br-int", "label": "tempest-TaggedAttachmentsTest-1844288661-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.47", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "e28b5baf29b14aa4b2b98cb1b615705f", "mtu": null, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape1f71ce6-58", "ovs_interfaceid": "e1f71ce6-58f3-4665-afc5-f92d1b6801c5", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "501886e8-de21-4108-86d4-d7a9138f8057", "address": "fa:16:3e:d3:09:2e", "network": {"id": "ca5d0d87-cb5f-4287-8438-a8ecc7d3fd33", "bridge": "br-int", "label": "tempest-tagged-attachments-test-net-1833288811", "subnets": [{"cidr": "10.10.10.0/24", "dns": [], "gateway": {"address": "10.10.10.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.10.10.149", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.10.10.1"}}], "meta": {"injected": false, "tenant_id": "e28b5baf29b14aa4b2b98cb1b615705f", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap501886e8-de", "ovs_interfaceid": "501886e8-de21-4108-86d4-d7a9138f8057", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:22:08 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-7860c3b0-ce88-4704-9331-30d017f21c6d tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] Releasing lock "refresh_cache-946b919b-39ae-4d23-aefd-01952f4b40b6" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:22:08 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-9fe5f931-56cd-42cf-818b-f22181f931cb req-ae3cae7d-899e-410b-b37d-6786d32ae5de service nova] Acquired lock "refresh_cache-946b919b-39ae-4d23-aefd-01952f4b40b6" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:22:08 user nova-compute[75234]: DEBUG nova.network.neutron [req-9fe5f931-56cd-42cf-818b-f22181f931cb req-ae3cae7d-899e-410b-b37d-6786d32ae5de service nova] [instance: 946b919b-39ae-4d23-aefd-01952f4b40b6] Refreshing network info cache for port 501886e8-de21-4108-86d4-d7a9138f8057 {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2006}}
Nov 29 03:22:08 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-7860c3b0-ce88-4704-9331-30d017f21c6d tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='True',created_at=2023-11-29T03:20:14Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-1777937262',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-device-tagging-server-1777937262',id=35,image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBJrVo4ngcPAPF4oP6xidv+ykI7O7g9y5APrTr7iXhruQjDIx7zqy1/omzWf7G2p8xg1QAr8Js8NrVgD6XT2ZMw0fSSCLDKoKJk+X3O2rM5aB1leyuLU4v60YiI/3ZRg9Ew==',key_name='tempest-keypair-1887924020',keypairs=<?>,launch_index=0,launched_at=2023-11-29T03:20:21Z,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=<?>,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=1,progress=0,project_id='e28b5baf29b14aa4b2b98cb1b615705f',ramdisk_id='',reservation_id='r-bfdfrqjg',resources=<?>,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-TaggedAttachmentsTest-399278293',owner_user_name='tempest-TaggedAttachmentsTest-399278293-project-member'},tags=<?>,task_state=None,terminated_at=None,trusted_certs=<?>,updated_at=2023-11-29T03:20:22Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='522c3c107614404fbe3af8c38fdd189e',uuid=946b919b-39ae-4d23-aefd-01952f4b40b6,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "501886e8-de21-4108-86d4-d7a9138f8057", "address": "fa:16:3e:d3:09:2e", "network": {"id": "ca5d0d87-cb5f-4287-8438-a8ecc7d3fd33", "bridge": "br-int", "label": "tempest-tagged-attachments-test-net-1833288811", "subnets": [{"cidr": "10.10.10.0/24", "dns": [], "gateway": {"address": "10.10.10.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.10.10.149", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.10.10.1"}}], "meta": {"injected": false, "tenant_id": "e28b5baf29b14aa4b2b98cb1b615705f", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap501886e8-de", "ovs_interfaceid": "501886e8-de21-4108-86d4-d7a9138f8057", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Nov 29 03:22:08 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-7860c3b0-ce88-4704-9331-30d017f21c6d tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] Converting VIF {"id": "501886e8-de21-4108-86d4-d7a9138f8057", "address": "fa:16:3e:d3:09:2e", "network": {"id": "ca5d0d87-cb5f-4287-8438-a8ecc7d3fd33", "bridge": "br-int", "label": "tempest-tagged-attachments-test-net-1833288811", "subnets": [{"cidr": "10.10.10.0/24", "dns": [], "gateway": {"address": "10.10.10.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.10.10.149", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.10.10.1"}}], "meta": {"injected": false, "tenant_id": "e28b5baf29b14aa4b2b98cb1b615705f", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap501886e8-de", "ovs_interfaceid": "501886e8-de21-4108-86d4-d7a9138f8057", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:22:08 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-7860c3b0-ce88-4704-9331-30d017f21c6d tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:d3:09:2e,bridge_name='br-int',has_traffic_filtering=True,id=501886e8-de21-4108-86d4-d7a9138f8057,network=Network(ca5d0d87-cb5f-4287-8438-a8ecc7d3fd33),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap501886e8-de') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:22:08 user nova-compute[75234]: DEBUG os_vif [None req-7860c3b0-ce88-4704-9331-30d017f21c6d tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:d3:09:2e,bridge_name='br-int',has_traffic_filtering=True,id=501886e8-de21-4108-86d4-d7a9138f8057,network=Network(ca5d0d87-cb5f-4287-8438-a8ecc7d3fd33),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap501886e8-de') {{(pid=75234) plug /opt/stack/data/venv/lib/python3.10/site-packages/os_vif/__init__.py:76}}
Nov 29 03:22:08 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:22:08 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:22:08 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Nov 29 03:22:08 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:22:08 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbCreateCommand(_result=None, table=QoS, columns={'type': 'linux-noop', 'external_ids': {'id': '150e19c1-5c3d-52c9-b6fa-92e1ea1e1605', '_type': 'linux-noop'}}, row=False) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:22:08 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:22:08 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:22:08 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:22:08 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tap501886e8-de, may_exist=True, interface_attrs={}) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:22:08 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Port, record=tap501886e8-de, col_values=(('qos', UUID('419d6a7a-9b01-4beb-a772-5b69d26cf945')),), if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:22:08 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DbSetCommand(_result=None, table=Interface, record=tap501886e8-de, col_values=(('external_ids', {'iface-id': '501886e8-de21-4108-86d4-d7a9138f8057', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:d3:09:2e', 'vm-uuid': '946b919b-39ae-4d23-aefd-01952f4b40b6'}),), if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:22:08 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:22:08 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:22:08 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:22:08 user nova-compute[75234]: INFO os_vif [None req-7860c3b0-ce88-4704-9331-30d017f21c6d tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:d3:09:2e,bridge_name='br-int',has_traffic_filtering=True,id=501886e8-de21-4108-86d4-d7a9138f8057,network=Network(ca5d0d87-cb5f-4287-8438-a8ecc7d3fd33),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap501886e8-de')
Nov 29 03:22:08 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-7860c3b0-ce88-4704-9331-30d017f21c6d tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='True',created_at=2023-11-29T03:20:14Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-1777937262',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-device-tagging-server-1777937262',id=35,image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBJrVo4ngcPAPF4oP6xidv+ykI7O7g9y5APrTr7iXhruQjDIx7zqy1/omzWf7G2p8xg1QAr8Js8NrVgD6XT2ZMw0fSSCLDKoKJk+X3O2rM5aB1leyuLU4v60YiI/3ZRg9Ew==',key_name='tempest-keypair-1887924020',keypairs=<?>,launch_index=0,launched_at=2023-11-29T03:20:21Z,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=<?>,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=1,progress=0,project_id='e28b5baf29b14aa4b2b98cb1b615705f',ramdisk_id='',reservation_id='r-bfdfrqjg',resources=<?>,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-TaggedAttachmentsTest-399278293',owner_user_name='tempest-TaggedAttachmentsTest-399278293-project-member'},tags=<?>,task_state=None,terminated_at=None,trusted_certs=<?>,updated_at=2023-11-29T03:20:22Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='522c3c107614404fbe3af8c38fdd189e',uuid=946b919b-39ae-4d23-aefd-01952f4b40b6,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "501886e8-de21-4108-86d4-d7a9138f8057", "address": "fa:16:3e:d3:09:2e", "network": {"id": "ca5d0d87-cb5f-4287-8438-a8ecc7d3fd33", "bridge": "br-int", "label": "tempest-tagged-attachments-test-net-1833288811", "subnets": [{"cidr": "10.10.10.0/24", "dns": [], "gateway": {"address": "10.10.10.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.10.10.149", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.10.10.1"}}], "meta": {"injected": false, "tenant_id": "e28b5baf29b14aa4b2b98cb1b615705f", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap501886e8-de", "ovs_interfaceid": "501886e8-de21-4108-86d4-d7a9138f8057", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=75234) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Nov 29 03:22:08 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-7860c3b0-ce88-4704-9331-30d017f21c6d tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] Converting VIF {"id": "501886e8-de21-4108-86d4-d7a9138f8057", "address": "fa:16:3e:d3:09:2e", "network": {"id": "ca5d0d87-cb5f-4287-8438-a8ecc7d3fd33", "bridge": "br-int", "label": "tempest-tagged-attachments-test-net-1833288811", "subnets": [{"cidr": "10.10.10.0/24", "dns": [], "gateway": {"address": "10.10.10.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.10.10.149", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.10.10.1"}}], "meta": {"injected": false, "tenant_id": "e28b5baf29b14aa4b2b98cb1b615705f", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap501886e8-de", "ovs_interfaceid": "501886e8-de21-4108-86d4-d7a9138f8057", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:22:08 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-7860c3b0-ce88-4704-9331-30d017f21c6d tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:d3:09:2e,bridge_name='br-int',has_traffic_filtering=True,id=501886e8-de21-4108-86d4-d7a9138f8057,network=Network(ca5d0d87-cb5f-4287-8438-a8ecc7d3fd33),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap501886e8-de') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:22:08 user nova-compute[75234]: DEBUG nova.virt.libvirt.guest [None req-7860c3b0-ce88-4704-9331-30d017f21c6d tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] attach device xml: <interface type="ethernet">
Nov 29 03:22:08 user nova-compute[75234]:   <mac address="fa:16:3e:d3:09:2e"/>
Nov 29 03:22:08 user nova-compute[75234]:   <model type="virtio"/>
Nov 29 03:22:08 user nova-compute[75234]:   <mtu size="1442"/>
Nov 29 03:22:08 user nova-compute[75234]:   <target dev="tap501886e8-de"/>
Nov 29 03:22:08 user nova-compute[75234]: </interface>
Nov 29 03:22:08 user nova-compute[75234]:  {{(pid=75234) attach_device /opt/stack/nova/nova/virt/libvirt/guest.py:339}}
Nov 29 03:22:08 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:22:09 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:22:09 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:22:09 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:22:09 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._sync_scheduler_instance_info {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:22:09 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:22:09 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:22:09 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._sync_power_states {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:22:09 user nova-compute[75234]: DEBUG nova.compute.manager [req-759253a5-0a9b-48d8-a08c-2def2daea040 req-4f743633-d5ef-415b-83e4-6600a3603279 service nova] [instance: 946b919b-39ae-4d23-aefd-01952f4b40b6] Received event network-vif-plugged-501886e8-de21-4108-86d4-d7a9138f8057 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:22:09 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-759253a5-0a9b-48d8-a08c-2def2daea040 req-4f743633-d5ef-415b-83e4-6600a3603279 service nova] Acquiring lock "946b919b-39ae-4d23-aefd-01952f4b40b6-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:22:09 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-759253a5-0a9b-48d8-a08c-2def2daea040 req-4f743633-d5ef-415b-83e4-6600a3603279 service nova] Lock "946b919b-39ae-4d23-aefd-01952f4b40b6-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:22:09 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-759253a5-0a9b-48d8-a08c-2def2daea040 req-4f743633-d5ef-415b-83e4-6600a3603279 service nova] Lock "946b919b-39ae-4d23-aefd-01952f4b40b6-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:22:09 user nova-compute[75234]: DEBUG nova.compute.manager [req-759253a5-0a9b-48d8-a08c-2def2daea040 req-4f743633-d5ef-415b-83e4-6600a3603279 service nova] [instance: 946b919b-39ae-4d23-aefd-01952f4b40b6] No waiting events found dispatching network-vif-plugged-501886e8-de21-4108-86d4-d7a9138f8057 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:22:09 user nova-compute[75234]: WARNING nova.compute.manager [req-759253a5-0a9b-48d8-a08c-2def2daea040 req-4f743633-d5ef-415b-83e4-6600a3603279 service nova] [instance: 946b919b-39ae-4d23-aefd-01952f4b40b6] Received unexpected event network-vif-plugged-501886e8-de21-4108-86d4-d7a9138f8057 for instance with vm_state active and task_state None.
Nov 29 03:22:09 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Triggering sync for uuid 21e9c6cd-c39f-41c0-8768-5dd64589c3f3 {{(pid=75234) _sync_power_states /opt/stack/nova/nova/compute/manager.py:10332}}
Nov 29 03:22:09 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Triggering sync for uuid 13e5da64-9adc-4479-81c7-d329e9b5f08f {{(pid=75234) _sync_power_states /opt/stack/nova/nova/compute/manager.py:10332}}
Nov 29 03:22:09 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Triggering sync for uuid a082c96e-e384-40b6-bcc8-fc5e0e104669 {{(pid=75234) _sync_power_states /opt/stack/nova/nova/compute/manager.py:10332}}
Nov 29 03:22:09 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Triggering sync for uuid 99d6f85f-1b77-4390-8b61-908c98f584c0 {{(pid=75234) _sync_power_states /opt/stack/nova/nova/compute/manager.py:10332}}
Nov 29 03:22:09 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Triggering sync for uuid af699bc2-9daa-4b5a-9b4a-ad92fe857d70 {{(pid=75234) _sync_power_states /opt/stack/nova/nova/compute/manager.py:10332}}
Nov 29 03:22:09 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Triggering sync for uuid 1df140f1-fbd9-4b5d-b356-d08b7e908f64 {{(pid=75234) _sync_power_states /opt/stack/nova/nova/compute/manager.py:10332}}
Nov 29 03:22:09 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Triggering sync for uuid 72ebd454-0275-4c3d-95d2-227ffb045a0c {{(pid=75234) _sync_power_states /opt/stack/nova/nova/compute/manager.py:10332}}
Nov 29 03:22:09 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Triggering sync for uuid 946b919b-39ae-4d23-aefd-01952f4b40b6 {{(pid=75234) _sync_power_states /opt/stack/nova/nova/compute/manager.py:10332}}
Nov 29 03:22:09 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Triggering sync for uuid 65d48c93-b6da-4f51-9d5a-0676dda62a60 {{(pid=75234) _sync_power_states /opt/stack/nova/nova/compute/manager.py:10332}}
Nov 29 03:22:09 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Triggering sync for uuid f6bc212d-85cd-479c-92ae-112c9f0101bc {{(pid=75234) _sync_power_states /opt/stack/nova/nova/compute/manager.py:10332}}
Nov 29 03:22:09 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Acquiring lock "21e9c6cd-c39f-41c0-8768-5dd64589c3f3" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:22:09 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "21e9c6cd-c39f-41c0-8768-5dd64589c3f3" acquired by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:22:09 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Acquiring lock "13e5da64-9adc-4479-81c7-d329e9b5f08f" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:22:09 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "13e5da64-9adc-4479-81c7-d329e9b5f08f" acquired by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:22:09 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Acquiring lock "a082c96e-e384-40b6-bcc8-fc5e0e104669" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:22:09 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "a082c96e-e384-40b6-bcc8-fc5e0e104669" acquired by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:22:09 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Acquiring lock "99d6f85f-1b77-4390-8b61-908c98f584c0" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:22:09 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "99d6f85f-1b77-4390-8b61-908c98f584c0" acquired by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:22:09 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Acquiring lock "af699bc2-9daa-4b5a-9b4a-ad92fe857d70" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:22:09 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "af699bc2-9daa-4b5a-9b4a-ad92fe857d70" acquired by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: waited 0.003s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:22:09 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Acquiring lock "1df140f1-fbd9-4b5d-b356-d08b7e908f64" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:22:09 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "1df140f1-fbd9-4b5d-b356-d08b7e908f64" acquired by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:22:09 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Acquiring lock "72ebd454-0275-4c3d-95d2-227ffb045a0c" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:22:09 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Acquiring lock "946b919b-39ae-4d23-aefd-01952f4b40b6" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:22:09 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "946b919b-39ae-4d23-aefd-01952f4b40b6" acquired by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:22:09 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Acquiring lock "65d48c93-b6da-4f51-9d5a-0676dda62a60" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:22:09 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "65d48c93-b6da-4f51-9d5a-0676dda62a60" acquired by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:22:09 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Acquiring lock "f6bc212d-85cd-479c-92ae-112c9f0101bc" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:22:09 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "f6bc212d-85cd-479c-92ae-112c9f0101bc" acquired by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:22:09 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:22:09 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=75234) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10541}}
Nov 29 03:22:09 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "21e9c6cd-c39f-41c0-8768-5dd64589c3f3" "released" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: held 0.064s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:22:09 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "13e5da64-9adc-4479-81c7-d329e9b5f08f" "released" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: held 0.064s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:22:09 user nova-compute[75234]: DEBUG nova.network.neutron [req-9fe5f931-56cd-42cf-818b-f22181f931cb req-ae3cae7d-899e-410b-b37d-6786d32ae5de service nova] [instance: 946b919b-39ae-4d23-aefd-01952f4b40b6] Updated VIF entry in instance network info cache for port 501886e8-de21-4108-86d4-d7a9138f8057. {{(pid=75234) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3481}}
Nov 29 03:22:09 user nova-compute[75234]: DEBUG nova.network.neutron [req-9fe5f931-56cd-42cf-818b-f22181f931cb req-ae3cae7d-899e-410b-b37d-6786d32ae5de service nova] [instance: 946b919b-39ae-4d23-aefd-01952f4b40b6] Updating instance_info_cache with network_info: [{"id": "e1f71ce6-58f3-4665-afc5-f92d1b6801c5", "address": "fa:16:3e:81:ca:ab", "network": {"id": "93ca21e4-138b-4f00-8d1d-b30d63dbe7aa", "bridge": "br-int", "label": "tempest-TaggedAttachmentsTest-1844288661-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.47", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "e28b5baf29b14aa4b2b98cb1b615705f", "mtu": null, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape1f71ce6-58", "ovs_interfaceid": "e1f71ce6-58f3-4665-afc5-f92d1b6801c5", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}, {"id": "501886e8-de21-4108-86d4-d7a9138f8057", "address": "fa:16:3e:d3:09:2e", "network": {"id": "ca5d0d87-cb5f-4287-8438-a8ecc7d3fd33", "bridge": "br-int", "label": "tempest-tagged-attachments-test-net-1833288811", "subnets": [{"cidr": "10.10.10.0/24", "dns": [], "gateway": {"address": "10.10.10.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.10.10.149", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.10.10.1"}}], "meta": {"injected": false, "tenant_id": "e28b5baf29b14aa4b2b98cb1b615705f", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap501886e8-de", "ovs_interfaceid": "501886e8-de21-4108-86d4-d7a9138f8057", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:22:09 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "a082c96e-e384-40b6-bcc8-fc5e0e104669" "released" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: held 0.072s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:22:09 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "99d6f85f-1b77-4390-8b61-908c98f584c0" "released" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: held 0.092s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:22:09 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-7860c3b0-ce88-4704-9331-30d017f21c6d tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] No BDM found with device name vda, not building metadata. {{(pid=75234) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12089}}
Nov 29 03:22:09 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-7860c3b0-ce88-4704-9331-30d017f21c6d tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] No BDM found with device name hda, not building metadata. {{(pid=75234) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12089}}
Nov 29 03:22:09 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-7860c3b0-ce88-4704-9331-30d017f21c6d tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] No VIF found with MAC fa:16:3e:81:ca:ab, not building metadata {{(pid=75234) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Nov 29 03:22:09 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-9fe5f931-56cd-42cf-818b-f22181f931cb req-ae3cae7d-899e-410b-b37d-6786d32ae5de service nova] Releasing lock "refresh_cache-946b919b-39ae-4d23-aefd-01952f4b40b6" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:22:09 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "af699bc2-9daa-4b5a-9b4a-ad92fe857d70" "released" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: held 0.100s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:22:09 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "1df140f1-fbd9-4b5d-b356-d08b7e908f64" "released" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: held 0.101s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:22:09 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "65d48c93-b6da-4f51-9d5a-0676dda62a60" "released" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: held 0.103s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:22:09 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "f6bc212d-85cd-479c-92ae-112c9f0101bc" "released" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: held 0.104s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:22:09 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "946b919b-39ae-4d23-aefd-01952f4b40b6" "released" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: held 0.112s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:22:09 user nova-compute[75234]: DEBUG nova.virt.libvirt.guest [None req-7860c3b0-ce88-4704-9331-30d017f21c6d tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] set metadata xml: <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Nov 29 03:22:09 user nova-compute[75234]:   <nova:package version="0.0.0"/>
Nov 29 03:22:09 user nova-compute[75234]:   <nova:name>tempest-device-tagging-server-1777937262</nova:name>
Nov 29 03:22:09 user nova-compute[75234]:   <nova:creationTime>2023-11-29 03:22:09</nova:creationTime>
Nov 29 03:22:09 user nova-compute[75234]:   <nova:flavor name="m1.tiny">
Nov 29 03:22:09 user nova-compute[75234]:     <nova:memory>512</nova:memory>
Nov 29 03:22:09 user nova-compute[75234]:     <nova:disk>1</nova:disk>
Nov 29 03:22:09 user nova-compute[75234]:     <nova:swap>0</nova:swap>
Nov 29 03:22:09 user nova-compute[75234]:     <nova:ephemeral>0</nova:ephemeral>
Nov 29 03:22:09 user nova-compute[75234]:     <nova:vcpus>1</nova:vcpus>
Nov 29 03:22:09 user nova-compute[75234]:   </nova:flavor>
Nov 29 03:22:09 user nova-compute[75234]:   <nova:owner>
Nov 29 03:22:09 user nova-compute[75234]:     <nova:user uuid="522c3c107614404fbe3af8c38fdd189e">tempest-TaggedAttachmentsTest-399278293-project-member</nova:user>
Nov 29 03:22:09 user nova-compute[75234]:     <nova:project uuid="e28b5baf29b14aa4b2b98cb1b615705f">tempest-TaggedAttachmentsTest-399278293</nova:project>
Nov 29 03:22:09 user nova-compute[75234]:   </nova:owner>
Nov 29 03:22:09 user nova-compute[75234]:   <nova:root type="image" uuid="96c30ba7-9b73-4a30-8ba4-7138879573b7"/>
Nov 29 03:22:09 user nova-compute[75234]:   <nova:ports>
Nov 29 03:22:09 user nova-compute[75234]:     <nova:port uuid="e1f71ce6-58f3-4665-afc5-f92d1b6801c5">
Nov 29 03:22:09 user nova-compute[75234]:       <nova:ip type="fixed" address="10.0.0.4" ipVersion="4"/>
Nov 29 03:22:09 user nova-compute[75234]:     </nova:port>
Nov 29 03:22:09 user nova-compute[75234]:     <nova:port uuid="501886e8-de21-4108-86d4-d7a9138f8057">
Nov 29 03:22:09 user nova-compute[75234]:       <nova:ip type="fixed" address="10.10.10.149" ipVersion="4"/>
Nov 29 03:22:09 user nova-compute[75234]:     </nova:port>
Nov 29 03:22:09 user nova-compute[75234]:   </nova:ports>
Nov 29 03:22:09 user nova-compute[75234]: </nova:instance>
Nov 29 03:22:09 user nova-compute[75234]:  {{(pid=75234) set_metadata /opt/stack/nova/nova/virt/libvirt/guest.py:359}}
Nov 29 03:22:09 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-7860c3b0-ce88-4704-9331-30d017f21c6d tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] Lock "interface-946b919b-39ae-4d23-aefd-01952f4b40b6-None" "released" by "nova.compute.manager.ComputeManager.attach_interface.<locals>.do_attach_interface" :: held 3.178s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:22:09 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-d2d0f5e8-4212-4ff8-9eb8-9f307567e392 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Waiting for libvirt event about the detach of device vdb with device alias virtio-disk1 from instance 72ebd454-0275-4c3d-95d2-227ffb045a0c is timed out.
Nov 29 03:22:09 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-d2d0f5e8-4212-4ff8-9eb8-9f307567e392 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Successfully detached device vdb from instance 72ebd454-0275-4c3d-95d2-227ffb045a0c from the live domain config.
Nov 29 03:22:09 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-d2d0f5e8-4212-4ff8-9eb8-9f307567e392 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] [instance: 72ebd454-0275-4c3d-95d2-227ffb045a0c] calling os-brick to detach iSCSI Volume {{(pid=75234) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Nov 29 03:22:09 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-d2d0f5e8-4212-4ff8-9eb8-9f307567e392 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f8a5050c9a0>, {'target_iqn': 'iqn.2010-10.org.openstack:0e8591d9-0c7f-4575-9e91-7fd99d87327a', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'yfqNpnzW', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdc'}, None), 'kwargs': False}}" {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:176}}
Nov 29 03:22:09 user nova-compute[75234]: WARNING os_brick.initiator.connectors.base [None req-d2d0f5e8-4212-4ff8-9eb8-9f307567e392 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Nov 29 03:22:09 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-d2d0f5e8-4212-4ff8-9eb8-9f307567e392 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:68}}
Nov 29 03:22:09 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-d2d0f5e8-4212-4ff8-9eb8-9f307567e392 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.002s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:73}}
Nov 29 03:22:09 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:22:09 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.023s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:22:09 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[545b9355-1e67-48a6-a37c-e346d3c18f31]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:cf6ab229-533a-424e-bd67-1d54d50ad4be\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:0e8591d9-0c7f-4575-9e91-7fd99d87327a\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:41f08854-31dd-4172-a5cf-01b0e695c10f\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:22:09 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-d2d0f5e8-4212-4ff8-9eb8-9f307567e392 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS:
Nov 29 03:22:09 user nova-compute[75234]: No targets found.
Nov 29 03:22:09 user nova-compute[75234]: iSNS:
Nov 29 03:22:09 user nova-compute[75234]: No targets found.
Nov 29 03:22:09 user nova-compute[75234]: STATIC:
Nov 29 03:22:09 user nova-compute[75234]: Target: iqn.2010-10.org.openstack:cf6ab229-533a-424e-bd67-1d54d50ad4be
Nov 29 03:22:09 user nova-compute[75234]:         Portal: 172.16.0.220:3260,-1
Nov 29 03:22:09 user nova-compute[75234]:                 Iface Name: default
Nov 29 03:22:09 user nova-compute[75234]: Target: iqn.2010-10.org.openstack:0e8591d9-0c7f-4575-9e91-7fd99d87327a
Nov 29 03:22:09 user nova-compute[75234]:         Portal: 172.16.0.220:3260,-1
Nov 29 03:22:09 user nova-compute[75234]:                 Iface Name: default
Nov 29 03:22:09 user nova-compute[75234]: Target: iqn.2010-10.org.openstack:41f08854-31dd-4172-a5cf-01b0e695c10f
Nov 29 03:22:09 user nova-compute[75234]:         Portal: 172.16.0.220:3260,-1
Nov 29 03:22:09 user nova-compute[75234]:                 Iface Name: default
Nov 29 03:22:09 user nova-compute[75234]: FIRMWARE:
Nov 29 03:22:09 user nova-compute[75234]: No targets found.
Nov 29 03:22:09 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:22:09 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-d2d0f5e8-4212-4ff8-9eb8-9f307567e392 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Regex to get portals from discoverydb: ^SENDTARGETS:
Nov 29 03:22:09 user nova-compute[75234]: .*?^DiscoveryAddress: 172.16.0.220,3260.*?
Nov 29 03:22:09 user nova-compute[75234]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=75234) _get_discoverydb_portals /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:382}}
Nov 29 03:22:09 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-d2d0f5e8-4212-4ff8-9eb8-9f307567e392 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:0e8591d9-0c7f-4575-9e91-7fd99d87327a', 0)] {{(pid=75234) _get_connection_devices /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:830}}
Nov 29 03:22:09 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:22:09 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.021s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:22:09 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[bcd77052-383a-4233-bce1-1bde3208bdc5]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:0e8591d9-0c7f-4575-9e91-7fd99d87327a\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:41f08854-31dd-4172-a5cf-01b0e695c10f\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:cf6ab229-533a-424e-bd67-1d54d50ad4be\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:22:09 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:22:09 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.026s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:22:09 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[78cf490a-3be2-437d-96e2-2291ef088c48]: (4, ('tcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:cf6ab229-533a-424e-bd67-1d54d50ad4be (non-flash)\ntcp: [38] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0e8591d9-0c7f-4575-9e91-7fd99d87327a (non-flash)\ntcp: [39] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:41f08854-31dd-4172-a5cf-01b0e695c10f (non-flash)\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:22:09 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-d2d0f5e8-4212-4ff8-9eb8-9f307567e392 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:cf6ab229-533a-424e-bd67-1d54d50ad4be (non-flash)
Nov 29 03:22:09 user nova-compute[75234]: tcp: [38] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0e8591d9-0c7f-4575-9e91-7fd99d87327a (non-flash)
Nov 29 03:22:09 user nova-compute[75234]: tcp: [39] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:41f08854-31dd-4172-a5cf-01b0e695c10f (non-flash)
Nov 29 03:22:09 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:22:09 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-d2d0f5e8-4212-4ff8-9eb8-9f307567e392 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] iscsi session list stdout=tcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:cf6ab229-533a-424e-bd67-1d54d50ad4be (non-flash)
Nov 29 03:22:09 user nova-compute[75234]: tcp: [38] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:0e8591d9-0c7f-4575-9e91-7fd99d87327a (non-flash)
Nov 29 03:22:09 user nova-compute[75234]: tcp: [39] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:41f08854-31dd-4172-a5cf-01b0e695c10f (non-flash)
Nov 29 03:22:09 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsi_session /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Nov 29 03:22:09 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-d2d0f5e8-4212-4ff8-9eb8-9f307567e392 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7f8a506da8c0>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:0e8591d9-0c7f-4575-9e91-7fd99d87327a'): ({'sdc'}, set())}) {{(pid=75234) _get_connection_devices /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:863}}
Nov 29 03:22:09 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-d2d0f5e8-4212-4ff8-9eb8-9f307567e392 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Removing single pathed devices sdc {{(pid=75234) remove_connection /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:374}}
Nov 29 03:22:09 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:22:09 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.023s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:22:09 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[cc168502-5bbf-4b7a-b97b-5dd0597ab334]: (4, ('path checker states:\nup                  3\n\npaths: 0\nbusy: False\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:22:09 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdc {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:22:09 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdc" returned: 0 in 0.019s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:22:09 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[6315bdf4-1569-4517-932e-f9c944bc8fbd]: (4, ('ok\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:22:09 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-d2d0f5e8-4212-4ff8-9eb8-9f307567e392 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Flushing IO for device /dev/sdc {{(pid=75234) flush_device_io /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:434}}
Nov 29 03:22:09 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:22:09 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdc {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:22:09 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:22:09 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdc" returned: 0 in 0.017s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:22:09 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[af074767-0bc6-4eca-b6b2-8c94f39b8d05]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:22:09 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-d2d0f5e8-4212-4ff8-9eb8-9f307567e392 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Remove SCSI device /dev/sdc with /sys/block/sdc/device/delete {{(pid=75234) remove_scsi_device /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:148}}
Nov 29 03:22:09 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdc/device/delete {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:22:09 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:22:09 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdc/device/delete" returned: 0 in 0.051s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:22:09 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[70d5aedf-b44e-40d3-be55-d784f1f89a6a]: (4, ('1', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:22:09 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-d2d0f5e8-4212-4ff8-9eb8-9f307567e392 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Checking to see if SCSI volumes sdc have been removed. {{(pid=75234) wait_for_volumes_removal /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:156}}
Nov 29 03:22:09 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-d2d0f5e8-4212-4ff8-9eb8-9f307567e392 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] SCSI volumes sdc have been removed. {{(pid=75234) wait_for_volumes_removal /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:166}}
Nov 29 03:22:09 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-d2d0f5e8-4212-4ff8-9eb8-9f307567e392 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:0e8591d9-0c7f-4575-9e91-7fd99d87327a')] {{(pid=75234) _disconnect_connection /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Nov 29 03:22:09 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:0e8591d9-0c7f-4575-9e91-7fd99d87327a -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:22:09 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:0e8591d9-0c7f-4575-9e91-7fd99d87327a -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.022s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:22:09 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[3a7c1ff6-b9e4-46bc-9ce1-25ed4063d4f6]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:22:09 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-d2d0f5e8-4212-4ff8-9eb8-9f307567e392 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:22:09 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:0e8591d9-0c7f-4575-9e91-7fd99d87327a -p 172.16.0.220:3260 --logout {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:22:09 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-3d578e7c-1357-4c08-b159-cba7de6ec18c tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] Acquiring lock "946b919b-39ae-4d23-aefd-01952f4b40b6" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:22:09 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-3d578e7c-1357-4c08-b159-cba7de6ec18c tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] Lock "946b919b-39ae-4d23-aefd-01952f4b40b6" acquired by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:22:09 user nova-compute[75234]: DEBUG nova.objects.instance [None req-3d578e7c-1357-4c08-b159-cba7de6ec18c tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] Lazy-loading 'flavor' on Instance uuid 946b919b-39ae-4d23-aefd-01952f4b40b6 {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:22:09 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:0e8591d9-0c7f-4575-9e91-7fd99d87327a -p 172.16.0.220:3260 --logout" returned: 0 in 0.070s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:22:09 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[0a459cda-cd9f-4125-b97a-f79bd005e37c]: (4, ('Logging out of session [sid: 38, target: iqn.2010-10.org.openstack:0e8591d9-0c7f-4575-9e91-7fd99d87327a, portal: 172.16.0.220,3260]\nLogout of [sid: 38, target: iqn.2010-10.org.openstack:0e8591d9-0c7f-4575-9e91-7fd99d87327a, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:22:09 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-d2d0f5e8-4212-4ff8-9eb8-9f307567e392 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 38, target: iqn.2010-10.org.openstack:0e8591d9-0c7f-4575-9e91-7fd99d87327a, portal: 172.16.0.220,3260]
Nov 29 03:22:09 user nova-compute[75234]: Logout of [sid: 38, target: iqn.2010-10.org.openstack:0e8591d9-0c7f-4575-9e91-7fd99d87327a, portal: 172.16.0.220,3260] successful.
Nov 29 03:22:09 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:22:09 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:0e8591d9-0c7f-4575-9e91-7fd99d87327a -p 172.16.0.220:3260 --op delete {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:22:09 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-3d578e7c-1357-4c08-b159-cba7de6ec18c tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] Lock "946b919b-39ae-4d23-aefd-01952f4b40b6" "released" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: held 0.056s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:22:09 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:0e8591d9-0c7f-4575-9e91-7fd99d87327a -p 172.16.0.220:3260 --op delete" returned: 0 in 0.026s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:22:09 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[024933e2-4a8a-43fa-b1a3-2d6e758e36cd]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:22:09 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-d2d0f5e8-4212-4ff8-9eb8-9f307567e392 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:22:09 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-d2d0f5e8-4212-4ff8-9eb8-9f307567e392 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.362s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:87}}
Nov 29 03:22:09 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-d2d0f5e8-4212-4ff8-9eb8-9f307567e392 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] <== disconnect_volume: return (364ms) None {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:203}}
Nov 29 03:22:09 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-d2d0f5e8-4212-4ff8-9eb8-9f307567e392 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] [instance: 72ebd454-0275-4c3d-95d2-227ffb045a0c] Disconnected iSCSI Volume {{(pid=75234) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:79}}
Nov 29 03:22:10 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-3d578e7c-1357-4c08-b159-cba7de6ec18c tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] Acquiring lock "946b919b-39ae-4d23-aefd-01952f4b40b6" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:22:10 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-3d578e7c-1357-4c08-b159-cba7de6ec18c tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] Lock "946b919b-39ae-4d23-aefd-01952f4b40b6" acquired by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:22:10 user nova-compute[75234]: INFO nova.compute.manager [None req-3d578e7c-1357-4c08-b159-cba7de6ec18c tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] [instance: 946b919b-39ae-4d23-aefd-01952f4b40b6] Attaching volume cc40439e-08c7-41c0-824c-f2aa5fbabc5d to /dev/vdb
Nov 29 03:22:10 user nova-compute[75234]: DEBUG os_brick.utils [None req-3d578e7c-1357-4c08-b159-cba7de6ec18c tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:176}}
Nov 29 03:22:10 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:22:10 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.013s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:22:10 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[6bdb12a0-25de-4547-b8e1-ff46ec710825]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:dd1368c5ba2c\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:22:10 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt -v / -n -o SOURCE {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:22:10 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt -v / -n -o SOURCE" returned: 0 in 0.017s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:22:10 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[ea903434-8c45-4722-a15c-2e77faa49f36]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:22:10 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:22:10 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.016s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:22:10 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[2ee4ca26-4b5a-4f85-9b23-9938972965b1]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:22:10 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[c385340b-10ec-4c40-b430-b1589ea680d1]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:22:10 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-3d578e7c-1357-4c08-b159-cba7de6ec18c tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] Running cmd (subprocess): nvme version {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:22:10 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-3d578e7c-1357-4c08-b159-cba7de6ec18c tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] 'nvme version' failed. Not Retrying. {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:473}}
Nov 29 03:22:10 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.nvmeof [None req-3d578e7c-1357-4c08-b159-cba7de6ec18c tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] nvme not present on system {{(pid=75234) nvme_present /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/nvmeof.py:756}}
Nov 29 03:22:10 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:22:10 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:473}}
Nov 29 03:22:10 user nova-compute[75234]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Nov 29 03:22:10 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[da8fadcf-ff20-4d0f-b2a7-478d02a3031b]: (4, '') {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:22:10 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.lightos [None req-3d578e7c-1357-4c08-b159-cba7de6ec18c tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=75234) find_dsc /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/lightos.py:98}}
Nov 29 03:22:10 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.lightos [None req-3d578e7c-1357-4c08-b159-cba7de6ec18c tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=75234) get_connector_properties /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/lightos.py:76}}
Nov 29 03:22:10 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.lightos [None req-3d578e7c-1357-4c08-b159-cba7de6ec18c tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] LIGHTOS: no hostnqn found. {{(pid=75234) get_connector_properties /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/lightos.py:84}}
Nov 29 03:22:10 user nova-compute[75234]: DEBUG os_brick.utils [None req-3d578e7c-1357-4c08-b159-cba7de6ec18c tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] <== get_connector_properties: return (122ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:dd1368c5ba2c', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:203}}
Nov 29 03:22:10 user nova-compute[75234]: DEBUG nova.virt.block_device [None req-3d578e7c-1357-4c08-b159-cba7de6ec18c tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] [instance: 946b919b-39ae-4d23-aefd-01952f4b40b6] Updating existing volume attachment record: a1a88572-2517-46e4-8e1d-dbc9e24db233 {{(pid=75234) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Nov 29 03:22:10 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:22:10 user nova-compute[75234]: DEBUG nova.objects.instance [None req-d2d0f5e8-4212-4ff8-9eb8-9f307567e392 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Lazy-loading 'flavor' on Instance uuid 72ebd454-0275-4c3d-95d2-227ffb045a0c {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:22:10 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-d2d0f5e8-4212-4ff8-9eb8-9f307567e392 tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Lock "72ebd454-0275-4c3d-95d2-227ffb045a0c" "released" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: held 21.813s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:22:10 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-9e4b0d4f-a264-497a-bcbf-17cceda54c9b tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Lock "72ebd454-0275-4c3d-95d2-227ffb045a0c" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 21.659s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:22:10 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-9e4b0d4f-a264-497a-bcbf-17cceda54c9b tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Acquiring lock "72ebd454-0275-4c3d-95d2-227ffb045a0c-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:22:10 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-9e4b0d4f-a264-497a-bcbf-17cceda54c9b tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Lock "72ebd454-0275-4c3d-95d2-227ffb045a0c-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:22:10 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-9e4b0d4f-a264-497a-bcbf-17cceda54c9b tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Lock "72ebd454-0275-4c3d-95d2-227ffb045a0c-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:22:10 user nova-compute[75234]: INFO nova.compute.manager [None req-9e4b0d4f-a264-497a-bcbf-17cceda54c9b tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] [instance: 72ebd454-0275-4c3d-95d2-227ffb045a0c] Terminating instance
Nov 29 03:22:10 user nova-compute[75234]: DEBUG nova.compute.manager [None req-9e4b0d4f-a264-497a-bcbf-17cceda54c9b tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] [instance: 72ebd454-0275-4c3d-95d2-227ffb045a0c] Start destroying the instance on the hypervisor. {{(pid=75234) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3140}}
Nov 29 03:22:10 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:22:10 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:22:10 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:22:11 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:22:11 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:22:11 user nova-compute[75234]: DEBUG nova.compute.manager [req-54a18b54-19c3-499d-bc6a-0a2ab1ff482f req-017e7124-9c1d-4b66-857a-eeb2714e11c8 service nova] [instance: 72ebd454-0275-4c3d-95d2-227ffb045a0c] Received event network-vif-unplugged-7b066f7f-bc63-4a5f-8b1b-20e4611cf387 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:22:11 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-54a18b54-19c3-499d-bc6a-0a2ab1ff482f req-017e7124-9c1d-4b66-857a-eeb2714e11c8 service nova] Acquiring lock "72ebd454-0275-4c3d-95d2-227ffb045a0c-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:22:11 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-54a18b54-19c3-499d-bc6a-0a2ab1ff482f req-017e7124-9c1d-4b66-857a-eeb2714e11c8 service nova] Lock "72ebd454-0275-4c3d-95d2-227ffb045a0c-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:22:11 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-54a18b54-19c3-499d-bc6a-0a2ab1ff482f req-017e7124-9c1d-4b66-857a-eeb2714e11c8 service nova] Lock "72ebd454-0275-4c3d-95d2-227ffb045a0c-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:22:11 user nova-compute[75234]: DEBUG nova.compute.manager [req-54a18b54-19c3-499d-bc6a-0a2ab1ff482f req-017e7124-9c1d-4b66-857a-eeb2714e11c8 service nova] [instance: 72ebd454-0275-4c3d-95d2-227ffb045a0c] No waiting events found dispatching network-vif-unplugged-7b066f7f-bc63-4a5f-8b1b-20e4611cf387 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:22:11 user nova-compute[75234]: DEBUG nova.compute.manager [req-54a18b54-19c3-499d-bc6a-0a2ab1ff482f req-017e7124-9c1d-4b66-857a-eeb2714e11c8 service nova] [instance: 72ebd454-0275-4c3d-95d2-227ffb045a0c] Received event network-vif-unplugged-7b066f7f-bc63-4a5f-8b1b-20e4611cf387 for instance with task_state deleting. {{(pid=75234) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10890}}
Nov 29 03:22:11 user nova-compute[75234]: DEBUG nova.compute.manager [req-9771d900-2458-49f3-ba8a-b1cbdd26ab2c req-7fe15f80-ea0e-4057-b50b-59ed344efcc9 service nova] [instance: 946b919b-39ae-4d23-aefd-01952f4b40b6] Received event network-vif-plugged-501886e8-de21-4108-86d4-d7a9138f8057 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:22:11 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-9771d900-2458-49f3-ba8a-b1cbdd26ab2c req-7fe15f80-ea0e-4057-b50b-59ed344efcc9 service nova] Acquiring lock "946b919b-39ae-4d23-aefd-01952f4b40b6-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:22:11 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-9771d900-2458-49f3-ba8a-b1cbdd26ab2c req-7fe15f80-ea0e-4057-b50b-59ed344efcc9 service nova] Lock "946b919b-39ae-4d23-aefd-01952f4b40b6-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:22:11 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-9771d900-2458-49f3-ba8a-b1cbdd26ab2c req-7fe15f80-ea0e-4057-b50b-59ed344efcc9 service nova] Lock "946b919b-39ae-4d23-aefd-01952f4b40b6-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:22:11 user nova-compute[75234]: DEBUG nova.compute.manager [req-9771d900-2458-49f3-ba8a-b1cbdd26ab2c req-7fe15f80-ea0e-4057-b50b-59ed344efcc9 service nova] [instance: 946b919b-39ae-4d23-aefd-01952f4b40b6] No waiting events found dispatching network-vif-plugged-501886e8-de21-4108-86d4-d7a9138f8057 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:22:11 user nova-compute[75234]: WARNING nova.compute.manager [req-9771d900-2458-49f3-ba8a-b1cbdd26ab2c req-7fe15f80-ea0e-4057-b50b-59ed344efcc9 service nova] [instance: 946b919b-39ae-4d23-aefd-01952f4b40b6] Received unexpected event network-vif-plugged-501886e8-de21-4108-86d4-d7a9138f8057 for instance with vm_state active and task_state None.
Nov 29 03:22:11 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._cleanup_expired_console_auth_tokens {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:22:11 user nova-compute[75234]: INFO nova.virt.libvirt.driver [-] [instance: 72ebd454-0275-4c3d-95d2-227ffb045a0c] Instance destroyed successfully.
Nov 29 03:22:11 user nova-compute[75234]: DEBUG nova.objects.instance [None req-9e4b0d4f-a264-497a-bcbf-17cceda54c9b tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Lazy-loading 'resources' on Instance uuid 72ebd454-0275-4c3d-95d2-227ffb045a0c {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:22:11 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-9e4b0d4f-a264-497a-bcbf-17cceda54c9b tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-11-29T03:19:16Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-VolumesSnapshotTestJSON-instance-1321793627',display_name='tempest-VolumesSnapshotTestJSON-instance-1321793627',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-volumessnapshottestjson-instance-1321793627',id=34,image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBMhLPqL+EpbVwJ3QgH6H+iq2siyx+OzEETODSHhyLDZVvMNvMkCCiM9MM+DdJnAGXLmWBUOWlS++Zcp6L8MZ4+GmIYRjpSS0hgQLzUmSsnG7tldFPkINDXoEPZUO6xjMqw==',key_name='tempest-keypair-1810678573',keypairs=<?>,launch_index=0,launched_at=2023-11-29T03:19:24Z,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='7e24dd8accea4c1e9598d643ea79c86d',ramdisk_id='',reservation_id='r-me7x9yk0',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-VolumesSnapshotTestJSON-902320119',owner_user_name='tempest-VolumesSnapshotTestJSON-902320119-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-11-29T03:19:24Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='899f638d0de34dcba53f4c2ef7f5e068',uuid=72ebd454-0275-4c3d-95d2-227ffb045a0c,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "7b066f7f-bc63-4a5f-8b1b-20e4611cf387", "address": "fa:16:3e:52:a7:f8", "network": {"id": "1898605d-00e7-4e81-964e-821e09f5421c", "bridge": "br-int", "label": "tempest-VolumesSnapshotTestJSON-1896954656-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.25", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "7e24dd8accea4c1e9598d643ea79c86d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap7b066f7f-bc", "ovs_interfaceid": "7b066f7f-bc63-4a5f-8b1b-20e4611cf387", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Nov 29 03:22:11 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-9e4b0d4f-a264-497a-bcbf-17cceda54c9b tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Converting VIF {"id": "7b066f7f-bc63-4a5f-8b1b-20e4611cf387", "address": "fa:16:3e:52:a7:f8", "network": {"id": "1898605d-00e7-4e81-964e-821e09f5421c", "bridge": "br-int", "label": "tempest-VolumesSnapshotTestJSON-1896954656-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.25", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "7e24dd8accea4c1e9598d643ea79c86d", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap7b066f7f-bc", "ovs_interfaceid": "7b066f7f-bc63-4a5f-8b1b-20e4611cf387", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:22:11 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-9e4b0d4f-a264-497a-bcbf-17cceda54c9b tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:52:a7:f8,bridge_name='br-int',has_traffic_filtering=True,id=7b066f7f-bc63-4a5f-8b1b-20e4611cf387,network=Network(1898605d-00e7-4e81-964e-821e09f5421c),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap7b066f7f-bc') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:22:11 user nova-compute[75234]: DEBUG os_vif [None req-9e4b0d4f-a264-497a-bcbf-17cceda54c9b tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Unplugging vif VIFOpenVSwitch(active=True,address=fa:16:3e:52:a7:f8,bridge_name='br-int',has_traffic_filtering=True,id=7b066f7f-bc63-4a5f-8b1b-20e4611cf387,network=Network(1898605d-00e7-4e81-964e-821e09f5421c),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap7b066f7f-bc') {{(pid=75234) unplug /opt/stack/data/venv/lib/python3.10/site-packages/os_vif/__init__.py:109}}
Nov 29 03:22:11 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:22:11 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tap7b066f7f-bc, bridge=br-int, if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:22:11 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:22:11 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:22:11 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:22:11 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbDestroyCommand(_result=None, table=QoS, record=d2acd126-5481-4982-8788-302bf299aab8) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:22:11 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:22:11 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:22:11 user nova-compute[75234]: INFO os_vif [None req-9e4b0d4f-a264-497a-bcbf-17cceda54c9b tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Successfully unplugged vif VIFOpenVSwitch(active=True,address=fa:16:3e:52:a7:f8,bridge_name='br-int',has_traffic_filtering=True,id=7b066f7f-bc63-4a5f-8b1b-20e4611cf387,network=Network(1898605d-00e7-4e81-964e-821e09f5421c),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap7b066f7f-bc')
Nov 29 03:22:11 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-9e4b0d4f-a264-497a-bcbf-17cceda54c9b tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] [instance: 72ebd454-0275-4c3d-95d2-227ffb045a0c] calling os-brick to detach iSCSI Volume {{(pid=75234) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Nov 29 03:22:11 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-9e4b0d4f-a264-497a-bcbf-17cceda54c9b tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f8a5050c9a0>, {'target_iqn': 'iqn.2010-10.org.openstack:0e8591d9-0c7f-4575-9e91-7fd99d87327a', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'yfqNpnzW', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdc'}, None), 'kwargs': True}}" {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:176}}
Nov 29 03:22:11 user nova-compute[75234]: WARNING os_brick.initiator.connectors.base [None req-9e4b0d4f-a264-497a-bcbf-17cceda54c9b tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Nov 29 03:22:11 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-9e4b0d4f-a264-497a-bcbf-17cceda54c9b tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:68}}
Nov 29 03:22:11 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-9e4b0d4f-a264-497a-bcbf-17cceda54c9b tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:73}}
Nov 29 03:22:11 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:22:11 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.023s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:22:11 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[ba99510b-fdd8-4f2d-b6da-a08c3d7827aa]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:cf6ab229-533a-424e-bd67-1d54d50ad4be\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:41f08854-31dd-4172-a5cf-01b0e695c10f\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:22:11 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-9e4b0d4f-a264-497a-bcbf-17cceda54c9b tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS:
Nov 29 03:22:11 user nova-compute[75234]: No targets found.
Nov 29 03:22:11 user nova-compute[75234]: iSNS:
Nov 29 03:22:11 user nova-compute[75234]: No targets found.
Nov 29 03:22:11 user nova-compute[75234]: STATIC:
Nov 29 03:22:11 user nova-compute[75234]: Target: iqn.2010-10.org.openstack:cf6ab229-533a-424e-bd67-1d54d50ad4be
Nov 29 03:22:11 user nova-compute[75234]:         Portal: 172.16.0.220:3260,-1
Nov 29 03:22:11 user nova-compute[75234]:                 Iface Name: default
Nov 29 03:22:11 user nova-compute[75234]: Target: iqn.2010-10.org.openstack:41f08854-31dd-4172-a5cf-01b0e695c10f
Nov 29 03:22:11 user nova-compute[75234]:         Portal: 172.16.0.220:3260,-1
Nov 29 03:22:11 user nova-compute[75234]:                 Iface Name: default
Nov 29 03:22:11 user nova-compute[75234]: FIRMWARE:
Nov 29 03:22:11 user nova-compute[75234]: No targets found.
Nov 29 03:22:11 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:22:11 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-9e4b0d4f-a264-497a-bcbf-17cceda54c9b tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Regex to get portals from discoverydb: ^SENDTARGETS:
Nov 29 03:22:11 user nova-compute[75234]: .*?^DiscoveryAddress: 172.16.0.220,3260.*?
Nov 29 03:22:11 user nova-compute[75234]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=75234) _get_discoverydb_portals /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:382}}
Nov 29 03:22:11 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-9e4b0d4f-a264-497a-bcbf-17cceda54c9b tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:0e8591d9-0c7f-4575-9e91-7fd99d87327a', 0)] {{(pid=75234) _get_connection_devices /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:830}}
Nov 29 03:22:11 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:22:11 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.020s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:22:11 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[993db27c-934b-4c4d-a973-9e27217aba5d]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:41f08854-31dd-4172-a5cf-01b0e695c10f\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:cf6ab229-533a-424e-bd67-1d54d50ad4be\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:22:11 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:22:12 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.022s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:22:12 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[4de0a3d9-1021-40bf-9a6e-12a194e47897]: (4, ('tcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:cf6ab229-533a-424e-bd67-1d54d50ad4be (non-flash)\ntcp: [39] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:41f08854-31dd-4172-a5cf-01b0e695c10f (non-flash)\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:22:12 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-9e4b0d4f-a264-497a-bcbf-17cceda54c9b tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:cf6ab229-533a-424e-bd67-1d54d50ad4be (non-flash)
Nov 29 03:22:12 user nova-compute[75234]: tcp: [39] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:41f08854-31dd-4172-a5cf-01b0e695c10f (non-flash)
Nov 29 03:22:12 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:22:12 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-9e4b0d4f-a264-497a-bcbf-17cceda54c9b tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] iscsi session list stdout=tcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:cf6ab229-533a-424e-bd67-1d54d50ad4be (non-flash)
Nov 29 03:22:12 user nova-compute[75234]: tcp: [39] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:41f08854-31dd-4172-a5cf-01b0e695c10f (non-flash)
Nov 29 03:22:12 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsi_session /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Nov 29 03:22:12 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-9e4b0d4f-a264-497a-bcbf-17cceda54c9b tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7f8a2c711f30>, {}) {{(pid=75234) _get_connection_devices /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:863}}
Nov 29 03:22:12 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-9e4b0d4f-a264-497a-bcbf-17cceda54c9b tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Disconnecting from: [] {{(pid=75234) _disconnect_connection /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Nov 29 03:22:12 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-9e4b0d4f-a264-497a-bcbf-17cceda54c9b tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.079s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:87}}
Nov 29 03:22:12 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-9e4b0d4f-a264-497a-bcbf-17cceda54c9b tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] <== disconnect_volume: return (80ms) None {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:203}}
Nov 29 03:22:12 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-9e4b0d4f-a264-497a-bcbf-17cceda54c9b tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] [instance: 72ebd454-0275-4c3d-95d2-227ffb045a0c] Disconnected iSCSI Volume {{(pid=75234) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:79}}
Nov 29 03:22:12 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-9e4b0d4f-a264-497a-bcbf-17cceda54c9b tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] [instance: 72ebd454-0275-4c3d-95d2-227ffb045a0c] Deleting instance files /opt/stack/data/nova/instances/72ebd454-0275-4c3d-95d2-227ffb045a0c_del
Nov 29 03:22:12 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-9e4b0d4f-a264-497a-bcbf-17cceda54c9b tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] [instance: 72ebd454-0275-4c3d-95d2-227ffb045a0c] Deletion of /opt/stack/data/nova/instances/72ebd454-0275-4c3d-95d2-227ffb045a0c_del complete
Nov 29 03:22:12 user nova-compute[75234]: INFO nova.compute.manager [None req-9e4b0d4f-a264-497a-bcbf-17cceda54c9b tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] [instance: 72ebd454-0275-4c3d-95d2-227ffb045a0c] Took 1.32 seconds to destroy the instance on the hypervisor.
Nov 29 03:22:12 user nova-compute[75234]: DEBUG oslo.service.loopingcall [None req-9e4b0d4f-a264-497a-bcbf-17cceda54c9b tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=75234) func /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/loopingcall.py:435}}
Nov 29 03:22:12 user nova-compute[75234]: DEBUG nova.compute.manager [-] [instance: 72ebd454-0275-4c3d-95d2-227ffb045a0c] Deallocating network for instance {{(pid=75234) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2279}}
Nov 29 03:22:12 user nova-compute[75234]: DEBUG nova.network.neutron [-] [instance: 72ebd454-0275-4c3d-95d2-227ffb045a0c] deallocate_for_instance() {{(pid=75234) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1802}}
Nov 29 03:22:12 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:22:12 user nova-compute[75234]: DEBUG nova.network.neutron [-] [instance: 72ebd454-0275-4c3d-95d2-227ffb045a0c] Updating instance_info_cache with network_info: [] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:22:12 user nova-compute[75234]: INFO nova.compute.manager [-] [instance: 72ebd454-0275-4c3d-95d2-227ffb045a0c] Took 0.79 seconds to deallocate network for instance.
Nov 29 03:22:12 user nova-compute[75234]: DEBUG nova.compute.manager [req-85c12f3c-fe6f-49b7-becb-857535ef0713 req-1d8d1b2b-fe60-49da-b0c8-44bc77ae6768 service nova] [instance: 72ebd454-0275-4c3d-95d2-227ffb045a0c] Received event network-vif-deleted-7b066f7f-bc63-4a5f-8b1b-20e4611cf387 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:22:12 user nova-compute[75234]: INFO nova.compute.manager [req-85c12f3c-fe6f-49b7-becb-857535ef0713 req-1d8d1b2b-fe60-49da-b0c8-44bc77ae6768 service nova] [instance: 72ebd454-0275-4c3d-95d2-227ffb045a0c] Neutron deleted interface 7b066f7f-bc63-4a5f-8b1b-20e4611cf387; detaching it from the instance and deleting it from the info cache
Nov 29 03:22:12 user nova-compute[75234]: DEBUG nova.network.neutron [req-85c12f3c-fe6f-49b7-becb-857535ef0713 req-1d8d1b2b-fe60-49da-b0c8-44bc77ae6768 service nova] [instance: 72ebd454-0275-4c3d-95d2-227ffb045a0c] Updating instance_info_cache with network_info: [] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:22:12 user nova-compute[75234]: WARNING nova.volume.cinder [None req-9e4b0d4f-a264-497a-bcbf-17cceda54c9b tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Attachment a9af2839-0f6b-4f49-bee0-e3d296464cba does not exist. Ignoring.: cinderclient.exceptions.NotFound: Volume attachment could not be found with filter: attachment_id = a9af2839-0f6b-4f49-bee0-e3d296464cba. (HTTP 404) (Request-ID: req-78a45849-18ab-4c78-98a3-12bc9741c0c3)
Nov 29 03:22:12 user nova-compute[75234]: INFO nova.compute.manager [None req-9e4b0d4f-a264-497a-bcbf-17cceda54c9b tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] [instance: 72ebd454-0275-4c3d-95d2-227ffb045a0c] Took 0.03 seconds to detach 1 volumes for instance.
Nov 29 03:22:12 user nova-compute[75234]: DEBUG nova.compute.manager [req-85c12f3c-fe6f-49b7-becb-857535ef0713 req-1d8d1b2b-fe60-49da-b0c8-44bc77ae6768 service nova] [instance: 72ebd454-0275-4c3d-95d2-227ffb045a0c] Detach interface failed, port_id=7b066f7f-bc63-4a5f-8b1b-20e4611cf387, reason: Instance 72ebd454-0275-4c3d-95d2-227ffb045a0c could not be found. {{(pid=75234) _process_instance_vif_deleted_event /opt/stack/nova/nova/compute/manager.py:10946}}
Nov 29 03:22:12 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-9e4b0d4f-a264-497a-bcbf-17cceda54c9b tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:22:12 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-9e4b0d4f-a264-497a-bcbf-17cceda54c9b tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:22:13 user nova-compute[75234]: DEBUG nova.scheduler.client.report [None req-9e4b0d4f-a264-497a-bcbf-17cceda54c9b tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Refreshing inventories for resource provider 4d291220-7c37-4839-a0e3-b23c66502b74 {{(pid=75234) _refresh_associations /opt/stack/nova/nova/scheduler/client/report.py:804}}
Nov 29 03:22:13 user nova-compute[75234]: DEBUG nova.scheduler.client.report [None req-9e4b0d4f-a264-497a-bcbf-17cceda54c9b tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Updating ProviderTree inventory for provider 4d291220-7c37-4839-a0e3-b23c66502b74 from _refresh_and_get_inventory using data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=75234) _refresh_and_get_inventory /opt/stack/nova/nova/scheduler/client/report.py:768}}
Nov 29 03:22:13 user nova-compute[75234]: DEBUG nova.compute.provider_tree [None req-9e4b0d4f-a264-497a-bcbf-17cceda54c9b tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Updating inventory in ProviderTree for provider 4d291220-7c37-4839-a0e3-b23c66502b74 with inventory: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=75234) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:176}}
Nov 29 03:22:13 user nova-compute[75234]: DEBUG nova.scheduler.client.report [None req-9e4b0d4f-a264-497a-bcbf-17cceda54c9b tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Refreshing aggregate associations for resource provider 4d291220-7c37-4839-a0e3-b23c66502b74, aggregates: None {{(pid=75234) _refresh_associations /opt/stack/nova/nova/scheduler/client/report.py:813}}
Nov 29 03:22:13 user nova-compute[75234]: DEBUG nova.scheduler.client.report [None req-9e4b0d4f-a264-497a-bcbf-17cceda54c9b tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Refreshing trait associations for resource provider 4d291220-7c37-4839-a0e3-b23c66502b74, traits: COMPUTE_IMAGE_TYPE_AMI,COMPUTE_GRAPHICS_MODEL_CIRRUS,COMPUTE_NET_VIF_MODEL_VMXNET3,COMPUTE_STORAGE_BUS_VIRTIO,COMPUTE_GRAPHICS_MODEL_NONE,COMPUTE_IMAGE_TYPE_ISO,COMPUTE_NET_VIF_MODEL_RTL8139,HW_CPU_X86_SSE,COMPUTE_IMAGE_TYPE_QCOW2,COMPUTE_SOCKET_PCI_NUMA_AFFINITY,COMPUTE_GRAPHICS_MODEL_BOCHS,COMPUTE_VOLUME_ATTACH_WITH_TAG,COMPUTE_ACCELERATORS,HW_CPU_X86_SSE2,COMPUTE_STORAGE_BUS_SCSI,COMPUTE_GRAPHICS_MODEL_VGA,COMPUTE_NET_ATTACH_INTERFACE_WITH_TAG,COMPUTE_SECURITY_UEFI_SECURE_BOOT,HW_CPU_X86_SSE41,COMPUTE_NET_VIF_MODEL_E1000E,COMPUTE_TRUSTED_CERTS,COMPUTE_STORAGE_BUS_FDC,COMPUTE_VIOMMU_MODEL_INTEL,COMPUTE_GRAPHICS_MODEL_VIRTIO,COMPUTE_VIOMMU_MODEL_AUTO,COMPUTE_IMAGE_TYPE_AKI,COMPUTE_NET_VIF_MODEL_PCNET,COMPUTE_GRAPHICS_MODEL_QXL,COMPUTE_NET_VIF_MODEL_VIRTIO,COMPUTE_GRAPHICS_MODEL_VMVGA,COMPUTE_VOLUME_MULTI_ATTACH,COMPUTE_STORAGE_BUS_SATA,HW_CPU_X86_MMX,COMPUTE_NET_VIF_MODEL_E1000,COMPUTE_IMAGE_TYPE_ARI,COMPUTE_STORAGE_BUS_IDE,COMPUTE_NODE,HW_CPU_X86_SSSE3,COMPUTE_RESCUE_BFV,COMPUTE_NET_ATTACH_INTERFACE,COMPUTE_STORAGE_BUS_USB,COMPUTE_NET_VIF_MODEL_SPAPR_VLAN,COMPUTE_IMAGE_TYPE_RAW,COMPUTE_NET_VIF_MODEL_NE2K_PCI,COMPUTE_VOLUME_EXTEND,COMPUTE_DEVICE_TAGGING,HW_CPU_X86_SSE42 {{(pid=75234) _refresh_associations /opt/stack/nova/nova/scheduler/client/report.py:825}}
Nov 29 03:22:13 user nova-compute[75234]: DEBUG nova.compute.manager [req-6cb4013f-364f-447d-a097-15f120d41855 req-2315568f-fff5-44c0-a193-7c380cef2b3b service nova] [instance: 72ebd454-0275-4c3d-95d2-227ffb045a0c] Received event network-vif-plugged-7b066f7f-bc63-4a5f-8b1b-20e4611cf387 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:22:13 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-6cb4013f-364f-447d-a097-15f120d41855 req-2315568f-fff5-44c0-a193-7c380cef2b3b service nova] Acquiring lock "72ebd454-0275-4c3d-95d2-227ffb045a0c-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:22:13 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-6cb4013f-364f-447d-a097-15f120d41855 req-2315568f-fff5-44c0-a193-7c380cef2b3b service nova] Lock "72ebd454-0275-4c3d-95d2-227ffb045a0c-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:22:13 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-6cb4013f-364f-447d-a097-15f120d41855 req-2315568f-fff5-44c0-a193-7c380cef2b3b service nova] Lock "72ebd454-0275-4c3d-95d2-227ffb045a0c-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:22:13 user nova-compute[75234]: DEBUG nova.compute.manager [req-6cb4013f-364f-447d-a097-15f120d41855 req-2315568f-fff5-44c0-a193-7c380cef2b3b service nova] [instance: 72ebd454-0275-4c3d-95d2-227ffb045a0c] No waiting events found dispatching network-vif-plugged-7b066f7f-bc63-4a5f-8b1b-20e4611cf387 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:22:13 user nova-compute[75234]: WARNING nova.compute.manager [req-6cb4013f-364f-447d-a097-15f120d41855 req-2315568f-fff5-44c0-a193-7c380cef2b3b service nova] [instance: 72ebd454-0275-4c3d-95d2-227ffb045a0c] Received unexpected event network-vif-plugged-7b066f7f-bc63-4a5f-8b1b-20e4611cf387 for instance with vm_state deleted and task_state None.
Nov 29 03:22:13 user nova-compute[75234]: DEBUG nova.compute.provider_tree [None req-9e4b0d4f-a264-497a-bcbf-17cceda54c9b tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Inventory has not changed in ProviderTree for provider: 4d291220-7c37-4839-a0e3-b23c66502b74 {{(pid=75234) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Nov 29 03:22:13 user nova-compute[75234]: DEBUG nova.scheduler.client.report [None req-9e4b0d4f-a264-497a-bcbf-17cceda54c9b tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Inventory has not changed for provider 4d291220-7c37-4839-a0e3-b23c66502b74 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=75234) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Nov 29 03:22:13 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-9e4b0d4f-a264-497a-bcbf-17cceda54c9b tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.433s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:22:13 user nova-compute[75234]: INFO nova.scheduler.client.report [None req-9e4b0d4f-a264-497a-bcbf-17cceda54c9b tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Deleted allocations for instance 72ebd454-0275-4c3d-95d2-227ffb045a0c
Nov 29 03:22:13 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-9e4b0d4f-a264-497a-bcbf-17cceda54c9b tempest-VolumesSnapshotTestJSON-902320119 tempest-VolumesSnapshotTestJSON-902320119-project-member] Lock "72ebd454-0275-4c3d-95d2-227ffb045a0c" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 2.756s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:22:13 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "72ebd454-0275-4c3d-95d2-227ffb045a0c" acquired by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: waited 4.177s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:22:13 user nova-compute[75234]: INFO nova.compute.manager [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] [instance: 72ebd454-0275-4c3d-95d2-227ffb045a0c] During sync_power_state the instance has a pending task (deleting). Skip.
Nov 29 03:22:13 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "72ebd454-0275-4c3d-95d2-227ffb045a0c" "released" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: held 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:22:14 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-bc8a3be3-2120-47a1-bbd0-43a63bccfa71 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Acquiring lock "99d6f85f-1b77-4390-8b61-908c98f584c0" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:22:14 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-bc8a3be3-2120-47a1-bbd0-43a63bccfa71 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Lock "99d6f85f-1b77-4390-8b61-908c98f584c0" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:22:14 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-bc8a3be3-2120-47a1-bbd0-43a63bccfa71 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Acquiring lock "99d6f85f-1b77-4390-8b61-908c98f584c0-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:22:14 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-bc8a3be3-2120-47a1-bbd0-43a63bccfa71 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Lock "99d6f85f-1b77-4390-8b61-908c98f584c0-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:22:14 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-bc8a3be3-2120-47a1-bbd0-43a63bccfa71 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Lock "99d6f85f-1b77-4390-8b61-908c98f584c0-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:22:14 user nova-compute[75234]: INFO nova.compute.manager [None req-bc8a3be3-2120-47a1-bbd0-43a63bccfa71 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] [instance: 99d6f85f-1b77-4390-8b61-908c98f584c0] Terminating instance
Nov 29 03:22:14 user nova-compute[75234]: DEBUG nova.compute.manager [None req-bc8a3be3-2120-47a1-bbd0-43a63bccfa71 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] [instance: 99d6f85f-1b77-4390-8b61-908c98f584c0] Start destroying the instance on the hypervisor. {{(pid=75234) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3140}}
Nov 29 03:22:14 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:22:14 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:22:14 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:22:14 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:22:14 user nova-compute[75234]: DEBUG nova.compute.manager [req-a512d5da-e9a6-4117-aa7a-c59bf08b90d9 req-92bd5763-065f-4dec-823b-ad0257ceba59 service nova] [instance: 99d6f85f-1b77-4390-8b61-908c98f584c0] Received event network-vif-unplugged-b68d722f-9020-4da4-8cdb-e2312ef06df9 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:22:14 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-a512d5da-e9a6-4117-aa7a-c59bf08b90d9 req-92bd5763-065f-4dec-823b-ad0257ceba59 service nova] Acquiring lock "99d6f85f-1b77-4390-8b61-908c98f584c0-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:22:14 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-a512d5da-e9a6-4117-aa7a-c59bf08b90d9 req-92bd5763-065f-4dec-823b-ad0257ceba59 service nova] Lock "99d6f85f-1b77-4390-8b61-908c98f584c0-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:22:14 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-a512d5da-e9a6-4117-aa7a-c59bf08b90d9 req-92bd5763-065f-4dec-823b-ad0257ceba59 service nova] Lock "99d6f85f-1b77-4390-8b61-908c98f584c0-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:22:14 user nova-compute[75234]: DEBUG nova.compute.manager [req-a512d5da-e9a6-4117-aa7a-c59bf08b90d9 req-92bd5763-065f-4dec-823b-ad0257ceba59 service nova] [instance: 99d6f85f-1b77-4390-8b61-908c98f584c0] No waiting events found dispatching network-vif-unplugged-b68d722f-9020-4da4-8cdb-e2312ef06df9 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:22:14 user nova-compute[75234]: DEBUG nova.compute.manager [req-a512d5da-e9a6-4117-aa7a-c59bf08b90d9 req-92bd5763-065f-4dec-823b-ad0257ceba59 service nova] [instance: 99d6f85f-1b77-4390-8b61-908c98f584c0] Received event network-vif-unplugged-b68d722f-9020-4da4-8cdb-e2312ef06df9 for instance with task_state deleting. {{(pid=75234) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10890}}
Nov 29 03:22:14 user nova-compute[75234]: INFO nova.virt.libvirt.driver [-] [instance: 99d6f85f-1b77-4390-8b61-908c98f584c0] Instance destroyed successfully.
Nov 29 03:22:14 user nova-compute[75234]: DEBUG nova.objects.instance [None req-bc8a3be3-2120-47a1-bbd0-43a63bccfa71 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Lazy-loading 'resources' on Instance uuid 99d6f85f-1b77-4390-8b61-908c98f584c0 {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:22:14 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-bc8a3be3-2120-47a1-bbd0-43a63bccfa71 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=True,compute_id=1,config_drive='',created_at=2023-11-29T03:16:35Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description=None,display_name='tempest-AttachVolumeShelveTestJSON-server-491132081',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-attachvolumeshelvetestjson-server-491132081',id=28,image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBICHY8pkIfnX3FeZlRYJkWHQpVAhCpz3AE8GcQPc0ndKAzNj2za38XOUjeohx2bMy5FXksYmprL/THiMHoZRTInmzdW4YHaLMkQCBMth1H/mEdsilbqzkRPZKyCYZLp9Bw==',key_name='tempest-keypair-1848667272',keypairs=<?>,launch_index=0,launched_at=2023-11-29T03:20:35Z,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='e17150005a9a449f9373b91e1c00083b',ramdisk_id='',reservation_id='r-79uedf60',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',clean_attempts='1',image_base_image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-AttachVolumeShelveTestJSON-1180535246',owner_user_name='tempest-AttachVolumeShelveTestJSON-1180535246-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-11-29T03:20:35Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='154f3f4f57514450baf8f22d67768bc9',uuid=99d6f85f-1b77-4390-8b61-908c98f584c0,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "b68d722f-9020-4da4-8cdb-e2312ef06df9", "address": "fa:16:3e:1e:90:83", "network": {"id": "aad085f7-106b-4de6-93f5-f86bb66ec962", "bridge": "br-int", "label": "tempest-AttachVolumeShelveTestJSON-775984268-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.193", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "e17150005a9a449f9373b91e1c00083b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapb68d722f-90", "ovs_interfaceid": "b68d722f-9020-4da4-8cdb-e2312ef06df9", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Nov 29 03:22:14 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-bc8a3be3-2120-47a1-bbd0-43a63bccfa71 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Converting VIF {"id": "b68d722f-9020-4da4-8cdb-e2312ef06df9", "address": "fa:16:3e:1e:90:83", "network": {"id": "aad085f7-106b-4de6-93f5-f86bb66ec962", "bridge": "br-int", "label": "tempest-AttachVolumeShelveTestJSON-775984268-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.193", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "e17150005a9a449f9373b91e1c00083b", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapb68d722f-90", "ovs_interfaceid": "b68d722f-9020-4da4-8cdb-e2312ef06df9", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:22:14 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-bc8a3be3-2120-47a1-bbd0-43a63bccfa71 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:1e:90:83,bridge_name='br-int',has_traffic_filtering=True,id=b68d722f-9020-4da4-8cdb-e2312ef06df9,network=Network(aad085f7-106b-4de6-93f5-f86bb66ec962),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapb68d722f-90') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:22:14 user nova-compute[75234]: DEBUG os_vif [None req-bc8a3be3-2120-47a1-bbd0-43a63bccfa71 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Unplugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:1e:90:83,bridge_name='br-int',has_traffic_filtering=True,id=b68d722f-9020-4da4-8cdb-e2312ef06df9,network=Network(aad085f7-106b-4de6-93f5-f86bb66ec962),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapb68d722f-90') {{(pid=75234) unplug /opt/stack/data/venv/lib/python3.10/site-packages/os_vif/__init__.py:109}}
Nov 29 03:22:14 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:22:14 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tapb68d722f-90, bridge=br-int, if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:22:14 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:22:14 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:22:14 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:22:14 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbDestroyCommand(_result=None, table=QoS, record=465e9465-20d7-40f4-a194-907ed3dbd63a) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:22:14 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:22:14 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:22:14 user nova-compute[75234]: INFO os_vif [None req-bc8a3be3-2120-47a1-bbd0-43a63bccfa71 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Successfully unplugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:1e:90:83,bridge_name='br-int',has_traffic_filtering=True,id=b68d722f-9020-4da4-8cdb-e2312ef06df9,network=Network(aad085f7-106b-4de6-93f5-f86bb66ec962),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapb68d722f-90')
Nov 29 03:22:14 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-bc8a3be3-2120-47a1-bbd0-43a63bccfa71 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] [instance: 99d6f85f-1b77-4390-8b61-908c98f584c0] Deleting instance files /opt/stack/data/nova/instances/99d6f85f-1b77-4390-8b61-908c98f584c0_del
Nov 29 03:22:14 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-bc8a3be3-2120-47a1-bbd0-43a63bccfa71 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] [instance: 99d6f85f-1b77-4390-8b61-908c98f584c0] Deletion of /opt/stack/data/nova/instances/99d6f85f-1b77-4390-8b61-908c98f584c0_del complete
Nov 29 03:22:15 user nova-compute[75234]: INFO nova.compute.manager [None req-bc8a3be3-2120-47a1-bbd0-43a63bccfa71 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] [instance: 99d6f85f-1b77-4390-8b61-908c98f584c0] Took 0.68 seconds to destroy the instance on the hypervisor.
Nov 29 03:22:15 user nova-compute[75234]: DEBUG oslo.service.loopingcall [None req-bc8a3be3-2120-47a1-bbd0-43a63bccfa71 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=75234) func /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/loopingcall.py:435}}
Nov 29 03:22:15 user nova-compute[75234]: DEBUG nova.compute.manager [-] [instance: 99d6f85f-1b77-4390-8b61-908c98f584c0] Deallocating network for instance {{(pid=75234) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2279}}
Nov 29 03:22:15 user nova-compute[75234]: DEBUG nova.network.neutron [-] [instance: 99d6f85f-1b77-4390-8b61-908c98f584c0] deallocate_for_instance() {{(pid=75234) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1802}}
Nov 29 03:22:15 user nova-compute[75234]: DEBUG nova.network.neutron [-] [instance: 99d6f85f-1b77-4390-8b61-908c98f584c0] Updating instance_info_cache with network_info: [] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:22:15 user nova-compute[75234]: INFO nova.compute.manager [-] [instance: 99d6f85f-1b77-4390-8b61-908c98f584c0] Took 0.69 seconds to deallocate network for instance.
Nov 29 03:22:15 user nova-compute[75234]: DEBUG nova.compute.manager [req-26486473-ffa5-4e00-ac8c-711063a97785 req-53243da5-e11d-470e-99c4-13104efbc4b9 service nova] [instance: 99d6f85f-1b77-4390-8b61-908c98f584c0] Received event network-vif-deleted-b68d722f-9020-4da4-8cdb-e2312ef06df9 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:22:15 user nova-compute[75234]: INFO nova.compute.manager [req-26486473-ffa5-4e00-ac8c-711063a97785 req-53243da5-e11d-470e-99c4-13104efbc4b9 service nova] [instance: 99d6f85f-1b77-4390-8b61-908c98f584c0] Neutron deleted interface b68d722f-9020-4da4-8cdb-e2312ef06df9; detaching it from the instance and deleting it from the info cache
Nov 29 03:22:15 user nova-compute[75234]: DEBUG nova.network.neutron [req-26486473-ffa5-4e00-ac8c-711063a97785 req-53243da5-e11d-470e-99c4-13104efbc4b9 service nova] [instance: 99d6f85f-1b77-4390-8b61-908c98f584c0] Updating instance_info_cache with network_info: [] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:22:15 user nova-compute[75234]: DEBUG nova.compute.manager [req-26486473-ffa5-4e00-ac8c-711063a97785 req-53243da5-e11d-470e-99c4-13104efbc4b9 service nova] [instance: 99d6f85f-1b77-4390-8b61-908c98f584c0] Detach interface failed, port_id=b68d722f-9020-4da4-8cdb-e2312ef06df9, reason: Instance 99d6f85f-1b77-4390-8b61-908c98f584c0 could not be found. {{(pid=75234) _process_instance_vif_deleted_event /opt/stack/nova/nova/compute/manager.py:10946}}
Nov 29 03:22:15 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-bc8a3be3-2120-47a1-bbd0-43a63bccfa71 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:22:15 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-bc8a3be3-2120-47a1-bbd0-43a63bccfa71 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:22:16 user nova-compute[75234]: DEBUG nova.compute.provider_tree [None req-bc8a3be3-2120-47a1-bbd0-43a63bccfa71 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Inventory has not changed in ProviderTree for provider: 4d291220-7c37-4839-a0e3-b23c66502b74 {{(pid=75234) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Nov 29 03:22:16 user nova-compute[75234]: DEBUG nova.scheduler.client.report [None req-bc8a3be3-2120-47a1-bbd0-43a63bccfa71 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Inventory has not changed for provider 4d291220-7c37-4839-a0e3-b23c66502b74 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=75234) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Nov 29 03:22:16 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-bc8a3be3-2120-47a1-bbd0-43a63bccfa71 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.396s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:22:16 user nova-compute[75234]: INFO nova.scheduler.client.report [None req-bc8a3be3-2120-47a1-bbd0-43a63bccfa71 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Deleted allocations for instance 99d6f85f-1b77-4390-8b61-908c98f584c0
Nov 29 03:22:16 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-bc8a3be3-2120-47a1-bbd0-43a63bccfa71 tempest-AttachVolumeShelveTestJSON-1180535246 tempest-AttachVolumeShelveTestJSON-1180535246-project-member] Lock "99d6f85f-1b77-4390-8b61-908c98f584c0" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 2.083s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:22:16 user nova-compute[75234]: DEBUG nova.compute.manager [req-ba97e5ad-b0b2-45d3-a645-6244b7f2016b req-f865678b-98dc-4466-9a43-bafa68a2a00f service nova] [instance: 99d6f85f-1b77-4390-8b61-908c98f584c0] Received event network-vif-plugged-b68d722f-9020-4da4-8cdb-e2312ef06df9 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:22:16 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-ba97e5ad-b0b2-45d3-a645-6244b7f2016b req-f865678b-98dc-4466-9a43-bafa68a2a00f service nova] Acquiring lock "99d6f85f-1b77-4390-8b61-908c98f584c0-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:22:16 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-ba97e5ad-b0b2-45d3-a645-6244b7f2016b req-f865678b-98dc-4466-9a43-bafa68a2a00f service nova] Lock "99d6f85f-1b77-4390-8b61-908c98f584c0-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:22:16 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-ba97e5ad-b0b2-45d3-a645-6244b7f2016b req-f865678b-98dc-4466-9a43-bafa68a2a00f service nova] Lock "99d6f85f-1b77-4390-8b61-908c98f584c0-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:22:16 user nova-compute[75234]: DEBUG nova.compute.manager [req-ba97e5ad-b0b2-45d3-a645-6244b7f2016b req-f865678b-98dc-4466-9a43-bafa68a2a00f service nova] [instance: 99d6f85f-1b77-4390-8b61-908c98f584c0] No waiting events found dispatching network-vif-plugged-b68d722f-9020-4da4-8cdb-e2312ef06df9 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:22:16 user nova-compute[75234]: WARNING nova.compute.manager [req-ba97e5ad-b0b2-45d3-a645-6244b7f2016b req-f865678b-98dc-4466-9a43-bafa68a2a00f service nova] [instance: 99d6f85f-1b77-4390-8b61-908c98f584c0] Received unexpected event network-vif-plugged-b68d722f-9020-4da4-8cdb-e2312ef06df9 for instance with vm_state deleted and task_state None.
Nov 29 03:22:17 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:22:19 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:22:21 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-3d578e7c-1357-4c08-b159-cba7de6ec18c tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] [instance: 946b919b-39ae-4d23-aefd-01952f4b40b6] Calling os-brick to attach iSCSI Volume {{(pid=75234) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Nov 29 03:22:21 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-3d578e7c-1357-4c08-b159-cba7de6ec18c tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f8a5050c9a0>, 'connection_properties': {'target_iqn': 'iqn.2010-10.org.openstack:cc40439e-08c7-41c0-824c-f2aa5fbabc5d', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'JhmYCkeU', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:176}}
Nov 29 03:22:21 user nova-compute[75234]: WARNING os_brick.initiator.connectors.base [None req-3d578e7c-1357-4c08-b159-cba7de6ec18c tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Nov 29 03:22:21 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-3d578e7c-1357-4c08-b159-cba7de6ec18c tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:68}}
Nov 29 03:22:21 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-3d578e7c-1357-4c08-b159-cba7de6ec18c tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:73}}
Nov 29 03:22:21 user nova-compute[75234]: WARNING os_brick.initiator.connectors.base [None req-3d578e7c-1357-4c08-b159-cba7de6ec18c tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Nov 29 03:22:21 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-3d578e7c-1357-4c08-b159-cba7de6ec18c tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:cc40439e-08c7-41c0-824c-f2aa5fbabc5d" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:68}}
Nov 29 03:22:21 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-3d578e7c-1357-4c08-b159-cba7de6ec18c tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:cc40439e-08c7-41c0-824c-f2aa5fbabc5d" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:73}}
Nov 29 03:22:21 user nova-compute[75234]: INFO os_brick.initiator.connectors.iscsi [None req-3d578e7c-1357-4c08-b159-cba7de6ec18c tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Nov 29 03:22:21 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:cc40439e-08c7-41c0-824c-f2aa5fbabc5d -p 172.16.0.220:3260 {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:22:21 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:cc40439e-08c7-41c0-824c-f2aa5fbabc5d -p 172.16.0.220:3260" returned: 21 in 0.018s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:22:21 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[f89a0269-c01e-495c-9961-429fa10a08e7]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:22:21 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-3d578e7c-1357-4c08-b159-cba7de6ec18c tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Nov 29 03:22:21 user nova-compute[75234]:  {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:22:21 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:cc40439e-08c7-41c0-824c-f2aa5fbabc5d -p 172.16.0.220:3260 --interface default --op new {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:22:21 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:cc40439e-08c7-41c0-824c-f2aa5fbabc5d -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.020s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:22:21 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[8bfaf5e1-ea4e-47b4-a80b-86312ee46982]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:cc40439e-08c7-41c0-824c-f2aa5fbabc5d] added\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:22:21 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-3d578e7c-1357-4c08-b159-cba7de6ec18c tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:cc40439e-08c7-41c0-824c-f2aa5fbabc5d] added
Nov 29 03:22:21 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:22:21 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:cc40439e-08c7-41c0-824c-f2aa5fbabc5d -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:22:21 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:cc40439e-08c7-41c0-824c-f2aa5fbabc5d -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.037s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:22:21 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[56a1360c-9a5a-4177-a690-cd3e966b4e52]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:22:21 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-3d578e7c-1357-4c08-b159-cba7de6ec18c tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:22:21 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:cc40439e-08c7-41c0-824c-f2aa5fbabc5d -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:22:21 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:cc40439e-08c7-41c0-824c-f2aa5fbabc5d -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.018s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:22:21 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[7e03fd1c-cfde-4a01-86bc-40fa7832c982]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:22:21 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-3d578e7c-1357-4c08-b159-cba7de6ec18c tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:22:21 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:cc40439e-08c7-41c0-824c-f2aa5fbabc5d -p 172.16.0.220:3260 --op update -n node.session.auth.username -v JhmYCkeU {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:22:21 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:cc40439e-08c7-41c0-824c-f2aa5fbabc5d -p 172.16.0.220:3260 --op update -n node.session.auth.username -v JhmYCkeU" returned: 0 in 0.018s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:22:21 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[c9de5c44-ab8b-4b09-b87f-b1c69d7252e2]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:22:21 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-3d578e7c-1357-4c08-b159-cba7de6ec18c tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'JhmYCkeU'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:22:21 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:cc40439e-08c7-41c0-824c-f2aa5fbabc5d -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:22:21 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:cc40439e-08c7-41c0-824c-f2aa5fbabc5d -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.022s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:22:21 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[40c2df35-7177-4f1f-be11-0207755286bb]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:22:21 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-3d578e7c-1357-4c08-b159-cba7de6ec18c tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:22:21 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:22:21 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.021s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:22:21 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[11e810e4-dcbb-42b2-bbae-eea6962fa2c7]: (4, ('tcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:cf6ab229-533a-424e-bd67-1d54d50ad4be (non-flash)\ntcp: [39] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:41f08854-31dd-4172-a5cf-01b0e695c10f (non-flash)\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:22:21 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-3d578e7c-1357-4c08-b159-cba7de6ec18c tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:cf6ab229-533a-424e-bd67-1d54d50ad4be (non-flash)
Nov 29 03:22:21 user nova-compute[75234]: tcp: [39] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:41f08854-31dd-4172-a5cf-01b0e695c10f (non-flash)
Nov 29 03:22:21 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:22:21 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-3d578e7c-1357-4c08-b159-cba7de6ec18c tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] iscsi session list stdout=tcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:cf6ab229-533a-424e-bd67-1d54d50ad4be (non-flash)
Nov 29 03:22:21 user nova-compute[75234]: tcp: [39] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:41f08854-31dd-4172-a5cf-01b0e695c10f (non-flash)
Nov 29 03:22:21 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsi_session /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Nov 29 03:22:21 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:cc40439e-08c7-41c0-824c-f2aa5fbabc5d -p 172.16.0.220:3260 --login {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:22:21 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:cc40439e-08c7-41c0-824c-f2aa5fbabc5d -p 172.16.0.220:3260 --login" returned: 0 in 0.058s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:22:21 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[34144cf1-6d33-4eb1-bc29-fbb7c533cd1f]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:cc40439e-08c7-41c0-824c-f2aa5fbabc5d, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:cc40439e-08c7-41c0-824c-f2aa5fbabc5d, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:22:21 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-3d578e7c-1357-4c08-b159-cba7de6ec18c tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:cc40439e-08c7-41c0-824c-f2aa5fbabc5d, portal: 172.16.0.220,3260]
Nov 29 03:22:21 user nova-compute[75234]: Login to [iface: default, target: iqn.2010-10.org.openstack:cc40439e-08c7-41c0-824c-f2aa5fbabc5d, portal: 172.16.0.220,3260] successful.
Nov 29 03:22:21 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:22:21 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:cc40439e-08c7-41c0-824c-f2aa5fbabc5d -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:22:21 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:cc40439e-08c7-41c0-824c-f2aa5fbabc5d -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.023s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:22:21 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[4b534161-565c-4296-9c32-395ecbdf57de]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:22:21 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-3d578e7c-1357-4c08-b159-cba7de6ec18c tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:22:21 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:22:21 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.023s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:22:21 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[6bfd6d68-c20d-4be6-a608-dfa245155b9a]: (4, ('tcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:cf6ab229-533a-424e-bd67-1d54d50ad4be (non-flash)\ntcp: [39] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:41f08854-31dd-4172-a5cf-01b0e695c10f (non-flash)\ntcp: [40] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:cc40439e-08c7-41c0-824c-f2aa5fbabc5d (non-flash)\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:22:21 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-3d578e7c-1357-4c08-b159-cba7de6ec18c tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:cf6ab229-533a-424e-bd67-1d54d50ad4be (non-flash)
Nov 29 03:22:21 user nova-compute[75234]: tcp: [39] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:41f08854-31dd-4172-a5cf-01b0e695c10f (non-flash)
Nov 29 03:22:21 user nova-compute[75234]: tcp: [40] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:cc40439e-08c7-41c0-824c-f2aa5fbabc5d (non-flash)
Nov 29 03:22:21 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:22:21 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-3d578e7c-1357-4c08-b159-cba7de6ec18c tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] iscsi session list stdout=tcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:cf6ab229-533a-424e-bd67-1d54d50ad4be (non-flash)
Nov 29 03:22:21 user nova-compute[75234]: tcp: [39] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:41f08854-31dd-4172-a5cf-01b0e695c10f (non-flash)
Nov 29 03:22:21 user nova-compute[75234]: tcp: [40] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:cc40439e-08c7-41c0-824c-f2aa5fbabc5d (non-flash)
Nov 29 03:22:21 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsi_session /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Nov 29 03:22:21 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-3d578e7c-1357-4c08-b159-cba7de6ec18c tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:cc40439e-08c7-41c0-824c-f2aa5fbabc5d" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.294s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:87}}
Nov 29 03:22:21 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-3d578e7c-1357-4c08-b159-cba7de6ec18c tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] Connected to 172.16.0.220:3260 {{(pid=75234) _connect_vol /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:644}}
Nov 29 03:22:21 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-3d578e7c-1357-4c08-b159-cba7de6ec18c tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] HCTL ('34', '-', '-', 0) found on session 40 with lun 0 {{(pid=75234) get_hctl /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:770}}
Nov 29 03:22:21 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-3d578e7c-1357-4c08-b159-cba7de6ec18c tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] Scanning host 34 c: -, t: -, l: 0) {{(pid=75234) scan_iscsi /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:795}}
Nov 29 03:22:21 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host34/scan {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:22:21 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host34/scan" returned: 0 in 0.017s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:22:21 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[835d61f6-364e-424b-bc42-b64eb6dc62b7]: (4, ('- - 0', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:22:21 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-3d578e7c-1357-4c08-b159-cba7de6ec18c tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] Searching for a device in session 40 and hctl ['34', '*', '*', 0] yield: sdc {{(pid=75234) device_name_by_hctl /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:789}}
Nov 29 03:22:21 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-3d578e7c-1357-4c08-b159-cba7de6ec18c tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] Connected to sdc using {'target_iqn': 'iqn.2010-10.org.openstack:cc40439e-08c7-41c0-824c-f2aa5fbabc5d', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'JhmYCkeU', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False} {{(pid=75234) _connect_vol /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:672}}
Nov 29 03:22:22 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:22:22 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-3d578e7c-1357-4c08-b159-cba7de6ec18c tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.328s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:87}}
Nov 29 03:22:22 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-3d578e7c-1357-4c08-b159-cba7de6ec18c tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] <== connect_volume: return (1330ms) {'type': 'block', 'scsi_wwn': '23661653734316531', 'path': '/dev/sdc'} {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:203}}
Nov 29 03:22:22 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-3d578e7c-1357-4c08-b159-cba7de6ec18c tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] [instance: 946b919b-39ae-4d23-aefd-01952f4b40b6] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '23661653734316531', 'path': '/dev/sdc'} {{(pid=75234) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Nov 29 03:22:22 user nova-compute[75234]: DEBUG nova.objects.instance [None req-3d578e7c-1357-4c08-b159-cba7de6ec18c tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] Lazy-loading 'flavor' on Instance uuid 946b919b-39ae-4d23-aefd-01952f4b40b6 {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:22:22 user nova-compute[75234]: DEBUG nova.virt.libvirt.guest [None req-3d578e7c-1357-4c08-b159-cba7de6ec18c tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] attach device xml: <disk type="block" device="disk">
Nov 29 03:22:22 user nova-compute[75234]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Nov 29 03:22:22 user nova-compute[75234]:   <source dev="/dev/sdc"/>
Nov 29 03:22:22 user nova-compute[75234]:   <target dev="vdb" bus="virtio"/>
Nov 29 03:22:22 user nova-compute[75234]:   <serial>cc40439e-08c7-41c0-824c-f2aa5fbabc5d</serial>
Nov 29 03:22:22 user nova-compute[75234]: </disk>
Nov 29 03:22:22 user nova-compute[75234]:  {{(pid=75234) attach_device /opt/stack/nova/nova/virt/libvirt/guest.py:339}}
Nov 29 03:22:23 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-3d578e7c-1357-4c08-b159-cba7de6ec18c tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] No BDM found with device name vda, not building metadata. {{(pid=75234) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12089}}
Nov 29 03:22:23 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-3d578e7c-1357-4c08-b159-cba7de6ec18c tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] No BDM found with device name hda, not building metadata. {{(pid=75234) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12089}}
Nov 29 03:22:23 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-3d578e7c-1357-4c08-b159-cba7de6ec18c tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] No VIF found with MAC fa:16:3e:81:ca:ab, not building metadata {{(pid=75234) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Nov 29 03:22:23 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-3d578e7c-1357-4c08-b159-cba7de6ec18c tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] Lock "946b919b-39ae-4d23-aefd-01952f4b40b6" "released" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: held 13.286s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:22:24 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._run_pending_deletes {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:22:24 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Cleaning up deleted instances {{(pid=75234) _run_pending_deletes /opt/stack/nova/nova/compute/manager.py:11209}}
Nov 29 03:22:24 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] There are 0 instances to clean {{(pid=75234) _run_pending_deletes /opt/stack/nova/nova/compute/manager.py:11218}}
Nov 29 03:22:24 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-32ae734d-32d1-4039-a184-5136d17975de tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] Acquiring lock "946b919b-39ae-4d23-aefd-01952f4b40b6" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:22:24 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-32ae734d-32d1-4039-a184-5136d17975de tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] Lock "946b919b-39ae-4d23-aefd-01952f4b40b6" acquired by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:22:24 user nova-compute[75234]: INFO nova.compute.manager [None req-32ae734d-32d1-4039-a184-5136d17975de tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] [instance: 946b919b-39ae-4d23-aefd-01952f4b40b6] Detaching volume cc40439e-08c7-41c0-824c-f2aa5fbabc5d
Nov 29 03:22:24 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:22:24 user nova-compute[75234]: INFO nova.virt.block_device [None req-32ae734d-32d1-4039-a184-5136d17975de tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] [instance: 946b919b-39ae-4d23-aefd-01952f4b40b6] Attempting to driver detach volume cc40439e-08c7-41c0-824c-f2aa5fbabc5d from mountpoint /dev/vdb
Nov 29 03:22:24 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-32ae734d-32d1-4039-a184-5136d17975de tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] Attempting to detach device vdb from instance 946b919b-39ae-4d23-aefd-01952f4b40b6 from the persistent domain config. {{(pid=75234) _detach_from_persistent /opt/stack/nova/nova/virt/libvirt/driver.py:2487}}
Nov 29 03:22:24 user nova-compute[75234]: DEBUG nova.virt.libvirt.guest [None req-32ae734d-32d1-4039-a184-5136d17975de tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] detach device xml: <disk type="block" device="disk">
Nov 29 03:22:24 user nova-compute[75234]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Nov 29 03:22:24 user nova-compute[75234]:   <source dev="/dev/sdc"/>
Nov 29 03:22:24 user nova-compute[75234]:   <target dev="vdb" bus="virtio"/>
Nov 29 03:22:24 user nova-compute[75234]:   <serial>cc40439e-08c7-41c0-824c-f2aa5fbabc5d</serial>
Nov 29 03:22:24 user nova-compute[75234]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x08" function="0x0"/>
Nov 29 03:22:24 user nova-compute[75234]: </disk>
Nov 29 03:22:24 user nova-compute[75234]:  {{(pid=75234) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Nov 29 03:22:24 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-32ae734d-32d1-4039-a184-5136d17975de tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] Successfully detached device vdb from instance 946b919b-39ae-4d23-aefd-01952f4b40b6 from the persistent domain config.
Nov 29 03:22:24 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-32ae734d-32d1-4039-a184-5136d17975de tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] (1/8): Attempting to detach device vdb with device alias virtio-disk1 from instance 946b919b-39ae-4d23-aefd-01952f4b40b6 from the live domain config. {{(pid=75234) _detach_from_live_with_retry /opt/stack/nova/nova/virt/libvirt/driver.py:2523}}
Nov 29 03:22:24 user nova-compute[75234]: DEBUG nova.virt.libvirt.guest [None req-32ae734d-32d1-4039-a184-5136d17975de tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] detach device xml: <disk type="block" device="disk">
Nov 29 03:22:24 user nova-compute[75234]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Nov 29 03:22:24 user nova-compute[75234]:   <source dev="/dev/sdc"/>
Nov 29 03:22:24 user nova-compute[75234]:   <target dev="vdb" bus="virtio"/>
Nov 29 03:22:24 user nova-compute[75234]:   <serial>cc40439e-08c7-41c0-824c-f2aa5fbabc5d</serial>
Nov 29 03:22:24 user nova-compute[75234]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x08" function="0x0"/>
Nov 29 03:22:24 user nova-compute[75234]: </disk>
Nov 29 03:22:24 user nova-compute[75234]:  {{(pid=75234) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Nov 29 03:22:25 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-32ae734d-32d1-4039-a184-5136d17975de tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] Start waiting for the detach event from libvirt for device vdb with device alias virtio-disk1 for instance 946b919b-39ae-4d23-aefd-01952f4b40b6 {{(pid=75234) _detach_from_live_and_wait_for_event /opt/stack/nova/nova/virt/libvirt/driver.py:2599}}
Nov 29 03:22:27 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:22:29 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:22:33 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-5e4923f2-23b8-4726-be85-ce4e4e87ed39 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Acquiring lock "060c15ec-9012-471a-984a-14c9fa6ce23a" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:22:33 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-5e4923f2-23b8-4726-be85-ce4e4e87ed39 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Lock "060c15ec-9012-471a-984a-14c9fa6ce23a" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.002s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:22:33 user nova-compute[75234]: DEBUG nova.compute.manager [None req-5e4923f2-23b8-4726-be85-ce4e4e87ed39 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] [instance: 060c15ec-9012-471a-984a-14c9fa6ce23a] Starting instance... {{(pid=75234) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2422}}
Nov 29 03:22:33 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-5e4923f2-23b8-4726-be85-ce4e4e87ed39 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:22:33 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-5e4923f2-23b8-4726-be85-ce4e4e87ed39 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:22:33 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-5e4923f2-23b8-4726-be85-ce4e4e87ed39 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=75234) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2366}}
Nov 29 03:22:33 user nova-compute[75234]: INFO nova.compute.claims [None req-5e4923f2-23b8-4726-be85-ce4e4e87ed39 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] [instance: 060c15ec-9012-471a-984a-14c9fa6ce23a] Claim successful on node user
Nov 29 03:22:34 user nova-compute[75234]: DEBUG nova.compute.provider_tree [None req-5e4923f2-23b8-4726-be85-ce4e4e87ed39 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Inventory has not changed in ProviderTree for provider: 4d291220-7c37-4839-a0e3-b23c66502b74 {{(pid=75234) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Nov 29 03:22:34 user nova-compute[75234]: DEBUG nova.scheduler.client.report [None req-5e4923f2-23b8-4726-be85-ce4e4e87ed39 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Inventory has not changed for provider 4d291220-7c37-4839-a0e3-b23c66502b74 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=75234) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Nov 29 03:22:34 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-5e4923f2-23b8-4726-be85-ce4e4e87ed39 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.512s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:22:34 user nova-compute[75234]: DEBUG nova.compute.manager [None req-5e4923f2-23b8-4726-be85-ce4e4e87ed39 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] [instance: 060c15ec-9012-471a-984a-14c9fa6ce23a] Start building networks asynchronously for instance. {{(pid=75234) _build_resources /opt/stack/nova/nova/compute/manager.py:2819}}
Nov 29 03:22:34 user nova-compute[75234]: DEBUG nova.compute.manager [None req-5e4923f2-23b8-4726-be85-ce4e4e87ed39 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] [instance: 060c15ec-9012-471a-984a-14c9fa6ce23a] Allocating IP information in the background. {{(pid=75234) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1972}}
Nov 29 03:22:34 user nova-compute[75234]: DEBUG nova.network.neutron [None req-5e4923f2-23b8-4726-be85-ce4e4e87ed39 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] [instance: 060c15ec-9012-471a-984a-14c9fa6ce23a] allocate_for_instance() {{(pid=75234) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1156}}
Nov 29 03:22:34 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-5e4923f2-23b8-4726-be85-ce4e4e87ed39 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] [instance: 060c15ec-9012-471a-984a-14c9fa6ce23a] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Nov 29 03:22:34 user nova-compute[75234]: DEBUG nova.compute.manager [None req-5e4923f2-23b8-4726-be85-ce4e4e87ed39 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] [instance: 060c15ec-9012-471a-984a-14c9fa6ce23a] Start building block device mappings for instance. {{(pid=75234) _build_resources /opt/stack/nova/nova/compute/manager.py:2854}}
Nov 29 03:22:34 user nova-compute[75234]: DEBUG nova.policy [None req-5e4923f2-23b8-4726-be85-ce4e4e87ed39 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '66686114450f4f31b3a917667a152d8b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '315732b6e3ee44d584b8e510ef199a90', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=75234) authorize /opt/stack/nova/nova/policy.py:203}}
Nov 29 03:22:34 user nova-compute[75234]: DEBUG nova.compute.manager [None req-5e4923f2-23b8-4726-be85-ce4e4e87ed39 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] [instance: 060c15ec-9012-471a-984a-14c9fa6ce23a] Start spawning the instance on the hypervisor. {{(pid=75234) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2628}}
Nov 29 03:22:34 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-5e4923f2-23b8-4726-be85-ce4e4e87ed39 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] [instance: 060c15ec-9012-471a-984a-14c9fa6ce23a] Creating instance directory {{(pid=75234) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4727}}
Nov 29 03:22:34 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-5e4923f2-23b8-4726-be85-ce4e4e87ed39 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] [instance: 060c15ec-9012-471a-984a-14c9fa6ce23a] Creating image(s)
Nov 29 03:22:34 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-5e4923f2-23b8-4726-be85-ce4e4e87ed39 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Acquiring lock "/opt/stack/data/nova/instances/060c15ec-9012-471a-984a-14c9fa6ce23a/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:22:34 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-5e4923f2-23b8-4726-be85-ce4e4e87ed39 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Lock "/opt/stack/data/nova/instances/060c15ec-9012-471a-984a-14c9fa6ce23a/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:22:34 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-5e4923f2-23b8-4726-be85-ce4e4e87ed39 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Lock "/opt/stack/data/nova/instances/060c15ec-9012-471a-984a-14c9fa6ce23a/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.007s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:22:34 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-5e4923f2-23b8-4726-be85-ce4e4e87ed39 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:22:34 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-5e4923f2-23b8-4726-be85-ce4e4e87ed39 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json" returned: 0 in 0.177s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:22:34 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-5e4923f2-23b8-4726-be85-ce4e4e87ed39 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Acquiring lock "80686e29dc9db6600d03e85cb62af298e12d7918" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:22:34 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-5e4923f2-23b8-4726-be85-ce4e4e87ed39 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Lock "80686e29dc9db6600d03e85cb62af298e12d7918" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:22:34 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-5e4923f2-23b8-4726-be85-ce4e4e87ed39 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:22:34 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-5e4923f2-23b8-4726-be85-ce4e4e87ed39 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json" returned: 0 in 0.225s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:22:34 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-5e4923f2-23b8-4726-be85-ce4e4e87ed39 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918,backing_fmt=raw /opt/stack/data/nova/instances/060c15ec-9012-471a-984a-14c9fa6ce23a/disk 1073741824 {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:22:34 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-5e4923f2-23b8-4726-be85-ce4e4e87ed39 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918,backing_fmt=raw /opt/stack/data/nova/instances/060c15ec-9012-471a-984a-14c9fa6ce23a/disk 1073741824" returned: 0 in 0.051s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:22:34 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-5e4923f2-23b8-4726-be85-ce4e4e87ed39 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Lock "80686e29dc9db6600d03e85cb62af298e12d7918" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.283s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:22:34 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-5e4923f2-23b8-4726-be85-ce4e4e87ed39 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:22:34 user nova-compute[75234]: DEBUG nova.network.neutron [None req-5e4923f2-23b8-4726-be85-ce4e4e87ed39 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] [instance: 060c15ec-9012-471a-984a-14c9fa6ce23a] Successfully created port: 3835164f-aa54-41fa-adf1-d10ae894fba5 {{(pid=75234) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:548}}
Nov 29 03:22:34 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:22:35 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-5e4923f2-23b8-4726-be85-ce4e4e87ed39 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json" returned: 0 in 0.204s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:22:35 user nova-compute[75234]: DEBUG nova.virt.disk.api [None req-5e4923f2-23b8-4726-be85-ce4e4e87ed39 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Checking if we can resize image /opt/stack/data/nova/instances/060c15ec-9012-471a-984a-14c9fa6ce23a/disk. size=1073741824 {{(pid=75234) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:166}}
Nov 29 03:22:35 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-5e4923f2-23b8-4726-be85-ce4e4e87ed39 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/060c15ec-9012-471a-984a-14c9fa6ce23a/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:22:35 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-5e4923f2-23b8-4726-be85-ce4e4e87ed39 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/060c15ec-9012-471a-984a-14c9fa6ce23a/disk --force-share --output=json" returned: 0 in 0.182s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:22:35 user nova-compute[75234]: DEBUG nova.virt.disk.api [None req-5e4923f2-23b8-4726-be85-ce4e4e87ed39 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Cannot resize image /opt/stack/data/nova/instances/060c15ec-9012-471a-984a-14c9fa6ce23a/disk to a smaller size. {{(pid=75234) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:172}}
Nov 29 03:22:35 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-5e4923f2-23b8-4726-be85-ce4e4e87ed39 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] [instance: 060c15ec-9012-471a-984a-14c9fa6ce23a] Created local disks {{(pid=75234) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4861}}
Nov 29 03:22:35 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-5e4923f2-23b8-4726-be85-ce4e4e87ed39 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] [instance: 060c15ec-9012-471a-984a-14c9fa6ce23a] Ensure instance console log exists: /opt/stack/data/nova/instances/060c15ec-9012-471a-984a-14c9fa6ce23a/console.log {{(pid=75234) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4613}}
Nov 29 03:22:35 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-5e4923f2-23b8-4726-be85-ce4e4e87ed39 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:22:35 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-5e4923f2-23b8-4726-be85-ce4e4e87ed39 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:22:35 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-5e4923f2-23b8-4726-be85-ce4e4e87ed39 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:22:35 user nova-compute[75234]: DEBUG nova.network.neutron [None req-5e4923f2-23b8-4726-be85-ce4e4e87ed39 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] [instance: 060c15ec-9012-471a-984a-14c9fa6ce23a] Successfully updated port: 3835164f-aa54-41fa-adf1-d10ae894fba5 {{(pid=75234) _update_port /opt/stack/nova/nova/network/neutron.py:586}}
Nov 29 03:22:35 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-5e4923f2-23b8-4726-be85-ce4e4e87ed39 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Acquiring lock "refresh_cache-060c15ec-9012-471a-984a-14c9fa6ce23a" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:22:35 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-5e4923f2-23b8-4726-be85-ce4e4e87ed39 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Acquired lock "refresh_cache-060c15ec-9012-471a-984a-14c9fa6ce23a" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:22:35 user nova-compute[75234]: DEBUG nova.network.neutron [None req-5e4923f2-23b8-4726-be85-ce4e4e87ed39 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] [instance: 060c15ec-9012-471a-984a-14c9fa6ce23a] Building network info cache for instance {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2009}}
Nov 29 03:22:35 user nova-compute[75234]: DEBUG nova.compute.manager [req-828a4a4e-a8ab-4b90-990e-5dac7d9b2c0f req-09b29131-231e-4b06-bb60-4033fd38be07 service nova] [instance: 060c15ec-9012-471a-984a-14c9fa6ce23a] Received event network-changed-3835164f-aa54-41fa-adf1-d10ae894fba5 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:22:35 user nova-compute[75234]: DEBUG nova.compute.manager [req-828a4a4e-a8ab-4b90-990e-5dac7d9b2c0f req-09b29131-231e-4b06-bb60-4033fd38be07 service nova] [instance: 060c15ec-9012-471a-984a-14c9fa6ce23a] Refreshing instance network info cache due to event network-changed-3835164f-aa54-41fa-adf1-d10ae894fba5. {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11117}}
Nov 29 03:22:35 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-828a4a4e-a8ab-4b90-990e-5dac7d9b2c0f req-09b29131-231e-4b06-bb60-4033fd38be07 service nova] Acquiring lock "refresh_cache-060c15ec-9012-471a-984a-14c9fa6ce23a" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:22:35 user nova-compute[75234]: DEBUG nova.network.neutron [None req-5e4923f2-23b8-4726-be85-ce4e4e87ed39 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] [instance: 060c15ec-9012-471a-984a-14c9fa6ce23a] Instance cache missing network info. {{(pid=75234) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3322}}
Nov 29 03:22:36 user nova-compute[75234]: DEBUG nova.network.neutron [None req-5e4923f2-23b8-4726-be85-ce4e4e87ed39 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] [instance: 060c15ec-9012-471a-984a-14c9fa6ce23a] Updating instance_info_cache with network_info: [{"id": "3835164f-aa54-41fa-adf1-d10ae894fba5", "address": "fa:16:3e:40:80:df", "network": {"id": "0f289171-3d73-4054-95a0-2515b4f206a0", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-1787960284-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "315732b6e3ee44d584b8e510ef199a90", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap3835164f-aa", "ovs_interfaceid": "3835164f-aa54-41fa-adf1-d10ae894fba5", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:22:36 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-5e4923f2-23b8-4726-be85-ce4e4e87ed39 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Releasing lock "refresh_cache-060c15ec-9012-471a-984a-14c9fa6ce23a" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:22:36 user nova-compute[75234]: DEBUG nova.compute.manager [None req-5e4923f2-23b8-4726-be85-ce4e4e87ed39 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] [instance: 060c15ec-9012-471a-984a-14c9fa6ce23a] Instance network_info: |[{"id": "3835164f-aa54-41fa-adf1-d10ae894fba5", "address": "fa:16:3e:40:80:df", "network": {"id": "0f289171-3d73-4054-95a0-2515b4f206a0", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-1787960284-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "315732b6e3ee44d584b8e510ef199a90", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap3835164f-aa", "ovs_interfaceid": "3835164f-aa54-41fa-adf1-d10ae894fba5", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=75234) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1987}}
Nov 29 03:22:36 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-828a4a4e-a8ab-4b90-990e-5dac7d9b2c0f req-09b29131-231e-4b06-bb60-4033fd38be07 service nova] Acquired lock "refresh_cache-060c15ec-9012-471a-984a-14c9fa6ce23a" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:22:36 user nova-compute[75234]: DEBUG nova.network.neutron [req-828a4a4e-a8ab-4b90-990e-5dac7d9b2c0f req-09b29131-231e-4b06-bb60-4033fd38be07 service nova] [instance: 060c15ec-9012-471a-984a-14c9fa6ce23a] Refreshing network info cache for port 3835164f-aa54-41fa-adf1-d10ae894fba5 {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2006}}
Nov 29 03:22:36 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-5e4923f2-23b8-4726-be85-ce4e4e87ed39 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] [instance: 060c15ec-9012-471a-984a-14c9fa6ce23a] Start _get_guest_xml network_info=[{"id": "3835164f-aa54-41fa-adf1-d10ae894fba5", "address": "fa:16:3e:40:80:df", "network": {"id": "0f289171-3d73-4054-95a0-2515b4f206a0", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-1787960284-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "315732b6e3ee44d584b8e510ef199a90", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap3835164f-aa", "ovs_interfaceid": "3835164f-aa54-41fa-adf1-d10ae894fba5", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum='c8fc807773e5354afe61636071771906',container_format='bare',created_at=2023-11-29T03:02:15Z,direct_url=<?>,disk_format='qcow2',id=96c30ba7-9b73-4a30-8ba4-7138879573b7,min_disk=0,min_ram=0,name='cirros-0.6.2-x86_64-disk',owner='b18e380921c748b78517b6f1211cd0f1',properties=ImageMetaProps,protected=<?>,size=21430272,status='active',tags=<?>,updated_at=2023-11-29T03:02:17Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [{'device_name': '/dev/vda', 'encryption_format': None, 'encrypted': False, 'device_type': 'disk', 'size': 0, 'disk_bus': 'virtio', 'boot_index': 0, 'encryption_options': None, 'encryption_secret_uuid': None, 'guest_format': None, 'image_id': '96c30ba7-9b73-4a30-8ba4-7138879573b7'}], 'ephemerals': [], 'block_device_mapping': [], 'swap': None} {{(pid=75234) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7562}}
Nov 29 03:22:36 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-5e4923f2-23b8-4726-be85-ce4e4e87ed39 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:22:36 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-5e4923f2-23b8-4726-be85-ce4e4e87ed39 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:22:36 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-5e4923f2-23b8-4726-be85-ce4e4e87ed39 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Searching host: 'user' for CPU controller through CGroups V1... {{(pid=75234) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1652}}
Nov 29 03:22:36 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-5e4923f2-23b8-4726-be85-ce4e4e87ed39 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] CPU controller missing on host. {{(pid=75234) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1662}}
Nov 29 03:22:36 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-5e4923f2-23b8-4726-be85-ce4e4e87ed39 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Searching host: 'user' for CPU controller through CGroups V2... {{(pid=75234) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1671}}
Nov 29 03:22:36 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-5e4923f2-23b8-4726-be85-ce4e4e87ed39 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] CPU controller found on host. {{(pid=75234) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1678}}
Nov 29 03:22:36 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-5e4923f2-23b8-4726-be85-ce4e4e87ed39 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=75234) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5400}}
Nov 29 03:22:36 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-5e4923f2-23b8-4726-be85-ce4e4e87ed39 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-11-29T03:02:04Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='1',id=6,is_public=True,memory_mb=512,name='m1.tiny',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='c8fc807773e5354afe61636071771906',container_format='bare',created_at=2023-11-29T03:02:15Z,direct_url=<?>,disk_format='qcow2',id=96c30ba7-9b73-4a30-8ba4-7138879573b7,min_disk=0,min_ram=0,name='cirros-0.6.2-x86_64-disk',owner='b18e380921c748b78517b6f1211cd0f1',properties=ImageMetaProps,protected=<?>,size=21430272,status='active',tags=<?>,updated_at=2023-11-29T03:02:17Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:563}}
Nov 29 03:22:36 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-5e4923f2-23b8-4726-be85-ce4e4e87ed39 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Flavor limits 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:348}}
Nov 29 03:22:36 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-5e4923f2-23b8-4726-be85-ce4e4e87ed39 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Image limits 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:352}}
Nov 29 03:22:36 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-5e4923f2-23b8-4726-be85-ce4e4e87ed39 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Flavor pref 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:388}}
Nov 29 03:22:36 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-5e4923f2-23b8-4726-be85-ce4e4e87ed39 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Image pref 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:392}}
Nov 29 03:22:36 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-5e4923f2-23b8-4726-be85-ce4e4e87ed39 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:430}}
Nov 29 03:22:36 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-5e4923f2-23b8-4726-be85-ce4e4e87ed39 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:569}}
Nov 29 03:22:36 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-5e4923f2-23b8-4726-be85-ce4e4e87ed39 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=75234) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:471}}
Nov 29 03:22:36 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-5e4923f2-23b8-4726-be85-ce4e4e87ed39 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Got 1 possible topologies {{(pid=75234) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:501}}
Nov 29 03:22:36 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-5e4923f2-23b8-4726-be85-ce4e4e87ed39 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:575}}
Nov 29 03:22:36 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-5e4923f2-23b8-4726-be85-ce4e4e87ed39 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:577}}
Nov 29 03:22:36 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-5e4923f2-23b8-4726-be85-ce4e4e87ed39 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-11-29T03:22:32Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-AttachVolumeNegativeTest-server-1614204486',display_name='tempest-AttachVolumeNegativeTest-server-1614204486',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-attachvolumenegativetest-server-1614204486',id=38,image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBACqQfFLGg5k9dVSlMtreyy4nYgd0k+k8JsZp3H7sIImHG7V9LfBMqYLdLqfJnxtCeXAGpm1MDcSX+HtAljaNui83OKS0FLyT7TjorcVMVGV7urkT0H9swBQ45aBBw/5Qw==',key_name='tempest-keypair-2051938263',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='315732b6e3ee44d584b8e510ef199a90',ramdisk_id='',reservation_id='r-b8r0t0y0',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-AttachVolumeNegativeTest-1582496806',owner_user_name='tempest-AttachVolumeNegativeTest-1582496806-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-11-29T03:22:34Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='66686114450f4f31b3a917667a152d8b',uuid=060c15ec-9012-471a-984a-14c9fa6ce23a,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "3835164f-aa54-41fa-adf1-d10ae894fba5", "address": "fa:16:3e:40:80:df", "network": {"id": "0f289171-3d73-4054-95a0-2515b4f206a0", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-1787960284-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "315732b6e3ee44d584b8e510ef199a90", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap3835164f-aa", "ovs_interfaceid": "3835164f-aa54-41fa-adf1-d10ae894fba5", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=75234) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Nov 29 03:22:36 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-5e4923f2-23b8-4726-be85-ce4e4e87ed39 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Converting VIF {"id": "3835164f-aa54-41fa-adf1-d10ae894fba5", "address": "fa:16:3e:40:80:df", "network": {"id": "0f289171-3d73-4054-95a0-2515b4f206a0", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-1787960284-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "315732b6e3ee44d584b8e510ef199a90", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap3835164f-aa", "ovs_interfaceid": "3835164f-aa54-41fa-adf1-d10ae894fba5", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:22:36 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-5e4923f2-23b8-4726-be85-ce4e4e87ed39 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:40:80:df,bridge_name='br-int',has_traffic_filtering=True,id=3835164f-aa54-41fa-adf1-d10ae894fba5,network=Network(0f289171-3d73-4054-95a0-2515b4f206a0),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap3835164f-aa') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:22:36 user nova-compute[75234]: DEBUG nova.objects.instance [None req-5e4923f2-23b8-4726-be85-ce4e4e87ed39 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Lazy-loading 'pci_devices' on Instance uuid 060c15ec-9012-471a-984a-14c9fa6ce23a {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:22:36 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-5e4923f2-23b8-4726-be85-ce4e4e87ed39 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] [instance: 060c15ec-9012-471a-984a-14c9fa6ce23a] End _get_guest_xml xml=<domain type="kvm">
Nov 29 03:22:36 user nova-compute[75234]:   <uuid>060c15ec-9012-471a-984a-14c9fa6ce23a</uuid>
Nov 29 03:22:36 user nova-compute[75234]:   <name>instance-00000026</name>
Nov 29 03:22:36 user nova-compute[75234]:   <memory>524288</memory>
Nov 29 03:22:36 user nova-compute[75234]:   <vcpu>1</vcpu>
Nov 29 03:22:36 user nova-compute[75234]:   <metadata>
Nov 29 03:22:36 user nova-compute[75234]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Nov 29 03:22:36 user nova-compute[75234]:       <nova:package version="0.0.0"/>
Nov 29 03:22:36 user nova-compute[75234]:       <nova:name>tempest-AttachVolumeNegativeTest-server-1614204486</nova:name>
Nov 29 03:22:36 user nova-compute[75234]:       <nova:creationTime>2023-11-29 03:22:36</nova:creationTime>
Nov 29 03:22:36 user nova-compute[75234]:       <nova:flavor name="m1.tiny">
Nov 29 03:22:36 user nova-compute[75234]:         <nova:memory>512</nova:memory>
Nov 29 03:22:36 user nova-compute[75234]:         <nova:disk>1</nova:disk>
Nov 29 03:22:36 user nova-compute[75234]:         <nova:swap>0</nova:swap>
Nov 29 03:22:36 user nova-compute[75234]:         <nova:ephemeral>0</nova:ephemeral>
Nov 29 03:22:36 user nova-compute[75234]:         <nova:vcpus>1</nova:vcpus>
Nov 29 03:22:36 user nova-compute[75234]:       </nova:flavor>
Nov 29 03:22:36 user nova-compute[75234]:       <nova:owner>
Nov 29 03:22:36 user nova-compute[75234]:         <nova:user uuid="66686114450f4f31b3a917667a152d8b">tempest-AttachVolumeNegativeTest-1582496806-project-member</nova:user>
Nov 29 03:22:36 user nova-compute[75234]:         <nova:project uuid="315732b6e3ee44d584b8e510ef199a90">tempest-AttachVolumeNegativeTest-1582496806</nova:project>
Nov 29 03:22:36 user nova-compute[75234]:       </nova:owner>
Nov 29 03:22:36 user nova-compute[75234]:       <nova:root type="image" uuid="96c30ba7-9b73-4a30-8ba4-7138879573b7"/>
Nov 29 03:22:36 user nova-compute[75234]:       <nova:ports>
Nov 29 03:22:36 user nova-compute[75234]:         <nova:port uuid="3835164f-aa54-41fa-adf1-d10ae894fba5">
Nov 29 03:22:36 user nova-compute[75234]:           <nova:ip type="fixed" address="10.0.0.13" ipVersion="4"/>
Nov 29 03:22:36 user nova-compute[75234]:         </nova:port>
Nov 29 03:22:36 user nova-compute[75234]:       </nova:ports>
Nov 29 03:22:36 user nova-compute[75234]:     </nova:instance>
Nov 29 03:22:36 user nova-compute[75234]:   </metadata>
Nov 29 03:22:36 user nova-compute[75234]:   <sysinfo type="smbios">
Nov 29 03:22:36 user nova-compute[75234]:     <system>
Nov 29 03:22:36 user nova-compute[75234]:       <entry name="manufacturer">OpenStack Foundation</entry>
Nov 29 03:22:36 user nova-compute[75234]:       <entry name="product">OpenStack Nova</entry>
Nov 29 03:22:36 user nova-compute[75234]:       <entry name="version">0.0.0</entry>
Nov 29 03:22:36 user nova-compute[75234]:       <entry name="serial">060c15ec-9012-471a-984a-14c9fa6ce23a</entry>
Nov 29 03:22:36 user nova-compute[75234]:       <entry name="uuid">060c15ec-9012-471a-984a-14c9fa6ce23a</entry>
Nov 29 03:22:36 user nova-compute[75234]:       <entry name="family">Virtual Machine</entry>
Nov 29 03:22:36 user nova-compute[75234]:     </system>
Nov 29 03:22:36 user nova-compute[75234]:   </sysinfo>
Nov 29 03:22:36 user nova-compute[75234]:   <os>
Nov 29 03:22:36 user nova-compute[75234]:     <type arch="x86_64" machine="pc">hvm</type>
Nov 29 03:22:36 user nova-compute[75234]:     <boot dev="hd"/>
Nov 29 03:22:36 user nova-compute[75234]:     <smbios mode="sysinfo"/>
Nov 29 03:22:36 user nova-compute[75234]:   </os>
Nov 29 03:22:36 user nova-compute[75234]:   <features>
Nov 29 03:22:36 user nova-compute[75234]:     <acpi/>
Nov 29 03:22:36 user nova-compute[75234]:     <apic/>
Nov 29 03:22:36 user nova-compute[75234]:     <vmcoreinfo/>
Nov 29 03:22:36 user nova-compute[75234]:   </features>
Nov 29 03:22:36 user nova-compute[75234]:   <clock offset="utc">
Nov 29 03:22:36 user nova-compute[75234]:     <timer name="pit" tickpolicy="delay"/>
Nov 29 03:22:36 user nova-compute[75234]:     <timer name="rtc" tickpolicy="catchup"/>
Nov 29 03:22:36 user nova-compute[75234]:     <timer name="hpet" present="no"/>
Nov 29 03:22:36 user nova-compute[75234]:   </clock>
Nov 29 03:22:36 user nova-compute[75234]:   <cpu mode="custom" match="exact">
Nov 29 03:22:36 user nova-compute[75234]:     <model>Nehalem</model>
Nov 29 03:22:36 user nova-compute[75234]:     <topology sockets="1" cores="1" threads="1"/>
Nov 29 03:22:36 user nova-compute[75234]:   </cpu>
Nov 29 03:22:36 user nova-compute[75234]:   <devices>
Nov 29 03:22:36 user nova-compute[75234]:     <disk type="file" device="disk">
Nov 29 03:22:36 user nova-compute[75234]:       <driver name="qemu" type="qcow2" cache="none"/>
Nov 29 03:22:36 user nova-compute[75234]:       <source file="/opt/stack/data/nova/instances/060c15ec-9012-471a-984a-14c9fa6ce23a/disk"/>
Nov 29 03:22:36 user nova-compute[75234]:       <target dev="vda" bus="virtio"/>
Nov 29 03:22:36 user nova-compute[75234]:     </disk>
Nov 29 03:22:36 user nova-compute[75234]:     <interface type="ethernet">
Nov 29 03:22:36 user nova-compute[75234]:       <mac address="fa:16:3e:40:80:df"/>
Nov 29 03:22:36 user nova-compute[75234]:       <model type="virtio"/>
Nov 29 03:22:36 user nova-compute[75234]:       <mtu size="1442"/>
Nov 29 03:22:36 user nova-compute[75234]:       <target dev="tap3835164f-aa"/>
Nov 29 03:22:36 user nova-compute[75234]:     </interface>
Nov 29 03:22:36 user nova-compute[75234]:     <serial type="pty">
Nov 29 03:22:36 user nova-compute[75234]:       <log file="/opt/stack/data/nova/instances/060c15ec-9012-471a-984a-14c9fa6ce23a/console.log" append="off"/>
Nov 29 03:22:36 user nova-compute[75234]:     </serial>
Nov 29 03:22:36 user nova-compute[75234]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Nov 29 03:22:36 user nova-compute[75234]:     <video>
Nov 29 03:22:36 user nova-compute[75234]:       <model type="virtio"/>
Nov 29 03:22:36 user nova-compute[75234]:     </video>
Nov 29 03:22:36 user nova-compute[75234]:     <rng model="virtio">
Nov 29 03:22:36 user nova-compute[75234]:       <backend model="random">/dev/urandom</backend>
Nov 29 03:22:36 user nova-compute[75234]:     </rng>
Nov 29 03:22:36 user nova-compute[75234]:     <controller type="usb" index="0" model="none"/>
Nov 29 03:22:36 user nova-compute[75234]:     <memballoon model="virtio">
Nov 29 03:22:36 user nova-compute[75234]:       <stats period="10"/>
Nov 29 03:22:36 user nova-compute[75234]:     </memballoon>
Nov 29 03:22:36 user nova-compute[75234]:   </devices>
Nov 29 03:22:36 user nova-compute[75234]: </domain>
Nov 29 03:22:36 user nova-compute[75234]:  {{(pid=75234) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7568}}
Nov 29 03:22:36 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-5e4923f2-23b8-4726-be85-ce4e4e87ed39 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-11-29T03:22:32Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-AttachVolumeNegativeTest-server-1614204486',display_name='tempest-AttachVolumeNegativeTest-server-1614204486',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-attachvolumenegativetest-server-1614204486',id=38,image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBACqQfFLGg5k9dVSlMtreyy4nYgd0k+k8JsZp3H7sIImHG7V9LfBMqYLdLqfJnxtCeXAGpm1MDcSX+HtAljaNui83OKS0FLyT7TjorcVMVGV7urkT0H9swBQ45aBBw/5Qw==',key_name='tempest-keypair-2051938263',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='315732b6e3ee44d584b8e510ef199a90',ramdisk_id='',reservation_id='r-b8r0t0y0',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-AttachVolumeNegativeTest-1582496806',owner_user_name='tempest-AttachVolumeNegativeTest-1582496806-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-11-29T03:22:34Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='66686114450f4f31b3a917667a152d8b',uuid=060c15ec-9012-471a-984a-14c9fa6ce23a,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "3835164f-aa54-41fa-adf1-d10ae894fba5", "address": "fa:16:3e:40:80:df", "network": {"id": "0f289171-3d73-4054-95a0-2515b4f206a0", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-1787960284-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "315732b6e3ee44d584b8e510ef199a90", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap3835164f-aa", "ovs_interfaceid": "3835164f-aa54-41fa-adf1-d10ae894fba5", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Nov 29 03:22:36 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-5e4923f2-23b8-4726-be85-ce4e4e87ed39 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Converting VIF {"id": "3835164f-aa54-41fa-adf1-d10ae894fba5", "address": "fa:16:3e:40:80:df", "network": {"id": "0f289171-3d73-4054-95a0-2515b4f206a0", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-1787960284-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "315732b6e3ee44d584b8e510ef199a90", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap3835164f-aa", "ovs_interfaceid": "3835164f-aa54-41fa-adf1-d10ae894fba5", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:22:36 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-5e4923f2-23b8-4726-be85-ce4e4e87ed39 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:40:80:df,bridge_name='br-int',has_traffic_filtering=True,id=3835164f-aa54-41fa-adf1-d10ae894fba5,network=Network(0f289171-3d73-4054-95a0-2515b4f206a0),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap3835164f-aa') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:22:36 user nova-compute[75234]: DEBUG os_vif [None req-5e4923f2-23b8-4726-be85-ce4e4e87ed39 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:40:80:df,bridge_name='br-int',has_traffic_filtering=True,id=3835164f-aa54-41fa-adf1-d10ae894fba5,network=Network(0f289171-3d73-4054-95a0-2515b4f206a0),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap3835164f-aa') {{(pid=75234) plug /opt/stack/data/venv/lib/python3.10/site-packages/os_vif/__init__.py:76}}
Nov 29 03:22:36 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:22:36 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:22:36 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Nov 29 03:22:36 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:22:36 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbCreateCommand(_result=None, table=QoS, columns={'type': 'linux-noop', 'external_ids': {'id': '5dccd310-0e22-5828-8648-dbc3d5b39cd4', '_type': 'linux-noop'}}, row=False) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:22:36 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:22:36 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:22:36 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:22:36 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tap3835164f-aa, may_exist=True, interface_attrs={}) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:22:36 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Port, record=tap3835164f-aa, col_values=(('qos', UUID('774c631d-0397-4164-89e6-00c3150caae8')),), if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:22:36 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DbSetCommand(_result=None, table=Interface, record=tap3835164f-aa, col_values=(('external_ids', {'iface-id': '3835164f-aa54-41fa-adf1-d10ae894fba5', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:40:80:df', 'vm-uuid': '060c15ec-9012-471a-984a-14c9fa6ce23a'}),), if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:22:36 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:22:36 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:22:36 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:22:36 user nova-compute[75234]: INFO os_vif [None req-5e4923f2-23b8-4726-be85-ce4e4e87ed39 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:40:80:df,bridge_name='br-int',has_traffic_filtering=True,id=3835164f-aa54-41fa-adf1-d10ae894fba5,network=Network(0f289171-3d73-4054-95a0-2515b4f206a0),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap3835164f-aa')
Nov 29 03:22:36 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-5e4923f2-23b8-4726-be85-ce4e4e87ed39 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] No BDM found with device name vda, not building metadata. {{(pid=75234) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12089}}
Nov 29 03:22:36 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-5e4923f2-23b8-4726-be85-ce4e4e87ed39 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] No VIF found with MAC fa:16:3e:40:80:df, not building metadata {{(pid=75234) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Nov 29 03:22:36 user nova-compute[75234]: DEBUG nova.network.neutron [req-828a4a4e-a8ab-4b90-990e-5dac7d9b2c0f req-09b29131-231e-4b06-bb60-4033fd38be07 service nova] [instance: 060c15ec-9012-471a-984a-14c9fa6ce23a] Updated VIF entry in instance network info cache for port 3835164f-aa54-41fa-adf1-d10ae894fba5. {{(pid=75234) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3481}}
Nov 29 03:22:36 user nova-compute[75234]: DEBUG nova.network.neutron [req-828a4a4e-a8ab-4b90-990e-5dac7d9b2c0f req-09b29131-231e-4b06-bb60-4033fd38be07 service nova] [instance: 060c15ec-9012-471a-984a-14c9fa6ce23a] Updating instance_info_cache with network_info: [{"id": "3835164f-aa54-41fa-adf1-d10ae894fba5", "address": "fa:16:3e:40:80:df", "network": {"id": "0f289171-3d73-4054-95a0-2515b4f206a0", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-1787960284-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "315732b6e3ee44d584b8e510ef199a90", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap3835164f-aa", "ovs_interfaceid": "3835164f-aa54-41fa-adf1-d10ae894fba5", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:22:36 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-828a4a4e-a8ab-4b90-990e-5dac7d9b2c0f req-09b29131-231e-4b06-bb60-4033fd38be07 service nova] Releasing lock "refresh_cache-060c15ec-9012-471a-984a-14c9fa6ce23a" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:22:37 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:22:37 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:22:37 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:22:37 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:22:37 user nova-compute[75234]: DEBUG nova.compute.manager [req-2cb51014-7aa6-4038-a5a1-74d0a91b0566 req-a4ba637b-0e40-454c-98a0-82e3fe9b9163 service nova] [instance: 060c15ec-9012-471a-984a-14c9fa6ce23a] Received event network-vif-plugged-3835164f-aa54-41fa-adf1-d10ae894fba5 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:22:37 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-2cb51014-7aa6-4038-a5a1-74d0a91b0566 req-a4ba637b-0e40-454c-98a0-82e3fe9b9163 service nova] Acquiring lock "060c15ec-9012-471a-984a-14c9fa6ce23a-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:22:37 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-2cb51014-7aa6-4038-a5a1-74d0a91b0566 req-a4ba637b-0e40-454c-98a0-82e3fe9b9163 service nova] Lock "060c15ec-9012-471a-984a-14c9fa6ce23a-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:22:37 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-2cb51014-7aa6-4038-a5a1-74d0a91b0566 req-a4ba637b-0e40-454c-98a0-82e3fe9b9163 service nova] Lock "060c15ec-9012-471a-984a-14c9fa6ce23a-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:22:37 user nova-compute[75234]: DEBUG nova.compute.manager [req-2cb51014-7aa6-4038-a5a1-74d0a91b0566 req-a4ba637b-0e40-454c-98a0-82e3fe9b9163 service nova] [instance: 060c15ec-9012-471a-984a-14c9fa6ce23a] No waiting events found dispatching network-vif-plugged-3835164f-aa54-41fa-adf1-d10ae894fba5 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:22:37 user nova-compute[75234]: WARNING nova.compute.manager [req-2cb51014-7aa6-4038-a5a1-74d0a91b0566 req-a4ba637b-0e40-454c-98a0-82e3fe9b9163 service nova] [instance: 060c15ec-9012-471a-984a-14c9fa6ce23a] Received unexpected event network-vif-plugged-3835164f-aa54-41fa-adf1-d10ae894fba5 for instance with vm_state building and task_state spawning.
Nov 29 03:22:38 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:22:38 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:22:38 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:22:39 user nova-compute[75234]: DEBUG nova.compute.manager [None req-5e4923f2-23b8-4726-be85-ce4e4e87ed39 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] [instance: 060c15ec-9012-471a-984a-14c9fa6ce23a] Instance event wait completed in 0 seconds for  {{(pid=75234) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Nov 29 03:22:39 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-5e4923f2-23b8-4726-be85-ce4e4e87ed39 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] [instance: 060c15ec-9012-471a-984a-14c9fa6ce23a] Guest created on hypervisor {{(pid=75234) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4421}}
Nov 29 03:22:39 user nova-compute[75234]: INFO nova.virt.libvirt.driver [-] [instance: 060c15ec-9012-471a-984a-14c9fa6ce23a] Instance spawned successfully.
Nov 29 03:22:39 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-5e4923f2-23b8-4726-be85-ce4e4e87ed39 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] [instance: 060c15ec-9012-471a-984a-14c9fa6ce23a] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:904}}
Nov 29 03:22:39 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-5e4923f2-23b8-4726-be85-ce4e4e87ed39 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] [instance: 060c15ec-9012-471a-984a-14c9fa6ce23a] Found default for hw_cdrom_bus of ide {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:22:39 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-5e4923f2-23b8-4726-be85-ce4e4e87ed39 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] [instance: 060c15ec-9012-471a-984a-14c9fa6ce23a] Found default for hw_disk_bus of virtio {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:22:39 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-5e4923f2-23b8-4726-be85-ce4e4e87ed39 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] [instance: 060c15ec-9012-471a-984a-14c9fa6ce23a] Found default for hw_input_bus of None {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:22:39 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-5e4923f2-23b8-4726-be85-ce4e4e87ed39 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] [instance: 060c15ec-9012-471a-984a-14c9fa6ce23a] Found default for hw_pointer_model of None {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:22:39 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-5e4923f2-23b8-4726-be85-ce4e4e87ed39 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] [instance: 060c15ec-9012-471a-984a-14c9fa6ce23a] Found default for hw_video_model of virtio {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:22:39 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-5e4923f2-23b8-4726-be85-ce4e4e87ed39 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] [instance: 060c15ec-9012-471a-984a-14c9fa6ce23a] Found default for hw_vif_model of virtio {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:22:39 user nova-compute[75234]: INFO nova.compute.manager [None req-5e4923f2-23b8-4726-be85-ce4e4e87ed39 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] [instance: 060c15ec-9012-471a-984a-14c9fa6ce23a] Took 5.53 seconds to spawn the instance on the hypervisor.
Nov 29 03:22:39 user nova-compute[75234]: DEBUG nova.compute.manager [None req-5e4923f2-23b8-4726-be85-ce4e4e87ed39 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] [instance: 060c15ec-9012-471a-984a-14c9fa6ce23a] Checking state {{(pid=75234) _get_power_state /opt/stack/nova/nova/compute/manager.py:1782}}
Nov 29 03:22:39 user nova-compute[75234]: DEBUG nova.compute.manager [req-a3363f60-f669-45e4-9961-65e6ac5b4bf9 req-86bee95b-8b90-44ef-a275-4b33965f8c30 service nova] [instance: 060c15ec-9012-471a-984a-14c9fa6ce23a] Received event network-vif-plugged-3835164f-aa54-41fa-adf1-d10ae894fba5 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:22:39 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-a3363f60-f669-45e4-9961-65e6ac5b4bf9 req-86bee95b-8b90-44ef-a275-4b33965f8c30 service nova] Acquiring lock "060c15ec-9012-471a-984a-14c9fa6ce23a-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:22:39 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-a3363f60-f669-45e4-9961-65e6ac5b4bf9 req-86bee95b-8b90-44ef-a275-4b33965f8c30 service nova] Lock "060c15ec-9012-471a-984a-14c9fa6ce23a-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:22:39 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-a3363f60-f669-45e4-9961-65e6ac5b4bf9 req-86bee95b-8b90-44ef-a275-4b33965f8c30 service nova] Lock "060c15ec-9012-471a-984a-14c9fa6ce23a-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:22:39 user nova-compute[75234]: DEBUG nova.compute.manager [req-a3363f60-f669-45e4-9961-65e6ac5b4bf9 req-86bee95b-8b90-44ef-a275-4b33965f8c30 service nova] [instance: 060c15ec-9012-471a-984a-14c9fa6ce23a] No waiting events found dispatching network-vif-plugged-3835164f-aa54-41fa-adf1-d10ae894fba5 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:22:39 user nova-compute[75234]: WARNING nova.compute.manager [req-a3363f60-f669-45e4-9961-65e6ac5b4bf9 req-86bee95b-8b90-44ef-a275-4b33965f8c30 service nova] [instance: 060c15ec-9012-471a-984a-14c9fa6ce23a] Received unexpected event network-vif-plugged-3835164f-aa54-41fa-adf1-d10ae894fba5 for instance with vm_state building and task_state spawning.
Nov 29 03:22:39 user nova-compute[75234]: INFO nova.compute.manager [None req-5e4923f2-23b8-4726-be85-ce4e4e87ed39 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] [instance: 060c15ec-9012-471a-984a-14c9fa6ce23a] Took 6.42 seconds to build instance.
Nov 29 03:22:39 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-5e4923f2-23b8-4726-be85-ce4e4e87ed39 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Lock "060c15ec-9012-471a-984a-14c9fa6ce23a" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 6.660s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:22:41 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:22:42 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:22:45 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-32ae734d-32d1-4039-a184-5136d17975de tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] Waiting for libvirt event about the detach of device vdb with device alias virtio-disk1 from instance 946b919b-39ae-4d23-aefd-01952f4b40b6 is timed out.
Nov 29 03:22:45 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-32ae734d-32d1-4039-a184-5136d17975de tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] Successfully detached device vdb from instance 946b919b-39ae-4d23-aefd-01952f4b40b6 from the live domain config.
Nov 29 03:22:45 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-32ae734d-32d1-4039-a184-5136d17975de tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] [instance: 946b919b-39ae-4d23-aefd-01952f4b40b6] calling os-brick to detach iSCSI Volume {{(pid=75234) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Nov 29 03:22:45 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-32ae734d-32d1-4039-a184-5136d17975de tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f8a5050c9a0>, {'target_iqn': 'iqn.2010-10.org.openstack:cc40439e-08c7-41c0-824c-f2aa5fbabc5d', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'JhmYCkeU', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdc'}, None), 'kwargs': False}}" {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:176}}
Nov 29 03:22:45 user nova-compute[75234]: WARNING os_brick.initiator.connectors.base [None req-32ae734d-32d1-4039-a184-5136d17975de tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Nov 29 03:22:45 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-32ae734d-32d1-4039-a184-5136d17975de tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:68}}
Nov 29 03:22:45 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-32ae734d-32d1-4039-a184-5136d17975de tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:73}}
Nov 29 03:22:45 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:22:45 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.020s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:22:45 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[b558bae1-49d2-4723-949a-2119457d87c0]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:cf6ab229-533a-424e-bd67-1d54d50ad4be\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:cc40439e-08c7-41c0-824c-f2aa5fbabc5d\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:41f08854-31dd-4172-a5cf-01b0e695c10f\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:22:45 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-32ae734d-32d1-4039-a184-5136d17975de tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS:
Nov 29 03:22:45 user nova-compute[75234]: No targets found.
Nov 29 03:22:45 user nova-compute[75234]: iSNS:
Nov 29 03:22:45 user nova-compute[75234]: No targets found.
Nov 29 03:22:45 user nova-compute[75234]: STATIC:
Nov 29 03:22:45 user nova-compute[75234]: Target: iqn.2010-10.org.openstack:cf6ab229-533a-424e-bd67-1d54d50ad4be
Nov 29 03:22:45 user nova-compute[75234]:         Portal: 172.16.0.220:3260,-1
Nov 29 03:22:45 user nova-compute[75234]:                 Iface Name: default
Nov 29 03:22:45 user nova-compute[75234]: Target: iqn.2010-10.org.openstack:cc40439e-08c7-41c0-824c-f2aa5fbabc5d
Nov 29 03:22:45 user nova-compute[75234]:         Portal: 172.16.0.220:3260,-1
Nov 29 03:22:45 user nova-compute[75234]:                 Iface Name: default
Nov 29 03:22:45 user nova-compute[75234]: Target: iqn.2010-10.org.openstack:41f08854-31dd-4172-a5cf-01b0e695c10f
Nov 29 03:22:45 user nova-compute[75234]:         Portal: 172.16.0.220:3260,-1
Nov 29 03:22:45 user nova-compute[75234]:                 Iface Name: default
Nov 29 03:22:45 user nova-compute[75234]: FIRMWARE:
Nov 29 03:22:45 user nova-compute[75234]: No targets found.
Nov 29 03:22:45 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:22:45 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-32ae734d-32d1-4039-a184-5136d17975de tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] Regex to get portals from discoverydb: ^SENDTARGETS:
Nov 29 03:22:45 user nova-compute[75234]: .*?^DiscoveryAddress: 172.16.0.220,3260.*?
Nov 29 03:22:45 user nova-compute[75234]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=75234) _get_discoverydb_portals /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:382}}
Nov 29 03:22:45 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-32ae734d-32d1-4039-a184-5136d17975de tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:cc40439e-08c7-41c0-824c-f2aa5fbabc5d', 0)] {{(pid=75234) _get_connection_devices /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:830}}
Nov 29 03:22:45 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:22:45 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.024s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:22:45 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[8b0b17d4-5620-44dd-82d2-3ebbb0560e1f]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:41f08854-31dd-4172-a5cf-01b0e695c10f\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:cc40439e-08c7-41c0-824c-f2aa5fbabc5d\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:cf6ab229-533a-424e-bd67-1d54d50ad4be\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:22:45 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:22:45 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.025s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:22:45 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[494454b1-fcb7-414f-92e3-4ce62394bb64]: (4, ('tcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:cf6ab229-533a-424e-bd67-1d54d50ad4be (non-flash)\ntcp: [39] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:41f08854-31dd-4172-a5cf-01b0e695c10f (non-flash)\ntcp: [40] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:cc40439e-08c7-41c0-824c-f2aa5fbabc5d (non-flash)\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:22:45 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-32ae734d-32d1-4039-a184-5136d17975de tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:cf6ab229-533a-424e-bd67-1d54d50ad4be (non-flash)
Nov 29 03:22:45 user nova-compute[75234]: tcp: [39] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:41f08854-31dd-4172-a5cf-01b0e695c10f (non-flash)
Nov 29 03:22:45 user nova-compute[75234]: tcp: [40] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:cc40439e-08c7-41c0-824c-f2aa5fbabc5d (non-flash)
Nov 29 03:22:45 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:22:45 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-32ae734d-32d1-4039-a184-5136d17975de tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] iscsi session list stdout=tcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:cf6ab229-533a-424e-bd67-1d54d50ad4be (non-flash)
Nov 29 03:22:45 user nova-compute[75234]: tcp: [39] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:41f08854-31dd-4172-a5cf-01b0e695c10f (non-flash)
Nov 29 03:22:45 user nova-compute[75234]: tcp: [40] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:cc40439e-08c7-41c0-824c-f2aa5fbabc5d (non-flash)
Nov 29 03:22:45 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsi_session /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Nov 29 03:22:45 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-32ae734d-32d1-4039-a184-5136d17975de tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7f8a502c8a60>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:cc40439e-08c7-41c0-824c-f2aa5fbabc5d'): ({'sdc'}, set())}) {{(pid=75234) _get_connection_devices /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:863}}
Nov 29 03:22:45 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-32ae734d-32d1-4039-a184-5136d17975de tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] Removing single pathed devices sdc {{(pid=75234) remove_connection /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:374}}
Nov 29 03:22:45 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:22:45 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.069s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:22:45 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[a2b37b02-82a4-4021-bb58-2116b36a733c]: (4, ('path checker states:\nup                  2\npending             1\n\npaths: 0\nbusy: False\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:22:45 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdc {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:22:45 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdc" returned: 0 in 0.168s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:22:45 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[b1a1b1ff-dd92-46cc-9f72-5a7cb94687d3]: (4, ('ok\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:22:45 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-32ae734d-32d1-4039-a184-5136d17975de tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] Flushing IO for device /dev/sdc {{(pid=75234) flush_device_io /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:434}}
Nov 29 03:22:45 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdc {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:22:45 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdc" returned: 0 in 0.014s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:22:45 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[26856f72-e1c3-491d-9e4b-ddb9053ab987]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:22:45 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-32ae734d-32d1-4039-a184-5136d17975de tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] Remove SCSI device /dev/sdc with /sys/block/sdc/device/delete {{(pid=75234) remove_scsi_device /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:148}}
Nov 29 03:22:45 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdc/device/delete {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:22:45 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdc/device/delete" returned: 0 in 0.053s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:22:45 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[c5d7e1da-920b-457d-948b-3d515eeddbaf]: (4, ('1', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:22:45 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-32ae734d-32d1-4039-a184-5136d17975de tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] Checking to see if SCSI volumes sdc have been removed. {{(pid=75234) wait_for_volumes_removal /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:156}}
Nov 29 03:22:45 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-32ae734d-32d1-4039-a184-5136d17975de tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] SCSI volumes sdc have been removed. {{(pid=75234) wait_for_volumes_removal /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:166}}
Nov 29 03:22:45 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-32ae734d-32d1-4039-a184-5136d17975de tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:cc40439e-08c7-41c0-824c-f2aa5fbabc5d')] {{(pid=75234) _disconnect_connection /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Nov 29 03:22:45 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:cc40439e-08c7-41c0-824c-f2aa5fbabc5d -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:22:45 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:cc40439e-08c7-41c0-824c-f2aa5fbabc5d -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.026s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:22:45 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[7ff91ace-ea87-4b9d-b127-2c57e7148dad]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:22:45 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-32ae734d-32d1-4039-a184-5136d17975de tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:22:45 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:cc40439e-08c7-41c0-824c-f2aa5fbabc5d -p 172.16.0.220:3260 --logout {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:22:45 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:cc40439e-08c7-41c0-824c-f2aa5fbabc5d -p 172.16.0.220:3260 --logout" returned: 0 in 0.056s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:22:45 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[c2e29ae6-5f87-442e-a8aa-78f53aec9850]: (4, ('Logging out of session [sid: 40, target: iqn.2010-10.org.openstack:cc40439e-08c7-41c0-824c-f2aa5fbabc5d, portal: 172.16.0.220,3260]\nLogout of [sid: 40, target: iqn.2010-10.org.openstack:cc40439e-08c7-41c0-824c-f2aa5fbabc5d, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:22:45 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-32ae734d-32d1-4039-a184-5136d17975de tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 40, target: iqn.2010-10.org.openstack:cc40439e-08c7-41c0-824c-f2aa5fbabc5d, portal: 172.16.0.220,3260]
Nov 29 03:22:45 user nova-compute[75234]: Logout of [sid: 40, target: iqn.2010-10.org.openstack:cc40439e-08c7-41c0-824c-f2aa5fbabc5d, portal: 172.16.0.220,3260] successful.
Nov 29 03:22:45 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:22:45 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:cc40439e-08c7-41c0-824c-f2aa5fbabc5d -p 172.16.0.220:3260 --op delete {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:22:45 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:cc40439e-08c7-41c0-824c-f2aa5fbabc5d -p 172.16.0.220:3260 --op delete" returned: 0 in 0.030s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:22:45 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[522f30d7-a20d-46ca-bcd9-086a931f1557]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:22:45 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-32ae734d-32d1-4039-a184-5136d17975de tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:22:45 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-32ae734d-32d1-4039-a184-5136d17975de tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.536s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:87}}
Nov 29 03:22:45 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-32ae734d-32d1-4039-a184-5136d17975de tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] <== disconnect_volume: return (537ms) None {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:203}}
Nov 29 03:22:45 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-32ae734d-32d1-4039-a184-5136d17975de tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] [instance: 946b919b-39ae-4d23-aefd-01952f4b40b6] Disconnected iSCSI Volume {{(pid=75234) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:79}}
Nov 29 03:22:46 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:22:46 user nova-compute[75234]: DEBUG nova.objects.instance [None req-32ae734d-32d1-4039-a184-5136d17975de tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] Lazy-loading 'flavor' on Instance uuid 946b919b-39ae-4d23-aefd-01952f4b40b6 {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:22:46 user nova-compute[75234]: DEBUG nova.compute.manager [None req-f021bf2f-61ef-47a5-b59b-cb1386125d64 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] [instance: 65d48c93-b6da-4f51-9d5a-0676dda62a60] Checking state {{(pid=75234) _get_power_state /opt/stack/nova/nova/compute/manager.py:1782}}
Nov 29 03:22:46 user nova-compute[75234]: INFO nova.compute.manager [None req-f021bf2f-61ef-47a5-b59b-cb1386125d64 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] [instance: 65d48c93-b6da-4f51-9d5a-0676dda62a60] instance snapshotting
Nov 29 03:22:46 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-32ae734d-32d1-4039-a184-5136d17975de tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] Lock "946b919b-39ae-4d23-aefd-01952f4b40b6" "released" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: held 21.992s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:22:46 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-f021bf2f-61ef-47a5-b59b-cb1386125d64 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] [instance: 65d48c93-b6da-4f51-9d5a-0676dda62a60] Beginning live snapshot process
Nov 29 03:22:47 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-f021bf2f-61ef-47a5-b59b-cb1386125d64 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/65d48c93-b6da-4f51-9d5a-0676dda62a60/disk --force-share --output=json -f qcow2 {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:22:47 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-f021bf2f-61ef-47a5-b59b-cb1386125d64 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/65d48c93-b6da-4f51-9d5a-0676dda62a60/disk --force-share --output=json -f qcow2" returned: 0 in 0.175s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:22:47 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-f021bf2f-61ef-47a5-b59b-cb1386125d64 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/65d48c93-b6da-4f51-9d5a-0676dda62a60/disk --force-share --output=json -f qcow2 {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:22:47 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-f021bf2f-61ef-47a5-b59b-cb1386125d64 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/65d48c93-b6da-4f51-9d5a-0676dda62a60/disk --force-share --output=json -f qcow2" returned: 0 in 0.198s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:22:47 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-f021bf2f-61ef-47a5-b59b-cb1386125d64 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:22:47 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-1f033c13-c772-4284-adad-9d92f9362260 tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] Acquiring lock "interface-946b919b-39ae-4d23-aefd-01952f4b40b6-501886e8-de21-4108-86d4-d7a9138f8057" by "nova.compute.manager.ComputeManager.detach_interface.<locals>.do_detach_interface" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:22:47 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-1f033c13-c772-4284-adad-9d92f9362260 tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] Lock "interface-946b919b-39ae-4d23-aefd-01952f4b40b6-501886e8-de21-4108-86d4-d7a9138f8057" acquired by "nova.compute.manager.ComputeManager.detach_interface.<locals>.do_detach_interface" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:22:47 user nova-compute[75234]: DEBUG nova.objects.instance [None req-1f033c13-c772-4284-adad-9d92f9362260 tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] Lazy-loading 'flavor' on Instance uuid 946b919b-39ae-4d23-aefd-01952f4b40b6 {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:22:47 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-1f033c13-c772-4284-adad-9d92f9362260 tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='True',created_at=2023-11-29T03:20:14Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=InstanceDeviceMetadata,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-1777937262',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-device-tagging-server-1777937262',id=35,image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBJrVo4ngcPAPF4oP6xidv+ykI7O7g9y5APrTr7iXhruQjDIx7zqy1/omzWf7G2p8xg1QAr8Js8NrVgD6XT2ZMw0fSSCLDKoKJk+X3O2rM5aB1leyuLU4v60YiI/3ZRg9Ew==',key_name='tempest-keypair-1887924020',keypairs=<?>,launch_index=0,launched_at=2023-11-29T03:20:21Z,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=<?>,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='e28b5baf29b14aa4b2b98cb1b615705f',ramdisk_id='',reservation_id='r-bfdfrqjg',resources=<?>,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-TaggedAttachmentsTest-399278293',owner_user_name='tempest-TaggedAttachmentsTest-399278293-project-member'},tags=<?>,task_state=None,terminated_at=None,trusted_certs=<?>,updated_at=2023-11-29T03:20:22Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='522c3c107614404fbe3af8c38fdd189e',uuid=946b919b-39ae-4d23-aefd-01952f4b40b6,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "501886e8-de21-4108-86d4-d7a9138f8057", "address": "fa:16:3e:d3:09:2e", "network": {"id": "ca5d0d87-cb5f-4287-8438-a8ecc7d3fd33", "bridge": "br-int", "label": "tempest-tagged-attachments-test-net-1833288811", "subnets": [{"cidr": "10.10.10.0/24", "dns": [], "gateway": {"address": "10.10.10.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.10.10.149", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.10.10.1"}}], "meta": {"injected": false, "tenant_id": "e28b5baf29b14aa4b2b98cb1b615705f", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap501886e8-de", "ovs_interfaceid": "501886e8-de21-4108-86d4-d7a9138f8057", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=75234) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Nov 29 03:22:47 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-1f033c13-c772-4284-adad-9d92f9362260 tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] Converting VIF {"id": "501886e8-de21-4108-86d4-d7a9138f8057", "address": "fa:16:3e:d3:09:2e", "network": {"id": "ca5d0d87-cb5f-4287-8438-a8ecc7d3fd33", "bridge": "br-int", "label": "tempest-tagged-attachments-test-net-1833288811", "subnets": [{"cidr": "10.10.10.0/24", "dns": [], "gateway": {"address": "10.10.10.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.10.10.149", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.10.10.1"}}], "meta": {"injected": false, "tenant_id": "e28b5baf29b14aa4b2b98cb1b615705f", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap501886e8-de", "ovs_interfaceid": "501886e8-de21-4108-86d4-d7a9138f8057", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:22:47 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-1f033c13-c772-4284-adad-9d92f9362260 tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:d3:09:2e,bridge_name='br-int',has_traffic_filtering=True,id=501886e8-de21-4108-86d4-d7a9138f8057,network=Network(ca5d0d87-cb5f-4287-8438-a8ecc7d3fd33),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap501886e8-de') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:22:47 user nova-compute[75234]: DEBUG nova.virt.libvirt.guest [None req-1f033c13-c772-4284-adad-9d92f9362260 tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] looking for interface given config: <interface type="ethernet"><mac address="fa:16:3e:d3:09:2e"/><model type="virtio"/><mtu size="1442"/><target dev="tap501886e8-de"/></interface> {{(pid=75234) get_interface_by_cfg /opt/stack/nova/nova/virt/libvirt/guest.py:257}}
Nov 29 03:22:47 user nova-compute[75234]: DEBUG nova.virt.libvirt.guest [None req-1f033c13-c772-4284-adad-9d92f9362260 tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] looking for interface given config: <interface type="ethernet"><mac address="fa:16:3e:d3:09:2e"/><model type="virtio"/><mtu size="1442"/><target dev="tap501886e8-de"/></interface> {{(pid=75234) get_interface_by_cfg /opt/stack/nova/nova/virt/libvirt/guest.py:257}}
Nov 29 03:22:47 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-1f033c13-c772-4284-adad-9d92f9362260 tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] Attempting to detach device tap501886e8-de from instance 946b919b-39ae-4d23-aefd-01952f4b40b6 from the persistent domain config. {{(pid=75234) _detach_from_persistent /opt/stack/nova/nova/virt/libvirt/driver.py:2487}}
Nov 29 03:22:47 user nova-compute[75234]: DEBUG nova.virt.libvirt.guest [None req-1f033c13-c772-4284-adad-9d92f9362260 tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] detach device xml: <interface type="ethernet">
Nov 29 03:22:47 user nova-compute[75234]:   <mac address="fa:16:3e:d3:09:2e"/>
Nov 29 03:22:47 user nova-compute[75234]:   <model type="virtio"/>
Nov 29 03:22:47 user nova-compute[75234]:   <mtu size="1442"/>
Nov 29 03:22:47 user nova-compute[75234]:   <target dev="tap501886e8-de"/>
Nov 29 03:22:47 user nova-compute[75234]: </interface>
Nov 29 03:22:47 user nova-compute[75234]:  {{(pid=75234) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Nov 29 03:22:47 user nova-compute[75234]: DEBUG nova.virt.libvirt.guest [None req-1f033c13-c772-4284-adad-9d92f9362260 tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] looking for interface given config: <interface type="ethernet"><mac address="fa:16:3e:d3:09:2e"/><model type="virtio"/><mtu size="1442"/><target dev="tap501886e8-de"/></interface> {{(pid=75234) get_interface_by_cfg /opt/stack/nova/nova/virt/libvirt/guest.py:257}}
Nov 29 03:22:47 user nova-compute[75234]: DEBUG nova.virt.libvirt.guest [None req-1f033c13-c772-4284-adad-9d92f9362260 tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] interface for config: <interface type="ethernet"><mac address="fa:16:3e:d3:09:2e"/><model type="virtio"/><mtu size="1442"/><target dev="tap501886e8-de"/></interface>not found in domain: <domain type='kvm' id='48'>
Nov 29 03:22:47 user nova-compute[75234]:   <name>instance-00000023</name>
Nov 29 03:22:47 user nova-compute[75234]:   <uuid>946b919b-39ae-4d23-aefd-01952f4b40b6</uuid>
Nov 29 03:22:47 user nova-compute[75234]:   <metadata>
Nov 29 03:22:47 user nova-compute[75234]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1" xmlns:instance="http://openstack.org/xmlns/libvirt/nova/1.1">
Nov 29 03:22:47 user nova-compute[75234]:   <nova:package version="0.0.0"/>
Nov 29 03:22:47 user nova-compute[75234]:   <nova:name>tempest-device-tagging-server-1777937262</nova:name>
Nov 29 03:22:47 user nova-compute[75234]:   <nova:creationTime>2023-11-29 03:22:09</nova:creationTime>
Nov 29 03:22:47 user nova-compute[75234]:   <nova:flavor name="m1.tiny">
Nov 29 03:22:47 user nova-compute[75234]:     <nova:memory>512</nova:memory>
Nov 29 03:22:47 user nova-compute[75234]:     <nova:disk>1</nova:disk>
Nov 29 03:22:47 user nova-compute[75234]:     <nova:swap>0</nova:swap>
Nov 29 03:22:47 user nova-compute[75234]:     <nova:ephemeral>0</nova:ephemeral>
Nov 29 03:22:47 user nova-compute[75234]:     <nova:vcpus>1</nova:vcpus>
Nov 29 03:22:47 user nova-compute[75234]:   </nova:flavor>
Nov 29 03:22:47 user nova-compute[75234]:   <nova:owner>
Nov 29 03:22:47 user nova-compute[75234]:     <nova:user uuid="522c3c107614404fbe3af8c38fdd189e">tempest-TaggedAttachmentsTest-399278293-project-member</nova:user>
Nov 29 03:22:47 user nova-compute[75234]:     <nova:project uuid="e28b5baf29b14aa4b2b98cb1b615705f">tempest-TaggedAttachmentsTest-399278293</nova:project>
Nov 29 03:22:47 user nova-compute[75234]:   </nova:owner>
Nov 29 03:22:47 user nova-compute[75234]:   <nova:root type="image" uuid="96c30ba7-9b73-4a30-8ba4-7138879573b7"/>
Nov 29 03:22:47 user nova-compute[75234]:   <nova:ports>
Nov 29 03:22:47 user nova-compute[75234]:     <nova:port uuid="e1f71ce6-58f3-4665-afc5-f92d1b6801c5">
Nov 29 03:22:47 user nova-compute[75234]:       <nova:ip type="fixed" address="10.0.0.4" ipVersion="4"/>
Nov 29 03:22:47 user nova-compute[75234]:     </nova:port>
Nov 29 03:22:47 user nova-compute[75234]:     <nova:port uuid="501886e8-de21-4108-86d4-d7a9138f8057">
Nov 29 03:22:47 user nova-compute[75234]:       <nova:ip type="fixed" address="10.10.10.149" ipVersion="4"/>
Nov 29 03:22:47 user nova-compute[75234]:     </nova:port>
Nov 29 03:22:47 user nova-compute[75234]:   </nova:ports>
Nov 29 03:22:47 user nova-compute[75234]: </nova:instance>
Nov 29 03:22:47 user nova-compute[75234]:   </metadata>
Nov 29 03:22:47 user nova-compute[75234]:   <memory unit='KiB'>524288</memory>
Nov 29 03:22:47 user nova-compute[75234]:   <currentMemory unit='KiB'>524288</currentMemory>
Nov 29 03:22:47 user nova-compute[75234]:   <vcpu placement='static'>1</vcpu>
Nov 29 03:22:47 user nova-compute[75234]:   <resource>
Nov 29 03:22:47 user nova-compute[75234]:     <partition>/machine</partition>
Nov 29 03:22:47 user nova-compute[75234]:   </resource>
Nov 29 03:22:47 user nova-compute[75234]:   <sysinfo type='smbios'>
Nov 29 03:22:47 user nova-compute[75234]:     <system>
Nov 29 03:22:47 user nova-compute[75234]:       <entry name='manufacturer'>OpenStack Foundation</entry>
Nov 29 03:22:47 user nova-compute[75234]:       <entry name='product'>OpenStack Nova</entry>
Nov 29 03:22:47 user nova-compute[75234]:       <entry name='version'>0.0.0</entry>
Nov 29 03:22:47 user nova-compute[75234]:       <entry name='serial'>946b919b-39ae-4d23-aefd-01952f4b40b6</entry>
Nov 29 03:22:47 user nova-compute[75234]:       <entry name='uuid'>946b919b-39ae-4d23-aefd-01952f4b40b6</entry>
Nov 29 03:22:47 user nova-compute[75234]:       <entry name='family'>Virtual Machine</entry>
Nov 29 03:22:47 user nova-compute[75234]:     </system>
Nov 29 03:22:47 user nova-compute[75234]:   </sysinfo>
Nov 29 03:22:47 user nova-compute[75234]:   <os>
Nov 29 03:22:47 user nova-compute[75234]:     <type arch='x86_64' machine='pc-i440fx-6.2'>hvm</type>
Nov 29 03:22:47 user nova-compute[75234]:     <boot dev='hd'/>
Nov 29 03:22:47 user nova-compute[75234]:     <smbios mode='sysinfo'/>
Nov 29 03:22:47 user nova-compute[75234]:   </os>
Nov 29 03:22:47 user nova-compute[75234]:   <features>
Nov 29 03:22:47 user nova-compute[75234]:     <acpi/>
Nov 29 03:22:47 user nova-compute[75234]:     <apic/>
Nov 29 03:22:47 user nova-compute[75234]:     <vmcoreinfo state='on'/>
Nov 29 03:22:47 user nova-compute[75234]:   </features>
Nov 29 03:22:47 user nova-compute[75234]:   <cpu mode='custom' match='exact' check='full'>
Nov 29 03:22:47 user nova-compute[75234]:     <model fallback='forbid'>Nehalem</model>
Nov 29 03:22:47 user nova-compute[75234]:     <topology sockets='1' dies='1' cores='1' threads='1'/>
Nov 29 03:22:47 user nova-compute[75234]:     <feature policy='require' name='vme'/>
Nov 29 03:22:47 user nova-compute[75234]:     <feature policy='require' name='x2apic'/>
Nov 29 03:22:47 user nova-compute[75234]:     <feature policy='require' name='hypervisor'/>
Nov 29 03:22:47 user nova-compute[75234]:   </cpu>
Nov 29 03:22:47 user nova-compute[75234]:   <clock offset='utc'>
Nov 29 03:22:47 user nova-compute[75234]:     <timer name='pit' tickpolicy='delay'/>
Nov 29 03:22:47 user nova-compute[75234]:     <timer name='rtc' tickpolicy='catchup'/>
Nov 29 03:22:47 user nova-compute[75234]:     <timer name='hpet' present='no'/>
Nov 29 03:22:47 user nova-compute[75234]:   </clock>
Nov 29 03:22:47 user nova-compute[75234]:   <on_poweroff>destroy</on_poweroff>
Nov 29 03:22:47 user nova-compute[75234]:   <on_reboot>restart</on_reboot>
Nov 29 03:22:47 user nova-compute[75234]:   <on_crash>destroy</on_crash>
Nov 29 03:22:47 user nova-compute[75234]:   <devices>
Nov 29 03:22:47 user nova-compute[75234]:     <emulator>/usr/bin/qemu-system-x86_64</emulator>
Nov 29 03:22:47 user nova-compute[75234]:     <disk type='file' device='disk'>
Nov 29 03:22:47 user nova-compute[75234]:       <driver name='qemu' type='qcow2' cache='none'/>
Nov 29 03:22:47 user nova-compute[75234]:       <source file='/opt/stack/data/nova/instances/946b919b-39ae-4d23-aefd-01952f4b40b6/disk' index='2'/>
Nov 29 03:22:47 user nova-compute[75234]:       <backingStore type='file' index='3'>
Nov 29 03:22:47 user nova-compute[75234]:         <format type='raw'/>
Nov 29 03:22:47 user nova-compute[75234]:         <source file='/opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918'/>
Nov 29 03:22:47 user nova-compute[75234]:         <backingStore/>
Nov 29 03:22:47 user nova-compute[75234]:       </backingStore>
Nov 29 03:22:47 user nova-compute[75234]:       <target dev='vda' bus='virtio'/>
Nov 29 03:22:47 user nova-compute[75234]:       <alias name='virtio-disk0'/>
Nov 29 03:22:47 user nova-compute[75234]:       <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/>
Nov 29 03:22:47 user nova-compute[75234]:     </disk>
Nov 29 03:22:47 user nova-compute[75234]:     <disk type='file' device='cdrom'>
Nov 29 03:22:47 user nova-compute[75234]:       <driver name='qemu' type='raw' cache='none'/>
Nov 29 03:22:47 user nova-compute[75234]:       <source file='/opt/stack/data/nova/instances/946b919b-39ae-4d23-aefd-01952f4b40b6/disk.config' index='1'/>
Nov 29 03:22:47 user nova-compute[75234]:       <backingStore/>
Nov 29 03:22:47 user nova-compute[75234]:       <target dev='hda' bus='ide'/>
Nov 29 03:22:47 user nova-compute[75234]:       <readonly/>
Nov 29 03:22:47 user nova-compute[75234]:       <alias name='ide0-0-0'/>
Nov 29 03:22:47 user nova-compute[75234]:       <address type='drive' controller='0' bus='0' target='0' unit='0'/>
Nov 29 03:22:47 user nova-compute[75234]:     </disk>
Nov 29 03:22:47 user nova-compute[75234]:     <controller type='usb' index='0' model='none'>
Nov 29 03:22:47 user nova-compute[75234]:       <alias name='usb'/>
Nov 29 03:22:47 user nova-compute[75234]:     </controller>
Nov 29 03:22:47 user nova-compute[75234]:     <controller type='pci' index='0' model='pci-root'>
Nov 29 03:22:47 user nova-compute[75234]:       <alias name='pci.0'/>
Nov 29 03:22:47 user nova-compute[75234]:     </controller>
Nov 29 03:22:47 user nova-compute[75234]:     <controller type='ide' index='0'>
Nov 29 03:22:47 user nova-compute[75234]:       <alias name='ide'/>
Nov 29 03:22:47 user nova-compute[75234]:       <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x1'/>
Nov 29 03:22:47 user nova-compute[75234]:     </controller>
Nov 29 03:22:47 user nova-compute[75234]:     <interface type='ethernet'>
Nov 29 03:22:47 user nova-compute[75234]:       <mac address='fa:16:3e:81:ca:ab'/>
Nov 29 03:22:47 user nova-compute[75234]:       <target dev='tape1f71ce6-58'/>
Nov 29 03:22:47 user nova-compute[75234]:       <model type='virtio'/>
Nov 29 03:22:47 user nova-compute[75234]:       <mtu size='1442'/>
Nov 29 03:22:47 user nova-compute[75234]:       <alias name='net0'/>
Nov 29 03:22:47 user nova-compute[75234]:       <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>
Nov 29 03:22:47 user nova-compute[75234]:     </interface>
Nov 29 03:22:47 user nova-compute[75234]:     <interface type='ethernet'>
Nov 29 03:22:47 user nova-compute[75234]:       <mac address='fa:16:3e:d3:09:2e'/>
Nov 29 03:22:47 user nova-compute[75234]:       <target dev='tap501886e8-de'/>
Nov 29 03:22:47 user nova-compute[75234]:       <model type='virtio'/>
Nov 29 03:22:47 user nova-compute[75234]:       <mtu size='1442'/>
Nov 29 03:22:47 user nova-compute[75234]:       <alias name='net1'/>
Nov 29 03:22:47 user nova-compute[75234]:       <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x0'/>
Nov 29 03:22:47 user nova-compute[75234]:     </interface>
Nov 29 03:22:47 user nova-compute[75234]:     <serial type='pty'>
Nov 29 03:22:47 user nova-compute[75234]:       <source path='/dev/pts/3'/>
Nov 29 03:22:47 user nova-compute[75234]:       <log file='/opt/stack/data/nova/instances/946b919b-39ae-4d23-aefd-01952f4b40b6/console.log' append='off'/>
Nov 29 03:22:47 user nova-compute[75234]:       <target type='isa-serial' port='0'>
Nov 29 03:22:47 user nova-compute[75234]:         <model name='isa-serial'/>
Nov 29 03:22:47 user nova-compute[75234]:       </target>
Nov 29 03:22:47 user nova-compute[75234]:       <alias name='serial0'/>
Nov 29 03:22:47 user nova-compute[75234]:     </serial>
Nov 29 03:22:47 user nova-compute[75234]:     <console type='pty' tty='/dev/pts/3'>
Nov 29 03:22:47 user nova-compute[75234]:       <source path='/dev/pts/3'/>
Nov 29 03:22:47 user nova-compute[75234]:       <log file='/opt/stack/data/nova/instances/946b919b-39ae-4d23-aefd-01952f4b40b6/console.log' append='off'/>
Nov 29 03:22:47 user nova-compute[75234]:       <target type='serial' port='0'/>
Nov 29 03:22:47 user nova-compute[75234]:       <alias name='serial0'/>
Nov 29 03:22:47 user nova-compute[75234]:     </console>
Nov 29 03:22:47 user nova-compute[75234]:     <input type='mouse' bus='ps2'>
Nov 29 03:22:47 user nova-compute[75234]:       <alias name='input0'/>
Nov 29 03:22:47 user nova-compute[75234]:     </input>
Nov 29 03:22:47 user nova-compute[75234]:     <input type='keyboard' bus='ps2'>
Nov 29 03:22:47 user nova-compute[75234]:       <alias name='input1'/>
Nov 29 03:22:47 user nova-compute[75234]:     </input>
Nov 29 03:22:47 user nova-compute[75234]:     <graphics type='vnc' port='5903' autoport='yes' listen='0.0.0.0'>
Nov 29 03:22:47 user nova-compute[75234]:       <listen type='address' address='0.0.0.0'/>
Nov 29 03:22:47 user nova-compute[75234]:     </graphics>
Nov 29 03:22:47 user nova-compute[75234]:     <audio id='1' type='none'/>
Nov 29 03:22:47 user nova-compute[75234]:     <video>
Nov 29 03:22:47 user nova-compute[75234]:       <model type='virtio' heads='1' primary='yes'/>
Nov 29 03:22:47 user nova-compute[75234]:       <alias name='video0'/>
Nov 29 03:22:47 user nova-compute[75234]:       <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
Nov 29 03:22:47 user nova-compute[75234]:     </video>
Nov 29 03:22:47 user nova-compute[75234]:     <memballoon model='virtio'>
Nov 29 03:22:47 user nova-compute[75234]:       <stats period='10'/>
Nov 29 03:22:47 user nova-compute[75234]:       <alias name='balloon0'/>
Nov 29 03:22:47 user nova-compute[75234]:       <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/>
Nov 29 03:22:47 user nova-compute[75234]:     </memballoon>
Nov 29 03:22:47 user nova-compute[75234]:     <rng model='virtio'>
Nov 29 03:22:47 user nova-compute[75234]:       <backend model='random'>/dev/urandom</backend>
Nov 29 03:22:47 user nova-compute[75234]:       <alias name='rng0'/>
Nov 29 03:22:47 user nova-compute[75234]:       <address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0'/>
Nov 29 03:22:47 user nova-compute[75234]:     </rng>
Nov 29 03:22:47 user nova-compute[75234]:   </devices>
Nov 29 03:22:47 user nova-compute[75234]:   <seclabel type='dynamic' model='apparmor' relabel='yes'>
Nov 29 03:22:47 user nova-compute[75234]:     <label>libvirt-946b919b-39ae-4d23-aefd-01952f4b40b6</label>
Nov 29 03:22:47 user nova-compute[75234]:     <imagelabel>libvirt-946b919b-39ae-4d23-aefd-01952f4b40b6</imagelabel>
Nov 29 03:22:47 user nova-compute[75234]:   </seclabel>
Nov 29 03:22:47 user nova-compute[75234]:   <seclabel type='dynamic' model='dac' relabel='yes'>
Nov 29 03:22:47 user nova-compute[75234]:     <label>+64055:+108</label>
Nov 29 03:22:47 user nova-compute[75234]:     <imagelabel>+64055:+108</imagelabel>
Nov 29 03:22:47 user nova-compute[75234]:   </seclabel>
Nov 29 03:22:47 user nova-compute[75234]: </domain>
Nov 29 03:22:47 user nova-compute[75234]:  {{(pid=75234) get_interface_by_cfg /opt/stack/nova/nova/virt/libvirt/guest.py:282}}
Nov 29 03:22:47 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-1f033c13-c772-4284-adad-9d92f9362260 tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] Successfully detached device tap501886e8-de from instance 946b919b-39ae-4d23-aefd-01952f4b40b6 from the persistent domain config.
Nov 29 03:22:47 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-1f033c13-c772-4284-adad-9d92f9362260 tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] (1/8): Attempting to detach device tap501886e8-de with device alias net1 from instance 946b919b-39ae-4d23-aefd-01952f4b40b6 from the live domain config. {{(pid=75234) _detach_from_live_with_retry /opt/stack/nova/nova/virt/libvirt/driver.py:2523}}
Nov 29 03:22:47 user nova-compute[75234]: DEBUG nova.virt.libvirt.guest [None req-1f033c13-c772-4284-adad-9d92f9362260 tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] detach device xml: <interface type="ethernet">
Nov 29 03:22:47 user nova-compute[75234]:   <mac address="fa:16:3e:d3:09:2e"/>
Nov 29 03:22:47 user nova-compute[75234]:   <model type="virtio"/>
Nov 29 03:22:47 user nova-compute[75234]:   <mtu size="1442"/>
Nov 29 03:22:47 user nova-compute[75234]:   <target dev="tap501886e8-de"/>
Nov 29 03:22:47 user nova-compute[75234]: </interface>
Nov 29 03:22:47 user nova-compute[75234]:  {{(pid=75234) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Nov 29 03:22:47 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-f021bf2f-61ef-47a5-b59b-cb1386125d64 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json" returned: 0 in 0.241s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:22:47 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-f021bf2f-61ef-47a5-b59b-cb1386125d64 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918,backing_fmt=raw /opt/stack/data/nova/instances/snapshots/tmpz14i41gx/950fb7560fc94ae49a695ac489d92047.delta 1073741824 {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:22:47 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-f021bf2f-61ef-47a5-b59b-cb1386125d64 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918,backing_fmt=raw /opt/stack/data/nova/instances/snapshots/tmpz14i41gx/950fb7560fc94ae49a695ac489d92047.delta 1073741824" returned: 0 in 0.190s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:22:47 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-f021bf2f-61ef-47a5-b59b-cb1386125d64 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] [instance: 65d48c93-b6da-4f51-9d5a-0676dda62a60] Quiescing instance not available: QEMU guest agent is not enabled.
Nov 29 03:22:47 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:22:47 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:22:48 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:22:48 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-1f033c13-c772-4284-adad-9d92f9362260 tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] Start waiting for the detach event from libvirt for device tap501886e8-de with device alias net1 for instance 946b919b-39ae-4d23-aefd-01952f4b40b6 {{(pid=75234) _detach_from_live_and_wait_for_event /opt/stack/nova/nova/virt/libvirt/driver.py:2599}}
Nov 29 03:22:48 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:22:48 user nova-compute[75234]: DEBUG nova.compute.manager [req-ad2a1be9-1596-4e1d-88f9-d766093c12be req-b585d0ab-aec8-4eed-897b-1da960b7624c service nova] [instance: 946b919b-39ae-4d23-aefd-01952f4b40b6] Received event network-vif-unplugged-501886e8-de21-4108-86d4-d7a9138f8057 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:22:48 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-ad2a1be9-1596-4e1d-88f9-d766093c12be req-b585d0ab-aec8-4eed-897b-1da960b7624c service nova] Acquiring lock "946b919b-39ae-4d23-aefd-01952f4b40b6-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:22:48 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-ad2a1be9-1596-4e1d-88f9-d766093c12be req-b585d0ab-aec8-4eed-897b-1da960b7624c service nova] Lock "946b919b-39ae-4d23-aefd-01952f4b40b6-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:22:48 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-ad2a1be9-1596-4e1d-88f9-d766093c12be req-b585d0ab-aec8-4eed-897b-1da960b7624c service nova] Lock "946b919b-39ae-4d23-aefd-01952f4b40b6-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:22:48 user nova-compute[75234]: DEBUG nova.compute.manager [req-ad2a1be9-1596-4e1d-88f9-d766093c12be req-b585d0ab-aec8-4eed-897b-1da960b7624c service nova] [instance: 946b919b-39ae-4d23-aefd-01952f4b40b6] No waiting events found dispatching network-vif-unplugged-501886e8-de21-4108-86d4-d7a9138f8057 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:22:48 user nova-compute[75234]: WARNING nova.compute.manager [req-ad2a1be9-1596-4e1d-88f9-d766093c12be req-b585d0ab-aec8-4eed-897b-1da960b7624c service nova] [instance: 946b919b-39ae-4d23-aefd-01952f4b40b6] Received unexpected event network-vif-unplugged-501886e8-de21-4108-86d4-d7a9138f8057 for instance with vm_state active and task_state None.
Nov 29 03:22:48 user nova-compute[75234]: DEBUG nova.virt.libvirt.guest [None req-f021bf2f-61ef-47a5-b59b-cb1386125d64 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] COPY block job progress, current cursor: 0 final cursor: 75497472 {{(pid=75234) is_job_complete /opt/stack/nova/nova/virt/libvirt/guest.py:846}}
Nov 29 03:22:49 user nova-compute[75234]: DEBUG nova.virt.libvirt.guest [None req-f021bf2f-61ef-47a5-b59b-cb1386125d64 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] COPY block job progress, current cursor: 21626880 final cursor: 75497472 {{(pid=75234) is_job_complete /opt/stack/nova/nova/virt/libvirt/guest.py:846}}
Nov 29 03:22:49 user nova-compute[75234]: DEBUG nova.virt.libvirt.guest [None req-f021bf2f-61ef-47a5-b59b-cb1386125d64 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] COPY block job progress, current cursor: 75497472 final cursor: 75497472 {{(pid=75234) is_job_complete /opt/stack/nova/nova/virt/libvirt/guest.py:846}}
Nov 29 03:22:49 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-f021bf2f-61ef-47a5-b59b-cb1386125d64 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] [instance: 65d48c93-b6da-4f51-9d5a-0676dda62a60] Skipping quiescing instance: QEMU guest agent is not enabled.
Nov 29 03:22:49 user nova-compute[75234]: DEBUG nova.privsep.utils [None req-f021bf2f-61ef-47a5-b59b-cb1386125d64 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Path '/opt/stack/data/nova/instances' supports direct I/O {{(pid=75234) supports_direct_io /opt/stack/nova/nova/privsep/utils.py:63}}
Nov 29 03:22:49 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-f021bf2f-61ef-47a5-b59b-cb1386125d64 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Running cmd (subprocess): qemu-img convert -t none -O qcow2 -f qcow2 /opt/stack/data/nova/instances/snapshots/tmpz14i41gx/950fb7560fc94ae49a695ac489d92047.delta /opt/stack/data/nova/instances/snapshots/tmpz14i41gx/950fb7560fc94ae49a695ac489d92047 {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:22:50 user nova-compute[75234]: DEBUG nova.compute.manager [req-10eca90d-a992-47a6-b36c-8d7064ac2535 req-de5e3989-c9cd-411e-967e-4c326802bca5 service nova] [instance: 946b919b-39ae-4d23-aefd-01952f4b40b6] Received event network-vif-plugged-501886e8-de21-4108-86d4-d7a9138f8057 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:22:50 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-10eca90d-a992-47a6-b36c-8d7064ac2535 req-de5e3989-c9cd-411e-967e-4c326802bca5 service nova] Acquiring lock "946b919b-39ae-4d23-aefd-01952f4b40b6-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:22:50 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-10eca90d-a992-47a6-b36c-8d7064ac2535 req-de5e3989-c9cd-411e-967e-4c326802bca5 service nova] Lock "946b919b-39ae-4d23-aefd-01952f4b40b6-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:22:50 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-10eca90d-a992-47a6-b36c-8d7064ac2535 req-de5e3989-c9cd-411e-967e-4c326802bca5 service nova] Lock "946b919b-39ae-4d23-aefd-01952f4b40b6-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:22:50 user nova-compute[75234]: DEBUG nova.compute.manager [req-10eca90d-a992-47a6-b36c-8d7064ac2535 req-de5e3989-c9cd-411e-967e-4c326802bca5 service nova] [instance: 946b919b-39ae-4d23-aefd-01952f4b40b6] No waiting events found dispatching network-vif-plugged-501886e8-de21-4108-86d4-d7a9138f8057 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:22:50 user nova-compute[75234]: WARNING nova.compute.manager [req-10eca90d-a992-47a6-b36c-8d7064ac2535 req-de5e3989-c9cd-411e-967e-4c326802bca5 service nova] [instance: 946b919b-39ae-4d23-aefd-01952f4b40b6] Received unexpected event network-vif-plugged-501886e8-de21-4108-86d4-d7a9138f8057 for instance with vm_state active and task_state None.
Nov 29 03:22:50 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-f021bf2f-61ef-47a5-b59b-cb1386125d64 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] CMD "qemu-img convert -t none -O qcow2 -f qcow2 /opt/stack/data/nova/instances/snapshots/tmpz14i41gx/950fb7560fc94ae49a695ac489d92047.delta /opt/stack/data/nova/instances/snapshots/tmpz14i41gx/950fb7560fc94ae49a695ac489d92047" returned: 0 in 0.614s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:22:50 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-f021bf2f-61ef-47a5-b59b-cb1386125d64 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] [instance: 65d48c93-b6da-4f51-9d5a-0676dda62a60] Snapshot extracted, beginning image upload
Nov 29 03:22:51 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:22:52 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-f021bf2f-61ef-47a5-b59b-cb1386125d64 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] [instance: 65d48c93-b6da-4f51-9d5a-0676dda62a60] Snapshot image upload complete
Nov 29 03:22:52 user nova-compute[75234]: INFO nova.compute.manager [None req-f021bf2f-61ef-47a5-b59b-cb1386125d64 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] [instance: 65d48c93-b6da-4f51-9d5a-0676dda62a60] Took 5.95 seconds to snapshot the instance on the hypervisor.
Nov 29 03:22:56 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:22:57 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:23:00 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:23:01 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:23:01 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:23:02 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:23:03 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager.update_available_resource {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:23:03 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:23:03 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:23:03 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:23:03 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Auditing locally available compute resources for user (node: user) {{(pid=75234) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:907}}
Nov 29 03:23:03 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] skipping disk /dev/sdb (vda) as it is a volume {{(pid=75234) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11213}}
Nov 29 03:23:03 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/13e5da64-9adc-4479-81c7-d329e9b5f08f/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:23:03 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/13e5da64-9adc-4479-81c7-d329e9b5f08f/disk --force-share --output=json" returned: 0 in 0.181s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:23:03 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/13e5da64-9adc-4479-81c7-d329e9b5f08f/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:23:04 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/13e5da64-9adc-4479-81c7-d329e9b5f08f/disk --force-share --output=json" returned: 0 in 0.166s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:23:04 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/1df140f1-fbd9-4b5d-b356-d08b7e908f64/disk.rescue --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:23:04 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/1df140f1-fbd9-4b5d-b356-d08b7e908f64/disk.rescue --force-share --output=json" returned: 0 in 0.186s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:23:04 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/1df140f1-fbd9-4b5d-b356-d08b7e908f64/disk.rescue --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:23:04 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/1df140f1-fbd9-4b5d-b356-d08b7e908f64/disk.rescue --force-share --output=json" returned: 0 in 0.189s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:23:04 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] skipping disk /opt/stack/data/nova/instances/1df140f1-fbd9-4b5d-b356-d08b7e908f64/disk (vdb) as it is a volume {{(pid=75234) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11213}}
Nov 29 03:23:04 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f6bc212d-85cd-479c-92ae-112c9f0101bc/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:23:04 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f6bc212d-85cd-479c-92ae-112c9f0101bc/disk --force-share --output=json" returned: 0 in 0.170s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:23:04 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f6bc212d-85cd-479c-92ae-112c9f0101bc/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:23:04 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f6bc212d-85cd-479c-92ae-112c9f0101bc/disk --force-share --output=json" returned: 0 in 0.172s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:23:04 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/65d48c93-b6da-4f51-9d5a-0676dda62a60/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:23:04 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/65d48c93-b6da-4f51-9d5a-0676dda62a60/disk --force-share --output=json" returned: 0 in 0.168s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:23:04 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/65d48c93-b6da-4f51-9d5a-0676dda62a60/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:23:05 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/65d48c93-b6da-4f51-9d5a-0676dda62a60/disk --force-share --output=json" returned: 0 in 0.195s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:23:05 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/060c15ec-9012-471a-984a-14c9fa6ce23a/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:23:05 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/060c15ec-9012-471a-984a-14c9fa6ce23a/disk --force-share --output=json" returned: 0 in 0.176s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:23:05 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/060c15ec-9012-471a-984a-14c9fa6ce23a/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:23:05 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/060c15ec-9012-471a-984a-14c9fa6ce23a/disk --force-share --output=json" returned: 0 in 0.171s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:23:05 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/946b919b-39ae-4d23-aefd-01952f4b40b6/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:23:05 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/946b919b-39ae-4d23-aefd-01952f4b40b6/disk --force-share --output=json" returned: 0 in 0.200s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:23:05 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/946b919b-39ae-4d23-aefd-01952f4b40b6/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:23:05 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/946b919b-39ae-4d23-aefd-01952f4b40b6/disk --force-share --output=json" returned: 0 in 0.171s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:23:05 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a082c96e-e384-40b6-bcc8-fc5e0e104669/disk.rescue --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:23:06 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a082c96e-e384-40b6-bcc8-fc5e0e104669/disk.rescue --force-share --output=json" returned: 0 in 0.179s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:23:06 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a082c96e-e384-40b6-bcc8-fc5e0e104669/disk.rescue --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:23:06 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a082c96e-e384-40b6-bcc8-fc5e0e104669/disk.rescue --force-share --output=json" returned: 0 in 0.166s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:23:06 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a082c96e-e384-40b6-bcc8-fc5e0e104669/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:23:06 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:23:06 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a082c96e-e384-40b6-bcc8-fc5e0e104669/disk --force-share --output=json" returned: 0 in 0.215s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:23:06 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a082c96e-e384-40b6-bcc8-fc5e0e104669/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:23:06 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a082c96e-e384-40b6-bcc8-fc5e0e104669/disk --force-share --output=json" returned: 0 in 0.167s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:23:06 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/21e9c6cd-c39f-41c0-8768-5dd64589c3f3/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:23:06 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/21e9c6cd-c39f-41c0-8768-5dd64589c3f3/disk --force-share --output=json" returned: 0 in 0.185s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:23:06 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/21e9c6cd-c39f-41c0-8768-5dd64589c3f3/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:23:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/21e9c6cd-c39f-41c0-8768-5dd64589c3f3/disk --force-share --output=json" returned: 0 in 0.173s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:23:07 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:23:07 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:23:07 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:23:07 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Hypervisor/Node resource view: name=user free_ram=7710MB free_disk=25.16680145263672GB free_vcpus=3 pci_devices=[{"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}] {{(pid=75234) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1106}}
Nov 29 03:23:07 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:23:07 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:23:07 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance 21e9c6cd-c39f-41c0-8768-5dd64589c3f3 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:23:07 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance 13e5da64-9adc-4479-81c7-d329e9b5f08f actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:23:07 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance a082c96e-e384-40b6-bcc8-fc5e0e104669 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:23:07 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance 1df140f1-fbd9-4b5d-b356-d08b7e908f64 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:23:07 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance af699bc2-9daa-4b5a-9b4a-ad92fe857d70 actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 2048, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:23:07 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance 946b919b-39ae-4d23-aefd-01952f4b40b6 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:23:07 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance 65d48c93-b6da-4f51-9d5a-0676dda62a60 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:23:07 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance f6bc212d-85cd-479c-92ae-112c9f0101bc actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:23:07 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance 060c15ec-9012-471a-984a-14c9fa6ce23a actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:23:07 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Total usable vcpus: 12, total allocated vcpus: 9 {{(pid=75234) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1129}}
Nov 29 03:23:07 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Final resource view: name=user phys_ram=16011MB used_ram=6656MB phys_disk=40GB used_disk=8GB total_vcpus=12 used_vcpus=9 pci_stats=[] {{(pid=75234) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1138}}
Nov 29 03:23:07 user nova-compute[75234]: DEBUG nova.compute.provider_tree [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Inventory has not changed in ProviderTree for provider: 4d291220-7c37-4839-a0e3-b23c66502b74 {{(pid=75234) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Nov 29 03:23:07 user nova-compute[75234]: DEBUG nova.scheduler.client.report [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Inventory has not changed for provider 4d291220-7c37-4839-a0e3-b23c66502b74 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=75234) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Nov 29 03:23:07 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Compute_service record updated for user:user {{(pid=75234) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:1067}}
Nov 29 03:23:07 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.447s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:23:08 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-1f033c13-c772-4284-adad-9d92f9362260 tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] Waiting for libvirt event about the detach of device tap501886e8-de with device alias net1 from instance 946b919b-39ae-4d23-aefd-01952f4b40b6 is timed out.
Nov 29 03:23:08 user nova-compute[75234]: DEBUG nova.virt.libvirt.guest [None req-1f033c13-c772-4284-adad-9d92f9362260 tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] looking for interface given config: <interface type="ethernet"><mac address="fa:16:3e:d3:09:2e"/><model type="virtio"/><mtu size="1442"/><target dev="tap501886e8-de"/></interface> {{(pid=75234) get_interface_by_cfg /opt/stack/nova/nova/virt/libvirt/guest.py:257}}
Nov 29 03:23:08 user nova-compute[75234]: DEBUG nova.virt.libvirt.guest [None req-1f033c13-c772-4284-adad-9d92f9362260 tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] interface for config: <interface type="ethernet"><mac address="fa:16:3e:d3:09:2e"/><model type="virtio"/><mtu size="1442"/><target dev="tap501886e8-de"/></interface>not found in domain: <domain type='kvm' id='48'>
Nov 29 03:23:08 user nova-compute[75234]:   <name>instance-00000023</name>
Nov 29 03:23:08 user nova-compute[75234]:   <uuid>946b919b-39ae-4d23-aefd-01952f4b40b6</uuid>
Nov 29 03:23:08 user nova-compute[75234]:   <metadata>
Nov 29 03:23:08 user nova-compute[75234]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1" xmlns:instance="http://openstack.org/xmlns/libvirt/nova/1.1">
Nov 29 03:23:08 user nova-compute[75234]:   <nova:package version="0.0.0"/>
Nov 29 03:23:08 user nova-compute[75234]:   <nova:name>tempest-device-tagging-server-1777937262</nova:name>
Nov 29 03:23:08 user nova-compute[75234]:   <nova:creationTime>2023-11-29 03:22:09</nova:creationTime>
Nov 29 03:23:08 user nova-compute[75234]:   <nova:flavor name="m1.tiny">
Nov 29 03:23:08 user nova-compute[75234]:     <nova:memory>512</nova:memory>
Nov 29 03:23:08 user nova-compute[75234]:     <nova:disk>1</nova:disk>
Nov 29 03:23:08 user nova-compute[75234]:     <nova:swap>0</nova:swap>
Nov 29 03:23:08 user nova-compute[75234]:     <nova:ephemeral>0</nova:ephemeral>
Nov 29 03:23:08 user nova-compute[75234]:     <nova:vcpus>1</nova:vcpus>
Nov 29 03:23:08 user nova-compute[75234]:   </nova:flavor>
Nov 29 03:23:08 user nova-compute[75234]:   <nova:owner>
Nov 29 03:23:08 user nova-compute[75234]:     <nova:user uuid="522c3c107614404fbe3af8c38fdd189e">tempest-TaggedAttachmentsTest-399278293-project-member</nova:user>
Nov 29 03:23:08 user nova-compute[75234]:     <nova:project uuid="e28b5baf29b14aa4b2b98cb1b615705f">tempest-TaggedAttachmentsTest-399278293</nova:project>
Nov 29 03:23:08 user nova-compute[75234]:   </nova:owner>
Nov 29 03:23:08 user nova-compute[75234]:   <nova:root type="image" uuid="96c30ba7-9b73-4a30-8ba4-7138879573b7"/>
Nov 29 03:23:08 user nova-compute[75234]:   <nova:ports>
Nov 29 03:23:08 user nova-compute[75234]:     <nova:port uuid="e1f71ce6-58f3-4665-afc5-f92d1b6801c5">
Nov 29 03:23:08 user nova-compute[75234]:       <nova:ip type="fixed" address="10.0.0.4" ipVersion="4"/>
Nov 29 03:23:08 user nova-compute[75234]:     </nova:port>
Nov 29 03:23:08 user nova-compute[75234]:     <nova:port uuid="501886e8-de21-4108-86d4-d7a9138f8057">
Nov 29 03:23:08 user nova-compute[75234]:       <nova:ip type="fixed" address="10.10.10.149" ipVersion="4"/>
Nov 29 03:23:08 user nova-compute[75234]:     </nova:port>
Nov 29 03:23:08 user nova-compute[75234]:   </nova:ports>
Nov 29 03:23:08 user nova-compute[75234]: </nova:instance>
Nov 29 03:23:08 user nova-compute[75234]:   </metadata>
Nov 29 03:23:08 user nova-compute[75234]:   <memory unit='KiB'>524288</memory>
Nov 29 03:23:08 user nova-compute[75234]:   <currentMemory unit='KiB'>524288</currentMemory>
Nov 29 03:23:08 user nova-compute[75234]:   <vcpu placement='static'>1</vcpu>
Nov 29 03:23:08 user nova-compute[75234]:   <resource>
Nov 29 03:23:08 user nova-compute[75234]:     <partition>/machine</partition>
Nov 29 03:23:08 user nova-compute[75234]:   </resource>
Nov 29 03:23:08 user nova-compute[75234]:   <sysinfo type='smbios'>
Nov 29 03:23:08 user nova-compute[75234]:     <system>
Nov 29 03:23:08 user nova-compute[75234]:       <entry name='manufacturer'>OpenStack Foundation</entry>
Nov 29 03:23:08 user nova-compute[75234]:       <entry name='product'>OpenStack Nova</entry>
Nov 29 03:23:08 user nova-compute[75234]:       <entry name='version'>0.0.0</entry>
Nov 29 03:23:08 user nova-compute[75234]:       <entry name='serial'>946b919b-39ae-4d23-aefd-01952f4b40b6</entry>
Nov 29 03:23:08 user nova-compute[75234]:       <entry name='uuid'>946b919b-39ae-4d23-aefd-01952f4b40b6</entry>
Nov 29 03:23:08 user nova-compute[75234]:       <entry name='family'>Virtual Machine</entry>
Nov 29 03:23:08 user nova-compute[75234]:     </system>
Nov 29 03:23:08 user nova-compute[75234]:   </sysinfo>
Nov 29 03:23:08 user nova-compute[75234]:   <os>
Nov 29 03:23:08 user nova-compute[75234]:     <type arch='x86_64' machine='pc-i440fx-6.2'>hvm</type>
Nov 29 03:23:08 user nova-compute[75234]:     <boot dev='hd'/>
Nov 29 03:23:08 user nova-compute[75234]:     <smbios mode='sysinfo'/>
Nov 29 03:23:08 user nova-compute[75234]:   </os>
Nov 29 03:23:08 user nova-compute[75234]:   <features>
Nov 29 03:23:08 user nova-compute[75234]:     <acpi/>
Nov 29 03:23:08 user nova-compute[75234]:     <apic/>
Nov 29 03:23:08 user nova-compute[75234]:     <vmcoreinfo state='on'/>
Nov 29 03:23:08 user nova-compute[75234]:   </features>
Nov 29 03:23:08 user nova-compute[75234]:   <cpu mode='custom' match='exact' check='full'>
Nov 29 03:23:08 user nova-compute[75234]:     <model fallback='forbid'>Nehalem</model>
Nov 29 03:23:08 user nova-compute[75234]:     <topology sockets='1' dies='1' cores='1' threads='1'/>
Nov 29 03:23:08 user nova-compute[75234]:     <feature policy='require' name='vme'/>
Nov 29 03:23:08 user nova-compute[75234]:     <feature policy='require' name='x2apic'/>
Nov 29 03:23:08 user nova-compute[75234]:     <feature policy='require' name='hypervisor'/>
Nov 29 03:23:08 user nova-compute[75234]:   </cpu>
Nov 29 03:23:08 user nova-compute[75234]:   <clock offset='utc'>
Nov 29 03:23:08 user nova-compute[75234]:     <timer name='pit' tickpolicy='delay'/>
Nov 29 03:23:08 user nova-compute[75234]:     <timer name='rtc' tickpolicy='catchup'/>
Nov 29 03:23:08 user nova-compute[75234]:     <timer name='hpet' present='no'/>
Nov 29 03:23:08 user nova-compute[75234]:   </clock>
Nov 29 03:23:08 user nova-compute[75234]:   <on_poweroff>destroy</on_poweroff>
Nov 29 03:23:08 user nova-compute[75234]:   <on_reboot>restart</on_reboot>
Nov 29 03:23:08 user nova-compute[75234]:   <on_crash>destroy</on_crash>
Nov 29 03:23:08 user nova-compute[75234]:   <devices>
Nov 29 03:23:08 user nova-compute[75234]:     <emulator>/usr/bin/qemu-system-x86_64</emulator>
Nov 29 03:23:08 user nova-compute[75234]:     <disk type='file' device='disk'>
Nov 29 03:23:08 user nova-compute[75234]:       <driver name='qemu' type='qcow2' cache='none'/>
Nov 29 03:23:08 user nova-compute[75234]:       <source file='/opt/stack/data/nova/instances/946b919b-39ae-4d23-aefd-01952f4b40b6/disk' index='2'/>
Nov 29 03:23:08 user nova-compute[75234]:       <backingStore type='file' index='3'>
Nov 29 03:23:08 user nova-compute[75234]:         <format type='raw'/>
Nov 29 03:23:08 user nova-compute[75234]:         <source file='/opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918'/>
Nov 29 03:23:08 user nova-compute[75234]:         <backingStore/>
Nov 29 03:23:08 user nova-compute[75234]:       </backingStore>
Nov 29 03:23:08 user nova-compute[75234]:       <target dev='vda' bus='virtio'/>
Nov 29 03:23:08 user nova-compute[75234]:       <alias name='virtio-disk0'/>
Nov 29 03:23:08 user nova-compute[75234]:       <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/>
Nov 29 03:23:08 user nova-compute[75234]:     </disk>
Nov 29 03:23:08 user nova-compute[75234]:     <disk type='file' device='cdrom'>
Nov 29 03:23:08 user nova-compute[75234]:       <driver name='qemu' type='raw' cache='none'/>
Nov 29 03:23:08 user nova-compute[75234]:       <source file='/opt/stack/data/nova/instances/946b919b-39ae-4d23-aefd-01952f4b40b6/disk.config' index='1'/>
Nov 29 03:23:08 user nova-compute[75234]:       <backingStore/>
Nov 29 03:23:08 user nova-compute[75234]:       <target dev='hda' bus='ide'/>
Nov 29 03:23:08 user nova-compute[75234]:       <readonly/>
Nov 29 03:23:08 user nova-compute[75234]:       <alias name='ide0-0-0'/>
Nov 29 03:23:08 user nova-compute[75234]:       <address type='drive' controller='0' bus='0' target='0' unit='0'/>
Nov 29 03:23:08 user nova-compute[75234]:     </disk>
Nov 29 03:23:08 user nova-compute[75234]:     <controller type='usb' index='0' model='none'>
Nov 29 03:23:08 user nova-compute[75234]:       <alias name='usb'/>
Nov 29 03:23:08 user nova-compute[75234]:     </controller>
Nov 29 03:23:08 user nova-compute[75234]:     <controller type='pci' index='0' model='pci-root'>
Nov 29 03:23:08 user nova-compute[75234]:       <alias name='pci.0'/>
Nov 29 03:23:08 user nova-compute[75234]:     </controller>
Nov 29 03:23:08 user nova-compute[75234]:     <controller type='ide' index='0'>
Nov 29 03:23:08 user nova-compute[75234]:       <alias name='ide'/>
Nov 29 03:23:08 user nova-compute[75234]:       <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x1'/>
Nov 29 03:23:08 user nova-compute[75234]:     </controller>
Nov 29 03:23:08 user nova-compute[75234]:     <interface type='ethernet'>
Nov 29 03:23:08 user nova-compute[75234]:       <mac address='fa:16:3e:81:ca:ab'/>
Nov 29 03:23:08 user nova-compute[75234]:       <target dev='tape1f71ce6-58'/>
Nov 29 03:23:08 user nova-compute[75234]:       <model type='virtio'/>
Nov 29 03:23:08 user nova-compute[75234]:       <mtu size='1442'/>
Nov 29 03:23:08 user nova-compute[75234]:       <alias name='net0'/>
Nov 29 03:23:08 user nova-compute[75234]:       <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>
Nov 29 03:23:08 user nova-compute[75234]:     </interface>
Nov 29 03:23:08 user nova-compute[75234]:     <serial type='pty'>
Nov 29 03:23:08 user nova-compute[75234]:       <source path='/dev/pts/3'/>
Nov 29 03:23:08 user nova-compute[75234]:       <log file='/opt/stack/data/nova/instances/946b919b-39ae-4d23-aefd-01952f4b40b6/console.log' append='off'/>
Nov 29 03:23:08 user nova-compute[75234]:       <target type='isa-serial' port='0'>
Nov 29 03:23:08 user nova-compute[75234]:         <model name='isa-serial'/>
Nov 29 03:23:08 user nova-compute[75234]:       </target>
Nov 29 03:23:08 user nova-compute[75234]:       <alias name='serial0'/>
Nov 29 03:23:08 user nova-compute[75234]:     </serial>
Nov 29 03:23:08 user nova-compute[75234]:     <console type='pty' tty='/dev/pts/3'>
Nov 29 03:23:08 user nova-compute[75234]:       <source path='/dev/pts/3'/>
Nov 29 03:23:08 user nova-compute[75234]:       <log file='/opt/stack/data/nova/instances/946b919b-39ae-4d23-aefd-01952f4b40b6/console.log' append='off'/>
Nov 29 03:23:08 user nova-compute[75234]:       <target type='serial' port='0'/>
Nov 29 03:23:08 user nova-compute[75234]:       <alias name='serial0'/>
Nov 29 03:23:08 user nova-compute[75234]:     </console>
Nov 29 03:23:08 user nova-compute[75234]:     <input type='mouse' bus='ps2'>
Nov 29 03:23:08 user nova-compute[75234]:       <alias name='input0'/>
Nov 29 03:23:08 user nova-compute[75234]:     </input>
Nov 29 03:23:08 user nova-compute[75234]:     <input type='keyboard' bus='ps2'>
Nov 29 03:23:08 user nova-compute[75234]:       <alias name='input1'/>
Nov 29 03:23:08 user nova-compute[75234]:     </input>
Nov 29 03:23:08 user nova-compute[75234]:     <graphics type='vnc' port='5903' autoport='yes' listen='0.0.0.0'>
Nov 29 03:23:08 user nova-compute[75234]:       <listen type='address' address='0.0.0.0'/>
Nov 29 03:23:08 user nova-compute[75234]:     </graphics>
Nov 29 03:23:08 user nova-compute[75234]:     <audio id='1' type='none'/>
Nov 29 03:23:08 user nova-compute[75234]:     <video>
Nov 29 03:23:08 user nova-compute[75234]:       <model type='virtio' heads='1' primary='yes'/>
Nov 29 03:23:08 user nova-compute[75234]:       <alias name='video0'/>
Nov 29 03:23:08 user nova-compute[75234]:       <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
Nov 29 03:23:08 user nova-compute[75234]:     </video>
Nov 29 03:23:08 user nova-compute[75234]:     <memballoon model='virtio'>
Nov 29 03:23:08 user nova-compute[75234]:       <stats period='10'/>
Nov 29 03:23:08 user nova-compute[75234]:       <alias name='balloon0'/>
Nov 29 03:23:08 user nova-compute[75234]:       <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/>
Nov 29 03:23:08 user nova-compute[75234]:     </memballoon>
Nov 29 03:23:08 user nova-compute[75234]:     <rng model='virtio'>
Nov 29 03:23:08 user nova-compute[75234]:       <backend model='random'>/dev/urandom</backend>
Nov 29 03:23:08 user nova-compute[75234]:       <alias name='rng0'/>
Nov 29 03:23:08 user nova-compute[75234]:       <address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0'/>
Nov 29 03:23:08 user nova-compute[75234]:     </rng>
Nov 29 03:23:08 user nova-compute[75234]:   </devices>
Nov 29 03:23:08 user nova-compute[75234]:   <seclabel type='dynamic' model='apparmor' relabel='yes'>
Nov 29 03:23:08 user nova-compute[75234]:     <label>libvirt-946b919b-39ae-4d23-aefd-01952f4b40b6</label>
Nov 29 03:23:08 user nova-compute[75234]:     <imagelabel>libvirt-946b919b-39ae-4d23-aefd-01952f4b40b6</imagelabel>
Nov 29 03:23:08 user nova-compute[75234]:   </seclabel>
Nov 29 03:23:08 user nova-compute[75234]:   <seclabel type='dynamic' model='dac' relabel='yes'>
Nov 29 03:23:08 user nova-compute[75234]:     <label>+64055:+108</label>
Nov 29 03:23:08 user nova-compute[75234]:     <imagelabel>+64055:+108</imagelabel>
Nov 29 03:23:08 user nova-compute[75234]:   </seclabel>
Nov 29 03:23:08 user nova-compute[75234]: </domain>
Nov 29 03:23:08 user nova-compute[75234]:  {{(pid=75234) get_interface_by_cfg /opt/stack/nova/nova/virt/libvirt/guest.py:282}}
Nov 29 03:23:08 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-1f033c13-c772-4284-adad-9d92f9362260 tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] Successfully detached device tap501886e8-de from instance 946b919b-39ae-4d23-aefd-01952f4b40b6 from the live domain config.
Nov 29 03:23:08 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-1f033c13-c772-4284-adad-9d92f9362260 tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='True',created_at=2023-11-29T03:20:14Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=InstanceDeviceMetadata,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-1777937262',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-device-tagging-server-1777937262',id=35,image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBJrVo4ngcPAPF4oP6xidv+ykI7O7g9y5APrTr7iXhruQjDIx7zqy1/omzWf7G2p8xg1QAr8Js8NrVgD6XT2ZMw0fSSCLDKoKJk+X3O2rM5aB1leyuLU4v60YiI/3ZRg9Ew==',key_name='tempest-keypair-1887924020',keypairs=<?>,launch_index=0,launched_at=2023-11-29T03:20:21Z,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=<?>,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='e28b5baf29b14aa4b2b98cb1b615705f',ramdisk_id='',reservation_id='r-bfdfrqjg',resources=<?>,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-TaggedAttachmentsTest-399278293',owner_user_name='tempest-TaggedAttachmentsTest-399278293-project-member'},tags=<?>,task_state=None,terminated_at=None,trusted_certs=<?>,updated_at=2023-11-29T03:20:22Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='522c3c107614404fbe3af8c38fdd189e',uuid=946b919b-39ae-4d23-aefd-01952f4b40b6,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "501886e8-de21-4108-86d4-d7a9138f8057", "address": "fa:16:3e:d3:09:2e", "network": {"id": "ca5d0d87-cb5f-4287-8438-a8ecc7d3fd33", "bridge": "br-int", "label": "tempest-tagged-attachments-test-net-1833288811", "subnets": [{"cidr": "10.10.10.0/24", "dns": [], "gateway": {"address": "10.10.10.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.10.10.149", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.10.10.1"}}], "meta": {"injected": false, "tenant_id": "e28b5baf29b14aa4b2b98cb1b615705f", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap501886e8-de", "ovs_interfaceid": "501886e8-de21-4108-86d4-d7a9138f8057", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Nov 29 03:23:08 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-1f033c13-c772-4284-adad-9d92f9362260 tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] Converting VIF {"id": "501886e8-de21-4108-86d4-d7a9138f8057", "address": "fa:16:3e:d3:09:2e", "network": {"id": "ca5d0d87-cb5f-4287-8438-a8ecc7d3fd33", "bridge": "br-int", "label": "tempest-tagged-attachments-test-net-1833288811", "subnets": [{"cidr": "10.10.10.0/24", "dns": [], "gateway": {"address": "10.10.10.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.10.10.149", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.10.10.1"}}], "meta": {"injected": false, "tenant_id": "e28b5baf29b14aa4b2b98cb1b615705f", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap501886e8-de", "ovs_interfaceid": "501886e8-de21-4108-86d4-d7a9138f8057", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:23:08 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-1f033c13-c772-4284-adad-9d92f9362260 tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:d3:09:2e,bridge_name='br-int',has_traffic_filtering=True,id=501886e8-de21-4108-86d4-d7a9138f8057,network=Network(ca5d0d87-cb5f-4287-8438-a8ecc7d3fd33),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap501886e8-de') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:23:08 user nova-compute[75234]: DEBUG os_vif [None req-1f033c13-c772-4284-adad-9d92f9362260 tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] Unplugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:d3:09:2e,bridge_name='br-int',has_traffic_filtering=True,id=501886e8-de21-4108-86d4-d7a9138f8057,network=Network(ca5d0d87-cb5f-4287-8438-a8ecc7d3fd33),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap501886e8-de') {{(pid=75234) unplug /opt/stack/data/venv/lib/python3.10/site-packages/os_vif/__init__.py:109}}
Nov 29 03:23:08 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:23:08 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tap501886e8-de, bridge=br-int, if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:23:08 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:23:08 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:23:08 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:23:08 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbDestroyCommand(_result=None, table=QoS, record=419d6a7a-9b01-4beb-a772-5b69d26cf945) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:23:08 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:23:08 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:23:08 user nova-compute[75234]: INFO os_vif [None req-1f033c13-c772-4284-adad-9d92f9362260 tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] Successfully unplugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:d3:09:2e,bridge_name='br-int',has_traffic_filtering=True,id=501886e8-de21-4108-86d4-d7a9138f8057,network=Network(ca5d0d87-cb5f-4287-8438-a8ecc7d3fd33),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap501886e8-de')
Nov 29 03:23:08 user nova-compute[75234]: DEBUG nova.virt.libvirt.guest [None req-1f033c13-c772-4284-adad-9d92f9362260 tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] set metadata xml: <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Nov 29 03:23:08 user nova-compute[75234]:   <nova:package version="0.0.0"/>
Nov 29 03:23:08 user nova-compute[75234]:   <nova:name>tempest-device-tagging-server-1777937262</nova:name>
Nov 29 03:23:08 user nova-compute[75234]:   <nova:creationTime>2023-11-29 03:23:08</nova:creationTime>
Nov 29 03:23:08 user nova-compute[75234]:   <nova:flavor name="m1.tiny">
Nov 29 03:23:08 user nova-compute[75234]:     <nova:memory>512</nova:memory>
Nov 29 03:23:08 user nova-compute[75234]:     <nova:disk>1</nova:disk>
Nov 29 03:23:08 user nova-compute[75234]:     <nova:swap>0</nova:swap>
Nov 29 03:23:08 user nova-compute[75234]:     <nova:ephemeral>0</nova:ephemeral>
Nov 29 03:23:08 user nova-compute[75234]:     <nova:vcpus>1</nova:vcpus>
Nov 29 03:23:08 user nova-compute[75234]:   </nova:flavor>
Nov 29 03:23:08 user nova-compute[75234]:   <nova:owner>
Nov 29 03:23:08 user nova-compute[75234]:     <nova:user uuid="522c3c107614404fbe3af8c38fdd189e">tempest-TaggedAttachmentsTest-399278293-project-member</nova:user>
Nov 29 03:23:08 user nova-compute[75234]:     <nova:project uuid="e28b5baf29b14aa4b2b98cb1b615705f">tempest-TaggedAttachmentsTest-399278293</nova:project>
Nov 29 03:23:08 user nova-compute[75234]:   </nova:owner>
Nov 29 03:23:08 user nova-compute[75234]:   <nova:root type="image" uuid="96c30ba7-9b73-4a30-8ba4-7138879573b7"/>
Nov 29 03:23:08 user nova-compute[75234]:   <nova:ports>
Nov 29 03:23:08 user nova-compute[75234]:     <nova:port uuid="e1f71ce6-58f3-4665-afc5-f92d1b6801c5">
Nov 29 03:23:08 user nova-compute[75234]:       <nova:ip type="fixed" address="10.0.0.4" ipVersion="4"/>
Nov 29 03:23:08 user nova-compute[75234]:     </nova:port>
Nov 29 03:23:08 user nova-compute[75234]:   </nova:ports>
Nov 29 03:23:08 user nova-compute[75234]: </nova:instance>
Nov 29 03:23:08 user nova-compute[75234]:  {{(pid=75234) set_metadata /opt/stack/nova/nova/virt/libvirt/guest.py:359}}
Nov 29 03:23:08 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:23:08 user nova-compute[75234]: DEBUG nova.compute.manager [req-00008e2d-5dd7-4aa8-9235-ce1cb7bfcd16 req-39c352bb-c139-4de3-af8b-a7b0d98ea775 service nova] [instance: 946b919b-39ae-4d23-aefd-01952f4b40b6] Received event network-vif-deleted-501886e8-de21-4108-86d4-d7a9138f8057 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:23:08 user nova-compute[75234]: INFO nova.compute.manager [req-00008e2d-5dd7-4aa8-9235-ce1cb7bfcd16 req-39c352bb-c139-4de3-af8b-a7b0d98ea775 service nova] [instance: 946b919b-39ae-4d23-aefd-01952f4b40b6] Neutron deleted interface 501886e8-de21-4108-86d4-d7a9138f8057; detaching it from the instance and deleting it from the info cache
Nov 29 03:23:08 user nova-compute[75234]: DEBUG nova.network.neutron [req-00008e2d-5dd7-4aa8-9235-ce1cb7bfcd16 req-39c352bb-c139-4de3-af8b-a7b0d98ea775 service nova] [instance: 946b919b-39ae-4d23-aefd-01952f4b40b6] Updating instance_info_cache with network_info: [{"id": "e1f71ce6-58f3-4665-afc5-f92d1b6801c5", "address": "fa:16:3e:81:ca:ab", "network": {"id": "93ca21e4-138b-4f00-8d1d-b30d63dbe7aa", "bridge": "br-int", "label": "tempest-TaggedAttachmentsTest-1844288661-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.47", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "e28b5baf29b14aa4b2b98cb1b615705f", "mtu": null, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape1f71ce6-58", "ovs_interfaceid": "e1f71ce6-58f3-4665-afc5-f92d1b6801c5", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:23:08 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-1f033c13-c772-4284-adad-9d92f9362260 tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] Acquiring lock "refresh_cache-946b919b-39ae-4d23-aefd-01952f4b40b6" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:23:08 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-1f033c13-c772-4284-adad-9d92f9362260 tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] Acquired lock "refresh_cache-946b919b-39ae-4d23-aefd-01952f4b40b6" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:23:08 user nova-compute[75234]: DEBUG nova.network.neutron [None req-1f033c13-c772-4284-adad-9d92f9362260 tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] [instance: 946b919b-39ae-4d23-aefd-01952f4b40b6] Building network info cache for instance {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2009}}
Nov 29 03:23:08 user nova-compute[75234]: DEBUG nova.objects.instance [req-00008e2d-5dd7-4aa8-9235-ce1cb7bfcd16 req-39c352bb-c139-4de3-af8b-a7b0d98ea775 service nova] Lazy-loading 'system_metadata' on Instance uuid 946b919b-39ae-4d23-aefd-01952f4b40b6 {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:23:08 user nova-compute[75234]: DEBUG nova.objects.instance [req-00008e2d-5dd7-4aa8-9235-ce1cb7bfcd16 req-39c352bb-c139-4de3-af8b-a7b0d98ea775 service nova] Lazy-loading 'flavor' on Instance uuid 946b919b-39ae-4d23-aefd-01952f4b40b6 {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:23:08 user nova-compute[75234]: DEBUG nova.objects.base [req-00008e2d-5dd7-4aa8-9235-ce1cb7bfcd16 req-39c352bb-c139-4de3-af8b-a7b0d98ea775 service nova] Object Instance<946b919b-39ae-4d23-aefd-01952f4b40b6> lazy-loaded attributes: system_metadata,flavor {{(pid=75234) wrapper /opt/stack/nova/nova/objects/base.py:126}}
Nov 29 03:23:08 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [req-00008e2d-5dd7-4aa8-9235-ce1cb7bfcd16 req-39c352bb-c139-4de3-af8b-a7b0d98ea775 service nova] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='True',created_at=2023-11-29T03:20:14Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-1777937262',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-device-tagging-server-1777937262',id=35,image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBJrVo4ngcPAPF4oP6xidv+ykI7O7g9y5APrTr7iXhruQjDIx7zqy1/omzWf7G2p8xg1QAr8Js8NrVgD6XT2ZMw0fSSCLDKoKJk+X3O2rM5aB1leyuLU4v60YiI/3ZRg9Ew==',key_name='tempest-keypair-1887924020',keypairs=<?>,launch_index=0,launched_at=2023-11-29T03:20:21Z,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata=<?>,migration_context=None,new_flavor=None,node='user',numa_topology=<?>,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='e28b5baf29b14aa4b2b98cb1b615705f',ramdisk_id='',reservation_id='r-bfdfrqjg',resources=<?>,root_device_name='/dev/vda',root_gb=1,security_groups=<?>,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-TaggedAttachmentsTest-399278293',owner_user_name='tempest-TaggedAttachmentsTest-399278293-project-member'},tags=<?>,task_state=None,terminated_at=None,trusted_certs=<?>,updated_at=2023-11-29T03:20:22Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='522c3c107614404fbe3af8c38fdd189e',uuid=946b919b-39ae-4d23-aefd-01952f4b40b6,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "501886e8-de21-4108-86d4-d7a9138f8057", "address": "fa:16:3e:d3:09:2e", "network": {"id": "ca5d0d87-cb5f-4287-8438-a8ecc7d3fd33", "bridge": "br-int", "label": "tempest-tagged-attachments-test-net-1833288811", "subnets": [{"cidr": "10.10.10.0/24", "dns": [], "gateway": {"address": "10.10.10.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.10.10.149", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.10.10.1"}}], "meta": {"injected": false, "tenant_id": "e28b5baf29b14aa4b2b98cb1b615705f", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap501886e8-de", "ovs_interfaceid": "501886e8-de21-4108-86d4-d7a9138f8057", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=75234) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Nov 29 03:23:08 user nova-compute[75234]: DEBUG nova.network.os_vif_util [req-00008e2d-5dd7-4aa8-9235-ce1cb7bfcd16 req-39c352bb-c139-4de3-af8b-a7b0d98ea775 service nova] Converting VIF {"id": "501886e8-de21-4108-86d4-d7a9138f8057", "address": "fa:16:3e:d3:09:2e", "network": {"id": "ca5d0d87-cb5f-4287-8438-a8ecc7d3fd33", "bridge": "br-int", "label": "tempest-tagged-attachments-test-net-1833288811", "subnets": [{"cidr": "10.10.10.0/24", "dns": [], "gateway": {"address": "10.10.10.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.10.10.149", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.10.10.1"}}], "meta": {"injected": false, "tenant_id": "e28b5baf29b14aa4b2b98cb1b615705f", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap501886e8-de", "ovs_interfaceid": "501886e8-de21-4108-86d4-d7a9138f8057", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:23:08 user nova-compute[75234]: DEBUG nova.network.os_vif_util [req-00008e2d-5dd7-4aa8-9235-ce1cb7bfcd16 req-39c352bb-c139-4de3-af8b-a7b0d98ea775 service nova] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:d3:09:2e,bridge_name='br-int',has_traffic_filtering=True,id=501886e8-de21-4108-86d4-d7a9138f8057,network=Network(ca5d0d87-cb5f-4287-8438-a8ecc7d3fd33),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap501886e8-de') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:23:08 user nova-compute[75234]: DEBUG nova.virt.libvirt.guest [req-00008e2d-5dd7-4aa8-9235-ce1cb7bfcd16 req-39c352bb-c139-4de3-af8b-a7b0d98ea775 service nova] looking for interface given config: <interface type="ethernet"><mac address="fa:16:3e:d3:09:2e"/><model type="virtio"/><mtu size="1442"/><target dev="tap501886e8-de"/></interface> {{(pid=75234) get_interface_by_cfg /opt/stack/nova/nova/virt/libvirt/guest.py:257}}
Nov 29 03:23:08 user nova-compute[75234]: DEBUG nova.virt.libvirt.guest [req-00008e2d-5dd7-4aa8-9235-ce1cb7bfcd16 req-39c352bb-c139-4de3-af8b-a7b0d98ea775 service nova] interface for config: <interface type="ethernet"><mac address="fa:16:3e:d3:09:2e"/><model type="virtio"/><mtu size="1442"/><target dev="tap501886e8-de"/></interface>not found in domain: <domain type='kvm' id='48'>
Nov 29 03:23:08 user nova-compute[75234]:   <name>instance-00000023</name>
Nov 29 03:23:08 user nova-compute[75234]:   <uuid>946b919b-39ae-4d23-aefd-01952f4b40b6</uuid>
Nov 29 03:23:08 user nova-compute[75234]:   <metadata>
Nov 29 03:23:08 user nova-compute[75234]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1" xmlns:instance="http://openstack.org/xmlns/libvirt/nova/1.1">
Nov 29 03:23:08 user nova-compute[75234]:   <nova:package version="0.0.0"/>
Nov 29 03:23:08 user nova-compute[75234]:   <nova:name>tempest-device-tagging-server-1777937262</nova:name>
Nov 29 03:23:08 user nova-compute[75234]:   <nova:creationTime>2023-11-29 03:23:08</nova:creationTime>
Nov 29 03:23:08 user nova-compute[75234]:   <nova:flavor name="m1.tiny">
Nov 29 03:23:08 user nova-compute[75234]:     <nova:memory>512</nova:memory>
Nov 29 03:23:08 user nova-compute[75234]:     <nova:disk>1</nova:disk>
Nov 29 03:23:08 user nova-compute[75234]:     <nova:swap>0</nova:swap>
Nov 29 03:23:08 user nova-compute[75234]:     <nova:ephemeral>0</nova:ephemeral>
Nov 29 03:23:08 user nova-compute[75234]:     <nova:vcpus>1</nova:vcpus>
Nov 29 03:23:08 user nova-compute[75234]:   </nova:flavor>
Nov 29 03:23:08 user nova-compute[75234]:   <nova:owner>
Nov 29 03:23:08 user nova-compute[75234]:     <nova:user uuid="522c3c107614404fbe3af8c38fdd189e">tempest-TaggedAttachmentsTest-399278293-project-member</nova:user>
Nov 29 03:23:08 user nova-compute[75234]:     <nova:project uuid="e28b5baf29b14aa4b2b98cb1b615705f">tempest-TaggedAttachmentsTest-399278293</nova:project>
Nov 29 03:23:08 user nova-compute[75234]:   </nova:owner>
Nov 29 03:23:08 user nova-compute[75234]:   <nova:root type="image" uuid="96c30ba7-9b73-4a30-8ba4-7138879573b7"/>
Nov 29 03:23:08 user nova-compute[75234]:   <nova:ports>
Nov 29 03:23:08 user nova-compute[75234]:     <nova:port uuid="e1f71ce6-58f3-4665-afc5-f92d1b6801c5">
Nov 29 03:23:08 user nova-compute[75234]:       <nova:ip type="fixed" address="10.0.0.4" ipVersion="4"/>
Nov 29 03:23:08 user nova-compute[75234]:     </nova:port>
Nov 29 03:23:08 user nova-compute[75234]:   </nova:ports>
Nov 29 03:23:08 user nova-compute[75234]: </nova:instance>
Nov 29 03:23:08 user nova-compute[75234]:   </metadata>
Nov 29 03:23:08 user nova-compute[75234]:   <memory unit='KiB'>524288</memory>
Nov 29 03:23:08 user nova-compute[75234]:   <currentMemory unit='KiB'>524288</currentMemory>
Nov 29 03:23:08 user nova-compute[75234]:   <vcpu placement='static'>1</vcpu>
Nov 29 03:23:08 user nova-compute[75234]:   <resource>
Nov 29 03:23:08 user nova-compute[75234]:     <partition>/machine</partition>
Nov 29 03:23:08 user nova-compute[75234]:   </resource>
Nov 29 03:23:08 user nova-compute[75234]:   <sysinfo type='smbios'>
Nov 29 03:23:08 user nova-compute[75234]:     <system>
Nov 29 03:23:08 user nova-compute[75234]:       <entry name='manufacturer'>OpenStack Foundation</entry>
Nov 29 03:23:08 user nova-compute[75234]:       <entry name='product'>OpenStack Nova</entry>
Nov 29 03:23:08 user nova-compute[75234]:       <entry name='version'>0.0.0</entry>
Nov 29 03:23:08 user nova-compute[75234]:       <entry name='serial'>946b919b-39ae-4d23-aefd-01952f4b40b6</entry>
Nov 29 03:23:08 user nova-compute[75234]:       <entry name='uuid'>946b919b-39ae-4d23-aefd-01952f4b40b6</entry>
Nov 29 03:23:08 user nova-compute[75234]:       <entry name='family'>Virtual Machine</entry>
Nov 29 03:23:08 user nova-compute[75234]:     </system>
Nov 29 03:23:08 user nova-compute[75234]:   </sysinfo>
Nov 29 03:23:08 user nova-compute[75234]:   <os>
Nov 29 03:23:08 user nova-compute[75234]:     <type arch='x86_64' machine='pc-i440fx-6.2'>hvm</type>
Nov 29 03:23:08 user nova-compute[75234]:     <boot dev='hd'/>
Nov 29 03:23:08 user nova-compute[75234]:     <smbios mode='sysinfo'/>
Nov 29 03:23:08 user nova-compute[75234]:   </os>
Nov 29 03:23:08 user nova-compute[75234]:   <features>
Nov 29 03:23:08 user nova-compute[75234]:     <acpi/>
Nov 29 03:23:08 user nova-compute[75234]:     <apic/>
Nov 29 03:23:08 user nova-compute[75234]:     <vmcoreinfo state='on'/>
Nov 29 03:23:08 user nova-compute[75234]:   </features>
Nov 29 03:23:08 user nova-compute[75234]:   <cpu mode='custom' match='exact' check='full'>
Nov 29 03:23:08 user nova-compute[75234]:     <model fallback='forbid'>Nehalem</model>
Nov 29 03:23:08 user nova-compute[75234]:     <topology sockets='1' dies='1' cores='1' threads='1'/>
Nov 29 03:23:08 user nova-compute[75234]:     <feature policy='require' name='vme'/>
Nov 29 03:23:08 user nova-compute[75234]:     <feature policy='require' name='x2apic'/>
Nov 29 03:23:08 user nova-compute[75234]:     <feature policy='require' name='hypervisor'/>
Nov 29 03:23:08 user nova-compute[75234]:   </cpu>
Nov 29 03:23:08 user nova-compute[75234]:   <clock offset='utc'>
Nov 29 03:23:08 user nova-compute[75234]:     <timer name='pit' tickpolicy='delay'/>
Nov 29 03:23:08 user nova-compute[75234]:     <timer name='rtc' tickpolicy='catchup'/>
Nov 29 03:23:08 user nova-compute[75234]:     <timer name='hpet' present='no'/>
Nov 29 03:23:08 user nova-compute[75234]:   </clock>
Nov 29 03:23:08 user nova-compute[75234]:   <on_poweroff>destroy</on_poweroff>
Nov 29 03:23:08 user nova-compute[75234]:   <on_reboot>restart</on_reboot>
Nov 29 03:23:08 user nova-compute[75234]:   <on_crash>destroy</on_crash>
Nov 29 03:23:08 user nova-compute[75234]:   <devices>
Nov 29 03:23:08 user nova-compute[75234]:     <emulator>/usr/bin/qemu-system-x86_64</emulator>
Nov 29 03:23:08 user nova-compute[75234]:     <disk type='file' device='disk'>
Nov 29 03:23:08 user nova-compute[75234]:       <driver name='qemu' type='qcow2' cache='none'/>
Nov 29 03:23:08 user nova-compute[75234]:       <source file='/opt/stack/data/nova/instances/946b919b-39ae-4d23-aefd-01952f4b40b6/disk' index='2'/>
Nov 29 03:23:08 user nova-compute[75234]:       <backingStore type='file' index='3'>
Nov 29 03:23:08 user nova-compute[75234]:         <format type='raw'/>
Nov 29 03:23:08 user nova-compute[75234]:         <source file='/opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918'/>
Nov 29 03:23:08 user nova-compute[75234]:         <backingStore/>
Nov 29 03:23:08 user nova-compute[75234]:       </backingStore>
Nov 29 03:23:08 user nova-compute[75234]:       <target dev='vda' bus='virtio'/>
Nov 29 03:23:08 user nova-compute[75234]:       <alias name='virtio-disk0'/>
Nov 29 03:23:08 user nova-compute[75234]:       <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/>
Nov 29 03:23:08 user nova-compute[75234]:     </disk>
Nov 29 03:23:08 user nova-compute[75234]:     <disk type='file' device='cdrom'>
Nov 29 03:23:08 user nova-compute[75234]:       <driver name='qemu' type='raw' cache='none'/>
Nov 29 03:23:08 user nova-compute[75234]:       <source file='/opt/stack/data/nova/instances/946b919b-39ae-4d23-aefd-01952f4b40b6/disk.config' index='1'/>
Nov 29 03:23:08 user nova-compute[75234]:       <backingStore/>
Nov 29 03:23:08 user nova-compute[75234]:       <target dev='hda' bus='ide'/>
Nov 29 03:23:08 user nova-compute[75234]:       <readonly/>
Nov 29 03:23:08 user nova-compute[75234]:       <alias name='ide0-0-0'/>
Nov 29 03:23:08 user nova-compute[75234]:       <address type='drive' controller='0' bus='0' target='0' unit='0'/>
Nov 29 03:23:08 user nova-compute[75234]:     </disk>
Nov 29 03:23:08 user nova-compute[75234]:     <controller type='usb' index='0' model='none'>
Nov 29 03:23:08 user nova-compute[75234]:       <alias name='usb'/>
Nov 29 03:23:08 user nova-compute[75234]:     </controller>
Nov 29 03:23:08 user nova-compute[75234]:     <controller type='pci' index='0' model='pci-root'>
Nov 29 03:23:08 user nova-compute[75234]:       <alias name='pci.0'/>
Nov 29 03:23:08 user nova-compute[75234]:     </controller>
Nov 29 03:23:08 user nova-compute[75234]:     <controller type='ide' index='0'>
Nov 29 03:23:08 user nova-compute[75234]:       <alias name='ide'/>
Nov 29 03:23:08 user nova-compute[75234]:       <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x1'/>
Nov 29 03:23:08 user nova-compute[75234]:     </controller>
Nov 29 03:23:08 user nova-compute[75234]:     <interface type='ethernet'>
Nov 29 03:23:08 user nova-compute[75234]:       <mac address='fa:16:3e:81:ca:ab'/>
Nov 29 03:23:08 user nova-compute[75234]:       <target dev='tape1f71ce6-58'/>
Nov 29 03:23:08 user nova-compute[75234]:       <model type='virtio'/>
Nov 29 03:23:08 user nova-compute[75234]:       <mtu size='1442'/>
Nov 29 03:23:08 user nova-compute[75234]:       <alias name='net0'/>
Nov 29 03:23:08 user nova-compute[75234]:       <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>
Nov 29 03:23:08 user nova-compute[75234]:     </interface>
Nov 29 03:23:08 user nova-compute[75234]:     <serial type='pty'>
Nov 29 03:23:08 user nova-compute[75234]:       <source path='/dev/pts/3'/>
Nov 29 03:23:08 user nova-compute[75234]:       <log file='/opt/stack/data/nova/instances/946b919b-39ae-4d23-aefd-01952f4b40b6/console.log' append='off'/>
Nov 29 03:23:08 user nova-compute[75234]:       <target type='isa-serial' port='0'>
Nov 29 03:23:08 user nova-compute[75234]:         <model name='isa-serial'/>
Nov 29 03:23:08 user nova-compute[75234]:       </target>
Nov 29 03:23:08 user nova-compute[75234]:       <alias name='serial0'/>
Nov 29 03:23:08 user nova-compute[75234]:     </serial>
Nov 29 03:23:08 user nova-compute[75234]:     <console type='pty' tty='/dev/pts/3'>
Nov 29 03:23:08 user nova-compute[75234]:       <source path='/dev/pts/3'/>
Nov 29 03:23:08 user nova-compute[75234]:       <log file='/opt/stack/data/nova/instances/946b919b-39ae-4d23-aefd-01952f4b40b6/console.log' append='off'/>
Nov 29 03:23:08 user nova-compute[75234]:       <target type='serial' port='0'/>
Nov 29 03:23:08 user nova-compute[75234]:       <alias name='serial0'/>
Nov 29 03:23:08 user nova-compute[75234]:     </console>
Nov 29 03:23:08 user nova-compute[75234]:     <input type='mouse' bus='ps2'>
Nov 29 03:23:08 user nova-compute[75234]:       <alias name='input0'/>
Nov 29 03:23:08 user nova-compute[75234]:     </input>
Nov 29 03:23:08 user nova-compute[75234]:     <input type='keyboard' bus='ps2'>
Nov 29 03:23:08 user nova-compute[75234]:       <alias name='input1'/>
Nov 29 03:23:08 user nova-compute[75234]:     </input>
Nov 29 03:23:08 user nova-compute[75234]:     <graphics type='vnc' port='5903' autoport='yes' listen='0.0.0.0'>
Nov 29 03:23:08 user nova-compute[75234]:       <listen type='address' address='0.0.0.0'/>
Nov 29 03:23:08 user nova-compute[75234]:     </graphics>
Nov 29 03:23:08 user nova-compute[75234]:     <audio id='1' type='none'/>
Nov 29 03:23:08 user nova-compute[75234]:     <video>
Nov 29 03:23:08 user nova-compute[75234]:       <model type='virtio' heads='1' primary='yes'/>
Nov 29 03:23:08 user nova-compute[75234]:       <alias name='video0'/>
Nov 29 03:23:08 user nova-compute[75234]:       <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
Nov 29 03:23:08 user nova-compute[75234]:     </video>
Nov 29 03:23:08 user nova-compute[75234]:     <memballoon model='virtio'>
Nov 29 03:23:08 user nova-compute[75234]:       <stats period='10'/>
Nov 29 03:23:08 user nova-compute[75234]:       <alias name='balloon0'/>
Nov 29 03:23:08 user nova-compute[75234]:       <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/>
Nov 29 03:23:08 user nova-compute[75234]:     </memballoon>
Nov 29 03:23:08 user nova-compute[75234]:     <rng model='virtio'>
Nov 29 03:23:08 user nova-compute[75234]:       <backend model='random'>/dev/urandom</backend>
Nov 29 03:23:08 user nova-compute[75234]:       <alias name='rng0'/>
Nov 29 03:23:08 user nova-compute[75234]:       <address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0'/>
Nov 29 03:23:08 user nova-compute[75234]:     </rng>
Nov 29 03:23:08 user nova-compute[75234]:   </devices>
Nov 29 03:23:08 user nova-compute[75234]:   <seclabel type='dynamic' model='apparmor' relabel='yes'>
Nov 29 03:23:08 user nova-compute[75234]:     <label>libvirt-946b919b-39ae-4d23-aefd-01952f4b40b6</label>
Nov 29 03:23:08 user nova-compute[75234]:     <imagelabel>libvirt-946b919b-39ae-4d23-aefd-01952f4b40b6</imagelabel>
Nov 29 03:23:08 user nova-compute[75234]:   </seclabel>
Nov 29 03:23:08 user nova-compute[75234]:   <seclabel type='dynamic' model='dac' relabel='yes'>
Nov 29 03:23:08 user nova-compute[75234]:     <label>+64055:+108</label>
Nov 29 03:23:08 user nova-compute[75234]:     <imagelabel>+64055:+108</imagelabel>
Nov 29 03:23:08 user nova-compute[75234]:   </seclabel>
Nov 29 03:23:08 user nova-compute[75234]: </domain>
Nov 29 03:23:08 user nova-compute[75234]:  {{(pid=75234) get_interface_by_cfg /opt/stack/nova/nova/virt/libvirt/guest.py:282}}
Nov 29 03:23:08 user nova-compute[75234]: DEBUG nova.virt.libvirt.guest [req-00008e2d-5dd7-4aa8-9235-ce1cb7bfcd16 req-39c352bb-c139-4de3-af8b-a7b0d98ea775 service nova] looking for interface given config: <interface type="ethernet"><mac address="fa:16:3e:d3:09:2e"/><model type="virtio"/><mtu size="1442"/><target dev="tap501886e8-de"/></interface> {{(pid=75234) get_interface_by_cfg /opt/stack/nova/nova/virt/libvirt/guest.py:257}}
Nov 29 03:23:08 user nova-compute[75234]: DEBUG nova.virt.libvirt.guest [req-00008e2d-5dd7-4aa8-9235-ce1cb7bfcd16 req-39c352bb-c139-4de3-af8b-a7b0d98ea775 service nova] interface for config: <interface type="ethernet"><mac address="fa:16:3e:d3:09:2e"/><model type="virtio"/><mtu size="1442"/><target dev="tap501886e8-de"/></interface>not found in domain: <domain type='kvm' id='48'>
Nov 29 03:23:08 user nova-compute[75234]:   <name>instance-00000023</name>
Nov 29 03:23:08 user nova-compute[75234]:   <uuid>946b919b-39ae-4d23-aefd-01952f4b40b6</uuid>
Nov 29 03:23:08 user nova-compute[75234]:   <metadata>
Nov 29 03:23:08 user nova-compute[75234]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1" xmlns:instance="http://openstack.org/xmlns/libvirt/nova/1.1">
Nov 29 03:23:08 user nova-compute[75234]:   <nova:package version="0.0.0"/>
Nov 29 03:23:08 user nova-compute[75234]:   <nova:name>tempest-device-tagging-server-1777937262</nova:name>
Nov 29 03:23:08 user nova-compute[75234]:   <nova:creationTime>2023-11-29 03:23:08</nova:creationTime>
Nov 29 03:23:08 user nova-compute[75234]:   <nova:flavor name="m1.tiny">
Nov 29 03:23:08 user nova-compute[75234]:     <nova:memory>512</nova:memory>
Nov 29 03:23:08 user nova-compute[75234]:     <nova:disk>1</nova:disk>
Nov 29 03:23:08 user nova-compute[75234]:     <nova:swap>0</nova:swap>
Nov 29 03:23:08 user nova-compute[75234]:     <nova:ephemeral>0</nova:ephemeral>
Nov 29 03:23:08 user nova-compute[75234]:     <nova:vcpus>1</nova:vcpus>
Nov 29 03:23:08 user nova-compute[75234]:   </nova:flavor>
Nov 29 03:23:08 user nova-compute[75234]:   <nova:owner>
Nov 29 03:23:08 user nova-compute[75234]:     <nova:user uuid="522c3c107614404fbe3af8c38fdd189e">tempest-TaggedAttachmentsTest-399278293-project-member</nova:user>
Nov 29 03:23:08 user nova-compute[75234]:     <nova:project uuid="e28b5baf29b14aa4b2b98cb1b615705f">tempest-TaggedAttachmentsTest-399278293</nova:project>
Nov 29 03:23:08 user nova-compute[75234]:   </nova:owner>
Nov 29 03:23:08 user nova-compute[75234]:   <nova:root type="image" uuid="96c30ba7-9b73-4a30-8ba4-7138879573b7"/>
Nov 29 03:23:08 user nova-compute[75234]:   <nova:ports>
Nov 29 03:23:08 user nova-compute[75234]:     <nova:port uuid="e1f71ce6-58f3-4665-afc5-f92d1b6801c5">
Nov 29 03:23:08 user nova-compute[75234]:       <nova:ip type="fixed" address="10.0.0.4" ipVersion="4"/>
Nov 29 03:23:08 user nova-compute[75234]:     </nova:port>
Nov 29 03:23:08 user nova-compute[75234]:   </nova:ports>
Nov 29 03:23:08 user nova-compute[75234]: </nova:instance>
Nov 29 03:23:08 user nova-compute[75234]:   </metadata>
Nov 29 03:23:08 user nova-compute[75234]:   <memory unit='KiB'>524288</memory>
Nov 29 03:23:08 user nova-compute[75234]:   <currentMemory unit='KiB'>524288</currentMemory>
Nov 29 03:23:08 user nova-compute[75234]:   <vcpu placement='static'>1</vcpu>
Nov 29 03:23:08 user nova-compute[75234]:   <resource>
Nov 29 03:23:08 user nova-compute[75234]:     <partition>/machine</partition>
Nov 29 03:23:08 user nova-compute[75234]:   </resource>
Nov 29 03:23:08 user nova-compute[75234]:   <sysinfo type='smbios'>
Nov 29 03:23:08 user nova-compute[75234]:     <system>
Nov 29 03:23:08 user nova-compute[75234]:       <entry name='manufacturer'>OpenStack Foundation</entry>
Nov 29 03:23:08 user nova-compute[75234]:       <entry name='product'>OpenStack Nova</entry>
Nov 29 03:23:08 user nova-compute[75234]:       <entry name='version'>0.0.0</entry>
Nov 29 03:23:08 user nova-compute[75234]:       <entry name='serial'>946b919b-39ae-4d23-aefd-01952f4b40b6</entry>
Nov 29 03:23:08 user nova-compute[75234]:       <entry name='uuid'>946b919b-39ae-4d23-aefd-01952f4b40b6</entry>
Nov 29 03:23:08 user nova-compute[75234]:       <entry name='family'>Virtual Machine</entry>
Nov 29 03:23:08 user nova-compute[75234]:     </system>
Nov 29 03:23:08 user nova-compute[75234]:   </sysinfo>
Nov 29 03:23:08 user nova-compute[75234]:   <os>
Nov 29 03:23:08 user nova-compute[75234]:     <type arch='x86_64' machine='pc-i440fx-6.2'>hvm</type>
Nov 29 03:23:08 user nova-compute[75234]:     <boot dev='hd'/>
Nov 29 03:23:08 user nova-compute[75234]:     <smbios mode='sysinfo'/>
Nov 29 03:23:08 user nova-compute[75234]:   </os>
Nov 29 03:23:08 user nova-compute[75234]:   <features>
Nov 29 03:23:08 user nova-compute[75234]:     <acpi/>
Nov 29 03:23:08 user nova-compute[75234]:     <apic/>
Nov 29 03:23:08 user nova-compute[75234]:     <vmcoreinfo state='on'/>
Nov 29 03:23:08 user nova-compute[75234]:   </features>
Nov 29 03:23:08 user nova-compute[75234]:   <cpu mode='custom' match='exact' check='full'>
Nov 29 03:23:08 user nova-compute[75234]:     <model fallback='forbid'>Nehalem</model>
Nov 29 03:23:08 user nova-compute[75234]:     <topology sockets='1' dies='1' cores='1' threads='1'/>
Nov 29 03:23:08 user nova-compute[75234]:     <feature policy='require' name='vme'/>
Nov 29 03:23:08 user nova-compute[75234]:     <feature policy='require' name='x2apic'/>
Nov 29 03:23:08 user nova-compute[75234]:     <feature policy='require' name='hypervisor'/>
Nov 29 03:23:08 user nova-compute[75234]:   </cpu>
Nov 29 03:23:08 user nova-compute[75234]:   <clock offset='utc'>
Nov 29 03:23:08 user nova-compute[75234]:     <timer name='pit' tickpolicy='delay'/>
Nov 29 03:23:08 user nova-compute[75234]:     <timer name='rtc' tickpolicy='catchup'/>
Nov 29 03:23:08 user nova-compute[75234]:     <timer name='hpet' present='no'/>
Nov 29 03:23:08 user nova-compute[75234]:   </clock>
Nov 29 03:23:08 user nova-compute[75234]:   <on_poweroff>destroy</on_poweroff>
Nov 29 03:23:08 user nova-compute[75234]:   <on_reboot>restart</on_reboot>
Nov 29 03:23:08 user nova-compute[75234]:   <on_crash>destroy</on_crash>
Nov 29 03:23:08 user nova-compute[75234]:   <devices>
Nov 29 03:23:08 user nova-compute[75234]:     <emulator>/usr/bin/qemu-system-x86_64</emulator>
Nov 29 03:23:08 user nova-compute[75234]:     <disk type='file' device='disk'>
Nov 29 03:23:08 user nova-compute[75234]:       <driver name='qemu' type='qcow2' cache='none'/>
Nov 29 03:23:08 user nova-compute[75234]:       <source file='/opt/stack/data/nova/instances/946b919b-39ae-4d23-aefd-01952f4b40b6/disk' index='2'/>
Nov 29 03:23:08 user nova-compute[75234]:       <backingStore type='file' index='3'>
Nov 29 03:23:08 user nova-compute[75234]:         <format type='raw'/>
Nov 29 03:23:08 user nova-compute[75234]:         <source file='/opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918'/>
Nov 29 03:23:08 user nova-compute[75234]:         <backingStore/>
Nov 29 03:23:08 user nova-compute[75234]:       </backingStore>
Nov 29 03:23:08 user nova-compute[75234]:       <target dev='vda' bus='virtio'/>
Nov 29 03:23:08 user nova-compute[75234]:       <alias name='virtio-disk0'/>
Nov 29 03:23:08 user nova-compute[75234]:       <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/>
Nov 29 03:23:08 user nova-compute[75234]:     </disk>
Nov 29 03:23:08 user nova-compute[75234]:     <disk type='file' device='cdrom'>
Nov 29 03:23:08 user nova-compute[75234]:       <driver name='qemu' type='raw' cache='none'/>
Nov 29 03:23:08 user nova-compute[75234]:       <source file='/opt/stack/data/nova/instances/946b919b-39ae-4d23-aefd-01952f4b40b6/disk.config' index='1'/>
Nov 29 03:23:08 user nova-compute[75234]:       <backingStore/>
Nov 29 03:23:08 user nova-compute[75234]:       <target dev='hda' bus='ide'/>
Nov 29 03:23:08 user nova-compute[75234]:       <readonly/>
Nov 29 03:23:08 user nova-compute[75234]:       <alias name='ide0-0-0'/>
Nov 29 03:23:08 user nova-compute[75234]:       <address type='drive' controller='0' bus='0' target='0' unit='0'/>
Nov 29 03:23:08 user nova-compute[75234]:     </disk>
Nov 29 03:23:08 user nova-compute[75234]:     <controller type='usb' index='0' model='none'>
Nov 29 03:23:08 user nova-compute[75234]:       <alias name='usb'/>
Nov 29 03:23:08 user nova-compute[75234]:     </controller>
Nov 29 03:23:08 user nova-compute[75234]:     <controller type='pci' index='0' model='pci-root'>
Nov 29 03:23:08 user nova-compute[75234]:       <alias name='pci.0'/>
Nov 29 03:23:08 user nova-compute[75234]:     </controller>
Nov 29 03:23:08 user nova-compute[75234]:     <controller type='ide' index='0'>
Nov 29 03:23:08 user nova-compute[75234]:       <alias name='ide'/>
Nov 29 03:23:08 user nova-compute[75234]:       <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x1'/>
Nov 29 03:23:08 user nova-compute[75234]:     </controller>
Nov 29 03:23:08 user nova-compute[75234]:     <interface type='ethernet'>
Nov 29 03:23:08 user nova-compute[75234]:       <mac address='fa:16:3e:81:ca:ab'/>
Nov 29 03:23:08 user nova-compute[75234]:       <target dev='tape1f71ce6-58'/>
Nov 29 03:23:08 user nova-compute[75234]:       <model type='virtio'/>
Nov 29 03:23:08 user nova-compute[75234]:       <mtu size='1442'/>
Nov 29 03:23:08 user nova-compute[75234]:       <alias name='net0'/>
Nov 29 03:23:08 user nova-compute[75234]:       <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>
Nov 29 03:23:08 user nova-compute[75234]:     </interface>
Nov 29 03:23:08 user nova-compute[75234]:     <serial type='pty'>
Nov 29 03:23:08 user nova-compute[75234]:       <source path='/dev/pts/3'/>
Nov 29 03:23:08 user nova-compute[75234]:       <log file='/opt/stack/data/nova/instances/946b919b-39ae-4d23-aefd-01952f4b40b6/console.log' append='off'/>
Nov 29 03:23:08 user nova-compute[75234]:       <target type='isa-serial' port='0'>
Nov 29 03:23:08 user nova-compute[75234]:         <model name='isa-serial'/>
Nov 29 03:23:08 user nova-compute[75234]:       </target>
Nov 29 03:23:08 user nova-compute[75234]:       <alias name='serial0'/>
Nov 29 03:23:08 user nova-compute[75234]:     </serial>
Nov 29 03:23:08 user nova-compute[75234]:     <console type='pty' tty='/dev/pts/3'>
Nov 29 03:23:08 user nova-compute[75234]:       <source path='/dev/pts/3'/>
Nov 29 03:23:08 user nova-compute[75234]:       <log file='/opt/stack/data/nova/instances/946b919b-39ae-4d23-aefd-01952f4b40b6/console.log' append='off'/>
Nov 29 03:23:08 user nova-compute[75234]:       <target type='serial' port='0'/>
Nov 29 03:23:08 user nova-compute[75234]:       <alias name='serial0'/>
Nov 29 03:23:08 user nova-compute[75234]:     </console>
Nov 29 03:23:08 user nova-compute[75234]:     <input type='mouse' bus='ps2'>
Nov 29 03:23:08 user nova-compute[75234]:       <alias name='input0'/>
Nov 29 03:23:08 user nova-compute[75234]:     </input>
Nov 29 03:23:08 user nova-compute[75234]:     <input type='keyboard' bus='ps2'>
Nov 29 03:23:08 user nova-compute[75234]:       <alias name='input1'/>
Nov 29 03:23:08 user nova-compute[75234]:     </input>
Nov 29 03:23:08 user nova-compute[75234]:     <graphics type='vnc' port='5903' autoport='yes' listen='0.0.0.0'>
Nov 29 03:23:08 user nova-compute[75234]:       <listen type='address' address='0.0.0.0'/>
Nov 29 03:23:08 user nova-compute[75234]:     </graphics>
Nov 29 03:23:08 user nova-compute[75234]:     <audio id='1' type='none'/>
Nov 29 03:23:08 user nova-compute[75234]:     <video>
Nov 29 03:23:08 user nova-compute[75234]:       <model type='virtio' heads='1' primary='yes'/>
Nov 29 03:23:08 user nova-compute[75234]:       <alias name='video0'/>
Nov 29 03:23:08 user nova-compute[75234]:       <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
Nov 29 03:23:08 user nova-compute[75234]:     </video>
Nov 29 03:23:08 user nova-compute[75234]:     <memballoon model='virtio'>
Nov 29 03:23:08 user nova-compute[75234]:       <stats period='10'/>
Nov 29 03:23:08 user nova-compute[75234]:       <alias name='balloon0'/>
Nov 29 03:23:08 user nova-compute[75234]:       <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/>
Nov 29 03:23:08 user nova-compute[75234]:     </memballoon>
Nov 29 03:23:08 user nova-compute[75234]:     <rng model='virtio'>
Nov 29 03:23:08 user nova-compute[75234]:       <backend model='random'>/dev/urandom</backend>
Nov 29 03:23:08 user nova-compute[75234]:       <alias name='rng0'/>
Nov 29 03:23:08 user nova-compute[75234]:       <address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0'/>
Nov 29 03:23:08 user nova-compute[75234]:     </rng>
Nov 29 03:23:08 user nova-compute[75234]:   </devices>
Nov 29 03:23:08 user nova-compute[75234]:   <seclabel type='dynamic' model='apparmor' relabel='yes'>
Nov 29 03:23:08 user nova-compute[75234]:     <label>libvirt-946b919b-39ae-4d23-aefd-01952f4b40b6</label>
Nov 29 03:23:08 user nova-compute[75234]:     <imagelabel>libvirt-946b919b-39ae-4d23-aefd-01952f4b40b6</imagelabel>
Nov 29 03:23:08 user nova-compute[75234]:   </seclabel>
Nov 29 03:23:08 user nova-compute[75234]:   <seclabel type='dynamic' model='dac' relabel='yes'>
Nov 29 03:23:08 user nova-compute[75234]:     <label>+64055:+108</label>
Nov 29 03:23:08 user nova-compute[75234]:     <imagelabel>+64055:+108</imagelabel>
Nov 29 03:23:08 user nova-compute[75234]:   </seclabel>
Nov 29 03:23:08 user nova-compute[75234]: </domain>
Nov 29 03:23:08 user nova-compute[75234]:  {{(pid=75234) get_interface_by_cfg /opt/stack/nova/nova/virt/libvirt/guest.py:282}}
Nov 29 03:23:08 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [req-00008e2d-5dd7-4aa8-9235-ce1cb7bfcd16 req-39c352bb-c139-4de3-af8b-a7b0d98ea775 service nova] [instance: 946b919b-39ae-4d23-aefd-01952f4b40b6] Detaching interface fa:16:3e:d3:09:2e failed because the device is no longer found on the guest.: nova.exception.DeviceNotFound: Device 'tap501886e8-de' not found.
Nov 29 03:23:08 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [req-00008e2d-5dd7-4aa8-9235-ce1cb7bfcd16 req-39c352bb-c139-4de3-af8b-a7b0d98ea775 service nova] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='True',created_at=2023-11-29T03:20:14Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-1777937262',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-device-tagging-server-1777937262',id=35,image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBJrVo4ngcPAPF4oP6xidv+ykI7O7g9y5APrTr7iXhruQjDIx7zqy1/omzWf7G2p8xg1QAr8Js8NrVgD6XT2ZMw0fSSCLDKoKJk+X3O2rM5aB1leyuLU4v60YiI/3ZRg9Ew==',key_name='tempest-keypair-1887924020',keypairs=<?>,launch_index=0,launched_at=2023-11-29T03:20:21Z,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata=<?>,migration_context=None,new_flavor=None,node='user',numa_topology=<?>,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='e28b5baf29b14aa4b2b98cb1b615705f',ramdisk_id='',reservation_id='r-bfdfrqjg',resources=<?>,root_device_name='/dev/vda',root_gb=1,security_groups=<?>,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-TaggedAttachmentsTest-399278293',owner_user_name='tempest-TaggedAttachmentsTest-399278293-project-member'},tags=<?>,task_state=None,terminated_at=None,trusted_certs=<?>,updated_at=2023-11-29T03:20:22Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='522c3c107614404fbe3af8c38fdd189e',uuid=946b919b-39ae-4d23-aefd-01952f4b40b6,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "501886e8-de21-4108-86d4-d7a9138f8057", "address": "fa:16:3e:d3:09:2e", "network": {"id": "ca5d0d87-cb5f-4287-8438-a8ecc7d3fd33", "bridge": "br-int", "label": "tempest-tagged-attachments-test-net-1833288811", "subnets": [{"cidr": "10.10.10.0/24", "dns": [], "gateway": {"address": "10.10.10.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.10.10.149", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.10.10.1"}}], "meta": {"injected": false, "tenant_id": "e28b5baf29b14aa4b2b98cb1b615705f", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap501886e8-de", "ovs_interfaceid": "501886e8-de21-4108-86d4-d7a9138f8057", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Nov 29 03:23:08 user nova-compute[75234]: DEBUG nova.network.os_vif_util [req-00008e2d-5dd7-4aa8-9235-ce1cb7bfcd16 req-39c352bb-c139-4de3-af8b-a7b0d98ea775 service nova] Converting VIF {"id": "501886e8-de21-4108-86d4-d7a9138f8057", "address": "fa:16:3e:d3:09:2e", "network": {"id": "ca5d0d87-cb5f-4287-8438-a8ecc7d3fd33", "bridge": "br-int", "label": "tempest-tagged-attachments-test-net-1833288811", "subnets": [{"cidr": "10.10.10.0/24", "dns": [], "gateway": {"address": "10.10.10.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.10.10.149", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.10.10.1"}}], "meta": {"injected": false, "tenant_id": "e28b5baf29b14aa4b2b98cb1b615705f", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap501886e8-de", "ovs_interfaceid": "501886e8-de21-4108-86d4-d7a9138f8057", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:23:08 user nova-compute[75234]: DEBUG nova.network.os_vif_util [req-00008e2d-5dd7-4aa8-9235-ce1cb7bfcd16 req-39c352bb-c139-4de3-af8b-a7b0d98ea775 service nova] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:d3:09:2e,bridge_name='br-int',has_traffic_filtering=True,id=501886e8-de21-4108-86d4-d7a9138f8057,network=Network(ca5d0d87-cb5f-4287-8438-a8ecc7d3fd33),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap501886e8-de') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:23:08 user nova-compute[75234]: DEBUG os_vif [req-00008e2d-5dd7-4aa8-9235-ce1cb7bfcd16 req-39c352bb-c139-4de3-af8b-a7b0d98ea775 service nova] Unplugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:d3:09:2e,bridge_name='br-int',has_traffic_filtering=True,id=501886e8-de21-4108-86d4-d7a9138f8057,network=Network(ca5d0d87-cb5f-4287-8438-a8ecc7d3fd33),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap501886e8-de') {{(pid=75234) unplug /opt/stack/data/venv/lib/python3.10/site-packages/os_vif/__init__.py:109}}
Nov 29 03:23:08 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:23:08 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tap501886e8-de, bridge=br-int, if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:23:08 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Nov 29 03:23:08 user nova-compute[75234]: INFO os_vif [req-00008e2d-5dd7-4aa8-9235-ce1cb7bfcd16 req-39c352bb-c139-4de3-af8b-a7b0d98ea775 service nova] Successfully unplugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:d3:09:2e,bridge_name='br-int',has_traffic_filtering=True,id=501886e8-de21-4108-86d4-d7a9138f8057,network=Network(ca5d0d87-cb5f-4287-8438-a8ecc7d3fd33),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap501886e8-de')
Nov 29 03:23:08 user nova-compute[75234]: DEBUG nova.virt.libvirt.guest [req-00008e2d-5dd7-4aa8-9235-ce1cb7bfcd16 req-39c352bb-c139-4de3-af8b-a7b0d98ea775 service nova] set metadata xml: <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Nov 29 03:23:08 user nova-compute[75234]:   <nova:package version="0.0.0"/>
Nov 29 03:23:08 user nova-compute[75234]:   <nova:name>tempest-device-tagging-server-1777937262</nova:name>
Nov 29 03:23:08 user nova-compute[75234]:   <nova:creationTime>2023-11-29 03:23:08</nova:creationTime>
Nov 29 03:23:08 user nova-compute[75234]:   <nova:flavor name="m1.tiny">
Nov 29 03:23:08 user nova-compute[75234]:     <nova:memory>512</nova:memory>
Nov 29 03:23:08 user nova-compute[75234]:     <nova:disk>1</nova:disk>
Nov 29 03:23:08 user nova-compute[75234]:     <nova:swap>0</nova:swap>
Nov 29 03:23:08 user nova-compute[75234]:     <nova:ephemeral>0</nova:ephemeral>
Nov 29 03:23:08 user nova-compute[75234]:     <nova:vcpus>1</nova:vcpus>
Nov 29 03:23:08 user nova-compute[75234]:   </nova:flavor>
Nov 29 03:23:08 user nova-compute[75234]:   <nova:owner>
Nov 29 03:23:08 user nova-compute[75234]:     <nova:user uuid="522c3c107614404fbe3af8c38fdd189e">tempest-TaggedAttachmentsTest-399278293-project-member</nova:user>
Nov 29 03:23:08 user nova-compute[75234]:     <nova:project uuid="e28b5baf29b14aa4b2b98cb1b615705f">tempest-TaggedAttachmentsTest-399278293</nova:project>
Nov 29 03:23:08 user nova-compute[75234]:   </nova:owner>
Nov 29 03:23:08 user nova-compute[75234]:   <nova:root type="image" uuid="96c30ba7-9b73-4a30-8ba4-7138879573b7"/>
Nov 29 03:23:08 user nova-compute[75234]:   <nova:ports>
Nov 29 03:23:08 user nova-compute[75234]:     <nova:port uuid="e1f71ce6-58f3-4665-afc5-f92d1b6801c5">
Nov 29 03:23:08 user nova-compute[75234]:       <nova:ip type="fixed" address="10.0.0.4" ipVersion="4"/>
Nov 29 03:23:08 user nova-compute[75234]:     </nova:port>
Nov 29 03:23:08 user nova-compute[75234]:   </nova:ports>
Nov 29 03:23:08 user nova-compute[75234]: </nova:instance>
Nov 29 03:23:08 user nova-compute[75234]:  {{(pid=75234) set_metadata /opt/stack/nova/nova/virt/libvirt/guest.py:359}}
Nov 29 03:23:08 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:23:08 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:23:08 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Starting heal instance info cache {{(pid=75234) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9922}}
Nov 29 03:23:09 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Acquiring lock "refresh_cache-a082c96e-e384-40b6-bcc8-fc5e0e104669" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:23:09 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Acquired lock "refresh_cache-a082c96e-e384-40b6-bcc8-fc5e0e104669" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:23:09 user nova-compute[75234]: DEBUG nova.network.neutron [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] [instance: a082c96e-e384-40b6-bcc8-fc5e0e104669] Forcefully refreshing network info cache for instance {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2003}}
Nov 29 03:23:09 user nova-compute[75234]: DEBUG nova.network.neutron [None req-1f033c13-c772-4284-adad-9d92f9362260 tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] [instance: 946b919b-39ae-4d23-aefd-01952f4b40b6] Updating instance_info_cache with network_info: [{"id": "e1f71ce6-58f3-4665-afc5-f92d1b6801c5", "address": "fa:16:3e:81:ca:ab", "network": {"id": "93ca21e4-138b-4f00-8d1d-b30d63dbe7aa", "bridge": "br-int", "label": "tempest-TaggedAttachmentsTest-1844288661-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.47", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "e28b5baf29b14aa4b2b98cb1b615705f", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape1f71ce6-58", "ovs_interfaceid": "e1f71ce6-58f3-4665-afc5-f92d1b6801c5", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:23:09 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-1f033c13-c772-4284-adad-9d92f9362260 tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] Releasing lock "refresh_cache-946b919b-39ae-4d23-aefd-01952f4b40b6" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:23:09 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-1f033c13-c772-4284-adad-9d92f9362260 tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] Lock "interface-946b919b-39ae-4d23-aefd-01952f4b40b6-501886e8-de21-4108-86d4-d7a9138f8057" "released" by "nova.compute.manager.ComputeManager.detach_interface.<locals>.do_detach_interface" :: held 22.026s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:23:10 user nova-compute[75234]: DEBUG nova.network.neutron [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] [instance: a082c96e-e384-40b6-bcc8-fc5e0e104669] Updating instance_info_cache with network_info: [{"id": "0d131138-51b6-49f9-9a11-6b64e32e5796", "address": "fa:16:3e:ee:2e:82", "network": {"id": "2b3c3140-f3b6-45da-8202-bb4d7f0dd130", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-2043420778-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "f8a7884867c944e1ba1024270e4902f3", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap0d131138-51", "ovs_interfaceid": "0d131138-51b6-49f9-9a11-6b64e32e5796", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:23:10 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Releasing lock "refresh_cache-a082c96e-e384-40b6-bcc8-fc5e0e104669" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:23:10 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] [instance: a082c96e-e384-40b6-bcc8-fc5e0e104669] Updated the network info_cache for instance {{(pid=75234) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9993}}
Nov 29 03:23:10 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:23:10 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:23:10 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:23:10 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=75234) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10541}}
Nov 29 03:23:10 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:23:10 user nova-compute[75234]: DEBUG nova.compute.manager [req-27a4733b-ca9c-4e59-8da2-bc42642c8973 req-128b8437-2022-4f76-9ef7-70b1f36826f8 service nova] [instance: f6bc212d-85cd-479c-92ae-112c9f0101bc] Received event network-changed-e43ebf8d-4145-4a78-8be5-600ea409e775 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:23:10 user nova-compute[75234]: DEBUG nova.compute.manager [req-27a4733b-ca9c-4e59-8da2-bc42642c8973 req-128b8437-2022-4f76-9ef7-70b1f36826f8 service nova] [instance: f6bc212d-85cd-479c-92ae-112c9f0101bc] Refreshing instance network info cache due to event network-changed-e43ebf8d-4145-4a78-8be5-600ea409e775. {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11117}}
Nov 29 03:23:10 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-27a4733b-ca9c-4e59-8da2-bc42642c8973 req-128b8437-2022-4f76-9ef7-70b1f36826f8 service nova] Acquiring lock "refresh_cache-f6bc212d-85cd-479c-92ae-112c9f0101bc" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:23:10 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-27a4733b-ca9c-4e59-8da2-bc42642c8973 req-128b8437-2022-4f76-9ef7-70b1f36826f8 service nova] Acquired lock "refresh_cache-f6bc212d-85cd-479c-92ae-112c9f0101bc" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:23:10 user nova-compute[75234]: DEBUG nova.network.neutron [req-27a4733b-ca9c-4e59-8da2-bc42642c8973 req-128b8437-2022-4f76-9ef7-70b1f36826f8 service nova] [instance: f6bc212d-85cd-479c-92ae-112c9f0101bc] Refreshing network info cache for port e43ebf8d-4145-4a78-8be5-600ea409e775 {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2006}}
Nov 29 03:23:11 user nova-compute[75234]: DEBUG nova.network.neutron [req-27a4733b-ca9c-4e59-8da2-bc42642c8973 req-128b8437-2022-4f76-9ef7-70b1f36826f8 service nova] [instance: f6bc212d-85cd-479c-92ae-112c9f0101bc] Updated VIF entry in instance network info cache for port e43ebf8d-4145-4a78-8be5-600ea409e775. {{(pid=75234) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3481}}
Nov 29 03:23:11 user nova-compute[75234]: DEBUG nova.network.neutron [req-27a4733b-ca9c-4e59-8da2-bc42642c8973 req-128b8437-2022-4f76-9ef7-70b1f36826f8 service nova] [instance: f6bc212d-85cd-479c-92ae-112c9f0101bc] Updating instance_info_cache with network_info: [{"id": "e43ebf8d-4145-4a78-8be5-600ea409e775", "address": "fa:16:3e:4e:8e:1d", "network": {"id": "9247dc93-6a54-495e-b113-d1ff7c215178", "bridge": "br-int", "label": "tempest-SnapshotDataIntegrityTests-261469187-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.143", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "7c3b05b1968540dbbb40817c87846ff6", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape43ebf8d-41", "ovs_interfaceid": "e43ebf8d-4145-4a78-8be5-600ea409e775", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:23:11 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-27a4733b-ca9c-4e59-8da2-bc42642c8973 req-128b8437-2022-4f76-9ef7-70b1f36826f8 service nova] Releasing lock "refresh_cache-f6bc212d-85cd-479c-92ae-112c9f0101bc" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:23:12 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:23:13 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:23:13 user nova-compute[75234]: INFO nova.compute.manager [None req-307bdcfb-bba6-451f-b9a8-41275ba17324 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] [instance: 1df140f1-fbd9-4b5d-b356-d08b7e908f64] Unrescuing
Nov 29 03:23:13 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-307bdcfb-bba6-451f-b9a8-41275ba17324 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Acquiring lock "refresh_cache-1df140f1-fbd9-4b5d-b356-d08b7e908f64" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:23:13 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-307bdcfb-bba6-451f-b9a8-41275ba17324 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Acquired lock "refresh_cache-1df140f1-fbd9-4b5d-b356-d08b7e908f64" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:23:13 user nova-compute[75234]: DEBUG nova.network.neutron [None req-307bdcfb-bba6-451f-b9a8-41275ba17324 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] [instance: 1df140f1-fbd9-4b5d-b356-d08b7e908f64] Building network info cache for instance {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2009}}
Nov 29 03:23:13 user nova-compute[75234]: DEBUG nova.network.neutron [None req-307bdcfb-bba6-451f-b9a8-41275ba17324 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] [instance: 1df140f1-fbd9-4b5d-b356-d08b7e908f64] Updating instance_info_cache with network_info: [{"id": "ea9a3586-da52-49b7-8ced-5d27806885ad", "address": "fa:16:3e:80:25:b8", "network": {"id": "2b3c3140-f3b6-45da-8202-bb4d7f0dd130", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-2043420778-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.158", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "f8a7884867c944e1ba1024270e4902f3", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapea9a3586-da", "ovs_interfaceid": "ea9a3586-da52-49b7-8ced-5d27806885ad", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:23:13 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-307bdcfb-bba6-451f-b9a8-41275ba17324 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Releasing lock "refresh_cache-1df140f1-fbd9-4b5d-b356-d08b7e908f64" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:23:13 user nova-compute[75234]: DEBUG nova.objects.instance [None req-307bdcfb-bba6-451f-b9a8-41275ba17324 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Lazy-loading 'flavor' on Instance uuid 1df140f1-fbd9-4b5d-b356-d08b7e908f64 {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:23:13 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:23:13 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:23:14 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:23:14 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:23:14 user nova-compute[75234]: DEBUG nova.compute.manager [req-ccf5da5a-5845-481f-ab21-f3a8c49b431e req-ecb6caa0-76df-408a-b4ec-2a3da6680317 service nova] [instance: 1df140f1-fbd9-4b5d-b356-d08b7e908f64] Received event network-vif-unplugged-ea9a3586-da52-49b7-8ced-5d27806885ad {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:23:14 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-ccf5da5a-5845-481f-ab21-f3a8c49b431e req-ecb6caa0-76df-408a-b4ec-2a3da6680317 service nova] Acquiring lock "1df140f1-fbd9-4b5d-b356-d08b7e908f64-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:23:14 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-ccf5da5a-5845-481f-ab21-f3a8c49b431e req-ecb6caa0-76df-408a-b4ec-2a3da6680317 service nova] Lock "1df140f1-fbd9-4b5d-b356-d08b7e908f64-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:23:14 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-ccf5da5a-5845-481f-ab21-f3a8c49b431e req-ecb6caa0-76df-408a-b4ec-2a3da6680317 service nova] Lock "1df140f1-fbd9-4b5d-b356-d08b7e908f64-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:23:14 user nova-compute[75234]: DEBUG nova.compute.manager [req-ccf5da5a-5845-481f-ab21-f3a8c49b431e req-ecb6caa0-76df-408a-b4ec-2a3da6680317 service nova] [instance: 1df140f1-fbd9-4b5d-b356-d08b7e908f64] No waiting events found dispatching network-vif-unplugged-ea9a3586-da52-49b7-8ced-5d27806885ad {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:23:14 user nova-compute[75234]: WARNING nova.compute.manager [req-ccf5da5a-5845-481f-ab21-f3a8c49b431e req-ecb6caa0-76df-408a-b4ec-2a3da6680317 service nova] [instance: 1df140f1-fbd9-4b5d-b356-d08b7e908f64] Received unexpected event network-vif-unplugged-ea9a3586-da52-49b7-8ced-5d27806885ad for instance with vm_state rescued and task_state unrescuing.
Nov 29 03:23:14 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-44fd6cba-153e-40ee-9556-39e3b07eeedf tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Acquiring lock "f6bc212d-85cd-479c-92ae-112c9f0101bc" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:23:14 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-44fd6cba-153e-40ee-9556-39e3b07eeedf tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Lock "f6bc212d-85cd-479c-92ae-112c9f0101bc" acquired by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:23:14 user nova-compute[75234]: DEBUG nova.objects.instance [None req-44fd6cba-153e-40ee-9556-39e3b07eeedf tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Lazy-loading 'flavor' on Instance uuid f6bc212d-85cd-479c-92ae-112c9f0101bc {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:23:14 user nova-compute[75234]: INFO nova.virt.libvirt.driver [-] [instance: 1df140f1-fbd9-4b5d-b356-d08b7e908f64] Instance destroyed successfully.
Nov 29 03:23:14 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-44fd6cba-153e-40ee-9556-39e3b07eeedf tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Lock "f6bc212d-85cd-479c-92ae-112c9f0101bc" "released" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: held 0.100s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:23:14 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-44fd6cba-153e-40ee-9556-39e3b07eeedf tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Acquiring lock "f6bc212d-85cd-479c-92ae-112c9f0101bc" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:23:14 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-44fd6cba-153e-40ee-9556-39e3b07eeedf tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Lock "f6bc212d-85cd-479c-92ae-112c9f0101bc" acquired by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:23:14 user nova-compute[75234]: INFO nova.compute.manager [None req-44fd6cba-153e-40ee-9556-39e3b07eeedf tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] [instance: f6bc212d-85cd-479c-92ae-112c9f0101bc] Attaching volume 6af4b65b-b038-462a-b797-7c57c8e8d680 to /dev/vdb
Nov 29 03:23:14 user nova-compute[75234]: DEBUG os_brick.utils [None req-44fd6cba-153e-40ee-9556-39e3b07eeedf tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:176}}
Nov 29 03:23:14 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:23:14 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.014s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:23:14 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[36805b53-4dc0-4f64-8aff-741fa8d7bf85]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:dd1368c5ba2c\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:23:14 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt -v / -n -o SOURCE {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:23:14 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt -v / -n -o SOURCE" returned: 0 in 0.017s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:23:14 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[7b7b419a-3d4b-498e-8e24-bffdf8517e88]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:23:14 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:23:14 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.017s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:23:14 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[2326f1ae-df65-468d-af88-fc10928ec487]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:23:14 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[83aab54f-1e25-4750-888a-44b341d097ba]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:23:14 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-44fd6cba-153e-40ee-9556-39e3b07eeedf tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Running cmd (subprocess): nvme version {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:23:14 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-44fd6cba-153e-40ee-9556-39e3b07eeedf tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] 'nvme version' failed. Not Retrying. {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:473}}
Nov 29 03:23:14 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.nvmeof [None req-44fd6cba-153e-40ee-9556-39e3b07eeedf tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] nvme not present on system {{(pid=75234) nvme_present /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/nvmeof.py:756}}
Nov 29 03:23:14 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:23:14 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:473}}
Nov 29 03:23:14 user nova-compute[75234]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Nov 29 03:23:14 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[7a696b6a-85e8-4c86-a64f-3e10aecb9ae9]: (4, '') {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:23:14 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.lightos [None req-44fd6cba-153e-40ee-9556-39e3b07eeedf tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=75234) find_dsc /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/lightos.py:98}}
Nov 29 03:23:14 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.lightos [None req-44fd6cba-153e-40ee-9556-39e3b07eeedf tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=75234) get_connector_properties /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/lightos.py:76}}
Nov 29 03:23:14 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.lightos [None req-44fd6cba-153e-40ee-9556-39e3b07eeedf tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] LIGHTOS: no hostnqn found. {{(pid=75234) get_connector_properties /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/lightos.py:84}}
Nov 29 03:23:14 user nova-compute[75234]: DEBUG os_brick.utils [None req-44fd6cba-153e-40ee-9556-39e3b07eeedf tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] <== get_connector_properties: return (124ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:dd1368c5ba2c', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:203}}
Nov 29 03:23:14 user nova-compute[75234]: DEBUG nova.virt.block_device [None req-44fd6cba-153e-40ee-9556-39e3b07eeedf tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] [instance: f6bc212d-85cd-479c-92ae-112c9f0101bc] Updating existing volume attachment record: 0d0e3ff3-f004-4f82-854c-a4933e779087 {{(pid=75234) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Nov 29 03:23:15 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:23:15 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:23:15 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:23:15 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:23:15 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:23:15 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:23:16 user nova-compute[75234]: DEBUG nova.compute.manager [req-84a4e34e-b50e-4ab9-bedb-9bb5be3ad50f req-7a47cf69-2645-47f1-823e-f9081aeebbc1 service nova] [instance: 1df140f1-fbd9-4b5d-b356-d08b7e908f64] Received event network-vif-plugged-ea9a3586-da52-49b7-8ced-5d27806885ad {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:23:16 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-84a4e34e-b50e-4ab9-bedb-9bb5be3ad50f req-7a47cf69-2645-47f1-823e-f9081aeebbc1 service nova] Acquiring lock "1df140f1-fbd9-4b5d-b356-d08b7e908f64-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:23:16 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-84a4e34e-b50e-4ab9-bedb-9bb5be3ad50f req-7a47cf69-2645-47f1-823e-f9081aeebbc1 service nova] Lock "1df140f1-fbd9-4b5d-b356-d08b7e908f64-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:23:16 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-84a4e34e-b50e-4ab9-bedb-9bb5be3ad50f req-7a47cf69-2645-47f1-823e-f9081aeebbc1 service nova] Lock "1df140f1-fbd9-4b5d-b356-d08b7e908f64-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:23:16 user nova-compute[75234]: DEBUG nova.compute.manager [req-84a4e34e-b50e-4ab9-bedb-9bb5be3ad50f req-7a47cf69-2645-47f1-823e-f9081aeebbc1 service nova] [instance: 1df140f1-fbd9-4b5d-b356-d08b7e908f64] No waiting events found dispatching network-vif-plugged-ea9a3586-da52-49b7-8ced-5d27806885ad {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:23:16 user nova-compute[75234]: WARNING nova.compute.manager [req-84a4e34e-b50e-4ab9-bedb-9bb5be3ad50f req-7a47cf69-2645-47f1-823e-f9081aeebbc1 service nova] [instance: 1df140f1-fbd9-4b5d-b356-d08b7e908f64] Received unexpected event network-vif-plugged-ea9a3586-da52-49b7-8ced-5d27806885ad for instance with vm_state rescued and task_state unrescuing.
Nov 29 03:23:16 user nova-compute[75234]: DEBUG nova.compute.manager [req-84a4e34e-b50e-4ab9-bedb-9bb5be3ad50f req-7a47cf69-2645-47f1-823e-f9081aeebbc1 service nova] [instance: 1df140f1-fbd9-4b5d-b356-d08b7e908f64] Received event network-vif-plugged-ea9a3586-da52-49b7-8ced-5d27806885ad {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:23:16 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-84a4e34e-b50e-4ab9-bedb-9bb5be3ad50f req-7a47cf69-2645-47f1-823e-f9081aeebbc1 service nova] Acquiring lock "1df140f1-fbd9-4b5d-b356-d08b7e908f64-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:23:16 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-84a4e34e-b50e-4ab9-bedb-9bb5be3ad50f req-7a47cf69-2645-47f1-823e-f9081aeebbc1 service nova] Lock "1df140f1-fbd9-4b5d-b356-d08b7e908f64-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:23:16 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-84a4e34e-b50e-4ab9-bedb-9bb5be3ad50f req-7a47cf69-2645-47f1-823e-f9081aeebbc1 service nova] Lock "1df140f1-fbd9-4b5d-b356-d08b7e908f64-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:23:16 user nova-compute[75234]: DEBUG nova.compute.manager [req-84a4e34e-b50e-4ab9-bedb-9bb5be3ad50f req-7a47cf69-2645-47f1-823e-f9081aeebbc1 service nova] [instance: 1df140f1-fbd9-4b5d-b356-d08b7e908f64] No waiting events found dispatching network-vif-plugged-ea9a3586-da52-49b7-8ced-5d27806885ad {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:23:16 user nova-compute[75234]: WARNING nova.compute.manager [req-84a4e34e-b50e-4ab9-bedb-9bb5be3ad50f req-7a47cf69-2645-47f1-823e-f9081aeebbc1 service nova] [instance: 1df140f1-fbd9-4b5d-b356-d08b7e908f64] Received unexpected event network-vif-plugged-ea9a3586-da52-49b7-8ced-5d27806885ad for instance with vm_state rescued and task_state unrescuing.
Nov 29 03:23:16 user nova-compute[75234]: DEBUG nova.compute.manager [req-84a4e34e-b50e-4ab9-bedb-9bb5be3ad50f req-7a47cf69-2645-47f1-823e-f9081aeebbc1 service nova] [instance: 1df140f1-fbd9-4b5d-b356-d08b7e908f64] Received event network-vif-plugged-ea9a3586-da52-49b7-8ced-5d27806885ad {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:23:16 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-84a4e34e-b50e-4ab9-bedb-9bb5be3ad50f req-7a47cf69-2645-47f1-823e-f9081aeebbc1 service nova] Acquiring lock "1df140f1-fbd9-4b5d-b356-d08b7e908f64-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:23:16 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-84a4e34e-b50e-4ab9-bedb-9bb5be3ad50f req-7a47cf69-2645-47f1-823e-f9081aeebbc1 service nova] Lock "1df140f1-fbd9-4b5d-b356-d08b7e908f64-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:23:16 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-84a4e34e-b50e-4ab9-bedb-9bb5be3ad50f req-7a47cf69-2645-47f1-823e-f9081aeebbc1 service nova] Lock "1df140f1-fbd9-4b5d-b356-d08b7e908f64-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:23:16 user nova-compute[75234]: DEBUG nova.compute.manager [req-84a4e34e-b50e-4ab9-bedb-9bb5be3ad50f req-7a47cf69-2645-47f1-823e-f9081aeebbc1 service nova] [instance: 1df140f1-fbd9-4b5d-b356-d08b7e908f64] No waiting events found dispatching network-vif-plugged-ea9a3586-da52-49b7-8ced-5d27806885ad {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:23:16 user nova-compute[75234]: WARNING nova.compute.manager [req-84a4e34e-b50e-4ab9-bedb-9bb5be3ad50f req-7a47cf69-2645-47f1-823e-f9081aeebbc1 service nova] [instance: 1df140f1-fbd9-4b5d-b356-d08b7e908f64] Received unexpected event network-vif-plugged-ea9a3586-da52-49b7-8ced-5d27806885ad for instance with vm_state rescued and task_state unrescuing.
Nov 29 03:23:16 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:23:17 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:23:17 user nova-compute[75234]: DEBUG nova.compute.manager [None req-307bdcfb-bba6-451f-b9a8-41275ba17324 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] [instance: 1df140f1-fbd9-4b5d-b356-d08b7e908f64] Checking state {{(pid=75234) _get_power_state /opt/stack/nova/nova/compute/manager.py:1782}}
Nov 29 03:23:18 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:23:22 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:23:22 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-44fd6cba-153e-40ee-9556-39e3b07eeedf tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] [instance: f6bc212d-85cd-479c-92ae-112c9f0101bc] Calling os-brick to attach iSCSI Volume {{(pid=75234) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Nov 29 03:23:22 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-44fd6cba-153e-40ee-9556-39e3b07eeedf tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f8a5050c9a0>, 'connection_properties': {'target_iqn': 'iqn.2010-10.org.openstack:6af4b65b-b038-462a-b797-7c57c8e8d680', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'suRxdGNK', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:176}}
Nov 29 03:23:22 user nova-compute[75234]: WARNING os_brick.initiator.connectors.base [None req-44fd6cba-153e-40ee-9556-39e3b07eeedf tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Nov 29 03:23:22 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-44fd6cba-153e-40ee-9556-39e3b07eeedf tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:68}}
Nov 29 03:23:22 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-44fd6cba-153e-40ee-9556-39e3b07eeedf tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.002s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:73}}
Nov 29 03:23:22 user nova-compute[75234]: WARNING os_brick.initiator.connectors.base [None req-44fd6cba-153e-40ee-9556-39e3b07eeedf tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Nov 29 03:23:22 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-44fd6cba-153e-40ee-9556-39e3b07eeedf tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:6af4b65b-b038-462a-b797-7c57c8e8d680" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:68}}
Nov 29 03:23:22 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-44fd6cba-153e-40ee-9556-39e3b07eeedf tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:6af4b65b-b038-462a-b797-7c57c8e8d680" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:73}}
Nov 29 03:23:22 user nova-compute[75234]: INFO os_brick.initiator.connectors.iscsi [None req-44fd6cba-153e-40ee-9556-39e3b07eeedf tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Nov 29 03:23:22 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:6af4b65b-b038-462a-b797-7c57c8e8d680 -p 172.16.0.220:3260 {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:23:22 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:6af4b65b-b038-462a-b797-7c57c8e8d680 -p 172.16.0.220:3260" returned: 21 in 0.020s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:23:22 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[1f0a2c60-068b-442a-98a1-31fc149b822c]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:23:22 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-44fd6cba-153e-40ee-9556-39e3b07eeedf tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Nov 29 03:23:22 user nova-compute[75234]:  {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:23:22 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:6af4b65b-b038-462a-b797-7c57c8e8d680 -p 172.16.0.220:3260 --interface default --op new {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:23:22 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:6af4b65b-b038-462a-b797-7c57c8e8d680 -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.026s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:23:22 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[54aa1c85-f3dd-4f1e-a5cd-cb5c9b414892]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:6af4b65b-b038-462a-b797-7c57c8e8d680] added\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:23:22 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-44fd6cba-153e-40ee-9556-39e3b07eeedf tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:6af4b65b-b038-462a-b797-7c57c8e8d680] added
Nov 29 03:23:22 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:23:22 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:6af4b65b-b038-462a-b797-7c57c8e8d680 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:23:22 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:6af4b65b-b038-462a-b797-7c57c8e8d680 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.024s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:23:22 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[f9a54946-abca-456f-a24e-10a6baeba8fc]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:23:22 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-44fd6cba-153e-40ee-9556-39e3b07eeedf tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:23:22 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:6af4b65b-b038-462a-b797-7c57c8e8d680 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:23:22 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:6af4b65b-b038-462a-b797-7c57c8e8d680 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.034s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:23:22 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[82f38e87-da0a-41da-b117-66412a846713]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:23:22 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-44fd6cba-153e-40ee-9556-39e3b07eeedf tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:23:22 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:6af4b65b-b038-462a-b797-7c57c8e8d680 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v suRxdGNK {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:23:22 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:6af4b65b-b038-462a-b797-7c57c8e8d680 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v suRxdGNK" returned: 0 in 0.037s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:23:22 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[b9a656dd-a80e-4846-bbe3-f0a37d02406f]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:23:22 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-44fd6cba-153e-40ee-9556-39e3b07eeedf tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'suRxdGNK'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:23:22 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:6af4b65b-b038-462a-b797-7c57c8e8d680 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:23:22 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:6af4b65b-b038-462a-b797-7c57c8e8d680 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.021s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:23:22 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[39f4a0d6-6829-4054-8d6f-5a1a36072c2c]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:23:22 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-44fd6cba-153e-40ee-9556-39e3b07eeedf tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:23:22 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:23:22 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.022s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:23:22 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[5adf551d-0f01-4672-a714-9602e9480090]: (4, ('tcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:cf6ab229-533a-424e-bd67-1d54d50ad4be (non-flash)\ntcp: [39] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:41f08854-31dd-4172-a5cf-01b0e695c10f (non-flash)\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:23:22 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-44fd6cba-153e-40ee-9556-39e3b07eeedf tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:cf6ab229-533a-424e-bd67-1d54d50ad4be (non-flash)
Nov 29 03:23:22 user nova-compute[75234]: tcp: [39] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:41f08854-31dd-4172-a5cf-01b0e695c10f (non-flash)
Nov 29 03:23:22 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:23:22 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-44fd6cba-153e-40ee-9556-39e3b07eeedf tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] iscsi session list stdout=tcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:cf6ab229-533a-424e-bd67-1d54d50ad4be (non-flash)
Nov 29 03:23:22 user nova-compute[75234]: tcp: [39] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:41f08854-31dd-4172-a5cf-01b0e695c10f (non-flash)
Nov 29 03:23:22 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsi_session /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Nov 29 03:23:22 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:6af4b65b-b038-462a-b797-7c57c8e8d680 -p 172.16.0.220:3260 --login {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:23:22 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:6af4b65b-b038-462a-b797-7c57c8e8d680 -p 172.16.0.220:3260 --login" returned: 0 in 0.068s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:23:22 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[cdfc05eb-11a3-4a2d-af12-1253b2013dd2]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:6af4b65b-b038-462a-b797-7c57c8e8d680, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:6af4b65b-b038-462a-b797-7c57c8e8d680, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:23:22 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-44fd6cba-153e-40ee-9556-39e3b07eeedf tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:6af4b65b-b038-462a-b797-7c57c8e8d680, portal: 172.16.0.220,3260]
Nov 29 03:23:22 user nova-compute[75234]: Login to [iface: default, target: iqn.2010-10.org.openstack:6af4b65b-b038-462a-b797-7c57c8e8d680, portal: 172.16.0.220,3260] successful.
Nov 29 03:23:22 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:23:22 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:6af4b65b-b038-462a-b797-7c57c8e8d680 -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:23:23 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:6af4b65b-b038-462a-b797-7c57c8e8d680 -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.022s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:23:23 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[c96ec42f-1e41-40f4-b869-40416c049d9d]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:23:23 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-44fd6cba-153e-40ee-9556-39e3b07eeedf tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:23:23 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:23:23 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.027s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:23:23 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[e0454bef-9629-4ef6-b339-9a5b0ec1a7a2]: (4, ('tcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:cf6ab229-533a-424e-bd67-1d54d50ad4be (non-flash)\ntcp: [39] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:41f08854-31dd-4172-a5cf-01b0e695c10f (non-flash)\ntcp: [41] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6af4b65b-b038-462a-b797-7c57c8e8d680 (non-flash)\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:23:23 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-44fd6cba-153e-40ee-9556-39e3b07eeedf tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:cf6ab229-533a-424e-bd67-1d54d50ad4be (non-flash)
Nov 29 03:23:23 user nova-compute[75234]: tcp: [39] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:41f08854-31dd-4172-a5cf-01b0e695c10f (non-flash)
Nov 29 03:23:23 user nova-compute[75234]: tcp: [41] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6af4b65b-b038-462a-b797-7c57c8e8d680 (non-flash)
Nov 29 03:23:23 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:23:23 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-44fd6cba-153e-40ee-9556-39e3b07eeedf tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] iscsi session list stdout=tcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:cf6ab229-533a-424e-bd67-1d54d50ad4be (non-flash)
Nov 29 03:23:23 user nova-compute[75234]: tcp: [39] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:41f08854-31dd-4172-a5cf-01b0e695c10f (non-flash)
Nov 29 03:23:23 user nova-compute[75234]: tcp: [41] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6af4b65b-b038-462a-b797-7c57c8e8d680 (non-flash)
Nov 29 03:23:23 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsi_session /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Nov 29 03:23:23 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-44fd6cba-153e-40ee-9556-39e3b07eeedf tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:6af4b65b-b038-462a-b797-7c57c8e8d680" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.349s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:87}}
Nov 29 03:23:23 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-44fd6cba-153e-40ee-9556-39e3b07eeedf tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Connected to 172.16.0.220:3260 {{(pid=75234) _connect_vol /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:644}}
Nov 29 03:23:23 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-44fd6cba-153e-40ee-9556-39e3b07eeedf tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] HCTL ('34', '-', '-', 0) found on session 41 with lun 0 {{(pid=75234) get_hctl /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:770}}
Nov 29 03:23:23 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-44fd6cba-153e-40ee-9556-39e3b07eeedf tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Scanning host 34 c: -, t: -, l: 0) {{(pid=75234) scan_iscsi /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:795}}
Nov 29 03:23:23 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:23:23 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host34/scan {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:23:23 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host34/scan" returned: 0 in 0.019s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:23:23 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[ed0220da-32f2-42fc-843f-de9bc6111c24]: (4, ('- - 0', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:23:23 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-44fd6cba-153e-40ee-9556-39e3b07eeedf tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Searching for a device in session 41 and hctl ['34', '*', '*', 0] yield: sdc {{(pid=75234) device_name_by_hctl /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:789}}
Nov 29 03:23:23 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-44fd6cba-153e-40ee-9556-39e3b07eeedf tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Connected to sdc using {'target_iqn': 'iqn.2010-10.org.openstack:6af4b65b-b038-462a-b797-7c57c8e8d680', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'suRxdGNK', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False} {{(pid=75234) _connect_vol /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:672}}
Nov 29 03:23:24 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-44fd6cba-153e-40ee-9556-39e3b07eeedf tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.386s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:87}}
Nov 29 03:23:24 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-44fd6cba-153e-40ee-9556-39e3b07eeedf tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] <== connect_volume: return (1387ms) {'type': 'block', 'scsi_wwn': '23365303538626637', 'path': '/dev/sdc'} {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:203}}
Nov 29 03:23:24 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-44fd6cba-153e-40ee-9556-39e3b07eeedf tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] [instance: f6bc212d-85cd-479c-92ae-112c9f0101bc] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '23365303538626637', 'path': '/dev/sdc'} {{(pid=75234) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Nov 29 03:23:24 user nova-compute[75234]: DEBUG nova.objects.instance [None req-44fd6cba-153e-40ee-9556-39e3b07eeedf tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Lazy-loading 'flavor' on Instance uuid f6bc212d-85cd-479c-92ae-112c9f0101bc {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:23:24 user nova-compute[75234]: DEBUG nova.virt.libvirt.guest [None req-44fd6cba-153e-40ee-9556-39e3b07eeedf tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] attach device xml: <disk type="block" device="disk">
Nov 29 03:23:24 user nova-compute[75234]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Nov 29 03:23:24 user nova-compute[75234]:   <source dev="/dev/sdc"/>
Nov 29 03:23:24 user nova-compute[75234]:   <target dev="vdb" bus="virtio"/>
Nov 29 03:23:24 user nova-compute[75234]:   <serial>6af4b65b-b038-462a-b797-7c57c8e8d680</serial>
Nov 29 03:23:24 user nova-compute[75234]: </disk>
Nov 29 03:23:24 user nova-compute[75234]:  {{(pid=75234) attach_device /opt/stack/nova/nova/virt/libvirt/guest.py:339}}
Nov 29 03:23:24 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-44fd6cba-153e-40ee-9556-39e3b07eeedf tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] No BDM found with device name vda, not building metadata. {{(pid=75234) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12089}}
Nov 29 03:23:24 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-44fd6cba-153e-40ee-9556-39e3b07eeedf tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] No BDM found with device name vdb, not building metadata. {{(pid=75234) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12089}}
Nov 29 03:23:24 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-44fd6cba-153e-40ee-9556-39e3b07eeedf tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] No VIF found with MAC fa:16:3e:4e:8e:1d, not building metadata {{(pid=75234) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Nov 29 03:23:24 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-44fd6cba-153e-40ee-9556-39e3b07eeedf tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Lock "f6bc212d-85cd-479c-92ae-112c9f0101bc" "released" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: held 10.120s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:23:26 user nova-compute[75234]: INFO nova.compute.manager [None req-7484a23d-7521-43cd-ab79-89965c6a7eeb tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] [instance: af699bc2-9daa-4b5a-9b4a-ad92fe857d70] Get console output
Nov 29 03:23:26 user nova-compute[75234]: INFO nova.privsep.libvirt [-] Ignored error while reading from instance console pty: can't concat NoneType to bytes
Nov 29 03:23:26 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:23:27 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:23:28 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:23:32 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:23:32 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:23:33 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:23:35 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:23:37 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:23:38 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:23:38 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-d27d41dd-ba12-4572-bfbb-b7497dc432eb tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] Acquiring lock "946b919b-39ae-4d23-aefd-01952f4b40b6" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:23:38 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-d27d41dd-ba12-4572-bfbb-b7497dc432eb tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] Lock "946b919b-39ae-4d23-aefd-01952f4b40b6" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:23:38 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-d27d41dd-ba12-4572-bfbb-b7497dc432eb tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] Acquiring lock "946b919b-39ae-4d23-aefd-01952f4b40b6-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:23:38 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-d27d41dd-ba12-4572-bfbb-b7497dc432eb tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] Lock "946b919b-39ae-4d23-aefd-01952f4b40b6-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:23:38 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-d27d41dd-ba12-4572-bfbb-b7497dc432eb tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] Lock "946b919b-39ae-4d23-aefd-01952f4b40b6-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:23:38 user nova-compute[75234]: INFO nova.compute.manager [None req-d27d41dd-ba12-4572-bfbb-b7497dc432eb tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] [instance: 946b919b-39ae-4d23-aefd-01952f4b40b6] Terminating instance
Nov 29 03:23:38 user nova-compute[75234]: DEBUG nova.compute.manager [None req-d27d41dd-ba12-4572-bfbb-b7497dc432eb tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] [instance: 946b919b-39ae-4d23-aefd-01952f4b40b6] Start destroying the instance on the hypervisor. {{(pid=75234) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3140}}
Nov 29 03:23:38 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:23:38 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:23:38 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:23:38 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:23:39 user nova-compute[75234]: DEBUG nova.compute.manager [req-e5d06295-fe02-4b54-8c0b-9f5d54ed4229 req-d8027c68-1969-430f-b4a5-be861fec9a8b service nova] [instance: 946b919b-39ae-4d23-aefd-01952f4b40b6] Received event network-vif-unplugged-e1f71ce6-58f3-4665-afc5-f92d1b6801c5 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:23:39 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-e5d06295-fe02-4b54-8c0b-9f5d54ed4229 req-d8027c68-1969-430f-b4a5-be861fec9a8b service nova] Acquiring lock "946b919b-39ae-4d23-aefd-01952f4b40b6-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:23:39 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-e5d06295-fe02-4b54-8c0b-9f5d54ed4229 req-d8027c68-1969-430f-b4a5-be861fec9a8b service nova] Lock "946b919b-39ae-4d23-aefd-01952f4b40b6-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:23:39 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-e5d06295-fe02-4b54-8c0b-9f5d54ed4229 req-d8027c68-1969-430f-b4a5-be861fec9a8b service nova] Lock "946b919b-39ae-4d23-aefd-01952f4b40b6-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:23:39 user nova-compute[75234]: DEBUG nova.compute.manager [req-e5d06295-fe02-4b54-8c0b-9f5d54ed4229 req-d8027c68-1969-430f-b4a5-be861fec9a8b service nova] [instance: 946b919b-39ae-4d23-aefd-01952f4b40b6] No waiting events found dispatching network-vif-unplugged-e1f71ce6-58f3-4665-afc5-f92d1b6801c5 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:23:39 user nova-compute[75234]: DEBUG nova.compute.manager [req-e5d06295-fe02-4b54-8c0b-9f5d54ed4229 req-d8027c68-1969-430f-b4a5-be861fec9a8b service nova] [instance: 946b919b-39ae-4d23-aefd-01952f4b40b6] Received event network-vif-unplugged-e1f71ce6-58f3-4665-afc5-f92d1b6801c5 for instance with task_state deleting. {{(pid=75234) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10890}}
Nov 29 03:23:39 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:23:39 user nova-compute[75234]: INFO nova.virt.libvirt.driver [-] [instance: 946b919b-39ae-4d23-aefd-01952f4b40b6] Instance destroyed successfully.
Nov 29 03:23:39 user nova-compute[75234]: DEBUG nova.objects.instance [None req-d27d41dd-ba12-4572-bfbb-b7497dc432eb tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] Lazy-loading 'resources' on Instance uuid 946b919b-39ae-4d23-aefd-01952f4b40b6 {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:23:39 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-d27d41dd-ba12-4572-bfbb-b7497dc432eb tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='True',created_at=2023-11-29T03:20:14Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description=None,display_name='tempest-device-tagging-server-1777937262',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-device-tagging-server-1777937262',id=35,image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBJrVo4ngcPAPF4oP6xidv+ykI7O7g9y5APrTr7iXhruQjDIx7zqy1/omzWf7G2p8xg1QAr8Js8NrVgD6XT2ZMw0fSSCLDKoKJk+X3O2rM5aB1leyuLU4v60YiI/3ZRg9Ew==',key_name='tempest-keypair-1887924020',keypairs=<?>,launch_index=0,launched_at=2023-11-29T03:20:21Z,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='e28b5baf29b14aa4b2b98cb1b615705f',ramdisk_id='',reservation_id='r-bfdfrqjg',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-TaggedAttachmentsTest-399278293',owner_user_name='tempest-TaggedAttachmentsTest-399278293-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-11-29T03:20:22Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='522c3c107614404fbe3af8c38fdd189e',uuid=946b919b-39ae-4d23-aefd-01952f4b40b6,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "e1f71ce6-58f3-4665-afc5-f92d1b6801c5", "address": "fa:16:3e:81:ca:ab", "network": {"id": "93ca21e4-138b-4f00-8d1d-b30d63dbe7aa", "bridge": "br-int", "label": "tempest-TaggedAttachmentsTest-1844288661-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.47", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "e28b5baf29b14aa4b2b98cb1b615705f", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape1f71ce6-58", "ovs_interfaceid": "e1f71ce6-58f3-4665-afc5-f92d1b6801c5", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Nov 29 03:23:39 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-d27d41dd-ba12-4572-bfbb-b7497dc432eb tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] Converting VIF {"id": "e1f71ce6-58f3-4665-afc5-f92d1b6801c5", "address": "fa:16:3e:81:ca:ab", "network": {"id": "93ca21e4-138b-4f00-8d1d-b30d63dbe7aa", "bridge": "br-int", "label": "tempest-TaggedAttachmentsTest-1844288661-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.47", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "e28b5baf29b14aa4b2b98cb1b615705f", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape1f71ce6-58", "ovs_interfaceid": "e1f71ce6-58f3-4665-afc5-f92d1b6801c5", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:23:39 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-d27d41dd-ba12-4572-bfbb-b7497dc432eb tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:81:ca:ab,bridge_name='br-int',has_traffic_filtering=True,id=e1f71ce6-58f3-4665-afc5-f92d1b6801c5,network=Network(93ca21e4-138b-4f00-8d1d-b30d63dbe7aa),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tape1f71ce6-58') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:23:39 user nova-compute[75234]: DEBUG os_vif [None req-d27d41dd-ba12-4572-bfbb-b7497dc432eb tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] Unplugging vif VIFOpenVSwitch(active=True,address=fa:16:3e:81:ca:ab,bridge_name='br-int',has_traffic_filtering=True,id=e1f71ce6-58f3-4665-afc5-f92d1b6801c5,network=Network(93ca21e4-138b-4f00-8d1d-b30d63dbe7aa),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tape1f71ce6-58') {{(pid=75234) unplug /opt/stack/data/venv/lib/python3.10/site-packages/os_vif/__init__.py:109}}
Nov 29 03:23:39 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:23:39 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tape1f71ce6-58, bridge=br-int, if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:23:39 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:23:39 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:23:39 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:23:39 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbDestroyCommand(_result=None, table=QoS, record=2288fbe0-80b5-4935-b658-10771df36056) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:23:39 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:23:39 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:23:39 user nova-compute[75234]: INFO os_vif [None req-d27d41dd-ba12-4572-bfbb-b7497dc432eb tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] Successfully unplugged vif VIFOpenVSwitch(active=True,address=fa:16:3e:81:ca:ab,bridge_name='br-int',has_traffic_filtering=True,id=e1f71ce6-58f3-4665-afc5-f92d1b6801c5,network=Network(93ca21e4-138b-4f00-8d1d-b30d63dbe7aa),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tape1f71ce6-58')
Nov 29 03:23:39 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-d27d41dd-ba12-4572-bfbb-b7497dc432eb tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] [instance: 946b919b-39ae-4d23-aefd-01952f4b40b6] Deleting instance files /opt/stack/data/nova/instances/946b919b-39ae-4d23-aefd-01952f4b40b6_del
Nov 29 03:23:39 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-d27d41dd-ba12-4572-bfbb-b7497dc432eb tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] [instance: 946b919b-39ae-4d23-aefd-01952f4b40b6] Deletion of /opt/stack/data/nova/instances/946b919b-39ae-4d23-aefd-01952f4b40b6_del complete
Nov 29 03:23:39 user nova-compute[75234]: INFO nova.compute.manager [None req-d27d41dd-ba12-4572-bfbb-b7497dc432eb tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] [instance: 946b919b-39ae-4d23-aefd-01952f4b40b6] Took 0.91 seconds to destroy the instance on the hypervisor.
Nov 29 03:23:39 user nova-compute[75234]: DEBUG oslo.service.loopingcall [None req-d27d41dd-ba12-4572-bfbb-b7497dc432eb tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=75234) func /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/loopingcall.py:435}}
Nov 29 03:23:39 user nova-compute[75234]: DEBUG nova.compute.manager [-] [instance: 946b919b-39ae-4d23-aefd-01952f4b40b6] Deallocating network for instance {{(pid=75234) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2279}}
Nov 29 03:23:39 user nova-compute[75234]: DEBUG nova.network.neutron [-] [instance: 946b919b-39ae-4d23-aefd-01952f4b40b6] deallocate_for_instance() {{(pid=75234) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1802}}
Nov 29 03:23:40 user nova-compute[75234]: DEBUG nova.network.neutron [-] [instance: 946b919b-39ae-4d23-aefd-01952f4b40b6] Updating instance_info_cache with network_info: [] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:23:40 user nova-compute[75234]: INFO nova.compute.manager [-] [instance: 946b919b-39ae-4d23-aefd-01952f4b40b6] Took 0.69 seconds to deallocate network for instance.
Nov 29 03:23:40 user nova-compute[75234]: DEBUG nova.compute.manager [req-67432815-2ccc-43b1-b3ab-cbfc4fda448d req-918067ad-1788-4921-b329-dadae0a624bb service nova] [instance: 946b919b-39ae-4d23-aefd-01952f4b40b6] Received event network-vif-deleted-e1f71ce6-58f3-4665-afc5-f92d1b6801c5 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:23:40 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-d27d41dd-ba12-4572-bfbb-b7497dc432eb tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:23:40 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-d27d41dd-ba12-4572-bfbb-b7497dc432eb tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:23:40 user nova-compute[75234]: DEBUG nova.compute.provider_tree [None req-d27d41dd-ba12-4572-bfbb-b7497dc432eb tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] Inventory has not changed in ProviderTree for provider: 4d291220-7c37-4839-a0e3-b23c66502b74 {{(pid=75234) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Nov 29 03:23:40 user nova-compute[75234]: DEBUG nova.scheduler.client.report [None req-d27d41dd-ba12-4572-bfbb-b7497dc432eb tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] Inventory has not changed for provider 4d291220-7c37-4839-a0e3-b23c66502b74 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=75234) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Nov 29 03:23:40 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-d27d41dd-ba12-4572-bfbb-b7497dc432eb tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.439s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:23:40 user nova-compute[75234]: INFO nova.scheduler.client.report [None req-d27d41dd-ba12-4572-bfbb-b7497dc432eb tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] Deleted allocations for instance 946b919b-39ae-4d23-aefd-01952f4b40b6
Nov 29 03:23:40 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-d27d41dd-ba12-4572-bfbb-b7497dc432eb tempest-TaggedAttachmentsTest-399278293 tempest-TaggedAttachmentsTest-399278293-project-member] Lock "946b919b-39ae-4d23-aefd-01952f4b40b6" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 2.344s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:23:41 user nova-compute[75234]: DEBUG nova.compute.manager [req-bbc47d5e-629c-4b0e-8cea-69b7242848b7 req-088e1325-478a-452e-9e1f-dcfd4fb0e902 service nova] [instance: 946b919b-39ae-4d23-aefd-01952f4b40b6] Received event network-vif-plugged-e1f71ce6-58f3-4665-afc5-f92d1b6801c5 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:23:41 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-bbc47d5e-629c-4b0e-8cea-69b7242848b7 req-088e1325-478a-452e-9e1f-dcfd4fb0e902 service nova] Acquiring lock "946b919b-39ae-4d23-aefd-01952f4b40b6-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:23:41 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-bbc47d5e-629c-4b0e-8cea-69b7242848b7 req-088e1325-478a-452e-9e1f-dcfd4fb0e902 service nova] Lock "946b919b-39ae-4d23-aefd-01952f4b40b6-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:23:41 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-bbc47d5e-629c-4b0e-8cea-69b7242848b7 req-088e1325-478a-452e-9e1f-dcfd4fb0e902 service nova] Lock "946b919b-39ae-4d23-aefd-01952f4b40b6-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:23:41 user nova-compute[75234]: DEBUG nova.compute.manager [req-bbc47d5e-629c-4b0e-8cea-69b7242848b7 req-088e1325-478a-452e-9e1f-dcfd4fb0e902 service nova] [instance: 946b919b-39ae-4d23-aefd-01952f4b40b6] No waiting events found dispatching network-vif-plugged-e1f71ce6-58f3-4665-afc5-f92d1b6801c5 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:23:41 user nova-compute[75234]: WARNING nova.compute.manager [req-bbc47d5e-629c-4b0e-8cea-69b7242848b7 req-088e1325-478a-452e-9e1f-dcfd4fb0e902 service nova] [instance: 946b919b-39ae-4d23-aefd-01952f4b40b6] Received unexpected event network-vif-plugged-e1f71ce6-58f3-4665-afc5-f92d1b6801c5 for instance with vm_state deleted and task_state None.
Nov 29 03:23:41 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:23:42 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:23:43 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-529a682b-48a4-4421-aab5-f015238d97b8 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Acquiring lock "f6bc212d-85cd-479c-92ae-112c9f0101bc" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:23:43 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-529a682b-48a4-4421-aab5-f015238d97b8 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Lock "f6bc212d-85cd-479c-92ae-112c9f0101bc" acquired by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:23:43 user nova-compute[75234]: INFO nova.compute.manager [None req-529a682b-48a4-4421-aab5-f015238d97b8 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] [instance: f6bc212d-85cd-479c-92ae-112c9f0101bc] Detaching volume 6af4b65b-b038-462a-b797-7c57c8e8d680
Nov 29 03:23:43 user nova-compute[75234]: INFO nova.virt.block_device [None req-529a682b-48a4-4421-aab5-f015238d97b8 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] [instance: f6bc212d-85cd-479c-92ae-112c9f0101bc] Attempting to driver detach volume 6af4b65b-b038-462a-b797-7c57c8e8d680 from mountpoint /dev/vdb
Nov 29 03:23:43 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-529a682b-48a4-4421-aab5-f015238d97b8 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Attempting to detach device vdb from instance f6bc212d-85cd-479c-92ae-112c9f0101bc from the persistent domain config. {{(pid=75234) _detach_from_persistent /opt/stack/nova/nova/virt/libvirt/driver.py:2487}}
Nov 29 03:23:43 user nova-compute[75234]: DEBUG nova.virt.libvirt.guest [None req-529a682b-48a4-4421-aab5-f015238d97b8 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] detach device xml: <disk type="block" device="disk">
Nov 29 03:23:43 user nova-compute[75234]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Nov 29 03:23:43 user nova-compute[75234]:   <source dev="/dev/sdc"/>
Nov 29 03:23:43 user nova-compute[75234]:   <target dev="vdb" bus="virtio"/>
Nov 29 03:23:43 user nova-compute[75234]:   <serial>6af4b65b-b038-462a-b797-7c57c8e8d680</serial>
Nov 29 03:23:43 user nova-compute[75234]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x07" function="0x0"/>
Nov 29 03:23:43 user nova-compute[75234]: </disk>
Nov 29 03:23:43 user nova-compute[75234]:  {{(pid=75234) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Nov 29 03:23:43 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-529a682b-48a4-4421-aab5-f015238d97b8 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Successfully detached device vdb from instance f6bc212d-85cd-479c-92ae-112c9f0101bc from the persistent domain config.
Nov 29 03:23:43 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-529a682b-48a4-4421-aab5-f015238d97b8 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] (1/8): Attempting to detach device vdb with device alias virtio-disk1 from instance f6bc212d-85cd-479c-92ae-112c9f0101bc from the live domain config. {{(pid=75234) _detach_from_live_with_retry /opt/stack/nova/nova/virt/libvirt/driver.py:2523}}
Nov 29 03:23:43 user nova-compute[75234]: DEBUG nova.virt.libvirt.guest [None req-529a682b-48a4-4421-aab5-f015238d97b8 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] detach device xml: <disk type="block" device="disk">
Nov 29 03:23:43 user nova-compute[75234]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Nov 29 03:23:43 user nova-compute[75234]:   <source dev="/dev/sdc"/>
Nov 29 03:23:43 user nova-compute[75234]:   <target dev="vdb" bus="virtio"/>
Nov 29 03:23:43 user nova-compute[75234]:   <serial>6af4b65b-b038-462a-b797-7c57c8e8d680</serial>
Nov 29 03:23:43 user nova-compute[75234]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x07" function="0x0"/>
Nov 29 03:23:43 user nova-compute[75234]: </disk>
Nov 29 03:23:43 user nova-compute[75234]:  {{(pid=75234) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Nov 29 03:23:44 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-529a682b-48a4-4421-aab5-f015238d97b8 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Start waiting for the detach event from libvirt for device vdb with device alias virtio-disk1 for instance f6bc212d-85cd-479c-92ae-112c9f0101bc {{(pid=75234) _detach_from_live_and_wait_for_event /opt/stack/nova/nova/virt/libvirt/driver.py:2599}}
Nov 29 03:23:44 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:23:45 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-201f89b9-1324-48f4-9cdb-c94c9b58ffbd tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Acquiring lock "5d7d744e-1a5f-40ab-b961-4b58c183efe0" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:23:45 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-201f89b9-1324-48f4-9cdb-c94c9b58ffbd tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Lock "5d7d744e-1a5f-40ab-b961-4b58c183efe0" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:23:46 user nova-compute[75234]: DEBUG nova.compute.manager [None req-201f89b9-1324-48f4-9cdb-c94c9b58ffbd tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] [instance: 5d7d744e-1a5f-40ab-b961-4b58c183efe0] Starting instance... {{(pid=75234) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2422}}
Nov 29 03:23:46 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-201f89b9-1324-48f4-9cdb-c94c9b58ffbd tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:23:46 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-201f89b9-1324-48f4-9cdb-c94c9b58ffbd tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:23:46 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-201f89b9-1324-48f4-9cdb-c94c9b58ffbd tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=75234) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2366}}
Nov 29 03:23:46 user nova-compute[75234]: INFO nova.compute.claims [None req-201f89b9-1324-48f4-9cdb-c94c9b58ffbd tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] [instance: 5d7d744e-1a5f-40ab-b961-4b58c183efe0] Claim successful on node user
Nov 29 03:23:46 user nova-compute[75234]: DEBUG nova.compute.provider_tree [None req-201f89b9-1324-48f4-9cdb-c94c9b58ffbd tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Inventory has not changed in ProviderTree for provider: 4d291220-7c37-4839-a0e3-b23c66502b74 {{(pid=75234) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Nov 29 03:23:46 user nova-compute[75234]: DEBUG nova.scheduler.client.report [None req-201f89b9-1324-48f4-9cdb-c94c9b58ffbd tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Inventory has not changed for provider 4d291220-7c37-4839-a0e3-b23c66502b74 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=75234) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Nov 29 03:23:46 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-201f89b9-1324-48f4-9cdb-c94c9b58ffbd tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.367s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:23:46 user nova-compute[75234]: DEBUG nova.compute.manager [None req-201f89b9-1324-48f4-9cdb-c94c9b58ffbd tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] [instance: 5d7d744e-1a5f-40ab-b961-4b58c183efe0] Start building networks asynchronously for instance. {{(pid=75234) _build_resources /opt/stack/nova/nova/compute/manager.py:2819}}
Nov 29 03:23:46 user nova-compute[75234]: DEBUG nova.compute.manager [None req-201f89b9-1324-48f4-9cdb-c94c9b58ffbd tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] [instance: 5d7d744e-1a5f-40ab-b961-4b58c183efe0] Allocating IP information in the background. {{(pid=75234) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1972}}
Nov 29 03:23:46 user nova-compute[75234]: DEBUG nova.network.neutron [None req-201f89b9-1324-48f4-9cdb-c94c9b58ffbd tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] [instance: 5d7d744e-1a5f-40ab-b961-4b58c183efe0] allocate_for_instance() {{(pid=75234) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1156}}
Nov 29 03:23:46 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-201f89b9-1324-48f4-9cdb-c94c9b58ffbd tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] [instance: 5d7d744e-1a5f-40ab-b961-4b58c183efe0] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Nov 29 03:23:46 user nova-compute[75234]: DEBUG nova.compute.manager [None req-201f89b9-1324-48f4-9cdb-c94c9b58ffbd tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] [instance: 5d7d744e-1a5f-40ab-b961-4b58c183efe0] Start building block device mappings for instance. {{(pid=75234) _build_resources /opt/stack/nova/nova/compute/manager.py:2854}}
Nov 29 03:23:46 user nova-compute[75234]: DEBUG nova.policy [None req-201f89b9-1324-48f4-9cdb-c94c9b58ffbd tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '3a02618f56ea45898ab4783a4f198b5d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '74f132d6666c47b8b43869df6fee6de3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=75234) authorize /opt/stack/nova/nova/policy.py:203}}
Nov 29 03:23:46 user nova-compute[75234]: DEBUG nova.compute.manager [None req-201f89b9-1324-48f4-9cdb-c94c9b58ffbd tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] [instance: 5d7d744e-1a5f-40ab-b961-4b58c183efe0] Start spawning the instance on the hypervisor. {{(pid=75234) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2628}}
Nov 29 03:23:46 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-201f89b9-1324-48f4-9cdb-c94c9b58ffbd tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] [instance: 5d7d744e-1a5f-40ab-b961-4b58c183efe0] Creating instance directory {{(pid=75234) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4727}}
Nov 29 03:23:46 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-201f89b9-1324-48f4-9cdb-c94c9b58ffbd tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] [instance: 5d7d744e-1a5f-40ab-b961-4b58c183efe0] Creating image(s)
Nov 29 03:23:46 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-201f89b9-1324-48f4-9cdb-c94c9b58ffbd tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Acquiring lock "/opt/stack/data/nova/instances/5d7d744e-1a5f-40ab-b961-4b58c183efe0/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:23:46 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-201f89b9-1324-48f4-9cdb-c94c9b58ffbd tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Lock "/opt/stack/data/nova/instances/5d7d744e-1a5f-40ab-b961-4b58c183efe0/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:23:46 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-201f89b9-1324-48f4-9cdb-c94c9b58ffbd tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Lock "/opt/stack/data/nova/instances/5d7d744e-1a5f-40ab-b961-4b58c183efe0/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.002s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:23:46 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-201f89b9-1324-48f4-9cdb-c94c9b58ffbd tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Acquiring lock "4d5d356d29424aea82c5ad651579c796fc09ff4d" by "nova.virt.libvirt.imagebackend.Image.cache.<locals>.fetch_func_sync" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:23:46 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-201f89b9-1324-48f4-9cdb-c94c9b58ffbd tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Lock "4d5d356d29424aea82c5ad651579c796fc09ff4d" acquired by "nova.virt.libvirt.imagebackend.Image.cache.<locals>.fetch_func_sync" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:23:47 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-201f89b9-1324-48f4-9cdb-c94c9b58ffbd tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/4d5d356d29424aea82c5ad651579c796fc09ff4d.part --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:23:47 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:23:47 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-201f89b9-1324-48f4-9cdb-c94c9b58ffbd tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/4d5d356d29424aea82c5ad651579c796fc09ff4d.part --force-share --output=json" returned: 0 in 0.181s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:23:47 user nova-compute[75234]: DEBUG nova.virt.images [None req-201f89b9-1324-48f4-9cdb-c94c9b58ffbd tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] 7107c48b-05b9-4052-917d-814b721d169c was qcow2, converting to raw {{(pid=75234) fetch_to_raw /opt/stack/nova/nova/virt/images.py:165}}
Nov 29 03:23:47 user nova-compute[75234]: DEBUG nova.privsep.utils [None req-201f89b9-1324-48f4-9cdb-c94c9b58ffbd tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Path '/opt/stack/data/nova/instances' supports direct I/O {{(pid=75234) supports_direct_io /opt/stack/nova/nova/privsep/utils.py:63}}
Nov 29 03:23:47 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-201f89b9-1324-48f4-9cdb-c94c9b58ffbd tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Running cmd (subprocess): qemu-img convert -t none -O raw -f qcow2 /opt/stack/data/nova/instances/_base/4d5d356d29424aea82c5ad651579c796fc09ff4d.part /opt/stack/data/nova/instances/_base/4d5d356d29424aea82c5ad651579c796fc09ff4d.converted {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:23:47 user nova-compute[75234]: DEBUG nova.network.neutron [None req-201f89b9-1324-48f4-9cdb-c94c9b58ffbd tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] [instance: 5d7d744e-1a5f-40ab-b961-4b58c183efe0] Successfully created port: fa65b32f-b627-45fa-8d6d-d99012ca1262 {{(pid=75234) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:548}}
Nov 29 03:23:48 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-201f89b9-1324-48f4-9cdb-c94c9b58ffbd tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] CMD "qemu-img convert -t none -O raw -f qcow2 /opt/stack/data/nova/instances/_base/4d5d356d29424aea82c5ad651579c796fc09ff4d.part /opt/stack/data/nova/instances/_base/4d5d356d29424aea82c5ad651579c796fc09ff4d.converted" returned: 0 in 0.447s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:23:48 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-201f89b9-1324-48f4-9cdb-c94c9b58ffbd tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/4d5d356d29424aea82c5ad651579c796fc09ff4d.converted --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:23:48 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-201f89b9-1324-48f4-9cdb-c94c9b58ffbd tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/4d5d356d29424aea82c5ad651579c796fc09ff4d.converted --force-share --output=json" returned: 0 in 0.184s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:23:48 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-201f89b9-1324-48f4-9cdb-c94c9b58ffbd tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Lock "4d5d356d29424aea82c5ad651579c796fc09ff4d" "released" by "nova.virt.libvirt.imagebackend.Image.cache.<locals>.fetch_func_sync" :: held 1.359s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:23:48 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-201f89b9-1324-48f4-9cdb-c94c9b58ffbd tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/4d5d356d29424aea82c5ad651579c796fc09ff4d --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:23:48 user nova-compute[75234]: DEBUG nova.network.neutron [None req-201f89b9-1324-48f4-9cdb-c94c9b58ffbd tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] [instance: 5d7d744e-1a5f-40ab-b961-4b58c183efe0] Successfully updated port: fa65b32f-b627-45fa-8d6d-d99012ca1262 {{(pid=75234) _update_port /opt/stack/nova/nova/network/neutron.py:586}}
Nov 29 03:23:48 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-201f89b9-1324-48f4-9cdb-c94c9b58ffbd tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Acquiring lock "refresh_cache-5d7d744e-1a5f-40ab-b961-4b58c183efe0" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:23:48 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-201f89b9-1324-48f4-9cdb-c94c9b58ffbd tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Acquired lock "refresh_cache-5d7d744e-1a5f-40ab-b961-4b58c183efe0" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:23:48 user nova-compute[75234]: DEBUG nova.network.neutron [None req-201f89b9-1324-48f4-9cdb-c94c9b58ffbd tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] [instance: 5d7d744e-1a5f-40ab-b961-4b58c183efe0] Building network info cache for instance {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2009}}
Nov 29 03:23:48 user nova-compute[75234]: DEBUG nova.compute.manager [req-c6c1978c-2f08-40e0-89c0-002bcb7f03ee req-d343389e-ddba-4c8a-97ba-e220722d3f33 service nova] [instance: 5d7d744e-1a5f-40ab-b961-4b58c183efe0] Received event network-changed-fa65b32f-b627-45fa-8d6d-d99012ca1262 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:23:48 user nova-compute[75234]: DEBUG nova.compute.manager [req-c6c1978c-2f08-40e0-89c0-002bcb7f03ee req-d343389e-ddba-4c8a-97ba-e220722d3f33 service nova] [instance: 5d7d744e-1a5f-40ab-b961-4b58c183efe0] Refreshing instance network info cache due to event network-changed-fa65b32f-b627-45fa-8d6d-d99012ca1262. {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11117}}
Nov 29 03:23:48 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-c6c1978c-2f08-40e0-89c0-002bcb7f03ee req-d343389e-ddba-4c8a-97ba-e220722d3f33 service nova] Acquiring lock "refresh_cache-5d7d744e-1a5f-40ab-b961-4b58c183efe0" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:23:48 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-201f89b9-1324-48f4-9cdb-c94c9b58ffbd tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/4d5d356d29424aea82c5ad651579c796fc09ff4d --force-share --output=json" returned: 0 in 0.194s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:23:48 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-201f89b9-1324-48f4-9cdb-c94c9b58ffbd tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Acquiring lock "4d5d356d29424aea82c5ad651579c796fc09ff4d" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:23:48 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-201f89b9-1324-48f4-9cdb-c94c9b58ffbd tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Lock "4d5d356d29424aea82c5ad651579c796fc09ff4d" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:23:48 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-201f89b9-1324-48f4-9cdb-c94c9b58ffbd tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/4d5d356d29424aea82c5ad651579c796fc09ff4d --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:23:48 user nova-compute[75234]: DEBUG nova.network.neutron [None req-201f89b9-1324-48f4-9cdb-c94c9b58ffbd tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] [instance: 5d7d744e-1a5f-40ab-b961-4b58c183efe0] Instance cache missing network info. {{(pid=75234) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3322}}
Nov 29 03:23:48 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-201f89b9-1324-48f4-9cdb-c94c9b58ffbd tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/4d5d356d29424aea82c5ad651579c796fc09ff4d --force-share --output=json" returned: 0 in 0.172s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:23:48 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-201f89b9-1324-48f4-9cdb-c94c9b58ffbd tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/4d5d356d29424aea82c5ad651579c796fc09ff4d,backing_fmt=raw /opt/stack/data/nova/instances/5d7d744e-1a5f-40ab-b961-4b58c183efe0/disk 1073741824 {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:23:48 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-201f89b9-1324-48f4-9cdb-c94c9b58ffbd tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/4d5d356d29424aea82c5ad651579c796fc09ff4d,backing_fmt=raw /opt/stack/data/nova/instances/5d7d744e-1a5f-40ab-b961-4b58c183efe0/disk 1073741824" returned: 0 in 0.063s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:23:48 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-201f89b9-1324-48f4-9cdb-c94c9b58ffbd tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Lock "4d5d356d29424aea82c5ad651579c796fc09ff4d" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.240s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:23:48 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-201f89b9-1324-48f4-9cdb-c94c9b58ffbd tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/4d5d356d29424aea82c5ad651579c796fc09ff4d --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:23:48 user nova-compute[75234]: DEBUG nova.network.neutron [None req-201f89b9-1324-48f4-9cdb-c94c9b58ffbd tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] [instance: 5d7d744e-1a5f-40ab-b961-4b58c183efe0] Updating instance_info_cache with network_info: [{"id": "fa65b32f-b627-45fa-8d6d-d99012ca1262", "address": "fa:16:3e:2f:fa:59", "network": {"id": "6787e0a8-2980-4a2e-bf2a-583af955241d", "bridge": "br-int", "label": "tempest-TestMinimumBasicScenario-1273600998-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "74f132d6666c47b8b43869df6fee6de3", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapfa65b32f-b6", "ovs_interfaceid": "fa65b32f-b627-45fa-8d6d-d99012ca1262", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:23:48 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-201f89b9-1324-48f4-9cdb-c94c9b58ffbd tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Releasing lock "refresh_cache-5d7d744e-1a5f-40ab-b961-4b58c183efe0" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:23:48 user nova-compute[75234]: DEBUG nova.compute.manager [None req-201f89b9-1324-48f4-9cdb-c94c9b58ffbd tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] [instance: 5d7d744e-1a5f-40ab-b961-4b58c183efe0] Instance network_info: |[{"id": "fa65b32f-b627-45fa-8d6d-d99012ca1262", "address": "fa:16:3e:2f:fa:59", "network": {"id": "6787e0a8-2980-4a2e-bf2a-583af955241d", "bridge": "br-int", "label": "tempest-TestMinimumBasicScenario-1273600998-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "74f132d6666c47b8b43869df6fee6de3", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapfa65b32f-b6", "ovs_interfaceid": "fa65b32f-b627-45fa-8d6d-d99012ca1262", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=75234) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1987}}
Nov 29 03:23:48 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-c6c1978c-2f08-40e0-89c0-002bcb7f03ee req-d343389e-ddba-4c8a-97ba-e220722d3f33 service nova] Acquired lock "refresh_cache-5d7d744e-1a5f-40ab-b961-4b58c183efe0" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:23:48 user nova-compute[75234]: DEBUG nova.network.neutron [req-c6c1978c-2f08-40e0-89c0-002bcb7f03ee req-d343389e-ddba-4c8a-97ba-e220722d3f33 service nova] [instance: 5d7d744e-1a5f-40ab-b961-4b58c183efe0] Refreshing network info cache for port fa65b32f-b627-45fa-8d6d-d99012ca1262 {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2006}}
Nov 29 03:23:48 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-201f89b9-1324-48f4-9cdb-c94c9b58ffbd tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/4d5d356d29424aea82c5ad651579c796fc09ff4d --force-share --output=json" returned: 0 in 0.175s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:23:48 user nova-compute[75234]: DEBUG nova.virt.disk.api [None req-201f89b9-1324-48f4-9cdb-c94c9b58ffbd tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Checking if we can resize image /opt/stack/data/nova/instances/5d7d744e-1a5f-40ab-b961-4b58c183efe0/disk. size=1073741824 {{(pid=75234) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:166}}
Nov 29 03:23:48 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-201f89b9-1324-48f4-9cdb-c94c9b58ffbd tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/5d7d744e-1a5f-40ab-b961-4b58c183efe0/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:23:49 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-201f89b9-1324-48f4-9cdb-c94c9b58ffbd tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/5d7d744e-1a5f-40ab-b961-4b58c183efe0/disk --force-share --output=json" returned: 0 in 0.175s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:23:49 user nova-compute[75234]: DEBUG nova.virt.disk.api [None req-201f89b9-1324-48f4-9cdb-c94c9b58ffbd tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Cannot resize image /opt/stack/data/nova/instances/5d7d744e-1a5f-40ab-b961-4b58c183efe0/disk to a smaller size. {{(pid=75234) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:172}}
Nov 29 03:23:49 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-201f89b9-1324-48f4-9cdb-c94c9b58ffbd tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] [instance: 5d7d744e-1a5f-40ab-b961-4b58c183efe0] Created local disks {{(pid=75234) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4861}}
Nov 29 03:23:49 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-201f89b9-1324-48f4-9cdb-c94c9b58ffbd tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] [instance: 5d7d744e-1a5f-40ab-b961-4b58c183efe0] Ensure instance console log exists: /opt/stack/data/nova/instances/5d7d744e-1a5f-40ab-b961-4b58c183efe0/console.log {{(pid=75234) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4613}}
Nov 29 03:23:49 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-201f89b9-1324-48f4-9cdb-c94c9b58ffbd tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:23:49 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-201f89b9-1324-48f4-9cdb-c94c9b58ffbd tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:23:49 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-201f89b9-1324-48f4-9cdb-c94c9b58ffbd tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:23:49 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-201f89b9-1324-48f4-9cdb-c94c9b58ffbd tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] [instance: 5d7d744e-1a5f-40ab-b961-4b58c183efe0] Start _get_guest_xml network_info=[{"id": "fa65b32f-b627-45fa-8d6d-d99012ca1262", "address": "fa:16:3e:2f:fa:59", "network": {"id": "6787e0a8-2980-4a2e-bf2a-583af955241d", "bridge": "br-int", "label": "tempest-TestMinimumBasicScenario-1273600998-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "74f132d6666c47b8b43869df6fee6de3", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapfa65b32f-b6", "ovs_interfaceid": "fa65b32f-b627-45fa-8d6d-d99012ca1262", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum='c8fc807773e5354afe61636071771906',container_format='bare',created_at=2023-11-29T03:23:43Z,direct_url=<?>,disk_format='qcow2',id=7107c48b-05b9-4052-917d-814b721d169c,min_disk=0,min_ram=0,name='tempest-scenario-img--1303145367',owner='74f132d6666c47b8b43869df6fee6de3',properties=ImageMetaProps,protected=<?>,size=21430272,status='active',tags=<?>,updated_at=2023-11-29T03:23:45Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [{'device_name': '/dev/vda', 'encryption_format': None, 'encrypted': False, 'device_type': 'disk', 'size': 0, 'disk_bus': 'virtio', 'boot_index': 0, 'encryption_options': None, 'encryption_secret_uuid': None, 'guest_format': None, 'image_id': '7107c48b-05b9-4052-917d-814b721d169c'}], 'ephemerals': [], 'block_device_mapping': [], 'swap': None} {{(pid=75234) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7562}}
Nov 29 03:23:49 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-201f89b9-1324-48f4-9cdb-c94c9b58ffbd tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:23:49 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-201f89b9-1324-48f4-9cdb-c94c9b58ffbd tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:23:49 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-201f89b9-1324-48f4-9cdb-c94c9b58ffbd tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Searching host: 'user' for CPU controller through CGroups V1... {{(pid=75234) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1652}}
Nov 29 03:23:49 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-201f89b9-1324-48f4-9cdb-c94c9b58ffbd tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] CPU controller missing on host. {{(pid=75234) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1662}}
Nov 29 03:23:49 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-201f89b9-1324-48f4-9cdb-c94c9b58ffbd tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Searching host: 'user' for CPU controller through CGroups V2... {{(pid=75234) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1671}}
Nov 29 03:23:49 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-201f89b9-1324-48f4-9cdb-c94c9b58ffbd tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] CPU controller found on host. {{(pid=75234) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1678}}
Nov 29 03:23:49 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-201f89b9-1324-48f4-9cdb-c94c9b58ffbd tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=75234) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5400}}
Nov 29 03:23:49 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-201f89b9-1324-48f4-9cdb-c94c9b58ffbd tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-11-29T03:02:04Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='1',id=6,is_public=True,memory_mb=512,name='m1.tiny',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='c8fc807773e5354afe61636071771906',container_format='bare',created_at=2023-11-29T03:23:43Z,direct_url=<?>,disk_format='qcow2',id=7107c48b-05b9-4052-917d-814b721d169c,min_disk=0,min_ram=0,name='tempest-scenario-img--1303145367',owner='74f132d6666c47b8b43869df6fee6de3',properties=ImageMetaProps,protected=<?>,size=21430272,status='active',tags=<?>,updated_at=2023-11-29T03:23:45Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:563}}
Nov 29 03:23:49 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-201f89b9-1324-48f4-9cdb-c94c9b58ffbd tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Flavor limits 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:348}}
Nov 29 03:23:49 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-201f89b9-1324-48f4-9cdb-c94c9b58ffbd tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Image limits 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:352}}
Nov 29 03:23:49 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-201f89b9-1324-48f4-9cdb-c94c9b58ffbd tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Flavor pref 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:388}}
Nov 29 03:23:49 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-201f89b9-1324-48f4-9cdb-c94c9b58ffbd tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Image pref 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:392}}
Nov 29 03:23:49 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-201f89b9-1324-48f4-9cdb-c94c9b58ffbd tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:430}}
Nov 29 03:23:49 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-201f89b9-1324-48f4-9cdb-c94c9b58ffbd tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:569}}
Nov 29 03:23:49 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-201f89b9-1324-48f4-9cdb-c94c9b58ffbd tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=75234) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:471}}
Nov 29 03:23:49 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-201f89b9-1324-48f4-9cdb-c94c9b58ffbd tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Got 1 possible topologies {{(pid=75234) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:501}}
Nov 29 03:23:49 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-201f89b9-1324-48f4-9cdb-c94c9b58ffbd tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:575}}
Nov 29 03:23:49 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-201f89b9-1324-48f4-9cdb-c94c9b58ffbd tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:577}}
Nov 29 03:23:49 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-201f89b9-1324-48f4-9cdb-c94c9b58ffbd tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-11-29T03:23:45Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-TestMinimumBasicScenario-server-310577704',display_name='tempest-TestMinimumBasicScenario-server-310577704',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-testminimumbasicscenario-server-310577704',id=39,image_ref='7107c48b-05b9-4052-917d-814b721d169c',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBBTfFCQxMn5FN9HRzHRLmeLvGNhlvH8jkTHWDyEbYwE8hTZhRqGybcRgcmJ5xQB1zSttCcEaPSgc17S9QrgRI43nIUg3IFEEzmkOkz4/kzJTs5GShIWM4PGtgtnmC06Ugw==',key_name='tempest-TestMinimumBasicScenario-44553634',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='74f132d6666c47b8b43869df6fee6de3',ramdisk_id='',reservation_id='r-dj1tj6j2',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='7107c48b-05b9-4052-917d-814b721d169c',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_min_disk='1',image_min_ram='0',network_allocated='True',owner_project_name='tempest-TestMinimumBasicScenario-1635847764',owner_user_name='tempest-TestMinimumBasicScenario-1635847764-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-11-29T03:23:47Z,user_data=None,user_id='3a02618f56ea45898ab4783a4f198b5d',uuid=5d7d744e-1a5f-40ab-b961-4b58c183efe0,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "fa65b32f-b627-45fa-8d6d-d99012ca1262", "address": "fa:16:3e:2f:fa:59", "network": {"id": "6787e0a8-2980-4a2e-bf2a-583af955241d", "bridge": "br-int", "label": "tempest-TestMinimumBasicScenario-1273600998-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "74f132d6666c47b8b43869df6fee6de3", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapfa65b32f-b6", "ovs_interfaceid": "fa65b32f-b627-45fa-8d6d-d99012ca1262", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=75234) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Nov 29 03:23:49 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-201f89b9-1324-48f4-9cdb-c94c9b58ffbd tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Converting VIF {"id": "fa65b32f-b627-45fa-8d6d-d99012ca1262", "address": "fa:16:3e:2f:fa:59", "network": {"id": "6787e0a8-2980-4a2e-bf2a-583af955241d", "bridge": "br-int", "label": "tempest-TestMinimumBasicScenario-1273600998-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "74f132d6666c47b8b43869df6fee6de3", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapfa65b32f-b6", "ovs_interfaceid": "fa65b32f-b627-45fa-8d6d-d99012ca1262", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:23:49 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-201f89b9-1324-48f4-9cdb-c94c9b58ffbd tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:2f:fa:59,bridge_name='br-int',has_traffic_filtering=True,id=fa65b32f-b627-45fa-8d6d-d99012ca1262,network=Network(6787e0a8-2980-4a2e-bf2a-583af955241d),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapfa65b32f-b6') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:23:49 user nova-compute[75234]: DEBUG nova.objects.instance [None req-201f89b9-1324-48f4-9cdb-c94c9b58ffbd tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Lazy-loading 'pci_devices' on Instance uuid 5d7d744e-1a5f-40ab-b961-4b58c183efe0 {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:23:49 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-201f89b9-1324-48f4-9cdb-c94c9b58ffbd tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] [instance: 5d7d744e-1a5f-40ab-b961-4b58c183efe0] End _get_guest_xml xml=<domain type="kvm">
Nov 29 03:23:49 user nova-compute[75234]:   <uuid>5d7d744e-1a5f-40ab-b961-4b58c183efe0</uuid>
Nov 29 03:23:49 user nova-compute[75234]:   <name>instance-00000027</name>
Nov 29 03:23:49 user nova-compute[75234]:   <memory>524288</memory>
Nov 29 03:23:49 user nova-compute[75234]:   <vcpu>1</vcpu>
Nov 29 03:23:49 user nova-compute[75234]:   <metadata>
Nov 29 03:23:49 user nova-compute[75234]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Nov 29 03:23:49 user nova-compute[75234]:       <nova:package version="0.0.0"/>
Nov 29 03:23:49 user nova-compute[75234]:       <nova:name>tempest-TestMinimumBasicScenario-server-310577704</nova:name>
Nov 29 03:23:49 user nova-compute[75234]:       <nova:creationTime>2023-11-29 03:23:49</nova:creationTime>
Nov 29 03:23:49 user nova-compute[75234]:       <nova:flavor name="m1.tiny">
Nov 29 03:23:49 user nova-compute[75234]:         <nova:memory>512</nova:memory>
Nov 29 03:23:49 user nova-compute[75234]:         <nova:disk>1</nova:disk>
Nov 29 03:23:49 user nova-compute[75234]:         <nova:swap>0</nova:swap>
Nov 29 03:23:49 user nova-compute[75234]:         <nova:ephemeral>0</nova:ephemeral>
Nov 29 03:23:49 user nova-compute[75234]:         <nova:vcpus>1</nova:vcpus>
Nov 29 03:23:49 user nova-compute[75234]:       </nova:flavor>
Nov 29 03:23:49 user nova-compute[75234]:       <nova:owner>
Nov 29 03:23:49 user nova-compute[75234]:         <nova:user uuid="3a02618f56ea45898ab4783a4f198b5d">tempest-TestMinimumBasicScenario-1635847764-project-member</nova:user>
Nov 29 03:23:49 user nova-compute[75234]:         <nova:project uuid="74f132d6666c47b8b43869df6fee6de3">tempest-TestMinimumBasicScenario-1635847764</nova:project>
Nov 29 03:23:49 user nova-compute[75234]:       </nova:owner>
Nov 29 03:23:49 user nova-compute[75234]:       <nova:root type="image" uuid="7107c48b-05b9-4052-917d-814b721d169c"/>
Nov 29 03:23:49 user nova-compute[75234]:       <nova:ports>
Nov 29 03:23:49 user nova-compute[75234]:         <nova:port uuid="fa65b32f-b627-45fa-8d6d-d99012ca1262">
Nov 29 03:23:49 user nova-compute[75234]:           <nova:ip type="fixed" address="10.0.0.3" ipVersion="4"/>
Nov 29 03:23:49 user nova-compute[75234]:         </nova:port>
Nov 29 03:23:49 user nova-compute[75234]:       </nova:ports>
Nov 29 03:23:49 user nova-compute[75234]:     </nova:instance>
Nov 29 03:23:49 user nova-compute[75234]:   </metadata>
Nov 29 03:23:49 user nova-compute[75234]:   <sysinfo type="smbios">
Nov 29 03:23:49 user nova-compute[75234]:     <system>
Nov 29 03:23:49 user nova-compute[75234]:       <entry name="manufacturer">OpenStack Foundation</entry>
Nov 29 03:23:49 user nova-compute[75234]:       <entry name="product">OpenStack Nova</entry>
Nov 29 03:23:49 user nova-compute[75234]:       <entry name="version">0.0.0</entry>
Nov 29 03:23:49 user nova-compute[75234]:       <entry name="serial">5d7d744e-1a5f-40ab-b961-4b58c183efe0</entry>
Nov 29 03:23:49 user nova-compute[75234]:       <entry name="uuid">5d7d744e-1a5f-40ab-b961-4b58c183efe0</entry>
Nov 29 03:23:49 user nova-compute[75234]:       <entry name="family">Virtual Machine</entry>
Nov 29 03:23:49 user nova-compute[75234]:     </system>
Nov 29 03:23:49 user nova-compute[75234]:   </sysinfo>
Nov 29 03:23:49 user nova-compute[75234]:   <os>
Nov 29 03:23:49 user nova-compute[75234]:     <type arch="x86_64" machine="pc">hvm</type>
Nov 29 03:23:49 user nova-compute[75234]:     <boot dev="hd"/>
Nov 29 03:23:49 user nova-compute[75234]:     <smbios mode="sysinfo"/>
Nov 29 03:23:49 user nova-compute[75234]:   </os>
Nov 29 03:23:49 user nova-compute[75234]:   <features>
Nov 29 03:23:49 user nova-compute[75234]:     <acpi/>
Nov 29 03:23:49 user nova-compute[75234]:     <apic/>
Nov 29 03:23:49 user nova-compute[75234]:     <vmcoreinfo/>
Nov 29 03:23:49 user nova-compute[75234]:   </features>
Nov 29 03:23:49 user nova-compute[75234]:   <clock offset="utc">
Nov 29 03:23:49 user nova-compute[75234]:     <timer name="pit" tickpolicy="delay"/>
Nov 29 03:23:49 user nova-compute[75234]:     <timer name="rtc" tickpolicy="catchup"/>
Nov 29 03:23:49 user nova-compute[75234]:     <timer name="hpet" present="no"/>
Nov 29 03:23:49 user nova-compute[75234]:   </clock>
Nov 29 03:23:49 user nova-compute[75234]:   <cpu mode="custom" match="exact">
Nov 29 03:23:49 user nova-compute[75234]:     <model>Nehalem</model>
Nov 29 03:23:49 user nova-compute[75234]:     <topology sockets="1" cores="1" threads="1"/>
Nov 29 03:23:49 user nova-compute[75234]:   </cpu>
Nov 29 03:23:49 user nova-compute[75234]:   <devices>
Nov 29 03:23:49 user nova-compute[75234]:     <disk type="file" device="disk">
Nov 29 03:23:49 user nova-compute[75234]:       <driver name="qemu" type="qcow2" cache="none"/>
Nov 29 03:23:49 user nova-compute[75234]:       <source file="/opt/stack/data/nova/instances/5d7d744e-1a5f-40ab-b961-4b58c183efe0/disk"/>
Nov 29 03:23:49 user nova-compute[75234]:       <target dev="vda" bus="virtio"/>
Nov 29 03:23:49 user nova-compute[75234]:     </disk>
Nov 29 03:23:49 user nova-compute[75234]:     <interface type="ethernet">
Nov 29 03:23:49 user nova-compute[75234]:       <mac address="fa:16:3e:2f:fa:59"/>
Nov 29 03:23:49 user nova-compute[75234]:       <model type="virtio"/>
Nov 29 03:23:49 user nova-compute[75234]:       <mtu size="1442"/>
Nov 29 03:23:49 user nova-compute[75234]:       <target dev="tapfa65b32f-b6"/>
Nov 29 03:23:49 user nova-compute[75234]:     </interface>
Nov 29 03:23:49 user nova-compute[75234]:     <serial type="pty">
Nov 29 03:23:49 user nova-compute[75234]:       <log file="/opt/stack/data/nova/instances/5d7d744e-1a5f-40ab-b961-4b58c183efe0/console.log" append="off"/>
Nov 29 03:23:49 user nova-compute[75234]:     </serial>
Nov 29 03:23:49 user nova-compute[75234]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Nov 29 03:23:49 user nova-compute[75234]:     <video>
Nov 29 03:23:49 user nova-compute[75234]:       <model type="virtio"/>
Nov 29 03:23:49 user nova-compute[75234]:     </video>
Nov 29 03:23:49 user nova-compute[75234]:     <rng model="virtio">
Nov 29 03:23:49 user nova-compute[75234]:       <backend model="random">/dev/urandom</backend>
Nov 29 03:23:49 user nova-compute[75234]:     </rng>
Nov 29 03:23:49 user nova-compute[75234]:     <controller type="usb" index="0" model="none"/>
Nov 29 03:23:49 user nova-compute[75234]:     <memballoon model="virtio">
Nov 29 03:23:49 user nova-compute[75234]:       <stats period="10"/>
Nov 29 03:23:49 user nova-compute[75234]:     </memballoon>
Nov 29 03:23:49 user nova-compute[75234]:   </devices>
Nov 29 03:23:49 user nova-compute[75234]: </domain>
Nov 29 03:23:49 user nova-compute[75234]:  {{(pid=75234) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7568}}
Nov 29 03:23:49 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-201f89b9-1324-48f4-9cdb-c94c9b58ffbd tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-11-29T03:23:45Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-TestMinimumBasicScenario-server-310577704',display_name='tempest-TestMinimumBasicScenario-server-310577704',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-testminimumbasicscenario-server-310577704',id=39,image_ref='7107c48b-05b9-4052-917d-814b721d169c',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBBTfFCQxMn5FN9HRzHRLmeLvGNhlvH8jkTHWDyEbYwE8hTZhRqGybcRgcmJ5xQB1zSttCcEaPSgc17S9QrgRI43nIUg3IFEEzmkOkz4/kzJTs5GShIWM4PGtgtnmC06Ugw==',key_name='tempest-TestMinimumBasicScenario-44553634',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='74f132d6666c47b8b43869df6fee6de3',ramdisk_id='',reservation_id='r-dj1tj6j2',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='7107c48b-05b9-4052-917d-814b721d169c',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_min_disk='1',image_min_ram='0',network_allocated='True',owner_project_name='tempest-TestMinimumBasicScenario-1635847764',owner_user_name='tempest-TestMinimumBasicScenario-1635847764-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-11-29T03:23:47Z,user_data=None,user_id='3a02618f56ea45898ab4783a4f198b5d',uuid=5d7d744e-1a5f-40ab-b961-4b58c183efe0,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "fa65b32f-b627-45fa-8d6d-d99012ca1262", "address": "fa:16:3e:2f:fa:59", "network": {"id": "6787e0a8-2980-4a2e-bf2a-583af955241d", "bridge": "br-int", "label": "tempest-TestMinimumBasicScenario-1273600998-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "74f132d6666c47b8b43869df6fee6de3", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapfa65b32f-b6", "ovs_interfaceid": "fa65b32f-b627-45fa-8d6d-d99012ca1262", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Nov 29 03:23:49 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-201f89b9-1324-48f4-9cdb-c94c9b58ffbd tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Converting VIF {"id": "fa65b32f-b627-45fa-8d6d-d99012ca1262", "address": "fa:16:3e:2f:fa:59", "network": {"id": "6787e0a8-2980-4a2e-bf2a-583af955241d", "bridge": "br-int", "label": "tempest-TestMinimumBasicScenario-1273600998-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "74f132d6666c47b8b43869df6fee6de3", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapfa65b32f-b6", "ovs_interfaceid": "fa65b32f-b627-45fa-8d6d-d99012ca1262", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:23:49 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-201f89b9-1324-48f4-9cdb-c94c9b58ffbd tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:2f:fa:59,bridge_name='br-int',has_traffic_filtering=True,id=fa65b32f-b627-45fa-8d6d-d99012ca1262,network=Network(6787e0a8-2980-4a2e-bf2a-583af955241d),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapfa65b32f-b6') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:23:49 user nova-compute[75234]: DEBUG os_vif [None req-201f89b9-1324-48f4-9cdb-c94c9b58ffbd tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:2f:fa:59,bridge_name='br-int',has_traffic_filtering=True,id=fa65b32f-b627-45fa-8d6d-d99012ca1262,network=Network(6787e0a8-2980-4a2e-bf2a-583af955241d),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapfa65b32f-b6') {{(pid=75234) plug /opt/stack/data/venv/lib/python3.10/site-packages/os_vif/__init__.py:76}}
Nov 29 03:23:49 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:23:49 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:23:49 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Nov 29 03:23:49 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:23:49 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbCreateCommand(_result=None, table=QoS, columns={'type': 'linux-noop', 'external_ids': {'id': 'bc1d6bff-df18-5391-8f57-ec3d535703ea', '_type': 'linux-noop'}}, row=False) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:23:49 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:23:49 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:23:49 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:23:49 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tapfa65b32f-b6, may_exist=True, interface_attrs={}) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:23:49 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Port, record=tapfa65b32f-b6, col_values=(('qos', UUID('59df5b17-ee40-40de-bbb2-45629fc7b9e3')),), if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:23:49 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DbSetCommand(_result=None, table=Interface, record=tapfa65b32f-b6, col_values=(('external_ids', {'iface-id': 'fa65b32f-b627-45fa-8d6d-d99012ca1262', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:2f:fa:59', 'vm-uuid': '5d7d744e-1a5f-40ab-b961-4b58c183efe0'}),), if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:23:49 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:23:49 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:23:49 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:23:49 user nova-compute[75234]: INFO os_vif [None req-201f89b9-1324-48f4-9cdb-c94c9b58ffbd tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:2f:fa:59,bridge_name='br-int',has_traffic_filtering=True,id=fa65b32f-b627-45fa-8d6d-d99012ca1262,network=Network(6787e0a8-2980-4a2e-bf2a-583af955241d),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapfa65b32f-b6')
Nov 29 03:23:49 user nova-compute[75234]: DEBUG nova.network.neutron [req-c6c1978c-2f08-40e0-89c0-002bcb7f03ee req-d343389e-ddba-4c8a-97ba-e220722d3f33 service nova] [instance: 5d7d744e-1a5f-40ab-b961-4b58c183efe0] Updated VIF entry in instance network info cache for port fa65b32f-b627-45fa-8d6d-d99012ca1262. {{(pid=75234) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3481}}
Nov 29 03:23:49 user nova-compute[75234]: DEBUG nova.network.neutron [req-c6c1978c-2f08-40e0-89c0-002bcb7f03ee req-d343389e-ddba-4c8a-97ba-e220722d3f33 service nova] [instance: 5d7d744e-1a5f-40ab-b961-4b58c183efe0] Updating instance_info_cache with network_info: [{"id": "fa65b32f-b627-45fa-8d6d-d99012ca1262", "address": "fa:16:3e:2f:fa:59", "network": {"id": "6787e0a8-2980-4a2e-bf2a-583af955241d", "bridge": "br-int", "label": "tempest-TestMinimumBasicScenario-1273600998-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "74f132d6666c47b8b43869df6fee6de3", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapfa65b32f-b6", "ovs_interfaceid": "fa65b32f-b627-45fa-8d6d-d99012ca1262", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:23:49 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-c6c1978c-2f08-40e0-89c0-002bcb7f03ee req-d343389e-ddba-4c8a-97ba-e220722d3f33 service nova] Releasing lock "refresh_cache-5d7d744e-1a5f-40ab-b961-4b58c183efe0" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:23:49 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-201f89b9-1324-48f4-9cdb-c94c9b58ffbd tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] No BDM found with device name vda, not building metadata. {{(pid=75234) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12089}}
Nov 29 03:23:49 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-201f89b9-1324-48f4-9cdb-c94c9b58ffbd tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] No VIF found with MAC fa:16:3e:2f:fa:59, not building metadata {{(pid=75234) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Nov 29 03:23:50 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:23:50 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:23:50 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:23:50 user nova-compute[75234]: DEBUG nova.compute.manager [req-50de8c63-de32-447d-8a2f-60e3eefe62e3 req-0beb382c-a0b9-410f-8f8f-a6c4742e3168 service nova] [instance: 5d7d744e-1a5f-40ab-b961-4b58c183efe0] Received event network-vif-plugged-fa65b32f-b627-45fa-8d6d-d99012ca1262 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:23:50 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-50de8c63-de32-447d-8a2f-60e3eefe62e3 req-0beb382c-a0b9-410f-8f8f-a6c4742e3168 service nova] Acquiring lock "5d7d744e-1a5f-40ab-b961-4b58c183efe0-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:23:50 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-50de8c63-de32-447d-8a2f-60e3eefe62e3 req-0beb382c-a0b9-410f-8f8f-a6c4742e3168 service nova] Lock "5d7d744e-1a5f-40ab-b961-4b58c183efe0-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:23:50 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-50de8c63-de32-447d-8a2f-60e3eefe62e3 req-0beb382c-a0b9-410f-8f8f-a6c4742e3168 service nova] Lock "5d7d744e-1a5f-40ab-b961-4b58c183efe0-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:23:50 user nova-compute[75234]: DEBUG nova.compute.manager [req-50de8c63-de32-447d-8a2f-60e3eefe62e3 req-0beb382c-a0b9-410f-8f8f-a6c4742e3168 service nova] [instance: 5d7d744e-1a5f-40ab-b961-4b58c183efe0] No waiting events found dispatching network-vif-plugged-fa65b32f-b627-45fa-8d6d-d99012ca1262 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:23:50 user nova-compute[75234]: WARNING nova.compute.manager [req-50de8c63-de32-447d-8a2f-60e3eefe62e3 req-0beb382c-a0b9-410f-8f8f-a6c4742e3168 service nova] [instance: 5d7d744e-1a5f-40ab-b961-4b58c183efe0] Received unexpected event network-vif-plugged-fa65b32f-b627-45fa-8d6d-d99012ca1262 for instance with vm_state building and task_state spawning.
Nov 29 03:23:51 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:23:51 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:23:51 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:23:51 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:23:52 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:23:52 user nova-compute[75234]: DEBUG nova.compute.manager [None req-201f89b9-1324-48f4-9cdb-c94c9b58ffbd tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] [instance: 5d7d744e-1a5f-40ab-b961-4b58c183efe0] Instance event wait completed in 0 seconds for  {{(pid=75234) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Nov 29 03:23:52 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-201f89b9-1324-48f4-9cdb-c94c9b58ffbd tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] [instance: 5d7d744e-1a5f-40ab-b961-4b58c183efe0] Guest created on hypervisor {{(pid=75234) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4421}}
Nov 29 03:23:52 user nova-compute[75234]: INFO nova.virt.libvirt.driver [-] [instance: 5d7d744e-1a5f-40ab-b961-4b58c183efe0] Instance spawned successfully.
Nov 29 03:23:52 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-201f89b9-1324-48f4-9cdb-c94c9b58ffbd tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] [instance: 5d7d744e-1a5f-40ab-b961-4b58c183efe0] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:904}}
Nov 29 03:23:52 user nova-compute[75234]: DEBUG nova.compute.manager [req-8940178f-cc85-4b5c-841e-3fdcc45722fc req-4001c23a-b46f-495e-aa86-2bc45f19f191 service nova] [instance: 5d7d744e-1a5f-40ab-b961-4b58c183efe0] Received event network-vif-plugged-fa65b32f-b627-45fa-8d6d-d99012ca1262 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:23:52 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-8940178f-cc85-4b5c-841e-3fdcc45722fc req-4001c23a-b46f-495e-aa86-2bc45f19f191 service nova] Acquiring lock "5d7d744e-1a5f-40ab-b961-4b58c183efe0-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:23:52 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-8940178f-cc85-4b5c-841e-3fdcc45722fc req-4001c23a-b46f-495e-aa86-2bc45f19f191 service nova] Lock "5d7d744e-1a5f-40ab-b961-4b58c183efe0-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:23:52 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-8940178f-cc85-4b5c-841e-3fdcc45722fc req-4001c23a-b46f-495e-aa86-2bc45f19f191 service nova] Lock "5d7d744e-1a5f-40ab-b961-4b58c183efe0-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:23:52 user nova-compute[75234]: DEBUG nova.compute.manager [req-8940178f-cc85-4b5c-841e-3fdcc45722fc req-4001c23a-b46f-495e-aa86-2bc45f19f191 service nova] [instance: 5d7d744e-1a5f-40ab-b961-4b58c183efe0] No waiting events found dispatching network-vif-plugged-fa65b32f-b627-45fa-8d6d-d99012ca1262 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:23:52 user nova-compute[75234]: WARNING nova.compute.manager [req-8940178f-cc85-4b5c-841e-3fdcc45722fc req-4001c23a-b46f-495e-aa86-2bc45f19f191 service nova] [instance: 5d7d744e-1a5f-40ab-b961-4b58c183efe0] Received unexpected event network-vif-plugged-fa65b32f-b627-45fa-8d6d-d99012ca1262 for instance with vm_state building and task_state spawning.
Nov 29 03:23:52 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-201f89b9-1324-48f4-9cdb-c94c9b58ffbd tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] [instance: 5d7d744e-1a5f-40ab-b961-4b58c183efe0] Found default for hw_cdrom_bus of ide {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:23:52 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-201f89b9-1324-48f4-9cdb-c94c9b58ffbd tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] [instance: 5d7d744e-1a5f-40ab-b961-4b58c183efe0] Found default for hw_disk_bus of virtio {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:23:52 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-201f89b9-1324-48f4-9cdb-c94c9b58ffbd tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] [instance: 5d7d744e-1a5f-40ab-b961-4b58c183efe0] Found default for hw_input_bus of None {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:23:52 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-201f89b9-1324-48f4-9cdb-c94c9b58ffbd tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] [instance: 5d7d744e-1a5f-40ab-b961-4b58c183efe0] Found default for hw_pointer_model of None {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:23:52 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-201f89b9-1324-48f4-9cdb-c94c9b58ffbd tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] [instance: 5d7d744e-1a5f-40ab-b961-4b58c183efe0] Found default for hw_video_model of virtio {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:23:52 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-201f89b9-1324-48f4-9cdb-c94c9b58ffbd tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] [instance: 5d7d744e-1a5f-40ab-b961-4b58c183efe0] Found default for hw_vif_model of virtio {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:23:53 user nova-compute[75234]: INFO nova.compute.manager [None req-201f89b9-1324-48f4-9cdb-c94c9b58ffbd tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] [instance: 5d7d744e-1a5f-40ab-b961-4b58c183efe0] Took 6.22 seconds to spawn the instance on the hypervisor.
Nov 29 03:23:53 user nova-compute[75234]: DEBUG nova.compute.manager [None req-201f89b9-1324-48f4-9cdb-c94c9b58ffbd tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] [instance: 5d7d744e-1a5f-40ab-b961-4b58c183efe0] Checking state {{(pid=75234) _get_power_state /opt/stack/nova/nova/compute/manager.py:1782}}
Nov 29 03:23:53 user nova-compute[75234]: INFO nova.compute.manager [None req-201f89b9-1324-48f4-9cdb-c94c9b58ffbd tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] [instance: 5d7d744e-1a5f-40ab-b961-4b58c183efe0] Took 6.92 seconds to build instance.
Nov 29 03:23:53 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-201f89b9-1324-48f4-9cdb-c94c9b58ffbd tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Lock "5d7d744e-1a5f-40ab-b961-4b58c183efe0" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 7.253s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:23:54 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:23:54 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-757a640d-e8c9-410f-a5ea-edacecebd824 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Acquiring lock "d5096ca2-d005-45ac-a487-11a810446859" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:23:54 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-757a640d-e8c9-410f-a5ea-edacecebd824 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Lock "d5096ca2-d005-45ac-a487-11a810446859" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:23:55 user nova-compute[75234]: DEBUG nova.compute.manager [None req-757a640d-e8c9-410f-a5ea-edacecebd824 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] [instance: d5096ca2-d005-45ac-a487-11a810446859] Starting instance... {{(pid=75234) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2422}}
Nov 29 03:23:55 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-757a640d-e8c9-410f-a5ea-edacecebd824 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:23:55 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-757a640d-e8c9-410f-a5ea-edacecebd824 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:23:55 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-757a640d-e8c9-410f-a5ea-edacecebd824 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=75234) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2366}}
Nov 29 03:23:55 user nova-compute[75234]: INFO nova.compute.claims [None req-757a640d-e8c9-410f-a5ea-edacecebd824 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] [instance: d5096ca2-d005-45ac-a487-11a810446859] Claim successful on node user
Nov 29 03:23:55 user nova-compute[75234]: DEBUG nova.compute.provider_tree [None req-757a640d-e8c9-410f-a5ea-edacecebd824 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Inventory has not changed in ProviderTree for provider: 4d291220-7c37-4839-a0e3-b23c66502b74 {{(pid=75234) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Nov 29 03:23:55 user nova-compute[75234]: DEBUG nova.scheduler.client.report [None req-757a640d-e8c9-410f-a5ea-edacecebd824 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Inventory has not changed for provider 4d291220-7c37-4839-a0e3-b23c66502b74 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=75234) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Nov 29 03:23:55 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-757a640d-e8c9-410f-a5ea-edacecebd824 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.402s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:23:55 user nova-compute[75234]: DEBUG nova.compute.manager [None req-757a640d-e8c9-410f-a5ea-edacecebd824 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] [instance: d5096ca2-d005-45ac-a487-11a810446859] Start building networks asynchronously for instance. {{(pid=75234) _build_resources /opt/stack/nova/nova/compute/manager.py:2819}}
Nov 29 03:23:55 user nova-compute[75234]: DEBUG nova.compute.manager [None req-757a640d-e8c9-410f-a5ea-edacecebd824 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] [instance: d5096ca2-d005-45ac-a487-11a810446859] Allocating IP information in the background. {{(pid=75234) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1972}}
Nov 29 03:23:55 user nova-compute[75234]: DEBUG nova.network.neutron [None req-757a640d-e8c9-410f-a5ea-edacecebd824 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] [instance: d5096ca2-d005-45ac-a487-11a810446859] allocate_for_instance() {{(pid=75234) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1156}}
Nov 29 03:23:55 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-757a640d-e8c9-410f-a5ea-edacecebd824 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] [instance: d5096ca2-d005-45ac-a487-11a810446859] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Nov 29 03:23:55 user nova-compute[75234]: DEBUG nova.compute.manager [None req-757a640d-e8c9-410f-a5ea-edacecebd824 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] [instance: d5096ca2-d005-45ac-a487-11a810446859] Start building block device mappings for instance. {{(pid=75234) _build_resources /opt/stack/nova/nova/compute/manager.py:2854}}
Nov 29 03:23:55 user nova-compute[75234]: INFO nova.virt.block_device [None req-757a640d-e8c9-410f-a5ea-edacecebd824 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] [instance: d5096ca2-d005-45ac-a487-11a810446859] Booting with blank volume at /dev/vda
Nov 29 03:23:55 user nova-compute[75234]: DEBUG nova.policy [None req-757a640d-e8c9-410f-a5ea-edacecebd824 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '877aa877f13a4f84ac4725278010aea3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'db00bcd0c82c44f1bda739e9857f6e84', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=75234) authorize /opt/stack/nova/nova/policy.py:203}}
Nov 29 03:23:56 user nova-compute[75234]: DEBUG nova.network.neutron [None req-757a640d-e8c9-410f-a5ea-edacecebd824 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] [instance: d5096ca2-d005-45ac-a487-11a810446859] Successfully created port: 9266d021-6e83-4bfa-8151-66c5182e61a0 {{(pid=75234) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:548}}
Nov 29 03:23:57 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:23:57 user nova-compute[75234]: DEBUG nova.network.neutron [None req-757a640d-e8c9-410f-a5ea-edacecebd824 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] [instance: d5096ca2-d005-45ac-a487-11a810446859] Successfully updated port: 9266d021-6e83-4bfa-8151-66c5182e61a0 {{(pid=75234) _update_port /opt/stack/nova/nova/network/neutron.py:586}}
Nov 29 03:23:57 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-757a640d-e8c9-410f-a5ea-edacecebd824 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Acquiring lock "refresh_cache-d5096ca2-d005-45ac-a487-11a810446859" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:23:57 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-757a640d-e8c9-410f-a5ea-edacecebd824 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Acquired lock "refresh_cache-d5096ca2-d005-45ac-a487-11a810446859" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:23:57 user nova-compute[75234]: DEBUG nova.network.neutron [None req-757a640d-e8c9-410f-a5ea-edacecebd824 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] [instance: d5096ca2-d005-45ac-a487-11a810446859] Building network info cache for instance {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2009}}
Nov 29 03:23:57 user nova-compute[75234]: DEBUG nova.compute.manager [req-c68d8af5-b932-4d15-bbde-a9d58fef9463 req-fb9a9ed0-bd4c-4f23-b835-42c8bfbb8b93 service nova] [instance: d5096ca2-d005-45ac-a487-11a810446859] Received event network-changed-9266d021-6e83-4bfa-8151-66c5182e61a0 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:23:57 user nova-compute[75234]: DEBUG nova.compute.manager [req-c68d8af5-b932-4d15-bbde-a9d58fef9463 req-fb9a9ed0-bd4c-4f23-b835-42c8bfbb8b93 service nova] [instance: d5096ca2-d005-45ac-a487-11a810446859] Refreshing instance network info cache due to event network-changed-9266d021-6e83-4bfa-8151-66c5182e61a0. {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11117}}
Nov 29 03:23:57 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-c68d8af5-b932-4d15-bbde-a9d58fef9463 req-fb9a9ed0-bd4c-4f23-b835-42c8bfbb8b93 service nova] Acquiring lock "refresh_cache-d5096ca2-d005-45ac-a487-11a810446859" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:23:57 user nova-compute[75234]: DEBUG nova.network.neutron [None req-757a640d-e8c9-410f-a5ea-edacecebd824 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] [instance: d5096ca2-d005-45ac-a487-11a810446859] Instance cache missing network info. {{(pid=75234) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3322}}
Nov 29 03:23:57 user nova-compute[75234]: DEBUG nova.network.neutron [None req-757a640d-e8c9-410f-a5ea-edacecebd824 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] [instance: d5096ca2-d005-45ac-a487-11a810446859] Updating instance_info_cache with network_info: [{"id": "9266d021-6e83-4bfa-8151-66c5182e61a0", "address": "fa:16:3e:ec:fb:67", "network": {"id": "4d086e08-7798-4cfe-9365-86be6f0a52d2", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-305624008-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "db00bcd0c82c44f1bda739e9857f6e84", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap9266d021-6e", "ovs_interfaceid": "9266d021-6e83-4bfa-8151-66c5182e61a0", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:23:57 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-757a640d-e8c9-410f-a5ea-edacecebd824 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Releasing lock "refresh_cache-d5096ca2-d005-45ac-a487-11a810446859" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:23:57 user nova-compute[75234]: DEBUG nova.compute.manager [None req-757a640d-e8c9-410f-a5ea-edacecebd824 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] [instance: d5096ca2-d005-45ac-a487-11a810446859] Instance network_info: |[{"id": "9266d021-6e83-4bfa-8151-66c5182e61a0", "address": "fa:16:3e:ec:fb:67", "network": {"id": "4d086e08-7798-4cfe-9365-86be6f0a52d2", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-305624008-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "db00bcd0c82c44f1bda739e9857f6e84", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap9266d021-6e", "ovs_interfaceid": "9266d021-6e83-4bfa-8151-66c5182e61a0", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=75234) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1987}}
Nov 29 03:23:57 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-c68d8af5-b932-4d15-bbde-a9d58fef9463 req-fb9a9ed0-bd4c-4f23-b835-42c8bfbb8b93 service nova] Acquired lock "refresh_cache-d5096ca2-d005-45ac-a487-11a810446859" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:23:57 user nova-compute[75234]: DEBUG nova.network.neutron [req-c68d8af5-b932-4d15-bbde-a9d58fef9463 req-fb9a9ed0-bd4c-4f23-b835-42c8bfbb8b93 service nova] [instance: d5096ca2-d005-45ac-a487-11a810446859] Refreshing network info cache for port 9266d021-6e83-4bfa-8151-66c5182e61a0 {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2006}}
Nov 29 03:23:58 user nova-compute[75234]: DEBUG nova.network.neutron [req-c68d8af5-b932-4d15-bbde-a9d58fef9463 req-fb9a9ed0-bd4c-4f23-b835-42c8bfbb8b93 service nova] [instance: d5096ca2-d005-45ac-a487-11a810446859] Updated VIF entry in instance network info cache for port 9266d021-6e83-4bfa-8151-66c5182e61a0. {{(pid=75234) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3481}}
Nov 29 03:23:58 user nova-compute[75234]: DEBUG nova.network.neutron [req-c68d8af5-b932-4d15-bbde-a9d58fef9463 req-fb9a9ed0-bd4c-4f23-b835-42c8bfbb8b93 service nova] [instance: d5096ca2-d005-45ac-a487-11a810446859] Updating instance_info_cache with network_info: [{"id": "9266d021-6e83-4bfa-8151-66c5182e61a0", "address": "fa:16:3e:ec:fb:67", "network": {"id": "4d086e08-7798-4cfe-9365-86be6f0a52d2", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-305624008-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "db00bcd0c82c44f1bda739e9857f6e84", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap9266d021-6e", "ovs_interfaceid": "9266d021-6e83-4bfa-8151-66c5182e61a0", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:23:58 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-c68d8af5-b932-4d15-bbde-a9d58fef9463 req-fb9a9ed0-bd4c-4f23-b835-42c8bfbb8b93 service nova] Releasing lock "refresh_cache-d5096ca2-d005-45ac-a487-11a810446859" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:23:59 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:24:01 user nova-compute[75234]: DEBUG os_brick.utils [None req-757a640d-e8c9-410f-a5ea-edacecebd824 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:176}}
Nov 29 03:24:01 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:24:01 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:24:01 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.016s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:24:01 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[2952351b-bf7a-4056-8716-55288259cf88]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:dd1368c5ba2c\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:24:01 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt -v / -n -o SOURCE {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:24:01 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt -v / -n -o SOURCE" returned: 0 in 0.015s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:24:01 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[b7b87f1c-09e6-4615-85f9-e2b221b34bb5]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:24:01 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:24:01 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.015s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:24:01 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[725d8103-9bba-43b8-a9ed-844db62ea5f9]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:24:01 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[21b3781c-10a0-4d7b-923d-9103e08614c5]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:24:01 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-757a640d-e8c9-410f-a5ea-edacecebd824 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Running cmd (subprocess): nvme version {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:24:01 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-757a640d-e8c9-410f-a5ea-edacecebd824 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] 'nvme version' failed. Not Retrying. {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:473}}
Nov 29 03:24:01 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.nvmeof [None req-757a640d-e8c9-410f-a5ea-edacecebd824 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] nvme not present on system {{(pid=75234) nvme_present /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/nvmeof.py:756}}
Nov 29 03:24:01 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:24:01 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:473}}
Nov 29 03:24:01 user nova-compute[75234]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Nov 29 03:24:01 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[300fca78-8573-42fe-9484-fa7f300e535f]: (4, '') {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:24:01 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.lightos [None req-757a640d-e8c9-410f-a5ea-edacecebd824 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=75234) find_dsc /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/lightos.py:98}}
Nov 29 03:24:01 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.lightos [None req-757a640d-e8c9-410f-a5ea-edacecebd824 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=75234) get_connector_properties /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/lightos.py:76}}
Nov 29 03:24:01 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.lightos [None req-757a640d-e8c9-410f-a5ea-edacecebd824 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] LIGHTOS: no hostnqn found. {{(pid=75234) get_connector_properties /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/lightos.py:84}}
Nov 29 03:24:01 user nova-compute[75234]: DEBUG os_brick.utils [None req-757a640d-e8c9-410f-a5ea-edacecebd824 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] <== get_connector_properties: return (113ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:dd1368c5ba2c', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:203}}
Nov 29 03:24:01 user nova-compute[75234]: DEBUG nova.virt.block_device [None req-757a640d-e8c9-410f-a5ea-edacecebd824 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] [instance: d5096ca2-d005-45ac-a487-11a810446859] Updating existing volume attachment record: 8cc2f809-f19c-4d43-94b7-56037905c4d8 {{(pid=75234) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Nov 29 03:24:02 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:24:04 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:24:04 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-529a682b-48a4-4421-aab5-f015238d97b8 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Waiting for libvirt event about the detach of device vdb with device alias virtio-disk1 from instance f6bc212d-85cd-479c-92ae-112c9f0101bc is timed out.
Nov 29 03:24:04 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-529a682b-48a4-4421-aab5-f015238d97b8 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Successfully detached device vdb from instance f6bc212d-85cd-479c-92ae-112c9f0101bc from the live domain config.
Nov 29 03:24:04 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-529a682b-48a4-4421-aab5-f015238d97b8 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] [instance: f6bc212d-85cd-479c-92ae-112c9f0101bc] calling os-brick to detach iSCSI Volume {{(pid=75234) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Nov 29 03:24:04 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-529a682b-48a4-4421-aab5-f015238d97b8 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f8a5050c9a0>, {'target_iqn': 'iqn.2010-10.org.openstack:6af4b65b-b038-462a-b797-7c57c8e8d680', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'suRxdGNK', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdc'}, None), 'kwargs': False}}" {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:176}}
Nov 29 03:24:04 user nova-compute[75234]: WARNING os_brick.initiator.connectors.base [None req-529a682b-48a4-4421-aab5-f015238d97b8 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Nov 29 03:24:04 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-529a682b-48a4-4421-aab5-f015238d97b8 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:68}}
Nov 29 03:24:04 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-529a682b-48a4-4421-aab5-f015238d97b8 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.002s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:73}}
Nov 29 03:24:04 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:24:04 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.018s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:24:04 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[6ad64613-c1a1-4913-b94c-cf261a5f30af]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:cf6ab229-533a-424e-bd67-1d54d50ad4be\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:6af4b65b-b038-462a-b797-7c57c8e8d680\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:41f08854-31dd-4172-a5cf-01b0e695c10f\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:24:04 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-529a682b-48a4-4421-aab5-f015238d97b8 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS:
Nov 29 03:24:04 user nova-compute[75234]: No targets found.
Nov 29 03:24:04 user nova-compute[75234]: iSNS:
Nov 29 03:24:04 user nova-compute[75234]: No targets found.
Nov 29 03:24:04 user nova-compute[75234]: STATIC:
Nov 29 03:24:04 user nova-compute[75234]: Target: iqn.2010-10.org.openstack:cf6ab229-533a-424e-bd67-1d54d50ad4be
Nov 29 03:24:04 user nova-compute[75234]:         Portal: 172.16.0.220:3260,-1
Nov 29 03:24:04 user nova-compute[75234]:                 Iface Name: default
Nov 29 03:24:04 user nova-compute[75234]: Target: iqn.2010-10.org.openstack:6af4b65b-b038-462a-b797-7c57c8e8d680
Nov 29 03:24:04 user nova-compute[75234]:         Portal: 172.16.0.220:3260,-1
Nov 29 03:24:04 user nova-compute[75234]:                 Iface Name: default
Nov 29 03:24:04 user nova-compute[75234]: Target: iqn.2010-10.org.openstack:41f08854-31dd-4172-a5cf-01b0e695c10f
Nov 29 03:24:04 user nova-compute[75234]:         Portal: 172.16.0.220:3260,-1
Nov 29 03:24:04 user nova-compute[75234]:                 Iface Name: default
Nov 29 03:24:04 user nova-compute[75234]: FIRMWARE:
Nov 29 03:24:04 user nova-compute[75234]: No targets found.
Nov 29 03:24:04 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:24:04 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-529a682b-48a4-4421-aab5-f015238d97b8 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Regex to get portals from discoverydb: ^SENDTARGETS:
Nov 29 03:24:04 user nova-compute[75234]: .*?^DiscoveryAddress: 172.16.0.220,3260.*?
Nov 29 03:24:04 user nova-compute[75234]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=75234) _get_discoverydb_portals /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:382}}
Nov 29 03:24:04 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-529a682b-48a4-4421-aab5-f015238d97b8 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:6af4b65b-b038-462a-b797-7c57c8e8d680', 0)] {{(pid=75234) _get_connection_devices /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:830}}
Nov 29 03:24:04 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:24:04 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.017s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:24:04 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[e2d0ffd9-da37-45da-ac4a-ef63cffd3816]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:41f08854-31dd-4172-a5cf-01b0e695c10f\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:6af4b65b-b038-462a-b797-7c57c8e8d680\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:cf6ab229-533a-424e-bd67-1d54d50ad4be\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:24:04 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:24:04 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.023s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:24:04 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[272ce3d4-ae05-4441-9582-f301db3bbf25]: (4, ('tcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:cf6ab229-533a-424e-bd67-1d54d50ad4be (non-flash)\ntcp: [39] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:41f08854-31dd-4172-a5cf-01b0e695c10f (non-flash)\ntcp: [41] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6af4b65b-b038-462a-b797-7c57c8e8d680 (non-flash)\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:24:04 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-529a682b-48a4-4421-aab5-f015238d97b8 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:cf6ab229-533a-424e-bd67-1d54d50ad4be (non-flash)
Nov 29 03:24:04 user nova-compute[75234]: tcp: [39] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:41f08854-31dd-4172-a5cf-01b0e695c10f (non-flash)
Nov 29 03:24:04 user nova-compute[75234]: tcp: [41] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6af4b65b-b038-462a-b797-7c57c8e8d680 (non-flash)
Nov 29 03:24:04 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:24:04 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-529a682b-48a4-4421-aab5-f015238d97b8 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] iscsi session list stdout=tcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:cf6ab229-533a-424e-bd67-1d54d50ad4be (non-flash)
Nov 29 03:24:04 user nova-compute[75234]: tcp: [39] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:41f08854-31dd-4172-a5cf-01b0e695c10f (non-flash)
Nov 29 03:24:04 user nova-compute[75234]: tcp: [41] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:6af4b65b-b038-462a-b797-7c57c8e8d680 (non-flash)
Nov 29 03:24:04 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsi_session /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Nov 29 03:24:04 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-529a682b-48a4-4421-aab5-f015238d97b8 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7f8a50161ab0>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:6af4b65b-b038-462a-b797-7c57c8e8d680'): ({'sdc'}, set())}) {{(pid=75234) _get_connection_devices /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:863}}
Nov 29 03:24:04 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-529a682b-48a4-4421-aab5-f015238d97b8 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Removing single pathed devices sdc {{(pid=75234) remove_connection /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:374}}
Nov 29 03:24:04 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:24:04 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.018s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:24:04 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[b3a7c5df-0864-4b3b-8718-33b67a9f3e42]: (4, ('path checker states:\nup                  3\n\npaths: 0\nbusy: False\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:24:04 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdc {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:24:04 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:24:04 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:24:04 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=75234) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10541}}
Nov 29 03:24:04 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdc" returned: 0 in 0.019s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:24:04 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[6b601495-3d1a-421d-9a55-df0c61a28dc4]: (4, ('ok\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:24:04 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-529a682b-48a4-4421-aab5-f015238d97b8 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Flushing IO for device /dev/sdc {{(pid=75234) flush_device_io /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:434}}
Nov 29 03:24:04 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdc {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:24:04 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdc" returned: 0 in 0.014s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:24:04 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[52eb5b06-c1ff-4b3c-873b-6b2ff2037b70]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:24:04 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-529a682b-48a4-4421-aab5-f015238d97b8 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Remove SCSI device /dev/sdc with /sys/block/sdc/device/delete {{(pid=75234) remove_scsi_device /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:148}}
Nov 29 03:24:04 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdc/device/delete {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:24:04 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdc/device/delete" returned: 0 in 0.049s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:24:04 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[1fd18385-d2e8-4a0d-a2b4-ee0d9d7064de]: (4, ('1', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:24:04 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-529a682b-48a4-4421-aab5-f015238d97b8 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Checking to see if SCSI volumes sdc have been removed. {{(pid=75234) wait_for_volumes_removal /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:156}}
Nov 29 03:24:04 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-529a682b-48a4-4421-aab5-f015238d97b8 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] SCSI volumes sdc have been removed. {{(pid=75234) wait_for_volumes_removal /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:166}}
Nov 29 03:24:04 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-529a682b-48a4-4421-aab5-f015238d97b8 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:6af4b65b-b038-462a-b797-7c57c8e8d680')] {{(pid=75234) _disconnect_connection /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Nov 29 03:24:04 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:6af4b65b-b038-462a-b797-7c57c8e8d680 -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:24:04 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:6af4b65b-b038-462a-b797-7c57c8e8d680 -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.020s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:24:04 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[073aa5c7-88f8-4c4d-ac2e-73d661074759]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:24:04 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-529a682b-48a4-4421-aab5-f015238d97b8 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:24:04 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:6af4b65b-b038-462a-b797-7c57c8e8d680 -p 172.16.0.220:3260 --logout {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:24:04 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:6af4b65b-b038-462a-b797-7c57c8e8d680 -p 172.16.0.220:3260 --logout" returned: 0 in 0.099s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:24:04 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[d06645e6-0178-45a2-b37e-6fa06a542939]: (4, ('Logging out of session [sid: 41, target: iqn.2010-10.org.openstack:6af4b65b-b038-462a-b797-7c57c8e8d680, portal: 172.16.0.220,3260]\nLogout of [sid: 41, target: iqn.2010-10.org.openstack:6af4b65b-b038-462a-b797-7c57c8e8d680, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:24:04 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-529a682b-48a4-4421-aab5-f015238d97b8 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 41, target: iqn.2010-10.org.openstack:6af4b65b-b038-462a-b797-7c57c8e8d680, portal: 172.16.0.220,3260]
Nov 29 03:24:04 user nova-compute[75234]: Logout of [sid: 41, target: iqn.2010-10.org.openstack:6af4b65b-b038-462a-b797-7c57c8e8d680, portal: 172.16.0.220,3260] successful.
Nov 29 03:24:04 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:24:04 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:6af4b65b-b038-462a-b797-7c57c8e8d680 -p 172.16.0.220:3260 --op delete {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:24:04 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:6af4b65b-b038-462a-b797-7c57c8e8d680 -p 172.16.0.220:3260 --op delete" returned: 0 in 0.023s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:24:04 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[a85c5f21-87bc-47de-bd42-d398c47ec377]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:24:04 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-529a682b-48a4-4421-aab5-f015238d97b8 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:24:04 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-529a682b-48a4-4421-aab5-f015238d97b8 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.353s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:87}}
Nov 29 03:24:04 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-529a682b-48a4-4421-aab5-f015238d97b8 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] <== disconnect_volume: return (355ms) None {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:203}}
Nov 29 03:24:04 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-529a682b-48a4-4421-aab5-f015238d97b8 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] [instance: f6bc212d-85cd-479c-92ae-112c9f0101bc] Disconnected iSCSI Volume {{(pid=75234) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:79}}
Nov 29 03:24:05 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager.update_available_resource {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:24:05 user nova-compute[75234]: DEBUG nova.objects.instance [None req-529a682b-48a4-4421-aab5-f015238d97b8 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Lazy-loading 'flavor' on Instance uuid f6bc212d-85cd-479c-92ae-112c9f0101bc {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:24:05 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:24:05 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:24:05 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:24:05 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Auditing locally available compute resources for user (node: user) {{(pid=75234) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:907}}
Nov 29 03:24:05 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-529a682b-48a4-4421-aab5-f015238d97b8 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Lock "f6bc212d-85cd-479c-92ae-112c9f0101bc" "released" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: held 21.795s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:24:05 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] skipping disk /dev/sdb (vda) as it is a volume {{(pid=75234) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11213}}
Nov 29 03:24:05 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/13e5da64-9adc-4479-81c7-d329e9b5f08f/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:24:05 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/13e5da64-9adc-4479-81c7-d329e9b5f08f/disk --force-share --output=json" returned: 0 in 0.170s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:24:05 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/13e5da64-9adc-4479-81c7-d329e9b5f08f/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:24:06 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/13e5da64-9adc-4479-81c7-d329e9b5f08f/disk --force-share --output=json" returned: 0 in 0.185s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:24:06 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/5d7d744e-1a5f-40ab-b961-4b58c183efe0/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:24:06 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/5d7d744e-1a5f-40ab-b961-4b58c183efe0/disk --force-share --output=json" returned: 0 in 0.172s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:24:06 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/5d7d744e-1a5f-40ab-b961-4b58c183efe0/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:24:06 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/5d7d744e-1a5f-40ab-b961-4b58c183efe0/disk --force-share --output=json" returned: 0 in 0.166s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:24:06 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/1df140f1-fbd9-4b5d-b356-d08b7e908f64/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:24:06 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/1df140f1-fbd9-4b5d-b356-d08b7e908f64/disk --force-share --output=json" returned: 0 in 0.165s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:24:06 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/1df140f1-fbd9-4b5d-b356-d08b7e908f64/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:24:06 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/1df140f1-fbd9-4b5d-b356-d08b7e908f64/disk --force-share --output=json" returned: 0 in 0.175s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:24:06 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] skipping disk /dev/sdd (vdb) as it is a volume {{(pid=75234) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11213}}
Nov 29 03:24:06 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f6bc212d-85cd-479c-92ae-112c9f0101bc/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:24:06 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f6bc212d-85cd-479c-92ae-112c9f0101bc/disk --force-share --output=json" returned: 0 in 0.172s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:24:06 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f6bc212d-85cd-479c-92ae-112c9f0101bc/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:24:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f6bc212d-85cd-479c-92ae-112c9f0101bc/disk --force-share --output=json" returned: 0 in 0.180s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:24:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/65d48c93-b6da-4f51-9d5a-0676dda62a60/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:24:07 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:24:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/65d48c93-b6da-4f51-9d5a-0676dda62a60/disk --force-share --output=json" returned: 0 in 0.173s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:24:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/65d48c93-b6da-4f51-9d5a-0676dda62a60/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:24:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/65d48c93-b6da-4f51-9d5a-0676dda62a60/disk --force-share --output=json" returned: 0 in 0.171s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:24:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/060c15ec-9012-471a-984a-14c9fa6ce23a/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:24:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/060c15ec-9012-471a-984a-14c9fa6ce23a/disk --force-share --output=json" returned: 0 in 0.167s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:24:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/060c15ec-9012-471a-984a-14c9fa6ce23a/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:24:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/060c15ec-9012-471a-984a-14c9fa6ce23a/disk --force-share --output=json" returned: 0 in 0.173s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:24:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a082c96e-e384-40b6-bcc8-fc5e0e104669/disk.rescue --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:24:08 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a082c96e-e384-40b6-bcc8-fc5e0e104669/disk.rescue --force-share --output=json" returned: 0 in 0.171s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:24:08 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a082c96e-e384-40b6-bcc8-fc5e0e104669/disk.rescue --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:24:08 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a082c96e-e384-40b6-bcc8-fc5e0e104669/disk.rescue --force-share --output=json" returned: 0 in 0.174s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:24:08 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a082c96e-e384-40b6-bcc8-fc5e0e104669/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:24:08 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a082c96e-e384-40b6-bcc8-fc5e0e104669/disk --force-share --output=json" returned: 0 in 0.174s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:24:08 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a082c96e-e384-40b6-bcc8-fc5e0e104669/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:24:08 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a082c96e-e384-40b6-bcc8-fc5e0e104669/disk --force-share --output=json" returned: 0 in 0.168s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:24:08 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/21e9c6cd-c39f-41c0-8768-5dd64589c3f3/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:24:08 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/21e9c6cd-c39f-41c0-8768-5dd64589c3f3/disk --force-share --output=json" returned: 0 in 0.169s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:24:08 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/21e9c6cd-c39f-41c0-8768-5dd64589c3f3/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:24:08 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/21e9c6cd-c39f-41c0-8768-5dd64589c3f3/disk --force-share --output=json" returned: 0 in 0.173s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:24:09 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:24:09 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:24:09 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:24:09 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Hypervisor/Node resource view: name=user free_ram=7720MB free_disk=25.118911743164062GB free_vcpus=3 pci_devices=[{"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}] {{(pid=75234) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1106}}
Nov 29 03:24:09 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:24:09 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:24:09 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance 21e9c6cd-c39f-41c0-8768-5dd64589c3f3 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:24:09 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance 13e5da64-9adc-4479-81c7-d329e9b5f08f actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:24:09 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance a082c96e-e384-40b6-bcc8-fc5e0e104669 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:24:09 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance 1df140f1-fbd9-4b5d-b356-d08b7e908f64 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:24:09 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance af699bc2-9daa-4b5a-9b4a-ad92fe857d70 actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 2048, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:24:09 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance 65d48c93-b6da-4f51-9d5a-0676dda62a60 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:24:09 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance f6bc212d-85cd-479c-92ae-112c9f0101bc actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:24:09 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance 060c15ec-9012-471a-984a-14c9fa6ce23a actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:24:09 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance 5d7d744e-1a5f-40ab-b961-4b58c183efe0 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:24:09 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance d5096ca2-d005-45ac-a487-11a810446859 actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:24:09 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Total usable vcpus: 12, total allocated vcpus: 10 {{(pid=75234) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1129}}
Nov 29 03:24:09 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Final resource view: name=user phys_ram=16011MB used_ram=7168MB phys_disk=40GB used_disk=8GB total_vcpus=12 used_vcpus=10 pci_stats=[] {{(pid=75234) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1138}}
Nov 29 03:24:09 user nova-compute[75234]: DEBUG nova.compute.provider_tree [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Inventory has not changed in ProviderTree for provider: 4d291220-7c37-4839-a0e3-b23c66502b74 {{(pid=75234) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Nov 29 03:24:09 user nova-compute[75234]: DEBUG nova.scheduler.client.report [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Inventory has not changed for provider 4d291220-7c37-4839-a0e3-b23c66502b74 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=75234) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Nov 29 03:24:09 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Compute_service record updated for user:user {{(pid=75234) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:1067}}
Nov 29 03:24:09 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.458s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:24:10 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._sync_scheduler_instance_info {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:24:10 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:24:10 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Starting heal instance info cache {{(pid=75234) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9922}}
Nov 29 03:24:10 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Acquiring lock "refresh_cache-af699bc2-9daa-4b5a-9b4a-ad92fe857d70" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:24:10 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Acquired lock "refresh_cache-af699bc2-9daa-4b5a-9b4a-ad92fe857d70" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:24:10 user nova-compute[75234]: DEBUG nova.network.neutron [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] [instance: af699bc2-9daa-4b5a-9b4a-ad92fe857d70] Forcefully refreshing network info cache for instance {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2003}}
Nov 29 03:24:11 user nova-compute[75234]: DEBUG nova.network.neutron [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] [instance: af699bc2-9daa-4b5a-9b4a-ad92fe857d70] Updating instance_info_cache with network_info: [{"id": "66dcb0f9-3180-42f7-851d-19b0c4f4119c", "address": "fa:16:3e:dd:9e:50", "network": {"id": "1f946e69-dbec-429c-83f8-a186bad4f2cc", "bridge": "br-int", "label": "tempest-ServerActionsTestOtherA-486369423-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.64", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "c1cb3d1f12f242a785a688d59dbe3cd6", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap66dcb0f9-31", "ovs_interfaceid": "66dcb0f9-3180-42f7-851d-19b0c4f4119c", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:24:11 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Releasing lock "refresh_cache-af699bc2-9daa-4b5a-9b4a-ad92fe857d70" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:24:11 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] [instance: af699bc2-9daa-4b5a-9b4a-ad92fe857d70] Updated the network info_cache for instance {{(pid=75234) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9993}}
Nov 29 03:24:11 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:24:11 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:24:11 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:24:12 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:24:14 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:24:16 user nova-compute[75234]: DEBUG nova.compute.manager [None req-757a640d-e8c9-410f-a5ea-edacecebd824 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] [instance: d5096ca2-d005-45ac-a487-11a810446859] Start spawning the instance on the hypervisor. {{(pid=75234) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2628}}
Nov 29 03:24:16 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-757a640d-e8c9-410f-a5ea-edacecebd824 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] [instance: d5096ca2-d005-45ac-a487-11a810446859] Creating instance directory {{(pid=75234) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4727}}
Nov 29 03:24:16 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-757a640d-e8c9-410f-a5ea-edacecebd824 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] [instance: d5096ca2-d005-45ac-a487-11a810446859] Creating image(s)
Nov 29 03:24:16 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-757a640d-e8c9-410f-a5ea-edacecebd824 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] [instance: d5096ca2-d005-45ac-a487-11a810446859] Did not create local disks {{(pid=75234) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4863}}
Nov 29 03:24:16 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-757a640d-e8c9-410f-a5ea-edacecebd824 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] [instance: d5096ca2-d005-45ac-a487-11a810446859] Ensure instance console log exists: /opt/stack/data/nova/instances/d5096ca2-d005-45ac-a487-11a810446859/console.log {{(pid=75234) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4613}}
Nov 29 03:24:16 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-757a640d-e8c9-410f-a5ea-edacecebd824 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:24:16 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-757a640d-e8c9-410f-a5ea-edacecebd824 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:24:16 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-757a640d-e8c9-410f-a5ea-edacecebd824 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:24:16 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-757a640d-e8c9-410f-a5ea-edacecebd824 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] [instance: d5096ca2-d005-45ac-a487-11a810446859] Start _get_guest_xml network_info=[{"id": "9266d021-6e83-4bfa-8151-66c5182e61a0", "address": "fa:16:3e:ec:fb:67", "network": {"id": "4d086e08-7798-4cfe-9365-86be6f0a52d2", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-305624008-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "db00bcd0c82c44f1bda739e9857f6e84", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap9266d021-6e", "ovs_interfaceid": "9266d021-6e83-4bfa-8151-66c5182e61a0", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, '/dev/vda': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum='c8fc807773e5354afe61636071771906',container_format='bare',created_at=2023-11-29T03:02:15Z,direct_url=<?>,disk_format='qcow2',id=96c30ba7-9b73-4a30-8ba4-7138879573b7,min_disk=0,min_ram=0,name='cirros-0.6.2-x86_64-disk',owner='b18e380921c748b78517b6f1211cd0f1',properties=ImageMetaProps,protected=<?>,size=21430272,status='active',tags=<?>,updated_at=2023-11-29T03:02:17Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [], 'ephemerals': [], 'block_device_mapping': [{'mount_device': '/dev/vda', 'attachment_id': '8cc2f809-f19c-4d43-94b7-56037905c4d8', 'disk_bus': 'virtio', 'device_type': 'disk', 'boot_index': 0, 'guest_format': None, 'delete_on_termination': False, 'connection_info': {'driver_volume_type': 'iscsi', 'data': {'target_iqn': 'iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'BbOlTYfq', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}, 'status': 'reserved', 'instance': 'd5096ca2-d005-45ac-a487-11a810446859', 'attached_at': '', 'detached_at': '', 'volume_id': 'c12ce787-781c-4e31-adfa-5b51145bc1a0', 'serial': 'c12ce787-781c-4e31-adfa-5b51145bc1a0'}, 'volume_type': None} {{(pid=75234) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7562}}
Nov 29 03:24:16 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-757a640d-e8c9-410f-a5ea-edacecebd824 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:24:16 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-757a640d-e8c9-410f-a5ea-edacecebd824 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:24:16 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-757a640d-e8c9-410f-a5ea-edacecebd824 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Searching host: 'user' for CPU controller through CGroups V1... {{(pid=75234) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1652}}
Nov 29 03:24:16 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-757a640d-e8c9-410f-a5ea-edacecebd824 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] CPU controller missing on host. {{(pid=75234) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1662}}
Nov 29 03:24:16 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-757a640d-e8c9-410f-a5ea-edacecebd824 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Searching host: 'user' for CPU controller through CGroups V2... {{(pid=75234) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1671}}
Nov 29 03:24:16 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-757a640d-e8c9-410f-a5ea-edacecebd824 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] CPU controller found on host. {{(pid=75234) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1678}}
Nov 29 03:24:16 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-757a640d-e8c9-410f-a5ea-edacecebd824 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=75234) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5400}}
Nov 29 03:24:16 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-757a640d-e8c9-410f-a5ea-edacecebd824 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-11-29T03:02:04Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='1',id=6,is_public=True,memory_mb=512,name='m1.tiny',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='c8fc807773e5354afe61636071771906',container_format='bare',created_at=2023-11-29T03:02:15Z,direct_url=<?>,disk_format='qcow2',id=96c30ba7-9b73-4a30-8ba4-7138879573b7,min_disk=0,min_ram=0,name='cirros-0.6.2-x86_64-disk',owner='b18e380921c748b78517b6f1211cd0f1',properties=ImageMetaProps,protected=<?>,size=21430272,status='active',tags=<?>,updated_at=2023-11-29T03:02:17Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:563}}
Nov 29 03:24:16 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-757a640d-e8c9-410f-a5ea-edacecebd824 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Flavor limits 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:348}}
Nov 29 03:24:16 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-757a640d-e8c9-410f-a5ea-edacecebd824 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Image limits 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:352}}
Nov 29 03:24:16 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-757a640d-e8c9-410f-a5ea-edacecebd824 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Flavor pref 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:388}}
Nov 29 03:24:16 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-757a640d-e8c9-410f-a5ea-edacecebd824 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Image pref 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:392}}
Nov 29 03:24:16 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-757a640d-e8c9-410f-a5ea-edacecebd824 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:430}}
Nov 29 03:24:16 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-757a640d-e8c9-410f-a5ea-edacecebd824 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:569}}
Nov 29 03:24:16 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-757a640d-e8c9-410f-a5ea-edacecebd824 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=75234) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:471}}
Nov 29 03:24:16 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-757a640d-e8c9-410f-a5ea-edacecebd824 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Got 1 possible topologies {{(pid=75234) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:501}}
Nov 29 03:24:16 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-757a640d-e8c9-410f-a5ea-edacecebd824 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:575}}
Nov 29 03:24:16 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-757a640d-e8c9-410f-a5ea-edacecebd824 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:577}}
Nov 29 03:24:16 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-757a640d-e8c9-410f-a5ea-edacecebd824 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] [instance: d5096ca2-d005-45ac-a487-11a810446859] Calling os-brick to attach iSCSI Volume {{(pid=75234) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Nov 29 03:24:16 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-757a640d-e8c9-410f-a5ea-edacecebd824 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f8a5050c9a0>, 'connection_properties': {'target_iqn': 'iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'BbOlTYfq', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:176}}
Nov 29 03:24:16 user nova-compute[75234]: WARNING os_brick.initiator.connectors.base [None req-757a640d-e8c9-410f-a5ea-edacecebd824 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Nov 29 03:24:16 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-757a640d-e8c9-410f-a5ea-edacecebd824 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:68}}
Nov 29 03:24:16 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-757a640d-e8c9-410f-a5ea-edacecebd824 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.002s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:73}}
Nov 29 03:24:16 user nova-compute[75234]: WARNING os_brick.initiator.connectors.base [None req-757a640d-e8c9-410f-a5ea-edacecebd824 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Nov 29 03:24:16 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-757a640d-e8c9-410f-a5ea-edacecebd824 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:68}}
Nov 29 03:24:16 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-757a640d-e8c9-410f-a5ea-edacecebd824 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:73}}
Nov 29 03:24:16 user nova-compute[75234]: INFO os_brick.initiator.connectors.iscsi [None req-757a640d-e8c9-410f-a5ea-edacecebd824 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Nov 29 03:24:16 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0 -p 172.16.0.220:3260 {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:24:16 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0 -p 172.16.0.220:3260" returned: 21 in 0.019s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:24:16 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[8487243c-1fa7-412a-9a4c-1861eb6013cc]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:24:16 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-757a640d-e8c9-410f-a5ea-edacecebd824 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Nov 29 03:24:16 user nova-compute[75234]:  {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:24:16 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0 -p 172.16.0.220:3260 --interface default --op new {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:24:16 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0 -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.021s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:24:16 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[4dc1bed5-e07f-44ba-89e6-fb53617fa9fa]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0] added\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:24:16 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-757a640d-e8c9-410f-a5ea-edacecebd824 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0] added
Nov 29 03:24:16 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:24:16 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:24:16 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.030s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:24:16 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[141ea11d-28c6-49e9-b094-d0cfabfe4e99]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:24:16 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-757a640d-e8c9-410f-a5ea-edacecebd824 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:24:16 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:24:16 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.028s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:24:16 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[0f95dcec-d21c-4b4a-995c-3bec912a1c62]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:24:16 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-757a640d-e8c9-410f-a5ea-edacecebd824 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:24:16 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v BbOlTYfq {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:24:16 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v BbOlTYfq" returned: 0 in 0.020s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:24:16 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[91d57e75-41c4-4934-b4f3-4aac3e7b4b86]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:24:16 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-757a640d-e8c9-410f-a5ea-edacecebd824 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'BbOlTYfq'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:24:16 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:24:16 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.020s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:24:16 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[e02882c2-d286-413e-a9f2-dfa39a666360]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:24:16 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-757a640d-e8c9-410f-a5ea-edacecebd824 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:24:16 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:24:16 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.020s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:24:16 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[4d9b2398-e2a2-415c-8e2e-27057bd10296]: (4, ('tcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:cf6ab229-533a-424e-bd67-1d54d50ad4be (non-flash)\ntcp: [39] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:41f08854-31dd-4172-a5cf-01b0e695c10f (non-flash)\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:24:16 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-757a640d-e8c9-410f-a5ea-edacecebd824 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:cf6ab229-533a-424e-bd67-1d54d50ad4be (non-flash)
Nov 29 03:24:16 user nova-compute[75234]: tcp: [39] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:41f08854-31dd-4172-a5cf-01b0e695c10f (non-flash)
Nov 29 03:24:16 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:24:16 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-757a640d-e8c9-410f-a5ea-edacecebd824 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] iscsi session list stdout=tcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:cf6ab229-533a-424e-bd67-1d54d50ad4be (non-flash)
Nov 29 03:24:16 user nova-compute[75234]: tcp: [39] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:41f08854-31dd-4172-a5cf-01b0e695c10f (non-flash)
Nov 29 03:24:16 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsi_session /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Nov 29 03:24:16 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0 -p 172.16.0.220:3260 --login {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:24:16 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0 -p 172.16.0.220:3260 --login" returned: 0 in 0.064s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:24:16 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[b4585804-e916-4d17-9884-36006a534c4b]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:24:16 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-757a640d-e8c9-410f-a5ea-edacecebd824 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0, portal: 172.16.0.220,3260]
Nov 29 03:24:16 user nova-compute[75234]: Login to [iface: default, target: iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0, portal: 172.16.0.220,3260] successful.
Nov 29 03:24:16 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:24:16 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0 -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:24:16 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0 -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.025s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:24:16 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[d6e234dc-8d28-428a-9b3f-cd85abf48b79]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:24:16 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-757a640d-e8c9-410f-a5ea-edacecebd824 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:24:16 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:24:16 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.027s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:24:16 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[392f65ee-a277-4f2b-8946-c5c41305efe4]: (4, ('tcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:cf6ab229-533a-424e-bd67-1d54d50ad4be (non-flash)\ntcp: [39] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:41f08854-31dd-4172-a5cf-01b0e695c10f (non-flash)\ntcp: [42] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0 (non-flash)\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:24:16 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-757a640d-e8c9-410f-a5ea-edacecebd824 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:cf6ab229-533a-424e-bd67-1d54d50ad4be (non-flash)
Nov 29 03:24:16 user nova-compute[75234]: tcp: [39] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:41f08854-31dd-4172-a5cf-01b0e695c10f (non-flash)
Nov 29 03:24:16 user nova-compute[75234]: tcp: [42] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0 (non-flash)
Nov 29 03:24:16 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:24:16 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-757a640d-e8c9-410f-a5ea-edacecebd824 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] iscsi session list stdout=tcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:cf6ab229-533a-424e-bd67-1d54d50ad4be (non-flash)
Nov 29 03:24:16 user nova-compute[75234]: tcp: [39] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:41f08854-31dd-4172-a5cf-01b0e695c10f (non-flash)
Nov 29 03:24:16 user nova-compute[75234]: tcp: [42] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0 (non-flash)
Nov 29 03:24:16 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsi_session /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Nov 29 03:24:16 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-757a640d-e8c9-410f-a5ea-edacecebd824 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.317s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:87}}
Nov 29 03:24:16 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-757a640d-e8c9-410f-a5ea-edacecebd824 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Connected to 172.16.0.220:3260 {{(pid=75234) _connect_vol /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:644}}
Nov 29 03:24:16 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-757a640d-e8c9-410f-a5ea-edacecebd824 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] HCTL ('34', '-', '-', 0) found on session 42 with lun 0 {{(pid=75234) get_hctl /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:770}}
Nov 29 03:24:16 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-757a640d-e8c9-410f-a5ea-edacecebd824 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Scanning host 34 c: -, t: -, l: 0) {{(pid=75234) scan_iscsi /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:795}}
Nov 29 03:24:16 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host34/scan {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:24:16 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host34/scan" returned: 0 in 0.018s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:24:16 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[7766f870-d206-4a4d-a3a5-0a9e407699a7]: (4, ('- - 0', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:24:16 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-757a640d-e8c9-410f-a5ea-edacecebd824 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Searching for a device in session 42 and hctl ['34', '*', '*', 0] yield: sdc {{(pid=75234) device_name_by_hctl /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:789}}
Nov 29 03:24:16 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-757a640d-e8c9-410f-a5ea-edacecebd824 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Connected to sdc using {'target_iqn': 'iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'BbOlTYfq', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False} {{(pid=75234) _connect_vol /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:672}}
Nov 29 03:24:17 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:24:17 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-757a640d-e8c9-410f-a5ea-edacecebd824 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.355s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:87}}
Nov 29 03:24:17 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-757a640d-e8c9-410f-a5ea-edacecebd824 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] <== connect_volume: return (1358ms) {'type': 'block', 'scsi_wwn': '23433356563626439', 'path': '/dev/sdc'} {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:203}}
Nov 29 03:24:17 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-757a640d-e8c9-410f-a5ea-edacecebd824 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] [instance: d5096ca2-d005-45ac-a487-11a810446859] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '23433356563626439', 'path': '/dev/sdc'} {{(pid=75234) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Nov 29 03:24:17 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-757a640d-e8c9-410f-a5ea-edacecebd824 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-11-29T03:23:54Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description=None,display_name='tempest-ServerBootFromVolumeStableRescueTest-server-1791455206',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-serverbootfromvolumestablerescuetest-server-1791455206',id=40,image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data=None,key_name=None,keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='db00bcd0c82c44f1bda739e9857f6e84',ramdisk_id='',reservation_id='r-pix7zk6i',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-ServerBootFromVolumeStableRescueTest-1601266069',owner_user_name='tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-11-29T03:23:56Z,user_data=None,user_id='877aa877f13a4f84ac4725278010aea3',uuid=d5096ca2-d005-45ac-a487-11a810446859,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "9266d021-6e83-4bfa-8151-66c5182e61a0", "address": "fa:16:3e:ec:fb:67", "network": {"id": "4d086e08-7798-4cfe-9365-86be6f0a52d2", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-305624008-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "db00bcd0c82c44f1bda739e9857f6e84", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap9266d021-6e", "ovs_interfaceid": "9266d021-6e83-4bfa-8151-66c5182e61a0", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=75234) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Nov 29 03:24:17 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-757a640d-e8c9-410f-a5ea-edacecebd824 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Converting VIF {"id": "9266d021-6e83-4bfa-8151-66c5182e61a0", "address": "fa:16:3e:ec:fb:67", "network": {"id": "4d086e08-7798-4cfe-9365-86be6f0a52d2", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-305624008-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "db00bcd0c82c44f1bda739e9857f6e84", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap9266d021-6e", "ovs_interfaceid": "9266d021-6e83-4bfa-8151-66c5182e61a0", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:24:17 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-757a640d-e8c9-410f-a5ea-edacecebd824 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:ec:fb:67,bridge_name='br-int',has_traffic_filtering=True,id=9266d021-6e83-4bfa-8151-66c5182e61a0,network=Network(4d086e08-7798-4cfe-9365-86be6f0a52d2),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap9266d021-6e') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:24:17 user nova-compute[75234]: DEBUG nova.objects.instance [None req-757a640d-e8c9-410f-a5ea-edacecebd824 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Lazy-loading 'pci_devices' on Instance uuid d5096ca2-d005-45ac-a487-11a810446859 {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:24:17 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-757a640d-e8c9-410f-a5ea-edacecebd824 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] [instance: d5096ca2-d005-45ac-a487-11a810446859] End _get_guest_xml xml=<domain type="kvm">
Nov 29 03:24:17 user nova-compute[75234]:   <uuid>d5096ca2-d005-45ac-a487-11a810446859</uuid>
Nov 29 03:24:17 user nova-compute[75234]:   <name>instance-00000028</name>
Nov 29 03:24:17 user nova-compute[75234]:   <memory>524288</memory>
Nov 29 03:24:17 user nova-compute[75234]:   <vcpu>1</vcpu>
Nov 29 03:24:17 user nova-compute[75234]:   <metadata>
Nov 29 03:24:17 user nova-compute[75234]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Nov 29 03:24:17 user nova-compute[75234]:       <nova:package version="0.0.0"/>
Nov 29 03:24:17 user nova-compute[75234]:       <nova:name>tempest-ServerBootFromVolumeStableRescueTest-server-1791455206</nova:name>
Nov 29 03:24:17 user nova-compute[75234]:       <nova:creationTime>2023-11-29 03:24:16</nova:creationTime>
Nov 29 03:24:17 user nova-compute[75234]:       <nova:flavor name="m1.tiny">
Nov 29 03:24:17 user nova-compute[75234]:         <nova:memory>512</nova:memory>
Nov 29 03:24:17 user nova-compute[75234]:         <nova:disk>1</nova:disk>
Nov 29 03:24:17 user nova-compute[75234]:         <nova:swap>0</nova:swap>
Nov 29 03:24:17 user nova-compute[75234]:         <nova:ephemeral>0</nova:ephemeral>
Nov 29 03:24:17 user nova-compute[75234]:         <nova:vcpus>1</nova:vcpus>
Nov 29 03:24:17 user nova-compute[75234]:       </nova:flavor>
Nov 29 03:24:17 user nova-compute[75234]:       <nova:owner>
Nov 29 03:24:17 user nova-compute[75234]:         <nova:user uuid="877aa877f13a4f84ac4725278010aea3">tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member</nova:user>
Nov 29 03:24:17 user nova-compute[75234]:         <nova:project uuid="db00bcd0c82c44f1bda739e9857f6e84">tempest-ServerBootFromVolumeStableRescueTest-1601266069</nova:project>
Nov 29 03:24:17 user nova-compute[75234]:       </nova:owner>
Nov 29 03:24:17 user nova-compute[75234]:       <nova:root type="image" uuid="96c30ba7-9b73-4a30-8ba4-7138879573b7"/>
Nov 29 03:24:17 user nova-compute[75234]:       <nova:ports>
Nov 29 03:24:17 user nova-compute[75234]:         <nova:port uuid="9266d021-6e83-4bfa-8151-66c5182e61a0">
Nov 29 03:24:17 user nova-compute[75234]:           <nova:ip type="fixed" address="10.0.0.8" ipVersion="4"/>
Nov 29 03:24:17 user nova-compute[75234]:         </nova:port>
Nov 29 03:24:17 user nova-compute[75234]:       </nova:ports>
Nov 29 03:24:17 user nova-compute[75234]:     </nova:instance>
Nov 29 03:24:17 user nova-compute[75234]:   </metadata>
Nov 29 03:24:17 user nova-compute[75234]:   <sysinfo type="smbios">
Nov 29 03:24:17 user nova-compute[75234]:     <system>
Nov 29 03:24:17 user nova-compute[75234]:       <entry name="manufacturer">OpenStack Foundation</entry>
Nov 29 03:24:17 user nova-compute[75234]:       <entry name="product">OpenStack Nova</entry>
Nov 29 03:24:17 user nova-compute[75234]:       <entry name="version">0.0.0</entry>
Nov 29 03:24:17 user nova-compute[75234]:       <entry name="serial">d5096ca2-d005-45ac-a487-11a810446859</entry>
Nov 29 03:24:17 user nova-compute[75234]:       <entry name="uuid">d5096ca2-d005-45ac-a487-11a810446859</entry>
Nov 29 03:24:17 user nova-compute[75234]:       <entry name="family">Virtual Machine</entry>
Nov 29 03:24:17 user nova-compute[75234]:     </system>
Nov 29 03:24:17 user nova-compute[75234]:   </sysinfo>
Nov 29 03:24:17 user nova-compute[75234]:   <os>
Nov 29 03:24:17 user nova-compute[75234]:     <type arch="x86_64" machine="pc">hvm</type>
Nov 29 03:24:17 user nova-compute[75234]:     <boot dev="hd"/>
Nov 29 03:24:17 user nova-compute[75234]:     <smbios mode="sysinfo"/>
Nov 29 03:24:17 user nova-compute[75234]:   </os>
Nov 29 03:24:17 user nova-compute[75234]:   <features>
Nov 29 03:24:17 user nova-compute[75234]:     <acpi/>
Nov 29 03:24:17 user nova-compute[75234]:     <apic/>
Nov 29 03:24:17 user nova-compute[75234]:     <vmcoreinfo/>
Nov 29 03:24:17 user nova-compute[75234]:   </features>
Nov 29 03:24:17 user nova-compute[75234]:   <clock offset="utc">
Nov 29 03:24:17 user nova-compute[75234]:     <timer name="pit" tickpolicy="delay"/>
Nov 29 03:24:17 user nova-compute[75234]:     <timer name="rtc" tickpolicy="catchup"/>
Nov 29 03:24:17 user nova-compute[75234]:     <timer name="hpet" present="no"/>
Nov 29 03:24:17 user nova-compute[75234]:   </clock>
Nov 29 03:24:17 user nova-compute[75234]:   <cpu mode="custom" match="exact">
Nov 29 03:24:17 user nova-compute[75234]:     <model>Nehalem</model>
Nov 29 03:24:17 user nova-compute[75234]:     <topology sockets="1" cores="1" threads="1"/>
Nov 29 03:24:17 user nova-compute[75234]:   </cpu>
Nov 29 03:24:17 user nova-compute[75234]:   <devices>
Nov 29 03:24:17 user nova-compute[75234]:     <disk type="block" device="disk">
Nov 29 03:24:17 user nova-compute[75234]:       <driver name="qemu" type="raw" cache="none" io="native"/>
Nov 29 03:24:17 user nova-compute[75234]:       <source dev="/dev/sdc"/>
Nov 29 03:24:17 user nova-compute[75234]:       <target dev="vda" bus="virtio"/>
Nov 29 03:24:17 user nova-compute[75234]:       <serial>c12ce787-781c-4e31-adfa-5b51145bc1a0</serial>
Nov 29 03:24:17 user nova-compute[75234]:     </disk>
Nov 29 03:24:17 user nova-compute[75234]:     <interface type="ethernet">
Nov 29 03:24:17 user nova-compute[75234]:       <mac address="fa:16:3e:ec:fb:67"/>
Nov 29 03:24:17 user nova-compute[75234]:       <model type="virtio"/>
Nov 29 03:24:17 user nova-compute[75234]:       <mtu size="1442"/>
Nov 29 03:24:17 user nova-compute[75234]:       <target dev="tap9266d021-6e"/>
Nov 29 03:24:17 user nova-compute[75234]:     </interface>
Nov 29 03:24:17 user nova-compute[75234]:     <serial type="pty">
Nov 29 03:24:17 user nova-compute[75234]:       <log file="/opt/stack/data/nova/instances/d5096ca2-d005-45ac-a487-11a810446859/console.log" append="off"/>
Nov 29 03:24:17 user nova-compute[75234]:     </serial>
Nov 29 03:24:17 user nova-compute[75234]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Nov 29 03:24:17 user nova-compute[75234]:     <video>
Nov 29 03:24:17 user nova-compute[75234]:       <model type="virtio"/>
Nov 29 03:24:17 user nova-compute[75234]:     </video>
Nov 29 03:24:17 user nova-compute[75234]:     <rng model="virtio">
Nov 29 03:24:17 user nova-compute[75234]:       <backend model="random">/dev/urandom</backend>
Nov 29 03:24:17 user nova-compute[75234]:     </rng>
Nov 29 03:24:17 user nova-compute[75234]:     <controller type="usb" index="0" model="none"/>
Nov 29 03:24:17 user nova-compute[75234]:     <memballoon model="virtio">
Nov 29 03:24:17 user nova-compute[75234]:       <stats period="10"/>
Nov 29 03:24:17 user nova-compute[75234]:     </memballoon>
Nov 29 03:24:17 user nova-compute[75234]:   </devices>
Nov 29 03:24:17 user nova-compute[75234]: </domain>
Nov 29 03:24:17 user nova-compute[75234]:  {{(pid=75234) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7568}}
Nov 29 03:24:17 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-757a640d-e8c9-410f-a5ea-edacecebd824 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-11-29T03:23:54Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description=None,display_name='tempest-ServerBootFromVolumeStableRescueTest-server-1791455206',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-serverbootfromvolumestablerescuetest-server-1791455206',id=40,image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data=None,key_name=None,keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='db00bcd0c82c44f1bda739e9857f6e84',ramdisk_id='',reservation_id='r-pix7zk6i',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-ServerBootFromVolumeStableRescueTest-1601266069',owner_user_name='tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-11-29T03:23:56Z,user_data=None,user_id='877aa877f13a4f84ac4725278010aea3',uuid=d5096ca2-d005-45ac-a487-11a810446859,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "9266d021-6e83-4bfa-8151-66c5182e61a0", "address": "fa:16:3e:ec:fb:67", "network": {"id": "4d086e08-7798-4cfe-9365-86be6f0a52d2", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-305624008-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "db00bcd0c82c44f1bda739e9857f6e84", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap9266d021-6e", "ovs_interfaceid": "9266d021-6e83-4bfa-8151-66c5182e61a0", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Nov 29 03:24:17 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-757a640d-e8c9-410f-a5ea-edacecebd824 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Converting VIF {"id": "9266d021-6e83-4bfa-8151-66c5182e61a0", "address": "fa:16:3e:ec:fb:67", "network": {"id": "4d086e08-7798-4cfe-9365-86be6f0a52d2", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-305624008-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "db00bcd0c82c44f1bda739e9857f6e84", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap9266d021-6e", "ovs_interfaceid": "9266d021-6e83-4bfa-8151-66c5182e61a0", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:24:17 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-757a640d-e8c9-410f-a5ea-edacecebd824 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:ec:fb:67,bridge_name='br-int',has_traffic_filtering=True,id=9266d021-6e83-4bfa-8151-66c5182e61a0,network=Network(4d086e08-7798-4cfe-9365-86be6f0a52d2),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap9266d021-6e') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:24:17 user nova-compute[75234]: DEBUG os_vif [None req-757a640d-e8c9-410f-a5ea-edacecebd824 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:ec:fb:67,bridge_name='br-int',has_traffic_filtering=True,id=9266d021-6e83-4bfa-8151-66c5182e61a0,network=Network(4d086e08-7798-4cfe-9365-86be6f0a52d2),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap9266d021-6e') {{(pid=75234) plug /opt/stack/data/venv/lib/python3.10/site-packages/os_vif/__init__.py:76}}
Nov 29 03:24:17 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:24:17 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:24:17 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Nov 29 03:24:17 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:24:17 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbCreateCommand(_result=None, table=QoS, columns={'type': 'linux-noop', 'external_ids': {'id': '3b535b22-1d2f-5dcb-ba66-1211f2c36d8d', '_type': 'linux-noop'}}, row=False) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:24:17 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:24:17 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:24:17 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:24:17 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tap9266d021-6e, may_exist=True, interface_attrs={}) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:24:17 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Port, record=tap9266d021-6e, col_values=(('qos', UUID('0b6948f9-c40d-4b00-aacf-ed1db9c1975e')),), if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:24:17 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DbSetCommand(_result=None, table=Interface, record=tap9266d021-6e, col_values=(('external_ids', {'iface-id': '9266d021-6e83-4bfa-8151-66c5182e61a0', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:ec:fb:67', 'vm-uuid': 'd5096ca2-d005-45ac-a487-11a810446859'}),), if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:24:17 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:24:17 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:24:17 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:24:17 user nova-compute[75234]: INFO os_vif [None req-757a640d-e8c9-410f-a5ea-edacecebd824 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:ec:fb:67,bridge_name='br-int',has_traffic_filtering=True,id=9266d021-6e83-4bfa-8151-66c5182e61a0,network=Network(4d086e08-7798-4cfe-9365-86be6f0a52d2),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap9266d021-6e')
Nov 29 03:24:17 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-757a640d-e8c9-410f-a5ea-edacecebd824 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] No BDM found with device name vda, not building metadata. {{(pid=75234) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12089}}
Nov 29 03:24:17 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-757a640d-e8c9-410f-a5ea-edacecebd824 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] No VIF found with MAC fa:16:3e:ec:fb:67, not building metadata {{(pid=75234) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Nov 29 03:24:19 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:24:19 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:24:19 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:24:19 user nova-compute[75234]: DEBUG nova.compute.manager [req-619957a6-a147-4f6a-8e4a-54e2413c2e7a req-647a5a44-149d-4198-83de-f7fe592919d9 service nova] [instance: d5096ca2-d005-45ac-a487-11a810446859] Received event network-vif-plugged-9266d021-6e83-4bfa-8151-66c5182e61a0 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:24:19 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-619957a6-a147-4f6a-8e4a-54e2413c2e7a req-647a5a44-149d-4198-83de-f7fe592919d9 service nova] Acquiring lock "d5096ca2-d005-45ac-a487-11a810446859-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:24:19 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-619957a6-a147-4f6a-8e4a-54e2413c2e7a req-647a5a44-149d-4198-83de-f7fe592919d9 service nova] Lock "d5096ca2-d005-45ac-a487-11a810446859-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:24:19 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-619957a6-a147-4f6a-8e4a-54e2413c2e7a req-647a5a44-149d-4198-83de-f7fe592919d9 service nova] Lock "d5096ca2-d005-45ac-a487-11a810446859-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:24:19 user nova-compute[75234]: DEBUG nova.compute.manager [req-619957a6-a147-4f6a-8e4a-54e2413c2e7a req-647a5a44-149d-4198-83de-f7fe592919d9 service nova] [instance: d5096ca2-d005-45ac-a487-11a810446859] No waiting events found dispatching network-vif-plugged-9266d021-6e83-4bfa-8151-66c5182e61a0 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:24:19 user nova-compute[75234]: WARNING nova.compute.manager [req-619957a6-a147-4f6a-8e4a-54e2413c2e7a req-647a5a44-149d-4198-83de-f7fe592919d9 service nova] [instance: d5096ca2-d005-45ac-a487-11a810446859] Received unexpected event network-vif-plugged-9266d021-6e83-4bfa-8151-66c5182e61a0 for instance with vm_state building and task_state spawning.
Nov 29 03:24:19 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:24:19 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:24:21 user nova-compute[75234]: DEBUG nova.compute.manager [None req-757a640d-e8c9-410f-a5ea-edacecebd824 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] [instance: d5096ca2-d005-45ac-a487-11a810446859] Instance event wait completed in 0 seconds for  {{(pid=75234) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Nov 29 03:24:21 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-757a640d-e8c9-410f-a5ea-edacecebd824 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] [instance: d5096ca2-d005-45ac-a487-11a810446859] Guest created on hypervisor {{(pid=75234) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4421}}
Nov 29 03:24:21 user nova-compute[75234]: INFO nova.virt.libvirt.driver [-] [instance: d5096ca2-d005-45ac-a487-11a810446859] Instance spawned successfully.
Nov 29 03:24:21 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-757a640d-e8c9-410f-a5ea-edacecebd824 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] [instance: d5096ca2-d005-45ac-a487-11a810446859] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:904}}
Nov 29 03:24:21 user nova-compute[75234]: DEBUG nova.compute.manager [req-83256f30-cf94-4398-ab9f-0bc6b15bc297 req-1fd680d6-afa7-470c-9008-13626be4f780 service nova] [instance: d5096ca2-d005-45ac-a487-11a810446859] Received event network-vif-plugged-9266d021-6e83-4bfa-8151-66c5182e61a0 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:24:21 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-83256f30-cf94-4398-ab9f-0bc6b15bc297 req-1fd680d6-afa7-470c-9008-13626be4f780 service nova] Acquiring lock "d5096ca2-d005-45ac-a487-11a810446859-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:24:21 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-83256f30-cf94-4398-ab9f-0bc6b15bc297 req-1fd680d6-afa7-470c-9008-13626be4f780 service nova] Lock "d5096ca2-d005-45ac-a487-11a810446859-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:24:21 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-83256f30-cf94-4398-ab9f-0bc6b15bc297 req-1fd680d6-afa7-470c-9008-13626be4f780 service nova] Lock "d5096ca2-d005-45ac-a487-11a810446859-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:24:21 user nova-compute[75234]: DEBUG nova.compute.manager [req-83256f30-cf94-4398-ab9f-0bc6b15bc297 req-1fd680d6-afa7-470c-9008-13626be4f780 service nova] [instance: d5096ca2-d005-45ac-a487-11a810446859] No waiting events found dispatching network-vif-plugged-9266d021-6e83-4bfa-8151-66c5182e61a0 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:24:21 user nova-compute[75234]: WARNING nova.compute.manager [req-83256f30-cf94-4398-ab9f-0bc6b15bc297 req-1fd680d6-afa7-470c-9008-13626be4f780 service nova] [instance: d5096ca2-d005-45ac-a487-11a810446859] Received unexpected event network-vif-plugged-9266d021-6e83-4bfa-8151-66c5182e61a0 for instance with vm_state building and task_state spawning.
Nov 29 03:24:21 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-757a640d-e8c9-410f-a5ea-edacecebd824 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] [instance: d5096ca2-d005-45ac-a487-11a810446859] Found default for hw_cdrom_bus of ide {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:24:21 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-757a640d-e8c9-410f-a5ea-edacecebd824 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] [instance: d5096ca2-d005-45ac-a487-11a810446859] Found default for hw_disk_bus of virtio {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:24:21 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-757a640d-e8c9-410f-a5ea-edacecebd824 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] [instance: d5096ca2-d005-45ac-a487-11a810446859] Found default for hw_input_bus of None {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:24:21 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-757a640d-e8c9-410f-a5ea-edacecebd824 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] [instance: d5096ca2-d005-45ac-a487-11a810446859] Found default for hw_pointer_model of None {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:24:21 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-757a640d-e8c9-410f-a5ea-edacecebd824 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] [instance: d5096ca2-d005-45ac-a487-11a810446859] Found default for hw_video_model of virtio {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:24:21 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-757a640d-e8c9-410f-a5ea-edacecebd824 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] [instance: d5096ca2-d005-45ac-a487-11a810446859] Found default for hw_vif_model of virtio {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:24:21 user nova-compute[75234]: INFO nova.compute.manager [None req-757a640d-e8c9-410f-a5ea-edacecebd824 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] [instance: d5096ca2-d005-45ac-a487-11a810446859] Took 5.50 seconds to spawn the instance on the hypervisor.
Nov 29 03:24:21 user nova-compute[75234]: DEBUG nova.compute.manager [None req-757a640d-e8c9-410f-a5ea-edacecebd824 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] [instance: d5096ca2-d005-45ac-a487-11a810446859] Checking state {{(pid=75234) _get_power_state /opt/stack/nova/nova/compute/manager.py:1782}}
Nov 29 03:24:21 user nova-compute[75234]: INFO nova.compute.manager [None req-757a640d-e8c9-410f-a5ea-edacecebd824 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] [instance: d5096ca2-d005-45ac-a487-11a810446859] Took 26.59 seconds to build instance.
Nov 29 03:24:21 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-757a640d-e8c9-410f-a5ea-edacecebd824 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Lock "d5096ca2-d005-45ac-a487-11a810446859" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 27.208s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:24:22 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:24:22 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:24:24 user nova-compute[75234]: DEBUG nova.compute.manager [req-5218b475-b763-43f0-89ae-d4169abd20a6 req-4dadf785-9760-4159-a6b2-932b51925f4d service nova] [instance: 060c15ec-9012-471a-984a-14c9fa6ce23a] Received event network-changed-3835164f-aa54-41fa-adf1-d10ae894fba5 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:24:24 user nova-compute[75234]: DEBUG nova.compute.manager [req-5218b475-b763-43f0-89ae-d4169abd20a6 req-4dadf785-9760-4159-a6b2-932b51925f4d service nova] [instance: 060c15ec-9012-471a-984a-14c9fa6ce23a] Refreshing instance network info cache due to event network-changed-3835164f-aa54-41fa-adf1-d10ae894fba5. {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11117}}
Nov 29 03:24:24 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-5218b475-b763-43f0-89ae-d4169abd20a6 req-4dadf785-9760-4159-a6b2-932b51925f4d service nova] Acquiring lock "refresh_cache-060c15ec-9012-471a-984a-14c9fa6ce23a" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:24:24 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-5218b475-b763-43f0-89ae-d4169abd20a6 req-4dadf785-9760-4159-a6b2-932b51925f4d service nova] Acquired lock "refresh_cache-060c15ec-9012-471a-984a-14c9fa6ce23a" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:24:24 user nova-compute[75234]: DEBUG nova.network.neutron [req-5218b475-b763-43f0-89ae-d4169abd20a6 req-4dadf785-9760-4159-a6b2-932b51925f4d service nova] [instance: 060c15ec-9012-471a-984a-14c9fa6ce23a] Refreshing network info cache for port 3835164f-aa54-41fa-adf1-d10ae894fba5 {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2006}}
Nov 29 03:24:24 user nova-compute[75234]: DEBUG nova.network.neutron [req-5218b475-b763-43f0-89ae-d4169abd20a6 req-4dadf785-9760-4159-a6b2-932b51925f4d service nova] [instance: 060c15ec-9012-471a-984a-14c9fa6ce23a] Updated VIF entry in instance network info cache for port 3835164f-aa54-41fa-adf1-d10ae894fba5. {{(pid=75234) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3481}}
Nov 29 03:24:24 user nova-compute[75234]: DEBUG nova.network.neutron [req-5218b475-b763-43f0-89ae-d4169abd20a6 req-4dadf785-9760-4159-a6b2-932b51925f4d service nova] [instance: 060c15ec-9012-471a-984a-14c9fa6ce23a] Updating instance_info_cache with network_info: [{"id": "3835164f-aa54-41fa-adf1-d10ae894fba5", "address": "fa:16:3e:40:80:df", "network": {"id": "0f289171-3d73-4054-95a0-2515b4f206a0", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-1787960284-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.197", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "315732b6e3ee44d584b8e510ef199a90", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap3835164f-aa", "ovs_interfaceid": "3835164f-aa54-41fa-adf1-d10ae894fba5", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:24:24 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-5218b475-b763-43f0-89ae-d4169abd20a6 req-4dadf785-9760-4159-a6b2-932b51925f4d service nova] Releasing lock "refresh_cache-060c15ec-9012-471a-984a-14c9fa6ce23a" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:24:27 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-43b2b0a6-2018-4194-87c9-47e6716ae577 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Acquiring lock "060c15ec-9012-471a-984a-14c9fa6ce23a" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:24:27 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-43b2b0a6-2018-4194-87c9-47e6716ae577 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Lock "060c15ec-9012-471a-984a-14c9fa6ce23a" acquired by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:24:27 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-7b3bdd0d-2bcf-4fd3-a406-8344b8486454 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Acquiring lock "af699bc2-9daa-4b5a-9b4a-ad92fe857d70" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:24:27 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-7b3bdd0d-2bcf-4fd3-a406-8344b8486454 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Lock "af699bc2-9daa-4b5a-9b4a-ad92fe857d70" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:24:27 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-7b3bdd0d-2bcf-4fd3-a406-8344b8486454 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Acquiring lock "af699bc2-9daa-4b5a-9b4a-ad92fe857d70-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:24:27 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-7b3bdd0d-2bcf-4fd3-a406-8344b8486454 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Lock "af699bc2-9daa-4b5a-9b4a-ad92fe857d70-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:24:27 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-7b3bdd0d-2bcf-4fd3-a406-8344b8486454 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Lock "af699bc2-9daa-4b5a-9b4a-ad92fe857d70-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:24:27 user nova-compute[75234]: DEBUG nova.objects.instance [None req-43b2b0a6-2018-4194-87c9-47e6716ae577 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Lazy-loading 'flavor' on Instance uuid 060c15ec-9012-471a-984a-14c9fa6ce23a {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:24:27 user nova-compute[75234]: INFO nova.compute.manager [None req-7b3bdd0d-2bcf-4fd3-a406-8344b8486454 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] [instance: af699bc2-9daa-4b5a-9b4a-ad92fe857d70] Terminating instance
Nov 29 03:24:27 user nova-compute[75234]: DEBUG nova.compute.manager [None req-7b3bdd0d-2bcf-4fd3-a406-8344b8486454 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] [instance: af699bc2-9daa-4b5a-9b4a-ad92fe857d70] Start destroying the instance on the hypervisor. {{(pid=75234) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3140}}
Nov 29 03:24:27 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-43b2b0a6-2018-4194-87c9-47e6716ae577 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Lock "060c15ec-9012-471a-984a-14c9fa6ce23a" "released" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: held 0.052s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:24:27 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:24:27 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:24:27 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-43b2b0a6-2018-4194-87c9-47e6716ae577 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Acquiring lock "060c15ec-9012-471a-984a-14c9fa6ce23a" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:24:27 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-43b2b0a6-2018-4194-87c9-47e6716ae577 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Lock "060c15ec-9012-471a-984a-14c9fa6ce23a" acquired by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:24:27 user nova-compute[75234]: INFO nova.compute.manager [None req-43b2b0a6-2018-4194-87c9-47e6716ae577 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] [instance: 060c15ec-9012-471a-984a-14c9fa6ce23a] Attaching volume cebe775c-675b-4202-b042-dbeac82d6935 to /dev/vdb
Nov 29 03:24:27 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:24:27 user nova-compute[75234]: DEBUG os_brick.utils [None req-43b2b0a6-2018-4194-87c9-47e6716ae577 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:176}}
Nov 29 03:24:27 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:24:27 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.013s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:24:27 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[f4bf4deb-1cd9-405c-9c61-b407c419094a]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:dd1368c5ba2c\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:24:27 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt -v / -n -o SOURCE {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:24:27 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt -v / -n -o SOURCE" returned: 0 in 0.016s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:24:27 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[dae1a8c9-d50d-4a78-937a-87fd357df05a]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:24:27 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:24:27 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:24:27 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.015s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:24:27 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:24:27 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[ea79a638-0456-45a3-afa4-cf476370c20f]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:24:27 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[62b2f17a-4c24-445f-a7ce-7012bfb36aa9]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:24:27 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-43b2b0a6-2018-4194-87c9-47e6716ae577 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Running cmd (subprocess): nvme version {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:24:27 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-43b2b0a6-2018-4194-87c9-47e6716ae577 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] 'nvme version' failed. Not Retrying. {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:473}}
Nov 29 03:24:27 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.nvmeof [None req-43b2b0a6-2018-4194-87c9-47e6716ae577 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] nvme not present on system {{(pid=75234) nvme_present /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/nvmeof.py:756}}
Nov 29 03:24:27 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:24:27 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:473}}
Nov 29 03:24:27 user nova-compute[75234]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Nov 29 03:24:27 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[de398748-2efe-4574-aa16-2e6ad4360df3]: (4, '') {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:24:27 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.lightos [None req-43b2b0a6-2018-4194-87c9-47e6716ae577 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=75234) find_dsc /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/lightos.py:98}}
Nov 29 03:24:27 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.lightos [None req-43b2b0a6-2018-4194-87c9-47e6716ae577 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=75234) get_connector_properties /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/lightos.py:76}}
Nov 29 03:24:27 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.lightos [None req-43b2b0a6-2018-4194-87c9-47e6716ae577 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] LIGHTOS: no hostnqn found. {{(pid=75234) get_connector_properties /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/lightos.py:84}}
Nov 29 03:24:27 user nova-compute[75234]: DEBUG os_brick.utils [None req-43b2b0a6-2018-4194-87c9-47e6716ae577 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] <== get_connector_properties: return (121ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:dd1368c5ba2c', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:203}}
Nov 29 03:24:27 user nova-compute[75234]: DEBUG nova.virt.block_device [None req-43b2b0a6-2018-4194-87c9-47e6716ae577 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] [instance: 060c15ec-9012-471a-984a-14c9fa6ce23a] Updating existing volume attachment record: 84667d5d-a653-4189-b9c8-34ac3eafb067 {{(pid=75234) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Nov 29 03:24:27 user nova-compute[75234]: DEBUG nova.compute.manager [req-78264c25-09d9-456c-b009-a235be9d9472 req-58af70cd-311d-4a03-9ada-4637675cbd7e service nova] [instance: af699bc2-9daa-4b5a-9b4a-ad92fe857d70] Received event network-vif-unplugged-66dcb0f9-3180-42f7-851d-19b0c4f4119c {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:24:27 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-78264c25-09d9-456c-b009-a235be9d9472 req-58af70cd-311d-4a03-9ada-4637675cbd7e service nova] Acquiring lock "af699bc2-9daa-4b5a-9b4a-ad92fe857d70-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:24:27 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-78264c25-09d9-456c-b009-a235be9d9472 req-58af70cd-311d-4a03-9ada-4637675cbd7e service nova] Lock "af699bc2-9daa-4b5a-9b4a-ad92fe857d70-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:24:27 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-78264c25-09d9-456c-b009-a235be9d9472 req-58af70cd-311d-4a03-9ada-4637675cbd7e service nova] Lock "af699bc2-9daa-4b5a-9b4a-ad92fe857d70-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:24:27 user nova-compute[75234]: DEBUG nova.compute.manager [req-78264c25-09d9-456c-b009-a235be9d9472 req-58af70cd-311d-4a03-9ada-4637675cbd7e service nova] [instance: af699bc2-9daa-4b5a-9b4a-ad92fe857d70] No waiting events found dispatching network-vif-unplugged-66dcb0f9-3180-42f7-851d-19b0c4f4119c {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:24:27 user nova-compute[75234]: DEBUG nova.compute.manager [req-78264c25-09d9-456c-b009-a235be9d9472 req-58af70cd-311d-4a03-9ada-4637675cbd7e service nova] [instance: af699bc2-9daa-4b5a-9b4a-ad92fe857d70] Received event network-vif-unplugged-66dcb0f9-3180-42f7-851d-19b0c4f4119c for instance with task_state deleting. {{(pid=75234) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10890}}
Nov 29 03:24:27 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:24:27 user nova-compute[75234]: INFO nova.virt.libvirt.driver [-] [instance: af699bc2-9daa-4b5a-9b4a-ad92fe857d70] Instance destroyed successfully.
Nov 29 03:24:27 user nova-compute[75234]: DEBUG nova.objects.instance [None req-7b3bdd0d-2bcf-4fd3-a406-8344b8486454 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Lazy-loading 'resources' on Instance uuid af699bc2-9daa-4b5a-9b4a-ad92fe857d70 {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:24:27 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-7b3bdd0d-2bcf-4fd3-a406-8344b8486454 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-11-29T03:17:51Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-ServerActionsTestOtherA-server-1736967407',display_name='tempest-ServerActionsTestOtherA-server-1736967407',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(7),hidden=False,host='user',hostname='tempest-serveractionstestothera-server-1736967407',id=31,image_ref='',info_cache=InstanceInfoCache,instance_type_id=7,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBGysIZGz8py0LojrFdRCkkIN/DpphiAjtTxjJQY7OVo0ZaKHLKAtSriy5/opnav7HzQyT3MeG6wW+2nQfBNyK0gW9HDU5His6qvZYEbhyEyMiaUzdXksC9FbE2bauudLhQ==',key_name='tempest-keypair-1767034584',keypairs=<?>,launch_index=0,launched_at=2023-11-29T03:20:03Z,launched_on='user',locked=False,locked_by=None,memory_mb=2048,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='c1cb3d1f12f242a785a688d59dbe3cd6',ramdisk_id='',reservation_id='r-p258pz1s',resources=None,root_device_name='/dev/vda',root_gb=20,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-ServerActionsTestOtherA-1495067208',owner_user_name='tempest-ServerActionsTestOtherA-1495067208-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-11-29T03:21:36Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='d611826f3be947c0947b99c763c26d44',uuid=af699bc2-9daa-4b5a-9b4a-ad92fe857d70,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "66dcb0f9-3180-42f7-851d-19b0c4f4119c", "address": "fa:16:3e:dd:9e:50", "network": {"id": "1f946e69-dbec-429c-83f8-a186bad4f2cc", "bridge": "br-int", "label": "tempest-ServerActionsTestOtherA-486369423-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.64", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "c1cb3d1f12f242a785a688d59dbe3cd6", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap66dcb0f9-31", "ovs_interfaceid": "66dcb0f9-3180-42f7-851d-19b0c4f4119c", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Nov 29 03:24:27 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-7b3bdd0d-2bcf-4fd3-a406-8344b8486454 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Converting VIF {"id": "66dcb0f9-3180-42f7-851d-19b0c4f4119c", "address": "fa:16:3e:dd:9e:50", "network": {"id": "1f946e69-dbec-429c-83f8-a186bad4f2cc", "bridge": "br-int", "label": "tempest-ServerActionsTestOtherA-486369423-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.64", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "c1cb3d1f12f242a785a688d59dbe3cd6", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap66dcb0f9-31", "ovs_interfaceid": "66dcb0f9-3180-42f7-851d-19b0c4f4119c", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:24:27 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-7b3bdd0d-2bcf-4fd3-a406-8344b8486454 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:dd:9e:50,bridge_name='br-int',has_traffic_filtering=True,id=66dcb0f9-3180-42f7-851d-19b0c4f4119c,network=Network(1f946e69-dbec-429c-83f8-a186bad4f2cc),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap66dcb0f9-31') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:24:27 user nova-compute[75234]: DEBUG os_vif [None req-7b3bdd0d-2bcf-4fd3-a406-8344b8486454 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Unplugging vif VIFOpenVSwitch(active=True,address=fa:16:3e:dd:9e:50,bridge_name='br-int',has_traffic_filtering=True,id=66dcb0f9-3180-42f7-851d-19b0c4f4119c,network=Network(1f946e69-dbec-429c-83f8-a186bad4f2cc),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap66dcb0f9-31') {{(pid=75234) unplug /opt/stack/data/venv/lib/python3.10/site-packages/os_vif/__init__.py:109}}
Nov 29 03:24:27 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:24:27 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tap66dcb0f9-31, bridge=br-int, if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:24:27 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:24:27 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:24:27 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:24:27 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbDestroyCommand(_result=None, table=QoS, record=2b57c14a-84ee-47a7-9057-0ca1849af5f4) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:24:27 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:24:27 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:24:27 user nova-compute[75234]: INFO os_vif [None req-7b3bdd0d-2bcf-4fd3-a406-8344b8486454 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Successfully unplugged vif VIFOpenVSwitch(active=True,address=fa:16:3e:dd:9e:50,bridge_name='br-int',has_traffic_filtering=True,id=66dcb0f9-3180-42f7-851d-19b0c4f4119c,network=Network(1f946e69-dbec-429c-83f8-a186bad4f2cc),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap66dcb0f9-31')
Nov 29 03:24:27 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-7b3bdd0d-2bcf-4fd3-a406-8344b8486454 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] [instance: af699bc2-9daa-4b5a-9b4a-ad92fe857d70] calling os-brick to detach iSCSI Volume {{(pid=75234) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Nov 29 03:24:27 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-7b3bdd0d-2bcf-4fd3-a406-8344b8486454 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f8a5050c9a0>, {'target_iqn': 'iqn.2010-10.org.openstack:cf6ab229-533a-424e-bd67-1d54d50ad4be', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'wodfcngk', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdb'}, None), 'kwargs': True}}" {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:176}}
Nov 29 03:24:27 user nova-compute[75234]: WARNING os_brick.initiator.connectors.base [None req-7b3bdd0d-2bcf-4fd3-a406-8344b8486454 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Nov 29 03:24:27 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-7b3bdd0d-2bcf-4fd3-a406-8344b8486454 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:68}}
Nov 29 03:24:27 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-7b3bdd0d-2bcf-4fd3-a406-8344b8486454 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.002s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:73}}
Nov 29 03:24:27 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:24:27 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.019s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:24:27 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[e5e606fe-b708-44f6-9b71-27f5ce2bc3d8]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:cf6ab229-533a-424e-bd67-1d54d50ad4be\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:41f08854-31dd-4172-a5cf-01b0e695c10f\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:24:27 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-7b3bdd0d-2bcf-4fd3-a406-8344b8486454 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS:
Nov 29 03:24:27 user nova-compute[75234]: No targets found.
Nov 29 03:24:27 user nova-compute[75234]: iSNS:
Nov 29 03:24:27 user nova-compute[75234]: No targets found.
Nov 29 03:24:27 user nova-compute[75234]: STATIC:
Nov 29 03:24:27 user nova-compute[75234]: Target: iqn.2010-10.org.openstack:cf6ab229-533a-424e-bd67-1d54d50ad4be
Nov 29 03:24:27 user nova-compute[75234]:         Portal: 172.16.0.220:3260,-1
Nov 29 03:24:27 user nova-compute[75234]:                 Iface Name: default
Nov 29 03:24:27 user nova-compute[75234]: Target: iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0
Nov 29 03:24:27 user nova-compute[75234]:         Portal: 172.16.0.220:3260,-1
Nov 29 03:24:27 user nova-compute[75234]:                 Iface Name: default
Nov 29 03:24:27 user nova-compute[75234]: Target: iqn.2010-10.org.openstack:41f08854-31dd-4172-a5cf-01b0e695c10f
Nov 29 03:24:27 user nova-compute[75234]:         Portal: 172.16.0.220:3260,-1
Nov 29 03:24:27 user nova-compute[75234]:                 Iface Name: default
Nov 29 03:24:27 user nova-compute[75234]: FIRMWARE:
Nov 29 03:24:27 user nova-compute[75234]: No targets found.
Nov 29 03:24:27 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:24:27 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-7b3bdd0d-2bcf-4fd3-a406-8344b8486454 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Regex to get portals from discoverydb: ^SENDTARGETS:
Nov 29 03:24:27 user nova-compute[75234]: .*?^DiscoveryAddress: 172.16.0.220,3260.*?
Nov 29 03:24:27 user nova-compute[75234]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=75234) _get_discoverydb_portals /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:382}}
Nov 29 03:24:27 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-7b3bdd0d-2bcf-4fd3-a406-8344b8486454 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:cf6ab229-533a-424e-bd67-1d54d50ad4be', 0)] {{(pid=75234) _get_connection_devices /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:830}}
Nov 29 03:24:27 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:24:27 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.018s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:24:27 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[35936459-c64d-42c6-987d-7818ca7ee249]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:41f08854-31dd-4172-a5cf-01b0e695c10f\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:cf6ab229-533a-424e-bd67-1d54d50ad4be\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:24:27 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:24:27 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.024s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:24:27 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[0f5681e9-7d5c-432b-bec0-3186f09f3988]: (4, ('tcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:cf6ab229-533a-424e-bd67-1d54d50ad4be (non-flash)\ntcp: [39] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:41f08854-31dd-4172-a5cf-01b0e695c10f (non-flash)\ntcp: [42] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0 (non-flash)\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:24:27 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-7b3bdd0d-2bcf-4fd3-a406-8344b8486454 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:cf6ab229-533a-424e-bd67-1d54d50ad4be (non-flash)
Nov 29 03:24:27 user nova-compute[75234]: tcp: [39] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:41f08854-31dd-4172-a5cf-01b0e695c10f (non-flash)
Nov 29 03:24:27 user nova-compute[75234]: tcp: [42] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0 (non-flash)
Nov 29 03:24:27 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:24:27 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-7b3bdd0d-2bcf-4fd3-a406-8344b8486454 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] iscsi session list stdout=tcp: [37] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:cf6ab229-533a-424e-bd67-1d54d50ad4be (non-flash)
Nov 29 03:24:27 user nova-compute[75234]: tcp: [39] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:41f08854-31dd-4172-a5cf-01b0e695c10f (non-flash)
Nov 29 03:24:27 user nova-compute[75234]: tcp: [42] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0 (non-flash)
Nov 29 03:24:27 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsi_session /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Nov 29 03:24:27 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-7b3bdd0d-2bcf-4fd3-a406-8344b8486454 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7f8a6037e0e0>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:cf6ab229-533a-424e-bd67-1d54d50ad4be'): ({'sdb'}, set())}) {{(pid=75234) _get_connection_devices /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:863}}
Nov 29 03:24:27 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-7b3bdd0d-2bcf-4fd3-a406-8344b8486454 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Removing single pathed devices sdb {{(pid=75234) remove_connection /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:374}}
Nov 29 03:24:27 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:24:28 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.019s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:24:28 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[12a38382-da4d-4dfa-beb1-de9bbdc3ca31]: (4, ('path checker states:\nup                  3\n\npaths: 0\nbusy: False\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:24:28 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdb {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:24:28 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdb" returned: 0 in 0.020s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:24:28 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[4a7d55b5-984e-409c-823b-3f6d62cef775]: (4, ('ok\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:24:28 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-7b3bdd0d-2bcf-4fd3-a406-8344b8486454 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Flushing IO for device /dev/sdb {{(pid=75234) flush_device_io /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:434}}
Nov 29 03:24:28 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdb {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:24:28 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdb" returned: 0 in 0.019s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:24:28 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[b36628fd-fc82-4717-8b8a-9212a08cb3a6]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:24:28 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-7b3bdd0d-2bcf-4fd3-a406-8344b8486454 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Remove SCSI device /dev/sdb with /sys/block/sdb/device/delete {{(pid=75234) remove_scsi_device /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:148}}
Nov 29 03:24:28 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdb/device/delete {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:24:28 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdb/device/delete" returned: 0 in 0.040s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:24:28 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[6d64b0ad-ec84-4c9c-b740-799dc91ca094]: (4, ('1', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:24:28 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-7b3bdd0d-2bcf-4fd3-a406-8344b8486454 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Checking to see if SCSI volumes sdb have been removed. {{(pid=75234) wait_for_volumes_removal /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:156}}
Nov 29 03:24:28 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-7b3bdd0d-2bcf-4fd3-a406-8344b8486454 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] SCSI volumes sdb have been removed. {{(pid=75234) wait_for_volumes_removal /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:166}}
Nov 29 03:24:28 user nova-compute[75234]: DEBUG os_brick.privileged.rootwrap [-] Unlinking ('/dev/disk/by-id/scsi-SSCST_BIO_Storage_71b33293', '/dev/disk/by-id/scsi-23365386266373437', '/dev/disk/by-id/scsi-1SCST_BIO71b33293-3e8bf747541ac246') {{(pid=86324) unlink_root /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/privileged/rootwrap.py:219}}
Nov 29 03:24:28 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[002e8f3e-7b9b-45a0-ad89-5083f1087f04]: (4, None) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:24:28 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-7b3bdd0d-2bcf-4fd3-a406-8344b8486454 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:cf6ab229-533a-424e-bd67-1d54d50ad4be')] {{(pid=75234) _disconnect_connection /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Nov 29 03:24:28 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:cf6ab229-533a-424e-bd67-1d54d50ad4be -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:24:28 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:cf6ab229-533a-424e-bd67-1d54d50ad4be -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.020s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:24:28 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[6183b496-a02c-4b0c-96cb-0cdcb7b54d7b]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:24:28 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-7b3bdd0d-2bcf-4fd3-a406-8344b8486454 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:24:28 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:cf6ab229-533a-424e-bd67-1d54d50ad4be -p 172.16.0.220:3260 --logout {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:24:28 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:cf6ab229-533a-424e-bd67-1d54d50ad4be -p 172.16.0.220:3260 --logout" returned: 0 in 0.057s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:24:28 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[57add643-d0eb-4f2a-aaa8-5e4400db936d]: (4, ('Logging out of session [sid: 37, target: iqn.2010-10.org.openstack:cf6ab229-533a-424e-bd67-1d54d50ad4be, portal: 172.16.0.220,3260]\nLogout of [sid: 37, target: iqn.2010-10.org.openstack:cf6ab229-533a-424e-bd67-1d54d50ad4be, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:24:28 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-7b3bdd0d-2bcf-4fd3-a406-8344b8486454 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 37, target: iqn.2010-10.org.openstack:cf6ab229-533a-424e-bd67-1d54d50ad4be, portal: 172.16.0.220,3260]
Nov 29 03:24:28 user nova-compute[75234]: Logout of [sid: 37, target: iqn.2010-10.org.openstack:cf6ab229-533a-424e-bd67-1d54d50ad4be, portal: 172.16.0.220,3260] successful.
Nov 29 03:24:28 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:24:28 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:cf6ab229-533a-424e-bd67-1d54d50ad4be -p 172.16.0.220:3260 --op delete {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:24:28 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:cf6ab229-533a-424e-bd67-1d54d50ad4be -p 172.16.0.220:3260 --op delete" returned: 0 in 0.022s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:24:28 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[f43df440-3edb-4441-b276-b2f79b89f05f]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:24:28 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-7b3bdd0d-2bcf-4fd3-a406-8344b8486454 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:24:28 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-7b3bdd0d-2bcf-4fd3-a406-8344b8486454 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.318s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:87}}
Nov 29 03:24:28 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-7b3bdd0d-2bcf-4fd3-a406-8344b8486454 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] <== disconnect_volume: return (319ms) None {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:203}}
Nov 29 03:24:28 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-7b3bdd0d-2bcf-4fd3-a406-8344b8486454 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] [instance: af699bc2-9daa-4b5a-9b4a-ad92fe857d70] Disconnected iSCSI Volume {{(pid=75234) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:79}}
Nov 29 03:24:28 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-7b3bdd0d-2bcf-4fd3-a406-8344b8486454 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] [instance: af699bc2-9daa-4b5a-9b4a-ad92fe857d70] Deleting instance files /opt/stack/data/nova/instances/af699bc2-9daa-4b5a-9b4a-ad92fe857d70_del
Nov 29 03:24:28 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-7b3bdd0d-2bcf-4fd3-a406-8344b8486454 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] [instance: af699bc2-9daa-4b5a-9b4a-ad92fe857d70] Deletion of /opt/stack/data/nova/instances/af699bc2-9daa-4b5a-9b4a-ad92fe857d70_del complete
Nov 29 03:24:28 user nova-compute[75234]: INFO nova.compute.manager [None req-7b3bdd0d-2bcf-4fd3-a406-8344b8486454 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] [instance: af699bc2-9daa-4b5a-9b4a-ad92fe857d70] Took 1.20 seconds to destroy the instance on the hypervisor.
Nov 29 03:24:28 user nova-compute[75234]: DEBUG oslo.service.loopingcall [None req-7b3bdd0d-2bcf-4fd3-a406-8344b8486454 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=75234) func /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/loopingcall.py:435}}
Nov 29 03:24:28 user nova-compute[75234]: DEBUG nova.compute.manager [-] [instance: af699bc2-9daa-4b5a-9b4a-ad92fe857d70] Deallocating network for instance {{(pid=75234) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2279}}
Nov 29 03:24:28 user nova-compute[75234]: DEBUG nova.network.neutron [-] [instance: af699bc2-9daa-4b5a-9b4a-ad92fe857d70] deallocate_for_instance() {{(pid=75234) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1802}}
Nov 29 03:24:28 user nova-compute[75234]: DEBUG nova.network.neutron [-] [instance: af699bc2-9daa-4b5a-9b4a-ad92fe857d70] Updating instance_info_cache with network_info: [] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:24:28 user nova-compute[75234]: INFO nova.compute.manager [-] [instance: af699bc2-9daa-4b5a-9b4a-ad92fe857d70] Took 0.63 seconds to deallocate network for instance.
Nov 29 03:24:29 user nova-compute[75234]: DEBUG nova.compute.manager [req-479a6647-6ac8-43fb-8195-de9a4973d1c0 req-54279c34-3764-4aa8-85dc-fade22ee9158 service nova] [instance: af699bc2-9daa-4b5a-9b4a-ad92fe857d70] Received event network-vif-plugged-66dcb0f9-3180-42f7-851d-19b0c4f4119c {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:24:29 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-479a6647-6ac8-43fb-8195-de9a4973d1c0 req-54279c34-3764-4aa8-85dc-fade22ee9158 service nova] Acquiring lock "af699bc2-9daa-4b5a-9b4a-ad92fe857d70-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:24:29 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-479a6647-6ac8-43fb-8195-de9a4973d1c0 req-54279c34-3764-4aa8-85dc-fade22ee9158 service nova] Lock "af699bc2-9daa-4b5a-9b4a-ad92fe857d70-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:24:29 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-479a6647-6ac8-43fb-8195-de9a4973d1c0 req-54279c34-3764-4aa8-85dc-fade22ee9158 service nova] Lock "af699bc2-9daa-4b5a-9b4a-ad92fe857d70-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:24:29 user nova-compute[75234]: DEBUG nova.compute.manager [req-479a6647-6ac8-43fb-8195-de9a4973d1c0 req-54279c34-3764-4aa8-85dc-fade22ee9158 service nova] [instance: af699bc2-9daa-4b5a-9b4a-ad92fe857d70] No waiting events found dispatching network-vif-plugged-66dcb0f9-3180-42f7-851d-19b0c4f4119c {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:24:29 user nova-compute[75234]: WARNING nova.compute.manager [req-479a6647-6ac8-43fb-8195-de9a4973d1c0 req-54279c34-3764-4aa8-85dc-fade22ee9158 service nova] [instance: af699bc2-9daa-4b5a-9b4a-ad92fe857d70] Received unexpected event network-vif-plugged-66dcb0f9-3180-42f7-851d-19b0c4f4119c for instance with vm_state active and task_state deleting.
Nov 29 03:24:29 user nova-compute[75234]: DEBUG nova.compute.manager [req-479a6647-6ac8-43fb-8195-de9a4973d1c0 req-54279c34-3764-4aa8-85dc-fade22ee9158 service nova] [instance: af699bc2-9daa-4b5a-9b4a-ad92fe857d70] Received event network-vif-deleted-66dcb0f9-3180-42f7-851d-19b0c4f4119c {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:24:29 user nova-compute[75234]: INFO nova.compute.manager [None req-7b3bdd0d-2bcf-4fd3-a406-8344b8486454 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] [instance: af699bc2-9daa-4b5a-9b4a-ad92fe857d70] Took 0.80 seconds to detach 1 volumes for instance.
Nov 29 03:24:29 user nova-compute[75234]: DEBUG nova.compute.manager [None req-7b3bdd0d-2bcf-4fd3-a406-8344b8486454 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] [instance: af699bc2-9daa-4b5a-9b4a-ad92fe857d70] Deleting volume: cf6ab229-533a-424e-bd67-1d54d50ad4be {{(pid=75234) _cleanup_volumes /opt/stack/nova/nova/compute/manager.py:3237}}
Nov 29 03:24:29 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-7b3bdd0d-2bcf-4fd3-a406-8344b8486454 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:24:29 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-7b3bdd0d-2bcf-4fd3-a406-8344b8486454 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:24:30 user nova-compute[75234]: DEBUG nova.compute.provider_tree [None req-7b3bdd0d-2bcf-4fd3-a406-8344b8486454 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Inventory has not changed in ProviderTree for provider: 4d291220-7c37-4839-a0e3-b23c66502b74 {{(pid=75234) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Nov 29 03:24:30 user nova-compute[75234]: DEBUG nova.scheduler.client.report [None req-7b3bdd0d-2bcf-4fd3-a406-8344b8486454 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Inventory has not changed for provider 4d291220-7c37-4839-a0e3-b23c66502b74 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=75234) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Nov 29 03:24:30 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-7b3bdd0d-2bcf-4fd3-a406-8344b8486454 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.346s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:24:30 user nova-compute[75234]: INFO nova.scheduler.client.report [None req-7b3bdd0d-2bcf-4fd3-a406-8344b8486454 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Deleted allocations for instance af699bc2-9daa-4b5a-9b4a-ad92fe857d70
Nov 29 03:24:30 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-7b3bdd0d-2bcf-4fd3-a406-8344b8486454 tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Lock "af699bc2-9daa-4b5a-9b4a-ad92fe857d70" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 3.248s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:24:32 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:24:32 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:24:35 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-43b2b0a6-2018-4194-87c9-47e6716ae577 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] [instance: 060c15ec-9012-471a-984a-14c9fa6ce23a] Calling os-brick to attach iSCSI Volume {{(pid=75234) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Nov 29 03:24:35 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-43b2b0a6-2018-4194-87c9-47e6716ae577 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f8a5050c9a0>, 'connection_properties': {'target_iqn': 'iqn.2010-10.org.openstack:cebe775c-675b-4202-b042-dbeac82d6935', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'UGxIRjPu', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:176}}
Nov 29 03:24:35 user nova-compute[75234]: WARNING os_brick.initiator.connectors.base [None req-43b2b0a6-2018-4194-87c9-47e6716ae577 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Nov 29 03:24:35 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-43b2b0a6-2018-4194-87c9-47e6716ae577 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:68}}
Nov 29 03:24:35 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-43b2b0a6-2018-4194-87c9-47e6716ae577 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:73}}
Nov 29 03:24:35 user nova-compute[75234]: WARNING os_brick.initiator.connectors.base [None req-43b2b0a6-2018-4194-87c9-47e6716ae577 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Nov 29 03:24:35 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-43b2b0a6-2018-4194-87c9-47e6716ae577 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:cebe775c-675b-4202-b042-dbeac82d6935" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:68}}
Nov 29 03:24:35 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-43b2b0a6-2018-4194-87c9-47e6716ae577 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:cebe775c-675b-4202-b042-dbeac82d6935" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:73}}
Nov 29 03:24:35 user nova-compute[75234]: INFO os_brick.initiator.connectors.iscsi [None req-43b2b0a6-2018-4194-87c9-47e6716ae577 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Nov 29 03:24:35 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:cebe775c-675b-4202-b042-dbeac82d6935 -p 172.16.0.220:3260 {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:24:35 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:cebe775c-675b-4202-b042-dbeac82d6935 -p 172.16.0.220:3260" returned: 21 in 0.020s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:24:35 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[0444c451-6de0-4d16-bcbf-2eccc6b15d9f]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:24:35 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-43b2b0a6-2018-4194-87c9-47e6716ae577 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Nov 29 03:24:35 user nova-compute[75234]:  {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:24:35 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:cebe775c-675b-4202-b042-dbeac82d6935 -p 172.16.0.220:3260 --interface default --op new {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:24:35 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:cebe775c-675b-4202-b042-dbeac82d6935 -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.019s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:24:35 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[bd4dbc85-4e87-47aa-933c-ed9b5964099e]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:cebe775c-675b-4202-b042-dbeac82d6935] added\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:24:35 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-43b2b0a6-2018-4194-87c9-47e6716ae577 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:cebe775c-675b-4202-b042-dbeac82d6935] added
Nov 29 03:24:35 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:24:35 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:cebe775c-675b-4202-b042-dbeac82d6935 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:24:35 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:cebe775c-675b-4202-b042-dbeac82d6935 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.018s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:24:35 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[0cbff939-b90f-403c-889a-741eafa0f738]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:24:35 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-43b2b0a6-2018-4194-87c9-47e6716ae577 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:24:35 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:cebe775c-675b-4202-b042-dbeac82d6935 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:24:35 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:cebe775c-675b-4202-b042-dbeac82d6935 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.019s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:24:35 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[731ec03c-7ae4-425d-8eff-5439403e7dad]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:24:35 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-43b2b0a6-2018-4194-87c9-47e6716ae577 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:24:35 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:cebe775c-675b-4202-b042-dbeac82d6935 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v UGxIRjPu {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:24:35 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:cebe775c-675b-4202-b042-dbeac82d6935 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v UGxIRjPu" returned: 0 in 0.025s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:24:35 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[4faa9312-c92d-47c8-9878-bb6e8ced41b3]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:24:35 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-43b2b0a6-2018-4194-87c9-47e6716ae577 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'UGxIRjPu'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:24:35 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:cebe775c-675b-4202-b042-dbeac82d6935 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:24:35 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:cebe775c-675b-4202-b042-dbeac82d6935 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.020s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:24:35 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[e4d72917-daa9-4ed8-a922-49acea17adb2]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:24:35 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-43b2b0a6-2018-4194-87c9-47e6716ae577 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:24:35 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:24:35 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.021s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:24:35 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[26934ef4-995e-49e7-b3c3-6f509ed851b0]: (4, ('tcp: [39] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:41f08854-31dd-4172-a5cf-01b0e695c10f (non-flash)\ntcp: [42] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0 (non-flash)\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:24:35 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-43b2b0a6-2018-4194-87c9-47e6716ae577 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [39] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:41f08854-31dd-4172-a5cf-01b0e695c10f (non-flash)
Nov 29 03:24:35 user nova-compute[75234]: tcp: [42] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0 (non-flash)
Nov 29 03:24:35 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:24:35 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-43b2b0a6-2018-4194-87c9-47e6716ae577 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] iscsi session list stdout=tcp: [39] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:41f08854-31dd-4172-a5cf-01b0e695c10f (non-flash)
Nov 29 03:24:35 user nova-compute[75234]: tcp: [42] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0 (non-flash)
Nov 29 03:24:35 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsi_session /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Nov 29 03:24:35 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:cebe775c-675b-4202-b042-dbeac82d6935 -p 172.16.0.220:3260 --login {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:24:36 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:cebe775c-675b-4202-b042-dbeac82d6935 -p 172.16.0.220:3260 --login" returned: 0 in 0.058s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:24:36 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[129c92cc-5ce5-4b9a-b433-5f11afe1c974]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:cebe775c-675b-4202-b042-dbeac82d6935, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:cebe775c-675b-4202-b042-dbeac82d6935, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:24:36 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-43b2b0a6-2018-4194-87c9-47e6716ae577 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:cebe775c-675b-4202-b042-dbeac82d6935, portal: 172.16.0.220,3260]
Nov 29 03:24:36 user nova-compute[75234]: Login to [iface: default, target: iqn.2010-10.org.openstack:cebe775c-675b-4202-b042-dbeac82d6935, portal: 172.16.0.220,3260] successful.
Nov 29 03:24:36 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:24:36 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:cebe775c-675b-4202-b042-dbeac82d6935 -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:24:36 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:cebe775c-675b-4202-b042-dbeac82d6935 -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.017s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:24:36 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[8952ad66-ec5a-4b7c-a982-2d1cd9924900]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:24:36 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-43b2b0a6-2018-4194-87c9-47e6716ae577 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:24:36 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:24:36 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.021s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:24:36 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[eceb8b3e-0850-4e62-a68c-61ac28b6edb5]: (4, ('tcp: [39] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:41f08854-31dd-4172-a5cf-01b0e695c10f (non-flash)\ntcp: [42] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0 (non-flash)\ntcp: [43] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:cebe775c-675b-4202-b042-dbeac82d6935 (non-flash)\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:24:36 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-43b2b0a6-2018-4194-87c9-47e6716ae577 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [39] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:41f08854-31dd-4172-a5cf-01b0e695c10f (non-flash)
Nov 29 03:24:36 user nova-compute[75234]: tcp: [42] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0 (non-flash)
Nov 29 03:24:36 user nova-compute[75234]: tcp: [43] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:cebe775c-675b-4202-b042-dbeac82d6935 (non-flash)
Nov 29 03:24:36 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:24:36 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-43b2b0a6-2018-4194-87c9-47e6716ae577 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] iscsi session list stdout=tcp: [39] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:41f08854-31dd-4172-a5cf-01b0e695c10f (non-flash)
Nov 29 03:24:36 user nova-compute[75234]: tcp: [42] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0 (non-flash)
Nov 29 03:24:36 user nova-compute[75234]: tcp: [43] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:cebe775c-675b-4202-b042-dbeac82d6935 (non-flash)
Nov 29 03:24:36 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsi_session /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Nov 29 03:24:36 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-43b2b0a6-2018-4194-87c9-47e6716ae577 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:cebe775c-675b-4202-b042-dbeac82d6935" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.279s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:87}}
Nov 29 03:24:36 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-43b2b0a6-2018-4194-87c9-47e6716ae577 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Connected to 172.16.0.220:3260 {{(pid=75234) _connect_vol /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:644}}
Nov 29 03:24:36 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-43b2b0a6-2018-4194-87c9-47e6716ae577 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] HCTL ('33', '-', '-', 0) found on session 43 with lun 0 {{(pid=75234) get_hctl /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:770}}
Nov 29 03:24:36 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-43b2b0a6-2018-4194-87c9-47e6716ae577 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Scanning host 33 c: -, t: -, l: 0) {{(pid=75234) scan_iscsi /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:795}}
Nov 29 03:24:36 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host33/scan {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:24:36 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host33/scan" returned: 0 in 0.017s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:24:36 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[b5f44dd7-db42-43c9-ad39-c0181643306b]: (4, ('- - 0', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:24:36 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-43b2b0a6-2018-4194-87c9-47e6716ae577 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Searching for a device in session 43 and hctl ['33', '*', '*', 0] yield: None {{(pid=75234) device_name_by_hctl /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:789}}
Nov 29 03:24:37 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-43b2b0a6-2018-4194-87c9-47e6716ae577 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Searching for a device in session 43 and hctl ['33', '*', '*', 0] yield: sdb {{(pid=75234) device_name_by_hctl /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:789}}
Nov 29 03:24:37 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-43b2b0a6-2018-4194-87c9-47e6716ae577 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Connected to sdb using {'target_iqn': 'iqn.2010-10.org.openstack:cebe775c-675b-4202-b042-dbeac82d6935', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'UGxIRjPu', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False} {{(pid=75234) _connect_vol /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:672}}
Nov 29 03:24:37 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-43b2b0a6-2018-4194-87c9-47e6716ae577 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.310s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:87}}
Nov 29 03:24:37 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-43b2b0a6-2018-4194-87c9-47e6716ae577 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] <== connect_volume: return (1311ms) {'type': 'block', 'scsi_wwn': '23564613937613030', 'path': '/dev/sdb'} {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:203}}
Nov 29 03:24:37 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-43b2b0a6-2018-4194-87c9-47e6716ae577 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] [instance: 060c15ec-9012-471a-984a-14c9fa6ce23a] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '23564613937613030', 'path': '/dev/sdb'} {{(pid=75234) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Nov 29 03:24:37 user nova-compute[75234]: DEBUG nova.objects.instance [None req-43b2b0a6-2018-4194-87c9-47e6716ae577 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Lazy-loading 'flavor' on Instance uuid 060c15ec-9012-471a-984a-14c9fa6ce23a {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:24:37 user nova-compute[75234]: DEBUG nova.virt.libvirt.guest [None req-43b2b0a6-2018-4194-87c9-47e6716ae577 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] attach device xml: <disk type="block" device="disk">
Nov 29 03:24:37 user nova-compute[75234]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Nov 29 03:24:37 user nova-compute[75234]:   <source dev="/dev/sdb"/>
Nov 29 03:24:37 user nova-compute[75234]:   <target dev="vdb" bus="virtio"/>
Nov 29 03:24:37 user nova-compute[75234]:   <serial>cebe775c-675b-4202-b042-dbeac82d6935</serial>
Nov 29 03:24:37 user nova-compute[75234]: </disk>
Nov 29 03:24:37 user nova-compute[75234]:  {{(pid=75234) attach_device /opt/stack/nova/nova/virt/libvirt/guest.py:339}}
Nov 29 03:24:37 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:24:37 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-43b2b0a6-2018-4194-87c9-47e6716ae577 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] No BDM found with device name vda, not building metadata. {{(pid=75234) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12089}}
Nov 29 03:24:37 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-43b2b0a6-2018-4194-87c9-47e6716ae577 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] No BDM found with device name vdb, not building metadata. {{(pid=75234) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12089}}
Nov 29 03:24:37 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-43b2b0a6-2018-4194-87c9-47e6716ae577 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] No VIF found with MAC fa:16:3e:40:80:df, not building metadata {{(pid=75234) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Nov 29 03:24:38 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-43b2b0a6-2018-4194-87c9-47e6716ae577 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Lock "060c15ec-9012-471a-984a-14c9fa6ce23a" "released" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: held 10.797s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:24:38 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-8ae021bf-1b04-4a75-a501-bbbc3050b131 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Acquiring lock "060c15ec-9012-471a-984a-14c9fa6ce23a" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:24:38 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-8ae021bf-1b04-4a75-a501-bbbc3050b131 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Lock "060c15ec-9012-471a-984a-14c9fa6ce23a" acquired by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:24:38 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:24:38 user nova-compute[75234]: INFO nova.compute.manager [None req-8ae021bf-1b04-4a75-a501-bbbc3050b131 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] [instance: 060c15ec-9012-471a-984a-14c9fa6ce23a] Detaching volume cebe775c-675b-4202-b042-dbeac82d6935
Nov 29 03:24:38 user nova-compute[75234]: INFO nova.virt.block_device [None req-8ae021bf-1b04-4a75-a501-bbbc3050b131 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] [instance: 060c15ec-9012-471a-984a-14c9fa6ce23a] Attempting to driver detach volume cebe775c-675b-4202-b042-dbeac82d6935 from mountpoint /dev/vdb
Nov 29 03:24:38 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-8ae021bf-1b04-4a75-a501-bbbc3050b131 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Attempting to detach device vdb from instance 060c15ec-9012-471a-984a-14c9fa6ce23a from the persistent domain config. {{(pid=75234) _detach_from_persistent /opt/stack/nova/nova/virt/libvirt/driver.py:2487}}
Nov 29 03:24:38 user nova-compute[75234]: DEBUG nova.virt.libvirt.guest [None req-8ae021bf-1b04-4a75-a501-bbbc3050b131 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] detach device xml: <disk type="block" device="disk">
Nov 29 03:24:38 user nova-compute[75234]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Nov 29 03:24:38 user nova-compute[75234]:   <source dev="/dev/sdb"/>
Nov 29 03:24:38 user nova-compute[75234]:   <target dev="vdb" bus="virtio"/>
Nov 29 03:24:38 user nova-compute[75234]:   <serial>cebe775c-675b-4202-b042-dbeac82d6935</serial>
Nov 29 03:24:38 user nova-compute[75234]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x07" function="0x0"/>
Nov 29 03:24:38 user nova-compute[75234]: </disk>
Nov 29 03:24:38 user nova-compute[75234]:  {{(pid=75234) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Nov 29 03:24:38 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-8ae021bf-1b04-4a75-a501-bbbc3050b131 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Successfully detached device vdb from instance 060c15ec-9012-471a-984a-14c9fa6ce23a from the persistent domain config.
Nov 29 03:24:38 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-8ae021bf-1b04-4a75-a501-bbbc3050b131 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] (1/8): Attempting to detach device vdb with device alias virtio-disk1 from instance 060c15ec-9012-471a-984a-14c9fa6ce23a from the live domain config. {{(pid=75234) _detach_from_live_with_retry /opt/stack/nova/nova/virt/libvirt/driver.py:2523}}
Nov 29 03:24:38 user nova-compute[75234]: DEBUG nova.virt.libvirt.guest [None req-8ae021bf-1b04-4a75-a501-bbbc3050b131 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] detach device xml: <disk type="block" device="disk">
Nov 29 03:24:38 user nova-compute[75234]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Nov 29 03:24:38 user nova-compute[75234]:   <source dev="/dev/sdb"/>
Nov 29 03:24:38 user nova-compute[75234]:   <target dev="vdb" bus="virtio"/>
Nov 29 03:24:38 user nova-compute[75234]:   <serial>cebe775c-675b-4202-b042-dbeac82d6935</serial>
Nov 29 03:24:38 user nova-compute[75234]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x07" function="0x0"/>
Nov 29 03:24:38 user nova-compute[75234]: </disk>
Nov 29 03:24:38 user nova-compute[75234]:  {{(pid=75234) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Nov 29 03:24:39 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-75769adb-154d-4655-b244-6c6ef2eaf6e8 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Acquiring lock "f6bc212d-85cd-479c-92ae-112c9f0101bc" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:24:39 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-75769adb-154d-4655-b244-6c6ef2eaf6e8 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Lock "f6bc212d-85cd-479c-92ae-112c9f0101bc" acquired by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:24:39 user nova-compute[75234]: DEBUG nova.objects.instance [None req-75769adb-154d-4655-b244-6c6ef2eaf6e8 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Lazy-loading 'flavor' on Instance uuid f6bc212d-85cd-479c-92ae-112c9f0101bc {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:24:39 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-75769adb-154d-4655-b244-6c6ef2eaf6e8 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Lock "f6bc212d-85cd-479c-92ae-112c9f0101bc" "released" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: held 0.056s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:24:39 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-75769adb-154d-4655-b244-6c6ef2eaf6e8 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Acquiring lock "f6bc212d-85cd-479c-92ae-112c9f0101bc" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:24:39 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-75769adb-154d-4655-b244-6c6ef2eaf6e8 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Lock "f6bc212d-85cd-479c-92ae-112c9f0101bc" acquired by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:24:39 user nova-compute[75234]: INFO nova.compute.manager [None req-75769adb-154d-4655-b244-6c6ef2eaf6e8 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] [instance: f6bc212d-85cd-479c-92ae-112c9f0101bc] Attaching volume 023a5970-c0e3-48f5-a8ef-d9d90fed8b37 to /dev/vdb
Nov 29 03:24:39 user nova-compute[75234]: DEBUG os_brick.utils [None req-75769adb-154d-4655-b244-6c6ef2eaf6e8 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:176}}
Nov 29 03:24:39 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:24:39 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.014s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:24:39 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[a87b8739-8a2c-49c6-a5c4-bccd0a82a612]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:dd1368c5ba2c\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:24:39 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-8ae021bf-1b04-4a75-a501-bbbc3050b131 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Start waiting for the detach event from libvirt for device vdb with device alias virtio-disk1 for instance 060c15ec-9012-471a-984a-14c9fa6ce23a {{(pid=75234) _detach_from_live_and_wait_for_event /opt/stack/nova/nova/virt/libvirt/driver.py:2599}}
Nov 29 03:24:39 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt -v / -n -o SOURCE {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:24:39 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt -v / -n -o SOURCE" returned: 0 in 0.015s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:24:39 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[a649d1e8-dd73-4259-a975-5e2f307e8ae5]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:24:39 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:24:39 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.015s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:24:39 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[cbe0a2ae-51c5-46b6-8db0-210ca91da491]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:24:39 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[d3ab66ae-dd96-4bcc-b12c-989eb54f9e72]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:24:39 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-75769adb-154d-4655-b244-6c6ef2eaf6e8 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Running cmd (subprocess): nvme version {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:24:39 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-75769adb-154d-4655-b244-6c6ef2eaf6e8 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] 'nvme version' failed. Not Retrying. {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:473}}
Nov 29 03:24:39 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.nvmeof [None req-75769adb-154d-4655-b244-6c6ef2eaf6e8 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] nvme not present on system {{(pid=75234) nvme_present /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/nvmeof.py:756}}
Nov 29 03:24:39 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:24:39 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:473}}
Nov 29 03:24:39 user nova-compute[75234]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Nov 29 03:24:39 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[9d579d37-e231-43ef-93d8-06e85a8c9c73]: (4, '') {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:24:39 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.lightos [None req-75769adb-154d-4655-b244-6c6ef2eaf6e8 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=75234) find_dsc /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/lightos.py:98}}
Nov 29 03:24:39 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.lightos [None req-75769adb-154d-4655-b244-6c6ef2eaf6e8 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=75234) get_connector_properties /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/lightos.py:76}}
Nov 29 03:24:39 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.lightos [None req-75769adb-154d-4655-b244-6c6ef2eaf6e8 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] LIGHTOS: no hostnqn found. {{(pid=75234) get_connector_properties /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/lightos.py:84}}
Nov 29 03:24:39 user nova-compute[75234]: DEBUG os_brick.utils [None req-75769adb-154d-4655-b244-6c6ef2eaf6e8 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] <== get_connector_properties: return (125ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:dd1368c5ba2c', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:203}}
Nov 29 03:24:39 user nova-compute[75234]: DEBUG nova.virt.block_device [None req-75769adb-154d-4655-b244-6c6ef2eaf6e8 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] [instance: f6bc212d-85cd-479c-92ae-112c9f0101bc] Updating existing volume attachment record: 0a84500e-14a8-4826-810c-942e62bf8ac0 {{(pid=75234) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Nov 29 03:24:42 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:24:42 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:24:42 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:24:46 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-75769adb-154d-4655-b244-6c6ef2eaf6e8 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] [instance: f6bc212d-85cd-479c-92ae-112c9f0101bc] Calling os-brick to attach iSCSI Volume {{(pid=75234) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Nov 29 03:24:46 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-75769adb-154d-4655-b244-6c6ef2eaf6e8 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f8a5050c9a0>, 'connection_properties': {'target_iqn': 'iqn.2010-10.org.openstack:023a5970-c0e3-48f5-a8ef-d9d90fed8b37', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'hjrHYOxB', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:176}}
Nov 29 03:24:46 user nova-compute[75234]: WARNING os_brick.initiator.connectors.base [None req-75769adb-154d-4655-b244-6c6ef2eaf6e8 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Nov 29 03:24:46 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-75769adb-154d-4655-b244-6c6ef2eaf6e8 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:68}}
Nov 29 03:24:46 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-75769adb-154d-4655-b244-6c6ef2eaf6e8 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.002s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:73}}
Nov 29 03:24:46 user nova-compute[75234]: WARNING os_brick.initiator.connectors.base [None req-75769adb-154d-4655-b244-6c6ef2eaf6e8 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Nov 29 03:24:46 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-75769adb-154d-4655-b244-6c6ef2eaf6e8 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:023a5970-c0e3-48f5-a8ef-d9d90fed8b37" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:68}}
Nov 29 03:24:46 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-75769adb-154d-4655-b244-6c6ef2eaf6e8 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:023a5970-c0e3-48f5-a8ef-d9d90fed8b37" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:73}}
Nov 29 03:24:46 user nova-compute[75234]: INFO os_brick.initiator.connectors.iscsi [None req-75769adb-154d-4655-b244-6c6ef2eaf6e8 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Nov 29 03:24:46 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:023a5970-c0e3-48f5-a8ef-d9d90fed8b37 -p 172.16.0.220:3260 {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:24:47 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:023a5970-c0e3-48f5-a8ef-d9d90fed8b37 -p 172.16.0.220:3260" returned: 21 in 0.017s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:24:47 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[501f17bb-be6a-40fa-b6b9-e9b996926af0]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:24:47 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-75769adb-154d-4655-b244-6c6ef2eaf6e8 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Nov 29 03:24:47 user nova-compute[75234]:  {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:24:47 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:023a5970-c0e3-48f5-a8ef-d9d90fed8b37 -p 172.16.0.220:3260 --interface default --op new {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:24:47 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:023a5970-c0e3-48f5-a8ef-d9d90fed8b37 -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.019s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:24:47 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[53feab49-9cb4-4fe7-99d2-4cd824eb2f26]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:023a5970-c0e3-48f5-a8ef-d9d90fed8b37] added\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:24:47 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-75769adb-154d-4655-b244-6c6ef2eaf6e8 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:023a5970-c0e3-48f5-a8ef-d9d90fed8b37] added
Nov 29 03:24:47 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:24:47 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:023a5970-c0e3-48f5-a8ef-d9d90fed8b37 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:24:47 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:023a5970-c0e3-48f5-a8ef-d9d90fed8b37 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.024s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:24:47 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[1bd48c8e-b269-4853-b0ca-93dee657c499]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:24:47 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-75769adb-154d-4655-b244-6c6ef2eaf6e8 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:24:47 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:023a5970-c0e3-48f5-a8ef-d9d90fed8b37 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:24:47 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:023a5970-c0e3-48f5-a8ef-d9d90fed8b37 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.017s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:24:47 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[7e0f9a57-e330-492e-92ea-60a821f2257c]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:24:47 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-75769adb-154d-4655-b244-6c6ef2eaf6e8 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:24:47 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:023a5970-c0e3-48f5-a8ef-d9d90fed8b37 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v hjrHYOxB {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:24:47 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:023a5970-c0e3-48f5-a8ef-d9d90fed8b37 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v hjrHYOxB" returned: 0 in 0.017s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:24:47 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[58b34b03-a8e1-4fa9-8a76-a75a9f883e7d]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:24:47 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-75769adb-154d-4655-b244-6c6ef2eaf6e8 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'hjrHYOxB'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:24:47 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:023a5970-c0e3-48f5-a8ef-d9d90fed8b37 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:24:47 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:023a5970-c0e3-48f5-a8ef-d9d90fed8b37 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.018s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:24:47 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[40b218aa-bc5e-44a8-9b0d-13278181e668]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:24:47 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-75769adb-154d-4655-b244-6c6ef2eaf6e8 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:24:47 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:24:47 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.021s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:24:47 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[cb0a399a-8a90-49c8-ad1c-414a83f2b834]: (4, ('tcp: [39] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:41f08854-31dd-4172-a5cf-01b0e695c10f (non-flash)\ntcp: [42] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0 (non-flash)\ntcp: [43] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:cebe775c-675b-4202-b042-dbeac82d6935 (non-flash)\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:24:47 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-75769adb-154d-4655-b244-6c6ef2eaf6e8 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [39] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:41f08854-31dd-4172-a5cf-01b0e695c10f (non-flash)
Nov 29 03:24:47 user nova-compute[75234]: tcp: [42] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0 (non-flash)
Nov 29 03:24:47 user nova-compute[75234]: tcp: [43] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:cebe775c-675b-4202-b042-dbeac82d6935 (non-flash)
Nov 29 03:24:47 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:24:47 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-75769adb-154d-4655-b244-6c6ef2eaf6e8 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] iscsi session list stdout=tcp: [39] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:41f08854-31dd-4172-a5cf-01b0e695c10f (non-flash)
Nov 29 03:24:47 user nova-compute[75234]: tcp: [42] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0 (non-flash)
Nov 29 03:24:47 user nova-compute[75234]: tcp: [43] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:cebe775c-675b-4202-b042-dbeac82d6935 (non-flash)
Nov 29 03:24:47 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsi_session /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Nov 29 03:24:47 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:023a5970-c0e3-48f5-a8ef-d9d90fed8b37 -p 172.16.0.220:3260 --login {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:24:47 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:023a5970-c0e3-48f5-a8ef-d9d90fed8b37 -p 172.16.0.220:3260 --login" returned: 0 in 0.062s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:24:47 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[45b34eee-43c6-4071-90a0-f9f8b71feee5]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:023a5970-c0e3-48f5-a8ef-d9d90fed8b37, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:023a5970-c0e3-48f5-a8ef-d9d90fed8b37, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:24:47 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-75769adb-154d-4655-b244-6c6ef2eaf6e8 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:023a5970-c0e3-48f5-a8ef-d9d90fed8b37, portal: 172.16.0.220,3260]
Nov 29 03:24:47 user nova-compute[75234]: Login to [iface: default, target: iqn.2010-10.org.openstack:023a5970-c0e3-48f5-a8ef-d9d90fed8b37, portal: 172.16.0.220,3260] successful.
Nov 29 03:24:47 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:24:47 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:023a5970-c0e3-48f5-a8ef-d9d90fed8b37 -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:24:47 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:023a5970-c0e3-48f5-a8ef-d9d90fed8b37 -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.019s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:24:47 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[5da36758-4b04-476f-b040-72a9aea4fad8]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:24:47 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-75769adb-154d-4655-b244-6c6ef2eaf6e8 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:24:47 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:24:47 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.022s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:24:47 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[0abbe511-57ff-4764-b630-0d7cbf1d56b6]: (4, ('tcp: [39] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:41f08854-31dd-4172-a5cf-01b0e695c10f (non-flash)\ntcp: [42] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0 (non-flash)\ntcp: [43] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:cebe775c-675b-4202-b042-dbeac82d6935 (non-flash)\ntcp: [44] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:023a5970-c0e3-48f5-a8ef-d9d90fed8b37 (non-flash)\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:24:47 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-75769adb-154d-4655-b244-6c6ef2eaf6e8 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [39] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:41f08854-31dd-4172-a5cf-01b0e695c10f (non-flash)
Nov 29 03:24:47 user nova-compute[75234]: tcp: [42] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0 (non-flash)
Nov 29 03:24:47 user nova-compute[75234]: tcp: [43] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:cebe775c-675b-4202-b042-dbeac82d6935 (non-flash)
Nov 29 03:24:47 user nova-compute[75234]: tcp: [44] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:023a5970-c0e3-48f5-a8ef-d9d90fed8b37 (non-flash)
Nov 29 03:24:47 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:24:47 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-75769adb-154d-4655-b244-6c6ef2eaf6e8 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] iscsi session list stdout=tcp: [39] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:41f08854-31dd-4172-a5cf-01b0e695c10f (non-flash)
Nov 29 03:24:47 user nova-compute[75234]: tcp: [42] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0 (non-flash)
Nov 29 03:24:47 user nova-compute[75234]: tcp: [43] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:cebe775c-675b-4202-b042-dbeac82d6935 (non-flash)
Nov 29 03:24:47 user nova-compute[75234]: tcp: [44] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:023a5970-c0e3-48f5-a8ef-d9d90fed8b37 (non-flash)
Nov 29 03:24:47 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsi_session /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Nov 29 03:24:47 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-75769adb-154d-4655-b244-6c6ef2eaf6e8 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:023a5970-c0e3-48f5-a8ef-d9d90fed8b37" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.271s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:87}}
Nov 29 03:24:47 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-75769adb-154d-4655-b244-6c6ef2eaf6e8 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Connected to 172.16.0.220:3260 {{(pid=75234) _connect_vol /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:644}}
Nov 29 03:24:47 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-75769adb-154d-4655-b244-6c6ef2eaf6e8 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] HCTL ('36', '-', '-', 0) found on session 44 with lun 0 {{(pid=75234) get_hctl /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:770}}
Nov 29 03:24:47 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-75769adb-154d-4655-b244-6c6ef2eaf6e8 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Scanning host 36 c: -, t: -, l: 0) {{(pid=75234) scan_iscsi /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:795}}
Nov 29 03:24:47 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host36/scan {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:24:47 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host36/scan" returned: 0 in 0.017s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:24:47 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[27137fb3-c488-4806-bacd-6692fb057f7d]: (4, ('- - 0', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:24:47 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-75769adb-154d-4655-b244-6c6ef2eaf6e8 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Searching for a device in session 44 and hctl ['36', '*', '*', 0] yield: None {{(pid=75234) device_name_by_hctl /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:789}}
Nov 29 03:24:47 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:24:48 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-75769adb-154d-4655-b244-6c6ef2eaf6e8 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Searching for a device in session 44 and hctl ['36', '*', '*', 0] yield: sde {{(pid=75234) device_name_by_hctl /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:789}}
Nov 29 03:24:48 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-75769adb-154d-4655-b244-6c6ef2eaf6e8 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Connected to sde using {'target_iqn': 'iqn.2010-10.org.openstack:023a5970-c0e3-48f5-a8ef-d9d90fed8b37', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'hjrHYOxB', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False} {{(pid=75234) _connect_vol /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:672}}
Nov 29 03:24:48 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-75769adb-154d-4655-b244-6c6ef2eaf6e8 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.302s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:87}}
Nov 29 03:24:48 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-75769adb-154d-4655-b244-6c6ef2eaf6e8 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] <== connect_volume: return (1304ms) {'type': 'block', 'scsi_wwn': '26536353262373232', 'path': '/dev/sde'} {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:203}}
Nov 29 03:24:48 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-75769adb-154d-4655-b244-6c6ef2eaf6e8 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] [instance: f6bc212d-85cd-479c-92ae-112c9f0101bc] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '26536353262373232', 'path': '/dev/sde'} {{(pid=75234) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Nov 29 03:24:48 user nova-compute[75234]: DEBUG nova.objects.instance [None req-75769adb-154d-4655-b244-6c6ef2eaf6e8 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Lazy-loading 'flavor' on Instance uuid f6bc212d-85cd-479c-92ae-112c9f0101bc {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:24:48 user nova-compute[75234]: DEBUG nova.virt.libvirt.guest [None req-75769adb-154d-4655-b244-6c6ef2eaf6e8 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] attach device xml: <disk type="block" device="disk">
Nov 29 03:24:48 user nova-compute[75234]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Nov 29 03:24:48 user nova-compute[75234]:   <source dev="/dev/sde"/>
Nov 29 03:24:48 user nova-compute[75234]:   <target dev="vdb" bus="virtio"/>
Nov 29 03:24:48 user nova-compute[75234]:   <serial>023a5970-c0e3-48f5-a8ef-d9d90fed8b37</serial>
Nov 29 03:24:48 user nova-compute[75234]: </disk>
Nov 29 03:24:48 user nova-compute[75234]:  {{(pid=75234) attach_device /opt/stack/nova/nova/virt/libvirt/guest.py:339}}
Nov 29 03:24:48 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-75769adb-154d-4655-b244-6c6ef2eaf6e8 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] No BDM found with device name vda, not building metadata. {{(pid=75234) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12089}}
Nov 29 03:24:48 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-75769adb-154d-4655-b244-6c6ef2eaf6e8 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] No BDM found with device name vdb, not building metadata. {{(pid=75234) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12089}}
Nov 29 03:24:48 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-75769adb-154d-4655-b244-6c6ef2eaf6e8 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] No VIF found with MAC fa:16:3e:4e:8e:1d, not building metadata {{(pid=75234) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Nov 29 03:24:48 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-75769adb-154d-4655-b244-6c6ef2eaf6e8 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Lock "f6bc212d-85cd-479c-92ae-112c9f0101bc" "released" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: held 9.570s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:24:50 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-bb39c935-0ccf-48b3-85a9-574b89670de2 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Acquiring lock "f6bc212d-85cd-479c-92ae-112c9f0101bc" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:24:50 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-bb39c935-0ccf-48b3-85a9-574b89670de2 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Lock "f6bc212d-85cd-479c-92ae-112c9f0101bc" acquired by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:24:50 user nova-compute[75234]: INFO nova.compute.manager [None req-bb39c935-0ccf-48b3-85a9-574b89670de2 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] [instance: f6bc212d-85cd-479c-92ae-112c9f0101bc] Detaching volume 023a5970-c0e3-48f5-a8ef-d9d90fed8b37
Nov 29 03:24:50 user nova-compute[75234]: INFO nova.virt.block_device [None req-bb39c935-0ccf-48b3-85a9-574b89670de2 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] [instance: f6bc212d-85cd-479c-92ae-112c9f0101bc] Attempting to driver detach volume 023a5970-c0e3-48f5-a8ef-d9d90fed8b37 from mountpoint /dev/vdb
Nov 29 03:24:50 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-bb39c935-0ccf-48b3-85a9-574b89670de2 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Attempting to detach device vdb from instance f6bc212d-85cd-479c-92ae-112c9f0101bc from the persistent domain config. {{(pid=75234) _detach_from_persistent /opt/stack/nova/nova/virt/libvirt/driver.py:2487}}
Nov 29 03:24:50 user nova-compute[75234]: DEBUG nova.virt.libvirt.guest [None req-bb39c935-0ccf-48b3-85a9-574b89670de2 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] detach device xml: <disk type="block" device="disk">
Nov 29 03:24:50 user nova-compute[75234]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Nov 29 03:24:50 user nova-compute[75234]:   <source dev="/dev/sde"/>
Nov 29 03:24:50 user nova-compute[75234]:   <target dev="vdb" bus="virtio"/>
Nov 29 03:24:50 user nova-compute[75234]:   <serial>023a5970-c0e3-48f5-a8ef-d9d90fed8b37</serial>
Nov 29 03:24:50 user nova-compute[75234]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x07" function="0x0"/>
Nov 29 03:24:50 user nova-compute[75234]: </disk>
Nov 29 03:24:50 user nova-compute[75234]:  {{(pid=75234) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Nov 29 03:24:50 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-bb39c935-0ccf-48b3-85a9-574b89670de2 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Successfully detached device vdb from instance f6bc212d-85cd-479c-92ae-112c9f0101bc from the persistent domain config.
Nov 29 03:24:50 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-bb39c935-0ccf-48b3-85a9-574b89670de2 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] (1/8): Attempting to detach device vdb with device alias virtio-disk1 from instance f6bc212d-85cd-479c-92ae-112c9f0101bc from the live domain config. {{(pid=75234) _detach_from_live_with_retry /opt/stack/nova/nova/virt/libvirt/driver.py:2523}}
Nov 29 03:24:50 user nova-compute[75234]: DEBUG nova.virt.libvirt.guest [None req-bb39c935-0ccf-48b3-85a9-574b89670de2 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] detach device xml: <disk type="block" device="disk">
Nov 29 03:24:50 user nova-compute[75234]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Nov 29 03:24:50 user nova-compute[75234]:   <source dev="/dev/sde"/>
Nov 29 03:24:50 user nova-compute[75234]:   <target dev="vdb" bus="virtio"/>
Nov 29 03:24:50 user nova-compute[75234]:   <serial>023a5970-c0e3-48f5-a8ef-d9d90fed8b37</serial>
Nov 29 03:24:50 user nova-compute[75234]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x07" function="0x0"/>
Nov 29 03:24:50 user nova-compute[75234]: </disk>
Nov 29 03:24:50 user nova-compute[75234]:  {{(pid=75234) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Nov 29 03:24:51 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-bb39c935-0ccf-48b3-85a9-574b89670de2 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Start waiting for the detach event from libvirt for device vdb with device alias virtio-disk1 for instance f6bc212d-85cd-479c-92ae-112c9f0101bc {{(pid=75234) _detach_from_live_and_wait_for_event /opt/stack/nova/nova/virt/libvirt/driver.py:2599}}
Nov 29 03:24:52 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:24:52 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:24:57 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:24:57 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:24:59 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-8ae021bf-1b04-4a75-a501-bbbc3050b131 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Waiting for libvirt event about the detach of device vdb with device alias virtio-disk1 from instance 060c15ec-9012-471a-984a-14c9fa6ce23a is timed out.
Nov 29 03:24:59 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-8ae021bf-1b04-4a75-a501-bbbc3050b131 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Successfully detached device vdb from instance 060c15ec-9012-471a-984a-14c9fa6ce23a from the live domain config.
Nov 29 03:24:59 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-8ae021bf-1b04-4a75-a501-bbbc3050b131 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] [instance: 060c15ec-9012-471a-984a-14c9fa6ce23a] calling os-brick to detach iSCSI Volume {{(pid=75234) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Nov 29 03:24:59 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-8ae021bf-1b04-4a75-a501-bbbc3050b131 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f8a5050c9a0>, {'target_iqn': 'iqn.2010-10.org.openstack:cebe775c-675b-4202-b042-dbeac82d6935', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'UGxIRjPu', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdb'}, None), 'kwargs': False}}" {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:176}}
Nov 29 03:24:59 user nova-compute[75234]: WARNING os_brick.initiator.connectors.base [None req-8ae021bf-1b04-4a75-a501-bbbc3050b131 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Nov 29 03:24:59 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-8ae021bf-1b04-4a75-a501-bbbc3050b131 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:68}}
Nov 29 03:24:59 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-8ae021bf-1b04-4a75-a501-bbbc3050b131 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:73}}
Nov 29 03:24:59 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:24:59 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.021s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:24:59 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[86805544-5bf2-4a7e-889d-ac10020342ba]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:cebe775c-675b-4202-b042-dbeac82d6935\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:023a5970-c0e3-48f5-a8ef-d9d90fed8b37\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:41f08854-31dd-4172-a5cf-01b0e695c10f\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:24:59 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-8ae021bf-1b04-4a75-a501-bbbc3050b131 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS:
Nov 29 03:24:59 user nova-compute[75234]: No targets found.
Nov 29 03:24:59 user nova-compute[75234]: iSNS:
Nov 29 03:24:59 user nova-compute[75234]: No targets found.
Nov 29 03:24:59 user nova-compute[75234]: STATIC:
Nov 29 03:24:59 user nova-compute[75234]: Target: iqn.2010-10.org.openstack:cebe775c-675b-4202-b042-dbeac82d6935
Nov 29 03:24:59 user nova-compute[75234]:         Portal: 172.16.0.220:3260,-1
Nov 29 03:24:59 user nova-compute[75234]:                 Iface Name: default
Nov 29 03:24:59 user nova-compute[75234]: Target: iqn.2010-10.org.openstack:023a5970-c0e3-48f5-a8ef-d9d90fed8b37
Nov 29 03:24:59 user nova-compute[75234]:         Portal: 172.16.0.220:3260,-1
Nov 29 03:24:59 user nova-compute[75234]:                 Iface Name: default
Nov 29 03:24:59 user nova-compute[75234]: Target: iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0
Nov 29 03:24:59 user nova-compute[75234]:         Portal: 172.16.0.220:3260,-1
Nov 29 03:24:59 user nova-compute[75234]:                 Iface Name: default
Nov 29 03:24:59 user nova-compute[75234]: Target: iqn.2010-10.org.openstack:41f08854-31dd-4172-a5cf-01b0e695c10f
Nov 29 03:24:59 user nova-compute[75234]:         Portal: 172.16.0.220:3260,-1
Nov 29 03:24:59 user nova-compute[75234]:                 Iface Name: default
Nov 29 03:24:59 user nova-compute[75234]: FIRMWARE:
Nov 29 03:24:59 user nova-compute[75234]: No targets found.
Nov 29 03:24:59 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:24:59 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-8ae021bf-1b04-4a75-a501-bbbc3050b131 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Regex to get portals from discoverydb: ^SENDTARGETS:
Nov 29 03:24:59 user nova-compute[75234]: .*?^DiscoveryAddress: 172.16.0.220,3260.*?
Nov 29 03:24:59 user nova-compute[75234]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=75234) _get_discoverydb_portals /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:382}}
Nov 29 03:24:59 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-8ae021bf-1b04-4a75-a501-bbbc3050b131 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:cebe775c-675b-4202-b042-dbeac82d6935', 0)] {{(pid=75234) _get_connection_devices /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:830}}
Nov 29 03:24:59 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:24:59 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.022s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:24:59 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[d84a14c8-e382-4dd9-987d-ea664fcb157a]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:023a5970-c0e3-48f5-a8ef-d9d90fed8b37\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:41f08854-31dd-4172-a5cf-01b0e695c10f\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:cebe775c-675b-4202-b042-dbeac82d6935\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:24:59 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:24:59 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.025s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:24:59 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[237d75ac-f647-460e-bdbc-24ef38c5745a]: (4, ('tcp: [39] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:41f08854-31dd-4172-a5cf-01b0e695c10f (non-flash)\ntcp: [42] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0 (non-flash)\ntcp: [43] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:cebe775c-675b-4202-b042-dbeac82d6935 (non-flash)\ntcp: [44] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:023a5970-c0e3-48f5-a8ef-d9d90fed8b37 (non-flash)\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:24:59 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-8ae021bf-1b04-4a75-a501-bbbc3050b131 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [39] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:41f08854-31dd-4172-a5cf-01b0e695c10f (non-flash)
Nov 29 03:24:59 user nova-compute[75234]: tcp: [42] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0 (non-flash)
Nov 29 03:24:59 user nova-compute[75234]: tcp: [43] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:cebe775c-675b-4202-b042-dbeac82d6935 (non-flash)
Nov 29 03:24:59 user nova-compute[75234]: tcp: [44] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:023a5970-c0e3-48f5-a8ef-d9d90fed8b37 (non-flash)
Nov 29 03:24:59 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:24:59 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-8ae021bf-1b04-4a75-a501-bbbc3050b131 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] iscsi session list stdout=tcp: [39] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:41f08854-31dd-4172-a5cf-01b0e695c10f (non-flash)
Nov 29 03:24:59 user nova-compute[75234]: tcp: [42] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0 (non-flash)
Nov 29 03:24:59 user nova-compute[75234]: tcp: [43] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:cebe775c-675b-4202-b042-dbeac82d6935 (non-flash)
Nov 29 03:24:59 user nova-compute[75234]: tcp: [44] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:023a5970-c0e3-48f5-a8ef-d9d90fed8b37 (non-flash)
Nov 29 03:24:59 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsi_session /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Nov 29 03:24:59 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-8ae021bf-1b04-4a75-a501-bbbc3050b131 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7f8a501609d0>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:cebe775c-675b-4202-b042-dbeac82d6935'): ({'sdb'}, set())}) {{(pid=75234) _get_connection_devices /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:863}}
Nov 29 03:24:59 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-8ae021bf-1b04-4a75-a501-bbbc3050b131 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Removing single pathed devices sdb {{(pid=75234) remove_connection /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:374}}
Nov 29 03:24:59 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:24:59 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.022s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:24:59 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[77668de1-2004-4419-861c-79ae2d669f1e]: (4, ('path checker states:\nup                  4\n\npaths: 0\nbusy: False\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:24:59 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdb {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:24:59 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdb" returned: 0 in 0.019s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:24:59 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[32ec5b57-caa6-42b7-88d2-5d3309edfd07]: (4, ('ok\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:24:59 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-8ae021bf-1b04-4a75-a501-bbbc3050b131 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Flushing IO for device /dev/sdb {{(pid=75234) flush_device_io /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:434}}
Nov 29 03:24:59 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdb {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:24:59 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdb" returned: 0 in 0.013s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:24:59 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[6a257b77-6208-4cd5-8d6e-bd1ca80702ae]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:24:59 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-8ae021bf-1b04-4a75-a501-bbbc3050b131 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Remove SCSI device /dev/sdb with /sys/block/sdb/device/delete {{(pid=75234) remove_scsi_device /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:148}}
Nov 29 03:24:59 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdb/device/delete {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:24:59 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdb/device/delete" returned: 0 in 0.046s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:24:59 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[fbf1e841-4fda-4972-bd14-25efa099a19b]: (4, ('1', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:24:59 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-8ae021bf-1b04-4a75-a501-bbbc3050b131 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Checking to see if SCSI volumes sdb have been removed. {{(pid=75234) wait_for_volumes_removal /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:156}}
Nov 29 03:24:59 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-8ae021bf-1b04-4a75-a501-bbbc3050b131 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] SCSI volumes sdb have been removed. {{(pid=75234) wait_for_volumes_removal /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:166}}
Nov 29 03:24:59 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-8ae021bf-1b04-4a75-a501-bbbc3050b131 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:cebe775c-675b-4202-b042-dbeac82d6935')] {{(pid=75234) _disconnect_connection /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Nov 29 03:24:59 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:cebe775c-675b-4202-b042-dbeac82d6935 -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:24:59 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:cebe775c-675b-4202-b042-dbeac82d6935 -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.021s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:24:59 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[25333955-915f-4b69-87f0-8b6d41addb9f]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:24:59 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-8ae021bf-1b04-4a75-a501-bbbc3050b131 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:24:59 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:cebe775c-675b-4202-b042-dbeac82d6935 -p 172.16.0.220:3260 --logout {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:24:59 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:cebe775c-675b-4202-b042-dbeac82d6935 -p 172.16.0.220:3260 --logout" returned: 0 in 0.048s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:24:59 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[41af3be5-05f4-4ff5-8896-67ff9534027c]: (4, ('Logging out of session [sid: 43, target: iqn.2010-10.org.openstack:cebe775c-675b-4202-b042-dbeac82d6935, portal: 172.16.0.220,3260]\nLogout of [sid: 43, target: iqn.2010-10.org.openstack:cebe775c-675b-4202-b042-dbeac82d6935, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:24:59 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-8ae021bf-1b04-4a75-a501-bbbc3050b131 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 43, target: iqn.2010-10.org.openstack:cebe775c-675b-4202-b042-dbeac82d6935, portal: 172.16.0.220,3260]
Nov 29 03:24:59 user nova-compute[75234]: Logout of [sid: 43, target: iqn.2010-10.org.openstack:cebe775c-675b-4202-b042-dbeac82d6935, portal: 172.16.0.220,3260] successful.
Nov 29 03:24:59 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:24:59 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:cebe775c-675b-4202-b042-dbeac82d6935 -p 172.16.0.220:3260 --op delete {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:24:59 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:cebe775c-675b-4202-b042-dbeac82d6935 -p 172.16.0.220:3260 --op delete" returned: 0 in 0.025s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:24:59 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[32459e2d-5dd4-42bd-b54f-e2a1bb6be9ae]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:24:59 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-8ae021bf-1b04-4a75-a501-bbbc3050b131 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:24:59 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-8ae021bf-1b04-4a75-a501-bbbc3050b131 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.319s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:87}}
Nov 29 03:24:59 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-8ae021bf-1b04-4a75-a501-bbbc3050b131 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] <== disconnect_volume: return (321ms) None {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:203}}
Nov 29 03:24:59 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-8ae021bf-1b04-4a75-a501-bbbc3050b131 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] [instance: 060c15ec-9012-471a-984a-14c9fa6ce23a] Disconnected iSCSI Volume {{(pid=75234) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:79}}
Nov 29 03:25:00 user nova-compute[75234]: DEBUG nova.objects.instance [None req-8ae021bf-1b04-4a75-a501-bbbc3050b131 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Lazy-loading 'flavor' on Instance uuid 060c15ec-9012-471a-984a-14c9fa6ce23a {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:25:00 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-8ae021bf-1b04-4a75-a501-bbbc3050b131 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Lock "060c15ec-9012-471a-984a-14c9fa6ce23a" "released" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: held 21.701s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:25:02 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:25:02 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:25:03 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:25:04 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:25:04 user nova-compute[75234]: DEBUG nova.compute.manager [req-5bae42e0-1c70-4654-966b-2b660eff0790 req-a8147cc8-36ba-4d78-82f4-fdced720747e service nova] [instance: 1df140f1-fbd9-4b5d-b356-d08b7e908f64] Received event network-changed-ea9a3586-da52-49b7-8ced-5d27806885ad {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:25:04 user nova-compute[75234]: DEBUG nova.compute.manager [req-5bae42e0-1c70-4654-966b-2b660eff0790 req-a8147cc8-36ba-4d78-82f4-fdced720747e service nova] [instance: 1df140f1-fbd9-4b5d-b356-d08b7e908f64] Refreshing instance network info cache due to event network-changed-ea9a3586-da52-49b7-8ced-5d27806885ad. {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11117}}
Nov 29 03:25:04 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-5bae42e0-1c70-4654-966b-2b660eff0790 req-a8147cc8-36ba-4d78-82f4-fdced720747e service nova] Acquiring lock "refresh_cache-1df140f1-fbd9-4b5d-b356-d08b7e908f64" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:25:04 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-5bae42e0-1c70-4654-966b-2b660eff0790 req-a8147cc8-36ba-4d78-82f4-fdced720747e service nova] Acquired lock "refresh_cache-1df140f1-fbd9-4b5d-b356-d08b7e908f64" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:25:04 user nova-compute[75234]: DEBUG nova.network.neutron [req-5bae42e0-1c70-4654-966b-2b660eff0790 req-a8147cc8-36ba-4d78-82f4-fdced720747e service nova] [instance: 1df140f1-fbd9-4b5d-b356-d08b7e908f64] Refreshing network info cache for port ea9a3586-da52-49b7-8ced-5d27806885ad {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2006}}
Nov 29 03:25:05 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-50a7db2f-4e8c-42d5-aa8d-9b2f13457b00 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Acquiring lock "1df140f1-fbd9-4b5d-b356-d08b7e908f64" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:25:05 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-50a7db2f-4e8c-42d5-aa8d-9b2f13457b00 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Lock "1df140f1-fbd9-4b5d-b356-d08b7e908f64" acquired by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:25:05 user nova-compute[75234]: INFO nova.compute.manager [None req-50a7db2f-4e8c-42d5-aa8d-9b2f13457b00 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] [instance: 1df140f1-fbd9-4b5d-b356-d08b7e908f64] Detaching volume 41f08854-31dd-4172-a5cf-01b0e695c10f
Nov 29 03:25:05 user nova-compute[75234]: DEBUG nova.network.neutron [req-5bae42e0-1c70-4654-966b-2b660eff0790 req-a8147cc8-36ba-4d78-82f4-fdced720747e service nova] [instance: 1df140f1-fbd9-4b5d-b356-d08b7e908f64] Updated VIF entry in instance network info cache for port ea9a3586-da52-49b7-8ced-5d27806885ad. {{(pid=75234) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3481}}
Nov 29 03:25:05 user nova-compute[75234]: DEBUG nova.network.neutron [req-5bae42e0-1c70-4654-966b-2b660eff0790 req-a8147cc8-36ba-4d78-82f4-fdced720747e service nova] [instance: 1df140f1-fbd9-4b5d-b356-d08b7e908f64] Updating instance_info_cache with network_info: [{"id": "ea9a3586-da52-49b7-8ced-5d27806885ad", "address": "fa:16:3e:80:25:b8", "network": {"id": "2b3c3140-f3b6-45da-8202-bb4d7f0dd130", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-2043420778-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.158", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "f8a7884867c944e1ba1024270e4902f3", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapea9a3586-da", "ovs_interfaceid": "ea9a3586-da52-49b7-8ced-5d27806885ad", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:25:05 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-5bae42e0-1c70-4654-966b-2b660eff0790 req-a8147cc8-36ba-4d78-82f4-fdced720747e service nova] Releasing lock "refresh_cache-1df140f1-fbd9-4b5d-b356-d08b7e908f64" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:25:05 user nova-compute[75234]: INFO nova.virt.block_device [None req-50a7db2f-4e8c-42d5-aa8d-9b2f13457b00 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] [instance: 1df140f1-fbd9-4b5d-b356-d08b7e908f64] Attempting to driver detach volume 41f08854-31dd-4172-a5cf-01b0e695c10f from mountpoint /dev/vdb
Nov 29 03:25:05 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-50a7db2f-4e8c-42d5-aa8d-9b2f13457b00 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Attempting to detach device vdb from instance 1df140f1-fbd9-4b5d-b356-d08b7e908f64 from the persistent domain config. {{(pid=75234) _detach_from_persistent /opt/stack/nova/nova/virt/libvirt/driver.py:2487}}
Nov 29 03:25:05 user nova-compute[75234]: DEBUG nova.virt.libvirt.guest [None req-50a7db2f-4e8c-42d5-aa8d-9b2f13457b00 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] detach device xml: <disk type="block" device="disk">
Nov 29 03:25:05 user nova-compute[75234]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Nov 29 03:25:05 user nova-compute[75234]:   <source dev="/dev/sdd"/>
Nov 29 03:25:05 user nova-compute[75234]:   <target dev="vdb" bus="virtio"/>
Nov 29 03:25:05 user nova-compute[75234]:   <serial>41f08854-31dd-4172-a5cf-01b0e695c10f</serial>
Nov 29 03:25:05 user nova-compute[75234]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x07" function="0x0"/>
Nov 29 03:25:05 user nova-compute[75234]: </disk>
Nov 29 03:25:05 user nova-compute[75234]:  {{(pid=75234) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Nov 29 03:25:05 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-50a7db2f-4e8c-42d5-aa8d-9b2f13457b00 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Successfully detached device vdb from instance 1df140f1-fbd9-4b5d-b356-d08b7e908f64 from the persistent domain config.
Nov 29 03:25:05 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-50a7db2f-4e8c-42d5-aa8d-9b2f13457b00 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] (1/8): Attempting to detach device vdb with device alias virtio-disk1 from instance 1df140f1-fbd9-4b5d-b356-d08b7e908f64 from the live domain config. {{(pid=75234) _detach_from_live_with_retry /opt/stack/nova/nova/virt/libvirt/driver.py:2523}}
Nov 29 03:25:05 user nova-compute[75234]: DEBUG nova.virt.libvirt.guest [None req-50a7db2f-4e8c-42d5-aa8d-9b2f13457b00 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] detach device xml: <disk type="block" device="disk">
Nov 29 03:25:05 user nova-compute[75234]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Nov 29 03:25:05 user nova-compute[75234]:   <source dev="/dev/sdd"/>
Nov 29 03:25:05 user nova-compute[75234]:   <target dev="vdb" bus="virtio"/>
Nov 29 03:25:05 user nova-compute[75234]:   <serial>41f08854-31dd-4172-a5cf-01b0e695c10f</serial>
Nov 29 03:25:05 user nova-compute[75234]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x07" function="0x0"/>
Nov 29 03:25:05 user nova-compute[75234]: </disk>
Nov 29 03:25:05 user nova-compute[75234]:  {{(pid=75234) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Nov 29 03:25:05 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-50a7db2f-4e8c-42d5-aa8d-9b2f13457b00 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Start waiting for the detach event from libvirt for device vdb with device alias virtio-disk1 for instance 1df140f1-fbd9-4b5d-b356-d08b7e908f64 {{(pid=75234) _detach_from_live_and_wait_for_event /opt/stack/nova/nova/virt/libvirt/driver.py:2599}}
Nov 29 03:25:06 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:25:06 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=75234) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10541}}
Nov 29 03:25:06 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager.update_available_resource {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:25:06 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:25:06 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:25:06 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:25:06 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Auditing locally available compute resources for user (node: user) {{(pid=75234) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:907}}
Nov 29 03:25:06 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/13e5da64-9adc-4479-81c7-d329e9b5f08f/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:25:06 user nova-compute[75234]: DEBUG nova.compute.manager [req-1b77935c-0329-4467-a303-6a5adbd7dbe1 req-9da27166-6952-4cd0-bec3-69c3d6773b8f service nova] [instance: 1df140f1-fbd9-4b5d-b356-d08b7e908f64] Received event network-changed-ea9a3586-da52-49b7-8ced-5d27806885ad {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:25:06 user nova-compute[75234]: DEBUG nova.compute.manager [req-1b77935c-0329-4467-a303-6a5adbd7dbe1 req-9da27166-6952-4cd0-bec3-69c3d6773b8f service nova] [instance: 1df140f1-fbd9-4b5d-b356-d08b7e908f64] Refreshing instance network info cache due to event network-changed-ea9a3586-da52-49b7-8ced-5d27806885ad. {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11117}}
Nov 29 03:25:06 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-1b77935c-0329-4467-a303-6a5adbd7dbe1 req-9da27166-6952-4cd0-bec3-69c3d6773b8f service nova] Acquiring lock "refresh_cache-1df140f1-fbd9-4b5d-b356-d08b7e908f64" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:25:06 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-1b77935c-0329-4467-a303-6a5adbd7dbe1 req-9da27166-6952-4cd0-bec3-69c3d6773b8f service nova] Acquired lock "refresh_cache-1df140f1-fbd9-4b5d-b356-d08b7e908f64" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:25:06 user nova-compute[75234]: DEBUG nova.network.neutron [req-1b77935c-0329-4467-a303-6a5adbd7dbe1 req-9da27166-6952-4cd0-bec3-69c3d6773b8f service nova] [instance: 1df140f1-fbd9-4b5d-b356-d08b7e908f64] Refreshing network info cache for port ea9a3586-da52-49b7-8ced-5d27806885ad {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2006}}
Nov 29 03:25:06 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/13e5da64-9adc-4479-81c7-d329e9b5f08f/disk --force-share --output=json" returned: 0 in 0.171s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:25:06 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/13e5da64-9adc-4479-81c7-d329e9b5f08f/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:25:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/13e5da64-9adc-4479-81c7-d329e9b5f08f/disk --force-share --output=json" returned: 0 in 0.179s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:25:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/5d7d744e-1a5f-40ab-b961-4b58c183efe0/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:25:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/5d7d744e-1a5f-40ab-b961-4b58c183efe0/disk --force-share --output=json" returned: 0 in 0.173s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:25:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/5d7d744e-1a5f-40ab-b961-4b58c183efe0/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:25:07 user nova-compute[75234]: DEBUG nova.network.neutron [req-1b77935c-0329-4467-a303-6a5adbd7dbe1 req-9da27166-6952-4cd0-bec3-69c3d6773b8f service nova] [instance: 1df140f1-fbd9-4b5d-b356-d08b7e908f64] Updated VIF entry in instance network info cache for port ea9a3586-da52-49b7-8ced-5d27806885ad. {{(pid=75234) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3481}}
Nov 29 03:25:07 user nova-compute[75234]: DEBUG nova.network.neutron [req-1b77935c-0329-4467-a303-6a5adbd7dbe1 req-9da27166-6952-4cd0-bec3-69c3d6773b8f service nova] [instance: 1df140f1-fbd9-4b5d-b356-d08b7e908f64] Updating instance_info_cache with network_info: [{"id": "ea9a3586-da52-49b7-8ced-5d27806885ad", "address": "fa:16:3e:80:25:b8", "network": {"id": "2b3c3140-f3b6-45da-8202-bb4d7f0dd130", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-2043420778-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.158", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "f8a7884867c944e1ba1024270e4902f3", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapea9a3586-da", "ovs_interfaceid": "ea9a3586-da52-49b7-8ced-5d27806885ad", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:25:07 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-1b77935c-0329-4467-a303-6a5adbd7dbe1 req-9da27166-6952-4cd0-bec3-69c3d6773b8f service nova] Releasing lock "refresh_cache-1df140f1-fbd9-4b5d-b356-d08b7e908f64" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:25:07 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:25:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/5d7d744e-1a5f-40ab-b961-4b58c183efe0/disk --force-share --output=json" returned: 0 in 0.169s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:25:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/1df140f1-fbd9-4b5d-b356-d08b7e908f64/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:25:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/1df140f1-fbd9-4b5d-b356-d08b7e908f64/disk --force-share --output=json" returned: 0 in 0.181s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:25:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/1df140f1-fbd9-4b5d-b356-d08b7e908f64/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:25:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/1df140f1-fbd9-4b5d-b356-d08b7e908f64/disk --force-share --output=json" returned: 0 in 0.183s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:25:07 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] skipping disk /dev/sdc (vda) as it is a volume {{(pid=75234) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11213}}
Nov 29 03:25:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f6bc212d-85cd-479c-92ae-112c9f0101bc/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:25:07 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:25:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f6bc212d-85cd-479c-92ae-112c9f0101bc/disk --force-share --output=json" returned: 0 in 0.182s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:25:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f6bc212d-85cd-479c-92ae-112c9f0101bc/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:25:08 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f6bc212d-85cd-479c-92ae-112c9f0101bc/disk --force-share --output=json" returned: 0 in 0.173s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:25:08 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/65d48c93-b6da-4f51-9d5a-0676dda62a60/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:25:08 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/65d48c93-b6da-4f51-9d5a-0676dda62a60/disk --force-share --output=json" returned: 0 in 0.206s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:25:08 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/65d48c93-b6da-4f51-9d5a-0676dda62a60/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:25:08 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/65d48c93-b6da-4f51-9d5a-0676dda62a60/disk --force-share --output=json" returned: 0 in 0.172s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:25:08 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/060c15ec-9012-471a-984a-14c9fa6ce23a/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:25:08 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/060c15ec-9012-471a-984a-14c9fa6ce23a/disk --force-share --output=json" returned: 0 in 0.169s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:25:08 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/060c15ec-9012-471a-984a-14c9fa6ce23a/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:25:08 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/060c15ec-9012-471a-984a-14c9fa6ce23a/disk --force-share --output=json" returned: 0 in 0.176s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:25:08 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a082c96e-e384-40b6-bcc8-fc5e0e104669/disk.rescue --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:25:09 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a082c96e-e384-40b6-bcc8-fc5e0e104669/disk.rescue --force-share --output=json" returned: 0 in 0.180s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:25:09 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a082c96e-e384-40b6-bcc8-fc5e0e104669/disk.rescue --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:25:09 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a082c96e-e384-40b6-bcc8-fc5e0e104669/disk.rescue --force-share --output=json" returned: 0 in 0.170s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:25:09 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a082c96e-e384-40b6-bcc8-fc5e0e104669/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:25:09 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a082c96e-e384-40b6-bcc8-fc5e0e104669/disk --force-share --output=json" returned: 0 in 0.197s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:25:09 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a082c96e-e384-40b6-bcc8-fc5e0e104669/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:25:09 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a082c96e-e384-40b6-bcc8-fc5e0e104669/disk --force-share --output=json" returned: 0 in 0.261s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:25:09 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/21e9c6cd-c39f-41c0-8768-5dd64589c3f3/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:25:09 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/21e9c6cd-c39f-41c0-8768-5dd64589c3f3/disk --force-share --output=json" returned: 0 in 0.166s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:25:09 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/21e9c6cd-c39f-41c0-8768-5dd64589c3f3/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:25:10 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/21e9c6cd-c39f-41c0-8768-5dd64589c3f3/disk --force-share --output=json" returned: 0 in 0.177s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:25:10 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:25:10 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:25:10 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Hypervisor/Node resource view: name=user free_ram=7917MB free_disk=25.079071044921875GB free_vcpus=3 pci_devices=[{"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}] {{(pid=75234) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1106}}
Nov 29 03:25:10 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:25:10 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:25:10 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance 21e9c6cd-c39f-41c0-8768-5dd64589c3f3 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:25:10 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance 13e5da64-9adc-4479-81c7-d329e9b5f08f actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:25:10 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance a082c96e-e384-40b6-bcc8-fc5e0e104669 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:25:10 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance 1df140f1-fbd9-4b5d-b356-d08b7e908f64 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:25:10 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance 65d48c93-b6da-4f51-9d5a-0676dda62a60 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:25:10 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance f6bc212d-85cd-479c-92ae-112c9f0101bc actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:25:10 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance 060c15ec-9012-471a-984a-14c9fa6ce23a actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:25:10 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance 5d7d744e-1a5f-40ab-b961-4b58c183efe0 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:25:10 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance d5096ca2-d005-45ac-a487-11a810446859 actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:25:10 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Total usable vcpus: 12, total allocated vcpus: 9 {{(pid=75234) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1129}}
Nov 29 03:25:10 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Final resource view: name=user phys_ram=16011MB used_ram=5120MB phys_disk=40GB used_disk=8GB total_vcpus=12 used_vcpus=9 pci_stats=[] {{(pid=75234) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1138}}
Nov 29 03:25:10 user nova-compute[75234]: DEBUG nova.compute.provider_tree [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Inventory has not changed in ProviderTree for provider: 4d291220-7c37-4839-a0e3-b23c66502b74 {{(pid=75234) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Nov 29 03:25:10 user nova-compute[75234]: DEBUG nova.scheduler.client.report [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Inventory has not changed for provider 4d291220-7c37-4839-a0e3-b23c66502b74 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=75234) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Nov 29 03:25:10 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Compute_service record updated for user:user {{(pid=75234) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:1067}}
Nov 29 03:25:10 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.428s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:25:11 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-bb39c935-0ccf-48b3-85a9-574b89670de2 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Waiting for libvirt event about the detach of device vdb with device alias virtio-disk1 from instance f6bc212d-85cd-479c-92ae-112c9f0101bc is timed out.
Nov 29 03:25:11 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-bb39c935-0ccf-48b3-85a9-574b89670de2 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Successfully detached device vdb from instance f6bc212d-85cd-479c-92ae-112c9f0101bc from the live domain config.
Nov 29 03:25:11 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-bb39c935-0ccf-48b3-85a9-574b89670de2 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] [instance: f6bc212d-85cd-479c-92ae-112c9f0101bc] calling os-brick to detach iSCSI Volume {{(pid=75234) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Nov 29 03:25:11 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-bb39c935-0ccf-48b3-85a9-574b89670de2 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f8a5050c9a0>, {'target_iqn': 'iqn.2010-10.org.openstack:023a5970-c0e3-48f5-a8ef-d9d90fed8b37', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'hjrHYOxB', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sde'}, None), 'kwargs': False}}" {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:176}}
Nov 29 03:25:11 user nova-compute[75234]: WARNING os_brick.initiator.connectors.base [None req-bb39c935-0ccf-48b3-85a9-574b89670de2 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Nov 29 03:25:11 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-bb39c935-0ccf-48b3-85a9-574b89670de2 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:68}}
Nov 29 03:25:11 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-bb39c935-0ccf-48b3-85a9-574b89670de2 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:73}}
Nov 29 03:25:11 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:25:11 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.019s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:25:11 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[8d87b0b0-22d8-41cc-ac08-8acc53f69330]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:023a5970-c0e3-48f5-a8ef-d9d90fed8b37\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:41f08854-31dd-4172-a5cf-01b0e695c10f\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:25:11 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-bb39c935-0ccf-48b3-85a9-574b89670de2 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS:
Nov 29 03:25:11 user nova-compute[75234]: No targets found.
Nov 29 03:25:11 user nova-compute[75234]: iSNS:
Nov 29 03:25:11 user nova-compute[75234]: No targets found.
Nov 29 03:25:11 user nova-compute[75234]: STATIC:
Nov 29 03:25:11 user nova-compute[75234]: Target: iqn.2010-10.org.openstack:023a5970-c0e3-48f5-a8ef-d9d90fed8b37
Nov 29 03:25:11 user nova-compute[75234]:         Portal: 172.16.0.220:3260,-1
Nov 29 03:25:11 user nova-compute[75234]:                 Iface Name: default
Nov 29 03:25:11 user nova-compute[75234]: Target: iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0
Nov 29 03:25:11 user nova-compute[75234]:         Portal: 172.16.0.220:3260,-1
Nov 29 03:25:11 user nova-compute[75234]:                 Iface Name: default
Nov 29 03:25:11 user nova-compute[75234]: Target: iqn.2010-10.org.openstack:41f08854-31dd-4172-a5cf-01b0e695c10f
Nov 29 03:25:11 user nova-compute[75234]:         Portal: 172.16.0.220:3260,-1
Nov 29 03:25:11 user nova-compute[75234]:                 Iface Name: default
Nov 29 03:25:11 user nova-compute[75234]: FIRMWARE:
Nov 29 03:25:11 user nova-compute[75234]: No targets found.
Nov 29 03:25:11 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:25:11 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-bb39c935-0ccf-48b3-85a9-574b89670de2 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Regex to get portals from discoverydb: ^SENDTARGETS:
Nov 29 03:25:11 user nova-compute[75234]: .*?^DiscoveryAddress: 172.16.0.220,3260.*?
Nov 29 03:25:11 user nova-compute[75234]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=75234) _get_discoverydb_portals /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:382}}
Nov 29 03:25:11 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-bb39c935-0ccf-48b3-85a9-574b89670de2 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:023a5970-c0e3-48f5-a8ef-d9d90fed8b37', 0)] {{(pid=75234) _get_connection_devices /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:830}}
Nov 29 03:25:11 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:25:11 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.018s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:25:11 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[3d9d9f20-41bd-44d1-87be-4e5ed7e6e417]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:023a5970-c0e3-48f5-a8ef-d9d90fed8b37\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:41f08854-31dd-4172-a5cf-01b0e695c10f\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:25:11 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:25:11 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.020s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:25:11 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[7235e127-84a4-41a9-8862-029b4d2c7652]: (4, ('tcp: [39] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:41f08854-31dd-4172-a5cf-01b0e695c10f (non-flash)\ntcp: [42] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0 (non-flash)\ntcp: [44] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:023a5970-c0e3-48f5-a8ef-d9d90fed8b37 (non-flash)\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:25:11 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-bb39c935-0ccf-48b3-85a9-574b89670de2 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [39] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:41f08854-31dd-4172-a5cf-01b0e695c10f (non-flash)
Nov 29 03:25:11 user nova-compute[75234]: tcp: [42] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0 (non-flash)
Nov 29 03:25:11 user nova-compute[75234]: tcp: [44] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:023a5970-c0e3-48f5-a8ef-d9d90fed8b37 (non-flash)
Nov 29 03:25:11 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:25:11 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-bb39c935-0ccf-48b3-85a9-574b89670de2 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] iscsi session list stdout=tcp: [39] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:41f08854-31dd-4172-a5cf-01b0e695c10f (non-flash)
Nov 29 03:25:11 user nova-compute[75234]: tcp: [42] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0 (non-flash)
Nov 29 03:25:11 user nova-compute[75234]: tcp: [44] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:023a5970-c0e3-48f5-a8ef-d9d90fed8b37 (non-flash)
Nov 29 03:25:11 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsi_session /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Nov 29 03:25:11 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-bb39c935-0ccf-48b3-85a9-574b89670de2 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7f8a2c712d40>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:023a5970-c0e3-48f5-a8ef-d9d90fed8b37'): ({'sde'}, set())}) {{(pid=75234) _get_connection_devices /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:863}}
Nov 29 03:25:11 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-bb39c935-0ccf-48b3-85a9-574b89670de2 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Removing single pathed devices sde {{(pid=75234) remove_connection /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:374}}
Nov 29 03:25:11 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:25:11 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.019s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:25:11 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[4a067d22-311d-44cc-9b50-862095bccc9f]: (4, ('path checker states:\nup                  3\n\npaths: 0\nbusy: False\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:25:11 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sde {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:25:11 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sde" returned: 0 in 0.022s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:25:11 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[452cc98d-6317-40ea-a6c1-a3e14e3bc799]: (4, ('ok\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:25:11 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-bb39c935-0ccf-48b3-85a9-574b89670de2 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Flushing IO for device /dev/sde {{(pid=75234) flush_device_io /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:434}}
Nov 29 03:25:11 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sde {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:25:11 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sde" returned: 0 in 0.013s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:25:11 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[1fb2bad9-e3f8-40a4-a163-bdd760750328]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:25:11 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-bb39c935-0ccf-48b3-85a9-574b89670de2 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Remove SCSI device /dev/sde with /sys/block/sde/device/delete {{(pid=75234) remove_scsi_device /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:148}}
Nov 29 03:25:11 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sde/device/delete {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:25:11 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sde/device/delete" returned: 0 in 0.042s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:25:11 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[b029319f-ce76-4e15-9832-33b62c057e9a]: (4, ('1', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:25:11 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-bb39c935-0ccf-48b3-85a9-574b89670de2 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Checking to see if SCSI volumes sde have been removed. {{(pid=75234) wait_for_volumes_removal /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:156}}
Nov 29 03:25:11 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-bb39c935-0ccf-48b3-85a9-574b89670de2 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] SCSI volumes sde have been removed. {{(pid=75234) wait_for_volumes_removal /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:166}}
Nov 29 03:25:11 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-bb39c935-0ccf-48b3-85a9-574b89670de2 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:023a5970-c0e3-48f5-a8ef-d9d90fed8b37')] {{(pid=75234) _disconnect_connection /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Nov 29 03:25:11 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:023a5970-c0e3-48f5-a8ef-d9d90fed8b37 -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:25:11 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:023a5970-c0e3-48f5-a8ef-d9d90fed8b37 -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.021s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:25:11 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[bb430cf4-3e1c-4f1e-b7da-1cb16f0ccfa7]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:25:11 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-bb39c935-0ccf-48b3-85a9-574b89670de2 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:25:11 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:023a5970-c0e3-48f5-a8ef-d9d90fed8b37 -p 172.16.0.220:3260 --logout {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:25:11 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:023a5970-c0e3-48f5-a8ef-d9d90fed8b37 -p 172.16.0.220:3260 --logout" returned: 0 in 0.059s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:25:11 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[2fa50920-b677-4648-ad8f-8ba4f4b00d8f]: (4, ('Logging out of session [sid: 44, target: iqn.2010-10.org.openstack:023a5970-c0e3-48f5-a8ef-d9d90fed8b37, portal: 172.16.0.220,3260]\nLogout of [sid: 44, target: iqn.2010-10.org.openstack:023a5970-c0e3-48f5-a8ef-d9d90fed8b37, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:25:11 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-bb39c935-0ccf-48b3-85a9-574b89670de2 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 44, target: iqn.2010-10.org.openstack:023a5970-c0e3-48f5-a8ef-d9d90fed8b37, portal: 172.16.0.220,3260]
Nov 29 03:25:11 user nova-compute[75234]: Logout of [sid: 44, target: iqn.2010-10.org.openstack:023a5970-c0e3-48f5-a8ef-d9d90fed8b37, portal: 172.16.0.220,3260] successful.
Nov 29 03:25:11 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:25:11 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:023a5970-c0e3-48f5-a8ef-d9d90fed8b37 -p 172.16.0.220:3260 --op delete {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:25:11 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:023a5970-c0e3-48f5-a8ef-d9d90fed8b37 -p 172.16.0.220:3260 --op delete" returned: 0 in 0.023s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:25:11 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[cc91a099-4b50-49cc-92e5-4a0d13371684]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:25:11 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-bb39c935-0ccf-48b3-85a9-574b89670de2 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:25:11 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-bb39c935-0ccf-48b3-85a9-574b89670de2 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.299s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:87}}
Nov 29 03:25:11 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-bb39c935-0ccf-48b3-85a9-574b89670de2 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] <== disconnect_volume: return (301ms) None {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:203}}
Nov 29 03:25:11 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-bb39c935-0ccf-48b3-85a9-574b89670de2 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] [instance: f6bc212d-85cd-479c-92ae-112c9f0101bc] Disconnected iSCSI Volume {{(pid=75234) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:79}}
Nov 29 03:25:11 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:25:11 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Starting heal instance info cache {{(pid=75234) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9922}}
Nov 29 03:25:11 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Acquiring lock "refresh_cache-1df140f1-fbd9-4b5d-b356-d08b7e908f64" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:25:11 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Acquired lock "refresh_cache-1df140f1-fbd9-4b5d-b356-d08b7e908f64" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:25:12 user nova-compute[75234]: DEBUG nova.network.neutron [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] [instance: 1df140f1-fbd9-4b5d-b356-d08b7e908f64] Forcefully refreshing network info cache for instance {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2003}}
Nov 29 03:25:12 user nova-compute[75234]: DEBUG nova.network.neutron [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] [instance: 1df140f1-fbd9-4b5d-b356-d08b7e908f64] Updating instance_info_cache with network_info: [{"id": "ea9a3586-da52-49b7-8ced-5d27806885ad", "address": "fa:16:3e:80:25:b8", "network": {"id": "2b3c3140-f3b6-45da-8202-bb4d7f0dd130", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-2043420778-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.158", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "f8a7884867c944e1ba1024270e4902f3", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapea9a3586-da", "ovs_interfaceid": "ea9a3586-da52-49b7-8ced-5d27806885ad", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:25:12 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Releasing lock "refresh_cache-1df140f1-fbd9-4b5d-b356-d08b7e908f64" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:25:12 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] [instance: 1df140f1-fbd9-4b5d-b356-d08b7e908f64] Updated the network info_cache for instance {{(pid=75234) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9993}}
Nov 29 03:25:12 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:25:12 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:25:12 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:25:13 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:25:17 user nova-compute[75234]: DEBUG nova.objects.instance [None req-bb39c935-0ccf-48b3-85a9-574b89670de2 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Lazy-loading 'flavor' on Instance uuid f6bc212d-85cd-479c-92ae-112c9f0101bc {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:25:17 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-bb39c935-0ccf-48b3-85a9-574b89670de2 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Lock "f6bc212d-85cd-479c-92ae-112c9f0101bc" "released" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: held 27.379s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:25:17 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:25:21 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-37436628-6dfd-4196-81bb-8f8dc1910e9c tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Acquiring lock "21e9c6cd-c39f-41c0-8768-5dd64589c3f3" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:25:21 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-37436628-6dfd-4196-81bb-8f8dc1910e9c tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Lock "21e9c6cd-c39f-41c0-8768-5dd64589c3f3" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:25:21 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-37436628-6dfd-4196-81bb-8f8dc1910e9c tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Acquiring lock "21e9c6cd-c39f-41c0-8768-5dd64589c3f3-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:25:21 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-37436628-6dfd-4196-81bb-8f8dc1910e9c tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Lock "21e9c6cd-c39f-41c0-8768-5dd64589c3f3-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:25:21 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-37436628-6dfd-4196-81bb-8f8dc1910e9c tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Lock "21e9c6cd-c39f-41c0-8768-5dd64589c3f3-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:25:21 user nova-compute[75234]: INFO nova.compute.manager [None req-37436628-6dfd-4196-81bb-8f8dc1910e9c tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] [instance: 21e9c6cd-c39f-41c0-8768-5dd64589c3f3] Terminating instance
Nov 29 03:25:21 user nova-compute[75234]: DEBUG nova.compute.manager [None req-37436628-6dfd-4196-81bb-8f8dc1910e9c tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] [instance: 21e9c6cd-c39f-41c0-8768-5dd64589c3f3] Start destroying the instance on the hypervisor. {{(pid=75234) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3140}}
Nov 29 03:25:21 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:25:21 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:25:21 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:25:21 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:25:21 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:25:21 user nova-compute[75234]: DEBUG nova.compute.manager [req-dbf5a183-0fe4-47e2-a0bc-661c80abd132 req-1fa55105-c03c-4105-b2e2-c583cfc22b2c service nova] [instance: 21e9c6cd-c39f-41c0-8768-5dd64589c3f3] Received event network-vif-unplugged-87f1e736-6665-4945-bd19-0cf93bff905e {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:25:21 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-dbf5a183-0fe4-47e2-a0bc-661c80abd132 req-1fa55105-c03c-4105-b2e2-c583cfc22b2c service nova] Acquiring lock "21e9c6cd-c39f-41c0-8768-5dd64589c3f3-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:25:21 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-dbf5a183-0fe4-47e2-a0bc-661c80abd132 req-1fa55105-c03c-4105-b2e2-c583cfc22b2c service nova] Lock "21e9c6cd-c39f-41c0-8768-5dd64589c3f3-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:25:21 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-dbf5a183-0fe4-47e2-a0bc-661c80abd132 req-1fa55105-c03c-4105-b2e2-c583cfc22b2c service nova] Lock "21e9c6cd-c39f-41c0-8768-5dd64589c3f3-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:25:21 user nova-compute[75234]: DEBUG nova.compute.manager [req-dbf5a183-0fe4-47e2-a0bc-661c80abd132 req-1fa55105-c03c-4105-b2e2-c583cfc22b2c service nova] [instance: 21e9c6cd-c39f-41c0-8768-5dd64589c3f3] No waiting events found dispatching network-vif-unplugged-87f1e736-6665-4945-bd19-0cf93bff905e {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:25:21 user nova-compute[75234]: DEBUG nova.compute.manager [req-dbf5a183-0fe4-47e2-a0bc-661c80abd132 req-1fa55105-c03c-4105-b2e2-c583cfc22b2c service nova] [instance: 21e9c6cd-c39f-41c0-8768-5dd64589c3f3] Received event network-vif-unplugged-87f1e736-6665-4945-bd19-0cf93bff905e for instance with task_state deleting. {{(pid=75234) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10890}}
Nov 29 03:25:21 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:25:21 user nova-compute[75234]: INFO nova.virt.libvirt.driver [-] [instance: 21e9c6cd-c39f-41c0-8768-5dd64589c3f3] Instance destroyed successfully.
Nov 29 03:25:21 user nova-compute[75234]: DEBUG nova.objects.instance [None req-37436628-6dfd-4196-81bb-8f8dc1910e9c tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Lazy-loading 'resources' on Instance uuid 21e9c6cd-c39f-41c0-8768-5dd64589c3f3 {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:25:21 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-37436628-6dfd-4196-81bb-8f8dc1910e9c tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-11-29T03:06:05Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-ServerActionsTestOtherA-server-1105149564',display_name='tempest-ServerActionsTestOtherA-server-1105149564',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-serveractionstestothera-server-1105149564',id=3,image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBGysIZGz8py0LojrFdRCkkIN/DpphiAjtTxjJQY7OVo0ZaKHLKAtSriy5/opnav7HzQyT3MeG6wW+2nQfBNyK0gW9HDU5His6qvZYEbhyEyMiaUzdXksC9FbE2bauudLhQ==',key_name='tempest-keypair-1767034584',keypairs=<?>,launch_index=0,launched_at=2023-11-29T03:06:20Z,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='c1cb3d1f12f242a785a688d59dbe3cd6',ramdisk_id='',reservation_id='r-nnmj1n3l',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-ServerActionsTestOtherA-1495067208',owner_user_name='tempest-ServerActionsTestOtherA-1495067208-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-11-29T03:06:21Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='d611826f3be947c0947b99c763c26d44',uuid=21e9c6cd-c39f-41c0-8768-5dd64589c3f3,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "87f1e736-6665-4945-bd19-0cf93bff905e", "address": "fa:16:3e:40:14:df", "network": {"id": "1f946e69-dbec-429c-83f8-a186bad4f2cc", "bridge": "br-int", "label": "tempest-ServerActionsTestOtherA-486369423-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "c1cb3d1f12f242a785a688d59dbe3cd6", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap87f1e736-66", "ovs_interfaceid": "87f1e736-6665-4945-bd19-0cf93bff905e", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Nov 29 03:25:21 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-37436628-6dfd-4196-81bb-8f8dc1910e9c tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Converting VIF {"id": "87f1e736-6665-4945-bd19-0cf93bff905e", "address": "fa:16:3e:40:14:df", "network": {"id": "1f946e69-dbec-429c-83f8-a186bad4f2cc", "bridge": "br-int", "label": "tempest-ServerActionsTestOtherA-486369423-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.5", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "c1cb3d1f12f242a785a688d59dbe3cd6", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap87f1e736-66", "ovs_interfaceid": "87f1e736-6665-4945-bd19-0cf93bff905e", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:25:21 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-37436628-6dfd-4196-81bb-8f8dc1910e9c tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:40:14:df,bridge_name='br-int',has_traffic_filtering=True,id=87f1e736-6665-4945-bd19-0cf93bff905e,network=Network(1f946e69-dbec-429c-83f8-a186bad4f2cc),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap87f1e736-66') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:25:21 user nova-compute[75234]: DEBUG os_vif [None req-37436628-6dfd-4196-81bb-8f8dc1910e9c tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Unplugging vif VIFOpenVSwitch(active=True,address=fa:16:3e:40:14:df,bridge_name='br-int',has_traffic_filtering=True,id=87f1e736-6665-4945-bd19-0cf93bff905e,network=Network(1f946e69-dbec-429c-83f8-a186bad4f2cc),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap87f1e736-66') {{(pid=75234) unplug /opt/stack/data/venv/lib/python3.10/site-packages/os_vif/__init__.py:109}}
Nov 29 03:25:21 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:25:21 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tap87f1e736-66, bridge=br-int, if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:25:21 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:25:21 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:25:21 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbDestroyCommand(_result=None, table=QoS, record=70224693-48a0-479d-be80-bc6803e99fb2) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:25:21 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:25:21 user nova-compute[75234]: INFO os_vif [None req-37436628-6dfd-4196-81bb-8f8dc1910e9c tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Successfully unplugged vif VIFOpenVSwitch(active=True,address=fa:16:3e:40:14:df,bridge_name='br-int',has_traffic_filtering=True,id=87f1e736-6665-4945-bd19-0cf93bff905e,network=Network(1f946e69-dbec-429c-83f8-a186bad4f2cc),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap87f1e736-66')
Nov 29 03:25:21 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-37436628-6dfd-4196-81bb-8f8dc1910e9c tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] [instance: 21e9c6cd-c39f-41c0-8768-5dd64589c3f3] Deleting instance files /opt/stack/data/nova/instances/21e9c6cd-c39f-41c0-8768-5dd64589c3f3_del
Nov 29 03:25:21 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-37436628-6dfd-4196-81bb-8f8dc1910e9c tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] [instance: 21e9c6cd-c39f-41c0-8768-5dd64589c3f3] Deletion of /opt/stack/data/nova/instances/21e9c6cd-c39f-41c0-8768-5dd64589c3f3_del complete
Nov 29 03:25:21 user nova-compute[75234]: INFO nova.compute.manager [None req-37436628-6dfd-4196-81bb-8f8dc1910e9c tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] [instance: 21e9c6cd-c39f-41c0-8768-5dd64589c3f3] Took 0.89 seconds to destroy the instance on the hypervisor.
Nov 29 03:25:21 user nova-compute[75234]: DEBUG oslo.service.loopingcall [None req-37436628-6dfd-4196-81bb-8f8dc1910e9c tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=75234) func /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/loopingcall.py:435}}
Nov 29 03:25:21 user nova-compute[75234]: DEBUG nova.compute.manager [-] [instance: 21e9c6cd-c39f-41c0-8768-5dd64589c3f3] Deallocating network for instance {{(pid=75234) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2279}}
Nov 29 03:25:21 user nova-compute[75234]: DEBUG nova.network.neutron [-] [instance: 21e9c6cd-c39f-41c0-8768-5dd64589c3f3] deallocate_for_instance() {{(pid=75234) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1802}}
Nov 29 03:25:22 user nova-compute[75234]: DEBUG nova.network.neutron [-] [instance: 21e9c6cd-c39f-41c0-8768-5dd64589c3f3] Updating instance_info_cache with network_info: [] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:25:22 user nova-compute[75234]: INFO nova.compute.manager [-] [instance: 21e9c6cd-c39f-41c0-8768-5dd64589c3f3] Took 0.45 seconds to deallocate network for instance.
Nov 29 03:25:22 user nova-compute[75234]: DEBUG nova.compute.manager [req-3c61b3ef-8c40-4782-ac6c-21c8eb7e2224 req-d4396ce8-492a-41f0-8a88-9e0dd36e17e7 service nova] [instance: 21e9c6cd-c39f-41c0-8768-5dd64589c3f3] Received event network-vif-deleted-87f1e736-6665-4945-bd19-0cf93bff905e {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:25:22 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-37436628-6dfd-4196-81bb-8f8dc1910e9c tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:25:22 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-37436628-6dfd-4196-81bb-8f8dc1910e9c tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:25:22 user nova-compute[75234]: DEBUG nova.compute.provider_tree [None req-37436628-6dfd-4196-81bb-8f8dc1910e9c tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Inventory has not changed in ProviderTree for provider: 4d291220-7c37-4839-a0e3-b23c66502b74 {{(pid=75234) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Nov 29 03:25:22 user nova-compute[75234]: DEBUG nova.scheduler.client.report [None req-37436628-6dfd-4196-81bb-8f8dc1910e9c tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Inventory has not changed for provider 4d291220-7c37-4839-a0e3-b23c66502b74 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=75234) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Nov 29 03:25:22 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-37436628-6dfd-4196-81bb-8f8dc1910e9c tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.334s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:25:22 user nova-compute[75234]: INFO nova.scheduler.client.report [None req-37436628-6dfd-4196-81bb-8f8dc1910e9c tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Deleted allocations for instance 21e9c6cd-c39f-41c0-8768-5dd64589c3f3
Nov 29 03:25:22 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-37436628-6dfd-4196-81bb-8f8dc1910e9c tempest-ServerActionsTestOtherA-1495067208 tempest-ServerActionsTestOtherA-1495067208-project-member] Lock "21e9c6cd-c39f-41c0-8768-5dd64589c3f3" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 1.870s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:25:23 user nova-compute[75234]: DEBUG nova.compute.manager [req-d660fe8a-7fdb-4945-8017-4c31b0c9f2ac req-e1375530-4d52-43c0-9a97-28395c20168f service nova] [instance: 21e9c6cd-c39f-41c0-8768-5dd64589c3f3] Received event network-vif-plugged-87f1e736-6665-4945-bd19-0cf93bff905e {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:25:23 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-d660fe8a-7fdb-4945-8017-4c31b0c9f2ac req-e1375530-4d52-43c0-9a97-28395c20168f service nova] Acquiring lock "21e9c6cd-c39f-41c0-8768-5dd64589c3f3-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:25:23 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-d660fe8a-7fdb-4945-8017-4c31b0c9f2ac req-e1375530-4d52-43c0-9a97-28395c20168f service nova] Lock "21e9c6cd-c39f-41c0-8768-5dd64589c3f3-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:25:23 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-d660fe8a-7fdb-4945-8017-4c31b0c9f2ac req-e1375530-4d52-43c0-9a97-28395c20168f service nova] Lock "21e9c6cd-c39f-41c0-8768-5dd64589c3f3-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:25:23 user nova-compute[75234]: DEBUG nova.compute.manager [req-d660fe8a-7fdb-4945-8017-4c31b0c9f2ac req-e1375530-4d52-43c0-9a97-28395c20168f service nova] [instance: 21e9c6cd-c39f-41c0-8768-5dd64589c3f3] No waiting events found dispatching network-vif-plugged-87f1e736-6665-4945-bd19-0cf93bff905e {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:25:23 user nova-compute[75234]: WARNING nova.compute.manager [req-d660fe8a-7fdb-4945-8017-4c31b0c9f2ac req-e1375530-4d52-43c0-9a97-28395c20168f service nova] [instance: 21e9c6cd-c39f-41c0-8768-5dd64589c3f3] Received unexpected event network-vif-plugged-87f1e736-6665-4945-bd19-0cf93bff905e for instance with vm_state deleted and task_state None.
Nov 29 03:25:25 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-50a7db2f-4e8c-42d5-aa8d-9b2f13457b00 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Waiting for libvirt event about the detach of device vdb with device alias virtio-disk1 from instance 1df140f1-fbd9-4b5d-b356-d08b7e908f64 is timed out.
Nov 29 03:25:25 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-50a7db2f-4e8c-42d5-aa8d-9b2f13457b00 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Successfully detached device vdb from instance 1df140f1-fbd9-4b5d-b356-d08b7e908f64 from the live domain config.
Nov 29 03:25:25 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-50a7db2f-4e8c-42d5-aa8d-9b2f13457b00 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] [instance: 1df140f1-fbd9-4b5d-b356-d08b7e908f64] calling os-brick to detach iSCSI Volume {{(pid=75234) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Nov 29 03:25:25 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-50a7db2f-4e8c-42d5-aa8d-9b2f13457b00 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f8a5050c9a0>, {'target_iqn': 'iqn.2010-10.org.openstack:41f08854-31dd-4172-a5cf-01b0e695c10f', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'XswhptcD', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdd'}, None), 'kwargs': False}}" {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:176}}
Nov 29 03:25:25 user nova-compute[75234]: WARNING os_brick.initiator.connectors.base [None req-50a7db2f-4e8c-42d5-aa8d-9b2f13457b00 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Nov 29 03:25:25 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-50a7db2f-4e8c-42d5-aa8d-9b2f13457b00 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:68}}
Nov 29 03:25:25 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-50a7db2f-4e8c-42d5-aa8d-9b2f13457b00 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:73}}
Nov 29 03:25:25 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:25:25 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.018s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:25:25 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[305cb95c-30f4-4f3b-8667-60513c9f2387]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:41f08854-31dd-4172-a5cf-01b0e695c10f\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:25:25 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-50a7db2f-4e8c-42d5-aa8d-9b2f13457b00 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS:
Nov 29 03:25:25 user nova-compute[75234]: No targets found.
Nov 29 03:25:25 user nova-compute[75234]: iSNS:
Nov 29 03:25:25 user nova-compute[75234]: No targets found.
Nov 29 03:25:25 user nova-compute[75234]: STATIC:
Nov 29 03:25:25 user nova-compute[75234]: Target: iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0
Nov 29 03:25:25 user nova-compute[75234]:         Portal: 172.16.0.220:3260,-1
Nov 29 03:25:25 user nova-compute[75234]:                 Iface Name: default
Nov 29 03:25:25 user nova-compute[75234]: Target: iqn.2010-10.org.openstack:41f08854-31dd-4172-a5cf-01b0e695c10f
Nov 29 03:25:25 user nova-compute[75234]:         Portal: 172.16.0.220:3260,-1
Nov 29 03:25:25 user nova-compute[75234]:                 Iface Name: default
Nov 29 03:25:25 user nova-compute[75234]: FIRMWARE:
Nov 29 03:25:25 user nova-compute[75234]: No targets found.
Nov 29 03:25:25 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:25:25 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-50a7db2f-4e8c-42d5-aa8d-9b2f13457b00 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Regex to get portals from discoverydb: ^SENDTARGETS:
Nov 29 03:25:25 user nova-compute[75234]: .*?^DiscoveryAddress: 172.16.0.220,3260.*?
Nov 29 03:25:25 user nova-compute[75234]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=75234) _get_discoverydb_portals /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:382}}
Nov 29 03:25:25 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-50a7db2f-4e8c-42d5-aa8d-9b2f13457b00 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:41f08854-31dd-4172-a5cf-01b0e695c10f', 0)] {{(pid=75234) _get_connection_devices /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:830}}
Nov 29 03:25:25 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:25:25 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.018s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:25:25 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[9f3406aa-7abe-494b-9052-be9d82a4a503]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:41f08854-31dd-4172-a5cf-01b0e695c10f\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:25:25 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:25:25 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.020s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:25:25 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[bd03ce04-9eb8-4a06-8d82-b05792630329]: (4, ('tcp: [39] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:41f08854-31dd-4172-a5cf-01b0e695c10f (non-flash)\ntcp: [42] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0 (non-flash)\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:25:25 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-50a7db2f-4e8c-42d5-aa8d-9b2f13457b00 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [39] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:41f08854-31dd-4172-a5cf-01b0e695c10f (non-flash)
Nov 29 03:25:25 user nova-compute[75234]: tcp: [42] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0 (non-flash)
Nov 29 03:25:25 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:25:25 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-50a7db2f-4e8c-42d5-aa8d-9b2f13457b00 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] iscsi session list stdout=tcp: [39] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:41f08854-31dd-4172-a5cf-01b0e695c10f (non-flash)
Nov 29 03:25:25 user nova-compute[75234]: tcp: [42] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0 (non-flash)
Nov 29 03:25:25 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsi_session /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Nov 29 03:25:25 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-50a7db2f-4e8c-42d5-aa8d-9b2f13457b00 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7f8a2c78e680>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:41f08854-31dd-4172-a5cf-01b0e695c10f'): ({'sdd'}, set())}) {{(pid=75234) _get_connection_devices /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:863}}
Nov 29 03:25:25 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-50a7db2f-4e8c-42d5-aa8d-9b2f13457b00 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Removing single pathed devices sdd {{(pid=75234) remove_connection /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:374}}
Nov 29 03:25:25 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:25:25 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.021s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:25:25 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[e5ae2ee0-5598-4234-91d0-8f2f82c2be57]: (4, ('path checker states:\nup                  2\n\npaths: 0\nbusy: False\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:25:25 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdd {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:25:25 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdd" returned: 0 in 0.018s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:25:25 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[3f7ca06d-cc12-4ac4-aebf-c835edd44639]: (4, ('ok\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:25:25 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-50a7db2f-4e8c-42d5-aa8d-9b2f13457b00 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Flushing IO for device /dev/sdd {{(pid=75234) flush_device_io /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:434}}
Nov 29 03:25:25 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdd {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:25:25 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdd" returned: 0 in 0.015s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:25:25 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[b78e2433-5c5a-49da-b3a6-f30eeb3e0704]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:25:25 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-50a7db2f-4e8c-42d5-aa8d-9b2f13457b00 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Remove SCSI device /dev/sdd with /sys/block/sdd/device/delete {{(pid=75234) remove_scsi_device /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:148}}
Nov 29 03:25:25 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdd/device/delete {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:25:25 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdd/device/delete" returned: 0 in 0.070s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:25:25 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[64e82877-391c-41d6-bcd0-ee311635c65b]: (4, ('1', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:25:25 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-50a7db2f-4e8c-42d5-aa8d-9b2f13457b00 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Checking to see if SCSI volumes sdd have been removed. {{(pid=75234) wait_for_volumes_removal /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:156}}
Nov 29 03:25:25 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-50a7db2f-4e8c-42d5-aa8d-9b2f13457b00 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] SCSI volumes sdd have been removed. {{(pid=75234) wait_for_volumes_removal /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:166}}
Nov 29 03:25:25 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-50a7db2f-4e8c-42d5-aa8d-9b2f13457b00 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:41f08854-31dd-4172-a5cf-01b0e695c10f')] {{(pid=75234) _disconnect_connection /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Nov 29 03:25:25 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:41f08854-31dd-4172-a5cf-01b0e695c10f -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:25:26 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:41f08854-31dd-4172-a5cf-01b0e695c10f -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.020s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:25:26 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[68bda810-c4c0-457c-91f0-bbdde99002d7]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:25:26 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-50a7db2f-4e8c-42d5-aa8d-9b2f13457b00 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:25:26 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:41f08854-31dd-4172-a5cf-01b0e695c10f -p 172.16.0.220:3260 --logout {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:25:26 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:41f08854-31dd-4172-a5cf-01b0e695c10f -p 172.16.0.220:3260 --logout" returned: 0 in 0.047s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:25:26 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[67ca881b-76e1-4e4e-95ff-898d7620cc05]: (4, ('Logging out of session [sid: 39, target: iqn.2010-10.org.openstack:41f08854-31dd-4172-a5cf-01b0e695c10f, portal: 172.16.0.220,3260]\nLogout of [sid: 39, target: iqn.2010-10.org.openstack:41f08854-31dd-4172-a5cf-01b0e695c10f, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:25:26 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-50a7db2f-4e8c-42d5-aa8d-9b2f13457b00 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 39, target: iqn.2010-10.org.openstack:41f08854-31dd-4172-a5cf-01b0e695c10f, portal: 172.16.0.220,3260]
Nov 29 03:25:26 user nova-compute[75234]: Logout of [sid: 39, target: iqn.2010-10.org.openstack:41f08854-31dd-4172-a5cf-01b0e695c10f, portal: 172.16.0.220,3260] successful.
Nov 29 03:25:26 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:25:26 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:41f08854-31dd-4172-a5cf-01b0e695c10f -p 172.16.0.220:3260 --op delete {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:25:26 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:41f08854-31dd-4172-a5cf-01b0e695c10f -p 172.16.0.220:3260 --op delete" returned: 0 in 0.022s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:25:26 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[71b681c3-76f1-4785-be83-a1cc4431990d]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:25:26 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-50a7db2f-4e8c-42d5-aa8d-9b2f13457b00 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:25:26 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-50a7db2f-4e8c-42d5-aa8d-9b2f13457b00 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.317s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:87}}
Nov 29 03:25:26 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-50a7db2f-4e8c-42d5-aa8d-9b2f13457b00 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] <== disconnect_volume: return (319ms) None {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:203}}
Nov 29 03:25:26 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-50a7db2f-4e8c-42d5-aa8d-9b2f13457b00 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] [instance: 1df140f1-fbd9-4b5d-b356-d08b7e908f64] Disconnected iSCSI Volume {{(pid=75234) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:79}}
Nov 29 03:25:26 user nova-compute[75234]: DEBUG nova.objects.instance [None req-50a7db2f-4e8c-42d5-aa8d-9b2f13457b00 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Lazy-loading 'flavor' on Instance uuid 1df140f1-fbd9-4b5d-b356-d08b7e908f64 {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:25:26 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-50a7db2f-4e8c-42d5-aa8d-9b2f13457b00 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Lock "1df140f1-fbd9-4b5d-b356-d08b7e908f64" "released" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: held 21.669s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:25:26 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:25:29 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-fa82c34e-2935-4be9-96d5-7ffc65d53717 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Acquiring lock "060c15ec-9012-471a-984a-14c9fa6ce23a" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:25:29 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-fa82c34e-2935-4be9-96d5-7ffc65d53717 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Lock "060c15ec-9012-471a-984a-14c9fa6ce23a" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:25:29 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-fa82c34e-2935-4be9-96d5-7ffc65d53717 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Acquiring lock "060c15ec-9012-471a-984a-14c9fa6ce23a-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:25:29 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-fa82c34e-2935-4be9-96d5-7ffc65d53717 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Lock "060c15ec-9012-471a-984a-14c9fa6ce23a-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:25:29 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-fa82c34e-2935-4be9-96d5-7ffc65d53717 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Lock "060c15ec-9012-471a-984a-14c9fa6ce23a-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:25:29 user nova-compute[75234]: INFO nova.compute.manager [None req-fa82c34e-2935-4be9-96d5-7ffc65d53717 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] [instance: 060c15ec-9012-471a-984a-14c9fa6ce23a] Terminating instance
Nov 29 03:25:29 user nova-compute[75234]: DEBUG nova.compute.manager [None req-fa82c34e-2935-4be9-96d5-7ffc65d53717 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] [instance: 060c15ec-9012-471a-984a-14c9fa6ce23a] Start destroying the instance on the hypervisor. {{(pid=75234) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3140}}
Nov 29 03:25:29 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:25:29 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:25:29 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:25:29 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:25:29 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:25:29 user nova-compute[75234]: DEBUG nova.compute.manager [req-375ac58a-4bdf-4fa9-8b80-8c493d3144ac req-209b88ab-b8bf-42ad-b534-f76c0a3b5e08 service nova] [instance: 060c15ec-9012-471a-984a-14c9fa6ce23a] Received event network-vif-unplugged-3835164f-aa54-41fa-adf1-d10ae894fba5 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:25:29 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-375ac58a-4bdf-4fa9-8b80-8c493d3144ac req-209b88ab-b8bf-42ad-b534-f76c0a3b5e08 service nova] Acquiring lock "060c15ec-9012-471a-984a-14c9fa6ce23a-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:25:29 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-375ac58a-4bdf-4fa9-8b80-8c493d3144ac req-209b88ab-b8bf-42ad-b534-f76c0a3b5e08 service nova] Lock "060c15ec-9012-471a-984a-14c9fa6ce23a-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:25:29 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-375ac58a-4bdf-4fa9-8b80-8c493d3144ac req-209b88ab-b8bf-42ad-b534-f76c0a3b5e08 service nova] Lock "060c15ec-9012-471a-984a-14c9fa6ce23a-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:25:29 user nova-compute[75234]: DEBUG nova.compute.manager [req-375ac58a-4bdf-4fa9-8b80-8c493d3144ac req-209b88ab-b8bf-42ad-b534-f76c0a3b5e08 service nova] [instance: 060c15ec-9012-471a-984a-14c9fa6ce23a] No waiting events found dispatching network-vif-unplugged-3835164f-aa54-41fa-adf1-d10ae894fba5 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:25:29 user nova-compute[75234]: DEBUG nova.compute.manager [req-375ac58a-4bdf-4fa9-8b80-8c493d3144ac req-209b88ab-b8bf-42ad-b534-f76c0a3b5e08 service nova] [instance: 060c15ec-9012-471a-984a-14c9fa6ce23a] Received event network-vif-unplugged-3835164f-aa54-41fa-adf1-d10ae894fba5 for instance with task_state deleting. {{(pid=75234) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10890}}
Nov 29 03:25:29 user nova-compute[75234]: INFO nova.virt.libvirt.driver [-] [instance: 060c15ec-9012-471a-984a-14c9fa6ce23a] Instance destroyed successfully.
Nov 29 03:25:29 user nova-compute[75234]: DEBUG nova.objects.instance [None req-fa82c34e-2935-4be9-96d5-7ffc65d53717 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Lazy-loading 'resources' on Instance uuid 060c15ec-9012-471a-984a-14c9fa6ce23a {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:25:29 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-fa82c34e-2935-4be9-96d5-7ffc65d53717 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-11-29T03:22:32Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-AttachVolumeNegativeTest-server-1614204486',display_name='tempest-AttachVolumeNegativeTest-server-1614204486',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-attachvolumenegativetest-server-1614204486',id=38,image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBACqQfFLGg5k9dVSlMtreyy4nYgd0k+k8JsZp3H7sIImHG7V9LfBMqYLdLqfJnxtCeXAGpm1MDcSX+HtAljaNui83OKS0FLyT7TjorcVMVGV7urkT0H9swBQ45aBBw/5Qw==',key_name='tempest-keypair-2051938263',keypairs=<?>,launch_index=0,launched_at=2023-11-29T03:22:39Z,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='315732b6e3ee44d584b8e510ef199a90',ramdisk_id='',reservation_id='r-b8r0t0y0',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-AttachVolumeNegativeTest-1582496806',owner_user_name='tempest-AttachVolumeNegativeTest-1582496806-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-11-29T03:22:40Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='66686114450f4f31b3a917667a152d8b',uuid=060c15ec-9012-471a-984a-14c9fa6ce23a,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "3835164f-aa54-41fa-adf1-d10ae894fba5", "address": "fa:16:3e:40:80:df", "network": {"id": "0f289171-3d73-4054-95a0-2515b4f206a0", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-1787960284-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.197", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "315732b6e3ee44d584b8e510ef199a90", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap3835164f-aa", "ovs_interfaceid": "3835164f-aa54-41fa-adf1-d10ae894fba5", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Nov 29 03:25:29 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-fa82c34e-2935-4be9-96d5-7ffc65d53717 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Converting VIF {"id": "3835164f-aa54-41fa-adf1-d10ae894fba5", "address": "fa:16:3e:40:80:df", "network": {"id": "0f289171-3d73-4054-95a0-2515b4f206a0", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-1787960284-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.197", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "315732b6e3ee44d584b8e510ef199a90", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap3835164f-aa", "ovs_interfaceid": "3835164f-aa54-41fa-adf1-d10ae894fba5", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:25:29 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-fa82c34e-2935-4be9-96d5-7ffc65d53717 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:40:80:df,bridge_name='br-int',has_traffic_filtering=True,id=3835164f-aa54-41fa-adf1-d10ae894fba5,network=Network(0f289171-3d73-4054-95a0-2515b4f206a0),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap3835164f-aa') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:25:29 user nova-compute[75234]: DEBUG os_vif [None req-fa82c34e-2935-4be9-96d5-7ffc65d53717 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Unplugging vif VIFOpenVSwitch(active=True,address=fa:16:3e:40:80:df,bridge_name='br-int',has_traffic_filtering=True,id=3835164f-aa54-41fa-adf1-d10ae894fba5,network=Network(0f289171-3d73-4054-95a0-2515b4f206a0),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap3835164f-aa') {{(pid=75234) unplug /opt/stack/data/venv/lib/python3.10/site-packages/os_vif/__init__.py:109}}
Nov 29 03:25:29 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:25:29 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tap3835164f-aa, bridge=br-int, if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:25:29 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:25:29 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:25:29 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:25:29 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbDestroyCommand(_result=None, table=QoS, record=774c631d-0397-4164-89e6-00c3150caae8) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:25:29 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:25:29 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:25:29 user nova-compute[75234]: INFO os_vif [None req-fa82c34e-2935-4be9-96d5-7ffc65d53717 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Successfully unplugged vif VIFOpenVSwitch(active=True,address=fa:16:3e:40:80:df,bridge_name='br-int',has_traffic_filtering=True,id=3835164f-aa54-41fa-adf1-d10ae894fba5,network=Network(0f289171-3d73-4054-95a0-2515b4f206a0),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap3835164f-aa')
Nov 29 03:25:29 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-fa82c34e-2935-4be9-96d5-7ffc65d53717 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] [instance: 060c15ec-9012-471a-984a-14c9fa6ce23a] Deleting instance files /opt/stack/data/nova/instances/060c15ec-9012-471a-984a-14c9fa6ce23a_del
Nov 29 03:25:29 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-fa82c34e-2935-4be9-96d5-7ffc65d53717 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] [instance: 060c15ec-9012-471a-984a-14c9fa6ce23a] Deletion of /opt/stack/data/nova/instances/060c15ec-9012-471a-984a-14c9fa6ce23a_del complete
Nov 29 03:25:30 user nova-compute[75234]: INFO nova.compute.manager [None req-fa82c34e-2935-4be9-96d5-7ffc65d53717 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] [instance: 060c15ec-9012-471a-984a-14c9fa6ce23a] Took 0.83 seconds to destroy the instance on the hypervisor.
Nov 29 03:25:30 user nova-compute[75234]: DEBUG oslo.service.loopingcall [None req-fa82c34e-2935-4be9-96d5-7ffc65d53717 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=75234) func /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/loopingcall.py:435}}
Nov 29 03:25:30 user nova-compute[75234]: DEBUG nova.compute.manager [-] [instance: 060c15ec-9012-471a-984a-14c9fa6ce23a] Deallocating network for instance {{(pid=75234) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2279}}
Nov 29 03:25:30 user nova-compute[75234]: DEBUG nova.network.neutron [-] [instance: 060c15ec-9012-471a-984a-14c9fa6ce23a] deallocate_for_instance() {{(pid=75234) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1802}}
Nov 29 03:25:30 user nova-compute[75234]: DEBUG nova.network.neutron [-] [instance: 060c15ec-9012-471a-984a-14c9fa6ce23a] Updating instance_info_cache with network_info: [] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:25:30 user nova-compute[75234]: INFO nova.compute.manager [-] [instance: 060c15ec-9012-471a-984a-14c9fa6ce23a] Took 0.66 seconds to deallocate network for instance.
Nov 29 03:25:30 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-fa82c34e-2935-4be9-96d5-7ffc65d53717 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:25:30 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-fa82c34e-2935-4be9-96d5-7ffc65d53717 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:25:30 user nova-compute[75234]: DEBUG nova.compute.provider_tree [None req-fa82c34e-2935-4be9-96d5-7ffc65d53717 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Inventory has not changed in ProviderTree for provider: 4d291220-7c37-4839-a0e3-b23c66502b74 {{(pid=75234) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Nov 29 03:25:30 user nova-compute[75234]: DEBUG nova.scheduler.client.report [None req-fa82c34e-2935-4be9-96d5-7ffc65d53717 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Inventory has not changed for provider 4d291220-7c37-4839-a0e3-b23c66502b74 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=75234) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Nov 29 03:25:31 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-fa82c34e-2935-4be9-96d5-7ffc65d53717 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.282s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:25:31 user nova-compute[75234]: INFO nova.scheduler.client.report [None req-fa82c34e-2935-4be9-96d5-7ffc65d53717 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Deleted allocations for instance 060c15ec-9012-471a-984a-14c9fa6ce23a
Nov 29 03:25:31 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-fa82c34e-2935-4be9-96d5-7ffc65d53717 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Lock "060c15ec-9012-471a-984a-14c9fa6ce23a" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 2.017s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:25:31 user nova-compute[75234]: DEBUG nova.compute.manager [req-eda5249f-66fa-4dd5-b1cc-00934f58d4b7 req-e2ece5c2-e3d4-498f-9a45-af4af6edbd0f service nova] [instance: 060c15ec-9012-471a-984a-14c9fa6ce23a] Received event network-vif-plugged-3835164f-aa54-41fa-adf1-d10ae894fba5 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:25:31 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-eda5249f-66fa-4dd5-b1cc-00934f58d4b7 req-e2ece5c2-e3d4-498f-9a45-af4af6edbd0f service nova] Acquiring lock "060c15ec-9012-471a-984a-14c9fa6ce23a-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:25:31 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-eda5249f-66fa-4dd5-b1cc-00934f58d4b7 req-e2ece5c2-e3d4-498f-9a45-af4af6edbd0f service nova] Lock "060c15ec-9012-471a-984a-14c9fa6ce23a-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:25:31 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-eda5249f-66fa-4dd5-b1cc-00934f58d4b7 req-e2ece5c2-e3d4-498f-9a45-af4af6edbd0f service nova] Lock "060c15ec-9012-471a-984a-14c9fa6ce23a-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:25:31 user nova-compute[75234]: DEBUG nova.compute.manager [req-eda5249f-66fa-4dd5-b1cc-00934f58d4b7 req-e2ece5c2-e3d4-498f-9a45-af4af6edbd0f service nova] [instance: 060c15ec-9012-471a-984a-14c9fa6ce23a] No waiting events found dispatching network-vif-plugged-3835164f-aa54-41fa-adf1-d10ae894fba5 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:25:31 user nova-compute[75234]: WARNING nova.compute.manager [req-eda5249f-66fa-4dd5-b1cc-00934f58d4b7 req-e2ece5c2-e3d4-498f-9a45-af4af6edbd0f service nova] [instance: 060c15ec-9012-471a-984a-14c9fa6ce23a] Received unexpected event network-vif-plugged-3835164f-aa54-41fa-adf1-d10ae894fba5 for instance with vm_state deleted and task_state None.
Nov 29 03:25:31 user nova-compute[75234]: DEBUG nova.compute.manager [req-eda5249f-66fa-4dd5-b1cc-00934f58d4b7 req-e2ece5c2-e3d4-498f-9a45-af4af6edbd0f service nova] [instance: 060c15ec-9012-471a-984a-14c9fa6ce23a] Received event network-vif-deleted-3835164f-aa54-41fa-adf1-d10ae894fba5 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:25:34 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:25:39 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-acf58aba-d11f-4a3d-ac0b-46d4f62a74a2 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Acquiring lock "5d7d744e-1a5f-40ab-b961-4b58c183efe0" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:25:39 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-acf58aba-d11f-4a3d-ac0b-46d4f62a74a2 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Lock "5d7d744e-1a5f-40ab-b961-4b58c183efe0" acquired by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:25:39 user nova-compute[75234]: DEBUG nova.objects.instance [None req-acf58aba-d11f-4a3d-ac0b-46d4f62a74a2 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Lazy-loading 'flavor' on Instance uuid 5d7d744e-1a5f-40ab-b961-4b58c183efe0 {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:25:39 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-acf58aba-d11f-4a3d-ac0b-46d4f62a74a2 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Lock "5d7d744e-1a5f-40ab-b961-4b58c183efe0" "released" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: held 0.050s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:25:39 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-acf58aba-d11f-4a3d-ac0b-46d4f62a74a2 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Acquiring lock "5d7d744e-1a5f-40ab-b961-4b58c183efe0" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:25:39 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-acf58aba-d11f-4a3d-ac0b-46d4f62a74a2 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Lock "5d7d744e-1a5f-40ab-b961-4b58c183efe0" acquired by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:25:39 user nova-compute[75234]: INFO nova.compute.manager [None req-acf58aba-d11f-4a3d-ac0b-46d4f62a74a2 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] [instance: 5d7d744e-1a5f-40ab-b961-4b58c183efe0] Attaching volume 65acc1f5-8dbf-457c-a978-510436bccf11 to /dev/vdb
Nov 29 03:25:39 user nova-compute[75234]: DEBUG os_brick.utils [None req-acf58aba-d11f-4a3d-ac0b-46d4f62a74a2 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:176}}
Nov 29 03:25:39 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:25:39 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.016s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:25:39 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[67388e03-60e1-4e5e-9d69-c46c2eff7c3b]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:dd1368c5ba2c\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:25:39 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt -v / -n -o SOURCE {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:25:39 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt -v / -n -o SOURCE" returned: 0 in 0.016s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:25:39 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[f859ddae-545d-431f-b2fa-6f6f84555d93]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:25:39 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:25:39 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.018s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:25:39 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[64a7a2e8-ce08-4be6-8a85-7cd49f4acc51]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:25:39 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[47e1bf37-9f75-4af9-a839-f1ad1b3084f8]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:25:39 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-acf58aba-d11f-4a3d-ac0b-46d4f62a74a2 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Running cmd (subprocess): nvme version {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:25:39 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-acf58aba-d11f-4a3d-ac0b-46d4f62a74a2 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] 'nvme version' failed. Not Retrying. {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:473}}
Nov 29 03:25:39 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.nvmeof [None req-acf58aba-d11f-4a3d-ac0b-46d4f62a74a2 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] nvme not present on system {{(pid=75234) nvme_present /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/nvmeof.py:756}}
Nov 29 03:25:39 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:25:39 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:473}}
Nov 29 03:25:39 user nova-compute[75234]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Nov 29 03:25:39 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[a279f146-efcd-4a11-800f-3826d203a716]: (4, '') {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:25:39 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.lightos [None req-acf58aba-d11f-4a3d-ac0b-46d4f62a74a2 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=75234) find_dsc /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/lightos.py:98}}
Nov 29 03:25:39 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.lightos [None req-acf58aba-d11f-4a3d-ac0b-46d4f62a74a2 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=75234) get_connector_properties /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/lightos.py:76}}
Nov 29 03:25:39 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.lightos [None req-acf58aba-d11f-4a3d-ac0b-46d4f62a74a2 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] LIGHTOS: no hostnqn found. {{(pid=75234) get_connector_properties /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/lightos.py:84}}
Nov 29 03:25:39 user nova-compute[75234]: DEBUG os_brick.utils [None req-acf58aba-d11f-4a3d-ac0b-46d4f62a74a2 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] <== get_connector_properties: return (125ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:dd1368c5ba2c', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:203}}
Nov 29 03:25:39 user nova-compute[75234]: DEBUG nova.virt.block_device [None req-acf58aba-d11f-4a3d-ac0b-46d4f62a74a2 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] [instance: 5d7d744e-1a5f-40ab-b961-4b58c183efe0] Updating existing volume attachment record: c22690b0-7791-46dd-a849-00a4321d5278 {{(pid=75234) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Nov 29 03:25:39 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:25:43 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-62b72a41-ee1e-42e2-b155-631bd04b679f tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Acquiring lock "f6bc212d-85cd-479c-92ae-112c9f0101bc" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:25:43 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-62b72a41-ee1e-42e2-b155-631bd04b679f tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Lock "f6bc212d-85cd-479c-92ae-112c9f0101bc" acquired by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:25:43 user nova-compute[75234]: DEBUG nova.objects.instance [None req-62b72a41-ee1e-42e2-b155-631bd04b679f tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Lazy-loading 'flavor' on Instance uuid f6bc212d-85cd-479c-92ae-112c9f0101bc {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:25:43 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-62b72a41-ee1e-42e2-b155-631bd04b679f tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Lock "f6bc212d-85cd-479c-92ae-112c9f0101bc" "released" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: held 0.055s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:25:43 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-62b72a41-ee1e-42e2-b155-631bd04b679f tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Acquiring lock "f6bc212d-85cd-479c-92ae-112c9f0101bc" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:25:43 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-62b72a41-ee1e-42e2-b155-631bd04b679f tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Lock "f6bc212d-85cd-479c-92ae-112c9f0101bc" acquired by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:25:43 user nova-compute[75234]: INFO nova.compute.manager [None req-62b72a41-ee1e-42e2-b155-631bd04b679f tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] [instance: f6bc212d-85cd-479c-92ae-112c9f0101bc] Attaching volume ee672802-fcc5-4892-8a23-91763600da13 to /dev/vdb
Nov 29 03:25:43 user nova-compute[75234]: DEBUG os_brick.utils [None req-62b72a41-ee1e-42e2-b155-631bd04b679f tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:176}}
Nov 29 03:25:43 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:25:43 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.015s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:25:43 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[1718be8d-8ced-4773-b305-e30552fd5011]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:dd1368c5ba2c\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:25:43 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt -v / -n -o SOURCE {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:25:43 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt -v / -n -o SOURCE" returned: 0 in 0.017s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:25:43 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[2e808bc4-956d-423d-b30b-bcec2b7694ae]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:25:43 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:25:43 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.019s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:25:43 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[5c388eea-f204-4fa6-9f28-7f4ba483f55c]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:25:43 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[7fc50c4c-f27d-4090-8311-fa0d7f09a3e5]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:25:43 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-62b72a41-ee1e-42e2-b155-631bd04b679f tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Running cmd (subprocess): nvme version {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:25:43 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-62b72a41-ee1e-42e2-b155-631bd04b679f tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] 'nvme version' failed. Not Retrying. {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:473}}
Nov 29 03:25:43 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.nvmeof [None req-62b72a41-ee1e-42e2-b155-631bd04b679f tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] nvme not present on system {{(pid=75234) nvme_present /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/nvmeof.py:756}}
Nov 29 03:25:43 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:25:43 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:473}}
Nov 29 03:25:43 user nova-compute[75234]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Nov 29 03:25:43 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[2fa8d19b-c00a-4bb1-a7bd-188a3ed0adb0]: (4, '') {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:25:43 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.lightos [None req-62b72a41-ee1e-42e2-b155-631bd04b679f tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=75234) find_dsc /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/lightos.py:98}}
Nov 29 03:25:43 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.lightos [None req-62b72a41-ee1e-42e2-b155-631bd04b679f tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=75234) get_connector_properties /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/lightos.py:76}}
Nov 29 03:25:43 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.lightos [None req-62b72a41-ee1e-42e2-b155-631bd04b679f tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] LIGHTOS: no hostnqn found. {{(pid=75234) get_connector_properties /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/lightos.py:84}}
Nov 29 03:25:43 user nova-compute[75234]: DEBUG os_brick.utils [None req-62b72a41-ee1e-42e2-b155-631bd04b679f tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] <== get_connector_properties: return (124ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:dd1368c5ba2c', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:203}}
Nov 29 03:25:43 user nova-compute[75234]: DEBUG nova.virt.block_device [None req-62b72a41-ee1e-42e2-b155-631bd04b679f tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] [instance: f6bc212d-85cd-479c-92ae-112c9f0101bc] Updating existing volume attachment record: 4bc3302c-38ba-48bb-9910-9f08d271adce {{(pid=75234) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Nov 29 03:25:44 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:25:45 user nova-compute[75234]: INFO nova.compute.manager [None req-b9b9d25d-5b47-43a7-8141-8b1b683c24b8 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] [instance: d5096ca2-d005-45ac-a487-11a810446859] Rescuing
Nov 29 03:25:45 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-b9b9d25d-5b47-43a7-8141-8b1b683c24b8 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Acquiring lock "refresh_cache-d5096ca2-d005-45ac-a487-11a810446859" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:25:45 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-b9b9d25d-5b47-43a7-8141-8b1b683c24b8 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Acquired lock "refresh_cache-d5096ca2-d005-45ac-a487-11a810446859" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:25:45 user nova-compute[75234]: DEBUG nova.network.neutron [None req-b9b9d25d-5b47-43a7-8141-8b1b683c24b8 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] [instance: d5096ca2-d005-45ac-a487-11a810446859] Building network info cache for instance {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2009}}
Nov 29 03:25:46 user nova-compute[75234]: DEBUG nova.network.neutron [None req-b9b9d25d-5b47-43a7-8141-8b1b683c24b8 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] [instance: d5096ca2-d005-45ac-a487-11a810446859] Updating instance_info_cache with network_info: [{"id": "9266d021-6e83-4bfa-8151-66c5182e61a0", "address": "fa:16:3e:ec:fb:67", "network": {"id": "4d086e08-7798-4cfe-9365-86be6f0a52d2", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-305624008-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "db00bcd0c82c44f1bda739e9857f6e84", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap9266d021-6e", "ovs_interfaceid": "9266d021-6e83-4bfa-8151-66c5182e61a0", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:25:46 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-b9b9d25d-5b47-43a7-8141-8b1b683c24b8 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Releasing lock "refresh_cache-d5096ca2-d005-45ac-a487-11a810446859" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:25:46 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:25:46 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:25:46 user nova-compute[75234]: DEBUG nova.compute.manager [req-b2a4baf4-b4f0-4778-8588-04061e00e4c6 req-9c5aa9bd-890d-4b7a-8de2-f1a1b8c705d3 service nova] [instance: d5096ca2-d005-45ac-a487-11a810446859] Received event network-vif-unplugged-9266d021-6e83-4bfa-8151-66c5182e61a0 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:25:46 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-b2a4baf4-b4f0-4778-8588-04061e00e4c6 req-9c5aa9bd-890d-4b7a-8de2-f1a1b8c705d3 service nova] Acquiring lock "d5096ca2-d005-45ac-a487-11a810446859-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:25:46 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-b2a4baf4-b4f0-4778-8588-04061e00e4c6 req-9c5aa9bd-890d-4b7a-8de2-f1a1b8c705d3 service nova] Lock "d5096ca2-d005-45ac-a487-11a810446859-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:25:46 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-b2a4baf4-b4f0-4778-8588-04061e00e4c6 req-9c5aa9bd-890d-4b7a-8de2-f1a1b8c705d3 service nova] Lock "d5096ca2-d005-45ac-a487-11a810446859-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:25:46 user nova-compute[75234]: DEBUG nova.compute.manager [req-b2a4baf4-b4f0-4778-8588-04061e00e4c6 req-9c5aa9bd-890d-4b7a-8de2-f1a1b8c705d3 service nova] [instance: d5096ca2-d005-45ac-a487-11a810446859] No waiting events found dispatching network-vif-unplugged-9266d021-6e83-4bfa-8151-66c5182e61a0 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:25:46 user nova-compute[75234]: WARNING nova.compute.manager [req-b2a4baf4-b4f0-4778-8588-04061e00e4c6 req-9c5aa9bd-890d-4b7a-8de2-f1a1b8c705d3 service nova] [instance: d5096ca2-d005-45ac-a487-11a810446859] Received unexpected event network-vif-unplugged-9266d021-6e83-4bfa-8151-66c5182e61a0 for instance with vm_state active and task_state rescuing.
Nov 29 03:25:46 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:25:46 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:25:46 user nova-compute[75234]: INFO nova.virt.libvirt.driver [-] [instance: d5096ca2-d005-45ac-a487-11a810446859] Instance destroyed successfully.
Nov 29 03:25:46 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-b9b9d25d-5b47-43a7-8141-8b1b683c24b8 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] [instance: d5096ca2-d005-45ac-a487-11a810446859] Attempting a stable device rescue
Nov 29 03:25:47 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-b9b9d25d-5b47-43a7-8141-8b1b683c24b8 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] rescue generated disk_info: {'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, '/dev/vda': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk.rescue': {'bus': 'virtio', 'dev': 'vdb', 'type': 'disk'}}} {{(pid=75234) rescue /opt/stack/nova/nova/virt/libvirt/driver.py:4318}}
Nov 29 03:25:47 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-b9b9d25d-5b47-43a7-8141-8b1b683c24b8 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] [instance: d5096ca2-d005-45ac-a487-11a810446859] Instance directory exists: not creating {{(pid=75234) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4723}}
Nov 29 03:25:47 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-b9b9d25d-5b47-43a7-8141-8b1b683c24b8 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] [instance: d5096ca2-d005-45ac-a487-11a810446859] Creating image(s)
Nov 29 03:25:47 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-b9b9d25d-5b47-43a7-8141-8b1b683c24b8 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Acquiring lock "/opt/stack/data/nova/instances/d5096ca2-d005-45ac-a487-11a810446859/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:25:47 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-b9b9d25d-5b47-43a7-8141-8b1b683c24b8 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Lock "/opt/stack/data/nova/instances/d5096ca2-d005-45ac-a487-11a810446859/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:25:47 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-b9b9d25d-5b47-43a7-8141-8b1b683c24b8 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Lock "/opt/stack/data/nova/instances/d5096ca2-d005-45ac-a487-11a810446859/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:25:47 user nova-compute[75234]: DEBUG nova.objects.instance [None req-b9b9d25d-5b47-43a7-8141-8b1b683c24b8 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Lazy-loading 'trusted_certs' on Instance uuid d5096ca2-d005-45ac-a487-11a810446859 {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:25:47 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-b9b9d25d-5b47-43a7-8141-8b1b683c24b8 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Acquiring lock "f30afef49830ef804887cdc7e9ea0848425eefd7" by "nova.virt.libvirt.imagebackend.Image.cache.<locals>.fetch_func_sync" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:25:47 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-b9b9d25d-5b47-43a7-8141-8b1b683c24b8 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Lock "f30afef49830ef804887cdc7e9ea0848425eefd7" acquired by "nova.virt.libvirt.imagebackend.Image.cache.<locals>.fetch_func_sync" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:25:47 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-acf58aba-d11f-4a3d-ac0b-46d4f62a74a2 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] [instance: 5d7d744e-1a5f-40ab-b961-4b58c183efe0] Calling os-brick to attach iSCSI Volume {{(pid=75234) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Nov 29 03:25:47 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-acf58aba-d11f-4a3d-ac0b-46d4f62a74a2 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f8a5050c9a0>, 'connection_properties': {'target_iqn': 'iqn.2010-10.org.openstack:65acc1f5-8dbf-457c-a978-510436bccf11', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'jqCadzEM', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:176}}
Nov 29 03:25:47 user nova-compute[75234]: WARNING os_brick.initiator.connectors.base [None req-acf58aba-d11f-4a3d-ac0b-46d4f62a74a2 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Nov 29 03:25:47 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-acf58aba-d11f-4a3d-ac0b-46d4f62a74a2 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:68}}
Nov 29 03:25:47 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-acf58aba-d11f-4a3d-ac0b-46d4f62a74a2 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.002s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:73}}
Nov 29 03:25:47 user nova-compute[75234]: WARNING os_brick.initiator.connectors.base [None req-acf58aba-d11f-4a3d-ac0b-46d4f62a74a2 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Nov 29 03:25:47 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-acf58aba-d11f-4a3d-ac0b-46d4f62a74a2 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:65acc1f5-8dbf-457c-a978-510436bccf11" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:68}}
Nov 29 03:25:47 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-acf58aba-d11f-4a3d-ac0b-46d4f62a74a2 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:65acc1f5-8dbf-457c-a978-510436bccf11" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:73}}
Nov 29 03:25:47 user nova-compute[75234]: INFO os_brick.initiator.connectors.iscsi [None req-acf58aba-d11f-4a3d-ac0b-46d4f62a74a2 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Nov 29 03:25:48 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-b9b9d25d-5b47-43a7-8141-8b1b683c24b8 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f30afef49830ef804887cdc7e9ea0848425eefd7.part --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:25:48 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:65acc1f5-8dbf-457c-a978-510436bccf11 -p 172.16.0.220:3260 {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:25:48 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:65acc1f5-8dbf-457c-a978-510436bccf11 -p 172.16.0.220:3260" returned: 21 in 0.032s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:25:48 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[db8ed429-e4bc-47a5-8b3e-01572155d12f]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:25:48 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:25:48 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-acf58aba-d11f-4a3d-ac0b-46d4f62a74a2 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Nov 29 03:25:48 user nova-compute[75234]:  {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:25:48 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:65acc1f5-8dbf-457c-a978-510436bccf11 -p 172.16.0.220:3260 --interface default --op new {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:25:48 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:65acc1f5-8dbf-457c-a978-510436bccf11 -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.020s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:25:48 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[dce0b509-96b2-468b-baeb-8f59767d3857]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:65acc1f5-8dbf-457c-a978-510436bccf11] added\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:25:48 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-acf58aba-d11f-4a3d-ac0b-46d4f62a74a2 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:65acc1f5-8dbf-457c-a978-510436bccf11] added
Nov 29 03:25:48 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:25:48 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:65acc1f5-8dbf-457c-a978-510436bccf11 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:25:48 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:65acc1f5-8dbf-457c-a978-510436bccf11 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.025s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:25:48 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[dead5707-c79c-450c-8bee-849f95af3691]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:25:48 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-acf58aba-d11f-4a3d-ac0b-46d4f62a74a2 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:25:48 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:65acc1f5-8dbf-457c-a978-510436bccf11 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:25:48 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:65acc1f5-8dbf-457c-a978-510436bccf11 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.027s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:25:48 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[73ee8b65-3ab1-4dc2-be43-a1a636f9ba20]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:25:48 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-acf58aba-d11f-4a3d-ac0b-46d4f62a74a2 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:25:48 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:65acc1f5-8dbf-457c-a978-510436bccf11 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v jqCadzEM {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:25:48 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:65acc1f5-8dbf-457c-a978-510436bccf11 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v jqCadzEM" returned: 0 in 0.018s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:25:48 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[55283ea1-1794-4cb8-94ef-0f707a16b2fe]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:25:48 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-acf58aba-d11f-4a3d-ac0b-46d4f62a74a2 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'jqCadzEM'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:25:48 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:65acc1f5-8dbf-457c-a978-510436bccf11 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:25:48 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:65acc1f5-8dbf-457c-a978-510436bccf11 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.028s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:25:48 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[ac67a87a-9678-4868-8c47-7410fae682c2]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:25:48 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-acf58aba-d11f-4a3d-ac0b-46d4f62a74a2 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:25:48 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:25:48 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-b9b9d25d-5b47-43a7-8141-8b1b683c24b8 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f30afef49830ef804887cdc7e9ea0848425eefd7.part --force-share --output=json" returned: 0 in 0.197s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:25:48 user nova-compute[75234]: DEBUG nova.virt.images [None req-b9b9d25d-5b47-43a7-8141-8b1b683c24b8 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] a2efcc3e-53b8-4d95-8340-8ec22f580f16 was qcow2, converting to raw {{(pid=75234) fetch_to_raw /opt/stack/nova/nova/virt/images.py:165}}
Nov 29 03:25:48 user nova-compute[75234]: DEBUG nova.privsep.utils [None req-b9b9d25d-5b47-43a7-8141-8b1b683c24b8 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Path '/opt/stack/data/nova/instances' supports direct I/O {{(pid=75234) supports_direct_io /opt/stack/nova/nova/privsep/utils.py:63}}
Nov 29 03:25:48 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-b9b9d25d-5b47-43a7-8141-8b1b683c24b8 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Running cmd (subprocess): qemu-img convert -t none -O raw -f qcow2 /opt/stack/data/nova/instances/_base/f30afef49830ef804887cdc7e9ea0848425eefd7.part /opt/stack/data/nova/instances/_base/f30afef49830ef804887cdc7e9ea0848425eefd7.converted {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:25:48 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.020s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:25:48 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[2f1d91eb-0cae-43a9-a50f-8a39ca6981de]: (4, ('tcp: [42] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0 (non-flash)\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:25:48 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-acf58aba-d11f-4a3d-ac0b-46d4f62a74a2 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [42] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0 (non-flash)
Nov 29 03:25:48 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:25:48 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-acf58aba-d11f-4a3d-ac0b-46d4f62a74a2 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] iscsi session list stdout=tcp: [42] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0 (non-flash)
Nov 29 03:25:48 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsi_session /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Nov 29 03:25:48 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:65acc1f5-8dbf-457c-a978-510436bccf11 -p 172.16.0.220:3260 --login {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:25:48 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:65acc1f5-8dbf-457c-a978-510436bccf11 -p 172.16.0.220:3260 --login" returned: 0 in 0.084s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:25:48 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[60ad31b1-d8e9-4f4a-b6e6-5fa0a8c0ff08]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:65acc1f5-8dbf-457c-a978-510436bccf11, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:65acc1f5-8dbf-457c-a978-510436bccf11, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:25:48 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-acf58aba-d11f-4a3d-ac0b-46d4f62a74a2 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:65acc1f5-8dbf-457c-a978-510436bccf11, portal: 172.16.0.220,3260]
Nov 29 03:25:48 user nova-compute[75234]: Login to [iface: default, target: iqn.2010-10.org.openstack:65acc1f5-8dbf-457c-a978-510436bccf11, portal: 172.16.0.220,3260] successful.
Nov 29 03:25:48 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:25:48 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:65acc1f5-8dbf-457c-a978-510436bccf11 -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:25:48 user nova-compute[75234]: DEBUG nova.compute.manager [req-f4d7e9e2-00d2-49fe-b21b-3138c1c216cb req-bf357468-4ff6-40b5-8212-bb55bbfb100a service nova] [instance: d5096ca2-d005-45ac-a487-11a810446859] Received event network-vif-plugged-9266d021-6e83-4bfa-8151-66c5182e61a0 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:25:48 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-f4d7e9e2-00d2-49fe-b21b-3138c1c216cb req-bf357468-4ff6-40b5-8212-bb55bbfb100a service nova] Acquiring lock "d5096ca2-d005-45ac-a487-11a810446859-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:25:48 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-f4d7e9e2-00d2-49fe-b21b-3138c1c216cb req-bf357468-4ff6-40b5-8212-bb55bbfb100a service nova] Lock "d5096ca2-d005-45ac-a487-11a810446859-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:25:48 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-f4d7e9e2-00d2-49fe-b21b-3138c1c216cb req-bf357468-4ff6-40b5-8212-bb55bbfb100a service nova] Lock "d5096ca2-d005-45ac-a487-11a810446859-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:25:48 user nova-compute[75234]: DEBUG nova.compute.manager [req-f4d7e9e2-00d2-49fe-b21b-3138c1c216cb req-bf357468-4ff6-40b5-8212-bb55bbfb100a service nova] [instance: d5096ca2-d005-45ac-a487-11a810446859] No waiting events found dispatching network-vif-plugged-9266d021-6e83-4bfa-8151-66c5182e61a0 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:25:48 user nova-compute[75234]: WARNING nova.compute.manager [req-f4d7e9e2-00d2-49fe-b21b-3138c1c216cb req-bf357468-4ff6-40b5-8212-bb55bbfb100a service nova] [instance: d5096ca2-d005-45ac-a487-11a810446859] Received unexpected event network-vif-plugged-9266d021-6e83-4bfa-8151-66c5182e61a0 for instance with vm_state active and task_state rescuing.
Nov 29 03:25:48 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:65acc1f5-8dbf-457c-a978-510436bccf11 -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.028s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:25:48 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[e9f9f192-d61b-4a5b-8b69-e3cc267f8b6e]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:25:48 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-acf58aba-d11f-4a3d-ac0b-46d4f62a74a2 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:25:48 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:25:48 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.021s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:25:48 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[da2db83d-3278-48e0-acd0-b7ace45dc6ee]: (4, ('tcp: [42] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0 (non-flash)\ntcp: [45] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:65acc1f5-8dbf-457c-a978-510436bccf11 (non-flash)\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:25:48 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-acf58aba-d11f-4a3d-ac0b-46d4f62a74a2 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [42] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0 (non-flash)
Nov 29 03:25:48 user nova-compute[75234]: tcp: [45] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:65acc1f5-8dbf-457c-a978-510436bccf11 (non-flash)
Nov 29 03:25:48 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:25:48 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-acf58aba-d11f-4a3d-ac0b-46d4f62a74a2 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] iscsi session list stdout=tcp: [42] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0 (non-flash)
Nov 29 03:25:48 user nova-compute[75234]: tcp: [45] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:65acc1f5-8dbf-457c-a978-510436bccf11 (non-flash)
Nov 29 03:25:48 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsi_session /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Nov 29 03:25:48 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-acf58aba-d11f-4a3d-ac0b-46d4f62a74a2 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:65acc1f5-8dbf-457c-a978-510436bccf11" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 1.409s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:87}}
Nov 29 03:25:48 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-acf58aba-d11f-4a3d-ac0b-46d4f62a74a2 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Connected to 172.16.0.220:3260 {{(pid=75234) _connect_vol /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:644}}
Nov 29 03:25:48 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-acf58aba-d11f-4a3d-ac0b-46d4f62a74a2 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] HCTL ('33', '-', '-', 0) found on session 45 with lun 0 {{(pid=75234) get_hctl /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:770}}
Nov 29 03:25:48 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-acf58aba-d11f-4a3d-ac0b-46d4f62a74a2 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Scanning host 33 c: -, t: -, l: 0) {{(pid=75234) scan_iscsi /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:795}}
Nov 29 03:25:48 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host33/scan {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:25:48 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host33/scan" returned: 0 in 0.035s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:25:48 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[ac99528f-2da5-4b48-999c-17fd2e1acd36]: (4, ('- - 0', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:25:48 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-acf58aba-d11f-4a3d-ac0b-46d4f62a74a2 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Searching for a device in session 45 and hctl ['33', '*', '*', 0] yield: sdb {{(pid=75234) device_name_by_hctl /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:789}}
Nov 29 03:25:48 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-acf58aba-d11f-4a3d-ac0b-46d4f62a74a2 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Connected to sdb using {'target_iqn': 'iqn.2010-10.org.openstack:65acc1f5-8dbf-457c-a978-510436bccf11', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'jqCadzEM', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False} {{(pid=75234) _connect_vol /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:672}}
Nov 29 03:25:48 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-b9b9d25d-5b47-43a7-8141-8b1b683c24b8 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] CMD "qemu-img convert -t none -O raw -f qcow2 /opt/stack/data/nova/instances/_base/f30afef49830ef804887cdc7e9ea0848425eefd7.part /opt/stack/data/nova/instances/_base/f30afef49830ef804887cdc7e9ea0848425eefd7.converted" returned: 0 in 0.407s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:25:48 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-b9b9d25d-5b47-43a7-8141-8b1b683c24b8 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f30afef49830ef804887cdc7e9ea0848425eefd7.converted --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:25:49 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-b9b9d25d-5b47-43a7-8141-8b1b683c24b8 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f30afef49830ef804887cdc7e9ea0848425eefd7.converted --force-share --output=json" returned: 0 in 0.180s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:25:49 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-b9b9d25d-5b47-43a7-8141-8b1b683c24b8 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Lock "f30afef49830ef804887cdc7e9ea0848425eefd7" "released" by "nova.virt.libvirt.imagebackend.Image.cache.<locals>.fetch_func_sync" :: held 1.992s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:25:49 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-b9b9d25d-5b47-43a7-8141-8b1b683c24b8 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Acquiring lock "f30afef49830ef804887cdc7e9ea0848425eefd7" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:25:49 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-b9b9d25d-5b47-43a7-8141-8b1b683c24b8 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Lock "f30afef49830ef804887cdc7e9ea0848425eefd7" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:25:49 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-b9b9d25d-5b47-43a7-8141-8b1b683c24b8 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f30afef49830ef804887cdc7e9ea0848425eefd7 --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:25:49 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-b9b9d25d-5b47-43a7-8141-8b1b683c24b8 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/f30afef49830ef804887cdc7e9ea0848425eefd7 --force-share --output=json" returned: 0 in 0.180s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:25:49 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-b9b9d25d-5b47-43a7-8141-8b1b683c24b8 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/f30afef49830ef804887cdc7e9ea0848425eefd7,backing_fmt=raw /opt/stack/data/nova/instances/d5096ca2-d005-45ac-a487-11a810446859/disk.rescue {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:25:49 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-b9b9d25d-5b47-43a7-8141-8b1b683c24b8 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/f30afef49830ef804887cdc7e9ea0848425eefd7,backing_fmt=raw /opt/stack/data/nova/instances/d5096ca2-d005-45ac-a487-11a810446859/disk.rescue" returned: 0 in 0.057s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:25:49 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-b9b9d25d-5b47-43a7-8141-8b1b683c24b8 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Lock "f30afef49830ef804887cdc7e9ea0848425eefd7" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.244s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:25:49 user nova-compute[75234]: DEBUG nova.objects.instance [None req-b9b9d25d-5b47-43a7-8141-8b1b683c24b8 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Lazy-loading 'migration_context' on Instance uuid d5096ca2-d005-45ac-a487-11a810446859 {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:25:49 user nova-compute[75234]: DEBUG nova.objects.base [None req-b9b9d25d-5b47-43a7-8141-8b1b683c24b8 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Object Instance<d5096ca2-d005-45ac-a487-11a810446859> lazy-loaded attributes: trusted_certs,migration_context {{(pid=75234) wrapper /opt/stack/nova/nova/objects/base.py:126}}
Nov 29 03:25:49 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-b9b9d25d-5b47-43a7-8141-8b1b683c24b8 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] [instance: d5096ca2-d005-45ac-a487-11a810446859] Created local disks {{(pid=75234) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4861}}
Nov 29 03:25:49 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-b9b9d25d-5b47-43a7-8141-8b1b683c24b8 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] [instance: d5096ca2-d005-45ac-a487-11a810446859] Start _get_guest_xml network_info=[{"id": "9266d021-6e83-4bfa-8151-66c5182e61a0", "address": "fa:16:3e:ec:fb:67", "network": {"id": "4d086e08-7798-4cfe-9365-86be6f0a52d2", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-305624008-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [], "label": "tempest-ServerBootFromVolumeStableRescueTest-305624008-network", "vif_mac": "fa:16:3e:ec:fb:67"}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "db00bcd0c82c44f1bda739e9857f6e84", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap9266d021-6e", "ovs_interfaceid": "9266d021-6e83-4bfa-8151-66c5182e61a0", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, '/dev/vda': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk.rescue': {'bus': 'virtio', 'dev': 'vdb', 'type': 'disk'}}} image_meta=ImageMeta(checksum='c8fc807773e5354afe61636071771906',container_format='bare',created_at=2023-11-29T03:02:15Z,direct_url=<?>,disk_format='qcow2',id=96c30ba7-9b73-4a30-8ba4-7138879573b7,min_disk=0,min_ram=0,name='cirros-0.6.2-x86_64-disk',owner='b18e380921c748b78517b6f1211cd0f1',properties=ImageMetaProps,protected=<?>,size=21430272,status='active',tags=<?>,updated_at=2023-11-29T03:02:17Z,virtual_size=<?>,visibility=<?>) rescue={'image_id': 'a2efcc3e-53b8-4d95-8340-8ec22f580f16', 'kernel_id': '', 'ramdisk_id': ''} block_device_info={'root_device_name': '/dev/vda', 'image': [], 'ephemerals': [], 'block_device_mapping': [{'mount_device': '/dev/vda', 'attachment_id': '8cc2f809-f19c-4d43-94b7-56037905c4d8', 'disk_bus': 'virtio', 'device_type': 'disk', 'boot_index': 0, 'guest_format': None, 'delete_on_termination': False, 'connection_info': {'driver_volume_type': 'iscsi', 'data': {'target_iqn': 'iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'BbOlTYfq', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdc'}, 'status': 'reserved', 'instance': 'd5096ca2-d005-45ac-a487-11a810446859', 'attached_at': '', 'detached_at': '', 'volume_id': 'c12ce787-781c-4e31-adfa-5b51145bc1a0', 'serial': 'c12ce787-781c-4e31-adfa-5b51145bc1a0'}, 'volume_type': None} {{(pid=75234) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7562}}
Nov 29 03:25:49 user nova-compute[75234]: DEBUG nova.objects.instance [None req-b9b9d25d-5b47-43a7-8141-8b1b683c24b8 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Lazy-loading 'resources' on Instance uuid d5096ca2-d005-45ac-a487-11a810446859 {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:25:49 user nova-compute[75234]: DEBUG nova.objects.base [None req-b9b9d25d-5b47-43a7-8141-8b1b683c24b8 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Object Instance<d5096ca2-d005-45ac-a487-11a810446859> lazy-loaded attributes: trusted_certs,migration_context,resources {{(pid=75234) wrapper /opt/stack/nova/nova/objects/base.py:126}}
Nov 29 03:25:49 user nova-compute[75234]: DEBUG nova.objects.instance [None req-b9b9d25d-5b47-43a7-8141-8b1b683c24b8 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Lazy-loading 'numa_topology' on Instance uuid d5096ca2-d005-45ac-a487-11a810446859 {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:25:49 user nova-compute[75234]: DEBUG nova.objects.base [None req-b9b9d25d-5b47-43a7-8141-8b1b683c24b8 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Object Instance<d5096ca2-d005-45ac-a487-11a810446859> lazy-loaded attributes: trusted_certs,migration_context,resources,numa_topology {{(pid=75234) wrapper /opt/stack/nova/nova/objects/base.py:126}}
Nov 29 03:25:49 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-b9b9d25d-5b47-43a7-8141-8b1b683c24b8 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:25:49 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-b9b9d25d-5b47-43a7-8141-8b1b683c24b8 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:25:49 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-b9b9d25d-5b47-43a7-8141-8b1b683c24b8 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Searching host: 'user' for CPU controller through CGroups V1... {{(pid=75234) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1652}}
Nov 29 03:25:49 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-b9b9d25d-5b47-43a7-8141-8b1b683c24b8 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] CPU controller missing on host. {{(pid=75234) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1662}}
Nov 29 03:25:49 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-b9b9d25d-5b47-43a7-8141-8b1b683c24b8 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Searching host: 'user' for CPU controller through CGroups V2... {{(pid=75234) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1671}}
Nov 29 03:25:49 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-b9b9d25d-5b47-43a7-8141-8b1b683c24b8 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] CPU controller found on host. {{(pid=75234) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1678}}
Nov 29 03:25:49 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-b9b9d25d-5b47-43a7-8141-8b1b683c24b8 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=75234) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5400}}
Nov 29 03:25:49 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-b9b9d25d-5b47-43a7-8141-8b1b683c24b8 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-11-29T03:02:04Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='1',id=6,is_public=True,memory_mb=512,name='m1.tiny',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='c8fc807773e5354afe61636071771906',container_format='bare',created_at=2023-11-29T03:02:15Z,direct_url=<?>,disk_format='qcow2',id=96c30ba7-9b73-4a30-8ba4-7138879573b7,min_disk=0,min_ram=0,name='cirros-0.6.2-x86_64-disk',owner='b18e380921c748b78517b6f1211cd0f1',properties=ImageMetaProps,protected=<?>,size=21430272,status='active',tags=<?>,updated_at=2023-11-29T03:02:17Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:563}}
Nov 29 03:25:49 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-b9b9d25d-5b47-43a7-8141-8b1b683c24b8 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Flavor limits 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:348}}
Nov 29 03:25:49 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-b9b9d25d-5b47-43a7-8141-8b1b683c24b8 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Image limits 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:352}}
Nov 29 03:25:49 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-b9b9d25d-5b47-43a7-8141-8b1b683c24b8 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Flavor pref 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:388}}
Nov 29 03:25:49 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-b9b9d25d-5b47-43a7-8141-8b1b683c24b8 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Image pref 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:392}}
Nov 29 03:25:49 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-b9b9d25d-5b47-43a7-8141-8b1b683c24b8 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:430}}
Nov 29 03:25:49 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-b9b9d25d-5b47-43a7-8141-8b1b683c24b8 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:569}}
Nov 29 03:25:49 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-b9b9d25d-5b47-43a7-8141-8b1b683c24b8 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=75234) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:471}}
Nov 29 03:25:49 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-b9b9d25d-5b47-43a7-8141-8b1b683c24b8 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Got 1 possible topologies {{(pid=75234) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:501}}
Nov 29 03:25:49 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-b9b9d25d-5b47-43a7-8141-8b1b683c24b8 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:575}}
Nov 29 03:25:49 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-b9b9d25d-5b47-43a7-8141-8b1b683c24b8 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:577}}
Nov 29 03:25:49 user nova-compute[75234]: DEBUG nova.objects.instance [None req-b9b9d25d-5b47-43a7-8141-8b1b683c24b8 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Lazy-loading 'vcpu_model' on Instance uuid d5096ca2-d005-45ac-a487-11a810446859 {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:25:49 user nova-compute[75234]: DEBUG nova.objects.base [None req-b9b9d25d-5b47-43a7-8141-8b1b683c24b8 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Object Instance<d5096ca2-d005-45ac-a487-11a810446859> lazy-loaded attributes: trusted_certs,migration_context,resources,numa_topology,vcpu_model {{(pid=75234) wrapper /opt/stack/nova/nova/objects/base.py:126}}
Nov 29 03:25:49 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-b9b9d25d-5b47-43a7-8141-8b1b683c24b8 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] [instance: d5096ca2-d005-45ac-a487-11a810446859] Calling os-brick to attach iSCSI Volume {{(pid=75234) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Nov 29 03:25:49 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-b9b9d25d-5b47-43a7-8141-8b1b683c24b8 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f8a5050c9a0>, 'connection_properties': {'target_iqn': 'iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'BbOlTYfq', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path'}}" {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:176}}
Nov 29 03:25:49 user nova-compute[75234]: WARNING os_brick.initiator.connectors.base [None req-b9b9d25d-5b47-43a7-8141-8b1b683c24b8 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Nov 29 03:25:49 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-b9b9d25d-5b47-43a7-8141-8b1b683c24b8 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:68}}
Nov 29 03:25:49 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-acf58aba-d11f-4a3d-ac0b-46d4f62a74a2 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 2.460s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:87}}
Nov 29 03:25:49 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-acf58aba-d11f-4a3d-ac0b-46d4f62a74a2 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] <== connect_volume: return (2463ms) {'type': 'block', 'scsi_wwn': '23236633763636566', 'path': '/dev/sdb'} {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:203}}
Nov 29 03:25:49 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-acf58aba-d11f-4a3d-ac0b-46d4f62a74a2 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] [instance: 5d7d744e-1a5f-40ab-b961-4b58c183efe0] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '23236633763636566', 'path': '/dev/sdb'} {{(pid=75234) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Nov 29 03:25:49 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-b9b9d25d-5b47-43a7-8141-8b1b683c24b8 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.240s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:73}}
Nov 29 03:25:49 user nova-compute[75234]: WARNING os_brick.initiator.connectors.base [None req-b9b9d25d-5b47-43a7-8141-8b1b683c24b8 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Nov 29 03:25:49 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-b9b9d25d-5b47-43a7-8141-8b1b683c24b8 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:68}}
Nov 29 03:25:49 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-b9b9d25d-5b47-43a7-8141-8b1b683c24b8 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:73}}
Nov 29 03:25:49 user nova-compute[75234]: INFO os_brick.initiator.connectors.iscsi [None req-b9b9d25d-5b47-43a7-8141-8b1b683c24b8 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Nov 29 03:25:49 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0 -p 172.16.0.220:3260 {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:25:49 user nova-compute[75234]: DEBUG nova.objects.instance [None req-acf58aba-d11f-4a3d-ac0b-46d4f62a74a2 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Lazy-loading 'flavor' on Instance uuid 5d7d744e-1a5f-40ab-b961-4b58c183efe0 {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:25:49 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0 -p 172.16.0.220:3260" returned: 0 in 0.024s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:25:49 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[66cf9134-4528-49f0-ae79-22c74ebe4ec7]: (4, ('# BEGIN RECORD 2.1.5\nnode.name = iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0\nnode.tpgt = -1\nnode.startup = automatic\nnode.leading_login = No\niface.iscsi_ifacename = default\niface.net_ifacename = <empty>\niface.ipaddress = <empty>\niface.prefix_len = 0\niface.hwaddress = <empty>\niface.transport_name = tcp\niface.initiatorname = <empty>\niface.state = <empty>\niface.vlan_id = 0\niface.vlan_priority = 0\niface.vlan_state = <empty>\niface.iface_num = 0\niface.mtu = 0\niface.port = 0\niface.bootproto = <empty>\niface.subnet_mask = <empty>\niface.gateway = <empty>\niface.dhcp_alt_client_id_state = <empty>\niface.dhcp_alt_client_id = <empty>\niface.dhcp_dns = <empty>\niface.dhcp_learn_iqn = <empty>\niface.dhcp_req_vendor_id_state = <empty>\niface.dhcp_vendor_id_state = <empty>\niface.dhcp_vendor_id = <empty>\niface.dhcp_slp_da = <empty>\niface.fragmentation = <empty>\niface.gratuitous_arp = <empty>\niface.incoming_forwarding = <empty>\niface.tos_state = <empty>\niface.tos = 0\niface.ttl = 0\niface.delayed_ack = <empty>\niface.tcp_nagle = <empty>\niface.tcp_wsf_state = <empty>\niface.tcp_wsf = 0\niface.tcp_timer_scale = 0\niface.tcp_timestamp = <empty>\niface.redirect = <empty>\niface.def_task_mgmt_timeout = 0\niface.header_digest = <empty>\niface.data_digest = <empty>\niface.immediate_data = <empty>\niface.initial_r2t = <empty>\niface.data_seq_inorder = <empty>\niface.data_pdu_inorder = <empty>\niface.erl = 0\niface.max_receive_data_len = 0\niface.first_burst_len = 0\niface.max_outstanding_r2t = 0\niface.max_burst_len = 0\niface.chap_auth = <empty>\niface.bidi_chap = <empty>\niface.strict_login_compliance = <empty>\niface.discovery_auth = <empty>\niface.discovery_logout = <empty>\nnode.discovery_address = <empty>\nnode.discovery_port = 0\nnode.discovery_type = static\nnode.session.initial_cmdsn = 0\nnode.session.initial_login_retry_max = 8\nnode.session.xmit_thread_priority = -20\nnode.session.cmds_max = 128\nnode.session.queue_depth = 32\nnode.session.nr_sessions = 1\nnode.session.auth.authmethod = CHAP\nnode.session.auth.username = BbOlTYfq\nnode.session.auth.password = ********\nnode.session.auth.username_in = <empty>\nnode.session.auth.password_in = <empty>\nnode.session.auth.chap_algs = MD5\nnode.session.timeo.replacement_timeout = 120\nnode.session.err_timeo.abort_timeout = 15\nnode.session.err_timeo.lu_reset_timeout = 30\nnode.session.err_timeo.tgt_reset_timeout = 30\nnode.session.err_timeo.host_reset_timeout = 60\nnode.session.iscsi.FastAbort = Yes\nnode.session.iscsi.InitialR2T = No\nnode.session.iscsi.ImmediateData = Yes\nnode.session.iscsi.FirstBurstLength = 262144\nnode.session.iscsi.MaxBurstLength = 16776192\nnode.session.iscsi.DefaultTime2Retain = 0\nnode.session.iscsi.DefaultTime2Wait = 2\nnode.session.iscsi.MaxConnections = 1\nnode.session.iscsi.MaxOutstandingR2T = 1\nnode.session.iscsi.ERL = 0\nnode.session.scan = manual\nnode.session.reopen_max = 0\nnode.conn[0].address = 172.16.0.220\nnode.conn[0].port = 3260\nnode.conn[0].startup = manual\nnode.conn[0].tcp.window_size = 524288\nnode.conn[0].tcp.type_of_service = 0\nnode.conn[0].timeo.logout_timeout = 15\nnode.conn[0].timeo.login_timeout = 15\nnode.conn[0].timeo.auth_timeout = 45\nnode.conn[0].timeo.noop_out_interval = 5\nnode.conn[0].timeo.noop_out_timeout = 5\nnode.conn[0].iscsi.MaxXmitDataSegmentLength = 0\nnode.conn[0].iscsi.MaxRecvDataSegmentLength = 262144\nnode.conn[0].iscsi.HeaderDigest = None\nnode.conn[0].iscsi.DataDigest = None\nnode.conn[0].iscsi.IFMarker = No\nnode.conn[0].iscsi.OFMarker = No\n# END RECORD\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:25:49 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-b9b9d25d-5b47-43a7-8141-8b1b683c24b8 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] iscsiadm (): stdout=# BEGIN RECORD 2.1.5
Nov 29 03:25:49 user nova-compute[75234]: node.name = iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0
Nov 29 03:25:49 user nova-compute[75234]: node.tpgt = -1
Nov 29 03:25:49 user nova-compute[75234]: node.startup = automatic
Nov 29 03:25:49 user nova-compute[75234]: node.leading_login = No
Nov 29 03:25:49 user nova-compute[75234]: iface.iscsi_ifacename = default
Nov 29 03:25:49 user nova-compute[75234]: iface.net_ifacename = <empty>
Nov 29 03:25:49 user nova-compute[75234]: iface.ipaddress = <empty>
Nov 29 03:25:49 user nova-compute[75234]: iface.prefix_len = 0
Nov 29 03:25:49 user nova-compute[75234]: iface.hwaddress = <empty>
Nov 29 03:25:49 user nova-compute[75234]: iface.transport_name = tcp
Nov 29 03:25:49 user nova-compute[75234]: iface.initiatorname = <empty>
Nov 29 03:25:49 user nova-compute[75234]: iface.state = <empty>
Nov 29 03:25:49 user nova-compute[75234]: iface.vlan_id = 0
Nov 29 03:25:49 user nova-compute[75234]: iface.vlan_priority = 0
Nov 29 03:25:49 user nova-compute[75234]: iface.vlan_state = <empty>
Nov 29 03:25:49 user nova-compute[75234]: iface.iface_num = 0
Nov 29 03:25:49 user nova-compute[75234]: iface.mtu = 0
Nov 29 03:25:49 user nova-compute[75234]: iface.port = 0
Nov 29 03:25:49 user nova-compute[75234]: iface.bootproto = <empty>
Nov 29 03:25:49 user nova-compute[75234]: iface.subnet_mask = <empty>
Nov 29 03:25:49 user nova-compute[75234]: iface.gateway = <empty>
Nov 29 03:25:49 user nova-compute[75234]: iface.dhcp_alt_client_id_state = <empty>
Nov 29 03:25:49 user nova-compute[75234]: iface.dhcp_alt_client_id = <empty>
Nov 29 03:25:49 user nova-compute[75234]: iface.dhcp_dns = <empty>
Nov 29 03:25:49 user nova-compute[75234]: iface.dhcp_learn_iqn = <empty>
Nov 29 03:25:49 user nova-compute[75234]: iface.dhcp_req_vendor_id_state = <empty>
Nov 29 03:25:49 user nova-compute[75234]: iface.dhcp_vendor_id_state = <empty>
Nov 29 03:25:49 user nova-compute[75234]: iface.dhcp_vendor_id = <empty>
Nov 29 03:25:49 user nova-compute[75234]: iface.dhcp_slp_da = <empty>
Nov 29 03:25:49 user nova-compute[75234]: iface.fragmentation = <empty>
Nov 29 03:25:49 user nova-compute[75234]: iface.gratuitous_arp = <empty>
Nov 29 03:25:49 user nova-compute[75234]: iface.incoming_forwarding = <empty>
Nov 29 03:25:49 user nova-compute[75234]: iface.tos_state = <empty>
Nov 29 03:25:49 user nova-compute[75234]: iface.tos = 0
Nov 29 03:25:49 user nova-compute[75234]: iface.ttl = 0
Nov 29 03:25:49 user nova-compute[75234]: iface.delayed_ack = <empty>
Nov 29 03:25:49 user nova-compute[75234]: iface.tcp_nagle = <empty>
Nov 29 03:25:49 user nova-compute[75234]: iface.tcp_wsf_state = <empty>
Nov 29 03:25:49 user nova-compute[75234]: iface.tcp_wsf = 0
Nov 29 03:25:49 user nova-compute[75234]: iface.tcp_timer_scale = 0
Nov 29 03:25:49 user nova-compute[75234]: iface.tcp_timestamp = <empty>
Nov 29 03:25:49 user nova-compute[75234]: iface.redirect = <empty>
Nov 29 03:25:49 user nova-compute[75234]: iface.def_task_mgmt_timeout = 0
Nov 29 03:25:49 user nova-compute[75234]: iface.header_digest = <empty>
Nov 29 03:25:49 user nova-compute[75234]: iface.data_digest = <empty>
Nov 29 03:25:49 user nova-compute[75234]: iface.immediate_data = <empty>
Nov 29 03:25:49 user nova-compute[75234]: iface.initial_r2t = <empty>
Nov 29 03:25:49 user nova-compute[75234]: iface.data_seq_inorder = <empty>
Nov 29 03:25:49 user nova-compute[75234]: iface.data_pdu_inorder = <empty>
Nov 29 03:25:49 user nova-compute[75234]: iface.erl = 0
Nov 29 03:25:49 user nova-compute[75234]: iface.max_receive_data_len = 0
Nov 29 03:25:49 user nova-compute[75234]: iface.first_burst_len = 0
Nov 29 03:25:49 user nova-compute[75234]: iface.max_outstanding_r2t = 0
Nov 29 03:25:49 user nova-compute[75234]: iface.max_burst_len = 0
Nov 29 03:25:49 user nova-compute[75234]: iface.chap_auth = <empty>
Nov 29 03:25:49 user nova-compute[75234]: iface.bidi_chap = <empty>
Nov 29 03:25:49 user nova-compute[75234]: iface.strict_login_compliance = <empty>
Nov 29 03:25:49 user nova-compute[75234]: iface.discovery_auth = <empty>
Nov 29 03:25:49 user nova-compute[75234]: iface.discovery_logout = <empty>
Nov 29 03:25:49 user nova-compute[75234]: node.discovery_address = <empty>
Nov 29 03:25:49 user nova-compute[75234]: node.discovery_port = 0
Nov 29 03:25:49 user nova-compute[75234]: node.discovery_type = static
Nov 29 03:25:49 user nova-compute[75234]: node.session.initial_cmdsn = 0
Nov 29 03:25:49 user nova-compute[75234]: node.session.initial_login_retry_max = 8
Nov 29 03:25:49 user nova-compute[75234]: node.session.xmit_thread_priority = -20
Nov 29 03:25:49 user nova-compute[75234]: node.session.cmds_max = 128
Nov 29 03:25:49 user nova-compute[75234]: node.session.queue_depth = 32
Nov 29 03:25:49 user nova-compute[75234]: node.session.nr_sessions = 1
Nov 29 03:25:49 user nova-compute[75234]: node.session.auth.authmethod = CHAP
Nov 29 03:25:49 user nova-compute[75234]: node.session.auth.username = BbOlTYfq
Nov 29 03:25:49 user nova-compute[75234]: node.session.auth.password = ***
Nov 29 03:25:49 user nova-compute[75234]: node.session.auth.username_in = <empty>
Nov 29 03:25:49 user nova-compute[75234]: node.session.auth.password_in = <empty>
Nov 29 03:25:49 user nova-compute[75234]: node.session.auth.chap_algs = MD5
Nov 29 03:25:49 user nova-compute[75234]: node.session.timeo.replacement_timeout = 120
Nov 29 03:25:49 user nova-compute[75234]: node.session.err_timeo.abort_timeout = 15
Nov 29 03:25:49 user nova-compute[75234]: node.session.err_timeo.lu_reset_timeout = 30
Nov 29 03:25:49 user nova-compute[75234]: node.session.err_timeo.tgt_reset_timeout = 30
Nov 29 03:25:49 user nova-compute[75234]: node.session.err_timeo.host_reset_timeout = 60
Nov 29 03:25:49 user nova-compute[75234]: node.session.iscsi.FastAbort = Yes
Nov 29 03:25:49 user nova-compute[75234]: node.session.iscsi.InitialR2T = No
Nov 29 03:25:49 user nova-compute[75234]: node.session.iscsi.ImmediateData = Yes
Nov 29 03:25:49 user nova-compute[75234]: node.session.iscsi.FirstBurstLength = 262144
Nov 29 03:25:49 user nova-compute[75234]: node.session.iscsi.MaxBurstLength = 16776192
Nov 29 03:25:49 user nova-compute[75234]: node.session.iscsi.DefaultTime2Retain = 0
Nov 29 03:25:49 user nova-compute[75234]: node.session.iscsi.DefaultTime2Wait = 2
Nov 29 03:25:49 user nova-compute[75234]: node.session.iscsi.MaxConnections = 1
Nov 29 03:25:49 user nova-compute[75234]: node.session.iscsi.MaxOutstandingR2T = 1
Nov 29 03:25:49 user nova-compute[75234]: node.session.iscsi.ERL = 0
Nov 29 03:25:49 user nova-compute[75234]: node.session.scan = manual
Nov 29 03:25:49 user nova-compute[75234]: node.session.reopen_max = 0
Nov 29 03:25:49 user nova-compute[75234]: node.conn[0].address = 172.16.0.220
Nov 29 03:25:49 user nova-compute[75234]: node.conn[0].port = 3260
Nov 29 03:25:49 user nova-compute[75234]: node.conn[0].startup = manual
Nov 29 03:25:49 user nova-compute[75234]: node.conn[0].tcp.window_size = 524288
Nov 29 03:25:49 user nova-compute[75234]: node.conn[0].tcp.type_of_service = 0
Nov 29 03:25:49 user nova-compute[75234]: node.conn[0].timeo.logout_timeout = 15
Nov 29 03:25:49 user nova-compute[75234]: node.conn[0].timeo.login_timeout = 15
Nov 29 03:25:49 user nova-compute[75234]: node.conn[0].timeo.auth_timeout = 45
Nov 29 03:25:49 user nova-compute[75234]: node.conn[0].timeo.noop_out_interval = 5
Nov 29 03:25:49 user nova-compute[75234]: node.conn[0].timeo.noop_out_timeout = 5
Nov 29 03:25:49 user nova-compute[75234]: node.conn[0].iscsi.MaxXmitDataSegmentLength = 0
Nov 29 03:25:49 user nova-compute[75234]: node.conn[0].iscsi.MaxRecvDataSegmentLength = 262144
Nov 29 03:25:49 user nova-compute[75234]: node.conn[0].iscsi.HeaderDigest = None
Nov 29 03:25:49 user nova-compute[75234]: node.conn[0].iscsi.DataDigest = None
Nov 29 03:25:49 user nova-compute[75234]: node.conn[0].iscsi.IFMarker = No
Nov 29 03:25:49 user nova-compute[75234]: node.conn[0].iscsi.OFMarker = No
Nov 29 03:25:49 user nova-compute[75234]: # END RECORD
Nov 29 03:25:49 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:25:49 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:25:49 user nova-compute[75234]: DEBUG nova.virt.libvirt.guest [None req-acf58aba-d11f-4a3d-ac0b-46d4f62a74a2 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] attach device xml: <disk type="block" device="disk">
Nov 29 03:25:49 user nova-compute[75234]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Nov 29 03:25:49 user nova-compute[75234]:   <source dev="/dev/sdb"/>
Nov 29 03:25:49 user nova-compute[75234]:   <target dev="vdb" bus="virtio"/>
Nov 29 03:25:49 user nova-compute[75234]:   <serial>65acc1f5-8dbf-457c-a978-510436bccf11</serial>
Nov 29 03:25:49 user nova-compute[75234]: </disk>
Nov 29 03:25:49 user nova-compute[75234]:  {{(pid=75234) attach_device /opt/stack/nova/nova/virt/libvirt/guest.py:339}}
Nov 29 03:25:49 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.020s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:25:49 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[18353473-f0c2-49c8-946c-4911d2a74678]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:25:49 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-b9b9d25d-5b47-43a7-8141-8b1b683c24b8 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:25:49 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:25:49 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.028s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:25:49 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[0ad1310b-0408-4ae0-8ed6-b64f6f522181]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:25:49 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-b9b9d25d-5b47-43a7-8141-8b1b683c24b8 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:25:49 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v BbOlTYfq {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:25:49 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v BbOlTYfq" returned: 0 in 0.030s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:25:49 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[2788fc17-2ce5-4eba-8bf9-fc6bd75b0cb7]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:25:49 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-b9b9d25d-5b47-43a7-8141-8b1b683c24b8 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'BbOlTYfq'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:25:49 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:25:49 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.018s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:25:49 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[05231fb1-6d2c-41d1-bb4e-24d647591220]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:25:49 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-b9b9d25d-5b47-43a7-8141-8b1b683c24b8 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:25:49 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:25:49 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.021s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:25:49 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[049355fa-b087-49b1-89ca-456656380fc1]: (4, ('tcp: [42] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0 (non-flash)\ntcp: [45] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:65acc1f5-8dbf-457c-a978-510436bccf11 (non-flash)\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:25:49 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-b9b9d25d-5b47-43a7-8141-8b1b683c24b8 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [42] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0 (non-flash)
Nov 29 03:25:49 user nova-compute[75234]: tcp: [45] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:65acc1f5-8dbf-457c-a978-510436bccf11 (non-flash)
Nov 29 03:25:49 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:25:49 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-b9b9d25d-5b47-43a7-8141-8b1b683c24b8 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] iscsi session list stdout=tcp: [42] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0 (non-flash)
Nov 29 03:25:49 user nova-compute[75234]: tcp: [45] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:65acc1f5-8dbf-457c-a978-510436bccf11 (non-flash)
Nov 29 03:25:49 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsi_session /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Nov 29 03:25:49 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-b9b9d25d-5b47-43a7-8141-8b1b683c24b8 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.169s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:87}}
Nov 29 03:25:49 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-b9b9d25d-5b47-43a7-8141-8b1b683c24b8 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Connected to 172.16.0.220:3260 {{(pid=75234) _connect_vol /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:644}}
Nov 29 03:25:49 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-b9b9d25d-5b47-43a7-8141-8b1b683c24b8 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] HCTL ('34', '0', '0', 0) found on session 42 with lun 0 {{(pid=75234) get_hctl /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:770}}
Nov 29 03:25:49 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-b9b9d25d-5b47-43a7-8141-8b1b683c24b8 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Scanning host 34 c: 0, t: 0, l: 0) {{(pid=75234) scan_iscsi /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:795}}
Nov 29 03:25:49 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host34/scan {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:25:49 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host34/scan" returned: 0 in 0.014s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:25:49 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[eef27fec-446b-4675-9b00-7fd35e063a1b]: (4, ('0 0 0', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:25:49 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-b9b9d25d-5b47-43a7-8141-8b1b683c24b8 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Searching for a device in session 42 and hctl ('34', '0', '0', 0) yield: sdc {{(pid=75234) device_name_by_hctl /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:789}}
Nov 29 03:25:49 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-b9b9d25d-5b47-43a7-8141-8b1b683c24b8 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Connected to sdc using {'target_iqn': 'iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'BbOlTYfq', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path'} {{(pid=75234) _connect_vol /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:672}}
Nov 29 03:25:49 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-b9b9d25d-5b47-43a7-8141-8b1b683c24b8 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 0.192s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:87}}
Nov 29 03:25:49 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-b9b9d25d-5b47-43a7-8141-8b1b683c24b8 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] <== connect_volume: return (431ms) {'type': 'block', 'scsi_wwn': '23433356563626439', 'path': '/dev/sdc'} {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:203}}
Nov 29 03:25:49 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-b9b9d25d-5b47-43a7-8141-8b1b683c24b8 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] [instance: d5096ca2-d005-45ac-a487-11a810446859] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '23433356563626439', 'path': '/dev/sdc'} {{(pid=75234) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Nov 29 03:25:49 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-b9b9d25d-5b47-43a7-8141-8b1b683c24b8 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-11-29T03:23:54Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description=None,display_name='tempest-ServerBootFromVolumeStableRescueTest-server-1791455206',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-serverbootfromvolumestablerescuetest-server-1791455206',id=40,image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data=None,key_name=None,keypairs=<?>,launch_index=0,launched_at=2023-11-29T03:24:21Z,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='db00bcd0c82c44f1bda739e9857f6e84',ramdisk_id='',reservation_id='r-pix7zk6i',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-ServerBootFromVolumeStableRescueTest-1601266069',owner_user_name='tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member'},tags=<?>,task_state='rescuing',terminated_at=None,trusted_certs=None,updated_at=2023-11-29T03:24:22Z,user_data=None,user_id='877aa877f13a4f84ac4725278010aea3',uuid=d5096ca2-d005-45ac-a487-11a810446859,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "9266d021-6e83-4bfa-8151-66c5182e61a0", "address": "fa:16:3e:ec:fb:67", "network": {"id": "4d086e08-7798-4cfe-9365-86be6f0a52d2", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-305624008-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [], "label": "tempest-ServerBootFromVolumeStableRescueTest-305624008-network", "vif_mac": "fa:16:3e:ec:fb:67"}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "db00bcd0c82c44f1bda739e9857f6e84", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap9266d021-6e", "ovs_interfaceid": "9266d021-6e83-4bfa-8151-66c5182e61a0", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=75234) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Nov 29 03:25:49 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-b9b9d25d-5b47-43a7-8141-8b1b683c24b8 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Converting VIF {"id": "9266d021-6e83-4bfa-8151-66c5182e61a0", "address": "fa:16:3e:ec:fb:67", "network": {"id": "4d086e08-7798-4cfe-9365-86be6f0a52d2", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-305624008-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [], "label": "tempest-ServerBootFromVolumeStableRescueTest-305624008-network", "vif_mac": "fa:16:3e:ec:fb:67"}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "db00bcd0c82c44f1bda739e9857f6e84", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap9266d021-6e", "ovs_interfaceid": "9266d021-6e83-4bfa-8151-66c5182e61a0", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:25:49 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-b9b9d25d-5b47-43a7-8141-8b1b683c24b8 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:ec:fb:67,bridge_name='br-int',has_traffic_filtering=True,id=9266d021-6e83-4bfa-8151-66c5182e61a0,network=Network(4d086e08-7798-4cfe-9365-86be6f0a52d2),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap9266d021-6e') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:25:49 user nova-compute[75234]: DEBUG nova.objects.instance [None req-b9b9d25d-5b47-43a7-8141-8b1b683c24b8 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Lazy-loading 'pci_devices' on Instance uuid d5096ca2-d005-45ac-a487-11a810446859 {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:25:49 user nova-compute[75234]: DEBUG nova.objects.base [None req-b9b9d25d-5b47-43a7-8141-8b1b683c24b8 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Object Instance<d5096ca2-d005-45ac-a487-11a810446859> lazy-loaded attributes: trusted_certs,migration_context,resources,numa_topology,vcpu_model,pci_devices {{(pid=75234) wrapper /opt/stack/nova/nova/objects/base.py:126}}
Nov 29 03:25:49 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-b9b9d25d-5b47-43a7-8141-8b1b683c24b8 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] [instance: d5096ca2-d005-45ac-a487-11a810446859] End _get_guest_xml xml=<domain type="kvm">
Nov 29 03:25:49 user nova-compute[75234]:   <uuid>d5096ca2-d005-45ac-a487-11a810446859</uuid>
Nov 29 03:25:49 user nova-compute[75234]:   <name>instance-00000028</name>
Nov 29 03:25:49 user nova-compute[75234]:   <memory>524288</memory>
Nov 29 03:25:49 user nova-compute[75234]:   <vcpu>1</vcpu>
Nov 29 03:25:49 user nova-compute[75234]:   <metadata>
Nov 29 03:25:49 user nova-compute[75234]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Nov 29 03:25:49 user nova-compute[75234]:       <nova:package version="0.0.0"/>
Nov 29 03:25:49 user nova-compute[75234]:       <nova:name>tempest-ServerBootFromVolumeStableRescueTest-server-1791455206</nova:name>
Nov 29 03:25:49 user nova-compute[75234]:       <nova:creationTime>2023-11-29 03:25:49</nova:creationTime>
Nov 29 03:25:49 user nova-compute[75234]:       <nova:flavor name="m1.tiny">
Nov 29 03:25:49 user nova-compute[75234]:         <nova:memory>512</nova:memory>
Nov 29 03:25:49 user nova-compute[75234]:         <nova:disk>1</nova:disk>
Nov 29 03:25:49 user nova-compute[75234]:         <nova:swap>0</nova:swap>
Nov 29 03:25:49 user nova-compute[75234]:         <nova:ephemeral>0</nova:ephemeral>
Nov 29 03:25:49 user nova-compute[75234]:         <nova:vcpus>1</nova:vcpus>
Nov 29 03:25:49 user nova-compute[75234]:       </nova:flavor>
Nov 29 03:25:49 user nova-compute[75234]:       <nova:owner>
Nov 29 03:25:49 user nova-compute[75234]:         <nova:user uuid="877aa877f13a4f84ac4725278010aea3">tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member</nova:user>
Nov 29 03:25:49 user nova-compute[75234]:         <nova:project uuid="db00bcd0c82c44f1bda739e9857f6e84">tempest-ServerBootFromVolumeStableRescueTest-1601266069</nova:project>
Nov 29 03:25:49 user nova-compute[75234]:       </nova:owner>
Nov 29 03:25:49 user nova-compute[75234]:       <nova:root type="image" uuid="96c30ba7-9b73-4a30-8ba4-7138879573b7"/>
Nov 29 03:25:49 user nova-compute[75234]:       <nova:ports>
Nov 29 03:25:49 user nova-compute[75234]:         <nova:port uuid="9266d021-6e83-4bfa-8151-66c5182e61a0">
Nov 29 03:25:49 user nova-compute[75234]:           <nova:ip type="fixed" address="10.0.0.8" ipVersion="4"/>
Nov 29 03:25:49 user nova-compute[75234]:         </nova:port>
Nov 29 03:25:49 user nova-compute[75234]:       </nova:ports>
Nov 29 03:25:49 user nova-compute[75234]:     </nova:instance>
Nov 29 03:25:49 user nova-compute[75234]:   </metadata>
Nov 29 03:25:49 user nova-compute[75234]:   <sysinfo type="smbios">
Nov 29 03:25:49 user nova-compute[75234]:     <system>
Nov 29 03:25:49 user nova-compute[75234]:       <entry name="manufacturer">OpenStack Foundation</entry>
Nov 29 03:25:49 user nova-compute[75234]:       <entry name="product">OpenStack Nova</entry>
Nov 29 03:25:49 user nova-compute[75234]:       <entry name="version">0.0.0</entry>
Nov 29 03:25:49 user nova-compute[75234]:       <entry name="serial">d5096ca2-d005-45ac-a487-11a810446859</entry>
Nov 29 03:25:49 user nova-compute[75234]:       <entry name="uuid">d5096ca2-d005-45ac-a487-11a810446859</entry>
Nov 29 03:25:49 user nova-compute[75234]:       <entry name="family">Virtual Machine</entry>
Nov 29 03:25:49 user nova-compute[75234]:     </system>
Nov 29 03:25:49 user nova-compute[75234]:   </sysinfo>
Nov 29 03:25:49 user nova-compute[75234]:   <os>
Nov 29 03:25:49 user nova-compute[75234]:     <type arch="x86_64" machine="pc">hvm</type>
Nov 29 03:25:49 user nova-compute[75234]:     <smbios mode="sysinfo"/>
Nov 29 03:25:49 user nova-compute[75234]:   </os>
Nov 29 03:25:49 user nova-compute[75234]:   <features>
Nov 29 03:25:49 user nova-compute[75234]:     <acpi/>
Nov 29 03:25:49 user nova-compute[75234]:     <apic/>
Nov 29 03:25:49 user nova-compute[75234]:     <vmcoreinfo/>
Nov 29 03:25:49 user nova-compute[75234]:   </features>
Nov 29 03:25:49 user nova-compute[75234]:   <clock offset="utc">
Nov 29 03:25:49 user nova-compute[75234]:     <timer name="pit" tickpolicy="delay"/>
Nov 29 03:25:49 user nova-compute[75234]:     <timer name="rtc" tickpolicy="catchup"/>
Nov 29 03:25:49 user nova-compute[75234]:     <timer name="hpet" present="no"/>
Nov 29 03:25:49 user nova-compute[75234]:   </clock>
Nov 29 03:25:49 user nova-compute[75234]:   <cpu mode="custom" match="exact">
Nov 29 03:25:49 user nova-compute[75234]:     <model>Nehalem</model>
Nov 29 03:25:49 user nova-compute[75234]:     <topology sockets="1" cores="1" threads="1"/>
Nov 29 03:25:49 user nova-compute[75234]:   </cpu>
Nov 29 03:25:49 user nova-compute[75234]:   <devices>
Nov 29 03:25:49 user nova-compute[75234]:     <disk type="block" device="disk">
Nov 29 03:25:49 user nova-compute[75234]:       <driver name="qemu" type="raw" cache="none" io="native"/>
Nov 29 03:25:49 user nova-compute[75234]:       <source dev="/dev/sdc"/>
Nov 29 03:25:49 user nova-compute[75234]:       <target dev="vda" bus="virtio"/>
Nov 29 03:25:49 user nova-compute[75234]:       <serial>c12ce787-781c-4e31-adfa-5b51145bc1a0</serial>
Nov 29 03:25:49 user nova-compute[75234]:     </disk>
Nov 29 03:25:49 user nova-compute[75234]:     <disk type="file" device="disk">
Nov 29 03:25:49 user nova-compute[75234]:       <driver name="qemu" type="qcow2" cache="none"/>
Nov 29 03:25:49 user nova-compute[75234]:       <source file="/opt/stack/data/nova/instances/d5096ca2-d005-45ac-a487-11a810446859/disk.rescue"/>
Nov 29 03:25:49 user nova-compute[75234]:       <target dev="vdb" bus="virtio"/>
Nov 29 03:25:49 user nova-compute[75234]:       <boot order="1"/>
Nov 29 03:25:49 user nova-compute[75234]:     </disk>
Nov 29 03:25:49 user nova-compute[75234]:     <interface type="ethernet">
Nov 29 03:25:49 user nova-compute[75234]:       <mac address="fa:16:3e:ec:fb:67"/>
Nov 29 03:25:49 user nova-compute[75234]:       <model type="virtio"/>
Nov 29 03:25:49 user nova-compute[75234]:       <mtu size="1442"/>
Nov 29 03:25:49 user nova-compute[75234]:       <target dev="tap9266d021-6e"/>
Nov 29 03:25:49 user nova-compute[75234]:     </interface>
Nov 29 03:25:49 user nova-compute[75234]:     <serial type="pty">
Nov 29 03:25:49 user nova-compute[75234]:       <log file="/opt/stack/data/nova/instances/d5096ca2-d005-45ac-a487-11a810446859/console.log" append="off"/>
Nov 29 03:25:49 user nova-compute[75234]:     </serial>
Nov 29 03:25:49 user nova-compute[75234]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Nov 29 03:25:49 user nova-compute[75234]:     <video>
Nov 29 03:25:49 user nova-compute[75234]:       <model type="virtio"/>
Nov 29 03:25:49 user nova-compute[75234]:     </video>
Nov 29 03:25:49 user nova-compute[75234]:     <rng model="virtio">
Nov 29 03:25:49 user nova-compute[75234]:       <backend model="random">/dev/urandom</backend>
Nov 29 03:25:49 user nova-compute[75234]:     </rng>
Nov 29 03:25:49 user nova-compute[75234]:     <controller type="usb" index="0" model="none"/>
Nov 29 03:25:49 user nova-compute[75234]:     <memballoon model="virtio">
Nov 29 03:25:49 user nova-compute[75234]:       <stats period="10"/>
Nov 29 03:25:49 user nova-compute[75234]:     </memballoon>
Nov 29 03:25:49 user nova-compute[75234]:   </devices>
Nov 29 03:25:49 user nova-compute[75234]: </domain>
Nov 29 03:25:49 user nova-compute[75234]:  {{(pid=75234) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7568}}
Nov 29 03:25:49 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:25:49 user nova-compute[75234]: INFO nova.virt.libvirt.driver [-] [instance: d5096ca2-d005-45ac-a487-11a810446859] Instance destroyed successfully.
Nov 29 03:25:50 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-b9b9d25d-5b47-43a7-8141-8b1b683c24b8 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] No BDM found with device name vda, not building metadata. {{(pid=75234) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12089}}
Nov 29 03:25:50 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-b9b9d25d-5b47-43a7-8141-8b1b683c24b8 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] No BDM found with device name vdb, not building metadata. {{(pid=75234) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12089}}
Nov 29 03:25:50 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-b9b9d25d-5b47-43a7-8141-8b1b683c24b8 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] No VIF found with MAC fa:16:3e:ec:fb:67, not building metadata {{(pid=75234) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Nov 29 03:25:50 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-acf58aba-d11f-4a3d-ac0b-46d4f62a74a2 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] No BDM found with device name vda, not building metadata. {{(pid=75234) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12089}}
Nov 29 03:25:50 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-acf58aba-d11f-4a3d-ac0b-46d4f62a74a2 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] No BDM found with device name vdb, not building metadata. {{(pid=75234) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12089}}
Nov 29 03:25:50 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-acf58aba-d11f-4a3d-ac0b-46d4f62a74a2 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] No VIF found with MAC fa:16:3e:2f:fa:59, not building metadata {{(pid=75234) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Nov 29 03:25:50 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-acf58aba-d11f-4a3d-ac0b-46d4f62a74a2 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Lock "5d7d744e-1a5f-40ab-b961-4b58c183efe0" "released" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: held 10.882s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:25:51 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:25:51 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:25:51 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:25:51 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:25:51 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-62b72a41-ee1e-42e2-b155-631bd04b679f tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] [instance: f6bc212d-85cd-479c-92ae-112c9f0101bc] Calling os-brick to attach iSCSI Volume {{(pid=75234) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Nov 29 03:25:51 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-62b72a41-ee1e-42e2-b155-631bd04b679f tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f8a5050c9a0>, 'connection_properties': {'target_iqn': 'iqn.2010-10.org.openstack:ee672802-fcc5-4892-8a23-91763600da13', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'ixauTRwW', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:176}}
Nov 29 03:25:51 user nova-compute[75234]: WARNING os_brick.initiator.connectors.base [None req-62b72a41-ee1e-42e2-b155-631bd04b679f tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Nov 29 03:25:51 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-62b72a41-ee1e-42e2-b155-631bd04b679f tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:68}}
Nov 29 03:25:51 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-62b72a41-ee1e-42e2-b155-631bd04b679f tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:73}}
Nov 29 03:25:51 user nova-compute[75234]: WARNING os_brick.initiator.connectors.base [None req-62b72a41-ee1e-42e2-b155-631bd04b679f tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Nov 29 03:25:51 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-62b72a41-ee1e-42e2-b155-631bd04b679f tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:ee672802-fcc5-4892-8a23-91763600da13" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:68}}
Nov 29 03:25:51 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-62b72a41-ee1e-42e2-b155-631bd04b679f tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:ee672802-fcc5-4892-8a23-91763600da13" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:73}}
Nov 29 03:25:51 user nova-compute[75234]: INFO os_brick.initiator.connectors.iscsi [None req-62b72a41-ee1e-42e2-b155-631bd04b679f tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Nov 29 03:25:51 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:ee672802-fcc5-4892-8a23-91763600da13 -p 172.16.0.220:3260 {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:25:51 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:ee672802-fcc5-4892-8a23-91763600da13 -p 172.16.0.220:3260" returned: 21 in 0.018s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:25:51 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[aa126ebd-c54e-4098-a051-2d10c2e076fe]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:25:51 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-62b72a41-ee1e-42e2-b155-631bd04b679f tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Nov 29 03:25:51 user nova-compute[75234]:  {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:25:51 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:ee672802-fcc5-4892-8a23-91763600da13 -p 172.16.0.220:3260 --interface default --op new {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:25:51 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:ee672802-fcc5-4892-8a23-91763600da13 -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.020s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:25:51 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[91428d12-269e-4be5-8eda-c1f15369f5db]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:ee672802-fcc5-4892-8a23-91763600da13] added\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:25:51 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-62b72a41-ee1e-42e2-b155-631bd04b679f tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:ee672802-fcc5-4892-8a23-91763600da13] added
Nov 29 03:25:51 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:25:51 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:ee672802-fcc5-4892-8a23-91763600da13 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:25:51 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:ee672802-fcc5-4892-8a23-91763600da13 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.024s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:25:51 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[14f987a7-808e-4c4a-9952-a718164ffe2f]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:25:51 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-62b72a41-ee1e-42e2-b155-631bd04b679f tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:25:51 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:ee672802-fcc5-4892-8a23-91763600da13 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:25:51 user nova-compute[75234]: DEBUG nova.compute.manager [req-95e22312-7656-41ff-b133-4c814819316a req-486345eb-55c5-4861-8b95-0663a533b1f1 service nova] [instance: d5096ca2-d005-45ac-a487-11a810446859] Received event network-vif-plugged-9266d021-6e83-4bfa-8151-66c5182e61a0 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:25:51 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-95e22312-7656-41ff-b133-4c814819316a req-486345eb-55c5-4861-8b95-0663a533b1f1 service nova] Acquiring lock "d5096ca2-d005-45ac-a487-11a810446859-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:25:51 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-95e22312-7656-41ff-b133-4c814819316a req-486345eb-55c5-4861-8b95-0663a533b1f1 service nova] Lock "d5096ca2-d005-45ac-a487-11a810446859-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:25:51 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-95e22312-7656-41ff-b133-4c814819316a req-486345eb-55c5-4861-8b95-0663a533b1f1 service nova] Lock "d5096ca2-d005-45ac-a487-11a810446859-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:25:51 user nova-compute[75234]: DEBUG nova.compute.manager [req-95e22312-7656-41ff-b133-4c814819316a req-486345eb-55c5-4861-8b95-0663a533b1f1 service nova] [instance: d5096ca2-d005-45ac-a487-11a810446859] No waiting events found dispatching network-vif-plugged-9266d021-6e83-4bfa-8151-66c5182e61a0 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:25:51 user nova-compute[75234]: WARNING nova.compute.manager [req-95e22312-7656-41ff-b133-4c814819316a req-486345eb-55c5-4861-8b95-0663a533b1f1 service nova] [instance: d5096ca2-d005-45ac-a487-11a810446859] Received unexpected event network-vif-plugged-9266d021-6e83-4bfa-8151-66c5182e61a0 for instance with vm_state active and task_state rescuing.
Nov 29 03:25:51 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:ee672802-fcc5-4892-8a23-91763600da13 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.018s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:25:51 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[8310ab8f-1372-45b7-aca8-e02939868b52]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:25:51 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-62b72a41-ee1e-42e2-b155-631bd04b679f tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:25:51 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:ee672802-fcc5-4892-8a23-91763600da13 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v ixauTRwW {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:25:51 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:ee672802-fcc5-4892-8a23-91763600da13 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v ixauTRwW" returned: 0 in 0.020s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:25:51 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[8fb8e52a-df8c-4f22-ac37-04ad64c059f1]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:25:51 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-62b72a41-ee1e-42e2-b155-631bd04b679f tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'ixauTRwW'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:25:51 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:ee672802-fcc5-4892-8a23-91763600da13 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:25:51 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:ee672802-fcc5-4892-8a23-91763600da13 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.024s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:25:51 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[b99bf555-e6c0-4f74-9604-1f9ca7913f96]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:25:51 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-62b72a41-ee1e-42e2-b155-631bd04b679f tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:25:51 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:25:51 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:25:51 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:25:51 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.025s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:25:51 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[1c8872d8-b2b8-4287-927f-97474d41fb8c]: (4, ('tcp: [42] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0 (non-flash)\ntcp: [45] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:65acc1f5-8dbf-457c-a978-510436bccf11 (non-flash)\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:25:51 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-62b72a41-ee1e-42e2-b155-631bd04b679f tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [42] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0 (non-flash)
Nov 29 03:25:51 user nova-compute[75234]: tcp: [45] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:65acc1f5-8dbf-457c-a978-510436bccf11 (non-flash)
Nov 29 03:25:51 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:25:51 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-62b72a41-ee1e-42e2-b155-631bd04b679f tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] iscsi session list stdout=tcp: [42] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0 (non-flash)
Nov 29 03:25:51 user nova-compute[75234]: tcp: [45] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:65acc1f5-8dbf-457c-a978-510436bccf11 (non-flash)
Nov 29 03:25:51 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsi_session /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Nov 29 03:25:51 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:ee672802-fcc5-4892-8a23-91763600da13 -p 172.16.0.220:3260 --login {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:25:51 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:ee672802-fcc5-4892-8a23-91763600da13 -p 172.16.0.220:3260 --login" returned: 0 in 0.057s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:25:51 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[fa830079-f29a-42ea-acb7-f98d87a0373c]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:ee672802-fcc5-4892-8a23-91763600da13, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:ee672802-fcc5-4892-8a23-91763600da13, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:25:51 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-62b72a41-ee1e-42e2-b155-631bd04b679f tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:ee672802-fcc5-4892-8a23-91763600da13, portal: 172.16.0.220,3260]
Nov 29 03:25:51 user nova-compute[75234]: Login to [iface: default, target: iqn.2010-10.org.openstack:ee672802-fcc5-4892-8a23-91763600da13, portal: 172.16.0.220,3260] successful.
Nov 29 03:25:51 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:25:51 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:ee672802-fcc5-4892-8a23-91763600da13 -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:25:51 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:ee672802-fcc5-4892-8a23-91763600da13 -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.103s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:25:51 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[9ba88570-9134-4943-a755-dfb5ca3ddeb1]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:25:51 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-62b72a41-ee1e-42e2-b155-631bd04b679f tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:25:51 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:25:51 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.023s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:25:51 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[8f77fb07-dadf-46e1-873b-44e0591d9b32]: (4, ('tcp: [42] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0 (non-flash)\ntcp: [45] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:65acc1f5-8dbf-457c-a978-510436bccf11 (non-flash)\ntcp: [46] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ee672802-fcc5-4892-8a23-91763600da13 (non-flash)\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:25:51 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-62b72a41-ee1e-42e2-b155-631bd04b679f tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [42] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0 (non-flash)
Nov 29 03:25:51 user nova-compute[75234]: tcp: [45] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:65acc1f5-8dbf-457c-a978-510436bccf11 (non-flash)
Nov 29 03:25:51 user nova-compute[75234]: tcp: [46] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ee672802-fcc5-4892-8a23-91763600da13 (non-flash)
Nov 29 03:25:51 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:25:51 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-62b72a41-ee1e-42e2-b155-631bd04b679f tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] iscsi session list stdout=tcp: [42] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0 (non-flash)
Nov 29 03:25:51 user nova-compute[75234]: tcp: [45] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:65acc1f5-8dbf-457c-a978-510436bccf11 (non-flash)
Nov 29 03:25:51 user nova-compute[75234]: tcp: [46] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ee672802-fcc5-4892-8a23-91763600da13 (non-flash)
Nov 29 03:25:51 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsi_session /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Nov 29 03:25:51 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-62b72a41-ee1e-42e2-b155-631bd04b679f tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:ee672802-fcc5-4892-8a23-91763600da13" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.428s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:87}}
Nov 29 03:25:51 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-62b72a41-ee1e-42e2-b155-631bd04b679f tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Connected to 172.16.0.220:3260 {{(pid=75234) _connect_vol /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:644}}
Nov 29 03:25:51 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-62b72a41-ee1e-42e2-b155-631bd04b679f tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] HCTL ('35', '-', '-', 0) found on session 46 with lun 0 {{(pid=75234) get_hctl /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:770}}
Nov 29 03:25:51 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-62b72a41-ee1e-42e2-b155-631bd04b679f tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Scanning host 35 c: -, t: -, l: 0) {{(pid=75234) scan_iscsi /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:795}}
Nov 29 03:25:51 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host35/scan {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:25:52 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host35/scan" returned: 0 in 0.020s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:25:52 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[08e2184f-0878-4804-bd6f-5c6674888bb7]: (4, ('- - 0', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:25:52 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-62b72a41-ee1e-42e2-b155-631bd04b679f tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Searching for a device in session 46 and hctl ['35', '*', '*', 0] yield: None {{(pid=75234) device_name_by_hctl /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:789}}
Nov 29 03:25:52 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:25:52 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-62b72a41-ee1e-42e2-b155-631bd04b679f tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Searching for a device in session 46 and hctl ['35', '*', '*', 0] yield: sdd {{(pid=75234) device_name_by_hctl /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:789}}
Nov 29 03:25:52 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-62b72a41-ee1e-42e2-b155-631bd04b679f tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Connected to sdd using {'target_iqn': 'iqn.2010-10.org.openstack:ee672802-fcc5-4892-8a23-91763600da13', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'ixauTRwW', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False} {{(pid=75234) _connect_vol /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:672}}
Nov 29 03:25:52 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-62b72a41-ee1e-42e2-b155-631bd04b679f tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.459s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:87}}
Nov 29 03:25:52 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-62b72a41-ee1e-42e2-b155-631bd04b679f tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] <== connect_volume: return (1460ms) {'type': 'block', 'scsi_wwn': '23535613061393563', 'path': '/dev/sdd'} {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:203}}
Nov 29 03:25:52 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-62b72a41-ee1e-42e2-b155-631bd04b679f tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] [instance: f6bc212d-85cd-479c-92ae-112c9f0101bc] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '23535613061393563', 'path': '/dev/sdd'} {{(pid=75234) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Nov 29 03:25:53 user nova-compute[75234]: DEBUG nova.objects.instance [None req-62b72a41-ee1e-42e2-b155-631bd04b679f tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Lazy-loading 'flavor' on Instance uuid f6bc212d-85cd-479c-92ae-112c9f0101bc {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:25:53 user nova-compute[75234]: DEBUG nova.virt.libvirt.guest [None req-62b72a41-ee1e-42e2-b155-631bd04b679f tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] attach device xml: <disk type="block" device="disk">
Nov 29 03:25:53 user nova-compute[75234]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Nov 29 03:25:53 user nova-compute[75234]:   <source dev="/dev/sdd"/>
Nov 29 03:25:53 user nova-compute[75234]:   <target dev="vdb" bus="virtio"/>
Nov 29 03:25:53 user nova-compute[75234]:   <serial>ee672802-fcc5-4892-8a23-91763600da13</serial>
Nov 29 03:25:53 user nova-compute[75234]: </disk>
Nov 29 03:25:53 user nova-compute[75234]:  {{(pid=75234) attach_device /opt/stack/nova/nova/virt/libvirt/guest.py:339}}
Nov 29 03:25:53 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-62b72a41-ee1e-42e2-b155-631bd04b679f tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] No BDM found with device name vda, not building metadata. {{(pid=75234) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12089}}
Nov 29 03:25:53 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-62b72a41-ee1e-42e2-b155-631bd04b679f tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] No BDM found with device name vdb, not building metadata. {{(pid=75234) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12089}}
Nov 29 03:25:53 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-62b72a41-ee1e-42e2-b155-631bd04b679f tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] No VIF found with MAC fa:16:3e:4e:8e:1d, not building metadata {{(pid=75234) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Nov 29 03:25:53 user nova-compute[75234]: DEBUG nova.compute.manager [req-a253aff3-d3b4-49bd-9210-e59f9e2f093b req-5a0d6259-e7fe-466b-8cb2-3d952afb1272 service nova] [instance: d5096ca2-d005-45ac-a487-11a810446859] Received event network-vif-plugged-9266d021-6e83-4bfa-8151-66c5182e61a0 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:25:53 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-a253aff3-d3b4-49bd-9210-e59f9e2f093b req-5a0d6259-e7fe-466b-8cb2-3d952afb1272 service nova] Acquiring lock "d5096ca2-d005-45ac-a487-11a810446859-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:25:53 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-a253aff3-d3b4-49bd-9210-e59f9e2f093b req-5a0d6259-e7fe-466b-8cb2-3d952afb1272 service nova] Lock "d5096ca2-d005-45ac-a487-11a810446859-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:25:53 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-a253aff3-d3b4-49bd-9210-e59f9e2f093b req-5a0d6259-e7fe-466b-8cb2-3d952afb1272 service nova] Lock "d5096ca2-d005-45ac-a487-11a810446859-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:25:53 user nova-compute[75234]: DEBUG nova.compute.manager [req-a253aff3-d3b4-49bd-9210-e59f9e2f093b req-5a0d6259-e7fe-466b-8cb2-3d952afb1272 service nova] [instance: d5096ca2-d005-45ac-a487-11a810446859] No waiting events found dispatching network-vif-plugged-9266d021-6e83-4bfa-8151-66c5182e61a0 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:25:53 user nova-compute[75234]: WARNING nova.compute.manager [req-a253aff3-d3b4-49bd-9210-e59f9e2f093b req-5a0d6259-e7fe-466b-8cb2-3d952afb1272 service nova] [instance: d5096ca2-d005-45ac-a487-11a810446859] Received unexpected event network-vif-plugged-9266d021-6e83-4bfa-8151-66c5182e61a0 for instance with vm_state active and task_state rescuing.
Nov 29 03:25:53 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-62b72a41-ee1e-42e2-b155-631bd04b679f tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Lock "f6bc212d-85cd-479c-92ae-112c9f0101bc" "released" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: held 9.902s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:25:54 user nova-compute[75234]: DEBUG nova.compute.manager [None req-b9b9d25d-5b47-43a7-8141-8b1b683c24b8 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] [instance: d5096ca2-d005-45ac-a487-11a810446859] Checking state {{(pid=75234) _get_power_state /opt/stack/nova/nova/compute/manager.py:1782}}
Nov 29 03:25:54 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:25:55 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-9711f432-4a82-43c1-b2d6-764d1942af1f tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Acquiring lock "1df140f1-fbd9-4b5d-b356-d08b7e908f64" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:25:55 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-9711f432-4a82-43c1-b2d6-764d1942af1f tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Lock "1df140f1-fbd9-4b5d-b356-d08b7e908f64" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:25:55 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-9711f432-4a82-43c1-b2d6-764d1942af1f tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Acquiring lock "1df140f1-fbd9-4b5d-b356-d08b7e908f64-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:25:55 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-9711f432-4a82-43c1-b2d6-764d1942af1f tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Lock "1df140f1-fbd9-4b5d-b356-d08b7e908f64-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:25:55 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-9711f432-4a82-43c1-b2d6-764d1942af1f tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Lock "1df140f1-fbd9-4b5d-b356-d08b7e908f64-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:25:55 user nova-compute[75234]: INFO nova.compute.manager [None req-9711f432-4a82-43c1-b2d6-764d1942af1f tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] [instance: 1df140f1-fbd9-4b5d-b356-d08b7e908f64] Terminating instance
Nov 29 03:25:55 user nova-compute[75234]: DEBUG nova.compute.manager [None req-9711f432-4a82-43c1-b2d6-764d1942af1f tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] [instance: 1df140f1-fbd9-4b5d-b356-d08b7e908f64] Start destroying the instance on the hypervisor. {{(pid=75234) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3140}}
Nov 29 03:25:55 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:25:55 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:25:55 user nova-compute[75234]: DEBUG nova.compute.manager [req-1e399479-d160-435d-98e1-26fd69273c59 req-5858ba2b-fce6-4cf7-9a8d-f05f48b04a00 service nova] [instance: 1df140f1-fbd9-4b5d-b356-d08b7e908f64] Received event network-vif-unplugged-ea9a3586-da52-49b7-8ced-5d27806885ad {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:25:55 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-1e399479-d160-435d-98e1-26fd69273c59 req-5858ba2b-fce6-4cf7-9a8d-f05f48b04a00 service nova] Acquiring lock "1df140f1-fbd9-4b5d-b356-d08b7e908f64-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:25:55 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-1e399479-d160-435d-98e1-26fd69273c59 req-5858ba2b-fce6-4cf7-9a8d-f05f48b04a00 service nova] Lock "1df140f1-fbd9-4b5d-b356-d08b7e908f64-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:25:55 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-1e399479-d160-435d-98e1-26fd69273c59 req-5858ba2b-fce6-4cf7-9a8d-f05f48b04a00 service nova] Lock "1df140f1-fbd9-4b5d-b356-d08b7e908f64-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:25:55 user nova-compute[75234]: DEBUG nova.compute.manager [req-1e399479-d160-435d-98e1-26fd69273c59 req-5858ba2b-fce6-4cf7-9a8d-f05f48b04a00 service nova] [instance: 1df140f1-fbd9-4b5d-b356-d08b7e908f64] No waiting events found dispatching network-vif-unplugged-ea9a3586-da52-49b7-8ced-5d27806885ad {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:25:55 user nova-compute[75234]: DEBUG nova.compute.manager [req-1e399479-d160-435d-98e1-26fd69273c59 req-5858ba2b-fce6-4cf7-9a8d-f05f48b04a00 service nova] [instance: 1df140f1-fbd9-4b5d-b356-d08b7e908f64] Received event network-vif-unplugged-ea9a3586-da52-49b7-8ced-5d27806885ad for instance with task_state deleting. {{(pid=75234) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10890}}
Nov 29 03:25:55 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:25:55 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:25:56 user nova-compute[75234]: INFO nova.virt.libvirt.driver [-] [instance: 1df140f1-fbd9-4b5d-b356-d08b7e908f64] Instance destroyed successfully.
Nov 29 03:25:56 user nova-compute[75234]: DEBUG nova.objects.instance [None req-9711f432-4a82-43c1-b2d6-764d1942af1f tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Lazy-loading 'resources' on Instance uuid 1df140f1-fbd9-4b5d-b356-d08b7e908f64 {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:25:56 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-9711f432-4a82-43c1-b2d6-764d1942af1f tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-11-29T03:19:10Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-ServerRescueNegativeTestJSON-server-278539854',display_name='tempest-ServerRescueNegativeTestJSON-server-278539854',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-serverrescuenegativetestjson-server-278539854',id=33,image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBKySS+rrPWcijvoGGhKEPiK1xSDIj74zuelaF50GYBXUvR1Yet4BFb/2sYsZA/5NzUIJ43mYGg7Ig5cf2ANImb2nTP65fl4DOr9JzbUOlYxAqRtjhkf+S+OjV52cB6+jDw==',key_name='tempest-keypair-605296868',keypairs=<?>,launch_index=0,launched_at=2023-11-29T03:21:28Z,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='f8a7884867c944e1ba1024270e4902f3',ramdisk_id='',reservation_id='r-s6t6psv1',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-ServerRescueNegativeTestJSON-1182706601',owner_user_name='tempest-ServerRescueNegativeTestJSON-1182706601-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-11-29T03:23:18Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='6792e866677645c598ddb7d2c1723a1a',uuid=1df140f1-fbd9-4b5d-b356-d08b7e908f64,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "ea9a3586-da52-49b7-8ced-5d27806885ad", "address": "fa:16:3e:80:25:b8", "network": {"id": "2b3c3140-f3b6-45da-8202-bb4d7f0dd130", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-2043420778-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.158", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "f8a7884867c944e1ba1024270e4902f3", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapea9a3586-da", "ovs_interfaceid": "ea9a3586-da52-49b7-8ced-5d27806885ad", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Nov 29 03:25:56 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-9711f432-4a82-43c1-b2d6-764d1942af1f tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Converting VIF {"id": "ea9a3586-da52-49b7-8ced-5d27806885ad", "address": "fa:16:3e:80:25:b8", "network": {"id": "2b3c3140-f3b6-45da-8202-bb4d7f0dd130", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-2043420778-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.158", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "f8a7884867c944e1ba1024270e4902f3", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapea9a3586-da", "ovs_interfaceid": "ea9a3586-da52-49b7-8ced-5d27806885ad", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:25:56 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-9711f432-4a82-43c1-b2d6-764d1942af1f tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:80:25:b8,bridge_name='br-int',has_traffic_filtering=True,id=ea9a3586-da52-49b7-8ced-5d27806885ad,network=Network(2b3c3140-f3b6-45da-8202-bb4d7f0dd130),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapea9a3586-da') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:25:56 user nova-compute[75234]: DEBUG os_vif [None req-9711f432-4a82-43c1-b2d6-764d1942af1f tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Unplugging vif VIFOpenVSwitch(active=True,address=fa:16:3e:80:25:b8,bridge_name='br-int',has_traffic_filtering=True,id=ea9a3586-da52-49b7-8ced-5d27806885ad,network=Network(2b3c3140-f3b6-45da-8202-bb4d7f0dd130),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapea9a3586-da') {{(pid=75234) unplug /opt/stack/data/venv/lib/python3.10/site-packages/os_vif/__init__.py:109}}
Nov 29 03:25:56 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:25:56 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tapea9a3586-da, bridge=br-int, if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:25:56 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:25:56 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:25:56 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:25:56 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbDestroyCommand(_result=None, table=QoS, record=67201c1a-050b-493c-9462-e0f47e8035fd) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:25:56 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:25:56 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:25:56 user nova-compute[75234]: INFO os_vif [None req-9711f432-4a82-43c1-b2d6-764d1942af1f tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Successfully unplugged vif VIFOpenVSwitch(active=True,address=fa:16:3e:80:25:b8,bridge_name='br-int',has_traffic_filtering=True,id=ea9a3586-da52-49b7-8ced-5d27806885ad,network=Network(2b3c3140-f3b6-45da-8202-bb4d7f0dd130),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapea9a3586-da')
Nov 29 03:25:56 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-9711f432-4a82-43c1-b2d6-764d1942af1f tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] [instance: 1df140f1-fbd9-4b5d-b356-d08b7e908f64] Deleting instance files /opt/stack/data/nova/instances/1df140f1-fbd9-4b5d-b356-d08b7e908f64_del
Nov 29 03:25:56 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-9711f432-4a82-43c1-b2d6-764d1942af1f tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] [instance: 1df140f1-fbd9-4b5d-b356-d08b7e908f64] Deletion of /opt/stack/data/nova/instances/1df140f1-fbd9-4b5d-b356-d08b7e908f64_del complete
Nov 29 03:25:56 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-367d2a35-cd8d-427c-ae01-7a32522fce38 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Acquiring lock "f6bc212d-85cd-479c-92ae-112c9f0101bc" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:25:56 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-367d2a35-cd8d-427c-ae01-7a32522fce38 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Lock "f6bc212d-85cd-479c-92ae-112c9f0101bc" acquired by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:25:56 user nova-compute[75234]: INFO nova.compute.manager [None req-367d2a35-cd8d-427c-ae01-7a32522fce38 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] [instance: f6bc212d-85cd-479c-92ae-112c9f0101bc] Detaching volume ee672802-fcc5-4892-8a23-91763600da13
Nov 29 03:25:56 user nova-compute[75234]: INFO nova.compute.manager [None req-9711f432-4a82-43c1-b2d6-764d1942af1f tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] [instance: 1df140f1-fbd9-4b5d-b356-d08b7e908f64] Took 0.88 seconds to destroy the instance on the hypervisor.
Nov 29 03:25:56 user nova-compute[75234]: DEBUG oslo.service.loopingcall [None req-9711f432-4a82-43c1-b2d6-764d1942af1f tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=75234) func /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/loopingcall.py:435}}
Nov 29 03:25:56 user nova-compute[75234]: DEBUG nova.compute.manager [-] [instance: 1df140f1-fbd9-4b5d-b356-d08b7e908f64] Deallocating network for instance {{(pid=75234) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2279}}
Nov 29 03:25:56 user nova-compute[75234]: DEBUG nova.network.neutron [-] [instance: 1df140f1-fbd9-4b5d-b356-d08b7e908f64] deallocate_for_instance() {{(pid=75234) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1802}}
Nov 29 03:25:56 user nova-compute[75234]: INFO nova.virt.block_device [None req-367d2a35-cd8d-427c-ae01-7a32522fce38 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] [instance: f6bc212d-85cd-479c-92ae-112c9f0101bc] Attempting to driver detach volume ee672802-fcc5-4892-8a23-91763600da13 from mountpoint /dev/vdb
Nov 29 03:25:56 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-367d2a35-cd8d-427c-ae01-7a32522fce38 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Attempting to detach device vdb from instance f6bc212d-85cd-479c-92ae-112c9f0101bc from the persistent domain config. {{(pid=75234) _detach_from_persistent /opt/stack/nova/nova/virt/libvirt/driver.py:2487}}
Nov 29 03:25:56 user nova-compute[75234]: DEBUG nova.virt.libvirt.guest [None req-367d2a35-cd8d-427c-ae01-7a32522fce38 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] detach device xml: <disk type="block" device="disk">
Nov 29 03:25:56 user nova-compute[75234]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Nov 29 03:25:56 user nova-compute[75234]:   <source dev="/dev/sdd"/>
Nov 29 03:25:56 user nova-compute[75234]:   <target dev="vdb" bus="virtio"/>
Nov 29 03:25:56 user nova-compute[75234]:   <serial>ee672802-fcc5-4892-8a23-91763600da13</serial>
Nov 29 03:25:56 user nova-compute[75234]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x07" function="0x0"/>
Nov 29 03:25:56 user nova-compute[75234]: </disk>
Nov 29 03:25:56 user nova-compute[75234]:  {{(pid=75234) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Nov 29 03:25:56 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-367d2a35-cd8d-427c-ae01-7a32522fce38 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Successfully detached device vdb from instance f6bc212d-85cd-479c-92ae-112c9f0101bc from the persistent domain config.
Nov 29 03:25:56 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-367d2a35-cd8d-427c-ae01-7a32522fce38 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] (1/8): Attempting to detach device vdb with device alias virtio-disk1 from instance f6bc212d-85cd-479c-92ae-112c9f0101bc from the live domain config. {{(pid=75234) _detach_from_live_with_retry /opt/stack/nova/nova/virt/libvirt/driver.py:2523}}
Nov 29 03:25:56 user nova-compute[75234]: DEBUG nova.virt.libvirt.guest [None req-367d2a35-cd8d-427c-ae01-7a32522fce38 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] detach device xml: <disk type="block" device="disk">
Nov 29 03:25:56 user nova-compute[75234]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Nov 29 03:25:56 user nova-compute[75234]:   <source dev="/dev/sdd"/>
Nov 29 03:25:56 user nova-compute[75234]:   <target dev="vdb" bus="virtio"/>
Nov 29 03:25:56 user nova-compute[75234]:   <serial>ee672802-fcc5-4892-8a23-91763600da13</serial>
Nov 29 03:25:56 user nova-compute[75234]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x07" function="0x0"/>
Nov 29 03:25:56 user nova-compute[75234]: </disk>
Nov 29 03:25:56 user nova-compute[75234]:  {{(pid=75234) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Nov 29 03:25:56 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-367d2a35-cd8d-427c-ae01-7a32522fce38 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Start waiting for the detach event from libvirt for device vdb with device alias virtio-disk1 for instance f6bc212d-85cd-479c-92ae-112c9f0101bc {{(pid=75234) _detach_from_live_and_wait_for_event /opt/stack/nova/nova/virt/libvirt/driver.py:2599}}
Nov 29 03:25:57 user nova-compute[75234]: DEBUG nova.network.neutron [-] [instance: 1df140f1-fbd9-4b5d-b356-d08b7e908f64] Updating instance_info_cache with network_info: [] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:25:57 user nova-compute[75234]: INFO nova.compute.manager [-] [instance: 1df140f1-fbd9-4b5d-b356-d08b7e908f64] Took 0.83 seconds to deallocate network for instance.
Nov 29 03:25:57 user nova-compute[75234]: DEBUG nova.compute.manager [req-cd55d979-8780-451d-8966-f7507e12ff6d req-1c1e4a8f-c66b-4d29-84f0-2dbc1813d64b service nova] [instance: 1df140f1-fbd9-4b5d-b356-d08b7e908f64] Received event network-vif-deleted-ea9a3586-da52-49b7-8ced-5d27806885ad {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:25:57 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-9711f432-4a82-43c1-b2d6-764d1942af1f tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:25:57 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-9711f432-4a82-43c1-b2d6-764d1942af1f tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:25:57 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:25:57 user nova-compute[75234]: DEBUG nova.compute.provider_tree [None req-9711f432-4a82-43c1-b2d6-764d1942af1f tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Inventory has not changed in ProviderTree for provider: 4d291220-7c37-4839-a0e3-b23c66502b74 {{(pid=75234) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Nov 29 03:25:57 user nova-compute[75234]: DEBUG nova.scheduler.client.report [None req-9711f432-4a82-43c1-b2d6-764d1942af1f tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Inventory has not changed for provider 4d291220-7c37-4839-a0e3-b23c66502b74 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=75234) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Nov 29 03:25:57 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-9711f432-4a82-43c1-b2d6-764d1942af1f tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.280s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:25:57 user nova-compute[75234]: INFO nova.scheduler.client.report [None req-9711f432-4a82-43c1-b2d6-764d1942af1f tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Deleted allocations for instance 1df140f1-fbd9-4b5d-b356-d08b7e908f64
Nov 29 03:25:57 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-9711f432-4a82-43c1-b2d6-764d1942af1f tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Lock "1df140f1-fbd9-4b5d-b356-d08b7e908f64" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 2.322s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:25:57 user nova-compute[75234]: DEBUG nova.compute.manager [req-4f3e6444-8a73-4670-88d1-461160faef66 req-ec995262-8657-45a9-b111-8075d211e97e service nova] [instance: 1df140f1-fbd9-4b5d-b356-d08b7e908f64] Received event network-vif-plugged-ea9a3586-da52-49b7-8ced-5d27806885ad {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:25:57 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-4f3e6444-8a73-4670-88d1-461160faef66 req-ec995262-8657-45a9-b111-8075d211e97e service nova] Acquiring lock "1df140f1-fbd9-4b5d-b356-d08b7e908f64-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:25:57 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-4f3e6444-8a73-4670-88d1-461160faef66 req-ec995262-8657-45a9-b111-8075d211e97e service nova] Lock "1df140f1-fbd9-4b5d-b356-d08b7e908f64-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:25:57 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-4f3e6444-8a73-4670-88d1-461160faef66 req-ec995262-8657-45a9-b111-8075d211e97e service nova] Lock "1df140f1-fbd9-4b5d-b356-d08b7e908f64-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:25:57 user nova-compute[75234]: DEBUG nova.compute.manager [req-4f3e6444-8a73-4670-88d1-461160faef66 req-ec995262-8657-45a9-b111-8075d211e97e service nova] [instance: 1df140f1-fbd9-4b5d-b356-d08b7e908f64] No waiting events found dispatching network-vif-plugged-ea9a3586-da52-49b7-8ced-5d27806885ad {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:25:57 user nova-compute[75234]: WARNING nova.compute.manager [req-4f3e6444-8a73-4670-88d1-461160faef66 req-ec995262-8657-45a9-b111-8075d211e97e service nova] [instance: 1df140f1-fbd9-4b5d-b356-d08b7e908f64] Received unexpected event network-vif-plugged-ea9a3586-da52-49b7-8ced-5d27806885ad for instance with vm_state deleted and task_state None.
Nov 29 03:25:58 user nova-compute[75234]: DEBUG nova.compute.manager [req-19e81bc9-8935-4432-aba6-51645accc623 req-bd2aaa1b-56a3-4e7f-8373-5e0ac2fcc2c3 service nova] [instance: 5d7d744e-1a5f-40ab-b961-4b58c183efe0] Received event network-changed-fa65b32f-b627-45fa-8d6d-d99012ca1262 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:25:58 user nova-compute[75234]: DEBUG nova.compute.manager [req-19e81bc9-8935-4432-aba6-51645accc623 req-bd2aaa1b-56a3-4e7f-8373-5e0ac2fcc2c3 service nova] [instance: 5d7d744e-1a5f-40ab-b961-4b58c183efe0] Refreshing instance network info cache due to event network-changed-fa65b32f-b627-45fa-8d6d-d99012ca1262. {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11117}}
Nov 29 03:25:58 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-19e81bc9-8935-4432-aba6-51645accc623 req-bd2aaa1b-56a3-4e7f-8373-5e0ac2fcc2c3 service nova] Acquiring lock "refresh_cache-5d7d744e-1a5f-40ab-b961-4b58c183efe0" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:25:58 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-19e81bc9-8935-4432-aba6-51645accc623 req-bd2aaa1b-56a3-4e7f-8373-5e0ac2fcc2c3 service nova] Acquired lock "refresh_cache-5d7d744e-1a5f-40ab-b961-4b58c183efe0" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:25:58 user nova-compute[75234]: DEBUG nova.network.neutron [req-19e81bc9-8935-4432-aba6-51645accc623 req-bd2aaa1b-56a3-4e7f-8373-5e0ac2fcc2c3 service nova] [instance: 5d7d744e-1a5f-40ab-b961-4b58c183efe0] Refreshing network info cache for port fa65b32f-b627-45fa-8d6d-d99012ca1262 {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2006}}
Nov 29 03:25:58 user nova-compute[75234]: DEBUG nova.network.neutron [req-19e81bc9-8935-4432-aba6-51645accc623 req-bd2aaa1b-56a3-4e7f-8373-5e0ac2fcc2c3 service nova] [instance: 5d7d744e-1a5f-40ab-b961-4b58c183efe0] Updated VIF entry in instance network info cache for port fa65b32f-b627-45fa-8d6d-d99012ca1262. {{(pid=75234) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3481}}
Nov 29 03:25:58 user nova-compute[75234]: DEBUG nova.network.neutron [req-19e81bc9-8935-4432-aba6-51645accc623 req-bd2aaa1b-56a3-4e7f-8373-5e0ac2fcc2c3 service nova] [instance: 5d7d744e-1a5f-40ab-b961-4b58c183efe0] Updating instance_info_cache with network_info: [{"id": "fa65b32f-b627-45fa-8d6d-d99012ca1262", "address": "fa:16:3e:2f:fa:59", "network": {"id": "6787e0a8-2980-4a2e-bf2a-583af955241d", "bridge": "br-int", "label": "tempest-TestMinimumBasicScenario-1273600998-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.111", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "74f132d6666c47b8b43869df6fee6de3", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapfa65b32f-b6", "ovs_interfaceid": "fa65b32f-b627-45fa-8d6d-d99012ca1262", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:25:58 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-19e81bc9-8935-4432-aba6-51645accc623 req-bd2aaa1b-56a3-4e7f-8373-5e0ac2fcc2c3 service nova] Releasing lock "refresh_cache-5d7d744e-1a5f-40ab-b961-4b58c183efe0" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:26:00 user nova-compute[75234]: DEBUG nova.compute.manager [req-42c3ecda-f96d-49bd-97c6-eb8a37bc82c5 req-53b7f989-c5b7-4145-ae75-a9c80b1d7574 service nova] [instance: 5d7d744e-1a5f-40ab-b961-4b58c183efe0] Received event network-changed-fa65b32f-b627-45fa-8d6d-d99012ca1262 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:26:00 user nova-compute[75234]: DEBUG nova.compute.manager [req-42c3ecda-f96d-49bd-97c6-eb8a37bc82c5 req-53b7f989-c5b7-4145-ae75-a9c80b1d7574 service nova] [instance: 5d7d744e-1a5f-40ab-b961-4b58c183efe0] Refreshing instance network info cache due to event network-changed-fa65b32f-b627-45fa-8d6d-d99012ca1262. {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11117}}
Nov 29 03:26:00 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-42c3ecda-f96d-49bd-97c6-eb8a37bc82c5 req-53b7f989-c5b7-4145-ae75-a9c80b1d7574 service nova] Acquiring lock "refresh_cache-5d7d744e-1a5f-40ab-b961-4b58c183efe0" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:26:00 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-42c3ecda-f96d-49bd-97c6-eb8a37bc82c5 req-53b7f989-c5b7-4145-ae75-a9c80b1d7574 service nova] Acquired lock "refresh_cache-5d7d744e-1a5f-40ab-b961-4b58c183efe0" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:26:00 user nova-compute[75234]: DEBUG nova.network.neutron [req-42c3ecda-f96d-49bd-97c6-eb8a37bc82c5 req-53b7f989-c5b7-4145-ae75-a9c80b1d7574 service nova] [instance: 5d7d744e-1a5f-40ab-b961-4b58c183efe0] Refreshing network info cache for port fa65b32f-b627-45fa-8d6d-d99012ca1262 {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2006}}
Nov 29 03:26:01 user nova-compute[75234]: DEBUG nova.network.neutron [req-42c3ecda-f96d-49bd-97c6-eb8a37bc82c5 req-53b7f989-c5b7-4145-ae75-a9c80b1d7574 service nova] [instance: 5d7d744e-1a5f-40ab-b961-4b58c183efe0] Updated VIF entry in instance network info cache for port fa65b32f-b627-45fa-8d6d-d99012ca1262. {{(pid=75234) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3481}}
Nov 29 03:26:01 user nova-compute[75234]: DEBUG nova.network.neutron [req-42c3ecda-f96d-49bd-97c6-eb8a37bc82c5 req-53b7f989-c5b7-4145-ae75-a9c80b1d7574 service nova] [instance: 5d7d744e-1a5f-40ab-b961-4b58c183efe0] Updating instance_info_cache with network_info: [{"id": "fa65b32f-b627-45fa-8d6d-d99012ca1262", "address": "fa:16:3e:2f:fa:59", "network": {"id": "6787e0a8-2980-4a2e-bf2a-583af955241d", "bridge": "br-int", "label": "tempest-TestMinimumBasicScenario-1273600998-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.111", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "74f132d6666c47b8b43869df6fee6de3", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapfa65b32f-b6", "ovs_interfaceid": "fa65b32f-b627-45fa-8d6d-d99012ca1262", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:26:01 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-42c3ecda-f96d-49bd-97c6-eb8a37bc82c5 req-53b7f989-c5b7-4145-ae75-a9c80b1d7574 service nova] Releasing lock "refresh_cache-5d7d744e-1a5f-40ab-b961-4b58c183efe0" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:26:01 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:26:02 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:26:03 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:26:04 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-b8d2f0de-5dee-497f-bfb2-fe1fc354fec8 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Acquiring lock "5d7d744e-1a5f-40ab-b961-4b58c183efe0" by "nova.compute.manager.ComputeManager.reboot_instance.<locals>.do_reboot_instance" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:26:04 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-b8d2f0de-5dee-497f-bfb2-fe1fc354fec8 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Lock "5d7d744e-1a5f-40ab-b961-4b58c183efe0" acquired by "nova.compute.manager.ComputeManager.reboot_instance.<locals>.do_reboot_instance" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:26:04 user nova-compute[75234]: INFO nova.compute.manager [None req-b8d2f0de-5dee-497f-bfb2-fe1fc354fec8 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] [instance: 5d7d744e-1a5f-40ab-b961-4b58c183efe0] Rebooting instance
Nov 29 03:26:05 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-b8d2f0de-5dee-497f-bfb2-fe1fc354fec8 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Acquiring lock "refresh_cache-5d7d744e-1a5f-40ab-b961-4b58c183efe0" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:26:05 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-b8d2f0de-5dee-497f-bfb2-fe1fc354fec8 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Acquired lock "refresh_cache-5d7d744e-1a5f-40ab-b961-4b58c183efe0" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:26:05 user nova-compute[75234]: DEBUG nova.network.neutron [None req-b8d2f0de-5dee-497f-bfb2-fe1fc354fec8 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] [instance: 5d7d744e-1a5f-40ab-b961-4b58c183efe0] Building network info cache for instance {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2009}}
Nov 29 03:26:05 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:26:05 user nova-compute[75234]: DEBUG nova.network.neutron [None req-b8d2f0de-5dee-497f-bfb2-fe1fc354fec8 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] [instance: 5d7d744e-1a5f-40ab-b961-4b58c183efe0] Updating instance_info_cache with network_info: [{"id": "fa65b32f-b627-45fa-8d6d-d99012ca1262", "address": "fa:16:3e:2f:fa:59", "network": {"id": "6787e0a8-2980-4a2e-bf2a-583af955241d", "bridge": "br-int", "label": "tempest-TestMinimumBasicScenario-1273600998-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.111", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "74f132d6666c47b8b43869df6fee6de3", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapfa65b32f-b6", "ovs_interfaceid": "fa65b32f-b627-45fa-8d6d-d99012ca1262", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:26:05 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-b8d2f0de-5dee-497f-bfb2-fe1fc354fec8 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Releasing lock "refresh_cache-5d7d744e-1a5f-40ab-b961-4b58c183efe0" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:26:05 user nova-compute[75234]: DEBUG nova.compute.manager [None req-b8d2f0de-5dee-497f-bfb2-fe1fc354fec8 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] [instance: 5d7d744e-1a5f-40ab-b961-4b58c183efe0] Checking state {{(pid=75234) _get_power_state /opt/stack/nova/nova/compute/manager.py:1782}}
Nov 29 03:26:06 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:26:06 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:26:06 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:26:06 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:26:06 user nova-compute[75234]: DEBUG nova.compute.manager [req-ca3372b6-a927-4d92-8466-f5c8d87ba2ab req-f99c955c-5178-4e94-af63-79dd53efb56b service nova] [instance: 5d7d744e-1a5f-40ab-b961-4b58c183efe0] Received event network-vif-unplugged-fa65b32f-b627-45fa-8d6d-d99012ca1262 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:26:06 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-ca3372b6-a927-4d92-8466-f5c8d87ba2ab req-f99c955c-5178-4e94-af63-79dd53efb56b service nova] Acquiring lock "5d7d744e-1a5f-40ab-b961-4b58c183efe0-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:26:06 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-ca3372b6-a927-4d92-8466-f5c8d87ba2ab req-f99c955c-5178-4e94-af63-79dd53efb56b service nova] Lock "5d7d744e-1a5f-40ab-b961-4b58c183efe0-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:26:06 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-ca3372b6-a927-4d92-8466-f5c8d87ba2ab req-f99c955c-5178-4e94-af63-79dd53efb56b service nova] Lock "5d7d744e-1a5f-40ab-b961-4b58c183efe0-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:26:06 user nova-compute[75234]: DEBUG nova.compute.manager [req-ca3372b6-a927-4d92-8466-f5c8d87ba2ab req-f99c955c-5178-4e94-af63-79dd53efb56b service nova] [instance: 5d7d744e-1a5f-40ab-b961-4b58c183efe0] No waiting events found dispatching network-vif-unplugged-fa65b32f-b627-45fa-8d6d-d99012ca1262 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:26:06 user nova-compute[75234]: WARNING nova.compute.manager [req-ca3372b6-a927-4d92-8466-f5c8d87ba2ab req-f99c955c-5178-4e94-af63-79dd53efb56b service nova] [instance: 5d7d744e-1a5f-40ab-b961-4b58c183efe0] Received unexpected event network-vif-unplugged-fa65b32f-b627-45fa-8d6d-d99012ca1262 for instance with vm_state active and task_state reboot_started_hard.
Nov 29 03:26:06 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:26:06 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:26:06 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:26:06 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=75234) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10541}}
Nov 29 03:26:06 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager.update_available_resource {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:26:06 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:26:06 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:26:06 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:26:06 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Auditing locally available compute resources for user (node: user) {{(pid=75234) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:907}}
Nov 29 03:26:06 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:26:06 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:26:06 user nova-compute[75234]: INFO nova.virt.libvirt.driver [-] [instance: 5d7d744e-1a5f-40ab-b961-4b58c183efe0] Instance destroyed successfully.
Nov 29 03:26:06 user nova-compute[75234]: DEBUG nova.objects.instance [None req-b8d2f0de-5dee-497f-bfb2-fe1fc354fec8 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Lazy-loading 'resources' on Instance uuid 5d7d744e-1a5f-40ab-b961-4b58c183efe0 {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:26:07 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-b8d2f0de-5dee-497f-bfb2-fe1fc354fec8 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-11-29T03:23:45Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-TestMinimumBasicScenario-server-310577704',display_name='tempest-TestMinimumBasicScenario-server-310577704',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-testminimumbasicscenario-server-310577704',id=39,image_ref='7107c48b-05b9-4052-917d-814b721d169c',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBBTfFCQxMn5FN9HRzHRLmeLvGNhlvH8jkTHWDyEbYwE8hTZhRqGybcRgcmJ5xQB1zSttCcEaPSgc17S9QrgRI43nIUg3IFEEzmkOkz4/kzJTs5GShIWM4PGtgtnmC06Ugw==',key_name='tempest-TestMinimumBasicScenario-44553634',keypairs=<?>,launch_index=0,launched_at=2023-11-29T03:23:53Z,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='74f132d6666c47b8b43869df6fee6de3',ramdisk_id='',reservation_id='r-dj1tj6j2',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='7107c48b-05b9-4052-917d-814b721d169c',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',owner_project_name='tempest-TestMinimumBasicScenario-1635847764',owner_user_name='tempest-TestMinimumBasicScenario-1635847764-project-member'},tags=<?>,task_state='reboot_started_hard',terminated_at=None,trusted_certs=<?>,updated_at=2023-11-29T03:26:06Z,user_data=None,user_id='3a02618f56ea45898ab4783a4f198b5d',uuid=5d7d744e-1a5f-40ab-b961-4b58c183efe0,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "fa65b32f-b627-45fa-8d6d-d99012ca1262", "address": "fa:16:3e:2f:fa:59", "network": {"id": "6787e0a8-2980-4a2e-bf2a-583af955241d", "bridge": "br-int", "label": "tempest-TestMinimumBasicScenario-1273600998-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.111", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "74f132d6666c47b8b43869df6fee6de3", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapfa65b32f-b6", "ovs_interfaceid": "fa65b32f-b627-45fa-8d6d-d99012ca1262", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Nov 29 03:26:07 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-b8d2f0de-5dee-497f-bfb2-fe1fc354fec8 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Converting VIF {"id": "fa65b32f-b627-45fa-8d6d-d99012ca1262", "address": "fa:16:3e:2f:fa:59", "network": {"id": "6787e0a8-2980-4a2e-bf2a-583af955241d", "bridge": "br-int", "label": "tempest-TestMinimumBasicScenario-1273600998-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.111", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "74f132d6666c47b8b43869df6fee6de3", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapfa65b32f-b6", "ovs_interfaceid": "fa65b32f-b627-45fa-8d6d-d99012ca1262", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:26:07 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-b8d2f0de-5dee-497f-bfb2-fe1fc354fec8 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:2f:fa:59,bridge_name='br-int',has_traffic_filtering=True,id=fa65b32f-b627-45fa-8d6d-d99012ca1262,network=Network(6787e0a8-2980-4a2e-bf2a-583af955241d),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapfa65b32f-b6') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:26:07 user nova-compute[75234]: DEBUG os_vif [None req-b8d2f0de-5dee-497f-bfb2-fe1fc354fec8 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Unplugging vif VIFOpenVSwitch(active=True,address=fa:16:3e:2f:fa:59,bridge_name='br-int',has_traffic_filtering=True,id=fa65b32f-b627-45fa-8d6d-d99012ca1262,network=Network(6787e0a8-2980-4a2e-bf2a-583af955241d),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapfa65b32f-b6') {{(pid=75234) unplug /opt/stack/data/venv/lib/python3.10/site-packages/os_vif/__init__.py:109}}
Nov 29 03:26:07 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:26:07 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tapfa65b32f-b6, bridge=br-int, if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:26:07 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:26:07 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:26:07 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbDestroyCommand(_result=None, table=QoS, record=59df5b17-ee40-40de-bbb2-45629fc7b9e3) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:26:07 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:26:07 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:26:07 user nova-compute[75234]: INFO os_vif [None req-b8d2f0de-5dee-497f-bfb2-fe1fc354fec8 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Successfully unplugged vif VIFOpenVSwitch(active=True,address=fa:16:3e:2f:fa:59,bridge_name='br-int',has_traffic_filtering=True,id=fa65b32f-b627-45fa-8d6d-d99012ca1262,network=Network(6787e0a8-2980-4a2e-bf2a-583af955241d),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapfa65b32f-b6')
Nov 29 03:26:07 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-b8d2f0de-5dee-497f-bfb2-fe1fc354fec8 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] [instance: 5d7d744e-1a5f-40ab-b961-4b58c183efe0] calling os-brick to detach iSCSI Volume {{(pid=75234) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Nov 29 03:26:07 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-b8d2f0de-5dee-497f-bfb2-fe1fc354fec8 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f8a5050c9a0>, {'target_iqn': 'iqn.2010-10.org.openstack:65acc1f5-8dbf-457c-a978-510436bccf11', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'jqCadzEM', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdb'}, None), 'kwargs': True}}" {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:176}}
Nov 29 03:26:07 user nova-compute[75234]: WARNING os_brick.initiator.connectors.base [None req-b8d2f0de-5dee-497f-bfb2-fe1fc354fec8 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Nov 29 03:26:07 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-b8d2f0de-5dee-497f-bfb2-fe1fc354fec8 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:68}}
Nov 29 03:26:07 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-b8d2f0de-5dee-497f-bfb2-fe1fc354fec8 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:73}}
Nov 29 03:26:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:26:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.023s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:26:07 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[f43b9eb4-c0d4-4111-b1f3-843db0c4ed84]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:ee672802-fcc5-4892-8a23-91763600da13\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:65acc1f5-8dbf-457c-a978-510436bccf11\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:26:07 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-b8d2f0de-5dee-497f-bfb2-fe1fc354fec8 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS:
Nov 29 03:26:07 user nova-compute[75234]: No targets found.
Nov 29 03:26:07 user nova-compute[75234]: iSNS:
Nov 29 03:26:07 user nova-compute[75234]: No targets found.
Nov 29 03:26:07 user nova-compute[75234]: STATIC:
Nov 29 03:26:07 user nova-compute[75234]: Target: iqn.2010-10.org.openstack:ee672802-fcc5-4892-8a23-91763600da13
Nov 29 03:26:07 user nova-compute[75234]:         Portal: 172.16.0.220:3260,-1
Nov 29 03:26:07 user nova-compute[75234]:                 Iface Name: default
Nov 29 03:26:07 user nova-compute[75234]: Target: iqn.2010-10.org.openstack:65acc1f5-8dbf-457c-a978-510436bccf11
Nov 29 03:26:07 user nova-compute[75234]:         Portal: 172.16.0.220:3260,-1
Nov 29 03:26:07 user nova-compute[75234]:                 Iface Name: default
Nov 29 03:26:07 user nova-compute[75234]: Target: iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0
Nov 29 03:26:07 user nova-compute[75234]:         Portal: 172.16.0.220:3260,-1
Nov 29 03:26:07 user nova-compute[75234]:                 Iface Name: default
Nov 29 03:26:07 user nova-compute[75234]: FIRMWARE:
Nov 29 03:26:07 user nova-compute[75234]: No targets found.
Nov 29 03:26:07 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:26:07 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-b8d2f0de-5dee-497f-bfb2-fe1fc354fec8 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Regex to get portals from discoverydb: ^SENDTARGETS:
Nov 29 03:26:07 user nova-compute[75234]: .*?^DiscoveryAddress: 172.16.0.220,3260.*?
Nov 29 03:26:07 user nova-compute[75234]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=75234) _get_discoverydb_portals /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:382}}
Nov 29 03:26:07 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-b8d2f0de-5dee-497f-bfb2-fe1fc354fec8 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:65acc1f5-8dbf-457c-a978-510436bccf11', 0)] {{(pid=75234) _get_connection_devices /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:830}}
Nov 29 03:26:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:26:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.022s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:26:07 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[60654381-f883-4c44-904a-d0676bfa1a99]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:65acc1f5-8dbf-457c-a978-510436bccf11\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:ee672802-fcc5-4892-8a23-91763600da13\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:26:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:26:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.024s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:26:07 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[961ab8ea-71d9-42c2-b3dc-3c87484e9405]: (4, ('tcp: [42] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0 (non-flash)\ntcp: [45] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:65acc1f5-8dbf-457c-a978-510436bccf11 (non-flash)\ntcp: [46] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ee672802-fcc5-4892-8a23-91763600da13 (non-flash)\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:26:07 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-b8d2f0de-5dee-497f-bfb2-fe1fc354fec8 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [42] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0 (non-flash)
Nov 29 03:26:07 user nova-compute[75234]: tcp: [45] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:65acc1f5-8dbf-457c-a978-510436bccf11 (non-flash)
Nov 29 03:26:07 user nova-compute[75234]: tcp: [46] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ee672802-fcc5-4892-8a23-91763600da13 (non-flash)
Nov 29 03:26:07 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:26:07 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-b8d2f0de-5dee-497f-bfb2-fe1fc354fec8 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] iscsi session list stdout=tcp: [42] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0 (non-flash)
Nov 29 03:26:07 user nova-compute[75234]: tcp: [45] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:65acc1f5-8dbf-457c-a978-510436bccf11 (non-flash)
Nov 29 03:26:07 user nova-compute[75234]: tcp: [46] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ee672802-fcc5-4892-8a23-91763600da13 (non-flash)
Nov 29 03:26:07 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsi_session /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Nov 29 03:26:07 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-b8d2f0de-5dee-497f-bfb2-fe1fc354fec8 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7f8a50748ee0>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:65acc1f5-8dbf-457c-a978-510436bccf11'): ({'sdb'}, set())}) {{(pid=75234) _get_connection_devices /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:863}}
Nov 29 03:26:07 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-b8d2f0de-5dee-497f-bfb2-fe1fc354fec8 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Removing single pathed devices sdb {{(pid=75234) remove_connection /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:374}}
Nov 29 03:26:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:26:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f6bc212d-85cd-479c-92ae-112c9f0101bc/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:26:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.019s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:26:07 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[f8959c21-d83b-48da-a7e7-065c13a5457e]: (4, ('path checker states:\nup                  3\n\npaths: 0\nbusy: False\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:26:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdb {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:26:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdb" returned: 0 in 0.021s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:26:07 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[5bfebaeb-3d3e-461b-9175-f19d842928b6]: (4, ('ok\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:26:07 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-b8d2f0de-5dee-497f-bfb2-fe1fc354fec8 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Flushing IO for device /dev/sdb {{(pid=75234) flush_device_io /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:434}}
Nov 29 03:26:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdb {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:26:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdb" returned: 0 in 0.017s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:26:07 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[e36cd4cb-ddf5-47d6-98ba-a3937dba6559]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:26:07 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-b8d2f0de-5dee-497f-bfb2-fe1fc354fec8 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Remove SCSI device /dev/sdb with /sys/block/sdb/device/delete {{(pid=75234) remove_scsi_device /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:148}}
Nov 29 03:26:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdb/device/delete {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:26:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdb/device/delete" returned: 0 in 0.057s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:26:07 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[0252bf2b-fa35-42d1-9e26-bc883b9ee7f4]: (4, ('1', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:26:07 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-b8d2f0de-5dee-497f-bfb2-fe1fc354fec8 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Checking to see if SCSI volumes sdb have been removed. {{(pid=75234) wait_for_volumes_removal /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:156}}
Nov 29 03:26:07 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-b8d2f0de-5dee-497f-bfb2-fe1fc354fec8 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] SCSI volumes sdb have been removed. {{(pid=75234) wait_for_volumes_removal /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:166}}
Nov 29 03:26:07 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-b8d2f0de-5dee-497f-bfb2-fe1fc354fec8 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:65acc1f5-8dbf-457c-a978-510436bccf11')] {{(pid=75234) _disconnect_connection /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Nov 29 03:26:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:65acc1f5-8dbf-457c-a978-510436bccf11 -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:26:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:65acc1f5-8dbf-457c-a978-510436bccf11 -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.020s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:26:07 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[189d9f03-1c31-4c63-9c8d-3a686b084cca]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:26:07 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-b8d2f0de-5dee-497f-bfb2-fe1fc354fec8 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:26:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:65acc1f5-8dbf-457c-a978-510436bccf11 -p 172.16.0.220:3260 --logout {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:26:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f6bc212d-85cd-479c-92ae-112c9f0101bc/disk --force-share --output=json" returned: 0 in 0.184s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:26:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f6bc212d-85cd-479c-92ae-112c9f0101bc/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:26:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:65acc1f5-8dbf-457c-a978-510436bccf11 -p 172.16.0.220:3260 --logout" returned: 0 in 0.071s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:26:07 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[bb6e6545-0991-4725-bf9a-d5a3a9509b26]: (4, ('Logging out of session [sid: 45, target: iqn.2010-10.org.openstack:65acc1f5-8dbf-457c-a978-510436bccf11, portal: 172.16.0.220,3260]\nLogout of [sid: 45, target: iqn.2010-10.org.openstack:65acc1f5-8dbf-457c-a978-510436bccf11, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:26:07 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-b8d2f0de-5dee-497f-bfb2-fe1fc354fec8 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 45, target: iqn.2010-10.org.openstack:65acc1f5-8dbf-457c-a978-510436bccf11, portal: 172.16.0.220,3260]
Nov 29 03:26:07 user nova-compute[75234]: Logout of [sid: 45, target: iqn.2010-10.org.openstack:65acc1f5-8dbf-457c-a978-510436bccf11, portal: 172.16.0.220,3260] successful.
Nov 29 03:26:07 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:26:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:65acc1f5-8dbf-457c-a978-510436bccf11 -p 172.16.0.220:3260 --op delete {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:26:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:65acc1f5-8dbf-457c-a978-510436bccf11 -p 172.16.0.220:3260 --op delete" returned: 0 in 0.028s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:26:07 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[85ed3a31-25c3-4d56-99f9-d4c7590c77fd]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:26:07 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-b8d2f0de-5dee-497f-bfb2-fe1fc354fec8 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:26:07 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-b8d2f0de-5dee-497f-bfb2-fe1fc354fec8 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.408s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:87}}
Nov 29 03:26:07 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-b8d2f0de-5dee-497f-bfb2-fe1fc354fec8 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] <== disconnect_volume: return (409ms) None {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:203}}
Nov 29 03:26:07 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-b8d2f0de-5dee-497f-bfb2-fe1fc354fec8 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] [instance: 5d7d744e-1a5f-40ab-b961-4b58c183efe0] Disconnected iSCSI Volume {{(pid=75234) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:79}}
Nov 29 03:26:07 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-b8d2f0de-5dee-497f-bfb2-fe1fc354fec8 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] [instance: 5d7d744e-1a5f-40ab-b961-4b58c183efe0] Start _get_guest_xml network_info=[{"id": "fa65b32f-b627-45fa-8d6d-d99012ca1262", "address": "fa:16:3e:2f:fa:59", "network": {"id": "6787e0a8-2980-4a2e-bf2a-583af955241d", "bridge": "br-int", "label": "tempest-TestMinimumBasicScenario-1273600998-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.111", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "74f132d6666c47b8b43869df6fee6de3", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapfa65b32f-b6", "ovs_interfaceid": "fa65b32f-b627-45fa-8d6d-d99012ca1262", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, '/dev/vdb': {'bus': 'virtio', 'dev': 'vdb', 'type': 'disk'}}} image_meta=ImageMeta(checksum=<?>,container_format='bare',created_at=<?>,direct_url=<?>,disk_format='qcow2',id=7107c48b-05b9-4052-917d-814b721d169c,min_disk=1,min_ram=0,name=<?>,owner=<?>,properties=ImageMetaProps,protected=<?>,size=<?>,status=<?>,tags=<?>,updated_at=<?>,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [{'device_name': '/dev/vda', 'encryption_format': None, 'encrypted': False, 'device_type': 'disk', 'size': 0, 'disk_bus': 'virtio', 'boot_index': 0, 'encryption_options': None, 'encryption_secret_uuid': None, 'guest_format': None, 'image_id': '7107c48b-05b9-4052-917d-814b721d169c'}], 'ephemerals': [], 'block_device_mapping': [{'mount_device': '/dev/vdb', 'attachment_id': 'c22690b0-7791-46dd-a849-00a4321d5278', 'disk_bus': 'virtio', 'device_type': 'disk', 'boot_index': None, 'guest_format': None, 'delete_on_termination': False, 'connection_info': {'driver_volume_type': 'iscsi', 'data': {'target_iqn': 'iqn.2010-10.org.openstack:65acc1f5-8dbf-457c-a978-510436bccf11', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'jqCadzEM', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdb'}, 'status': 'reserved', 'instance': '5d7d744e-1a5f-40ab-b961-4b58c183efe0', 'attached_at': '', 'detached_at': '', 'volume_id': '65acc1f5-8dbf-457c-a978-510436bccf11', 'serial': '65acc1f5-8dbf-457c-a978-510436bccf11'}, 'volume_type': None} {{(pid=75234) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7562}}
Nov 29 03:26:07 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:26:07 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-b8d2f0de-5dee-497f-bfb2-fe1fc354fec8 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:26:07 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-b8d2f0de-5dee-497f-bfb2-fe1fc354fec8 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:26:07 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-b8d2f0de-5dee-497f-bfb2-fe1fc354fec8 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Searching host: 'user' for CPU controller through CGroups V1... {{(pid=75234) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1652}}
Nov 29 03:26:07 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-b8d2f0de-5dee-497f-bfb2-fe1fc354fec8 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] CPU controller missing on host. {{(pid=75234) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1662}}
Nov 29 03:26:07 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-b8d2f0de-5dee-497f-bfb2-fe1fc354fec8 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Searching host: 'user' for CPU controller through CGroups V2... {{(pid=75234) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1671}}
Nov 29 03:26:07 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-b8d2f0de-5dee-497f-bfb2-fe1fc354fec8 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] CPU controller found on host. {{(pid=75234) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1678}}
Nov 29 03:26:07 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-b8d2f0de-5dee-497f-bfb2-fe1fc354fec8 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=75234) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5400}}
Nov 29 03:26:07 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-b8d2f0de-5dee-497f-bfb2-fe1fc354fec8 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-11-29T03:02:04Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='1',id=6,is_public=True,memory_mb=512,name='m1.tiny',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum=<?>,container_format='bare',created_at=<?>,direct_url=<?>,disk_format='qcow2',id=7107c48b-05b9-4052-917d-814b721d169c,min_disk=1,min_ram=0,name=<?>,owner=<?>,properties=ImageMetaProps,protected=<?>,size=<?>,status=<?>,tags=<?>,updated_at=<?>,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:563}}
Nov 29 03:26:07 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-b8d2f0de-5dee-497f-bfb2-fe1fc354fec8 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Flavor limits 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:348}}
Nov 29 03:26:07 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-b8d2f0de-5dee-497f-bfb2-fe1fc354fec8 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Image limits 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:352}}
Nov 29 03:26:07 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-b8d2f0de-5dee-497f-bfb2-fe1fc354fec8 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Flavor pref 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:388}}
Nov 29 03:26:07 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-b8d2f0de-5dee-497f-bfb2-fe1fc354fec8 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Image pref 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:392}}
Nov 29 03:26:07 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-b8d2f0de-5dee-497f-bfb2-fe1fc354fec8 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:430}}
Nov 29 03:26:07 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-b8d2f0de-5dee-497f-bfb2-fe1fc354fec8 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:569}}
Nov 29 03:26:07 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-b8d2f0de-5dee-497f-bfb2-fe1fc354fec8 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=75234) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:471}}
Nov 29 03:26:07 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-b8d2f0de-5dee-497f-bfb2-fe1fc354fec8 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Got 1 possible topologies {{(pid=75234) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:501}}
Nov 29 03:26:07 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-b8d2f0de-5dee-497f-bfb2-fe1fc354fec8 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:575}}
Nov 29 03:26:07 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-b8d2f0de-5dee-497f-bfb2-fe1fc354fec8 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:577}}
Nov 29 03:26:07 user nova-compute[75234]: DEBUG nova.objects.instance [None req-b8d2f0de-5dee-497f-bfb2-fe1fc354fec8 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Lazy-loading 'vcpu_model' on Instance uuid 5d7d744e-1a5f-40ab-b961-4b58c183efe0 {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:26:07 user nova-compute[75234]: DEBUG nova.objects.base [None req-b8d2f0de-5dee-497f-bfb2-fe1fc354fec8 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Object Instance<5d7d744e-1a5f-40ab-b961-4b58c183efe0> lazy-loaded attributes: resources,vcpu_model {{(pid=75234) wrapper /opt/stack/nova/nova/objects/base.py:126}}
Nov 29 03:26:07 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-b8d2f0de-5dee-497f-bfb2-fe1fc354fec8 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] [instance: 5d7d744e-1a5f-40ab-b961-4b58c183efe0] Calling os-brick to attach iSCSI Volume {{(pid=75234) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Nov 29 03:26:07 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-b8d2f0de-5dee-497f-bfb2-fe1fc354fec8 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f8a5050c9a0>, 'connection_properties': {'target_iqn': 'iqn.2010-10.org.openstack:65acc1f5-8dbf-457c-a978-510436bccf11', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'jqCadzEM', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path'}}" {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:176}}
Nov 29 03:26:07 user nova-compute[75234]: WARNING os_brick.initiator.connectors.base [None req-b8d2f0de-5dee-497f-bfb2-fe1fc354fec8 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Nov 29 03:26:07 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-b8d2f0de-5dee-497f-bfb2-fe1fc354fec8 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:68}}
Nov 29 03:26:07 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-b8d2f0de-5dee-497f-bfb2-fe1fc354fec8 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.002s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:73}}
Nov 29 03:26:07 user nova-compute[75234]: WARNING os_brick.initiator.connectors.base [None req-b8d2f0de-5dee-497f-bfb2-fe1fc354fec8 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Nov 29 03:26:07 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-b8d2f0de-5dee-497f-bfb2-fe1fc354fec8 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:65acc1f5-8dbf-457c-a978-510436bccf11" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:68}}
Nov 29 03:26:07 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-b8d2f0de-5dee-497f-bfb2-fe1fc354fec8 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:65acc1f5-8dbf-457c-a978-510436bccf11" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:73}}
Nov 29 03:26:07 user nova-compute[75234]: INFO os_brick.initiator.connectors.iscsi [None req-b8d2f0de-5dee-497f-bfb2-fe1fc354fec8 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Nov 29 03:26:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f6bc212d-85cd-479c-92ae-112c9f0101bc/disk --force-share --output=json" returned: 0 in 0.223s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:26:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:65acc1f5-8dbf-457c-a978-510436bccf11 -p 172.16.0.220:3260 {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:26:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/65d48c93-b6da-4f51-9d5a-0676dda62a60/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:26:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:65acc1f5-8dbf-457c-a978-510436bccf11 -p 172.16.0.220:3260" returned: 21 in 0.017s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:26:07 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[5d4b7831-77ae-4fd2-b5d7-d70a93efeb2b]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:26:07 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-b8d2f0de-5dee-497f-bfb2-fe1fc354fec8 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Nov 29 03:26:07 user nova-compute[75234]:  {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:26:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:65acc1f5-8dbf-457c-a978-510436bccf11 -p 172.16.0.220:3260 --interface default --op new {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:26:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:65acc1f5-8dbf-457c-a978-510436bccf11 -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.020s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:26:07 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[2a69f0c3-d581-4aa0-8092-d8f1b0eed01b]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:65acc1f5-8dbf-457c-a978-510436bccf11] added\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:26:07 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-b8d2f0de-5dee-497f-bfb2-fe1fc354fec8 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:65acc1f5-8dbf-457c-a978-510436bccf11] added
Nov 29 03:26:07 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:26:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:65acc1f5-8dbf-457c-a978-510436bccf11 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:26:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:65acc1f5-8dbf-457c-a978-510436bccf11 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.018s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:26:07 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[71b7c66b-56b1-447f-ab01-a2a757c2bed9]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:26:07 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-b8d2f0de-5dee-497f-bfb2-fe1fc354fec8 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:26:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:65acc1f5-8dbf-457c-a978-510436bccf11 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:26:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:65acc1f5-8dbf-457c-a978-510436bccf11 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.019s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:26:07 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[9bf0707a-050f-4679-b4b9-ec3a4b9e2f59]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:26:07 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-b8d2f0de-5dee-497f-bfb2-fe1fc354fec8 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:26:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:65acc1f5-8dbf-457c-a978-510436bccf11 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v jqCadzEM {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:26:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:65acc1f5-8dbf-457c-a978-510436bccf11 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v jqCadzEM" returned: 0 in 0.019s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:26:07 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[305b0847-d5c1-4cb3-9eee-3a062dbdbb13]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:26:07 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-b8d2f0de-5dee-497f-bfb2-fe1fc354fec8 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'jqCadzEM'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:26:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:65acc1f5-8dbf-457c-a978-510436bccf11 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:26:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:65acc1f5-8dbf-457c-a978-510436bccf11 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.018s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:26:07 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[38b1115a-4f4f-4434-9108-43c786de3f2c]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:26:07 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-b8d2f0de-5dee-497f-bfb2-fe1fc354fec8 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:26:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:26:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/65d48c93-b6da-4f51-9d5a-0676dda62a60/disk --force-share --output=json" returned: 0 in 0.178s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:26:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/65d48c93-b6da-4f51-9d5a-0676dda62a60/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:26:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.024s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:26:07 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[f1d0fa9f-fef1-4517-ac5d-f41f1cffb84c]: (4, ('tcp: [42] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0 (non-flash)\ntcp: [46] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ee672802-fcc5-4892-8a23-91763600da13 (non-flash)\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:26:07 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-b8d2f0de-5dee-497f-bfb2-fe1fc354fec8 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [42] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0 (non-flash)
Nov 29 03:26:07 user nova-compute[75234]: tcp: [46] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ee672802-fcc5-4892-8a23-91763600da13 (non-flash)
Nov 29 03:26:07 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:26:07 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-b8d2f0de-5dee-497f-bfb2-fe1fc354fec8 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] iscsi session list stdout=tcp: [42] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0 (non-flash)
Nov 29 03:26:07 user nova-compute[75234]: tcp: [46] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ee672802-fcc5-4892-8a23-91763600da13 (non-flash)
Nov 29 03:26:07 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsi_session /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Nov 29 03:26:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:65acc1f5-8dbf-457c-a978-510436bccf11 -p 172.16.0.220:3260 --login {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:26:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:65acc1f5-8dbf-457c-a978-510436bccf11 -p 172.16.0.220:3260 --login" returned: 0 in 0.058s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:26:07 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[9112d5c0-d1f7-4349-b5b3-5d1d3e2a6624]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:65acc1f5-8dbf-457c-a978-510436bccf11, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:65acc1f5-8dbf-457c-a978-510436bccf11, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:26:07 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-b8d2f0de-5dee-497f-bfb2-fe1fc354fec8 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:65acc1f5-8dbf-457c-a978-510436bccf11, portal: 172.16.0.220,3260]
Nov 29 03:26:07 user nova-compute[75234]: Login to [iface: default, target: iqn.2010-10.org.openstack:65acc1f5-8dbf-457c-a978-510436bccf11, portal: 172.16.0.220,3260] successful.
Nov 29 03:26:07 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:26:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:65acc1f5-8dbf-457c-a978-510436bccf11 -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:26:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:65acc1f5-8dbf-457c-a978-510436bccf11 -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.018s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:26:07 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[8b4e0532-6983-4fb7-bf9a-b777eaa1308f]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:26:07 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-b8d2f0de-5dee-497f-bfb2-fe1fc354fec8 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:26:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:26:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.023s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:26:07 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[c3d548ad-5076-4a5e-b6d5-0b3869540179]: (4, ('tcp: [42] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0 (non-flash)\ntcp: [46] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ee672802-fcc5-4892-8a23-91763600da13 (non-flash)\ntcp: [47] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:65acc1f5-8dbf-457c-a978-510436bccf11 (non-flash)\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:26:07 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-b8d2f0de-5dee-497f-bfb2-fe1fc354fec8 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [42] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0 (non-flash)
Nov 29 03:26:07 user nova-compute[75234]: tcp: [46] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ee672802-fcc5-4892-8a23-91763600da13 (non-flash)
Nov 29 03:26:07 user nova-compute[75234]: tcp: [47] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:65acc1f5-8dbf-457c-a978-510436bccf11 (non-flash)
Nov 29 03:26:07 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:26:07 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-b8d2f0de-5dee-497f-bfb2-fe1fc354fec8 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] iscsi session list stdout=tcp: [42] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0 (non-flash)
Nov 29 03:26:07 user nova-compute[75234]: tcp: [46] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ee672802-fcc5-4892-8a23-91763600da13 (non-flash)
Nov 29 03:26:07 user nova-compute[75234]: tcp: [47] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:65acc1f5-8dbf-457c-a978-510436bccf11 (non-flash)
Nov 29 03:26:07 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsi_session /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Nov 29 03:26:07 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-b8d2f0de-5dee-497f-bfb2-fe1fc354fec8 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:65acc1f5-8dbf-457c-a978-510436bccf11" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.344s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:87}}
Nov 29 03:26:07 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-b8d2f0de-5dee-497f-bfb2-fe1fc354fec8 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Connected to 172.16.0.220:3260 {{(pid=75234) _connect_vol /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:644}}
Nov 29 03:26:07 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-b8d2f0de-5dee-497f-bfb2-fe1fc354fec8 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] HCTL ('33', '-', '-', 0) found on session 47 with lun 0 {{(pid=75234) get_hctl /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:770}}
Nov 29 03:26:07 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-b8d2f0de-5dee-497f-bfb2-fe1fc354fec8 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Scanning host 33 c: -, t: -, l: 0) {{(pid=75234) scan_iscsi /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:795}}
Nov 29 03:26:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host33/scan {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:26:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/65d48c93-b6da-4f51-9d5a-0676dda62a60/disk --force-share --output=json" returned: 0 in 0.190s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:26:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host33/scan" returned: 0 in 0.034s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:26:07 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[1137a611-1f84-4ff4-9694-34c4981a14e0]: (4, ('- - 0', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:26:07 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-b8d2f0de-5dee-497f-bfb2-fe1fc354fec8 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Searching for a device in session 47 and hctl ['33', '*', '*', 0] yield: sdb {{(pid=75234) device_name_by_hctl /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:789}}
Nov 29 03:26:07 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-b8d2f0de-5dee-497f-bfb2-fe1fc354fec8 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Connected to sdb using {'target_iqn': 'iqn.2010-10.org.openstack:65acc1f5-8dbf-457c-a978-510436bccf11', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'jqCadzEM', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path'} {{(pid=75234) _connect_vol /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:672}}
Nov 29 03:26:07 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] skipping disk /dev/sdc (vda) as it is a volume {{(pid=75234) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11213}}
Nov 29 03:26:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/d5096ca2-d005-45ac-a487-11a810446859/disk.rescue --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:26:08 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/d5096ca2-d005-45ac-a487-11a810446859/disk.rescue --force-share --output=json" returned: 0 in 0.181s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:26:08 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/d5096ca2-d005-45ac-a487-11a810446859/disk.rescue --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:26:08 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/d5096ca2-d005-45ac-a487-11a810446859/disk.rescue --force-share --output=json" returned: 0 in 0.185s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:26:08 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/13e5da64-9adc-4479-81c7-d329e9b5f08f/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:26:08 user nova-compute[75234]: DEBUG nova.compute.manager [req-2f0a2493-e71e-4d52-9140-699f26a19ffa req-3e59a375-da88-482c-88c4-e648e82403e7 service nova] [instance: 5d7d744e-1a5f-40ab-b961-4b58c183efe0] Received event network-vif-plugged-fa65b32f-b627-45fa-8d6d-d99012ca1262 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:26:08 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-2f0a2493-e71e-4d52-9140-699f26a19ffa req-3e59a375-da88-482c-88c4-e648e82403e7 service nova] Acquiring lock "5d7d744e-1a5f-40ab-b961-4b58c183efe0-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:26:08 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-2f0a2493-e71e-4d52-9140-699f26a19ffa req-3e59a375-da88-482c-88c4-e648e82403e7 service nova] Lock "5d7d744e-1a5f-40ab-b961-4b58c183efe0-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:26:08 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-2f0a2493-e71e-4d52-9140-699f26a19ffa req-3e59a375-da88-482c-88c4-e648e82403e7 service nova] Lock "5d7d744e-1a5f-40ab-b961-4b58c183efe0-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:26:08 user nova-compute[75234]: DEBUG nova.compute.manager [req-2f0a2493-e71e-4d52-9140-699f26a19ffa req-3e59a375-da88-482c-88c4-e648e82403e7 service nova] [instance: 5d7d744e-1a5f-40ab-b961-4b58c183efe0] No waiting events found dispatching network-vif-plugged-fa65b32f-b627-45fa-8d6d-d99012ca1262 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:26:08 user nova-compute[75234]: WARNING nova.compute.manager [req-2f0a2493-e71e-4d52-9140-699f26a19ffa req-3e59a375-da88-482c-88c4-e648e82403e7 service nova] [instance: 5d7d744e-1a5f-40ab-b961-4b58c183efe0] Received unexpected event network-vif-plugged-fa65b32f-b627-45fa-8d6d-d99012ca1262 for instance with vm_state active and task_state reboot_started_hard.
Nov 29 03:26:08 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/13e5da64-9adc-4479-81c7-d329e9b5f08f/disk --force-share --output=json" returned: 0 in 0.169s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:26:08 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/13e5da64-9adc-4479-81c7-d329e9b5f08f/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:26:08 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/13e5da64-9adc-4479-81c7-d329e9b5f08f/disk --force-share --output=json" returned: 0 in 0.169s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:26:08 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a082c96e-e384-40b6-bcc8-fc5e0e104669/disk.rescue --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:26:08 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a082c96e-e384-40b6-bcc8-fc5e0e104669/disk.rescue --force-share --output=json" returned: 0 in 0.181s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:26:08 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a082c96e-e384-40b6-bcc8-fc5e0e104669/disk.rescue --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:26:08 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-b8d2f0de-5dee-497f-bfb2-fe1fc354fec8 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.402s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:87}}
Nov 29 03:26:08 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-b8d2f0de-5dee-497f-bfb2-fe1fc354fec8 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] <== connect_volume: return (1406ms) {'type': 'block', 'scsi_wwn': '23236633763636566', 'path': '/dev/sdb'} {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:203}}
Nov 29 03:26:08 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-b8d2f0de-5dee-497f-bfb2-fe1fc354fec8 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] [instance: 5d7d744e-1a5f-40ab-b961-4b58c183efe0] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '23236633763636566', 'path': '/dev/sdb'} {{(pid=75234) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Nov 29 03:26:08 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-b8d2f0de-5dee-497f-bfb2-fe1fc354fec8 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-11-29T03:23:45Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-TestMinimumBasicScenario-server-310577704',display_name='tempest-TestMinimumBasicScenario-server-310577704',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-testminimumbasicscenario-server-310577704',id=39,image_ref='7107c48b-05b9-4052-917d-814b721d169c',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBBTfFCQxMn5FN9HRzHRLmeLvGNhlvH8jkTHWDyEbYwE8hTZhRqGybcRgcmJ5xQB1zSttCcEaPSgc17S9QrgRI43nIUg3IFEEzmkOkz4/kzJTs5GShIWM4PGtgtnmC06Ugw==',key_name='tempest-TestMinimumBasicScenario-44553634',keypairs=<?>,launch_index=0,launched_at=2023-11-29T03:23:53Z,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='74f132d6666c47b8b43869df6fee6de3',ramdisk_id='',reservation_id='r-dj1tj6j2',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='7107c48b-05b9-4052-917d-814b721d169c',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',owner_project_name='tempest-TestMinimumBasicScenario-1635847764',owner_user_name='tempest-TestMinimumBasicScenario-1635847764-project-member'},tags=<?>,task_state='reboot_started_hard',terminated_at=None,trusted_certs=<?>,updated_at=2023-11-29T03:26:06Z,user_data=None,user_id='3a02618f56ea45898ab4783a4f198b5d',uuid=5d7d744e-1a5f-40ab-b961-4b58c183efe0,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "fa65b32f-b627-45fa-8d6d-d99012ca1262", "address": "fa:16:3e:2f:fa:59", "network": {"id": "6787e0a8-2980-4a2e-bf2a-583af955241d", "bridge": "br-int", "label": "tempest-TestMinimumBasicScenario-1273600998-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.111", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "74f132d6666c47b8b43869df6fee6de3", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapfa65b32f-b6", "ovs_interfaceid": "fa65b32f-b627-45fa-8d6d-d99012ca1262", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=75234) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Nov 29 03:26:08 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-b8d2f0de-5dee-497f-bfb2-fe1fc354fec8 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Converting VIF {"id": "fa65b32f-b627-45fa-8d6d-d99012ca1262", "address": "fa:16:3e:2f:fa:59", "network": {"id": "6787e0a8-2980-4a2e-bf2a-583af955241d", "bridge": "br-int", "label": "tempest-TestMinimumBasicScenario-1273600998-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.111", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "74f132d6666c47b8b43869df6fee6de3", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapfa65b32f-b6", "ovs_interfaceid": "fa65b32f-b627-45fa-8d6d-d99012ca1262", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:26:08 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-b8d2f0de-5dee-497f-bfb2-fe1fc354fec8 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:2f:fa:59,bridge_name='br-int',has_traffic_filtering=True,id=fa65b32f-b627-45fa-8d6d-d99012ca1262,network=Network(6787e0a8-2980-4a2e-bf2a-583af955241d),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapfa65b32f-b6') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:26:09 user nova-compute[75234]: DEBUG nova.objects.instance [None req-b8d2f0de-5dee-497f-bfb2-fe1fc354fec8 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Lazy-loading 'pci_devices' on Instance uuid 5d7d744e-1a5f-40ab-b961-4b58c183efe0 {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:26:09 user nova-compute[75234]: DEBUG nova.objects.base [None req-b8d2f0de-5dee-497f-bfb2-fe1fc354fec8 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Object Instance<5d7d744e-1a5f-40ab-b961-4b58c183efe0> lazy-loaded attributes: resources,vcpu_model,pci_devices {{(pid=75234) wrapper /opt/stack/nova/nova/objects/base.py:126}}
Nov 29 03:26:09 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-b8d2f0de-5dee-497f-bfb2-fe1fc354fec8 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] [instance: 5d7d744e-1a5f-40ab-b961-4b58c183efe0] End _get_guest_xml xml=<domain type="kvm">
Nov 29 03:26:09 user nova-compute[75234]:   <uuid>5d7d744e-1a5f-40ab-b961-4b58c183efe0</uuid>
Nov 29 03:26:09 user nova-compute[75234]:   <name>instance-00000027</name>
Nov 29 03:26:09 user nova-compute[75234]:   <memory>524288</memory>
Nov 29 03:26:09 user nova-compute[75234]:   <vcpu>1</vcpu>
Nov 29 03:26:09 user nova-compute[75234]:   <metadata>
Nov 29 03:26:09 user nova-compute[75234]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Nov 29 03:26:09 user nova-compute[75234]:       <nova:package version="0.0.0"/>
Nov 29 03:26:09 user nova-compute[75234]:       <nova:name>tempest-TestMinimumBasicScenario-server-310577704</nova:name>
Nov 29 03:26:09 user nova-compute[75234]:       <nova:creationTime>2023-11-29 03:26:07</nova:creationTime>
Nov 29 03:26:09 user nova-compute[75234]:       <nova:flavor name="m1.tiny">
Nov 29 03:26:09 user nova-compute[75234]:         <nova:memory>512</nova:memory>
Nov 29 03:26:09 user nova-compute[75234]:         <nova:disk>1</nova:disk>
Nov 29 03:26:09 user nova-compute[75234]:         <nova:swap>0</nova:swap>
Nov 29 03:26:09 user nova-compute[75234]:         <nova:ephemeral>0</nova:ephemeral>
Nov 29 03:26:09 user nova-compute[75234]:         <nova:vcpus>1</nova:vcpus>
Nov 29 03:26:09 user nova-compute[75234]:       </nova:flavor>
Nov 29 03:26:09 user nova-compute[75234]:       <nova:owner>
Nov 29 03:26:09 user nova-compute[75234]:         <nova:user uuid="3a02618f56ea45898ab4783a4f198b5d">tempest-TestMinimumBasicScenario-1635847764-project-member</nova:user>
Nov 29 03:26:09 user nova-compute[75234]:         <nova:project uuid="74f132d6666c47b8b43869df6fee6de3">tempest-TestMinimumBasicScenario-1635847764</nova:project>
Nov 29 03:26:09 user nova-compute[75234]:       </nova:owner>
Nov 29 03:26:09 user nova-compute[75234]:       <nova:root type="image" uuid="7107c48b-05b9-4052-917d-814b721d169c"/>
Nov 29 03:26:09 user nova-compute[75234]:       <nova:ports>
Nov 29 03:26:09 user nova-compute[75234]:         <nova:port uuid="fa65b32f-b627-45fa-8d6d-d99012ca1262">
Nov 29 03:26:09 user nova-compute[75234]:           <nova:ip type="fixed" address="10.0.0.3" ipVersion="4"/>
Nov 29 03:26:09 user nova-compute[75234]:         </nova:port>
Nov 29 03:26:09 user nova-compute[75234]:       </nova:ports>
Nov 29 03:26:09 user nova-compute[75234]:     </nova:instance>
Nov 29 03:26:09 user nova-compute[75234]:   </metadata>
Nov 29 03:26:09 user nova-compute[75234]:   <sysinfo type="smbios">
Nov 29 03:26:09 user nova-compute[75234]:     <system>
Nov 29 03:26:09 user nova-compute[75234]:       <entry name="manufacturer">OpenStack Foundation</entry>
Nov 29 03:26:09 user nova-compute[75234]:       <entry name="product">OpenStack Nova</entry>
Nov 29 03:26:09 user nova-compute[75234]:       <entry name="version">0.0.0</entry>
Nov 29 03:26:09 user nova-compute[75234]:       <entry name="serial">5d7d744e-1a5f-40ab-b961-4b58c183efe0</entry>
Nov 29 03:26:09 user nova-compute[75234]:       <entry name="uuid">5d7d744e-1a5f-40ab-b961-4b58c183efe0</entry>
Nov 29 03:26:09 user nova-compute[75234]:       <entry name="family">Virtual Machine</entry>
Nov 29 03:26:09 user nova-compute[75234]:     </system>
Nov 29 03:26:09 user nova-compute[75234]:   </sysinfo>
Nov 29 03:26:09 user nova-compute[75234]:   <os>
Nov 29 03:26:09 user nova-compute[75234]:     <type arch="x86_64" machine="pc">hvm</type>
Nov 29 03:26:09 user nova-compute[75234]:     <boot dev="hd"/>
Nov 29 03:26:09 user nova-compute[75234]:     <smbios mode="sysinfo"/>
Nov 29 03:26:09 user nova-compute[75234]:   </os>
Nov 29 03:26:09 user nova-compute[75234]:   <features>
Nov 29 03:26:09 user nova-compute[75234]:     <acpi/>
Nov 29 03:26:09 user nova-compute[75234]:     <apic/>
Nov 29 03:26:09 user nova-compute[75234]:     <vmcoreinfo/>
Nov 29 03:26:09 user nova-compute[75234]:   </features>
Nov 29 03:26:09 user nova-compute[75234]:   <clock offset="utc">
Nov 29 03:26:09 user nova-compute[75234]:     <timer name="pit" tickpolicy="delay"/>
Nov 29 03:26:09 user nova-compute[75234]:     <timer name="rtc" tickpolicy="catchup"/>
Nov 29 03:26:09 user nova-compute[75234]:     <timer name="hpet" present="no"/>
Nov 29 03:26:09 user nova-compute[75234]:   </clock>
Nov 29 03:26:09 user nova-compute[75234]:   <cpu mode="custom" match="exact">
Nov 29 03:26:09 user nova-compute[75234]:     <model>Nehalem</model>
Nov 29 03:26:09 user nova-compute[75234]:     <topology sockets="1" cores="1" threads="1"/>
Nov 29 03:26:09 user nova-compute[75234]:   </cpu>
Nov 29 03:26:09 user nova-compute[75234]:   <devices>
Nov 29 03:26:09 user nova-compute[75234]:     <disk type="file" device="disk">
Nov 29 03:26:09 user nova-compute[75234]:       <driver name="qemu" type="qcow2" cache="none"/>
Nov 29 03:26:09 user nova-compute[75234]:       <source file="/opt/stack/data/nova/instances/5d7d744e-1a5f-40ab-b961-4b58c183efe0/disk"/>
Nov 29 03:26:09 user nova-compute[75234]:       <target dev="vda" bus="virtio"/>
Nov 29 03:26:09 user nova-compute[75234]:     </disk>
Nov 29 03:26:09 user nova-compute[75234]:     <disk type="block" device="disk">
Nov 29 03:26:09 user nova-compute[75234]:       <driver name="qemu" type="raw" cache="none" io="native"/>
Nov 29 03:26:09 user nova-compute[75234]:       <source dev="/dev/sdb"/>
Nov 29 03:26:09 user nova-compute[75234]:       <target dev="vdb" bus="virtio"/>
Nov 29 03:26:09 user nova-compute[75234]:       <serial>65acc1f5-8dbf-457c-a978-510436bccf11</serial>
Nov 29 03:26:09 user nova-compute[75234]:     </disk>
Nov 29 03:26:09 user nova-compute[75234]:     <interface type="ethernet">
Nov 29 03:26:09 user nova-compute[75234]:       <mac address="fa:16:3e:2f:fa:59"/>
Nov 29 03:26:09 user nova-compute[75234]:       <model type="virtio"/>
Nov 29 03:26:09 user nova-compute[75234]:       <mtu size="1442"/>
Nov 29 03:26:09 user nova-compute[75234]:       <target dev="tapfa65b32f-b6"/>
Nov 29 03:26:09 user nova-compute[75234]:     </interface>
Nov 29 03:26:09 user nova-compute[75234]:     <serial type="pty">
Nov 29 03:26:09 user nova-compute[75234]:       <log file="/opt/stack/data/nova/instances/5d7d744e-1a5f-40ab-b961-4b58c183efe0/console.log" append="off"/>
Nov 29 03:26:09 user nova-compute[75234]:     </serial>
Nov 29 03:26:09 user nova-compute[75234]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Nov 29 03:26:09 user nova-compute[75234]:     <video>
Nov 29 03:26:09 user nova-compute[75234]:       <model type="virtio"/>
Nov 29 03:26:09 user nova-compute[75234]:     </video>
Nov 29 03:26:09 user nova-compute[75234]:     <rng model="virtio">
Nov 29 03:26:09 user nova-compute[75234]:       <backend model="random">/dev/urandom</backend>
Nov 29 03:26:09 user nova-compute[75234]:     </rng>
Nov 29 03:26:09 user nova-compute[75234]:     <controller type="usb" index="0" model="none"/>
Nov 29 03:26:09 user nova-compute[75234]:     <memballoon model="virtio">
Nov 29 03:26:09 user nova-compute[75234]:       <stats period="10"/>
Nov 29 03:26:09 user nova-compute[75234]:     </memballoon>
Nov 29 03:26:09 user nova-compute[75234]:   </devices>
Nov 29 03:26:09 user nova-compute[75234]: </domain>
Nov 29 03:26:09 user nova-compute[75234]:  {{(pid=75234) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7568}}
Nov 29 03:26:09 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-b8d2f0de-5dee-497f-bfb2-fe1fc354fec8 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/5d7d744e-1a5f-40ab-b961-4b58c183efe0/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:26:09 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a082c96e-e384-40b6-bcc8-fc5e0e104669/disk.rescue --force-share --output=json" returned: 0 in 0.202s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:26:09 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a082c96e-e384-40b6-bcc8-fc5e0e104669/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:26:09 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-b8d2f0de-5dee-497f-bfb2-fe1fc354fec8 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/5d7d744e-1a5f-40ab-b961-4b58c183efe0/disk --force-share --output=json" returned: 0 in 0.184s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:26:09 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-b8d2f0de-5dee-497f-bfb2-fe1fc354fec8 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/5d7d744e-1a5f-40ab-b961-4b58c183efe0/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:26:09 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a082c96e-e384-40b6-bcc8-fc5e0e104669/disk --force-share --output=json" returned: 0 in 0.193s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:26:09 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a082c96e-e384-40b6-bcc8-fc5e0e104669/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:26:09 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-b8d2f0de-5dee-497f-bfb2-fe1fc354fec8 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/5d7d744e-1a5f-40ab-b961-4b58c183efe0/disk --force-share --output=json" returned: 0 in 0.187s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:26:09 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-b8d2f0de-5dee-497f-bfb2-fe1fc354fec8 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] skipping disk /dev/sdb (vdb) as it is a volume {{(pid=75234) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11213}}
Nov 29 03:26:09 user nova-compute[75234]: DEBUG nova.objects.instance [None req-b8d2f0de-5dee-497f-bfb2-fe1fc354fec8 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Lazy-loading 'trusted_certs' on Instance uuid 5d7d744e-1a5f-40ab-b961-4b58c183efe0 {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:26:09 user nova-compute[75234]: DEBUG nova.objects.base [None req-b8d2f0de-5dee-497f-bfb2-fe1fc354fec8 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Object Instance<5d7d744e-1a5f-40ab-b961-4b58c183efe0> lazy-loaded attributes: resources,vcpu_model,pci_devices,trusted_certs {{(pid=75234) wrapper /opt/stack/nova/nova/objects/base.py:126}}
Nov 29 03:26:09 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-b8d2f0de-5dee-497f-bfb2-fe1fc354fec8 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/4d5d356d29424aea82c5ad651579c796fc09ff4d --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:26:09 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a082c96e-e384-40b6-bcc8-fc5e0e104669/disk --force-share --output=json" returned: 0 in 0.201s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:26:09 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Error from libvirt while getting description of instance-00000027: [Error Code 42] Domain not found: no domain with matching uuid '5d7d744e-1a5f-40ab-b961-4b58c183efe0' (instance-00000027): libvirt.libvirtError: Domain not found: no domain with matching uuid '5d7d744e-1a5f-40ab-b961-4b58c183efe0' (instance-00000027)
Nov 29 03:26:09 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-b8d2f0de-5dee-497f-bfb2-fe1fc354fec8 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/4d5d356d29424aea82c5ad651579c796fc09ff4d --force-share --output=json" returned: 0 in 0.492s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:26:09 user nova-compute[75234]: DEBUG nova.virt.disk.api [None req-b8d2f0de-5dee-497f-bfb2-fe1fc354fec8 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Checking if we can resize image /opt/stack/data/nova/instances/5d7d744e-1a5f-40ab-b961-4b58c183efe0/disk. size=1073741824 {{(pid=75234) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:166}}
Nov 29 03:26:09 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-b8d2f0de-5dee-497f-bfb2-fe1fc354fec8 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/5d7d744e-1a5f-40ab-b961-4b58c183efe0/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:26:09 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:26:09 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:26:09 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Hypervisor/Node resource view: name=user free_ram=8264MB free_disk=25.063480377197266GB free_vcpus=7 pci_devices=[{"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}] {{(pid=75234) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1106}}
Nov 29 03:26:09 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:26:09 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:26:10 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance 13e5da64-9adc-4479-81c7-d329e9b5f08f actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 512, 'DISK_GB': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:26:10 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance a082c96e-e384-40b6-bcc8-fc5e0e104669 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 512, 'DISK_GB': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:26:10 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance 65d48c93-b6da-4f51-9d5a-0676dda62a60 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 512, 'DISK_GB': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:26:10 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance f6bc212d-85cd-479c-92ae-112c9f0101bc actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 512, 'DISK_GB': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:26:10 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance 5d7d744e-1a5f-40ab-b961-4b58c183efe0 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 512, 'DISK_GB': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:26:10 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance d5096ca2-d005-45ac-a487-11a810446859 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 512}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:26:10 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Total usable vcpus: 12, total allocated vcpus: 6 {{(pid=75234) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1129}}
Nov 29 03:26:10 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Final resource view: name=user phys_ram=16011MB used_ram=3584MB phys_disk=40GB used_disk=5GB total_vcpus=12 used_vcpus=6 pci_stats=[] {{(pid=75234) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1138}}
Nov 29 03:26:10 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-b8d2f0de-5dee-497f-bfb2-fe1fc354fec8 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/5d7d744e-1a5f-40ab-b961-4b58c183efe0/disk --force-share --output=json" returned: 0 in 0.206s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:26:10 user nova-compute[75234]: DEBUG nova.virt.disk.api [None req-b8d2f0de-5dee-497f-bfb2-fe1fc354fec8 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Cannot resize image /opt/stack/data/nova/instances/5d7d744e-1a5f-40ab-b961-4b58c183efe0/disk to a smaller size. {{(pid=75234) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:172}}
Nov 29 03:26:10 user nova-compute[75234]: DEBUG nova.objects.instance [None req-b8d2f0de-5dee-497f-bfb2-fe1fc354fec8 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Lazy-loading 'migration_context' on Instance uuid 5d7d744e-1a5f-40ab-b961-4b58c183efe0 {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:26:10 user nova-compute[75234]: DEBUG nova.objects.base [None req-b8d2f0de-5dee-497f-bfb2-fe1fc354fec8 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Object Instance<5d7d744e-1a5f-40ab-b961-4b58c183efe0> lazy-loaded attributes: resources,vcpu_model,pci_devices,trusted_certs,migration_context {{(pid=75234) wrapper /opt/stack/nova/nova/objects/base.py:126}}
Nov 29 03:26:10 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-b8d2f0de-5dee-497f-bfb2-fe1fc354fec8 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-11-29T03:23:45Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-TestMinimumBasicScenario-server-310577704',display_name='tempest-TestMinimumBasicScenario-server-310577704',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-testminimumbasicscenario-server-310577704',id=39,image_ref='7107c48b-05b9-4052-917d-814b721d169c',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBBTfFCQxMn5FN9HRzHRLmeLvGNhlvH8jkTHWDyEbYwE8hTZhRqGybcRgcmJ5xQB1zSttCcEaPSgc17S9QrgRI43nIUg3IFEEzmkOkz4/kzJTs5GShIWM4PGtgtnmC06Ugw==',key_name='tempest-TestMinimumBasicScenario-44553634',keypairs=<?>,launch_index=0,launched_at=2023-11-29T03:23:53Z,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=<?>,power_state=1,progress=0,project_id='74f132d6666c47b8b43869df6fee6de3',ramdisk_id='',reservation_id='r-dj1tj6j2',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='7107c48b-05b9-4052-917d-814b721d169c',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',owner_project_name='tempest-TestMinimumBasicScenario-1635847764',owner_user_name='tempest-TestMinimumBasicScenario-1635847764-project-member'},tags=<?>,task_state='reboot_started_hard',terminated_at=None,trusted_certs=None,updated_at=2023-11-29T03:26:06Z,user_data=None,user_id='3a02618f56ea45898ab4783a4f198b5d',uuid=5d7d744e-1a5f-40ab-b961-4b58c183efe0,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "fa65b32f-b627-45fa-8d6d-d99012ca1262", "address": "fa:16:3e:2f:fa:59", "network": {"id": "6787e0a8-2980-4a2e-bf2a-583af955241d", "bridge": "br-int", "label": "tempest-TestMinimumBasicScenario-1273600998-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.111", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "74f132d6666c47b8b43869df6fee6de3", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapfa65b32f-b6", "ovs_interfaceid": "fa65b32f-b627-45fa-8d6d-d99012ca1262", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Nov 29 03:26:10 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-b8d2f0de-5dee-497f-bfb2-fe1fc354fec8 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Converting VIF {"id": "fa65b32f-b627-45fa-8d6d-d99012ca1262", "address": "fa:16:3e:2f:fa:59", "network": {"id": "6787e0a8-2980-4a2e-bf2a-583af955241d", "bridge": "br-int", "label": "tempest-TestMinimumBasicScenario-1273600998-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.111", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "74f132d6666c47b8b43869df6fee6de3", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapfa65b32f-b6", "ovs_interfaceid": "fa65b32f-b627-45fa-8d6d-d99012ca1262", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:26:10 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-b8d2f0de-5dee-497f-bfb2-fe1fc354fec8 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:2f:fa:59,bridge_name='br-int',has_traffic_filtering=True,id=fa65b32f-b627-45fa-8d6d-d99012ca1262,network=Network(6787e0a8-2980-4a2e-bf2a-583af955241d),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapfa65b32f-b6') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:26:10 user nova-compute[75234]: DEBUG os_vif [None req-b8d2f0de-5dee-497f-bfb2-fe1fc354fec8 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Plugging vif VIFOpenVSwitch(active=True,address=fa:16:3e:2f:fa:59,bridge_name='br-int',has_traffic_filtering=True,id=fa65b32f-b627-45fa-8d6d-d99012ca1262,network=Network(6787e0a8-2980-4a2e-bf2a-583af955241d),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapfa65b32f-b6') {{(pid=75234) plug /opt/stack/data/venv/lib/python3.10/site-packages/os_vif/__init__.py:76}}
Nov 29 03:26:10 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:26:10 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:26:10 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Nov 29 03:26:10 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:26:10 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbCreateCommand(_result=None, table=QoS, columns={'type': 'linux-noop', 'external_ids': {'id': 'bc1d6bff-df18-5391-8f57-ec3d535703ea', '_type': 'linux-noop'}}, row=False) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:26:10 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:26:10 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:26:10 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:26:10 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tapfa65b32f-b6, may_exist=True, interface_attrs={}) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:26:10 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Port, record=tapfa65b32f-b6, col_values=(('qos', UUID('6ad2a50c-0354-48b2-9234-7766b33e3a56')),), if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:26:10 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DbSetCommand(_result=None, table=Interface, record=tapfa65b32f-b6, col_values=(('external_ids', {'iface-id': 'fa65b32f-b627-45fa-8d6d-d99012ca1262', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:2f:fa:59', 'vm-uuid': '5d7d744e-1a5f-40ab-b961-4b58c183efe0'}),), if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:26:10 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:26:10 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:26:10 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:26:10 user nova-compute[75234]: INFO os_vif [None req-b8d2f0de-5dee-497f-bfb2-fe1fc354fec8 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Successfully plugged vif VIFOpenVSwitch(active=True,address=fa:16:3e:2f:fa:59,bridge_name='br-int',has_traffic_filtering=True,id=fa65b32f-b627-45fa-8d6d-d99012ca1262,network=Network(6787e0a8-2980-4a2e-bf2a-583af955241d),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapfa65b32f-b6')
Nov 29 03:26:10 user nova-compute[75234]: DEBUG nova.compute.provider_tree [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Inventory has not changed in ProviderTree for provider: 4d291220-7c37-4839-a0e3-b23c66502b74 {{(pid=75234) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Nov 29 03:26:10 user nova-compute[75234]: DEBUG nova.scheduler.client.report [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Inventory has not changed for provider 4d291220-7c37-4839-a0e3-b23c66502b74 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=75234) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Nov 29 03:26:10 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Compute_service record updated for user:user {{(pid=75234) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:1067}}
Nov 29 03:26:10 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.373s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:26:11 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:26:11 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Starting heal instance info cache {{(pid=75234) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9922}}
Nov 29 03:26:11 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Didn't find any instances for network info cache update. {{(pid=75234) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:10008}}
Nov 29 03:26:11 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:26:11 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:26:11 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:26:11 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:26:11 user nova-compute[75234]: DEBUG nova.compute.manager [req-2b2da49e-31ec-40fc-905b-61556ddc3b51 req-3456e6b2-6ed8-4571-a701-525165fe63bd service nova] [instance: 5d7d744e-1a5f-40ab-b961-4b58c183efe0] Received event network-vif-plugged-fa65b32f-b627-45fa-8d6d-d99012ca1262 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:26:11 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-2b2da49e-31ec-40fc-905b-61556ddc3b51 req-3456e6b2-6ed8-4571-a701-525165fe63bd service nova] Acquiring lock "5d7d744e-1a5f-40ab-b961-4b58c183efe0-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:26:11 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-2b2da49e-31ec-40fc-905b-61556ddc3b51 req-3456e6b2-6ed8-4571-a701-525165fe63bd service nova] Lock "5d7d744e-1a5f-40ab-b961-4b58c183efe0-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:26:11 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-2b2da49e-31ec-40fc-905b-61556ddc3b51 req-3456e6b2-6ed8-4571-a701-525165fe63bd service nova] Lock "5d7d744e-1a5f-40ab-b961-4b58c183efe0-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:26:11 user nova-compute[75234]: DEBUG nova.compute.manager [req-2b2da49e-31ec-40fc-905b-61556ddc3b51 req-3456e6b2-6ed8-4571-a701-525165fe63bd service nova] [instance: 5d7d744e-1a5f-40ab-b961-4b58c183efe0] No waiting events found dispatching network-vif-plugged-fa65b32f-b627-45fa-8d6d-d99012ca1262 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:26:11 user nova-compute[75234]: WARNING nova.compute.manager [req-2b2da49e-31ec-40fc-905b-61556ddc3b51 req-3456e6b2-6ed8-4571-a701-525165fe63bd service nova] [instance: 5d7d744e-1a5f-40ab-b961-4b58c183efe0] Received unexpected event network-vif-plugged-fa65b32f-b627-45fa-8d6d-d99012ca1262 for instance with vm_state active and task_state reboot_started_hard.
Nov 29 03:26:12 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:26:12 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:26:12 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:26:12 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:26:12 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:26:12 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:26:13 user nova-compute[75234]: DEBUG nova.compute.manager [req-85010a7c-d4fc-48dd-885f-974f00556b6a req-1ff9d4de-c8e7-41cd-b9c0-4f38fee3219b service nova] [instance: 5d7d744e-1a5f-40ab-b961-4b58c183efe0] Received event network-vif-plugged-fa65b32f-b627-45fa-8d6d-d99012ca1262 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:26:13 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-85010a7c-d4fc-48dd-885f-974f00556b6a req-1ff9d4de-c8e7-41cd-b9c0-4f38fee3219b service nova] Acquiring lock "5d7d744e-1a5f-40ab-b961-4b58c183efe0-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:26:13 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-85010a7c-d4fc-48dd-885f-974f00556b6a req-1ff9d4de-c8e7-41cd-b9c0-4f38fee3219b service nova] Lock "5d7d744e-1a5f-40ab-b961-4b58c183efe0-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:26:13 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-85010a7c-d4fc-48dd-885f-974f00556b6a req-1ff9d4de-c8e7-41cd-b9c0-4f38fee3219b service nova] Lock "5d7d744e-1a5f-40ab-b961-4b58c183efe0-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:26:13 user nova-compute[75234]: DEBUG nova.compute.manager [req-85010a7c-d4fc-48dd-885f-974f00556b6a req-1ff9d4de-c8e7-41cd-b9c0-4f38fee3219b service nova] [instance: 5d7d744e-1a5f-40ab-b961-4b58c183efe0] No waiting events found dispatching network-vif-plugged-fa65b32f-b627-45fa-8d6d-d99012ca1262 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:26:13 user nova-compute[75234]: WARNING nova.compute.manager [req-85010a7c-d4fc-48dd-885f-974f00556b6a req-1ff9d4de-c8e7-41cd-b9c0-4f38fee3219b service nova] [instance: 5d7d744e-1a5f-40ab-b961-4b58c183efe0] Received unexpected event network-vif-plugged-fa65b32f-b627-45fa-8d6d-d99012ca1262 for instance with vm_state active and task_state reboot_started_hard.
Nov 29 03:26:13 user nova-compute[75234]: DEBUG nova.compute.manager [None req-b8d2f0de-5dee-497f-bfb2-fe1fc354fec8 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] [instance: 5d7d744e-1a5f-40ab-b961-4b58c183efe0] Instance event wait completed in 0 seconds for  {{(pid=75234) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Nov 29 03:26:13 user nova-compute[75234]: INFO nova.virt.libvirt.driver [-] [instance: 5d7d744e-1a5f-40ab-b961-4b58c183efe0] Instance rebooted successfully.
Nov 29 03:26:13 user nova-compute[75234]: DEBUG nova.compute.manager [None req-b8d2f0de-5dee-497f-bfb2-fe1fc354fec8 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] [instance: 5d7d744e-1a5f-40ab-b961-4b58c183efe0] Checking state {{(pid=75234) _get_power_state /opt/stack/nova/nova/compute/manager.py:1782}}
Nov 29 03:26:13 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-b8d2f0de-5dee-497f-bfb2-fe1fc354fec8 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Lock "5d7d744e-1a5f-40ab-b961-4b58c183efe0" "released" by "nova.compute.manager.ComputeManager.reboot_instance.<locals>.do_reboot_instance" :: held 9.034s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:26:14 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._sync_scheduler_instance_info {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:26:14 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:26:15 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:26:15 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:26:16 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-0a0433bb-8c43-4ec4-84a5-c17b7d061e19 tempest-VolumesNegativeTest-814389221 tempest-VolumesNegativeTest-814389221-project-member] Acquiring lock "a4897e0e-d29c-45b6-8f2f-8f5ff55d6a8f" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:26:16 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-0a0433bb-8c43-4ec4-84a5-c17b7d061e19 tempest-VolumesNegativeTest-814389221 tempest-VolumesNegativeTest-814389221-project-member] Lock "a4897e0e-d29c-45b6-8f2f-8f5ff55d6a8f" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.002s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:26:16 user nova-compute[75234]: DEBUG nova.compute.manager [None req-0a0433bb-8c43-4ec4-84a5-c17b7d061e19 tempest-VolumesNegativeTest-814389221 tempest-VolumesNegativeTest-814389221-project-member] [instance: a4897e0e-d29c-45b6-8f2f-8f5ff55d6a8f] Starting instance... {{(pid=75234) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2422}}
Nov 29 03:26:16 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-0a0433bb-8c43-4ec4-84a5-c17b7d061e19 tempest-VolumesNegativeTest-814389221 tempest-VolumesNegativeTest-814389221-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:26:16 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-0a0433bb-8c43-4ec4-84a5-c17b7d061e19 tempest-VolumesNegativeTest-814389221 tempest-VolumesNegativeTest-814389221-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:26:16 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-0a0433bb-8c43-4ec4-84a5-c17b7d061e19 tempest-VolumesNegativeTest-814389221 tempest-VolumesNegativeTest-814389221-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=75234) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2366}}
Nov 29 03:26:16 user nova-compute[75234]: INFO nova.compute.claims [None req-0a0433bb-8c43-4ec4-84a5-c17b7d061e19 tempest-VolumesNegativeTest-814389221 tempest-VolumesNegativeTest-814389221-project-member] [instance: a4897e0e-d29c-45b6-8f2f-8f5ff55d6a8f] Claim successful on node user
Nov 29 03:26:16 user nova-compute[75234]: DEBUG nova.compute.provider_tree [None req-0a0433bb-8c43-4ec4-84a5-c17b7d061e19 tempest-VolumesNegativeTest-814389221 tempest-VolumesNegativeTest-814389221-project-member] Inventory has not changed in ProviderTree for provider: 4d291220-7c37-4839-a0e3-b23c66502b74 {{(pid=75234) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Nov 29 03:26:16 user nova-compute[75234]: DEBUG nova.scheduler.client.report [None req-0a0433bb-8c43-4ec4-84a5-c17b7d061e19 tempest-VolumesNegativeTest-814389221 tempest-VolumesNegativeTest-814389221-project-member] Inventory has not changed for provider 4d291220-7c37-4839-a0e3-b23c66502b74 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=75234) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Nov 29 03:26:16 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-0a0433bb-8c43-4ec4-84a5-c17b7d061e19 tempest-VolumesNegativeTest-814389221 tempest-VolumesNegativeTest-814389221-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.416s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:26:16 user nova-compute[75234]: DEBUG nova.compute.manager [None req-0a0433bb-8c43-4ec4-84a5-c17b7d061e19 tempest-VolumesNegativeTest-814389221 tempest-VolumesNegativeTest-814389221-project-member] [instance: a4897e0e-d29c-45b6-8f2f-8f5ff55d6a8f] Start building networks asynchronously for instance. {{(pid=75234) _build_resources /opt/stack/nova/nova/compute/manager.py:2819}}
Nov 29 03:26:16 user nova-compute[75234]: DEBUG nova.compute.manager [None req-0a0433bb-8c43-4ec4-84a5-c17b7d061e19 tempest-VolumesNegativeTest-814389221 tempest-VolumesNegativeTest-814389221-project-member] [instance: a4897e0e-d29c-45b6-8f2f-8f5ff55d6a8f] Allocating IP information in the background. {{(pid=75234) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1972}}
Nov 29 03:26:16 user nova-compute[75234]: DEBUG nova.network.neutron [None req-0a0433bb-8c43-4ec4-84a5-c17b7d061e19 tempest-VolumesNegativeTest-814389221 tempest-VolumesNegativeTest-814389221-project-member] [instance: a4897e0e-d29c-45b6-8f2f-8f5ff55d6a8f] allocate_for_instance() {{(pid=75234) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1156}}
Nov 29 03:26:16 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-0a0433bb-8c43-4ec4-84a5-c17b7d061e19 tempest-VolumesNegativeTest-814389221 tempest-VolumesNegativeTest-814389221-project-member] [instance: a4897e0e-d29c-45b6-8f2f-8f5ff55d6a8f] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Nov 29 03:26:16 user nova-compute[75234]: DEBUG nova.compute.manager [None req-0a0433bb-8c43-4ec4-84a5-c17b7d061e19 tempest-VolumesNegativeTest-814389221 tempest-VolumesNegativeTest-814389221-project-member] [instance: a4897e0e-d29c-45b6-8f2f-8f5ff55d6a8f] Start building block device mappings for instance. {{(pid=75234) _build_resources /opt/stack/nova/nova/compute/manager.py:2854}}
Nov 29 03:26:16 user nova-compute[75234]: DEBUG nova.compute.manager [None req-0a0433bb-8c43-4ec4-84a5-c17b7d061e19 tempest-VolumesNegativeTest-814389221 tempest-VolumesNegativeTest-814389221-project-member] [instance: a4897e0e-d29c-45b6-8f2f-8f5ff55d6a8f] Start spawning the instance on the hypervisor. {{(pid=75234) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2628}}
Nov 29 03:26:16 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-0a0433bb-8c43-4ec4-84a5-c17b7d061e19 tempest-VolumesNegativeTest-814389221 tempest-VolumesNegativeTest-814389221-project-member] [instance: a4897e0e-d29c-45b6-8f2f-8f5ff55d6a8f] Creating instance directory {{(pid=75234) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4727}}
Nov 29 03:26:16 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-0a0433bb-8c43-4ec4-84a5-c17b7d061e19 tempest-VolumesNegativeTest-814389221 tempest-VolumesNegativeTest-814389221-project-member] [instance: a4897e0e-d29c-45b6-8f2f-8f5ff55d6a8f] Creating image(s)
Nov 29 03:26:16 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-0a0433bb-8c43-4ec4-84a5-c17b7d061e19 tempest-VolumesNegativeTest-814389221 tempest-VolumesNegativeTest-814389221-project-member] Acquiring lock "/opt/stack/data/nova/instances/a4897e0e-d29c-45b6-8f2f-8f5ff55d6a8f/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:26:16 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-0a0433bb-8c43-4ec4-84a5-c17b7d061e19 tempest-VolumesNegativeTest-814389221 tempest-VolumesNegativeTest-814389221-project-member] Lock "/opt/stack/data/nova/instances/a4897e0e-d29c-45b6-8f2f-8f5ff55d6a8f/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:26:16 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-0a0433bb-8c43-4ec4-84a5-c17b7d061e19 tempest-VolumesNegativeTest-814389221 tempest-VolumesNegativeTest-814389221-project-member] Lock "/opt/stack/data/nova/instances/a4897e0e-d29c-45b6-8f2f-8f5ff55d6a8f/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.003s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:26:16 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-0a0433bb-8c43-4ec4-84a5-c17b7d061e19 tempest-VolumesNegativeTest-814389221 tempest-VolumesNegativeTest-814389221-project-member] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:26:17 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-367d2a35-cd8d-427c-ae01-7a32522fce38 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Waiting for libvirt event about the detach of device vdb with device alias virtio-disk1 from instance f6bc212d-85cd-479c-92ae-112c9f0101bc is timed out.
Nov 29 03:26:17 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-367d2a35-cd8d-427c-ae01-7a32522fce38 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Successfully detached device vdb from instance f6bc212d-85cd-479c-92ae-112c9f0101bc from the live domain config.
Nov 29 03:26:17 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-367d2a35-cd8d-427c-ae01-7a32522fce38 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] [instance: f6bc212d-85cd-479c-92ae-112c9f0101bc] calling os-brick to detach iSCSI Volume {{(pid=75234) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Nov 29 03:26:17 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-367d2a35-cd8d-427c-ae01-7a32522fce38 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f8a5050c9a0>, {'target_iqn': 'iqn.2010-10.org.openstack:ee672802-fcc5-4892-8a23-91763600da13', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'ixauTRwW', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdd'}, None), 'kwargs': False}}" {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:176}}
Nov 29 03:26:17 user nova-compute[75234]: WARNING os_brick.initiator.connectors.base [None req-367d2a35-cd8d-427c-ae01-7a32522fce38 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Nov 29 03:26:17 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-367d2a35-cd8d-427c-ae01-7a32522fce38 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:68}}
Nov 29 03:26:17 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-367d2a35-cd8d-427c-ae01-7a32522fce38 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.002s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:73}}
Nov 29 03:26:17 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:26:17 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.022s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:26:17 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[c23748de-ee6c-45a0-8a75-57f6af43bc9e]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:ee672802-fcc5-4892-8a23-91763600da13\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:65acc1f5-8dbf-457c-a978-510436bccf11\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:26:17 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-367d2a35-cd8d-427c-ae01-7a32522fce38 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS:
Nov 29 03:26:17 user nova-compute[75234]: No targets found.
Nov 29 03:26:17 user nova-compute[75234]: iSNS:
Nov 29 03:26:17 user nova-compute[75234]: No targets found.
Nov 29 03:26:17 user nova-compute[75234]: STATIC:
Nov 29 03:26:17 user nova-compute[75234]: Target: iqn.2010-10.org.openstack:ee672802-fcc5-4892-8a23-91763600da13
Nov 29 03:26:17 user nova-compute[75234]:         Portal: 172.16.0.220:3260,-1
Nov 29 03:26:17 user nova-compute[75234]:                 Iface Name: default
Nov 29 03:26:17 user nova-compute[75234]: Target: iqn.2010-10.org.openstack:65acc1f5-8dbf-457c-a978-510436bccf11
Nov 29 03:26:17 user nova-compute[75234]:         Portal: 172.16.0.220:3260,-1
Nov 29 03:26:17 user nova-compute[75234]:                 Iface Name: default
Nov 29 03:26:17 user nova-compute[75234]: Target: iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0
Nov 29 03:26:17 user nova-compute[75234]:         Portal: 172.16.0.220:3260,-1
Nov 29 03:26:17 user nova-compute[75234]:                 Iface Name: default
Nov 29 03:26:17 user nova-compute[75234]: FIRMWARE:
Nov 29 03:26:17 user nova-compute[75234]: No targets found.
Nov 29 03:26:17 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:26:17 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-367d2a35-cd8d-427c-ae01-7a32522fce38 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Regex to get portals from discoverydb: ^SENDTARGETS:
Nov 29 03:26:17 user nova-compute[75234]: .*?^DiscoveryAddress: 172.16.0.220,3260.*?
Nov 29 03:26:17 user nova-compute[75234]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=75234) _get_discoverydb_portals /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:382}}
Nov 29 03:26:17 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-367d2a35-cd8d-427c-ae01-7a32522fce38 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:ee672802-fcc5-4892-8a23-91763600da13', 0)] {{(pid=75234) _get_connection_devices /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:830}}
Nov 29 03:26:17 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:26:17 user nova-compute[75234]: DEBUG nova.policy [None req-0a0433bb-8c43-4ec4-84a5-c17b7d061e19 tempest-VolumesNegativeTest-814389221 tempest-VolumesNegativeTest-814389221-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '3a791e338de34f48b8512360b2422c11', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5da24ca4ca0d43d2849bf605ee87fa64', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=75234) authorize /opt/stack/nova/nova/policy.py:203}}
Nov 29 03:26:17 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.018s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:26:17 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[86f1256d-0958-4a96-90d2-187d76df9596]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:65acc1f5-8dbf-457c-a978-510436bccf11\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:ee672802-fcc5-4892-8a23-91763600da13\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:26:17 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:26:17 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.023s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:26:17 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[d8ba44c8-eed7-48b1-8158-50b6cf55c627]: (4, ('tcp: [42] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0 (non-flash)\ntcp: [46] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ee672802-fcc5-4892-8a23-91763600da13 (non-flash)\ntcp: [47] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:65acc1f5-8dbf-457c-a978-510436bccf11 (non-flash)\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:26:17 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-367d2a35-cd8d-427c-ae01-7a32522fce38 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [42] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0 (non-flash)
Nov 29 03:26:17 user nova-compute[75234]: tcp: [46] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ee672802-fcc5-4892-8a23-91763600da13 (non-flash)
Nov 29 03:26:17 user nova-compute[75234]: tcp: [47] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:65acc1f5-8dbf-457c-a978-510436bccf11 (non-flash)
Nov 29 03:26:17 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:26:17 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-367d2a35-cd8d-427c-ae01-7a32522fce38 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] iscsi session list stdout=tcp: [42] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0 (non-flash)
Nov 29 03:26:17 user nova-compute[75234]: tcp: [46] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:ee672802-fcc5-4892-8a23-91763600da13 (non-flash)
Nov 29 03:26:17 user nova-compute[75234]: tcp: [47] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:65acc1f5-8dbf-457c-a978-510436bccf11 (non-flash)
Nov 29 03:26:17 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsi_session /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Nov 29 03:26:17 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-367d2a35-cd8d-427c-ae01-7a32522fce38 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7f8a5074a170>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:ee672802-fcc5-4892-8a23-91763600da13'): ({'sdd'}, set())}) {{(pid=75234) _get_connection_devices /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:863}}
Nov 29 03:26:17 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-367d2a35-cd8d-427c-ae01-7a32522fce38 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Removing single pathed devices sdd {{(pid=75234) remove_connection /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:374}}
Nov 29 03:26:17 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:26:17 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.020s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:26:17 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[bf1d55bd-cc37-463c-b222-b1a9a4d25392]: (4, ('path checker states:\nup                  3\n\npaths: 0\nbusy: False\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:26:17 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdd {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:26:17 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-0a0433bb-8c43-4ec4-84a5-c17b7d061e19 tempest-VolumesNegativeTest-814389221 tempest-VolumesNegativeTest-814389221-project-member] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json" returned: 0 in 0.195s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:26:17 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-0a0433bb-8c43-4ec4-84a5-c17b7d061e19 tempest-VolumesNegativeTest-814389221 tempest-VolumesNegativeTest-814389221-project-member] Acquiring lock "80686e29dc9db6600d03e85cb62af298e12d7918" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:26:17 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-0a0433bb-8c43-4ec4-84a5-c17b7d061e19 tempest-VolumesNegativeTest-814389221 tempest-VolumesNegativeTest-814389221-project-member] Lock "80686e29dc9db6600d03e85cb62af298e12d7918" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:26:17 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-0a0433bb-8c43-4ec4-84a5-c17b7d061e19 tempest-VolumesNegativeTest-814389221 tempest-VolumesNegativeTest-814389221-project-member] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:26:17 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdd" returned: 0 in 0.021s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:26:17 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[867c1b97-31e2-4dce-9a84-49a26d7cba22]: (4, ('ok\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:26:17 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-367d2a35-cd8d-427c-ae01-7a32522fce38 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Flushing IO for device /dev/sdd {{(pid=75234) flush_device_io /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:434}}
Nov 29 03:26:17 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdd {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:26:17 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdd" returned: 0 in 0.014s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:26:17 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[7d437ac7-b70f-4d8d-afd5-aba1f0c2122b]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:26:17 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-367d2a35-cd8d-427c-ae01-7a32522fce38 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Remove SCSI device /dev/sdd with /sys/block/sdd/device/delete {{(pid=75234) remove_scsi_device /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:148}}
Nov 29 03:26:17 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdd/device/delete {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:26:17 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdd/device/delete" returned: 0 in 0.050s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:26:17 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[0a15bb9e-aaa7-46e7-a36c-bb2ba5aaf8db]: (4, ('1', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:26:17 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-367d2a35-cd8d-427c-ae01-7a32522fce38 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Checking to see if SCSI volumes sdd have been removed. {{(pid=75234) wait_for_volumes_removal /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:156}}
Nov 29 03:26:17 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-367d2a35-cd8d-427c-ae01-7a32522fce38 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] SCSI volumes sdd have been removed. {{(pid=75234) wait_for_volumes_removal /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:166}}
Nov 29 03:26:17 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-367d2a35-cd8d-427c-ae01-7a32522fce38 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:ee672802-fcc5-4892-8a23-91763600da13')] {{(pid=75234) _disconnect_connection /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Nov 29 03:26:17 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:ee672802-fcc5-4892-8a23-91763600da13 -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:26:17 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:ee672802-fcc5-4892-8a23-91763600da13 -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.025s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:26:17 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[13b6647a-d275-4960-bbe5-3946e2b2f008]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:26:17 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-367d2a35-cd8d-427c-ae01-7a32522fce38 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:26:17 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:ee672802-fcc5-4892-8a23-91763600da13 -p 172.16.0.220:3260 --logout {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:26:17 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-0a0433bb-8c43-4ec4-84a5-c17b7d061e19 tempest-VolumesNegativeTest-814389221 tempest-VolumesNegativeTest-814389221-project-member] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json" returned: 0 in 0.218s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:26:17 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-0a0433bb-8c43-4ec4-84a5-c17b7d061e19 tempest-VolumesNegativeTest-814389221 tempest-VolumesNegativeTest-814389221-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918,backing_fmt=raw /opt/stack/data/nova/instances/a4897e0e-d29c-45b6-8f2f-8f5ff55d6a8f/disk 1073741824 {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:26:17 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:ee672802-fcc5-4892-8a23-91763600da13 -p 172.16.0.220:3260 --logout" returned: 0 in 0.064s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:26:17 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[19e3f2ee-a3ac-4301-901e-db433470183b]: (4, ('Logging out of session [sid: 46, target: iqn.2010-10.org.openstack:ee672802-fcc5-4892-8a23-91763600da13, portal: 172.16.0.220,3260]\nLogout of [sid: 46, target: iqn.2010-10.org.openstack:ee672802-fcc5-4892-8a23-91763600da13, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:26:17 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-367d2a35-cd8d-427c-ae01-7a32522fce38 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 46, target: iqn.2010-10.org.openstack:ee672802-fcc5-4892-8a23-91763600da13, portal: 172.16.0.220,3260]
Nov 29 03:26:17 user nova-compute[75234]: Logout of [sid: 46, target: iqn.2010-10.org.openstack:ee672802-fcc5-4892-8a23-91763600da13, portal: 172.16.0.220,3260] successful.
Nov 29 03:26:17 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:26:17 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:ee672802-fcc5-4892-8a23-91763600da13 -p 172.16.0.220:3260 --op delete {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:26:17 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:26:17 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-0a0433bb-8c43-4ec4-84a5-c17b7d061e19 tempest-VolumesNegativeTest-814389221 tempest-VolumesNegativeTest-814389221-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918,backing_fmt=raw /opt/stack/data/nova/instances/a4897e0e-d29c-45b6-8f2f-8f5ff55d6a8f/disk 1073741824" returned: 0 in 0.077s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:26:17 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-0a0433bb-8c43-4ec4-84a5-c17b7d061e19 tempest-VolumesNegativeTest-814389221 tempest-VolumesNegativeTest-814389221-project-member] Lock "80686e29dc9db6600d03e85cb62af298e12d7918" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.299s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:26:17 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-0a0433bb-8c43-4ec4-84a5-c17b7d061e19 tempest-VolumesNegativeTest-814389221 tempest-VolumesNegativeTest-814389221-project-member] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:26:17 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:ee672802-fcc5-4892-8a23-91763600da13 -p 172.16.0.220:3260 --op delete" returned: 0 in 0.024s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:26:17 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[99bdcb9c-5a30-4e5e-998b-68ef1b83e818]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:26:17 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-367d2a35-cd8d-427c-ae01-7a32522fce38 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:26:17 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-367d2a35-cd8d-427c-ae01-7a32522fce38 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.489s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:87}}
Nov 29 03:26:17 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-367d2a35-cd8d-427c-ae01-7a32522fce38 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] <== disconnect_volume: return (491ms) None {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:203}}
Nov 29 03:26:17 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-367d2a35-cd8d-427c-ae01-7a32522fce38 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] [instance: f6bc212d-85cd-479c-92ae-112c9f0101bc] Disconnected iSCSI Volume {{(pid=75234) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:79}}
Nov 29 03:26:17 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-0a0433bb-8c43-4ec4-84a5-c17b7d061e19 tempest-VolumesNegativeTest-814389221 tempest-VolumesNegativeTest-814389221-project-member] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json" returned: 0 in 0.243s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:26:17 user nova-compute[75234]: DEBUG nova.virt.disk.api [None req-0a0433bb-8c43-4ec4-84a5-c17b7d061e19 tempest-VolumesNegativeTest-814389221 tempest-VolumesNegativeTest-814389221-project-member] Checking if we can resize image /opt/stack/data/nova/instances/a4897e0e-d29c-45b6-8f2f-8f5ff55d6a8f/disk. size=1073741824 {{(pid=75234) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:166}}
Nov 29 03:26:17 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-0a0433bb-8c43-4ec4-84a5-c17b7d061e19 tempest-VolumesNegativeTest-814389221 tempest-VolumesNegativeTest-814389221-project-member] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a4897e0e-d29c-45b6-8f2f-8f5ff55d6a8f/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:26:17 user nova-compute[75234]: DEBUG nova.network.neutron [None req-0a0433bb-8c43-4ec4-84a5-c17b7d061e19 tempest-VolumesNegativeTest-814389221 tempest-VolumesNegativeTest-814389221-project-member] [instance: a4897e0e-d29c-45b6-8f2f-8f5ff55d6a8f] Successfully created port: 55748abc-3e82-40c3-a7ba-b16327b28e9e {{(pid=75234) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:548}}
Nov 29 03:26:17 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-0a0433bb-8c43-4ec4-84a5-c17b7d061e19 tempest-VolumesNegativeTest-814389221 tempest-VolumesNegativeTest-814389221-project-member] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a4897e0e-d29c-45b6-8f2f-8f5ff55d6a8f/disk --force-share --output=json" returned: 0 in 0.215s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:26:17 user nova-compute[75234]: DEBUG nova.virt.disk.api [None req-0a0433bb-8c43-4ec4-84a5-c17b7d061e19 tempest-VolumesNegativeTest-814389221 tempest-VolumesNegativeTest-814389221-project-member] Cannot resize image /opt/stack/data/nova/instances/a4897e0e-d29c-45b6-8f2f-8f5ff55d6a8f/disk to a smaller size. {{(pid=75234) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:172}}
Nov 29 03:26:17 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-0a0433bb-8c43-4ec4-84a5-c17b7d061e19 tempest-VolumesNegativeTest-814389221 tempest-VolumesNegativeTest-814389221-project-member] [instance: a4897e0e-d29c-45b6-8f2f-8f5ff55d6a8f] Created local disks {{(pid=75234) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4861}}
Nov 29 03:26:17 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-0a0433bb-8c43-4ec4-84a5-c17b7d061e19 tempest-VolumesNegativeTest-814389221 tempest-VolumesNegativeTest-814389221-project-member] [instance: a4897e0e-d29c-45b6-8f2f-8f5ff55d6a8f] Ensure instance console log exists: /opt/stack/data/nova/instances/a4897e0e-d29c-45b6-8f2f-8f5ff55d6a8f/console.log {{(pid=75234) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4613}}
Nov 29 03:26:17 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-0a0433bb-8c43-4ec4-84a5-c17b7d061e19 tempest-VolumesNegativeTest-814389221 tempest-VolumesNegativeTest-814389221-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:26:17 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-0a0433bb-8c43-4ec4-84a5-c17b7d061e19 tempest-VolumesNegativeTest-814389221 tempest-VolumesNegativeTest-814389221-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:26:17 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-0a0433bb-8c43-4ec4-84a5-c17b7d061e19 tempest-VolumesNegativeTest-814389221 tempest-VolumesNegativeTest-814389221-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:26:18 user nova-compute[75234]: DEBUG nova.objects.instance [None req-367d2a35-cd8d-427c-ae01-7a32522fce38 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Lazy-loading 'flavor' on Instance uuid f6bc212d-85cd-479c-92ae-112c9f0101bc {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:26:18 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-367d2a35-cd8d-427c-ae01-7a32522fce38 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Lock "f6bc212d-85cd-479c-92ae-112c9f0101bc" "released" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: held 22.283s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:26:18 user nova-compute[75234]: DEBUG nova.network.neutron [None req-0a0433bb-8c43-4ec4-84a5-c17b7d061e19 tempest-VolumesNegativeTest-814389221 tempest-VolumesNegativeTest-814389221-project-member] [instance: a4897e0e-d29c-45b6-8f2f-8f5ff55d6a8f] Successfully updated port: 55748abc-3e82-40c3-a7ba-b16327b28e9e {{(pid=75234) _update_port /opt/stack/nova/nova/network/neutron.py:586}}
Nov 29 03:26:18 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-0a0433bb-8c43-4ec4-84a5-c17b7d061e19 tempest-VolumesNegativeTest-814389221 tempest-VolumesNegativeTest-814389221-project-member] Acquiring lock "refresh_cache-a4897e0e-d29c-45b6-8f2f-8f5ff55d6a8f" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:26:18 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-0a0433bb-8c43-4ec4-84a5-c17b7d061e19 tempest-VolumesNegativeTest-814389221 tempest-VolumesNegativeTest-814389221-project-member] Acquired lock "refresh_cache-a4897e0e-d29c-45b6-8f2f-8f5ff55d6a8f" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:26:18 user nova-compute[75234]: DEBUG nova.network.neutron [None req-0a0433bb-8c43-4ec4-84a5-c17b7d061e19 tempest-VolumesNegativeTest-814389221 tempest-VolumesNegativeTest-814389221-project-member] [instance: a4897e0e-d29c-45b6-8f2f-8f5ff55d6a8f] Building network info cache for instance {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2009}}
Nov 29 03:26:18 user nova-compute[75234]: DEBUG nova.compute.manager [req-9211d3f4-f6c2-4c1b-ad3f-adb03debc9c2 req-6b74d337-f3fe-4ce0-b215-7c88294e99fd service nova] [instance: a4897e0e-d29c-45b6-8f2f-8f5ff55d6a8f] Received event network-changed-55748abc-3e82-40c3-a7ba-b16327b28e9e {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:26:18 user nova-compute[75234]: DEBUG nova.compute.manager [req-9211d3f4-f6c2-4c1b-ad3f-adb03debc9c2 req-6b74d337-f3fe-4ce0-b215-7c88294e99fd service nova] [instance: a4897e0e-d29c-45b6-8f2f-8f5ff55d6a8f] Refreshing instance network info cache due to event network-changed-55748abc-3e82-40c3-a7ba-b16327b28e9e. {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11117}}
Nov 29 03:26:18 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-9211d3f4-f6c2-4c1b-ad3f-adb03debc9c2 req-6b74d337-f3fe-4ce0-b215-7c88294e99fd service nova] Acquiring lock "refresh_cache-a4897e0e-d29c-45b6-8f2f-8f5ff55d6a8f" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:26:18 user nova-compute[75234]: DEBUG nova.network.neutron [None req-0a0433bb-8c43-4ec4-84a5-c17b7d061e19 tempest-VolumesNegativeTest-814389221 tempest-VolumesNegativeTest-814389221-project-member] [instance: a4897e0e-d29c-45b6-8f2f-8f5ff55d6a8f] Instance cache missing network info. {{(pid=75234) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3322}}
Nov 29 03:26:19 user nova-compute[75234]: DEBUG nova.network.neutron [None req-0a0433bb-8c43-4ec4-84a5-c17b7d061e19 tempest-VolumesNegativeTest-814389221 tempest-VolumesNegativeTest-814389221-project-member] [instance: a4897e0e-d29c-45b6-8f2f-8f5ff55d6a8f] Updating instance_info_cache with network_info: [{"id": "55748abc-3e82-40c3-a7ba-b16327b28e9e", "address": "fa:16:3e:e8:7d:56", "network": {"id": "3b52ddb6-adad-4b84-99e0-6fb38ab8f76b", "bridge": "br-int", "label": "shared", "subnets": [{"cidr": "192.168.233.0/24", "dns": [], "gateway": {"address": "192.168.233.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "192.168.233.161", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "192.168.233.1"}}], "meta": {"injected": false, "tenant_id": "b18e380921c748b78517b6f1211cd0f1", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap55748abc-3e", "ovs_interfaceid": "55748abc-3e82-40c3-a7ba-b16327b28e9e", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:26:19 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-0a0433bb-8c43-4ec4-84a5-c17b7d061e19 tempest-VolumesNegativeTest-814389221 tempest-VolumesNegativeTest-814389221-project-member] Releasing lock "refresh_cache-a4897e0e-d29c-45b6-8f2f-8f5ff55d6a8f" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:26:19 user nova-compute[75234]: DEBUG nova.compute.manager [None req-0a0433bb-8c43-4ec4-84a5-c17b7d061e19 tempest-VolumesNegativeTest-814389221 tempest-VolumesNegativeTest-814389221-project-member] [instance: a4897e0e-d29c-45b6-8f2f-8f5ff55d6a8f] Instance network_info: |[{"id": "55748abc-3e82-40c3-a7ba-b16327b28e9e", "address": "fa:16:3e:e8:7d:56", "network": {"id": "3b52ddb6-adad-4b84-99e0-6fb38ab8f76b", "bridge": "br-int", "label": "shared", "subnets": [{"cidr": "192.168.233.0/24", "dns": [], "gateway": {"address": "192.168.233.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "192.168.233.161", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "192.168.233.1"}}], "meta": {"injected": false, "tenant_id": "b18e380921c748b78517b6f1211cd0f1", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap55748abc-3e", "ovs_interfaceid": "55748abc-3e82-40c3-a7ba-b16327b28e9e", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=75234) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1987}}
Nov 29 03:26:19 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-9211d3f4-f6c2-4c1b-ad3f-adb03debc9c2 req-6b74d337-f3fe-4ce0-b215-7c88294e99fd service nova] Acquired lock "refresh_cache-a4897e0e-d29c-45b6-8f2f-8f5ff55d6a8f" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:26:19 user nova-compute[75234]: DEBUG nova.network.neutron [req-9211d3f4-f6c2-4c1b-ad3f-adb03debc9c2 req-6b74d337-f3fe-4ce0-b215-7c88294e99fd service nova] [instance: a4897e0e-d29c-45b6-8f2f-8f5ff55d6a8f] Refreshing network info cache for port 55748abc-3e82-40c3-a7ba-b16327b28e9e {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2006}}
Nov 29 03:26:19 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-0a0433bb-8c43-4ec4-84a5-c17b7d061e19 tempest-VolumesNegativeTest-814389221 tempest-VolumesNegativeTest-814389221-project-member] [instance: a4897e0e-d29c-45b6-8f2f-8f5ff55d6a8f] Start _get_guest_xml network_info=[{"id": "55748abc-3e82-40c3-a7ba-b16327b28e9e", "address": "fa:16:3e:e8:7d:56", "network": {"id": "3b52ddb6-adad-4b84-99e0-6fb38ab8f76b", "bridge": "br-int", "label": "shared", "subnets": [{"cidr": "192.168.233.0/24", "dns": [], "gateway": {"address": "192.168.233.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "192.168.233.161", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "192.168.233.1"}}], "meta": {"injected": false, "tenant_id": "b18e380921c748b78517b6f1211cd0f1", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap55748abc-3e", "ovs_interfaceid": "55748abc-3e82-40c3-a7ba-b16327b28e9e", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum='c8fc807773e5354afe61636071771906',container_format='bare',created_at=2023-11-29T03:02:15Z,direct_url=<?>,disk_format='qcow2',id=96c30ba7-9b73-4a30-8ba4-7138879573b7,min_disk=0,min_ram=0,name='cirros-0.6.2-x86_64-disk',owner='b18e380921c748b78517b6f1211cd0f1',properties=ImageMetaProps,protected=<?>,size=21430272,status='active',tags=<?>,updated_at=2023-11-29T03:02:17Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [{'device_name': '/dev/vda', 'encryption_format': None, 'encrypted': False, 'device_type': 'disk', 'size': 0, 'disk_bus': 'virtio', 'boot_index': 0, 'encryption_options': None, 'encryption_secret_uuid': None, 'guest_format': None, 'image_id': '96c30ba7-9b73-4a30-8ba4-7138879573b7'}], 'ephemerals': [], 'block_device_mapping': [], 'swap': None} {{(pid=75234) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7562}}
Nov 29 03:26:19 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-0a0433bb-8c43-4ec4-84a5-c17b7d061e19 tempest-VolumesNegativeTest-814389221 tempest-VolumesNegativeTest-814389221-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:26:19 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-0a0433bb-8c43-4ec4-84a5-c17b7d061e19 tempest-VolumesNegativeTest-814389221 tempest-VolumesNegativeTest-814389221-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:26:19 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-0a0433bb-8c43-4ec4-84a5-c17b7d061e19 tempest-VolumesNegativeTest-814389221 tempest-VolumesNegativeTest-814389221-project-member] Searching host: 'user' for CPU controller through CGroups V1... {{(pid=75234) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1652}}
Nov 29 03:26:19 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-0a0433bb-8c43-4ec4-84a5-c17b7d061e19 tempest-VolumesNegativeTest-814389221 tempest-VolumesNegativeTest-814389221-project-member] CPU controller missing on host. {{(pid=75234) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1662}}
Nov 29 03:26:19 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-0a0433bb-8c43-4ec4-84a5-c17b7d061e19 tempest-VolumesNegativeTest-814389221 tempest-VolumesNegativeTest-814389221-project-member] Searching host: 'user' for CPU controller through CGroups V2... {{(pid=75234) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1671}}
Nov 29 03:26:19 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-0a0433bb-8c43-4ec4-84a5-c17b7d061e19 tempest-VolumesNegativeTest-814389221 tempest-VolumesNegativeTest-814389221-project-member] CPU controller found on host. {{(pid=75234) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1678}}
Nov 29 03:26:19 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-0a0433bb-8c43-4ec4-84a5-c17b7d061e19 tempest-VolumesNegativeTest-814389221 tempest-VolumesNegativeTest-814389221-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=75234) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5400}}
Nov 29 03:26:19 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-0a0433bb-8c43-4ec4-84a5-c17b7d061e19 tempest-VolumesNegativeTest-814389221 tempest-VolumesNegativeTest-814389221-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-11-29T03:02:04Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='1',id=6,is_public=True,memory_mb=512,name='m1.tiny',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='c8fc807773e5354afe61636071771906',container_format='bare',created_at=2023-11-29T03:02:15Z,direct_url=<?>,disk_format='qcow2',id=96c30ba7-9b73-4a30-8ba4-7138879573b7,min_disk=0,min_ram=0,name='cirros-0.6.2-x86_64-disk',owner='b18e380921c748b78517b6f1211cd0f1',properties=ImageMetaProps,protected=<?>,size=21430272,status='active',tags=<?>,updated_at=2023-11-29T03:02:17Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:563}}
Nov 29 03:26:19 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-0a0433bb-8c43-4ec4-84a5-c17b7d061e19 tempest-VolumesNegativeTest-814389221 tempest-VolumesNegativeTest-814389221-project-member] Flavor limits 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:348}}
Nov 29 03:26:19 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-0a0433bb-8c43-4ec4-84a5-c17b7d061e19 tempest-VolumesNegativeTest-814389221 tempest-VolumesNegativeTest-814389221-project-member] Image limits 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:352}}
Nov 29 03:26:19 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-0a0433bb-8c43-4ec4-84a5-c17b7d061e19 tempest-VolumesNegativeTest-814389221 tempest-VolumesNegativeTest-814389221-project-member] Flavor pref 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:388}}
Nov 29 03:26:19 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-0a0433bb-8c43-4ec4-84a5-c17b7d061e19 tempest-VolumesNegativeTest-814389221 tempest-VolumesNegativeTest-814389221-project-member] Image pref 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:392}}
Nov 29 03:26:19 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-0a0433bb-8c43-4ec4-84a5-c17b7d061e19 tempest-VolumesNegativeTest-814389221 tempest-VolumesNegativeTest-814389221-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:430}}
Nov 29 03:26:19 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-0a0433bb-8c43-4ec4-84a5-c17b7d061e19 tempest-VolumesNegativeTest-814389221 tempest-VolumesNegativeTest-814389221-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:569}}
Nov 29 03:26:19 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-0a0433bb-8c43-4ec4-84a5-c17b7d061e19 tempest-VolumesNegativeTest-814389221 tempest-VolumesNegativeTest-814389221-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=75234) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:471}}
Nov 29 03:26:19 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-0a0433bb-8c43-4ec4-84a5-c17b7d061e19 tempest-VolumesNegativeTest-814389221 tempest-VolumesNegativeTest-814389221-project-member] Got 1 possible topologies {{(pid=75234) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:501}}
Nov 29 03:26:19 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-0a0433bb-8c43-4ec4-84a5-c17b7d061e19 tempest-VolumesNegativeTest-814389221 tempest-VolumesNegativeTest-814389221-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:575}}
Nov 29 03:26:19 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-0a0433bb-8c43-4ec4-84a5-c17b7d061e19 tempest-VolumesNegativeTest-814389221 tempest-VolumesNegativeTest-814389221-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:577}}
Nov 29 03:26:19 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-0a0433bb-8c43-4ec4-84a5-c17b7d061e19 tempest-VolumesNegativeTest-814389221 tempest-VolumesNegativeTest-814389221-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-11-29T03:26:15Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-VolumesNegativeTest-instance-1349352086',display_name='tempest-VolumesNegativeTest-instance-1349352086',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-volumesnegativetest-instance-1349352086',id=41,image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data=None,key_name=None,keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='5da24ca4ca0d43d2849bf605ee87fa64',ramdisk_id='',reservation_id='r-7kuexd4k',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-VolumesNegativeTest-814389221',owner_user_name='tempest-VolumesNegativeTest-814389221-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-11-29T03:26:17Z,user_data=None,user_id='3a791e338de34f48b8512360b2422c11',uuid=a4897e0e-d29c-45b6-8f2f-8f5ff55d6a8f,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "55748abc-3e82-40c3-a7ba-b16327b28e9e", "address": "fa:16:3e:e8:7d:56", "network": {"id": "3b52ddb6-adad-4b84-99e0-6fb38ab8f76b", "bridge": "br-int", "label": "shared", "subnets": [{"cidr": "192.168.233.0/24", "dns": [], "gateway": {"address": "192.168.233.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "192.168.233.161", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "192.168.233.1"}}], "meta": {"injected": false, "tenant_id": "b18e380921c748b78517b6f1211cd0f1", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap55748abc-3e", "ovs_interfaceid": "55748abc-3e82-40c3-a7ba-b16327b28e9e", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=75234) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Nov 29 03:26:19 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-0a0433bb-8c43-4ec4-84a5-c17b7d061e19 tempest-VolumesNegativeTest-814389221 tempest-VolumesNegativeTest-814389221-project-member] Converting VIF {"id": "55748abc-3e82-40c3-a7ba-b16327b28e9e", "address": "fa:16:3e:e8:7d:56", "network": {"id": "3b52ddb6-adad-4b84-99e0-6fb38ab8f76b", "bridge": "br-int", "label": "shared", "subnets": [{"cidr": "192.168.233.0/24", "dns": [], "gateway": {"address": "192.168.233.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "192.168.233.161", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "192.168.233.1"}}], "meta": {"injected": false, "tenant_id": "b18e380921c748b78517b6f1211cd0f1", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap55748abc-3e", "ovs_interfaceid": "55748abc-3e82-40c3-a7ba-b16327b28e9e", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:26:19 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-0a0433bb-8c43-4ec4-84a5-c17b7d061e19 tempest-VolumesNegativeTest-814389221 tempest-VolumesNegativeTest-814389221-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:e8:7d:56,bridge_name='br-int',has_traffic_filtering=True,id=55748abc-3e82-40c3-a7ba-b16327b28e9e,network=Network(3b52ddb6-adad-4b84-99e0-6fb38ab8f76b),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap55748abc-3e') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:26:19 user nova-compute[75234]: DEBUG nova.objects.instance [None req-0a0433bb-8c43-4ec4-84a5-c17b7d061e19 tempest-VolumesNegativeTest-814389221 tempest-VolumesNegativeTest-814389221-project-member] Lazy-loading 'pci_devices' on Instance uuid a4897e0e-d29c-45b6-8f2f-8f5ff55d6a8f {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:26:19 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-0a0433bb-8c43-4ec4-84a5-c17b7d061e19 tempest-VolumesNegativeTest-814389221 tempest-VolumesNegativeTest-814389221-project-member] [instance: a4897e0e-d29c-45b6-8f2f-8f5ff55d6a8f] End _get_guest_xml xml=<domain type="kvm">
Nov 29 03:26:19 user nova-compute[75234]:   <uuid>a4897e0e-d29c-45b6-8f2f-8f5ff55d6a8f</uuid>
Nov 29 03:26:19 user nova-compute[75234]:   <name>instance-00000029</name>
Nov 29 03:26:19 user nova-compute[75234]:   <memory>524288</memory>
Nov 29 03:26:19 user nova-compute[75234]:   <vcpu>1</vcpu>
Nov 29 03:26:19 user nova-compute[75234]:   <metadata>
Nov 29 03:26:19 user nova-compute[75234]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Nov 29 03:26:19 user nova-compute[75234]:       <nova:package version="0.0.0"/>
Nov 29 03:26:19 user nova-compute[75234]:       <nova:name>tempest-VolumesNegativeTest-instance-1349352086</nova:name>
Nov 29 03:26:19 user nova-compute[75234]:       <nova:creationTime>2023-11-29 03:26:19</nova:creationTime>
Nov 29 03:26:19 user nova-compute[75234]:       <nova:flavor name="m1.tiny">
Nov 29 03:26:19 user nova-compute[75234]:         <nova:memory>512</nova:memory>
Nov 29 03:26:19 user nova-compute[75234]:         <nova:disk>1</nova:disk>
Nov 29 03:26:19 user nova-compute[75234]:         <nova:swap>0</nova:swap>
Nov 29 03:26:19 user nova-compute[75234]:         <nova:ephemeral>0</nova:ephemeral>
Nov 29 03:26:19 user nova-compute[75234]:         <nova:vcpus>1</nova:vcpus>
Nov 29 03:26:19 user nova-compute[75234]:       </nova:flavor>
Nov 29 03:26:19 user nova-compute[75234]:       <nova:owner>
Nov 29 03:26:19 user nova-compute[75234]:         <nova:user uuid="3a791e338de34f48b8512360b2422c11">tempest-VolumesNegativeTest-814389221-project-member</nova:user>
Nov 29 03:26:19 user nova-compute[75234]:         <nova:project uuid="5da24ca4ca0d43d2849bf605ee87fa64">tempest-VolumesNegativeTest-814389221</nova:project>
Nov 29 03:26:19 user nova-compute[75234]:       </nova:owner>
Nov 29 03:26:19 user nova-compute[75234]:       <nova:root type="image" uuid="96c30ba7-9b73-4a30-8ba4-7138879573b7"/>
Nov 29 03:26:19 user nova-compute[75234]:       <nova:ports>
Nov 29 03:26:19 user nova-compute[75234]:         <nova:port uuid="55748abc-3e82-40c3-a7ba-b16327b28e9e">
Nov 29 03:26:19 user nova-compute[75234]:           <nova:ip type="fixed" address="192.168.233.161" ipVersion="4"/>
Nov 29 03:26:19 user nova-compute[75234]:         </nova:port>
Nov 29 03:26:19 user nova-compute[75234]:       </nova:ports>
Nov 29 03:26:19 user nova-compute[75234]:     </nova:instance>
Nov 29 03:26:19 user nova-compute[75234]:   </metadata>
Nov 29 03:26:19 user nova-compute[75234]:   <sysinfo type="smbios">
Nov 29 03:26:19 user nova-compute[75234]:     <system>
Nov 29 03:26:19 user nova-compute[75234]:       <entry name="manufacturer">OpenStack Foundation</entry>
Nov 29 03:26:19 user nova-compute[75234]:       <entry name="product">OpenStack Nova</entry>
Nov 29 03:26:19 user nova-compute[75234]:       <entry name="version">0.0.0</entry>
Nov 29 03:26:19 user nova-compute[75234]:       <entry name="serial">a4897e0e-d29c-45b6-8f2f-8f5ff55d6a8f</entry>
Nov 29 03:26:19 user nova-compute[75234]:       <entry name="uuid">a4897e0e-d29c-45b6-8f2f-8f5ff55d6a8f</entry>
Nov 29 03:26:19 user nova-compute[75234]:       <entry name="family">Virtual Machine</entry>
Nov 29 03:26:19 user nova-compute[75234]:     </system>
Nov 29 03:26:19 user nova-compute[75234]:   </sysinfo>
Nov 29 03:26:19 user nova-compute[75234]:   <os>
Nov 29 03:26:19 user nova-compute[75234]:     <type arch="x86_64" machine="pc">hvm</type>
Nov 29 03:26:19 user nova-compute[75234]:     <boot dev="hd"/>
Nov 29 03:26:19 user nova-compute[75234]:     <smbios mode="sysinfo"/>
Nov 29 03:26:19 user nova-compute[75234]:   </os>
Nov 29 03:26:19 user nova-compute[75234]:   <features>
Nov 29 03:26:19 user nova-compute[75234]:     <acpi/>
Nov 29 03:26:19 user nova-compute[75234]:     <apic/>
Nov 29 03:26:19 user nova-compute[75234]:     <vmcoreinfo/>
Nov 29 03:26:19 user nova-compute[75234]:   </features>
Nov 29 03:26:19 user nova-compute[75234]:   <clock offset="utc">
Nov 29 03:26:19 user nova-compute[75234]:     <timer name="pit" tickpolicy="delay"/>
Nov 29 03:26:19 user nova-compute[75234]:     <timer name="rtc" tickpolicy="catchup"/>
Nov 29 03:26:19 user nova-compute[75234]:     <timer name="hpet" present="no"/>
Nov 29 03:26:19 user nova-compute[75234]:   </clock>
Nov 29 03:26:19 user nova-compute[75234]:   <cpu mode="custom" match="exact">
Nov 29 03:26:19 user nova-compute[75234]:     <model>Nehalem</model>
Nov 29 03:26:19 user nova-compute[75234]:     <topology sockets="1" cores="1" threads="1"/>
Nov 29 03:26:19 user nova-compute[75234]:   </cpu>
Nov 29 03:26:19 user nova-compute[75234]:   <devices>
Nov 29 03:26:19 user nova-compute[75234]:     <disk type="file" device="disk">
Nov 29 03:26:19 user nova-compute[75234]:       <driver name="qemu" type="qcow2" cache="none"/>
Nov 29 03:26:19 user nova-compute[75234]:       <source file="/opt/stack/data/nova/instances/a4897e0e-d29c-45b6-8f2f-8f5ff55d6a8f/disk"/>
Nov 29 03:26:19 user nova-compute[75234]:       <target dev="vda" bus="virtio"/>
Nov 29 03:26:19 user nova-compute[75234]:     </disk>
Nov 29 03:26:19 user nova-compute[75234]:     <interface type="ethernet">
Nov 29 03:26:19 user nova-compute[75234]:       <mac address="fa:16:3e:e8:7d:56"/>
Nov 29 03:26:19 user nova-compute[75234]:       <model type="virtio"/>
Nov 29 03:26:19 user nova-compute[75234]:       <mtu size="1442"/>
Nov 29 03:26:19 user nova-compute[75234]:       <target dev="tap55748abc-3e"/>
Nov 29 03:26:19 user nova-compute[75234]:     </interface>
Nov 29 03:26:19 user nova-compute[75234]:     <serial type="pty">
Nov 29 03:26:19 user nova-compute[75234]:       <log file="/opt/stack/data/nova/instances/a4897e0e-d29c-45b6-8f2f-8f5ff55d6a8f/console.log" append="off"/>
Nov 29 03:26:19 user nova-compute[75234]:     </serial>
Nov 29 03:26:19 user nova-compute[75234]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Nov 29 03:26:19 user nova-compute[75234]:     <video>
Nov 29 03:26:19 user nova-compute[75234]:       <model type="virtio"/>
Nov 29 03:26:19 user nova-compute[75234]:     </video>
Nov 29 03:26:19 user nova-compute[75234]:     <rng model="virtio">
Nov 29 03:26:19 user nova-compute[75234]:       <backend model="random">/dev/urandom</backend>
Nov 29 03:26:19 user nova-compute[75234]:     </rng>
Nov 29 03:26:19 user nova-compute[75234]:     <controller type="usb" index="0" model="none"/>
Nov 29 03:26:19 user nova-compute[75234]:     <memballoon model="virtio">
Nov 29 03:26:19 user nova-compute[75234]:       <stats period="10"/>
Nov 29 03:26:19 user nova-compute[75234]:     </memballoon>
Nov 29 03:26:19 user nova-compute[75234]:   </devices>
Nov 29 03:26:19 user nova-compute[75234]: </domain>
Nov 29 03:26:19 user nova-compute[75234]:  {{(pid=75234) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7568}}
Nov 29 03:26:19 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-0a0433bb-8c43-4ec4-84a5-c17b7d061e19 tempest-VolumesNegativeTest-814389221 tempest-VolumesNegativeTest-814389221-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-11-29T03:26:15Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-VolumesNegativeTest-instance-1349352086',display_name='tempest-VolumesNegativeTest-instance-1349352086',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-volumesnegativetest-instance-1349352086',id=41,image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data=None,key_name=None,keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='5da24ca4ca0d43d2849bf605ee87fa64',ramdisk_id='',reservation_id='r-7kuexd4k',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-VolumesNegativeTest-814389221',owner_user_name='tempest-VolumesNegativeTest-814389221-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-11-29T03:26:17Z,user_data=None,user_id='3a791e338de34f48b8512360b2422c11',uuid=a4897e0e-d29c-45b6-8f2f-8f5ff55d6a8f,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "55748abc-3e82-40c3-a7ba-b16327b28e9e", "address": "fa:16:3e:e8:7d:56", "network": {"id": "3b52ddb6-adad-4b84-99e0-6fb38ab8f76b", "bridge": "br-int", "label": "shared", "subnets": [{"cidr": "192.168.233.0/24", "dns": [], "gateway": {"address": "192.168.233.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "192.168.233.161", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "192.168.233.1"}}], "meta": {"injected": false, "tenant_id": "b18e380921c748b78517b6f1211cd0f1", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap55748abc-3e", "ovs_interfaceid": "55748abc-3e82-40c3-a7ba-b16327b28e9e", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Nov 29 03:26:19 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-0a0433bb-8c43-4ec4-84a5-c17b7d061e19 tempest-VolumesNegativeTest-814389221 tempest-VolumesNegativeTest-814389221-project-member] Converting VIF {"id": "55748abc-3e82-40c3-a7ba-b16327b28e9e", "address": "fa:16:3e:e8:7d:56", "network": {"id": "3b52ddb6-adad-4b84-99e0-6fb38ab8f76b", "bridge": "br-int", "label": "shared", "subnets": [{"cidr": "192.168.233.0/24", "dns": [], "gateway": {"address": "192.168.233.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "192.168.233.161", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "192.168.233.1"}}], "meta": {"injected": false, "tenant_id": "b18e380921c748b78517b6f1211cd0f1", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap55748abc-3e", "ovs_interfaceid": "55748abc-3e82-40c3-a7ba-b16327b28e9e", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:26:19 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-0a0433bb-8c43-4ec4-84a5-c17b7d061e19 tempest-VolumesNegativeTest-814389221 tempest-VolumesNegativeTest-814389221-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:e8:7d:56,bridge_name='br-int',has_traffic_filtering=True,id=55748abc-3e82-40c3-a7ba-b16327b28e9e,network=Network(3b52ddb6-adad-4b84-99e0-6fb38ab8f76b),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap55748abc-3e') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:26:19 user nova-compute[75234]: DEBUG os_vif [None req-0a0433bb-8c43-4ec4-84a5-c17b7d061e19 tempest-VolumesNegativeTest-814389221 tempest-VolumesNegativeTest-814389221-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:e8:7d:56,bridge_name='br-int',has_traffic_filtering=True,id=55748abc-3e82-40c3-a7ba-b16327b28e9e,network=Network(3b52ddb6-adad-4b84-99e0-6fb38ab8f76b),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap55748abc-3e') {{(pid=75234) plug /opt/stack/data/venv/lib/python3.10/site-packages/os_vif/__init__.py:76}}
Nov 29 03:26:19 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:26:19 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:26:19 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Nov 29 03:26:19 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:26:19 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbCreateCommand(_result=None, table=QoS, columns={'type': 'linux-noop', 'external_ids': {'id': '24e0167e-0823-5a4b-b8b6-79bd1014c1c3', '_type': 'linux-noop'}}, row=False) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:26:19 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:26:19 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:26:19 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:26:19 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tap55748abc-3e, may_exist=True, interface_attrs={}) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:26:19 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Port, record=tap55748abc-3e, col_values=(('qos', UUID('6afe4725-9987-4656-b61b-2d7fecde2ac9')),), if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:26:19 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DbSetCommand(_result=None, table=Interface, record=tap55748abc-3e, col_values=(('external_ids', {'iface-id': '55748abc-3e82-40c3-a7ba-b16327b28e9e', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:e8:7d:56', 'vm-uuid': 'a4897e0e-d29c-45b6-8f2f-8f5ff55d6a8f'}),), if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:26:19 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:26:19 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:26:19 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:26:19 user nova-compute[75234]: INFO os_vif [None req-0a0433bb-8c43-4ec4-84a5-c17b7d061e19 tempest-VolumesNegativeTest-814389221 tempest-VolumesNegativeTest-814389221-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:e8:7d:56,bridge_name='br-int',has_traffic_filtering=True,id=55748abc-3e82-40c3-a7ba-b16327b28e9e,network=Network(3b52ddb6-adad-4b84-99e0-6fb38ab8f76b),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap55748abc-3e')
Nov 29 03:26:19 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-0a0433bb-8c43-4ec4-84a5-c17b7d061e19 tempest-VolumesNegativeTest-814389221 tempest-VolumesNegativeTest-814389221-project-member] No BDM found with device name vda, not building metadata. {{(pid=75234) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12089}}
Nov 29 03:26:19 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-0a0433bb-8c43-4ec4-84a5-c17b7d061e19 tempest-VolumesNegativeTest-814389221 tempest-VolumesNegativeTest-814389221-project-member] No VIF found with MAC fa:16:3e:e8:7d:56, not building metadata {{(pid=75234) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Nov 29 03:26:19 user nova-compute[75234]: DEBUG nova.network.neutron [req-9211d3f4-f6c2-4c1b-ad3f-adb03debc9c2 req-6b74d337-f3fe-4ce0-b215-7c88294e99fd service nova] [instance: a4897e0e-d29c-45b6-8f2f-8f5ff55d6a8f] Updated VIF entry in instance network info cache for port 55748abc-3e82-40c3-a7ba-b16327b28e9e. {{(pid=75234) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3481}}
Nov 29 03:26:19 user nova-compute[75234]: DEBUG nova.network.neutron [req-9211d3f4-f6c2-4c1b-ad3f-adb03debc9c2 req-6b74d337-f3fe-4ce0-b215-7c88294e99fd service nova] [instance: a4897e0e-d29c-45b6-8f2f-8f5ff55d6a8f] Updating instance_info_cache with network_info: [{"id": "55748abc-3e82-40c3-a7ba-b16327b28e9e", "address": "fa:16:3e:e8:7d:56", "network": {"id": "3b52ddb6-adad-4b84-99e0-6fb38ab8f76b", "bridge": "br-int", "label": "shared", "subnets": [{"cidr": "192.168.233.0/24", "dns": [], "gateway": {"address": "192.168.233.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "192.168.233.161", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "192.168.233.1"}}], "meta": {"injected": false, "tenant_id": "b18e380921c748b78517b6f1211cd0f1", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap55748abc-3e", "ovs_interfaceid": "55748abc-3e82-40c3-a7ba-b16327b28e9e", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:26:19 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-9211d3f4-f6c2-4c1b-ad3f-adb03debc9c2 req-6b74d337-f3fe-4ce0-b215-7c88294e99fd service nova] Releasing lock "refresh_cache-a4897e0e-d29c-45b6-8f2f-8f5ff55d6a8f" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:26:20 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:26:20 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:26:20 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:26:21 user nova-compute[75234]: DEBUG nova.compute.manager [req-54048cd4-bf95-4c15-9b2c-3eb52f81ac4e req-ba976967-763e-4e7b-89c5-e333ac1639db service nova] [instance: a4897e0e-d29c-45b6-8f2f-8f5ff55d6a8f] Received event network-vif-plugged-55748abc-3e82-40c3-a7ba-b16327b28e9e {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:26:21 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-54048cd4-bf95-4c15-9b2c-3eb52f81ac4e req-ba976967-763e-4e7b-89c5-e333ac1639db service nova] Acquiring lock "a4897e0e-d29c-45b6-8f2f-8f5ff55d6a8f-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:26:21 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-54048cd4-bf95-4c15-9b2c-3eb52f81ac4e req-ba976967-763e-4e7b-89c5-e333ac1639db service nova] Lock "a4897e0e-d29c-45b6-8f2f-8f5ff55d6a8f-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:26:21 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-54048cd4-bf95-4c15-9b2c-3eb52f81ac4e req-ba976967-763e-4e7b-89c5-e333ac1639db service nova] Lock "a4897e0e-d29c-45b6-8f2f-8f5ff55d6a8f-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:26:21 user nova-compute[75234]: DEBUG nova.compute.manager [req-54048cd4-bf95-4c15-9b2c-3eb52f81ac4e req-ba976967-763e-4e7b-89c5-e333ac1639db service nova] [instance: a4897e0e-d29c-45b6-8f2f-8f5ff55d6a8f] No waiting events found dispatching network-vif-plugged-55748abc-3e82-40c3-a7ba-b16327b28e9e {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:26:21 user nova-compute[75234]: WARNING nova.compute.manager [req-54048cd4-bf95-4c15-9b2c-3eb52f81ac4e req-ba976967-763e-4e7b-89c5-e333ac1639db service nova] [instance: a4897e0e-d29c-45b6-8f2f-8f5ff55d6a8f] Received unexpected event network-vif-plugged-55748abc-3e82-40c3-a7ba-b16327b28e9e for instance with vm_state building and task_state spawning.
Nov 29 03:26:21 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:26:21 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:26:22 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:26:23 user nova-compute[75234]: DEBUG nova.compute.manager [None req-0a0433bb-8c43-4ec4-84a5-c17b7d061e19 tempest-VolumesNegativeTest-814389221 tempest-VolumesNegativeTest-814389221-project-member] [instance: a4897e0e-d29c-45b6-8f2f-8f5ff55d6a8f] Instance event wait completed in 0 seconds for  {{(pid=75234) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Nov 29 03:26:23 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-0a0433bb-8c43-4ec4-84a5-c17b7d061e19 tempest-VolumesNegativeTest-814389221 tempest-VolumesNegativeTest-814389221-project-member] [instance: a4897e0e-d29c-45b6-8f2f-8f5ff55d6a8f] Guest created on hypervisor {{(pid=75234) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4421}}
Nov 29 03:26:23 user nova-compute[75234]: INFO nova.virt.libvirt.driver [-] [instance: a4897e0e-d29c-45b6-8f2f-8f5ff55d6a8f] Instance spawned successfully.
Nov 29 03:26:23 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-0a0433bb-8c43-4ec4-84a5-c17b7d061e19 tempest-VolumesNegativeTest-814389221 tempest-VolumesNegativeTest-814389221-project-member] [instance: a4897e0e-d29c-45b6-8f2f-8f5ff55d6a8f] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:904}}
Nov 29 03:26:23 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-0a0433bb-8c43-4ec4-84a5-c17b7d061e19 tempest-VolumesNegativeTest-814389221 tempest-VolumesNegativeTest-814389221-project-member] [instance: a4897e0e-d29c-45b6-8f2f-8f5ff55d6a8f] Found default for hw_cdrom_bus of ide {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:26:23 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-0a0433bb-8c43-4ec4-84a5-c17b7d061e19 tempest-VolumesNegativeTest-814389221 tempest-VolumesNegativeTest-814389221-project-member] [instance: a4897e0e-d29c-45b6-8f2f-8f5ff55d6a8f] Found default for hw_disk_bus of virtio {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:26:23 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-0a0433bb-8c43-4ec4-84a5-c17b7d061e19 tempest-VolumesNegativeTest-814389221 tempest-VolumesNegativeTest-814389221-project-member] [instance: a4897e0e-d29c-45b6-8f2f-8f5ff55d6a8f] Found default for hw_input_bus of None {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:26:23 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-0a0433bb-8c43-4ec4-84a5-c17b7d061e19 tempest-VolumesNegativeTest-814389221 tempest-VolumesNegativeTest-814389221-project-member] [instance: a4897e0e-d29c-45b6-8f2f-8f5ff55d6a8f] Found default for hw_pointer_model of None {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:26:23 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-0a0433bb-8c43-4ec4-84a5-c17b7d061e19 tempest-VolumesNegativeTest-814389221 tempest-VolumesNegativeTest-814389221-project-member] [instance: a4897e0e-d29c-45b6-8f2f-8f5ff55d6a8f] Found default for hw_video_model of virtio {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:26:23 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-0a0433bb-8c43-4ec4-84a5-c17b7d061e19 tempest-VolumesNegativeTest-814389221 tempest-VolumesNegativeTest-814389221-project-member] [instance: a4897e0e-d29c-45b6-8f2f-8f5ff55d6a8f] Found default for hw_vif_model of virtio {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:26:23 user nova-compute[75234]: INFO nova.compute.manager [None req-0a0433bb-8c43-4ec4-84a5-c17b7d061e19 tempest-VolumesNegativeTest-814389221 tempest-VolumesNegativeTest-814389221-project-member] [instance: a4897e0e-d29c-45b6-8f2f-8f5ff55d6a8f] Took 6.32 seconds to spawn the instance on the hypervisor.
Nov 29 03:26:23 user nova-compute[75234]: DEBUG nova.compute.manager [None req-0a0433bb-8c43-4ec4-84a5-c17b7d061e19 tempest-VolumesNegativeTest-814389221 tempest-VolumesNegativeTest-814389221-project-member] [instance: a4897e0e-d29c-45b6-8f2f-8f5ff55d6a8f] Checking state {{(pid=75234) _get_power_state /opt/stack/nova/nova/compute/manager.py:1782}}
Nov 29 03:26:23 user nova-compute[75234]: DEBUG nova.compute.manager [req-4b169db5-db07-47c9-9732-7159e9634afe req-cd5cea51-6133-4eb2-8781-eddc37218177 service nova] [instance: a4897e0e-d29c-45b6-8f2f-8f5ff55d6a8f] Received event network-vif-plugged-55748abc-3e82-40c3-a7ba-b16327b28e9e {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:26:23 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-4b169db5-db07-47c9-9732-7159e9634afe req-cd5cea51-6133-4eb2-8781-eddc37218177 service nova] Acquiring lock "a4897e0e-d29c-45b6-8f2f-8f5ff55d6a8f-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:26:23 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-4b169db5-db07-47c9-9732-7159e9634afe req-cd5cea51-6133-4eb2-8781-eddc37218177 service nova] Lock "a4897e0e-d29c-45b6-8f2f-8f5ff55d6a8f-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:26:23 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-4b169db5-db07-47c9-9732-7159e9634afe req-cd5cea51-6133-4eb2-8781-eddc37218177 service nova] Lock "a4897e0e-d29c-45b6-8f2f-8f5ff55d6a8f-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:26:23 user nova-compute[75234]: DEBUG nova.compute.manager [req-4b169db5-db07-47c9-9732-7159e9634afe req-cd5cea51-6133-4eb2-8781-eddc37218177 service nova] [instance: a4897e0e-d29c-45b6-8f2f-8f5ff55d6a8f] No waiting events found dispatching network-vif-plugged-55748abc-3e82-40c3-a7ba-b16327b28e9e {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:26:23 user nova-compute[75234]: WARNING nova.compute.manager [req-4b169db5-db07-47c9-9732-7159e9634afe req-cd5cea51-6133-4eb2-8781-eddc37218177 service nova] [instance: a4897e0e-d29c-45b6-8f2f-8f5ff55d6a8f] Received unexpected event network-vif-plugged-55748abc-3e82-40c3-a7ba-b16327b28e9e for instance with vm_state building and task_state spawning.
Nov 29 03:26:23 user nova-compute[75234]: INFO nova.compute.manager [None req-0a0433bb-8c43-4ec4-84a5-c17b7d061e19 tempest-VolumesNegativeTest-814389221 tempest-VolumesNegativeTest-814389221-project-member] [instance: a4897e0e-d29c-45b6-8f2f-8f5ff55d6a8f] Took 7.10 seconds to build instance.
Nov 29 03:26:23 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-0a0433bb-8c43-4ec4-84a5-c17b7d061e19 tempest-VolumesNegativeTest-814389221 tempest-VolumesNegativeTest-814389221-project-member] Lock "a4897e0e-d29c-45b6-8f2f-8f5ff55d6a8f" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 7.201s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:26:23 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-a73e47e1-e5c8-4b33-84ee-dd89b7d8033b tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Acquiring lock "499676de-0e76-45ae-864c-44e55e3d7c7f" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:26:23 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-a73e47e1-e5c8-4b33-84ee-dd89b7d8033b tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Lock "499676de-0e76-45ae-864c-44e55e3d7c7f" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:26:23 user nova-compute[75234]: DEBUG nova.compute.manager [None req-a73e47e1-e5c8-4b33-84ee-dd89b7d8033b tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] [instance: 499676de-0e76-45ae-864c-44e55e3d7c7f] Starting instance... {{(pid=75234) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2422}}
Nov 29 03:26:23 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-a73e47e1-e5c8-4b33-84ee-dd89b7d8033b tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:26:23 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-a73e47e1-e5c8-4b33-84ee-dd89b7d8033b tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:26:23 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-a73e47e1-e5c8-4b33-84ee-dd89b7d8033b tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=75234) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2366}}
Nov 29 03:26:23 user nova-compute[75234]: INFO nova.compute.claims [None req-a73e47e1-e5c8-4b33-84ee-dd89b7d8033b tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] [instance: 499676de-0e76-45ae-864c-44e55e3d7c7f] Claim successful on node user
Nov 29 03:26:24 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:26:24 user nova-compute[75234]: DEBUG nova.compute.provider_tree [None req-a73e47e1-e5c8-4b33-84ee-dd89b7d8033b tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Inventory has not changed in ProviderTree for provider: 4d291220-7c37-4839-a0e3-b23c66502b74 {{(pid=75234) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Nov 29 03:26:24 user nova-compute[75234]: DEBUG nova.scheduler.client.report [None req-a73e47e1-e5c8-4b33-84ee-dd89b7d8033b tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Inventory has not changed for provider 4d291220-7c37-4839-a0e3-b23c66502b74 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=75234) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Nov 29 03:26:24 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-a73e47e1-e5c8-4b33-84ee-dd89b7d8033b tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.464s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:26:24 user nova-compute[75234]: DEBUG nova.compute.manager [None req-a73e47e1-e5c8-4b33-84ee-dd89b7d8033b tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] [instance: 499676de-0e76-45ae-864c-44e55e3d7c7f] Start building networks asynchronously for instance. {{(pid=75234) _build_resources /opt/stack/nova/nova/compute/manager.py:2819}}
Nov 29 03:26:24 user nova-compute[75234]: DEBUG nova.compute.manager [None req-a73e47e1-e5c8-4b33-84ee-dd89b7d8033b tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] [instance: 499676de-0e76-45ae-864c-44e55e3d7c7f] Allocating IP information in the background. {{(pid=75234) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1972}}
Nov 29 03:26:24 user nova-compute[75234]: DEBUG nova.network.neutron [None req-a73e47e1-e5c8-4b33-84ee-dd89b7d8033b tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] [instance: 499676de-0e76-45ae-864c-44e55e3d7c7f] allocate_for_instance() {{(pid=75234) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1156}}
Nov 29 03:26:24 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-a73e47e1-e5c8-4b33-84ee-dd89b7d8033b tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] [instance: 499676de-0e76-45ae-864c-44e55e3d7c7f] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Nov 29 03:26:24 user nova-compute[75234]: DEBUG nova.compute.manager [None req-a73e47e1-e5c8-4b33-84ee-dd89b7d8033b tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] [instance: 499676de-0e76-45ae-864c-44e55e3d7c7f] Start building block device mappings for instance. {{(pid=75234) _build_resources /opt/stack/nova/nova/compute/manager.py:2854}}
Nov 29 03:26:24 user nova-compute[75234]: DEBUG nova.policy [None req-a73e47e1-e5c8-4b33-84ee-dd89b7d8033b tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '66686114450f4f31b3a917667a152d8b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '315732b6e3ee44d584b8e510ef199a90', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=75234) authorize /opt/stack/nova/nova/policy.py:203}}
Nov 29 03:26:24 user nova-compute[75234]: DEBUG nova.compute.manager [None req-a73e47e1-e5c8-4b33-84ee-dd89b7d8033b tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] [instance: 499676de-0e76-45ae-864c-44e55e3d7c7f] Start spawning the instance on the hypervisor. {{(pid=75234) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2628}}
Nov 29 03:26:24 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-a73e47e1-e5c8-4b33-84ee-dd89b7d8033b tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] [instance: 499676de-0e76-45ae-864c-44e55e3d7c7f] Creating instance directory {{(pid=75234) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4727}}
Nov 29 03:26:24 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-a73e47e1-e5c8-4b33-84ee-dd89b7d8033b tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] [instance: 499676de-0e76-45ae-864c-44e55e3d7c7f] Creating image(s)
Nov 29 03:26:24 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-a73e47e1-e5c8-4b33-84ee-dd89b7d8033b tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Acquiring lock "/opt/stack/data/nova/instances/499676de-0e76-45ae-864c-44e55e3d7c7f/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:26:24 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-a73e47e1-e5c8-4b33-84ee-dd89b7d8033b tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Lock "/opt/stack/data/nova/instances/499676de-0e76-45ae-864c-44e55e3d7c7f/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:26:24 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-a73e47e1-e5c8-4b33-84ee-dd89b7d8033b tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Lock "/opt/stack/data/nova/instances/499676de-0e76-45ae-864c-44e55e3d7c7f/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.002s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:26:24 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-a73e47e1-e5c8-4b33-84ee-dd89b7d8033b tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:26:24 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:26:24 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-a73e47e1-e5c8-4b33-84ee-dd89b7d8033b tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json" returned: 0 in 0.209s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:26:24 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-a73e47e1-e5c8-4b33-84ee-dd89b7d8033b tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Acquiring lock "80686e29dc9db6600d03e85cb62af298e12d7918" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:26:24 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-a73e47e1-e5c8-4b33-84ee-dd89b7d8033b tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Lock "80686e29dc9db6600d03e85cb62af298e12d7918" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.002s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:26:24 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-a73e47e1-e5c8-4b33-84ee-dd89b7d8033b tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:26:25 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-a73e47e1-e5c8-4b33-84ee-dd89b7d8033b tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json" returned: 0 in 0.189s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:26:25 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-a73e47e1-e5c8-4b33-84ee-dd89b7d8033b tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918,backing_fmt=raw /opt/stack/data/nova/instances/499676de-0e76-45ae-864c-44e55e3d7c7f/disk 1073741824 {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:26:25 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-a73e47e1-e5c8-4b33-84ee-dd89b7d8033b tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918,backing_fmt=raw /opt/stack/data/nova/instances/499676de-0e76-45ae-864c-44e55e3d7c7f/disk 1073741824" returned: 0 in 0.067s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:26:25 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-a73e47e1-e5c8-4b33-84ee-dd89b7d8033b tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Lock "80686e29dc9db6600d03e85cb62af298e12d7918" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.267s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:26:25 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-a73e47e1-e5c8-4b33-84ee-dd89b7d8033b tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:26:25 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-a73e47e1-e5c8-4b33-84ee-dd89b7d8033b tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json" returned: 0 in 0.214s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:26:25 user nova-compute[75234]: DEBUG nova.virt.disk.api [None req-a73e47e1-e5c8-4b33-84ee-dd89b7d8033b tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Checking if we can resize image /opt/stack/data/nova/instances/499676de-0e76-45ae-864c-44e55e3d7c7f/disk. size=1073741824 {{(pid=75234) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:166}}
Nov 29 03:26:25 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-a73e47e1-e5c8-4b33-84ee-dd89b7d8033b tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/499676de-0e76-45ae-864c-44e55e3d7c7f/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:26:25 user nova-compute[75234]: DEBUG nova.network.neutron [None req-a73e47e1-e5c8-4b33-84ee-dd89b7d8033b tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] [instance: 499676de-0e76-45ae-864c-44e55e3d7c7f] Successfully created port: 720988c2-e733-41cb-b3ea-c215cfe46bd6 {{(pid=75234) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:548}}
Nov 29 03:26:25 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-a73e47e1-e5c8-4b33-84ee-dd89b7d8033b tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/499676de-0e76-45ae-864c-44e55e3d7c7f/disk --force-share --output=json" returned: 0 in 0.213s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:26:25 user nova-compute[75234]: DEBUG nova.virt.disk.api [None req-a73e47e1-e5c8-4b33-84ee-dd89b7d8033b tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Cannot resize image /opt/stack/data/nova/instances/499676de-0e76-45ae-864c-44e55e3d7c7f/disk to a smaller size. {{(pid=75234) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:172}}
Nov 29 03:26:25 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-a73e47e1-e5c8-4b33-84ee-dd89b7d8033b tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] [instance: 499676de-0e76-45ae-864c-44e55e3d7c7f] Created local disks {{(pid=75234) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4861}}
Nov 29 03:26:25 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-a73e47e1-e5c8-4b33-84ee-dd89b7d8033b tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] [instance: 499676de-0e76-45ae-864c-44e55e3d7c7f] Ensure instance console log exists: /opt/stack/data/nova/instances/499676de-0e76-45ae-864c-44e55e3d7c7f/console.log {{(pid=75234) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4613}}
Nov 29 03:26:25 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-a73e47e1-e5c8-4b33-84ee-dd89b7d8033b tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:26:25 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-a73e47e1-e5c8-4b33-84ee-dd89b7d8033b tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:26:25 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-a73e47e1-e5c8-4b33-84ee-dd89b7d8033b tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:26:26 user nova-compute[75234]: DEBUG nova.network.neutron [None req-a73e47e1-e5c8-4b33-84ee-dd89b7d8033b tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] [instance: 499676de-0e76-45ae-864c-44e55e3d7c7f] Successfully updated port: 720988c2-e733-41cb-b3ea-c215cfe46bd6 {{(pid=75234) _update_port /opt/stack/nova/nova/network/neutron.py:586}}
Nov 29 03:26:26 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-a73e47e1-e5c8-4b33-84ee-dd89b7d8033b tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Acquiring lock "refresh_cache-499676de-0e76-45ae-864c-44e55e3d7c7f" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:26:26 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-a73e47e1-e5c8-4b33-84ee-dd89b7d8033b tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Acquired lock "refresh_cache-499676de-0e76-45ae-864c-44e55e3d7c7f" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:26:26 user nova-compute[75234]: DEBUG nova.network.neutron [None req-a73e47e1-e5c8-4b33-84ee-dd89b7d8033b tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] [instance: 499676de-0e76-45ae-864c-44e55e3d7c7f] Building network info cache for instance {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2009}}
Nov 29 03:26:26 user nova-compute[75234]: DEBUG nova.network.neutron [None req-a73e47e1-e5c8-4b33-84ee-dd89b7d8033b tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] [instance: 499676de-0e76-45ae-864c-44e55e3d7c7f] Instance cache missing network info. {{(pid=75234) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3322}}
Nov 29 03:26:26 user nova-compute[75234]: DEBUG nova.compute.manager [req-1646d138-328c-4963-9e3c-55462273d73b req-4483bbcf-4a4a-41c5-976f-908a4c86a753 service nova] [instance: 499676de-0e76-45ae-864c-44e55e3d7c7f] Received event network-changed-720988c2-e733-41cb-b3ea-c215cfe46bd6 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:26:26 user nova-compute[75234]: DEBUG nova.compute.manager [req-1646d138-328c-4963-9e3c-55462273d73b req-4483bbcf-4a4a-41c5-976f-908a4c86a753 service nova] [instance: 499676de-0e76-45ae-864c-44e55e3d7c7f] Refreshing instance network info cache due to event network-changed-720988c2-e733-41cb-b3ea-c215cfe46bd6. {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11117}}
Nov 29 03:26:26 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-1646d138-328c-4963-9e3c-55462273d73b req-4483bbcf-4a4a-41c5-976f-908a4c86a753 service nova] Acquiring lock "refresh_cache-499676de-0e76-45ae-864c-44e55e3d7c7f" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:26:26 user nova-compute[75234]: DEBUG nova.network.neutron [None req-a73e47e1-e5c8-4b33-84ee-dd89b7d8033b tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] [instance: 499676de-0e76-45ae-864c-44e55e3d7c7f] Updating instance_info_cache with network_info: [{"id": "720988c2-e733-41cb-b3ea-c215cfe46bd6", "address": "fa:16:3e:0d:b8:b3", "network": {"id": "0f289171-3d73-4054-95a0-2515b4f206a0", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-1787960284-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "315732b6e3ee44d584b8e510ef199a90", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap720988c2-e7", "ovs_interfaceid": "720988c2-e733-41cb-b3ea-c215cfe46bd6", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:26:26 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-a73e47e1-e5c8-4b33-84ee-dd89b7d8033b tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Releasing lock "refresh_cache-499676de-0e76-45ae-864c-44e55e3d7c7f" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:26:26 user nova-compute[75234]: DEBUG nova.compute.manager [None req-a73e47e1-e5c8-4b33-84ee-dd89b7d8033b tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] [instance: 499676de-0e76-45ae-864c-44e55e3d7c7f] Instance network_info: |[{"id": "720988c2-e733-41cb-b3ea-c215cfe46bd6", "address": "fa:16:3e:0d:b8:b3", "network": {"id": "0f289171-3d73-4054-95a0-2515b4f206a0", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-1787960284-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "315732b6e3ee44d584b8e510ef199a90", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap720988c2-e7", "ovs_interfaceid": "720988c2-e733-41cb-b3ea-c215cfe46bd6", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=75234) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1987}}
Nov 29 03:26:26 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-1646d138-328c-4963-9e3c-55462273d73b req-4483bbcf-4a4a-41c5-976f-908a4c86a753 service nova] Acquired lock "refresh_cache-499676de-0e76-45ae-864c-44e55e3d7c7f" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:26:26 user nova-compute[75234]: DEBUG nova.network.neutron [req-1646d138-328c-4963-9e3c-55462273d73b req-4483bbcf-4a4a-41c5-976f-908a4c86a753 service nova] [instance: 499676de-0e76-45ae-864c-44e55e3d7c7f] Refreshing network info cache for port 720988c2-e733-41cb-b3ea-c215cfe46bd6 {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2006}}
Nov 29 03:26:26 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-a73e47e1-e5c8-4b33-84ee-dd89b7d8033b tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] [instance: 499676de-0e76-45ae-864c-44e55e3d7c7f] Start _get_guest_xml network_info=[{"id": "720988c2-e733-41cb-b3ea-c215cfe46bd6", "address": "fa:16:3e:0d:b8:b3", "network": {"id": "0f289171-3d73-4054-95a0-2515b4f206a0", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-1787960284-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "315732b6e3ee44d584b8e510ef199a90", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap720988c2-e7", "ovs_interfaceid": "720988c2-e733-41cb-b3ea-c215cfe46bd6", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum='c8fc807773e5354afe61636071771906',container_format='bare',created_at=2023-11-29T03:02:15Z,direct_url=<?>,disk_format='qcow2',id=96c30ba7-9b73-4a30-8ba4-7138879573b7,min_disk=0,min_ram=0,name='cirros-0.6.2-x86_64-disk',owner='b18e380921c748b78517b6f1211cd0f1',properties=ImageMetaProps,protected=<?>,size=21430272,status='active',tags=<?>,updated_at=2023-11-29T03:02:17Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [{'device_name': '/dev/vda', 'encryption_format': None, 'encrypted': False, 'device_type': 'disk', 'size': 0, 'disk_bus': 'virtio', 'boot_index': 0, 'encryption_options': None, 'encryption_secret_uuid': None, 'guest_format': None, 'image_id': '96c30ba7-9b73-4a30-8ba4-7138879573b7'}], 'ephemerals': [], 'block_device_mapping': [], 'swap': None} {{(pid=75234) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7562}}
Nov 29 03:26:26 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-a73e47e1-e5c8-4b33-84ee-dd89b7d8033b tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:26:26 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-a73e47e1-e5c8-4b33-84ee-dd89b7d8033b tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:26:26 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-a73e47e1-e5c8-4b33-84ee-dd89b7d8033b tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Searching host: 'user' for CPU controller through CGroups V1... {{(pid=75234) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1652}}
Nov 29 03:26:26 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-a73e47e1-e5c8-4b33-84ee-dd89b7d8033b tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] CPU controller missing on host. {{(pid=75234) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1662}}
Nov 29 03:26:26 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-a73e47e1-e5c8-4b33-84ee-dd89b7d8033b tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Searching host: 'user' for CPU controller through CGroups V2... {{(pid=75234) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1671}}
Nov 29 03:26:26 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-a73e47e1-e5c8-4b33-84ee-dd89b7d8033b tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] CPU controller found on host. {{(pid=75234) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1678}}
Nov 29 03:26:26 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-a73e47e1-e5c8-4b33-84ee-dd89b7d8033b tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=75234) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5400}}
Nov 29 03:26:26 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-a73e47e1-e5c8-4b33-84ee-dd89b7d8033b tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-11-29T03:02:04Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='1',id=6,is_public=True,memory_mb=512,name='m1.tiny',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='c8fc807773e5354afe61636071771906',container_format='bare',created_at=2023-11-29T03:02:15Z,direct_url=<?>,disk_format='qcow2',id=96c30ba7-9b73-4a30-8ba4-7138879573b7,min_disk=0,min_ram=0,name='cirros-0.6.2-x86_64-disk',owner='b18e380921c748b78517b6f1211cd0f1',properties=ImageMetaProps,protected=<?>,size=21430272,status='active',tags=<?>,updated_at=2023-11-29T03:02:17Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:563}}
Nov 29 03:26:26 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-a73e47e1-e5c8-4b33-84ee-dd89b7d8033b tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Flavor limits 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:348}}
Nov 29 03:26:26 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-a73e47e1-e5c8-4b33-84ee-dd89b7d8033b tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Image limits 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:352}}
Nov 29 03:26:26 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-a73e47e1-e5c8-4b33-84ee-dd89b7d8033b tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Flavor pref 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:388}}
Nov 29 03:26:26 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-a73e47e1-e5c8-4b33-84ee-dd89b7d8033b tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Image pref 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:392}}
Nov 29 03:26:26 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-a73e47e1-e5c8-4b33-84ee-dd89b7d8033b tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:430}}
Nov 29 03:26:26 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-a73e47e1-e5c8-4b33-84ee-dd89b7d8033b tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:569}}
Nov 29 03:26:26 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-a73e47e1-e5c8-4b33-84ee-dd89b7d8033b tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=75234) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:471}}
Nov 29 03:26:26 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-a73e47e1-e5c8-4b33-84ee-dd89b7d8033b tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Got 1 possible topologies {{(pid=75234) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:501}}
Nov 29 03:26:26 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-a73e47e1-e5c8-4b33-84ee-dd89b7d8033b tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:575}}
Nov 29 03:26:26 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-a73e47e1-e5c8-4b33-84ee-dd89b7d8033b tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:577}}
Nov 29 03:26:26 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-a73e47e1-e5c8-4b33-84ee-dd89b7d8033b tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-11-29T03:26:23Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-AttachVolumeNegativeTest-server-272620904',display_name='tempest-AttachVolumeNegativeTest-server-272620904',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-attachvolumenegativetest-server-272620904',id=42,image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBAeV+xn9UGsWgfasuo5ci4YCLCfVHxnk9mWnUGlN4CXUvM0yE0RrqSC8l9glhTxJAVaU31Y7rLe4uk5w24n4N4cYiV7rwY+sOMhAYj/kLr660fRVmge9Wnqb+j9IA4vy5Q==',key_name='tempest-keypair-374385187',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='315732b6e3ee44d584b8e510ef199a90',ramdisk_id='',reservation_id='r-brbd5rm9',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-AttachVolumeNegativeTest-1582496806',owner_user_name='tempest-AttachVolumeNegativeTest-1582496806-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-11-29T03:26:25Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='66686114450f4f31b3a917667a152d8b',uuid=499676de-0e76-45ae-864c-44e55e3d7c7f,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "720988c2-e733-41cb-b3ea-c215cfe46bd6", "address": "fa:16:3e:0d:b8:b3", "network": {"id": "0f289171-3d73-4054-95a0-2515b4f206a0", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-1787960284-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "315732b6e3ee44d584b8e510ef199a90", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap720988c2-e7", "ovs_interfaceid": "720988c2-e733-41cb-b3ea-c215cfe46bd6", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=75234) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Nov 29 03:26:26 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-a73e47e1-e5c8-4b33-84ee-dd89b7d8033b tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Converting VIF {"id": "720988c2-e733-41cb-b3ea-c215cfe46bd6", "address": "fa:16:3e:0d:b8:b3", "network": {"id": "0f289171-3d73-4054-95a0-2515b4f206a0", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-1787960284-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "315732b6e3ee44d584b8e510ef199a90", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap720988c2-e7", "ovs_interfaceid": "720988c2-e733-41cb-b3ea-c215cfe46bd6", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:26:26 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-a73e47e1-e5c8-4b33-84ee-dd89b7d8033b tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:0d:b8:b3,bridge_name='br-int',has_traffic_filtering=True,id=720988c2-e733-41cb-b3ea-c215cfe46bd6,network=Network(0f289171-3d73-4054-95a0-2515b4f206a0),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap720988c2-e7') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:26:26 user nova-compute[75234]: DEBUG nova.objects.instance [None req-a73e47e1-e5c8-4b33-84ee-dd89b7d8033b tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Lazy-loading 'pci_devices' on Instance uuid 499676de-0e76-45ae-864c-44e55e3d7c7f {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:26:26 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-a73e47e1-e5c8-4b33-84ee-dd89b7d8033b tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] [instance: 499676de-0e76-45ae-864c-44e55e3d7c7f] End _get_guest_xml xml=<domain type="kvm">
Nov 29 03:26:26 user nova-compute[75234]:   <uuid>499676de-0e76-45ae-864c-44e55e3d7c7f</uuid>
Nov 29 03:26:26 user nova-compute[75234]:   <name>instance-0000002a</name>
Nov 29 03:26:26 user nova-compute[75234]:   <memory>524288</memory>
Nov 29 03:26:26 user nova-compute[75234]:   <vcpu>1</vcpu>
Nov 29 03:26:26 user nova-compute[75234]:   <metadata>
Nov 29 03:26:26 user nova-compute[75234]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Nov 29 03:26:26 user nova-compute[75234]:       <nova:package version="0.0.0"/>
Nov 29 03:26:26 user nova-compute[75234]:       <nova:name>tempest-AttachVolumeNegativeTest-server-272620904</nova:name>
Nov 29 03:26:26 user nova-compute[75234]:       <nova:creationTime>2023-11-29 03:26:26</nova:creationTime>
Nov 29 03:26:26 user nova-compute[75234]:       <nova:flavor name="m1.tiny">
Nov 29 03:26:26 user nova-compute[75234]:         <nova:memory>512</nova:memory>
Nov 29 03:26:26 user nova-compute[75234]:         <nova:disk>1</nova:disk>
Nov 29 03:26:26 user nova-compute[75234]:         <nova:swap>0</nova:swap>
Nov 29 03:26:26 user nova-compute[75234]:         <nova:ephemeral>0</nova:ephemeral>
Nov 29 03:26:26 user nova-compute[75234]:         <nova:vcpus>1</nova:vcpus>
Nov 29 03:26:26 user nova-compute[75234]:       </nova:flavor>
Nov 29 03:26:26 user nova-compute[75234]:       <nova:owner>
Nov 29 03:26:26 user nova-compute[75234]:         <nova:user uuid="66686114450f4f31b3a917667a152d8b">tempest-AttachVolumeNegativeTest-1582496806-project-member</nova:user>
Nov 29 03:26:26 user nova-compute[75234]:         <nova:project uuid="315732b6e3ee44d584b8e510ef199a90">tempest-AttachVolumeNegativeTest-1582496806</nova:project>
Nov 29 03:26:26 user nova-compute[75234]:       </nova:owner>
Nov 29 03:26:26 user nova-compute[75234]:       <nova:root type="image" uuid="96c30ba7-9b73-4a30-8ba4-7138879573b7"/>
Nov 29 03:26:26 user nova-compute[75234]:       <nova:ports>
Nov 29 03:26:26 user nova-compute[75234]:         <nova:port uuid="720988c2-e733-41cb-b3ea-c215cfe46bd6">
Nov 29 03:26:26 user nova-compute[75234]:           <nova:ip type="fixed" address="10.0.0.4" ipVersion="4"/>
Nov 29 03:26:26 user nova-compute[75234]:         </nova:port>
Nov 29 03:26:26 user nova-compute[75234]:       </nova:ports>
Nov 29 03:26:26 user nova-compute[75234]:     </nova:instance>
Nov 29 03:26:26 user nova-compute[75234]:   </metadata>
Nov 29 03:26:26 user nova-compute[75234]:   <sysinfo type="smbios">
Nov 29 03:26:26 user nova-compute[75234]:     <system>
Nov 29 03:26:26 user nova-compute[75234]:       <entry name="manufacturer">OpenStack Foundation</entry>
Nov 29 03:26:26 user nova-compute[75234]:       <entry name="product">OpenStack Nova</entry>
Nov 29 03:26:26 user nova-compute[75234]:       <entry name="version">0.0.0</entry>
Nov 29 03:26:26 user nova-compute[75234]:       <entry name="serial">499676de-0e76-45ae-864c-44e55e3d7c7f</entry>
Nov 29 03:26:26 user nova-compute[75234]:       <entry name="uuid">499676de-0e76-45ae-864c-44e55e3d7c7f</entry>
Nov 29 03:26:26 user nova-compute[75234]:       <entry name="family">Virtual Machine</entry>
Nov 29 03:26:26 user nova-compute[75234]:     </system>
Nov 29 03:26:26 user nova-compute[75234]:   </sysinfo>
Nov 29 03:26:26 user nova-compute[75234]:   <os>
Nov 29 03:26:26 user nova-compute[75234]:     <type arch="x86_64" machine="pc">hvm</type>
Nov 29 03:26:26 user nova-compute[75234]:     <boot dev="hd"/>
Nov 29 03:26:26 user nova-compute[75234]:     <smbios mode="sysinfo"/>
Nov 29 03:26:26 user nova-compute[75234]:   </os>
Nov 29 03:26:26 user nova-compute[75234]:   <features>
Nov 29 03:26:26 user nova-compute[75234]:     <acpi/>
Nov 29 03:26:26 user nova-compute[75234]:     <apic/>
Nov 29 03:26:26 user nova-compute[75234]:     <vmcoreinfo/>
Nov 29 03:26:26 user nova-compute[75234]:   </features>
Nov 29 03:26:26 user nova-compute[75234]:   <clock offset="utc">
Nov 29 03:26:26 user nova-compute[75234]:     <timer name="pit" tickpolicy="delay"/>
Nov 29 03:26:26 user nova-compute[75234]:     <timer name="rtc" tickpolicy="catchup"/>
Nov 29 03:26:26 user nova-compute[75234]:     <timer name="hpet" present="no"/>
Nov 29 03:26:26 user nova-compute[75234]:   </clock>
Nov 29 03:26:26 user nova-compute[75234]:   <cpu mode="custom" match="exact">
Nov 29 03:26:26 user nova-compute[75234]:     <model>Nehalem</model>
Nov 29 03:26:26 user nova-compute[75234]:     <topology sockets="1" cores="1" threads="1"/>
Nov 29 03:26:26 user nova-compute[75234]:   </cpu>
Nov 29 03:26:26 user nova-compute[75234]:   <devices>
Nov 29 03:26:26 user nova-compute[75234]:     <disk type="file" device="disk">
Nov 29 03:26:26 user nova-compute[75234]:       <driver name="qemu" type="qcow2" cache="none"/>
Nov 29 03:26:26 user nova-compute[75234]:       <source file="/opt/stack/data/nova/instances/499676de-0e76-45ae-864c-44e55e3d7c7f/disk"/>
Nov 29 03:26:26 user nova-compute[75234]:       <target dev="vda" bus="virtio"/>
Nov 29 03:26:26 user nova-compute[75234]:     </disk>
Nov 29 03:26:26 user nova-compute[75234]:     <interface type="ethernet">
Nov 29 03:26:26 user nova-compute[75234]:       <mac address="fa:16:3e:0d:b8:b3"/>
Nov 29 03:26:26 user nova-compute[75234]:       <model type="virtio"/>
Nov 29 03:26:26 user nova-compute[75234]:       <mtu size="1442"/>
Nov 29 03:26:26 user nova-compute[75234]:       <target dev="tap720988c2-e7"/>
Nov 29 03:26:26 user nova-compute[75234]:     </interface>
Nov 29 03:26:26 user nova-compute[75234]:     <serial type="pty">
Nov 29 03:26:26 user nova-compute[75234]:       <log file="/opt/stack/data/nova/instances/499676de-0e76-45ae-864c-44e55e3d7c7f/console.log" append="off"/>
Nov 29 03:26:26 user nova-compute[75234]:     </serial>
Nov 29 03:26:26 user nova-compute[75234]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Nov 29 03:26:26 user nova-compute[75234]:     <video>
Nov 29 03:26:26 user nova-compute[75234]:       <model type="virtio"/>
Nov 29 03:26:26 user nova-compute[75234]:     </video>
Nov 29 03:26:26 user nova-compute[75234]:     <rng model="virtio">
Nov 29 03:26:26 user nova-compute[75234]:       <backend model="random">/dev/urandom</backend>
Nov 29 03:26:26 user nova-compute[75234]:     </rng>
Nov 29 03:26:26 user nova-compute[75234]:     <controller type="usb" index="0" model="none"/>
Nov 29 03:26:26 user nova-compute[75234]:     <memballoon model="virtio">
Nov 29 03:26:26 user nova-compute[75234]:       <stats period="10"/>
Nov 29 03:26:26 user nova-compute[75234]:     </memballoon>
Nov 29 03:26:26 user nova-compute[75234]:   </devices>
Nov 29 03:26:26 user nova-compute[75234]: </domain>
Nov 29 03:26:26 user nova-compute[75234]:  {{(pid=75234) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7568}}
Nov 29 03:26:26 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-a73e47e1-e5c8-4b33-84ee-dd89b7d8033b tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-11-29T03:26:23Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-AttachVolumeNegativeTest-server-272620904',display_name='tempest-AttachVolumeNegativeTest-server-272620904',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-attachvolumenegativetest-server-272620904',id=42,image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBAeV+xn9UGsWgfasuo5ci4YCLCfVHxnk9mWnUGlN4CXUvM0yE0RrqSC8l9glhTxJAVaU31Y7rLe4uk5w24n4N4cYiV7rwY+sOMhAYj/kLr660fRVmge9Wnqb+j9IA4vy5Q==',key_name='tempest-keypair-374385187',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='315732b6e3ee44d584b8e510ef199a90',ramdisk_id='',reservation_id='r-brbd5rm9',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-AttachVolumeNegativeTest-1582496806',owner_user_name='tempest-AttachVolumeNegativeTest-1582496806-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-11-29T03:26:25Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='66686114450f4f31b3a917667a152d8b',uuid=499676de-0e76-45ae-864c-44e55e3d7c7f,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "720988c2-e733-41cb-b3ea-c215cfe46bd6", "address": "fa:16:3e:0d:b8:b3", "network": {"id": "0f289171-3d73-4054-95a0-2515b4f206a0", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-1787960284-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "315732b6e3ee44d584b8e510ef199a90", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap720988c2-e7", "ovs_interfaceid": "720988c2-e733-41cb-b3ea-c215cfe46bd6", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Nov 29 03:26:26 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-a73e47e1-e5c8-4b33-84ee-dd89b7d8033b tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Converting VIF {"id": "720988c2-e733-41cb-b3ea-c215cfe46bd6", "address": "fa:16:3e:0d:b8:b3", "network": {"id": "0f289171-3d73-4054-95a0-2515b4f206a0", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-1787960284-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "315732b6e3ee44d584b8e510ef199a90", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap720988c2-e7", "ovs_interfaceid": "720988c2-e733-41cb-b3ea-c215cfe46bd6", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:26:26 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-a73e47e1-e5c8-4b33-84ee-dd89b7d8033b tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:0d:b8:b3,bridge_name='br-int',has_traffic_filtering=True,id=720988c2-e733-41cb-b3ea-c215cfe46bd6,network=Network(0f289171-3d73-4054-95a0-2515b4f206a0),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap720988c2-e7') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:26:26 user nova-compute[75234]: DEBUG os_vif [None req-a73e47e1-e5c8-4b33-84ee-dd89b7d8033b tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:0d:b8:b3,bridge_name='br-int',has_traffic_filtering=True,id=720988c2-e733-41cb-b3ea-c215cfe46bd6,network=Network(0f289171-3d73-4054-95a0-2515b4f206a0),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap720988c2-e7') {{(pid=75234) plug /opt/stack/data/venv/lib/python3.10/site-packages/os_vif/__init__.py:76}}
Nov 29 03:26:26 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:26:26 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:26:26 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Nov 29 03:26:26 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:26:26 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbCreateCommand(_result=None, table=QoS, columns={'type': 'linux-noop', 'external_ids': {'id': '06165367-058e-5779-abfd-f0141f9d6534', '_type': 'linux-noop'}}, row=False) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:26:26 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:26:26 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:26:26 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:26:26 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tap720988c2-e7, may_exist=True, interface_attrs={}) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:26:26 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Port, record=tap720988c2-e7, col_values=(('qos', UUID('0ad6b090-4acd-4240-93c8-d0ce6745dbb7')),), if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:26:26 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DbSetCommand(_result=None, table=Interface, record=tap720988c2-e7, col_values=(('external_ids', {'iface-id': '720988c2-e733-41cb-b3ea-c215cfe46bd6', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:0d:b8:b3', 'vm-uuid': '499676de-0e76-45ae-864c-44e55e3d7c7f'}),), if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:26:26 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:26:26 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:26:26 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:26:26 user nova-compute[75234]: INFO os_vif [None req-a73e47e1-e5c8-4b33-84ee-dd89b7d8033b tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:0d:b8:b3,bridge_name='br-int',has_traffic_filtering=True,id=720988c2-e733-41cb-b3ea-c215cfe46bd6,network=Network(0f289171-3d73-4054-95a0-2515b4f206a0),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap720988c2-e7')
Nov 29 03:26:26 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-a73e47e1-e5c8-4b33-84ee-dd89b7d8033b tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] No BDM found with device name vda, not building metadata. {{(pid=75234) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12089}}
Nov 29 03:26:26 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-a73e47e1-e5c8-4b33-84ee-dd89b7d8033b tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] No VIF found with MAC fa:16:3e:0d:b8:b3, not building metadata {{(pid=75234) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Nov 29 03:26:27 user nova-compute[75234]: DEBUG nova.network.neutron [req-1646d138-328c-4963-9e3c-55462273d73b req-4483bbcf-4a4a-41c5-976f-908a4c86a753 service nova] [instance: 499676de-0e76-45ae-864c-44e55e3d7c7f] Updated VIF entry in instance network info cache for port 720988c2-e733-41cb-b3ea-c215cfe46bd6. {{(pid=75234) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3481}}
Nov 29 03:26:27 user nova-compute[75234]: DEBUG nova.network.neutron [req-1646d138-328c-4963-9e3c-55462273d73b req-4483bbcf-4a4a-41c5-976f-908a4c86a753 service nova] [instance: 499676de-0e76-45ae-864c-44e55e3d7c7f] Updating instance_info_cache with network_info: [{"id": "720988c2-e733-41cb-b3ea-c215cfe46bd6", "address": "fa:16:3e:0d:b8:b3", "network": {"id": "0f289171-3d73-4054-95a0-2515b4f206a0", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-1787960284-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "315732b6e3ee44d584b8e510ef199a90", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap720988c2-e7", "ovs_interfaceid": "720988c2-e733-41cb-b3ea-c215cfe46bd6", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:26:27 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-1646d138-328c-4963-9e3c-55462273d73b req-4483bbcf-4a4a-41c5-976f-908a4c86a753 service nova] Releasing lock "refresh_cache-499676de-0e76-45ae-864c-44e55e3d7c7f" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:26:27 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:26:28 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:26:28 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:26:28 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:26:28 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:26:28 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:26:29 user nova-compute[75234]: DEBUG nova.compute.manager [req-79f9b314-8862-4ea2-98b7-37c357e12a99 req-ba63f41f-f9fe-4a53-addd-daf71f15c385 service nova] [instance: 499676de-0e76-45ae-864c-44e55e3d7c7f] Received event network-vif-plugged-720988c2-e733-41cb-b3ea-c215cfe46bd6 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:26:29 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-79f9b314-8862-4ea2-98b7-37c357e12a99 req-ba63f41f-f9fe-4a53-addd-daf71f15c385 service nova] Acquiring lock "499676de-0e76-45ae-864c-44e55e3d7c7f-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:26:29 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-79f9b314-8862-4ea2-98b7-37c357e12a99 req-ba63f41f-f9fe-4a53-addd-daf71f15c385 service nova] Lock "499676de-0e76-45ae-864c-44e55e3d7c7f-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:26:29 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-79f9b314-8862-4ea2-98b7-37c357e12a99 req-ba63f41f-f9fe-4a53-addd-daf71f15c385 service nova] Lock "499676de-0e76-45ae-864c-44e55e3d7c7f-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:26:29 user nova-compute[75234]: DEBUG nova.compute.manager [req-79f9b314-8862-4ea2-98b7-37c357e12a99 req-ba63f41f-f9fe-4a53-addd-daf71f15c385 service nova] [instance: 499676de-0e76-45ae-864c-44e55e3d7c7f] No waiting events found dispatching network-vif-plugged-720988c2-e733-41cb-b3ea-c215cfe46bd6 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:26:29 user nova-compute[75234]: WARNING nova.compute.manager [req-79f9b314-8862-4ea2-98b7-37c357e12a99 req-ba63f41f-f9fe-4a53-addd-daf71f15c385 service nova] [instance: 499676de-0e76-45ae-864c-44e55e3d7c7f] Received unexpected event network-vif-plugged-720988c2-e733-41cb-b3ea-c215cfe46bd6 for instance with vm_state building and task_state spawning.
Nov 29 03:26:30 user nova-compute[75234]: DEBUG nova.compute.manager [None req-a73e47e1-e5c8-4b33-84ee-dd89b7d8033b tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] [instance: 499676de-0e76-45ae-864c-44e55e3d7c7f] Instance event wait completed in 0 seconds for  {{(pid=75234) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Nov 29 03:26:30 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-a73e47e1-e5c8-4b33-84ee-dd89b7d8033b tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] [instance: 499676de-0e76-45ae-864c-44e55e3d7c7f] Guest created on hypervisor {{(pid=75234) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4421}}
Nov 29 03:26:30 user nova-compute[75234]: INFO nova.virt.libvirt.driver [-] [instance: 499676de-0e76-45ae-864c-44e55e3d7c7f] Instance spawned successfully.
Nov 29 03:26:30 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-a73e47e1-e5c8-4b33-84ee-dd89b7d8033b tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] [instance: 499676de-0e76-45ae-864c-44e55e3d7c7f] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:904}}
Nov 29 03:26:30 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-a73e47e1-e5c8-4b33-84ee-dd89b7d8033b tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] [instance: 499676de-0e76-45ae-864c-44e55e3d7c7f] Found default for hw_cdrom_bus of ide {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:26:30 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-a73e47e1-e5c8-4b33-84ee-dd89b7d8033b tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] [instance: 499676de-0e76-45ae-864c-44e55e3d7c7f] Found default for hw_disk_bus of virtio {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:26:30 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-a73e47e1-e5c8-4b33-84ee-dd89b7d8033b tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] [instance: 499676de-0e76-45ae-864c-44e55e3d7c7f] Found default for hw_input_bus of None {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:26:30 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-a73e47e1-e5c8-4b33-84ee-dd89b7d8033b tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] [instance: 499676de-0e76-45ae-864c-44e55e3d7c7f] Found default for hw_pointer_model of None {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:26:30 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-a73e47e1-e5c8-4b33-84ee-dd89b7d8033b tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] [instance: 499676de-0e76-45ae-864c-44e55e3d7c7f] Found default for hw_video_model of virtio {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:26:30 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-a73e47e1-e5c8-4b33-84ee-dd89b7d8033b tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] [instance: 499676de-0e76-45ae-864c-44e55e3d7c7f] Found default for hw_vif_model of virtio {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:26:30 user nova-compute[75234]: INFO nova.compute.manager [None req-a73e47e1-e5c8-4b33-84ee-dd89b7d8033b tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] [instance: 499676de-0e76-45ae-864c-44e55e3d7c7f] Took 5.94 seconds to spawn the instance on the hypervisor.
Nov 29 03:26:30 user nova-compute[75234]: DEBUG nova.compute.manager [None req-a73e47e1-e5c8-4b33-84ee-dd89b7d8033b tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] [instance: 499676de-0e76-45ae-864c-44e55e3d7c7f] Checking state {{(pid=75234) _get_power_state /opt/stack/nova/nova/compute/manager.py:1782}}
Nov 29 03:26:30 user nova-compute[75234]: INFO nova.compute.manager [None req-a73e47e1-e5c8-4b33-84ee-dd89b7d8033b tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] [instance: 499676de-0e76-45ae-864c-44e55e3d7c7f] Took 6.78 seconds to build instance.
Nov 29 03:26:30 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-a73e47e1-e5c8-4b33-84ee-dd89b7d8033b tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Lock "499676de-0e76-45ae-864c-44e55e3d7c7f" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 6.902s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:26:31 user nova-compute[75234]: DEBUG nova.compute.manager [req-412f1e40-4c17-4de0-a70a-6bd92f1a8e2a req-1dc624d2-f017-4b11-af06-71fdadc31e13 service nova] [instance: 499676de-0e76-45ae-864c-44e55e3d7c7f] Received event network-vif-plugged-720988c2-e733-41cb-b3ea-c215cfe46bd6 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:26:31 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-412f1e40-4c17-4de0-a70a-6bd92f1a8e2a req-1dc624d2-f017-4b11-af06-71fdadc31e13 service nova] Acquiring lock "499676de-0e76-45ae-864c-44e55e3d7c7f-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:26:31 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-412f1e40-4c17-4de0-a70a-6bd92f1a8e2a req-1dc624d2-f017-4b11-af06-71fdadc31e13 service nova] Lock "499676de-0e76-45ae-864c-44e55e3d7c7f-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:26:31 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-412f1e40-4c17-4de0-a70a-6bd92f1a8e2a req-1dc624d2-f017-4b11-af06-71fdadc31e13 service nova] Lock "499676de-0e76-45ae-864c-44e55e3d7c7f-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:26:31 user nova-compute[75234]: DEBUG nova.compute.manager [req-412f1e40-4c17-4de0-a70a-6bd92f1a8e2a req-1dc624d2-f017-4b11-af06-71fdadc31e13 service nova] [instance: 499676de-0e76-45ae-864c-44e55e3d7c7f] No waiting events found dispatching network-vif-plugged-720988c2-e733-41cb-b3ea-c215cfe46bd6 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:26:31 user nova-compute[75234]: WARNING nova.compute.manager [req-412f1e40-4c17-4de0-a70a-6bd92f1a8e2a req-1dc624d2-f017-4b11-af06-71fdadc31e13 service nova] [instance: 499676de-0e76-45ae-864c-44e55e3d7c7f] Received unexpected event network-vif-plugged-720988c2-e733-41cb-b3ea-c215cfe46bd6 for instance with vm_state active and task_state None.
Nov 29 03:26:31 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:26:32 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:26:36 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:26:37 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:26:41 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:26:42 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:26:46 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:26:47 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:26:51 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-0e3dcae5-c1fb-4d02-9b3f-4d5cb6932574 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Acquiring lock "f6bc212d-85cd-479c-92ae-112c9f0101bc" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:26:51 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-0e3dcae5-c1fb-4d02-9b3f-4d5cb6932574 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Lock "f6bc212d-85cd-479c-92ae-112c9f0101bc" acquired by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:26:51 user nova-compute[75234]: DEBUG nova.objects.instance [None req-0e3dcae5-c1fb-4d02-9b3f-4d5cb6932574 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Lazy-loading 'flavor' on Instance uuid f6bc212d-85cd-479c-92ae-112c9f0101bc {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:26:51 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-0e3dcae5-c1fb-4d02-9b3f-4d5cb6932574 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Lock "f6bc212d-85cd-479c-92ae-112c9f0101bc" "released" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: held 0.055s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:26:51 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-0e3dcae5-c1fb-4d02-9b3f-4d5cb6932574 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Acquiring lock "f6bc212d-85cd-479c-92ae-112c9f0101bc" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:26:51 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-0e3dcae5-c1fb-4d02-9b3f-4d5cb6932574 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Lock "f6bc212d-85cd-479c-92ae-112c9f0101bc" acquired by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:26:51 user nova-compute[75234]: INFO nova.compute.manager [None req-0e3dcae5-c1fb-4d02-9b3f-4d5cb6932574 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] [instance: f6bc212d-85cd-479c-92ae-112c9f0101bc] Attaching volume 5b9c21f2-f2fd-4675-b2b9-09b63a5f2132 to /dev/vdb
Nov 29 03:26:51 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:26:51 user nova-compute[75234]: DEBUG os_brick.utils [None req-0e3dcae5-c1fb-4d02-9b3f-4d5cb6932574 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:176}}
Nov 29 03:26:51 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:26:51 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.017s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:26:51 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[a32f2f82-269e-4ef4-a283-b3e56a394a73]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:dd1368c5ba2c\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:26:51 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt -v / -n -o SOURCE {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:26:51 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt -v / -n -o SOURCE" returned: 0 in 0.015s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:26:51 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[36640ffa-ffec-41d2-a712-7503eeab84ca]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:26:51 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:26:51 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.016s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:26:51 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[8bf901fc-76e1-4cf3-a6c9-df537a0d06ad]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:26:51 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[d7324a20-d414-4d2d-9851-de35ccebc8d9]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:26:51 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-0e3dcae5-c1fb-4d02-9b3f-4d5cb6932574 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Running cmd (subprocess): nvme version {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:26:51 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-0e3dcae5-c1fb-4d02-9b3f-4d5cb6932574 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] 'nvme version' failed. Not Retrying. {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:473}}
Nov 29 03:26:51 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.nvmeof [None req-0e3dcae5-c1fb-4d02-9b3f-4d5cb6932574 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] nvme not present on system {{(pid=75234) nvme_present /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/nvmeof.py:756}}
Nov 29 03:26:51 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:26:51 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:473}}
Nov 29 03:26:51 user nova-compute[75234]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Nov 29 03:26:51 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[6df9d6a1-5ddd-40e4-88c4-6c6c2ffec001]: (4, '') {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:26:52 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.lightos [None req-0e3dcae5-c1fb-4d02-9b3f-4d5cb6932574 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=75234) find_dsc /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/lightos.py:98}}
Nov 29 03:26:52 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.lightos [None req-0e3dcae5-c1fb-4d02-9b3f-4d5cb6932574 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=75234) get_connector_properties /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/lightos.py:76}}
Nov 29 03:26:52 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.lightos [None req-0e3dcae5-c1fb-4d02-9b3f-4d5cb6932574 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] LIGHTOS: no hostnqn found. {{(pid=75234) get_connector_properties /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/lightos.py:84}}
Nov 29 03:26:52 user nova-compute[75234]: DEBUG os_brick.utils [None req-0e3dcae5-c1fb-4d02-9b3f-4d5cb6932574 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] <== get_connector_properties: return (128ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:dd1368c5ba2c', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:203}}
Nov 29 03:26:52 user nova-compute[75234]: DEBUG nova.virt.block_device [None req-0e3dcae5-c1fb-4d02-9b3f-4d5cb6932574 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] [instance: f6bc212d-85cd-479c-92ae-112c9f0101bc] Updating existing volume attachment record: 707d5410-50a4-4f75-b440-de0577e0fb59 {{(pid=75234) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Nov 29 03:26:52 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:26:56 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:26:57 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:27:01 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:27:02 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:27:02 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-aabb9b15-b98b-419b-8adf-bf98fc1559d8 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Acquiring lock "a082c96e-e384-40b6-bcc8-fc5e0e104669" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:27:02 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-aabb9b15-b98b-419b-8adf-bf98fc1559d8 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Lock "a082c96e-e384-40b6-bcc8-fc5e0e104669" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:27:02 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-aabb9b15-b98b-419b-8adf-bf98fc1559d8 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Acquiring lock "a082c96e-e384-40b6-bcc8-fc5e0e104669-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:27:02 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-aabb9b15-b98b-419b-8adf-bf98fc1559d8 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Lock "a082c96e-e384-40b6-bcc8-fc5e0e104669-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:27:02 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-aabb9b15-b98b-419b-8adf-bf98fc1559d8 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Lock "a082c96e-e384-40b6-bcc8-fc5e0e104669-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:27:02 user nova-compute[75234]: INFO nova.compute.manager [None req-aabb9b15-b98b-419b-8adf-bf98fc1559d8 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] [instance: a082c96e-e384-40b6-bcc8-fc5e0e104669] Terminating instance
Nov 29 03:27:02 user nova-compute[75234]: DEBUG nova.compute.manager [None req-aabb9b15-b98b-419b-8adf-bf98fc1559d8 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] [instance: a082c96e-e384-40b6-bcc8-fc5e0e104669] Start destroying the instance on the hypervisor. {{(pid=75234) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3140}}
Nov 29 03:27:02 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:27:03 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:27:03 user nova-compute[75234]: DEBUG nova.compute.manager [req-72a3f728-3d9b-48ee-a12c-ec1a37c5762e req-877c6184-45a8-4adf-a876-cd911c862c76 service nova] [instance: a082c96e-e384-40b6-bcc8-fc5e0e104669] Received event network-vif-unplugged-0d131138-51b6-49f9-9a11-6b64e32e5796 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:27:03 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-72a3f728-3d9b-48ee-a12c-ec1a37c5762e req-877c6184-45a8-4adf-a876-cd911c862c76 service nova] Acquiring lock "a082c96e-e384-40b6-bcc8-fc5e0e104669-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:27:03 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-72a3f728-3d9b-48ee-a12c-ec1a37c5762e req-877c6184-45a8-4adf-a876-cd911c862c76 service nova] Lock "a082c96e-e384-40b6-bcc8-fc5e0e104669-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:27:03 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-72a3f728-3d9b-48ee-a12c-ec1a37c5762e req-877c6184-45a8-4adf-a876-cd911c862c76 service nova] Lock "a082c96e-e384-40b6-bcc8-fc5e0e104669-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:27:03 user nova-compute[75234]: DEBUG nova.compute.manager [req-72a3f728-3d9b-48ee-a12c-ec1a37c5762e req-877c6184-45a8-4adf-a876-cd911c862c76 service nova] [instance: a082c96e-e384-40b6-bcc8-fc5e0e104669] No waiting events found dispatching network-vif-unplugged-0d131138-51b6-49f9-9a11-6b64e32e5796 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:27:03 user nova-compute[75234]: DEBUG nova.compute.manager [req-72a3f728-3d9b-48ee-a12c-ec1a37c5762e req-877c6184-45a8-4adf-a876-cd911c862c76 service nova] [instance: a082c96e-e384-40b6-bcc8-fc5e0e104669] Received event network-vif-unplugged-0d131138-51b6-49f9-9a11-6b64e32e5796 for instance with task_state deleting. {{(pid=75234) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10890}}
Nov 29 03:27:03 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:27:03 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:27:03 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:27:03 user nova-compute[75234]: INFO nova.virt.libvirt.driver [-] [instance: a082c96e-e384-40b6-bcc8-fc5e0e104669] Instance destroyed successfully.
Nov 29 03:27:03 user nova-compute[75234]: DEBUG nova.objects.instance [None req-aabb9b15-b98b-419b-8adf-bf98fc1559d8 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Lazy-loading 'resources' on Instance uuid a082c96e-e384-40b6-bcc8-fc5e0e104669 {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:27:03 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-aabb9b15-b98b-419b-8adf-bf98fc1559d8 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-11-29T03:10:26Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-ServerRescueNegativeTestJSON-server-2058873758',display_name='tempest-ServerRescueNegativeTestJSON-server-2058873758',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-serverrescuenegativetestjson-server-2058873758',id=19,image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data=None,key_name=None,keypairs=<?>,launch_index=0,launched_at=2023-11-29T03:12:27Z,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='f8a7884867c944e1ba1024270e4902f3',ramdisk_id='',reservation_id='r-58ukk6ag',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-ServerRescueNegativeTestJSON-1182706601',owner_user_name='tempest-ServerRescueNegativeTestJSON-1182706601-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-11-29T03:12:28Z,user_data=None,user_id='6792e866677645c598ddb7d2c1723a1a',uuid=a082c96e-e384-40b6-bcc8-fc5e0e104669,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='rescued') vif={"id": "0d131138-51b6-49f9-9a11-6b64e32e5796", "address": "fa:16:3e:ee:2e:82", "network": {"id": "2b3c3140-f3b6-45da-8202-bb4d7f0dd130", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-2043420778-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "f8a7884867c944e1ba1024270e4902f3", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap0d131138-51", "ovs_interfaceid": "0d131138-51b6-49f9-9a11-6b64e32e5796", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Nov 29 03:27:03 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-aabb9b15-b98b-419b-8adf-bf98fc1559d8 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Converting VIF {"id": "0d131138-51b6-49f9-9a11-6b64e32e5796", "address": "fa:16:3e:ee:2e:82", "network": {"id": "2b3c3140-f3b6-45da-8202-bb4d7f0dd130", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-2043420778-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "f8a7884867c944e1ba1024270e4902f3", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap0d131138-51", "ovs_interfaceid": "0d131138-51b6-49f9-9a11-6b64e32e5796", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:27:03 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-aabb9b15-b98b-419b-8adf-bf98fc1559d8 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:ee:2e:82,bridge_name='br-int',has_traffic_filtering=True,id=0d131138-51b6-49f9-9a11-6b64e32e5796,network=Network(2b3c3140-f3b6-45da-8202-bb4d7f0dd130),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap0d131138-51') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:27:03 user nova-compute[75234]: DEBUG os_vif [None req-aabb9b15-b98b-419b-8adf-bf98fc1559d8 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Unplugging vif VIFOpenVSwitch(active=True,address=fa:16:3e:ee:2e:82,bridge_name='br-int',has_traffic_filtering=True,id=0d131138-51b6-49f9-9a11-6b64e32e5796,network=Network(2b3c3140-f3b6-45da-8202-bb4d7f0dd130),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap0d131138-51') {{(pid=75234) unplug /opt/stack/data/venv/lib/python3.10/site-packages/os_vif/__init__.py:109}}
Nov 29 03:27:03 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:27:03 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tap0d131138-51, bridge=br-int, if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:27:03 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:27:03 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:27:03 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:27:03 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbDestroyCommand(_result=None, table=QoS, record=0651645b-eb78-4bea-9b8d-25f0c14ee539) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:27:03 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:27:03 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:27:03 user nova-compute[75234]: INFO os_vif [None req-aabb9b15-b98b-419b-8adf-bf98fc1559d8 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Successfully unplugged vif VIFOpenVSwitch(active=True,address=fa:16:3e:ee:2e:82,bridge_name='br-int',has_traffic_filtering=True,id=0d131138-51b6-49f9-9a11-6b64e32e5796,network=Network(2b3c3140-f3b6-45da-8202-bb4d7f0dd130),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap0d131138-51')
Nov 29 03:27:03 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-aabb9b15-b98b-419b-8adf-bf98fc1559d8 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] [instance: a082c96e-e384-40b6-bcc8-fc5e0e104669] Deleting instance files /opt/stack/data/nova/instances/a082c96e-e384-40b6-bcc8-fc5e0e104669_del
Nov 29 03:27:03 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-aabb9b15-b98b-419b-8adf-bf98fc1559d8 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] [instance: a082c96e-e384-40b6-bcc8-fc5e0e104669] Deletion of /opt/stack/data/nova/instances/a082c96e-e384-40b6-bcc8-fc5e0e104669_del complete
Nov 29 03:27:03 user nova-compute[75234]: INFO nova.compute.manager [None req-aabb9b15-b98b-419b-8adf-bf98fc1559d8 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] [instance: a082c96e-e384-40b6-bcc8-fc5e0e104669] Took 0.97 seconds to destroy the instance on the hypervisor.
Nov 29 03:27:03 user nova-compute[75234]: DEBUG oslo.service.loopingcall [None req-aabb9b15-b98b-419b-8adf-bf98fc1559d8 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=75234) func /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/loopingcall.py:435}}
Nov 29 03:27:03 user nova-compute[75234]: DEBUG nova.compute.manager [-] [instance: a082c96e-e384-40b6-bcc8-fc5e0e104669] Deallocating network for instance {{(pid=75234) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2279}}
Nov 29 03:27:03 user nova-compute[75234]: DEBUG nova.network.neutron [-] [instance: a082c96e-e384-40b6-bcc8-fc5e0e104669] deallocate_for_instance() {{(pid=75234) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1802}}
Nov 29 03:27:04 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:27:04 user nova-compute[75234]: DEBUG nova.network.neutron [-] [instance: a082c96e-e384-40b6-bcc8-fc5e0e104669] Updating instance_info_cache with network_info: [] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:27:04 user nova-compute[75234]: INFO nova.compute.manager [-] [instance: a082c96e-e384-40b6-bcc8-fc5e0e104669] Took 0.56 seconds to deallocate network for instance.
Nov 29 03:27:04 user nova-compute[75234]: DEBUG nova.compute.manager [req-ea261e70-fb88-41c5-bcf5-2e0d23cdca52 req-e8c1f153-2c6d-4682-a1ef-167939295483 service nova] [instance: a082c96e-e384-40b6-bcc8-fc5e0e104669] Received event network-vif-deleted-0d131138-51b6-49f9-9a11-6b64e32e5796 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:27:04 user nova-compute[75234]: INFO nova.compute.manager [req-ea261e70-fb88-41c5-bcf5-2e0d23cdca52 req-e8c1f153-2c6d-4682-a1ef-167939295483 service nova] [instance: a082c96e-e384-40b6-bcc8-fc5e0e104669] Neutron deleted interface 0d131138-51b6-49f9-9a11-6b64e32e5796; detaching it from the instance and deleting it from the info cache
Nov 29 03:27:04 user nova-compute[75234]: DEBUG nova.network.neutron [req-ea261e70-fb88-41c5-bcf5-2e0d23cdca52 req-e8c1f153-2c6d-4682-a1ef-167939295483 service nova] [instance: a082c96e-e384-40b6-bcc8-fc5e0e104669] Updating instance_info_cache with network_info: [] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:27:04 user nova-compute[75234]: DEBUG nova.compute.manager [req-ea261e70-fb88-41c5-bcf5-2e0d23cdca52 req-e8c1f153-2c6d-4682-a1ef-167939295483 service nova] [instance: a082c96e-e384-40b6-bcc8-fc5e0e104669] Detach interface failed, port_id=0d131138-51b6-49f9-9a11-6b64e32e5796, reason: Instance a082c96e-e384-40b6-bcc8-fc5e0e104669 could not be found. {{(pid=75234) _process_instance_vif_deleted_event /opt/stack/nova/nova/compute/manager.py:10946}}
Nov 29 03:27:04 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._cleanup_incomplete_migrations {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:27:04 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Cleaning up deleted instances with incomplete migration  {{(pid=75234) _cleanup_incomplete_migrations /opt/stack/nova/nova/compute/manager.py:11247}}
Nov 29 03:27:04 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-aabb9b15-b98b-419b-8adf-bf98fc1559d8 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:27:04 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-aabb9b15-b98b-419b-8adf-bf98fc1559d8 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:27:04 user nova-compute[75234]: DEBUG nova.compute.provider_tree [None req-aabb9b15-b98b-419b-8adf-bf98fc1559d8 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Inventory has not changed in ProviderTree for provider: 4d291220-7c37-4839-a0e3-b23c66502b74 {{(pid=75234) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Nov 29 03:27:04 user nova-compute[75234]: DEBUG nova.scheduler.client.report [None req-aabb9b15-b98b-419b-8adf-bf98fc1559d8 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Inventory has not changed for provider 4d291220-7c37-4839-a0e3-b23c66502b74 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=75234) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Nov 29 03:27:04 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-aabb9b15-b98b-419b-8adf-bf98fc1559d8 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.371s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:27:04 user nova-compute[75234]: INFO nova.scheduler.client.report [None req-aabb9b15-b98b-419b-8adf-bf98fc1559d8 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Deleted allocations for instance a082c96e-e384-40b6-bcc8-fc5e0e104669
Nov 29 03:27:04 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-aabb9b15-b98b-419b-8adf-bf98fc1559d8 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Lock "a082c96e-e384-40b6-bcc8-fc5e0e104669" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 2.099s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:27:05 user nova-compute[75234]: DEBUG nova.compute.manager [req-12532201-a0e3-4872-8a64-1599e40c3b33 req-0c241975-e59d-4305-9264-c09f633763a6 service nova] [instance: a082c96e-e384-40b6-bcc8-fc5e0e104669] Received event network-vif-plugged-0d131138-51b6-49f9-9a11-6b64e32e5796 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:27:05 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-12532201-a0e3-4872-8a64-1599e40c3b33 req-0c241975-e59d-4305-9264-c09f633763a6 service nova] Acquiring lock "a082c96e-e384-40b6-bcc8-fc5e0e104669-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:27:05 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-12532201-a0e3-4872-8a64-1599e40c3b33 req-0c241975-e59d-4305-9264-c09f633763a6 service nova] Lock "a082c96e-e384-40b6-bcc8-fc5e0e104669-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:27:05 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-12532201-a0e3-4872-8a64-1599e40c3b33 req-0c241975-e59d-4305-9264-c09f633763a6 service nova] Lock "a082c96e-e384-40b6-bcc8-fc5e0e104669-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:27:05 user nova-compute[75234]: DEBUG nova.compute.manager [req-12532201-a0e3-4872-8a64-1599e40c3b33 req-0c241975-e59d-4305-9264-c09f633763a6 service nova] [instance: a082c96e-e384-40b6-bcc8-fc5e0e104669] No waiting events found dispatching network-vif-plugged-0d131138-51b6-49f9-9a11-6b64e32e5796 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:27:05 user nova-compute[75234]: WARNING nova.compute.manager [req-12532201-a0e3-4872-8a64-1599e40c3b33 req-0c241975-e59d-4305-9264-c09f633763a6 service nova] [instance: a082c96e-e384-40b6-bcc8-fc5e0e104669] Received unexpected event network-vif-plugged-0d131138-51b6-49f9-9a11-6b64e32e5796 for instance with vm_state deleted and task_state None.
Nov 29 03:27:06 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:27:07 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:27:07 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:27:07 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=75234) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10541}}
Nov 29 03:27:07 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager.update_available_resource {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:27:07 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:27:07 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.002s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:27:07 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:27:07 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Auditing locally available compute resources for user (node: user) {{(pid=75234) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:907}}
Nov 29 03:27:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f6bc212d-85cd-479c-92ae-112c9f0101bc/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:27:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f6bc212d-85cd-479c-92ae-112c9f0101bc/disk --force-share --output=json" returned: 0 in 0.176s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:27:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f6bc212d-85cd-479c-92ae-112c9f0101bc/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:27:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f6bc212d-85cd-479c-92ae-112c9f0101bc/disk --force-share --output=json" returned: 0 in 0.172s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:27:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/65d48c93-b6da-4f51-9d5a-0676dda62a60/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:27:08 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/65d48c93-b6da-4f51-9d5a-0676dda62a60/disk --force-share --output=json" returned: 0 in 0.176s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:27:08 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/65d48c93-b6da-4f51-9d5a-0676dda62a60/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:27:08 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/65d48c93-b6da-4f51-9d5a-0676dda62a60/disk --force-share --output=json" returned: 0 in 0.175s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:27:08 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] skipping disk /dev/sdc (vda) as it is a volume {{(pid=75234) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11213}}
Nov 29 03:27:08 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/d5096ca2-d005-45ac-a487-11a810446859/disk.rescue --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:27:08 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/d5096ca2-d005-45ac-a487-11a810446859/disk.rescue --force-share --output=json" returned: 0 in 0.167s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:27:08 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/d5096ca2-d005-45ac-a487-11a810446859/disk.rescue --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:27:08 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/d5096ca2-d005-45ac-a487-11a810446859/disk.rescue --force-share --output=json" returned: 0 in 0.171s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:27:08 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/13e5da64-9adc-4479-81c7-d329e9b5f08f/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:27:08 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:27:08 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/13e5da64-9adc-4479-81c7-d329e9b5f08f/disk --force-share --output=json" returned: 0 in 0.170s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:27:08 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/13e5da64-9adc-4479-81c7-d329e9b5f08f/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:27:09 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/13e5da64-9adc-4479-81c7-d329e9b5f08f/disk --force-share --output=json" returned: 0 in 0.178s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:27:09 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a4897e0e-d29c-45b6-8f2f-8f5ff55d6a8f/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:27:09 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a4897e0e-d29c-45b6-8f2f-8f5ff55d6a8f/disk --force-share --output=json" returned: 0 in 0.173s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:27:09 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a4897e0e-d29c-45b6-8f2f-8f5ff55d6a8f/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:27:09 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/a4897e0e-d29c-45b6-8f2f-8f5ff55d6a8f/disk --force-share --output=json" returned: 0 in 0.175s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:27:09 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/5d7d744e-1a5f-40ab-b961-4b58c183efe0/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:27:09 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/5d7d744e-1a5f-40ab-b961-4b58c183efe0/disk --force-share --output=json" returned: 0 in 0.177s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:27:09 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/5d7d744e-1a5f-40ab-b961-4b58c183efe0/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:27:09 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/5d7d744e-1a5f-40ab-b961-4b58c183efe0/disk --force-share --output=json" returned: 0 in 0.173s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:27:09 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] skipping disk /dev/sdb (vdb) as it is a volume {{(pid=75234) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11213}}
Nov 29 03:27:09 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/499676de-0e76-45ae-864c-44e55e3d7c7f/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:27:09 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/499676de-0e76-45ae-864c-44e55e3d7c7f/disk --force-share --output=json" returned: 0 in 0.187s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:27:09 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/499676de-0e76-45ae-864c-44e55e3d7c7f/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:27:10 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/499676de-0e76-45ae-864c-44e55e3d7c7f/disk --force-share --output=json" returned: 0 in 0.170s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:27:10 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:27:10 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:27:10 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Hypervisor/Node resource view: name=user free_ram=7899MB free_disk=25.02054214477539GB free_vcpus=5 pci_devices=[{"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}] {{(pid=75234) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1106}}
Nov 29 03:27:10 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:27:10 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:27:10 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance 13e5da64-9adc-4479-81c7-d329e9b5f08f actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:27:10 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance 65d48c93-b6da-4f51-9d5a-0676dda62a60 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:27:10 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance f6bc212d-85cd-479c-92ae-112c9f0101bc actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:27:10 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance 5d7d744e-1a5f-40ab-b961-4b58c183efe0 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:27:10 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance d5096ca2-d005-45ac-a487-11a810446859 actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:27:10 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance a4897e0e-d29c-45b6-8f2f-8f5ff55d6a8f actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:27:10 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance 499676de-0e76-45ae-864c-44e55e3d7c7f actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:27:10 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Total usable vcpus: 12, total allocated vcpus: 7 {{(pid=75234) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1129}}
Nov 29 03:27:10 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Final resource view: name=user phys_ram=16011MB used_ram=4096MB phys_disk=40GB used_disk=6GB total_vcpus=12 used_vcpus=7 pci_stats=[] {{(pid=75234) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1138}}
Nov 29 03:27:10 user nova-compute[75234]: DEBUG nova.compute.provider_tree [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Inventory has not changed in ProviderTree for provider: 4d291220-7c37-4839-a0e3-b23c66502b74 {{(pid=75234) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Nov 29 03:27:11 user nova-compute[75234]: DEBUG nova.scheduler.client.report [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Inventory has not changed for provider 4d291220-7c37-4839-a0e3-b23c66502b74 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=75234) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Nov 29 03:27:11 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Compute_service record updated for user:user {{(pid=75234) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:1067}}
Nov 29 03:27:11 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.506s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:27:11 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-0e3dcae5-c1fb-4d02-9b3f-4d5cb6932574 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] [instance: f6bc212d-85cd-479c-92ae-112c9f0101bc] Calling os-brick to attach iSCSI Volume {{(pid=75234) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Nov 29 03:27:11 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-0e3dcae5-c1fb-4d02-9b3f-4d5cb6932574 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f8a5050c9a0>, 'connection_properties': {'target_iqn': 'iqn.2010-10.org.openstack:5b9c21f2-f2fd-4675-b2b9-09b63a5f2132', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'YGwHprTv', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:176}}
Nov 29 03:27:11 user nova-compute[75234]: WARNING os_brick.initiator.connectors.base [None req-0e3dcae5-c1fb-4d02-9b3f-4d5cb6932574 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Nov 29 03:27:11 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-0e3dcae5-c1fb-4d02-9b3f-4d5cb6932574 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:68}}
Nov 29 03:27:11 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-0e3dcae5-c1fb-4d02-9b3f-4d5cb6932574 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:73}}
Nov 29 03:27:11 user nova-compute[75234]: WARNING os_brick.initiator.connectors.base [None req-0e3dcae5-c1fb-4d02-9b3f-4d5cb6932574 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Nov 29 03:27:11 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-0e3dcae5-c1fb-4d02-9b3f-4d5cb6932574 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:5b9c21f2-f2fd-4675-b2b9-09b63a5f2132" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:68}}
Nov 29 03:27:11 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-0e3dcae5-c1fb-4d02-9b3f-4d5cb6932574 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:5b9c21f2-f2fd-4675-b2b9-09b63a5f2132" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:73}}
Nov 29 03:27:11 user nova-compute[75234]: INFO os_brick.initiator.connectors.iscsi [None req-0e3dcae5-c1fb-4d02-9b3f-4d5cb6932574 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Nov 29 03:27:11 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:5b9c21f2-f2fd-4675-b2b9-09b63a5f2132 -p 172.16.0.220:3260 {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:27:11 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:5b9c21f2-f2fd-4675-b2b9-09b63a5f2132 -p 172.16.0.220:3260" returned: 21 in 0.019s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:27:11 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[3ae3ca9a-4a9e-421c-a865-0f897c80a854]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:27:11 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-0e3dcae5-c1fb-4d02-9b3f-4d5cb6932574 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Nov 29 03:27:11 user nova-compute[75234]:  {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:27:11 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:5b9c21f2-f2fd-4675-b2b9-09b63a5f2132 -p 172.16.0.220:3260 --interface default --op new {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:27:11 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:5b9c21f2-f2fd-4675-b2b9-09b63a5f2132 -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.019s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:27:11 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[c59a4530-4428-4b97-bb26-658584b7a464]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:5b9c21f2-f2fd-4675-b2b9-09b63a5f2132] added\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:27:11 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-0e3dcae5-c1fb-4d02-9b3f-4d5cb6932574 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:5b9c21f2-f2fd-4675-b2b9-09b63a5f2132] added
Nov 29 03:27:11 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:27:11 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:5b9c21f2-f2fd-4675-b2b9-09b63a5f2132 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:27:11 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:5b9c21f2-f2fd-4675-b2b9-09b63a5f2132 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.024s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:27:11 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[04d31fae-9710-4554-a481-fe0e14661840]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:27:11 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-0e3dcae5-c1fb-4d02-9b3f-4d5cb6932574 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:27:11 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:5b9c21f2-f2fd-4675-b2b9-09b63a5f2132 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:27:11 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:5b9c21f2-f2fd-4675-b2b9-09b63a5f2132 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.019s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:27:11 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[7fc3c2eb-4524-4c25-8837-9d1c08cece28]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:27:11 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-0e3dcae5-c1fb-4d02-9b3f-4d5cb6932574 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:27:11 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:5b9c21f2-f2fd-4675-b2b9-09b63a5f2132 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v YGwHprTv {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:27:11 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:5b9c21f2-f2fd-4675-b2b9-09b63a5f2132 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v YGwHprTv" returned: 0 in 0.017s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:27:11 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[2c747d1a-fb06-4c81-b8eb-ef5711f2f570]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:27:11 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-0e3dcae5-c1fb-4d02-9b3f-4d5cb6932574 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'YGwHprTv'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:27:11 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:5b9c21f2-f2fd-4675-b2b9-09b63a5f2132 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:27:11 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:5b9c21f2-f2fd-4675-b2b9-09b63a5f2132 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.018s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:27:11 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[74f30468-53b5-483b-9070-d19e4976d734]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:27:11 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-0e3dcae5-c1fb-4d02-9b3f-4d5cb6932574 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:27:12 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:27:12 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.021s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:27:12 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[d0a3a155-0bd7-4eda-8e4f-0fe98839913b]: (4, ('tcp: [42] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0 (non-flash)\ntcp: [47] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:65acc1f5-8dbf-457c-a978-510436bccf11 (non-flash)\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:27:12 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-0e3dcae5-c1fb-4d02-9b3f-4d5cb6932574 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [42] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0 (non-flash)
Nov 29 03:27:12 user nova-compute[75234]: tcp: [47] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:65acc1f5-8dbf-457c-a978-510436bccf11 (non-flash)
Nov 29 03:27:12 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:27:12 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-0e3dcae5-c1fb-4d02-9b3f-4d5cb6932574 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] iscsi session list stdout=tcp: [42] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0 (non-flash)
Nov 29 03:27:12 user nova-compute[75234]: tcp: [47] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:65acc1f5-8dbf-457c-a978-510436bccf11 (non-flash)
Nov 29 03:27:12 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsi_session /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Nov 29 03:27:12 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:5b9c21f2-f2fd-4675-b2b9-09b63a5f2132 -p 172.16.0.220:3260 --login {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:27:12 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:5b9c21f2-f2fd-4675-b2b9-09b63a5f2132 -p 172.16.0.220:3260 --login" returned: 0 in 0.059s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:27:12 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[76c5c90b-20d5-4c08-bd4b-7ee542e1d5ed]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:5b9c21f2-f2fd-4675-b2b9-09b63a5f2132, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:5b9c21f2-f2fd-4675-b2b9-09b63a5f2132, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:27:12 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-0e3dcae5-c1fb-4d02-9b3f-4d5cb6932574 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:5b9c21f2-f2fd-4675-b2b9-09b63a5f2132, portal: 172.16.0.220,3260]
Nov 29 03:27:12 user nova-compute[75234]: Login to [iface: default, target: iqn.2010-10.org.openstack:5b9c21f2-f2fd-4675-b2b9-09b63a5f2132, portal: 172.16.0.220,3260] successful.
Nov 29 03:27:12 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:27:12 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:5b9c21f2-f2fd-4675-b2b9-09b63a5f2132 -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:27:12 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:5b9c21f2-f2fd-4675-b2b9-09b63a5f2132 -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.019s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:27:12 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[148dcb71-8fa0-49c3-9130-fb66677e71c1]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:27:12 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-0e3dcae5-c1fb-4d02-9b3f-4d5cb6932574 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:27:12 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:27:12 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.020s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:27:12 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[0f8f765d-b74a-4361-b2c6-8e4855148461]: (4, ('tcp: [42] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0 (non-flash)\ntcp: [47] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:65acc1f5-8dbf-457c-a978-510436bccf11 (non-flash)\ntcp: [48] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5b9c21f2-f2fd-4675-b2b9-09b63a5f2132 (non-flash)\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:27:12 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-0e3dcae5-c1fb-4d02-9b3f-4d5cb6932574 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [42] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0 (non-flash)
Nov 29 03:27:12 user nova-compute[75234]: tcp: [47] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:65acc1f5-8dbf-457c-a978-510436bccf11 (non-flash)
Nov 29 03:27:12 user nova-compute[75234]: tcp: [48] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5b9c21f2-f2fd-4675-b2b9-09b63a5f2132 (non-flash)
Nov 29 03:27:12 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:27:12 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-0e3dcae5-c1fb-4d02-9b3f-4d5cb6932574 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] iscsi session list stdout=tcp: [42] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0 (non-flash)
Nov 29 03:27:12 user nova-compute[75234]: tcp: [47] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:65acc1f5-8dbf-457c-a978-510436bccf11 (non-flash)
Nov 29 03:27:12 user nova-compute[75234]: tcp: [48] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5b9c21f2-f2fd-4675-b2b9-09b63a5f2132 (non-flash)
Nov 29 03:27:12 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsi_session /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Nov 29 03:27:12 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-0e3dcae5-c1fb-4d02-9b3f-4d5cb6932574 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:5b9c21f2-f2fd-4675-b2b9-09b63a5f2132" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.275s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:87}}
Nov 29 03:27:12 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-0e3dcae5-c1fb-4d02-9b3f-4d5cb6932574 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Connected to 172.16.0.220:3260 {{(pid=75234) _connect_vol /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:644}}
Nov 29 03:27:12 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-0e3dcae5-c1fb-4d02-9b3f-4d5cb6932574 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] HCTL ('35', '-', '-', 0) found on session 48 with lun 0 {{(pid=75234) get_hctl /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:770}}
Nov 29 03:27:12 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-0e3dcae5-c1fb-4d02-9b3f-4d5cb6932574 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Scanning host 35 c: -, t: -, l: 0) {{(pid=75234) scan_iscsi /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:795}}
Nov 29 03:27:12 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host35/scan {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:27:12 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host35/scan" returned: 0 in 0.017s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:27:12 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[7a851960-f433-4797-9286-d0951649602c]: (4, ('- - 0', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:27:12 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-0e3dcae5-c1fb-4d02-9b3f-4d5cb6932574 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Searching for a device in session 48 and hctl ['35', '*', '*', 0] yield: sdd {{(pid=75234) device_name_by_hctl /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:789}}
Nov 29 03:27:12 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-0e3dcae5-c1fb-4d02-9b3f-4d5cb6932574 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Connected to sdd using {'target_iqn': 'iqn.2010-10.org.openstack:5b9c21f2-f2fd-4675-b2b9-09b63a5f2132', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'YGwHprTv', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False} {{(pid=75234) _connect_vol /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:672}}
Nov 29 03:27:13 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:27:13 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Starting heal instance info cache {{(pid=75234) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9922}}
Nov 29 03:27:13 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Rebuilding the list of instances to heal {{(pid=75234) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9926}}
Nov 29 03:27:13 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Acquiring lock "refresh_cache-13e5da64-9adc-4479-81c7-d329e9b5f08f" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:27:13 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Acquired lock "refresh_cache-13e5da64-9adc-4479-81c7-d329e9b5f08f" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:27:13 user nova-compute[75234]: DEBUG nova.network.neutron [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] [instance: 13e5da64-9adc-4479-81c7-d329e9b5f08f] Forcefully refreshing network info cache for instance {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2003}}
Nov 29 03:27:13 user nova-compute[75234]: DEBUG nova.objects.instance [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lazy-loading 'info_cache' on Instance uuid 13e5da64-9adc-4479-81c7-d329e9b5f08f {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:27:13 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-0e3dcae5-c1fb-4d02-9b3f-4d5cb6932574 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.307s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:87}}
Nov 29 03:27:13 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-0e3dcae5-c1fb-4d02-9b3f-4d5cb6932574 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] <== connect_volume: return (1308ms) {'type': 'block', 'scsi_wwn': '23236326534353833', 'path': '/dev/sdd'} {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:203}}
Nov 29 03:27:13 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-0e3dcae5-c1fb-4d02-9b3f-4d5cb6932574 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] [instance: f6bc212d-85cd-479c-92ae-112c9f0101bc] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '23236326534353833', 'path': '/dev/sdd'} {{(pid=75234) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Nov 29 03:27:13 user nova-compute[75234]: DEBUG nova.objects.instance [None req-0e3dcae5-c1fb-4d02-9b3f-4d5cb6932574 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Lazy-loading 'flavor' on Instance uuid f6bc212d-85cd-479c-92ae-112c9f0101bc {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:27:13 user nova-compute[75234]: DEBUG nova.virt.libvirt.guest [None req-0e3dcae5-c1fb-4d02-9b3f-4d5cb6932574 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] attach device xml: <disk type="block" device="disk">
Nov 29 03:27:13 user nova-compute[75234]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Nov 29 03:27:13 user nova-compute[75234]:   <source dev="/dev/sdd"/>
Nov 29 03:27:13 user nova-compute[75234]:   <target dev="vdb" bus="virtio"/>
Nov 29 03:27:13 user nova-compute[75234]:   <serial>5b9c21f2-f2fd-4675-b2b9-09b63a5f2132</serial>
Nov 29 03:27:13 user nova-compute[75234]: </disk>
Nov 29 03:27:13 user nova-compute[75234]:  {{(pid=75234) attach_device /opt/stack/nova/nova/virt/libvirt/guest.py:339}}
Nov 29 03:27:13 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-0e3dcae5-c1fb-4d02-9b3f-4d5cb6932574 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] No BDM found with device name vda, not building metadata. {{(pid=75234) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12089}}
Nov 29 03:27:13 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-0e3dcae5-c1fb-4d02-9b3f-4d5cb6932574 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] No BDM found with device name vdb, not building metadata. {{(pid=75234) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12089}}
Nov 29 03:27:13 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-0e3dcae5-c1fb-4d02-9b3f-4d5cb6932574 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] No VIF found with MAC fa:16:3e:4e:8e:1d, not building metadata {{(pid=75234) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Nov 29 03:27:13 user nova-compute[75234]: DEBUG nova.network.neutron [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] [instance: 13e5da64-9adc-4479-81c7-d329e9b5f08f] Updating instance_info_cache with network_info: [{"id": "2abf9e25-d1d9-4893-899e-912d41b7c38f", "address": "fa:16:3e:59:79:35", "network": {"id": "2b3c3140-f3b6-45da-8202-bb4d7f0dd130", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-2043420778-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "f8a7884867c944e1ba1024270e4902f3", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2abf9e25-d1", "ovs_interfaceid": "2abf9e25-d1d9-4893-899e-912d41b7c38f", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:27:13 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-0e3dcae5-c1fb-4d02-9b3f-4d5cb6932574 tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Lock "f6bc212d-85cd-479c-92ae-112c9f0101bc" "released" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: held 21.938s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:27:13 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Releasing lock "refresh_cache-13e5da64-9adc-4479-81c7-d329e9b5f08f" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:27:13 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] [instance: 13e5da64-9adc-4479-81c7-d329e9b5f08f] Updated the network info_cache for instance {{(pid=75234) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9993}}
Nov 29 03:27:13 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:27:13 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:27:13 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:27:13 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5002 ms, sending inactivity probe {{(pid=75234) run /opt/stack/data/venv/lib/python3.10/site-packages/ovs/reconnect.py:117}}
Nov 29 03:27:13 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=75234) _transition /opt/stack/data/venv/lib/python3.10/site-packages/ovs/reconnect.py:519}}
Nov 29 03:27:13 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=75234) _transition /opt/stack/data/venv/lib/python3.10/site-packages/ovs/reconnect.py:519}}
Nov 29 03:27:13 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:27:14 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:27:15 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-2ca8a86d-b63b-4468-9add-3dc4730658eb tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Acquiring lock "f6bc212d-85cd-479c-92ae-112c9f0101bc" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:27:15 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-2ca8a86d-b63b-4468-9add-3dc4730658eb tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Lock "f6bc212d-85cd-479c-92ae-112c9f0101bc" acquired by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:27:15 user nova-compute[75234]: INFO nova.compute.manager [None req-2ca8a86d-b63b-4468-9add-3dc4730658eb tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] [instance: f6bc212d-85cd-479c-92ae-112c9f0101bc] Detaching volume 5b9c21f2-f2fd-4675-b2b9-09b63a5f2132
Nov 29 03:27:16 user nova-compute[75234]: INFO nova.virt.block_device [None req-2ca8a86d-b63b-4468-9add-3dc4730658eb tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] [instance: f6bc212d-85cd-479c-92ae-112c9f0101bc] Attempting to driver detach volume 5b9c21f2-f2fd-4675-b2b9-09b63a5f2132 from mountpoint /dev/vdb
Nov 29 03:27:16 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-2ca8a86d-b63b-4468-9add-3dc4730658eb tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Attempting to detach device vdb from instance f6bc212d-85cd-479c-92ae-112c9f0101bc from the persistent domain config. {{(pid=75234) _detach_from_persistent /opt/stack/nova/nova/virt/libvirt/driver.py:2487}}
Nov 29 03:27:16 user nova-compute[75234]: DEBUG nova.virt.libvirt.guest [None req-2ca8a86d-b63b-4468-9add-3dc4730658eb tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] detach device xml: <disk type="block" device="disk">
Nov 29 03:27:16 user nova-compute[75234]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Nov 29 03:27:16 user nova-compute[75234]:   <source dev="/dev/sdd"/>
Nov 29 03:27:16 user nova-compute[75234]:   <target dev="vdb" bus="virtio"/>
Nov 29 03:27:16 user nova-compute[75234]:   <serial>5b9c21f2-f2fd-4675-b2b9-09b63a5f2132</serial>
Nov 29 03:27:16 user nova-compute[75234]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x07" function="0x0"/>
Nov 29 03:27:16 user nova-compute[75234]: </disk>
Nov 29 03:27:16 user nova-compute[75234]:  {{(pid=75234) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Nov 29 03:27:16 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-2ca8a86d-b63b-4468-9add-3dc4730658eb tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Successfully detached device vdb from instance f6bc212d-85cd-479c-92ae-112c9f0101bc from the persistent domain config.
Nov 29 03:27:16 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-2ca8a86d-b63b-4468-9add-3dc4730658eb tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] (1/8): Attempting to detach device vdb with device alias virtio-disk1 from instance f6bc212d-85cd-479c-92ae-112c9f0101bc from the live domain config. {{(pid=75234) _detach_from_live_with_retry /opt/stack/nova/nova/virt/libvirt/driver.py:2523}}
Nov 29 03:27:16 user nova-compute[75234]: DEBUG nova.virt.libvirt.guest [None req-2ca8a86d-b63b-4468-9add-3dc4730658eb tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] detach device xml: <disk type="block" device="disk">
Nov 29 03:27:16 user nova-compute[75234]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Nov 29 03:27:16 user nova-compute[75234]:   <source dev="/dev/sdd"/>
Nov 29 03:27:16 user nova-compute[75234]:   <target dev="vdb" bus="virtio"/>
Nov 29 03:27:16 user nova-compute[75234]:   <serial>5b9c21f2-f2fd-4675-b2b9-09b63a5f2132</serial>
Nov 29 03:27:16 user nova-compute[75234]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x07" function="0x0"/>
Nov 29 03:27:16 user nova-compute[75234]: </disk>
Nov 29 03:27:16 user nova-compute[75234]:  {{(pid=75234) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Nov 29 03:27:16 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-2ca8a86d-b63b-4468-9add-3dc4730658eb tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Start waiting for the detach event from libvirt for device vdb with device alias virtio-disk1 for instance f6bc212d-85cd-479c-92ae-112c9f0101bc {{(pid=75234) _detach_from_live_and_wait_for_event /opt/stack/nova/nova/virt/libvirt/driver.py:2599}}
Nov 29 03:27:16 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:27:17 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:27:18 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._cleanup_expired_console_auth_tokens {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:27:18 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:27:23 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:27:28 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:27:29 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._run_pending_deletes {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:27:29 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Cleaning up deleted instances {{(pid=75234) _run_pending_deletes /opt/stack/nova/nova/compute/manager.py:11209}}
Nov 29 03:27:29 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] There are 0 instances to clean {{(pid=75234) _run_pending_deletes /opt/stack/nova/nova/compute/manager.py:11218}}
Nov 29 03:27:33 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:27:36 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-2ca8a86d-b63b-4468-9add-3dc4730658eb tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Waiting for libvirt event about the detach of device vdb with device alias virtio-disk1 from instance f6bc212d-85cd-479c-92ae-112c9f0101bc is timed out.
Nov 29 03:27:36 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-2ca8a86d-b63b-4468-9add-3dc4730658eb tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Successfully detached device vdb from instance f6bc212d-85cd-479c-92ae-112c9f0101bc from the live domain config.
Nov 29 03:27:36 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-2ca8a86d-b63b-4468-9add-3dc4730658eb tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] [instance: f6bc212d-85cd-479c-92ae-112c9f0101bc] calling os-brick to detach iSCSI Volume {{(pid=75234) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Nov 29 03:27:36 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-2ca8a86d-b63b-4468-9add-3dc4730658eb tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f8a5050c9a0>, {'target_iqn': 'iqn.2010-10.org.openstack:5b9c21f2-f2fd-4675-b2b9-09b63a5f2132', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'YGwHprTv', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdd'}, None), 'kwargs': False}}" {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:176}}
Nov 29 03:27:36 user nova-compute[75234]: WARNING os_brick.initiator.connectors.base [None req-2ca8a86d-b63b-4468-9add-3dc4730658eb tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Nov 29 03:27:36 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-2ca8a86d-b63b-4468-9add-3dc4730658eb tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:68}}
Nov 29 03:27:36 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-2ca8a86d-b63b-4468-9add-3dc4730658eb tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:73}}
Nov 29 03:27:36 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:27:36 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.022s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:27:36 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[ffd6afa4-e1b4-4b6a-8d73-870373fd0ac4]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:65acc1f5-8dbf-457c-a978-510436bccf11\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:5b9c21f2-f2fd-4675-b2b9-09b63a5f2132\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:27:36 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-2ca8a86d-b63b-4468-9add-3dc4730658eb tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS:
Nov 29 03:27:36 user nova-compute[75234]: No targets found.
Nov 29 03:27:36 user nova-compute[75234]: iSNS:
Nov 29 03:27:36 user nova-compute[75234]: No targets found.
Nov 29 03:27:36 user nova-compute[75234]: STATIC:
Nov 29 03:27:36 user nova-compute[75234]: Target: iqn.2010-10.org.openstack:65acc1f5-8dbf-457c-a978-510436bccf11
Nov 29 03:27:36 user nova-compute[75234]:         Portal: 172.16.0.220:3260,-1
Nov 29 03:27:36 user nova-compute[75234]:                 Iface Name: default
Nov 29 03:27:36 user nova-compute[75234]: Target: iqn.2010-10.org.openstack:5b9c21f2-f2fd-4675-b2b9-09b63a5f2132
Nov 29 03:27:36 user nova-compute[75234]:         Portal: 172.16.0.220:3260,-1
Nov 29 03:27:36 user nova-compute[75234]:                 Iface Name: default
Nov 29 03:27:36 user nova-compute[75234]: Target: iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0
Nov 29 03:27:36 user nova-compute[75234]:         Portal: 172.16.0.220:3260,-1
Nov 29 03:27:36 user nova-compute[75234]:                 Iface Name: default
Nov 29 03:27:36 user nova-compute[75234]: FIRMWARE:
Nov 29 03:27:36 user nova-compute[75234]: No targets found.
Nov 29 03:27:36 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:27:36 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-2ca8a86d-b63b-4468-9add-3dc4730658eb tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Regex to get portals from discoverydb: ^SENDTARGETS:
Nov 29 03:27:36 user nova-compute[75234]: .*?^DiscoveryAddress: 172.16.0.220,3260.*?
Nov 29 03:27:36 user nova-compute[75234]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=75234) _get_discoverydb_portals /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:382}}
Nov 29 03:27:36 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-2ca8a86d-b63b-4468-9add-3dc4730658eb tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:5b9c21f2-f2fd-4675-b2b9-09b63a5f2132', 0)] {{(pid=75234) _get_connection_devices /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:830}}
Nov 29 03:27:36 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:27:36 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.020s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:27:36 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[323833da-4368-44fa-a66d-ed9c4ce9c855]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:5b9c21f2-f2fd-4675-b2b9-09b63a5f2132\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:65acc1f5-8dbf-457c-a978-510436bccf11\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:27:36 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:27:36 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.023s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:27:36 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[06e4a441-ffb6-4347-bf80-5fab43e8ba7c]: (4, ('tcp: [42] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0 (non-flash)\ntcp: [47] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:65acc1f5-8dbf-457c-a978-510436bccf11 (non-flash)\ntcp: [48] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5b9c21f2-f2fd-4675-b2b9-09b63a5f2132 (non-flash)\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:27:36 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-2ca8a86d-b63b-4468-9add-3dc4730658eb tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [42] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0 (non-flash)
Nov 29 03:27:36 user nova-compute[75234]: tcp: [47] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:65acc1f5-8dbf-457c-a978-510436bccf11 (non-flash)
Nov 29 03:27:36 user nova-compute[75234]: tcp: [48] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5b9c21f2-f2fd-4675-b2b9-09b63a5f2132 (non-flash)
Nov 29 03:27:36 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:27:36 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-2ca8a86d-b63b-4468-9add-3dc4730658eb tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] iscsi session list stdout=tcp: [42] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0 (non-flash)
Nov 29 03:27:36 user nova-compute[75234]: tcp: [47] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:65acc1f5-8dbf-457c-a978-510436bccf11 (non-flash)
Nov 29 03:27:36 user nova-compute[75234]: tcp: [48] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:5b9c21f2-f2fd-4675-b2b9-09b63a5f2132 (non-flash)
Nov 29 03:27:36 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsi_session /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Nov 29 03:27:36 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-2ca8a86d-b63b-4468-9add-3dc4730658eb tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7f8a2c60c700>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:5b9c21f2-f2fd-4675-b2b9-09b63a5f2132'): ({'sdd'}, set())}) {{(pid=75234) _get_connection_devices /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:863}}
Nov 29 03:27:36 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-2ca8a86d-b63b-4468-9add-3dc4730658eb tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Removing single pathed devices sdd {{(pid=75234) remove_connection /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:374}}
Nov 29 03:27:36 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:27:36 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.026s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:27:36 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[030627fd-9dba-4ac8-96c3-d242affb5074]: (4, ('path checker states:\nup                  3\n\npaths: 0\nbusy: False\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:27:36 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdd {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:27:36 user nova-compute[75234]: INFO nova.compute.manager [None req-be0242cb-f449-431a-ad3b-ef3810076d4a tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] [instance: d5096ca2-d005-45ac-a487-11a810446859] Unrescuing
Nov 29 03:27:36 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-be0242cb-f449-431a-ad3b-ef3810076d4a tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Acquiring lock "refresh_cache-d5096ca2-d005-45ac-a487-11a810446859" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:27:36 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-be0242cb-f449-431a-ad3b-ef3810076d4a tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Acquired lock "refresh_cache-d5096ca2-d005-45ac-a487-11a810446859" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:27:36 user nova-compute[75234]: DEBUG nova.network.neutron [None req-be0242cb-f449-431a-ad3b-ef3810076d4a tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] [instance: d5096ca2-d005-45ac-a487-11a810446859] Building network info cache for instance {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2009}}
Nov 29 03:27:36 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdd" returned: 0 in 0.023s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:27:36 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[3ecc27be-cac8-4c6a-9f21-f5238b2e924c]: (4, ('ok\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:27:36 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-2ca8a86d-b63b-4468-9add-3dc4730658eb tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Flushing IO for device /dev/sdd {{(pid=75234) flush_device_io /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:434}}
Nov 29 03:27:36 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdd {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:27:36 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdd" returned: 0 in 0.019s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:27:36 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[f1d4e378-99de-40d3-a181-6ef3b746b7d3]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:27:36 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-2ca8a86d-b63b-4468-9add-3dc4730658eb tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Remove SCSI device /dev/sdd with /sys/block/sdd/device/delete {{(pid=75234) remove_scsi_device /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:148}}
Nov 29 03:27:36 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdd/device/delete {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:27:36 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdd/device/delete" returned: 0 in 0.046s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:27:36 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[5309ab7e-32d3-40e3-9c4d-3a8a1982d184]: (4, ('1', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:27:36 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-2ca8a86d-b63b-4468-9add-3dc4730658eb tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Checking to see if SCSI volumes sdd have been removed. {{(pid=75234) wait_for_volumes_removal /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:156}}
Nov 29 03:27:36 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-2ca8a86d-b63b-4468-9add-3dc4730658eb tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] SCSI volumes sdd have been removed. {{(pid=75234) wait_for_volumes_removal /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:166}}
Nov 29 03:27:36 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-2ca8a86d-b63b-4468-9add-3dc4730658eb tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:5b9c21f2-f2fd-4675-b2b9-09b63a5f2132')] {{(pid=75234) _disconnect_connection /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Nov 29 03:27:36 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:5b9c21f2-f2fd-4675-b2b9-09b63a5f2132 -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:27:36 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:5b9c21f2-f2fd-4675-b2b9-09b63a5f2132 -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.070s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:27:36 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[2565f508-e1ac-4de4-933b-c13017b2f728]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:27:36 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-2ca8a86d-b63b-4468-9add-3dc4730658eb tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:27:36 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:5b9c21f2-f2fd-4675-b2b9-09b63a5f2132 -p 172.16.0.220:3260 --logout {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:27:36 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:5b9c21f2-f2fd-4675-b2b9-09b63a5f2132 -p 172.16.0.220:3260 --logout" returned: 0 in 0.066s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:27:36 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[64fbccf8-9963-4873-8c58-a98c4d35720c]: (4, ('Logging out of session [sid: 48, target: iqn.2010-10.org.openstack:5b9c21f2-f2fd-4675-b2b9-09b63a5f2132, portal: 172.16.0.220,3260]\nLogout of [sid: 48, target: iqn.2010-10.org.openstack:5b9c21f2-f2fd-4675-b2b9-09b63a5f2132, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:27:36 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-2ca8a86d-b63b-4468-9add-3dc4730658eb tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 48, target: iqn.2010-10.org.openstack:5b9c21f2-f2fd-4675-b2b9-09b63a5f2132, portal: 172.16.0.220,3260]
Nov 29 03:27:36 user nova-compute[75234]: Logout of [sid: 48, target: iqn.2010-10.org.openstack:5b9c21f2-f2fd-4675-b2b9-09b63a5f2132, portal: 172.16.0.220,3260] successful.
Nov 29 03:27:36 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:27:36 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:5b9c21f2-f2fd-4675-b2b9-09b63a5f2132 -p 172.16.0.220:3260 --op delete {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:27:36 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:5b9c21f2-f2fd-4675-b2b9-09b63a5f2132 -p 172.16.0.220:3260 --op delete" returned: 0 in 0.029s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:27:36 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[9f67772d-8f39-47d2-a769-9c0da2ee74c9]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:27:36 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-2ca8a86d-b63b-4468-9add-3dc4730658eb tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:27:36 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-2ca8a86d-b63b-4468-9add-3dc4730658eb tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.399s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:87}}
Nov 29 03:27:36 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-2ca8a86d-b63b-4468-9add-3dc4730658eb tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] <== disconnect_volume: return (400ms) None {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:203}}
Nov 29 03:27:36 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-2ca8a86d-b63b-4468-9add-3dc4730658eb tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] [instance: f6bc212d-85cd-479c-92ae-112c9f0101bc] Disconnected iSCSI Volume {{(pid=75234) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:79}}
Nov 29 03:27:37 user nova-compute[75234]: DEBUG nova.network.neutron [None req-be0242cb-f449-431a-ad3b-ef3810076d4a tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] [instance: d5096ca2-d005-45ac-a487-11a810446859] Updating instance_info_cache with network_info: [{"id": "9266d021-6e83-4bfa-8151-66c5182e61a0", "address": "fa:16:3e:ec:fb:67", "network": {"id": "4d086e08-7798-4cfe-9365-86be6f0a52d2", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-305624008-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "db00bcd0c82c44f1bda739e9857f6e84", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap9266d021-6e", "ovs_interfaceid": "9266d021-6e83-4bfa-8151-66c5182e61a0", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:27:37 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-be0242cb-f449-431a-ad3b-ef3810076d4a tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Releasing lock "refresh_cache-d5096ca2-d005-45ac-a487-11a810446859" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:27:37 user nova-compute[75234]: DEBUG nova.objects.instance [None req-be0242cb-f449-431a-ad3b-ef3810076d4a tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Lazy-loading 'flavor' on Instance uuid d5096ca2-d005-45ac-a487-11a810446859 {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:27:37 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:27:37 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:27:37 user nova-compute[75234]: DEBUG nova.compute.manager [req-aebdf2e2-6072-4553-80d0-14d5cd699559 req-5434ca27-bcd9-4991-bf3e-8c143e0cf71a service nova] [instance: d5096ca2-d005-45ac-a487-11a810446859] Received event network-vif-unplugged-9266d021-6e83-4bfa-8151-66c5182e61a0 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:27:37 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-aebdf2e2-6072-4553-80d0-14d5cd699559 req-5434ca27-bcd9-4991-bf3e-8c143e0cf71a service nova] Acquiring lock "d5096ca2-d005-45ac-a487-11a810446859-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:27:37 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-aebdf2e2-6072-4553-80d0-14d5cd699559 req-5434ca27-bcd9-4991-bf3e-8c143e0cf71a service nova] Lock "d5096ca2-d005-45ac-a487-11a810446859-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:27:37 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-aebdf2e2-6072-4553-80d0-14d5cd699559 req-5434ca27-bcd9-4991-bf3e-8c143e0cf71a service nova] Lock "d5096ca2-d005-45ac-a487-11a810446859-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:27:37 user nova-compute[75234]: DEBUG nova.compute.manager [req-aebdf2e2-6072-4553-80d0-14d5cd699559 req-5434ca27-bcd9-4991-bf3e-8c143e0cf71a service nova] [instance: d5096ca2-d005-45ac-a487-11a810446859] No waiting events found dispatching network-vif-unplugged-9266d021-6e83-4bfa-8151-66c5182e61a0 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:27:37 user nova-compute[75234]: WARNING nova.compute.manager [req-aebdf2e2-6072-4553-80d0-14d5cd699559 req-5434ca27-bcd9-4991-bf3e-8c143e0cf71a service nova] [instance: d5096ca2-d005-45ac-a487-11a810446859] Received unexpected event network-vif-unplugged-9266d021-6e83-4bfa-8151-66c5182e61a0 for instance with vm_state rescued and task_state unrescuing.
Nov 29 03:27:37 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:27:37 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:27:37 user nova-compute[75234]: DEBUG nova.objects.instance [None req-2ca8a86d-b63b-4468-9add-3dc4730658eb tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Lazy-loading 'flavor' on Instance uuid f6bc212d-85cd-479c-92ae-112c9f0101bc {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:27:37 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-2ca8a86d-b63b-4468-9add-3dc4730658eb tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Lock "f6bc212d-85cd-479c-92ae-112c9f0101bc" "released" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: held 21.841s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:27:37 user nova-compute[75234]: INFO nova.virt.libvirt.driver [-] [instance: d5096ca2-d005-45ac-a487-11a810446859] Instance destroyed successfully.
Nov 29 03:27:38 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:27:39 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:27:39 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:27:39 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:27:39 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:27:39 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:27:39 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:27:39 user nova-compute[75234]: DEBUG nova.compute.manager [req-92177304-fccd-48ac-8918-d83c65a053ba req-aa09ffa8-7f0c-4531-99e4-c1698f65843a service nova] [instance: d5096ca2-d005-45ac-a487-11a810446859] Received event network-vif-plugged-9266d021-6e83-4bfa-8151-66c5182e61a0 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:27:39 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-92177304-fccd-48ac-8918-d83c65a053ba req-aa09ffa8-7f0c-4531-99e4-c1698f65843a service nova] Acquiring lock "d5096ca2-d005-45ac-a487-11a810446859-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:27:39 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-92177304-fccd-48ac-8918-d83c65a053ba req-aa09ffa8-7f0c-4531-99e4-c1698f65843a service nova] Lock "d5096ca2-d005-45ac-a487-11a810446859-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:27:39 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-92177304-fccd-48ac-8918-d83c65a053ba req-aa09ffa8-7f0c-4531-99e4-c1698f65843a service nova] Lock "d5096ca2-d005-45ac-a487-11a810446859-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:27:39 user nova-compute[75234]: DEBUG nova.compute.manager [req-92177304-fccd-48ac-8918-d83c65a053ba req-aa09ffa8-7f0c-4531-99e4-c1698f65843a service nova] [instance: d5096ca2-d005-45ac-a487-11a810446859] No waiting events found dispatching network-vif-plugged-9266d021-6e83-4bfa-8151-66c5182e61a0 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:27:39 user nova-compute[75234]: WARNING nova.compute.manager [req-92177304-fccd-48ac-8918-d83c65a053ba req-aa09ffa8-7f0c-4531-99e4-c1698f65843a service nova] [instance: d5096ca2-d005-45ac-a487-11a810446859] Received unexpected event network-vif-plugged-9266d021-6e83-4bfa-8151-66c5182e61a0 for instance with vm_state rescued and task_state unrescuing.
Nov 29 03:27:39 user nova-compute[75234]: DEBUG nova.compute.manager [req-92177304-fccd-48ac-8918-d83c65a053ba req-aa09ffa8-7f0c-4531-99e4-c1698f65843a service nova] [instance: d5096ca2-d005-45ac-a487-11a810446859] Received event network-vif-plugged-9266d021-6e83-4bfa-8151-66c5182e61a0 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:27:39 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-92177304-fccd-48ac-8918-d83c65a053ba req-aa09ffa8-7f0c-4531-99e4-c1698f65843a service nova] Acquiring lock "d5096ca2-d005-45ac-a487-11a810446859-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:27:39 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-92177304-fccd-48ac-8918-d83c65a053ba req-aa09ffa8-7f0c-4531-99e4-c1698f65843a service nova] Lock "d5096ca2-d005-45ac-a487-11a810446859-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:27:39 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-92177304-fccd-48ac-8918-d83c65a053ba req-aa09ffa8-7f0c-4531-99e4-c1698f65843a service nova] Lock "d5096ca2-d005-45ac-a487-11a810446859-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:27:39 user nova-compute[75234]: DEBUG nova.compute.manager [req-92177304-fccd-48ac-8918-d83c65a053ba req-aa09ffa8-7f0c-4531-99e4-c1698f65843a service nova] [instance: d5096ca2-d005-45ac-a487-11a810446859] No waiting events found dispatching network-vif-plugged-9266d021-6e83-4bfa-8151-66c5182e61a0 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:27:39 user nova-compute[75234]: WARNING nova.compute.manager [req-92177304-fccd-48ac-8918-d83c65a053ba req-aa09ffa8-7f0c-4531-99e4-c1698f65843a service nova] [instance: d5096ca2-d005-45ac-a487-11a810446859] Received unexpected event network-vif-plugged-9266d021-6e83-4bfa-8151-66c5182e61a0 for instance with vm_state rescued and task_state unrescuing.
Nov 29 03:27:41 user nova-compute[75234]: DEBUG nova.compute.manager [None req-be0242cb-f449-431a-ad3b-ef3810076d4a tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] [instance: d5096ca2-d005-45ac-a487-11a810446859] Checking state {{(pid=75234) _get_power_state /opt/stack/nova/nova/compute/manager.py:1782}}
Nov 29 03:27:41 user nova-compute[75234]: DEBUG nova.compute.manager [req-b098bc6e-804f-4a4e-8b9e-649c5103d2d3 req-f84bf03a-0331-401d-8153-3e1aa3f58146 service nova] [instance: d5096ca2-d005-45ac-a487-11a810446859] Received event network-vif-plugged-9266d021-6e83-4bfa-8151-66c5182e61a0 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:27:41 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-b098bc6e-804f-4a4e-8b9e-649c5103d2d3 req-f84bf03a-0331-401d-8153-3e1aa3f58146 service nova] Acquiring lock "d5096ca2-d005-45ac-a487-11a810446859-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:27:41 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-b098bc6e-804f-4a4e-8b9e-649c5103d2d3 req-f84bf03a-0331-401d-8153-3e1aa3f58146 service nova] Lock "d5096ca2-d005-45ac-a487-11a810446859-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:27:41 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-b098bc6e-804f-4a4e-8b9e-649c5103d2d3 req-f84bf03a-0331-401d-8153-3e1aa3f58146 service nova] Lock "d5096ca2-d005-45ac-a487-11a810446859-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:27:41 user nova-compute[75234]: DEBUG nova.compute.manager [req-b098bc6e-804f-4a4e-8b9e-649c5103d2d3 req-f84bf03a-0331-401d-8153-3e1aa3f58146 service nova] [instance: d5096ca2-d005-45ac-a487-11a810446859] No waiting events found dispatching network-vif-plugged-9266d021-6e83-4bfa-8151-66c5182e61a0 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:27:41 user nova-compute[75234]: WARNING nova.compute.manager [req-b098bc6e-804f-4a4e-8b9e-649c5103d2d3 req-f84bf03a-0331-401d-8153-3e1aa3f58146 service nova] [instance: d5096ca2-d005-45ac-a487-11a810446859] Received unexpected event network-vif-plugged-9266d021-6e83-4bfa-8151-66c5182e61a0 for instance with vm_state active and task_state None.
Nov 29 03:27:42 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:27:42 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:27:43 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:27:48 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:27:53 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-53df7e7e-9c29-40dd-9de0-683ba9933ae6 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Acquiring lock "13e5da64-9adc-4479-81c7-d329e9b5f08f" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:27:53 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-53df7e7e-9c29-40dd-9de0-683ba9933ae6 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Lock "13e5da64-9adc-4479-81c7-d329e9b5f08f" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:27:53 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-53df7e7e-9c29-40dd-9de0-683ba9933ae6 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Acquiring lock "13e5da64-9adc-4479-81c7-d329e9b5f08f-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:27:53 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-53df7e7e-9c29-40dd-9de0-683ba9933ae6 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Lock "13e5da64-9adc-4479-81c7-d329e9b5f08f-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:27:53 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-53df7e7e-9c29-40dd-9de0-683ba9933ae6 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Lock "13e5da64-9adc-4479-81c7-d329e9b5f08f-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:27:53 user nova-compute[75234]: INFO nova.compute.manager [None req-53df7e7e-9c29-40dd-9de0-683ba9933ae6 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] [instance: 13e5da64-9adc-4479-81c7-d329e9b5f08f] Terminating instance
Nov 29 03:27:53 user nova-compute[75234]: DEBUG nova.compute.manager [None req-53df7e7e-9c29-40dd-9de0-683ba9933ae6 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] [instance: 13e5da64-9adc-4479-81c7-d329e9b5f08f] Start destroying the instance on the hypervisor. {{(pid=75234) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3140}}
Nov 29 03:27:53 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:27:53 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:27:53 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:27:53 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:27:53 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:27:53 user nova-compute[75234]: DEBUG nova.compute.manager [req-a2a683cd-9ea0-49d2-8e10-6e8773fac49f req-56316c70-6bbd-49f3-bc9d-354bab86a874 service nova] [instance: 13e5da64-9adc-4479-81c7-d329e9b5f08f] Received event network-vif-unplugged-2abf9e25-d1d9-4893-899e-912d41b7c38f {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:27:53 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-a2a683cd-9ea0-49d2-8e10-6e8773fac49f req-56316c70-6bbd-49f3-bc9d-354bab86a874 service nova] Acquiring lock "13e5da64-9adc-4479-81c7-d329e9b5f08f-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:27:53 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-a2a683cd-9ea0-49d2-8e10-6e8773fac49f req-56316c70-6bbd-49f3-bc9d-354bab86a874 service nova] Lock "13e5da64-9adc-4479-81c7-d329e9b5f08f-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:27:53 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-a2a683cd-9ea0-49d2-8e10-6e8773fac49f req-56316c70-6bbd-49f3-bc9d-354bab86a874 service nova] Lock "13e5da64-9adc-4479-81c7-d329e9b5f08f-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:27:53 user nova-compute[75234]: DEBUG nova.compute.manager [req-a2a683cd-9ea0-49d2-8e10-6e8773fac49f req-56316c70-6bbd-49f3-bc9d-354bab86a874 service nova] [instance: 13e5da64-9adc-4479-81c7-d329e9b5f08f] No waiting events found dispatching network-vif-unplugged-2abf9e25-d1d9-4893-899e-912d41b7c38f {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:27:53 user nova-compute[75234]: DEBUG nova.compute.manager [req-a2a683cd-9ea0-49d2-8e10-6e8773fac49f req-56316c70-6bbd-49f3-bc9d-354bab86a874 service nova] [instance: 13e5da64-9adc-4479-81c7-d329e9b5f08f] Received event network-vif-unplugged-2abf9e25-d1d9-4893-899e-912d41b7c38f for instance with task_state deleting. {{(pid=75234) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10890}}
Nov 29 03:27:53 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:27:54 user nova-compute[75234]: INFO nova.virt.libvirt.driver [-] [instance: 13e5da64-9adc-4479-81c7-d329e9b5f08f] Instance destroyed successfully.
Nov 29 03:27:54 user nova-compute[75234]: DEBUG nova.objects.instance [None req-53df7e7e-9c29-40dd-9de0-683ba9933ae6 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Lazy-loading 'resources' on Instance uuid 13e5da64-9adc-4479-81c7-d329e9b5f08f {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:27:54 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-53df7e7e-9c29-40dd-9de0-683ba9933ae6 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-11-29T03:10:25Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-ServerRescueNegativeTestJSON-server-715053465',display_name='tempest-ServerRescueNegativeTestJSON-server-715053465',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-serverrescuenegativetestjson-server-715053465',id=18,image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data=None,key_name=None,keypairs=<?>,launch_index=0,launched_at=2023-11-29T03:15:18Z,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='f8a7884867c944e1ba1024270e4902f3',ramdisk_id='',reservation_id='r-e0thim3u',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-ServerRescueNegativeTestJSON-1182706601',owner_user_name='tempest-ServerRescueNegativeTestJSON-1182706601-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-11-29T03:17:09Z,user_data=None,user_id='6792e866677645c598ddb7d2c1723a1a',uuid=13e5da64-9adc-4479-81c7-d329e9b5f08f,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "2abf9e25-d1d9-4893-899e-912d41b7c38f", "address": "fa:16:3e:59:79:35", "network": {"id": "2b3c3140-f3b6-45da-8202-bb4d7f0dd130", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-2043420778-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "f8a7884867c944e1ba1024270e4902f3", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2abf9e25-d1", "ovs_interfaceid": "2abf9e25-d1d9-4893-899e-912d41b7c38f", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Nov 29 03:27:54 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-53df7e7e-9c29-40dd-9de0-683ba9933ae6 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Converting VIF {"id": "2abf9e25-d1d9-4893-899e-912d41b7c38f", "address": "fa:16:3e:59:79:35", "network": {"id": "2b3c3140-f3b6-45da-8202-bb4d7f0dd130", "bridge": "br-int", "label": "tempest-ServerRescueNegativeTestJSON-2043420778-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "f8a7884867c944e1ba1024270e4902f3", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap2abf9e25-d1", "ovs_interfaceid": "2abf9e25-d1d9-4893-899e-912d41b7c38f", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:27:54 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-53df7e7e-9c29-40dd-9de0-683ba9933ae6 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:59:79:35,bridge_name='br-int',has_traffic_filtering=True,id=2abf9e25-d1d9-4893-899e-912d41b7c38f,network=Network(2b3c3140-f3b6-45da-8202-bb4d7f0dd130),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap2abf9e25-d1') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:27:54 user nova-compute[75234]: DEBUG os_vif [None req-53df7e7e-9c29-40dd-9de0-683ba9933ae6 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Unplugging vif VIFOpenVSwitch(active=True,address=fa:16:3e:59:79:35,bridge_name='br-int',has_traffic_filtering=True,id=2abf9e25-d1d9-4893-899e-912d41b7c38f,network=Network(2b3c3140-f3b6-45da-8202-bb4d7f0dd130),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap2abf9e25-d1') {{(pid=75234) unplug /opt/stack/data/venv/lib/python3.10/site-packages/os_vif/__init__.py:109}}
Nov 29 03:27:54 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:27:54 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tap2abf9e25-d1, bridge=br-int, if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:27:54 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:27:54 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:27:54 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:27:54 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbDestroyCommand(_result=None, table=QoS, record=268ca68b-d061-4990-a3d9-a154adf42b8b) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:27:54 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:27:54 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:27:54 user nova-compute[75234]: INFO os_vif [None req-53df7e7e-9c29-40dd-9de0-683ba9933ae6 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Successfully unplugged vif VIFOpenVSwitch(active=True,address=fa:16:3e:59:79:35,bridge_name='br-int',has_traffic_filtering=True,id=2abf9e25-d1d9-4893-899e-912d41b7c38f,network=Network(2b3c3140-f3b6-45da-8202-bb4d7f0dd130),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap2abf9e25-d1')
Nov 29 03:27:54 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-53df7e7e-9c29-40dd-9de0-683ba9933ae6 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] [instance: 13e5da64-9adc-4479-81c7-d329e9b5f08f] Deleting instance files /opt/stack/data/nova/instances/13e5da64-9adc-4479-81c7-d329e9b5f08f_del
Nov 29 03:27:54 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-53df7e7e-9c29-40dd-9de0-683ba9933ae6 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] [instance: 13e5da64-9adc-4479-81c7-d329e9b5f08f] Deletion of /opt/stack/data/nova/instances/13e5da64-9adc-4479-81c7-d329e9b5f08f_del complete
Nov 29 03:27:54 user nova-compute[75234]: INFO nova.compute.manager [None req-53df7e7e-9c29-40dd-9de0-683ba9933ae6 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] [instance: 13e5da64-9adc-4479-81c7-d329e9b5f08f] Took 0.88 seconds to destroy the instance on the hypervisor.
Nov 29 03:27:54 user nova-compute[75234]: DEBUG oslo.service.loopingcall [None req-53df7e7e-9c29-40dd-9de0-683ba9933ae6 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=75234) func /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/loopingcall.py:435}}
Nov 29 03:27:54 user nova-compute[75234]: DEBUG nova.compute.manager [-] [instance: 13e5da64-9adc-4479-81c7-d329e9b5f08f] Deallocating network for instance {{(pid=75234) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2279}}
Nov 29 03:27:54 user nova-compute[75234]: DEBUG nova.network.neutron [-] [instance: 13e5da64-9adc-4479-81c7-d329e9b5f08f] deallocate_for_instance() {{(pid=75234) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1802}}
Nov 29 03:27:54 user nova-compute[75234]: DEBUG nova.network.neutron [-] [instance: 13e5da64-9adc-4479-81c7-d329e9b5f08f] Updating instance_info_cache with network_info: [] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:27:54 user nova-compute[75234]: INFO nova.compute.manager [-] [instance: 13e5da64-9adc-4479-81c7-d329e9b5f08f] Took 0.42 seconds to deallocate network for instance.
Nov 29 03:27:54 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-53df7e7e-9c29-40dd-9de0-683ba9933ae6 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:27:54 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-53df7e7e-9c29-40dd-9de0-683ba9933ae6 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:27:54 user nova-compute[75234]: DEBUG nova.scheduler.client.report [None req-53df7e7e-9c29-40dd-9de0-683ba9933ae6 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Refreshing inventories for resource provider 4d291220-7c37-4839-a0e3-b23c66502b74 {{(pid=75234) _refresh_associations /opt/stack/nova/nova/scheduler/client/report.py:804}}
Nov 29 03:27:54 user nova-compute[75234]: DEBUG nova.scheduler.client.report [None req-53df7e7e-9c29-40dd-9de0-683ba9933ae6 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Updating ProviderTree inventory for provider 4d291220-7c37-4839-a0e3-b23c66502b74 from _refresh_and_get_inventory using data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=75234) _refresh_and_get_inventory /opt/stack/nova/nova/scheduler/client/report.py:768}}
Nov 29 03:27:54 user nova-compute[75234]: DEBUG nova.compute.provider_tree [None req-53df7e7e-9c29-40dd-9de0-683ba9933ae6 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Updating inventory in ProviderTree for provider 4d291220-7c37-4839-a0e3-b23c66502b74 with inventory: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=75234) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:176}}
Nov 29 03:27:54 user nova-compute[75234]: DEBUG nova.scheduler.client.report [None req-53df7e7e-9c29-40dd-9de0-683ba9933ae6 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Refreshing aggregate associations for resource provider 4d291220-7c37-4839-a0e3-b23c66502b74, aggregates: None {{(pid=75234) _refresh_associations /opt/stack/nova/nova/scheduler/client/report.py:813}}
Nov 29 03:27:54 user nova-compute[75234]: DEBUG nova.scheduler.client.report [None req-53df7e7e-9c29-40dd-9de0-683ba9933ae6 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Refreshing trait associations for resource provider 4d291220-7c37-4839-a0e3-b23c66502b74, traits: COMPUTE_IMAGE_TYPE_AMI,COMPUTE_GRAPHICS_MODEL_CIRRUS,COMPUTE_NET_VIF_MODEL_VMXNET3,COMPUTE_STORAGE_BUS_VIRTIO,COMPUTE_GRAPHICS_MODEL_NONE,COMPUTE_IMAGE_TYPE_ISO,COMPUTE_NET_VIF_MODEL_RTL8139,HW_CPU_X86_SSE,COMPUTE_IMAGE_TYPE_QCOW2,COMPUTE_SOCKET_PCI_NUMA_AFFINITY,COMPUTE_GRAPHICS_MODEL_BOCHS,COMPUTE_VOLUME_ATTACH_WITH_TAG,COMPUTE_ACCELERATORS,HW_CPU_X86_SSE2,COMPUTE_STORAGE_BUS_SCSI,COMPUTE_GRAPHICS_MODEL_VGA,COMPUTE_NET_ATTACH_INTERFACE_WITH_TAG,COMPUTE_SECURITY_UEFI_SECURE_BOOT,HW_CPU_X86_SSE41,COMPUTE_NET_VIF_MODEL_E1000E,COMPUTE_TRUSTED_CERTS,COMPUTE_STORAGE_BUS_FDC,COMPUTE_VIOMMU_MODEL_INTEL,COMPUTE_GRAPHICS_MODEL_VIRTIO,COMPUTE_VIOMMU_MODEL_AUTO,COMPUTE_IMAGE_TYPE_AKI,COMPUTE_NET_VIF_MODEL_PCNET,COMPUTE_GRAPHICS_MODEL_QXL,COMPUTE_NET_VIF_MODEL_VIRTIO,COMPUTE_GRAPHICS_MODEL_VMVGA,COMPUTE_VOLUME_MULTI_ATTACH,COMPUTE_STORAGE_BUS_SATA,HW_CPU_X86_MMX,COMPUTE_NET_VIF_MODEL_E1000,COMPUTE_IMAGE_TYPE_ARI,COMPUTE_STORAGE_BUS_IDE,COMPUTE_NODE,HW_CPU_X86_SSSE3,COMPUTE_RESCUE_BFV,COMPUTE_NET_ATTACH_INTERFACE,COMPUTE_STORAGE_BUS_USB,COMPUTE_NET_VIF_MODEL_SPAPR_VLAN,COMPUTE_IMAGE_TYPE_RAW,COMPUTE_NET_VIF_MODEL_NE2K_PCI,COMPUTE_VOLUME_EXTEND,COMPUTE_DEVICE_TAGGING,HW_CPU_X86_SSE42 {{(pid=75234) _refresh_associations /opt/stack/nova/nova/scheduler/client/report.py:825}}
Nov 29 03:27:55 user nova-compute[75234]: DEBUG nova.compute.provider_tree [None req-53df7e7e-9c29-40dd-9de0-683ba9933ae6 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Inventory has not changed in ProviderTree for provider: 4d291220-7c37-4839-a0e3-b23c66502b74 {{(pid=75234) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Nov 29 03:27:55 user nova-compute[75234]: DEBUG nova.scheduler.client.report [None req-53df7e7e-9c29-40dd-9de0-683ba9933ae6 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Inventory has not changed for provider 4d291220-7c37-4839-a0e3-b23c66502b74 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=75234) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Nov 29 03:27:55 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-53df7e7e-9c29-40dd-9de0-683ba9933ae6 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.356s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:27:55 user nova-compute[75234]: INFO nova.scheduler.client.report [None req-53df7e7e-9c29-40dd-9de0-683ba9933ae6 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Deleted allocations for instance 13e5da64-9adc-4479-81c7-d329e9b5f08f
Nov 29 03:27:55 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-53df7e7e-9c29-40dd-9de0-683ba9933ae6 tempest-ServerRescueNegativeTestJSON-1182706601 tempest-ServerRescueNegativeTestJSON-1182706601-project-member] Lock "13e5da64-9adc-4479-81c7-d329e9b5f08f" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 1.856s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:27:55 user nova-compute[75234]: DEBUG nova.compute.manager [req-85f94b65-f858-43dc-b821-34e913fcf274 req-88167391-f24c-44fa-8646-0198858c40d8 service nova] [instance: 13e5da64-9adc-4479-81c7-d329e9b5f08f] Received event network-vif-plugged-2abf9e25-d1d9-4893-899e-912d41b7c38f {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:27:55 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-85f94b65-f858-43dc-b821-34e913fcf274 req-88167391-f24c-44fa-8646-0198858c40d8 service nova] Acquiring lock "13e5da64-9adc-4479-81c7-d329e9b5f08f-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:27:55 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-85f94b65-f858-43dc-b821-34e913fcf274 req-88167391-f24c-44fa-8646-0198858c40d8 service nova] Lock "13e5da64-9adc-4479-81c7-d329e9b5f08f-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:27:55 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-85f94b65-f858-43dc-b821-34e913fcf274 req-88167391-f24c-44fa-8646-0198858c40d8 service nova] Lock "13e5da64-9adc-4479-81c7-d329e9b5f08f-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:27:55 user nova-compute[75234]: DEBUG nova.compute.manager [req-85f94b65-f858-43dc-b821-34e913fcf274 req-88167391-f24c-44fa-8646-0198858c40d8 service nova] [instance: 13e5da64-9adc-4479-81c7-d329e9b5f08f] No waiting events found dispatching network-vif-plugged-2abf9e25-d1d9-4893-899e-912d41b7c38f {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:27:55 user nova-compute[75234]: WARNING nova.compute.manager [req-85f94b65-f858-43dc-b821-34e913fcf274 req-88167391-f24c-44fa-8646-0198858c40d8 service nova] [instance: 13e5da64-9adc-4479-81c7-d329e9b5f08f] Received unexpected event network-vif-plugged-2abf9e25-d1d9-4893-899e-912d41b7c38f for instance with vm_state deleted and task_state None.
Nov 29 03:27:55 user nova-compute[75234]: DEBUG nova.compute.manager [req-85f94b65-f858-43dc-b821-34e913fcf274 req-88167391-f24c-44fa-8646-0198858c40d8 service nova] [instance: 13e5da64-9adc-4479-81c7-d329e9b5f08f] Received event network-vif-deleted-2abf9e25-d1d9-4893-899e-912d41b7c38f {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:27:57 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:27:57 user nova-compute[75234]: DEBUG nova.compute.manager [req-fd2a6866-150b-4125-a0b4-5e23384fd9b9 req-b2f104d0-aafc-46e8-b82d-bff7933bf230 service nova] [instance: 5d7d744e-1a5f-40ab-b961-4b58c183efe0] Received event network-changed-fa65b32f-b627-45fa-8d6d-d99012ca1262 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:27:57 user nova-compute[75234]: DEBUG nova.compute.manager [req-fd2a6866-150b-4125-a0b4-5e23384fd9b9 req-b2f104d0-aafc-46e8-b82d-bff7933bf230 service nova] [instance: 5d7d744e-1a5f-40ab-b961-4b58c183efe0] Refreshing instance network info cache due to event network-changed-fa65b32f-b627-45fa-8d6d-d99012ca1262. {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11117}}
Nov 29 03:27:57 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-fd2a6866-150b-4125-a0b4-5e23384fd9b9 req-b2f104d0-aafc-46e8-b82d-bff7933bf230 service nova] Acquiring lock "refresh_cache-5d7d744e-1a5f-40ab-b961-4b58c183efe0" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:27:57 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-fd2a6866-150b-4125-a0b4-5e23384fd9b9 req-b2f104d0-aafc-46e8-b82d-bff7933bf230 service nova] Acquired lock "refresh_cache-5d7d744e-1a5f-40ab-b961-4b58c183efe0" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:27:57 user nova-compute[75234]: DEBUG nova.network.neutron [req-fd2a6866-150b-4125-a0b4-5e23384fd9b9 req-b2f104d0-aafc-46e8-b82d-bff7933bf230 service nova] [instance: 5d7d744e-1a5f-40ab-b961-4b58c183efe0] Refreshing network info cache for port fa65b32f-b627-45fa-8d6d-d99012ca1262 {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2006}}
Nov 29 03:27:58 user nova-compute[75234]: DEBUG nova.compute.manager [req-4a4480e7-b917-485f-92e7-cdd5d8ab7ebc req-0a475129-bd0e-42f9-9660-468d8bcb8a14 service nova] [instance: 5d7d744e-1a5f-40ab-b961-4b58c183efe0] Received event network-changed-fa65b32f-b627-45fa-8d6d-d99012ca1262 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:27:58 user nova-compute[75234]: DEBUG nova.compute.manager [req-4a4480e7-b917-485f-92e7-cdd5d8ab7ebc req-0a475129-bd0e-42f9-9660-468d8bcb8a14 service nova] [instance: 5d7d744e-1a5f-40ab-b961-4b58c183efe0] Refreshing instance network info cache due to event network-changed-fa65b32f-b627-45fa-8d6d-d99012ca1262. {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11117}}
Nov 29 03:27:58 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-4a4480e7-b917-485f-92e7-cdd5d8ab7ebc req-0a475129-bd0e-42f9-9660-468d8bcb8a14 service nova] Acquiring lock "refresh_cache-5d7d744e-1a5f-40ab-b961-4b58c183efe0" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:27:58 user nova-compute[75234]: DEBUG nova.network.neutron [req-fd2a6866-150b-4125-a0b4-5e23384fd9b9 req-b2f104d0-aafc-46e8-b82d-bff7933bf230 service nova] [instance: 5d7d744e-1a5f-40ab-b961-4b58c183efe0] Updated VIF entry in instance network info cache for port fa65b32f-b627-45fa-8d6d-d99012ca1262. {{(pid=75234) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3481}}
Nov 29 03:27:58 user nova-compute[75234]: DEBUG nova.network.neutron [req-fd2a6866-150b-4125-a0b4-5e23384fd9b9 req-b2f104d0-aafc-46e8-b82d-bff7933bf230 service nova] [instance: 5d7d744e-1a5f-40ab-b961-4b58c183efe0] Updating instance_info_cache with network_info: [{"id": "fa65b32f-b627-45fa-8d6d-d99012ca1262", "address": "fa:16:3e:2f:fa:59", "network": {"id": "6787e0a8-2980-4a2e-bf2a-583af955241d", "bridge": "br-int", "label": "tempest-TestMinimumBasicScenario-1273600998-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.111", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "74f132d6666c47b8b43869df6fee6de3", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapfa65b32f-b6", "ovs_interfaceid": "fa65b32f-b627-45fa-8d6d-d99012ca1262", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:27:58 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-fd2a6866-150b-4125-a0b4-5e23384fd9b9 req-b2f104d0-aafc-46e8-b82d-bff7933bf230 service nova] Releasing lock "refresh_cache-5d7d744e-1a5f-40ab-b961-4b58c183efe0" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:27:58 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-4a4480e7-b917-485f-92e7-cdd5d8ab7ebc req-0a475129-bd0e-42f9-9660-468d8bcb8a14 service nova] Acquired lock "refresh_cache-5d7d744e-1a5f-40ab-b961-4b58c183efe0" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:27:58 user nova-compute[75234]: DEBUG nova.network.neutron [req-4a4480e7-b917-485f-92e7-cdd5d8ab7ebc req-0a475129-bd0e-42f9-9660-468d8bcb8a14 service nova] [instance: 5d7d744e-1a5f-40ab-b961-4b58c183efe0] Refreshing network info cache for port fa65b32f-b627-45fa-8d6d-d99012ca1262 {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2006}}
Nov 29 03:27:58 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-a993e9c4-6849-45ce-ba40-fe3da865f35c tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Acquiring lock "5d7d744e-1a5f-40ab-b961-4b58c183efe0" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:27:58 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-a993e9c4-6849-45ce-ba40-fe3da865f35c tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Lock "5d7d744e-1a5f-40ab-b961-4b58c183efe0" acquired by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:27:58 user nova-compute[75234]: INFO nova.compute.manager [None req-a993e9c4-6849-45ce-ba40-fe3da865f35c tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] [instance: 5d7d744e-1a5f-40ab-b961-4b58c183efe0] Detaching volume 65acc1f5-8dbf-457c-a978-510436bccf11
Nov 29 03:27:58 user nova-compute[75234]: INFO nova.virt.block_device [None req-a993e9c4-6849-45ce-ba40-fe3da865f35c tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] [instance: 5d7d744e-1a5f-40ab-b961-4b58c183efe0] Attempting to driver detach volume 65acc1f5-8dbf-457c-a978-510436bccf11 from mountpoint /dev/vdb
Nov 29 03:27:58 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-a993e9c4-6849-45ce-ba40-fe3da865f35c tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Attempting to detach device vdb from instance 5d7d744e-1a5f-40ab-b961-4b58c183efe0 from the persistent domain config. {{(pid=75234) _detach_from_persistent /opt/stack/nova/nova/virt/libvirt/driver.py:2487}}
Nov 29 03:27:58 user nova-compute[75234]: DEBUG nova.virt.libvirt.guest [None req-a993e9c4-6849-45ce-ba40-fe3da865f35c tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] detach device xml: <disk type="block" device="disk">
Nov 29 03:27:58 user nova-compute[75234]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Nov 29 03:27:58 user nova-compute[75234]:   <source dev="/dev/sdb"/>
Nov 29 03:27:58 user nova-compute[75234]:   <target dev="vdb" bus="virtio"/>
Nov 29 03:27:58 user nova-compute[75234]:   <serial>65acc1f5-8dbf-457c-a978-510436bccf11</serial>
Nov 29 03:27:58 user nova-compute[75234]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x05" function="0x0"/>
Nov 29 03:27:58 user nova-compute[75234]: </disk>
Nov 29 03:27:58 user nova-compute[75234]:  {{(pid=75234) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Nov 29 03:27:58 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-a993e9c4-6849-45ce-ba40-fe3da865f35c tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Successfully detached device vdb from instance 5d7d744e-1a5f-40ab-b961-4b58c183efe0 from the persistent domain config.
Nov 29 03:27:58 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-a993e9c4-6849-45ce-ba40-fe3da865f35c tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] (1/8): Attempting to detach device vdb with device alias virtio-disk1 from instance 5d7d744e-1a5f-40ab-b961-4b58c183efe0 from the live domain config. {{(pid=75234) _detach_from_live_with_retry /opt/stack/nova/nova/virt/libvirt/driver.py:2523}}
Nov 29 03:27:58 user nova-compute[75234]: DEBUG nova.virt.libvirt.guest [None req-a993e9c4-6849-45ce-ba40-fe3da865f35c tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] detach device xml: <disk type="block" device="disk">
Nov 29 03:27:58 user nova-compute[75234]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Nov 29 03:27:58 user nova-compute[75234]:   <source dev="/dev/sdb"/>
Nov 29 03:27:58 user nova-compute[75234]:   <target dev="vdb" bus="virtio"/>
Nov 29 03:27:58 user nova-compute[75234]:   <serial>65acc1f5-8dbf-457c-a978-510436bccf11</serial>
Nov 29 03:27:58 user nova-compute[75234]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x05" function="0x0"/>
Nov 29 03:27:58 user nova-compute[75234]: </disk>
Nov 29 03:27:58 user nova-compute[75234]:  {{(pid=75234) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Nov 29 03:27:58 user nova-compute[75234]: DEBUG nova.network.neutron [req-4a4480e7-b917-485f-92e7-cdd5d8ab7ebc req-0a475129-bd0e-42f9-9660-468d8bcb8a14 service nova] [instance: 5d7d744e-1a5f-40ab-b961-4b58c183efe0] Updated VIF entry in instance network info cache for port fa65b32f-b627-45fa-8d6d-d99012ca1262. {{(pid=75234) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3481}}
Nov 29 03:27:58 user nova-compute[75234]: DEBUG nova.network.neutron [req-4a4480e7-b917-485f-92e7-cdd5d8ab7ebc req-0a475129-bd0e-42f9-9660-468d8bcb8a14 service nova] [instance: 5d7d744e-1a5f-40ab-b961-4b58c183efe0] Updating instance_info_cache with network_info: [{"id": "fa65b32f-b627-45fa-8d6d-d99012ca1262", "address": "fa:16:3e:2f:fa:59", "network": {"id": "6787e0a8-2980-4a2e-bf2a-583af955241d", "bridge": "br-int", "label": "tempest-TestMinimumBasicScenario-1273600998-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "74f132d6666c47b8b43869df6fee6de3", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapfa65b32f-b6", "ovs_interfaceid": "fa65b32f-b627-45fa-8d6d-d99012ca1262", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:27:58 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-4a4480e7-b917-485f-92e7-cdd5d8ab7ebc req-0a475129-bd0e-42f9-9660-468d8bcb8a14 service nova] Releasing lock "refresh_cache-5d7d744e-1a5f-40ab-b961-4b58c183efe0" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:27:58 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-a993e9c4-6849-45ce-ba40-fe3da865f35c tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Start waiting for the detach event from libvirt for device vdb with device alias virtio-disk1 for instance 5d7d744e-1a5f-40ab-b961-4b58c183efe0 {{(pid=75234) _detach_from_live_and_wait_for_event /opt/stack/nova/nova/virt/libvirt/driver.py:2599}}
Nov 29 03:27:59 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:28:04 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:28:04 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:28:04 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5001 ms, sending inactivity probe {{(pid=75234) run /opt/stack/data/venv/lib/python3.10/site-packages/ovs/reconnect.py:117}}
Nov 29 03:28:04 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=75234) _transition /opt/stack/data/venv/lib/python3.10/site-packages/ovs/reconnect.py:519}}
Nov 29 03:28:04 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=75234) _transition /opt/stack/data/venv/lib/python3.10/site-packages/ovs/reconnect.py:519}}
Nov 29 03:28:04 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:28:05 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:28:06 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-303cf5f0-9f3d-49c8-9da0-75d0bf6fc272 tempest-VolumesNegativeTest-814389221 tempest-VolumesNegativeTest-814389221-project-member] Acquiring lock "a4897e0e-d29c-45b6-8f2f-8f5ff55d6a8f" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:28:06 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-303cf5f0-9f3d-49c8-9da0-75d0bf6fc272 tempest-VolumesNegativeTest-814389221 tempest-VolumesNegativeTest-814389221-project-member] Lock "a4897e0e-d29c-45b6-8f2f-8f5ff55d6a8f" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:28:06 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-303cf5f0-9f3d-49c8-9da0-75d0bf6fc272 tempest-VolumesNegativeTest-814389221 tempest-VolumesNegativeTest-814389221-project-member] Acquiring lock "a4897e0e-d29c-45b6-8f2f-8f5ff55d6a8f-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:28:06 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-303cf5f0-9f3d-49c8-9da0-75d0bf6fc272 tempest-VolumesNegativeTest-814389221 tempest-VolumesNegativeTest-814389221-project-member] Lock "a4897e0e-d29c-45b6-8f2f-8f5ff55d6a8f-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:28:06 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-303cf5f0-9f3d-49c8-9da0-75d0bf6fc272 tempest-VolumesNegativeTest-814389221 tempest-VolumesNegativeTest-814389221-project-member] Lock "a4897e0e-d29c-45b6-8f2f-8f5ff55d6a8f-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:28:06 user nova-compute[75234]: INFO nova.compute.manager [None req-303cf5f0-9f3d-49c8-9da0-75d0bf6fc272 tempest-VolumesNegativeTest-814389221 tempest-VolumesNegativeTest-814389221-project-member] [instance: a4897e0e-d29c-45b6-8f2f-8f5ff55d6a8f] Terminating instance
Nov 29 03:28:06 user nova-compute[75234]: DEBUG nova.compute.manager [None req-303cf5f0-9f3d-49c8-9da0-75d0bf6fc272 tempest-VolumesNegativeTest-814389221 tempest-VolumesNegativeTest-814389221-project-member] [instance: a4897e0e-d29c-45b6-8f2f-8f5ff55d6a8f] Start destroying the instance on the hypervisor. {{(pid=75234) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3140}}
Nov 29 03:28:06 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:28:06 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:28:06 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:28:06 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:28:06 user nova-compute[75234]: DEBUG nova.compute.manager [req-f62805f6-730f-41ee-af9a-1a41adf3dd97 req-d7b9d2f2-767f-4afc-bf9b-11f05021957c service nova] [instance: a4897e0e-d29c-45b6-8f2f-8f5ff55d6a8f] Received event network-vif-unplugged-55748abc-3e82-40c3-a7ba-b16327b28e9e {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:28:06 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-f62805f6-730f-41ee-af9a-1a41adf3dd97 req-d7b9d2f2-767f-4afc-bf9b-11f05021957c service nova] Acquiring lock "a4897e0e-d29c-45b6-8f2f-8f5ff55d6a8f-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:28:06 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-f62805f6-730f-41ee-af9a-1a41adf3dd97 req-d7b9d2f2-767f-4afc-bf9b-11f05021957c service nova] Lock "a4897e0e-d29c-45b6-8f2f-8f5ff55d6a8f-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:28:06 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-f62805f6-730f-41ee-af9a-1a41adf3dd97 req-d7b9d2f2-767f-4afc-bf9b-11f05021957c service nova] Lock "a4897e0e-d29c-45b6-8f2f-8f5ff55d6a8f-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.003s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:28:06 user nova-compute[75234]: DEBUG nova.compute.manager [req-f62805f6-730f-41ee-af9a-1a41adf3dd97 req-d7b9d2f2-767f-4afc-bf9b-11f05021957c service nova] [instance: a4897e0e-d29c-45b6-8f2f-8f5ff55d6a8f] No waiting events found dispatching network-vif-unplugged-55748abc-3e82-40c3-a7ba-b16327b28e9e {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:28:06 user nova-compute[75234]: DEBUG nova.compute.manager [req-f62805f6-730f-41ee-af9a-1a41adf3dd97 req-d7b9d2f2-767f-4afc-bf9b-11f05021957c service nova] [instance: a4897e0e-d29c-45b6-8f2f-8f5ff55d6a8f] Received event network-vif-unplugged-55748abc-3e82-40c3-a7ba-b16327b28e9e for instance with task_state deleting. {{(pid=75234) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10890}}
Nov 29 03:28:07 user nova-compute[75234]: INFO nova.virt.libvirt.driver [-] [instance: a4897e0e-d29c-45b6-8f2f-8f5ff55d6a8f] Instance destroyed successfully.
Nov 29 03:28:07 user nova-compute[75234]: DEBUG nova.objects.instance [None req-303cf5f0-9f3d-49c8-9da0-75d0bf6fc272 tempest-VolumesNegativeTest-814389221 tempest-VolumesNegativeTest-814389221-project-member] Lazy-loading 'resources' on Instance uuid a4897e0e-d29c-45b6-8f2f-8f5ff55d6a8f {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:28:07 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-303cf5f0-9f3d-49c8-9da0-75d0bf6fc272 tempest-VolumesNegativeTest-814389221 tempest-VolumesNegativeTest-814389221-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-11-29T03:26:15Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-VolumesNegativeTest-instance-1349352086',display_name='tempest-VolumesNegativeTest-instance-1349352086',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-volumesnegativetest-instance-1349352086',id=41,image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data=None,key_name=None,keypairs=<?>,launch_index=0,launched_at=2023-11-29T03:26:23Z,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='5da24ca4ca0d43d2849bf605ee87fa64',ramdisk_id='',reservation_id='r-7kuexd4k',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-VolumesNegativeTest-814389221',owner_user_name='tempest-VolumesNegativeTest-814389221-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-11-29T03:26:23Z,user_data=None,user_id='3a791e338de34f48b8512360b2422c11',uuid=a4897e0e-d29c-45b6-8f2f-8f5ff55d6a8f,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "55748abc-3e82-40c3-a7ba-b16327b28e9e", "address": "fa:16:3e:e8:7d:56", "network": {"id": "3b52ddb6-adad-4b84-99e0-6fb38ab8f76b", "bridge": "br-int", "label": "shared", "subnets": [{"cidr": "192.168.233.0/24", "dns": [], "gateway": {"address": "192.168.233.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "192.168.233.161", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "192.168.233.1"}}], "meta": {"injected": false, "tenant_id": "b18e380921c748b78517b6f1211cd0f1", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap55748abc-3e", "ovs_interfaceid": "55748abc-3e82-40c3-a7ba-b16327b28e9e", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Nov 29 03:28:07 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-303cf5f0-9f3d-49c8-9da0-75d0bf6fc272 tempest-VolumesNegativeTest-814389221 tempest-VolumesNegativeTest-814389221-project-member] Converting VIF {"id": "55748abc-3e82-40c3-a7ba-b16327b28e9e", "address": "fa:16:3e:e8:7d:56", "network": {"id": "3b52ddb6-adad-4b84-99e0-6fb38ab8f76b", "bridge": "br-int", "label": "shared", "subnets": [{"cidr": "192.168.233.0/24", "dns": [], "gateway": {"address": "192.168.233.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "192.168.233.161", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "192.168.233.1"}}], "meta": {"injected": false, "tenant_id": "b18e380921c748b78517b6f1211cd0f1", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap55748abc-3e", "ovs_interfaceid": "55748abc-3e82-40c3-a7ba-b16327b28e9e", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:28:07 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-303cf5f0-9f3d-49c8-9da0-75d0bf6fc272 tempest-VolumesNegativeTest-814389221 tempest-VolumesNegativeTest-814389221-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:e8:7d:56,bridge_name='br-int',has_traffic_filtering=True,id=55748abc-3e82-40c3-a7ba-b16327b28e9e,network=Network(3b52ddb6-adad-4b84-99e0-6fb38ab8f76b),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap55748abc-3e') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:28:07 user nova-compute[75234]: DEBUG os_vif [None req-303cf5f0-9f3d-49c8-9da0-75d0bf6fc272 tempest-VolumesNegativeTest-814389221 tempest-VolumesNegativeTest-814389221-project-member] Unplugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:e8:7d:56,bridge_name='br-int',has_traffic_filtering=True,id=55748abc-3e82-40c3-a7ba-b16327b28e9e,network=Network(3b52ddb6-adad-4b84-99e0-6fb38ab8f76b),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap55748abc-3e') {{(pid=75234) unplug /opt/stack/data/venv/lib/python3.10/site-packages/os_vif/__init__.py:109}}
Nov 29 03:28:07 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:28:07 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tap55748abc-3e, bridge=br-int, if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:28:07 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:28:07 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:28:07 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:28:07 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbDestroyCommand(_result=None, table=QoS, record=6afe4725-9987-4656-b61b-2d7fecde2ac9) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:28:07 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:28:07 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:28:07 user nova-compute[75234]: INFO os_vif [None req-303cf5f0-9f3d-49c8-9da0-75d0bf6fc272 tempest-VolumesNegativeTest-814389221 tempest-VolumesNegativeTest-814389221-project-member] Successfully unplugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:e8:7d:56,bridge_name='br-int',has_traffic_filtering=True,id=55748abc-3e82-40c3-a7ba-b16327b28e9e,network=Network(3b52ddb6-adad-4b84-99e0-6fb38ab8f76b),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap55748abc-3e')
Nov 29 03:28:07 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-303cf5f0-9f3d-49c8-9da0-75d0bf6fc272 tempest-VolumesNegativeTest-814389221 tempest-VolumesNegativeTest-814389221-project-member] [instance: a4897e0e-d29c-45b6-8f2f-8f5ff55d6a8f] Deleting instance files /opt/stack/data/nova/instances/a4897e0e-d29c-45b6-8f2f-8f5ff55d6a8f_del
Nov 29 03:28:07 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-303cf5f0-9f3d-49c8-9da0-75d0bf6fc272 tempest-VolumesNegativeTest-814389221 tempest-VolumesNegativeTest-814389221-project-member] [instance: a4897e0e-d29c-45b6-8f2f-8f5ff55d6a8f] Deletion of /opt/stack/data/nova/instances/a4897e0e-d29c-45b6-8f2f-8f5ff55d6a8f_del complete
Nov 29 03:28:07 user nova-compute[75234]: INFO nova.compute.manager [None req-303cf5f0-9f3d-49c8-9da0-75d0bf6fc272 tempest-VolumesNegativeTest-814389221 tempest-VolumesNegativeTest-814389221-project-member] [instance: a4897e0e-d29c-45b6-8f2f-8f5ff55d6a8f] Took 0.67 seconds to destroy the instance on the hypervisor.
Nov 29 03:28:07 user nova-compute[75234]: DEBUG oslo.service.loopingcall [None req-303cf5f0-9f3d-49c8-9da0-75d0bf6fc272 tempest-VolumesNegativeTest-814389221 tempest-VolumesNegativeTest-814389221-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=75234) func /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/loopingcall.py:435}}
Nov 29 03:28:07 user nova-compute[75234]: DEBUG nova.compute.manager [-] [instance: a4897e0e-d29c-45b6-8f2f-8f5ff55d6a8f] Deallocating network for instance {{(pid=75234) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2279}}
Nov 29 03:28:07 user nova-compute[75234]: DEBUG nova.network.neutron [-] [instance: a4897e0e-d29c-45b6-8f2f-8f5ff55d6a8f] deallocate_for_instance() {{(pid=75234) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1802}}
Nov 29 03:28:07 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:28:07 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=75234) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10541}}
Nov 29 03:28:07 user nova-compute[75234]: DEBUG nova.network.neutron [-] [instance: a4897e0e-d29c-45b6-8f2f-8f5ff55d6a8f] Updating instance_info_cache with network_info: [] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:28:07 user nova-compute[75234]: INFO nova.compute.manager [-] [instance: a4897e0e-d29c-45b6-8f2f-8f5ff55d6a8f] Took 0.49 seconds to deallocate network for instance.
Nov 29 03:28:07 user nova-compute[75234]: DEBUG nova.compute.manager [req-1b1b1110-8efa-4218-b7d4-332d8fc4cbcf req-31cec122-677a-44ff-8493-7f2730308e8b service nova] [instance: a4897e0e-d29c-45b6-8f2f-8f5ff55d6a8f] Received event network-vif-deleted-55748abc-3e82-40c3-a7ba-b16327b28e9e {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:28:07 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-303cf5f0-9f3d-49c8-9da0-75d0bf6fc272 tempest-VolumesNegativeTest-814389221 tempest-VolumesNegativeTest-814389221-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:28:07 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-303cf5f0-9f3d-49c8-9da0-75d0bf6fc272 tempest-VolumesNegativeTest-814389221 tempest-VolumesNegativeTest-814389221-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:28:07 user nova-compute[75234]: DEBUG nova.compute.provider_tree [None req-303cf5f0-9f3d-49c8-9da0-75d0bf6fc272 tempest-VolumesNegativeTest-814389221 tempest-VolumesNegativeTest-814389221-project-member] Inventory has not changed in ProviderTree for provider: 4d291220-7c37-4839-a0e3-b23c66502b74 {{(pid=75234) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Nov 29 03:28:07 user nova-compute[75234]: DEBUG nova.scheduler.client.report [None req-303cf5f0-9f3d-49c8-9da0-75d0bf6fc272 tempest-VolumesNegativeTest-814389221 tempest-VolumesNegativeTest-814389221-project-member] Inventory has not changed for provider 4d291220-7c37-4839-a0e3-b23c66502b74 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=75234) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Nov 29 03:28:07 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-303cf5f0-9f3d-49c8-9da0-75d0bf6fc272 tempest-VolumesNegativeTest-814389221 tempest-VolumesNegativeTest-814389221-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.247s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:28:08 user nova-compute[75234]: INFO nova.scheduler.client.report [None req-303cf5f0-9f3d-49c8-9da0-75d0bf6fc272 tempest-VolumesNegativeTest-814389221 tempest-VolumesNegativeTest-814389221-project-member] Deleted allocations for instance a4897e0e-d29c-45b6-8f2f-8f5ff55d6a8f
Nov 29 03:28:08 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-303cf5f0-9f3d-49c8-9da0-75d0bf6fc272 tempest-VolumesNegativeTest-814389221 tempest-VolumesNegativeTest-814389221-project-member] Lock "a4897e0e-d29c-45b6-8f2f-8f5ff55d6a8f" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 1.606s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:28:08 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:28:08 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:28:08 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager.update_available_resource {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:28:08 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:28:08 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:28:08 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:28:08 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Auditing locally available compute resources for user (node: user) {{(pid=75234) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:907}}
Nov 29 03:28:08 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f6bc212d-85cd-479c-92ae-112c9f0101bc/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:28:08 user nova-compute[75234]: DEBUG nova.compute.manager [req-2ee40540-5a7f-4f4e-885d-b27f50d07803 req-d9e32a0f-b322-4e77-8cd9-338ee1ffec83 service nova] [instance: a4897e0e-d29c-45b6-8f2f-8f5ff55d6a8f] Received event network-vif-plugged-55748abc-3e82-40c3-a7ba-b16327b28e9e {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:28:08 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-2ee40540-5a7f-4f4e-885d-b27f50d07803 req-d9e32a0f-b322-4e77-8cd9-338ee1ffec83 service nova] Acquiring lock "a4897e0e-d29c-45b6-8f2f-8f5ff55d6a8f-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:28:08 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-2ee40540-5a7f-4f4e-885d-b27f50d07803 req-d9e32a0f-b322-4e77-8cd9-338ee1ffec83 service nova] Lock "a4897e0e-d29c-45b6-8f2f-8f5ff55d6a8f-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:28:08 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-2ee40540-5a7f-4f4e-885d-b27f50d07803 req-d9e32a0f-b322-4e77-8cd9-338ee1ffec83 service nova] Lock "a4897e0e-d29c-45b6-8f2f-8f5ff55d6a8f-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:28:08 user nova-compute[75234]: DEBUG nova.compute.manager [req-2ee40540-5a7f-4f4e-885d-b27f50d07803 req-d9e32a0f-b322-4e77-8cd9-338ee1ffec83 service nova] [instance: a4897e0e-d29c-45b6-8f2f-8f5ff55d6a8f] No waiting events found dispatching network-vif-plugged-55748abc-3e82-40c3-a7ba-b16327b28e9e {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:28:08 user nova-compute[75234]: WARNING nova.compute.manager [req-2ee40540-5a7f-4f4e-885d-b27f50d07803 req-d9e32a0f-b322-4e77-8cd9-338ee1ffec83 service nova] [instance: a4897e0e-d29c-45b6-8f2f-8f5ff55d6a8f] Received unexpected event network-vif-plugged-55748abc-3e82-40c3-a7ba-b16327b28e9e for instance with vm_state deleted and task_state None.
Nov 29 03:28:08 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f6bc212d-85cd-479c-92ae-112c9f0101bc/disk --force-share --output=json" returned: 0 in 0.213s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:28:08 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f6bc212d-85cd-479c-92ae-112c9f0101bc/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:28:09 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/f6bc212d-85cd-479c-92ae-112c9f0101bc/disk --force-share --output=json" returned: 0 in 0.177s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:28:09 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/65d48c93-b6da-4f51-9d5a-0676dda62a60/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:28:09 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/65d48c93-b6da-4f51-9d5a-0676dda62a60/disk --force-share --output=json" returned: 0 in 0.172s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:28:09 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/65d48c93-b6da-4f51-9d5a-0676dda62a60/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:28:09 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/65d48c93-b6da-4f51-9d5a-0676dda62a60/disk --force-share --output=json" returned: 0 in 0.192s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:28:09 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] skipping disk /dev/sdc (vda) as it is a volume {{(pid=75234) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11213}}
Nov 29 03:28:09 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/5d7d744e-1a5f-40ab-b961-4b58c183efe0/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:28:09 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/5d7d744e-1a5f-40ab-b961-4b58c183efe0/disk --force-share --output=json" returned: 0 in 0.173s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:28:09 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/5d7d744e-1a5f-40ab-b961-4b58c183efe0/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:28:09 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/5d7d744e-1a5f-40ab-b961-4b58c183efe0/disk --force-share --output=json" returned: 0 in 0.178s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:28:09 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/499676de-0e76-45ae-864c-44e55e3d7c7f/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:28:09 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/499676de-0e76-45ae-864c-44e55e3d7c7f/disk --force-share --output=json" returned: 0 in 0.188s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:28:09 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/499676de-0e76-45ae-864c-44e55e3d7c7f/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:28:10 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/499676de-0e76-45ae-864c-44e55e3d7c7f/disk --force-share --output=json" returned: 0 in 0.169s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:28:10 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:28:10 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:28:10 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Hypervisor/Node resource view: name=user free_ram=8382MB free_disk=25.074310302734375GB free_vcpus=7 pci_devices=[{"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}] {{(pid=75234) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1106}}
Nov 29 03:28:10 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:28:10 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:28:10 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance 65d48c93-b6da-4f51-9d5a-0676dda62a60 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:28:10 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance f6bc212d-85cd-479c-92ae-112c9f0101bc actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:28:10 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance 5d7d744e-1a5f-40ab-b961-4b58c183efe0 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:28:10 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance d5096ca2-d005-45ac-a487-11a810446859 actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:28:10 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance 499676de-0e76-45ae-864c-44e55e3d7c7f actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:28:10 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Total usable vcpus: 12, total allocated vcpus: 5 {{(pid=75234) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1129}}
Nov 29 03:28:10 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Final resource view: name=user phys_ram=16011MB used_ram=3072MB phys_disk=40GB used_disk=4GB total_vcpus=12 used_vcpus=5 pci_stats=[] {{(pid=75234) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1138}}
Nov 29 03:28:10 user nova-compute[75234]: DEBUG nova.compute.provider_tree [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Inventory has not changed in ProviderTree for provider: 4d291220-7c37-4839-a0e3-b23c66502b74 {{(pid=75234) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Nov 29 03:28:10 user nova-compute[75234]: DEBUG nova.scheduler.client.report [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Inventory has not changed for provider 4d291220-7c37-4839-a0e3-b23c66502b74 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=75234) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Nov 29 03:28:10 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Compute_service record updated for user:user {{(pid=75234) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:1067}}
Nov 29 03:28:10 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.320s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:28:11 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:28:11 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Starting heal instance info cache {{(pid=75234) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9922}}
Nov 29 03:28:11 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Acquiring lock "refresh_cache-65d48c93-b6da-4f51-9d5a-0676dda62a60" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:28:11 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Acquired lock "refresh_cache-65d48c93-b6da-4f51-9d5a-0676dda62a60" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:28:11 user nova-compute[75234]: DEBUG nova.network.neutron [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] [instance: 65d48c93-b6da-4f51-9d5a-0676dda62a60] Forcefully refreshing network info cache for instance {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2003}}
Nov 29 03:28:12 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:28:12 user nova-compute[75234]: DEBUG nova.network.neutron [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] [instance: 65d48c93-b6da-4f51-9d5a-0676dda62a60] Updating instance_info_cache with network_info: [{"id": "967835a2-4b55-4ae6-b071-91233af412fd", "address": "fa:16:3e:ee:3c:e2", "network": {"id": "4d086e08-7798-4cfe-9365-86be6f0a52d2", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-305624008-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "db00bcd0c82c44f1bda739e9857f6e84", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap967835a2-4b", "ovs_interfaceid": "967835a2-4b55-4ae6-b071-91233af412fd", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:28:12 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Releasing lock "refresh_cache-65d48c93-b6da-4f51-9d5a-0676dda62a60" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:28:12 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] [instance: 65d48c93-b6da-4f51-9d5a-0676dda62a60] Updated the network info_cache for instance {{(pid=75234) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9993}}
Nov 29 03:28:12 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:28:15 user nova-compute[75234]: DEBUG nova.compute.manager [req-9676a243-1168-4f40-a4d3-36258a083ab5 req-9bdd2c83-9482-4f9f-ad20-09a12f45eb29 service nova] [instance: 499676de-0e76-45ae-864c-44e55e3d7c7f] Received event network-changed-720988c2-e733-41cb-b3ea-c215cfe46bd6 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:28:15 user nova-compute[75234]: DEBUG nova.compute.manager [req-9676a243-1168-4f40-a4d3-36258a083ab5 req-9bdd2c83-9482-4f9f-ad20-09a12f45eb29 service nova] [instance: 499676de-0e76-45ae-864c-44e55e3d7c7f] Refreshing instance network info cache due to event network-changed-720988c2-e733-41cb-b3ea-c215cfe46bd6. {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11117}}
Nov 29 03:28:15 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-9676a243-1168-4f40-a4d3-36258a083ab5 req-9bdd2c83-9482-4f9f-ad20-09a12f45eb29 service nova] Acquiring lock "refresh_cache-499676de-0e76-45ae-864c-44e55e3d7c7f" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:28:15 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-9676a243-1168-4f40-a4d3-36258a083ab5 req-9bdd2c83-9482-4f9f-ad20-09a12f45eb29 service nova] Acquired lock "refresh_cache-499676de-0e76-45ae-864c-44e55e3d7c7f" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:28:15 user nova-compute[75234]: DEBUG nova.network.neutron [req-9676a243-1168-4f40-a4d3-36258a083ab5 req-9bdd2c83-9482-4f9f-ad20-09a12f45eb29 service nova] [instance: 499676de-0e76-45ae-864c-44e55e3d7c7f] Refreshing network info cache for port 720988c2-e733-41cb-b3ea-c215cfe46bd6 {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2006}}
Nov 29 03:28:15 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:28:15 user nova-compute[75234]: DEBUG nova.network.neutron [req-9676a243-1168-4f40-a4d3-36258a083ab5 req-9bdd2c83-9482-4f9f-ad20-09a12f45eb29 service nova] [instance: 499676de-0e76-45ae-864c-44e55e3d7c7f] Updated VIF entry in instance network info cache for port 720988c2-e733-41cb-b3ea-c215cfe46bd6. {{(pid=75234) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3481}}
Nov 29 03:28:15 user nova-compute[75234]: DEBUG nova.network.neutron [req-9676a243-1168-4f40-a4d3-36258a083ab5 req-9bdd2c83-9482-4f9f-ad20-09a12f45eb29 service nova] [instance: 499676de-0e76-45ae-864c-44e55e3d7c7f] Updating instance_info_cache with network_info: [{"id": "720988c2-e733-41cb-b3ea-c215cfe46bd6", "address": "fa:16:3e:0d:b8:b3", "network": {"id": "0f289171-3d73-4054-95a0-2515b4f206a0", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-1787960284-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.10", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "315732b6e3ee44d584b8e510ef199a90", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap720988c2-e7", "ovs_interfaceid": "720988c2-e733-41cb-b3ea-c215cfe46bd6", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:28:15 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-9676a243-1168-4f40-a4d3-36258a083ab5 req-9bdd2c83-9482-4f9f-ad20-09a12f45eb29 service nova] Releasing lock "refresh_cache-499676de-0e76-45ae-864c-44e55e3d7c7f" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:28:16 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:28:17 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:28:17 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:28:17 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-8f88db6e-4d5f-45e8-aff3-a3558bcc5b40 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Acquiring lock "499676de-0e76-45ae-864c-44e55e3d7c7f" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:28:17 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-8f88db6e-4d5f-45e8-aff3-a3558bcc5b40 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Lock "499676de-0e76-45ae-864c-44e55e3d7c7f" acquired by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:28:17 user nova-compute[75234]: DEBUG nova.objects.instance [None req-8f88db6e-4d5f-45e8-aff3-a3558bcc5b40 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Lazy-loading 'flavor' on Instance uuid 499676de-0e76-45ae-864c-44e55e3d7c7f {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:28:17 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-8f88db6e-4d5f-45e8-aff3-a3558bcc5b40 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Lock "499676de-0e76-45ae-864c-44e55e3d7c7f" "released" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: held 0.071s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:28:18 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-8f88db6e-4d5f-45e8-aff3-a3558bcc5b40 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Acquiring lock "499676de-0e76-45ae-864c-44e55e3d7c7f" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:28:18 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-8f88db6e-4d5f-45e8-aff3-a3558bcc5b40 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Lock "499676de-0e76-45ae-864c-44e55e3d7c7f" acquired by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:28:18 user nova-compute[75234]: INFO nova.compute.manager [None req-8f88db6e-4d5f-45e8-aff3-a3558bcc5b40 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] [instance: 499676de-0e76-45ae-864c-44e55e3d7c7f] Attaching volume fbebef3b-05f6-495b-8523-660f550506fb to /dev/vdb
Nov 29 03:28:18 user nova-compute[75234]: DEBUG os_brick.utils [None req-8f88db6e-4d5f-45e8-aff3-a3558bcc5b40 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:176}}
Nov 29 03:28:18 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:28:18 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.015s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:28:18 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[9dd69631-55f2-49f7-85ab-cabf1177b4a9]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:dd1368c5ba2c\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:28:18 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt -v / -n -o SOURCE {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:28:18 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt -v / -n -o SOURCE" returned: 0 in 0.015s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:28:18 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[154a1eda-db02-4382-a094-0216826c02a8]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:28:18 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:28:18 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.016s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:28:18 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[933645fe-2350-4ef8-91d7-58a59edd5d52]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:28:18 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[6b3f3417-ea46-4d5e-9957-ec70991ca096]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:28:18 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-8f88db6e-4d5f-45e8-aff3-a3558bcc5b40 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Running cmd (subprocess): nvme version {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:28:18 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-8f88db6e-4d5f-45e8-aff3-a3558bcc5b40 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] 'nvme version' failed. Not Retrying. {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:473}}
Nov 29 03:28:18 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.nvmeof [None req-8f88db6e-4d5f-45e8-aff3-a3558bcc5b40 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] nvme not present on system {{(pid=75234) nvme_present /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/nvmeof.py:756}}
Nov 29 03:28:18 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:28:18 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:473}}
Nov 29 03:28:18 user nova-compute[75234]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Nov 29 03:28:18 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[46ba2935-15c7-41c0-b180-97fab87847ee]: (4, '') {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:28:18 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.lightos [None req-8f88db6e-4d5f-45e8-aff3-a3558bcc5b40 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=75234) find_dsc /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/lightos.py:98}}
Nov 29 03:28:18 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.lightos [None req-8f88db6e-4d5f-45e8-aff3-a3558bcc5b40 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=75234) get_connector_properties /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/lightos.py:76}}
Nov 29 03:28:18 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.lightos [None req-8f88db6e-4d5f-45e8-aff3-a3558bcc5b40 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] LIGHTOS: no hostnqn found. {{(pid=75234) get_connector_properties /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/lightos.py:84}}
Nov 29 03:28:18 user nova-compute[75234]: DEBUG os_brick.utils [None req-8f88db6e-4d5f-45e8-aff3-a3558bcc5b40 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] <== get_connector_properties: return (122ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:dd1368c5ba2c', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:203}}
Nov 29 03:28:18 user nova-compute[75234]: DEBUG nova.virt.block_device [None req-8f88db6e-4d5f-45e8-aff3-a3558bcc5b40 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] [instance: 499676de-0e76-45ae-864c-44e55e3d7c7f] Updating existing volume attachment record: 9f051790-c01f-4e6b-aa33-1848e48059fe {{(pid=75234) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Nov 29 03:28:18 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-a993e9c4-6849-45ce-ba40-fe3da865f35c tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Waiting for libvirt event about the detach of device vdb with device alias virtio-disk1 from instance 5d7d744e-1a5f-40ab-b961-4b58c183efe0 is timed out.
Nov 29 03:28:18 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-a993e9c4-6849-45ce-ba40-fe3da865f35c tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Successfully detached device vdb from instance 5d7d744e-1a5f-40ab-b961-4b58c183efe0 from the live domain config.
Nov 29 03:28:18 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-a993e9c4-6849-45ce-ba40-fe3da865f35c tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] [instance: 5d7d744e-1a5f-40ab-b961-4b58c183efe0] calling os-brick to detach iSCSI Volume {{(pid=75234) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Nov 29 03:28:18 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-a993e9c4-6849-45ce-ba40-fe3da865f35c tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f8a5050c9a0>, {'target_iqn': 'iqn.2010-10.org.openstack:65acc1f5-8dbf-457c-a978-510436bccf11', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'jqCadzEM', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdb'}, None), 'kwargs': False}}" {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:176}}
Nov 29 03:28:18 user nova-compute[75234]: WARNING os_brick.initiator.connectors.base [None req-a993e9c4-6849-45ce-ba40-fe3da865f35c tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Nov 29 03:28:18 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-a993e9c4-6849-45ce-ba40-fe3da865f35c tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:68}}
Nov 29 03:28:18 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-a993e9c4-6849-45ce-ba40-fe3da865f35c tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.003s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:73}}
Nov 29 03:28:18 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:28:19 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.019s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:28:19 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[b1fa13fd-e421-48dd-a9b9-318287a2dd7e]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:65acc1f5-8dbf-457c-a978-510436bccf11\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:28:19 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-a993e9c4-6849-45ce-ba40-fe3da865f35c tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS:
Nov 29 03:28:19 user nova-compute[75234]: No targets found.
Nov 29 03:28:19 user nova-compute[75234]: iSNS:
Nov 29 03:28:19 user nova-compute[75234]: No targets found.
Nov 29 03:28:19 user nova-compute[75234]: STATIC:
Nov 29 03:28:19 user nova-compute[75234]: Target: iqn.2010-10.org.openstack:65acc1f5-8dbf-457c-a978-510436bccf11
Nov 29 03:28:19 user nova-compute[75234]:         Portal: 172.16.0.220:3260,-1
Nov 29 03:28:19 user nova-compute[75234]:                 Iface Name: default
Nov 29 03:28:19 user nova-compute[75234]: Target: iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0
Nov 29 03:28:19 user nova-compute[75234]:         Portal: 172.16.0.220:3260,-1
Nov 29 03:28:19 user nova-compute[75234]:                 Iface Name: default
Nov 29 03:28:19 user nova-compute[75234]: FIRMWARE:
Nov 29 03:28:19 user nova-compute[75234]: No targets found.
Nov 29 03:28:19 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:28:19 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-a993e9c4-6849-45ce-ba40-fe3da865f35c tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Regex to get portals from discoverydb: ^SENDTARGETS:
Nov 29 03:28:19 user nova-compute[75234]: .*?^DiscoveryAddress: 172.16.0.220,3260.*?
Nov 29 03:28:19 user nova-compute[75234]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=75234) _get_discoverydb_portals /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:382}}
Nov 29 03:28:19 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-a993e9c4-6849-45ce-ba40-fe3da865f35c tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:65acc1f5-8dbf-457c-a978-510436bccf11', 0)] {{(pid=75234) _get_connection_devices /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:830}}
Nov 29 03:28:19 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:28:19 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.022s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:28:19 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[7c495380-ae89-4f92-8769-c4ffc07ab775]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:65acc1f5-8dbf-457c-a978-510436bccf11\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:28:19 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:28:19 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.022s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:28:19 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[3890ffbd-d993-4495-ab11-f0414a29bbbe]: (4, ('tcp: [42] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0 (non-flash)\ntcp: [47] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:65acc1f5-8dbf-457c-a978-510436bccf11 (non-flash)\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:28:19 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-a993e9c4-6849-45ce-ba40-fe3da865f35c tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [42] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0 (non-flash)
Nov 29 03:28:19 user nova-compute[75234]: tcp: [47] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:65acc1f5-8dbf-457c-a978-510436bccf11 (non-flash)
Nov 29 03:28:19 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:28:19 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-a993e9c4-6849-45ce-ba40-fe3da865f35c tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] iscsi session list stdout=tcp: [42] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0 (non-flash)
Nov 29 03:28:19 user nova-compute[75234]: tcp: [47] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:65acc1f5-8dbf-457c-a978-510436bccf11 (non-flash)
Nov 29 03:28:19 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsi_session /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Nov 29 03:28:19 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-a993e9c4-6849-45ce-ba40-fe3da865f35c tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7f8a502c72e0>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:65acc1f5-8dbf-457c-a978-510436bccf11'): ({'sdb'}, set())}) {{(pid=75234) _get_connection_devices /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:863}}
Nov 29 03:28:19 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-a993e9c4-6849-45ce-ba40-fe3da865f35c tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Removing single pathed devices sdb {{(pid=75234) remove_connection /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:374}}
Nov 29 03:28:19 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:28:19 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.020s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:28:19 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[42dbf4d5-a51c-4078-ba31-4b77e2b8b023]: (4, ('path checker states:\nup                  2\n\npaths: 0\nbusy: False\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:28:19 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdb {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:28:19 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdb" returned: 0 in 0.021s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:28:19 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[3aec4893-fc0f-49e7-964a-76d0cedf58fe]: (4, ('ok\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:28:19 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-a993e9c4-6849-45ce-ba40-fe3da865f35c tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Flushing IO for device /dev/sdb {{(pid=75234) flush_device_io /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:434}}
Nov 29 03:28:19 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdb {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:28:19 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdb" returned: 0 in 0.014s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:28:19 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[0dd50c14-225f-4e6a-b918-8fd91d0e1e3c]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:28:19 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-a993e9c4-6849-45ce-ba40-fe3da865f35c tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Remove SCSI device /dev/sdb with /sys/block/sdb/device/delete {{(pid=75234) remove_scsi_device /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:148}}
Nov 29 03:28:19 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdb/device/delete {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:28:19 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdb/device/delete" returned: 0 in 0.055s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:28:19 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[d34c9760-e70d-4c00-a73c-61d1d7dce53b]: (4, ('1', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:28:19 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-a993e9c4-6849-45ce-ba40-fe3da865f35c tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Checking to see if SCSI volumes sdb have been removed. {{(pid=75234) wait_for_volumes_removal /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:156}}
Nov 29 03:28:19 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-a993e9c4-6849-45ce-ba40-fe3da865f35c tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] SCSI volumes sdb have been removed. {{(pid=75234) wait_for_volumes_removal /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:166}}
Nov 29 03:28:19 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-a993e9c4-6849-45ce-ba40-fe3da865f35c tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:65acc1f5-8dbf-457c-a978-510436bccf11')] {{(pid=75234) _disconnect_connection /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Nov 29 03:28:19 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:65acc1f5-8dbf-457c-a978-510436bccf11 -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:28:19 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:65acc1f5-8dbf-457c-a978-510436bccf11 -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.042s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:28:19 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[d068aa5f-1861-4599-82fa-5764cce9ac33]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:28:19 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-a993e9c4-6849-45ce-ba40-fe3da865f35c tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:28:19 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:65acc1f5-8dbf-457c-a978-510436bccf11 -p 172.16.0.220:3260 --logout {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:28:19 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:65acc1f5-8dbf-457c-a978-510436bccf11 -p 172.16.0.220:3260 --logout" returned: 0 in 0.062s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:28:19 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[d4e2b69a-e24c-40c9-83c6-edb8196daec8]: (4, ('Logging out of session [sid: 47, target: iqn.2010-10.org.openstack:65acc1f5-8dbf-457c-a978-510436bccf11, portal: 172.16.0.220,3260]\nLogout of [sid: 47, target: iqn.2010-10.org.openstack:65acc1f5-8dbf-457c-a978-510436bccf11, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:28:19 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-a993e9c4-6849-45ce-ba40-fe3da865f35c tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 47, target: iqn.2010-10.org.openstack:65acc1f5-8dbf-457c-a978-510436bccf11, portal: 172.16.0.220,3260]
Nov 29 03:28:19 user nova-compute[75234]: Logout of [sid: 47, target: iqn.2010-10.org.openstack:65acc1f5-8dbf-457c-a978-510436bccf11, portal: 172.16.0.220,3260] successful.
Nov 29 03:28:19 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:28:19 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:65acc1f5-8dbf-457c-a978-510436bccf11 -p 172.16.0.220:3260 --op delete {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:28:19 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:65acc1f5-8dbf-457c-a978-510436bccf11 -p 172.16.0.220:3260 --op delete" returned: 0 in 0.023s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:28:19 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[e24519e8-9f1e-416a-a21c-4b603dba62d6]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:28:19 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-a993e9c4-6849-45ce-ba40-fe3da865f35c tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:28:19 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-a993e9c4-6849-45ce-ba40-fe3da865f35c tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.354s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:87}}
Nov 29 03:28:19 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-a993e9c4-6849-45ce-ba40-fe3da865f35c tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] <== disconnect_volume: return (356ms) None {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:203}}
Nov 29 03:28:19 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-a993e9c4-6849-45ce-ba40-fe3da865f35c tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] [instance: 5d7d744e-1a5f-40ab-b961-4b58c183efe0] Disconnected iSCSI Volume {{(pid=75234) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:79}}
Nov 29 03:28:19 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._sync_scheduler_instance_info {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:28:21 user nova-compute[75234]: DEBUG nova.objects.instance [None req-a993e9c4-6849-45ce-ba40-fe3da865f35c tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Lazy-loading 'flavor' on Instance uuid 5d7d744e-1a5f-40ab-b961-4b58c183efe0 {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:28:21 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-a993e9c4-6849-45ce-ba40-fe3da865f35c tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Lock "5d7d744e-1a5f-40ab-b961-4b58c183efe0" "released" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: held 22.886s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:28:22 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:28:22 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:28:25 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-8f88db6e-4d5f-45e8-aff3-a3558bcc5b40 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] [instance: 499676de-0e76-45ae-864c-44e55e3d7c7f] Calling os-brick to attach iSCSI Volume {{(pid=75234) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Nov 29 03:28:25 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-8f88db6e-4d5f-45e8-aff3-a3558bcc5b40 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f8a5050c9a0>, 'connection_properties': {'target_iqn': 'iqn.2010-10.org.openstack:fbebef3b-05f6-495b-8523-660f550506fb', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'AvlTztYZ', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:176}}
Nov 29 03:28:25 user nova-compute[75234]: WARNING os_brick.initiator.connectors.base [None req-8f88db6e-4d5f-45e8-aff3-a3558bcc5b40 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Nov 29 03:28:25 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-8f88db6e-4d5f-45e8-aff3-a3558bcc5b40 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:68}}
Nov 29 03:28:25 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-8f88db6e-4d5f-45e8-aff3-a3558bcc5b40 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:73}}
Nov 29 03:28:25 user nova-compute[75234]: WARNING os_brick.initiator.connectors.base [None req-8f88db6e-4d5f-45e8-aff3-a3558bcc5b40 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Nov 29 03:28:25 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-8f88db6e-4d5f-45e8-aff3-a3558bcc5b40 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:fbebef3b-05f6-495b-8523-660f550506fb" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:68}}
Nov 29 03:28:25 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-8f88db6e-4d5f-45e8-aff3-a3558bcc5b40 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:fbebef3b-05f6-495b-8523-660f550506fb" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:73}}
Nov 29 03:28:25 user nova-compute[75234]: INFO os_brick.initiator.connectors.iscsi [None req-8f88db6e-4d5f-45e8-aff3-a3558bcc5b40 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Nov 29 03:28:25 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:fbebef3b-05f6-495b-8523-660f550506fb -p 172.16.0.220:3260 {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:28:25 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:fbebef3b-05f6-495b-8523-660f550506fb -p 172.16.0.220:3260" returned: 21 in 0.018s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:28:25 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[1320fce9-b1fe-4ada-8960-db97bff48600]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:28:25 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-8f88db6e-4d5f-45e8-aff3-a3558bcc5b40 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Nov 29 03:28:25 user nova-compute[75234]:  {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:28:25 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:fbebef3b-05f6-495b-8523-660f550506fb -p 172.16.0.220:3260 --interface default --op new {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:28:25 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:fbebef3b-05f6-495b-8523-660f550506fb -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.021s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:28:25 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[1c118ffd-9a5e-4cc6-b57e-72b0e4641aa5]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:fbebef3b-05f6-495b-8523-660f550506fb] added\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:28:25 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-8f88db6e-4d5f-45e8-aff3-a3558bcc5b40 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:fbebef3b-05f6-495b-8523-660f550506fb] added
Nov 29 03:28:25 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:28:25 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:fbebef3b-05f6-495b-8523-660f550506fb -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:28:25 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:fbebef3b-05f6-495b-8523-660f550506fb -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.020s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:28:25 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[160fafb1-e6a8-4714-8154-51040eff8a23]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:28:25 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-8f88db6e-4d5f-45e8-aff3-a3558bcc5b40 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:28:25 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:fbebef3b-05f6-495b-8523-660f550506fb -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:28:25 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:fbebef3b-05f6-495b-8523-660f550506fb -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.020s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:28:25 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[12b8fade-19e1-4f3d-9e88-4910f64c5035]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:28:25 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-8f88db6e-4d5f-45e8-aff3-a3558bcc5b40 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:28:25 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:fbebef3b-05f6-495b-8523-660f550506fb -p 172.16.0.220:3260 --op update -n node.session.auth.username -v AvlTztYZ {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:28:25 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:fbebef3b-05f6-495b-8523-660f550506fb -p 172.16.0.220:3260 --op update -n node.session.auth.username -v AvlTztYZ" returned: 0 in 0.019s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:28:25 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[b073ffca-3dfc-4709-b0c2-d9d2f717303d]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:28:25 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-8f88db6e-4d5f-45e8-aff3-a3558bcc5b40 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'AvlTztYZ'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:28:25 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:fbebef3b-05f6-495b-8523-660f550506fb -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:28:25 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:fbebef3b-05f6-495b-8523-660f550506fb -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.017s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:28:25 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[2f8142b7-e885-49e6-b1cf-c14f4f939f0d]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:28:25 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-8f88db6e-4d5f-45e8-aff3-a3558bcc5b40 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:28:25 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:28:25 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.018s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:28:25 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[4a189422-b1cb-47de-b902-53d447c27c91]: (4, ('tcp: [42] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0 (non-flash)\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:28:25 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-8f88db6e-4d5f-45e8-aff3-a3558bcc5b40 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [42] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0 (non-flash)
Nov 29 03:28:25 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:28:25 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-8f88db6e-4d5f-45e8-aff3-a3558bcc5b40 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] iscsi session list stdout=tcp: [42] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0 (non-flash)
Nov 29 03:28:25 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsi_session /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Nov 29 03:28:25 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:fbebef3b-05f6-495b-8523-660f550506fb -p 172.16.0.220:3260 --login {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:28:25 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:fbebef3b-05f6-495b-8523-660f550506fb -p 172.16.0.220:3260 --login" returned: 0 in 0.052s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:28:25 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[34ba4a4d-32a8-4c1f-994c-1efbbfe5c759]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:fbebef3b-05f6-495b-8523-660f550506fb, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:fbebef3b-05f6-495b-8523-660f550506fb, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:28:25 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-8f88db6e-4d5f-45e8-aff3-a3558bcc5b40 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:fbebef3b-05f6-495b-8523-660f550506fb, portal: 172.16.0.220,3260]
Nov 29 03:28:25 user nova-compute[75234]: Login to [iface: default, target: iqn.2010-10.org.openstack:fbebef3b-05f6-495b-8523-660f550506fb, portal: 172.16.0.220,3260] successful.
Nov 29 03:28:25 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:28:26 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:fbebef3b-05f6-495b-8523-660f550506fb -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:28:26 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:fbebef3b-05f6-495b-8523-660f550506fb -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.018s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:28:26 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[ac272bfb-436b-4d89-8f74-3fbfb9cdf380]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:28:26 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-8f88db6e-4d5f-45e8-aff3-a3558bcc5b40 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:28:26 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:28:26 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.020s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:28:26 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[45bb810d-8a94-494b-b32f-04e5e3ba0dd0]: (4, ('tcp: [42] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0 (non-flash)\ntcp: [49] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:fbebef3b-05f6-495b-8523-660f550506fb (non-flash)\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:28:26 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-8f88db6e-4d5f-45e8-aff3-a3558bcc5b40 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [42] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0 (non-flash)
Nov 29 03:28:26 user nova-compute[75234]: tcp: [49] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:fbebef3b-05f6-495b-8523-660f550506fb (non-flash)
Nov 29 03:28:26 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:28:26 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-8f88db6e-4d5f-45e8-aff3-a3558bcc5b40 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] iscsi session list stdout=tcp: [42] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0 (non-flash)
Nov 29 03:28:26 user nova-compute[75234]: tcp: [49] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:fbebef3b-05f6-495b-8523-660f550506fb (non-flash)
Nov 29 03:28:26 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsi_session /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Nov 29 03:28:26 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-8f88db6e-4d5f-45e8-aff3-a3558bcc5b40 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:fbebef3b-05f6-495b-8523-660f550506fb" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.263s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:87}}
Nov 29 03:28:26 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-8f88db6e-4d5f-45e8-aff3-a3558bcc5b40 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Connected to 172.16.0.220:3260 {{(pid=75234) _connect_vol /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:644}}
Nov 29 03:28:26 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-8f88db6e-4d5f-45e8-aff3-a3558bcc5b40 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] HCTL ('33', '-', '-', 0) found on session 49 with lun 0 {{(pid=75234) get_hctl /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:770}}
Nov 29 03:28:26 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-8f88db6e-4d5f-45e8-aff3-a3558bcc5b40 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Scanning host 33 c: -, t: -, l: 0) {{(pid=75234) scan_iscsi /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:795}}
Nov 29 03:28:26 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host33/scan {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:28:26 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host33/scan" returned: 0 in 0.016s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:28:26 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[6b8e7bff-4e45-498e-b7ed-a5b70f417e6e]: (4, ('- - 0', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:28:26 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-8f88db6e-4d5f-45e8-aff3-a3558bcc5b40 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Searching for a device in session 49 and hctl ['33', '*', '*', 0] yield: sdb {{(pid=75234) device_name_by_hctl /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:789}}
Nov 29 03:28:26 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-8f88db6e-4d5f-45e8-aff3-a3558bcc5b40 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Connected to sdb using {'target_iqn': 'iqn.2010-10.org.openstack:fbebef3b-05f6-495b-8523-660f550506fb', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'AvlTztYZ', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False} {{(pid=75234) _connect_vol /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:672}}
Nov 29 03:28:27 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-8f88db6e-4d5f-45e8-aff3-a3558bcc5b40 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.293s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:87}}
Nov 29 03:28:27 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-8f88db6e-4d5f-45e8-aff3-a3558bcc5b40 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] <== connect_volume: return (1294ms) {'type': 'block', 'scsi_wwn': '23830333963396564', 'path': '/dev/sdb'} {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:203}}
Nov 29 03:28:27 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-8f88db6e-4d5f-45e8-aff3-a3558bcc5b40 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] [instance: 499676de-0e76-45ae-864c-44e55e3d7c7f] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '23830333963396564', 'path': '/dev/sdb'} {{(pid=75234) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Nov 29 03:28:27 user nova-compute[75234]: DEBUG nova.objects.instance [None req-8f88db6e-4d5f-45e8-aff3-a3558bcc5b40 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Lazy-loading 'flavor' on Instance uuid 499676de-0e76-45ae-864c-44e55e3d7c7f {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:28:27 user nova-compute[75234]: DEBUG nova.virt.libvirt.guest [None req-8f88db6e-4d5f-45e8-aff3-a3558bcc5b40 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] attach device xml: <disk type="block" device="disk">
Nov 29 03:28:27 user nova-compute[75234]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Nov 29 03:28:27 user nova-compute[75234]:   <source dev="/dev/sdb"/>
Nov 29 03:28:27 user nova-compute[75234]:   <target dev="vdb" bus="virtio"/>
Nov 29 03:28:27 user nova-compute[75234]:   <serial>fbebef3b-05f6-495b-8523-660f550506fb</serial>
Nov 29 03:28:27 user nova-compute[75234]: </disk>
Nov 29 03:28:27 user nova-compute[75234]:  {{(pid=75234) attach_device /opt/stack/nova/nova/virt/libvirt/guest.py:339}}
Nov 29 03:28:27 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:28:27 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-8f88db6e-4d5f-45e8-aff3-a3558bcc5b40 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] No BDM found with device name vda, not building metadata. {{(pid=75234) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12089}}
Nov 29 03:28:27 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-8f88db6e-4d5f-45e8-aff3-a3558bcc5b40 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] No BDM found with device name vdb, not building metadata. {{(pid=75234) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12089}}
Nov 29 03:28:27 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-8f88db6e-4d5f-45e8-aff3-a3558bcc5b40 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] No VIF found with MAC fa:16:3e:0d:b8:b3, not building metadata {{(pid=75234) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Nov 29 03:28:27 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-8f88db6e-4d5f-45e8-aff3-a3558bcc5b40 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Lock "499676de-0e76-45ae-864c-44e55e3d7c7f" "released" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: held 9.635s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:28:28 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-39f1dba3-f38c-468a-809a-11333ff841e1 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Acquiring lock "499676de-0e76-45ae-864c-44e55e3d7c7f" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:28:28 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-39f1dba3-f38c-468a-809a-11333ff841e1 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Lock "499676de-0e76-45ae-864c-44e55e3d7c7f" acquired by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:28:28 user nova-compute[75234]: INFO nova.compute.manager [None req-39f1dba3-f38c-468a-809a-11333ff841e1 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] [instance: 499676de-0e76-45ae-864c-44e55e3d7c7f] Detaching volume fbebef3b-05f6-495b-8523-660f550506fb
Nov 29 03:28:28 user nova-compute[75234]: INFO nova.virt.block_device [None req-39f1dba3-f38c-468a-809a-11333ff841e1 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] [instance: 499676de-0e76-45ae-864c-44e55e3d7c7f] Attempting to driver detach volume fbebef3b-05f6-495b-8523-660f550506fb from mountpoint /dev/vdb
Nov 29 03:28:28 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-39f1dba3-f38c-468a-809a-11333ff841e1 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Attempting to detach device vdb from instance 499676de-0e76-45ae-864c-44e55e3d7c7f from the persistent domain config. {{(pid=75234) _detach_from_persistent /opt/stack/nova/nova/virt/libvirt/driver.py:2487}}
Nov 29 03:28:28 user nova-compute[75234]: DEBUG nova.virt.libvirt.guest [None req-39f1dba3-f38c-468a-809a-11333ff841e1 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] detach device xml: <disk type="block" device="disk">
Nov 29 03:28:28 user nova-compute[75234]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Nov 29 03:28:28 user nova-compute[75234]:   <source dev="/dev/sdb"/>
Nov 29 03:28:28 user nova-compute[75234]:   <target dev="vdb" bus="virtio"/>
Nov 29 03:28:28 user nova-compute[75234]:   <serial>fbebef3b-05f6-495b-8523-660f550506fb</serial>
Nov 29 03:28:28 user nova-compute[75234]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x07" function="0x0"/>
Nov 29 03:28:28 user nova-compute[75234]: </disk>
Nov 29 03:28:28 user nova-compute[75234]:  {{(pid=75234) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Nov 29 03:28:28 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-39f1dba3-f38c-468a-809a-11333ff841e1 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Successfully detached device vdb from instance 499676de-0e76-45ae-864c-44e55e3d7c7f from the persistent domain config.
Nov 29 03:28:28 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-39f1dba3-f38c-468a-809a-11333ff841e1 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] (1/8): Attempting to detach device vdb with device alias virtio-disk1 from instance 499676de-0e76-45ae-864c-44e55e3d7c7f from the live domain config. {{(pid=75234) _detach_from_live_with_retry /opt/stack/nova/nova/virt/libvirt/driver.py:2523}}
Nov 29 03:28:28 user nova-compute[75234]: DEBUG nova.virt.libvirt.guest [None req-39f1dba3-f38c-468a-809a-11333ff841e1 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] detach device xml: <disk type="block" device="disk">
Nov 29 03:28:28 user nova-compute[75234]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Nov 29 03:28:28 user nova-compute[75234]:   <source dev="/dev/sdb"/>
Nov 29 03:28:28 user nova-compute[75234]:   <target dev="vdb" bus="virtio"/>
Nov 29 03:28:28 user nova-compute[75234]:   <serial>fbebef3b-05f6-495b-8523-660f550506fb</serial>
Nov 29 03:28:28 user nova-compute[75234]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x07" function="0x0"/>
Nov 29 03:28:28 user nova-compute[75234]: </disk>
Nov 29 03:28:28 user nova-compute[75234]:  {{(pid=75234) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Nov 29 03:28:28 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-39f1dba3-f38c-468a-809a-11333ff841e1 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Start waiting for the detach event from libvirt for device vdb with device alias virtio-disk1 for instance 499676de-0e76-45ae-864c-44e55e3d7c7f {{(pid=75234) _detach_from_live_and_wait_for_event /opt/stack/nova/nova/virt/libvirt/driver.py:2599}}
Nov 29 03:28:32 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:28:32 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:28:34 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-89f4d392-bf87-4c48-b688-1dfb2d29f7ef tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Acquiring lock "f6bc212d-85cd-479c-92ae-112c9f0101bc" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:28:34 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-89f4d392-bf87-4c48-b688-1dfb2d29f7ef tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Lock "f6bc212d-85cd-479c-92ae-112c9f0101bc" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:28:34 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-89f4d392-bf87-4c48-b688-1dfb2d29f7ef tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Acquiring lock "f6bc212d-85cd-479c-92ae-112c9f0101bc-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:28:34 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-89f4d392-bf87-4c48-b688-1dfb2d29f7ef tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Lock "f6bc212d-85cd-479c-92ae-112c9f0101bc-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:28:34 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-89f4d392-bf87-4c48-b688-1dfb2d29f7ef tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Lock "f6bc212d-85cd-479c-92ae-112c9f0101bc-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:28:34 user nova-compute[75234]: INFO nova.compute.manager [None req-89f4d392-bf87-4c48-b688-1dfb2d29f7ef tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] [instance: f6bc212d-85cd-479c-92ae-112c9f0101bc] Terminating instance
Nov 29 03:28:34 user nova-compute[75234]: DEBUG nova.compute.manager [None req-89f4d392-bf87-4c48-b688-1dfb2d29f7ef tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] [instance: f6bc212d-85cd-479c-92ae-112c9f0101bc] Start destroying the instance on the hypervisor. {{(pid=75234) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3140}}
Nov 29 03:28:34 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:28:34 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:28:34 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:28:34 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:28:35 user nova-compute[75234]: DEBUG nova.compute.manager [req-c013fffa-3562-47df-ad50-5923ffb9eefa req-d5ba1b57-42e8-4b9d-b1be-560dd85f4fd8 service nova] [instance: f6bc212d-85cd-479c-92ae-112c9f0101bc] Received event network-vif-unplugged-e43ebf8d-4145-4a78-8be5-600ea409e775 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:28:35 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-c013fffa-3562-47df-ad50-5923ffb9eefa req-d5ba1b57-42e8-4b9d-b1be-560dd85f4fd8 service nova] Acquiring lock "f6bc212d-85cd-479c-92ae-112c9f0101bc-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:28:35 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-c013fffa-3562-47df-ad50-5923ffb9eefa req-d5ba1b57-42e8-4b9d-b1be-560dd85f4fd8 service nova] Lock "f6bc212d-85cd-479c-92ae-112c9f0101bc-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:28:35 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-c013fffa-3562-47df-ad50-5923ffb9eefa req-d5ba1b57-42e8-4b9d-b1be-560dd85f4fd8 service nova] Lock "f6bc212d-85cd-479c-92ae-112c9f0101bc-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:28:35 user nova-compute[75234]: DEBUG nova.compute.manager [req-c013fffa-3562-47df-ad50-5923ffb9eefa req-d5ba1b57-42e8-4b9d-b1be-560dd85f4fd8 service nova] [instance: f6bc212d-85cd-479c-92ae-112c9f0101bc] No waiting events found dispatching network-vif-unplugged-e43ebf8d-4145-4a78-8be5-600ea409e775 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:28:35 user nova-compute[75234]: DEBUG nova.compute.manager [req-c013fffa-3562-47df-ad50-5923ffb9eefa req-d5ba1b57-42e8-4b9d-b1be-560dd85f4fd8 service nova] [instance: f6bc212d-85cd-479c-92ae-112c9f0101bc] Received event network-vif-unplugged-e43ebf8d-4145-4a78-8be5-600ea409e775 for instance with task_state deleting. {{(pid=75234) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10890}}
Nov 29 03:28:35 user nova-compute[75234]: INFO nova.virt.libvirt.driver [-] [instance: f6bc212d-85cd-479c-92ae-112c9f0101bc] Instance destroyed successfully.
Nov 29 03:28:35 user nova-compute[75234]: DEBUG nova.objects.instance [None req-89f4d392-bf87-4c48-b688-1dfb2d29f7ef tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Lazy-loading 'resources' on Instance uuid f6bc212d-85cd-479c-92ae-112c9f0101bc {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:28:35 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-89f4d392-bf87-4c48-b688-1dfb2d29f7ef tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-11-29T03:21:17Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-SnapshotDataIntegrityTests-server-744808219',display_name='tempest-SnapshotDataIntegrityTests-server-744808219',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-snapshotdataintegritytests-server-744808219',id=37,image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBLcl+lLJR5zldffje/+52KkXZkw4CLRUBrN092/lLCyYU8Rz6cZyByDOlFzBvjoKUTlEBjgZpSHtec+jLeJl2h2rkNlWQ+il4II7lulZv0B8LAiMbGsvFeL2w1CWre/RQw==',key_name='tempest-keypair-1458143874',keypairs=<?>,launch_index=0,launched_at=2023-11-29T03:21:26Z,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='7c3b05b1968540dbbb40817c87846ff6',ramdisk_id='',reservation_id='r-vu6047ud',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-SnapshotDataIntegrityTests-1450496038',owner_user_name='tempest-SnapshotDataIntegrityTests-1450496038-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-11-29T03:21:26Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='e4aed253decd45a4b1f9e42550f7dd81',uuid=f6bc212d-85cd-479c-92ae-112c9f0101bc,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "e43ebf8d-4145-4a78-8be5-600ea409e775", "address": "fa:16:3e:4e:8e:1d", "network": {"id": "9247dc93-6a54-495e-b113-d1ff7c215178", "bridge": "br-int", "label": "tempest-SnapshotDataIntegrityTests-261469187-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.143", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "7c3b05b1968540dbbb40817c87846ff6", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape43ebf8d-41", "ovs_interfaceid": "e43ebf8d-4145-4a78-8be5-600ea409e775", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Nov 29 03:28:35 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-89f4d392-bf87-4c48-b688-1dfb2d29f7ef tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Converting VIF {"id": "e43ebf8d-4145-4a78-8be5-600ea409e775", "address": "fa:16:3e:4e:8e:1d", "network": {"id": "9247dc93-6a54-495e-b113-d1ff7c215178", "bridge": "br-int", "label": "tempest-SnapshotDataIntegrityTests-261469187-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.143", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "7c3b05b1968540dbbb40817c87846ff6", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tape43ebf8d-41", "ovs_interfaceid": "e43ebf8d-4145-4a78-8be5-600ea409e775", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:28:35 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-89f4d392-bf87-4c48-b688-1dfb2d29f7ef tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:4e:8e:1d,bridge_name='br-int',has_traffic_filtering=True,id=e43ebf8d-4145-4a78-8be5-600ea409e775,network=Network(9247dc93-6a54-495e-b113-d1ff7c215178),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tape43ebf8d-41') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:28:35 user nova-compute[75234]: DEBUG os_vif [None req-89f4d392-bf87-4c48-b688-1dfb2d29f7ef tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Unplugging vif VIFOpenVSwitch(active=True,address=fa:16:3e:4e:8e:1d,bridge_name='br-int',has_traffic_filtering=True,id=e43ebf8d-4145-4a78-8be5-600ea409e775,network=Network(9247dc93-6a54-495e-b113-d1ff7c215178),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tape43ebf8d-41') {{(pid=75234) unplug /opt/stack/data/venv/lib/python3.10/site-packages/os_vif/__init__.py:109}}
Nov 29 03:28:35 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:28:35 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tape43ebf8d-41, bridge=br-int, if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:28:35 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:28:35 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:28:35 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:28:35 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbDestroyCommand(_result=None, table=QoS, record=b0d5fe14-fca7-4a5d-8797-b3197b45ace5) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:28:35 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:28:35 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:28:35 user nova-compute[75234]: INFO os_vif [None req-89f4d392-bf87-4c48-b688-1dfb2d29f7ef tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Successfully unplugged vif VIFOpenVSwitch(active=True,address=fa:16:3e:4e:8e:1d,bridge_name='br-int',has_traffic_filtering=True,id=e43ebf8d-4145-4a78-8be5-600ea409e775,network=Network(9247dc93-6a54-495e-b113-d1ff7c215178),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tape43ebf8d-41')
Nov 29 03:28:35 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-89f4d392-bf87-4c48-b688-1dfb2d29f7ef tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] [instance: f6bc212d-85cd-479c-92ae-112c9f0101bc] Deleting instance files /opt/stack/data/nova/instances/f6bc212d-85cd-479c-92ae-112c9f0101bc_del
Nov 29 03:28:35 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-89f4d392-bf87-4c48-b688-1dfb2d29f7ef tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] [instance: f6bc212d-85cd-479c-92ae-112c9f0101bc] Deletion of /opt/stack/data/nova/instances/f6bc212d-85cd-479c-92ae-112c9f0101bc_del complete
Nov 29 03:28:35 user nova-compute[75234]: INFO nova.compute.manager [None req-89f4d392-bf87-4c48-b688-1dfb2d29f7ef tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] [instance: f6bc212d-85cd-479c-92ae-112c9f0101bc] Took 0.89 seconds to destroy the instance on the hypervisor.
Nov 29 03:28:35 user nova-compute[75234]: DEBUG oslo.service.loopingcall [None req-89f4d392-bf87-4c48-b688-1dfb2d29f7ef tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=75234) func /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/loopingcall.py:435}}
Nov 29 03:28:35 user nova-compute[75234]: DEBUG nova.compute.manager [-] [instance: f6bc212d-85cd-479c-92ae-112c9f0101bc] Deallocating network for instance {{(pid=75234) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2279}}
Nov 29 03:28:35 user nova-compute[75234]: DEBUG nova.network.neutron [-] [instance: f6bc212d-85cd-479c-92ae-112c9f0101bc] deallocate_for_instance() {{(pid=75234) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1802}}
Nov 29 03:28:37 user nova-compute[75234]: DEBUG nova.compute.manager [req-095f12cb-1555-4b5a-8881-12bb2fafd6eb req-e2e02334-a3a3-4492-ae76-d446b4c7bde9 service nova] [instance: f6bc212d-85cd-479c-92ae-112c9f0101bc] Received event network-vif-plugged-e43ebf8d-4145-4a78-8be5-600ea409e775 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:28:37 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-095f12cb-1555-4b5a-8881-12bb2fafd6eb req-e2e02334-a3a3-4492-ae76-d446b4c7bde9 service nova] Acquiring lock "f6bc212d-85cd-479c-92ae-112c9f0101bc-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:28:37 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-095f12cb-1555-4b5a-8881-12bb2fafd6eb req-e2e02334-a3a3-4492-ae76-d446b4c7bde9 service nova] Lock "f6bc212d-85cd-479c-92ae-112c9f0101bc-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:28:37 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-095f12cb-1555-4b5a-8881-12bb2fafd6eb req-e2e02334-a3a3-4492-ae76-d446b4c7bde9 service nova] Lock "f6bc212d-85cd-479c-92ae-112c9f0101bc-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:28:37 user nova-compute[75234]: DEBUG nova.compute.manager [req-095f12cb-1555-4b5a-8881-12bb2fafd6eb req-e2e02334-a3a3-4492-ae76-d446b4c7bde9 service nova] [instance: f6bc212d-85cd-479c-92ae-112c9f0101bc] No waiting events found dispatching network-vif-plugged-e43ebf8d-4145-4a78-8be5-600ea409e775 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:28:37 user nova-compute[75234]: WARNING nova.compute.manager [req-095f12cb-1555-4b5a-8881-12bb2fafd6eb req-e2e02334-a3a3-4492-ae76-d446b4c7bde9 service nova] [instance: f6bc212d-85cd-479c-92ae-112c9f0101bc] Received unexpected event network-vif-plugged-e43ebf8d-4145-4a78-8be5-600ea409e775 for instance with vm_state active and task_state deleting.
Nov 29 03:28:37 user nova-compute[75234]: DEBUG nova.compute.manager [req-095f12cb-1555-4b5a-8881-12bb2fafd6eb req-e2e02334-a3a3-4492-ae76-d446b4c7bde9 service nova] [instance: f6bc212d-85cd-479c-92ae-112c9f0101bc] Received event network-vif-deleted-e43ebf8d-4145-4a78-8be5-600ea409e775 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:28:37 user nova-compute[75234]: INFO nova.compute.manager [req-095f12cb-1555-4b5a-8881-12bb2fafd6eb req-e2e02334-a3a3-4492-ae76-d446b4c7bde9 service nova] [instance: f6bc212d-85cd-479c-92ae-112c9f0101bc] Neutron deleted interface e43ebf8d-4145-4a78-8be5-600ea409e775; detaching it from the instance and deleting it from the info cache
Nov 29 03:28:37 user nova-compute[75234]: DEBUG nova.network.neutron [req-095f12cb-1555-4b5a-8881-12bb2fafd6eb req-e2e02334-a3a3-4492-ae76-d446b4c7bde9 service nova] [instance: f6bc212d-85cd-479c-92ae-112c9f0101bc] Updating instance_info_cache with network_info: [] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:28:37 user nova-compute[75234]: DEBUG nova.network.neutron [-] [instance: f6bc212d-85cd-479c-92ae-112c9f0101bc] Updating instance_info_cache with network_info: [] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:28:37 user nova-compute[75234]: INFO nova.compute.manager [-] [instance: f6bc212d-85cd-479c-92ae-112c9f0101bc] Took 1.64 seconds to deallocate network for instance.
Nov 29 03:28:37 user nova-compute[75234]: DEBUG nova.compute.manager [req-095f12cb-1555-4b5a-8881-12bb2fafd6eb req-e2e02334-a3a3-4492-ae76-d446b4c7bde9 service nova] [instance: f6bc212d-85cd-479c-92ae-112c9f0101bc] Detach interface failed, port_id=e43ebf8d-4145-4a78-8be5-600ea409e775, reason: Instance f6bc212d-85cd-479c-92ae-112c9f0101bc could not be found. {{(pid=75234) _process_instance_vif_deleted_event /opt/stack/nova/nova/compute/manager.py:10946}}
Nov 29 03:28:37 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-89f4d392-bf87-4c48-b688-1dfb2d29f7ef tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:28:37 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-89f4d392-bf87-4c48-b688-1dfb2d29f7ef tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:28:37 user nova-compute[75234]: DEBUG nova.compute.provider_tree [None req-89f4d392-bf87-4c48-b688-1dfb2d29f7ef tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Inventory has not changed in ProviderTree for provider: 4d291220-7c37-4839-a0e3-b23c66502b74 {{(pid=75234) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Nov 29 03:28:37 user nova-compute[75234]: DEBUG nova.scheduler.client.report [None req-89f4d392-bf87-4c48-b688-1dfb2d29f7ef tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Inventory has not changed for provider 4d291220-7c37-4839-a0e3-b23c66502b74 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=75234) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Nov 29 03:28:37 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-89f4d392-bf87-4c48-b688-1dfb2d29f7ef tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.209s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:28:37 user nova-compute[75234]: INFO nova.scheduler.client.report [None req-89f4d392-bf87-4c48-b688-1dfb2d29f7ef tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Deleted allocations for instance f6bc212d-85cd-479c-92ae-112c9f0101bc
Nov 29 03:28:37 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-89f4d392-bf87-4c48-b688-1dfb2d29f7ef tempest-SnapshotDataIntegrityTests-1450496038 tempest-SnapshotDataIntegrityTests-1450496038-project-member] Lock "f6bc212d-85cd-479c-92ae-112c9f0101bc" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 2.939s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:28:40 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:28:45 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:28:45 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:28:46 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:28:47 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:28:48 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-39f1dba3-f38c-468a-809a-11333ff841e1 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Waiting for libvirt event about the detach of device vdb with device alias virtio-disk1 from instance 499676de-0e76-45ae-864c-44e55e3d7c7f is timed out.
Nov 29 03:28:48 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-39f1dba3-f38c-468a-809a-11333ff841e1 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Successfully detached device vdb from instance 499676de-0e76-45ae-864c-44e55e3d7c7f from the live domain config.
Nov 29 03:28:48 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-39f1dba3-f38c-468a-809a-11333ff841e1 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] [instance: 499676de-0e76-45ae-864c-44e55e3d7c7f] calling os-brick to detach iSCSI Volume {{(pid=75234) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Nov 29 03:28:48 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-39f1dba3-f38c-468a-809a-11333ff841e1 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f8a5050c9a0>, {'target_iqn': 'iqn.2010-10.org.openstack:fbebef3b-05f6-495b-8523-660f550506fb', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'AvlTztYZ', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdb'}, None), 'kwargs': False}}" {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:176}}
Nov 29 03:28:48 user nova-compute[75234]: WARNING os_brick.initiator.connectors.base [None req-39f1dba3-f38c-468a-809a-11333ff841e1 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Nov 29 03:28:48 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-39f1dba3-f38c-468a-809a-11333ff841e1 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:68}}
Nov 29 03:28:48 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-39f1dba3-f38c-468a-809a-11333ff841e1 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:73}}
Nov 29 03:28:48 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:28:48 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.020s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:28:48 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[e311429c-c4e7-4171-bcd7-da56640b77dd]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:fbebef3b-05f6-495b-8523-660f550506fb\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:28:48 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-39f1dba3-f38c-468a-809a-11333ff841e1 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS:
Nov 29 03:28:48 user nova-compute[75234]: No targets found.
Nov 29 03:28:48 user nova-compute[75234]: iSNS:
Nov 29 03:28:48 user nova-compute[75234]: No targets found.
Nov 29 03:28:48 user nova-compute[75234]: STATIC:
Nov 29 03:28:48 user nova-compute[75234]: Target: iqn.2010-10.org.openstack:fbebef3b-05f6-495b-8523-660f550506fb
Nov 29 03:28:48 user nova-compute[75234]:         Portal: 172.16.0.220:3260,-1
Nov 29 03:28:48 user nova-compute[75234]:                 Iface Name: default
Nov 29 03:28:48 user nova-compute[75234]: Target: iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0
Nov 29 03:28:48 user nova-compute[75234]:         Portal: 172.16.0.220:3260,-1
Nov 29 03:28:48 user nova-compute[75234]:                 Iface Name: default
Nov 29 03:28:48 user nova-compute[75234]: FIRMWARE:
Nov 29 03:28:48 user nova-compute[75234]: No targets found.
Nov 29 03:28:48 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:28:48 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-39f1dba3-f38c-468a-809a-11333ff841e1 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Regex to get portals from discoverydb: ^SENDTARGETS:
Nov 29 03:28:48 user nova-compute[75234]: .*?^DiscoveryAddress: 172.16.0.220,3260.*?
Nov 29 03:28:48 user nova-compute[75234]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=75234) _get_discoverydb_portals /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:382}}
Nov 29 03:28:48 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-39f1dba3-f38c-468a-809a-11333ff841e1 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:fbebef3b-05f6-495b-8523-660f550506fb', 0)] {{(pid=75234) _get_connection_devices /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:830}}
Nov 29 03:28:48 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:28:48 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.017s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:28:48 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[b4952fe7-aae1-4f5e-a095-013cd7d5860c]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:fbebef3b-05f6-495b-8523-660f550506fb\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:28:48 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:28:48 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.027s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:28:48 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[158f72e8-fc6e-4e46-b41e-1cb34897c09c]: (4, ('tcp: [42] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0 (non-flash)\ntcp: [49] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:fbebef3b-05f6-495b-8523-660f550506fb (non-flash)\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:28:48 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-39f1dba3-f38c-468a-809a-11333ff841e1 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [42] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0 (non-flash)
Nov 29 03:28:48 user nova-compute[75234]: tcp: [49] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:fbebef3b-05f6-495b-8523-660f550506fb (non-flash)
Nov 29 03:28:48 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:28:48 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-39f1dba3-f38c-468a-809a-11333ff841e1 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] iscsi session list stdout=tcp: [42] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0 (non-flash)
Nov 29 03:28:48 user nova-compute[75234]: tcp: [49] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:fbebef3b-05f6-495b-8523-660f550506fb (non-flash)
Nov 29 03:28:48 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsi_session /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Nov 29 03:28:48 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-39f1dba3-f38c-468a-809a-11333ff841e1 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7f8a6838b6d0>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:fbebef3b-05f6-495b-8523-660f550506fb'): ({'sdb'}, set())}) {{(pid=75234) _get_connection_devices /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:863}}
Nov 29 03:28:48 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-39f1dba3-f38c-468a-809a-11333ff841e1 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Removing single pathed devices sdb {{(pid=75234) remove_connection /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:374}}
Nov 29 03:28:48 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:28:48 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.020s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:28:48 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[9339d7cf-d168-4f55-b484-6bf5c7778bd4]: (4, ('path checker states:\nup                  2\n\npaths: 0\nbusy: False\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:28:48 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdb {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:28:48 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdb" returned: 0 in 0.020s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:28:48 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[b31fb68a-3198-4a94-a7c5-0efcf47ffbc2]: (4, ('ok\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:28:48 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-39f1dba3-f38c-468a-809a-11333ff841e1 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Flushing IO for device /dev/sdb {{(pid=75234) flush_device_io /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:434}}
Nov 29 03:28:48 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdb {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:28:48 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdb" returned: 0 in 0.016s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:28:48 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[f95fa100-9b4e-4dd4-9281-1cc49c204233]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:28:48 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-39f1dba3-f38c-468a-809a-11333ff841e1 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Remove SCSI device /dev/sdb with /sys/block/sdb/device/delete {{(pid=75234) remove_scsi_device /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:148}}
Nov 29 03:28:48 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdb/device/delete {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:28:49 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdb/device/delete" returned: 0 in 0.043s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:28:49 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[eac3c8fb-43e3-440e-b4fd-58e5ee0230a0]: (4, ('1', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:28:49 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-39f1dba3-f38c-468a-809a-11333ff841e1 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Checking to see if SCSI volumes sdb have been removed. {{(pid=75234) wait_for_volumes_removal /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:156}}
Nov 29 03:28:49 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-39f1dba3-f38c-468a-809a-11333ff841e1 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] SCSI volumes sdb have been removed. {{(pid=75234) wait_for_volumes_removal /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:166}}
Nov 29 03:28:49 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-39f1dba3-f38c-468a-809a-11333ff841e1 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:fbebef3b-05f6-495b-8523-660f550506fb')] {{(pid=75234) _disconnect_connection /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Nov 29 03:28:49 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:fbebef3b-05f6-495b-8523-660f550506fb -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:28:49 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:fbebef3b-05f6-495b-8523-660f550506fb -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.019s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:28:49 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[51246574-f503-4657-8224-77957d7b0e2d]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:28:49 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-39f1dba3-f38c-468a-809a-11333ff841e1 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:28:49 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:fbebef3b-05f6-495b-8523-660f550506fb -p 172.16.0.220:3260 --logout {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:28:49 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:fbebef3b-05f6-495b-8523-660f550506fb -p 172.16.0.220:3260 --logout" returned: 0 in 0.061s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:28:49 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[1bd3730a-d549-4f01-8de0-0d6932190a10]: (4, ('Logging out of session [sid: 49, target: iqn.2010-10.org.openstack:fbebef3b-05f6-495b-8523-660f550506fb, portal: 172.16.0.220,3260]\nLogout of [sid: 49, target: iqn.2010-10.org.openstack:fbebef3b-05f6-495b-8523-660f550506fb, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:28:49 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-39f1dba3-f38c-468a-809a-11333ff841e1 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 49, target: iqn.2010-10.org.openstack:fbebef3b-05f6-495b-8523-660f550506fb, portal: 172.16.0.220,3260]
Nov 29 03:28:49 user nova-compute[75234]: Logout of [sid: 49, target: iqn.2010-10.org.openstack:fbebef3b-05f6-495b-8523-660f550506fb, portal: 172.16.0.220,3260] successful.
Nov 29 03:28:49 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:28:49 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:fbebef3b-05f6-495b-8523-660f550506fb -p 172.16.0.220:3260 --op delete {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:28:49 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:fbebef3b-05f6-495b-8523-660f550506fb -p 172.16.0.220:3260 --op delete" returned: 0 in 0.021s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:28:49 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[8091ded2-026c-4a68-aeab-7de28d7711c7]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:28:49 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-39f1dba3-f38c-468a-809a-11333ff841e1 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:28:49 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-39f1dba3-f38c-468a-809a-11333ff841e1 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.315s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:87}}
Nov 29 03:28:49 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-39f1dba3-f38c-468a-809a-11333ff841e1 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] <== disconnect_volume: return (316ms) None {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:203}}
Nov 29 03:28:49 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-39f1dba3-f38c-468a-809a-11333ff841e1 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] [instance: 499676de-0e76-45ae-864c-44e55e3d7c7f] Disconnected iSCSI Volume {{(pid=75234) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:79}}
Nov 29 03:28:49 user nova-compute[75234]: DEBUG nova.objects.instance [None req-39f1dba3-f38c-468a-809a-11333ff841e1 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Lazy-loading 'flavor' on Instance uuid 499676de-0e76-45ae-864c-44e55e3d7c7f {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:28:50 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-39f1dba3-f38c-468a-809a-11333ff841e1 tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Lock "499676de-0e76-45ae-864c-44e55e3d7c7f" "released" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: held 21.765s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:28:50 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:28:52 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-32ae2a8a-ca25-43e0-938a-c37c27d3eb6c tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Acquiring lock "5d7d744e-1a5f-40ab-b961-4b58c183efe0" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:28:52 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-32ae2a8a-ca25-43e0-938a-c37c27d3eb6c tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Lock "5d7d744e-1a5f-40ab-b961-4b58c183efe0" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:28:52 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-32ae2a8a-ca25-43e0-938a-c37c27d3eb6c tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Acquiring lock "5d7d744e-1a5f-40ab-b961-4b58c183efe0-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:28:52 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-32ae2a8a-ca25-43e0-938a-c37c27d3eb6c tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Lock "5d7d744e-1a5f-40ab-b961-4b58c183efe0-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:28:52 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-32ae2a8a-ca25-43e0-938a-c37c27d3eb6c tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Lock "5d7d744e-1a5f-40ab-b961-4b58c183efe0-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:28:52 user nova-compute[75234]: INFO nova.compute.manager [None req-32ae2a8a-ca25-43e0-938a-c37c27d3eb6c tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] [instance: 5d7d744e-1a5f-40ab-b961-4b58c183efe0] Terminating instance
Nov 29 03:28:52 user nova-compute[75234]: DEBUG nova.compute.manager [None req-32ae2a8a-ca25-43e0-938a-c37c27d3eb6c tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] [instance: 5d7d744e-1a5f-40ab-b961-4b58c183efe0] Start destroying the instance on the hypervisor. {{(pid=75234) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3140}}
Nov 29 03:28:52 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:28:52 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:28:52 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:28:52 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:28:52 user nova-compute[75234]: DEBUG nova.compute.manager [req-28772f00-3d31-461d-9387-34bd687da48e req-e4e91051-45e8-46a3-873e-5330107ccdc5 service nova] [instance: 5d7d744e-1a5f-40ab-b961-4b58c183efe0] Received event network-vif-unplugged-fa65b32f-b627-45fa-8d6d-d99012ca1262 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:28:52 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-28772f00-3d31-461d-9387-34bd687da48e req-e4e91051-45e8-46a3-873e-5330107ccdc5 service nova] Acquiring lock "5d7d744e-1a5f-40ab-b961-4b58c183efe0-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:28:52 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-28772f00-3d31-461d-9387-34bd687da48e req-e4e91051-45e8-46a3-873e-5330107ccdc5 service nova] Lock "5d7d744e-1a5f-40ab-b961-4b58c183efe0-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:28:52 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-28772f00-3d31-461d-9387-34bd687da48e req-e4e91051-45e8-46a3-873e-5330107ccdc5 service nova] Lock "5d7d744e-1a5f-40ab-b961-4b58c183efe0-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:28:52 user nova-compute[75234]: DEBUG nova.compute.manager [req-28772f00-3d31-461d-9387-34bd687da48e req-e4e91051-45e8-46a3-873e-5330107ccdc5 service nova] [instance: 5d7d744e-1a5f-40ab-b961-4b58c183efe0] No waiting events found dispatching network-vif-unplugged-fa65b32f-b627-45fa-8d6d-d99012ca1262 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:28:52 user nova-compute[75234]: DEBUG nova.compute.manager [req-28772f00-3d31-461d-9387-34bd687da48e req-e4e91051-45e8-46a3-873e-5330107ccdc5 service nova] [instance: 5d7d744e-1a5f-40ab-b961-4b58c183efe0] Received event network-vif-unplugged-fa65b32f-b627-45fa-8d6d-d99012ca1262 for instance with task_state deleting. {{(pid=75234) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10890}}
Nov 29 03:28:53 user nova-compute[75234]: INFO nova.virt.libvirt.driver [-] [instance: 5d7d744e-1a5f-40ab-b961-4b58c183efe0] Instance destroyed successfully.
Nov 29 03:28:53 user nova-compute[75234]: DEBUG nova.objects.instance [None req-32ae2a8a-ca25-43e0-938a-c37c27d3eb6c tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Lazy-loading 'resources' on Instance uuid 5d7d744e-1a5f-40ab-b961-4b58c183efe0 {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:28:53 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-32ae2a8a-ca25-43e0-938a-c37c27d3eb6c tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-11-29T03:23:45Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-TestMinimumBasicScenario-server-310577704',display_name='tempest-TestMinimumBasicScenario-server-310577704',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-testminimumbasicscenario-server-310577704',id=39,image_ref='7107c48b-05b9-4052-917d-814b721d169c',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBBTfFCQxMn5FN9HRzHRLmeLvGNhlvH8jkTHWDyEbYwE8hTZhRqGybcRgcmJ5xQB1zSttCcEaPSgc17S9QrgRI43nIUg3IFEEzmkOkz4/kzJTs5GShIWM4PGtgtnmC06Ugw==',key_name='tempest-TestMinimumBasicScenario-44553634',keypairs=<?>,launch_index=0,launched_at=2023-11-29T03:23:53Z,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='74f132d6666c47b8b43869df6fee6de3',ramdisk_id='',reservation_id='r-dj1tj6j2',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='7107c48b-05b9-4052-917d-814b721d169c',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',owner_project_name='tempest-TestMinimumBasicScenario-1635847764',owner_user_name='tempest-TestMinimumBasicScenario-1635847764-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-11-29T03:26:14Z,user_data=None,user_id='3a02618f56ea45898ab4783a4f198b5d',uuid=5d7d744e-1a5f-40ab-b961-4b58c183efe0,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "fa65b32f-b627-45fa-8d6d-d99012ca1262", "address": "fa:16:3e:2f:fa:59", "network": {"id": "6787e0a8-2980-4a2e-bf2a-583af955241d", "bridge": "br-int", "label": "tempest-TestMinimumBasicScenario-1273600998-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "74f132d6666c47b8b43869df6fee6de3", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapfa65b32f-b6", "ovs_interfaceid": "fa65b32f-b627-45fa-8d6d-d99012ca1262", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Nov 29 03:28:53 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-32ae2a8a-ca25-43e0-938a-c37c27d3eb6c tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Converting VIF {"id": "fa65b32f-b627-45fa-8d6d-d99012ca1262", "address": "fa:16:3e:2f:fa:59", "network": {"id": "6787e0a8-2980-4a2e-bf2a-583af955241d", "bridge": "br-int", "label": "tempest-TestMinimumBasicScenario-1273600998-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.3", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "74f132d6666c47b8b43869df6fee6de3", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapfa65b32f-b6", "ovs_interfaceid": "fa65b32f-b627-45fa-8d6d-d99012ca1262", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:28:53 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-32ae2a8a-ca25-43e0-938a-c37c27d3eb6c tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:2f:fa:59,bridge_name='br-int',has_traffic_filtering=True,id=fa65b32f-b627-45fa-8d6d-d99012ca1262,network=Network(6787e0a8-2980-4a2e-bf2a-583af955241d),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapfa65b32f-b6') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:28:53 user nova-compute[75234]: DEBUG os_vif [None req-32ae2a8a-ca25-43e0-938a-c37c27d3eb6c tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Unplugging vif VIFOpenVSwitch(active=True,address=fa:16:3e:2f:fa:59,bridge_name='br-int',has_traffic_filtering=True,id=fa65b32f-b627-45fa-8d6d-d99012ca1262,network=Network(6787e0a8-2980-4a2e-bf2a-583af955241d),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapfa65b32f-b6') {{(pid=75234) unplug /opt/stack/data/venv/lib/python3.10/site-packages/os_vif/__init__.py:109}}
Nov 29 03:28:53 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:28:53 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tapfa65b32f-b6, bridge=br-int, if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:28:53 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:28:53 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:28:53 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbDestroyCommand(_result=None, table=QoS, record=6ad2a50c-0354-48b2-9234-7766b33e3a56) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:28:53 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:28:53 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:28:53 user nova-compute[75234]: INFO os_vif [None req-32ae2a8a-ca25-43e0-938a-c37c27d3eb6c tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Successfully unplugged vif VIFOpenVSwitch(active=True,address=fa:16:3e:2f:fa:59,bridge_name='br-int',has_traffic_filtering=True,id=fa65b32f-b627-45fa-8d6d-d99012ca1262,network=Network(6787e0a8-2980-4a2e-bf2a-583af955241d),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapfa65b32f-b6')
Nov 29 03:28:53 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-32ae2a8a-ca25-43e0-938a-c37c27d3eb6c tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] [instance: 5d7d744e-1a5f-40ab-b961-4b58c183efe0] Deleting instance files /opt/stack/data/nova/instances/5d7d744e-1a5f-40ab-b961-4b58c183efe0_del
Nov 29 03:28:53 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-32ae2a8a-ca25-43e0-938a-c37c27d3eb6c tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] [instance: 5d7d744e-1a5f-40ab-b961-4b58c183efe0] Deletion of /opt/stack/data/nova/instances/5d7d744e-1a5f-40ab-b961-4b58c183efe0_del complete
Nov 29 03:28:53 user nova-compute[75234]: INFO nova.compute.manager [None req-32ae2a8a-ca25-43e0-938a-c37c27d3eb6c tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] [instance: 5d7d744e-1a5f-40ab-b961-4b58c183efe0] Took 0.88 seconds to destroy the instance on the hypervisor.
Nov 29 03:28:53 user nova-compute[75234]: DEBUG oslo.service.loopingcall [None req-32ae2a8a-ca25-43e0-938a-c37c27d3eb6c tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=75234) func /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/loopingcall.py:435}}
Nov 29 03:28:53 user nova-compute[75234]: DEBUG nova.compute.manager [-] [instance: 5d7d744e-1a5f-40ab-b961-4b58c183efe0] Deallocating network for instance {{(pid=75234) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2279}}
Nov 29 03:28:53 user nova-compute[75234]: DEBUG nova.network.neutron [-] [instance: 5d7d744e-1a5f-40ab-b961-4b58c183efe0] deallocate_for_instance() {{(pid=75234) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1802}}
Nov 29 03:28:53 user nova-compute[75234]: DEBUG nova.network.neutron [-] [instance: 5d7d744e-1a5f-40ab-b961-4b58c183efe0] Updating instance_info_cache with network_info: [] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:28:53 user nova-compute[75234]: INFO nova.compute.manager [-] [instance: 5d7d744e-1a5f-40ab-b961-4b58c183efe0] Took 0.49 seconds to deallocate network for instance.
Nov 29 03:28:53 user nova-compute[75234]: DEBUG nova.compute.manager [req-8eec727e-824b-4a36-8024-dbb7081dd99a req-3c23bb5a-c91e-466f-a759-51889822786a service nova] [instance: 5d7d744e-1a5f-40ab-b961-4b58c183efe0] Received event network-vif-deleted-fa65b32f-b627-45fa-8d6d-d99012ca1262 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:28:53 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-32ae2a8a-ca25-43e0-938a-c37c27d3eb6c tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:28:53 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-32ae2a8a-ca25-43e0-938a-c37c27d3eb6c tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:28:53 user nova-compute[75234]: DEBUG nova.compute.provider_tree [None req-32ae2a8a-ca25-43e0-938a-c37c27d3eb6c tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Inventory has not changed in ProviderTree for provider: 4d291220-7c37-4839-a0e3-b23c66502b74 {{(pid=75234) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Nov 29 03:28:53 user nova-compute[75234]: DEBUG nova.scheduler.client.report [None req-32ae2a8a-ca25-43e0-938a-c37c27d3eb6c tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Inventory has not changed for provider 4d291220-7c37-4839-a0e3-b23c66502b74 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=75234) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Nov 29 03:28:53 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-32ae2a8a-ca25-43e0-938a-c37c27d3eb6c tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.193s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:28:53 user nova-compute[75234]: INFO nova.scheduler.client.report [None req-32ae2a8a-ca25-43e0-938a-c37c27d3eb6c tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Deleted allocations for instance 5d7d744e-1a5f-40ab-b961-4b58c183efe0
Nov 29 03:28:54 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-32ae2a8a-ca25-43e0-938a-c37c27d3eb6c tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Lock "5d7d744e-1a5f-40ab-b961-4b58c183efe0" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 1.768s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:28:54 user nova-compute[75234]: DEBUG nova.compute.manager [req-936ef86d-eb5c-4ac0-8860-2004dda08ca6 req-250c7f21-c7ae-4dab-b087-50e27ef5f7d9 service nova] [instance: 5d7d744e-1a5f-40ab-b961-4b58c183efe0] Received event network-vif-plugged-fa65b32f-b627-45fa-8d6d-d99012ca1262 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:28:54 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-936ef86d-eb5c-4ac0-8860-2004dda08ca6 req-250c7f21-c7ae-4dab-b087-50e27ef5f7d9 service nova] Acquiring lock "5d7d744e-1a5f-40ab-b961-4b58c183efe0-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:28:54 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-936ef86d-eb5c-4ac0-8860-2004dda08ca6 req-250c7f21-c7ae-4dab-b087-50e27ef5f7d9 service nova] Lock "5d7d744e-1a5f-40ab-b961-4b58c183efe0-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:28:54 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-936ef86d-eb5c-4ac0-8860-2004dda08ca6 req-250c7f21-c7ae-4dab-b087-50e27ef5f7d9 service nova] Lock "5d7d744e-1a5f-40ab-b961-4b58c183efe0-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:28:54 user nova-compute[75234]: DEBUG nova.compute.manager [req-936ef86d-eb5c-4ac0-8860-2004dda08ca6 req-250c7f21-c7ae-4dab-b087-50e27ef5f7d9 service nova] [instance: 5d7d744e-1a5f-40ab-b961-4b58c183efe0] No waiting events found dispatching network-vif-plugged-fa65b32f-b627-45fa-8d6d-d99012ca1262 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:28:54 user nova-compute[75234]: WARNING nova.compute.manager [req-936ef86d-eb5c-4ac0-8860-2004dda08ca6 req-250c7f21-c7ae-4dab-b087-50e27ef5f7d9 service nova] [instance: 5d7d744e-1a5f-40ab-b961-4b58c183efe0] Received unexpected event network-vif-plugged-fa65b32f-b627-45fa-8d6d-d99012ca1262 for instance with vm_state deleted and task_state None.
Nov 29 03:28:58 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:29:03 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:29:05 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:29:08 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:29:08 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:29:08 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:29:08 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:29:08 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=75234) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10541}}
Nov 29 03:29:10 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager.update_available_resource {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:29:10 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:29:10 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:29:10 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:29:10 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Auditing locally available compute resources for user (node: user) {{(pid=75234) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:907}}
Nov 29 03:29:10 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/65d48c93-b6da-4f51-9d5a-0676dda62a60/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:29:10 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/65d48c93-b6da-4f51-9d5a-0676dda62a60/disk --force-share --output=json" returned: 0 in 0.177s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:29:10 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/65d48c93-b6da-4f51-9d5a-0676dda62a60/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:29:10 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:29:10 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/65d48c93-b6da-4f51-9d5a-0676dda62a60/disk --force-share --output=json" returned: 0 in 0.191s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:29:10 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] skipping disk /dev/sdc (vda) as it is a volume {{(pid=75234) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11213}}
Nov 29 03:29:10 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/499676de-0e76-45ae-864c-44e55e3d7c7f/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:29:11 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/499676de-0e76-45ae-864c-44e55e3d7c7f/disk --force-share --output=json" returned: 0 in 0.171s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:29:11 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/499676de-0e76-45ae-864c-44e55e3d7c7f/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:29:11 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/499676de-0e76-45ae-864c-44e55e3d7c7f/disk --force-share --output=json" returned: 0 in 0.178s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:29:11 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:29:11 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:29:11 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Hypervisor/Node resource view: name=user free_ram=8645MB free_disk=25.101726531982422GB free_vcpus=9 pci_devices=[{"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}] {{(pid=75234) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1106}}
Nov 29 03:29:11 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:29:11 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:29:11 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance 65d48c93-b6da-4f51-9d5a-0676dda62a60 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 512, 'DISK_GB': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:29:11 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance d5096ca2-d005-45ac-a487-11a810446859 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 512}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:29:11 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance 499676de-0e76-45ae-864c-44e55e3d7c7f actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 512, 'DISK_GB': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:29:11 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Total usable vcpus: 12, total allocated vcpus: 3 {{(pid=75234) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1129}}
Nov 29 03:29:11 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Final resource view: name=user phys_ram=16011MB used_ram=2048MB phys_disk=40GB used_disk=2GB total_vcpus=12 used_vcpus=3 pci_stats=[] {{(pid=75234) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1138}}
Nov 29 03:29:11 user nova-compute[75234]: DEBUG nova.compute.provider_tree [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Inventory has not changed in ProviderTree for provider: 4d291220-7c37-4839-a0e3-b23c66502b74 {{(pid=75234) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Nov 29 03:29:11 user nova-compute[75234]: DEBUG nova.scheduler.client.report [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Inventory has not changed for provider 4d291220-7c37-4839-a0e3-b23c66502b74 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=75234) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Nov 29 03:29:11 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Compute_service record updated for user:user {{(pid=75234) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:1067}}
Nov 29 03:29:11 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.264s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:29:12 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:29:12 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:29:12 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Starting heal instance info cache {{(pid=75234) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9922}}
Nov 29 03:29:13 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Acquiring lock "refresh_cache-d5096ca2-d005-45ac-a487-11a810446859" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:29:13 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Acquired lock "refresh_cache-d5096ca2-d005-45ac-a487-11a810446859" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:29:13 user nova-compute[75234]: DEBUG nova.network.neutron [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] [instance: d5096ca2-d005-45ac-a487-11a810446859] Forcefully refreshing network info cache for instance {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2003}}
Nov 29 03:29:13 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:29:13 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:29:13 user nova-compute[75234]: DEBUG nova.network.neutron [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] [instance: d5096ca2-d005-45ac-a487-11a810446859] Updating instance_info_cache with network_info: [{"id": "9266d021-6e83-4bfa-8151-66c5182e61a0", "address": "fa:16:3e:ec:fb:67", "network": {"id": "4d086e08-7798-4cfe-9365-86be6f0a52d2", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-305624008-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "db00bcd0c82c44f1bda739e9857f6e84", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap9266d021-6e", "ovs_interfaceid": "9266d021-6e83-4bfa-8151-66c5182e61a0", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:29:13 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Releasing lock "refresh_cache-d5096ca2-d005-45ac-a487-11a810446859" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:29:13 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] [instance: d5096ca2-d005-45ac-a487-11a810446859] Updated the network info_cache for instance {{(pid=75234) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9993}}
Nov 29 03:29:13 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:29:14 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:29:15 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:29:16 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:29:16 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:29:17 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:29:17 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:29:18 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:29:18 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-fcbadf8d-b262-4f0b-9087-89fbd84cab0e tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Acquiring lock "499676de-0e76-45ae-864c-44e55e3d7c7f" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:29:18 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-fcbadf8d-b262-4f0b-9087-89fbd84cab0e tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Lock "499676de-0e76-45ae-864c-44e55e3d7c7f" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:29:18 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-fcbadf8d-b262-4f0b-9087-89fbd84cab0e tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Acquiring lock "499676de-0e76-45ae-864c-44e55e3d7c7f-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:29:18 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-fcbadf8d-b262-4f0b-9087-89fbd84cab0e tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Lock "499676de-0e76-45ae-864c-44e55e3d7c7f-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:29:18 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-fcbadf8d-b262-4f0b-9087-89fbd84cab0e tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Lock "499676de-0e76-45ae-864c-44e55e3d7c7f-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:29:18 user nova-compute[75234]: INFO nova.compute.manager [None req-fcbadf8d-b262-4f0b-9087-89fbd84cab0e tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] [instance: 499676de-0e76-45ae-864c-44e55e3d7c7f] Terminating instance
Nov 29 03:29:18 user nova-compute[75234]: DEBUG nova.compute.manager [None req-fcbadf8d-b262-4f0b-9087-89fbd84cab0e tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] [instance: 499676de-0e76-45ae-864c-44e55e3d7c7f] Start destroying the instance on the hypervisor. {{(pid=75234) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3140}}
Nov 29 03:29:18 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:29:18 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:29:18 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:29:18 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:29:18 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:29:19 user nova-compute[75234]: DEBUG nova.compute.manager [req-8ff1d138-08a6-4b3a-8f9f-4f2b9af52497 req-fb93277f-0f46-44e4-b553-e0d19435df37 service nova] [instance: 499676de-0e76-45ae-864c-44e55e3d7c7f] Received event network-vif-unplugged-720988c2-e733-41cb-b3ea-c215cfe46bd6 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:29:19 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-8ff1d138-08a6-4b3a-8f9f-4f2b9af52497 req-fb93277f-0f46-44e4-b553-e0d19435df37 service nova] Acquiring lock "499676de-0e76-45ae-864c-44e55e3d7c7f-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:29:19 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-8ff1d138-08a6-4b3a-8f9f-4f2b9af52497 req-fb93277f-0f46-44e4-b553-e0d19435df37 service nova] Lock "499676de-0e76-45ae-864c-44e55e3d7c7f-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:29:19 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-8ff1d138-08a6-4b3a-8f9f-4f2b9af52497 req-fb93277f-0f46-44e4-b553-e0d19435df37 service nova] Lock "499676de-0e76-45ae-864c-44e55e3d7c7f-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:29:19 user nova-compute[75234]: DEBUG nova.compute.manager [req-8ff1d138-08a6-4b3a-8f9f-4f2b9af52497 req-fb93277f-0f46-44e4-b553-e0d19435df37 service nova] [instance: 499676de-0e76-45ae-864c-44e55e3d7c7f] No waiting events found dispatching network-vif-unplugged-720988c2-e733-41cb-b3ea-c215cfe46bd6 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:29:19 user nova-compute[75234]: DEBUG nova.compute.manager [req-8ff1d138-08a6-4b3a-8f9f-4f2b9af52497 req-fb93277f-0f46-44e4-b553-e0d19435df37 service nova] [instance: 499676de-0e76-45ae-864c-44e55e3d7c7f] Received event network-vif-unplugged-720988c2-e733-41cb-b3ea-c215cfe46bd6 for instance with task_state deleting. {{(pid=75234) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10890}}
Nov 29 03:29:19 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:29:19 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:29:19 user nova-compute[75234]: INFO nova.virt.libvirt.driver [-] [instance: 499676de-0e76-45ae-864c-44e55e3d7c7f] Instance destroyed successfully.
Nov 29 03:29:19 user nova-compute[75234]: DEBUG nova.objects.instance [None req-fcbadf8d-b262-4f0b-9087-89fbd84cab0e tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Lazy-loading 'resources' on Instance uuid 499676de-0e76-45ae-864c-44e55e3d7c7f {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:29:19 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-fcbadf8d-b262-4f0b-9087-89fbd84cab0e tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-11-29T03:26:23Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-AttachVolumeNegativeTest-server-272620904',display_name='tempest-AttachVolumeNegativeTest-server-272620904',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-attachvolumenegativetest-server-272620904',id=42,image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBAeV+xn9UGsWgfasuo5ci4YCLCfVHxnk9mWnUGlN4CXUvM0yE0RrqSC8l9glhTxJAVaU31Y7rLe4uk5w24n4N4cYiV7rwY+sOMhAYj/kLr660fRVmge9Wnqb+j9IA4vy5Q==',key_name='tempest-keypair-374385187',keypairs=<?>,launch_index=0,launched_at=2023-11-29T03:26:30Z,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='315732b6e3ee44d584b8e510ef199a90',ramdisk_id='',reservation_id='r-brbd5rm9',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-AttachVolumeNegativeTest-1582496806',owner_user_name='tempest-AttachVolumeNegativeTest-1582496806-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-11-29T03:26:31Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='66686114450f4f31b3a917667a152d8b',uuid=499676de-0e76-45ae-864c-44e55e3d7c7f,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "720988c2-e733-41cb-b3ea-c215cfe46bd6", "address": "fa:16:3e:0d:b8:b3", "network": {"id": "0f289171-3d73-4054-95a0-2515b4f206a0", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-1787960284-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.10", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "315732b6e3ee44d584b8e510ef199a90", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap720988c2-e7", "ovs_interfaceid": "720988c2-e733-41cb-b3ea-c215cfe46bd6", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Nov 29 03:29:19 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-fcbadf8d-b262-4f0b-9087-89fbd84cab0e tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Converting VIF {"id": "720988c2-e733-41cb-b3ea-c215cfe46bd6", "address": "fa:16:3e:0d:b8:b3", "network": {"id": "0f289171-3d73-4054-95a0-2515b4f206a0", "bridge": "br-int", "label": "tempest-AttachVolumeNegativeTest-1787960284-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.4", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.10", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "315732b6e3ee44d584b8e510ef199a90", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap720988c2-e7", "ovs_interfaceid": "720988c2-e733-41cb-b3ea-c215cfe46bd6", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:29:19 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-fcbadf8d-b262-4f0b-9087-89fbd84cab0e tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:0d:b8:b3,bridge_name='br-int',has_traffic_filtering=True,id=720988c2-e733-41cb-b3ea-c215cfe46bd6,network=Network(0f289171-3d73-4054-95a0-2515b4f206a0),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap720988c2-e7') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:29:19 user nova-compute[75234]: DEBUG os_vif [None req-fcbadf8d-b262-4f0b-9087-89fbd84cab0e tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Unplugging vif VIFOpenVSwitch(active=True,address=fa:16:3e:0d:b8:b3,bridge_name='br-int',has_traffic_filtering=True,id=720988c2-e733-41cb-b3ea-c215cfe46bd6,network=Network(0f289171-3d73-4054-95a0-2515b4f206a0),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap720988c2-e7') {{(pid=75234) unplug /opt/stack/data/venv/lib/python3.10/site-packages/os_vif/__init__.py:109}}
Nov 29 03:29:19 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:29:19 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tap720988c2-e7, bridge=br-int, if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:29:19 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:29:19 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:29:19 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:29:19 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbDestroyCommand(_result=None, table=QoS, record=0ad6b090-4acd-4240-93c8-d0ce6745dbb7) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:29:19 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:29:19 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:29:19 user nova-compute[75234]: INFO os_vif [None req-fcbadf8d-b262-4f0b-9087-89fbd84cab0e tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Successfully unplugged vif VIFOpenVSwitch(active=True,address=fa:16:3e:0d:b8:b3,bridge_name='br-int',has_traffic_filtering=True,id=720988c2-e733-41cb-b3ea-c215cfe46bd6,network=Network(0f289171-3d73-4054-95a0-2515b4f206a0),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap720988c2-e7')
Nov 29 03:29:19 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-fcbadf8d-b262-4f0b-9087-89fbd84cab0e tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] [instance: 499676de-0e76-45ae-864c-44e55e3d7c7f] Deleting instance files /opt/stack/data/nova/instances/499676de-0e76-45ae-864c-44e55e3d7c7f_del
Nov 29 03:29:19 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-fcbadf8d-b262-4f0b-9087-89fbd84cab0e tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] [instance: 499676de-0e76-45ae-864c-44e55e3d7c7f] Deletion of /opt/stack/data/nova/instances/499676de-0e76-45ae-864c-44e55e3d7c7f_del complete
Nov 29 03:29:19 user nova-compute[75234]: INFO nova.compute.manager [None req-fcbadf8d-b262-4f0b-9087-89fbd84cab0e tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] [instance: 499676de-0e76-45ae-864c-44e55e3d7c7f] Took 0.90 seconds to destroy the instance on the hypervisor.
Nov 29 03:29:19 user nova-compute[75234]: DEBUG oslo.service.loopingcall [None req-fcbadf8d-b262-4f0b-9087-89fbd84cab0e tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=75234) func /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/loopingcall.py:435}}
Nov 29 03:29:19 user nova-compute[75234]: DEBUG nova.compute.manager [-] [instance: 499676de-0e76-45ae-864c-44e55e3d7c7f] Deallocating network for instance {{(pid=75234) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2279}}
Nov 29 03:29:19 user nova-compute[75234]: DEBUG nova.network.neutron [-] [instance: 499676de-0e76-45ae-864c-44e55e3d7c7f] deallocate_for_instance() {{(pid=75234) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1802}}
Nov 29 03:29:19 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:29:20 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:29:20 user nova-compute[75234]: DEBUG nova.network.neutron [-] [instance: 499676de-0e76-45ae-864c-44e55e3d7c7f] Updating instance_info_cache with network_info: [] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:29:20 user nova-compute[75234]: INFO nova.compute.manager [-] [instance: 499676de-0e76-45ae-864c-44e55e3d7c7f] Took 0.93 seconds to deallocate network for instance.
Nov 29 03:29:20 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-fcbadf8d-b262-4f0b-9087-89fbd84cab0e tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:29:20 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-fcbadf8d-b262-4f0b-9087-89fbd84cab0e tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:29:20 user nova-compute[75234]: DEBUG nova.compute.provider_tree [None req-fcbadf8d-b262-4f0b-9087-89fbd84cab0e tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Inventory has not changed in ProviderTree for provider: 4d291220-7c37-4839-a0e3-b23c66502b74 {{(pid=75234) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Nov 29 03:29:20 user nova-compute[75234]: DEBUG nova.scheduler.client.report [None req-fcbadf8d-b262-4f0b-9087-89fbd84cab0e tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Inventory has not changed for provider 4d291220-7c37-4839-a0e3-b23c66502b74 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=75234) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Nov 29 03:29:20 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-fcbadf8d-b262-4f0b-9087-89fbd84cab0e tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.155s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:29:20 user nova-compute[75234]: INFO nova.scheduler.client.report [None req-fcbadf8d-b262-4f0b-9087-89fbd84cab0e tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Deleted allocations for instance 499676de-0e76-45ae-864c-44e55e3d7c7f
Nov 29 03:29:20 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-fcbadf8d-b262-4f0b-9087-89fbd84cab0e tempest-AttachVolumeNegativeTest-1582496806 tempest-AttachVolumeNegativeTest-1582496806-project-member] Lock "499676de-0e76-45ae-864c-44e55e3d7c7f" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 2.210s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:29:21 user nova-compute[75234]: DEBUG nova.compute.manager [req-23aa2184-414a-490b-b3b2-2a4b2dd84c7f req-f322755c-3620-4370-baec-871573c8ea54 service nova] [instance: 499676de-0e76-45ae-864c-44e55e3d7c7f] Received event network-vif-plugged-720988c2-e733-41cb-b3ea-c215cfe46bd6 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:29:21 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-23aa2184-414a-490b-b3b2-2a4b2dd84c7f req-f322755c-3620-4370-baec-871573c8ea54 service nova] Acquiring lock "499676de-0e76-45ae-864c-44e55e3d7c7f-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:29:21 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-23aa2184-414a-490b-b3b2-2a4b2dd84c7f req-f322755c-3620-4370-baec-871573c8ea54 service nova] Lock "499676de-0e76-45ae-864c-44e55e3d7c7f-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:29:21 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-23aa2184-414a-490b-b3b2-2a4b2dd84c7f req-f322755c-3620-4370-baec-871573c8ea54 service nova] Lock "499676de-0e76-45ae-864c-44e55e3d7c7f-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:29:21 user nova-compute[75234]: DEBUG nova.compute.manager [req-23aa2184-414a-490b-b3b2-2a4b2dd84c7f req-f322755c-3620-4370-baec-871573c8ea54 service nova] [instance: 499676de-0e76-45ae-864c-44e55e3d7c7f] No waiting events found dispatching network-vif-plugged-720988c2-e733-41cb-b3ea-c215cfe46bd6 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:29:21 user nova-compute[75234]: WARNING nova.compute.manager [req-23aa2184-414a-490b-b3b2-2a4b2dd84c7f req-f322755c-3620-4370-baec-871573c8ea54 service nova] [instance: 499676de-0e76-45ae-864c-44e55e3d7c7f] Received unexpected event network-vif-plugged-720988c2-e733-41cb-b3ea-c215cfe46bd6 for instance with vm_state deleted and task_state None.
Nov 29 03:29:21 user nova-compute[75234]: DEBUG nova.compute.manager [req-23aa2184-414a-490b-b3b2-2a4b2dd84c7f req-f322755c-3620-4370-baec-871573c8ea54 service nova] [instance: 499676de-0e76-45ae-864c-44e55e3d7c7f] Received event network-vif-deleted-720988c2-e733-41cb-b3ea-c215cfe46bd6 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:29:22 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:29:22 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:29:24 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:29:27 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:29:28 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-59d92465-53a2-4fe5-a842-2343f91e5e6b tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Acquiring lock "abb01889-df4b-44a6-9827-87056d7ca125" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:29:28 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-59d92465-53a2-4fe5-a842-2343f91e5e6b tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Lock "abb01889-df4b-44a6-9827-87056d7ca125" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:29:28 user nova-compute[75234]: DEBUG nova.compute.manager [None req-59d92465-53a2-4fe5-a842-2343f91e5e6b tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] [instance: abb01889-df4b-44a6-9827-87056d7ca125] Starting instance... {{(pid=75234) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2422}}
Nov 29 03:29:29 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:29:29 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-59d92465-53a2-4fe5-a842-2343f91e5e6b tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:29:29 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-59d92465-53a2-4fe5-a842-2343f91e5e6b tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:29:29 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-59d92465-53a2-4fe5-a842-2343f91e5e6b tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=75234) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2366}}
Nov 29 03:29:29 user nova-compute[75234]: INFO nova.compute.claims [None req-59d92465-53a2-4fe5-a842-2343f91e5e6b tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] [instance: abb01889-df4b-44a6-9827-87056d7ca125] Claim successful on node user
Nov 29 03:29:29 user nova-compute[75234]: DEBUG nova.compute.provider_tree [None req-59d92465-53a2-4fe5-a842-2343f91e5e6b tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Inventory has not changed in ProviderTree for provider: 4d291220-7c37-4839-a0e3-b23c66502b74 {{(pid=75234) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Nov 29 03:29:29 user nova-compute[75234]: DEBUG nova.scheduler.client.report [None req-59d92465-53a2-4fe5-a842-2343f91e5e6b tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Inventory has not changed for provider 4d291220-7c37-4839-a0e3-b23c66502b74 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=75234) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Nov 29 03:29:29 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-59d92465-53a2-4fe5-a842-2343f91e5e6b tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.300s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:29:29 user nova-compute[75234]: DEBUG nova.compute.manager [None req-59d92465-53a2-4fe5-a842-2343f91e5e6b tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] [instance: abb01889-df4b-44a6-9827-87056d7ca125] Start building networks asynchronously for instance. {{(pid=75234) _build_resources /opt/stack/nova/nova/compute/manager.py:2819}}
Nov 29 03:29:29 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-dda9b2f7-7676-4d13-99ef-fdc6b1e3b994 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] Acquiring lock "01d62e54-9957-40b1-8dfe-1783b64ffabf" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:29:29 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-dda9b2f7-7676-4d13-99ef-fdc6b1e3b994 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] Lock "01d62e54-9957-40b1-8dfe-1783b64ffabf" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:29:29 user nova-compute[75234]: DEBUG nova.compute.manager [None req-dda9b2f7-7676-4d13-99ef-fdc6b1e3b994 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] [instance: 01d62e54-9957-40b1-8dfe-1783b64ffabf] Starting instance... {{(pid=75234) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2422}}
Nov 29 03:29:29 user nova-compute[75234]: DEBUG nova.compute.manager [None req-59d92465-53a2-4fe5-a842-2343f91e5e6b tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] [instance: abb01889-df4b-44a6-9827-87056d7ca125] Allocating IP information in the background. {{(pid=75234) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1972}}
Nov 29 03:29:29 user nova-compute[75234]: DEBUG nova.network.neutron [None req-59d92465-53a2-4fe5-a842-2343f91e5e6b tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] [instance: abb01889-df4b-44a6-9827-87056d7ca125] allocate_for_instance() {{(pid=75234) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1156}}
Nov 29 03:29:29 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-59d92465-53a2-4fe5-a842-2343f91e5e6b tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] [instance: abb01889-df4b-44a6-9827-87056d7ca125] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Nov 29 03:29:29 user nova-compute[75234]: DEBUG nova.compute.manager [None req-59d92465-53a2-4fe5-a842-2343f91e5e6b tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] [instance: abb01889-df4b-44a6-9827-87056d7ca125] Start building block device mappings for instance. {{(pid=75234) _build_resources /opt/stack/nova/nova/compute/manager.py:2854}}
Nov 29 03:29:29 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:29:29 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-dda9b2f7-7676-4d13-99ef-fdc6b1e3b994 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:29:29 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-dda9b2f7-7676-4d13-99ef-fdc6b1e3b994 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:29:29 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-dda9b2f7-7676-4d13-99ef-fdc6b1e3b994 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=75234) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2366}}
Nov 29 03:29:29 user nova-compute[75234]: INFO nova.compute.claims [None req-dda9b2f7-7676-4d13-99ef-fdc6b1e3b994 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] [instance: 01d62e54-9957-40b1-8dfe-1783b64ffabf] Claim successful on node user
Nov 29 03:29:29 user nova-compute[75234]: DEBUG nova.compute.manager [None req-59d92465-53a2-4fe5-a842-2343f91e5e6b tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] [instance: abb01889-df4b-44a6-9827-87056d7ca125] Start spawning the instance on the hypervisor. {{(pid=75234) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2628}}
Nov 29 03:29:29 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-59d92465-53a2-4fe5-a842-2343f91e5e6b tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] [instance: abb01889-df4b-44a6-9827-87056d7ca125] Creating instance directory {{(pid=75234) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4727}}
Nov 29 03:29:29 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-59d92465-53a2-4fe5-a842-2343f91e5e6b tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] [instance: abb01889-df4b-44a6-9827-87056d7ca125] Creating image(s)
Nov 29 03:29:29 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-59d92465-53a2-4fe5-a842-2343f91e5e6b tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Acquiring lock "/opt/stack/data/nova/instances/abb01889-df4b-44a6-9827-87056d7ca125/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:29:29 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-59d92465-53a2-4fe5-a842-2343f91e5e6b tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Lock "/opt/stack/data/nova/instances/abb01889-df4b-44a6-9827-87056d7ca125/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:29:29 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-59d92465-53a2-4fe5-a842-2343f91e5e6b tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Lock "/opt/stack/data/nova/instances/abb01889-df4b-44a6-9827-87056d7ca125/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.009s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:29:29 user nova-compute[75234]: DEBUG nova.policy [None req-59d92465-53a2-4fe5-a842-2343f91e5e6b tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '877aa877f13a4f84ac4725278010aea3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'db00bcd0c82c44f1bda739e9857f6e84', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=75234) authorize /opt/stack/nova/nova/policy.py:203}}
Nov 29 03:29:29 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-59d92465-53a2-4fe5-a842-2343f91e5e6b tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:29:29 user nova-compute[75234]: DEBUG nova.compute.provider_tree [None req-dda9b2f7-7676-4d13-99ef-fdc6b1e3b994 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] Inventory has not changed in ProviderTree for provider: 4d291220-7c37-4839-a0e3-b23c66502b74 {{(pid=75234) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Nov 29 03:29:29 user nova-compute[75234]: DEBUG nova.scheduler.client.report [None req-dda9b2f7-7676-4d13-99ef-fdc6b1e3b994 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] Inventory has not changed for provider 4d291220-7c37-4839-a0e3-b23c66502b74 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=75234) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Nov 29 03:29:29 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-59d92465-53a2-4fe5-a842-2343f91e5e6b tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json" returned: 0 in 0.224s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:29:29 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-59d92465-53a2-4fe5-a842-2343f91e5e6b tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Acquiring lock "80686e29dc9db6600d03e85cb62af298e12d7918" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:29:29 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-59d92465-53a2-4fe5-a842-2343f91e5e6b tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Lock "80686e29dc9db6600d03e85cb62af298e12d7918" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.002s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:29:29 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-59d92465-53a2-4fe5-a842-2343f91e5e6b tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:29:29 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-dda9b2f7-7676-4d13-99ef-fdc6b1e3b994 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.409s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:29:29 user nova-compute[75234]: DEBUG nova.compute.manager [None req-dda9b2f7-7676-4d13-99ef-fdc6b1e3b994 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] [instance: 01d62e54-9957-40b1-8dfe-1783b64ffabf] Start building networks asynchronously for instance. {{(pid=75234) _build_resources /opt/stack/nova/nova/compute/manager.py:2819}}
Nov 29 03:29:29 user nova-compute[75234]: DEBUG nova.compute.manager [None req-dda9b2f7-7676-4d13-99ef-fdc6b1e3b994 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] [instance: 01d62e54-9957-40b1-8dfe-1783b64ffabf] Allocating IP information in the background. {{(pid=75234) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1972}}
Nov 29 03:29:29 user nova-compute[75234]: DEBUG nova.network.neutron [None req-dda9b2f7-7676-4d13-99ef-fdc6b1e3b994 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] [instance: 01d62e54-9957-40b1-8dfe-1783b64ffabf] allocate_for_instance() {{(pid=75234) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1156}}
Nov 29 03:29:29 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-dda9b2f7-7676-4d13-99ef-fdc6b1e3b994 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] [instance: 01d62e54-9957-40b1-8dfe-1783b64ffabf] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Nov 29 03:29:30 user nova-compute[75234]: DEBUG nova.compute.manager [None req-dda9b2f7-7676-4d13-99ef-fdc6b1e3b994 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] [instance: 01d62e54-9957-40b1-8dfe-1783b64ffabf] Start building block device mappings for instance. {{(pid=75234) _build_resources /opt/stack/nova/nova/compute/manager.py:2854}}
Nov 29 03:29:30 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-59d92465-53a2-4fe5-a842-2343f91e5e6b tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json" returned: 0 in 0.200s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:29:30 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-59d92465-53a2-4fe5-a842-2343f91e5e6b tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918,backing_fmt=raw /opt/stack/data/nova/instances/abb01889-df4b-44a6-9827-87056d7ca125/disk 1073741824 {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:29:30 user nova-compute[75234]: INFO nova.virt.block_device [None req-dda9b2f7-7676-4d13-99ef-fdc6b1e3b994 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] [instance: 01d62e54-9957-40b1-8dfe-1783b64ffabf] Booting with volume 43e92d4a-7e98-4777-8fa5-3cec396b86ce at /dev/vda
Nov 29 03:29:30 user nova-compute[75234]: DEBUG nova.policy [None req-dda9b2f7-7676-4d13-99ef-fdc6b1e3b994 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '5728da78dc074c1993a8a2cc9b821ed6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '53e55fcb8cd74895baec6f9528770bbc', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=75234) authorize /opt/stack/nova/nova/policy.py:203}}
Nov 29 03:29:30 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-59d92465-53a2-4fe5-a842-2343f91e5e6b tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918,backing_fmt=raw /opt/stack/data/nova/instances/abb01889-df4b-44a6-9827-87056d7ca125/disk 1073741824" returned: 0 in 0.063s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:29:30 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-59d92465-53a2-4fe5-a842-2343f91e5e6b tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Lock "80686e29dc9db6600d03e85cb62af298e12d7918" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.267s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:29:30 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-59d92465-53a2-4fe5-a842-2343f91e5e6b tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:29:30 user nova-compute[75234]: DEBUG os_brick.utils [None req-dda9b2f7-7676-4d13-99ef-fdc6b1e3b994 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:176}}
Nov 29 03:29:30 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:29:30 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.014s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:29:30 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[74e9cd5b-dc04-4878-a226-1944fb6e6fa5]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:dd1368c5ba2c\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:29:30 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt -v / -n -o SOURCE {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:29:30 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt -v / -n -o SOURCE" returned: 0 in 0.017s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:29:30 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[b331ddaa-92c6-4ebb-91fc-2cb2de3da34a]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:29:30 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:29:30 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.020s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:29:30 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[944e00ed-6c47-4acc-9a61-234baeec0c91]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:29:30 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[f3d89c4b-eab3-4def-aa35-0bc7beb358e0]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:29:30 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-dda9b2f7-7676-4d13-99ef-fdc6b1e3b994 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] Running cmd (subprocess): nvme version {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:29:30 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-dda9b2f7-7676-4d13-99ef-fdc6b1e3b994 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] 'nvme version' failed. Not Retrying. {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:473}}
Nov 29 03:29:30 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.nvmeof [None req-dda9b2f7-7676-4d13-99ef-fdc6b1e3b994 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] nvme not present on system {{(pid=75234) nvme_present /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/nvmeof.py:756}}
Nov 29 03:29:30 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-59d92465-53a2-4fe5-a842-2343f91e5e6b tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json" returned: 0 in 0.204s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:29:30 user nova-compute[75234]: DEBUG nova.virt.disk.api [None req-59d92465-53a2-4fe5-a842-2343f91e5e6b tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Checking if we can resize image /opt/stack/data/nova/instances/abb01889-df4b-44a6-9827-87056d7ca125/disk. size=1073741824 {{(pid=75234) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:166}}
Nov 29 03:29:30 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-59d92465-53a2-4fe5-a842-2343f91e5e6b tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/abb01889-df4b-44a6-9827-87056d7ca125/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:29:30 user nova-compute[75234]: DEBUG nova.network.neutron [None req-59d92465-53a2-4fe5-a842-2343f91e5e6b tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] [instance: abb01889-df4b-44a6-9827-87056d7ca125] Successfully created port: fec40d24-caf1-48f7-8565-8fffdbdad60e {{(pid=75234) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:548}}
Nov 29 03:29:30 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:29:30 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:473}}
Nov 29 03:29:30 user nova-compute[75234]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Nov 29 03:29:30 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[125a8794-ab40-498c-b6c1-88db7ff5edad]: (4, '') {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:29:30 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.lightos [None req-dda9b2f7-7676-4d13-99ef-fdc6b1e3b994 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=75234) find_dsc /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/lightos.py:98}}
Nov 29 03:29:30 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.lightos [None req-dda9b2f7-7676-4d13-99ef-fdc6b1e3b994 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=75234) get_connector_properties /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/lightos.py:76}}
Nov 29 03:29:30 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.lightos [None req-dda9b2f7-7676-4d13-99ef-fdc6b1e3b994 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] LIGHTOS: no hostnqn found. {{(pid=75234) get_connector_properties /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/lightos.py:84}}
Nov 29 03:29:30 user nova-compute[75234]: DEBUG os_brick.utils [None req-dda9b2f7-7676-4d13-99ef-fdc6b1e3b994 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] <== get_connector_properties: return (168ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:dd1368c5ba2c', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:203}}
Nov 29 03:29:30 user nova-compute[75234]: DEBUG nova.virt.block_device [None req-dda9b2f7-7676-4d13-99ef-fdc6b1e3b994 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] [instance: 01d62e54-9957-40b1-8dfe-1783b64ffabf] Updating existing volume attachment record: c5d90cb4-d632-48d4-8e96-c3d53b1ae303 {{(pid=75234) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Nov 29 03:29:30 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-59d92465-53a2-4fe5-a842-2343f91e5e6b tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/abb01889-df4b-44a6-9827-87056d7ca125/disk --force-share --output=json" returned: 0 in 0.178s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:29:30 user nova-compute[75234]: DEBUG nova.virt.disk.api [None req-59d92465-53a2-4fe5-a842-2343f91e5e6b tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Cannot resize image /opt/stack/data/nova/instances/abb01889-df4b-44a6-9827-87056d7ca125/disk to a smaller size. {{(pid=75234) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:172}}
Nov 29 03:29:30 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-59d92465-53a2-4fe5-a842-2343f91e5e6b tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] [instance: abb01889-df4b-44a6-9827-87056d7ca125] Created local disks {{(pid=75234) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4861}}
Nov 29 03:29:30 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-59d92465-53a2-4fe5-a842-2343f91e5e6b tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] [instance: abb01889-df4b-44a6-9827-87056d7ca125] Ensure instance console log exists: /opt/stack/data/nova/instances/abb01889-df4b-44a6-9827-87056d7ca125/console.log {{(pid=75234) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4613}}
Nov 29 03:29:30 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-59d92465-53a2-4fe5-a842-2343f91e5e6b tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:29:30 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-59d92465-53a2-4fe5-a842-2343f91e5e6b tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:29:30 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-59d92465-53a2-4fe5-a842-2343f91e5e6b tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:29:31 user nova-compute[75234]: DEBUG nova.network.neutron [None req-dda9b2f7-7676-4d13-99ef-fdc6b1e3b994 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] [instance: 01d62e54-9957-40b1-8dfe-1783b64ffabf] Successfully created port: 6dbd7c27-31c9-4ddc-a81b-85f1e742d160 {{(pid=75234) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:548}}
Nov 29 03:29:31 user nova-compute[75234]: DEBUG nova.network.neutron [None req-59d92465-53a2-4fe5-a842-2343f91e5e6b tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] [instance: abb01889-df4b-44a6-9827-87056d7ca125] Successfully updated port: fec40d24-caf1-48f7-8565-8fffdbdad60e {{(pid=75234) _update_port /opt/stack/nova/nova/network/neutron.py:586}}
Nov 29 03:29:31 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-59d92465-53a2-4fe5-a842-2343f91e5e6b tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Acquiring lock "refresh_cache-abb01889-df4b-44a6-9827-87056d7ca125" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:29:31 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-59d92465-53a2-4fe5-a842-2343f91e5e6b tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Acquired lock "refresh_cache-abb01889-df4b-44a6-9827-87056d7ca125" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:29:31 user nova-compute[75234]: DEBUG nova.network.neutron [None req-59d92465-53a2-4fe5-a842-2343f91e5e6b tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] [instance: abb01889-df4b-44a6-9827-87056d7ca125] Building network info cache for instance {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2009}}
Nov 29 03:29:31 user nova-compute[75234]: DEBUG nova.compute.manager [req-637f4f85-8e4a-437d-91b2-436000a87ac6 req-cfd115ad-4be9-4549-94ab-f2e43cdcc97a service nova] [instance: abb01889-df4b-44a6-9827-87056d7ca125] Received event network-changed-fec40d24-caf1-48f7-8565-8fffdbdad60e {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:29:31 user nova-compute[75234]: DEBUG nova.compute.manager [req-637f4f85-8e4a-437d-91b2-436000a87ac6 req-cfd115ad-4be9-4549-94ab-f2e43cdcc97a service nova] [instance: abb01889-df4b-44a6-9827-87056d7ca125] Refreshing instance network info cache due to event network-changed-fec40d24-caf1-48f7-8565-8fffdbdad60e. {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11117}}
Nov 29 03:29:31 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-637f4f85-8e4a-437d-91b2-436000a87ac6 req-cfd115ad-4be9-4549-94ab-f2e43cdcc97a service nova] Acquiring lock "refresh_cache-abb01889-df4b-44a6-9827-87056d7ca125" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:29:31 user nova-compute[75234]: DEBUG nova.network.neutron [None req-59d92465-53a2-4fe5-a842-2343f91e5e6b tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] [instance: abb01889-df4b-44a6-9827-87056d7ca125] Instance cache missing network info. {{(pid=75234) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3322}}
Nov 29 03:29:31 user nova-compute[75234]: DEBUG nova.network.neutron [None req-59d92465-53a2-4fe5-a842-2343f91e5e6b tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] [instance: abb01889-df4b-44a6-9827-87056d7ca125] Updating instance_info_cache with network_info: [{"id": "fec40d24-caf1-48f7-8565-8fffdbdad60e", "address": "fa:16:3e:e3:bd:f8", "network": {"id": "4d086e08-7798-4cfe-9365-86be6f0a52d2", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-305624008-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "db00bcd0c82c44f1bda739e9857f6e84", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapfec40d24-ca", "ovs_interfaceid": "fec40d24-caf1-48f7-8565-8fffdbdad60e", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:29:31 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-59d92465-53a2-4fe5-a842-2343f91e5e6b tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Releasing lock "refresh_cache-abb01889-df4b-44a6-9827-87056d7ca125" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:29:31 user nova-compute[75234]: DEBUG nova.compute.manager [None req-59d92465-53a2-4fe5-a842-2343f91e5e6b tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] [instance: abb01889-df4b-44a6-9827-87056d7ca125] Instance network_info: |[{"id": "fec40d24-caf1-48f7-8565-8fffdbdad60e", "address": "fa:16:3e:e3:bd:f8", "network": {"id": "4d086e08-7798-4cfe-9365-86be6f0a52d2", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-305624008-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "db00bcd0c82c44f1bda739e9857f6e84", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapfec40d24-ca", "ovs_interfaceid": "fec40d24-caf1-48f7-8565-8fffdbdad60e", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=75234) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1987}}
Nov 29 03:29:31 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-637f4f85-8e4a-437d-91b2-436000a87ac6 req-cfd115ad-4be9-4549-94ab-f2e43cdcc97a service nova] Acquired lock "refresh_cache-abb01889-df4b-44a6-9827-87056d7ca125" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:29:31 user nova-compute[75234]: DEBUG nova.network.neutron [req-637f4f85-8e4a-437d-91b2-436000a87ac6 req-cfd115ad-4be9-4549-94ab-f2e43cdcc97a service nova] [instance: abb01889-df4b-44a6-9827-87056d7ca125] Refreshing network info cache for port fec40d24-caf1-48f7-8565-8fffdbdad60e {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2006}}
Nov 29 03:29:31 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-59d92465-53a2-4fe5-a842-2343f91e5e6b tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] [instance: abb01889-df4b-44a6-9827-87056d7ca125] Start _get_guest_xml network_info=[{"id": "fec40d24-caf1-48f7-8565-8fffdbdad60e", "address": "fa:16:3e:e3:bd:f8", "network": {"id": "4d086e08-7798-4cfe-9365-86be6f0a52d2", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-305624008-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "db00bcd0c82c44f1bda739e9857f6e84", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapfec40d24-ca", "ovs_interfaceid": "fec40d24-caf1-48f7-8565-8fffdbdad60e", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum='c8fc807773e5354afe61636071771906',container_format='bare',created_at=2023-11-29T03:02:15Z,direct_url=<?>,disk_format='qcow2',id=96c30ba7-9b73-4a30-8ba4-7138879573b7,min_disk=0,min_ram=0,name='cirros-0.6.2-x86_64-disk',owner='b18e380921c748b78517b6f1211cd0f1',properties=ImageMetaProps,protected=<?>,size=21430272,status='active',tags=<?>,updated_at=2023-11-29T03:02:17Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [{'device_name': '/dev/vda', 'encryption_format': None, 'encrypted': False, 'device_type': 'disk', 'size': 0, 'disk_bus': 'virtio', 'boot_index': 0, 'encryption_options': None, 'encryption_secret_uuid': None, 'guest_format': None, 'image_id': '96c30ba7-9b73-4a30-8ba4-7138879573b7'}], 'ephemerals': [], 'block_device_mapping': [], 'swap': None} {{(pid=75234) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7562}}
Nov 29 03:29:31 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-59d92465-53a2-4fe5-a842-2343f91e5e6b tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:29:31 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-59d92465-53a2-4fe5-a842-2343f91e5e6b tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:29:31 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-59d92465-53a2-4fe5-a842-2343f91e5e6b tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Searching host: 'user' for CPU controller through CGroups V1... {{(pid=75234) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1652}}
Nov 29 03:29:31 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-59d92465-53a2-4fe5-a842-2343f91e5e6b tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] CPU controller missing on host. {{(pid=75234) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1662}}
Nov 29 03:29:31 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-59d92465-53a2-4fe5-a842-2343f91e5e6b tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Searching host: 'user' for CPU controller through CGroups V2... {{(pid=75234) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1671}}
Nov 29 03:29:31 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-59d92465-53a2-4fe5-a842-2343f91e5e6b tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] CPU controller found on host. {{(pid=75234) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1678}}
Nov 29 03:29:31 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-59d92465-53a2-4fe5-a842-2343f91e5e6b tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=75234) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5400}}
Nov 29 03:29:31 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-59d92465-53a2-4fe5-a842-2343f91e5e6b tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-11-29T03:02:04Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='1',id=6,is_public=True,memory_mb=512,name='m1.tiny',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='c8fc807773e5354afe61636071771906',container_format='bare',created_at=2023-11-29T03:02:15Z,direct_url=<?>,disk_format='qcow2',id=96c30ba7-9b73-4a30-8ba4-7138879573b7,min_disk=0,min_ram=0,name='cirros-0.6.2-x86_64-disk',owner='b18e380921c748b78517b6f1211cd0f1',properties=ImageMetaProps,protected=<?>,size=21430272,status='active',tags=<?>,updated_at=2023-11-29T03:02:17Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:563}}
Nov 29 03:29:31 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-59d92465-53a2-4fe5-a842-2343f91e5e6b tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Flavor limits 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:348}}
Nov 29 03:29:31 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-59d92465-53a2-4fe5-a842-2343f91e5e6b tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Image limits 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:352}}
Nov 29 03:29:31 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-59d92465-53a2-4fe5-a842-2343f91e5e6b tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Flavor pref 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:388}}
Nov 29 03:29:31 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-59d92465-53a2-4fe5-a842-2343f91e5e6b tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Image pref 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:392}}
Nov 29 03:29:31 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-59d92465-53a2-4fe5-a842-2343f91e5e6b tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:430}}
Nov 29 03:29:31 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-59d92465-53a2-4fe5-a842-2343f91e5e6b tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:569}}
Nov 29 03:29:31 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-59d92465-53a2-4fe5-a842-2343f91e5e6b tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=75234) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:471}}
Nov 29 03:29:31 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-59d92465-53a2-4fe5-a842-2343f91e5e6b tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Got 1 possible topologies {{(pid=75234) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:501}}
Nov 29 03:29:31 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-59d92465-53a2-4fe5-a842-2343f91e5e6b tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:575}}
Nov 29 03:29:31 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-59d92465-53a2-4fe5-a842-2343f91e5e6b tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:577}}
Nov 29 03:29:32 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-59d92465-53a2-4fe5-a842-2343f91e5e6b tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-11-29T03:29:28Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description=None,display_name='tempest-ServerBootFromVolumeStableRescueTest-server-1187610459',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-serverbootfromvolumestablerescuetest-server-1187610459',id=43,image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data=None,key_name=None,keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='db00bcd0c82c44f1bda739e9857f6e84',ramdisk_id='',reservation_id='r-ft0p330b',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-ServerBootFromVolumeStableRescueTest-1601266069',owner_user_name='tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-11-29T03:29:29Z,user_data=None,user_id='877aa877f13a4f84ac4725278010aea3',uuid=abb01889-df4b-44a6-9827-87056d7ca125,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "fec40d24-caf1-48f7-8565-8fffdbdad60e", "address": "fa:16:3e:e3:bd:f8", "network": {"id": "4d086e08-7798-4cfe-9365-86be6f0a52d2", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-305624008-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "db00bcd0c82c44f1bda739e9857f6e84", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapfec40d24-ca", "ovs_interfaceid": "fec40d24-caf1-48f7-8565-8fffdbdad60e", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=75234) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Nov 29 03:29:32 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-59d92465-53a2-4fe5-a842-2343f91e5e6b tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Converting VIF {"id": "fec40d24-caf1-48f7-8565-8fffdbdad60e", "address": "fa:16:3e:e3:bd:f8", "network": {"id": "4d086e08-7798-4cfe-9365-86be6f0a52d2", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-305624008-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "db00bcd0c82c44f1bda739e9857f6e84", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapfec40d24-ca", "ovs_interfaceid": "fec40d24-caf1-48f7-8565-8fffdbdad60e", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:29:32 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-59d92465-53a2-4fe5-a842-2343f91e5e6b tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:e3:bd:f8,bridge_name='br-int',has_traffic_filtering=True,id=fec40d24-caf1-48f7-8565-8fffdbdad60e,network=Network(4d086e08-7798-4cfe-9365-86be6f0a52d2),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapfec40d24-ca') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:29:32 user nova-compute[75234]: DEBUG nova.objects.instance [None req-59d92465-53a2-4fe5-a842-2343f91e5e6b tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Lazy-loading 'pci_devices' on Instance uuid abb01889-df4b-44a6-9827-87056d7ca125 {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:29:32 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-59d92465-53a2-4fe5-a842-2343f91e5e6b tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] [instance: abb01889-df4b-44a6-9827-87056d7ca125] End _get_guest_xml xml=<domain type="kvm">
Nov 29 03:29:32 user nova-compute[75234]:   <uuid>abb01889-df4b-44a6-9827-87056d7ca125</uuid>
Nov 29 03:29:32 user nova-compute[75234]:   <name>instance-0000002b</name>
Nov 29 03:29:32 user nova-compute[75234]:   <memory>524288</memory>
Nov 29 03:29:32 user nova-compute[75234]:   <vcpu>1</vcpu>
Nov 29 03:29:32 user nova-compute[75234]:   <metadata>
Nov 29 03:29:32 user nova-compute[75234]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Nov 29 03:29:32 user nova-compute[75234]:       <nova:package version="0.0.0"/>
Nov 29 03:29:32 user nova-compute[75234]:       <nova:name>tempest-ServerBootFromVolumeStableRescueTest-server-1187610459</nova:name>
Nov 29 03:29:32 user nova-compute[75234]:       <nova:creationTime>2023-11-29 03:29:31</nova:creationTime>
Nov 29 03:29:32 user nova-compute[75234]:       <nova:flavor name="m1.tiny">
Nov 29 03:29:32 user nova-compute[75234]:         <nova:memory>512</nova:memory>
Nov 29 03:29:32 user nova-compute[75234]:         <nova:disk>1</nova:disk>
Nov 29 03:29:32 user nova-compute[75234]:         <nova:swap>0</nova:swap>
Nov 29 03:29:32 user nova-compute[75234]:         <nova:ephemeral>0</nova:ephemeral>
Nov 29 03:29:32 user nova-compute[75234]:         <nova:vcpus>1</nova:vcpus>
Nov 29 03:29:32 user nova-compute[75234]:       </nova:flavor>
Nov 29 03:29:32 user nova-compute[75234]:       <nova:owner>
Nov 29 03:29:32 user nova-compute[75234]:         <nova:user uuid="877aa877f13a4f84ac4725278010aea3">tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member</nova:user>
Nov 29 03:29:32 user nova-compute[75234]:         <nova:project uuid="db00bcd0c82c44f1bda739e9857f6e84">tempest-ServerBootFromVolumeStableRescueTest-1601266069</nova:project>
Nov 29 03:29:32 user nova-compute[75234]:       </nova:owner>
Nov 29 03:29:32 user nova-compute[75234]:       <nova:root type="image" uuid="96c30ba7-9b73-4a30-8ba4-7138879573b7"/>
Nov 29 03:29:32 user nova-compute[75234]:       <nova:ports>
Nov 29 03:29:32 user nova-compute[75234]:         <nova:port uuid="fec40d24-caf1-48f7-8565-8fffdbdad60e">
Nov 29 03:29:32 user nova-compute[75234]:           <nova:ip type="fixed" address="10.0.0.9" ipVersion="4"/>
Nov 29 03:29:32 user nova-compute[75234]:         </nova:port>
Nov 29 03:29:32 user nova-compute[75234]:       </nova:ports>
Nov 29 03:29:32 user nova-compute[75234]:     </nova:instance>
Nov 29 03:29:32 user nova-compute[75234]:   </metadata>
Nov 29 03:29:32 user nova-compute[75234]:   <sysinfo type="smbios">
Nov 29 03:29:32 user nova-compute[75234]:     <system>
Nov 29 03:29:32 user nova-compute[75234]:       <entry name="manufacturer">OpenStack Foundation</entry>
Nov 29 03:29:32 user nova-compute[75234]:       <entry name="product">OpenStack Nova</entry>
Nov 29 03:29:32 user nova-compute[75234]:       <entry name="version">0.0.0</entry>
Nov 29 03:29:32 user nova-compute[75234]:       <entry name="serial">abb01889-df4b-44a6-9827-87056d7ca125</entry>
Nov 29 03:29:32 user nova-compute[75234]:       <entry name="uuid">abb01889-df4b-44a6-9827-87056d7ca125</entry>
Nov 29 03:29:32 user nova-compute[75234]:       <entry name="family">Virtual Machine</entry>
Nov 29 03:29:32 user nova-compute[75234]:     </system>
Nov 29 03:29:32 user nova-compute[75234]:   </sysinfo>
Nov 29 03:29:32 user nova-compute[75234]:   <os>
Nov 29 03:29:32 user nova-compute[75234]:     <type arch="x86_64" machine="pc">hvm</type>
Nov 29 03:29:32 user nova-compute[75234]:     <boot dev="hd"/>
Nov 29 03:29:32 user nova-compute[75234]:     <smbios mode="sysinfo"/>
Nov 29 03:29:32 user nova-compute[75234]:   </os>
Nov 29 03:29:32 user nova-compute[75234]:   <features>
Nov 29 03:29:32 user nova-compute[75234]:     <acpi/>
Nov 29 03:29:32 user nova-compute[75234]:     <apic/>
Nov 29 03:29:32 user nova-compute[75234]:     <vmcoreinfo/>
Nov 29 03:29:32 user nova-compute[75234]:   </features>
Nov 29 03:29:32 user nova-compute[75234]:   <clock offset="utc">
Nov 29 03:29:32 user nova-compute[75234]:     <timer name="pit" tickpolicy="delay"/>
Nov 29 03:29:32 user nova-compute[75234]:     <timer name="rtc" tickpolicy="catchup"/>
Nov 29 03:29:32 user nova-compute[75234]:     <timer name="hpet" present="no"/>
Nov 29 03:29:32 user nova-compute[75234]:   </clock>
Nov 29 03:29:32 user nova-compute[75234]:   <cpu mode="custom" match="exact">
Nov 29 03:29:32 user nova-compute[75234]:     <model>Nehalem</model>
Nov 29 03:29:32 user nova-compute[75234]:     <topology sockets="1" cores="1" threads="1"/>
Nov 29 03:29:32 user nova-compute[75234]:   </cpu>
Nov 29 03:29:32 user nova-compute[75234]:   <devices>
Nov 29 03:29:32 user nova-compute[75234]:     <disk type="file" device="disk">
Nov 29 03:29:32 user nova-compute[75234]:       <driver name="qemu" type="qcow2" cache="none"/>
Nov 29 03:29:32 user nova-compute[75234]:       <source file="/opt/stack/data/nova/instances/abb01889-df4b-44a6-9827-87056d7ca125/disk"/>
Nov 29 03:29:32 user nova-compute[75234]:       <target dev="vda" bus="virtio"/>
Nov 29 03:29:32 user nova-compute[75234]:     </disk>
Nov 29 03:29:32 user nova-compute[75234]:     <interface type="ethernet">
Nov 29 03:29:32 user nova-compute[75234]:       <mac address="fa:16:3e:e3:bd:f8"/>
Nov 29 03:29:32 user nova-compute[75234]:       <model type="virtio"/>
Nov 29 03:29:32 user nova-compute[75234]:       <mtu size="1442"/>
Nov 29 03:29:32 user nova-compute[75234]:       <target dev="tapfec40d24-ca"/>
Nov 29 03:29:32 user nova-compute[75234]:     </interface>
Nov 29 03:29:32 user nova-compute[75234]:     <serial type="pty">
Nov 29 03:29:32 user nova-compute[75234]:       <log file="/opt/stack/data/nova/instances/abb01889-df4b-44a6-9827-87056d7ca125/console.log" append="off"/>
Nov 29 03:29:32 user nova-compute[75234]:     </serial>
Nov 29 03:29:32 user nova-compute[75234]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Nov 29 03:29:32 user nova-compute[75234]:     <video>
Nov 29 03:29:32 user nova-compute[75234]:       <model type="virtio"/>
Nov 29 03:29:32 user nova-compute[75234]:     </video>
Nov 29 03:29:32 user nova-compute[75234]:     <rng model="virtio">
Nov 29 03:29:32 user nova-compute[75234]:       <backend model="random">/dev/urandom</backend>
Nov 29 03:29:32 user nova-compute[75234]:     </rng>
Nov 29 03:29:32 user nova-compute[75234]:     <controller type="usb" index="0" model="none"/>
Nov 29 03:29:32 user nova-compute[75234]:     <memballoon model="virtio">
Nov 29 03:29:32 user nova-compute[75234]:       <stats period="10"/>
Nov 29 03:29:32 user nova-compute[75234]:     </memballoon>
Nov 29 03:29:32 user nova-compute[75234]:   </devices>
Nov 29 03:29:32 user nova-compute[75234]: </domain>
Nov 29 03:29:32 user nova-compute[75234]:  {{(pid=75234) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7568}}
Nov 29 03:29:32 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-59d92465-53a2-4fe5-a842-2343f91e5e6b tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-11-29T03:29:28Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description=None,display_name='tempest-ServerBootFromVolumeStableRescueTest-server-1187610459',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-serverbootfromvolumestablerescuetest-server-1187610459',id=43,image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data=None,key_name=None,keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='db00bcd0c82c44f1bda739e9857f6e84',ramdisk_id='',reservation_id='r-ft0p330b',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-ServerBootFromVolumeStableRescueTest-1601266069',owner_user_name='tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-11-29T03:29:29Z,user_data=None,user_id='877aa877f13a4f84ac4725278010aea3',uuid=abb01889-df4b-44a6-9827-87056d7ca125,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "fec40d24-caf1-48f7-8565-8fffdbdad60e", "address": "fa:16:3e:e3:bd:f8", "network": {"id": "4d086e08-7798-4cfe-9365-86be6f0a52d2", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-305624008-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "db00bcd0c82c44f1bda739e9857f6e84", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapfec40d24-ca", "ovs_interfaceid": "fec40d24-caf1-48f7-8565-8fffdbdad60e", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Nov 29 03:29:32 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-59d92465-53a2-4fe5-a842-2343f91e5e6b tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Converting VIF {"id": "fec40d24-caf1-48f7-8565-8fffdbdad60e", "address": "fa:16:3e:e3:bd:f8", "network": {"id": "4d086e08-7798-4cfe-9365-86be6f0a52d2", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-305624008-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "db00bcd0c82c44f1bda739e9857f6e84", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapfec40d24-ca", "ovs_interfaceid": "fec40d24-caf1-48f7-8565-8fffdbdad60e", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:29:32 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-59d92465-53a2-4fe5-a842-2343f91e5e6b tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:e3:bd:f8,bridge_name='br-int',has_traffic_filtering=True,id=fec40d24-caf1-48f7-8565-8fffdbdad60e,network=Network(4d086e08-7798-4cfe-9365-86be6f0a52d2),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapfec40d24-ca') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:29:32 user nova-compute[75234]: DEBUG os_vif [None req-59d92465-53a2-4fe5-a842-2343f91e5e6b tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:e3:bd:f8,bridge_name='br-int',has_traffic_filtering=True,id=fec40d24-caf1-48f7-8565-8fffdbdad60e,network=Network(4d086e08-7798-4cfe-9365-86be6f0a52d2),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapfec40d24-ca') {{(pid=75234) plug /opt/stack/data/venv/lib/python3.10/site-packages/os_vif/__init__.py:76}}
Nov 29 03:29:32 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:29:32 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:29:32 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Nov 29 03:29:32 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:29:32 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbCreateCommand(_result=None, table=QoS, columns={'type': 'linux-noop', 'external_ids': {'id': '801746b4-789c-59d5-9ab5-d96de23e9462', '_type': 'linux-noop'}}, row=False) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:29:32 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:29:32 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:29:32 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:29:32 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tapfec40d24-ca, may_exist=True, interface_attrs={}) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:29:32 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Port, record=tapfec40d24-ca, col_values=(('qos', UUID('816c4e61-ce5c-4c8a-83cd-b4eeadf527bc')),), if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:29:32 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DbSetCommand(_result=None, table=Interface, record=tapfec40d24-ca, col_values=(('external_ids', {'iface-id': 'fec40d24-caf1-48f7-8565-8fffdbdad60e', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:e3:bd:f8', 'vm-uuid': 'abb01889-df4b-44a6-9827-87056d7ca125'}),), if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:29:32 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:29:32 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:29:32 user nova-compute[75234]: INFO os_vif [None req-59d92465-53a2-4fe5-a842-2343f91e5e6b tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:e3:bd:f8,bridge_name='br-int',has_traffic_filtering=True,id=fec40d24-caf1-48f7-8565-8fffdbdad60e,network=Network(4d086e08-7798-4cfe-9365-86be6f0a52d2),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapfec40d24-ca')
Nov 29 03:29:32 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-59d92465-53a2-4fe5-a842-2343f91e5e6b tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] No BDM found with device name vda, not building metadata. {{(pid=75234) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12089}}
Nov 29 03:29:32 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-59d92465-53a2-4fe5-a842-2343f91e5e6b tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] No VIF found with MAC fa:16:3e:e3:bd:f8, not building metadata {{(pid=75234) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Nov 29 03:29:32 user nova-compute[75234]: DEBUG nova.network.neutron [None req-dda9b2f7-7676-4d13-99ef-fdc6b1e3b994 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] [instance: 01d62e54-9957-40b1-8dfe-1783b64ffabf] Successfully updated port: 6dbd7c27-31c9-4ddc-a81b-85f1e742d160 {{(pid=75234) _update_port /opt/stack/nova/nova/network/neutron.py:586}}
Nov 29 03:29:32 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-dda9b2f7-7676-4d13-99ef-fdc6b1e3b994 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] Acquiring lock "refresh_cache-01d62e54-9957-40b1-8dfe-1783b64ffabf" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:29:32 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-dda9b2f7-7676-4d13-99ef-fdc6b1e3b994 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] Acquired lock "refresh_cache-01d62e54-9957-40b1-8dfe-1783b64ffabf" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:29:32 user nova-compute[75234]: DEBUG nova.network.neutron [None req-dda9b2f7-7676-4d13-99ef-fdc6b1e3b994 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] [instance: 01d62e54-9957-40b1-8dfe-1783b64ffabf] Building network info cache for instance {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2009}}
Nov 29 03:29:32 user nova-compute[75234]: DEBUG nova.compute.manager [req-defaadbc-5ac1-44cd-b193-8019e7714485 req-bc4237ac-1045-4d5d-9488-6300aeaa85d4 service nova] [instance: 01d62e54-9957-40b1-8dfe-1783b64ffabf] Received event network-changed-6dbd7c27-31c9-4ddc-a81b-85f1e742d160 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:29:32 user nova-compute[75234]: DEBUG nova.compute.manager [req-defaadbc-5ac1-44cd-b193-8019e7714485 req-bc4237ac-1045-4d5d-9488-6300aeaa85d4 service nova] [instance: 01d62e54-9957-40b1-8dfe-1783b64ffabf] Refreshing instance network info cache due to event network-changed-6dbd7c27-31c9-4ddc-a81b-85f1e742d160. {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11117}}
Nov 29 03:29:32 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-defaadbc-5ac1-44cd-b193-8019e7714485 req-bc4237ac-1045-4d5d-9488-6300aeaa85d4 service nova] Acquiring lock "refresh_cache-01d62e54-9957-40b1-8dfe-1783b64ffabf" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:29:32 user nova-compute[75234]: DEBUG nova.network.neutron [None req-dda9b2f7-7676-4d13-99ef-fdc6b1e3b994 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] [instance: 01d62e54-9957-40b1-8dfe-1783b64ffabf] Instance cache missing network info. {{(pid=75234) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3322}}
Nov 29 03:29:32 user nova-compute[75234]: DEBUG nova.network.neutron [None req-dda9b2f7-7676-4d13-99ef-fdc6b1e3b994 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] [instance: 01d62e54-9957-40b1-8dfe-1783b64ffabf] Updating instance_info_cache with network_info: [{"id": "6dbd7c27-31c9-4ddc-a81b-85f1e742d160", "address": "fa:16:3e:57:6a:26", "network": {"id": "504616f5-0d6b-47a6-aaf5-663983428b41", "bridge": "br-int", "label": "tempest-TestShelveInstance-438404840-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "53e55fcb8cd74895baec6f9528770bbc", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap6dbd7c27-31", "ovs_interfaceid": "6dbd7c27-31c9-4ddc-a81b-85f1e742d160", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:29:32 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-dda9b2f7-7676-4d13-99ef-fdc6b1e3b994 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] Releasing lock "refresh_cache-01d62e54-9957-40b1-8dfe-1783b64ffabf" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:29:32 user nova-compute[75234]: DEBUG nova.compute.manager [None req-dda9b2f7-7676-4d13-99ef-fdc6b1e3b994 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] [instance: 01d62e54-9957-40b1-8dfe-1783b64ffabf] Instance network_info: |[{"id": "6dbd7c27-31c9-4ddc-a81b-85f1e742d160", "address": "fa:16:3e:57:6a:26", "network": {"id": "504616f5-0d6b-47a6-aaf5-663983428b41", "bridge": "br-int", "label": "tempest-TestShelveInstance-438404840-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "53e55fcb8cd74895baec6f9528770bbc", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap6dbd7c27-31", "ovs_interfaceid": "6dbd7c27-31c9-4ddc-a81b-85f1e742d160", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=75234) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1987}}
Nov 29 03:29:32 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-defaadbc-5ac1-44cd-b193-8019e7714485 req-bc4237ac-1045-4d5d-9488-6300aeaa85d4 service nova] Acquired lock "refresh_cache-01d62e54-9957-40b1-8dfe-1783b64ffabf" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:29:32 user nova-compute[75234]: DEBUG nova.network.neutron [req-defaadbc-5ac1-44cd-b193-8019e7714485 req-bc4237ac-1045-4d5d-9488-6300aeaa85d4 service nova] [instance: 01d62e54-9957-40b1-8dfe-1783b64ffabf] Refreshing network info cache for port 6dbd7c27-31c9-4ddc-a81b-85f1e742d160 {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2006}}
Nov 29 03:29:32 user nova-compute[75234]: DEBUG nova.network.neutron [req-637f4f85-8e4a-437d-91b2-436000a87ac6 req-cfd115ad-4be9-4549-94ab-f2e43cdcc97a service nova] [instance: abb01889-df4b-44a6-9827-87056d7ca125] Updated VIF entry in instance network info cache for port fec40d24-caf1-48f7-8565-8fffdbdad60e. {{(pid=75234) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3481}}
Nov 29 03:29:32 user nova-compute[75234]: DEBUG nova.network.neutron [req-637f4f85-8e4a-437d-91b2-436000a87ac6 req-cfd115ad-4be9-4549-94ab-f2e43cdcc97a service nova] [instance: abb01889-df4b-44a6-9827-87056d7ca125] Updating instance_info_cache with network_info: [{"id": "fec40d24-caf1-48f7-8565-8fffdbdad60e", "address": "fa:16:3e:e3:bd:f8", "network": {"id": "4d086e08-7798-4cfe-9365-86be6f0a52d2", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-305624008-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "db00bcd0c82c44f1bda739e9857f6e84", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapfec40d24-ca", "ovs_interfaceid": "fec40d24-caf1-48f7-8565-8fffdbdad60e", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:29:32 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-637f4f85-8e4a-437d-91b2-436000a87ac6 req-cfd115ad-4be9-4549-94ab-f2e43cdcc97a service nova] Releasing lock "refresh_cache-abb01889-df4b-44a6-9827-87056d7ca125" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:29:32 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:29:32 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:29:33 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:29:33 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:29:33 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:29:33 user nova-compute[75234]: DEBUG nova.network.neutron [req-defaadbc-5ac1-44cd-b193-8019e7714485 req-bc4237ac-1045-4d5d-9488-6300aeaa85d4 service nova] [instance: 01d62e54-9957-40b1-8dfe-1783b64ffabf] Updated VIF entry in instance network info cache for port 6dbd7c27-31c9-4ddc-a81b-85f1e742d160. {{(pid=75234) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3481}}
Nov 29 03:29:33 user nova-compute[75234]: DEBUG nova.network.neutron [req-defaadbc-5ac1-44cd-b193-8019e7714485 req-bc4237ac-1045-4d5d-9488-6300aeaa85d4 service nova] [instance: 01d62e54-9957-40b1-8dfe-1783b64ffabf] Updating instance_info_cache with network_info: [{"id": "6dbd7c27-31c9-4ddc-a81b-85f1e742d160", "address": "fa:16:3e:57:6a:26", "network": {"id": "504616f5-0d6b-47a6-aaf5-663983428b41", "bridge": "br-int", "label": "tempest-TestShelveInstance-438404840-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "53e55fcb8cd74895baec6f9528770bbc", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap6dbd7c27-31", "ovs_interfaceid": "6dbd7c27-31c9-4ddc-a81b-85f1e742d160", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:29:33 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-defaadbc-5ac1-44cd-b193-8019e7714485 req-bc4237ac-1045-4d5d-9488-6300aeaa85d4 service nova] Releasing lock "refresh_cache-01d62e54-9957-40b1-8dfe-1783b64ffabf" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:29:33 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:29:34 user nova-compute[75234]: DEBUG nova.compute.manager [req-ee2ef95c-2a15-48a5-a300-ad60b498d33e req-6114f28e-0980-4c7e-a85e-a22867ff2b38 service nova] [instance: abb01889-df4b-44a6-9827-87056d7ca125] Received event network-vif-plugged-fec40d24-caf1-48f7-8565-8fffdbdad60e {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:29:34 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-ee2ef95c-2a15-48a5-a300-ad60b498d33e req-6114f28e-0980-4c7e-a85e-a22867ff2b38 service nova] Acquiring lock "abb01889-df4b-44a6-9827-87056d7ca125-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:29:34 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-ee2ef95c-2a15-48a5-a300-ad60b498d33e req-6114f28e-0980-4c7e-a85e-a22867ff2b38 service nova] Lock "abb01889-df4b-44a6-9827-87056d7ca125-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:29:34 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-ee2ef95c-2a15-48a5-a300-ad60b498d33e req-6114f28e-0980-4c7e-a85e-a22867ff2b38 service nova] Lock "abb01889-df4b-44a6-9827-87056d7ca125-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:29:34 user nova-compute[75234]: DEBUG nova.compute.manager [req-ee2ef95c-2a15-48a5-a300-ad60b498d33e req-6114f28e-0980-4c7e-a85e-a22867ff2b38 service nova] [instance: abb01889-df4b-44a6-9827-87056d7ca125] No waiting events found dispatching network-vif-plugged-fec40d24-caf1-48f7-8565-8fffdbdad60e {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:29:34 user nova-compute[75234]: WARNING nova.compute.manager [req-ee2ef95c-2a15-48a5-a300-ad60b498d33e req-6114f28e-0980-4c7e-a85e-a22867ff2b38 service nova] [instance: abb01889-df4b-44a6-9827-87056d7ca125] Received unexpected event network-vif-plugged-fec40d24-caf1-48f7-8565-8fffdbdad60e for instance with vm_state building and task_state spawning.
Nov 29 03:29:34 user nova-compute[75234]: DEBUG nova.compute.manager [req-ee2ef95c-2a15-48a5-a300-ad60b498d33e req-6114f28e-0980-4c7e-a85e-a22867ff2b38 service nova] [instance: abb01889-df4b-44a6-9827-87056d7ca125] Received event network-vif-plugged-fec40d24-caf1-48f7-8565-8fffdbdad60e {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:29:34 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-ee2ef95c-2a15-48a5-a300-ad60b498d33e req-6114f28e-0980-4c7e-a85e-a22867ff2b38 service nova] Acquiring lock "abb01889-df4b-44a6-9827-87056d7ca125-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:29:34 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-ee2ef95c-2a15-48a5-a300-ad60b498d33e req-6114f28e-0980-4c7e-a85e-a22867ff2b38 service nova] Lock "abb01889-df4b-44a6-9827-87056d7ca125-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:29:34 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-ee2ef95c-2a15-48a5-a300-ad60b498d33e req-6114f28e-0980-4c7e-a85e-a22867ff2b38 service nova] Lock "abb01889-df4b-44a6-9827-87056d7ca125-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:29:34 user nova-compute[75234]: DEBUG nova.compute.manager [req-ee2ef95c-2a15-48a5-a300-ad60b498d33e req-6114f28e-0980-4c7e-a85e-a22867ff2b38 service nova] [instance: abb01889-df4b-44a6-9827-87056d7ca125] No waiting events found dispatching network-vif-plugged-fec40d24-caf1-48f7-8565-8fffdbdad60e {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:29:34 user nova-compute[75234]: WARNING nova.compute.manager [req-ee2ef95c-2a15-48a5-a300-ad60b498d33e req-6114f28e-0980-4c7e-a85e-a22867ff2b38 service nova] [instance: abb01889-df4b-44a6-9827-87056d7ca125] Received unexpected event network-vif-plugged-fec40d24-caf1-48f7-8565-8fffdbdad60e for instance with vm_state building and task_state spawning.
Nov 29 03:29:34 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-cbb5b5ed-441c-455b-9e87-49b73c2c4cb0 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Acquiring lock "1e9263fc-5735-420f-a116-8f60c8736faf" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:29:34 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-cbb5b5ed-441c-455b-9e87-49b73c2c4cb0 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Lock "1e9263fc-5735-420f-a116-8f60c8736faf" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:29:34 user nova-compute[75234]: DEBUG nova.compute.manager [None req-cbb5b5ed-441c-455b-9e87-49b73c2c4cb0 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: 1e9263fc-5735-420f-a116-8f60c8736faf] Starting instance... {{(pid=75234) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2422}}
Nov 29 03:29:34 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-cbb5b5ed-441c-455b-9e87-49b73c2c4cb0 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:29:34 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-cbb5b5ed-441c-455b-9e87-49b73c2c4cb0 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:29:34 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-cbb5b5ed-441c-455b-9e87-49b73c2c4cb0 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=75234) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2366}}
Nov 29 03:29:34 user nova-compute[75234]: INFO nova.compute.claims [None req-cbb5b5ed-441c-455b-9e87-49b73c2c4cb0 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: 1e9263fc-5735-420f-a116-8f60c8736faf] Claim successful on node user
Nov 29 03:29:35 user nova-compute[75234]: DEBUG nova.compute.provider_tree [None req-cbb5b5ed-441c-455b-9e87-49b73c2c4cb0 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Inventory has not changed in ProviderTree for provider: 4d291220-7c37-4839-a0e3-b23c66502b74 {{(pid=75234) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Nov 29 03:29:35 user nova-compute[75234]: DEBUG nova.scheduler.client.report [None req-cbb5b5ed-441c-455b-9e87-49b73c2c4cb0 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Inventory has not changed for provider 4d291220-7c37-4839-a0e3-b23c66502b74 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=75234) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Nov 29 03:29:35 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-cbb5b5ed-441c-455b-9e87-49b73c2c4cb0 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.437s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:29:35 user nova-compute[75234]: DEBUG nova.compute.manager [None req-cbb5b5ed-441c-455b-9e87-49b73c2c4cb0 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: 1e9263fc-5735-420f-a116-8f60c8736faf] Start building networks asynchronously for instance. {{(pid=75234) _build_resources /opt/stack/nova/nova/compute/manager.py:2819}}
Nov 29 03:29:35 user nova-compute[75234]: DEBUG nova.compute.manager [None req-cbb5b5ed-441c-455b-9e87-49b73c2c4cb0 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: 1e9263fc-5735-420f-a116-8f60c8736faf] Allocating IP information in the background. {{(pid=75234) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1972}}
Nov 29 03:29:35 user nova-compute[75234]: DEBUG nova.network.neutron [None req-cbb5b5ed-441c-455b-9e87-49b73c2c4cb0 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: 1e9263fc-5735-420f-a116-8f60c8736faf] allocate_for_instance() {{(pid=75234) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1156}}
Nov 29 03:29:35 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-cbb5b5ed-441c-455b-9e87-49b73c2c4cb0 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: 1e9263fc-5735-420f-a116-8f60c8736faf] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Nov 29 03:29:35 user nova-compute[75234]: DEBUG nova.compute.manager [None req-cbb5b5ed-441c-455b-9e87-49b73c2c4cb0 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: 1e9263fc-5735-420f-a116-8f60c8736faf] Start building block device mappings for instance. {{(pid=75234) _build_resources /opt/stack/nova/nova/compute/manager.py:2854}}
Nov 29 03:29:35 user nova-compute[75234]: DEBUG nova.policy [None req-cbb5b5ed-441c-455b-9e87-49b73c2c4cb0 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '7d1f1f11a9aa4c1db8627413eb963fa4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '81a265d3c15b4e9d826b35ddbd0ff443', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=75234) authorize /opt/stack/nova/nova/policy.py:203}}
Nov 29 03:29:35 user nova-compute[75234]: INFO nova.virt.block_device [None req-cbb5b5ed-441c-455b-9e87-49b73c2c4cb0 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: 1e9263fc-5735-420f-a116-8f60c8736faf] Booting with volume snapshot 98fc8b27-739d-471f-85e1-5ebe49bcfce4 at /dev/vda
Nov 29 03:29:36 user nova-compute[75234]: DEBUG nova.compute.manager [None req-59d92465-53a2-4fe5-a842-2343f91e5e6b tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] [instance: abb01889-df4b-44a6-9827-87056d7ca125] Instance event wait completed in 0 seconds for  {{(pid=75234) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Nov 29 03:29:36 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-59d92465-53a2-4fe5-a842-2343f91e5e6b tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] [instance: abb01889-df4b-44a6-9827-87056d7ca125] Guest created on hypervisor {{(pid=75234) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4421}}
Nov 29 03:29:36 user nova-compute[75234]: INFO nova.virt.libvirt.driver [-] [instance: abb01889-df4b-44a6-9827-87056d7ca125] Instance spawned successfully.
Nov 29 03:29:36 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-59d92465-53a2-4fe5-a842-2343f91e5e6b tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] [instance: abb01889-df4b-44a6-9827-87056d7ca125] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:904}}
Nov 29 03:29:36 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-59d92465-53a2-4fe5-a842-2343f91e5e6b tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] [instance: abb01889-df4b-44a6-9827-87056d7ca125] Found default for hw_cdrom_bus of ide {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:29:36 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-59d92465-53a2-4fe5-a842-2343f91e5e6b tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] [instance: abb01889-df4b-44a6-9827-87056d7ca125] Found default for hw_disk_bus of virtio {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:29:36 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-59d92465-53a2-4fe5-a842-2343f91e5e6b tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] [instance: abb01889-df4b-44a6-9827-87056d7ca125] Found default for hw_input_bus of None {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:29:36 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-59d92465-53a2-4fe5-a842-2343f91e5e6b tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] [instance: abb01889-df4b-44a6-9827-87056d7ca125] Found default for hw_pointer_model of None {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:29:36 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-59d92465-53a2-4fe5-a842-2343f91e5e6b tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] [instance: abb01889-df4b-44a6-9827-87056d7ca125] Found default for hw_video_model of virtio {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:29:36 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-59d92465-53a2-4fe5-a842-2343f91e5e6b tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] [instance: abb01889-df4b-44a6-9827-87056d7ca125] Found default for hw_vif_model of virtio {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:29:36 user nova-compute[75234]: DEBUG nova.network.neutron [None req-cbb5b5ed-441c-455b-9e87-49b73c2c4cb0 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: 1e9263fc-5735-420f-a116-8f60c8736faf] Successfully created port: f836309d-8f97-4521-aa57-d6516a9e7528 {{(pid=75234) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:548}}
Nov 29 03:29:36 user nova-compute[75234]: INFO nova.compute.manager [None req-59d92465-53a2-4fe5-a842-2343f91e5e6b tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] [instance: abb01889-df4b-44a6-9827-87056d7ca125] Took 6.55 seconds to spawn the instance on the hypervisor.
Nov 29 03:29:36 user nova-compute[75234]: DEBUG nova.compute.manager [None req-59d92465-53a2-4fe5-a842-2343f91e5e6b tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] [instance: abb01889-df4b-44a6-9827-87056d7ca125] Checking state {{(pid=75234) _get_power_state /opt/stack/nova/nova/compute/manager.py:1782}}
Nov 29 03:29:36 user nova-compute[75234]: INFO nova.compute.manager [None req-59d92465-53a2-4fe5-a842-2343f91e5e6b tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] [instance: abb01889-df4b-44a6-9827-87056d7ca125] Took 7.23 seconds to build instance.
Nov 29 03:29:36 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-59d92465-53a2-4fe5-a842-2343f91e5e6b tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Lock "abb01889-df4b-44a6-9827-87056d7ca125" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 7.386s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:29:36 user nova-compute[75234]: DEBUG nova.network.neutron [None req-cbb5b5ed-441c-455b-9e87-49b73c2c4cb0 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: 1e9263fc-5735-420f-a116-8f60c8736faf] Successfully updated port: f836309d-8f97-4521-aa57-d6516a9e7528 {{(pid=75234) _update_port /opt/stack/nova/nova/network/neutron.py:586}}
Nov 29 03:29:36 user nova-compute[75234]: DEBUG nova.compute.manager [req-84b23e74-86b4-4254-aac8-4eb3751a96ee req-0b23b4b8-71c3-4c3e-908a-11bddff73108 service nova] [instance: 1e9263fc-5735-420f-a116-8f60c8736faf] Received event network-changed-f836309d-8f97-4521-aa57-d6516a9e7528 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:29:36 user nova-compute[75234]: DEBUG nova.compute.manager [req-84b23e74-86b4-4254-aac8-4eb3751a96ee req-0b23b4b8-71c3-4c3e-908a-11bddff73108 service nova] [instance: 1e9263fc-5735-420f-a116-8f60c8736faf] Refreshing instance network info cache due to event network-changed-f836309d-8f97-4521-aa57-d6516a9e7528. {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11117}}
Nov 29 03:29:36 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-84b23e74-86b4-4254-aac8-4eb3751a96ee req-0b23b4b8-71c3-4c3e-908a-11bddff73108 service nova] Acquiring lock "refresh_cache-1e9263fc-5735-420f-a116-8f60c8736faf" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:29:36 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-84b23e74-86b4-4254-aac8-4eb3751a96ee req-0b23b4b8-71c3-4c3e-908a-11bddff73108 service nova] Acquired lock "refresh_cache-1e9263fc-5735-420f-a116-8f60c8736faf" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:29:36 user nova-compute[75234]: DEBUG nova.network.neutron [req-84b23e74-86b4-4254-aac8-4eb3751a96ee req-0b23b4b8-71c3-4c3e-908a-11bddff73108 service nova] [instance: 1e9263fc-5735-420f-a116-8f60c8736faf] Refreshing network info cache for port f836309d-8f97-4521-aa57-d6516a9e7528 {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2006}}
Nov 29 03:29:36 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-cbb5b5ed-441c-455b-9e87-49b73c2c4cb0 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Acquiring lock "refresh_cache-1e9263fc-5735-420f-a116-8f60c8736faf" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:29:36 user nova-compute[75234]: DEBUG nova.network.neutron [req-84b23e74-86b4-4254-aac8-4eb3751a96ee req-0b23b4b8-71c3-4c3e-908a-11bddff73108 service nova] [instance: 1e9263fc-5735-420f-a116-8f60c8736faf] Instance cache missing network info. {{(pid=75234) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3322}}
Nov 29 03:29:37 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:29:37 user nova-compute[75234]: DEBUG nova.network.neutron [req-84b23e74-86b4-4254-aac8-4eb3751a96ee req-0b23b4b8-71c3-4c3e-908a-11bddff73108 service nova] [instance: 1e9263fc-5735-420f-a116-8f60c8736faf] Updating instance_info_cache with network_info: [] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:29:37 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-84b23e74-86b4-4254-aac8-4eb3751a96ee req-0b23b4b8-71c3-4c3e-908a-11bddff73108 service nova] Releasing lock "refresh_cache-1e9263fc-5735-420f-a116-8f60c8736faf" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:29:37 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-cbb5b5ed-441c-455b-9e87-49b73c2c4cb0 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Acquired lock "refresh_cache-1e9263fc-5735-420f-a116-8f60c8736faf" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:29:37 user nova-compute[75234]: DEBUG nova.network.neutron [None req-cbb5b5ed-441c-455b-9e87-49b73c2c4cb0 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: 1e9263fc-5735-420f-a116-8f60c8736faf] Building network info cache for instance {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2009}}
Nov 29 03:29:37 user nova-compute[75234]: DEBUG nova.network.neutron [None req-cbb5b5ed-441c-455b-9e87-49b73c2c4cb0 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: 1e9263fc-5735-420f-a116-8f60c8736faf] Instance cache missing network info. {{(pid=75234) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3322}}
Nov 29 03:29:37 user nova-compute[75234]: DEBUG nova.network.neutron [None req-cbb5b5ed-441c-455b-9e87-49b73c2c4cb0 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: 1e9263fc-5735-420f-a116-8f60c8736faf] Updating instance_info_cache with network_info: [{"id": "f836309d-8f97-4521-aa57-d6516a9e7528", "address": "fa:16:3e:1e:0c:76", "network": {"id": "a3f397d3-fd75-4a6e-a815-cfcedcca15fb", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-678570085-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "81a265d3c15b4e9d826b35ddbd0ff443", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapf836309d-8f", "ovs_interfaceid": "f836309d-8f97-4521-aa57-d6516a9e7528", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:29:37 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-cbb5b5ed-441c-455b-9e87-49b73c2c4cb0 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Releasing lock "refresh_cache-1e9263fc-5735-420f-a116-8f60c8736faf" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:29:37 user nova-compute[75234]: DEBUG nova.compute.manager [None req-cbb5b5ed-441c-455b-9e87-49b73c2c4cb0 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: 1e9263fc-5735-420f-a116-8f60c8736faf] Instance network_info: |[{"id": "f836309d-8f97-4521-aa57-d6516a9e7528", "address": "fa:16:3e:1e:0c:76", "network": {"id": "a3f397d3-fd75-4a6e-a815-cfcedcca15fb", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-678570085-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "81a265d3c15b4e9d826b35ddbd0ff443", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapf836309d-8f", "ovs_interfaceid": "f836309d-8f97-4521-aa57-d6516a9e7528", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=75234) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1987}}
Nov 29 03:29:37 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:29:38 user nova-compute[75234]: DEBUG nova.compute.manager [None req-dda9b2f7-7676-4d13-99ef-fdc6b1e3b994 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] [instance: 01d62e54-9957-40b1-8dfe-1783b64ffabf] Start spawning the instance on the hypervisor. {{(pid=75234) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2628}}
Nov 29 03:29:38 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-dda9b2f7-7676-4d13-99ef-fdc6b1e3b994 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] [instance: 01d62e54-9957-40b1-8dfe-1783b64ffabf] Creating instance directory {{(pid=75234) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4727}}
Nov 29 03:29:38 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-dda9b2f7-7676-4d13-99ef-fdc6b1e3b994 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] [instance: 01d62e54-9957-40b1-8dfe-1783b64ffabf] Creating image(s)
Nov 29 03:29:38 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-dda9b2f7-7676-4d13-99ef-fdc6b1e3b994 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] [instance: 01d62e54-9957-40b1-8dfe-1783b64ffabf] Did not create local disks {{(pid=75234) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4863}}
Nov 29 03:29:38 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-dda9b2f7-7676-4d13-99ef-fdc6b1e3b994 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] [instance: 01d62e54-9957-40b1-8dfe-1783b64ffabf] Ensure instance console log exists: /opt/stack/data/nova/instances/01d62e54-9957-40b1-8dfe-1783b64ffabf/console.log {{(pid=75234) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4613}}
Nov 29 03:29:38 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-dda9b2f7-7676-4d13-99ef-fdc6b1e3b994 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:29:38 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-dda9b2f7-7676-4d13-99ef-fdc6b1e3b994 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:29:38 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-dda9b2f7-7676-4d13-99ef-fdc6b1e3b994 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:29:38 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-dda9b2f7-7676-4d13-99ef-fdc6b1e3b994 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] [instance: 01d62e54-9957-40b1-8dfe-1783b64ffabf] Start _get_guest_xml network_info=[{"id": "6dbd7c27-31c9-4ddc-a81b-85f1e742d160", "address": "fa:16:3e:57:6a:26", "network": {"id": "504616f5-0d6b-47a6-aaf5-663983428b41", "bridge": "br-int", "label": "tempest-TestShelveInstance-438404840-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "53e55fcb8cd74895baec6f9528770bbc", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap6dbd7c27-31", "ovs_interfaceid": "6dbd7c27-31c9-4ddc-a81b-85f1e742d160", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, '/dev/vda': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum=<?>,container_format=<?>,created_at=<?>,direct_url=<?>,disk_format=<?>,id=<?>,min_disk=0,min_ram=0,name=<?>,owner=<?>,properties=ImageMetaProps,protected=<?>,size=1073741824,status='active',tags=<?>,updated_at=<?>,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [], 'ephemerals': [], 'block_device_mapping': [{'mount_device': '/dev/vda', 'attachment_id': 'c5d90cb4-d632-48d4-8e96-c3d53b1ae303', 'disk_bus': 'virtio', 'device_type': 'disk', 'boot_index': 0, 'guest_format': None, 'delete_on_termination': True, 'connection_info': {'driver_volume_type': 'iscsi', 'data': {'target_iqn': 'iqn.2010-10.org.openstack:43e92d4a-7e98-4777-8fa5-3cec396b86ce', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'AQnkxjFp', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}, 'status': 'reserved', 'instance': '01d62e54-9957-40b1-8dfe-1783b64ffabf', 'attached_at': '', 'detached_at': '', 'volume_id': '43e92d4a-7e98-4777-8fa5-3cec396b86ce', 'serial': '43e92d4a-7e98-4777-8fa5-3cec396b86ce'}, 'volume_type': None} {{(pid=75234) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7562}}
Nov 29 03:29:38 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-dda9b2f7-7676-4d13-99ef-fdc6b1e3b994 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:29:38 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-dda9b2f7-7676-4d13-99ef-fdc6b1e3b994 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:29:38 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-dda9b2f7-7676-4d13-99ef-fdc6b1e3b994 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] Searching host: 'user' for CPU controller through CGroups V1... {{(pid=75234) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1652}}
Nov 29 03:29:38 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-dda9b2f7-7676-4d13-99ef-fdc6b1e3b994 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] CPU controller missing on host. {{(pid=75234) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1662}}
Nov 29 03:29:38 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-dda9b2f7-7676-4d13-99ef-fdc6b1e3b994 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] Searching host: 'user' for CPU controller through CGroups V2... {{(pid=75234) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1671}}
Nov 29 03:29:38 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-dda9b2f7-7676-4d13-99ef-fdc6b1e3b994 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] CPU controller found on host. {{(pid=75234) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1678}}
Nov 29 03:29:38 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-dda9b2f7-7676-4d13-99ef-fdc6b1e3b994 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=75234) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5400}}
Nov 29 03:29:38 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-dda9b2f7-7676-4d13-99ef-fdc6b1e3b994 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-11-29T03:02:04Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='1',id=6,is_public=True,memory_mb=512,name='m1.tiny',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum=<?>,container_format=<?>,created_at=<?>,direct_url=<?>,disk_format=<?>,id=<?>,min_disk=0,min_ram=0,name=<?>,owner=<?>,properties=ImageMetaProps,protected=<?>,size=1073741824,status='active',tags=<?>,updated_at=<?>,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:563}}
Nov 29 03:29:38 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-dda9b2f7-7676-4d13-99ef-fdc6b1e3b994 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] Flavor limits 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:348}}
Nov 29 03:29:38 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-dda9b2f7-7676-4d13-99ef-fdc6b1e3b994 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] Image limits 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:352}}
Nov 29 03:29:38 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-dda9b2f7-7676-4d13-99ef-fdc6b1e3b994 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] Flavor pref 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:388}}
Nov 29 03:29:38 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-dda9b2f7-7676-4d13-99ef-fdc6b1e3b994 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] Image pref 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:392}}
Nov 29 03:29:38 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-dda9b2f7-7676-4d13-99ef-fdc6b1e3b994 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:430}}
Nov 29 03:29:38 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-dda9b2f7-7676-4d13-99ef-fdc6b1e3b994 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:569}}
Nov 29 03:29:38 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-dda9b2f7-7676-4d13-99ef-fdc6b1e3b994 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=75234) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:471}}
Nov 29 03:29:38 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-dda9b2f7-7676-4d13-99ef-fdc6b1e3b994 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] Got 1 possible topologies {{(pid=75234) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:501}}
Nov 29 03:29:38 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-dda9b2f7-7676-4d13-99ef-fdc6b1e3b994 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:575}}
Nov 29 03:29:38 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-dda9b2f7-7676-4d13-99ef-fdc6b1e3b994 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:577}}
Nov 29 03:29:38 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-dda9b2f7-7676-4d13-99ef-fdc6b1e3b994 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] [instance: 01d62e54-9957-40b1-8dfe-1783b64ffabf] Calling os-brick to attach iSCSI Volume {{(pid=75234) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Nov 29 03:29:38 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-dda9b2f7-7676-4d13-99ef-fdc6b1e3b994 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f8a5050c9a0>, 'connection_properties': {'target_iqn': 'iqn.2010-10.org.openstack:43e92d4a-7e98-4777-8fa5-3cec396b86ce', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'AQnkxjFp', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:176}}
Nov 29 03:29:38 user nova-compute[75234]: WARNING os_brick.initiator.connectors.base [None req-dda9b2f7-7676-4d13-99ef-fdc6b1e3b994 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Nov 29 03:29:38 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-dda9b2f7-7676-4d13-99ef-fdc6b1e3b994 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:68}}
Nov 29 03:29:38 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-dda9b2f7-7676-4d13-99ef-fdc6b1e3b994 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:73}}
Nov 29 03:29:38 user nova-compute[75234]: WARNING os_brick.initiator.connectors.base [None req-dda9b2f7-7676-4d13-99ef-fdc6b1e3b994 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Nov 29 03:29:38 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-dda9b2f7-7676-4d13-99ef-fdc6b1e3b994 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:43e92d4a-7e98-4777-8fa5-3cec396b86ce" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:68}}
Nov 29 03:29:38 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-dda9b2f7-7676-4d13-99ef-fdc6b1e3b994 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:43e92d4a-7e98-4777-8fa5-3cec396b86ce" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:73}}
Nov 29 03:29:38 user nova-compute[75234]: INFO os_brick.initiator.connectors.iscsi [None req-dda9b2f7-7676-4d13-99ef-fdc6b1e3b994 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Nov 29 03:29:38 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:43e92d4a-7e98-4777-8fa5-3cec396b86ce -p 172.16.0.220:3260 {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:29:38 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:43e92d4a-7e98-4777-8fa5-3cec396b86ce -p 172.16.0.220:3260" returned: 21 in 0.019s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:29:38 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[ed7bbae6-03fa-4d02-ad1e-f1e9092c7e36]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:29:38 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-dda9b2f7-7676-4d13-99ef-fdc6b1e3b994 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Nov 29 03:29:38 user nova-compute[75234]:  {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:29:38 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:43e92d4a-7e98-4777-8fa5-3cec396b86ce -p 172.16.0.220:3260 --interface default --op new {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:29:38 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:43e92d4a-7e98-4777-8fa5-3cec396b86ce -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.020s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:29:38 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[d4dcc673-f866-44e0-b830-99247888a91d]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:43e92d4a-7e98-4777-8fa5-3cec396b86ce] added\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:29:38 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-dda9b2f7-7676-4d13-99ef-fdc6b1e3b994 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:43e92d4a-7e98-4777-8fa5-3cec396b86ce] added
Nov 29 03:29:38 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:29:38 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:43e92d4a-7e98-4777-8fa5-3cec396b86ce -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:29:38 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:43e92d4a-7e98-4777-8fa5-3cec396b86ce -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.019s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:29:38 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[e1990a32-58a8-4d63-bc59-56adfac2bc61]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:29:38 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-dda9b2f7-7676-4d13-99ef-fdc6b1e3b994 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:29:38 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:43e92d4a-7e98-4777-8fa5-3cec396b86ce -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:29:38 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:43e92d4a-7e98-4777-8fa5-3cec396b86ce -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.028s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:29:38 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[4f45a2c0-accf-41a5-98a4-3a0f35568ea6]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:29:38 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-dda9b2f7-7676-4d13-99ef-fdc6b1e3b994 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:29:38 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:43e92d4a-7e98-4777-8fa5-3cec396b86ce -p 172.16.0.220:3260 --op update -n node.session.auth.username -v AQnkxjFp {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:29:38 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:43e92d4a-7e98-4777-8fa5-3cec396b86ce -p 172.16.0.220:3260 --op update -n node.session.auth.username -v AQnkxjFp" returned: 0 in 0.018s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:29:38 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[f6159cbe-ca98-4ad7-90d8-eb5f82e7f3ff]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:29:38 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-dda9b2f7-7676-4d13-99ef-fdc6b1e3b994 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'AQnkxjFp'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:29:38 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:43e92d4a-7e98-4777-8fa5-3cec396b86ce -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:29:38 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:43e92d4a-7e98-4777-8fa5-3cec396b86ce -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.020s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:29:38 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[3283b665-bad8-403d-9e38-7c1e76ef90c5]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:29:38 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-dda9b2f7-7676-4d13-99ef-fdc6b1e3b994 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:29:38 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:29:38 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.020s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:29:38 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[2d4f98b5-b9c2-4f71-af0a-50ad34e17e8c]: (4, ('tcp: [42] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0 (non-flash)\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:29:38 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-dda9b2f7-7676-4d13-99ef-fdc6b1e3b994 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [42] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0 (non-flash)
Nov 29 03:29:38 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:29:38 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-dda9b2f7-7676-4d13-99ef-fdc6b1e3b994 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] iscsi session list stdout=tcp: [42] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0 (non-flash)
Nov 29 03:29:38 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsi_session /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Nov 29 03:29:38 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:43e92d4a-7e98-4777-8fa5-3cec396b86ce -p 172.16.0.220:3260 --login {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:29:38 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:43e92d4a-7e98-4777-8fa5-3cec396b86ce -p 172.16.0.220:3260 --login" returned: 0 in 0.064s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:29:38 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[3762121a-3218-467e-be00-9bd6594d77be]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:43e92d4a-7e98-4777-8fa5-3cec396b86ce, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:43e92d4a-7e98-4777-8fa5-3cec396b86ce, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:29:38 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-dda9b2f7-7676-4d13-99ef-fdc6b1e3b994 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:43e92d4a-7e98-4777-8fa5-3cec396b86ce, portal: 172.16.0.220,3260]
Nov 29 03:29:38 user nova-compute[75234]: Login to [iface: default, target: iqn.2010-10.org.openstack:43e92d4a-7e98-4777-8fa5-3cec396b86ce, portal: 172.16.0.220,3260] successful.
Nov 29 03:29:38 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:29:38 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:43e92d4a-7e98-4777-8fa5-3cec396b86ce -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:29:38 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:43e92d4a-7e98-4777-8fa5-3cec396b86ce -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.021s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:29:38 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[afeab867-7d52-44a9-924e-ead00c479d7c]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:29:38 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-dda9b2f7-7676-4d13-99ef-fdc6b1e3b994 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:29:38 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:29:38 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.027s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:29:38 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[d444e0e3-c32a-494d-b275-6b2fec3578d1]: (4, ('tcp: [42] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0 (non-flash)\ntcp: [50] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:43e92d4a-7e98-4777-8fa5-3cec396b86ce (non-flash)\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:29:38 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-dda9b2f7-7676-4d13-99ef-fdc6b1e3b994 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [42] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0 (non-flash)
Nov 29 03:29:38 user nova-compute[75234]: tcp: [50] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:43e92d4a-7e98-4777-8fa5-3cec396b86ce (non-flash)
Nov 29 03:29:38 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:29:38 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-dda9b2f7-7676-4d13-99ef-fdc6b1e3b994 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] iscsi session list stdout=tcp: [42] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0 (non-flash)
Nov 29 03:29:38 user nova-compute[75234]: tcp: [50] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:43e92d4a-7e98-4777-8fa5-3cec396b86ce (non-flash)
Nov 29 03:29:38 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsi_session /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Nov 29 03:29:38 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-dda9b2f7-7676-4d13-99ef-fdc6b1e3b994 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:43e92d4a-7e98-4777-8fa5-3cec396b86ce" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.294s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:87}}
Nov 29 03:29:38 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-dda9b2f7-7676-4d13-99ef-fdc6b1e3b994 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] Connected to 172.16.0.220:3260 {{(pid=75234) _connect_vol /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:644}}
Nov 29 03:29:38 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-dda9b2f7-7676-4d13-99ef-fdc6b1e3b994 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] HCTL ('33', '-', '-', 0) found on session 50 with lun 0 {{(pid=75234) get_hctl /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:770}}
Nov 29 03:29:38 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-dda9b2f7-7676-4d13-99ef-fdc6b1e3b994 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] Scanning host 33 c: -, t: -, l: 0) {{(pid=75234) scan_iscsi /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:795}}
Nov 29 03:29:38 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host33/scan {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:29:38 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host33/scan" returned: 0 in 0.018s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:29:38 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[47aa765c-7601-426c-8e47-ae0566d3b847]: (4, ('- - 0', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:29:38 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-dda9b2f7-7676-4d13-99ef-fdc6b1e3b994 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] Searching for a device in session 50 and hctl ['33', '*', '*', 0] yield: sdb {{(pid=75234) device_name_by_hctl /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:789}}
Nov 29 03:29:38 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-dda9b2f7-7676-4d13-99ef-fdc6b1e3b994 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] Connected to sdb using {'target_iqn': 'iqn.2010-10.org.openstack:43e92d4a-7e98-4777-8fa5-3cec396b86ce', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'AQnkxjFp', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False} {{(pid=75234) _connect_vol /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:672}}
Nov 29 03:29:39 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-dda9b2f7-7676-4d13-99ef-fdc6b1e3b994 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.330s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:87}}
Nov 29 03:29:39 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-dda9b2f7-7676-4d13-99ef-fdc6b1e3b994 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] <== connect_volume: return (1332ms) {'type': 'block', 'scsi_wwn': '23237313364396533', 'path': '/dev/sdb'} {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:203}}
Nov 29 03:29:39 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-dda9b2f7-7676-4d13-99ef-fdc6b1e3b994 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] [instance: 01d62e54-9957-40b1-8dfe-1783b64ffabf] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '23237313364396533', 'path': '/dev/sdb'} {{(pid=75234) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Nov 29 03:29:39 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-dda9b2f7-7676-4d13-99ef-fdc6b1e3b994 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-11-29T03:29:28Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-TestShelveInstance-server-1849172102',display_name='tempest-TestShelveInstance-server-1849172102',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-testshelveinstance-server-1849172102',id=44,image_ref='',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBPnJOw+AusvgDl0zYq6G0Nlkai7pjeaHxMjEe+mgW4X2uDyLlbOtVi5OTxN9x2/pgEhihEfkWnF2CarcswPn27UxxBjasnu8imtsEqjFxc/RzHA5mgFRB5eUL6c9v+ODYA==',key_name='tempest-TestShelveInstance-1141474138',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='53e55fcb8cd74895baec6f9528770bbc',ramdisk_id='',reservation_id='r-vhus4691',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TestShelveInstance-1683282958',owner_user_name='tempest-TestShelveInstance-1683282958-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-11-29T03:29:30Z,user_data=None,user_id='5728da78dc074c1993a8a2cc9b821ed6',uuid=01d62e54-9957-40b1-8dfe-1783b64ffabf,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "6dbd7c27-31c9-4ddc-a81b-85f1e742d160", "address": "fa:16:3e:57:6a:26", "network": {"id": "504616f5-0d6b-47a6-aaf5-663983428b41", "bridge": "br-int", "label": "tempest-TestShelveInstance-438404840-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "53e55fcb8cd74895baec6f9528770bbc", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap6dbd7c27-31", "ovs_interfaceid": "6dbd7c27-31c9-4ddc-a81b-85f1e742d160", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=75234) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Nov 29 03:29:39 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-dda9b2f7-7676-4d13-99ef-fdc6b1e3b994 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] Converting VIF {"id": "6dbd7c27-31c9-4ddc-a81b-85f1e742d160", "address": "fa:16:3e:57:6a:26", "network": {"id": "504616f5-0d6b-47a6-aaf5-663983428b41", "bridge": "br-int", "label": "tempest-TestShelveInstance-438404840-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "53e55fcb8cd74895baec6f9528770bbc", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap6dbd7c27-31", "ovs_interfaceid": "6dbd7c27-31c9-4ddc-a81b-85f1e742d160", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:29:39 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-dda9b2f7-7676-4d13-99ef-fdc6b1e3b994 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:57:6a:26,bridge_name='br-int',has_traffic_filtering=True,id=6dbd7c27-31c9-4ddc-a81b-85f1e742d160,network=Network(504616f5-0d6b-47a6-aaf5-663983428b41),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap6dbd7c27-31') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:29:39 user nova-compute[75234]: DEBUG nova.objects.instance [None req-dda9b2f7-7676-4d13-99ef-fdc6b1e3b994 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] Lazy-loading 'pci_devices' on Instance uuid 01d62e54-9957-40b1-8dfe-1783b64ffabf {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:29:39 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-dda9b2f7-7676-4d13-99ef-fdc6b1e3b994 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] [instance: 01d62e54-9957-40b1-8dfe-1783b64ffabf] End _get_guest_xml xml=<domain type="kvm">
Nov 29 03:29:39 user nova-compute[75234]:   <uuid>01d62e54-9957-40b1-8dfe-1783b64ffabf</uuid>
Nov 29 03:29:39 user nova-compute[75234]:   <name>instance-0000002c</name>
Nov 29 03:29:39 user nova-compute[75234]:   <memory>524288</memory>
Nov 29 03:29:39 user nova-compute[75234]:   <vcpu>1</vcpu>
Nov 29 03:29:39 user nova-compute[75234]:   <metadata>
Nov 29 03:29:39 user nova-compute[75234]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Nov 29 03:29:39 user nova-compute[75234]:       <nova:package version="0.0.0"/>
Nov 29 03:29:39 user nova-compute[75234]:       <nova:name>tempest-TestShelveInstance-server-1849172102</nova:name>
Nov 29 03:29:39 user nova-compute[75234]:       <nova:creationTime>2023-11-29 03:29:38</nova:creationTime>
Nov 29 03:29:39 user nova-compute[75234]:       <nova:flavor name="m1.tiny">
Nov 29 03:29:39 user nova-compute[75234]:         <nova:memory>512</nova:memory>
Nov 29 03:29:39 user nova-compute[75234]:         <nova:disk>1</nova:disk>
Nov 29 03:29:39 user nova-compute[75234]:         <nova:swap>0</nova:swap>
Nov 29 03:29:39 user nova-compute[75234]:         <nova:ephemeral>0</nova:ephemeral>
Nov 29 03:29:39 user nova-compute[75234]:         <nova:vcpus>1</nova:vcpus>
Nov 29 03:29:39 user nova-compute[75234]:       </nova:flavor>
Nov 29 03:29:39 user nova-compute[75234]:       <nova:owner>
Nov 29 03:29:39 user nova-compute[75234]:         <nova:user uuid="5728da78dc074c1993a8a2cc9b821ed6">tempest-TestShelveInstance-1683282958-project-member</nova:user>
Nov 29 03:29:39 user nova-compute[75234]:         <nova:project uuid="53e55fcb8cd74895baec6f9528770bbc">tempest-TestShelveInstance-1683282958</nova:project>
Nov 29 03:29:39 user nova-compute[75234]:       </nova:owner>
Nov 29 03:29:39 user nova-compute[75234]:       <nova:ports>
Nov 29 03:29:39 user nova-compute[75234]:         <nova:port uuid="6dbd7c27-31c9-4ddc-a81b-85f1e742d160">
Nov 29 03:29:39 user nova-compute[75234]:           <nova:ip type="fixed" address="10.0.0.13" ipVersion="4"/>
Nov 29 03:29:39 user nova-compute[75234]:         </nova:port>
Nov 29 03:29:39 user nova-compute[75234]:       </nova:ports>
Nov 29 03:29:39 user nova-compute[75234]:     </nova:instance>
Nov 29 03:29:39 user nova-compute[75234]:   </metadata>
Nov 29 03:29:39 user nova-compute[75234]:   <sysinfo type="smbios">
Nov 29 03:29:39 user nova-compute[75234]:     <system>
Nov 29 03:29:39 user nova-compute[75234]:       <entry name="manufacturer">OpenStack Foundation</entry>
Nov 29 03:29:39 user nova-compute[75234]:       <entry name="product">OpenStack Nova</entry>
Nov 29 03:29:39 user nova-compute[75234]:       <entry name="version">0.0.0</entry>
Nov 29 03:29:39 user nova-compute[75234]:       <entry name="serial">01d62e54-9957-40b1-8dfe-1783b64ffabf</entry>
Nov 29 03:29:39 user nova-compute[75234]:       <entry name="uuid">01d62e54-9957-40b1-8dfe-1783b64ffabf</entry>
Nov 29 03:29:39 user nova-compute[75234]:       <entry name="family">Virtual Machine</entry>
Nov 29 03:29:39 user nova-compute[75234]:     </system>
Nov 29 03:29:39 user nova-compute[75234]:   </sysinfo>
Nov 29 03:29:39 user nova-compute[75234]:   <os>
Nov 29 03:29:39 user nova-compute[75234]:     <type arch="x86_64" machine="pc">hvm</type>
Nov 29 03:29:39 user nova-compute[75234]:     <boot dev="hd"/>
Nov 29 03:29:39 user nova-compute[75234]:     <smbios mode="sysinfo"/>
Nov 29 03:29:39 user nova-compute[75234]:   </os>
Nov 29 03:29:39 user nova-compute[75234]:   <features>
Nov 29 03:29:39 user nova-compute[75234]:     <acpi/>
Nov 29 03:29:39 user nova-compute[75234]:     <apic/>
Nov 29 03:29:39 user nova-compute[75234]:     <vmcoreinfo/>
Nov 29 03:29:39 user nova-compute[75234]:   </features>
Nov 29 03:29:39 user nova-compute[75234]:   <clock offset="utc">
Nov 29 03:29:39 user nova-compute[75234]:     <timer name="pit" tickpolicy="delay"/>
Nov 29 03:29:39 user nova-compute[75234]:     <timer name="rtc" tickpolicy="catchup"/>
Nov 29 03:29:39 user nova-compute[75234]:     <timer name="hpet" present="no"/>
Nov 29 03:29:39 user nova-compute[75234]:   </clock>
Nov 29 03:29:39 user nova-compute[75234]:   <cpu mode="custom" match="exact">
Nov 29 03:29:39 user nova-compute[75234]:     <model>Nehalem</model>
Nov 29 03:29:39 user nova-compute[75234]:     <topology sockets="1" cores="1" threads="1"/>
Nov 29 03:29:39 user nova-compute[75234]:   </cpu>
Nov 29 03:29:39 user nova-compute[75234]:   <devices>
Nov 29 03:29:39 user nova-compute[75234]:     <disk type="block" device="disk">
Nov 29 03:29:39 user nova-compute[75234]:       <driver name="qemu" type="raw" cache="none" io="native"/>
Nov 29 03:29:39 user nova-compute[75234]:       <source dev="/dev/sdb"/>
Nov 29 03:29:39 user nova-compute[75234]:       <target dev="vda" bus="virtio"/>
Nov 29 03:29:39 user nova-compute[75234]:       <serial>43e92d4a-7e98-4777-8fa5-3cec396b86ce</serial>
Nov 29 03:29:39 user nova-compute[75234]:     </disk>
Nov 29 03:29:39 user nova-compute[75234]:     <interface type="ethernet">
Nov 29 03:29:39 user nova-compute[75234]:       <mac address="fa:16:3e:57:6a:26"/>
Nov 29 03:29:39 user nova-compute[75234]:       <model type="virtio"/>
Nov 29 03:29:39 user nova-compute[75234]:       <mtu size="1442"/>
Nov 29 03:29:39 user nova-compute[75234]:       <target dev="tap6dbd7c27-31"/>
Nov 29 03:29:39 user nova-compute[75234]:     </interface>
Nov 29 03:29:39 user nova-compute[75234]:     <serial type="pty">
Nov 29 03:29:39 user nova-compute[75234]:       <log file="/opt/stack/data/nova/instances/01d62e54-9957-40b1-8dfe-1783b64ffabf/console.log" append="off"/>
Nov 29 03:29:39 user nova-compute[75234]:     </serial>
Nov 29 03:29:39 user nova-compute[75234]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Nov 29 03:29:39 user nova-compute[75234]:     <video>
Nov 29 03:29:39 user nova-compute[75234]:       <model type="virtio"/>
Nov 29 03:29:39 user nova-compute[75234]:     </video>
Nov 29 03:29:39 user nova-compute[75234]:     <rng model="virtio">
Nov 29 03:29:39 user nova-compute[75234]:       <backend model="random">/dev/urandom</backend>
Nov 29 03:29:39 user nova-compute[75234]:     </rng>
Nov 29 03:29:39 user nova-compute[75234]:     <controller type="usb" index="0" model="none"/>
Nov 29 03:29:39 user nova-compute[75234]:     <memballoon model="virtio">
Nov 29 03:29:39 user nova-compute[75234]:       <stats period="10"/>
Nov 29 03:29:39 user nova-compute[75234]:     </memballoon>
Nov 29 03:29:39 user nova-compute[75234]:   </devices>
Nov 29 03:29:39 user nova-compute[75234]: </domain>
Nov 29 03:29:39 user nova-compute[75234]:  {{(pid=75234) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7568}}
Nov 29 03:29:39 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-dda9b2f7-7676-4d13-99ef-fdc6b1e3b994 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-11-29T03:29:28Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-TestShelveInstance-server-1849172102',display_name='tempest-TestShelveInstance-server-1849172102',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-testshelveinstance-server-1849172102',id=44,image_ref='',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBPnJOw+AusvgDl0zYq6G0Nlkai7pjeaHxMjEe+mgW4X2uDyLlbOtVi5OTxN9x2/pgEhihEfkWnF2CarcswPn27UxxBjasnu8imtsEqjFxc/RzHA5mgFRB5eUL6c9v+ODYA==',key_name='tempest-TestShelveInstance-1141474138',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='53e55fcb8cd74895baec6f9528770bbc',ramdisk_id='',reservation_id='r-vhus4691',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TestShelveInstance-1683282958',owner_user_name='tempest-TestShelveInstance-1683282958-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-11-29T03:29:30Z,user_data=None,user_id='5728da78dc074c1993a8a2cc9b821ed6',uuid=01d62e54-9957-40b1-8dfe-1783b64ffabf,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "6dbd7c27-31c9-4ddc-a81b-85f1e742d160", "address": "fa:16:3e:57:6a:26", "network": {"id": "504616f5-0d6b-47a6-aaf5-663983428b41", "bridge": "br-int", "label": "tempest-TestShelveInstance-438404840-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "53e55fcb8cd74895baec6f9528770bbc", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap6dbd7c27-31", "ovs_interfaceid": "6dbd7c27-31c9-4ddc-a81b-85f1e742d160", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Nov 29 03:29:39 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-dda9b2f7-7676-4d13-99ef-fdc6b1e3b994 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] Converting VIF {"id": "6dbd7c27-31c9-4ddc-a81b-85f1e742d160", "address": "fa:16:3e:57:6a:26", "network": {"id": "504616f5-0d6b-47a6-aaf5-663983428b41", "bridge": "br-int", "label": "tempest-TestShelveInstance-438404840-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "53e55fcb8cd74895baec6f9528770bbc", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap6dbd7c27-31", "ovs_interfaceid": "6dbd7c27-31c9-4ddc-a81b-85f1e742d160", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:29:39 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-dda9b2f7-7676-4d13-99ef-fdc6b1e3b994 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:57:6a:26,bridge_name='br-int',has_traffic_filtering=True,id=6dbd7c27-31c9-4ddc-a81b-85f1e742d160,network=Network(504616f5-0d6b-47a6-aaf5-663983428b41),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap6dbd7c27-31') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:29:39 user nova-compute[75234]: DEBUG os_vif [None req-dda9b2f7-7676-4d13-99ef-fdc6b1e3b994 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:57:6a:26,bridge_name='br-int',has_traffic_filtering=True,id=6dbd7c27-31c9-4ddc-a81b-85f1e742d160,network=Network(504616f5-0d6b-47a6-aaf5-663983428b41),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap6dbd7c27-31') {{(pid=75234) plug /opt/stack/data/venv/lib/python3.10/site-packages/os_vif/__init__.py:76}}
Nov 29 03:29:39 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:29:39 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:29:39 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Nov 29 03:29:39 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:29:39 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbCreateCommand(_result=None, table=QoS, columns={'type': 'linux-noop', 'external_ids': {'id': '94f3d8db-41ef-5088-9907-ebfda2e89d5f', '_type': 'linux-noop'}}, row=False) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:29:39 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:29:39 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:29:39 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:29:39 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tap6dbd7c27-31, may_exist=True, interface_attrs={}) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:29:39 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Port, record=tap6dbd7c27-31, col_values=(('qos', UUID('b6b3be26-a6af-4c78-9637-a24eb15e301a')),), if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:29:39 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DbSetCommand(_result=None, table=Interface, record=tap6dbd7c27-31, col_values=(('external_ids', {'iface-id': '6dbd7c27-31c9-4ddc-a81b-85f1e742d160', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:57:6a:26', 'vm-uuid': '01d62e54-9957-40b1-8dfe-1783b64ffabf'}),), if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:29:39 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:29:39 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:29:39 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:29:39 user nova-compute[75234]: INFO os_vif [None req-dda9b2f7-7676-4d13-99ef-fdc6b1e3b994 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:57:6a:26,bridge_name='br-int',has_traffic_filtering=True,id=6dbd7c27-31c9-4ddc-a81b-85f1e742d160,network=Network(504616f5-0d6b-47a6-aaf5-663983428b41),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap6dbd7c27-31')
Nov 29 03:29:39 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-dda9b2f7-7676-4d13-99ef-fdc6b1e3b994 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] No BDM found with device name vda, not building metadata. {{(pid=75234) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12089}}
Nov 29 03:29:39 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-dda9b2f7-7676-4d13-99ef-fdc6b1e3b994 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] No VIF found with MAC fa:16:3e:57:6a:26, not building metadata {{(pid=75234) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Nov 29 03:29:41 user nova-compute[75234]: DEBUG os_brick.utils [None req-cbb5b5ed-441c-455b-9e87-49b73c2c4cb0 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:176}}
Nov 29 03:29:41 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:29:41 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.013s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:29:41 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[a8397730-6410-4ec8-a6f1-8b657918a35e]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:dd1368c5ba2c\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:29:41 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt -v / -n -o SOURCE {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:29:41 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:29:41 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt -v / -n -o SOURCE" returned: 0 in 0.026s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:29:41 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[4d35a3b4-10c6-4221-b07a-3e4949f8ecc5]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:29:41 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:29:41 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.017s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:29:41 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[1e55515f-2a9c-40ad-a5cd-fac9f803ce2f]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:29:41 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[7917fd33-0f4c-449a-a118-44170d9e948c]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:29:41 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-cbb5b5ed-441c-455b-9e87-49b73c2c4cb0 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Running cmd (subprocess): nvme version {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:29:41 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:29:41 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-cbb5b5ed-441c-455b-9e87-49b73c2c4cb0 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] 'nvme version' failed. Not Retrying. {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:473}}
Nov 29 03:29:41 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.nvmeof [None req-cbb5b5ed-441c-455b-9e87-49b73c2c4cb0 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] nvme not present on system {{(pid=75234) nvme_present /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/nvmeof.py:756}}
Nov 29 03:29:41 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:29:41 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:473}}
Nov 29 03:29:41 user nova-compute[75234]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Nov 29 03:29:41 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[75879d51-016f-4bf8-a284-8407dcb6f1ce]: (4, '') {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:29:41 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.lightos [None req-cbb5b5ed-441c-455b-9e87-49b73c2c4cb0 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=75234) find_dsc /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/lightos.py:98}}
Nov 29 03:29:41 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.lightos [None req-cbb5b5ed-441c-455b-9e87-49b73c2c4cb0 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=75234) get_connector_properties /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/lightos.py:76}}
Nov 29 03:29:41 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.lightos [None req-cbb5b5ed-441c-455b-9e87-49b73c2c4cb0 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] LIGHTOS: no hostnqn found. {{(pid=75234) get_connector_properties /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/lightos.py:84}}
Nov 29 03:29:41 user nova-compute[75234]: DEBUG os_brick.utils [None req-cbb5b5ed-441c-455b-9e87-49b73c2c4cb0 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] <== get_connector_properties: return (150ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:dd1368c5ba2c', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:203}}
Nov 29 03:29:41 user nova-compute[75234]: DEBUG nova.virt.block_device [None req-cbb5b5ed-441c-455b-9e87-49b73c2c4cb0 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: 1e9263fc-5735-420f-a116-8f60c8736faf] Updating existing volume attachment record: c6bcc9c9-0bc3-4c41-9b8e-2208c430653c {{(pid=75234) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Nov 29 03:29:41 user nova-compute[75234]: DEBUG nova.compute.manager [req-d2bbd23b-af21-44ff-a744-3633067d20ee req-3d8fc3d5-94c2-4e53-aa0e-28655bad9648 service nova] [instance: 01d62e54-9957-40b1-8dfe-1783b64ffabf] Received event network-vif-plugged-6dbd7c27-31c9-4ddc-a81b-85f1e742d160 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:29:41 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-d2bbd23b-af21-44ff-a744-3633067d20ee req-3d8fc3d5-94c2-4e53-aa0e-28655bad9648 service nova] Acquiring lock "01d62e54-9957-40b1-8dfe-1783b64ffabf-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:29:41 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-d2bbd23b-af21-44ff-a744-3633067d20ee req-3d8fc3d5-94c2-4e53-aa0e-28655bad9648 service nova] Lock "01d62e54-9957-40b1-8dfe-1783b64ffabf-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:29:41 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-d2bbd23b-af21-44ff-a744-3633067d20ee req-3d8fc3d5-94c2-4e53-aa0e-28655bad9648 service nova] Lock "01d62e54-9957-40b1-8dfe-1783b64ffabf-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:29:41 user nova-compute[75234]: DEBUG nova.compute.manager [req-d2bbd23b-af21-44ff-a744-3633067d20ee req-3d8fc3d5-94c2-4e53-aa0e-28655bad9648 service nova] [instance: 01d62e54-9957-40b1-8dfe-1783b64ffabf] No waiting events found dispatching network-vif-plugged-6dbd7c27-31c9-4ddc-a81b-85f1e742d160 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:29:41 user nova-compute[75234]: WARNING nova.compute.manager [req-d2bbd23b-af21-44ff-a744-3633067d20ee req-3d8fc3d5-94c2-4e53-aa0e-28655bad9648 service nova] [instance: 01d62e54-9957-40b1-8dfe-1783b64ffabf] Received unexpected event network-vif-plugged-6dbd7c27-31c9-4ddc-a81b-85f1e742d160 for instance with vm_state building and task_state spawning.
Nov 29 03:29:41 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:29:41 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:29:41 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:29:42 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:29:43 user nova-compute[75234]: DEBUG nova.compute.manager [None req-dda9b2f7-7676-4d13-99ef-fdc6b1e3b994 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] [instance: 01d62e54-9957-40b1-8dfe-1783b64ffabf] Instance event wait completed in 0 seconds for  {{(pid=75234) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Nov 29 03:29:43 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-dda9b2f7-7676-4d13-99ef-fdc6b1e3b994 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] [instance: 01d62e54-9957-40b1-8dfe-1783b64ffabf] Guest created on hypervisor {{(pid=75234) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4421}}
Nov 29 03:29:43 user nova-compute[75234]: INFO nova.virt.libvirt.driver [-] [instance: 01d62e54-9957-40b1-8dfe-1783b64ffabf] Instance spawned successfully.
Nov 29 03:29:43 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-dda9b2f7-7676-4d13-99ef-fdc6b1e3b994 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] [instance: 01d62e54-9957-40b1-8dfe-1783b64ffabf] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:904}}
Nov 29 03:29:43 user nova-compute[75234]: DEBUG nova.compute.manager [req-b26b79c6-5c9e-4a96-873b-82ecefa444ee req-a862bca8-4d7c-40c5-b59e-617d7c504fa3 service nova] [instance: 01d62e54-9957-40b1-8dfe-1783b64ffabf] Received event network-vif-plugged-6dbd7c27-31c9-4ddc-a81b-85f1e742d160 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:29:43 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-b26b79c6-5c9e-4a96-873b-82ecefa444ee req-a862bca8-4d7c-40c5-b59e-617d7c504fa3 service nova] Acquiring lock "01d62e54-9957-40b1-8dfe-1783b64ffabf-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:29:43 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-b26b79c6-5c9e-4a96-873b-82ecefa444ee req-a862bca8-4d7c-40c5-b59e-617d7c504fa3 service nova] Lock "01d62e54-9957-40b1-8dfe-1783b64ffabf-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:29:43 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-b26b79c6-5c9e-4a96-873b-82ecefa444ee req-a862bca8-4d7c-40c5-b59e-617d7c504fa3 service nova] Lock "01d62e54-9957-40b1-8dfe-1783b64ffabf-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:29:43 user nova-compute[75234]: DEBUG nova.compute.manager [req-b26b79c6-5c9e-4a96-873b-82ecefa444ee req-a862bca8-4d7c-40c5-b59e-617d7c504fa3 service nova] [instance: 01d62e54-9957-40b1-8dfe-1783b64ffabf] No waiting events found dispatching network-vif-plugged-6dbd7c27-31c9-4ddc-a81b-85f1e742d160 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:29:43 user nova-compute[75234]: WARNING nova.compute.manager [req-b26b79c6-5c9e-4a96-873b-82ecefa444ee req-a862bca8-4d7c-40c5-b59e-617d7c504fa3 service nova] [instance: 01d62e54-9957-40b1-8dfe-1783b64ffabf] Received unexpected event network-vif-plugged-6dbd7c27-31c9-4ddc-a81b-85f1e742d160 for instance with vm_state building and task_state spawning.
Nov 29 03:29:43 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-dda9b2f7-7676-4d13-99ef-fdc6b1e3b994 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] [instance: 01d62e54-9957-40b1-8dfe-1783b64ffabf] Found default for hw_cdrom_bus of ide {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:29:43 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-dda9b2f7-7676-4d13-99ef-fdc6b1e3b994 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] [instance: 01d62e54-9957-40b1-8dfe-1783b64ffabf] Found default for hw_disk_bus of virtio {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:29:43 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-dda9b2f7-7676-4d13-99ef-fdc6b1e3b994 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] [instance: 01d62e54-9957-40b1-8dfe-1783b64ffabf] Found default for hw_input_bus of None {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:29:43 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-dda9b2f7-7676-4d13-99ef-fdc6b1e3b994 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] [instance: 01d62e54-9957-40b1-8dfe-1783b64ffabf] Found default for hw_pointer_model of None {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:29:43 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-dda9b2f7-7676-4d13-99ef-fdc6b1e3b994 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] [instance: 01d62e54-9957-40b1-8dfe-1783b64ffabf] Found default for hw_video_model of virtio {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:29:43 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-dda9b2f7-7676-4d13-99ef-fdc6b1e3b994 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] [instance: 01d62e54-9957-40b1-8dfe-1783b64ffabf] Found default for hw_vif_model of virtio {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:29:43 user nova-compute[75234]: INFO nova.compute.manager [None req-dda9b2f7-7676-4d13-99ef-fdc6b1e3b994 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] [instance: 01d62e54-9957-40b1-8dfe-1783b64ffabf] Took 5.45 seconds to spawn the instance on the hypervisor.
Nov 29 03:29:43 user nova-compute[75234]: DEBUG nova.compute.manager [None req-dda9b2f7-7676-4d13-99ef-fdc6b1e3b994 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] [instance: 01d62e54-9957-40b1-8dfe-1783b64ffabf] Checking state {{(pid=75234) _get_power_state /opt/stack/nova/nova/compute/manager.py:1782}}
Nov 29 03:29:43 user nova-compute[75234]: INFO nova.compute.manager [None req-dda9b2f7-7676-4d13-99ef-fdc6b1e3b994 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] [instance: 01d62e54-9957-40b1-8dfe-1783b64ffabf] Took 14.32 seconds to build instance.
Nov 29 03:29:43 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-dda9b2f7-7676-4d13-99ef-fdc6b1e3b994 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] Lock "01d62e54-9957-40b1-8dfe-1783b64ffabf" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 14.435s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:29:44 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:29:46 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-a124e922-fc6a-431a-ba05-92a9e593c1a7 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Acquiring lock "cb0f042e-43c0-43aa-b243-cfc78a7b2695" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:29:46 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-a124e922-fc6a-431a-ba05-92a9e593c1a7 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Lock "cb0f042e-43c0-43aa-b243-cfc78a7b2695" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:29:46 user nova-compute[75234]: DEBUG nova.compute.manager [None req-a124e922-fc6a-431a-ba05-92a9e593c1a7 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] [instance: cb0f042e-43c0-43aa-b243-cfc78a7b2695] Starting instance... {{(pid=75234) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2422}}
Nov 29 03:29:47 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-a124e922-fc6a-431a-ba05-92a9e593c1a7 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:29:47 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-a124e922-fc6a-431a-ba05-92a9e593c1a7 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:29:47 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-a124e922-fc6a-431a-ba05-92a9e593c1a7 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=75234) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2366}}
Nov 29 03:29:47 user nova-compute[75234]: INFO nova.compute.claims [None req-a124e922-fc6a-431a-ba05-92a9e593c1a7 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] [instance: cb0f042e-43c0-43aa-b243-cfc78a7b2695] Claim successful on node user
Nov 29 03:29:47 user nova-compute[75234]: DEBUG nova.compute.provider_tree [None req-a124e922-fc6a-431a-ba05-92a9e593c1a7 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Inventory has not changed in ProviderTree for provider: 4d291220-7c37-4839-a0e3-b23c66502b74 {{(pid=75234) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Nov 29 03:29:47 user nova-compute[75234]: DEBUG nova.scheduler.client.report [None req-a124e922-fc6a-431a-ba05-92a9e593c1a7 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Inventory has not changed for provider 4d291220-7c37-4839-a0e3-b23c66502b74 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=75234) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Nov 29 03:29:47 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-a124e922-fc6a-431a-ba05-92a9e593c1a7 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.366s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:29:47 user nova-compute[75234]: DEBUG nova.compute.manager [None req-a124e922-fc6a-431a-ba05-92a9e593c1a7 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] [instance: cb0f042e-43c0-43aa-b243-cfc78a7b2695] Start building networks asynchronously for instance. {{(pid=75234) _build_resources /opt/stack/nova/nova/compute/manager.py:2819}}
Nov 29 03:29:47 user nova-compute[75234]: DEBUG nova.compute.manager [None req-a124e922-fc6a-431a-ba05-92a9e593c1a7 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] [instance: cb0f042e-43c0-43aa-b243-cfc78a7b2695] Allocating IP information in the background. {{(pid=75234) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1972}}
Nov 29 03:29:47 user nova-compute[75234]: DEBUG nova.network.neutron [None req-a124e922-fc6a-431a-ba05-92a9e593c1a7 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] [instance: cb0f042e-43c0-43aa-b243-cfc78a7b2695] allocate_for_instance() {{(pid=75234) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1156}}
Nov 29 03:29:47 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-a124e922-fc6a-431a-ba05-92a9e593c1a7 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] [instance: cb0f042e-43c0-43aa-b243-cfc78a7b2695] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Nov 29 03:29:47 user nova-compute[75234]: DEBUG nova.compute.manager [None req-a124e922-fc6a-431a-ba05-92a9e593c1a7 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] [instance: cb0f042e-43c0-43aa-b243-cfc78a7b2695] Start building block device mappings for instance. {{(pid=75234) _build_resources /opt/stack/nova/nova/compute/manager.py:2854}}
Nov 29 03:29:47 user nova-compute[75234]: DEBUG nova.policy [None req-a124e922-fc6a-431a-ba05-92a9e593c1a7 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '3a02618f56ea45898ab4783a4f198b5d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '74f132d6666c47b8b43869df6fee6de3', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=75234) authorize /opt/stack/nova/nova/policy.py:203}}
Nov 29 03:29:47 user nova-compute[75234]: DEBUG nova.compute.manager [None req-a124e922-fc6a-431a-ba05-92a9e593c1a7 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] [instance: cb0f042e-43c0-43aa-b243-cfc78a7b2695] Start spawning the instance on the hypervisor. {{(pid=75234) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2628}}
Nov 29 03:29:47 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-a124e922-fc6a-431a-ba05-92a9e593c1a7 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] [instance: cb0f042e-43c0-43aa-b243-cfc78a7b2695] Creating instance directory {{(pid=75234) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4727}}
Nov 29 03:29:47 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-a124e922-fc6a-431a-ba05-92a9e593c1a7 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] [instance: cb0f042e-43c0-43aa-b243-cfc78a7b2695] Creating image(s)
Nov 29 03:29:47 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-a124e922-fc6a-431a-ba05-92a9e593c1a7 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Acquiring lock "/opt/stack/data/nova/instances/cb0f042e-43c0-43aa-b243-cfc78a7b2695/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:29:47 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-a124e922-fc6a-431a-ba05-92a9e593c1a7 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Lock "/opt/stack/data/nova/instances/cb0f042e-43c0-43aa-b243-cfc78a7b2695/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:29:47 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-a124e922-fc6a-431a-ba05-92a9e593c1a7 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Lock "/opt/stack/data/nova/instances/cb0f042e-43c0-43aa-b243-cfc78a7b2695/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.002s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:29:47 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-a124e922-fc6a-431a-ba05-92a9e593c1a7 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Acquiring lock "d54adabc8d95c75567f7840e57b0a84314361a7e" by "nova.virt.libvirt.imagebackend.Image.cache.<locals>.fetch_func_sync" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:29:47 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-a124e922-fc6a-431a-ba05-92a9e593c1a7 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Lock "d54adabc8d95c75567f7840e57b0a84314361a7e" acquired by "nova.virt.libvirt.imagebackend.Image.cache.<locals>.fetch_func_sync" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:29:47 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:29:48 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-a124e922-fc6a-431a-ba05-92a9e593c1a7 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/d54adabc8d95c75567f7840e57b0a84314361a7e.part --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:29:48 user nova-compute[75234]: DEBUG nova.network.neutron [None req-a124e922-fc6a-431a-ba05-92a9e593c1a7 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] [instance: cb0f042e-43c0-43aa-b243-cfc78a7b2695] Successfully created port: 8f9642c3-9e10-4aa7-b5a2-53bb68ad96dc {{(pid=75234) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:548}}
Nov 29 03:29:48 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-a124e922-fc6a-431a-ba05-92a9e593c1a7 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/d54adabc8d95c75567f7840e57b0a84314361a7e.part --force-share --output=json" returned: 0 in 0.201s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:29:48 user nova-compute[75234]: DEBUG nova.virt.images [None req-a124e922-fc6a-431a-ba05-92a9e593c1a7 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] 39c09b70-6b0e-4a2d-b9f7-1f76a50a094d was qcow2, converting to raw {{(pid=75234) fetch_to_raw /opt/stack/nova/nova/virt/images.py:165}}
Nov 29 03:29:48 user nova-compute[75234]: DEBUG nova.privsep.utils [None req-a124e922-fc6a-431a-ba05-92a9e593c1a7 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Path '/opt/stack/data/nova/instances' supports direct I/O {{(pid=75234) supports_direct_io /opt/stack/nova/nova/privsep/utils.py:63}}
Nov 29 03:29:48 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-a124e922-fc6a-431a-ba05-92a9e593c1a7 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Running cmd (subprocess): qemu-img convert -t none -O raw -f qcow2 /opt/stack/data/nova/instances/_base/d54adabc8d95c75567f7840e57b0a84314361a7e.part /opt/stack/data/nova/instances/_base/d54adabc8d95c75567f7840e57b0a84314361a7e.converted {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:29:49 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-a124e922-fc6a-431a-ba05-92a9e593c1a7 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] CMD "qemu-img convert -t none -O raw -f qcow2 /opt/stack/data/nova/instances/_base/d54adabc8d95c75567f7840e57b0a84314361a7e.part /opt/stack/data/nova/instances/_base/d54adabc8d95c75567f7840e57b0a84314361a7e.converted" returned: 0 in 0.428s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:29:49 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-a124e922-fc6a-431a-ba05-92a9e593c1a7 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/d54adabc8d95c75567f7840e57b0a84314361a7e.converted --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:29:49 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-a124e922-fc6a-431a-ba05-92a9e593c1a7 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/d54adabc8d95c75567f7840e57b0a84314361a7e.converted --force-share --output=json" returned: 0 in 0.240s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:29:49 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-a124e922-fc6a-431a-ba05-92a9e593c1a7 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Lock "d54adabc8d95c75567f7840e57b0a84314361a7e" "released" by "nova.virt.libvirt.imagebackend.Image.cache.<locals>.fetch_func_sync" :: held 1.736s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:29:49 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-a124e922-fc6a-431a-ba05-92a9e593c1a7 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/d54adabc8d95c75567f7840e57b0a84314361a7e --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:29:49 user nova-compute[75234]: DEBUG nova.network.neutron [None req-a124e922-fc6a-431a-ba05-92a9e593c1a7 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] [instance: cb0f042e-43c0-43aa-b243-cfc78a7b2695] Successfully updated port: 8f9642c3-9e10-4aa7-b5a2-53bb68ad96dc {{(pid=75234) _update_port /opt/stack/nova/nova/network/neutron.py:586}}
Nov 29 03:29:49 user nova-compute[75234]: DEBUG nova.compute.manager [None req-cbb5b5ed-441c-455b-9e87-49b73c2c4cb0 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: 1e9263fc-5735-420f-a116-8f60c8736faf] Start spawning the instance on the hypervisor. {{(pid=75234) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2628}}
Nov 29 03:29:49 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-cbb5b5ed-441c-455b-9e87-49b73c2c4cb0 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: 1e9263fc-5735-420f-a116-8f60c8736faf] Creating instance directory {{(pid=75234) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4727}}
Nov 29 03:29:49 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-cbb5b5ed-441c-455b-9e87-49b73c2c4cb0 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: 1e9263fc-5735-420f-a116-8f60c8736faf] Creating image(s)
Nov 29 03:29:49 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-cbb5b5ed-441c-455b-9e87-49b73c2c4cb0 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: 1e9263fc-5735-420f-a116-8f60c8736faf] Did not create local disks {{(pid=75234) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4863}}
Nov 29 03:29:49 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-cbb5b5ed-441c-455b-9e87-49b73c2c4cb0 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: 1e9263fc-5735-420f-a116-8f60c8736faf] Ensure instance console log exists: /opt/stack/data/nova/instances/1e9263fc-5735-420f-a116-8f60c8736faf/console.log {{(pid=75234) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4613}}
Nov 29 03:29:49 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-cbb5b5ed-441c-455b-9e87-49b73c2c4cb0 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:29:49 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-cbb5b5ed-441c-455b-9e87-49b73c2c4cb0 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:29:49 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-cbb5b5ed-441c-455b-9e87-49b73c2c4cb0 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:29:49 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-cbb5b5ed-441c-455b-9e87-49b73c2c4cb0 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: 1e9263fc-5735-420f-a116-8f60c8736faf] Start _get_guest_xml network_info=[{"id": "f836309d-8f97-4521-aa57-d6516a9e7528", "address": "fa:16:3e:1e:0c:76", "network": {"id": "a3f397d3-fd75-4a6e-a815-cfcedcca15fb", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-678570085-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "81a265d3c15b4e9d826b35ddbd0ff443", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapf836309d-8f", "ovs_interfaceid": "f836309d-8f97-4521-aa57-d6516a9e7528", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, '/dev/vda': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum=<?>,container_format=<?>,created_at=<?>,direct_url=<?>,disk_format=<?>,id=<?>,min_disk=0,min_ram=0,name=<?>,owner=<?>,properties=ImageMetaProps,protected=<?>,size=1073741824,status='active',tags=<?>,updated_at=<?>,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [], 'ephemerals': [], 'block_device_mapping': [{'mount_device': '/dev/vda', 'attachment_id': 'c6bcc9c9-0bc3-4c41-9b8e-2208c430653c', 'disk_bus': 'virtio', 'device_type': 'disk', 'boot_index': 0, 'guest_format': None, 'delete_on_termination': True, 'connection_info': {'driver_volume_type': 'iscsi', 'data': {'target_iqn': 'iqn.2010-10.org.openstack:d1f13879-6e1b-4fe3-b97d-066a9e657ef7', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'VpKRqEQa', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}, 'status': 'reserved', 'instance': '1e9263fc-5735-420f-a116-8f60c8736faf', 'attached_at': '', 'detached_at': '', 'volume_id': 'd1f13879-6e1b-4fe3-b97d-066a9e657ef7', 'serial': 'd1f13879-6e1b-4fe3-b97d-066a9e657ef7'}, 'volume_type': None} {{(pid=75234) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7562}}
Nov 29 03:29:49 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-a124e922-fc6a-431a-ba05-92a9e593c1a7 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Acquiring lock "refresh_cache-cb0f042e-43c0-43aa-b243-cfc78a7b2695" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:29:49 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-a124e922-fc6a-431a-ba05-92a9e593c1a7 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Acquired lock "refresh_cache-cb0f042e-43c0-43aa-b243-cfc78a7b2695" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:29:49 user nova-compute[75234]: DEBUG nova.network.neutron [None req-a124e922-fc6a-431a-ba05-92a9e593c1a7 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] [instance: cb0f042e-43c0-43aa-b243-cfc78a7b2695] Building network info cache for instance {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2009}}
Nov 29 03:29:49 user nova-compute[75234]: DEBUG nova.compute.manager [req-1a263c33-061a-4149-bf2a-a8070f5bf538 req-97feda1a-8c3f-4eb3-81e2-a9998d4ec0b4 service nova] [instance: cb0f042e-43c0-43aa-b243-cfc78a7b2695] Received event network-changed-8f9642c3-9e10-4aa7-b5a2-53bb68ad96dc {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:29:49 user nova-compute[75234]: DEBUG nova.compute.manager [req-1a263c33-061a-4149-bf2a-a8070f5bf538 req-97feda1a-8c3f-4eb3-81e2-a9998d4ec0b4 service nova] [instance: cb0f042e-43c0-43aa-b243-cfc78a7b2695] Refreshing instance network info cache due to event network-changed-8f9642c3-9e10-4aa7-b5a2-53bb68ad96dc. {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11117}}
Nov 29 03:29:49 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-1a263c33-061a-4149-bf2a-a8070f5bf538 req-97feda1a-8c3f-4eb3-81e2-a9998d4ec0b4 service nova] Acquiring lock "refresh_cache-cb0f042e-43c0-43aa-b243-cfc78a7b2695" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:29:49 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-cbb5b5ed-441c-455b-9e87-49b73c2c4cb0 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:29:49 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-cbb5b5ed-441c-455b-9e87-49b73c2c4cb0 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:29:49 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-cbb5b5ed-441c-455b-9e87-49b73c2c4cb0 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Searching host: 'user' for CPU controller through CGroups V1... {{(pid=75234) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1652}}
Nov 29 03:29:49 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-cbb5b5ed-441c-455b-9e87-49b73c2c4cb0 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] CPU controller missing on host. {{(pid=75234) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1662}}
Nov 29 03:29:49 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-cbb5b5ed-441c-455b-9e87-49b73c2c4cb0 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Searching host: 'user' for CPU controller through CGroups V2... {{(pid=75234) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1671}}
Nov 29 03:29:49 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-cbb5b5ed-441c-455b-9e87-49b73c2c4cb0 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] CPU controller found on host. {{(pid=75234) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1678}}
Nov 29 03:29:49 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-cbb5b5ed-441c-455b-9e87-49b73c2c4cb0 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=75234) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5400}}
Nov 29 03:29:49 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-cbb5b5ed-441c-455b-9e87-49b73c2c4cb0 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-11-29T03:02:04Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='1',id=6,is_public=True,memory_mb=512,name='m1.tiny',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum=<?>,container_format=<?>,created_at=<?>,direct_url=<?>,disk_format=<?>,id=<?>,min_disk=0,min_ram=0,name=<?>,owner=<?>,properties=ImageMetaProps,protected=<?>,size=1073741824,status='active',tags=<?>,updated_at=<?>,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:563}}
Nov 29 03:29:49 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-cbb5b5ed-441c-455b-9e87-49b73c2c4cb0 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Flavor limits 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:348}}
Nov 29 03:29:49 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-cbb5b5ed-441c-455b-9e87-49b73c2c4cb0 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Image limits 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:352}}
Nov 29 03:29:49 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-cbb5b5ed-441c-455b-9e87-49b73c2c4cb0 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Flavor pref 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:388}}
Nov 29 03:29:49 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-cbb5b5ed-441c-455b-9e87-49b73c2c4cb0 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Image pref 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:392}}
Nov 29 03:29:49 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-cbb5b5ed-441c-455b-9e87-49b73c2c4cb0 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:430}}
Nov 29 03:29:49 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-cbb5b5ed-441c-455b-9e87-49b73c2c4cb0 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:569}}
Nov 29 03:29:49 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-cbb5b5ed-441c-455b-9e87-49b73c2c4cb0 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=75234) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:471}}
Nov 29 03:29:49 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-cbb5b5ed-441c-455b-9e87-49b73c2c4cb0 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Got 1 possible topologies {{(pid=75234) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:501}}
Nov 29 03:29:49 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-cbb5b5ed-441c-455b-9e87-49b73c2c4cb0 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:575}}
Nov 29 03:29:49 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-cbb5b5ed-441c-455b-9e87-49b73c2c4cb0 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:577}}
Nov 29 03:29:49 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-cbb5b5ed-441c-455b-9e87-49b73c2c4cb0 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: 1e9263fc-5735-420f-a116-8f60c8736faf] Calling os-brick to attach iSCSI Volume {{(pid=75234) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Nov 29 03:29:49 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-cbb5b5ed-441c-455b-9e87-49b73c2c4cb0 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f8a5050c9a0>, 'connection_properties': {'target_iqn': 'iqn.2010-10.org.openstack:d1f13879-6e1b-4fe3-b97d-066a9e657ef7', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'VpKRqEQa', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:176}}
Nov 29 03:29:49 user nova-compute[75234]: WARNING os_brick.initiator.connectors.base [None req-cbb5b5ed-441c-455b-9e87-49b73c2c4cb0 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Nov 29 03:29:49 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-cbb5b5ed-441c-455b-9e87-49b73c2c4cb0 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:68}}
Nov 29 03:29:49 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-cbb5b5ed-441c-455b-9e87-49b73c2c4cb0 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.002s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:73}}
Nov 29 03:29:49 user nova-compute[75234]: WARNING os_brick.initiator.connectors.base [None req-cbb5b5ed-441c-455b-9e87-49b73c2c4cb0 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Nov 29 03:29:49 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-cbb5b5ed-441c-455b-9e87-49b73c2c4cb0 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:d1f13879-6e1b-4fe3-b97d-066a9e657ef7" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:68}}
Nov 29 03:29:49 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-cbb5b5ed-441c-455b-9e87-49b73c2c4cb0 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:d1f13879-6e1b-4fe3-b97d-066a9e657ef7" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:73}}
Nov 29 03:29:49 user nova-compute[75234]: INFO os_brick.initiator.connectors.iscsi [None req-cbb5b5ed-441c-455b-9e87-49b73c2c4cb0 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Nov 29 03:29:49 user nova-compute[75234]: DEBUG nova.network.neutron [None req-a124e922-fc6a-431a-ba05-92a9e593c1a7 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] [instance: cb0f042e-43c0-43aa-b243-cfc78a7b2695] Instance cache missing network info. {{(pid=75234) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3322}}
Nov 29 03:29:49 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:d1f13879-6e1b-4fe3-b97d-066a9e657ef7 -p 172.16.0.220:3260 {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:29:49 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:d1f13879-6e1b-4fe3-b97d-066a9e657ef7 -p 172.16.0.220:3260" returned: 21 in 0.019s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:29:49 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[e332f33a-7a71-4fbc-872c-2180d9f18564]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:29:49 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-cbb5b5ed-441c-455b-9e87-49b73c2c4cb0 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Nov 29 03:29:49 user nova-compute[75234]:  {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:29:49 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:d1f13879-6e1b-4fe3-b97d-066a9e657ef7 -p 172.16.0.220:3260 --interface default --op new {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:29:49 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-a124e922-fc6a-431a-ba05-92a9e593c1a7 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/d54adabc8d95c75567f7840e57b0a84314361a7e --force-share --output=json" returned: 0 in 0.200s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:29:49 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-a124e922-fc6a-431a-ba05-92a9e593c1a7 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Acquiring lock "d54adabc8d95c75567f7840e57b0a84314361a7e" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:29:49 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-a124e922-fc6a-431a-ba05-92a9e593c1a7 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Lock "d54adabc8d95c75567f7840e57b0a84314361a7e" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:29:49 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-a124e922-fc6a-431a-ba05-92a9e593c1a7 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/d54adabc8d95c75567f7840e57b0a84314361a7e --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:29:49 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:d1f13879-6e1b-4fe3-b97d-066a9e657ef7 -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.030s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:29:49 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[159ca19b-b4fa-44b3-829d-8acdabab82e2]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:d1f13879-6e1b-4fe3-b97d-066a9e657ef7] added\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:29:49 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-cbb5b5ed-441c-455b-9e87-49b73c2c4cb0 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:d1f13879-6e1b-4fe3-b97d-066a9e657ef7] added
Nov 29 03:29:49 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:29:49 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:d1f13879-6e1b-4fe3-b97d-066a9e657ef7 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:29:49 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:d1f13879-6e1b-4fe3-b97d-066a9e657ef7 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.020s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:29:49 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[8c8da523-b905-4735-bad0-74f9b7693f5d]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:29:49 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-cbb5b5ed-441c-455b-9e87-49b73c2c4cb0 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:29:49 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:d1f13879-6e1b-4fe3-b97d-066a9e657ef7 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:29:49 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:d1f13879-6e1b-4fe3-b97d-066a9e657ef7 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.021s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:29:49 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[f2b771b7-495a-49d7-afc7-2e1edee881b8]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:29:49 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-cbb5b5ed-441c-455b-9e87-49b73c2c4cb0 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:29:49 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:d1f13879-6e1b-4fe3-b97d-066a9e657ef7 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v VpKRqEQa {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:29:49 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:d1f13879-6e1b-4fe3-b97d-066a9e657ef7 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v VpKRqEQa" returned: 0 in 0.018s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:29:49 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[14f08278-f5a5-4ca1-b9b5-c1d5ee82f15f]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:29:49 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-cbb5b5ed-441c-455b-9e87-49b73c2c4cb0 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'VpKRqEQa'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:29:49 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:d1f13879-6e1b-4fe3-b97d-066a9e657ef7 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:29:49 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:d1f13879-6e1b-4fe3-b97d-066a9e657ef7 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.019s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:29:49 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[a2be2376-2e20-4273-bcf8-5dcb90ba44f2]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:29:49 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-cbb5b5ed-441c-455b-9e87-49b73c2c4cb0 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:29:49 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:29:49 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.021s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:29:49 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[6c0198f8-4a01-4306-8e85-27598b75c967]: (4, ('tcp: [42] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0 (non-flash)\ntcp: [50] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:43e92d4a-7e98-4777-8fa5-3cec396b86ce (non-flash)\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:29:49 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-cbb5b5ed-441c-455b-9e87-49b73c2c4cb0 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [42] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0 (non-flash)
Nov 29 03:29:49 user nova-compute[75234]: tcp: [50] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:43e92d4a-7e98-4777-8fa5-3cec396b86ce (non-flash)
Nov 29 03:29:49 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:29:49 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-cbb5b5ed-441c-455b-9e87-49b73c2c4cb0 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] iscsi session list stdout=tcp: [42] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0 (non-flash)
Nov 29 03:29:49 user nova-compute[75234]: tcp: [50] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:43e92d4a-7e98-4777-8fa5-3cec396b86ce (non-flash)
Nov 29 03:29:49 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsi_session /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Nov 29 03:29:49 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:29:49 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-a124e922-fc6a-431a-ba05-92a9e593c1a7 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/d54adabc8d95c75567f7840e57b0a84314361a7e --force-share --output=json" returned: 0 in 0.182s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:29:49 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-a124e922-fc6a-431a-ba05-92a9e593c1a7 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/d54adabc8d95c75567f7840e57b0a84314361a7e,backing_fmt=raw /opt/stack/data/nova/instances/cb0f042e-43c0-43aa-b243-cfc78a7b2695/disk 1073741824 {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:29:49 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:d1f13879-6e1b-4fe3-b97d-066a9e657ef7 -p 172.16.0.220:3260 --login {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:29:49 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-a124e922-fc6a-431a-ba05-92a9e593c1a7 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/d54adabc8d95c75567f7840e57b0a84314361a7e,backing_fmt=raw /opt/stack/data/nova/instances/cb0f042e-43c0-43aa-b243-cfc78a7b2695/disk 1073741824" returned: 0 in 0.074s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:29:49 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-a124e922-fc6a-431a-ba05-92a9e593c1a7 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Lock "d54adabc8d95c75567f7840e57b0a84314361a7e" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.260s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:29:49 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-a124e922-fc6a-431a-ba05-92a9e593c1a7 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/d54adabc8d95c75567f7840e57b0a84314361a7e --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:29:49 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:d1f13879-6e1b-4fe3-b97d-066a9e657ef7 -p 172.16.0.220:3260 --login" returned: 0 in 0.110s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:29:49 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[595c05ca-c717-4722-be19-a678727e0053]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:d1f13879-6e1b-4fe3-b97d-066a9e657ef7, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:d1f13879-6e1b-4fe3-b97d-066a9e657ef7, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:29:49 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-cbb5b5ed-441c-455b-9e87-49b73c2c4cb0 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:d1f13879-6e1b-4fe3-b97d-066a9e657ef7, portal: 172.16.0.220,3260]
Nov 29 03:29:49 user nova-compute[75234]: Login to [iface: default, target: iqn.2010-10.org.openstack:d1f13879-6e1b-4fe3-b97d-066a9e657ef7, portal: 172.16.0.220,3260] successful.
Nov 29 03:29:49 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:29:49 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:d1f13879-6e1b-4fe3-b97d-066a9e657ef7 -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:29:49 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:d1f13879-6e1b-4fe3-b97d-066a9e657ef7 -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.019s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:29:49 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[b5739c1d-ea6a-4407-af07-7aaaffdaccd3]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:29:49 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-cbb5b5ed-441c-455b-9e87-49b73c2c4cb0 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:29:49 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:29:49 user nova-compute[75234]: DEBUG nova.network.neutron [None req-a124e922-fc6a-431a-ba05-92a9e593c1a7 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] [instance: cb0f042e-43c0-43aa-b243-cfc78a7b2695] Updating instance_info_cache with network_info: [{"id": "8f9642c3-9e10-4aa7-b5a2-53bb68ad96dc", "address": "fa:16:3e:31:19:9f", "network": {"id": "6787e0a8-2980-4a2e-bf2a-583af955241d", "bridge": "br-int", "label": "tempest-TestMinimumBasicScenario-1273600998-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "74f132d6666c47b8b43869df6fee6de3", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap8f9642c3-9e", "ovs_interfaceid": "8f9642c3-9e10-4aa7-b5a2-53bb68ad96dc", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:29:49 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.023s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:29:49 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[a516fe76-e784-4f47-a295-da914265f672]: (4, ('tcp: [42] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0 (non-flash)\ntcp: [50] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:43e92d4a-7e98-4777-8fa5-3cec396b86ce (non-flash)\ntcp: [51] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d1f13879-6e1b-4fe3-b97d-066a9e657ef7 (non-flash)\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:29:49 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-cbb5b5ed-441c-455b-9e87-49b73c2c4cb0 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [42] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0 (non-flash)
Nov 29 03:29:49 user nova-compute[75234]: tcp: [50] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:43e92d4a-7e98-4777-8fa5-3cec396b86ce (non-flash)
Nov 29 03:29:49 user nova-compute[75234]: tcp: [51] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d1f13879-6e1b-4fe3-b97d-066a9e657ef7 (non-flash)
Nov 29 03:29:49 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:29:49 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-cbb5b5ed-441c-455b-9e87-49b73c2c4cb0 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] iscsi session list stdout=tcp: [42] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0 (non-flash)
Nov 29 03:29:49 user nova-compute[75234]: tcp: [50] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:43e92d4a-7e98-4777-8fa5-3cec396b86ce (non-flash)
Nov 29 03:29:49 user nova-compute[75234]: tcp: [51] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d1f13879-6e1b-4fe3-b97d-066a9e657ef7 (non-flash)
Nov 29 03:29:49 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsi_session /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Nov 29 03:29:49 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-cbb5b5ed-441c-455b-9e87-49b73c2c4cb0 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:d1f13879-6e1b-4fe3-b97d-066a9e657ef7" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.388s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:87}}
Nov 29 03:29:49 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-cbb5b5ed-441c-455b-9e87-49b73c2c4cb0 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Connected to 172.16.0.220:3260 {{(pid=75234) _connect_vol /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:644}}
Nov 29 03:29:49 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-cbb5b5ed-441c-455b-9e87-49b73c2c4cb0 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] HCTL ('35', '-', '-', 0) found on session 51 with lun 0 {{(pid=75234) get_hctl /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:770}}
Nov 29 03:29:49 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-cbb5b5ed-441c-455b-9e87-49b73c2c4cb0 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Scanning host 35 c: -, t: -, l: 0) {{(pid=75234) scan_iscsi /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:795}}
Nov 29 03:29:49 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-a124e922-fc6a-431a-ba05-92a9e593c1a7 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Releasing lock "refresh_cache-cb0f042e-43c0-43aa-b243-cfc78a7b2695" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:29:49 user nova-compute[75234]: DEBUG nova.compute.manager [None req-a124e922-fc6a-431a-ba05-92a9e593c1a7 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] [instance: cb0f042e-43c0-43aa-b243-cfc78a7b2695] Instance network_info: |[{"id": "8f9642c3-9e10-4aa7-b5a2-53bb68ad96dc", "address": "fa:16:3e:31:19:9f", "network": {"id": "6787e0a8-2980-4a2e-bf2a-583af955241d", "bridge": "br-int", "label": "tempest-TestMinimumBasicScenario-1273600998-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "74f132d6666c47b8b43869df6fee6de3", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap8f9642c3-9e", "ovs_interfaceid": "8f9642c3-9e10-4aa7-b5a2-53bb68ad96dc", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=75234) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1987}}
Nov 29 03:29:49 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host35/scan {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:29:49 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-1a263c33-061a-4149-bf2a-a8070f5bf538 req-97feda1a-8c3f-4eb3-81e2-a9998d4ec0b4 service nova] Acquired lock "refresh_cache-cb0f042e-43c0-43aa-b243-cfc78a7b2695" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:29:49 user nova-compute[75234]: DEBUG nova.network.neutron [req-1a263c33-061a-4149-bf2a-a8070f5bf538 req-97feda1a-8c3f-4eb3-81e2-a9998d4ec0b4 service nova] [instance: cb0f042e-43c0-43aa-b243-cfc78a7b2695] Refreshing network info cache for port 8f9642c3-9e10-4aa7-b5a2-53bb68ad96dc {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2006}}
Nov 29 03:29:49 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host35/scan" returned: 0 in 0.027s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:29:49 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[6d99ba87-0190-4d48-95da-36d811c32fcf]: (4, ('- - 0', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:29:49 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-cbb5b5ed-441c-455b-9e87-49b73c2c4cb0 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Searching for a device in session 51 and hctl ['35', '*', '*', 0] yield: sdd {{(pid=75234) device_name_by_hctl /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:789}}
Nov 29 03:29:49 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-cbb5b5ed-441c-455b-9e87-49b73c2c4cb0 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Connected to sdd using {'target_iqn': 'iqn.2010-10.org.openstack:d1f13879-6e1b-4fe3-b97d-066a9e657ef7', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'VpKRqEQa', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False} {{(pid=75234) _connect_vol /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:672}}
Nov 29 03:29:50 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-a124e922-fc6a-431a-ba05-92a9e593c1a7 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/d54adabc8d95c75567f7840e57b0a84314361a7e --force-share --output=json" returned: 0 in 0.194s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:29:50 user nova-compute[75234]: DEBUG nova.virt.disk.api [None req-a124e922-fc6a-431a-ba05-92a9e593c1a7 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Checking if we can resize image /opt/stack/data/nova/instances/cb0f042e-43c0-43aa-b243-cfc78a7b2695/disk. size=1073741824 {{(pid=75234) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:166}}
Nov 29 03:29:50 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-a124e922-fc6a-431a-ba05-92a9e593c1a7 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/cb0f042e-43c0-43aa-b243-cfc78a7b2695/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:29:50 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-a124e922-fc6a-431a-ba05-92a9e593c1a7 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/cb0f042e-43c0-43aa-b243-cfc78a7b2695/disk --force-share --output=json" returned: 0 in 0.229s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:29:50 user nova-compute[75234]: DEBUG nova.virt.disk.api [None req-a124e922-fc6a-431a-ba05-92a9e593c1a7 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Cannot resize image /opt/stack/data/nova/instances/cb0f042e-43c0-43aa-b243-cfc78a7b2695/disk to a smaller size. {{(pid=75234) can_resize_image /opt/stack/nova/nova/virt/disk/api.py:172}}
Nov 29 03:29:50 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-a124e922-fc6a-431a-ba05-92a9e593c1a7 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] [instance: cb0f042e-43c0-43aa-b243-cfc78a7b2695] Created local disks {{(pid=75234) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4861}}
Nov 29 03:29:50 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-a124e922-fc6a-431a-ba05-92a9e593c1a7 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] [instance: cb0f042e-43c0-43aa-b243-cfc78a7b2695] Ensure instance console log exists: /opt/stack/data/nova/instances/cb0f042e-43c0-43aa-b243-cfc78a7b2695/console.log {{(pid=75234) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4613}}
Nov 29 03:29:50 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-a124e922-fc6a-431a-ba05-92a9e593c1a7 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:29:50 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-a124e922-fc6a-431a-ba05-92a9e593c1a7 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:29:50 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-a124e922-fc6a-431a-ba05-92a9e593c1a7 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:29:50 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-a124e922-fc6a-431a-ba05-92a9e593c1a7 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] [instance: cb0f042e-43c0-43aa-b243-cfc78a7b2695] Start _get_guest_xml network_info=[{"id": "8f9642c3-9e10-4aa7-b5a2-53bb68ad96dc", "address": "fa:16:3e:31:19:9f", "network": {"id": "6787e0a8-2980-4a2e-bf2a-583af955241d", "bridge": "br-int", "label": "tempest-TestMinimumBasicScenario-1273600998-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "74f132d6666c47b8b43869df6fee6de3", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap8f9642c3-9e", "ovs_interfaceid": "8f9642c3-9e10-4aa7-b5a2-53bb68ad96dc", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum='c8fc807773e5354afe61636071771906',container_format='bare',created_at=2023-11-29T03:29:44Z,direct_url=<?>,disk_format='qcow2',id=39c09b70-6b0e-4a2d-b9f7-1f76a50a094d,min_disk=0,min_ram=0,name='tempest-scenario-img--735160832',owner='74f132d6666c47b8b43869df6fee6de3',properties=ImageMetaProps,protected=<?>,size=21430272,status='active',tags=<?>,updated_at=2023-11-29T03:29:46Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [{'device_name': '/dev/vda', 'encryption_format': None, 'encrypted': False, 'device_type': 'disk', 'size': 0, 'disk_bus': 'virtio', 'boot_index': 0, 'encryption_options': None, 'encryption_secret_uuid': None, 'guest_format': None, 'image_id': '39c09b70-6b0e-4a2d-b9f7-1f76a50a094d'}], 'ephemerals': [], 'block_device_mapping': [], 'swap': None} {{(pid=75234) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7562}}
Nov 29 03:29:50 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-a124e922-fc6a-431a-ba05-92a9e593c1a7 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:29:50 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-a124e922-fc6a-431a-ba05-92a9e593c1a7 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:29:50 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-a124e922-fc6a-431a-ba05-92a9e593c1a7 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Searching host: 'user' for CPU controller through CGroups V1... {{(pid=75234) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1652}}
Nov 29 03:29:50 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-a124e922-fc6a-431a-ba05-92a9e593c1a7 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] CPU controller missing on host. {{(pid=75234) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1662}}
Nov 29 03:29:50 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-a124e922-fc6a-431a-ba05-92a9e593c1a7 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Searching host: 'user' for CPU controller through CGroups V2... {{(pid=75234) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1671}}
Nov 29 03:29:50 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-a124e922-fc6a-431a-ba05-92a9e593c1a7 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] CPU controller found on host. {{(pid=75234) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1678}}
Nov 29 03:29:50 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-a124e922-fc6a-431a-ba05-92a9e593c1a7 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=75234) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5400}}
Nov 29 03:29:50 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-a124e922-fc6a-431a-ba05-92a9e593c1a7 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-11-29T03:02:04Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='1',id=6,is_public=True,memory_mb=512,name='m1.tiny',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='c8fc807773e5354afe61636071771906',container_format='bare',created_at=2023-11-29T03:29:44Z,direct_url=<?>,disk_format='qcow2',id=39c09b70-6b0e-4a2d-b9f7-1f76a50a094d,min_disk=0,min_ram=0,name='tempest-scenario-img--735160832',owner='74f132d6666c47b8b43869df6fee6de3',properties=ImageMetaProps,protected=<?>,size=21430272,status='active',tags=<?>,updated_at=2023-11-29T03:29:46Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:563}}
Nov 29 03:29:50 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-a124e922-fc6a-431a-ba05-92a9e593c1a7 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Flavor limits 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:348}}
Nov 29 03:29:50 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-a124e922-fc6a-431a-ba05-92a9e593c1a7 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Image limits 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:352}}
Nov 29 03:29:50 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-a124e922-fc6a-431a-ba05-92a9e593c1a7 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Flavor pref 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:388}}
Nov 29 03:29:50 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-a124e922-fc6a-431a-ba05-92a9e593c1a7 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Image pref 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:392}}
Nov 29 03:29:50 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-a124e922-fc6a-431a-ba05-92a9e593c1a7 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:430}}
Nov 29 03:29:50 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-a124e922-fc6a-431a-ba05-92a9e593c1a7 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:569}}
Nov 29 03:29:50 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-a124e922-fc6a-431a-ba05-92a9e593c1a7 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=75234) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:471}}
Nov 29 03:29:50 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-a124e922-fc6a-431a-ba05-92a9e593c1a7 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Got 1 possible topologies {{(pid=75234) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:501}}
Nov 29 03:29:50 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-a124e922-fc6a-431a-ba05-92a9e593c1a7 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:575}}
Nov 29 03:29:50 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-a124e922-fc6a-431a-ba05-92a9e593c1a7 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:577}}
Nov 29 03:29:50 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-a124e922-fc6a-431a-ba05-92a9e593c1a7 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-11-29T03:29:46Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-TestMinimumBasicScenario-server-1999691651',display_name='tempest-TestMinimumBasicScenario-server-1999691651',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-testminimumbasicscenario-server-1999691651',id=46,image_ref='39c09b70-6b0e-4a2d-b9f7-1f76a50a094d',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBLqZTjJ3Wp5EB1mGeMlVGHzydzMhKmKzqzC+PvIWz1jwo/eWvz8BbybL4HoY/jqWWGyeee/UtM8csTyDO1CCy9W96A60d4tN6E3Pg7qFv3rg6kLzbOVwZtACkYPfit/ztw==',key_name='tempest-TestMinimumBasicScenario-967805131',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='74f132d6666c47b8b43869df6fee6de3',ramdisk_id='',reservation_id='r-cfgmj1kq',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='39c09b70-6b0e-4a2d-b9f7-1f76a50a094d',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_min_disk='1',image_min_ram='0',network_allocated='True',owner_project_name='tempest-TestMinimumBasicScenario-1635847764',owner_user_name='tempest-TestMinimumBasicScenario-1635847764-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-11-29T03:29:48Z,user_data=None,user_id='3a02618f56ea45898ab4783a4f198b5d',uuid=cb0f042e-43c0-43aa-b243-cfc78a7b2695,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "8f9642c3-9e10-4aa7-b5a2-53bb68ad96dc", "address": "fa:16:3e:31:19:9f", "network": {"id": "6787e0a8-2980-4a2e-bf2a-583af955241d", "bridge": "br-int", "label": "tempest-TestMinimumBasicScenario-1273600998-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "74f132d6666c47b8b43869df6fee6de3", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap8f9642c3-9e", "ovs_interfaceid": "8f9642c3-9e10-4aa7-b5a2-53bb68ad96dc", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=75234) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Nov 29 03:29:50 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-a124e922-fc6a-431a-ba05-92a9e593c1a7 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Converting VIF {"id": "8f9642c3-9e10-4aa7-b5a2-53bb68ad96dc", "address": "fa:16:3e:31:19:9f", "network": {"id": "6787e0a8-2980-4a2e-bf2a-583af955241d", "bridge": "br-int", "label": "tempest-TestMinimumBasicScenario-1273600998-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "74f132d6666c47b8b43869df6fee6de3", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap8f9642c3-9e", "ovs_interfaceid": "8f9642c3-9e10-4aa7-b5a2-53bb68ad96dc", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:29:50 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-a124e922-fc6a-431a-ba05-92a9e593c1a7 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:31:19:9f,bridge_name='br-int',has_traffic_filtering=True,id=8f9642c3-9e10-4aa7-b5a2-53bb68ad96dc,network=Network(6787e0a8-2980-4a2e-bf2a-583af955241d),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap8f9642c3-9e') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:29:50 user nova-compute[75234]: DEBUG nova.objects.instance [None req-a124e922-fc6a-431a-ba05-92a9e593c1a7 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Lazy-loading 'pci_devices' on Instance uuid cb0f042e-43c0-43aa-b243-cfc78a7b2695 {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:29:50 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-a124e922-fc6a-431a-ba05-92a9e593c1a7 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] [instance: cb0f042e-43c0-43aa-b243-cfc78a7b2695] End _get_guest_xml xml=<domain type="kvm">
Nov 29 03:29:50 user nova-compute[75234]:   <uuid>cb0f042e-43c0-43aa-b243-cfc78a7b2695</uuid>
Nov 29 03:29:50 user nova-compute[75234]:   <name>instance-0000002e</name>
Nov 29 03:29:50 user nova-compute[75234]:   <memory>524288</memory>
Nov 29 03:29:50 user nova-compute[75234]:   <vcpu>1</vcpu>
Nov 29 03:29:50 user nova-compute[75234]:   <metadata>
Nov 29 03:29:50 user nova-compute[75234]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Nov 29 03:29:50 user nova-compute[75234]:       <nova:package version="0.0.0"/>
Nov 29 03:29:50 user nova-compute[75234]:       <nova:name>tempest-TestMinimumBasicScenario-server-1999691651</nova:name>
Nov 29 03:29:50 user nova-compute[75234]:       <nova:creationTime>2023-11-29 03:29:50</nova:creationTime>
Nov 29 03:29:50 user nova-compute[75234]:       <nova:flavor name="m1.tiny">
Nov 29 03:29:50 user nova-compute[75234]:         <nova:memory>512</nova:memory>
Nov 29 03:29:50 user nova-compute[75234]:         <nova:disk>1</nova:disk>
Nov 29 03:29:50 user nova-compute[75234]:         <nova:swap>0</nova:swap>
Nov 29 03:29:50 user nova-compute[75234]:         <nova:ephemeral>0</nova:ephemeral>
Nov 29 03:29:50 user nova-compute[75234]:         <nova:vcpus>1</nova:vcpus>
Nov 29 03:29:50 user nova-compute[75234]:       </nova:flavor>
Nov 29 03:29:50 user nova-compute[75234]:       <nova:owner>
Nov 29 03:29:50 user nova-compute[75234]:         <nova:user uuid="3a02618f56ea45898ab4783a4f198b5d">tempest-TestMinimumBasicScenario-1635847764-project-member</nova:user>
Nov 29 03:29:50 user nova-compute[75234]:         <nova:project uuid="74f132d6666c47b8b43869df6fee6de3">tempest-TestMinimumBasicScenario-1635847764</nova:project>
Nov 29 03:29:50 user nova-compute[75234]:       </nova:owner>
Nov 29 03:29:50 user nova-compute[75234]:       <nova:root type="image" uuid="39c09b70-6b0e-4a2d-b9f7-1f76a50a094d"/>
Nov 29 03:29:50 user nova-compute[75234]:       <nova:ports>
Nov 29 03:29:50 user nova-compute[75234]:         <nova:port uuid="8f9642c3-9e10-4aa7-b5a2-53bb68ad96dc">
Nov 29 03:29:50 user nova-compute[75234]:           <nova:ip type="fixed" address="10.0.0.11" ipVersion="4"/>
Nov 29 03:29:50 user nova-compute[75234]:         </nova:port>
Nov 29 03:29:50 user nova-compute[75234]:       </nova:ports>
Nov 29 03:29:50 user nova-compute[75234]:     </nova:instance>
Nov 29 03:29:50 user nova-compute[75234]:   </metadata>
Nov 29 03:29:50 user nova-compute[75234]:   <sysinfo type="smbios">
Nov 29 03:29:50 user nova-compute[75234]:     <system>
Nov 29 03:29:50 user nova-compute[75234]:       <entry name="manufacturer">OpenStack Foundation</entry>
Nov 29 03:29:50 user nova-compute[75234]:       <entry name="product">OpenStack Nova</entry>
Nov 29 03:29:50 user nova-compute[75234]:       <entry name="version">0.0.0</entry>
Nov 29 03:29:50 user nova-compute[75234]:       <entry name="serial">cb0f042e-43c0-43aa-b243-cfc78a7b2695</entry>
Nov 29 03:29:50 user nova-compute[75234]:       <entry name="uuid">cb0f042e-43c0-43aa-b243-cfc78a7b2695</entry>
Nov 29 03:29:50 user nova-compute[75234]:       <entry name="family">Virtual Machine</entry>
Nov 29 03:29:50 user nova-compute[75234]:     </system>
Nov 29 03:29:50 user nova-compute[75234]:   </sysinfo>
Nov 29 03:29:50 user nova-compute[75234]:   <os>
Nov 29 03:29:50 user nova-compute[75234]:     <type arch="x86_64" machine="pc">hvm</type>
Nov 29 03:29:50 user nova-compute[75234]:     <boot dev="hd"/>
Nov 29 03:29:50 user nova-compute[75234]:     <smbios mode="sysinfo"/>
Nov 29 03:29:50 user nova-compute[75234]:   </os>
Nov 29 03:29:50 user nova-compute[75234]:   <features>
Nov 29 03:29:50 user nova-compute[75234]:     <acpi/>
Nov 29 03:29:50 user nova-compute[75234]:     <apic/>
Nov 29 03:29:50 user nova-compute[75234]:     <vmcoreinfo/>
Nov 29 03:29:50 user nova-compute[75234]:   </features>
Nov 29 03:29:50 user nova-compute[75234]:   <clock offset="utc">
Nov 29 03:29:50 user nova-compute[75234]:     <timer name="pit" tickpolicy="delay"/>
Nov 29 03:29:50 user nova-compute[75234]:     <timer name="rtc" tickpolicy="catchup"/>
Nov 29 03:29:50 user nova-compute[75234]:     <timer name="hpet" present="no"/>
Nov 29 03:29:50 user nova-compute[75234]:   </clock>
Nov 29 03:29:50 user nova-compute[75234]:   <cpu mode="custom" match="exact">
Nov 29 03:29:50 user nova-compute[75234]:     <model>Nehalem</model>
Nov 29 03:29:50 user nova-compute[75234]:     <topology sockets="1" cores="1" threads="1"/>
Nov 29 03:29:50 user nova-compute[75234]:   </cpu>
Nov 29 03:29:50 user nova-compute[75234]:   <devices>
Nov 29 03:29:50 user nova-compute[75234]:     <disk type="file" device="disk">
Nov 29 03:29:50 user nova-compute[75234]:       <driver name="qemu" type="qcow2" cache="none"/>
Nov 29 03:29:50 user nova-compute[75234]:       <source file="/opt/stack/data/nova/instances/cb0f042e-43c0-43aa-b243-cfc78a7b2695/disk"/>
Nov 29 03:29:50 user nova-compute[75234]:       <target dev="vda" bus="virtio"/>
Nov 29 03:29:50 user nova-compute[75234]:     </disk>
Nov 29 03:29:50 user nova-compute[75234]:     <interface type="ethernet">
Nov 29 03:29:50 user nova-compute[75234]:       <mac address="fa:16:3e:31:19:9f"/>
Nov 29 03:29:50 user nova-compute[75234]:       <model type="virtio"/>
Nov 29 03:29:50 user nova-compute[75234]:       <mtu size="1442"/>
Nov 29 03:29:50 user nova-compute[75234]:       <target dev="tap8f9642c3-9e"/>
Nov 29 03:29:50 user nova-compute[75234]:     </interface>
Nov 29 03:29:50 user nova-compute[75234]:     <serial type="pty">
Nov 29 03:29:50 user nova-compute[75234]:       <log file="/opt/stack/data/nova/instances/cb0f042e-43c0-43aa-b243-cfc78a7b2695/console.log" append="off"/>
Nov 29 03:29:50 user nova-compute[75234]:     </serial>
Nov 29 03:29:50 user nova-compute[75234]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Nov 29 03:29:50 user nova-compute[75234]:     <video>
Nov 29 03:29:50 user nova-compute[75234]:       <model type="virtio"/>
Nov 29 03:29:50 user nova-compute[75234]:     </video>
Nov 29 03:29:50 user nova-compute[75234]:     <rng model="virtio">
Nov 29 03:29:50 user nova-compute[75234]:       <backend model="random">/dev/urandom</backend>
Nov 29 03:29:50 user nova-compute[75234]:     </rng>
Nov 29 03:29:50 user nova-compute[75234]:     <controller type="usb" index="0" model="none"/>
Nov 29 03:29:50 user nova-compute[75234]:     <memballoon model="virtio">
Nov 29 03:29:50 user nova-compute[75234]:       <stats period="10"/>
Nov 29 03:29:50 user nova-compute[75234]:     </memballoon>
Nov 29 03:29:50 user nova-compute[75234]:   </devices>
Nov 29 03:29:50 user nova-compute[75234]: </domain>
Nov 29 03:29:50 user nova-compute[75234]:  {{(pid=75234) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7568}}
Nov 29 03:29:50 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-a124e922-fc6a-431a-ba05-92a9e593c1a7 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-11-29T03:29:46Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-TestMinimumBasicScenario-server-1999691651',display_name='tempest-TestMinimumBasicScenario-server-1999691651',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-testminimumbasicscenario-server-1999691651',id=46,image_ref='39c09b70-6b0e-4a2d-b9f7-1f76a50a094d',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBLqZTjJ3Wp5EB1mGeMlVGHzydzMhKmKzqzC+PvIWz1jwo/eWvz8BbybL4HoY/jqWWGyeee/UtM8csTyDO1CCy9W96A60d4tN6E3Pg7qFv3rg6kLzbOVwZtACkYPfit/ztw==',key_name='tempest-TestMinimumBasicScenario-967805131',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='74f132d6666c47b8b43869df6fee6de3',ramdisk_id='',reservation_id='r-cfgmj1kq',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='39c09b70-6b0e-4a2d-b9f7-1f76a50a094d',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_min_disk='1',image_min_ram='0',network_allocated='True',owner_project_name='tempest-TestMinimumBasicScenario-1635847764',owner_user_name='tempest-TestMinimumBasicScenario-1635847764-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-11-29T03:29:48Z,user_data=None,user_id='3a02618f56ea45898ab4783a4f198b5d',uuid=cb0f042e-43c0-43aa-b243-cfc78a7b2695,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "8f9642c3-9e10-4aa7-b5a2-53bb68ad96dc", "address": "fa:16:3e:31:19:9f", "network": {"id": "6787e0a8-2980-4a2e-bf2a-583af955241d", "bridge": "br-int", "label": "tempest-TestMinimumBasicScenario-1273600998-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "74f132d6666c47b8b43869df6fee6de3", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap8f9642c3-9e", "ovs_interfaceid": "8f9642c3-9e10-4aa7-b5a2-53bb68ad96dc", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Nov 29 03:29:50 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-a124e922-fc6a-431a-ba05-92a9e593c1a7 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Converting VIF {"id": "8f9642c3-9e10-4aa7-b5a2-53bb68ad96dc", "address": "fa:16:3e:31:19:9f", "network": {"id": "6787e0a8-2980-4a2e-bf2a-583af955241d", "bridge": "br-int", "label": "tempest-TestMinimumBasicScenario-1273600998-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "74f132d6666c47b8b43869df6fee6de3", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap8f9642c3-9e", "ovs_interfaceid": "8f9642c3-9e10-4aa7-b5a2-53bb68ad96dc", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:29:50 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-a124e922-fc6a-431a-ba05-92a9e593c1a7 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:31:19:9f,bridge_name='br-int',has_traffic_filtering=True,id=8f9642c3-9e10-4aa7-b5a2-53bb68ad96dc,network=Network(6787e0a8-2980-4a2e-bf2a-583af955241d),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap8f9642c3-9e') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:29:50 user nova-compute[75234]: DEBUG os_vif [None req-a124e922-fc6a-431a-ba05-92a9e593c1a7 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:31:19:9f,bridge_name='br-int',has_traffic_filtering=True,id=8f9642c3-9e10-4aa7-b5a2-53bb68ad96dc,network=Network(6787e0a8-2980-4a2e-bf2a-583af955241d),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap8f9642c3-9e') {{(pid=75234) plug /opt/stack/data/venv/lib/python3.10/site-packages/os_vif/__init__.py:76}}
Nov 29 03:29:50 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:29:50 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:29:50 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Nov 29 03:29:50 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:29:50 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbCreateCommand(_result=None, table=QoS, columns={'type': 'linux-noop', 'external_ids': {'id': '4db40271-bb28-5e0f-9999-a744bde9c548', '_type': 'linux-noop'}}, row=False) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:29:50 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:29:50 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:29:50 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:29:50 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tap8f9642c3-9e, may_exist=True, interface_attrs={}) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:29:50 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Port, record=tap8f9642c3-9e, col_values=(('qos', UUID('72db526e-a12f-4edc-857c-2aee24308393')),), if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:29:50 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DbSetCommand(_result=None, table=Interface, record=tap8f9642c3-9e, col_values=(('external_ids', {'iface-id': '8f9642c3-9e10-4aa7-b5a2-53bb68ad96dc', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:31:19:9f', 'vm-uuid': 'cb0f042e-43c0-43aa-b243-cfc78a7b2695'}),), if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:29:50 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:29:50 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:29:50 user nova-compute[75234]: INFO os_vif [None req-a124e922-fc6a-431a-ba05-92a9e593c1a7 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:31:19:9f,bridge_name='br-int',has_traffic_filtering=True,id=8f9642c3-9e10-4aa7-b5a2-53bb68ad96dc,network=Network(6787e0a8-2980-4a2e-bf2a-583af955241d),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap8f9642c3-9e')
Nov 29 03:29:50 user nova-compute[75234]: DEBUG nova.network.neutron [req-1a263c33-061a-4149-bf2a-a8070f5bf538 req-97feda1a-8c3f-4eb3-81e2-a9998d4ec0b4 service nova] [instance: cb0f042e-43c0-43aa-b243-cfc78a7b2695] Updated VIF entry in instance network info cache for port 8f9642c3-9e10-4aa7-b5a2-53bb68ad96dc. {{(pid=75234) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3481}}
Nov 29 03:29:50 user nova-compute[75234]: DEBUG nova.network.neutron [req-1a263c33-061a-4149-bf2a-a8070f5bf538 req-97feda1a-8c3f-4eb3-81e2-a9998d4ec0b4 service nova] [instance: cb0f042e-43c0-43aa-b243-cfc78a7b2695] Updating instance_info_cache with network_info: [{"id": "8f9642c3-9e10-4aa7-b5a2-53bb68ad96dc", "address": "fa:16:3e:31:19:9f", "network": {"id": "6787e0a8-2980-4a2e-bf2a-583af955241d", "bridge": "br-int", "label": "tempest-TestMinimumBasicScenario-1273600998-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "74f132d6666c47b8b43869df6fee6de3", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap8f9642c3-9e", "ovs_interfaceid": "8f9642c3-9e10-4aa7-b5a2-53bb68ad96dc", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:29:50 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-1a263c33-061a-4149-bf2a-a8070f5bf538 req-97feda1a-8c3f-4eb3-81e2-a9998d4ec0b4 service nova] Releasing lock "refresh_cache-cb0f042e-43c0-43aa-b243-cfc78a7b2695" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:29:50 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-a124e922-fc6a-431a-ba05-92a9e593c1a7 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] No BDM found with device name vda, not building metadata. {{(pid=75234) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12089}}
Nov 29 03:29:50 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-a124e922-fc6a-431a-ba05-92a9e593c1a7 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] No VIF found with MAC fa:16:3e:31:19:9f, not building metadata {{(pid=75234) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Nov 29 03:29:50 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-cbb5b5ed-441c-455b-9e87-49b73c2c4cb0 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.436s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:87}}
Nov 29 03:29:50 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-cbb5b5ed-441c-455b-9e87-49b73c2c4cb0 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] <== connect_volume: return (1437ms) {'type': 'block', 'scsi_wwn': '26431316137313461', 'path': '/dev/sdd'} {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:203}}
Nov 29 03:29:50 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-cbb5b5ed-441c-455b-9e87-49b73c2c4cb0 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: 1e9263fc-5735-420f-a116-8f60c8736faf] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '26431316137313461', 'path': '/dev/sdd'} {{(pid=75234) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Nov 29 03:29:51 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-cbb5b5ed-441c-455b-9e87-49b73c2c4cb0 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-11-29T03:29:34Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-TestVolumeBootPattern-server-180638926',display_name='tempest-TestVolumeBootPattern-server-180638926',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-testvolumebootpattern-server-180638926',id=45,image_ref='',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data=None,key_name=None,keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='81a265d3c15b4e9d826b35ddbd0ff443',ramdisk_id='',reservation_id='r-or8ryn80',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TestVolumeBootPattern-1244492388',owner_user_name='tempest-TestVolumeBootPattern-1244492388-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-11-29T03:29:35Z,user_data=None,user_id='7d1f1f11a9aa4c1db8627413eb963fa4',uuid=1e9263fc-5735-420f-a116-8f60c8736faf,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "f836309d-8f97-4521-aa57-d6516a9e7528", "address": "fa:16:3e:1e:0c:76", "network": {"id": "a3f397d3-fd75-4a6e-a815-cfcedcca15fb", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-678570085-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "81a265d3c15b4e9d826b35ddbd0ff443", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapf836309d-8f", "ovs_interfaceid": "f836309d-8f97-4521-aa57-d6516a9e7528", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=75234) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Nov 29 03:29:51 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-cbb5b5ed-441c-455b-9e87-49b73c2c4cb0 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Converting VIF {"id": "f836309d-8f97-4521-aa57-d6516a9e7528", "address": "fa:16:3e:1e:0c:76", "network": {"id": "a3f397d3-fd75-4a6e-a815-cfcedcca15fb", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-678570085-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "81a265d3c15b4e9d826b35ddbd0ff443", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapf836309d-8f", "ovs_interfaceid": "f836309d-8f97-4521-aa57-d6516a9e7528", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:29:51 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-cbb5b5ed-441c-455b-9e87-49b73c2c4cb0 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:1e:0c:76,bridge_name='br-int',has_traffic_filtering=True,id=f836309d-8f97-4521-aa57-d6516a9e7528,network=Network(a3f397d3-fd75-4a6e-a815-cfcedcca15fb),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapf836309d-8f') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:29:51 user nova-compute[75234]: DEBUG nova.objects.instance [None req-cbb5b5ed-441c-455b-9e87-49b73c2c4cb0 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Lazy-loading 'pci_devices' on Instance uuid 1e9263fc-5735-420f-a116-8f60c8736faf {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:29:51 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-cbb5b5ed-441c-455b-9e87-49b73c2c4cb0 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: 1e9263fc-5735-420f-a116-8f60c8736faf] End _get_guest_xml xml=<domain type="kvm">
Nov 29 03:29:51 user nova-compute[75234]:   <uuid>1e9263fc-5735-420f-a116-8f60c8736faf</uuid>
Nov 29 03:29:51 user nova-compute[75234]:   <name>instance-0000002d</name>
Nov 29 03:29:51 user nova-compute[75234]:   <memory>524288</memory>
Nov 29 03:29:51 user nova-compute[75234]:   <vcpu>1</vcpu>
Nov 29 03:29:51 user nova-compute[75234]:   <metadata>
Nov 29 03:29:51 user nova-compute[75234]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Nov 29 03:29:51 user nova-compute[75234]:       <nova:package version="0.0.0"/>
Nov 29 03:29:51 user nova-compute[75234]:       <nova:name>tempest-TestVolumeBootPattern-server-180638926</nova:name>
Nov 29 03:29:51 user nova-compute[75234]:       <nova:creationTime>2023-11-29 03:29:49</nova:creationTime>
Nov 29 03:29:51 user nova-compute[75234]:       <nova:flavor name="m1.tiny">
Nov 29 03:29:51 user nova-compute[75234]:         <nova:memory>512</nova:memory>
Nov 29 03:29:51 user nova-compute[75234]:         <nova:disk>1</nova:disk>
Nov 29 03:29:51 user nova-compute[75234]:         <nova:swap>0</nova:swap>
Nov 29 03:29:51 user nova-compute[75234]:         <nova:ephemeral>0</nova:ephemeral>
Nov 29 03:29:51 user nova-compute[75234]:         <nova:vcpus>1</nova:vcpus>
Nov 29 03:29:51 user nova-compute[75234]:       </nova:flavor>
Nov 29 03:29:51 user nova-compute[75234]:       <nova:owner>
Nov 29 03:29:51 user nova-compute[75234]:         <nova:user uuid="7d1f1f11a9aa4c1db8627413eb963fa4">tempest-TestVolumeBootPattern-1244492388-project-member</nova:user>
Nov 29 03:29:51 user nova-compute[75234]:         <nova:project uuid="81a265d3c15b4e9d826b35ddbd0ff443">tempest-TestVolumeBootPattern-1244492388</nova:project>
Nov 29 03:29:51 user nova-compute[75234]:       </nova:owner>
Nov 29 03:29:51 user nova-compute[75234]:       <nova:ports>
Nov 29 03:29:51 user nova-compute[75234]:         <nova:port uuid="f836309d-8f97-4521-aa57-d6516a9e7528">
Nov 29 03:29:51 user nova-compute[75234]:           <nova:ip type="fixed" address="10.0.0.10" ipVersion="4"/>
Nov 29 03:29:51 user nova-compute[75234]:         </nova:port>
Nov 29 03:29:51 user nova-compute[75234]:       </nova:ports>
Nov 29 03:29:51 user nova-compute[75234]:     </nova:instance>
Nov 29 03:29:51 user nova-compute[75234]:   </metadata>
Nov 29 03:29:51 user nova-compute[75234]:   <sysinfo type="smbios">
Nov 29 03:29:51 user nova-compute[75234]:     <system>
Nov 29 03:29:51 user nova-compute[75234]:       <entry name="manufacturer">OpenStack Foundation</entry>
Nov 29 03:29:51 user nova-compute[75234]:       <entry name="product">OpenStack Nova</entry>
Nov 29 03:29:51 user nova-compute[75234]:       <entry name="version">0.0.0</entry>
Nov 29 03:29:51 user nova-compute[75234]:       <entry name="serial">1e9263fc-5735-420f-a116-8f60c8736faf</entry>
Nov 29 03:29:51 user nova-compute[75234]:       <entry name="uuid">1e9263fc-5735-420f-a116-8f60c8736faf</entry>
Nov 29 03:29:51 user nova-compute[75234]:       <entry name="family">Virtual Machine</entry>
Nov 29 03:29:51 user nova-compute[75234]:     </system>
Nov 29 03:29:51 user nova-compute[75234]:   </sysinfo>
Nov 29 03:29:51 user nova-compute[75234]:   <os>
Nov 29 03:29:51 user nova-compute[75234]:     <type arch="x86_64" machine="pc">hvm</type>
Nov 29 03:29:51 user nova-compute[75234]:     <boot dev="hd"/>
Nov 29 03:29:51 user nova-compute[75234]:     <smbios mode="sysinfo"/>
Nov 29 03:29:51 user nova-compute[75234]:   </os>
Nov 29 03:29:51 user nova-compute[75234]:   <features>
Nov 29 03:29:51 user nova-compute[75234]:     <acpi/>
Nov 29 03:29:51 user nova-compute[75234]:     <apic/>
Nov 29 03:29:51 user nova-compute[75234]:     <vmcoreinfo/>
Nov 29 03:29:51 user nova-compute[75234]:   </features>
Nov 29 03:29:51 user nova-compute[75234]:   <clock offset="utc">
Nov 29 03:29:51 user nova-compute[75234]:     <timer name="pit" tickpolicy="delay"/>
Nov 29 03:29:51 user nova-compute[75234]:     <timer name="rtc" tickpolicy="catchup"/>
Nov 29 03:29:51 user nova-compute[75234]:     <timer name="hpet" present="no"/>
Nov 29 03:29:51 user nova-compute[75234]:   </clock>
Nov 29 03:29:51 user nova-compute[75234]:   <cpu mode="custom" match="exact">
Nov 29 03:29:51 user nova-compute[75234]:     <model>Nehalem</model>
Nov 29 03:29:51 user nova-compute[75234]:     <topology sockets="1" cores="1" threads="1"/>
Nov 29 03:29:51 user nova-compute[75234]:   </cpu>
Nov 29 03:29:51 user nova-compute[75234]:   <devices>
Nov 29 03:29:51 user nova-compute[75234]:     <disk type="block" device="disk">
Nov 29 03:29:51 user nova-compute[75234]:       <driver name="qemu" type="raw" cache="none" io="native"/>
Nov 29 03:29:51 user nova-compute[75234]:       <source dev="/dev/sdd"/>
Nov 29 03:29:51 user nova-compute[75234]:       <target dev="vda" bus="virtio"/>
Nov 29 03:29:51 user nova-compute[75234]:       <serial>d1f13879-6e1b-4fe3-b97d-066a9e657ef7</serial>
Nov 29 03:29:51 user nova-compute[75234]:     </disk>
Nov 29 03:29:51 user nova-compute[75234]:     <interface type="ethernet">
Nov 29 03:29:51 user nova-compute[75234]:       <mac address="fa:16:3e:1e:0c:76"/>
Nov 29 03:29:51 user nova-compute[75234]:       <model type="virtio"/>
Nov 29 03:29:51 user nova-compute[75234]:       <mtu size="1442"/>
Nov 29 03:29:51 user nova-compute[75234]:       <target dev="tapf836309d-8f"/>
Nov 29 03:29:51 user nova-compute[75234]:     </interface>
Nov 29 03:29:51 user nova-compute[75234]:     <serial type="pty">
Nov 29 03:29:51 user nova-compute[75234]:       <log file="/opt/stack/data/nova/instances/1e9263fc-5735-420f-a116-8f60c8736faf/console.log" append="off"/>
Nov 29 03:29:51 user nova-compute[75234]:     </serial>
Nov 29 03:29:51 user nova-compute[75234]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Nov 29 03:29:51 user nova-compute[75234]:     <video>
Nov 29 03:29:51 user nova-compute[75234]:       <model type="virtio"/>
Nov 29 03:29:51 user nova-compute[75234]:     </video>
Nov 29 03:29:51 user nova-compute[75234]:     <rng model="virtio">
Nov 29 03:29:51 user nova-compute[75234]:       <backend model="random">/dev/urandom</backend>
Nov 29 03:29:51 user nova-compute[75234]:     </rng>
Nov 29 03:29:51 user nova-compute[75234]:     <controller type="usb" index="0" model="none"/>
Nov 29 03:29:51 user nova-compute[75234]:     <memballoon model="virtio">
Nov 29 03:29:51 user nova-compute[75234]:       <stats period="10"/>
Nov 29 03:29:51 user nova-compute[75234]:     </memballoon>
Nov 29 03:29:51 user nova-compute[75234]:   </devices>
Nov 29 03:29:51 user nova-compute[75234]: </domain>
Nov 29 03:29:51 user nova-compute[75234]:  {{(pid=75234) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7568}}
Nov 29 03:29:51 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-cbb5b5ed-441c-455b-9e87-49b73c2c4cb0 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-11-29T03:29:34Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-TestVolumeBootPattern-server-180638926',display_name='tempest-TestVolumeBootPattern-server-180638926',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-testvolumebootpattern-server-180638926',id=45,image_ref='',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data=None,key_name=None,keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='81a265d3c15b4e9d826b35ddbd0ff443',ramdisk_id='',reservation_id='r-or8ryn80',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TestVolumeBootPattern-1244492388',owner_user_name='tempest-TestVolumeBootPattern-1244492388-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-11-29T03:29:35Z,user_data=None,user_id='7d1f1f11a9aa4c1db8627413eb963fa4',uuid=1e9263fc-5735-420f-a116-8f60c8736faf,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "f836309d-8f97-4521-aa57-d6516a9e7528", "address": "fa:16:3e:1e:0c:76", "network": {"id": "a3f397d3-fd75-4a6e-a815-cfcedcca15fb", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-678570085-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "81a265d3c15b4e9d826b35ddbd0ff443", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapf836309d-8f", "ovs_interfaceid": "f836309d-8f97-4521-aa57-d6516a9e7528", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Nov 29 03:29:51 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-cbb5b5ed-441c-455b-9e87-49b73c2c4cb0 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Converting VIF {"id": "f836309d-8f97-4521-aa57-d6516a9e7528", "address": "fa:16:3e:1e:0c:76", "network": {"id": "a3f397d3-fd75-4a6e-a815-cfcedcca15fb", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-678570085-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "81a265d3c15b4e9d826b35ddbd0ff443", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapf836309d-8f", "ovs_interfaceid": "f836309d-8f97-4521-aa57-d6516a9e7528", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:29:51 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-cbb5b5ed-441c-455b-9e87-49b73c2c4cb0 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:1e:0c:76,bridge_name='br-int',has_traffic_filtering=True,id=f836309d-8f97-4521-aa57-d6516a9e7528,network=Network(a3f397d3-fd75-4a6e-a815-cfcedcca15fb),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapf836309d-8f') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:29:51 user nova-compute[75234]: DEBUG os_vif [None req-cbb5b5ed-441c-455b-9e87-49b73c2c4cb0 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:1e:0c:76,bridge_name='br-int',has_traffic_filtering=True,id=f836309d-8f97-4521-aa57-d6516a9e7528,network=Network(a3f397d3-fd75-4a6e-a815-cfcedcca15fb),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapf836309d-8f') {{(pid=75234) plug /opt/stack/data/venv/lib/python3.10/site-packages/os_vif/__init__.py:76}}
Nov 29 03:29:51 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:29:51 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:29:51 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Nov 29 03:29:51 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:29:51 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbCreateCommand(_result=None, table=QoS, columns={'type': 'linux-noop', 'external_ids': {'id': 'ace76d78-cf7a-5f96-a4e7-11652718ac29', '_type': 'linux-noop'}}, row=False) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:29:51 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:29:51 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:29:51 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:29:51 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:29:51 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tapf836309d-8f, may_exist=True, interface_attrs={}) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:29:51 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Port, record=tapf836309d-8f, col_values=(('qos', UUID('355af910-5588-4b9b-aecf-a05b8f27ead1')),), if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:29:51 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DbSetCommand(_result=None, table=Interface, record=tapf836309d-8f, col_values=(('external_ids', {'iface-id': 'f836309d-8f97-4521-aa57-d6516a9e7528', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:1e:0c:76', 'vm-uuid': '1e9263fc-5735-420f-a116-8f60c8736faf'}),), if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:29:51 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:29:51 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:29:51 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:29:51 user nova-compute[75234]: INFO os_vif [None req-cbb5b5ed-441c-455b-9e87-49b73c2c4cb0 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:1e:0c:76,bridge_name='br-int',has_traffic_filtering=True,id=f836309d-8f97-4521-aa57-d6516a9e7528,network=Network(a3f397d3-fd75-4a6e-a815-cfcedcca15fb),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapf836309d-8f')
Nov 29 03:29:51 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-cbb5b5ed-441c-455b-9e87-49b73c2c4cb0 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] No BDM found with device name vda, not building metadata. {{(pid=75234) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12089}}
Nov 29 03:29:51 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-cbb5b5ed-441c-455b-9e87-49b73c2c4cb0 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] No VIF found with MAC fa:16:3e:1e:0c:76, not building metadata {{(pid=75234) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Nov 29 03:29:52 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:29:52 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:29:52 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:29:52 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:29:52 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:29:53 user nova-compute[75234]: DEBUG nova.compute.manager [req-8e302b52-e737-4f39-8a13-be1c892c3f44 req-8e89bd01-5499-4aeb-a08c-160817cf06d5 service nova] [instance: cb0f042e-43c0-43aa-b243-cfc78a7b2695] Received event network-vif-plugged-8f9642c3-9e10-4aa7-b5a2-53bb68ad96dc {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:29:53 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-8e302b52-e737-4f39-8a13-be1c892c3f44 req-8e89bd01-5499-4aeb-a08c-160817cf06d5 service nova] Acquiring lock "cb0f042e-43c0-43aa-b243-cfc78a7b2695-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:29:53 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-8e302b52-e737-4f39-8a13-be1c892c3f44 req-8e89bd01-5499-4aeb-a08c-160817cf06d5 service nova] Lock "cb0f042e-43c0-43aa-b243-cfc78a7b2695-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:29:53 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-8e302b52-e737-4f39-8a13-be1c892c3f44 req-8e89bd01-5499-4aeb-a08c-160817cf06d5 service nova] Lock "cb0f042e-43c0-43aa-b243-cfc78a7b2695-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:29:53 user nova-compute[75234]: DEBUG nova.compute.manager [req-8e302b52-e737-4f39-8a13-be1c892c3f44 req-8e89bd01-5499-4aeb-a08c-160817cf06d5 service nova] [instance: cb0f042e-43c0-43aa-b243-cfc78a7b2695] No waiting events found dispatching network-vif-plugged-8f9642c3-9e10-4aa7-b5a2-53bb68ad96dc {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:29:53 user nova-compute[75234]: WARNING nova.compute.manager [req-8e302b52-e737-4f39-8a13-be1c892c3f44 req-8e89bd01-5499-4aeb-a08c-160817cf06d5 service nova] [instance: cb0f042e-43c0-43aa-b243-cfc78a7b2695] Received unexpected event network-vif-plugged-8f9642c3-9e10-4aa7-b5a2-53bb68ad96dc for instance with vm_state building and task_state spawning.
Nov 29 03:29:53 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:29:53 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:29:53 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:29:53 user nova-compute[75234]: DEBUG nova.compute.manager [req-9f1a2a7b-2782-4c19-85a0-0ca28167ebfd req-80e3e862-4623-4445-8b90-e2d282d39b4b service nova] [instance: 1e9263fc-5735-420f-a116-8f60c8736faf] Received event network-vif-plugged-f836309d-8f97-4521-aa57-d6516a9e7528 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:29:53 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-9f1a2a7b-2782-4c19-85a0-0ca28167ebfd req-80e3e862-4623-4445-8b90-e2d282d39b4b service nova] Acquiring lock "1e9263fc-5735-420f-a116-8f60c8736faf-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:29:53 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-9f1a2a7b-2782-4c19-85a0-0ca28167ebfd req-80e3e862-4623-4445-8b90-e2d282d39b4b service nova] Lock "1e9263fc-5735-420f-a116-8f60c8736faf-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:29:53 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-9f1a2a7b-2782-4c19-85a0-0ca28167ebfd req-80e3e862-4623-4445-8b90-e2d282d39b4b service nova] Lock "1e9263fc-5735-420f-a116-8f60c8736faf-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:29:53 user nova-compute[75234]: DEBUG nova.compute.manager [req-9f1a2a7b-2782-4c19-85a0-0ca28167ebfd req-80e3e862-4623-4445-8b90-e2d282d39b4b service nova] [instance: 1e9263fc-5735-420f-a116-8f60c8736faf] No waiting events found dispatching network-vif-plugged-f836309d-8f97-4521-aa57-d6516a9e7528 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:29:53 user nova-compute[75234]: WARNING nova.compute.manager [req-9f1a2a7b-2782-4c19-85a0-0ca28167ebfd req-80e3e862-4623-4445-8b90-e2d282d39b4b service nova] [instance: 1e9263fc-5735-420f-a116-8f60c8736faf] Received unexpected event network-vif-plugged-f836309d-8f97-4521-aa57-d6516a9e7528 for instance with vm_state building and task_state spawning.
Nov 29 03:29:53 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:29:53 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:29:53 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:29:53 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:29:54 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:29:54 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:29:55 user nova-compute[75234]: DEBUG nova.compute.manager [req-94729a2e-9b83-447e-833d-11bcbc1a524e req-0adabfa8-96fa-4981-b6d3-282956ae2ab5 service nova] [instance: cb0f042e-43c0-43aa-b243-cfc78a7b2695] Received event network-vif-plugged-8f9642c3-9e10-4aa7-b5a2-53bb68ad96dc {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:29:55 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-94729a2e-9b83-447e-833d-11bcbc1a524e req-0adabfa8-96fa-4981-b6d3-282956ae2ab5 service nova] Acquiring lock "cb0f042e-43c0-43aa-b243-cfc78a7b2695-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:29:55 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-94729a2e-9b83-447e-833d-11bcbc1a524e req-0adabfa8-96fa-4981-b6d3-282956ae2ab5 service nova] Lock "cb0f042e-43c0-43aa-b243-cfc78a7b2695-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:29:55 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-94729a2e-9b83-447e-833d-11bcbc1a524e req-0adabfa8-96fa-4981-b6d3-282956ae2ab5 service nova] Lock "cb0f042e-43c0-43aa-b243-cfc78a7b2695-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:29:55 user nova-compute[75234]: DEBUG nova.compute.manager [req-94729a2e-9b83-447e-833d-11bcbc1a524e req-0adabfa8-96fa-4981-b6d3-282956ae2ab5 service nova] [instance: cb0f042e-43c0-43aa-b243-cfc78a7b2695] No waiting events found dispatching network-vif-plugged-8f9642c3-9e10-4aa7-b5a2-53bb68ad96dc {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:29:55 user nova-compute[75234]: WARNING nova.compute.manager [req-94729a2e-9b83-447e-833d-11bcbc1a524e req-0adabfa8-96fa-4981-b6d3-282956ae2ab5 service nova] [instance: cb0f042e-43c0-43aa-b243-cfc78a7b2695] Received unexpected event network-vif-plugged-8f9642c3-9e10-4aa7-b5a2-53bb68ad96dc for instance with vm_state building and task_state spawning.
Nov 29 03:29:55 user nova-compute[75234]: DEBUG nova.compute.manager [req-47bced33-d8fd-4f4d-b9c5-efdd3707f8e5 req-b208437b-6386-4c5a-838e-1f43ee81f480 service nova] [instance: 1e9263fc-5735-420f-a116-8f60c8736faf] Received event network-vif-plugged-f836309d-8f97-4521-aa57-d6516a9e7528 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:29:55 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-47bced33-d8fd-4f4d-b9c5-efdd3707f8e5 req-b208437b-6386-4c5a-838e-1f43ee81f480 service nova] Acquiring lock "1e9263fc-5735-420f-a116-8f60c8736faf-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:29:55 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-47bced33-d8fd-4f4d-b9c5-efdd3707f8e5 req-b208437b-6386-4c5a-838e-1f43ee81f480 service nova] Lock "1e9263fc-5735-420f-a116-8f60c8736faf-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:29:55 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-47bced33-d8fd-4f4d-b9c5-efdd3707f8e5 req-b208437b-6386-4c5a-838e-1f43ee81f480 service nova] Lock "1e9263fc-5735-420f-a116-8f60c8736faf-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:29:55 user nova-compute[75234]: DEBUG nova.compute.manager [req-47bced33-d8fd-4f4d-b9c5-efdd3707f8e5 req-b208437b-6386-4c5a-838e-1f43ee81f480 service nova] [instance: 1e9263fc-5735-420f-a116-8f60c8736faf] No waiting events found dispatching network-vif-plugged-f836309d-8f97-4521-aa57-d6516a9e7528 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:29:55 user nova-compute[75234]: WARNING nova.compute.manager [req-47bced33-d8fd-4f4d-b9c5-efdd3707f8e5 req-b208437b-6386-4c5a-838e-1f43ee81f480 service nova] [instance: 1e9263fc-5735-420f-a116-8f60c8736faf] Received unexpected event network-vif-plugged-f836309d-8f97-4521-aa57-d6516a9e7528 for instance with vm_state building and task_state spawning.
Nov 29 03:29:55 user nova-compute[75234]: DEBUG nova.compute.manager [None req-a124e922-fc6a-431a-ba05-92a9e593c1a7 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] [instance: cb0f042e-43c0-43aa-b243-cfc78a7b2695] Instance event wait completed in 0 seconds for  {{(pid=75234) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Nov 29 03:29:55 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-a124e922-fc6a-431a-ba05-92a9e593c1a7 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] [instance: cb0f042e-43c0-43aa-b243-cfc78a7b2695] Guest created on hypervisor {{(pid=75234) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4421}}
Nov 29 03:29:55 user nova-compute[75234]: INFO nova.virt.libvirt.driver [-] [instance: cb0f042e-43c0-43aa-b243-cfc78a7b2695] Instance spawned successfully.
Nov 29 03:29:55 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-a124e922-fc6a-431a-ba05-92a9e593c1a7 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] [instance: cb0f042e-43c0-43aa-b243-cfc78a7b2695] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:904}}
Nov 29 03:29:55 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-a124e922-fc6a-431a-ba05-92a9e593c1a7 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] [instance: cb0f042e-43c0-43aa-b243-cfc78a7b2695] Found default for hw_cdrom_bus of ide {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:29:55 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-a124e922-fc6a-431a-ba05-92a9e593c1a7 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] [instance: cb0f042e-43c0-43aa-b243-cfc78a7b2695] Found default for hw_disk_bus of virtio {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:29:55 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-a124e922-fc6a-431a-ba05-92a9e593c1a7 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] [instance: cb0f042e-43c0-43aa-b243-cfc78a7b2695] Found default for hw_input_bus of None {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:29:55 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-a124e922-fc6a-431a-ba05-92a9e593c1a7 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] [instance: cb0f042e-43c0-43aa-b243-cfc78a7b2695] Found default for hw_pointer_model of None {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:29:55 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-a124e922-fc6a-431a-ba05-92a9e593c1a7 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] [instance: cb0f042e-43c0-43aa-b243-cfc78a7b2695] Found default for hw_video_model of virtio {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:29:55 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-a124e922-fc6a-431a-ba05-92a9e593c1a7 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] [instance: cb0f042e-43c0-43aa-b243-cfc78a7b2695] Found default for hw_vif_model of virtio {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:29:55 user nova-compute[75234]: DEBUG nova.compute.manager [None req-cbb5b5ed-441c-455b-9e87-49b73c2c4cb0 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: 1e9263fc-5735-420f-a116-8f60c8736faf] Instance event wait completed in 0 seconds for  {{(pid=75234) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Nov 29 03:29:55 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-cbb5b5ed-441c-455b-9e87-49b73c2c4cb0 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: 1e9263fc-5735-420f-a116-8f60c8736faf] Guest created on hypervisor {{(pid=75234) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4421}}
Nov 29 03:29:55 user nova-compute[75234]: INFO nova.virt.libvirt.driver [-] [instance: 1e9263fc-5735-420f-a116-8f60c8736faf] Instance spawned successfully.
Nov 29 03:29:55 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-cbb5b5ed-441c-455b-9e87-49b73c2c4cb0 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: 1e9263fc-5735-420f-a116-8f60c8736faf] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:904}}
Nov 29 03:29:55 user nova-compute[75234]: INFO nova.compute.manager [None req-a124e922-fc6a-431a-ba05-92a9e593c1a7 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] [instance: cb0f042e-43c0-43aa-b243-cfc78a7b2695] Took 8.34 seconds to spawn the instance on the hypervisor.
Nov 29 03:29:55 user nova-compute[75234]: DEBUG nova.compute.manager [None req-a124e922-fc6a-431a-ba05-92a9e593c1a7 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] [instance: cb0f042e-43c0-43aa-b243-cfc78a7b2695] Checking state {{(pid=75234) _get_power_state /opt/stack/nova/nova/compute/manager.py:1782}}
Nov 29 03:29:56 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-cbb5b5ed-441c-455b-9e87-49b73c2c4cb0 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: 1e9263fc-5735-420f-a116-8f60c8736faf] Found default for hw_cdrom_bus of ide {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:29:56 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-cbb5b5ed-441c-455b-9e87-49b73c2c4cb0 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: 1e9263fc-5735-420f-a116-8f60c8736faf] Found default for hw_disk_bus of virtio {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:29:56 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-cbb5b5ed-441c-455b-9e87-49b73c2c4cb0 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: 1e9263fc-5735-420f-a116-8f60c8736faf] Found default for hw_input_bus of None {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:29:56 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-cbb5b5ed-441c-455b-9e87-49b73c2c4cb0 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: 1e9263fc-5735-420f-a116-8f60c8736faf] Found default for hw_pointer_model of None {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:29:56 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-cbb5b5ed-441c-455b-9e87-49b73c2c4cb0 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: 1e9263fc-5735-420f-a116-8f60c8736faf] Found default for hw_video_model of virtio {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:29:56 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-cbb5b5ed-441c-455b-9e87-49b73c2c4cb0 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: 1e9263fc-5735-420f-a116-8f60c8736faf] Found default for hw_vif_model of virtio {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:29:56 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:29:56 user nova-compute[75234]: INFO nova.compute.manager [None req-cbb5b5ed-441c-455b-9e87-49b73c2c4cb0 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: 1e9263fc-5735-420f-a116-8f60c8736faf] Took 6.64 seconds to spawn the instance on the hypervisor.
Nov 29 03:29:56 user nova-compute[75234]: DEBUG nova.compute.manager [None req-cbb5b5ed-441c-455b-9e87-49b73c2c4cb0 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: 1e9263fc-5735-420f-a116-8f60c8736faf] Checking state {{(pid=75234) _get_power_state /opt/stack/nova/nova/compute/manager.py:1782}}
Nov 29 03:29:56 user nova-compute[75234]: INFO nova.compute.manager [None req-a124e922-fc6a-431a-ba05-92a9e593c1a7 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] [instance: cb0f042e-43c0-43aa-b243-cfc78a7b2695] Took 9.09 seconds to build instance.
Nov 29 03:29:56 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-a124e922-fc6a-431a-ba05-92a9e593c1a7 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Lock "cb0f042e-43c0-43aa-b243-cfc78a7b2695" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 9.188s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:29:56 user nova-compute[75234]: INFO nova.compute.manager [None req-cbb5b5ed-441c-455b-9e87-49b73c2c4cb0 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: 1e9263fc-5735-420f-a116-8f60c8736faf] Took 21.46 seconds to build instance.
Nov 29 03:29:56 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-cbb5b5ed-441c-455b-9e87-49b73c2c4cb0 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Lock "1e9263fc-5735-420f-a116-8f60c8736faf" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 21.557s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:29:57 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:30:01 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:30:02 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:30:06 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:30:07 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:30:07 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:30:08 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:30:08 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:30:09 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:30:10 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:30:10 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=75234) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10541}}
Nov 29 03:30:11 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:30:11 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager.update_available_resource {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:30:11 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:30:11 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:30:11 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:30:11 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Auditing locally available compute resources for user (node: user) {{(pid=75234) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:907}}
Nov 29 03:30:11 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/65d48c93-b6da-4f51-9d5a-0676dda62a60/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:30:11 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/65d48c93-b6da-4f51-9d5a-0676dda62a60/disk --force-share --output=json" returned: 0 in 0.165s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:30:11 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/65d48c93-b6da-4f51-9d5a-0676dda62a60/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:30:11 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/65d48c93-b6da-4f51-9d5a-0676dda62a60/disk --force-share --output=json" returned: 0 in 0.175s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:30:11 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] skipping disk /dev/sdd (vda) as it is a volume {{(pid=75234) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11213}}
Nov 29 03:30:12 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] skipping disk /dev/sdc (vda) as it is a volume {{(pid=75234) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11213}}
Nov 29 03:30:12 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] skipping disk /dev/sdb (vda) as it is a volume {{(pid=75234) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11213}}
Nov 29 03:30:12 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/abb01889-df4b-44a6-9827-87056d7ca125/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:30:12 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/abb01889-df4b-44a6-9827-87056d7ca125/disk --force-share --output=json" returned: 0 in 0.196s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:30:12 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/abb01889-df4b-44a6-9827-87056d7ca125/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:30:12 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/abb01889-df4b-44a6-9827-87056d7ca125/disk --force-share --output=json" returned: 0 in 0.188s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:30:12 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/cb0f042e-43c0-43aa-b243-cfc78a7b2695/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:30:12 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/cb0f042e-43c0-43aa-b243-cfc78a7b2695/disk --force-share --output=json" returned: 0 in 0.189s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:30:12 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/cb0f042e-43c0-43aa-b243-cfc78a7b2695/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:30:12 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:30:12 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/cb0f042e-43c0-43aa-b243-cfc78a7b2695/disk --force-share --output=json" returned: 0 in 0.184s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:30:13 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:30:13 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:30:13 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Hypervisor/Node resource view: name=user free_ram=8149MB free_disk=25.039077758789062GB free_vcpus=6 pci_devices=[{"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}] {{(pid=75234) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1106}}
Nov 29 03:30:13 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:30:13 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:30:13 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance 65d48c93-b6da-4f51-9d5a-0676dda62a60 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 512, 'DISK_GB': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:30:13 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance d5096ca2-d005-45ac-a487-11a810446859 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 512}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:30:13 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance abb01889-df4b-44a6-9827-87056d7ca125 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 512, 'DISK_GB': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:30:13 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance 01d62e54-9957-40b1-8dfe-1783b64ffabf actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 512}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:30:13 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance 1e9263fc-5735-420f-a116-8f60c8736faf actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 512}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:30:13 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance cb0f042e-43c0-43aa-b243-cfc78a7b2695 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 512, 'DISK_GB': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:30:13 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Total usable vcpus: 12, total allocated vcpus: 6 {{(pid=75234) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1129}}
Nov 29 03:30:13 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Final resource view: name=user phys_ram=16011MB used_ram=3584MB phys_disk=40GB used_disk=3GB total_vcpus=12 used_vcpus=6 pci_stats=[] {{(pid=75234) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1138}}
Nov 29 03:30:13 user nova-compute[75234]: DEBUG nova.compute.provider_tree [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Inventory has not changed in ProviderTree for provider: 4d291220-7c37-4839-a0e3-b23c66502b74 {{(pid=75234) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Nov 29 03:30:13 user nova-compute[75234]: DEBUG nova.scheduler.client.report [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Inventory has not changed for provider 4d291220-7c37-4839-a0e3-b23c66502b74 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=75234) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Nov 29 03:30:13 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Compute_service record updated for user:user {{(pid=75234) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:1067}}
Nov 29 03:30:13 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.346s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:30:15 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:30:15 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Starting heal instance info cache {{(pid=75234) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9922}}
Nov 29 03:30:15 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Didn't find any instances for network info cache update. {{(pid=75234) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:10008}}
Nov 29 03:30:15 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:30:15 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:30:16 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:30:17 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:30:19 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:30:21 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:30:22 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:30:23 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._sync_scheduler_instance_info {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:30:24 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:30:26 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:30:27 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:30:31 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:30:32 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:30:36 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:30:41 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:30:41 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:30:41 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5002 ms, sending inactivity probe {{(pid=75234) run /opt/stack/data/venv/lib/python3.10/site-packages/ovs/reconnect.py:117}}
Nov 29 03:30:41 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=75234) _transition /opt/stack/data/venv/lib/python3.10/site-packages/ovs/reconnect.py:519}}
Nov 29 03:30:41 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=75234) _transition /opt/stack/data/venv/lib/python3.10/site-packages/ovs/reconnect.py:519}}
Nov 29 03:30:41 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:30:46 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:30:51 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:30:56 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:31:01 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:31:06 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:31:09 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:31:09 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:31:09 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:31:11 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:31:11 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:31:11 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5002 ms, sending inactivity probe {{(pid=75234) run /opt/stack/data/venv/lib/python3.10/site-packages/ovs/reconnect.py:117}}
Nov 29 03:31:11 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=75234) _transition /opt/stack/data/venv/lib/python3.10/site-packages/ovs/reconnect.py:519}}
Nov 29 03:31:11 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=75234) _transition /opt/stack/data/venv/lib/python3.10/site-packages/ovs/reconnect.py:519}}
Nov 29 03:31:11 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:31:11 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager.update_available_resource {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:31:11 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:31:11 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:31:11 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:31:11 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Auditing locally available compute resources for user (node: user) {{(pid=75234) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:907}}
Nov 29 03:31:11 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/65d48c93-b6da-4f51-9d5a-0676dda62a60/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:31:11 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/65d48c93-b6da-4f51-9d5a-0676dda62a60/disk --force-share --output=json" returned: 0 in 0.171s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:31:11 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/65d48c93-b6da-4f51-9d5a-0676dda62a60/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:31:11 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/65d48c93-b6da-4f51-9d5a-0676dda62a60/disk --force-share --output=json" returned: 0 in 0.168s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:31:11 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] skipping disk /dev/sdd (vda) as it is a volume {{(pid=75234) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11213}}
Nov 29 03:31:11 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] skipping disk /dev/sdc (vda) as it is a volume {{(pid=75234) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11213}}
Nov 29 03:31:11 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] skipping disk /dev/sdb (vda) as it is a volume {{(pid=75234) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11213}}
Nov 29 03:31:11 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/abb01889-df4b-44a6-9827-87056d7ca125/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:31:12 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/abb01889-df4b-44a6-9827-87056d7ca125/disk --force-share --output=json" returned: 0 in 0.180s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:31:12 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/abb01889-df4b-44a6-9827-87056d7ca125/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:31:12 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/abb01889-df4b-44a6-9827-87056d7ca125/disk --force-share --output=json" returned: 0 in 0.167s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:31:12 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/cb0f042e-43c0-43aa-b243-cfc78a7b2695/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:31:12 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/cb0f042e-43c0-43aa-b243-cfc78a7b2695/disk --force-share --output=json" returned: 0 in 0.169s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:31:12 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/cb0f042e-43c0-43aa-b243-cfc78a7b2695/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:31:12 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/cb0f042e-43c0-43aa-b243-cfc78a7b2695/disk --force-share --output=json" returned: 0 in 0.244s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:31:13 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:31:13 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:31:13 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Hypervisor/Node resource view: name=user free_ram=8299MB free_disk=25.034774780273438GB free_vcpus=6 pci_devices=[{"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}] {{(pid=75234) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1106}}
Nov 29 03:31:13 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:31:13 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:31:13 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance 65d48c93-b6da-4f51-9d5a-0676dda62a60 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 512, 'DISK_GB': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:31:13 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance d5096ca2-d005-45ac-a487-11a810446859 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 512}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:31:13 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance abb01889-df4b-44a6-9827-87056d7ca125 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 512, 'DISK_GB': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:31:13 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance 01d62e54-9957-40b1-8dfe-1783b64ffabf actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 512}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:31:13 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance 1e9263fc-5735-420f-a116-8f60c8736faf actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 512}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:31:13 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance cb0f042e-43c0-43aa-b243-cfc78a7b2695 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 512, 'DISK_GB': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:31:13 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Total usable vcpus: 12, total allocated vcpus: 6 {{(pid=75234) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1129}}
Nov 29 03:31:13 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Final resource view: name=user phys_ram=16011MB used_ram=3584MB phys_disk=40GB used_disk=3GB total_vcpus=12 used_vcpus=6 pci_stats=[] {{(pid=75234) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1138}}
Nov 29 03:31:13 user nova-compute[75234]: DEBUG nova.compute.provider_tree [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Inventory has not changed in ProviderTree for provider: 4d291220-7c37-4839-a0e3-b23c66502b74 {{(pid=75234) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Nov 29 03:31:13 user nova-compute[75234]: DEBUG nova.scheduler.client.report [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Inventory has not changed for provider 4d291220-7c37-4839-a0e3-b23c66502b74 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=75234) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Nov 29 03:31:13 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Compute_service record updated for user:user {{(pid=75234) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:1067}}
Nov 29 03:31:13 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.335s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:31:14 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:31:14 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Starting heal instance info cache {{(pid=75234) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9922}}
Nov 29 03:31:14 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Rebuilding the list of instances to heal {{(pid=75234) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9926}}
Nov 29 03:31:14 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Acquiring lock "refresh_cache-65d48c93-b6da-4f51-9d5a-0676dda62a60" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:31:14 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Acquired lock "refresh_cache-65d48c93-b6da-4f51-9d5a-0676dda62a60" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:31:14 user nova-compute[75234]: DEBUG nova.network.neutron [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] [instance: 65d48c93-b6da-4f51-9d5a-0676dda62a60] Forcefully refreshing network info cache for instance {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2003}}
Nov 29 03:31:14 user nova-compute[75234]: DEBUG nova.objects.instance [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lazy-loading 'info_cache' on Instance uuid 65d48c93-b6da-4f51-9d5a-0676dda62a60 {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:31:15 user nova-compute[75234]: DEBUG nova.network.neutron [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] [instance: 65d48c93-b6da-4f51-9d5a-0676dda62a60] Updating instance_info_cache with network_info: [{"id": "967835a2-4b55-4ae6-b071-91233af412fd", "address": "fa:16:3e:ee:3c:e2", "network": {"id": "4d086e08-7798-4cfe-9365-86be6f0a52d2", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-305624008-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "db00bcd0c82c44f1bda739e9857f6e84", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap967835a2-4b", "ovs_interfaceid": "967835a2-4b55-4ae6-b071-91233af412fd", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:31:15 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Releasing lock "refresh_cache-65d48c93-b6da-4f51-9d5a-0676dda62a60" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:31:15 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] [instance: 65d48c93-b6da-4f51-9d5a-0676dda62a60] Updated the network info_cache for instance {{(pid=75234) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9993}}
Nov 29 03:31:15 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:31:15 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=75234) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10541}}
Nov 29 03:31:16 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:31:17 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:31:17 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:31:20 user nova-compute[75234]: DEBUG nova.compute.manager [None req-4c39de24-7e12-41de-a2da-7c56dc78ca87 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] [instance: abb01889-df4b-44a6-9827-87056d7ca125] Checking state {{(pid=75234) _get_power_state /opt/stack/nova/nova/compute/manager.py:1782}}
Nov 29 03:31:20 user nova-compute[75234]: INFO nova.compute.manager [None req-4c39de24-7e12-41de-a2da-7c56dc78ca87 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] [instance: abb01889-df4b-44a6-9827-87056d7ca125] instance snapshotting
Nov 29 03:31:20 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-4c39de24-7e12-41de-a2da-7c56dc78ca87 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] [instance: abb01889-df4b-44a6-9827-87056d7ca125] Beginning live snapshot process
Nov 29 03:31:20 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-4c39de24-7e12-41de-a2da-7c56dc78ca87 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/abb01889-df4b-44a6-9827-87056d7ca125/disk --force-share --output=json -f qcow2 {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:31:20 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-4c39de24-7e12-41de-a2da-7c56dc78ca87 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/abb01889-df4b-44a6-9827-87056d7ca125/disk --force-share --output=json -f qcow2" returned: 0 in 0.196s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:31:20 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-4c39de24-7e12-41de-a2da-7c56dc78ca87 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/abb01889-df4b-44a6-9827-87056d7ca125/disk --force-share --output=json -f qcow2 {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:31:20 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-4c39de24-7e12-41de-a2da-7c56dc78ca87 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/abb01889-df4b-44a6-9827-87056d7ca125/disk --force-share --output=json -f qcow2" returned: 0 in 0.205s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:31:20 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-4c39de24-7e12-41de-a2da-7c56dc78ca87 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:31:20 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:31:21 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-4c39de24-7e12-41de-a2da-7c56dc78ca87 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918 --force-share --output=json" returned: 0 in 0.227s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:31:21 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-4c39de24-7e12-41de-a2da-7c56dc78ca87 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918,backing_fmt=raw /opt/stack/data/nova/instances/snapshots/tmp0swo8c4g/85d713bb6a3048eb8d7dc887cb5adacc.delta 1073741824 {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:31:21 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-4c39de24-7e12-41de-a2da-7c56dc78ca87 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918,backing_fmt=raw /opt/stack/data/nova/instances/snapshots/tmp0swo8c4g/85d713bb6a3048eb8d7dc887cb5adacc.delta 1073741824" returned: 0 in 0.062s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:31:21 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-4c39de24-7e12-41de-a2da-7c56dc78ca87 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] [instance: abb01889-df4b-44a6-9827-87056d7ca125] Quiescing instance not available: QEMU guest agent is not enabled.
Nov 29 03:31:21 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:31:21 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:31:21 user nova-compute[75234]: DEBUG nova.virt.libvirt.guest [None req-4c39de24-7e12-41de-a2da-7c56dc78ca87 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] COPY block job progress, current cursor: 0 final cursor: 75497472 {{(pid=75234) is_job_complete /opt/stack/nova/nova/virt/libvirt/guest.py:846}}
Nov 29 03:31:22 user nova-compute[75234]: DEBUG nova.compute.manager [req-4cae09d6-5497-4e96-b9d6-37f22d159756 req-595cf23d-0dfe-4f6e-86b8-fe1fd5ed9fa9 service nova] [instance: 01d62e54-9957-40b1-8dfe-1783b64ffabf] Received event network-changed-6dbd7c27-31c9-4ddc-a81b-85f1e742d160 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:31:22 user nova-compute[75234]: DEBUG nova.compute.manager [req-4cae09d6-5497-4e96-b9d6-37f22d159756 req-595cf23d-0dfe-4f6e-86b8-fe1fd5ed9fa9 service nova] [instance: 01d62e54-9957-40b1-8dfe-1783b64ffabf] Refreshing instance network info cache due to event network-changed-6dbd7c27-31c9-4ddc-a81b-85f1e742d160. {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11117}}
Nov 29 03:31:22 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-4cae09d6-5497-4e96-b9d6-37f22d159756 req-595cf23d-0dfe-4f6e-86b8-fe1fd5ed9fa9 service nova] Acquiring lock "refresh_cache-01d62e54-9957-40b1-8dfe-1783b64ffabf" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:31:22 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-4cae09d6-5497-4e96-b9d6-37f22d159756 req-595cf23d-0dfe-4f6e-86b8-fe1fd5ed9fa9 service nova] Acquired lock "refresh_cache-01d62e54-9957-40b1-8dfe-1783b64ffabf" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:31:22 user nova-compute[75234]: DEBUG nova.network.neutron [req-4cae09d6-5497-4e96-b9d6-37f22d159756 req-595cf23d-0dfe-4f6e-86b8-fe1fd5ed9fa9 service nova] [instance: 01d62e54-9957-40b1-8dfe-1783b64ffabf] Refreshing network info cache for port 6dbd7c27-31c9-4ddc-a81b-85f1e742d160 {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2006}}
Nov 29 03:31:22 user nova-compute[75234]: DEBUG nova.virt.libvirt.guest [None req-4c39de24-7e12-41de-a2da-7c56dc78ca87 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] COPY block job progress, current cursor: 27066368 final cursor: 75497472 {{(pid=75234) is_job_complete /opt/stack/nova/nova/virt/libvirt/guest.py:846}}
Nov 29 03:31:22 user nova-compute[75234]: DEBUG nova.network.neutron [req-4cae09d6-5497-4e96-b9d6-37f22d159756 req-595cf23d-0dfe-4f6e-86b8-fe1fd5ed9fa9 service nova] [instance: 01d62e54-9957-40b1-8dfe-1783b64ffabf] Updated VIF entry in instance network info cache for port 6dbd7c27-31c9-4ddc-a81b-85f1e742d160. {{(pid=75234) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3481}}
Nov 29 03:31:22 user nova-compute[75234]: DEBUG nova.network.neutron [req-4cae09d6-5497-4e96-b9d6-37f22d159756 req-595cf23d-0dfe-4f6e-86b8-fe1fd5ed9fa9 service nova] [instance: 01d62e54-9957-40b1-8dfe-1783b64ffabf] Updating instance_info_cache with network_info: [{"id": "6dbd7c27-31c9-4ddc-a81b-85f1e742d160", "address": "fa:16:3e:57:6a:26", "network": {"id": "504616f5-0d6b-47a6-aaf5-663983428b41", "bridge": "br-int", "label": "tempest-TestShelveInstance-438404840-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.210", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "53e55fcb8cd74895baec6f9528770bbc", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap6dbd7c27-31", "ovs_interfaceid": "6dbd7c27-31c9-4ddc-a81b-85f1e742d160", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:31:22 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-4cae09d6-5497-4e96-b9d6-37f22d159756 req-595cf23d-0dfe-4f6e-86b8-fe1fd5ed9fa9 service nova] Releasing lock "refresh_cache-01d62e54-9957-40b1-8dfe-1783b64ffabf" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:31:22 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:31:22 user nova-compute[75234]: DEBUG nova.virt.libvirt.guest [None req-4c39de24-7e12-41de-a2da-7c56dc78ca87 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] COPY block job progress, current cursor: 75497472 final cursor: 75497472 {{(pid=75234) is_job_complete /opt/stack/nova/nova/virt/libvirt/guest.py:846}}
Nov 29 03:31:22 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-4c39de24-7e12-41de-a2da-7c56dc78ca87 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] [instance: abb01889-df4b-44a6-9827-87056d7ca125] Skipping quiescing instance: QEMU guest agent is not enabled.
Nov 29 03:31:22 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-4d7abd60-3c0e-4be4-a5be-3af6bcac3c17 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] Acquiring lock "01d62e54-9957-40b1-8dfe-1783b64ffabf" by "nova.compute.manager.ComputeManager.shelve_offload_instance.<locals>.do_shelve_offload_instance" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:31:22 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-4d7abd60-3c0e-4be4-a5be-3af6bcac3c17 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] Lock "01d62e54-9957-40b1-8dfe-1783b64ffabf" acquired by "nova.compute.manager.ComputeManager.shelve_offload_instance.<locals>.do_shelve_offload_instance" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:31:22 user nova-compute[75234]: INFO nova.compute.manager [None req-4d7abd60-3c0e-4be4-a5be-3af6bcac3c17 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] [instance: 01d62e54-9957-40b1-8dfe-1783b64ffabf] Shelve offloading
Nov 29 03:31:22 user nova-compute[75234]: DEBUG nova.privsep.utils [None req-4c39de24-7e12-41de-a2da-7c56dc78ca87 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Path '/opt/stack/data/nova/instances' supports direct I/O {{(pid=75234) supports_direct_io /opt/stack/nova/nova/privsep/utils.py:63}}
Nov 29 03:31:22 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-4c39de24-7e12-41de-a2da-7c56dc78ca87 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Running cmd (subprocess): qemu-img convert -t none -O qcow2 -f qcow2 /opt/stack/data/nova/instances/snapshots/tmp0swo8c4g/85d713bb6a3048eb8d7dc887cb5adacc.delta /opt/stack/data/nova/instances/snapshots/tmp0swo8c4g/85d713bb6a3048eb8d7dc887cb5adacc {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:31:22 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:31:22 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:31:22 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:31:23 user nova-compute[75234]: DEBUG nova.compute.manager [req-e67b2f74-0ef5-4b5f-aa87-172203a49506 req-14f35fec-125f-4960-a311-2e53325d6634 service nova] [instance: 01d62e54-9957-40b1-8dfe-1783b64ffabf] Received event network-vif-unplugged-6dbd7c27-31c9-4ddc-a81b-85f1e742d160 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:31:23 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-e67b2f74-0ef5-4b5f-aa87-172203a49506 req-14f35fec-125f-4960-a311-2e53325d6634 service nova] Acquiring lock "01d62e54-9957-40b1-8dfe-1783b64ffabf-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:31:23 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-e67b2f74-0ef5-4b5f-aa87-172203a49506 req-14f35fec-125f-4960-a311-2e53325d6634 service nova] Lock "01d62e54-9957-40b1-8dfe-1783b64ffabf-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:31:23 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-e67b2f74-0ef5-4b5f-aa87-172203a49506 req-14f35fec-125f-4960-a311-2e53325d6634 service nova] Lock "01d62e54-9957-40b1-8dfe-1783b64ffabf-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:31:23 user nova-compute[75234]: DEBUG nova.compute.manager [req-e67b2f74-0ef5-4b5f-aa87-172203a49506 req-14f35fec-125f-4960-a311-2e53325d6634 service nova] [instance: 01d62e54-9957-40b1-8dfe-1783b64ffabf] No waiting events found dispatching network-vif-unplugged-6dbd7c27-31c9-4ddc-a81b-85f1e742d160 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:31:23 user nova-compute[75234]: WARNING nova.compute.manager [req-e67b2f74-0ef5-4b5f-aa87-172203a49506 req-14f35fec-125f-4960-a311-2e53325d6634 service nova] [instance: 01d62e54-9957-40b1-8dfe-1783b64ffabf] Received unexpected event network-vif-unplugged-6dbd7c27-31c9-4ddc-a81b-85f1e742d160 for instance with vm_state active and task_state shelving.
Nov 29 03:31:23 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-4c39de24-7e12-41de-a2da-7c56dc78ca87 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] CMD "qemu-img convert -t none -O qcow2 -f qcow2 /opt/stack/data/nova/instances/snapshots/tmp0swo8c4g/85d713bb6a3048eb8d7dc887cb5adacc.delta /opt/stack/data/nova/instances/snapshots/tmp0swo8c4g/85d713bb6a3048eb8d7dc887cb5adacc" returned: 0 in 0.400s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:31:23 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-4c39de24-7e12-41de-a2da-7c56dc78ca87 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] [instance: abb01889-df4b-44a6-9827-87056d7ca125] Snapshot extracted, beginning image upload
Nov 29 03:31:23 user nova-compute[75234]: INFO nova.virt.libvirt.driver [-] [instance: 01d62e54-9957-40b1-8dfe-1783b64ffabf] Instance destroyed successfully.
Nov 29 03:31:23 user nova-compute[75234]: DEBUG nova.compute.manager [None req-4d7abd60-3c0e-4be4-a5be-3af6bcac3c17 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] [instance: 01d62e54-9957-40b1-8dfe-1783b64ffabf] Checking state {{(pid=75234) _get_power_state /opt/stack/nova/nova/compute/manager.py:1782}}
Nov 29 03:31:23 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-4d7abd60-3c0e-4be4-a5be-3af6bcac3c17 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] Acquiring lock "refresh_cache-01d62e54-9957-40b1-8dfe-1783b64ffabf" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:31:23 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-4d7abd60-3c0e-4be4-a5be-3af6bcac3c17 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] Acquired lock "refresh_cache-01d62e54-9957-40b1-8dfe-1783b64ffabf" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:31:23 user nova-compute[75234]: DEBUG nova.network.neutron [None req-4d7abd60-3c0e-4be4-a5be-3af6bcac3c17 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] [instance: 01d62e54-9957-40b1-8dfe-1783b64ffabf] Building network info cache for instance {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2009}}
Nov 29 03:31:24 user nova-compute[75234]: DEBUG nova.network.neutron [None req-4d7abd60-3c0e-4be4-a5be-3af6bcac3c17 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] [instance: 01d62e54-9957-40b1-8dfe-1783b64ffabf] Updating instance_info_cache with network_info: [{"id": "6dbd7c27-31c9-4ddc-a81b-85f1e742d160", "address": "fa:16:3e:57:6a:26", "network": {"id": "504616f5-0d6b-47a6-aaf5-663983428b41", "bridge": "br-int", "label": "tempest-TestShelveInstance-438404840-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.210", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "53e55fcb8cd74895baec6f9528770bbc", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap6dbd7c27-31", "ovs_interfaceid": "6dbd7c27-31c9-4ddc-a81b-85f1e742d160", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:31:24 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-4d7abd60-3c0e-4be4-a5be-3af6bcac3c17 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] Releasing lock "refresh_cache-01d62e54-9957-40b1-8dfe-1783b64ffabf" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:31:24 user nova-compute[75234]: INFO nova.virt.libvirt.driver [-] [instance: 01d62e54-9957-40b1-8dfe-1783b64ffabf] Instance destroyed successfully.
Nov 29 03:31:24 user nova-compute[75234]: DEBUG nova.objects.instance [None req-4d7abd60-3c0e-4be4-a5be-3af6bcac3c17 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] Lazy-loading 'resources' on Instance uuid 01d62e54-9957-40b1-8dfe-1783b64ffabf {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:31:24 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-4d7abd60-3c0e-4be4-a5be-3af6bcac3c17 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-11-29T03:29:28Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-TestShelveInstance-server-1849172102',display_name='tempest-TestShelveInstance-server-1849172102',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-testshelveinstance-server-1849172102',id=44,image_ref='',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBPnJOw+AusvgDl0zYq6G0Nlkai7pjeaHxMjEe+mgW4X2uDyLlbOtVi5OTxN9x2/pgEhihEfkWnF2CarcswPn27UxxBjasnu8imtsEqjFxc/RzHA5mgFRB5eUL6c9v+ODYA==',key_name='tempest-TestShelveInstance-1141474138',keypairs=<?>,launch_index=0,launched_at=2023-11-29T03:29:43Z,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=<?>,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='53e55fcb8cd74895baec6f9528770bbc',ramdisk_id='',reservation_id='r-vhus4691',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-TestShelveInstance-1683282958',owner_user_name='tempest-TestShelveInstance-1683282958-project-member'},tags=<?>,task_state='shelving',terminated_at=None,trusted_certs=<?>,updated_at=2023-11-29T03:29:44Z,user_data=None,user_id='5728da78dc074c1993a8a2cc9b821ed6',uuid=01d62e54-9957-40b1-8dfe-1783b64ffabf,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "6dbd7c27-31c9-4ddc-a81b-85f1e742d160", "address": "fa:16:3e:57:6a:26", "network": {"id": "504616f5-0d6b-47a6-aaf5-663983428b41", "bridge": "br-int", "label": "tempest-TestShelveInstance-438404840-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.210", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "53e55fcb8cd74895baec6f9528770bbc", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap6dbd7c27-31", "ovs_interfaceid": "6dbd7c27-31c9-4ddc-a81b-85f1e742d160", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Nov 29 03:31:24 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-4d7abd60-3c0e-4be4-a5be-3af6bcac3c17 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] Converting VIF {"id": "6dbd7c27-31c9-4ddc-a81b-85f1e742d160", "address": "fa:16:3e:57:6a:26", "network": {"id": "504616f5-0d6b-47a6-aaf5-663983428b41", "bridge": "br-int", "label": "tempest-TestShelveInstance-438404840-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.210", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "53e55fcb8cd74895baec6f9528770bbc", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap6dbd7c27-31", "ovs_interfaceid": "6dbd7c27-31c9-4ddc-a81b-85f1e742d160", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:31:24 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-4d7abd60-3c0e-4be4-a5be-3af6bcac3c17 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:57:6a:26,bridge_name='br-int',has_traffic_filtering=True,id=6dbd7c27-31c9-4ddc-a81b-85f1e742d160,network=Network(504616f5-0d6b-47a6-aaf5-663983428b41),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap6dbd7c27-31') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:31:24 user nova-compute[75234]: DEBUG os_vif [None req-4d7abd60-3c0e-4be4-a5be-3af6bcac3c17 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] Unplugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:57:6a:26,bridge_name='br-int',has_traffic_filtering=True,id=6dbd7c27-31c9-4ddc-a81b-85f1e742d160,network=Network(504616f5-0d6b-47a6-aaf5-663983428b41),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap6dbd7c27-31') {{(pid=75234) unplug /opt/stack/data/venv/lib/python3.10/site-packages/os_vif/__init__.py:109}}
Nov 29 03:31:24 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:31:24 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tap6dbd7c27-31, bridge=br-int, if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:31:24 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:31:24 user nova-compute[75234]: DEBUG nova.compute.manager [req-34eb9f87-ff3a-4ccb-9570-b2f06486f18c req-db1f2b52-09d3-4f0f-be97-6e65cfabecc5 service nova] [instance: 01d62e54-9957-40b1-8dfe-1783b64ffabf] Received event network-changed-6dbd7c27-31c9-4ddc-a81b-85f1e742d160 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:31:24 user nova-compute[75234]: DEBUG nova.compute.manager [req-34eb9f87-ff3a-4ccb-9570-b2f06486f18c req-db1f2b52-09d3-4f0f-be97-6e65cfabecc5 service nova] [instance: 01d62e54-9957-40b1-8dfe-1783b64ffabf] Refreshing instance network info cache due to event network-changed-6dbd7c27-31c9-4ddc-a81b-85f1e742d160. {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11117}}
Nov 29 03:31:24 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-34eb9f87-ff3a-4ccb-9570-b2f06486f18c req-db1f2b52-09d3-4f0f-be97-6e65cfabecc5 service nova] Acquiring lock "refresh_cache-01d62e54-9957-40b1-8dfe-1783b64ffabf" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:31:24 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-34eb9f87-ff3a-4ccb-9570-b2f06486f18c req-db1f2b52-09d3-4f0f-be97-6e65cfabecc5 service nova] Acquired lock "refresh_cache-01d62e54-9957-40b1-8dfe-1783b64ffabf" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:31:24 user nova-compute[75234]: DEBUG nova.network.neutron [req-34eb9f87-ff3a-4ccb-9570-b2f06486f18c req-db1f2b52-09d3-4f0f-be97-6e65cfabecc5 service nova] [instance: 01d62e54-9957-40b1-8dfe-1783b64ffabf] Refreshing network info cache for port 6dbd7c27-31c9-4ddc-a81b-85f1e742d160 {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2006}}
Nov 29 03:31:24 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:31:24 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:31:24 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbDestroyCommand(_result=None, table=QoS, record=b6b3be26-a6af-4c78-9637-a24eb15e301a) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:31:24 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:31:24 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:31:24 user nova-compute[75234]: INFO os_vif [None req-4d7abd60-3c0e-4be4-a5be-3af6bcac3c17 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] Successfully unplugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:57:6a:26,bridge_name='br-int',has_traffic_filtering=True,id=6dbd7c27-31c9-4ddc-a81b-85f1e742d160,network=Network(504616f5-0d6b-47a6-aaf5-663983428b41),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap6dbd7c27-31')
Nov 29 03:31:24 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-4d7abd60-3c0e-4be4-a5be-3af6bcac3c17 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] [instance: 01d62e54-9957-40b1-8dfe-1783b64ffabf] calling os-brick to detach iSCSI Volume {{(pid=75234) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Nov 29 03:31:24 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-4d7abd60-3c0e-4be4-a5be-3af6bcac3c17 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f8a5050c9a0>, {'target_iqn': 'iqn.2010-10.org.openstack:43e92d4a-7e98-4777-8fa5-3cec396b86ce', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'AQnkxjFp', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdb'}, None), 'kwargs': True}}" {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:176}}
Nov 29 03:31:24 user nova-compute[75234]: WARNING os_brick.initiator.connectors.base [None req-4d7abd60-3c0e-4be4-a5be-3af6bcac3c17 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Nov 29 03:31:24 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-4d7abd60-3c0e-4be4-a5be-3af6bcac3c17 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:68}}
Nov 29 03:31:24 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-4d7abd60-3c0e-4be4-a5be-3af6bcac3c17 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:73}}
Nov 29 03:31:24 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:31:24 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.019s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:31:24 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[b3e3769a-be14-4765-9405-8599295fefcc]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:43e92d4a-7e98-4777-8fa5-3cec396b86ce\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:d1f13879-6e1b-4fe3-b97d-066a9e657ef7\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:31:24 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-4d7abd60-3c0e-4be4-a5be-3af6bcac3c17 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS:
Nov 29 03:31:24 user nova-compute[75234]: No targets found.
Nov 29 03:31:24 user nova-compute[75234]: iSNS:
Nov 29 03:31:24 user nova-compute[75234]: No targets found.
Nov 29 03:31:24 user nova-compute[75234]: STATIC:
Nov 29 03:31:24 user nova-compute[75234]: Target: iqn.2010-10.org.openstack:43e92d4a-7e98-4777-8fa5-3cec396b86ce
Nov 29 03:31:24 user nova-compute[75234]:         Portal: 172.16.0.220:3260,-1
Nov 29 03:31:24 user nova-compute[75234]:                 Iface Name: default
Nov 29 03:31:24 user nova-compute[75234]: Target: iqn.2010-10.org.openstack:d1f13879-6e1b-4fe3-b97d-066a9e657ef7
Nov 29 03:31:24 user nova-compute[75234]:         Portal: 172.16.0.220:3260,-1
Nov 29 03:31:24 user nova-compute[75234]:                 Iface Name: default
Nov 29 03:31:24 user nova-compute[75234]: Target: iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0
Nov 29 03:31:24 user nova-compute[75234]:         Portal: 172.16.0.220:3260,-1
Nov 29 03:31:24 user nova-compute[75234]:                 Iface Name: default
Nov 29 03:31:24 user nova-compute[75234]: FIRMWARE:
Nov 29 03:31:24 user nova-compute[75234]: No targets found.
Nov 29 03:31:24 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:31:24 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-4d7abd60-3c0e-4be4-a5be-3af6bcac3c17 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] Regex to get portals from discoverydb: ^SENDTARGETS:
Nov 29 03:31:24 user nova-compute[75234]: .*?^DiscoveryAddress: 172.16.0.220,3260.*?
Nov 29 03:31:24 user nova-compute[75234]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=75234) _get_discoverydb_portals /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:382}}
Nov 29 03:31:24 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-4d7abd60-3c0e-4be4-a5be-3af6bcac3c17 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:43e92d4a-7e98-4777-8fa5-3cec396b86ce', 0)] {{(pid=75234) _get_connection_devices /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:830}}
Nov 29 03:31:24 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:31:24 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.018s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:31:24 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[66822184-02f9-4da7-8116-0088b9e79b5e]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:43e92d4a-7e98-4777-8fa5-3cec396b86ce\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:d1f13879-6e1b-4fe3-b97d-066a9e657ef7\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:31:24 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:31:24 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.030s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:31:24 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[ed0864b8-2067-4735-857e-7243db2c8cf8]: (4, ('tcp: [42] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0 (non-flash)\ntcp: [50] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:43e92d4a-7e98-4777-8fa5-3cec396b86ce (non-flash)\ntcp: [51] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d1f13879-6e1b-4fe3-b97d-066a9e657ef7 (non-flash)\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:31:24 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-4d7abd60-3c0e-4be4-a5be-3af6bcac3c17 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [42] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0 (non-flash)
Nov 29 03:31:24 user nova-compute[75234]: tcp: [50] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:43e92d4a-7e98-4777-8fa5-3cec396b86ce (non-flash)
Nov 29 03:31:24 user nova-compute[75234]: tcp: [51] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d1f13879-6e1b-4fe3-b97d-066a9e657ef7 (non-flash)
Nov 29 03:31:24 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:31:24 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-4d7abd60-3c0e-4be4-a5be-3af6bcac3c17 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] iscsi session list stdout=tcp: [42] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0 (non-flash)
Nov 29 03:31:24 user nova-compute[75234]: tcp: [50] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:43e92d4a-7e98-4777-8fa5-3cec396b86ce (non-flash)
Nov 29 03:31:24 user nova-compute[75234]: tcp: [51] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d1f13879-6e1b-4fe3-b97d-066a9e657ef7 (non-flash)
Nov 29 03:31:24 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsi_session /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Nov 29 03:31:24 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-4d7abd60-3c0e-4be4-a5be-3af6bcac3c17 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7f8a506279a0>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:43e92d4a-7e98-4777-8fa5-3cec396b86ce'): ({'sdb'}, set())}) {{(pid=75234) _get_connection_devices /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:863}}
Nov 29 03:31:24 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-4d7abd60-3c0e-4be4-a5be-3af6bcac3c17 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] Removing single pathed devices sdb {{(pid=75234) remove_connection /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:374}}
Nov 29 03:31:24 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:31:24 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.019s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:31:24 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[afa7954a-60b9-4cdb-8b0b-5dcf995e9306]: (4, ('path checker states:\nup                  3\n\npaths: 0\nbusy: False\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:31:24 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdb {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:31:24 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdb" returned: 0 in 0.020s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:31:24 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[5fa04e2a-5960-4c36-b2c2-3d0ac4548770]: (4, ('ok\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:31:24 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-4d7abd60-3c0e-4be4-a5be-3af6bcac3c17 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] Flushing IO for device /dev/sdb {{(pid=75234) flush_device_io /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:434}}
Nov 29 03:31:24 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdb {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:31:24 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdb" returned: 0 in 0.015s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:31:24 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[0b3f6a8c-c059-4dec-821d-ff4e6f349821]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:31:24 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-4d7abd60-3c0e-4be4-a5be-3af6bcac3c17 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] Remove SCSI device /dev/sdb with /sys/block/sdb/device/delete {{(pid=75234) remove_scsi_device /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:148}}
Nov 29 03:31:24 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdb/device/delete {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:31:24 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdb/device/delete" returned: 0 in 0.056s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:31:24 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[17ef65b1-a90e-4ec0-b955-c5a752fedafe]: (4, ('1', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:31:24 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-4d7abd60-3c0e-4be4-a5be-3af6bcac3c17 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] Checking to see if SCSI volumes sdb have been removed. {{(pid=75234) wait_for_volumes_removal /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:156}}
Nov 29 03:31:24 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-4d7abd60-3c0e-4be4-a5be-3af6bcac3c17 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] SCSI volumes sdb have been removed. {{(pid=75234) wait_for_volumes_removal /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:166}}
Nov 29 03:31:24 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-4d7abd60-3c0e-4be4-a5be-3af6bcac3c17 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:43e92d4a-7e98-4777-8fa5-3cec396b86ce')] {{(pid=75234) _disconnect_connection /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Nov 29 03:31:24 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:43e92d4a-7e98-4777-8fa5-3cec396b86ce -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:31:24 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:43e92d4a-7e98-4777-8fa5-3cec396b86ce -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.021s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:31:24 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[662071e7-8deb-4779-add5-248cf8474b5d]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:31:24 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-4d7abd60-3c0e-4be4-a5be-3af6bcac3c17 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:31:24 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:43e92d4a-7e98-4777-8fa5-3cec396b86ce -p 172.16.0.220:3260 --logout {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:31:24 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:43e92d4a-7e98-4777-8fa5-3cec396b86ce -p 172.16.0.220:3260 --logout" returned: 0 in 0.058s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:31:24 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[21e05225-b779-427f-9b91-747e845a394f]: (4, ('Logging out of session [sid: 50, target: iqn.2010-10.org.openstack:43e92d4a-7e98-4777-8fa5-3cec396b86ce, portal: 172.16.0.220,3260]\nLogout of [sid: 50, target: iqn.2010-10.org.openstack:43e92d4a-7e98-4777-8fa5-3cec396b86ce, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:31:24 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-4d7abd60-3c0e-4be4-a5be-3af6bcac3c17 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 50, target: iqn.2010-10.org.openstack:43e92d4a-7e98-4777-8fa5-3cec396b86ce, portal: 172.16.0.220,3260]
Nov 29 03:31:24 user nova-compute[75234]: Logout of [sid: 50, target: iqn.2010-10.org.openstack:43e92d4a-7e98-4777-8fa5-3cec396b86ce, portal: 172.16.0.220,3260] successful.
Nov 29 03:31:24 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:31:24 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:43e92d4a-7e98-4777-8fa5-3cec396b86ce -p 172.16.0.220:3260 --op delete {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:31:24 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:43e92d4a-7e98-4777-8fa5-3cec396b86ce -p 172.16.0.220:3260 --op delete" returned: 0 in 0.025s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:31:24 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[79c9fde8-b274-43e2-8700-272c5bb681e1]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:31:24 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-4d7abd60-3c0e-4be4-a5be-3af6bcac3c17 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:31:24 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-4d7abd60-3c0e-4be4-a5be-3af6bcac3c17 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.336s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:87}}
Nov 29 03:31:24 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-4d7abd60-3c0e-4be4-a5be-3af6bcac3c17 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] <== disconnect_volume: return (337ms) None {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:203}}
Nov 29 03:31:24 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-4d7abd60-3c0e-4be4-a5be-3af6bcac3c17 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] [instance: 01d62e54-9957-40b1-8dfe-1783b64ffabf] Disconnected iSCSI Volume {{(pid=75234) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:79}}
Nov 29 03:31:24 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-4d7abd60-3c0e-4be4-a5be-3af6bcac3c17 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] [instance: 01d62e54-9957-40b1-8dfe-1783b64ffabf] Deleting instance files /opt/stack/data/nova/instances/01d62e54-9957-40b1-8dfe-1783b64ffabf_del
Nov 29 03:31:24 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-4d7abd60-3c0e-4be4-a5be-3af6bcac3c17 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] [instance: 01d62e54-9957-40b1-8dfe-1783b64ffabf] Deletion of /opt/stack/data/nova/instances/01d62e54-9957-40b1-8dfe-1783b64ffabf_del complete
Nov 29 03:31:25 user nova-compute[75234]: DEBUG nova.network.neutron [req-34eb9f87-ff3a-4ccb-9570-b2f06486f18c req-db1f2b52-09d3-4f0f-be97-6e65cfabecc5 service nova] [instance: 01d62e54-9957-40b1-8dfe-1783b64ffabf] Updated VIF entry in instance network info cache for port 6dbd7c27-31c9-4ddc-a81b-85f1e742d160. {{(pid=75234) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3481}}
Nov 29 03:31:25 user nova-compute[75234]: DEBUG nova.network.neutron [req-34eb9f87-ff3a-4ccb-9570-b2f06486f18c req-db1f2b52-09d3-4f0f-be97-6e65cfabecc5 service nova] [instance: 01d62e54-9957-40b1-8dfe-1783b64ffabf] Updating instance_info_cache with network_info: [{"id": "6dbd7c27-31c9-4ddc-a81b-85f1e742d160", "address": "fa:16:3e:57:6a:26", "network": {"id": "504616f5-0d6b-47a6-aaf5-663983428b41", "bridge": null, "label": "tempest-TestShelveInstance-438404840-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.210", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "53e55fcb8cd74895baec6f9528770bbc", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "unbound", "details": {}, "devname": "tap6dbd7c27-31", "ovs_interfaceid": null, "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:31:25 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-34eb9f87-ff3a-4ccb-9570-b2f06486f18c req-db1f2b52-09d3-4f0f-be97-6e65cfabecc5 service nova] Releasing lock "refresh_cache-01d62e54-9957-40b1-8dfe-1783b64ffabf" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:31:25 user nova-compute[75234]: DEBUG nova.compute.manager [req-0da89ad8-0a38-4830-812a-b58518b11922 req-4430a8a3-eccc-4ccd-9c3b-565529514af4 service nova] [instance: 01d62e54-9957-40b1-8dfe-1783b64ffabf] Received event network-vif-plugged-6dbd7c27-31c9-4ddc-a81b-85f1e742d160 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:31:25 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-0da89ad8-0a38-4830-812a-b58518b11922 req-4430a8a3-eccc-4ccd-9c3b-565529514af4 service nova] Acquiring lock "01d62e54-9957-40b1-8dfe-1783b64ffabf-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:31:25 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-0da89ad8-0a38-4830-812a-b58518b11922 req-4430a8a3-eccc-4ccd-9c3b-565529514af4 service nova] Lock "01d62e54-9957-40b1-8dfe-1783b64ffabf-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:31:25 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-0da89ad8-0a38-4830-812a-b58518b11922 req-4430a8a3-eccc-4ccd-9c3b-565529514af4 service nova] Lock "01d62e54-9957-40b1-8dfe-1783b64ffabf-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:31:25 user nova-compute[75234]: DEBUG nova.compute.manager [req-0da89ad8-0a38-4830-812a-b58518b11922 req-4430a8a3-eccc-4ccd-9c3b-565529514af4 service nova] [instance: 01d62e54-9957-40b1-8dfe-1783b64ffabf] No waiting events found dispatching network-vif-plugged-6dbd7c27-31c9-4ddc-a81b-85f1e742d160 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:31:25 user nova-compute[75234]: WARNING nova.compute.manager [req-0da89ad8-0a38-4830-812a-b58518b11922 req-4430a8a3-eccc-4ccd-9c3b-565529514af4 service nova] [instance: 01d62e54-9957-40b1-8dfe-1783b64ffabf] Received unexpected event network-vif-plugged-6dbd7c27-31c9-4ddc-a81b-85f1e742d160 for instance with vm_state active and task_state shelving.
Nov 29 03:31:25 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-cf508c25-d18c-404c-ab8d-b75800bbd752 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Acquiring lock "1e9263fc-5735-420f-a116-8f60c8736faf" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:31:25 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-cf508c25-d18c-404c-ab8d-b75800bbd752 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Lock "1e9263fc-5735-420f-a116-8f60c8736faf" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:31:25 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-cf508c25-d18c-404c-ab8d-b75800bbd752 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Acquiring lock "1e9263fc-5735-420f-a116-8f60c8736faf-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:31:25 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-cf508c25-d18c-404c-ab8d-b75800bbd752 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Lock "1e9263fc-5735-420f-a116-8f60c8736faf-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:31:25 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-cf508c25-d18c-404c-ab8d-b75800bbd752 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Lock "1e9263fc-5735-420f-a116-8f60c8736faf-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:31:25 user nova-compute[75234]: INFO nova.compute.manager [None req-cf508c25-d18c-404c-ab8d-b75800bbd752 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: 1e9263fc-5735-420f-a116-8f60c8736faf] Terminating instance
Nov 29 03:31:25 user nova-compute[75234]: DEBUG nova.compute.manager [None req-cf508c25-d18c-404c-ab8d-b75800bbd752 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: 1e9263fc-5735-420f-a116-8f60c8736faf] Start destroying the instance on the hypervisor. {{(pid=75234) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3140}}
Nov 29 03:31:25 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:31:25 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:31:25 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:31:25 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:31:26 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-4c39de24-7e12-41de-a2da-7c56dc78ca87 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] [instance: abb01889-df4b-44a6-9827-87056d7ca125] Snapshot image upload complete
Nov 29 03:31:26 user nova-compute[75234]: INFO nova.compute.manager [None req-4c39de24-7e12-41de-a2da-7c56dc78ca87 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] [instance: abb01889-df4b-44a6-9827-87056d7ca125] Took 6.06 seconds to snapshot the instance on the hypervisor.
Nov 29 03:31:26 user nova-compute[75234]: INFO nova.scheduler.client.report [None req-4d7abd60-3c0e-4be4-a5be-3af6bcac3c17 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] Deleted allocations for instance 01d62e54-9957-40b1-8dfe-1783b64ffabf
Nov 29 03:31:26 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-4d7abd60-3c0e-4be4-a5be-3af6bcac3c17 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:31:26 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-4d7abd60-3c0e-4be4-a5be-3af6bcac3c17 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:31:26 user nova-compute[75234]: DEBUG nova.objects.instance [None req-4d7abd60-3c0e-4be4-a5be-3af6bcac3c17 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] Lazy-loading 'numa_topology' on Instance uuid 01d62e54-9957-40b1-8dfe-1783b64ffabf {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:31:26 user nova-compute[75234]: DEBUG nova.objects.base [None req-4d7abd60-3c0e-4be4-a5be-3af6bcac3c17 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] Object Instance<01d62e54-9957-40b1-8dfe-1783b64ffabf> lazy-loaded attributes: resources,numa_topology {{(pid=75234) wrapper /opt/stack/nova/nova/objects/base.py:126}}
Nov 29 03:31:26 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:31:26 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:31:26 user nova-compute[75234]: INFO nova.virt.libvirt.driver [-] [instance: 1e9263fc-5735-420f-a116-8f60c8736faf] Instance destroyed successfully.
Nov 29 03:31:26 user nova-compute[75234]: DEBUG nova.objects.instance [None req-cf508c25-d18c-404c-ab8d-b75800bbd752 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Lazy-loading 'resources' on Instance uuid 1e9263fc-5735-420f-a116-8f60c8736faf {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:31:26 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-cf508c25-d18c-404c-ab8d-b75800bbd752 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-11-29T03:29:34Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-TestVolumeBootPattern-server-180638926',display_name='tempest-TestVolumeBootPattern-server-180638926',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-testvolumebootpattern-server-180638926',id=45,image_ref='',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data=None,key_name=None,keypairs=<?>,launch_index=0,launched_at=2023-11-29T03:29:56Z,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='81a265d3c15b4e9d826b35ddbd0ff443',ramdisk_id='',reservation_id='r-or8ryn80',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-TestVolumeBootPattern-1244492388',owner_user_name='tempest-TestVolumeBootPattern-1244492388-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-11-29T03:29:56Z,user_data=None,user_id='7d1f1f11a9aa4c1db8627413eb963fa4',uuid=1e9263fc-5735-420f-a116-8f60c8736faf,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "f836309d-8f97-4521-aa57-d6516a9e7528", "address": "fa:16:3e:1e:0c:76", "network": {"id": "a3f397d3-fd75-4a6e-a815-cfcedcca15fb", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-678570085-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "81a265d3c15b4e9d826b35ddbd0ff443", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapf836309d-8f", "ovs_interfaceid": "f836309d-8f97-4521-aa57-d6516a9e7528", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Nov 29 03:31:26 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-cf508c25-d18c-404c-ab8d-b75800bbd752 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Converting VIF {"id": "f836309d-8f97-4521-aa57-d6516a9e7528", "address": "fa:16:3e:1e:0c:76", "network": {"id": "a3f397d3-fd75-4a6e-a815-cfcedcca15fb", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-678570085-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.10", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "81a265d3c15b4e9d826b35ddbd0ff443", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapf836309d-8f", "ovs_interfaceid": "f836309d-8f97-4521-aa57-d6516a9e7528", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:31:26 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-cf508c25-d18c-404c-ab8d-b75800bbd752 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:1e:0c:76,bridge_name='br-int',has_traffic_filtering=True,id=f836309d-8f97-4521-aa57-d6516a9e7528,network=Network(a3f397d3-fd75-4a6e-a815-cfcedcca15fb),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapf836309d-8f') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:31:26 user nova-compute[75234]: DEBUG os_vif [None req-cf508c25-d18c-404c-ab8d-b75800bbd752 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Unplugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:1e:0c:76,bridge_name='br-int',has_traffic_filtering=True,id=f836309d-8f97-4521-aa57-d6516a9e7528,network=Network(a3f397d3-fd75-4a6e-a815-cfcedcca15fb),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapf836309d-8f') {{(pid=75234) unplug /opt/stack/data/venv/lib/python3.10/site-packages/os_vif/__init__.py:109}}
Nov 29 03:31:26 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:31:26 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tapf836309d-8f, bridge=br-int, if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:31:26 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:31:26 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:31:26 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:31:26 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbDestroyCommand(_result=None, table=QoS, record=355af910-5588-4b9b-aecf-a05b8f27ead1) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:31:26 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:31:26 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:31:26 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:31:26 user nova-compute[75234]: INFO os_vif [None req-cf508c25-d18c-404c-ab8d-b75800bbd752 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Successfully unplugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:1e:0c:76,bridge_name='br-int',has_traffic_filtering=True,id=f836309d-8f97-4521-aa57-d6516a9e7528,network=Network(a3f397d3-fd75-4a6e-a815-cfcedcca15fb),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapf836309d-8f')
Nov 29 03:31:26 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-cf508c25-d18c-404c-ab8d-b75800bbd752 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: 1e9263fc-5735-420f-a116-8f60c8736faf] calling os-brick to detach iSCSI Volume {{(pid=75234) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Nov 29 03:31:26 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-cf508c25-d18c-404c-ab8d-b75800bbd752 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f8a5050c9a0>, {'target_iqn': 'iqn.2010-10.org.openstack:d1f13879-6e1b-4fe3-b97d-066a9e657ef7', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'VpKRqEQa', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdd'}, None), 'kwargs': True}}" {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:176}}
Nov 29 03:31:26 user nova-compute[75234]: WARNING os_brick.initiator.connectors.base [None req-cf508c25-d18c-404c-ab8d-b75800bbd752 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Nov 29 03:31:26 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-cf508c25-d18c-404c-ab8d-b75800bbd752 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:68}}
Nov 29 03:31:26 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-cf508c25-d18c-404c-ab8d-b75800bbd752 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:73}}
Nov 29 03:31:26 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:31:26 user nova-compute[75234]: DEBUG nova.compute.provider_tree [None req-4d7abd60-3c0e-4be4-a5be-3af6bcac3c17 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] Inventory has not changed in ProviderTree for provider: 4d291220-7c37-4839-a0e3-b23c66502b74 {{(pid=75234) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Nov 29 03:31:26 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.018s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:31:26 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[8209cd70-72ce-4a98-b5d6-b4dbfa4dfcaf]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:d1f13879-6e1b-4fe3-b97d-066a9e657ef7\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:31:26 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-cf508c25-d18c-404c-ab8d-b75800bbd752 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS:
Nov 29 03:31:26 user nova-compute[75234]: No targets found.
Nov 29 03:31:26 user nova-compute[75234]: iSNS:
Nov 29 03:31:26 user nova-compute[75234]: No targets found.
Nov 29 03:31:26 user nova-compute[75234]: STATIC:
Nov 29 03:31:26 user nova-compute[75234]: Target: iqn.2010-10.org.openstack:d1f13879-6e1b-4fe3-b97d-066a9e657ef7
Nov 29 03:31:26 user nova-compute[75234]:         Portal: 172.16.0.220:3260,-1
Nov 29 03:31:26 user nova-compute[75234]:                 Iface Name: default
Nov 29 03:31:26 user nova-compute[75234]: Target: iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0
Nov 29 03:31:26 user nova-compute[75234]:         Portal: 172.16.0.220:3260,-1
Nov 29 03:31:26 user nova-compute[75234]:                 Iface Name: default
Nov 29 03:31:26 user nova-compute[75234]: FIRMWARE:
Nov 29 03:31:26 user nova-compute[75234]: No targets found.
Nov 29 03:31:26 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:31:26 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-cf508c25-d18c-404c-ab8d-b75800bbd752 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Regex to get portals from discoverydb: ^SENDTARGETS:
Nov 29 03:31:26 user nova-compute[75234]: .*?^DiscoveryAddress: 172.16.0.220,3260.*?
Nov 29 03:31:26 user nova-compute[75234]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=75234) _get_discoverydb_portals /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:382}}
Nov 29 03:31:26 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-cf508c25-d18c-404c-ab8d-b75800bbd752 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:d1f13879-6e1b-4fe3-b97d-066a9e657ef7', 0)] {{(pid=75234) _get_connection_devices /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:830}}
Nov 29 03:31:26 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:31:26 user nova-compute[75234]: DEBUG nova.scheduler.client.report [None req-4d7abd60-3c0e-4be4-a5be-3af6bcac3c17 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] Inventory has not changed for provider 4d291220-7c37-4839-a0e3-b23c66502b74 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=75234) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Nov 29 03:31:26 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.022s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:31:26 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[407a392d-7320-4e3a-8884-0144f876a2ec]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:d1f13879-6e1b-4fe3-b97d-066a9e657ef7\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:31:26 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:31:26 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-4d7abd60-3c0e-4be4-a5be-3af6bcac3c17 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.308s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:31:26 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.026s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:31:26 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[9d0b2e4e-d9b9-4a30-aa99-1d77d1c288c5]: (4, ('tcp: [42] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0 (non-flash)\ntcp: [51] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d1f13879-6e1b-4fe3-b97d-066a9e657ef7 (non-flash)\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:31:26 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-cf508c25-d18c-404c-ab8d-b75800bbd752 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [42] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0 (non-flash)
Nov 29 03:31:26 user nova-compute[75234]: tcp: [51] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d1f13879-6e1b-4fe3-b97d-066a9e657ef7 (non-flash)
Nov 29 03:31:26 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:31:26 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-cf508c25-d18c-404c-ab8d-b75800bbd752 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] iscsi session list stdout=tcp: [42] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0 (non-flash)
Nov 29 03:31:26 user nova-compute[75234]: tcp: [51] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d1f13879-6e1b-4fe3-b97d-066a9e657ef7 (non-flash)
Nov 29 03:31:26 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsi_session /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Nov 29 03:31:26 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-cf508c25-d18c-404c-ab8d-b75800bbd752 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7f8a2c6de830>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:d1f13879-6e1b-4fe3-b97d-066a9e657ef7'): ({'sdd'}, set())}) {{(pid=75234) _get_connection_devices /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:863}}
Nov 29 03:31:26 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-cf508c25-d18c-404c-ab8d-b75800bbd752 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Removing single pathed devices sdd {{(pid=75234) remove_connection /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:374}}
Nov 29 03:31:26 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:31:26 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.021s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:31:26 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[970eeed0-5cb7-458c-9623-a3ab83aa07ea]: (4, ('path checker states:\nup                  2\n\npaths: 0\nbusy: False\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:31:26 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdd {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:31:26 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdd" returned: 0 in 0.021s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:31:26 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[ee0b277b-4e67-4a9d-baaa-b908c7029612]: (4, ('ok\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:31:26 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-cf508c25-d18c-404c-ab8d-b75800bbd752 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Flushing IO for device /dev/sdd {{(pid=75234) flush_device_io /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:434}}
Nov 29 03:31:26 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-4d7abd60-3c0e-4be4-a5be-3af6bcac3c17 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] Lock "01d62e54-9957-40b1-8dfe-1783b64ffabf" "released" by "nova.compute.manager.ComputeManager.shelve_offload_instance.<locals>.do_shelve_offload_instance" :: held 3.885s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:31:26 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdd {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:31:26 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdd" returned: 0 in 0.018s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:31:26 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[12744869-64bd-4930-9f91-b8fc448fdcc0]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:31:26 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-cf508c25-d18c-404c-ab8d-b75800bbd752 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Remove SCSI device /dev/sdd with /sys/block/sdd/device/delete {{(pid=75234) remove_scsi_device /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:148}}
Nov 29 03:31:26 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdd/device/delete {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:31:26 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdd/device/delete" returned: 0 in 0.052s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:31:26 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[47a6a277-3e60-451d-a512-5325f86ba834]: (4, ('1', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:31:26 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-cf508c25-d18c-404c-ab8d-b75800bbd752 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Checking to see if SCSI volumes sdd have been removed. {{(pid=75234) wait_for_volumes_removal /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:156}}
Nov 29 03:31:26 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-cf508c25-d18c-404c-ab8d-b75800bbd752 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] SCSI volumes sdd have been removed. {{(pid=75234) wait_for_volumes_removal /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:166}}
Nov 29 03:31:26 user nova-compute[75234]: DEBUG os_brick.privileged.rootwrap [-] Unlinking ('/dev/disk/by-id/scsi-1SCST_BIOdbc3ecac-d11a714a71cf6fd6', '/dev/disk/by-id/scsi-26431316137313461', '/dev/disk/by-id/scsi-SSCST_BIO_Storage_dbc3ecac') {{(pid=86324) unlink_root /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/privileged/rootwrap.py:219}}
Nov 29 03:31:26 user nova-compute[75234]: WARNING os_brick.exception [-] Unlink failed for /dev/disk/by-id/scsi-1SCST_BIOdbc3ecac-d11a714a71cf6fd6
Nov 29 03:31:26 user nova-compute[75234]: WARNING os_brick.exception [-] Unlink failed for /dev/disk/by-id/scsi-26431316137313461
Nov 29 03:31:26 user nova-compute[75234]: WARNING os_brick.exception [-] Unlink failed for /dev/disk/by-id/scsi-SSCST_BIO_Storage_dbc3ecac
Nov 29 03:31:26 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[7e446e47-d61b-40e5-be5d-183f2a997afc]: (4, None) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:31:26 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-cf508c25-d18c-404c-ab8d-b75800bbd752 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:d1f13879-6e1b-4fe3-b97d-066a9e657ef7')] {{(pid=75234) _disconnect_connection /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Nov 29 03:31:26 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:d1f13879-6e1b-4fe3-b97d-066a9e657ef7 -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:31:26 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:d1f13879-6e1b-4fe3-b97d-066a9e657ef7 -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.024s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:31:26 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[af8e73a3-985d-4f76-8054-461484ac707a]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:31:26 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-cf508c25-d18c-404c-ab8d-b75800bbd752 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:31:26 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:d1f13879-6e1b-4fe3-b97d-066a9e657ef7 -p 172.16.0.220:3260 --logout {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:31:26 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:d1f13879-6e1b-4fe3-b97d-066a9e657ef7 -p 172.16.0.220:3260 --logout" returned: 0 in 0.040s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:31:26 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[add891e7-7295-4436-be29-9996e1eed30f]: (4, ('Logging out of session [sid: 51, target: iqn.2010-10.org.openstack:d1f13879-6e1b-4fe3-b97d-066a9e657ef7, portal: 172.16.0.220,3260]\nLogout of [sid: 51, target: iqn.2010-10.org.openstack:d1f13879-6e1b-4fe3-b97d-066a9e657ef7, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:31:26 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-cf508c25-d18c-404c-ab8d-b75800bbd752 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 51, target: iqn.2010-10.org.openstack:d1f13879-6e1b-4fe3-b97d-066a9e657ef7, portal: 172.16.0.220,3260]
Nov 29 03:31:26 user nova-compute[75234]: Logout of [sid: 51, target: iqn.2010-10.org.openstack:d1f13879-6e1b-4fe3-b97d-066a9e657ef7, portal: 172.16.0.220,3260] successful.
Nov 29 03:31:26 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:31:26 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:d1f13879-6e1b-4fe3-b97d-066a9e657ef7 -p 172.16.0.220:3260 --op delete {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:31:26 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:d1f13879-6e1b-4fe3-b97d-066a9e657ef7 -p 172.16.0.220:3260 --op delete" returned: 0 in 0.020s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:31:26 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[62e3356d-2630-42a6-aea2-a0624cb9f731]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:31:26 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-cf508c25-d18c-404c-ab8d-b75800bbd752 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:31:26 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-cf508c25-d18c-404c-ab8d-b75800bbd752 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.328s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:87}}
Nov 29 03:31:26 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-cf508c25-d18c-404c-ab8d-b75800bbd752 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] <== disconnect_volume: return (330ms) None {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:203}}
Nov 29 03:31:26 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-cf508c25-d18c-404c-ab8d-b75800bbd752 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: 1e9263fc-5735-420f-a116-8f60c8736faf] Disconnected iSCSI Volume {{(pid=75234) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:79}}
Nov 29 03:31:26 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-cf508c25-d18c-404c-ab8d-b75800bbd752 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: 1e9263fc-5735-420f-a116-8f60c8736faf] Deleting instance files /opt/stack/data/nova/instances/1e9263fc-5735-420f-a116-8f60c8736faf_del
Nov 29 03:31:26 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-cf508c25-d18c-404c-ab8d-b75800bbd752 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: 1e9263fc-5735-420f-a116-8f60c8736faf] Deletion of /opt/stack/data/nova/instances/1e9263fc-5735-420f-a116-8f60c8736faf_del complete
Nov 29 03:31:26 user nova-compute[75234]: INFO nova.compute.manager [None req-cf508c25-d18c-404c-ab8d-b75800bbd752 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: 1e9263fc-5735-420f-a116-8f60c8736faf] Took 1.25 seconds to destroy the instance on the hypervisor.
Nov 29 03:31:26 user nova-compute[75234]: DEBUG oslo.service.loopingcall [None req-cf508c25-d18c-404c-ab8d-b75800bbd752 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=75234) func /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/loopingcall.py:435}}
Nov 29 03:31:26 user nova-compute[75234]: DEBUG nova.compute.manager [-] [instance: 1e9263fc-5735-420f-a116-8f60c8736faf] Deallocating network for instance {{(pid=75234) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2279}}
Nov 29 03:31:26 user nova-compute[75234]: DEBUG nova.network.neutron [-] [instance: 1e9263fc-5735-420f-a116-8f60c8736faf] deallocate_for_instance() {{(pid=75234) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1802}}
Nov 29 03:31:27 user nova-compute[75234]: DEBUG nova.compute.manager [req-e3b528f6-7ed0-409c-b0e6-7692f7a358b8 req-b2b9732e-2fb4-4ebd-9e5b-5c8bb7694791 service nova] [instance: 1e9263fc-5735-420f-a116-8f60c8736faf] Received event network-vif-unplugged-f836309d-8f97-4521-aa57-d6516a9e7528 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:31:27 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-e3b528f6-7ed0-409c-b0e6-7692f7a358b8 req-b2b9732e-2fb4-4ebd-9e5b-5c8bb7694791 service nova] Acquiring lock "1e9263fc-5735-420f-a116-8f60c8736faf-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:31:27 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-e3b528f6-7ed0-409c-b0e6-7692f7a358b8 req-b2b9732e-2fb4-4ebd-9e5b-5c8bb7694791 service nova] Lock "1e9263fc-5735-420f-a116-8f60c8736faf-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:31:27 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-e3b528f6-7ed0-409c-b0e6-7692f7a358b8 req-b2b9732e-2fb4-4ebd-9e5b-5c8bb7694791 service nova] Lock "1e9263fc-5735-420f-a116-8f60c8736faf-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:31:27 user nova-compute[75234]: DEBUG nova.compute.manager [req-e3b528f6-7ed0-409c-b0e6-7692f7a358b8 req-b2b9732e-2fb4-4ebd-9e5b-5c8bb7694791 service nova] [instance: 1e9263fc-5735-420f-a116-8f60c8736faf] No waiting events found dispatching network-vif-unplugged-f836309d-8f97-4521-aa57-d6516a9e7528 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:31:27 user nova-compute[75234]: DEBUG nova.compute.manager [req-e3b528f6-7ed0-409c-b0e6-7692f7a358b8 req-b2b9732e-2fb4-4ebd-9e5b-5c8bb7694791 service nova] [instance: 1e9263fc-5735-420f-a116-8f60c8736faf] Received event network-vif-unplugged-f836309d-8f97-4521-aa57-d6516a9e7528 for instance with task_state deleting. {{(pid=75234) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10890}}
Nov 29 03:31:27 user nova-compute[75234]: DEBUG nova.compute.manager [req-e3b528f6-7ed0-409c-b0e6-7692f7a358b8 req-b2b9732e-2fb4-4ebd-9e5b-5c8bb7694791 service nova] [instance: 1e9263fc-5735-420f-a116-8f60c8736faf] Received event network-vif-plugged-f836309d-8f97-4521-aa57-d6516a9e7528 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:31:27 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-e3b528f6-7ed0-409c-b0e6-7692f7a358b8 req-b2b9732e-2fb4-4ebd-9e5b-5c8bb7694791 service nova] Acquiring lock "1e9263fc-5735-420f-a116-8f60c8736faf-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:31:27 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-e3b528f6-7ed0-409c-b0e6-7692f7a358b8 req-b2b9732e-2fb4-4ebd-9e5b-5c8bb7694791 service nova] Lock "1e9263fc-5735-420f-a116-8f60c8736faf-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:31:27 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-e3b528f6-7ed0-409c-b0e6-7692f7a358b8 req-b2b9732e-2fb4-4ebd-9e5b-5c8bb7694791 service nova] Lock "1e9263fc-5735-420f-a116-8f60c8736faf-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:31:27 user nova-compute[75234]: DEBUG nova.compute.manager [req-e3b528f6-7ed0-409c-b0e6-7692f7a358b8 req-b2b9732e-2fb4-4ebd-9e5b-5c8bb7694791 service nova] [instance: 1e9263fc-5735-420f-a116-8f60c8736faf] No waiting events found dispatching network-vif-plugged-f836309d-8f97-4521-aa57-d6516a9e7528 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:31:27 user nova-compute[75234]: WARNING nova.compute.manager [req-e3b528f6-7ed0-409c-b0e6-7692f7a358b8 req-b2b9732e-2fb4-4ebd-9e5b-5c8bb7694791 service nova] [instance: 1e9263fc-5735-420f-a116-8f60c8736faf] Received unexpected event network-vif-plugged-f836309d-8f97-4521-aa57-d6516a9e7528 for instance with vm_state active and task_state deleting.
Nov 29 03:31:27 user nova-compute[75234]: DEBUG nova.network.neutron [-] [instance: 1e9263fc-5735-420f-a116-8f60c8736faf] Updating instance_info_cache with network_info: [] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:31:27 user nova-compute[75234]: INFO nova.compute.manager [-] [instance: 1e9263fc-5735-420f-a116-8f60c8736faf] Took 0.42 seconds to deallocate network for instance.
Nov 29 03:31:27 user nova-compute[75234]: DEBUG nova.compute.manager [req-e0592aee-acd0-41cc-b5de-942327c5de1f req-39c89117-c69a-4358-9155-adead476e28d service nova] [instance: 1e9263fc-5735-420f-a116-8f60c8736faf] Received event network-vif-deleted-f836309d-8f97-4521-aa57-d6516a9e7528 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:31:27 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:31:28 user nova-compute[75234]: INFO nova.compute.manager [None req-cf508c25-d18c-404c-ab8d-b75800bbd752 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: 1e9263fc-5735-420f-a116-8f60c8736faf] Took 0.75 seconds to detach 1 volumes for instance.
Nov 29 03:31:28 user nova-compute[75234]: DEBUG nova.compute.manager [None req-cf508c25-d18c-404c-ab8d-b75800bbd752 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: 1e9263fc-5735-420f-a116-8f60c8736faf] Deleting volume: d1f13879-6e1b-4fe3-b97d-066a9e657ef7 {{(pid=75234) _cleanup_volumes /opt/stack/nova/nova/compute/manager.py:3237}}
Nov 29 03:31:28 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-cf508c25-d18c-404c-ab8d-b75800bbd752 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:31:28 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-cf508c25-d18c-404c-ab8d-b75800bbd752 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:31:28 user nova-compute[75234]: DEBUG nova.compute.provider_tree [None req-cf508c25-d18c-404c-ab8d-b75800bbd752 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Inventory has not changed in ProviderTree for provider: 4d291220-7c37-4839-a0e3-b23c66502b74 {{(pid=75234) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Nov 29 03:31:28 user nova-compute[75234]: DEBUG nova.scheduler.client.report [None req-cf508c25-d18c-404c-ab8d-b75800bbd752 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Inventory has not changed for provider 4d291220-7c37-4839-a0e3-b23c66502b74 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=75234) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Nov 29 03:31:28 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-cf508c25-d18c-404c-ab8d-b75800bbd752 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.231s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:31:28 user nova-compute[75234]: INFO nova.scheduler.client.report [None req-cf508c25-d18c-404c-ab8d-b75800bbd752 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Deleted allocations for instance 1e9263fc-5735-420f-a116-8f60c8736faf
Nov 29 03:31:28 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-cf508c25-d18c-404c-ab8d-b75800bbd752 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Lock "1e9263fc-5735-420f-a116-8f60c8736faf" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 2.920s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:31:31 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:31:32 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:31:36 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:31:37 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:31:40 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-6b0cedbf-3dda-45ea-bf77-296e8a582851 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Acquiring lock "cb0f042e-43c0-43aa-b243-cfc78a7b2695" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:31:40 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-6b0cedbf-3dda-45ea-bf77-296e8a582851 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Lock "cb0f042e-43c0-43aa-b243-cfc78a7b2695" acquired by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:31:40 user nova-compute[75234]: DEBUG nova.objects.instance [None req-6b0cedbf-3dda-45ea-bf77-296e8a582851 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Lazy-loading 'flavor' on Instance uuid cb0f042e-43c0-43aa-b243-cfc78a7b2695 {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:31:40 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-6b0cedbf-3dda-45ea-bf77-296e8a582851 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Lock "cb0f042e-43c0-43aa-b243-cfc78a7b2695" "released" by "nova.compute.manager.ComputeManager.reserve_block_device_name.<locals>.do_reserve" :: held 0.057s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:31:40 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-6b0cedbf-3dda-45ea-bf77-296e8a582851 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Acquiring lock "cb0f042e-43c0-43aa-b243-cfc78a7b2695" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:31:40 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-6b0cedbf-3dda-45ea-bf77-296e8a582851 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Lock "cb0f042e-43c0-43aa-b243-cfc78a7b2695" acquired by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:31:40 user nova-compute[75234]: INFO nova.compute.manager [None req-6b0cedbf-3dda-45ea-bf77-296e8a582851 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] [instance: cb0f042e-43c0-43aa-b243-cfc78a7b2695] Attaching volume d7d89638-2f88-47e2-a43c-b37e8919ecbc to /dev/vdb
Nov 29 03:31:40 user nova-compute[75234]: DEBUG os_brick.utils [None req-6b0cedbf-3dda-45ea-bf77-296e8a582851 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:176}}
Nov 29 03:31:40 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:31:40 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.013s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:31:40 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[659f0b21-e89c-48fe-918d-8a74770838c5]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:dd1368c5ba2c\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:31:40 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt -v / -n -o SOURCE {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:31:40 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt -v / -n -o SOURCE" returned: 0 in 0.016s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:31:40 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[f4569f58-15eb-4194-ab6d-2352a66463ce]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:31:40 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:31:40 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.016s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:31:40 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[c8bf24e3-e873-4fe8-8168-6dbf01ecd0fe]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:31:40 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[0c04705e-c8ed-420a-8233-b398d5094004]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:31:40 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-6b0cedbf-3dda-45ea-bf77-296e8a582851 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Running cmd (subprocess): nvme version {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:31:40 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-6b0cedbf-3dda-45ea-bf77-296e8a582851 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] 'nvme version' failed. Not Retrying. {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:473}}
Nov 29 03:31:40 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.nvmeof [None req-6b0cedbf-3dda-45ea-bf77-296e8a582851 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] nvme not present on system {{(pid=75234) nvme_present /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/nvmeof.py:756}}
Nov 29 03:31:40 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:31:40 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:473}}
Nov 29 03:31:40 user nova-compute[75234]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Nov 29 03:31:40 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[9fa870e5-d139-4976-a3c9-4f2ec0e6f87e]: (4, '') {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:31:40 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.lightos [None req-6b0cedbf-3dda-45ea-bf77-296e8a582851 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=75234) find_dsc /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/lightos.py:98}}
Nov 29 03:31:40 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.lightos [None req-6b0cedbf-3dda-45ea-bf77-296e8a582851 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=75234) get_connector_properties /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/lightos.py:76}}
Nov 29 03:31:40 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.lightos [None req-6b0cedbf-3dda-45ea-bf77-296e8a582851 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] LIGHTOS: no hostnqn found. {{(pid=75234) get_connector_properties /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/lightos.py:84}}
Nov 29 03:31:40 user nova-compute[75234]: DEBUG os_brick.utils [None req-6b0cedbf-3dda-45ea-bf77-296e8a582851 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] <== get_connector_properties: return (120ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:dd1368c5ba2c', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:203}}
Nov 29 03:31:40 user nova-compute[75234]: DEBUG nova.virt.block_device [None req-6b0cedbf-3dda-45ea-bf77-296e8a582851 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] [instance: cb0f042e-43c0-43aa-b243-cfc78a7b2695] Updating existing volume attachment record: 9914382f-4ae7-46da-9c49-6500547969ed {{(pid=75234) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Nov 29 03:31:41 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:31:46 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:31:47 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-6b0cedbf-3dda-45ea-bf77-296e8a582851 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] [instance: cb0f042e-43c0-43aa-b243-cfc78a7b2695] Calling os-brick to attach iSCSI Volume {{(pid=75234) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Nov 29 03:31:47 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-6b0cedbf-3dda-45ea-bf77-296e8a582851 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f8a5050c9a0>, 'connection_properties': {'target_iqn': 'iqn.2010-10.org.openstack:d7d89638-2f88-47e2-a43c-b37e8919ecbc', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'YPIlLaJA', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:176}}
Nov 29 03:31:47 user nova-compute[75234]: WARNING os_brick.initiator.connectors.base [None req-6b0cedbf-3dda-45ea-bf77-296e8a582851 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Nov 29 03:31:47 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-6b0cedbf-3dda-45ea-bf77-296e8a582851 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:68}}
Nov 29 03:31:47 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-6b0cedbf-3dda-45ea-bf77-296e8a582851 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.002s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:73}}
Nov 29 03:31:47 user nova-compute[75234]: WARNING os_brick.initiator.connectors.base [None req-6b0cedbf-3dda-45ea-bf77-296e8a582851 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Nov 29 03:31:47 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-6b0cedbf-3dda-45ea-bf77-296e8a582851 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:d7d89638-2f88-47e2-a43c-b37e8919ecbc" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:68}}
Nov 29 03:31:47 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-6b0cedbf-3dda-45ea-bf77-296e8a582851 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:d7d89638-2f88-47e2-a43c-b37e8919ecbc" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:73}}
Nov 29 03:31:47 user nova-compute[75234]: INFO os_brick.initiator.connectors.iscsi [None req-6b0cedbf-3dda-45ea-bf77-296e8a582851 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Nov 29 03:31:47 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:d7d89638-2f88-47e2-a43c-b37e8919ecbc -p 172.16.0.220:3260 {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:31:47 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:d7d89638-2f88-47e2-a43c-b37e8919ecbc -p 172.16.0.220:3260" returned: 21 in 0.021s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:31:47 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[09319f21-723b-43c8-9696-c8da340406b8]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:31:47 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-6b0cedbf-3dda-45ea-bf77-296e8a582851 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Nov 29 03:31:47 user nova-compute[75234]:  {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:31:47 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:d7d89638-2f88-47e2-a43c-b37e8919ecbc -p 172.16.0.220:3260 --interface default --op new {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:31:47 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:d7d89638-2f88-47e2-a43c-b37e8919ecbc -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.020s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:31:47 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[4116d489-262d-4498-9a66-0a77a15b9a68]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:d7d89638-2f88-47e2-a43c-b37e8919ecbc] added\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:31:47 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-6b0cedbf-3dda-45ea-bf77-296e8a582851 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:d7d89638-2f88-47e2-a43c-b37e8919ecbc] added
Nov 29 03:31:47 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:31:47 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:d7d89638-2f88-47e2-a43c-b37e8919ecbc -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:31:47 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:d7d89638-2f88-47e2-a43c-b37e8919ecbc -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.022s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:31:47 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[1ff241bd-926b-4c0b-90b3-d9dddc4bd084]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:31:47 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-6b0cedbf-3dda-45ea-bf77-296e8a582851 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:31:47 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:d7d89638-2f88-47e2-a43c-b37e8919ecbc -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:31:47 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:d7d89638-2f88-47e2-a43c-b37e8919ecbc -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.022s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:31:47 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[01faad7b-9af5-4b91-9317-22dff0358667]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:31:47 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-6b0cedbf-3dda-45ea-bf77-296e8a582851 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:31:47 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:d7d89638-2f88-47e2-a43c-b37e8919ecbc -p 172.16.0.220:3260 --op update -n node.session.auth.username -v YPIlLaJA {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:31:47 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:d7d89638-2f88-47e2-a43c-b37e8919ecbc -p 172.16.0.220:3260 --op update -n node.session.auth.username -v YPIlLaJA" returned: 0 in 0.021s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:31:47 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[cd62ee7f-6c9b-49d3-bb06-c8031553f339]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:31:47 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-6b0cedbf-3dda-45ea-bf77-296e8a582851 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'YPIlLaJA'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:31:47 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:d7d89638-2f88-47e2-a43c-b37e8919ecbc -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:31:47 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:d7d89638-2f88-47e2-a43c-b37e8919ecbc -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.024s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:31:47 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[6fcc30ae-6766-44c3-90d3-9bd7d7efdf5a]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:31:47 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-6b0cedbf-3dda-45ea-bf77-296e8a582851 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:31:47 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:31:47 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.025s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:31:47 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[ab5a3272-768f-4636-a022-e55933be289a]: (4, ('tcp: [42] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0 (non-flash)\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:31:47 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-6b0cedbf-3dda-45ea-bf77-296e8a582851 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [42] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0 (non-flash)
Nov 29 03:31:47 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:31:47 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-6b0cedbf-3dda-45ea-bf77-296e8a582851 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] iscsi session list stdout=tcp: [42] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0 (non-flash)
Nov 29 03:31:47 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsi_session /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Nov 29 03:31:47 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:d7d89638-2f88-47e2-a43c-b37e8919ecbc -p 172.16.0.220:3260 --login {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:31:47 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:d7d89638-2f88-47e2-a43c-b37e8919ecbc -p 172.16.0.220:3260 --login" returned: 0 in 0.049s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:31:47 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[347a7fd9-68d0-43ff-884f-06835540bdc9]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:d7d89638-2f88-47e2-a43c-b37e8919ecbc, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:d7d89638-2f88-47e2-a43c-b37e8919ecbc, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:31:47 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-6b0cedbf-3dda-45ea-bf77-296e8a582851 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:d7d89638-2f88-47e2-a43c-b37e8919ecbc, portal: 172.16.0.220,3260]
Nov 29 03:31:47 user nova-compute[75234]: Login to [iface: default, target: iqn.2010-10.org.openstack:d7d89638-2f88-47e2-a43c-b37e8919ecbc, portal: 172.16.0.220,3260] successful.
Nov 29 03:31:47 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:31:47 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:d7d89638-2f88-47e2-a43c-b37e8919ecbc -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:31:47 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:d7d89638-2f88-47e2-a43c-b37e8919ecbc -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.018s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:31:47 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[6fb49d2e-e0f0-471f-a6a3-a8e95bd9cb3c]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:31:47 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-6b0cedbf-3dda-45ea-bf77-296e8a582851 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:31:47 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:31:47 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.020s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:31:47 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[99db7975-7b36-4489-a67d-753eb6f0f73c]: (4, ('tcp: [42] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0 (non-flash)\ntcp: [53] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d7d89638-2f88-47e2-a43c-b37e8919ecbc (non-flash)\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:31:47 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-6b0cedbf-3dda-45ea-bf77-296e8a582851 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [42] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0 (non-flash)
Nov 29 03:31:47 user nova-compute[75234]: tcp: [53] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d7d89638-2f88-47e2-a43c-b37e8919ecbc (non-flash)
Nov 29 03:31:47 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:31:47 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-6b0cedbf-3dda-45ea-bf77-296e8a582851 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] iscsi session list stdout=tcp: [42] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0 (non-flash)
Nov 29 03:31:47 user nova-compute[75234]: tcp: [53] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d7d89638-2f88-47e2-a43c-b37e8919ecbc (non-flash)
Nov 29 03:31:47 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsi_session /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Nov 29 03:31:47 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-6b0cedbf-3dda-45ea-bf77-296e8a582851 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:d7d89638-2f88-47e2-a43c-b37e8919ecbc" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.288s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:87}}
Nov 29 03:31:47 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-6b0cedbf-3dda-45ea-bf77-296e8a582851 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Connected to 172.16.0.220:3260 {{(pid=75234) _connect_vol /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:644}}
Nov 29 03:31:47 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-6b0cedbf-3dda-45ea-bf77-296e8a582851 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] HCTL ('33', '-', '-', 0) found on session 53 with lun 0 {{(pid=75234) get_hctl /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:770}}
Nov 29 03:31:47 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-6b0cedbf-3dda-45ea-bf77-296e8a582851 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Scanning host 33 c: -, t: -, l: 0) {{(pid=75234) scan_iscsi /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:795}}
Nov 29 03:31:47 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host33/scan {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:31:47 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host33/scan" returned: 0 in 0.016s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:31:47 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[1af46ddd-511b-4b07-a606-b460c861f8c0]: (4, ('- - 0', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:31:47 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-6b0cedbf-3dda-45ea-bf77-296e8a582851 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Searching for a device in session 53 and hctl ['33', '*', '*', 0] yield: None {{(pid=75234) device_name_by_hctl /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:789}}
Nov 29 03:31:48 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-6b0cedbf-3dda-45ea-bf77-296e8a582851 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Searching for a device in session 53 and hctl ['33', '*', '*', 0] yield: sdb {{(pid=75234) device_name_by_hctl /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:789}}
Nov 29 03:31:48 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-6b0cedbf-3dda-45ea-bf77-296e8a582851 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Connected to sdb using {'target_iqn': 'iqn.2010-10.org.openstack:d7d89638-2f88-47e2-a43c-b37e8919ecbc', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'YPIlLaJA', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False} {{(pid=75234) _connect_vol /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:672}}
Nov 29 03:31:48 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-6b0cedbf-3dda-45ea-bf77-296e8a582851 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.315s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:87}}
Nov 29 03:31:48 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-6b0cedbf-3dda-45ea-bf77-296e8a582851 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] <== connect_volume: return (1317ms) {'type': 'block', 'scsi_wwn': '23465303335653463', 'path': '/dev/sdb'} {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:203}}
Nov 29 03:31:48 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-6b0cedbf-3dda-45ea-bf77-296e8a582851 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] [instance: cb0f042e-43c0-43aa-b243-cfc78a7b2695] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '23465303335653463', 'path': '/dev/sdb'} {{(pid=75234) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Nov 29 03:31:48 user nova-compute[75234]: DEBUG nova.objects.instance [None req-6b0cedbf-3dda-45ea-bf77-296e8a582851 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Lazy-loading 'flavor' on Instance uuid cb0f042e-43c0-43aa-b243-cfc78a7b2695 {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:31:48 user nova-compute[75234]: DEBUG nova.virt.libvirt.guest [None req-6b0cedbf-3dda-45ea-bf77-296e8a582851 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] attach device xml: <disk type="block" device="disk">
Nov 29 03:31:48 user nova-compute[75234]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Nov 29 03:31:48 user nova-compute[75234]:   <source dev="/dev/sdb"/>
Nov 29 03:31:48 user nova-compute[75234]:   <target dev="vdb" bus="virtio"/>
Nov 29 03:31:48 user nova-compute[75234]:   <serial>d7d89638-2f88-47e2-a43c-b37e8919ecbc</serial>
Nov 29 03:31:48 user nova-compute[75234]: </disk>
Nov 29 03:31:48 user nova-compute[75234]:  {{(pid=75234) attach_device /opt/stack/nova/nova/virt/libvirt/guest.py:339}}
Nov 29 03:31:48 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-6b0cedbf-3dda-45ea-bf77-296e8a582851 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] No BDM found with device name vda, not building metadata. {{(pid=75234) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12089}}
Nov 29 03:31:48 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-6b0cedbf-3dda-45ea-bf77-296e8a582851 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] No BDM found with device name vdb, not building metadata. {{(pid=75234) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12089}}
Nov 29 03:31:48 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-6b0cedbf-3dda-45ea-bf77-296e8a582851 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] No VIF found with MAC fa:16:3e:31:19:9f, not building metadata {{(pid=75234) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Nov 29 03:31:49 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-6b0cedbf-3dda-45ea-bf77-296e8a582851 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Lock "cb0f042e-43c0-43aa-b243-cfc78a7b2695" "released" by "nova.compute.manager.ComputeManager.attach_volume.<locals>.do_attach_volume" :: held 8.689s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:31:51 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:31:51 user nova-compute[75234]: DEBUG nova.compute.manager [req-da911b49-b699-4e87-9d72-28cab591c90c req-99f86f1c-3538-40df-b3c2-0280dc6a2bbc service nova] [instance: cb0f042e-43c0-43aa-b243-cfc78a7b2695] Received event network-changed-8f9642c3-9e10-4aa7-b5a2-53bb68ad96dc {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:31:51 user nova-compute[75234]: DEBUG nova.compute.manager [req-da911b49-b699-4e87-9d72-28cab591c90c req-99f86f1c-3538-40df-b3c2-0280dc6a2bbc service nova] [instance: cb0f042e-43c0-43aa-b243-cfc78a7b2695] Refreshing instance network info cache due to event network-changed-8f9642c3-9e10-4aa7-b5a2-53bb68ad96dc. {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11117}}
Nov 29 03:31:51 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-da911b49-b699-4e87-9d72-28cab591c90c req-99f86f1c-3538-40df-b3c2-0280dc6a2bbc service nova] Acquiring lock "refresh_cache-cb0f042e-43c0-43aa-b243-cfc78a7b2695" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:31:51 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-da911b49-b699-4e87-9d72-28cab591c90c req-99f86f1c-3538-40df-b3c2-0280dc6a2bbc service nova] Acquired lock "refresh_cache-cb0f042e-43c0-43aa-b243-cfc78a7b2695" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:31:51 user nova-compute[75234]: DEBUG nova.network.neutron [req-da911b49-b699-4e87-9d72-28cab591c90c req-99f86f1c-3538-40df-b3c2-0280dc6a2bbc service nova] [instance: cb0f042e-43c0-43aa-b243-cfc78a7b2695] Refreshing network info cache for port 8f9642c3-9e10-4aa7-b5a2-53bb68ad96dc {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2006}}
Nov 29 03:31:52 user nova-compute[75234]: DEBUG nova.network.neutron [req-da911b49-b699-4e87-9d72-28cab591c90c req-99f86f1c-3538-40df-b3c2-0280dc6a2bbc service nova] [instance: cb0f042e-43c0-43aa-b243-cfc78a7b2695] Updated VIF entry in instance network info cache for port 8f9642c3-9e10-4aa7-b5a2-53bb68ad96dc. {{(pid=75234) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3481}}
Nov 29 03:31:52 user nova-compute[75234]: DEBUG nova.network.neutron [req-da911b49-b699-4e87-9d72-28cab591c90c req-99f86f1c-3538-40df-b3c2-0280dc6a2bbc service nova] [instance: cb0f042e-43c0-43aa-b243-cfc78a7b2695] Updating instance_info_cache with network_info: [{"id": "8f9642c3-9e10-4aa7-b5a2-53bb68ad96dc", "address": "fa:16:3e:31:19:9f", "network": {"id": "6787e0a8-2980-4a2e-bf2a-583af955241d", "bridge": "br-int", "label": "tempest-TestMinimumBasicScenario-1273600998-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.168", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "74f132d6666c47b8b43869df6fee6de3", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap8f9642c3-9e", "ovs_interfaceid": "8f9642c3-9e10-4aa7-b5a2-53bb68ad96dc", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:31:52 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-da911b49-b699-4e87-9d72-28cab591c90c req-99f86f1c-3538-40df-b3c2-0280dc6a2bbc service nova] Releasing lock "refresh_cache-cb0f042e-43c0-43aa-b243-cfc78a7b2695" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:31:52 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:31:53 user nova-compute[75234]: DEBUG nova.compute.manager [req-a96b4b89-0833-4c78-97f8-0227aacea642 req-d3d6aa00-3d79-46d7-b954-f71743dd74a5 service nova] [instance: cb0f042e-43c0-43aa-b243-cfc78a7b2695] Received event network-changed-8f9642c3-9e10-4aa7-b5a2-53bb68ad96dc {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:31:53 user nova-compute[75234]: DEBUG nova.compute.manager [req-a96b4b89-0833-4c78-97f8-0227aacea642 req-d3d6aa00-3d79-46d7-b954-f71743dd74a5 service nova] [instance: cb0f042e-43c0-43aa-b243-cfc78a7b2695] Refreshing instance network info cache due to event network-changed-8f9642c3-9e10-4aa7-b5a2-53bb68ad96dc. {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11117}}
Nov 29 03:31:53 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-a96b4b89-0833-4c78-97f8-0227aacea642 req-d3d6aa00-3d79-46d7-b954-f71743dd74a5 service nova] Acquiring lock "refresh_cache-cb0f042e-43c0-43aa-b243-cfc78a7b2695" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:31:53 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-a96b4b89-0833-4c78-97f8-0227aacea642 req-d3d6aa00-3d79-46d7-b954-f71743dd74a5 service nova] Acquired lock "refresh_cache-cb0f042e-43c0-43aa-b243-cfc78a7b2695" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:31:53 user nova-compute[75234]: DEBUG nova.network.neutron [req-a96b4b89-0833-4c78-97f8-0227aacea642 req-d3d6aa00-3d79-46d7-b954-f71743dd74a5 service nova] [instance: cb0f042e-43c0-43aa-b243-cfc78a7b2695] Refreshing network info cache for port 8f9642c3-9e10-4aa7-b5a2-53bb68ad96dc {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2006}}
Nov 29 03:31:54 user nova-compute[75234]: DEBUG nova.network.neutron [req-a96b4b89-0833-4c78-97f8-0227aacea642 req-d3d6aa00-3d79-46d7-b954-f71743dd74a5 service nova] [instance: cb0f042e-43c0-43aa-b243-cfc78a7b2695] Updated VIF entry in instance network info cache for port 8f9642c3-9e10-4aa7-b5a2-53bb68ad96dc. {{(pid=75234) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3481}}
Nov 29 03:31:54 user nova-compute[75234]: DEBUG nova.network.neutron [req-a96b4b89-0833-4c78-97f8-0227aacea642 req-d3d6aa00-3d79-46d7-b954-f71743dd74a5 service nova] [instance: cb0f042e-43c0-43aa-b243-cfc78a7b2695] Updating instance_info_cache with network_info: [{"id": "8f9642c3-9e10-4aa7-b5a2-53bb68ad96dc", "address": "fa:16:3e:31:19:9f", "network": {"id": "6787e0a8-2980-4a2e-bf2a-583af955241d", "bridge": "br-int", "label": "tempest-TestMinimumBasicScenario-1273600998-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.168", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "74f132d6666c47b8b43869df6fee6de3", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap8f9642c3-9e", "ovs_interfaceid": "8f9642c3-9e10-4aa7-b5a2-53bb68ad96dc", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:31:54 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-a96b4b89-0833-4c78-97f8-0227aacea642 req-d3d6aa00-3d79-46d7-b954-f71743dd74a5 service nova] Releasing lock "refresh_cache-cb0f042e-43c0-43aa-b243-cfc78a7b2695" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:31:54 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-4178432c-317f-47ab-897d-32329a486723 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Acquiring lock "cb0f042e-43c0-43aa-b243-cfc78a7b2695" by "nova.compute.manager.ComputeManager.reboot_instance.<locals>.do_reboot_instance" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:31:54 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-4178432c-317f-47ab-897d-32329a486723 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Lock "cb0f042e-43c0-43aa-b243-cfc78a7b2695" acquired by "nova.compute.manager.ComputeManager.reboot_instance.<locals>.do_reboot_instance" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:31:54 user nova-compute[75234]: INFO nova.compute.manager [None req-4178432c-317f-47ab-897d-32329a486723 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] [instance: cb0f042e-43c0-43aa-b243-cfc78a7b2695] Rebooting instance
Nov 29 03:31:54 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-4178432c-317f-47ab-897d-32329a486723 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Acquiring lock "refresh_cache-cb0f042e-43c0-43aa-b243-cfc78a7b2695" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:31:54 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-4178432c-317f-47ab-897d-32329a486723 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Acquired lock "refresh_cache-cb0f042e-43c0-43aa-b243-cfc78a7b2695" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:31:54 user nova-compute[75234]: DEBUG nova.network.neutron [None req-4178432c-317f-47ab-897d-32329a486723 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] [instance: cb0f042e-43c0-43aa-b243-cfc78a7b2695] Building network info cache for instance {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2009}}
Nov 29 03:31:54 user nova-compute[75234]: DEBUG nova.network.neutron [None req-4178432c-317f-47ab-897d-32329a486723 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] [instance: cb0f042e-43c0-43aa-b243-cfc78a7b2695] Updating instance_info_cache with network_info: [{"id": "8f9642c3-9e10-4aa7-b5a2-53bb68ad96dc", "address": "fa:16:3e:31:19:9f", "network": {"id": "6787e0a8-2980-4a2e-bf2a-583af955241d", "bridge": "br-int", "label": "tempest-TestMinimumBasicScenario-1273600998-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.168", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "74f132d6666c47b8b43869df6fee6de3", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap8f9642c3-9e", "ovs_interfaceid": "8f9642c3-9e10-4aa7-b5a2-53bb68ad96dc", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:31:55 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-4178432c-317f-47ab-897d-32329a486723 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Releasing lock "refresh_cache-cb0f042e-43c0-43aa-b243-cfc78a7b2695" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:31:55 user nova-compute[75234]: DEBUG nova.compute.manager [None req-4178432c-317f-47ab-897d-32329a486723 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] [instance: cb0f042e-43c0-43aa-b243-cfc78a7b2695] Checking state {{(pid=75234) _get_power_state /opt/stack/nova/nova/compute/manager.py:1782}}
Nov 29 03:31:56 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:31:57 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:31:57 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:31:57 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:31:57 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:31:57 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:31:57 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:31:57 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:31:57 user nova-compute[75234]: DEBUG nova.compute.manager [req-61362f96-24d9-4a44-a1e6-19f75ed74a3d req-ccf0a5b6-f82b-441d-8485-54950aea4df6 service nova] [instance: cb0f042e-43c0-43aa-b243-cfc78a7b2695] Received event network-vif-unplugged-8f9642c3-9e10-4aa7-b5a2-53bb68ad96dc {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:31:57 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-61362f96-24d9-4a44-a1e6-19f75ed74a3d req-ccf0a5b6-f82b-441d-8485-54950aea4df6 service nova] Acquiring lock "cb0f042e-43c0-43aa-b243-cfc78a7b2695-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:31:57 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-61362f96-24d9-4a44-a1e6-19f75ed74a3d req-ccf0a5b6-f82b-441d-8485-54950aea4df6 service nova] Lock "cb0f042e-43c0-43aa-b243-cfc78a7b2695-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:31:57 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-61362f96-24d9-4a44-a1e6-19f75ed74a3d req-ccf0a5b6-f82b-441d-8485-54950aea4df6 service nova] Lock "cb0f042e-43c0-43aa-b243-cfc78a7b2695-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.002s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:31:57 user nova-compute[75234]: DEBUG nova.compute.manager [req-61362f96-24d9-4a44-a1e6-19f75ed74a3d req-ccf0a5b6-f82b-441d-8485-54950aea4df6 service nova] [instance: cb0f042e-43c0-43aa-b243-cfc78a7b2695] No waiting events found dispatching network-vif-unplugged-8f9642c3-9e10-4aa7-b5a2-53bb68ad96dc {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:31:57 user nova-compute[75234]: WARNING nova.compute.manager [req-61362f96-24d9-4a44-a1e6-19f75ed74a3d req-ccf0a5b6-f82b-441d-8485-54950aea4df6 service nova] [instance: cb0f042e-43c0-43aa-b243-cfc78a7b2695] Received unexpected event network-vif-unplugged-8f9642c3-9e10-4aa7-b5a2-53bb68ad96dc for instance with vm_state active and task_state reboot_started.
Nov 29 03:31:58 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-4178432c-317f-47ab-897d-32329a486723 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] [instance: cb0f042e-43c0-43aa-b243-cfc78a7b2695] Instance shutdown successfully.
Nov 29 03:31:59 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:31:59 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:31:59 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:31:59 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:31:59 user nova-compute[75234]: DEBUG nova.compute.manager [req-2b3c362a-ea2d-4c1b-8fab-4c4f5c448a9a req-0d64455f-9b75-4e4a-b0e9-e7c96d1b1567 service nova] [instance: cb0f042e-43c0-43aa-b243-cfc78a7b2695] Received event network-vif-plugged-8f9642c3-9e10-4aa7-b5a2-53bb68ad96dc {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:31:59 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-2b3c362a-ea2d-4c1b-8fab-4c4f5c448a9a req-0d64455f-9b75-4e4a-b0e9-e7c96d1b1567 service nova] Acquiring lock "cb0f042e-43c0-43aa-b243-cfc78a7b2695-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:31:59 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-2b3c362a-ea2d-4c1b-8fab-4c4f5c448a9a req-0d64455f-9b75-4e4a-b0e9-e7c96d1b1567 service nova] Lock "cb0f042e-43c0-43aa-b243-cfc78a7b2695-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:31:59 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-2b3c362a-ea2d-4c1b-8fab-4c4f5c448a9a req-0d64455f-9b75-4e4a-b0e9-e7c96d1b1567 service nova] Lock "cb0f042e-43c0-43aa-b243-cfc78a7b2695-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:31:59 user nova-compute[75234]: DEBUG nova.compute.manager [req-2b3c362a-ea2d-4c1b-8fab-4c4f5c448a9a req-0d64455f-9b75-4e4a-b0e9-e7c96d1b1567 service nova] [instance: cb0f042e-43c0-43aa-b243-cfc78a7b2695] No waiting events found dispatching network-vif-plugged-8f9642c3-9e10-4aa7-b5a2-53bb68ad96dc {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:31:59 user nova-compute[75234]: WARNING nova.compute.manager [req-2b3c362a-ea2d-4c1b-8fab-4c4f5c448a9a req-0d64455f-9b75-4e4a-b0e9-e7c96d1b1567 service nova] [instance: cb0f042e-43c0-43aa-b243-cfc78a7b2695] Received unexpected event network-vif-plugged-8f9642c3-9e10-4aa7-b5a2-53bb68ad96dc for instance with vm_state active and task_state reboot_started.
Nov 29 03:31:59 user nova-compute[75234]: DEBUG nova.compute.manager [req-2b3c362a-ea2d-4c1b-8fab-4c4f5c448a9a req-0d64455f-9b75-4e4a-b0e9-e7c96d1b1567 service nova] [instance: cb0f042e-43c0-43aa-b243-cfc78a7b2695] Received event network-vif-plugged-8f9642c3-9e10-4aa7-b5a2-53bb68ad96dc {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:31:59 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-2b3c362a-ea2d-4c1b-8fab-4c4f5c448a9a req-0d64455f-9b75-4e4a-b0e9-e7c96d1b1567 service nova] Acquiring lock "cb0f042e-43c0-43aa-b243-cfc78a7b2695-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:31:59 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-2b3c362a-ea2d-4c1b-8fab-4c4f5c448a9a req-0d64455f-9b75-4e4a-b0e9-e7c96d1b1567 service nova] Lock "cb0f042e-43c0-43aa-b243-cfc78a7b2695-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:31:59 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-2b3c362a-ea2d-4c1b-8fab-4c4f5c448a9a req-0d64455f-9b75-4e4a-b0e9-e7c96d1b1567 service nova] Lock "cb0f042e-43c0-43aa-b243-cfc78a7b2695-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:31:59 user nova-compute[75234]: DEBUG nova.compute.manager [req-2b3c362a-ea2d-4c1b-8fab-4c4f5c448a9a req-0d64455f-9b75-4e4a-b0e9-e7c96d1b1567 service nova] [instance: cb0f042e-43c0-43aa-b243-cfc78a7b2695] No waiting events found dispatching network-vif-plugged-8f9642c3-9e10-4aa7-b5a2-53bb68ad96dc {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:31:59 user nova-compute[75234]: WARNING nova.compute.manager [req-2b3c362a-ea2d-4c1b-8fab-4c4f5c448a9a req-0d64455f-9b75-4e4a-b0e9-e7c96d1b1567 service nova] [instance: cb0f042e-43c0-43aa-b243-cfc78a7b2695] Received unexpected event network-vif-plugged-8f9642c3-9e10-4aa7-b5a2-53bb68ad96dc for instance with vm_state active and task_state reboot_started.
Nov 29 03:31:59 user nova-compute[75234]: DEBUG nova.compute.manager [req-2b3c362a-ea2d-4c1b-8fab-4c4f5c448a9a req-0d64455f-9b75-4e4a-b0e9-e7c96d1b1567 service nova] [instance: cb0f042e-43c0-43aa-b243-cfc78a7b2695] Received event network-vif-plugged-8f9642c3-9e10-4aa7-b5a2-53bb68ad96dc {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:31:59 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-2b3c362a-ea2d-4c1b-8fab-4c4f5c448a9a req-0d64455f-9b75-4e4a-b0e9-e7c96d1b1567 service nova] Acquiring lock "cb0f042e-43c0-43aa-b243-cfc78a7b2695-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:31:59 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-2b3c362a-ea2d-4c1b-8fab-4c4f5c448a9a req-0d64455f-9b75-4e4a-b0e9-e7c96d1b1567 service nova] Lock "cb0f042e-43c0-43aa-b243-cfc78a7b2695-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:31:59 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-2b3c362a-ea2d-4c1b-8fab-4c4f5c448a9a req-0d64455f-9b75-4e4a-b0e9-e7c96d1b1567 service nova] Lock "cb0f042e-43c0-43aa-b243-cfc78a7b2695-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:31:59 user nova-compute[75234]: DEBUG nova.compute.manager [req-2b3c362a-ea2d-4c1b-8fab-4c4f5c448a9a req-0d64455f-9b75-4e4a-b0e9-e7c96d1b1567 service nova] [instance: cb0f042e-43c0-43aa-b243-cfc78a7b2695] No waiting events found dispatching network-vif-plugged-8f9642c3-9e10-4aa7-b5a2-53bb68ad96dc {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:31:59 user nova-compute[75234]: WARNING nova.compute.manager [req-2b3c362a-ea2d-4c1b-8fab-4c4f5c448a9a req-0d64455f-9b75-4e4a-b0e9-e7c96d1b1567 service nova] [instance: cb0f042e-43c0-43aa-b243-cfc78a7b2695] Received unexpected event network-vif-plugged-8f9642c3-9e10-4aa7-b5a2-53bb68ad96dc for instance with vm_state active and task_state reboot_started.
Nov 29 03:31:59 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:31:59 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:31:59 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:32:00 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:32:01 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:32:01 user nova-compute[75234]: INFO nova.virt.libvirt.driver [-] [instance: cb0f042e-43c0-43aa-b243-cfc78a7b2695] Instance running successfully.
Nov 29 03:32:01 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-4178432c-317f-47ab-897d-32329a486723 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] [instance: cb0f042e-43c0-43aa-b243-cfc78a7b2695] Instance soft rebooted successfully.
Nov 29 03:32:01 user nova-compute[75234]: DEBUG nova.compute.manager [None req-4178432c-317f-47ab-897d-32329a486723 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] [instance: cb0f042e-43c0-43aa-b243-cfc78a7b2695] Checking state {{(pid=75234) _get_power_state /opt/stack/nova/nova/compute/manager.py:1782}}
Nov 29 03:32:01 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-4178432c-317f-47ab-897d-32329a486723 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Lock "cb0f042e-43c0-43aa-b243-cfc78a7b2695" "released" by "nova.compute.manager.ComputeManager.reboot_instance.<locals>.do_reboot_instance" :: held 7.152s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:32:02 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:32:04 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._cleanup_incomplete_migrations {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:32:04 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Cleaning up deleted instances with incomplete migration  {{(pid=75234) _cleanup_incomplete_migrations /opt/stack/nova/nova/compute/manager.py:11247}}
Nov 29 03:32:04 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._cleanup_running_deleted_instances {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:32:06 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:32:07 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:32:10 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:32:10 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:32:11 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:32:11 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:32:12 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:32:12 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=75234) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10541}}
Nov 29 03:32:13 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager.update_available_resource {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:32:13 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:32:13 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:32:13 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:32:13 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Auditing locally available compute resources for user (node: user) {{(pid=75234) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:907}}
Nov 29 03:32:13 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/65d48c93-b6da-4f51-9d5a-0676dda62a60/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:32:13 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/65d48c93-b6da-4f51-9d5a-0676dda62a60/disk --force-share --output=json" returned: 0 in 0.178s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:32:13 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/65d48c93-b6da-4f51-9d5a-0676dda62a60/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:32:13 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/65d48c93-b6da-4f51-9d5a-0676dda62a60/disk --force-share --output=json" returned: 0 in 0.170s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:32:13 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] skipping disk /dev/sdc (vda) as it is a volume {{(pid=75234) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11213}}
Nov 29 03:32:13 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/abb01889-df4b-44a6-9827-87056d7ca125/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:32:14 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/abb01889-df4b-44a6-9827-87056d7ca125/disk --force-share --output=json" returned: 0 in 0.177s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:32:14 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/abb01889-df4b-44a6-9827-87056d7ca125/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:32:14 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/abb01889-df4b-44a6-9827-87056d7ca125/disk --force-share --output=json" returned: 0 in 0.170s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:32:14 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/cb0f042e-43c0-43aa-b243-cfc78a7b2695/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:32:14 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/cb0f042e-43c0-43aa-b243-cfc78a7b2695/disk --force-share --output=json" returned: 0 in 0.169s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:32:14 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/cb0f042e-43c0-43aa-b243-cfc78a7b2695/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:32:14 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/cb0f042e-43c0-43aa-b243-cfc78a7b2695/disk --force-share --output=json" returned: 0 in 0.169s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:32:14 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] skipping disk /dev/sdb (vdb) as it is a volume {{(pid=75234) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11213}}
Nov 29 03:32:15 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:32:15 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:32:15 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Hypervisor/Node resource view: name=user free_ram=8518MB free_disk=24.98299789428711GB free_vcpus=8 pci_devices=[{"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}] {{(pid=75234) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1106}}
Nov 29 03:32:15 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:32:15 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:32:15 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance 65d48c93-b6da-4f51-9d5a-0676dda62a60 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 512, 'DISK_GB': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:32:15 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance d5096ca2-d005-45ac-a487-11a810446859 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 512}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:32:15 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance abb01889-df4b-44a6-9827-87056d7ca125 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 512, 'DISK_GB': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:32:15 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance cb0f042e-43c0-43aa-b243-cfc78a7b2695 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 512, 'DISK_GB': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:32:15 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Total usable vcpus: 12, total allocated vcpus: 4 {{(pid=75234) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1129}}
Nov 29 03:32:15 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Final resource view: name=user phys_ram=16011MB used_ram=2560MB phys_disk=40GB used_disk=3GB total_vcpus=12 used_vcpus=4 pci_stats=[] {{(pid=75234) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1138}}
Nov 29 03:32:15 user nova-compute[75234]: DEBUG nova.compute.provider_tree [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Inventory has not changed in ProviderTree for provider: 4d291220-7c37-4839-a0e3-b23c66502b74 {{(pid=75234) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Nov 29 03:32:15 user nova-compute[75234]: DEBUG nova.scheduler.client.report [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Inventory has not changed for provider 4d291220-7c37-4839-a0e3-b23c66502b74 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=75234) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Nov 29 03:32:15 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Compute_service record updated for user:user {{(pid=75234) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:1067}}
Nov 29 03:32:15 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.485s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:32:16 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:32:17 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:32:17 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Starting heal instance info cache {{(pid=75234) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9922}}
Nov 29 03:32:17 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Acquiring lock "refresh_cache-d5096ca2-d005-45ac-a487-11a810446859" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:32:17 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Acquired lock "refresh_cache-d5096ca2-d005-45ac-a487-11a810446859" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:32:17 user nova-compute[75234]: DEBUG nova.network.neutron [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] [instance: d5096ca2-d005-45ac-a487-11a810446859] Forcefully refreshing network info cache for instance {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2003}}
Nov 29 03:32:17 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:32:18 user nova-compute[75234]: DEBUG nova.network.neutron [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] [instance: d5096ca2-d005-45ac-a487-11a810446859] Updating instance_info_cache with network_info: [{"id": "9266d021-6e83-4bfa-8151-66c5182e61a0", "address": "fa:16:3e:ec:fb:67", "network": {"id": "4d086e08-7798-4cfe-9365-86be6f0a52d2", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-305624008-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "db00bcd0c82c44f1bda739e9857f6e84", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap9266d021-6e", "ovs_interfaceid": "9266d021-6e83-4bfa-8151-66c5182e61a0", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:32:18 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Releasing lock "refresh_cache-d5096ca2-d005-45ac-a487-11a810446859" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:32:18 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] [instance: d5096ca2-d005-45ac-a487-11a810446859] Updated the network info_cache for instance {{(pid=75234) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9993}}
Nov 29 03:32:18 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._sync_power_states {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:32:18 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Triggering sync for uuid 65d48c93-b6da-4f51-9d5a-0676dda62a60 {{(pid=75234) _sync_power_states /opt/stack/nova/nova/compute/manager.py:10332}}
Nov 29 03:32:18 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Triggering sync for uuid d5096ca2-d005-45ac-a487-11a810446859 {{(pid=75234) _sync_power_states /opt/stack/nova/nova/compute/manager.py:10332}}
Nov 29 03:32:18 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Triggering sync for uuid abb01889-df4b-44a6-9827-87056d7ca125 {{(pid=75234) _sync_power_states /opt/stack/nova/nova/compute/manager.py:10332}}
Nov 29 03:32:18 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Triggering sync for uuid cb0f042e-43c0-43aa-b243-cfc78a7b2695 {{(pid=75234) _sync_power_states /opt/stack/nova/nova/compute/manager.py:10332}}
Nov 29 03:32:18 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Acquiring lock "65d48c93-b6da-4f51-9d5a-0676dda62a60" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:32:18 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "65d48c93-b6da-4f51-9d5a-0676dda62a60" acquired by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:32:18 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Acquiring lock "d5096ca2-d005-45ac-a487-11a810446859" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:32:18 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "d5096ca2-d005-45ac-a487-11a810446859" acquired by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:32:18 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Acquiring lock "abb01889-df4b-44a6-9827-87056d7ca125" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:32:18 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "abb01889-df4b-44a6-9827-87056d7ca125" acquired by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:32:18 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Acquiring lock "cb0f042e-43c0-43aa-b243-cfc78a7b2695" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:32:18 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "cb0f042e-43c0-43aa-b243-cfc78a7b2695" acquired by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:32:18 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "65d48c93-b6da-4f51-9d5a-0676dda62a60" "released" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: held 0.034s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:32:18 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "d5096ca2-d005-45ac-a487-11a810446859" "released" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: held 0.034s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:32:18 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "cb0f042e-43c0-43aa-b243-cfc78a7b2695" "released" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: held 0.037s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:32:18 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "abb01889-df4b-44a6-9827-87056d7ca125" "released" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: held 0.039s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:32:18 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:32:19 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:32:21 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:32:22 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:32:23 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._sync_scheduler_instance_info {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:32:23 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:32:26 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:32:27 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:32:27 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-d0cfb3d1-066f-4604-ac70-d86765dfbd73 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Acquiring lock "ebc3e3b2-037d-43ef-9426-caebe298ee3e" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:32:27 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-d0cfb3d1-066f-4604-ac70-d86765dfbd73 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Lock "ebc3e3b2-037d-43ef-9426-caebe298ee3e" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:32:27 user nova-compute[75234]: DEBUG nova.compute.manager [None req-d0cfb3d1-066f-4604-ac70-d86765dfbd73 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] [instance: ebc3e3b2-037d-43ef-9426-caebe298ee3e] Starting instance... {{(pid=75234) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2422}}
Nov 29 03:32:28 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-d0cfb3d1-066f-4604-ac70-d86765dfbd73 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:32:28 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-d0cfb3d1-066f-4604-ac70-d86765dfbd73 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:32:28 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-d0cfb3d1-066f-4604-ac70-d86765dfbd73 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=75234) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2366}}
Nov 29 03:32:28 user nova-compute[75234]: INFO nova.compute.claims [None req-d0cfb3d1-066f-4604-ac70-d86765dfbd73 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] [instance: ebc3e3b2-037d-43ef-9426-caebe298ee3e] Claim successful on node user
Nov 29 03:32:28 user nova-compute[75234]: DEBUG nova.compute.provider_tree [None req-d0cfb3d1-066f-4604-ac70-d86765dfbd73 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Inventory has not changed in ProviderTree for provider: 4d291220-7c37-4839-a0e3-b23c66502b74 {{(pid=75234) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Nov 29 03:32:28 user nova-compute[75234]: DEBUG nova.scheduler.client.report [None req-d0cfb3d1-066f-4604-ac70-d86765dfbd73 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Inventory has not changed for provider 4d291220-7c37-4839-a0e3-b23c66502b74 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=75234) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Nov 29 03:32:28 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-d0cfb3d1-066f-4604-ac70-d86765dfbd73 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.256s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:32:28 user nova-compute[75234]: DEBUG nova.compute.manager [None req-d0cfb3d1-066f-4604-ac70-d86765dfbd73 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] [instance: ebc3e3b2-037d-43ef-9426-caebe298ee3e] Start building networks asynchronously for instance. {{(pid=75234) _build_resources /opt/stack/nova/nova/compute/manager.py:2819}}
Nov 29 03:32:28 user nova-compute[75234]: DEBUG nova.compute.manager [None req-d0cfb3d1-066f-4604-ac70-d86765dfbd73 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] [instance: ebc3e3b2-037d-43ef-9426-caebe298ee3e] Allocating IP information in the background. {{(pid=75234) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1972}}
Nov 29 03:32:28 user nova-compute[75234]: DEBUG nova.network.neutron [None req-d0cfb3d1-066f-4604-ac70-d86765dfbd73 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] [instance: ebc3e3b2-037d-43ef-9426-caebe298ee3e] allocate_for_instance() {{(pid=75234) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1156}}
Nov 29 03:32:28 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-d0cfb3d1-066f-4604-ac70-d86765dfbd73 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] [instance: ebc3e3b2-037d-43ef-9426-caebe298ee3e] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Nov 29 03:32:28 user nova-compute[75234]: DEBUG nova.compute.manager [None req-d0cfb3d1-066f-4604-ac70-d86765dfbd73 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] [instance: ebc3e3b2-037d-43ef-9426-caebe298ee3e] Start building block device mappings for instance. {{(pid=75234) _build_resources /opt/stack/nova/nova/compute/manager.py:2854}}
Nov 29 03:32:28 user nova-compute[75234]: DEBUG nova.policy [None req-d0cfb3d1-066f-4604-ac70-d86765dfbd73 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '877aa877f13a4f84ac4725278010aea3', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'db00bcd0c82c44f1bda739e9857f6e84', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=75234) authorize /opt/stack/nova/nova/policy.py:203}}
Nov 29 03:32:28 user nova-compute[75234]: INFO nova.virt.block_device [None req-d0cfb3d1-066f-4604-ac70-d86765dfbd73 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] [instance: ebc3e3b2-037d-43ef-9426-caebe298ee3e] Booting with volume-backed-image 96c30ba7-9b73-4a30-8ba4-7138879573b7 at /dev/vda
Nov 29 03:32:28 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:32:29 user nova-compute[75234]: DEBUG nova.network.neutron [None req-d0cfb3d1-066f-4604-ac70-d86765dfbd73 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] [instance: ebc3e3b2-037d-43ef-9426-caebe298ee3e] Successfully created port: bd0a4e5b-69cd-4f5c-ae22-994e4d8f2682 {{(pid=75234) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:548}}
Nov 29 03:32:29 user nova-compute[75234]: DEBUG nova.network.neutron [None req-d0cfb3d1-066f-4604-ac70-d86765dfbd73 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] [instance: ebc3e3b2-037d-43ef-9426-caebe298ee3e] Successfully updated port: bd0a4e5b-69cd-4f5c-ae22-994e4d8f2682 {{(pid=75234) _update_port /opt/stack/nova/nova/network/neutron.py:586}}
Nov 29 03:32:29 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-d0cfb3d1-066f-4604-ac70-d86765dfbd73 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Acquiring lock "refresh_cache-ebc3e3b2-037d-43ef-9426-caebe298ee3e" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:32:29 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-d0cfb3d1-066f-4604-ac70-d86765dfbd73 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Acquired lock "refresh_cache-ebc3e3b2-037d-43ef-9426-caebe298ee3e" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:32:29 user nova-compute[75234]: DEBUG nova.network.neutron [None req-d0cfb3d1-066f-4604-ac70-d86765dfbd73 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] [instance: ebc3e3b2-037d-43ef-9426-caebe298ee3e] Building network info cache for instance {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2009}}
Nov 29 03:32:29 user nova-compute[75234]: DEBUG nova.compute.manager [req-187e2fa1-78ca-4d14-957c-3030a5f2dcd7 req-04515bf5-6ac4-4f5b-9836-45ab69b355e1 service nova] [instance: ebc3e3b2-037d-43ef-9426-caebe298ee3e] Received event network-changed-bd0a4e5b-69cd-4f5c-ae22-994e4d8f2682 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:32:29 user nova-compute[75234]: DEBUG nova.compute.manager [req-187e2fa1-78ca-4d14-957c-3030a5f2dcd7 req-04515bf5-6ac4-4f5b-9836-45ab69b355e1 service nova] [instance: ebc3e3b2-037d-43ef-9426-caebe298ee3e] Refreshing instance network info cache due to event network-changed-bd0a4e5b-69cd-4f5c-ae22-994e4d8f2682. {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11117}}
Nov 29 03:32:29 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-187e2fa1-78ca-4d14-957c-3030a5f2dcd7 req-04515bf5-6ac4-4f5b-9836-45ab69b355e1 service nova] Acquiring lock "refresh_cache-ebc3e3b2-037d-43ef-9426-caebe298ee3e" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:32:29 user nova-compute[75234]: DEBUG nova.network.neutron [None req-d0cfb3d1-066f-4604-ac70-d86765dfbd73 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] [instance: ebc3e3b2-037d-43ef-9426-caebe298ee3e] Instance cache missing network info. {{(pid=75234) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3322}}
Nov 29 03:32:30 user nova-compute[75234]: DEBUG nova.network.neutron [None req-d0cfb3d1-066f-4604-ac70-d86765dfbd73 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] [instance: ebc3e3b2-037d-43ef-9426-caebe298ee3e] Updating instance_info_cache with network_info: [{"id": "bd0a4e5b-69cd-4f5c-ae22-994e4d8f2682", "address": "fa:16:3e:c2:aa:3e", "network": {"id": "4d086e08-7798-4cfe-9365-86be6f0a52d2", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-305624008-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "db00bcd0c82c44f1bda739e9857f6e84", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapbd0a4e5b-69", "ovs_interfaceid": "bd0a4e5b-69cd-4f5c-ae22-994e4d8f2682", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:32:30 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-d0cfb3d1-066f-4604-ac70-d86765dfbd73 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Releasing lock "refresh_cache-ebc3e3b2-037d-43ef-9426-caebe298ee3e" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:32:30 user nova-compute[75234]: DEBUG nova.compute.manager [None req-d0cfb3d1-066f-4604-ac70-d86765dfbd73 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] [instance: ebc3e3b2-037d-43ef-9426-caebe298ee3e] Instance network_info: |[{"id": "bd0a4e5b-69cd-4f5c-ae22-994e4d8f2682", "address": "fa:16:3e:c2:aa:3e", "network": {"id": "4d086e08-7798-4cfe-9365-86be6f0a52d2", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-305624008-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "db00bcd0c82c44f1bda739e9857f6e84", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapbd0a4e5b-69", "ovs_interfaceid": "bd0a4e5b-69cd-4f5c-ae22-994e4d8f2682", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=75234) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1987}}
Nov 29 03:32:30 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-187e2fa1-78ca-4d14-957c-3030a5f2dcd7 req-04515bf5-6ac4-4f5b-9836-45ab69b355e1 service nova] Acquired lock "refresh_cache-ebc3e3b2-037d-43ef-9426-caebe298ee3e" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:32:30 user nova-compute[75234]: DEBUG nova.network.neutron [req-187e2fa1-78ca-4d14-957c-3030a5f2dcd7 req-04515bf5-6ac4-4f5b-9836-45ab69b355e1 service nova] [instance: ebc3e3b2-037d-43ef-9426-caebe298ee3e] Refreshing network info cache for port bd0a4e5b-69cd-4f5c-ae22-994e4d8f2682 {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2006}}
Nov 29 03:32:30 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._run_pending_deletes {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:32:30 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Cleaning up deleted instances {{(pid=75234) _run_pending_deletes /opt/stack/nova/nova/compute/manager.py:11209}}
Nov 29 03:32:30 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] There are 0 instances to clean {{(pid=75234) _run_pending_deletes /opt/stack/nova/nova/compute/manager.py:11218}}
Nov 29 03:32:30 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._cleanup_expired_console_auth_tokens {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:32:30 user nova-compute[75234]: DEBUG nova.network.neutron [req-187e2fa1-78ca-4d14-957c-3030a5f2dcd7 req-04515bf5-6ac4-4f5b-9836-45ab69b355e1 service nova] [instance: ebc3e3b2-037d-43ef-9426-caebe298ee3e] Updated VIF entry in instance network info cache for port bd0a4e5b-69cd-4f5c-ae22-994e4d8f2682. {{(pid=75234) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3481}}
Nov 29 03:32:30 user nova-compute[75234]: DEBUG nova.network.neutron [req-187e2fa1-78ca-4d14-957c-3030a5f2dcd7 req-04515bf5-6ac4-4f5b-9836-45ab69b355e1 service nova] [instance: ebc3e3b2-037d-43ef-9426-caebe298ee3e] Updating instance_info_cache with network_info: [{"id": "bd0a4e5b-69cd-4f5c-ae22-994e4d8f2682", "address": "fa:16:3e:c2:aa:3e", "network": {"id": "4d086e08-7798-4cfe-9365-86be6f0a52d2", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-305624008-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "db00bcd0c82c44f1bda739e9857f6e84", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapbd0a4e5b-69", "ovs_interfaceid": "bd0a4e5b-69cd-4f5c-ae22-994e4d8f2682", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:32:30 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-187e2fa1-78ca-4d14-957c-3030a5f2dcd7 req-04515bf5-6ac4-4f5b-9836-45ab69b355e1 service nova] Releasing lock "refresh_cache-ebc3e3b2-037d-43ef-9426-caebe298ee3e" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:32:31 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-3f2c9326-e9dc-46ad-b028-3d5aef2c14ea tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Acquiring lock "884589ec-b86c-4094-9a3f-54b641311d62" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:32:31 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-3f2c9326-e9dc-46ad-b028-3d5aef2c14ea tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Lock "884589ec-b86c-4094-9a3f-54b641311d62" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:32:31 user nova-compute[75234]: DEBUG nova.compute.manager [None req-3f2c9326-e9dc-46ad-b028-3d5aef2c14ea tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: 884589ec-b86c-4094-9a3f-54b641311d62] Starting instance... {{(pid=75234) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2422}}
Nov 29 03:32:31 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-3f2c9326-e9dc-46ad-b028-3d5aef2c14ea tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:32:31 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-3f2c9326-e9dc-46ad-b028-3d5aef2c14ea tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:32:31 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-3f2c9326-e9dc-46ad-b028-3d5aef2c14ea tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=75234) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2366}}
Nov 29 03:32:31 user nova-compute[75234]: INFO nova.compute.claims [None req-3f2c9326-e9dc-46ad-b028-3d5aef2c14ea tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: 884589ec-b86c-4094-9a3f-54b641311d62] Claim successful on node user
Nov 29 03:32:31 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:32:31 user nova-compute[75234]: DEBUG nova.compute.provider_tree [None req-3f2c9326-e9dc-46ad-b028-3d5aef2c14ea tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Inventory has not changed in ProviderTree for provider: 4d291220-7c37-4839-a0e3-b23c66502b74 {{(pid=75234) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Nov 29 03:32:31 user nova-compute[75234]: DEBUG nova.scheduler.client.report [None req-3f2c9326-e9dc-46ad-b028-3d5aef2c14ea tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Inventory has not changed for provider 4d291220-7c37-4839-a0e3-b23c66502b74 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=75234) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Nov 29 03:32:31 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-3f2c9326-e9dc-46ad-b028-3d5aef2c14ea tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.296s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:32:31 user nova-compute[75234]: DEBUG nova.compute.manager [None req-3f2c9326-e9dc-46ad-b028-3d5aef2c14ea tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: 884589ec-b86c-4094-9a3f-54b641311d62] Start building networks asynchronously for instance. {{(pid=75234) _build_resources /opt/stack/nova/nova/compute/manager.py:2819}}
Nov 29 03:32:31 user nova-compute[75234]: DEBUG nova.compute.manager [None req-3f2c9326-e9dc-46ad-b028-3d5aef2c14ea tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: 884589ec-b86c-4094-9a3f-54b641311d62] Allocating IP information in the background. {{(pid=75234) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1972}}
Nov 29 03:32:31 user nova-compute[75234]: DEBUG nova.network.neutron [None req-3f2c9326-e9dc-46ad-b028-3d5aef2c14ea tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: 884589ec-b86c-4094-9a3f-54b641311d62] allocate_for_instance() {{(pid=75234) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1156}}
Nov 29 03:32:32 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-3f2c9326-e9dc-46ad-b028-3d5aef2c14ea tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: 884589ec-b86c-4094-9a3f-54b641311d62] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Nov 29 03:32:32 user nova-compute[75234]: DEBUG nova.compute.manager [None req-3f2c9326-e9dc-46ad-b028-3d5aef2c14ea tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: 884589ec-b86c-4094-9a3f-54b641311d62] Start building block device mappings for instance. {{(pid=75234) _build_resources /opt/stack/nova/nova/compute/manager.py:2854}}
Nov 29 03:32:32 user nova-compute[75234]: INFO nova.virt.block_device [None req-3f2c9326-e9dc-46ad-b028-3d5aef2c14ea tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: 884589ec-b86c-4094-9a3f-54b641311d62] Booting with volume a3d30ae4-ca6d-4e06-8fc9-caf4d91ef900 at /dev/vda
Nov 29 03:32:32 user nova-compute[75234]: DEBUG nova.policy [None req-3f2c9326-e9dc-46ad-b028-3d5aef2c14ea tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '7d1f1f11a9aa4c1db8627413eb963fa4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '81a265d3c15b4e9d826b35ddbd0ff443', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=75234) authorize /opt/stack/nova/nova/policy.py:203}}
Nov 29 03:32:32 user nova-compute[75234]: DEBUG os_brick.utils [None req-3f2c9326-e9dc-46ad-b028-3d5aef2c14ea tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:176}}
Nov 29 03:32:32 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:32:32 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.016s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:32:32 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[ccfbcd49-3063-419e-ac3a-d5af33fbbf11]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:dd1368c5ba2c\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:32:32 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt -v / -n -o SOURCE {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:32:32 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt -v / -n -o SOURCE" returned: 0 in 0.014s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:32:32 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[85404181-9757-42ec-8c33-f5798d1dd76a]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:32:32 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:32:32 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.017s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:32:32 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[698311e2-7db3-4d5b-8269-4e6e8395843c]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:32:32 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[bef1be60-c1f1-4da9-987a-31ef0c5221c4]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:32:32 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-3f2c9326-e9dc-46ad-b028-3d5aef2c14ea tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Running cmd (subprocess): nvme version {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:32:32 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-3f2c9326-e9dc-46ad-b028-3d5aef2c14ea tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] 'nvme version' failed. Not Retrying. {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:473}}
Nov 29 03:32:32 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.nvmeof [None req-3f2c9326-e9dc-46ad-b028-3d5aef2c14ea tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] nvme not present on system {{(pid=75234) nvme_present /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/nvmeof.py:756}}
Nov 29 03:32:32 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:32:32 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:473}}
Nov 29 03:32:32 user nova-compute[75234]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Nov 29 03:32:32 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[3d6b1794-8686-47c5-85fb-71d2cb050bdc]: (4, '') {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:32:32 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.lightos [None req-3f2c9326-e9dc-46ad-b028-3d5aef2c14ea tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=75234) find_dsc /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/lightos.py:98}}
Nov 29 03:32:32 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.lightos [None req-3f2c9326-e9dc-46ad-b028-3d5aef2c14ea tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=75234) get_connector_properties /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/lightos.py:76}}
Nov 29 03:32:32 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.lightos [None req-3f2c9326-e9dc-46ad-b028-3d5aef2c14ea tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] LIGHTOS: no hostnqn found. {{(pid=75234) get_connector_properties /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/lightos.py:84}}
Nov 29 03:32:32 user nova-compute[75234]: DEBUG os_brick.utils [None req-3f2c9326-e9dc-46ad-b028-3d5aef2c14ea tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] <== get_connector_properties: return (151ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:dd1368c5ba2c', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:203}}
Nov 29 03:32:32 user nova-compute[75234]: DEBUG nova.virt.block_device [None req-3f2c9326-e9dc-46ad-b028-3d5aef2c14ea tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: 884589ec-b86c-4094-9a3f-54b641311d62] Updating existing volume attachment record: e4b3ea0b-a686-4154-9d19-42280681db55 {{(pid=75234) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Nov 29 03:32:32 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:32:32 user nova-compute[75234]: DEBUG nova.network.neutron [None req-3f2c9326-e9dc-46ad-b028-3d5aef2c14ea tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: 884589ec-b86c-4094-9a3f-54b641311d62] Successfully created port: 1deddce5-7279-44f9-9f1e-cd9ffddc5d6c {{(pid=75234) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:548}}
Nov 29 03:32:33 user nova-compute[75234]: DEBUG nova.network.neutron [None req-3f2c9326-e9dc-46ad-b028-3d5aef2c14ea tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: 884589ec-b86c-4094-9a3f-54b641311d62] Successfully updated port: 1deddce5-7279-44f9-9f1e-cd9ffddc5d6c {{(pid=75234) _update_port /opt/stack/nova/nova/network/neutron.py:586}}
Nov 29 03:32:33 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-3f2c9326-e9dc-46ad-b028-3d5aef2c14ea tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Acquiring lock "refresh_cache-884589ec-b86c-4094-9a3f-54b641311d62" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:32:33 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-3f2c9326-e9dc-46ad-b028-3d5aef2c14ea tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Acquired lock "refresh_cache-884589ec-b86c-4094-9a3f-54b641311d62" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:32:33 user nova-compute[75234]: DEBUG nova.network.neutron [None req-3f2c9326-e9dc-46ad-b028-3d5aef2c14ea tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: 884589ec-b86c-4094-9a3f-54b641311d62] Building network info cache for instance {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2009}}
Nov 29 03:32:33 user nova-compute[75234]: DEBUG nova.compute.manager [req-537a380e-8241-49f5-bfb4-06d651ccc7cd req-f6beeba1-9278-4000-86d2-06658efde833 service nova] [instance: 884589ec-b86c-4094-9a3f-54b641311d62] Received event network-changed-1deddce5-7279-44f9-9f1e-cd9ffddc5d6c {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:32:33 user nova-compute[75234]: DEBUG nova.compute.manager [req-537a380e-8241-49f5-bfb4-06d651ccc7cd req-f6beeba1-9278-4000-86d2-06658efde833 service nova] [instance: 884589ec-b86c-4094-9a3f-54b641311d62] Refreshing instance network info cache due to event network-changed-1deddce5-7279-44f9-9f1e-cd9ffddc5d6c. {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11117}}
Nov 29 03:32:33 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-537a380e-8241-49f5-bfb4-06d651ccc7cd req-f6beeba1-9278-4000-86d2-06658efde833 service nova] Acquiring lock "refresh_cache-884589ec-b86c-4094-9a3f-54b641311d62" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:32:33 user nova-compute[75234]: DEBUG nova.network.neutron [None req-3f2c9326-e9dc-46ad-b028-3d5aef2c14ea tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: 884589ec-b86c-4094-9a3f-54b641311d62] Instance cache missing network info. {{(pid=75234) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3322}}
Nov 29 03:32:34 user nova-compute[75234]: DEBUG nova.network.neutron [None req-3f2c9326-e9dc-46ad-b028-3d5aef2c14ea tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: 884589ec-b86c-4094-9a3f-54b641311d62] Updating instance_info_cache with network_info: [{"id": "1deddce5-7279-44f9-9f1e-cd9ffddc5d6c", "address": "fa:16:3e:c5:44:c3", "network": {"id": "a3f397d3-fd75-4a6e-a815-cfcedcca15fb", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-678570085-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "81a265d3c15b4e9d826b35ddbd0ff443", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap1deddce5-72", "ovs_interfaceid": "1deddce5-7279-44f9-9f1e-cd9ffddc5d6c", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:32:34 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-3f2c9326-e9dc-46ad-b028-3d5aef2c14ea tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Releasing lock "refresh_cache-884589ec-b86c-4094-9a3f-54b641311d62" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:32:34 user nova-compute[75234]: DEBUG nova.compute.manager [None req-3f2c9326-e9dc-46ad-b028-3d5aef2c14ea tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: 884589ec-b86c-4094-9a3f-54b641311d62] Instance network_info: |[{"id": "1deddce5-7279-44f9-9f1e-cd9ffddc5d6c", "address": "fa:16:3e:c5:44:c3", "network": {"id": "a3f397d3-fd75-4a6e-a815-cfcedcca15fb", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-678570085-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "81a265d3c15b4e9d826b35ddbd0ff443", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap1deddce5-72", "ovs_interfaceid": "1deddce5-7279-44f9-9f1e-cd9ffddc5d6c", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=75234) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1987}}
Nov 29 03:32:34 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-537a380e-8241-49f5-bfb4-06d651ccc7cd req-f6beeba1-9278-4000-86d2-06658efde833 service nova] Acquired lock "refresh_cache-884589ec-b86c-4094-9a3f-54b641311d62" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:32:34 user nova-compute[75234]: DEBUG nova.network.neutron [req-537a380e-8241-49f5-bfb4-06d651ccc7cd req-f6beeba1-9278-4000-86d2-06658efde833 service nova] [instance: 884589ec-b86c-4094-9a3f-54b641311d62] Refreshing network info cache for port 1deddce5-7279-44f9-9f1e-cd9ffddc5d6c {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2006}}
Nov 29 03:32:34 user nova-compute[75234]: DEBUG os_brick.utils [None req-d0cfb3d1-066f-4604-ac70-d86765dfbd73 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:176}}
Nov 29 03:32:34 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:32:34 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.014s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:32:34 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[1cafc72a-1565-4eb1-914a-c7e612b58b70]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:dd1368c5ba2c\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:32:34 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt -v / -n -o SOURCE {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:32:34 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt -v / -n -o SOURCE" returned: 0 in 0.015s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:32:34 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[69f064b2-34e9-43eb-bfb0-2266e738abe1]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:32:34 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:32:34 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.016s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:32:34 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[b895fad3-e3ce-436a-ad32-596b63048fef]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:32:34 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[6cc65097-797f-44df-a708-0e0e986f3cfd]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:32:34 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-d0cfb3d1-066f-4604-ac70-d86765dfbd73 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Running cmd (subprocess): nvme version {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:32:34 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-d0cfb3d1-066f-4604-ac70-d86765dfbd73 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] 'nvme version' failed. Not Retrying. {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:473}}
Nov 29 03:32:34 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.nvmeof [None req-d0cfb3d1-066f-4604-ac70-d86765dfbd73 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] nvme not present on system {{(pid=75234) nvme_present /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/nvmeof.py:756}}
Nov 29 03:32:34 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:32:34 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:473}}
Nov 29 03:32:34 user nova-compute[75234]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Nov 29 03:32:34 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[684bbc02-0092-411a-b7fd-ace61d92b83f]: (4, '') {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:32:34 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.lightos [None req-d0cfb3d1-066f-4604-ac70-d86765dfbd73 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=75234) find_dsc /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/lightos.py:98}}
Nov 29 03:32:34 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.lightos [None req-d0cfb3d1-066f-4604-ac70-d86765dfbd73 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=75234) get_connector_properties /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/lightos.py:76}}
Nov 29 03:32:34 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.lightos [None req-d0cfb3d1-066f-4604-ac70-d86765dfbd73 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] LIGHTOS: no hostnqn found. {{(pid=75234) get_connector_properties /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/lightos.py:84}}
Nov 29 03:32:34 user nova-compute[75234]: DEBUG os_brick.utils [None req-d0cfb3d1-066f-4604-ac70-d86765dfbd73 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] <== get_connector_properties: return (117ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:dd1368c5ba2c', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:203}}
Nov 29 03:32:34 user nova-compute[75234]: DEBUG nova.virt.block_device [None req-d0cfb3d1-066f-4604-ac70-d86765dfbd73 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] [instance: ebc3e3b2-037d-43ef-9426-caebe298ee3e] Updating existing volume attachment record: d004fe65-1313-4db0-8bed-ff0a443bf010 {{(pid=75234) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Nov 29 03:32:34 user nova-compute[75234]: DEBUG nova.network.neutron [req-537a380e-8241-49f5-bfb4-06d651ccc7cd req-f6beeba1-9278-4000-86d2-06658efde833 service nova] [instance: 884589ec-b86c-4094-9a3f-54b641311d62] Updated VIF entry in instance network info cache for port 1deddce5-7279-44f9-9f1e-cd9ffddc5d6c. {{(pid=75234) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3481}}
Nov 29 03:32:34 user nova-compute[75234]: DEBUG nova.network.neutron [req-537a380e-8241-49f5-bfb4-06d651ccc7cd req-f6beeba1-9278-4000-86d2-06658efde833 service nova] [instance: 884589ec-b86c-4094-9a3f-54b641311d62] Updating instance_info_cache with network_info: [{"id": "1deddce5-7279-44f9-9f1e-cd9ffddc5d6c", "address": "fa:16:3e:c5:44:c3", "network": {"id": "a3f397d3-fd75-4a6e-a815-cfcedcca15fb", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-678570085-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "81a265d3c15b4e9d826b35ddbd0ff443", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap1deddce5-72", "ovs_interfaceid": "1deddce5-7279-44f9-9f1e-cd9ffddc5d6c", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:32:34 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-537a380e-8241-49f5-bfb4-06d651ccc7cd req-f6beeba1-9278-4000-86d2-06658efde833 service nova] Releasing lock "refresh_cache-884589ec-b86c-4094-9a3f-54b641311d62" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:32:36 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:32:39 user nova-compute[75234]: DEBUG nova.compute.manager [None req-3f2c9326-e9dc-46ad-b028-3d5aef2c14ea tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: 884589ec-b86c-4094-9a3f-54b641311d62] Start spawning the instance on the hypervisor. {{(pid=75234) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2628}}
Nov 29 03:32:39 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-3f2c9326-e9dc-46ad-b028-3d5aef2c14ea tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: 884589ec-b86c-4094-9a3f-54b641311d62] Creating instance directory {{(pid=75234) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4727}}
Nov 29 03:32:39 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-3f2c9326-e9dc-46ad-b028-3d5aef2c14ea tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: 884589ec-b86c-4094-9a3f-54b641311d62] Creating image(s)
Nov 29 03:32:39 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-3f2c9326-e9dc-46ad-b028-3d5aef2c14ea tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: 884589ec-b86c-4094-9a3f-54b641311d62] Did not create local disks {{(pid=75234) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4863}}
Nov 29 03:32:39 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-3f2c9326-e9dc-46ad-b028-3d5aef2c14ea tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: 884589ec-b86c-4094-9a3f-54b641311d62] Ensure instance console log exists: /opt/stack/data/nova/instances/884589ec-b86c-4094-9a3f-54b641311d62/console.log {{(pid=75234) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4613}}
Nov 29 03:32:39 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-3f2c9326-e9dc-46ad-b028-3d5aef2c14ea tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:32:39 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-3f2c9326-e9dc-46ad-b028-3d5aef2c14ea tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:32:39 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-3f2c9326-e9dc-46ad-b028-3d5aef2c14ea tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:32:39 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-3f2c9326-e9dc-46ad-b028-3d5aef2c14ea tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: 884589ec-b86c-4094-9a3f-54b641311d62] Start _get_guest_xml network_info=[{"id": "1deddce5-7279-44f9-9f1e-cd9ffddc5d6c", "address": "fa:16:3e:c5:44:c3", "network": {"id": "a3f397d3-fd75-4a6e-a815-cfcedcca15fb", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-678570085-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "81a265d3c15b4e9d826b35ddbd0ff443", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap1deddce5-72", "ovs_interfaceid": "1deddce5-7279-44f9-9f1e-cd9ffddc5d6c", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, '/dev/vda': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum=<?>,container_format=<?>,created_at=<?>,direct_url=<?>,disk_format=<?>,id=<?>,min_disk=0,min_ram=0,name=<?>,owner=<?>,properties=ImageMetaProps,protected=<?>,size=1073741824,status='active',tags=<?>,updated_at=<?>,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [], 'ephemerals': [], 'block_device_mapping': [{'mount_device': '/dev/vda', 'attachment_id': 'e4b3ea0b-a686-4154-9d19-42280681db55', 'disk_bus': 'virtio', 'device_type': 'disk', 'boot_index': 0, 'guest_format': None, 'delete_on_termination': True, 'connection_info': {'driver_volume_type': 'iscsi', 'data': {'target_iqn': 'iqn.2010-10.org.openstack:a3d30ae4-ca6d-4e06-8fc9-caf4d91ef900', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'foMjDFpN', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}, 'status': 'reserved', 'instance': '884589ec-b86c-4094-9a3f-54b641311d62', 'attached_at': '', 'detached_at': '', 'volume_id': 'a3d30ae4-ca6d-4e06-8fc9-caf4d91ef900', 'serial': 'a3d30ae4-ca6d-4e06-8fc9-caf4d91ef900'}, 'volume_type': None} {{(pid=75234) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7562}}
Nov 29 03:32:39 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-3f2c9326-e9dc-46ad-b028-3d5aef2c14ea tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:32:39 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-3f2c9326-e9dc-46ad-b028-3d5aef2c14ea tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:32:39 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-3f2c9326-e9dc-46ad-b028-3d5aef2c14ea tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Searching host: 'user' for CPU controller through CGroups V1... {{(pid=75234) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1652}}
Nov 29 03:32:39 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-3f2c9326-e9dc-46ad-b028-3d5aef2c14ea tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] CPU controller missing on host. {{(pid=75234) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1662}}
Nov 29 03:32:39 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-3f2c9326-e9dc-46ad-b028-3d5aef2c14ea tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Searching host: 'user' for CPU controller through CGroups V2... {{(pid=75234) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1671}}
Nov 29 03:32:39 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-3f2c9326-e9dc-46ad-b028-3d5aef2c14ea tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] CPU controller found on host. {{(pid=75234) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1678}}
Nov 29 03:32:39 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-3f2c9326-e9dc-46ad-b028-3d5aef2c14ea tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=75234) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5400}}
Nov 29 03:32:39 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-3f2c9326-e9dc-46ad-b028-3d5aef2c14ea tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-11-29T03:02:04Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='1',id=6,is_public=True,memory_mb=512,name='m1.tiny',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum=<?>,container_format=<?>,created_at=<?>,direct_url=<?>,disk_format=<?>,id=<?>,min_disk=0,min_ram=0,name=<?>,owner=<?>,properties=ImageMetaProps,protected=<?>,size=1073741824,status='active',tags=<?>,updated_at=<?>,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:563}}
Nov 29 03:32:39 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-3f2c9326-e9dc-46ad-b028-3d5aef2c14ea tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Flavor limits 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:348}}
Nov 29 03:32:39 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-3f2c9326-e9dc-46ad-b028-3d5aef2c14ea tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Image limits 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:352}}
Nov 29 03:32:39 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-3f2c9326-e9dc-46ad-b028-3d5aef2c14ea tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Flavor pref 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:388}}
Nov 29 03:32:39 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-3f2c9326-e9dc-46ad-b028-3d5aef2c14ea tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Image pref 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:392}}
Nov 29 03:32:39 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-3f2c9326-e9dc-46ad-b028-3d5aef2c14ea tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:430}}
Nov 29 03:32:39 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-3f2c9326-e9dc-46ad-b028-3d5aef2c14ea tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:569}}
Nov 29 03:32:39 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-3f2c9326-e9dc-46ad-b028-3d5aef2c14ea tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=75234) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:471}}
Nov 29 03:32:39 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-3f2c9326-e9dc-46ad-b028-3d5aef2c14ea tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Got 1 possible topologies {{(pid=75234) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:501}}
Nov 29 03:32:39 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-3f2c9326-e9dc-46ad-b028-3d5aef2c14ea tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:575}}
Nov 29 03:32:39 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-3f2c9326-e9dc-46ad-b028-3d5aef2c14ea tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:577}}
Nov 29 03:32:39 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-3f2c9326-e9dc-46ad-b028-3d5aef2c14ea tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: 884589ec-b86c-4094-9a3f-54b641311d62] Calling os-brick to attach iSCSI Volume {{(pid=75234) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Nov 29 03:32:39 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-3f2c9326-e9dc-46ad-b028-3d5aef2c14ea tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f8a5050c9a0>, 'connection_properties': {'target_iqn': 'iqn.2010-10.org.openstack:a3d30ae4-ca6d-4e06-8fc9-caf4d91ef900', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'foMjDFpN', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:176}}
Nov 29 03:32:39 user nova-compute[75234]: WARNING os_brick.initiator.connectors.base [None req-3f2c9326-e9dc-46ad-b028-3d5aef2c14ea tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Nov 29 03:32:39 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-3f2c9326-e9dc-46ad-b028-3d5aef2c14ea tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:68}}
Nov 29 03:32:39 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-3f2c9326-e9dc-46ad-b028-3d5aef2c14ea tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:73}}
Nov 29 03:32:39 user nova-compute[75234]: WARNING os_brick.initiator.connectors.base [None req-3f2c9326-e9dc-46ad-b028-3d5aef2c14ea tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Nov 29 03:32:39 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-3f2c9326-e9dc-46ad-b028-3d5aef2c14ea tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:a3d30ae4-ca6d-4e06-8fc9-caf4d91ef900" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:68}}
Nov 29 03:32:39 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-3f2c9326-e9dc-46ad-b028-3d5aef2c14ea tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:a3d30ae4-ca6d-4e06-8fc9-caf4d91ef900" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:73}}
Nov 29 03:32:39 user nova-compute[75234]: INFO os_brick.initiator.connectors.iscsi [None req-3f2c9326-e9dc-46ad-b028-3d5aef2c14ea tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Nov 29 03:32:39 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:a3d30ae4-ca6d-4e06-8fc9-caf4d91ef900 -p 172.16.0.220:3260 {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:32:39 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:a3d30ae4-ca6d-4e06-8fc9-caf4d91ef900 -p 172.16.0.220:3260" returned: 21 in 0.018s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:32:39 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[1af47d4e-512d-4c0c-bdbc-ff564868ca77]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:32:39 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-3f2c9326-e9dc-46ad-b028-3d5aef2c14ea tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Nov 29 03:32:39 user nova-compute[75234]:  {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:32:39 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:a3d30ae4-ca6d-4e06-8fc9-caf4d91ef900 -p 172.16.0.220:3260 --interface default --op new {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:32:39 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:a3d30ae4-ca6d-4e06-8fc9-caf4d91ef900 -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.019s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:32:39 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[465ea91f-2dd0-408b-a94c-d045c100cb60]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:a3d30ae4-ca6d-4e06-8fc9-caf4d91ef900] added\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:32:39 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-3f2c9326-e9dc-46ad-b028-3d5aef2c14ea tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:a3d30ae4-ca6d-4e06-8fc9-caf4d91ef900] added
Nov 29 03:32:39 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:32:39 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:a3d30ae4-ca6d-4e06-8fc9-caf4d91ef900 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:32:39 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:a3d30ae4-ca6d-4e06-8fc9-caf4d91ef900 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.017s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:32:39 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[3670b380-c7aa-4ed1-9dc0-1c4d32399430]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:32:39 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-3f2c9326-e9dc-46ad-b028-3d5aef2c14ea tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:32:39 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:a3d30ae4-ca6d-4e06-8fc9-caf4d91ef900 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:32:39 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:a3d30ae4-ca6d-4e06-8fc9-caf4d91ef900 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.018s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:32:39 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[8b42506a-c7c2-4263-88c3-6b164bf03201]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:32:39 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-3f2c9326-e9dc-46ad-b028-3d5aef2c14ea tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:32:39 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:a3d30ae4-ca6d-4e06-8fc9-caf4d91ef900 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v foMjDFpN {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:32:39 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:a3d30ae4-ca6d-4e06-8fc9-caf4d91ef900 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v foMjDFpN" returned: 0 in 0.018s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:32:39 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[0a90950f-bc78-487d-a10e-3438ba0743ad]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:32:39 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-3f2c9326-e9dc-46ad-b028-3d5aef2c14ea tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'foMjDFpN'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:32:39 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:a3d30ae4-ca6d-4e06-8fc9-caf4d91ef900 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:32:39 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:a3d30ae4-ca6d-4e06-8fc9-caf4d91ef900 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.018s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:32:39 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[e21160a3-d3ae-49a1-800d-ad6041336a62]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:32:39 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-3f2c9326-e9dc-46ad-b028-3d5aef2c14ea tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:32:39 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:32:39 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.020s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:32:39 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[ab1e3ea4-5ee7-421d-8943-b759d507381c]: (4, ('tcp: [42] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0 (non-flash)\ntcp: [53] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d7d89638-2f88-47e2-a43c-b37e8919ecbc (non-flash)\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:32:39 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-3f2c9326-e9dc-46ad-b028-3d5aef2c14ea tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [42] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0 (non-flash)
Nov 29 03:32:39 user nova-compute[75234]: tcp: [53] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d7d89638-2f88-47e2-a43c-b37e8919ecbc (non-flash)
Nov 29 03:32:39 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:32:39 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-3f2c9326-e9dc-46ad-b028-3d5aef2c14ea tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] iscsi session list stdout=tcp: [42] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0 (non-flash)
Nov 29 03:32:39 user nova-compute[75234]: tcp: [53] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d7d89638-2f88-47e2-a43c-b37e8919ecbc (non-flash)
Nov 29 03:32:39 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsi_session /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Nov 29 03:32:39 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:a3d30ae4-ca6d-4e06-8fc9-caf4d91ef900 -p 172.16.0.220:3260 --login {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:32:39 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:a3d30ae4-ca6d-4e06-8fc9-caf4d91ef900 -p 172.16.0.220:3260 --login" returned: 0 in 0.061s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:32:39 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[8010984c-2466-40ac-9a9f-e0223274a398]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:a3d30ae4-ca6d-4e06-8fc9-caf4d91ef900, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:a3d30ae4-ca6d-4e06-8fc9-caf4d91ef900, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:32:39 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-3f2c9326-e9dc-46ad-b028-3d5aef2c14ea tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:a3d30ae4-ca6d-4e06-8fc9-caf4d91ef900, portal: 172.16.0.220,3260]
Nov 29 03:32:39 user nova-compute[75234]: Login to [iface: default, target: iqn.2010-10.org.openstack:a3d30ae4-ca6d-4e06-8fc9-caf4d91ef900, portal: 172.16.0.220,3260] successful.
Nov 29 03:32:39 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:32:39 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:a3d30ae4-ca6d-4e06-8fc9-caf4d91ef900 -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:32:39 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:a3d30ae4-ca6d-4e06-8fc9-caf4d91ef900 -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.023s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:32:39 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[0707babb-c99b-4307-ac07-7c9d1370187e]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:32:39 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-3f2c9326-e9dc-46ad-b028-3d5aef2c14ea tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:32:39 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:32:39 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.020s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:32:39 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[4a72053a-ba80-4b06-9545-4f0a959231f4]: (4, ('tcp: [42] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0 (non-flash)\ntcp: [53] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d7d89638-2f88-47e2-a43c-b37e8919ecbc (non-flash)\ntcp: [54] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a3d30ae4-ca6d-4e06-8fc9-caf4d91ef900 (non-flash)\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:32:39 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-3f2c9326-e9dc-46ad-b028-3d5aef2c14ea tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [42] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0 (non-flash)
Nov 29 03:32:39 user nova-compute[75234]: tcp: [53] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d7d89638-2f88-47e2-a43c-b37e8919ecbc (non-flash)
Nov 29 03:32:39 user nova-compute[75234]: tcp: [54] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a3d30ae4-ca6d-4e06-8fc9-caf4d91ef900 (non-flash)
Nov 29 03:32:39 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:32:39 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-3f2c9326-e9dc-46ad-b028-3d5aef2c14ea tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] iscsi session list stdout=tcp: [42] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0 (non-flash)
Nov 29 03:32:39 user nova-compute[75234]: tcp: [53] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d7d89638-2f88-47e2-a43c-b37e8919ecbc (non-flash)
Nov 29 03:32:39 user nova-compute[75234]: tcp: [54] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a3d30ae4-ca6d-4e06-8fc9-caf4d91ef900 (non-flash)
Nov 29 03:32:39 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsi_session /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Nov 29 03:32:39 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-3f2c9326-e9dc-46ad-b028-3d5aef2c14ea tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:a3d30ae4-ca6d-4e06-8fc9-caf4d91ef900" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.273s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:87}}
Nov 29 03:32:39 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-3f2c9326-e9dc-46ad-b028-3d5aef2c14ea tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Connected to 172.16.0.220:3260 {{(pid=75234) _connect_vol /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:644}}
Nov 29 03:32:39 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-3f2c9326-e9dc-46ad-b028-3d5aef2c14ea tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] HCTL ('35', '-', '-', 0) found on session 54 with lun 0 {{(pid=75234) get_hctl /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:770}}
Nov 29 03:32:39 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-3f2c9326-e9dc-46ad-b028-3d5aef2c14ea tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Scanning host 35 c: -, t: -, l: 0) {{(pid=75234) scan_iscsi /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:795}}
Nov 29 03:32:39 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host35/scan {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:32:39 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host35/scan" returned: 0 in 0.016s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:32:39 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[daf6e452-7532-4199-ab67-507a7fbc45e6]: (4, ('- - 0', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:32:39 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-3f2c9326-e9dc-46ad-b028-3d5aef2c14ea tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Searching for a device in session 54 and hctl ['35', '*', '*', 0] yield: None {{(pid=75234) device_name_by_hctl /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:789}}
Nov 29 03:32:40 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-3f2c9326-e9dc-46ad-b028-3d5aef2c14ea tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Searching for a device in session 54 and hctl ['35', '*', '*', 0] yield: sdd {{(pid=75234) device_name_by_hctl /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:789}}
Nov 29 03:32:40 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-3f2c9326-e9dc-46ad-b028-3d5aef2c14ea tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Connected to sdd using {'target_iqn': 'iqn.2010-10.org.openstack:a3d30ae4-ca6d-4e06-8fc9-caf4d91ef900', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'foMjDFpN', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False} {{(pid=75234) _connect_vol /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:672}}
Nov 29 03:32:40 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-3f2c9326-e9dc-46ad-b028-3d5aef2c14ea tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.303s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:87}}
Nov 29 03:32:40 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-3f2c9326-e9dc-46ad-b028-3d5aef2c14ea tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] <== connect_volume: return (1305ms) {'type': 'block', 'scsi_wwn': '23833663531653637', 'path': '/dev/sdd'} {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:203}}
Nov 29 03:32:40 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-3f2c9326-e9dc-46ad-b028-3d5aef2c14ea tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: 884589ec-b86c-4094-9a3f-54b641311d62] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '23833663531653637', 'path': '/dev/sdd'} {{(pid=75234) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Nov 29 03:32:40 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-3f2c9326-e9dc-46ad-b028-3d5aef2c14ea tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-11-29T03:32:30Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-TestVolumeBootPattern-volume-backed-server-1667553757',display_name='tempest-TestVolumeBootPattern-volume-backed-server-1667553757',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-testvolumebootpattern-volume-backed-server-1667553757',id=48,image_ref='',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBEkw5vg+737ESWS3QhcThM/Ju+W7vDKKOHu4jAL47D+/LrumwViYptAXsVBquZbfetengtms1FvxACUCBNg/diM/Oq5FnXWy4aGB+8HpGsjosjG8tDfDnQgbi6ZUnh66qQ==',key_name='tempest-keypair-185724684',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='81a265d3c15b4e9d826b35ddbd0ff443',ramdisk_id='',reservation_id='r-699s07kq',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TestVolumeBootPattern-1244492388',owner_user_name='tempest-TestVolumeBootPattern-1244492388-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-11-29T03:32:32Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='7d1f1f11a9aa4c1db8627413eb963fa4',uuid=884589ec-b86c-4094-9a3f-54b641311d62,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "1deddce5-7279-44f9-9f1e-cd9ffddc5d6c", "address": "fa:16:3e:c5:44:c3", "network": {"id": "a3f397d3-fd75-4a6e-a815-cfcedcca15fb", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-678570085-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "81a265d3c15b4e9d826b35ddbd0ff443", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap1deddce5-72", "ovs_interfaceid": "1deddce5-7279-44f9-9f1e-cd9ffddc5d6c", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=75234) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Nov 29 03:32:40 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-3f2c9326-e9dc-46ad-b028-3d5aef2c14ea tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Converting VIF {"id": "1deddce5-7279-44f9-9f1e-cd9ffddc5d6c", "address": "fa:16:3e:c5:44:c3", "network": {"id": "a3f397d3-fd75-4a6e-a815-cfcedcca15fb", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-678570085-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "81a265d3c15b4e9d826b35ddbd0ff443", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap1deddce5-72", "ovs_interfaceid": "1deddce5-7279-44f9-9f1e-cd9ffddc5d6c", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:32:40 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-3f2c9326-e9dc-46ad-b028-3d5aef2c14ea tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:c5:44:c3,bridge_name='br-int',has_traffic_filtering=True,id=1deddce5-7279-44f9-9f1e-cd9ffddc5d6c,network=Network(a3f397d3-fd75-4a6e-a815-cfcedcca15fb),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap1deddce5-72') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:32:40 user nova-compute[75234]: DEBUG nova.objects.instance [None req-3f2c9326-e9dc-46ad-b028-3d5aef2c14ea tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Lazy-loading 'pci_devices' on Instance uuid 884589ec-b86c-4094-9a3f-54b641311d62 {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:32:40 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-3f2c9326-e9dc-46ad-b028-3d5aef2c14ea tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: 884589ec-b86c-4094-9a3f-54b641311d62] End _get_guest_xml xml=<domain type="kvm">
Nov 29 03:32:40 user nova-compute[75234]:   <uuid>884589ec-b86c-4094-9a3f-54b641311d62</uuid>
Nov 29 03:32:40 user nova-compute[75234]:   <name>instance-00000030</name>
Nov 29 03:32:40 user nova-compute[75234]:   <memory>524288</memory>
Nov 29 03:32:40 user nova-compute[75234]:   <vcpu>1</vcpu>
Nov 29 03:32:40 user nova-compute[75234]:   <metadata>
Nov 29 03:32:40 user nova-compute[75234]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Nov 29 03:32:40 user nova-compute[75234]:       <nova:package version="0.0.0"/>
Nov 29 03:32:40 user nova-compute[75234]:       <nova:name>tempest-TestVolumeBootPattern-volume-backed-server-1667553757</nova:name>
Nov 29 03:32:40 user nova-compute[75234]:       <nova:creationTime>2023-11-29 03:32:39</nova:creationTime>
Nov 29 03:32:40 user nova-compute[75234]:       <nova:flavor name="m1.tiny">
Nov 29 03:32:40 user nova-compute[75234]:         <nova:memory>512</nova:memory>
Nov 29 03:32:40 user nova-compute[75234]:         <nova:disk>1</nova:disk>
Nov 29 03:32:40 user nova-compute[75234]:         <nova:swap>0</nova:swap>
Nov 29 03:32:40 user nova-compute[75234]:         <nova:ephemeral>0</nova:ephemeral>
Nov 29 03:32:40 user nova-compute[75234]:         <nova:vcpus>1</nova:vcpus>
Nov 29 03:32:40 user nova-compute[75234]:       </nova:flavor>
Nov 29 03:32:40 user nova-compute[75234]:       <nova:owner>
Nov 29 03:32:40 user nova-compute[75234]:         <nova:user uuid="7d1f1f11a9aa4c1db8627413eb963fa4">tempest-TestVolumeBootPattern-1244492388-project-member</nova:user>
Nov 29 03:32:40 user nova-compute[75234]:         <nova:project uuid="81a265d3c15b4e9d826b35ddbd0ff443">tempest-TestVolumeBootPattern-1244492388</nova:project>
Nov 29 03:32:40 user nova-compute[75234]:       </nova:owner>
Nov 29 03:32:40 user nova-compute[75234]:       <nova:ports>
Nov 29 03:32:40 user nova-compute[75234]:         <nova:port uuid="1deddce5-7279-44f9-9f1e-cd9ffddc5d6c">
Nov 29 03:32:40 user nova-compute[75234]:           <nova:ip type="fixed" address="10.0.0.6" ipVersion="4"/>
Nov 29 03:32:40 user nova-compute[75234]:         </nova:port>
Nov 29 03:32:40 user nova-compute[75234]:       </nova:ports>
Nov 29 03:32:40 user nova-compute[75234]:     </nova:instance>
Nov 29 03:32:40 user nova-compute[75234]:   </metadata>
Nov 29 03:32:40 user nova-compute[75234]:   <sysinfo type="smbios">
Nov 29 03:32:40 user nova-compute[75234]:     <system>
Nov 29 03:32:40 user nova-compute[75234]:       <entry name="manufacturer">OpenStack Foundation</entry>
Nov 29 03:32:40 user nova-compute[75234]:       <entry name="product">OpenStack Nova</entry>
Nov 29 03:32:40 user nova-compute[75234]:       <entry name="version">0.0.0</entry>
Nov 29 03:32:40 user nova-compute[75234]:       <entry name="serial">884589ec-b86c-4094-9a3f-54b641311d62</entry>
Nov 29 03:32:40 user nova-compute[75234]:       <entry name="uuid">884589ec-b86c-4094-9a3f-54b641311d62</entry>
Nov 29 03:32:40 user nova-compute[75234]:       <entry name="family">Virtual Machine</entry>
Nov 29 03:32:40 user nova-compute[75234]:     </system>
Nov 29 03:32:40 user nova-compute[75234]:   </sysinfo>
Nov 29 03:32:40 user nova-compute[75234]:   <os>
Nov 29 03:32:40 user nova-compute[75234]:     <type arch="x86_64" machine="pc">hvm</type>
Nov 29 03:32:40 user nova-compute[75234]:     <boot dev="hd"/>
Nov 29 03:32:40 user nova-compute[75234]:     <smbios mode="sysinfo"/>
Nov 29 03:32:40 user nova-compute[75234]:   </os>
Nov 29 03:32:40 user nova-compute[75234]:   <features>
Nov 29 03:32:40 user nova-compute[75234]:     <acpi/>
Nov 29 03:32:40 user nova-compute[75234]:     <apic/>
Nov 29 03:32:40 user nova-compute[75234]:     <vmcoreinfo/>
Nov 29 03:32:40 user nova-compute[75234]:   </features>
Nov 29 03:32:40 user nova-compute[75234]:   <clock offset="utc">
Nov 29 03:32:40 user nova-compute[75234]:     <timer name="pit" tickpolicy="delay"/>
Nov 29 03:32:40 user nova-compute[75234]:     <timer name="rtc" tickpolicy="catchup"/>
Nov 29 03:32:40 user nova-compute[75234]:     <timer name="hpet" present="no"/>
Nov 29 03:32:40 user nova-compute[75234]:   </clock>
Nov 29 03:32:40 user nova-compute[75234]:   <cpu mode="custom" match="exact">
Nov 29 03:32:40 user nova-compute[75234]:     <model>Nehalem</model>
Nov 29 03:32:40 user nova-compute[75234]:     <topology sockets="1" cores="1" threads="1"/>
Nov 29 03:32:40 user nova-compute[75234]:   </cpu>
Nov 29 03:32:40 user nova-compute[75234]:   <devices>
Nov 29 03:32:40 user nova-compute[75234]:     <disk type="block" device="disk">
Nov 29 03:32:40 user nova-compute[75234]:       <driver name="qemu" type="raw" cache="none" io="native"/>
Nov 29 03:32:40 user nova-compute[75234]:       <source dev="/dev/sdd"/>
Nov 29 03:32:40 user nova-compute[75234]:       <target dev="vda" bus="virtio"/>
Nov 29 03:32:40 user nova-compute[75234]:       <serial>a3d30ae4-ca6d-4e06-8fc9-caf4d91ef900</serial>
Nov 29 03:32:40 user nova-compute[75234]:     </disk>
Nov 29 03:32:40 user nova-compute[75234]:     <interface type="ethernet">
Nov 29 03:32:40 user nova-compute[75234]:       <mac address="fa:16:3e:c5:44:c3"/>
Nov 29 03:32:40 user nova-compute[75234]:       <model type="virtio"/>
Nov 29 03:32:40 user nova-compute[75234]:       <mtu size="1442"/>
Nov 29 03:32:40 user nova-compute[75234]:       <target dev="tap1deddce5-72"/>
Nov 29 03:32:40 user nova-compute[75234]:     </interface>
Nov 29 03:32:40 user nova-compute[75234]:     <serial type="pty">
Nov 29 03:32:40 user nova-compute[75234]:       <log file="/opt/stack/data/nova/instances/884589ec-b86c-4094-9a3f-54b641311d62/console.log" append="off"/>
Nov 29 03:32:40 user nova-compute[75234]:     </serial>
Nov 29 03:32:40 user nova-compute[75234]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Nov 29 03:32:40 user nova-compute[75234]:     <video>
Nov 29 03:32:40 user nova-compute[75234]:       <model type="virtio"/>
Nov 29 03:32:40 user nova-compute[75234]:     </video>
Nov 29 03:32:40 user nova-compute[75234]:     <rng model="virtio">
Nov 29 03:32:40 user nova-compute[75234]:       <backend model="random">/dev/urandom</backend>
Nov 29 03:32:40 user nova-compute[75234]:     </rng>
Nov 29 03:32:40 user nova-compute[75234]:     <controller type="usb" index="0" model="none"/>
Nov 29 03:32:40 user nova-compute[75234]:     <memballoon model="virtio">
Nov 29 03:32:40 user nova-compute[75234]:       <stats period="10"/>
Nov 29 03:32:40 user nova-compute[75234]:     </memballoon>
Nov 29 03:32:40 user nova-compute[75234]:   </devices>
Nov 29 03:32:40 user nova-compute[75234]: </domain>
Nov 29 03:32:40 user nova-compute[75234]:  {{(pid=75234) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7568}}
Nov 29 03:32:40 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-3f2c9326-e9dc-46ad-b028-3d5aef2c14ea tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-11-29T03:32:30Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-TestVolumeBootPattern-volume-backed-server-1667553757',display_name='tempest-TestVolumeBootPattern-volume-backed-server-1667553757',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-testvolumebootpattern-volume-backed-server-1667553757',id=48,image_ref='',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBEkw5vg+737ESWS3QhcThM/Ju+W7vDKKOHu4jAL47D+/LrumwViYptAXsVBquZbfetengtms1FvxACUCBNg/diM/Oq5FnXWy4aGB+8HpGsjosjG8tDfDnQgbi6ZUnh66qQ==',key_name='tempest-keypair-185724684',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='81a265d3c15b4e9d826b35ddbd0ff443',ramdisk_id='',reservation_id='r-699s07kq',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TestVolumeBootPattern-1244492388',owner_user_name='tempest-TestVolumeBootPattern-1244492388-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-11-29T03:32:32Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='7d1f1f11a9aa4c1db8627413eb963fa4',uuid=884589ec-b86c-4094-9a3f-54b641311d62,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "1deddce5-7279-44f9-9f1e-cd9ffddc5d6c", "address": "fa:16:3e:c5:44:c3", "network": {"id": "a3f397d3-fd75-4a6e-a815-cfcedcca15fb", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-678570085-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "81a265d3c15b4e9d826b35ddbd0ff443", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap1deddce5-72", "ovs_interfaceid": "1deddce5-7279-44f9-9f1e-cd9ffddc5d6c", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Nov 29 03:32:40 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-3f2c9326-e9dc-46ad-b028-3d5aef2c14ea tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Converting VIF {"id": "1deddce5-7279-44f9-9f1e-cd9ffddc5d6c", "address": "fa:16:3e:c5:44:c3", "network": {"id": "a3f397d3-fd75-4a6e-a815-cfcedcca15fb", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-678570085-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "81a265d3c15b4e9d826b35ddbd0ff443", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap1deddce5-72", "ovs_interfaceid": "1deddce5-7279-44f9-9f1e-cd9ffddc5d6c", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:32:40 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-3f2c9326-e9dc-46ad-b028-3d5aef2c14ea tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:c5:44:c3,bridge_name='br-int',has_traffic_filtering=True,id=1deddce5-7279-44f9-9f1e-cd9ffddc5d6c,network=Network(a3f397d3-fd75-4a6e-a815-cfcedcca15fb),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap1deddce5-72') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:32:40 user nova-compute[75234]: DEBUG os_vif [None req-3f2c9326-e9dc-46ad-b028-3d5aef2c14ea tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:c5:44:c3,bridge_name='br-int',has_traffic_filtering=True,id=1deddce5-7279-44f9-9f1e-cd9ffddc5d6c,network=Network(a3f397d3-fd75-4a6e-a815-cfcedcca15fb),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap1deddce5-72') {{(pid=75234) plug /opt/stack/data/venv/lib/python3.10/site-packages/os_vif/__init__.py:76}}
Nov 29 03:32:40 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:32:40 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:32:40 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Nov 29 03:32:40 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:32:40 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbCreateCommand(_result=None, table=QoS, columns={'type': 'linux-noop', 'external_ids': {'id': 'ea3344a4-2361-556d-905a-7d966dffe1ac', '_type': 'linux-noop'}}, row=False) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:32:40 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:32:40 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:32:40 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:32:40 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tap1deddce5-72, may_exist=True, interface_attrs={}) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:32:40 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Port, record=tap1deddce5-72, col_values=(('qos', UUID('90c3f023-0aab-4515-8ef4-215807178391')),), if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:32:40 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DbSetCommand(_result=None, table=Interface, record=tap1deddce5-72, col_values=(('external_ids', {'iface-id': '1deddce5-7279-44f9-9f1e-cd9ffddc5d6c', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:c5:44:c3', 'vm-uuid': '884589ec-b86c-4094-9a3f-54b641311d62'}),), if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:32:40 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:32:40 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:32:41 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:32:41 user nova-compute[75234]: INFO os_vif [None req-3f2c9326-e9dc-46ad-b028-3d5aef2c14ea tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:c5:44:c3,bridge_name='br-int',has_traffic_filtering=True,id=1deddce5-7279-44f9-9f1e-cd9ffddc5d6c,network=Network(a3f397d3-fd75-4a6e-a815-cfcedcca15fb),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap1deddce5-72')
Nov 29 03:32:41 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-3f2c9326-e9dc-46ad-b028-3d5aef2c14ea tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] No BDM found with device name vda, not building metadata. {{(pid=75234) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12089}}
Nov 29 03:32:41 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-3f2c9326-e9dc-46ad-b028-3d5aef2c14ea tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] No VIF found with MAC fa:16:3e:c5:44:c3, not building metadata {{(pid=75234) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Nov 29 03:32:41 user nova-compute[75234]: DEBUG nova.compute.manager [None req-d0cfb3d1-066f-4604-ac70-d86765dfbd73 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] [instance: ebc3e3b2-037d-43ef-9426-caebe298ee3e] Start spawning the instance on the hypervisor. {{(pid=75234) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2628}}
Nov 29 03:32:41 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-d0cfb3d1-066f-4604-ac70-d86765dfbd73 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] [instance: ebc3e3b2-037d-43ef-9426-caebe298ee3e] Creating instance directory {{(pid=75234) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4727}}
Nov 29 03:32:41 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-d0cfb3d1-066f-4604-ac70-d86765dfbd73 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] [instance: ebc3e3b2-037d-43ef-9426-caebe298ee3e] Creating image(s)
Nov 29 03:32:41 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-d0cfb3d1-066f-4604-ac70-d86765dfbd73 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] [instance: ebc3e3b2-037d-43ef-9426-caebe298ee3e] Did not create local disks {{(pid=75234) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4863}}
Nov 29 03:32:41 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-d0cfb3d1-066f-4604-ac70-d86765dfbd73 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] [instance: ebc3e3b2-037d-43ef-9426-caebe298ee3e] Ensure instance console log exists: /opt/stack/data/nova/instances/ebc3e3b2-037d-43ef-9426-caebe298ee3e/console.log {{(pid=75234) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4613}}
Nov 29 03:32:41 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-d0cfb3d1-066f-4604-ac70-d86765dfbd73 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:32:41 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-d0cfb3d1-066f-4604-ac70-d86765dfbd73 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:32:41 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-d0cfb3d1-066f-4604-ac70-d86765dfbd73 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:32:41 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-d0cfb3d1-066f-4604-ac70-d86765dfbd73 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] [instance: ebc3e3b2-037d-43ef-9426-caebe298ee3e] Start _get_guest_xml network_info=[{"id": "bd0a4e5b-69cd-4f5c-ae22-994e4d8f2682", "address": "fa:16:3e:c2:aa:3e", "network": {"id": "4d086e08-7798-4cfe-9365-86be6f0a52d2", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-305624008-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "db00bcd0c82c44f1bda739e9857f6e84", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapbd0a4e5b-69", "ovs_interfaceid": "bd0a4e5b-69cd-4f5c-ae22-994e4d8f2682", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, '/dev/vda': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum='c8fc807773e5354afe61636071771906',container_format='bare',created_at=2023-11-29T03:02:15Z,direct_url=<?>,disk_format='qcow2',id=96c30ba7-9b73-4a30-8ba4-7138879573b7,min_disk=0,min_ram=0,name='cirros-0.6.2-x86_64-disk',owner='b18e380921c748b78517b6f1211cd0f1',properties=ImageMetaProps,protected=<?>,size=21430272,status='active',tags=<?>,updated_at=2023-11-29T03:02:17Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [], 'ephemerals': [], 'block_device_mapping': [{'mount_device': '/dev/vda', 'attachment_id': 'd004fe65-1313-4db0-8bed-ff0a443bf010', 'disk_bus': 'virtio', 'device_type': 'disk', 'boot_index': 0, 'guest_format': None, 'delete_on_termination': False, 'connection_info': {'driver_volume_type': 'iscsi', 'data': {'target_iqn': 'iqn.2010-10.org.openstack:257d7345-5b29-4bc1-8844-e211864c06c3', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'uUeBnhLX', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}, 'status': 'reserved', 'instance': 'ebc3e3b2-037d-43ef-9426-caebe298ee3e', 'attached_at': '', 'detached_at': '', 'volume_id': '257d7345-5b29-4bc1-8844-e211864c06c3', 'serial': '257d7345-5b29-4bc1-8844-e211864c06c3'}, 'volume_type': None} {{(pid=75234) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7562}}
Nov 29 03:32:41 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-d0cfb3d1-066f-4604-ac70-d86765dfbd73 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:32:41 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-d0cfb3d1-066f-4604-ac70-d86765dfbd73 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:32:41 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-d0cfb3d1-066f-4604-ac70-d86765dfbd73 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Searching host: 'user' for CPU controller through CGroups V1... {{(pid=75234) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1652}}
Nov 29 03:32:41 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-d0cfb3d1-066f-4604-ac70-d86765dfbd73 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] CPU controller missing on host. {{(pid=75234) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1662}}
Nov 29 03:32:41 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-d0cfb3d1-066f-4604-ac70-d86765dfbd73 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Searching host: 'user' for CPU controller through CGroups V2... {{(pid=75234) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1671}}
Nov 29 03:32:41 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-d0cfb3d1-066f-4604-ac70-d86765dfbd73 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] CPU controller found on host. {{(pid=75234) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1678}}
Nov 29 03:32:41 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-d0cfb3d1-066f-4604-ac70-d86765dfbd73 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=75234) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5400}}
Nov 29 03:32:41 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-d0cfb3d1-066f-4604-ac70-d86765dfbd73 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-11-29T03:02:04Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='1',id=6,is_public=True,memory_mb=512,name='m1.tiny',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='c8fc807773e5354afe61636071771906',container_format='bare',created_at=2023-11-29T03:02:15Z,direct_url=<?>,disk_format='qcow2',id=96c30ba7-9b73-4a30-8ba4-7138879573b7,min_disk=0,min_ram=0,name='cirros-0.6.2-x86_64-disk',owner='b18e380921c748b78517b6f1211cd0f1',properties=ImageMetaProps,protected=<?>,size=21430272,status='active',tags=<?>,updated_at=2023-11-29T03:02:17Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:563}}
Nov 29 03:32:41 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-d0cfb3d1-066f-4604-ac70-d86765dfbd73 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Flavor limits 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:348}}
Nov 29 03:32:41 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-d0cfb3d1-066f-4604-ac70-d86765dfbd73 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Image limits 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:352}}
Nov 29 03:32:41 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-d0cfb3d1-066f-4604-ac70-d86765dfbd73 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Flavor pref 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:388}}
Nov 29 03:32:41 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-d0cfb3d1-066f-4604-ac70-d86765dfbd73 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Image pref 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:392}}
Nov 29 03:32:41 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-d0cfb3d1-066f-4604-ac70-d86765dfbd73 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:430}}
Nov 29 03:32:41 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-d0cfb3d1-066f-4604-ac70-d86765dfbd73 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:569}}
Nov 29 03:32:41 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-d0cfb3d1-066f-4604-ac70-d86765dfbd73 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=75234) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:471}}
Nov 29 03:32:41 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-d0cfb3d1-066f-4604-ac70-d86765dfbd73 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Got 1 possible topologies {{(pid=75234) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:501}}
Nov 29 03:32:41 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-d0cfb3d1-066f-4604-ac70-d86765dfbd73 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:575}}
Nov 29 03:32:41 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-d0cfb3d1-066f-4604-ac70-d86765dfbd73 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:577}}
Nov 29 03:32:41 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-d0cfb3d1-066f-4604-ac70-d86765dfbd73 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] [instance: ebc3e3b2-037d-43ef-9426-caebe298ee3e] Calling os-brick to attach iSCSI Volume {{(pid=75234) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Nov 29 03:32:41 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-d0cfb3d1-066f-4604-ac70-d86765dfbd73 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f8a5050c9a0>, 'connection_properties': {'target_iqn': 'iqn.2010-10.org.openstack:257d7345-5b29-4bc1-8844-e211864c06c3', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'uUeBnhLX', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:176}}
Nov 29 03:32:41 user nova-compute[75234]: WARNING os_brick.initiator.connectors.base [None req-d0cfb3d1-066f-4604-ac70-d86765dfbd73 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Nov 29 03:32:41 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-d0cfb3d1-066f-4604-ac70-d86765dfbd73 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:68}}
Nov 29 03:32:41 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-d0cfb3d1-066f-4604-ac70-d86765dfbd73 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:73}}
Nov 29 03:32:41 user nova-compute[75234]: WARNING os_brick.initiator.connectors.base [None req-d0cfb3d1-066f-4604-ac70-d86765dfbd73 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Nov 29 03:32:41 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-d0cfb3d1-066f-4604-ac70-d86765dfbd73 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:257d7345-5b29-4bc1-8844-e211864c06c3" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:68}}
Nov 29 03:32:41 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-d0cfb3d1-066f-4604-ac70-d86765dfbd73 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:257d7345-5b29-4bc1-8844-e211864c06c3" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:73}}
Nov 29 03:32:41 user nova-compute[75234]: INFO os_brick.initiator.connectors.iscsi [None req-d0cfb3d1-066f-4604-ac70-d86765dfbd73 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Nov 29 03:32:41 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:257d7345-5b29-4bc1-8844-e211864c06c3 -p 172.16.0.220:3260 {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:32:41 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:257d7345-5b29-4bc1-8844-e211864c06c3 -p 172.16.0.220:3260" returned: 21 in 0.020s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:32:41 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[502af2ac-5b5d-4761-a6e2-9c67b36868a6]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:32:41 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-d0cfb3d1-066f-4604-ac70-d86765dfbd73 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Nov 29 03:32:41 user nova-compute[75234]:  {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:32:41 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:257d7345-5b29-4bc1-8844-e211864c06c3 -p 172.16.0.220:3260 --interface default --op new {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:32:41 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:257d7345-5b29-4bc1-8844-e211864c06c3 -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.018s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:32:41 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[5991f2f8-aa01-4c86-a767-68277ad8d18c]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:257d7345-5b29-4bc1-8844-e211864c06c3] added\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:32:41 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-d0cfb3d1-066f-4604-ac70-d86765dfbd73 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:257d7345-5b29-4bc1-8844-e211864c06c3] added
Nov 29 03:32:41 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:32:41 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:257d7345-5b29-4bc1-8844-e211864c06c3 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:32:41 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:257d7345-5b29-4bc1-8844-e211864c06c3 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.019s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:32:41 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[41d768ae-9f73-4cae-a337-0e27271c29f1]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:32:41 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-d0cfb3d1-066f-4604-ac70-d86765dfbd73 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:32:41 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:257d7345-5b29-4bc1-8844-e211864c06c3 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:32:41 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:257d7345-5b29-4bc1-8844-e211864c06c3 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.018s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:32:41 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[f051cef2-22ab-47d5-91c4-391b7014d410]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:32:41 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-d0cfb3d1-066f-4604-ac70-d86765dfbd73 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:32:41 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:257d7345-5b29-4bc1-8844-e211864c06c3 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v uUeBnhLX {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:32:41 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:257d7345-5b29-4bc1-8844-e211864c06c3 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v uUeBnhLX" returned: 0 in 0.021s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:32:41 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[f6cbd26b-9bd3-4fbd-b609-0f2214657b0d]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:32:41 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-d0cfb3d1-066f-4604-ac70-d86765dfbd73 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'uUeBnhLX'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:32:41 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:257d7345-5b29-4bc1-8844-e211864c06c3 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:32:41 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:257d7345-5b29-4bc1-8844-e211864c06c3 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.018s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:32:41 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[9d1d6904-7231-44f0-9072-eb6ee0ac8da2]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:32:41 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-d0cfb3d1-066f-4604-ac70-d86765dfbd73 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:32:41 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:32:41 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.024s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:32:41 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[3034055e-a1f7-4d69-8927-f3be8fb18756]: (4, ('tcp: [42] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0 (non-flash)\ntcp: [53] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d7d89638-2f88-47e2-a43c-b37e8919ecbc (non-flash)\ntcp: [54] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a3d30ae4-ca6d-4e06-8fc9-caf4d91ef900 (non-flash)\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:32:41 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-d0cfb3d1-066f-4604-ac70-d86765dfbd73 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [42] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0 (non-flash)
Nov 29 03:32:41 user nova-compute[75234]: tcp: [53] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d7d89638-2f88-47e2-a43c-b37e8919ecbc (non-flash)
Nov 29 03:32:41 user nova-compute[75234]: tcp: [54] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a3d30ae4-ca6d-4e06-8fc9-caf4d91ef900 (non-flash)
Nov 29 03:32:41 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:32:41 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-d0cfb3d1-066f-4604-ac70-d86765dfbd73 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] iscsi session list stdout=tcp: [42] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0 (non-flash)
Nov 29 03:32:41 user nova-compute[75234]: tcp: [53] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d7d89638-2f88-47e2-a43c-b37e8919ecbc (non-flash)
Nov 29 03:32:41 user nova-compute[75234]: tcp: [54] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a3d30ae4-ca6d-4e06-8fc9-caf4d91ef900 (non-flash)
Nov 29 03:32:41 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsi_session /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Nov 29 03:32:41 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:257d7345-5b29-4bc1-8844-e211864c06c3 -p 172.16.0.220:3260 --login {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:32:41 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:257d7345-5b29-4bc1-8844-e211864c06c3 -p 172.16.0.220:3260 --login" returned: 0 in 0.061s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:32:41 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[6b754600-f7cb-475c-aea8-609112d1b209]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:257d7345-5b29-4bc1-8844-e211864c06c3, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:257d7345-5b29-4bc1-8844-e211864c06c3, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:32:41 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-d0cfb3d1-066f-4604-ac70-d86765dfbd73 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:257d7345-5b29-4bc1-8844-e211864c06c3, portal: 172.16.0.220,3260]
Nov 29 03:32:41 user nova-compute[75234]: Login to [iface: default, target: iqn.2010-10.org.openstack:257d7345-5b29-4bc1-8844-e211864c06c3, portal: 172.16.0.220,3260] successful.
Nov 29 03:32:41 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:32:41 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:257d7345-5b29-4bc1-8844-e211864c06c3 -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:32:41 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:257d7345-5b29-4bc1-8844-e211864c06c3 -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.019s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:32:41 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[5e91ff75-411f-4023-b7f5-ec80984b596d]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:32:41 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-d0cfb3d1-066f-4604-ac70-d86765dfbd73 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:32:41 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:32:41 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.023s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:32:41 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[c2954795-853a-4a69-93f7-15af264a6171]: (4, ('tcp: [42] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0 (non-flash)\ntcp: [53] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d7d89638-2f88-47e2-a43c-b37e8919ecbc (non-flash)\ntcp: [54] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a3d30ae4-ca6d-4e06-8fc9-caf4d91ef900 (non-flash)\ntcp: [55] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:257d7345-5b29-4bc1-8844-e211864c06c3 (non-flash)\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:32:41 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-d0cfb3d1-066f-4604-ac70-d86765dfbd73 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [42] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0 (non-flash)
Nov 29 03:32:41 user nova-compute[75234]: tcp: [53] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d7d89638-2f88-47e2-a43c-b37e8919ecbc (non-flash)
Nov 29 03:32:41 user nova-compute[75234]: tcp: [54] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a3d30ae4-ca6d-4e06-8fc9-caf4d91ef900 (non-flash)
Nov 29 03:32:41 user nova-compute[75234]: tcp: [55] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:257d7345-5b29-4bc1-8844-e211864c06c3 (non-flash)
Nov 29 03:32:41 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:32:41 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-d0cfb3d1-066f-4604-ac70-d86765dfbd73 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] iscsi session list stdout=tcp: [42] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0 (non-flash)
Nov 29 03:32:41 user nova-compute[75234]: tcp: [53] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d7d89638-2f88-47e2-a43c-b37e8919ecbc (non-flash)
Nov 29 03:32:41 user nova-compute[75234]: tcp: [54] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a3d30ae4-ca6d-4e06-8fc9-caf4d91ef900 (non-flash)
Nov 29 03:32:41 user nova-compute[75234]: tcp: [55] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:257d7345-5b29-4bc1-8844-e211864c06c3 (non-flash)
Nov 29 03:32:41 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsi_session /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Nov 29 03:32:41 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-d0cfb3d1-066f-4604-ac70-d86765dfbd73 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:257d7345-5b29-4bc1-8844-e211864c06c3" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.276s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:87}}
Nov 29 03:32:41 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-d0cfb3d1-066f-4604-ac70-d86765dfbd73 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Connected to 172.16.0.220:3260 {{(pid=75234) _connect_vol /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:644}}
Nov 29 03:32:41 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-d0cfb3d1-066f-4604-ac70-d86765dfbd73 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] HCTL ('36', '-', '-', 0) found on session 55 with lun 0 {{(pid=75234) get_hctl /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:770}}
Nov 29 03:32:41 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-d0cfb3d1-066f-4604-ac70-d86765dfbd73 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Scanning host 36 c: -, t: -, l: 0) {{(pid=75234) scan_iscsi /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:795}}
Nov 29 03:32:41 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host36/scan {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:32:41 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host36/scan" returned: 0 in 0.015s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:32:41 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[5ce9569b-5c62-41e8-94b9-87e6f5467d8a]: (4, ('- - 0', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:32:41 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-d0cfb3d1-066f-4604-ac70-d86765dfbd73 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Searching for a device in session 55 and hctl ['36', '*', '*', 0] yield: sde {{(pid=75234) device_name_by_hctl /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:789}}
Nov 29 03:32:41 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-d0cfb3d1-066f-4604-ac70-d86765dfbd73 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Connected to sde using {'target_iqn': 'iqn.2010-10.org.openstack:257d7345-5b29-4bc1-8844-e211864c06c3', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'uUeBnhLX', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False} {{(pid=75234) _connect_vol /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:672}}
Nov 29 03:32:42 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:32:42 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:32:42 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:32:42 user nova-compute[75234]: DEBUG nova.compute.manager [req-7ffd48dc-36ec-44bf-86c1-d75ce0172597 req-84474490-9bf2-474b-b9f6-66616d573b9a service nova] [instance: 884589ec-b86c-4094-9a3f-54b641311d62] Received event network-vif-plugged-1deddce5-7279-44f9-9f1e-cd9ffddc5d6c {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:32:42 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-7ffd48dc-36ec-44bf-86c1-d75ce0172597 req-84474490-9bf2-474b-b9f6-66616d573b9a service nova] Acquiring lock "884589ec-b86c-4094-9a3f-54b641311d62-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:32:42 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-7ffd48dc-36ec-44bf-86c1-d75ce0172597 req-84474490-9bf2-474b-b9f6-66616d573b9a service nova] Lock "884589ec-b86c-4094-9a3f-54b641311d62-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:32:42 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-7ffd48dc-36ec-44bf-86c1-d75ce0172597 req-84474490-9bf2-474b-b9f6-66616d573b9a service nova] Lock "884589ec-b86c-4094-9a3f-54b641311d62-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:32:42 user nova-compute[75234]: DEBUG nova.compute.manager [req-7ffd48dc-36ec-44bf-86c1-d75ce0172597 req-84474490-9bf2-474b-b9f6-66616d573b9a service nova] [instance: 884589ec-b86c-4094-9a3f-54b641311d62] No waiting events found dispatching network-vif-plugged-1deddce5-7279-44f9-9f1e-cd9ffddc5d6c {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:32:42 user nova-compute[75234]: WARNING nova.compute.manager [req-7ffd48dc-36ec-44bf-86c1-d75ce0172597 req-84474490-9bf2-474b-b9f6-66616d573b9a service nova] [instance: 884589ec-b86c-4094-9a3f-54b641311d62] Received unexpected event network-vif-plugged-1deddce5-7279-44f9-9f1e-cd9ffddc5d6c for instance with vm_state building and task_state spawning.
Nov 29 03:32:42 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-d0cfb3d1-066f-4604-ac70-d86765dfbd73 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.305s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:87}}
Nov 29 03:32:42 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-d0cfb3d1-066f-4604-ac70-d86765dfbd73 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] <== connect_volume: return (1306ms) {'type': 'block', 'scsi_wwn': '26239616134643263', 'path': '/dev/sde'} {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:203}}
Nov 29 03:32:42 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-d0cfb3d1-066f-4604-ac70-d86765dfbd73 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] [instance: ebc3e3b2-037d-43ef-9426-caebe298ee3e] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '26239616134643263', 'path': '/dev/sde'} {{(pid=75234) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Nov 29 03:32:42 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:32:43 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-d0cfb3d1-066f-4604-ac70-d86765dfbd73 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-11-29T03:32:27Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description=None,display_name='tempest-ServerBootFromVolumeStableRescueTest-server-1464602599',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-serverbootfromvolumestablerescuetest-server-1464602599',id=47,image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data=None,key_name=None,keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='db00bcd0c82c44f1bda739e9857f6e84',ramdisk_id='',reservation_id='r-ztz14xc0',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-ServerBootFromVolumeStableRescueTest-1601266069',owner_user_name='tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-11-29T03:32:28Z,user_data=None,user_id='877aa877f13a4f84ac4725278010aea3',uuid=ebc3e3b2-037d-43ef-9426-caebe298ee3e,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "bd0a4e5b-69cd-4f5c-ae22-994e4d8f2682", "address": "fa:16:3e:c2:aa:3e", "network": {"id": "4d086e08-7798-4cfe-9365-86be6f0a52d2", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-305624008-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "db00bcd0c82c44f1bda739e9857f6e84", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapbd0a4e5b-69", "ovs_interfaceid": "bd0a4e5b-69cd-4f5c-ae22-994e4d8f2682", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=75234) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Nov 29 03:32:43 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-d0cfb3d1-066f-4604-ac70-d86765dfbd73 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Converting VIF {"id": "bd0a4e5b-69cd-4f5c-ae22-994e4d8f2682", "address": "fa:16:3e:c2:aa:3e", "network": {"id": "4d086e08-7798-4cfe-9365-86be6f0a52d2", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-305624008-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "db00bcd0c82c44f1bda739e9857f6e84", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapbd0a4e5b-69", "ovs_interfaceid": "bd0a4e5b-69cd-4f5c-ae22-994e4d8f2682", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:32:43 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-d0cfb3d1-066f-4604-ac70-d86765dfbd73 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:c2:aa:3e,bridge_name='br-int',has_traffic_filtering=True,id=bd0a4e5b-69cd-4f5c-ae22-994e4d8f2682,network=Network(4d086e08-7798-4cfe-9365-86be6f0a52d2),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapbd0a4e5b-69') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:32:43 user nova-compute[75234]: DEBUG nova.objects.instance [None req-d0cfb3d1-066f-4604-ac70-d86765dfbd73 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Lazy-loading 'pci_devices' on Instance uuid ebc3e3b2-037d-43ef-9426-caebe298ee3e {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:32:43 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:32:43 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-d0cfb3d1-066f-4604-ac70-d86765dfbd73 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] [instance: ebc3e3b2-037d-43ef-9426-caebe298ee3e] End _get_guest_xml xml=<domain type="kvm">
Nov 29 03:32:43 user nova-compute[75234]:   <uuid>ebc3e3b2-037d-43ef-9426-caebe298ee3e</uuid>
Nov 29 03:32:43 user nova-compute[75234]:   <name>instance-0000002f</name>
Nov 29 03:32:43 user nova-compute[75234]:   <memory>524288</memory>
Nov 29 03:32:43 user nova-compute[75234]:   <vcpu>1</vcpu>
Nov 29 03:32:43 user nova-compute[75234]:   <metadata>
Nov 29 03:32:43 user nova-compute[75234]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Nov 29 03:32:43 user nova-compute[75234]:       <nova:package version="0.0.0"/>
Nov 29 03:32:43 user nova-compute[75234]:       <nova:name>tempest-ServerBootFromVolumeStableRescueTest-server-1464602599</nova:name>
Nov 29 03:32:43 user nova-compute[75234]:       <nova:creationTime>2023-11-29 03:32:41</nova:creationTime>
Nov 29 03:32:43 user nova-compute[75234]:       <nova:flavor name="m1.tiny">
Nov 29 03:32:43 user nova-compute[75234]:         <nova:memory>512</nova:memory>
Nov 29 03:32:43 user nova-compute[75234]:         <nova:disk>1</nova:disk>
Nov 29 03:32:43 user nova-compute[75234]:         <nova:swap>0</nova:swap>
Nov 29 03:32:43 user nova-compute[75234]:         <nova:ephemeral>0</nova:ephemeral>
Nov 29 03:32:43 user nova-compute[75234]:         <nova:vcpus>1</nova:vcpus>
Nov 29 03:32:43 user nova-compute[75234]:       </nova:flavor>
Nov 29 03:32:43 user nova-compute[75234]:       <nova:owner>
Nov 29 03:32:43 user nova-compute[75234]:         <nova:user uuid="877aa877f13a4f84ac4725278010aea3">tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member</nova:user>
Nov 29 03:32:43 user nova-compute[75234]:         <nova:project uuid="db00bcd0c82c44f1bda739e9857f6e84">tempest-ServerBootFromVolumeStableRescueTest-1601266069</nova:project>
Nov 29 03:32:43 user nova-compute[75234]:       </nova:owner>
Nov 29 03:32:43 user nova-compute[75234]:       <nova:root type="image" uuid="96c30ba7-9b73-4a30-8ba4-7138879573b7"/>
Nov 29 03:32:43 user nova-compute[75234]:       <nova:ports>
Nov 29 03:32:43 user nova-compute[75234]:         <nova:port uuid="bd0a4e5b-69cd-4f5c-ae22-994e4d8f2682">
Nov 29 03:32:43 user nova-compute[75234]:           <nova:ip type="fixed" address="10.0.0.14" ipVersion="4"/>
Nov 29 03:32:43 user nova-compute[75234]:         </nova:port>
Nov 29 03:32:43 user nova-compute[75234]:       </nova:ports>
Nov 29 03:32:43 user nova-compute[75234]:     </nova:instance>
Nov 29 03:32:43 user nova-compute[75234]:   </metadata>
Nov 29 03:32:43 user nova-compute[75234]:   <sysinfo type="smbios">
Nov 29 03:32:43 user nova-compute[75234]:     <system>
Nov 29 03:32:43 user nova-compute[75234]:       <entry name="manufacturer">OpenStack Foundation</entry>
Nov 29 03:32:43 user nova-compute[75234]:       <entry name="product">OpenStack Nova</entry>
Nov 29 03:32:43 user nova-compute[75234]:       <entry name="version">0.0.0</entry>
Nov 29 03:32:43 user nova-compute[75234]:       <entry name="serial">ebc3e3b2-037d-43ef-9426-caebe298ee3e</entry>
Nov 29 03:32:43 user nova-compute[75234]:       <entry name="uuid">ebc3e3b2-037d-43ef-9426-caebe298ee3e</entry>
Nov 29 03:32:43 user nova-compute[75234]:       <entry name="family">Virtual Machine</entry>
Nov 29 03:32:43 user nova-compute[75234]:     </system>
Nov 29 03:32:43 user nova-compute[75234]:   </sysinfo>
Nov 29 03:32:43 user nova-compute[75234]:   <os>
Nov 29 03:32:43 user nova-compute[75234]:     <type arch="x86_64" machine="pc">hvm</type>
Nov 29 03:32:43 user nova-compute[75234]:     <boot dev="hd"/>
Nov 29 03:32:43 user nova-compute[75234]:     <smbios mode="sysinfo"/>
Nov 29 03:32:43 user nova-compute[75234]:   </os>
Nov 29 03:32:43 user nova-compute[75234]:   <features>
Nov 29 03:32:43 user nova-compute[75234]:     <acpi/>
Nov 29 03:32:43 user nova-compute[75234]:     <apic/>
Nov 29 03:32:43 user nova-compute[75234]:     <vmcoreinfo/>
Nov 29 03:32:43 user nova-compute[75234]:   </features>
Nov 29 03:32:43 user nova-compute[75234]:   <clock offset="utc">
Nov 29 03:32:43 user nova-compute[75234]:     <timer name="pit" tickpolicy="delay"/>
Nov 29 03:32:43 user nova-compute[75234]:     <timer name="rtc" tickpolicy="catchup"/>
Nov 29 03:32:43 user nova-compute[75234]:     <timer name="hpet" present="no"/>
Nov 29 03:32:43 user nova-compute[75234]:   </clock>
Nov 29 03:32:43 user nova-compute[75234]:   <cpu mode="custom" match="exact">
Nov 29 03:32:43 user nova-compute[75234]:     <model>Nehalem</model>
Nov 29 03:32:43 user nova-compute[75234]:     <topology sockets="1" cores="1" threads="1"/>
Nov 29 03:32:43 user nova-compute[75234]:   </cpu>
Nov 29 03:32:43 user nova-compute[75234]:   <devices>
Nov 29 03:32:43 user nova-compute[75234]:     <disk type="block" device="disk">
Nov 29 03:32:43 user nova-compute[75234]:       <driver name="qemu" type="raw" cache="none" io="native"/>
Nov 29 03:32:43 user nova-compute[75234]:       <source dev="/dev/sde"/>
Nov 29 03:32:43 user nova-compute[75234]:       <target dev="vda" bus="virtio"/>
Nov 29 03:32:43 user nova-compute[75234]:       <serial>257d7345-5b29-4bc1-8844-e211864c06c3</serial>
Nov 29 03:32:43 user nova-compute[75234]:     </disk>
Nov 29 03:32:43 user nova-compute[75234]:     <interface type="ethernet">
Nov 29 03:32:43 user nova-compute[75234]:       <mac address="fa:16:3e:c2:aa:3e"/>
Nov 29 03:32:43 user nova-compute[75234]:       <model type="virtio"/>
Nov 29 03:32:43 user nova-compute[75234]:       <mtu size="1442"/>
Nov 29 03:32:43 user nova-compute[75234]:       <target dev="tapbd0a4e5b-69"/>
Nov 29 03:32:43 user nova-compute[75234]:     </interface>
Nov 29 03:32:43 user nova-compute[75234]:     <serial type="pty">
Nov 29 03:32:43 user nova-compute[75234]:       <log file="/opt/stack/data/nova/instances/ebc3e3b2-037d-43ef-9426-caebe298ee3e/console.log" append="off"/>
Nov 29 03:32:43 user nova-compute[75234]:     </serial>
Nov 29 03:32:43 user nova-compute[75234]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Nov 29 03:32:43 user nova-compute[75234]:     <video>
Nov 29 03:32:43 user nova-compute[75234]:       <model type="virtio"/>
Nov 29 03:32:43 user nova-compute[75234]:     </video>
Nov 29 03:32:43 user nova-compute[75234]:     <rng model="virtio">
Nov 29 03:32:43 user nova-compute[75234]:       <backend model="random">/dev/urandom</backend>
Nov 29 03:32:43 user nova-compute[75234]:     </rng>
Nov 29 03:32:43 user nova-compute[75234]:     <controller type="usb" index="0" model="none"/>
Nov 29 03:32:43 user nova-compute[75234]:     <memballoon model="virtio">
Nov 29 03:32:43 user nova-compute[75234]:       <stats period="10"/>
Nov 29 03:32:43 user nova-compute[75234]:     </memballoon>
Nov 29 03:32:43 user nova-compute[75234]:   </devices>
Nov 29 03:32:43 user nova-compute[75234]: </domain>
Nov 29 03:32:43 user nova-compute[75234]:  {{(pid=75234) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7568}}
Nov 29 03:32:43 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-d0cfb3d1-066f-4604-ac70-d86765dfbd73 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-11-29T03:32:27Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description=None,display_name='tempest-ServerBootFromVolumeStableRescueTest-server-1464602599',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-serverbootfromvolumestablerescuetest-server-1464602599',id=47,image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data=None,key_name=None,keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='db00bcd0c82c44f1bda739e9857f6e84',ramdisk_id='',reservation_id='r-ztz14xc0',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',image_container_format='bare',image_disk_format='qcow2',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-ServerBootFromVolumeStableRescueTest-1601266069',owner_user_name='tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-11-29T03:32:28Z,user_data=None,user_id='877aa877f13a4f84ac4725278010aea3',uuid=ebc3e3b2-037d-43ef-9426-caebe298ee3e,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "bd0a4e5b-69cd-4f5c-ae22-994e4d8f2682", "address": "fa:16:3e:c2:aa:3e", "network": {"id": "4d086e08-7798-4cfe-9365-86be6f0a52d2", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-305624008-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "db00bcd0c82c44f1bda739e9857f6e84", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapbd0a4e5b-69", "ovs_interfaceid": "bd0a4e5b-69cd-4f5c-ae22-994e4d8f2682", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Nov 29 03:32:43 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-d0cfb3d1-066f-4604-ac70-d86765dfbd73 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Converting VIF {"id": "bd0a4e5b-69cd-4f5c-ae22-994e4d8f2682", "address": "fa:16:3e:c2:aa:3e", "network": {"id": "4d086e08-7798-4cfe-9365-86be6f0a52d2", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-305624008-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "db00bcd0c82c44f1bda739e9857f6e84", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapbd0a4e5b-69", "ovs_interfaceid": "bd0a4e5b-69cd-4f5c-ae22-994e4d8f2682", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:32:43 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-d0cfb3d1-066f-4604-ac70-d86765dfbd73 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:c2:aa:3e,bridge_name='br-int',has_traffic_filtering=True,id=bd0a4e5b-69cd-4f5c-ae22-994e4d8f2682,network=Network(4d086e08-7798-4cfe-9365-86be6f0a52d2),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapbd0a4e5b-69') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:32:43 user nova-compute[75234]: DEBUG os_vif [None req-d0cfb3d1-066f-4604-ac70-d86765dfbd73 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:c2:aa:3e,bridge_name='br-int',has_traffic_filtering=True,id=bd0a4e5b-69cd-4f5c-ae22-994e4d8f2682,network=Network(4d086e08-7798-4cfe-9365-86be6f0a52d2),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapbd0a4e5b-69') {{(pid=75234) plug /opt/stack/data/venv/lib/python3.10/site-packages/os_vif/__init__.py:76}}
Nov 29 03:32:43 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:32:43 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:32:43 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Nov 29 03:32:43 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:32:43 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbCreateCommand(_result=None, table=QoS, columns={'type': 'linux-noop', 'external_ids': {'id': '39f0ffd2-8fcc-579a-b30d-d1c3dbf16640', '_type': 'linux-noop'}}, row=False) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:32:43 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:32:43 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:32:43 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:32:43 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:32:43 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tapbd0a4e5b-69, may_exist=True, interface_attrs={}) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:32:43 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Port, record=tapbd0a4e5b-69, col_values=(('qos', UUID('02065a19-7713-49ec-8e10-e7f7f21ec748')),), if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:32:43 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DbSetCommand(_result=None, table=Interface, record=tapbd0a4e5b-69, col_values=(('external_ids', {'iface-id': 'bd0a4e5b-69cd-4f5c-ae22-994e4d8f2682', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:c2:aa:3e', 'vm-uuid': 'ebc3e3b2-037d-43ef-9426-caebe298ee3e'}),), if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:32:43 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:32:43 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:32:43 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:32:43 user nova-compute[75234]: INFO os_vif [None req-d0cfb3d1-066f-4604-ac70-d86765dfbd73 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:c2:aa:3e,bridge_name='br-int',has_traffic_filtering=True,id=bd0a4e5b-69cd-4f5c-ae22-994e4d8f2682,network=Network(4d086e08-7798-4cfe-9365-86be6f0a52d2),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapbd0a4e5b-69')
Nov 29 03:32:43 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-d0cfb3d1-066f-4604-ac70-d86765dfbd73 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] No BDM found with device name vda, not building metadata. {{(pid=75234) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12089}}
Nov 29 03:32:43 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-d0cfb3d1-066f-4604-ac70-d86765dfbd73 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] No VIF found with MAC fa:16:3e:c2:aa:3e, not building metadata {{(pid=75234) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Nov 29 03:32:44 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:32:44 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:32:44 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:32:44 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:32:44 user nova-compute[75234]: DEBUG nova.compute.manager [None req-3f2c9326-e9dc-46ad-b028-3d5aef2c14ea tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: 884589ec-b86c-4094-9a3f-54b641311d62] Instance event wait completed in 0 seconds for  {{(pid=75234) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Nov 29 03:32:44 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-3f2c9326-e9dc-46ad-b028-3d5aef2c14ea tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: 884589ec-b86c-4094-9a3f-54b641311d62] Guest created on hypervisor {{(pid=75234) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4421}}
Nov 29 03:32:44 user nova-compute[75234]: DEBUG nova.compute.manager [req-fc9f18bc-3360-4b01-8d14-dadfc6c98e31 req-d832beef-415c-48f8-abfb-b39d0d8ed8d9 service nova] [instance: ebc3e3b2-037d-43ef-9426-caebe298ee3e] Received event network-vif-plugged-bd0a4e5b-69cd-4f5c-ae22-994e4d8f2682 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:32:44 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-fc9f18bc-3360-4b01-8d14-dadfc6c98e31 req-d832beef-415c-48f8-abfb-b39d0d8ed8d9 service nova] Acquiring lock "ebc3e3b2-037d-43ef-9426-caebe298ee3e-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:32:44 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-fc9f18bc-3360-4b01-8d14-dadfc6c98e31 req-d832beef-415c-48f8-abfb-b39d0d8ed8d9 service nova] Lock "ebc3e3b2-037d-43ef-9426-caebe298ee3e-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:32:44 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-fc9f18bc-3360-4b01-8d14-dadfc6c98e31 req-d832beef-415c-48f8-abfb-b39d0d8ed8d9 service nova] Lock "ebc3e3b2-037d-43ef-9426-caebe298ee3e-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:32:44 user nova-compute[75234]: DEBUG nova.compute.manager [req-fc9f18bc-3360-4b01-8d14-dadfc6c98e31 req-d832beef-415c-48f8-abfb-b39d0d8ed8d9 service nova] [instance: ebc3e3b2-037d-43ef-9426-caebe298ee3e] No waiting events found dispatching network-vif-plugged-bd0a4e5b-69cd-4f5c-ae22-994e4d8f2682 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:32:44 user nova-compute[75234]: WARNING nova.compute.manager [req-fc9f18bc-3360-4b01-8d14-dadfc6c98e31 req-d832beef-415c-48f8-abfb-b39d0d8ed8d9 service nova] [instance: ebc3e3b2-037d-43ef-9426-caebe298ee3e] Received unexpected event network-vif-plugged-bd0a4e5b-69cd-4f5c-ae22-994e4d8f2682 for instance with vm_state building and task_state spawning.
Nov 29 03:32:44 user nova-compute[75234]: INFO nova.virt.libvirt.driver [-] [instance: 884589ec-b86c-4094-9a3f-54b641311d62] Instance spawned successfully.
Nov 29 03:32:44 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-3f2c9326-e9dc-46ad-b028-3d5aef2c14ea tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: 884589ec-b86c-4094-9a3f-54b641311d62] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:904}}
Nov 29 03:32:44 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-3f2c9326-e9dc-46ad-b028-3d5aef2c14ea tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: 884589ec-b86c-4094-9a3f-54b641311d62] Found default for hw_cdrom_bus of ide {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:32:44 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-3f2c9326-e9dc-46ad-b028-3d5aef2c14ea tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: 884589ec-b86c-4094-9a3f-54b641311d62] Found default for hw_disk_bus of virtio {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:32:44 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-3f2c9326-e9dc-46ad-b028-3d5aef2c14ea tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: 884589ec-b86c-4094-9a3f-54b641311d62] Found default for hw_input_bus of None {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:32:44 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-3f2c9326-e9dc-46ad-b028-3d5aef2c14ea tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: 884589ec-b86c-4094-9a3f-54b641311d62] Found default for hw_pointer_model of None {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:32:44 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-3f2c9326-e9dc-46ad-b028-3d5aef2c14ea tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: 884589ec-b86c-4094-9a3f-54b641311d62] Found default for hw_video_model of virtio {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:32:44 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-3f2c9326-e9dc-46ad-b028-3d5aef2c14ea tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: 884589ec-b86c-4094-9a3f-54b641311d62] Found default for hw_vif_model of virtio {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:32:44 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:32:44 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:32:44 user nova-compute[75234]: INFO nova.compute.manager [None req-3f2c9326-e9dc-46ad-b028-3d5aef2c14ea tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: 884589ec-b86c-4094-9a3f-54b641311d62] Took 5.07 seconds to spawn the instance on the hypervisor.
Nov 29 03:32:44 user nova-compute[75234]: DEBUG nova.compute.manager [None req-3f2c9326-e9dc-46ad-b028-3d5aef2c14ea tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: 884589ec-b86c-4094-9a3f-54b641311d62] Checking state {{(pid=75234) _get_power_state /opt/stack/nova/nova/compute/manager.py:1782}}
Nov 29 03:32:44 user nova-compute[75234]: DEBUG nova.compute.manager [req-5dc4d033-5221-4108-920e-a4413c4d0906 req-27aa53c9-b6b0-4992-a73c-c26a01ff271e service nova] [instance: 884589ec-b86c-4094-9a3f-54b641311d62] Received event network-vif-plugged-1deddce5-7279-44f9-9f1e-cd9ffddc5d6c {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:32:44 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-5dc4d033-5221-4108-920e-a4413c4d0906 req-27aa53c9-b6b0-4992-a73c-c26a01ff271e service nova] Acquiring lock "884589ec-b86c-4094-9a3f-54b641311d62-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:32:44 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-5dc4d033-5221-4108-920e-a4413c4d0906 req-27aa53c9-b6b0-4992-a73c-c26a01ff271e service nova] Lock "884589ec-b86c-4094-9a3f-54b641311d62-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:32:44 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-5dc4d033-5221-4108-920e-a4413c4d0906 req-27aa53c9-b6b0-4992-a73c-c26a01ff271e service nova] Lock "884589ec-b86c-4094-9a3f-54b641311d62-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:32:44 user nova-compute[75234]: DEBUG nova.compute.manager [req-5dc4d033-5221-4108-920e-a4413c4d0906 req-27aa53c9-b6b0-4992-a73c-c26a01ff271e service nova] [instance: 884589ec-b86c-4094-9a3f-54b641311d62] No waiting events found dispatching network-vif-plugged-1deddce5-7279-44f9-9f1e-cd9ffddc5d6c {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:32:44 user nova-compute[75234]: WARNING nova.compute.manager [req-5dc4d033-5221-4108-920e-a4413c4d0906 req-27aa53c9-b6b0-4992-a73c-c26a01ff271e service nova] [instance: 884589ec-b86c-4094-9a3f-54b641311d62] Received unexpected event network-vif-plugged-1deddce5-7279-44f9-9f1e-cd9ffddc5d6c for instance with vm_state building and task_state spawning.
Nov 29 03:32:44 user nova-compute[75234]: INFO nova.compute.manager [None req-3f2c9326-e9dc-46ad-b028-3d5aef2c14ea tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: 884589ec-b86c-4094-9a3f-54b641311d62] Took 13.29 seconds to build instance.
Nov 29 03:32:44 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-3f2c9326-e9dc-46ad-b028-3d5aef2c14ea tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Lock "884589ec-b86c-4094-9a3f-54b641311d62" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 13.399s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:32:46 user nova-compute[75234]: DEBUG nova.compute.manager [None req-d0cfb3d1-066f-4604-ac70-d86765dfbd73 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] [instance: ebc3e3b2-037d-43ef-9426-caebe298ee3e] Instance event wait completed in 0 seconds for  {{(pid=75234) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Nov 29 03:32:46 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-d0cfb3d1-066f-4604-ac70-d86765dfbd73 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] [instance: ebc3e3b2-037d-43ef-9426-caebe298ee3e] Guest created on hypervisor {{(pid=75234) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4421}}
Nov 29 03:32:46 user nova-compute[75234]: INFO nova.virt.libvirt.driver [-] [instance: ebc3e3b2-037d-43ef-9426-caebe298ee3e] Instance spawned successfully.
Nov 29 03:32:46 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-d0cfb3d1-066f-4604-ac70-d86765dfbd73 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] [instance: ebc3e3b2-037d-43ef-9426-caebe298ee3e] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:904}}
Nov 29 03:32:46 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-d0cfb3d1-066f-4604-ac70-d86765dfbd73 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] [instance: ebc3e3b2-037d-43ef-9426-caebe298ee3e] Found default for hw_cdrom_bus of ide {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:32:46 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-d0cfb3d1-066f-4604-ac70-d86765dfbd73 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] [instance: ebc3e3b2-037d-43ef-9426-caebe298ee3e] Found default for hw_disk_bus of virtio {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:32:46 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-d0cfb3d1-066f-4604-ac70-d86765dfbd73 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] [instance: ebc3e3b2-037d-43ef-9426-caebe298ee3e] Found default for hw_input_bus of None {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:32:46 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-d0cfb3d1-066f-4604-ac70-d86765dfbd73 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] [instance: ebc3e3b2-037d-43ef-9426-caebe298ee3e] Found default for hw_pointer_model of None {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:32:46 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-d0cfb3d1-066f-4604-ac70-d86765dfbd73 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] [instance: ebc3e3b2-037d-43ef-9426-caebe298ee3e] Found default for hw_video_model of virtio {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:32:46 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-d0cfb3d1-066f-4604-ac70-d86765dfbd73 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] [instance: ebc3e3b2-037d-43ef-9426-caebe298ee3e] Found default for hw_vif_model of virtio {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:32:46 user nova-compute[75234]: DEBUG nova.compute.manager [req-e6c4cd30-6fad-4a33-ae2b-e149e19714d5 req-e702d0f0-47b5-4cc3-9264-e1262f28c5f9 service nova] [instance: ebc3e3b2-037d-43ef-9426-caebe298ee3e] Received event network-vif-plugged-bd0a4e5b-69cd-4f5c-ae22-994e4d8f2682 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:32:46 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-e6c4cd30-6fad-4a33-ae2b-e149e19714d5 req-e702d0f0-47b5-4cc3-9264-e1262f28c5f9 service nova] Acquiring lock "ebc3e3b2-037d-43ef-9426-caebe298ee3e-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:32:46 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-e6c4cd30-6fad-4a33-ae2b-e149e19714d5 req-e702d0f0-47b5-4cc3-9264-e1262f28c5f9 service nova] Lock "ebc3e3b2-037d-43ef-9426-caebe298ee3e-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:32:46 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-e6c4cd30-6fad-4a33-ae2b-e149e19714d5 req-e702d0f0-47b5-4cc3-9264-e1262f28c5f9 service nova] Lock "ebc3e3b2-037d-43ef-9426-caebe298ee3e-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:32:46 user nova-compute[75234]: DEBUG nova.compute.manager [req-e6c4cd30-6fad-4a33-ae2b-e149e19714d5 req-e702d0f0-47b5-4cc3-9264-e1262f28c5f9 service nova] [instance: ebc3e3b2-037d-43ef-9426-caebe298ee3e] No waiting events found dispatching network-vif-plugged-bd0a4e5b-69cd-4f5c-ae22-994e4d8f2682 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:32:46 user nova-compute[75234]: WARNING nova.compute.manager [req-e6c4cd30-6fad-4a33-ae2b-e149e19714d5 req-e702d0f0-47b5-4cc3-9264-e1262f28c5f9 service nova] [instance: ebc3e3b2-037d-43ef-9426-caebe298ee3e] Received unexpected event network-vif-plugged-bd0a4e5b-69cd-4f5c-ae22-994e4d8f2682 for instance with vm_state building and task_state spawning.
Nov 29 03:32:46 user nova-compute[75234]: INFO nova.compute.manager [None req-d0cfb3d1-066f-4604-ac70-d86765dfbd73 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] [instance: ebc3e3b2-037d-43ef-9426-caebe298ee3e] Took 5.03 seconds to spawn the instance on the hypervisor.
Nov 29 03:32:46 user nova-compute[75234]: DEBUG nova.compute.manager [None req-d0cfb3d1-066f-4604-ac70-d86765dfbd73 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] [instance: ebc3e3b2-037d-43ef-9426-caebe298ee3e] Checking state {{(pid=75234) _get_power_state /opt/stack/nova/nova/compute/manager.py:1782}}
Nov 29 03:32:46 user nova-compute[75234]: INFO nova.compute.manager [None req-d0cfb3d1-066f-4604-ac70-d86765dfbd73 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] [instance: ebc3e3b2-037d-43ef-9426-caebe298ee3e] Took 18.61 seconds to build instance.
Nov 29 03:32:46 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-d0cfb3d1-066f-4604-ac70-d86765dfbd73 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Lock "ebc3e3b2-037d-43ef-9426-caebe298ee3e" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 18.789s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:32:47 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:32:48 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:32:53 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:32:58 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:33:02 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:33:03 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:33:07 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:33:08 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:33:11 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:33:12 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:33:12 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:33:12 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=75234) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10541}}
Nov 29 03:33:12 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:33:13 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:33:13 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:33:14 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-641fca7b-031e-4b3b-9491-ee221dda4df1 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] Acquiring lock "01d62e54-9957-40b1-8dfe-1783b64ffabf" by "nova.compute.manager.ComputeManager.unshelve_instance.<locals>.do_unshelve_instance" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:33:14 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-641fca7b-031e-4b3b-9491-ee221dda4df1 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] Lock "01d62e54-9957-40b1-8dfe-1783b64ffabf" acquired by "nova.compute.manager.ComputeManager.unshelve_instance.<locals>.do_unshelve_instance" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:33:14 user nova-compute[75234]: INFO nova.compute.manager [None req-641fca7b-031e-4b3b-9491-ee221dda4df1 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] [instance: 01d62e54-9957-40b1-8dfe-1783b64ffabf] Unshelving
Nov 29 03:33:14 user nova-compute[75234]: INFO nova.virt.block_device [None req-641fca7b-031e-4b3b-9491-ee221dda4df1 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] [instance: 01d62e54-9957-40b1-8dfe-1783b64ffabf] Booting with volume 43e92d4a-7e98-4777-8fa5-3cec396b86ce at /dev/vda
Nov 29 03:33:14 user nova-compute[75234]: DEBUG os_brick.utils [None req-641fca7b-031e-4b3b-9491-ee221dda4df1 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:176}}
Nov 29 03:33:14 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:33:14 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.013s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:33:14 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[659f9e32-6a0b-40c0-8d09-4324d8cb40be]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:dd1368c5ba2c\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:33:14 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt -v / -n -o SOURCE {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:33:14 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt -v / -n -o SOURCE" returned: 0 in 0.015s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:33:14 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[3c6e3f91-b544-4692-b5c1-9725c6026b55]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:33:14 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:33:14 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.016s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:33:14 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[e5255915-c8b0-4d87-904d-41a80281f170]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:33:14 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[de59dce4-0bd0-4388-a8fe-883be4844b27]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:33:14 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-641fca7b-031e-4b3b-9491-ee221dda4df1 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] Running cmd (subprocess): nvme version {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:33:14 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-641fca7b-031e-4b3b-9491-ee221dda4df1 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] 'nvme version' failed. Not Retrying. {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:473}}
Nov 29 03:33:14 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.nvmeof [None req-641fca7b-031e-4b3b-9491-ee221dda4df1 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] nvme not present on system {{(pid=75234) nvme_present /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/nvmeof.py:756}}
Nov 29 03:33:14 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:33:14 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:473}}
Nov 29 03:33:14 user nova-compute[75234]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Nov 29 03:33:14 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[68c8f734-7899-4a49-b594-11f45f50eac5]: (4, '') {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:33:14 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.lightos [None req-641fca7b-031e-4b3b-9491-ee221dda4df1 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=75234) find_dsc /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/lightos.py:98}}
Nov 29 03:33:14 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.lightos [None req-641fca7b-031e-4b3b-9491-ee221dda4df1 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=75234) get_connector_properties /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/lightos.py:76}}
Nov 29 03:33:14 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.lightos [None req-641fca7b-031e-4b3b-9491-ee221dda4df1 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] LIGHTOS: no hostnqn found. {{(pid=75234) get_connector_properties /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/lightos.py:84}}
Nov 29 03:33:14 user nova-compute[75234]: DEBUG os_brick.utils [None req-641fca7b-031e-4b3b-9491-ee221dda4df1 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] <== get_connector_properties: return (114ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:dd1368c5ba2c', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:203}}
Nov 29 03:33:14 user nova-compute[75234]: DEBUG nova.virt.block_device [None req-641fca7b-031e-4b3b-9491-ee221dda4df1 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] [instance: 01d62e54-9957-40b1-8dfe-1783b64ffabf] Updating existing volume attachment record: 716d8d31-315d-4347-ab4e-217e4001ed8f {{(pid=75234) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Nov 29 03:33:15 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager.update_available_resource {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:33:15 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:33:15 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:33:15 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:33:15 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Auditing locally available compute resources for user (node: user) {{(pid=75234) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:907}}
Nov 29 03:33:15 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/65d48c93-b6da-4f51-9d5a-0676dda62a60/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:33:15 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/65d48c93-b6da-4f51-9d5a-0676dda62a60/disk --force-share --output=json" returned: 0 in 0.245s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:33:15 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/65d48c93-b6da-4f51-9d5a-0676dda62a60/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:33:16 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/65d48c93-b6da-4f51-9d5a-0676dda62a60/disk --force-share --output=json" returned: 0 in 0.168s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:33:16 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] skipping disk /dev/sde (vda) as it is a volume {{(pid=75234) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11213}}
Nov 29 03:33:16 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] skipping disk /dev/sdc (vda) as it is a volume {{(pid=75234) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11213}}
Nov 29 03:33:16 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] skipping disk /dev/sdd (vda) as it is a volume {{(pid=75234) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11213}}
Nov 29 03:33:16 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/abb01889-df4b-44a6-9827-87056d7ca125/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:33:16 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/abb01889-df4b-44a6-9827-87056d7ca125/disk --force-share --output=json" returned: 0 in 0.173s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:33:16 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/abb01889-df4b-44a6-9827-87056d7ca125/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:33:16 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/abb01889-df4b-44a6-9827-87056d7ca125/disk --force-share --output=json" returned: 0 in 0.169s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:33:16 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/cb0f042e-43c0-43aa-b243-cfc78a7b2695/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:33:16 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/cb0f042e-43c0-43aa-b243-cfc78a7b2695/disk --force-share --output=json" returned: 0 in 0.167s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:33:16 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/cb0f042e-43c0-43aa-b243-cfc78a7b2695/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:33:16 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/cb0f042e-43c0-43aa-b243-cfc78a7b2695/disk --force-share --output=json" returned: 0 in 0.168s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:33:16 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] skipping disk /dev/sdb (vdb) as it is a volume {{(pid=75234) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11213}}
Nov 29 03:33:17 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:33:17 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:33:17 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Hypervisor/Node resource view: name=user free_ram=8278MB free_disk=24.971546173095703GB free_vcpus=6 pci_devices=[{"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}] {{(pid=75234) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1106}}
Nov 29 03:33:17 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:33:17 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:33:17 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance 65d48c93-b6da-4f51-9d5a-0676dda62a60 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:33:17 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance d5096ca2-d005-45ac-a487-11a810446859 actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:33:17 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance abb01889-df4b-44a6-9827-87056d7ca125 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:33:17 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance cb0f042e-43c0-43aa-b243-cfc78a7b2695 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:33:17 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance ebc3e3b2-037d-43ef-9426-caebe298ee3e actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:33:17 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance 884589ec-b86c-4094-9a3f-54b641311d62 actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:33:17 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance 01d62e54-9957-40b1-8dfe-1783b64ffabf has been scheduled to this compute host, the scheduler has made an allocation against this compute node but the instance has yet to start. Skipping heal of allocation: {'resources': {'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1761}}
Nov 29 03:33:17 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Total usable vcpus: 12, total allocated vcpus: 6 {{(pid=75234) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1129}}
Nov 29 03:33:17 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Final resource view: name=user phys_ram=16011MB used_ram=3584MB phys_disk=40GB used_disk=3GB total_vcpus=12 used_vcpus=6 pci_stats=[] {{(pid=75234) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1138}}
Nov 29 03:33:17 user nova-compute[75234]: DEBUG nova.scheduler.client.report [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Refreshing inventories for resource provider 4d291220-7c37-4839-a0e3-b23c66502b74 {{(pid=75234) _refresh_associations /opt/stack/nova/nova/scheduler/client/report.py:804}}
Nov 29 03:33:17 user nova-compute[75234]: DEBUG nova.scheduler.client.report [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Updating ProviderTree inventory for provider 4d291220-7c37-4839-a0e3-b23c66502b74 from _refresh_and_get_inventory using data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=75234) _refresh_and_get_inventory /opt/stack/nova/nova/scheduler/client/report.py:768}}
Nov 29 03:33:17 user nova-compute[75234]: DEBUG nova.compute.provider_tree [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Updating inventory in ProviderTree for provider 4d291220-7c37-4839-a0e3-b23c66502b74 with inventory: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=75234) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:176}}
Nov 29 03:33:17 user nova-compute[75234]: DEBUG nova.scheduler.client.report [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Refreshing aggregate associations for resource provider 4d291220-7c37-4839-a0e3-b23c66502b74, aggregates: None {{(pid=75234) _refresh_associations /opt/stack/nova/nova/scheduler/client/report.py:813}}
Nov 29 03:33:17 user nova-compute[75234]: DEBUG nova.scheduler.client.report [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Refreshing trait associations for resource provider 4d291220-7c37-4839-a0e3-b23c66502b74, traits: COMPUTE_IMAGE_TYPE_AMI,COMPUTE_GRAPHICS_MODEL_CIRRUS,COMPUTE_NET_VIF_MODEL_VMXNET3,COMPUTE_STORAGE_BUS_VIRTIO,COMPUTE_GRAPHICS_MODEL_NONE,COMPUTE_IMAGE_TYPE_ISO,COMPUTE_NET_VIF_MODEL_RTL8139,HW_CPU_X86_SSE,COMPUTE_IMAGE_TYPE_QCOW2,COMPUTE_SOCKET_PCI_NUMA_AFFINITY,COMPUTE_GRAPHICS_MODEL_BOCHS,COMPUTE_VOLUME_ATTACH_WITH_TAG,COMPUTE_ACCELERATORS,HW_CPU_X86_SSE2,COMPUTE_STORAGE_BUS_SCSI,COMPUTE_GRAPHICS_MODEL_VGA,COMPUTE_NET_ATTACH_INTERFACE_WITH_TAG,COMPUTE_SECURITY_UEFI_SECURE_BOOT,HW_CPU_X86_SSE41,COMPUTE_NET_VIF_MODEL_E1000E,COMPUTE_TRUSTED_CERTS,COMPUTE_STORAGE_BUS_FDC,COMPUTE_VIOMMU_MODEL_INTEL,COMPUTE_GRAPHICS_MODEL_VIRTIO,COMPUTE_VIOMMU_MODEL_AUTO,COMPUTE_IMAGE_TYPE_AKI,COMPUTE_NET_VIF_MODEL_PCNET,COMPUTE_GRAPHICS_MODEL_QXL,COMPUTE_NET_VIF_MODEL_VIRTIO,COMPUTE_GRAPHICS_MODEL_VMVGA,COMPUTE_VOLUME_MULTI_ATTACH,COMPUTE_STORAGE_BUS_SATA,HW_CPU_X86_MMX,COMPUTE_NET_VIF_MODEL_E1000,COMPUTE_IMAGE_TYPE_ARI,COMPUTE_STORAGE_BUS_IDE,COMPUTE_NODE,HW_CPU_X86_SSSE3,COMPUTE_RESCUE_BFV,COMPUTE_NET_ATTACH_INTERFACE,COMPUTE_STORAGE_BUS_USB,COMPUTE_NET_VIF_MODEL_SPAPR_VLAN,COMPUTE_IMAGE_TYPE_RAW,COMPUTE_NET_VIF_MODEL_NE2K_PCI,COMPUTE_VOLUME_EXTEND,COMPUTE_DEVICE_TAGGING,HW_CPU_X86_SSE42 {{(pid=75234) _refresh_associations /opt/stack/nova/nova/scheduler/client/report.py:825}}
Nov 29 03:33:17 user nova-compute[75234]: DEBUG nova.compute.provider_tree [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Inventory has not changed in ProviderTree for provider: 4d291220-7c37-4839-a0e3-b23c66502b74 {{(pid=75234) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Nov 29 03:33:17 user nova-compute[75234]: DEBUG nova.scheduler.client.report [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Inventory has not changed for provider 4d291220-7c37-4839-a0e3-b23c66502b74 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=75234) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Nov 29 03:33:17 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Compute_service record updated for user:user {{(pid=75234) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:1067}}
Nov 29 03:33:17 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.518s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:33:17 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:33:18 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:33:19 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:33:19 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Starting heal instance info cache {{(pid=75234) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9922}}
Nov 29 03:33:19 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Acquiring lock "refresh_cache-abb01889-df4b-44a6-9827-87056d7ca125" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:33:19 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Acquired lock "refresh_cache-abb01889-df4b-44a6-9827-87056d7ca125" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:33:19 user nova-compute[75234]: DEBUG nova.network.neutron [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] [instance: abb01889-df4b-44a6-9827-87056d7ca125] Forcefully refreshing network info cache for instance {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2003}}
Nov 29 03:33:20 user nova-compute[75234]: DEBUG nova.network.neutron [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] [instance: abb01889-df4b-44a6-9827-87056d7ca125] Updating instance_info_cache with network_info: [{"id": "fec40d24-caf1-48f7-8565-8fffdbdad60e", "address": "fa:16:3e:e3:bd:f8", "network": {"id": "4d086e08-7798-4cfe-9365-86be6f0a52d2", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-305624008-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "db00bcd0c82c44f1bda739e9857f6e84", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapfec40d24-ca", "ovs_interfaceid": "fec40d24-caf1-48f7-8565-8fffdbdad60e", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:33:20 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Releasing lock "refresh_cache-abb01889-df4b-44a6-9827-87056d7ca125" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:33:20 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] [instance: abb01889-df4b-44a6-9827-87056d7ca125] Updated the network info_cache for instance {{(pid=75234) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9993}}
Nov 29 03:33:20 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:33:20 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:33:21 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-641fca7b-031e-4b3b-9491-ee221dda4df1 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:33:21 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-641fca7b-031e-4b3b-9491-ee221dda4df1 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:33:21 user nova-compute[75234]: DEBUG nova.objects.instance [None req-641fca7b-031e-4b3b-9491-ee221dda4df1 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] Lazy-loading 'pci_requests' on Instance uuid 01d62e54-9957-40b1-8dfe-1783b64ffabf {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:33:21 user nova-compute[75234]: DEBUG nova.objects.instance [None req-641fca7b-031e-4b3b-9491-ee221dda4df1 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] Lazy-loading 'numa_topology' on Instance uuid 01d62e54-9957-40b1-8dfe-1783b64ffabf {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:33:21 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-641fca7b-031e-4b3b-9491-ee221dda4df1 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=75234) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2366}}
Nov 29 03:33:21 user nova-compute[75234]: INFO nova.compute.claims [None req-641fca7b-031e-4b3b-9491-ee221dda4df1 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] [instance: 01d62e54-9957-40b1-8dfe-1783b64ffabf] Claim successful on node user
Nov 29 03:33:22 user nova-compute[75234]: DEBUG nova.compute.provider_tree [None req-641fca7b-031e-4b3b-9491-ee221dda4df1 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] Inventory has not changed in ProviderTree for provider: 4d291220-7c37-4839-a0e3-b23c66502b74 {{(pid=75234) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Nov 29 03:33:22 user nova-compute[75234]: DEBUG nova.scheduler.client.report [None req-641fca7b-031e-4b3b-9491-ee221dda4df1 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] Inventory has not changed for provider 4d291220-7c37-4839-a0e3-b23c66502b74 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=75234) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Nov 29 03:33:22 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-641fca7b-031e-4b3b-9491-ee221dda4df1 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.328s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:33:22 user nova-compute[75234]: INFO nova.network.neutron [None req-641fca7b-031e-4b3b-9491-ee221dda4df1 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] [instance: 01d62e54-9957-40b1-8dfe-1783b64ffabf] Updating port 6dbd7c27-31c9-4ddc-a81b-85f1e742d160 with attributes {'binding:host_id': 'user', 'device_owner': 'compute:nova'}
Nov 29 03:33:22 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:33:22 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-641fca7b-031e-4b3b-9491-ee221dda4df1 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] Acquiring lock "refresh_cache-01d62e54-9957-40b1-8dfe-1783b64ffabf" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:33:22 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-641fca7b-031e-4b3b-9491-ee221dda4df1 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] Acquired lock "refresh_cache-01d62e54-9957-40b1-8dfe-1783b64ffabf" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:33:22 user nova-compute[75234]: DEBUG nova.network.neutron [None req-641fca7b-031e-4b3b-9491-ee221dda4df1 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] [instance: 01d62e54-9957-40b1-8dfe-1783b64ffabf] Building network info cache for instance {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2009}}
Nov 29 03:33:22 user nova-compute[75234]: DEBUG nova.compute.manager [req-04ec5c54-99ce-4f40-812c-98c943bd9be8 req-d22de450-3bf7-4ac9-a3db-f4685e4e162a service nova] [instance: 01d62e54-9957-40b1-8dfe-1783b64ffabf] Received event network-changed-6dbd7c27-31c9-4ddc-a81b-85f1e742d160 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:33:22 user nova-compute[75234]: DEBUG nova.compute.manager [req-04ec5c54-99ce-4f40-812c-98c943bd9be8 req-d22de450-3bf7-4ac9-a3db-f4685e4e162a service nova] [instance: 01d62e54-9957-40b1-8dfe-1783b64ffabf] Refreshing instance network info cache due to event network-changed-6dbd7c27-31c9-4ddc-a81b-85f1e742d160. {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11117}}
Nov 29 03:33:22 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-04ec5c54-99ce-4f40-812c-98c943bd9be8 req-d22de450-3bf7-4ac9-a3db-f4685e4e162a service nova] Acquiring lock "refresh_cache-01d62e54-9957-40b1-8dfe-1783b64ffabf" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:33:23 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:33:23 user nova-compute[75234]: DEBUG nova.network.neutron [None req-641fca7b-031e-4b3b-9491-ee221dda4df1 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] [instance: 01d62e54-9957-40b1-8dfe-1783b64ffabf] Updating instance_info_cache with network_info: [{"id": "6dbd7c27-31c9-4ddc-a81b-85f1e742d160", "address": "fa:16:3e:57:6a:26", "network": {"id": "504616f5-0d6b-47a6-aaf5-663983428b41", "bridge": "br-int", "label": "tempest-TestShelveInstance-438404840-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.210", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "53e55fcb8cd74895baec6f9528770bbc", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap6dbd7c27-31", "ovs_interfaceid": "6dbd7c27-31c9-4ddc-a81b-85f1e742d160", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:33:23 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-641fca7b-031e-4b3b-9491-ee221dda4df1 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] Releasing lock "refresh_cache-01d62e54-9957-40b1-8dfe-1783b64ffabf" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:33:23 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-641fca7b-031e-4b3b-9491-ee221dda4df1 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] [instance: 01d62e54-9957-40b1-8dfe-1783b64ffabf] Creating instance directory {{(pid=75234) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4727}}
Nov 29 03:33:23 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-641fca7b-031e-4b3b-9491-ee221dda4df1 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] [instance: 01d62e54-9957-40b1-8dfe-1783b64ffabf] Creating image(s)
Nov 29 03:33:23 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-641fca7b-031e-4b3b-9491-ee221dda4df1 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] [instance: 01d62e54-9957-40b1-8dfe-1783b64ffabf] Did not create local disks {{(pid=75234) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4863}}
Nov 29 03:33:23 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-641fca7b-031e-4b3b-9491-ee221dda4df1 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] [instance: 01d62e54-9957-40b1-8dfe-1783b64ffabf] Ensure instance console log exists: /opt/stack/data/nova/instances/01d62e54-9957-40b1-8dfe-1783b64ffabf/console.log {{(pid=75234) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4613}}
Nov 29 03:33:23 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-641fca7b-031e-4b3b-9491-ee221dda4df1 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:33:23 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-641fca7b-031e-4b3b-9491-ee221dda4df1 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:33:23 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-641fca7b-031e-4b3b-9491-ee221dda4df1 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:33:23 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-641fca7b-031e-4b3b-9491-ee221dda4df1 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] [instance: 01d62e54-9957-40b1-8dfe-1783b64ffabf] Start _get_guest_xml network_info=[{"id": "6dbd7c27-31c9-4ddc-a81b-85f1e742d160", "address": "fa:16:3e:57:6a:26", "network": {"id": "504616f5-0d6b-47a6-aaf5-663983428b41", "bridge": "br-int", "label": "tempest-TestShelveInstance-438404840-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.210", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "53e55fcb8cd74895baec6f9528770bbc", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap6dbd7c27-31", "ovs_interfaceid": "6dbd7c27-31c9-4ddc-a81b-85f1e742d160", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, '/dev/vda': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum=<?>,container_format=<?>,created_at=<?>,direct_url=<?>,disk_format=<?>,id=<?>,min_disk=1,min_ram=0,name=<?>,owner=<?>,properties=ImageMetaProps,protected=<?>,size=<?>,status=<?>,tags=<?>,updated_at=<?>,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [], 'ephemerals': [], 'block_device_mapping': [{'mount_device': '/dev/vda', 'attachment_id': '716d8d31-315d-4347-ab4e-217e4001ed8f', 'disk_bus': 'virtio', 'device_type': 'disk', 'boot_index': 0, 'guest_format': None, 'delete_on_termination': True, 'connection_info': {'driver_volume_type': 'iscsi', 'data': {'target_iqn': 'iqn.2010-10.org.openstack:43e92d4a-7e98-4777-8fa5-3cec396b86ce', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'AQnkxjFp', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}, 'status': 'reserved', 'instance': '01d62e54-9957-40b1-8dfe-1783b64ffabf', 'attached_at': '', 'detached_at': '', 'volume_id': '43e92d4a-7e98-4777-8fa5-3cec396b86ce', 'serial': '43e92d4a-7e98-4777-8fa5-3cec396b86ce'}, 'volume_type': None} {{(pid=75234) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7562}}
Nov 29 03:33:23 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-04ec5c54-99ce-4f40-812c-98c943bd9be8 req-d22de450-3bf7-4ac9-a3db-f4685e4e162a service nova] Acquired lock "refresh_cache-01d62e54-9957-40b1-8dfe-1783b64ffabf" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:33:23 user nova-compute[75234]: DEBUG nova.network.neutron [req-04ec5c54-99ce-4f40-812c-98c943bd9be8 req-d22de450-3bf7-4ac9-a3db-f4685e4e162a service nova] [instance: 01d62e54-9957-40b1-8dfe-1783b64ffabf] Refreshing network info cache for port 6dbd7c27-31c9-4ddc-a81b-85f1e742d160 {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2006}}
Nov 29 03:33:23 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-641fca7b-031e-4b3b-9491-ee221dda4df1 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:33:23 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-641fca7b-031e-4b3b-9491-ee221dda4df1 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:33:23 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-641fca7b-031e-4b3b-9491-ee221dda4df1 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] Searching host: 'user' for CPU controller through CGroups V1... {{(pid=75234) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1652}}
Nov 29 03:33:23 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-641fca7b-031e-4b3b-9491-ee221dda4df1 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] CPU controller missing on host. {{(pid=75234) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1662}}
Nov 29 03:33:23 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-641fca7b-031e-4b3b-9491-ee221dda4df1 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] Searching host: 'user' for CPU controller through CGroups V2... {{(pid=75234) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1671}}
Nov 29 03:33:23 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-641fca7b-031e-4b3b-9491-ee221dda4df1 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] CPU controller found on host. {{(pid=75234) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1678}}
Nov 29 03:33:23 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-641fca7b-031e-4b3b-9491-ee221dda4df1 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=75234) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5400}}
Nov 29 03:33:23 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-641fca7b-031e-4b3b-9491-ee221dda4df1 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-11-29T03:02:04Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='1',id=6,is_public=True,memory_mb=512,name='m1.tiny',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum=<?>,container_format=<?>,created_at=<?>,direct_url=<?>,disk_format=<?>,id=<?>,min_disk=1,min_ram=0,name=<?>,owner=<?>,properties=ImageMetaProps,protected=<?>,size=<?>,status=<?>,tags=<?>,updated_at=<?>,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:563}}
Nov 29 03:33:23 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-641fca7b-031e-4b3b-9491-ee221dda4df1 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] Flavor limits 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:348}}
Nov 29 03:33:23 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-641fca7b-031e-4b3b-9491-ee221dda4df1 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] Image limits 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:352}}
Nov 29 03:33:23 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-641fca7b-031e-4b3b-9491-ee221dda4df1 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] Flavor pref 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:388}}
Nov 29 03:33:23 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-641fca7b-031e-4b3b-9491-ee221dda4df1 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] Image pref 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:392}}
Nov 29 03:33:23 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-641fca7b-031e-4b3b-9491-ee221dda4df1 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:430}}
Nov 29 03:33:23 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-641fca7b-031e-4b3b-9491-ee221dda4df1 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:569}}
Nov 29 03:33:23 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-641fca7b-031e-4b3b-9491-ee221dda4df1 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=75234) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:471}}
Nov 29 03:33:23 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-641fca7b-031e-4b3b-9491-ee221dda4df1 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] Got 1 possible topologies {{(pid=75234) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:501}}
Nov 29 03:33:23 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-641fca7b-031e-4b3b-9491-ee221dda4df1 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:575}}
Nov 29 03:33:23 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-641fca7b-031e-4b3b-9491-ee221dda4df1 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:577}}
Nov 29 03:33:23 user nova-compute[75234]: DEBUG nova.objects.instance [None req-641fca7b-031e-4b3b-9491-ee221dda4df1 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] Lazy-loading 'vcpu_model' on Instance uuid 01d62e54-9957-40b1-8dfe-1783b64ffabf {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:33:23 user nova-compute[75234]: DEBUG nova.objects.base [None req-641fca7b-031e-4b3b-9491-ee221dda4df1 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] Object Instance<01d62e54-9957-40b1-8dfe-1783b64ffabf> lazy-loaded attributes: pci_requests,vcpu_model {{(pid=75234) wrapper /opt/stack/nova/nova/objects/base.py:126}}
Nov 29 03:33:23 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-641fca7b-031e-4b3b-9491-ee221dda4df1 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] [instance: 01d62e54-9957-40b1-8dfe-1783b64ffabf] Calling os-brick to attach iSCSI Volume {{(pid=75234) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Nov 29 03:33:23 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-641fca7b-031e-4b3b-9491-ee221dda4df1 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f8a5050c9a0>, 'connection_properties': {'target_iqn': 'iqn.2010-10.org.openstack:43e92d4a-7e98-4777-8fa5-3cec396b86ce', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'AQnkxjFp', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:176}}
Nov 29 03:33:23 user nova-compute[75234]: WARNING os_brick.initiator.connectors.base [None req-641fca7b-031e-4b3b-9491-ee221dda4df1 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Nov 29 03:33:23 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-641fca7b-031e-4b3b-9491-ee221dda4df1 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:68}}
Nov 29 03:33:23 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-641fca7b-031e-4b3b-9491-ee221dda4df1 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:73}}
Nov 29 03:33:23 user nova-compute[75234]: WARNING os_brick.initiator.connectors.base [None req-641fca7b-031e-4b3b-9491-ee221dda4df1 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Nov 29 03:33:23 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-641fca7b-031e-4b3b-9491-ee221dda4df1 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:43e92d4a-7e98-4777-8fa5-3cec396b86ce" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:68}}
Nov 29 03:33:23 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-641fca7b-031e-4b3b-9491-ee221dda4df1 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:43e92d4a-7e98-4777-8fa5-3cec396b86ce" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:73}}
Nov 29 03:33:23 user nova-compute[75234]: INFO os_brick.initiator.connectors.iscsi [None req-641fca7b-031e-4b3b-9491-ee221dda4df1 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Nov 29 03:33:23 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:33:23 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:43e92d4a-7e98-4777-8fa5-3cec396b86ce -p 172.16.0.220:3260 {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:33:23 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:43e92d4a-7e98-4777-8fa5-3cec396b86ce -p 172.16.0.220:3260" returned: 21 in 0.018s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:33:23 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[80b2159d-c79e-4901-9bd2-d7cd8e1f2871]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:33:23 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-641fca7b-031e-4b3b-9491-ee221dda4df1 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Nov 29 03:33:23 user nova-compute[75234]:  {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:33:23 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:43e92d4a-7e98-4777-8fa5-3cec396b86ce -p 172.16.0.220:3260 --interface default --op new {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:33:23 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:43e92d4a-7e98-4777-8fa5-3cec396b86ce -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.019s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:33:23 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[e4bd65d9-aac9-4f80-a4a7-ee510103c8ec]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:43e92d4a-7e98-4777-8fa5-3cec396b86ce] added\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:33:23 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-641fca7b-031e-4b3b-9491-ee221dda4df1 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:43e92d4a-7e98-4777-8fa5-3cec396b86ce] added
Nov 29 03:33:23 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:33:23 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:43e92d4a-7e98-4777-8fa5-3cec396b86ce -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:33:23 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:43e92d4a-7e98-4777-8fa5-3cec396b86ce -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.019s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:33:23 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[e03abd83-a586-4f3d-850c-f82be3882643]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:33:23 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-641fca7b-031e-4b3b-9491-ee221dda4df1 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:33:23 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:43e92d4a-7e98-4777-8fa5-3cec396b86ce -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:33:23 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:43e92d4a-7e98-4777-8fa5-3cec396b86ce -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.026s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:33:23 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[83be4567-47de-41a7-bfa8-8884af8b246f]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:33:23 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-641fca7b-031e-4b3b-9491-ee221dda4df1 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:33:23 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:43e92d4a-7e98-4777-8fa5-3cec396b86ce -p 172.16.0.220:3260 --op update -n node.session.auth.username -v AQnkxjFp {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:33:23 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:43e92d4a-7e98-4777-8fa5-3cec396b86ce -p 172.16.0.220:3260 --op update -n node.session.auth.username -v AQnkxjFp" returned: 0 in 0.017s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:33:23 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[361b7991-f4b2-4c31-a135-a7f1cab9e20a]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:33:23 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-641fca7b-031e-4b3b-9491-ee221dda4df1 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'AQnkxjFp'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:33:23 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:43e92d4a-7e98-4777-8fa5-3cec396b86ce -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:33:23 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:43e92d4a-7e98-4777-8fa5-3cec396b86ce -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.020s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:33:23 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[697b9b3d-e2da-4bc5-944c-96ae2e36ec9f]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:33:23 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-641fca7b-031e-4b3b-9491-ee221dda4df1 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:33:23 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:33:23 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.022s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:33:23 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[2f1593b4-db1e-4445-bed9-ff120a376bef]: (4, ('tcp: [42] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0 (non-flash)\ntcp: [53] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d7d89638-2f88-47e2-a43c-b37e8919ecbc (non-flash)\ntcp: [54] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a3d30ae4-ca6d-4e06-8fc9-caf4d91ef900 (non-flash)\ntcp: [55] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:257d7345-5b29-4bc1-8844-e211864c06c3 (non-flash)\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:33:23 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-641fca7b-031e-4b3b-9491-ee221dda4df1 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [42] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0 (non-flash)
Nov 29 03:33:23 user nova-compute[75234]: tcp: [53] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d7d89638-2f88-47e2-a43c-b37e8919ecbc (non-flash)
Nov 29 03:33:23 user nova-compute[75234]: tcp: [54] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a3d30ae4-ca6d-4e06-8fc9-caf4d91ef900 (non-flash)
Nov 29 03:33:23 user nova-compute[75234]: tcp: [55] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:257d7345-5b29-4bc1-8844-e211864c06c3 (non-flash)
Nov 29 03:33:23 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:33:23 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-641fca7b-031e-4b3b-9491-ee221dda4df1 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] iscsi session list stdout=tcp: [42] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0 (non-flash)
Nov 29 03:33:23 user nova-compute[75234]: tcp: [53] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d7d89638-2f88-47e2-a43c-b37e8919ecbc (non-flash)
Nov 29 03:33:23 user nova-compute[75234]: tcp: [54] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a3d30ae4-ca6d-4e06-8fc9-caf4d91ef900 (non-flash)
Nov 29 03:33:23 user nova-compute[75234]: tcp: [55] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:257d7345-5b29-4bc1-8844-e211864c06c3 (non-flash)
Nov 29 03:33:23 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsi_session /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Nov 29 03:33:23 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:43e92d4a-7e98-4777-8fa5-3cec396b86ce -p 172.16.0.220:3260 --login {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:33:23 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:43e92d4a-7e98-4777-8fa5-3cec396b86ce -p 172.16.0.220:3260 --login" returned: 0 in 0.068s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:33:23 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[00df6fde-021e-48a1-bcb7-196b55cabd64]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:43e92d4a-7e98-4777-8fa5-3cec396b86ce, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:43e92d4a-7e98-4777-8fa5-3cec396b86ce, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:33:23 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-641fca7b-031e-4b3b-9491-ee221dda4df1 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:43e92d4a-7e98-4777-8fa5-3cec396b86ce, portal: 172.16.0.220,3260]
Nov 29 03:33:23 user nova-compute[75234]: Login to [iface: default, target: iqn.2010-10.org.openstack:43e92d4a-7e98-4777-8fa5-3cec396b86ce, portal: 172.16.0.220,3260] successful.
Nov 29 03:33:23 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:33:23 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:43e92d4a-7e98-4777-8fa5-3cec396b86ce -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:33:23 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:43e92d4a-7e98-4777-8fa5-3cec396b86ce -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.023s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:33:23 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[b7d90024-f51e-4fce-b8ef-d01112288fa3]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:33:23 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-641fca7b-031e-4b3b-9491-ee221dda4df1 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:33:23 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:33:23 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.028s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:33:23 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[d228a25c-64c7-4e5c-bb9c-41f6704167c7]: (4, ('tcp: [42] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0 (non-flash)\ntcp: [53] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d7d89638-2f88-47e2-a43c-b37e8919ecbc (non-flash)\ntcp: [54] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a3d30ae4-ca6d-4e06-8fc9-caf4d91ef900 (non-flash)\ntcp: [55] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:257d7345-5b29-4bc1-8844-e211864c06c3 (non-flash)\ntcp: [56] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:43e92d4a-7e98-4777-8fa5-3cec396b86ce (non-flash)\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:33:23 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-641fca7b-031e-4b3b-9491-ee221dda4df1 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [42] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0 (non-flash)
Nov 29 03:33:23 user nova-compute[75234]: tcp: [53] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d7d89638-2f88-47e2-a43c-b37e8919ecbc (non-flash)
Nov 29 03:33:23 user nova-compute[75234]: tcp: [54] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a3d30ae4-ca6d-4e06-8fc9-caf4d91ef900 (non-flash)
Nov 29 03:33:23 user nova-compute[75234]: tcp: [55] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:257d7345-5b29-4bc1-8844-e211864c06c3 (non-flash)
Nov 29 03:33:23 user nova-compute[75234]: tcp: [56] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:43e92d4a-7e98-4777-8fa5-3cec396b86ce (non-flash)
Nov 29 03:33:23 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:33:23 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-641fca7b-031e-4b3b-9491-ee221dda4df1 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] iscsi session list stdout=tcp: [42] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0 (non-flash)
Nov 29 03:33:23 user nova-compute[75234]: tcp: [53] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d7d89638-2f88-47e2-a43c-b37e8919ecbc (non-flash)
Nov 29 03:33:23 user nova-compute[75234]: tcp: [54] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a3d30ae4-ca6d-4e06-8fc9-caf4d91ef900 (non-flash)
Nov 29 03:33:23 user nova-compute[75234]: tcp: [55] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:257d7345-5b29-4bc1-8844-e211864c06c3 (non-flash)
Nov 29 03:33:23 user nova-compute[75234]: tcp: [56] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:43e92d4a-7e98-4777-8fa5-3cec396b86ce (non-flash)
Nov 29 03:33:23 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsi_session /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Nov 29 03:33:23 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-641fca7b-031e-4b3b-9491-ee221dda4df1 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:43e92d4a-7e98-4777-8fa5-3cec396b86ce" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.298s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:87}}
Nov 29 03:33:23 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-641fca7b-031e-4b3b-9491-ee221dda4df1 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] Connected to 172.16.0.220:3260 {{(pid=75234) _connect_vol /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:644}}
Nov 29 03:33:23 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-641fca7b-031e-4b3b-9491-ee221dda4df1 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] HCTL ('37', '-', '-', 0) found on session 56 with lun 0 {{(pid=75234) get_hctl /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:770}}
Nov 29 03:33:23 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-641fca7b-031e-4b3b-9491-ee221dda4df1 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] Scanning host 37 c: -, t: -, l: 0) {{(pid=75234) scan_iscsi /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:795}}
Nov 29 03:33:23 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host37/scan {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:33:23 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host37/scan" returned: 0 in 0.017s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:33:23 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[4c1d0138-ec8f-4094-b8e6-0ee681b7c90b]: (4, ('- - 0', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:33:23 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-641fca7b-031e-4b3b-9491-ee221dda4df1 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] Searching for a device in session 56 and hctl ['37', '*', '*', 0] yield: sdf {{(pid=75234) device_name_by_hctl /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:789}}
Nov 29 03:33:23 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-641fca7b-031e-4b3b-9491-ee221dda4df1 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] Connected to sdf using {'target_iqn': 'iqn.2010-10.org.openstack:43e92d4a-7e98-4777-8fa5-3cec396b86ce', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'AQnkxjFp', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False} {{(pid=75234) _connect_vol /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:672}}
Nov 29 03:33:23 user nova-compute[75234]: DEBUG nova.network.neutron [req-04ec5c54-99ce-4f40-812c-98c943bd9be8 req-d22de450-3bf7-4ac9-a3db-f4685e4e162a service nova] [instance: 01d62e54-9957-40b1-8dfe-1783b64ffabf] Updated VIF entry in instance network info cache for port 6dbd7c27-31c9-4ddc-a81b-85f1e742d160. {{(pid=75234) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3481}}
Nov 29 03:33:23 user nova-compute[75234]: DEBUG nova.network.neutron [req-04ec5c54-99ce-4f40-812c-98c943bd9be8 req-d22de450-3bf7-4ac9-a3db-f4685e4e162a service nova] [instance: 01d62e54-9957-40b1-8dfe-1783b64ffabf] Updating instance_info_cache with network_info: [{"id": "6dbd7c27-31c9-4ddc-a81b-85f1e742d160", "address": "fa:16:3e:57:6a:26", "network": {"id": "504616f5-0d6b-47a6-aaf5-663983428b41", "bridge": "br-int", "label": "tempest-TestShelveInstance-438404840-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.210", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "53e55fcb8cd74895baec6f9528770bbc", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap6dbd7c27-31", "ovs_interfaceid": "6dbd7c27-31c9-4ddc-a81b-85f1e742d160", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:33:23 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-04ec5c54-99ce-4f40-812c-98c943bd9be8 req-d22de450-3bf7-4ac9-a3db-f4685e4e162a service nova] Releasing lock "refresh_cache-01d62e54-9957-40b1-8dfe-1783b64ffabf" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:33:24 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-641fca7b-031e-4b3b-9491-ee221dda4df1 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.333s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:87}}
Nov 29 03:33:24 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-641fca7b-031e-4b3b-9491-ee221dda4df1 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] <== connect_volume: return (1334ms) {'type': 'block', 'scsi_wwn': '23237313364396533', 'path': '/dev/sdf'} {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:203}}
Nov 29 03:33:24 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-641fca7b-031e-4b3b-9491-ee221dda4df1 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] [instance: 01d62e54-9957-40b1-8dfe-1783b64ffabf] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '23237313364396533', 'path': '/dev/sdf'} {{(pid=75234) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Nov 29 03:33:24 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-641fca7b-031e-4b3b-9491-ee221dda4df1 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=True,compute_id=1,config_drive='',created_at=2023-11-29T03:29:28Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-TestShelveInstance-server-1849172102',display_name='tempest-TestShelveInstance-server-1849172102',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-testshelveinstance-server-1849172102',id=44,image_ref='',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data=None,key_name='tempest-TestShelveInstance-1141474138',keypairs=<?>,launch_index=0,launched_at=2023-11-29T03:29:43Z,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=4,progress=0,project_id='53e55fcb8cd74895baec6f9528770bbc',ramdisk_id='',reservation_id='r-vhus4691',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',clean_attempts='1',image_base_image_ref='',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-TestShelveInstance-1683282958',owner_user_name='tempest-TestShelveInstance-1683282958-project-member'},tags=<?>,task_state='spawning',terminated_at=None,trusted_certs=<?>,updated_at=2023-11-29T03:33:14Z,user_data=None,user_id='5728da78dc074c1993a8a2cc9b821ed6',uuid=01d62e54-9957-40b1-8dfe-1783b64ffabf,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='shelved_offloaded') vif={"id": "6dbd7c27-31c9-4ddc-a81b-85f1e742d160", "address": "fa:16:3e:57:6a:26", "network": {"id": "504616f5-0d6b-47a6-aaf5-663983428b41", "bridge": "br-int", "label": "tempest-TestShelveInstance-438404840-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.210", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "53e55fcb8cd74895baec6f9528770bbc", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap6dbd7c27-31", "ovs_interfaceid": "6dbd7c27-31c9-4ddc-a81b-85f1e742d160", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=75234) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Nov 29 03:33:24 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-641fca7b-031e-4b3b-9491-ee221dda4df1 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] Converting VIF {"id": "6dbd7c27-31c9-4ddc-a81b-85f1e742d160", "address": "fa:16:3e:57:6a:26", "network": {"id": "504616f5-0d6b-47a6-aaf5-663983428b41", "bridge": "br-int", "label": "tempest-TestShelveInstance-438404840-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.210", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "53e55fcb8cd74895baec6f9528770bbc", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap6dbd7c27-31", "ovs_interfaceid": "6dbd7c27-31c9-4ddc-a81b-85f1e742d160", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:33:24 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-641fca7b-031e-4b3b-9491-ee221dda4df1 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:57:6a:26,bridge_name='br-int',has_traffic_filtering=True,id=6dbd7c27-31c9-4ddc-a81b-85f1e742d160,network=Network(504616f5-0d6b-47a6-aaf5-663983428b41),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap6dbd7c27-31') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:33:24 user nova-compute[75234]: DEBUG nova.objects.instance [None req-641fca7b-031e-4b3b-9491-ee221dda4df1 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] Lazy-loading 'pci_devices' on Instance uuid 01d62e54-9957-40b1-8dfe-1783b64ffabf {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:33:24 user nova-compute[75234]: DEBUG nova.objects.base [None req-641fca7b-031e-4b3b-9491-ee221dda4df1 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] Object Instance<01d62e54-9957-40b1-8dfe-1783b64ffabf> lazy-loaded attributes: pci_requests,vcpu_model,pci_devices {{(pid=75234) wrapper /opt/stack/nova/nova/objects/base.py:126}}
Nov 29 03:33:24 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-641fca7b-031e-4b3b-9491-ee221dda4df1 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] [instance: 01d62e54-9957-40b1-8dfe-1783b64ffabf] End _get_guest_xml xml=<domain type="kvm">
Nov 29 03:33:24 user nova-compute[75234]:   <uuid>01d62e54-9957-40b1-8dfe-1783b64ffabf</uuid>
Nov 29 03:33:24 user nova-compute[75234]:   <name>instance-0000002c</name>
Nov 29 03:33:24 user nova-compute[75234]:   <memory>524288</memory>
Nov 29 03:33:24 user nova-compute[75234]:   <vcpu>1</vcpu>
Nov 29 03:33:24 user nova-compute[75234]:   <metadata>
Nov 29 03:33:24 user nova-compute[75234]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Nov 29 03:33:24 user nova-compute[75234]:       <nova:package version="0.0.0"/>
Nov 29 03:33:24 user nova-compute[75234]:       <nova:name>tempest-TestShelveInstance-server-1849172102</nova:name>
Nov 29 03:33:24 user nova-compute[75234]:       <nova:creationTime>2023-11-29 03:33:23</nova:creationTime>
Nov 29 03:33:24 user nova-compute[75234]:       <nova:flavor name="m1.tiny">
Nov 29 03:33:24 user nova-compute[75234]:         <nova:memory>512</nova:memory>
Nov 29 03:33:24 user nova-compute[75234]:         <nova:disk>1</nova:disk>
Nov 29 03:33:24 user nova-compute[75234]:         <nova:swap>0</nova:swap>
Nov 29 03:33:24 user nova-compute[75234]:         <nova:ephemeral>0</nova:ephemeral>
Nov 29 03:33:24 user nova-compute[75234]:         <nova:vcpus>1</nova:vcpus>
Nov 29 03:33:24 user nova-compute[75234]:       </nova:flavor>
Nov 29 03:33:24 user nova-compute[75234]:       <nova:owner>
Nov 29 03:33:24 user nova-compute[75234]:         <nova:user uuid="5728da78dc074c1993a8a2cc9b821ed6">tempest-TestShelveInstance-1683282958-project-member</nova:user>
Nov 29 03:33:24 user nova-compute[75234]:         <nova:project uuid="53e55fcb8cd74895baec6f9528770bbc">tempest-TestShelveInstance-1683282958</nova:project>
Nov 29 03:33:24 user nova-compute[75234]:       </nova:owner>
Nov 29 03:33:24 user nova-compute[75234]:       <nova:ports>
Nov 29 03:33:24 user nova-compute[75234]:         <nova:port uuid="6dbd7c27-31c9-4ddc-a81b-85f1e742d160">
Nov 29 03:33:24 user nova-compute[75234]:           <nova:ip type="fixed" address="10.0.0.13" ipVersion="4"/>
Nov 29 03:33:24 user nova-compute[75234]:         </nova:port>
Nov 29 03:33:24 user nova-compute[75234]:       </nova:ports>
Nov 29 03:33:24 user nova-compute[75234]:     </nova:instance>
Nov 29 03:33:24 user nova-compute[75234]:   </metadata>
Nov 29 03:33:24 user nova-compute[75234]:   <sysinfo type="smbios">
Nov 29 03:33:24 user nova-compute[75234]:     <system>
Nov 29 03:33:24 user nova-compute[75234]:       <entry name="manufacturer">OpenStack Foundation</entry>
Nov 29 03:33:24 user nova-compute[75234]:       <entry name="product">OpenStack Nova</entry>
Nov 29 03:33:24 user nova-compute[75234]:       <entry name="version">0.0.0</entry>
Nov 29 03:33:24 user nova-compute[75234]:       <entry name="serial">01d62e54-9957-40b1-8dfe-1783b64ffabf</entry>
Nov 29 03:33:24 user nova-compute[75234]:       <entry name="uuid">01d62e54-9957-40b1-8dfe-1783b64ffabf</entry>
Nov 29 03:33:24 user nova-compute[75234]:       <entry name="family">Virtual Machine</entry>
Nov 29 03:33:24 user nova-compute[75234]:     </system>
Nov 29 03:33:24 user nova-compute[75234]:   </sysinfo>
Nov 29 03:33:24 user nova-compute[75234]:   <os>
Nov 29 03:33:24 user nova-compute[75234]:     <type arch="x86_64" machine="pc">hvm</type>
Nov 29 03:33:24 user nova-compute[75234]:     <boot dev="hd"/>
Nov 29 03:33:24 user nova-compute[75234]:     <smbios mode="sysinfo"/>
Nov 29 03:33:24 user nova-compute[75234]:   </os>
Nov 29 03:33:24 user nova-compute[75234]:   <features>
Nov 29 03:33:24 user nova-compute[75234]:     <acpi/>
Nov 29 03:33:24 user nova-compute[75234]:     <apic/>
Nov 29 03:33:24 user nova-compute[75234]:     <vmcoreinfo/>
Nov 29 03:33:24 user nova-compute[75234]:   </features>
Nov 29 03:33:24 user nova-compute[75234]:   <clock offset="utc">
Nov 29 03:33:24 user nova-compute[75234]:     <timer name="pit" tickpolicy="delay"/>
Nov 29 03:33:24 user nova-compute[75234]:     <timer name="rtc" tickpolicy="catchup"/>
Nov 29 03:33:24 user nova-compute[75234]:     <timer name="hpet" present="no"/>
Nov 29 03:33:24 user nova-compute[75234]:   </clock>
Nov 29 03:33:24 user nova-compute[75234]:   <cpu mode="custom" match="exact">
Nov 29 03:33:24 user nova-compute[75234]:     <model>Nehalem</model>
Nov 29 03:33:24 user nova-compute[75234]:     <topology sockets="1" cores="1" threads="1"/>
Nov 29 03:33:24 user nova-compute[75234]:   </cpu>
Nov 29 03:33:24 user nova-compute[75234]:   <devices>
Nov 29 03:33:24 user nova-compute[75234]:     <disk type="block" device="disk">
Nov 29 03:33:24 user nova-compute[75234]:       <driver name="qemu" type="raw" cache="none" io="native"/>
Nov 29 03:33:24 user nova-compute[75234]:       <source dev="/dev/sdf"/>
Nov 29 03:33:24 user nova-compute[75234]:       <target dev="vda" bus="virtio"/>
Nov 29 03:33:24 user nova-compute[75234]:       <serial>43e92d4a-7e98-4777-8fa5-3cec396b86ce</serial>
Nov 29 03:33:24 user nova-compute[75234]:     </disk>
Nov 29 03:33:24 user nova-compute[75234]:     <interface type="ethernet">
Nov 29 03:33:24 user nova-compute[75234]:       <mac address="fa:16:3e:57:6a:26"/>
Nov 29 03:33:24 user nova-compute[75234]:       <model type="virtio"/>
Nov 29 03:33:24 user nova-compute[75234]:       <mtu size="1442"/>
Nov 29 03:33:24 user nova-compute[75234]:       <target dev="tap6dbd7c27-31"/>
Nov 29 03:33:24 user nova-compute[75234]:     </interface>
Nov 29 03:33:24 user nova-compute[75234]:     <serial type="pty">
Nov 29 03:33:24 user nova-compute[75234]:       <log file="/opt/stack/data/nova/instances/01d62e54-9957-40b1-8dfe-1783b64ffabf/console.log" append="off"/>
Nov 29 03:33:24 user nova-compute[75234]:     </serial>
Nov 29 03:33:24 user nova-compute[75234]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Nov 29 03:33:24 user nova-compute[75234]:     <video>
Nov 29 03:33:24 user nova-compute[75234]:       <model type="virtio"/>
Nov 29 03:33:24 user nova-compute[75234]:     </video>
Nov 29 03:33:24 user nova-compute[75234]:     <rng model="virtio">
Nov 29 03:33:24 user nova-compute[75234]:       <backend model="random">/dev/urandom</backend>
Nov 29 03:33:24 user nova-compute[75234]:     </rng>
Nov 29 03:33:24 user nova-compute[75234]:     <controller type="usb" index="0" model="none"/>
Nov 29 03:33:24 user nova-compute[75234]:     <memballoon model="virtio">
Nov 29 03:33:24 user nova-compute[75234]:       <stats period="10"/>
Nov 29 03:33:24 user nova-compute[75234]:     </memballoon>
Nov 29 03:33:24 user nova-compute[75234]:   </devices>
Nov 29 03:33:24 user nova-compute[75234]: </domain>
Nov 29 03:33:24 user nova-compute[75234]:  {{(pid=75234) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7568}}
Nov 29 03:33:24 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-641fca7b-031e-4b3b-9491-ee221dda4df1 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=True,compute_id=1,config_drive='',created_at=2023-11-29T03:29:28Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-TestShelveInstance-server-1849172102',display_name='tempest-TestShelveInstance-server-1849172102',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-testshelveinstance-server-1849172102',id=44,image_ref='',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data=None,key_name='tempest-TestShelveInstance-1141474138',keypairs=<?>,launch_index=0,launched_at=2023-11-29T03:29:43Z,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=4,progress=0,project_id='53e55fcb8cd74895baec6f9528770bbc',ramdisk_id='',reservation_id='r-vhus4691',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',clean_attempts='1',image_base_image_ref='',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-TestShelveInstance-1683282958',owner_user_name='tempest-TestShelveInstance-1683282958-project-member'},tags=<?>,task_state='spawning',terminated_at=None,trusted_certs=<?>,updated_at=2023-11-29T03:33:14Z,user_data=None,user_id='5728da78dc074c1993a8a2cc9b821ed6',uuid=01d62e54-9957-40b1-8dfe-1783b64ffabf,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='shelved_offloaded') vif={"id": "6dbd7c27-31c9-4ddc-a81b-85f1e742d160", "address": "fa:16:3e:57:6a:26", "network": {"id": "504616f5-0d6b-47a6-aaf5-663983428b41", "bridge": "br-int", "label": "tempest-TestShelveInstance-438404840-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.210", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "53e55fcb8cd74895baec6f9528770bbc", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap6dbd7c27-31", "ovs_interfaceid": "6dbd7c27-31c9-4ddc-a81b-85f1e742d160", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Nov 29 03:33:24 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-641fca7b-031e-4b3b-9491-ee221dda4df1 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] Converting VIF {"id": "6dbd7c27-31c9-4ddc-a81b-85f1e742d160", "address": "fa:16:3e:57:6a:26", "network": {"id": "504616f5-0d6b-47a6-aaf5-663983428b41", "bridge": "br-int", "label": "tempest-TestShelveInstance-438404840-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.210", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "53e55fcb8cd74895baec6f9528770bbc", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap6dbd7c27-31", "ovs_interfaceid": "6dbd7c27-31c9-4ddc-a81b-85f1e742d160", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:33:24 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-641fca7b-031e-4b3b-9491-ee221dda4df1 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:57:6a:26,bridge_name='br-int',has_traffic_filtering=True,id=6dbd7c27-31c9-4ddc-a81b-85f1e742d160,network=Network(504616f5-0d6b-47a6-aaf5-663983428b41),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap6dbd7c27-31') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:33:24 user nova-compute[75234]: DEBUG os_vif [None req-641fca7b-031e-4b3b-9491-ee221dda4df1 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:57:6a:26,bridge_name='br-int',has_traffic_filtering=True,id=6dbd7c27-31c9-4ddc-a81b-85f1e742d160,network=Network(504616f5-0d6b-47a6-aaf5-663983428b41),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap6dbd7c27-31') {{(pid=75234) plug /opt/stack/data/venv/lib/python3.10/site-packages/os_vif/__init__.py:76}}
Nov 29 03:33:24 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:33:24 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:33:24 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Nov 29 03:33:24 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:33:24 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbCreateCommand(_result=None, table=QoS, columns={'type': 'linux-noop', 'external_ids': {'id': '94f3d8db-41ef-5088-9907-ebfda2e89d5f', '_type': 'linux-noop'}}, row=False) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:33:24 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:33:24 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:33:24 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:33:24 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tap6dbd7c27-31, may_exist=True, interface_attrs={}) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:33:24 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Port, record=tap6dbd7c27-31, col_values=(('qos', UUID('25437cf9-68c3-4b1b-bb38-6fe1184ff97e')),), if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:33:24 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DbSetCommand(_result=None, table=Interface, record=tap6dbd7c27-31, col_values=(('external_ids', {'iface-id': '6dbd7c27-31c9-4ddc-a81b-85f1e742d160', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:57:6a:26', 'vm-uuid': '01d62e54-9957-40b1-8dfe-1783b64ffabf'}),), if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:33:24 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:33:24 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:33:24 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:33:24 user nova-compute[75234]: INFO os_vif [None req-641fca7b-031e-4b3b-9491-ee221dda4df1 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:57:6a:26,bridge_name='br-int',has_traffic_filtering=True,id=6dbd7c27-31c9-4ddc-a81b-85f1e742d160,network=Network(504616f5-0d6b-47a6-aaf5-663983428b41),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap6dbd7c27-31')
Nov 29 03:33:24 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-641fca7b-031e-4b3b-9491-ee221dda4df1 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] No BDM found with device name vda, not building metadata. {{(pid=75234) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12089}}
Nov 29 03:33:24 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-641fca7b-031e-4b3b-9491-ee221dda4df1 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] No VIF found with MAC fa:16:3e:57:6a:26, not building metadata {{(pid=75234) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Nov 29 03:33:26 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:33:26 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:33:26 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:33:26 user nova-compute[75234]: DEBUG nova.compute.manager [req-1dfb9280-156b-4714-b82a-6cabd6db63aa req-b860e83e-b0fb-4321-b7eb-569d0c9235aa service nova] [instance: 01d62e54-9957-40b1-8dfe-1783b64ffabf] Received event network-vif-plugged-6dbd7c27-31c9-4ddc-a81b-85f1e742d160 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:33:26 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-1dfb9280-156b-4714-b82a-6cabd6db63aa req-b860e83e-b0fb-4321-b7eb-569d0c9235aa service nova] Acquiring lock "01d62e54-9957-40b1-8dfe-1783b64ffabf-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:33:26 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-1dfb9280-156b-4714-b82a-6cabd6db63aa req-b860e83e-b0fb-4321-b7eb-569d0c9235aa service nova] Lock "01d62e54-9957-40b1-8dfe-1783b64ffabf-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:33:26 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-1dfb9280-156b-4714-b82a-6cabd6db63aa req-b860e83e-b0fb-4321-b7eb-569d0c9235aa service nova] Lock "01d62e54-9957-40b1-8dfe-1783b64ffabf-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:33:26 user nova-compute[75234]: DEBUG nova.compute.manager [req-1dfb9280-156b-4714-b82a-6cabd6db63aa req-b860e83e-b0fb-4321-b7eb-569d0c9235aa service nova] [instance: 01d62e54-9957-40b1-8dfe-1783b64ffabf] No waiting events found dispatching network-vif-plugged-6dbd7c27-31c9-4ddc-a81b-85f1e742d160 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:33:26 user nova-compute[75234]: WARNING nova.compute.manager [req-1dfb9280-156b-4714-b82a-6cabd6db63aa req-b860e83e-b0fb-4321-b7eb-569d0c9235aa service nova] [instance: 01d62e54-9957-40b1-8dfe-1783b64ffabf] Received unexpected event network-vif-plugged-6dbd7c27-31c9-4ddc-a81b-85f1e742d160 for instance with vm_state shelved_offloaded and task_state spawning.
Nov 29 03:33:26 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:33:26 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:33:26 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:33:27 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:33:28 user nova-compute[75234]: DEBUG nova.compute.manager [None req-641fca7b-031e-4b3b-9491-ee221dda4df1 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] [instance: 01d62e54-9957-40b1-8dfe-1783b64ffabf] Instance event wait completed in 0 seconds for  {{(pid=75234) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Nov 29 03:33:28 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-641fca7b-031e-4b3b-9491-ee221dda4df1 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] [instance: 01d62e54-9957-40b1-8dfe-1783b64ffabf] Guest created on hypervisor {{(pid=75234) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4421}}
Nov 29 03:33:28 user nova-compute[75234]: INFO nova.virt.libvirt.driver [-] [instance: 01d62e54-9957-40b1-8dfe-1783b64ffabf] Instance spawned successfully.
Nov 29 03:33:28 user nova-compute[75234]: DEBUG nova.compute.manager [None req-641fca7b-031e-4b3b-9491-ee221dda4df1 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] [instance: 01d62e54-9957-40b1-8dfe-1783b64ffabf] Checking state {{(pid=75234) _get_power_state /opt/stack/nova/nova/compute/manager.py:1782}}
Nov 29 03:33:28 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-641fca7b-031e-4b3b-9491-ee221dda4df1 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] Lock "01d62e54-9957-40b1-8dfe-1783b64ffabf" "released" by "nova.compute.manager.ComputeManager.unshelve_instance.<locals>.do_unshelve_instance" :: held 14.049s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:33:28 user nova-compute[75234]: DEBUG nova.compute.manager [req-4c080aae-451f-4703-8589-0a4b6ba2ace9 req-387a3f72-1149-4234-8d32-c2e07d682b6e service nova] [instance: 01d62e54-9957-40b1-8dfe-1783b64ffabf] Received event network-vif-plugged-6dbd7c27-31c9-4ddc-a81b-85f1e742d160 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:33:28 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-4c080aae-451f-4703-8589-0a4b6ba2ace9 req-387a3f72-1149-4234-8d32-c2e07d682b6e service nova] Acquiring lock "01d62e54-9957-40b1-8dfe-1783b64ffabf-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:33:28 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-4c080aae-451f-4703-8589-0a4b6ba2ace9 req-387a3f72-1149-4234-8d32-c2e07d682b6e service nova] Lock "01d62e54-9957-40b1-8dfe-1783b64ffabf-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:33:28 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-4c080aae-451f-4703-8589-0a4b6ba2ace9 req-387a3f72-1149-4234-8d32-c2e07d682b6e service nova] Lock "01d62e54-9957-40b1-8dfe-1783b64ffabf-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:33:28 user nova-compute[75234]: DEBUG nova.compute.manager [req-4c080aae-451f-4703-8589-0a4b6ba2ace9 req-387a3f72-1149-4234-8d32-c2e07d682b6e service nova] [instance: 01d62e54-9957-40b1-8dfe-1783b64ffabf] No waiting events found dispatching network-vif-plugged-6dbd7c27-31c9-4ddc-a81b-85f1e742d160 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:33:28 user nova-compute[75234]: WARNING nova.compute.manager [req-4c080aae-451f-4703-8589-0a4b6ba2ace9 req-387a3f72-1149-4234-8d32-c2e07d682b6e service nova] [instance: 01d62e54-9957-40b1-8dfe-1783b64ffabf] Received unexpected event network-vif-plugged-6dbd7c27-31c9-4ddc-a81b-85f1e742d160 for instance with vm_state active and task_state None.
Nov 29 03:33:29 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:33:32 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:33:34 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:33:39 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:33:39 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:33:39 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5002 ms, sending inactivity probe {{(pid=75234) run /opt/stack/data/venv/lib/python3.10/site-packages/ovs/reconnect.py:117}}
Nov 29 03:33:39 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=75234) _transition /opt/stack/data/venv/lib/python3.10/site-packages/ovs/reconnect.py:519}}
Nov 29 03:33:39 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=75234) _transition /opt/stack/data/venv/lib/python3.10/site-packages/ovs/reconnect.py:519}}
Nov 29 03:33:39 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:33:42 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:33:44 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:33:46 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:33:46 user nova-compute[75234]: DEBUG nova.compute.manager [req-d2122e6d-2e6a-4acc-8441-d006bbdc8ecf req-423f3c85-7a5e-4e1f-a9b4-6191e999ec7d service nova] [instance: cb0f042e-43c0-43aa-b243-cfc78a7b2695] Received event network-changed-8f9642c3-9e10-4aa7-b5a2-53bb68ad96dc {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:33:46 user nova-compute[75234]: DEBUG nova.compute.manager [req-d2122e6d-2e6a-4acc-8441-d006bbdc8ecf req-423f3c85-7a5e-4e1f-a9b4-6191e999ec7d service nova] [instance: cb0f042e-43c0-43aa-b243-cfc78a7b2695] Refreshing instance network info cache due to event network-changed-8f9642c3-9e10-4aa7-b5a2-53bb68ad96dc. {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11117}}
Nov 29 03:33:46 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-d2122e6d-2e6a-4acc-8441-d006bbdc8ecf req-423f3c85-7a5e-4e1f-a9b4-6191e999ec7d service nova] Acquiring lock "refresh_cache-cb0f042e-43c0-43aa-b243-cfc78a7b2695" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:33:46 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-d2122e6d-2e6a-4acc-8441-d006bbdc8ecf req-423f3c85-7a5e-4e1f-a9b4-6191e999ec7d service nova] Acquired lock "refresh_cache-cb0f042e-43c0-43aa-b243-cfc78a7b2695" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:33:46 user nova-compute[75234]: DEBUG nova.network.neutron [req-d2122e6d-2e6a-4acc-8441-d006bbdc8ecf req-423f3c85-7a5e-4e1f-a9b4-6191e999ec7d service nova] [instance: cb0f042e-43c0-43aa-b243-cfc78a7b2695] Refreshing network info cache for port 8f9642c3-9e10-4aa7-b5a2-53bb68ad96dc {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2006}}
Nov 29 03:33:46 user nova-compute[75234]: DEBUG nova.network.neutron [req-d2122e6d-2e6a-4acc-8441-d006bbdc8ecf req-423f3c85-7a5e-4e1f-a9b4-6191e999ec7d service nova] [instance: cb0f042e-43c0-43aa-b243-cfc78a7b2695] Updated VIF entry in instance network info cache for port 8f9642c3-9e10-4aa7-b5a2-53bb68ad96dc. {{(pid=75234) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3481}}
Nov 29 03:33:46 user nova-compute[75234]: DEBUG nova.network.neutron [req-d2122e6d-2e6a-4acc-8441-d006bbdc8ecf req-423f3c85-7a5e-4e1f-a9b4-6191e999ec7d service nova] [instance: cb0f042e-43c0-43aa-b243-cfc78a7b2695] Updating instance_info_cache with network_info: [{"id": "8f9642c3-9e10-4aa7-b5a2-53bb68ad96dc", "address": "fa:16:3e:31:19:9f", "network": {"id": "6787e0a8-2980-4a2e-bf2a-583af955241d", "bridge": "br-int", "label": "tempest-TestMinimumBasicScenario-1273600998-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.168", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "74f132d6666c47b8b43869df6fee6de3", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap8f9642c3-9e", "ovs_interfaceid": "8f9642c3-9e10-4aa7-b5a2-53bb68ad96dc", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:33:46 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-d2122e6d-2e6a-4acc-8441-d006bbdc8ecf req-423f3c85-7a5e-4e1f-a9b4-6191e999ec7d service nova] Releasing lock "refresh_cache-cb0f042e-43c0-43aa-b243-cfc78a7b2695" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:33:46 user nova-compute[75234]: DEBUG nova.compute.manager [req-c0e2f362-603a-48be-aee1-d205a60ee448 req-07e672f9-d688-41bb-acbb-ade34fed03da service nova] [instance: cb0f042e-43c0-43aa-b243-cfc78a7b2695] Received event network-changed-8f9642c3-9e10-4aa7-b5a2-53bb68ad96dc {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:33:46 user nova-compute[75234]: DEBUG nova.compute.manager [req-c0e2f362-603a-48be-aee1-d205a60ee448 req-07e672f9-d688-41bb-acbb-ade34fed03da service nova] [instance: cb0f042e-43c0-43aa-b243-cfc78a7b2695] Refreshing instance network info cache due to event network-changed-8f9642c3-9e10-4aa7-b5a2-53bb68ad96dc. {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11117}}
Nov 29 03:33:46 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-c0e2f362-603a-48be-aee1-d205a60ee448 req-07e672f9-d688-41bb-acbb-ade34fed03da service nova] Acquiring lock "refresh_cache-cb0f042e-43c0-43aa-b243-cfc78a7b2695" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:33:46 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-c0e2f362-603a-48be-aee1-d205a60ee448 req-07e672f9-d688-41bb-acbb-ade34fed03da service nova] Acquired lock "refresh_cache-cb0f042e-43c0-43aa-b243-cfc78a7b2695" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:33:46 user nova-compute[75234]: DEBUG nova.network.neutron [req-c0e2f362-603a-48be-aee1-d205a60ee448 req-07e672f9-d688-41bb-acbb-ade34fed03da service nova] [instance: cb0f042e-43c0-43aa-b243-cfc78a7b2695] Refreshing network info cache for port 8f9642c3-9e10-4aa7-b5a2-53bb68ad96dc {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2006}}
Nov 29 03:33:47 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-374f0ad9-c6c2-4f58-8d0d-ef12f98d4485 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Acquiring lock "cb0f042e-43c0-43aa-b243-cfc78a7b2695" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:33:47 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-374f0ad9-c6c2-4f58-8d0d-ef12f98d4485 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Lock "cb0f042e-43c0-43aa-b243-cfc78a7b2695" acquired by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:33:47 user nova-compute[75234]: INFO nova.compute.manager [None req-374f0ad9-c6c2-4f58-8d0d-ef12f98d4485 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] [instance: cb0f042e-43c0-43aa-b243-cfc78a7b2695] Detaching volume d7d89638-2f88-47e2-a43c-b37e8919ecbc
Nov 29 03:33:47 user nova-compute[75234]: INFO nova.virt.block_device [None req-374f0ad9-c6c2-4f58-8d0d-ef12f98d4485 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] [instance: cb0f042e-43c0-43aa-b243-cfc78a7b2695] Attempting to driver detach volume d7d89638-2f88-47e2-a43c-b37e8919ecbc from mountpoint /dev/vdb
Nov 29 03:33:47 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-374f0ad9-c6c2-4f58-8d0d-ef12f98d4485 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Attempting to detach device vdb from instance cb0f042e-43c0-43aa-b243-cfc78a7b2695 from the persistent domain config. {{(pid=75234) _detach_from_persistent /opt/stack/nova/nova/virt/libvirt/driver.py:2487}}
Nov 29 03:33:47 user nova-compute[75234]: DEBUG nova.virt.libvirt.guest [None req-374f0ad9-c6c2-4f58-8d0d-ef12f98d4485 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] detach device xml: <disk type="block" device="disk">
Nov 29 03:33:47 user nova-compute[75234]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Nov 29 03:33:47 user nova-compute[75234]:   <source dev="/dev/sdb"/>
Nov 29 03:33:47 user nova-compute[75234]:   <target dev="vdb" bus="virtio"/>
Nov 29 03:33:47 user nova-compute[75234]:   <serial>d7d89638-2f88-47e2-a43c-b37e8919ecbc</serial>
Nov 29 03:33:47 user nova-compute[75234]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x07" function="0x0"/>
Nov 29 03:33:47 user nova-compute[75234]: </disk>
Nov 29 03:33:47 user nova-compute[75234]:  {{(pid=75234) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Nov 29 03:33:47 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-374f0ad9-c6c2-4f58-8d0d-ef12f98d4485 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Successfully detached device vdb from instance cb0f042e-43c0-43aa-b243-cfc78a7b2695 from the persistent domain config.
Nov 29 03:33:47 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-374f0ad9-c6c2-4f58-8d0d-ef12f98d4485 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] (1/8): Attempting to detach device vdb with device alias virtio-disk1 from instance cb0f042e-43c0-43aa-b243-cfc78a7b2695 from the live domain config. {{(pid=75234) _detach_from_live_with_retry /opt/stack/nova/nova/virt/libvirt/driver.py:2523}}
Nov 29 03:33:47 user nova-compute[75234]: DEBUG nova.virt.libvirt.guest [None req-374f0ad9-c6c2-4f58-8d0d-ef12f98d4485 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] detach device xml: <disk type="block" device="disk">
Nov 29 03:33:47 user nova-compute[75234]:   <driver name="qemu" type="raw" cache="none" io="native"/>
Nov 29 03:33:47 user nova-compute[75234]:   <source dev="/dev/sdb"/>
Nov 29 03:33:47 user nova-compute[75234]:   <target dev="vdb" bus="virtio"/>
Nov 29 03:33:47 user nova-compute[75234]:   <serial>d7d89638-2f88-47e2-a43c-b37e8919ecbc</serial>
Nov 29 03:33:47 user nova-compute[75234]:   <address type="pci" domain="0x0000" bus="0x00" slot="0x07" function="0x0"/>
Nov 29 03:33:47 user nova-compute[75234]: </disk>
Nov 29 03:33:47 user nova-compute[75234]:  {{(pid=75234) detach_device /opt/stack/nova/nova/virt/libvirt/guest.py:465}}
Nov 29 03:33:47 user nova-compute[75234]: DEBUG nova.network.neutron [req-c0e2f362-603a-48be-aee1-d205a60ee448 req-07e672f9-d688-41bb-acbb-ade34fed03da service nova] [instance: cb0f042e-43c0-43aa-b243-cfc78a7b2695] Updated VIF entry in instance network info cache for port 8f9642c3-9e10-4aa7-b5a2-53bb68ad96dc. {{(pid=75234) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3481}}
Nov 29 03:33:47 user nova-compute[75234]: DEBUG nova.network.neutron [req-c0e2f362-603a-48be-aee1-d205a60ee448 req-07e672f9-d688-41bb-acbb-ade34fed03da service nova] [instance: cb0f042e-43c0-43aa-b243-cfc78a7b2695] Updating instance_info_cache with network_info: [{"id": "8f9642c3-9e10-4aa7-b5a2-53bb68ad96dc", "address": "fa:16:3e:31:19:9f", "network": {"id": "6787e0a8-2980-4a2e-bf2a-583af955241d", "bridge": "br-int", "label": "tempest-TestMinimumBasicScenario-1273600998-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "74f132d6666c47b8b43869df6fee6de3", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap8f9642c3-9e", "ovs_interfaceid": "8f9642c3-9e10-4aa7-b5a2-53bb68ad96dc", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:33:47 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-c0e2f362-603a-48be-aee1-d205a60ee448 req-07e672f9-d688-41bb-acbb-ade34fed03da service nova] Releasing lock "refresh_cache-cb0f042e-43c0-43aa-b243-cfc78a7b2695" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:33:47 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-374f0ad9-c6c2-4f58-8d0d-ef12f98d4485 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Start waiting for the detach event from libvirt for device vdb with device alias virtio-disk1 for instance cb0f042e-43c0-43aa-b243-cfc78a7b2695 {{(pid=75234) _detach_from_live_and_wait_for_event /opt/stack/nova/nova/virt/libvirt/driver.py:2599}}
Nov 29 03:33:47 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:33:49 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:33:52 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:33:54 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:33:57 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:33:59 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:34:04 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:34:07 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-374f0ad9-c6c2-4f58-8d0d-ef12f98d4485 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Waiting for libvirt event about the detach of device vdb with device alias virtio-disk1 from instance cb0f042e-43c0-43aa-b243-cfc78a7b2695 is timed out.
Nov 29 03:34:07 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-374f0ad9-c6c2-4f58-8d0d-ef12f98d4485 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Successfully detached device vdb from instance cb0f042e-43c0-43aa-b243-cfc78a7b2695 from the live domain config.
Nov 29 03:34:07 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-374f0ad9-c6c2-4f58-8d0d-ef12f98d4485 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] [instance: cb0f042e-43c0-43aa-b243-cfc78a7b2695] calling os-brick to detach iSCSI Volume {{(pid=75234) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Nov 29 03:34:07 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-374f0ad9-c6c2-4f58-8d0d-ef12f98d4485 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f8a5050c9a0>, {'target_iqn': 'iqn.2010-10.org.openstack:d7d89638-2f88-47e2-a43c-b37e8919ecbc', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'YPIlLaJA', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdb'}, None), 'kwargs': False}}" {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:176}}
Nov 29 03:34:07 user nova-compute[75234]: WARNING os_brick.initiator.connectors.base [None req-374f0ad9-c6c2-4f58-8d0d-ef12f98d4485 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Nov 29 03:34:07 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-374f0ad9-c6c2-4f58-8d0d-ef12f98d4485 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:68}}
Nov 29 03:34:07 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-374f0ad9-c6c2-4f58-8d0d-ef12f98d4485 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:73}}
Nov 29 03:34:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:34:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.022s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:34:07 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[274df1ff-1f3a-4907-a77d-d62c87c56826]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:a3d30ae4-ca6d-4e06-8fc9-caf4d91ef900\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:43e92d4a-7e98-4777-8fa5-3cec396b86ce\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:257d7345-5b29-4bc1-8844-e211864c06c3\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:d7d89638-2f88-47e2-a43c-b37e8919ecbc\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:34:07 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-374f0ad9-c6c2-4f58-8d0d-ef12f98d4485 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS:
Nov 29 03:34:07 user nova-compute[75234]: No targets found.
Nov 29 03:34:07 user nova-compute[75234]: iSNS:
Nov 29 03:34:07 user nova-compute[75234]: No targets found.
Nov 29 03:34:07 user nova-compute[75234]: STATIC:
Nov 29 03:34:07 user nova-compute[75234]: Target: iqn.2010-10.org.openstack:a3d30ae4-ca6d-4e06-8fc9-caf4d91ef900
Nov 29 03:34:07 user nova-compute[75234]:         Portal: 172.16.0.220:3260,-1
Nov 29 03:34:07 user nova-compute[75234]:                 Iface Name: default
Nov 29 03:34:07 user nova-compute[75234]: Target: iqn.2010-10.org.openstack:43e92d4a-7e98-4777-8fa5-3cec396b86ce
Nov 29 03:34:07 user nova-compute[75234]:         Portal: 172.16.0.220:3260,-1
Nov 29 03:34:07 user nova-compute[75234]:                 Iface Name: default
Nov 29 03:34:07 user nova-compute[75234]: Target: iqn.2010-10.org.openstack:257d7345-5b29-4bc1-8844-e211864c06c3
Nov 29 03:34:07 user nova-compute[75234]:         Portal: 172.16.0.220:3260,-1
Nov 29 03:34:07 user nova-compute[75234]:                 Iface Name: default
Nov 29 03:34:07 user nova-compute[75234]: Target: iqn.2010-10.org.openstack:d7d89638-2f88-47e2-a43c-b37e8919ecbc
Nov 29 03:34:07 user nova-compute[75234]:         Portal: 172.16.0.220:3260,-1
Nov 29 03:34:07 user nova-compute[75234]:                 Iface Name: default
Nov 29 03:34:07 user nova-compute[75234]: Target: iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0
Nov 29 03:34:07 user nova-compute[75234]:         Portal: 172.16.0.220:3260,-1
Nov 29 03:34:07 user nova-compute[75234]:                 Iface Name: default
Nov 29 03:34:07 user nova-compute[75234]: FIRMWARE:
Nov 29 03:34:07 user nova-compute[75234]: No targets found.
Nov 29 03:34:07 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:34:07 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-374f0ad9-c6c2-4f58-8d0d-ef12f98d4485 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Regex to get portals from discoverydb: ^SENDTARGETS:
Nov 29 03:34:07 user nova-compute[75234]: .*?^DiscoveryAddress: 172.16.0.220,3260.*?
Nov 29 03:34:07 user nova-compute[75234]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=75234) _get_discoverydb_portals /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:382}}
Nov 29 03:34:07 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-374f0ad9-c6c2-4f58-8d0d-ef12f98d4485 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:d7d89638-2f88-47e2-a43c-b37e8919ecbc', 0)] {{(pid=75234) _get_connection_devices /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:830}}
Nov 29 03:34:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:34:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.018s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:34:07 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[644ae07b-e24b-4467-b8ca-efb6c1885514]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:257d7345-5b29-4bc1-8844-e211864c06c3\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:43e92d4a-7e98-4777-8fa5-3cec396b86ce\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:a3d30ae4-ca6d-4e06-8fc9-caf4d91ef900\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:d7d89638-2f88-47e2-a43c-b37e8919ecbc\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:34:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:34:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.027s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:34:07 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[834ff795-2a74-41c5-b791-6c568adfb77f]: (4, ('tcp: [42] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0 (non-flash)\ntcp: [53] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d7d89638-2f88-47e2-a43c-b37e8919ecbc (non-flash)\ntcp: [54] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a3d30ae4-ca6d-4e06-8fc9-caf4d91ef900 (non-flash)\ntcp: [55] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:257d7345-5b29-4bc1-8844-e211864c06c3 (non-flash)\ntcp: [56] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:43e92d4a-7e98-4777-8fa5-3cec396b86ce (non-flash)\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:34:07 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-374f0ad9-c6c2-4f58-8d0d-ef12f98d4485 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [42] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0 (non-flash)
Nov 29 03:34:07 user nova-compute[75234]: tcp: [53] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d7d89638-2f88-47e2-a43c-b37e8919ecbc (non-flash)
Nov 29 03:34:07 user nova-compute[75234]: tcp: [54] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a3d30ae4-ca6d-4e06-8fc9-caf4d91ef900 (non-flash)
Nov 29 03:34:07 user nova-compute[75234]: tcp: [55] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:257d7345-5b29-4bc1-8844-e211864c06c3 (non-flash)
Nov 29 03:34:07 user nova-compute[75234]: tcp: [56] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:43e92d4a-7e98-4777-8fa5-3cec396b86ce (non-flash)
Nov 29 03:34:07 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:34:07 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-374f0ad9-c6c2-4f58-8d0d-ef12f98d4485 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] iscsi session list stdout=tcp: [42] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0 (non-flash)
Nov 29 03:34:07 user nova-compute[75234]: tcp: [53] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:d7d89638-2f88-47e2-a43c-b37e8919ecbc (non-flash)
Nov 29 03:34:07 user nova-compute[75234]: tcp: [54] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a3d30ae4-ca6d-4e06-8fc9-caf4d91ef900 (non-flash)
Nov 29 03:34:07 user nova-compute[75234]: tcp: [55] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:257d7345-5b29-4bc1-8844-e211864c06c3 (non-flash)
Nov 29 03:34:07 user nova-compute[75234]: tcp: [56] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:43e92d4a-7e98-4777-8fa5-3cec396b86ce (non-flash)
Nov 29 03:34:07 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsi_session /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Nov 29 03:34:07 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-374f0ad9-c6c2-4f58-8d0d-ef12f98d4485 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7f8a5032f5b0>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:d7d89638-2f88-47e2-a43c-b37e8919ecbc'): ({'sdb'}, set())}) {{(pid=75234) _get_connection_devices /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:863}}
Nov 29 03:34:07 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-374f0ad9-c6c2-4f58-8d0d-ef12f98d4485 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Removing single pathed devices sdb {{(pid=75234) remove_connection /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:374}}
Nov 29 03:34:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:34:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.018s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:34:07 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[aa4c7027-63e7-43db-bc86-380eafdb5b0b]: (4, ('path checker states:\nup                  5\n\npaths: 0\nbusy: False\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:34:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdb {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:34:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdb" returned: 0 in 0.020s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:34:07 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[818d2fea-c83d-4b21-b187-a034dd31f4d0]: (4, ('ok\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:34:07 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-374f0ad9-c6c2-4f58-8d0d-ef12f98d4485 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Flushing IO for device /dev/sdb {{(pid=75234) flush_device_io /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:434}}
Nov 29 03:34:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdb {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:34:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdb" returned: 0 in 0.017s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:34:07 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[a53cc6e4-9650-4300-9333-880bd6333c13]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:34:07 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-374f0ad9-c6c2-4f58-8d0d-ef12f98d4485 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Remove SCSI device /dev/sdb with /sys/block/sdb/device/delete {{(pid=75234) remove_scsi_device /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:148}}
Nov 29 03:34:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdb/device/delete {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:34:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdb/device/delete" returned: 0 in 0.039s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:34:07 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[2e743157-5506-4d90-a98d-70e338c76acd]: (4, ('1', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:34:07 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-374f0ad9-c6c2-4f58-8d0d-ef12f98d4485 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Checking to see if SCSI volumes sdb have been removed. {{(pid=75234) wait_for_volumes_removal /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:156}}
Nov 29 03:34:07 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-374f0ad9-c6c2-4f58-8d0d-ef12f98d4485 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] SCSI volumes sdb have been removed. {{(pid=75234) wait_for_volumes_removal /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:166}}
Nov 29 03:34:07 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-374f0ad9-c6c2-4f58-8d0d-ef12f98d4485 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:d7d89638-2f88-47e2-a43c-b37e8919ecbc')] {{(pid=75234) _disconnect_connection /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Nov 29 03:34:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:d7d89638-2f88-47e2-a43c-b37e8919ecbc -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:34:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:d7d89638-2f88-47e2-a43c-b37e8919ecbc -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.021s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:34:07 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[e913c14f-6310-45eb-a2d4-ff0d6b1f366f]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:34:07 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-374f0ad9-c6c2-4f58-8d0d-ef12f98d4485 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:34:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:d7d89638-2f88-47e2-a43c-b37e8919ecbc -p 172.16.0.220:3260 --logout {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:34:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:d7d89638-2f88-47e2-a43c-b37e8919ecbc -p 172.16.0.220:3260 --logout" returned: 0 in 0.049s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:34:07 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[7ecbd0f6-fbc1-47f3-a53d-24139f2199fe]: (4, ('Logging out of session [sid: 53, target: iqn.2010-10.org.openstack:d7d89638-2f88-47e2-a43c-b37e8919ecbc, portal: 172.16.0.220,3260]\nLogout of [sid: 53, target: iqn.2010-10.org.openstack:d7d89638-2f88-47e2-a43c-b37e8919ecbc, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:34:07 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-374f0ad9-c6c2-4f58-8d0d-ef12f98d4485 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 53, target: iqn.2010-10.org.openstack:d7d89638-2f88-47e2-a43c-b37e8919ecbc, portal: 172.16.0.220,3260]
Nov 29 03:34:07 user nova-compute[75234]: Logout of [sid: 53, target: iqn.2010-10.org.openstack:d7d89638-2f88-47e2-a43c-b37e8919ecbc, portal: 172.16.0.220,3260] successful.
Nov 29 03:34:07 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:34:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:d7d89638-2f88-47e2-a43c-b37e8919ecbc -p 172.16.0.220:3260 --op delete {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:34:08 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:d7d89638-2f88-47e2-a43c-b37e8919ecbc -p 172.16.0.220:3260 --op delete" returned: 0 in 0.027s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:34:08 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[822780c0-8c52-4bd8-aff0-be839fc53d72]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:34:08 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-374f0ad9-c6c2-4f58-8d0d-ef12f98d4485 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:34:08 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-374f0ad9-c6c2-4f58-8d0d-ef12f98d4485 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.307s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:87}}
Nov 29 03:34:08 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-374f0ad9-c6c2-4f58-8d0d-ef12f98d4485 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] <== disconnect_volume: return (307ms) None {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:203}}
Nov 29 03:34:08 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-374f0ad9-c6c2-4f58-8d0d-ef12f98d4485 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] [instance: cb0f042e-43c0-43aa-b243-cfc78a7b2695] Disconnected iSCSI Volume {{(pid=75234) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:79}}
Nov 29 03:34:08 user nova-compute[75234]: DEBUG nova.objects.instance [None req-374f0ad9-c6c2-4f58-8d0d-ef12f98d4485 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Lazy-loading 'flavor' on Instance uuid cb0f042e-43c0-43aa-b243-cfc78a7b2695 {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:34:09 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-374f0ad9-c6c2-4f58-8d0d-ef12f98d4485 tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Lock "cb0f042e-43c0-43aa-b243-cfc78a7b2695" "released" by "nova.compute.manager.ComputeManager.detach_volume.<locals>.do_detach_volume" :: held 21.877s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:34:09 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:34:13 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:34:13 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:34:13 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:34:13 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=75234) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10541}}
Nov 29 03:34:14 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:34:14 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:34:14 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:34:14 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5003 ms, sending inactivity probe {{(pid=75234) run /opt/stack/data/venv/lib/python3.10/site-packages/ovs/reconnect.py:117}}
Nov 29 03:34:14 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=75234) _transition /opt/stack/data/venv/lib/python3.10/site-packages/ovs/reconnect.py:519}}
Nov 29 03:34:14 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=75234) _transition /opt/stack/data/venv/lib/python3.10/site-packages/ovs/reconnect.py:519}}
Nov 29 03:34:14 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:34:16 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager.update_available_resource {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:34:16 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:34:16 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:34:16 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:34:16 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Auditing locally available compute resources for user (node: user) {{(pid=75234) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:907}}
Nov 29 03:34:16 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/65d48c93-b6da-4f51-9d5a-0676dda62a60/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:34:16 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/65d48c93-b6da-4f51-9d5a-0676dda62a60/disk --force-share --output=json" returned: 0 in 0.180s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:34:16 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/65d48c93-b6da-4f51-9d5a-0676dda62a60/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:34:17 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/65d48c93-b6da-4f51-9d5a-0676dda62a60/disk --force-share --output=json" returned: 0 in 0.171s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:34:17 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] skipping disk /dev/sdf (vda) as it is a volume {{(pid=75234) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11213}}
Nov 29 03:34:17 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] skipping disk /dev/sde (vda) as it is a volume {{(pid=75234) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11213}}
Nov 29 03:34:17 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] skipping disk /dev/sdc (vda) as it is a volume {{(pid=75234) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11213}}
Nov 29 03:34:17 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] skipping disk /dev/sdd (vda) as it is a volume {{(pid=75234) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11213}}
Nov 29 03:34:17 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/abb01889-df4b-44a6-9827-87056d7ca125/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:34:17 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/abb01889-df4b-44a6-9827-87056d7ca125/disk --force-share --output=json" returned: 0 in 0.165s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:34:17 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/abb01889-df4b-44a6-9827-87056d7ca125/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:34:17 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/abb01889-df4b-44a6-9827-87056d7ca125/disk --force-share --output=json" returned: 0 in 0.169s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:34:17 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/cb0f042e-43c0-43aa-b243-cfc78a7b2695/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:34:17 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/cb0f042e-43c0-43aa-b243-cfc78a7b2695/disk --force-share --output=json" returned: 0 in 0.175s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:34:17 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/cb0f042e-43c0-43aa-b243-cfc78a7b2695/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:34:17 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/cb0f042e-43c0-43aa-b243-cfc78a7b2695/disk --force-share --output=json" returned: 0 in 0.166s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:34:18 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:34:18 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:34:18 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Hypervisor/Node resource view: name=user free_ram=8215MB free_disk=24.969783782958984GB free_vcpus=5 pci_devices=[{"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}] {{(pid=75234) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1106}}
Nov 29 03:34:18 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:34:18 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:34:18 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance 65d48c93-b6da-4f51-9d5a-0676dda62a60 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 512, 'DISK_GB': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:34:18 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance d5096ca2-d005-45ac-a487-11a810446859 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 512}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:34:18 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance abb01889-df4b-44a6-9827-87056d7ca125 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 512, 'DISK_GB': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:34:18 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance cb0f042e-43c0-43aa-b243-cfc78a7b2695 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 512, 'DISK_GB': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:34:18 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance ebc3e3b2-037d-43ef-9426-caebe298ee3e actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 512}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:34:18 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance 884589ec-b86c-4094-9a3f-54b641311d62 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 512}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:34:18 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance 01d62e54-9957-40b1-8dfe-1783b64ffabf actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 512}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:34:18 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Total usable vcpus: 12, total allocated vcpus: 7 {{(pid=75234) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1129}}
Nov 29 03:34:18 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Final resource view: name=user phys_ram=16011MB used_ram=4096MB phys_disk=40GB used_disk=3GB total_vcpus=12 used_vcpus=7 pci_stats=[] {{(pid=75234) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1138}}
Nov 29 03:34:18 user nova-compute[75234]: DEBUG nova.compute.provider_tree [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Inventory has not changed in ProviderTree for provider: 4d291220-7c37-4839-a0e3-b23c66502b74 {{(pid=75234) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Nov 29 03:34:18 user nova-compute[75234]: DEBUG nova.scheduler.client.report [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Inventory has not changed for provider 4d291220-7c37-4839-a0e3-b23c66502b74 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=75234) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Nov 29 03:34:18 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Compute_service record updated for user:user {{(pid=75234) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:1067}}
Nov 29 03:34:18 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.342s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:34:18 user nova-compute[75234]: INFO nova.compute.manager [None req-3b177ed1-ffba-4658-a45d-a282d4e771bb tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] [instance: ebc3e3b2-037d-43ef-9426-caebe298ee3e] Rescuing
Nov 29 03:34:18 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-3b177ed1-ffba-4658-a45d-a282d4e771bb tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Acquiring lock "refresh_cache-ebc3e3b2-037d-43ef-9426-caebe298ee3e" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:34:18 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-3b177ed1-ffba-4658-a45d-a282d4e771bb tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Acquired lock "refresh_cache-ebc3e3b2-037d-43ef-9426-caebe298ee3e" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:34:18 user nova-compute[75234]: DEBUG nova.network.neutron [None req-3b177ed1-ffba-4658-a45d-a282d4e771bb tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] [instance: ebc3e3b2-037d-43ef-9426-caebe298ee3e] Building network info cache for instance {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2009}}
Nov 29 03:34:19 user nova-compute[75234]: DEBUG nova.network.neutron [None req-3b177ed1-ffba-4658-a45d-a282d4e771bb tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] [instance: ebc3e3b2-037d-43ef-9426-caebe298ee3e] Updating instance_info_cache with network_info: [{"id": "bd0a4e5b-69cd-4f5c-ae22-994e4d8f2682", "address": "fa:16:3e:c2:aa:3e", "network": {"id": "4d086e08-7798-4cfe-9365-86be6f0a52d2", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-305624008-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "db00bcd0c82c44f1bda739e9857f6e84", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapbd0a4e5b-69", "ovs_interfaceid": "bd0a4e5b-69cd-4f5c-ae22-994e4d8f2682", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:34:19 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-3b177ed1-ffba-4658-a45d-a282d4e771bb tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Releasing lock "refresh_cache-ebc3e3b2-037d-43ef-9426-caebe298ee3e" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:34:19 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:34:19 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:34:19 user nova-compute[75234]: DEBUG nova.compute.manager [req-a152115d-1a27-4a79-af5c-4261a5070ba9 req-42efb991-d706-4a63-91a5-b0613006b670 service nova] [instance: ebc3e3b2-037d-43ef-9426-caebe298ee3e] Received event network-vif-unplugged-bd0a4e5b-69cd-4f5c-ae22-994e4d8f2682 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:34:19 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-a152115d-1a27-4a79-af5c-4261a5070ba9 req-42efb991-d706-4a63-91a5-b0613006b670 service nova] Acquiring lock "ebc3e3b2-037d-43ef-9426-caebe298ee3e-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:34:19 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-a152115d-1a27-4a79-af5c-4261a5070ba9 req-42efb991-d706-4a63-91a5-b0613006b670 service nova] Lock "ebc3e3b2-037d-43ef-9426-caebe298ee3e-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:34:19 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-a152115d-1a27-4a79-af5c-4261a5070ba9 req-42efb991-d706-4a63-91a5-b0613006b670 service nova] Lock "ebc3e3b2-037d-43ef-9426-caebe298ee3e-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:34:19 user nova-compute[75234]: DEBUG nova.compute.manager [req-a152115d-1a27-4a79-af5c-4261a5070ba9 req-42efb991-d706-4a63-91a5-b0613006b670 service nova] [instance: ebc3e3b2-037d-43ef-9426-caebe298ee3e] No waiting events found dispatching network-vif-unplugged-bd0a4e5b-69cd-4f5c-ae22-994e4d8f2682 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:34:19 user nova-compute[75234]: WARNING nova.compute.manager [req-a152115d-1a27-4a79-af5c-4261a5070ba9 req-42efb991-d706-4a63-91a5-b0613006b670 service nova] [instance: ebc3e3b2-037d-43ef-9426-caebe298ee3e] Received unexpected event network-vif-unplugged-bd0a4e5b-69cd-4f5c-ae22-994e4d8f2682 for instance with vm_state active and task_state rescuing.
Nov 29 03:34:19 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:34:19 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:34:19 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:34:19 user nova-compute[75234]: INFO nova.virt.libvirt.driver [-] [instance: ebc3e3b2-037d-43ef-9426-caebe298ee3e] Instance destroyed successfully.
Nov 29 03:34:19 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-3b177ed1-ffba-4658-a45d-a282d4e771bb tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] [instance: ebc3e3b2-037d-43ef-9426-caebe298ee3e] Attempting a stable device rescue
Nov 29 03:34:20 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-3b177ed1-ffba-4658-a45d-a282d4e771bb tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] rescue generated disk_info: {'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, '/dev/vda': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk.rescue': {'bus': 'virtio', 'dev': 'vdb', 'type': 'disk'}}} {{(pid=75234) rescue /opt/stack/nova/nova/virt/libvirt/driver.py:4318}}
Nov 29 03:34:20 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-3b177ed1-ffba-4658-a45d-a282d4e771bb tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] [instance: ebc3e3b2-037d-43ef-9426-caebe298ee3e] Instance directory exists: not creating {{(pid=75234) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4723}}
Nov 29 03:34:20 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-3b177ed1-ffba-4658-a45d-a282d4e771bb tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] [instance: ebc3e3b2-037d-43ef-9426-caebe298ee3e] Creating image(s)
Nov 29 03:34:20 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-3b177ed1-ffba-4658-a45d-a282d4e771bb tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Acquiring lock "/opt/stack/data/nova/instances/ebc3e3b2-037d-43ef-9426-caebe298ee3e/disk.info" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:34:20 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-3b177ed1-ffba-4658-a45d-a282d4e771bb tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Lock "/opt/stack/data/nova/instances/ebc3e3b2-037d-43ef-9426-caebe298ee3e/disk.info" acquired by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:34:20 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-3b177ed1-ffba-4658-a45d-a282d4e771bb tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Lock "/opt/stack/data/nova/instances/ebc3e3b2-037d-43ef-9426-caebe298ee3e/disk.info" "released" by "nova.virt.libvirt.imagebackend.Image.resolve_driver_format.<locals>.write_to_disk_info_file" :: held 0.002s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:34:20 user nova-compute[75234]: DEBUG nova.objects.instance [None req-3b177ed1-ffba-4658-a45d-a282d4e771bb tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Lazy-loading 'trusted_certs' on Instance uuid ebc3e3b2-037d-43ef-9426-caebe298ee3e {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:34:20 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-3b177ed1-ffba-4658-a45d-a282d4e771bb tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Acquiring lock "a3fb13f48aa8955e4a2ec143d3ba27674e6f5fdd" by "nova.virt.libvirt.imagebackend.Image.cache.<locals>.fetch_func_sync" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:34:20 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-3b177ed1-ffba-4658-a45d-a282d4e771bb tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Lock "a3fb13f48aa8955e4a2ec143d3ba27674e6f5fdd" acquired by "nova.virt.libvirt.imagebackend.Image.cache.<locals>.fetch_func_sync" :: waited 0.002s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:34:21 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-3b177ed1-ffba-4658-a45d-a282d4e771bb tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/a3fb13f48aa8955e4a2ec143d3ba27674e6f5fdd.part --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:34:21 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:34:21 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Starting heal instance info cache {{(pid=75234) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9922}}
Nov 29 03:34:21 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Acquiring lock "refresh_cache-01d62e54-9957-40b1-8dfe-1783b64ffabf" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:34:21 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Acquired lock "refresh_cache-01d62e54-9957-40b1-8dfe-1783b64ffabf" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:34:21 user nova-compute[75234]: DEBUG nova.network.neutron [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] [instance: 01d62e54-9957-40b1-8dfe-1783b64ffabf] Forcefully refreshing network info cache for instance {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2003}}
Nov 29 03:34:21 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-3b177ed1-ffba-4658-a45d-a282d4e771bb tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/a3fb13f48aa8955e4a2ec143d3ba27674e6f5fdd.part --force-share --output=json" returned: 0 in 0.172s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:34:21 user nova-compute[75234]: DEBUG nova.virt.images [None req-3b177ed1-ffba-4658-a45d-a282d4e771bb tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] cdfa8f6f-8ff6-4e7c-b0e4-bd3e16db3789 was qcow2, converting to raw {{(pid=75234) fetch_to_raw /opt/stack/nova/nova/virt/images.py:165}}
Nov 29 03:34:21 user nova-compute[75234]: DEBUG nova.privsep.utils [None req-3b177ed1-ffba-4658-a45d-a282d4e771bb tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Path '/opt/stack/data/nova/instances' supports direct I/O {{(pid=75234) supports_direct_io /opt/stack/nova/nova/privsep/utils.py:63}}
Nov 29 03:34:21 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-3b177ed1-ffba-4658-a45d-a282d4e771bb tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Running cmd (subprocess): qemu-img convert -t none -O raw -f qcow2 /opt/stack/data/nova/instances/_base/a3fb13f48aa8955e4a2ec143d3ba27674e6f5fdd.part /opt/stack/data/nova/instances/_base/a3fb13f48aa8955e4a2ec143d3ba27674e6f5fdd.converted {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:34:21 user nova-compute[75234]: DEBUG nova.compute.manager [req-2348066c-3ce5-43db-8980-6e11ba0edfc6 req-524f735b-be45-4ba1-b563-0bd832515618 service nova] [instance: ebc3e3b2-037d-43ef-9426-caebe298ee3e] Received event network-vif-plugged-bd0a4e5b-69cd-4f5c-ae22-994e4d8f2682 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:34:21 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-2348066c-3ce5-43db-8980-6e11ba0edfc6 req-524f735b-be45-4ba1-b563-0bd832515618 service nova] Acquiring lock "ebc3e3b2-037d-43ef-9426-caebe298ee3e-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:34:21 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-2348066c-3ce5-43db-8980-6e11ba0edfc6 req-524f735b-be45-4ba1-b563-0bd832515618 service nova] Lock "ebc3e3b2-037d-43ef-9426-caebe298ee3e-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:34:21 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-2348066c-3ce5-43db-8980-6e11ba0edfc6 req-524f735b-be45-4ba1-b563-0bd832515618 service nova] Lock "ebc3e3b2-037d-43ef-9426-caebe298ee3e-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:34:21 user nova-compute[75234]: DEBUG nova.compute.manager [req-2348066c-3ce5-43db-8980-6e11ba0edfc6 req-524f735b-be45-4ba1-b563-0bd832515618 service nova] [instance: ebc3e3b2-037d-43ef-9426-caebe298ee3e] No waiting events found dispatching network-vif-plugged-bd0a4e5b-69cd-4f5c-ae22-994e4d8f2682 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:34:21 user nova-compute[75234]: WARNING nova.compute.manager [req-2348066c-3ce5-43db-8980-6e11ba0edfc6 req-524f735b-be45-4ba1-b563-0bd832515618 service nova] [instance: ebc3e3b2-037d-43ef-9426-caebe298ee3e] Received unexpected event network-vif-plugged-bd0a4e5b-69cd-4f5c-ae22-994e4d8f2682 for instance with vm_state active and task_state rescuing.
Nov 29 03:34:21 user nova-compute[75234]: DEBUG nova.network.neutron [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] [instance: 01d62e54-9957-40b1-8dfe-1783b64ffabf] Updating instance_info_cache with network_info: [{"id": "6dbd7c27-31c9-4ddc-a81b-85f1e742d160", "address": "fa:16:3e:57:6a:26", "network": {"id": "504616f5-0d6b-47a6-aaf5-663983428b41", "bridge": "br-int", "label": "tempest-TestShelveInstance-438404840-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.210", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "53e55fcb8cd74895baec6f9528770bbc", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap6dbd7c27-31", "ovs_interfaceid": "6dbd7c27-31c9-4ddc-a81b-85f1e742d160", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:34:21 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Releasing lock "refresh_cache-01d62e54-9957-40b1-8dfe-1783b64ffabf" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:34:21 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] [instance: 01d62e54-9957-40b1-8dfe-1783b64ffabf] Updated the network info_cache for instance {{(pid=75234) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9993}}
Nov 29 03:34:21 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-3b177ed1-ffba-4658-a45d-a282d4e771bb tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] CMD "qemu-img convert -t none -O raw -f qcow2 /opt/stack/data/nova/instances/_base/a3fb13f48aa8955e4a2ec143d3ba27674e6f5fdd.part /opt/stack/data/nova/instances/_base/a3fb13f48aa8955e4a2ec143d3ba27674e6f5fdd.converted" returned: 0 in 0.689s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:34:21 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:34:21 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-3b177ed1-ffba-4658-a45d-a282d4e771bb tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/a3fb13f48aa8955e4a2ec143d3ba27674e6f5fdd.converted --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:34:21 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:34:22 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-3b177ed1-ffba-4658-a45d-a282d4e771bb tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/a3fb13f48aa8955e4a2ec143d3ba27674e6f5fdd.converted --force-share --output=json" returned: 0 in 0.170s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:34:22 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-3b177ed1-ffba-4658-a45d-a282d4e771bb tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Lock "a3fb13f48aa8955e4a2ec143d3ba27674e6f5fdd" "released" by "nova.virt.libvirt.imagebackend.Image.cache.<locals>.fetch_func_sync" :: held 2.065s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:34:22 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-3b177ed1-ffba-4658-a45d-a282d4e771bb tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Acquiring lock "a3fb13f48aa8955e4a2ec143d3ba27674e6f5fdd" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:34:22 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-3b177ed1-ffba-4658-a45d-a282d4e771bb tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Lock "a3fb13f48aa8955e4a2ec143d3ba27674e6f5fdd" acquired by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:34:22 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-3b177ed1-ffba-4658-a45d-a282d4e771bb tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/a3fb13f48aa8955e4a2ec143d3ba27674e6f5fdd --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:34:22 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-3b177ed1-ffba-4658-a45d-a282d4e771bb tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/_base/a3fb13f48aa8955e4a2ec143d3ba27674e6f5fdd --force-share --output=json" returned: 0 in 0.175s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:34:22 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-3b177ed1-ffba-4658-a45d-a282d4e771bb tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/a3fb13f48aa8955e4a2ec143d3ba27674e6f5fdd,backing_fmt=raw /opt/stack/data/nova/instances/ebc3e3b2-037d-43ef-9426-caebe298ee3e/disk.rescue {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:34:22 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-3b177ed1-ffba-4658-a45d-a282d4e771bb tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] CMD "env LC_ALL=C LANG=C qemu-img create -f qcow2 -o backing_file=/opt/stack/data/nova/instances/_base/a3fb13f48aa8955e4a2ec143d3ba27674e6f5fdd,backing_fmt=raw /opt/stack/data/nova/instances/ebc3e3b2-037d-43ef-9426-caebe298ee3e/disk.rescue" returned: 0 in 0.048s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:34:22 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-3b177ed1-ffba-4658-a45d-a282d4e771bb tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Lock "a3fb13f48aa8955e4a2ec143d3ba27674e6f5fdd" "released" by "nova.virt.libvirt.imagebackend.Qcow2.create_image.<locals>.create_qcow2_image" :: held 0.230s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:34:22 user nova-compute[75234]: DEBUG nova.objects.instance [None req-3b177ed1-ffba-4658-a45d-a282d4e771bb tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Lazy-loading 'migration_context' on Instance uuid ebc3e3b2-037d-43ef-9426-caebe298ee3e {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:34:22 user nova-compute[75234]: DEBUG nova.objects.base [None req-3b177ed1-ffba-4658-a45d-a282d4e771bb tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Object Instance<ebc3e3b2-037d-43ef-9426-caebe298ee3e> lazy-loaded attributes: trusted_certs,migration_context {{(pid=75234) wrapper /opt/stack/nova/nova/objects/base.py:126}}
Nov 29 03:34:22 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-3b177ed1-ffba-4658-a45d-a282d4e771bb tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] [instance: ebc3e3b2-037d-43ef-9426-caebe298ee3e] Created local disks {{(pid=75234) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4861}}
Nov 29 03:34:22 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-3b177ed1-ffba-4658-a45d-a282d4e771bb tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] [instance: ebc3e3b2-037d-43ef-9426-caebe298ee3e] Start _get_guest_xml network_info=[{"id": "bd0a4e5b-69cd-4f5c-ae22-994e4d8f2682", "address": "fa:16:3e:c2:aa:3e", "network": {"id": "4d086e08-7798-4cfe-9365-86be6f0a52d2", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-305624008-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [], "label": "tempest-ServerBootFromVolumeStableRescueTest-305624008-network", "vif_mac": "fa:16:3e:c2:aa:3e"}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "db00bcd0c82c44f1bda739e9857f6e84", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapbd0a4e5b-69", "ovs_interfaceid": "bd0a4e5b-69cd-4f5c-ae22-994e4d8f2682", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, '/dev/vda': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, 'disk.rescue': {'bus': 'virtio', 'dev': 'vdb', 'type': 'disk'}}} image_meta=ImageMeta(checksum='c8fc807773e5354afe61636071771906',container_format='bare',created_at=2023-11-29T03:02:15Z,direct_url=<?>,disk_format='qcow2',id=96c30ba7-9b73-4a30-8ba4-7138879573b7,min_disk=0,min_ram=0,name='cirros-0.6.2-x86_64-disk',owner='b18e380921c748b78517b6f1211cd0f1',properties=ImageMetaProps,protected=<?>,size=21430272,status='active',tags=<?>,updated_at=2023-11-29T03:02:17Z,virtual_size=<?>,visibility=<?>) rescue={'image_id': 'cdfa8f6f-8ff6-4e7c-b0e4-bd3e16db3789', 'kernel_id': '', 'ramdisk_id': ''} block_device_info={'root_device_name': '/dev/vda', 'image': [], 'ephemerals': [], 'block_device_mapping': [{'mount_device': '/dev/vda', 'attachment_id': 'd004fe65-1313-4db0-8bed-ff0a443bf010', 'disk_bus': 'virtio', 'device_type': 'disk', 'boot_index': 0, 'guest_format': None, 'delete_on_termination': False, 'connection_info': {'driver_volume_type': 'iscsi', 'data': {'target_iqn': 'iqn.2010-10.org.openstack:257d7345-5b29-4bc1-8844-e211864c06c3', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'uUeBnhLX', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sde'}, 'status': 'reserved', 'instance': 'ebc3e3b2-037d-43ef-9426-caebe298ee3e', 'attached_at': '', 'detached_at': '', 'volume_id': '257d7345-5b29-4bc1-8844-e211864c06c3', 'serial': '257d7345-5b29-4bc1-8844-e211864c06c3'}, 'volume_type': None} {{(pid=75234) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7562}}
Nov 29 03:34:22 user nova-compute[75234]: DEBUG nova.objects.instance [None req-3b177ed1-ffba-4658-a45d-a282d4e771bb tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Lazy-loading 'resources' on Instance uuid ebc3e3b2-037d-43ef-9426-caebe298ee3e {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:34:22 user nova-compute[75234]: DEBUG nova.objects.base [None req-3b177ed1-ffba-4658-a45d-a282d4e771bb tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Object Instance<ebc3e3b2-037d-43ef-9426-caebe298ee3e> lazy-loaded attributes: trusted_certs,migration_context,resources {{(pid=75234) wrapper /opt/stack/nova/nova/objects/base.py:126}}
Nov 29 03:34:22 user nova-compute[75234]: DEBUG nova.objects.instance [None req-3b177ed1-ffba-4658-a45d-a282d4e771bb tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Lazy-loading 'numa_topology' on Instance uuid ebc3e3b2-037d-43ef-9426-caebe298ee3e {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:34:22 user nova-compute[75234]: DEBUG nova.objects.base [None req-3b177ed1-ffba-4658-a45d-a282d4e771bb tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Object Instance<ebc3e3b2-037d-43ef-9426-caebe298ee3e> lazy-loaded attributes: trusted_certs,migration_context,resources,numa_topology {{(pid=75234) wrapper /opt/stack/nova/nova/objects/base.py:126}}
Nov 29 03:34:22 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-3b177ed1-ffba-4658-a45d-a282d4e771bb tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:34:22 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-3b177ed1-ffba-4658-a45d-a282d4e771bb tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:34:22 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-3b177ed1-ffba-4658-a45d-a282d4e771bb tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Searching host: 'user' for CPU controller through CGroups V1... {{(pid=75234) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1652}}
Nov 29 03:34:22 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-3b177ed1-ffba-4658-a45d-a282d4e771bb tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] CPU controller missing on host. {{(pid=75234) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1662}}
Nov 29 03:34:22 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-3b177ed1-ffba-4658-a45d-a282d4e771bb tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Searching host: 'user' for CPU controller through CGroups V2... {{(pid=75234) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1671}}
Nov 29 03:34:22 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-3b177ed1-ffba-4658-a45d-a282d4e771bb tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] CPU controller found on host. {{(pid=75234) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1678}}
Nov 29 03:34:22 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-3b177ed1-ffba-4658-a45d-a282d4e771bb tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=75234) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5400}}
Nov 29 03:34:22 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-3b177ed1-ffba-4658-a45d-a282d4e771bb tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-11-29T03:02:04Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='1',id=6,is_public=True,memory_mb=512,name='m1.tiny',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='c8fc807773e5354afe61636071771906',container_format='bare',created_at=2023-11-29T03:02:15Z,direct_url=<?>,disk_format='qcow2',id=96c30ba7-9b73-4a30-8ba4-7138879573b7,min_disk=0,min_ram=0,name='cirros-0.6.2-x86_64-disk',owner='b18e380921c748b78517b6f1211cd0f1',properties=ImageMetaProps,protected=<?>,size=21430272,status='active',tags=<?>,updated_at=2023-11-29T03:02:17Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:563}}
Nov 29 03:34:22 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-3b177ed1-ffba-4658-a45d-a282d4e771bb tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Flavor limits 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:348}}
Nov 29 03:34:22 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-3b177ed1-ffba-4658-a45d-a282d4e771bb tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Image limits 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:352}}
Nov 29 03:34:22 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-3b177ed1-ffba-4658-a45d-a282d4e771bb tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Flavor pref 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:388}}
Nov 29 03:34:22 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-3b177ed1-ffba-4658-a45d-a282d4e771bb tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Image pref 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:392}}
Nov 29 03:34:22 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-3b177ed1-ffba-4658-a45d-a282d4e771bb tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:430}}
Nov 29 03:34:22 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-3b177ed1-ffba-4658-a45d-a282d4e771bb tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:569}}
Nov 29 03:34:22 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-3b177ed1-ffba-4658-a45d-a282d4e771bb tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=75234) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:471}}
Nov 29 03:34:22 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-3b177ed1-ffba-4658-a45d-a282d4e771bb tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Got 1 possible topologies {{(pid=75234) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:501}}
Nov 29 03:34:22 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-3b177ed1-ffba-4658-a45d-a282d4e771bb tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:575}}
Nov 29 03:34:22 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-3b177ed1-ffba-4658-a45d-a282d4e771bb tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:577}}
Nov 29 03:34:22 user nova-compute[75234]: DEBUG nova.objects.instance [None req-3b177ed1-ffba-4658-a45d-a282d4e771bb tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Lazy-loading 'vcpu_model' on Instance uuid ebc3e3b2-037d-43ef-9426-caebe298ee3e {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:34:22 user nova-compute[75234]: DEBUG nova.compute.manager [req-66e9aede-5077-4fc5-8f78-fc604f63b445 req-b9f5909c-39a2-44fc-aab2-3f9eff83ab6c service nova] [instance: 884589ec-b86c-4094-9a3f-54b641311d62] Received event network-changed-1deddce5-7279-44f9-9f1e-cd9ffddc5d6c {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:34:22 user nova-compute[75234]: DEBUG nova.compute.manager [req-66e9aede-5077-4fc5-8f78-fc604f63b445 req-b9f5909c-39a2-44fc-aab2-3f9eff83ab6c service nova] [instance: 884589ec-b86c-4094-9a3f-54b641311d62] Refreshing instance network info cache due to event network-changed-1deddce5-7279-44f9-9f1e-cd9ffddc5d6c. {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11117}}
Nov 29 03:34:22 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-66e9aede-5077-4fc5-8f78-fc604f63b445 req-b9f5909c-39a2-44fc-aab2-3f9eff83ab6c service nova] Acquiring lock "refresh_cache-884589ec-b86c-4094-9a3f-54b641311d62" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:34:22 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-66e9aede-5077-4fc5-8f78-fc604f63b445 req-b9f5909c-39a2-44fc-aab2-3f9eff83ab6c service nova] Acquired lock "refresh_cache-884589ec-b86c-4094-9a3f-54b641311d62" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:34:22 user nova-compute[75234]: DEBUG nova.network.neutron [req-66e9aede-5077-4fc5-8f78-fc604f63b445 req-b9f5909c-39a2-44fc-aab2-3f9eff83ab6c service nova] [instance: 884589ec-b86c-4094-9a3f-54b641311d62] Refreshing network info cache for port 1deddce5-7279-44f9-9f1e-cd9ffddc5d6c {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2006}}
Nov 29 03:34:22 user nova-compute[75234]: DEBUG nova.objects.base [None req-3b177ed1-ffba-4658-a45d-a282d4e771bb tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Object Instance<ebc3e3b2-037d-43ef-9426-caebe298ee3e> lazy-loaded attributes: trusted_certs,migration_context,resources,numa_topology,vcpu_model {{(pid=75234) wrapper /opt/stack/nova/nova/objects/base.py:126}}
Nov 29 03:34:22 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-3b177ed1-ffba-4658-a45d-a282d4e771bb tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] [instance: ebc3e3b2-037d-43ef-9426-caebe298ee3e] Calling os-brick to attach iSCSI Volume {{(pid=75234) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Nov 29 03:34:22 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-3b177ed1-ffba-4658-a45d-a282d4e771bb tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f8a5050c9a0>, 'connection_properties': {'target_iqn': 'iqn.2010-10.org.openstack:257d7345-5b29-4bc1-8844-e211864c06c3', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'uUeBnhLX', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path'}}" {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:176}}
Nov 29 03:34:22 user nova-compute[75234]: WARNING os_brick.initiator.connectors.base [None req-3b177ed1-ffba-4658-a45d-a282d4e771bb tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Nov 29 03:34:22 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-3b177ed1-ffba-4658-a45d-a282d4e771bb tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:68}}
Nov 29 03:34:22 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-3b177ed1-ffba-4658-a45d-a282d4e771bb tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:73}}
Nov 29 03:34:22 user nova-compute[75234]: WARNING os_brick.initiator.connectors.base [None req-3b177ed1-ffba-4658-a45d-a282d4e771bb tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Nov 29 03:34:22 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-3b177ed1-ffba-4658-a45d-a282d4e771bb tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:257d7345-5b29-4bc1-8844-e211864c06c3" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:68}}
Nov 29 03:34:22 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-3b177ed1-ffba-4658-a45d-a282d4e771bb tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:257d7345-5b29-4bc1-8844-e211864c06c3" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:73}}
Nov 29 03:34:22 user nova-compute[75234]: INFO os_brick.initiator.connectors.iscsi [None req-3b177ed1-ffba-4658-a45d-a282d4e771bb tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Nov 29 03:34:22 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:257d7345-5b29-4bc1-8844-e211864c06c3 -p 172.16.0.220:3260 {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:34:22 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:257d7345-5b29-4bc1-8844-e211864c06c3 -p 172.16.0.220:3260" returned: 0 in 0.027s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:34:22 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[3dc0c665-d1d9-4328-ac30-5e6eef85f43f]: (4, ('# BEGIN RECORD 2.1.5\nnode.name = iqn.2010-10.org.openstack:257d7345-5b29-4bc1-8844-e211864c06c3\nnode.tpgt = -1\nnode.startup = automatic\nnode.leading_login = No\niface.iscsi_ifacename = default\niface.net_ifacename = <empty>\niface.ipaddress = <empty>\niface.prefix_len = 0\niface.hwaddress = <empty>\niface.transport_name = tcp\niface.initiatorname = <empty>\niface.state = <empty>\niface.vlan_id = 0\niface.vlan_priority = 0\niface.vlan_state = <empty>\niface.iface_num = 0\niface.mtu = 0\niface.port = 0\niface.bootproto = <empty>\niface.subnet_mask = <empty>\niface.gateway = <empty>\niface.dhcp_alt_client_id_state = <empty>\niface.dhcp_alt_client_id = <empty>\niface.dhcp_dns = <empty>\niface.dhcp_learn_iqn = <empty>\niface.dhcp_req_vendor_id_state = <empty>\niface.dhcp_vendor_id_state = <empty>\niface.dhcp_vendor_id = <empty>\niface.dhcp_slp_da = <empty>\niface.fragmentation = <empty>\niface.gratuitous_arp = <empty>\niface.incoming_forwarding = <empty>\niface.tos_state = <empty>\niface.tos = 0\niface.ttl = 0\niface.delayed_ack = <empty>\niface.tcp_nagle = <empty>\niface.tcp_wsf_state = <empty>\niface.tcp_wsf = 0\niface.tcp_timer_scale = 0\niface.tcp_timestamp = <empty>\niface.redirect = <empty>\niface.def_task_mgmt_timeout = 0\niface.header_digest = <empty>\niface.data_digest = <empty>\niface.immediate_data = <empty>\niface.initial_r2t = <empty>\niface.data_seq_inorder = <empty>\niface.data_pdu_inorder = <empty>\niface.erl = 0\niface.max_receive_data_len = 0\niface.first_burst_len = 0\niface.max_outstanding_r2t = 0\niface.max_burst_len = 0\niface.chap_auth = <empty>\niface.bidi_chap = <empty>\niface.strict_login_compliance = <empty>\niface.discovery_auth = <empty>\niface.discovery_logout = <empty>\nnode.discovery_address = <empty>\nnode.discovery_port = 0\nnode.discovery_type = static\nnode.session.initial_cmdsn = 0\nnode.session.initial_login_retry_max = 8\nnode.session.xmit_thread_priority = -20\nnode.session.cmds_max = 128\nnode.session.queue_depth = 32\nnode.session.nr_sessions = 1\nnode.session.auth.authmethod = CHAP\nnode.session.auth.username = uUeBnhLX\nnode.session.auth.password = ********\nnode.session.auth.username_in = <empty>\nnode.session.auth.password_in = <empty>\nnode.session.auth.chap_algs = MD5\nnode.session.timeo.replacement_timeout = 120\nnode.session.err_timeo.abort_timeout = 15\nnode.session.err_timeo.lu_reset_timeout = 30\nnode.session.err_timeo.tgt_reset_timeout = 30\nnode.session.err_timeo.host_reset_timeout = 60\nnode.session.iscsi.FastAbort = Yes\nnode.session.iscsi.InitialR2T = No\nnode.session.iscsi.ImmediateData = Yes\nnode.session.iscsi.FirstBurstLength = 262144\nnode.session.iscsi.MaxBurstLength = 16776192\nnode.session.iscsi.DefaultTime2Retain = 0\nnode.session.iscsi.DefaultTime2Wait = 2\nnode.session.iscsi.MaxConnections = 1\nnode.session.iscsi.MaxOutstandingR2T = 1\nnode.session.iscsi.ERL = 0\nnode.session.scan = manual\nnode.session.reopen_max = 0\nnode.conn[0].address = 172.16.0.220\nnode.conn[0].port = 3260\nnode.conn[0].startup = manual\nnode.conn[0].tcp.window_size = 524288\nnode.conn[0].tcp.type_of_service = 0\nnode.conn[0].timeo.logout_timeout = 15\nnode.conn[0].timeo.login_timeout = 15\nnode.conn[0].timeo.auth_timeout = 45\nnode.conn[0].timeo.noop_out_interval = 5\nnode.conn[0].timeo.noop_out_timeout = 5\nnode.conn[0].iscsi.MaxXmitDataSegmentLength = 0\nnode.conn[0].iscsi.MaxRecvDataSegmentLength = 262144\nnode.conn[0].iscsi.HeaderDigest = None\nnode.conn[0].iscsi.DataDigest = None\nnode.conn[0].iscsi.IFMarker = No\nnode.conn[0].iscsi.OFMarker = No\n# END RECORD\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:34:22 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-3b177ed1-ffba-4658-a45d-a282d4e771bb tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] iscsiadm (): stdout=# BEGIN RECORD 2.1.5
Nov 29 03:34:22 user nova-compute[75234]: node.name = iqn.2010-10.org.openstack:257d7345-5b29-4bc1-8844-e211864c06c3
Nov 29 03:34:22 user nova-compute[75234]: node.tpgt = -1
Nov 29 03:34:22 user nova-compute[75234]: node.startup = automatic
Nov 29 03:34:22 user nova-compute[75234]: node.leading_login = No
Nov 29 03:34:22 user nova-compute[75234]: iface.iscsi_ifacename = default
Nov 29 03:34:22 user nova-compute[75234]: iface.net_ifacename = <empty>
Nov 29 03:34:22 user nova-compute[75234]: iface.ipaddress = <empty>
Nov 29 03:34:22 user nova-compute[75234]: iface.prefix_len = 0
Nov 29 03:34:22 user nova-compute[75234]: iface.hwaddress = <empty>
Nov 29 03:34:22 user nova-compute[75234]: iface.transport_name = tcp
Nov 29 03:34:22 user nova-compute[75234]: iface.initiatorname = <empty>
Nov 29 03:34:22 user nova-compute[75234]: iface.state = <empty>
Nov 29 03:34:22 user nova-compute[75234]: iface.vlan_id = 0
Nov 29 03:34:22 user nova-compute[75234]: iface.vlan_priority = 0
Nov 29 03:34:22 user nova-compute[75234]: iface.vlan_state = <empty>
Nov 29 03:34:22 user nova-compute[75234]: iface.iface_num = 0
Nov 29 03:34:22 user nova-compute[75234]: iface.mtu = 0
Nov 29 03:34:22 user nova-compute[75234]: iface.port = 0
Nov 29 03:34:22 user nova-compute[75234]: iface.bootproto = <empty>
Nov 29 03:34:22 user nova-compute[75234]: iface.subnet_mask = <empty>
Nov 29 03:34:22 user nova-compute[75234]: iface.gateway = <empty>
Nov 29 03:34:22 user nova-compute[75234]: iface.dhcp_alt_client_id_state = <empty>
Nov 29 03:34:22 user nova-compute[75234]: iface.dhcp_alt_client_id = <empty>
Nov 29 03:34:22 user nova-compute[75234]: iface.dhcp_dns = <empty>
Nov 29 03:34:22 user nova-compute[75234]: iface.dhcp_learn_iqn = <empty>
Nov 29 03:34:22 user nova-compute[75234]: iface.dhcp_req_vendor_id_state = <empty>
Nov 29 03:34:22 user nova-compute[75234]: iface.dhcp_vendor_id_state = <empty>
Nov 29 03:34:22 user nova-compute[75234]: iface.dhcp_vendor_id = <empty>
Nov 29 03:34:22 user nova-compute[75234]: iface.dhcp_slp_da = <empty>
Nov 29 03:34:22 user nova-compute[75234]: iface.fragmentation = <empty>
Nov 29 03:34:22 user nova-compute[75234]: iface.gratuitous_arp = <empty>
Nov 29 03:34:22 user nova-compute[75234]: iface.incoming_forwarding = <empty>
Nov 29 03:34:22 user nova-compute[75234]: iface.tos_state = <empty>
Nov 29 03:34:22 user nova-compute[75234]: iface.tos = 0
Nov 29 03:34:22 user nova-compute[75234]: iface.ttl = 0
Nov 29 03:34:22 user nova-compute[75234]: iface.delayed_ack = <empty>
Nov 29 03:34:22 user nova-compute[75234]: iface.tcp_nagle = <empty>
Nov 29 03:34:22 user nova-compute[75234]: iface.tcp_wsf_state = <empty>
Nov 29 03:34:22 user nova-compute[75234]: iface.tcp_wsf = 0
Nov 29 03:34:22 user nova-compute[75234]: iface.tcp_timer_scale = 0
Nov 29 03:34:22 user nova-compute[75234]: iface.tcp_timestamp = <empty>
Nov 29 03:34:22 user nova-compute[75234]: iface.redirect = <empty>
Nov 29 03:34:22 user nova-compute[75234]: iface.def_task_mgmt_timeout = 0
Nov 29 03:34:22 user nova-compute[75234]: iface.header_digest = <empty>
Nov 29 03:34:22 user nova-compute[75234]: iface.data_digest = <empty>
Nov 29 03:34:22 user nova-compute[75234]: iface.immediate_data = <empty>
Nov 29 03:34:22 user nova-compute[75234]: iface.initial_r2t = <empty>
Nov 29 03:34:22 user nova-compute[75234]: iface.data_seq_inorder = <empty>
Nov 29 03:34:22 user nova-compute[75234]: iface.data_pdu_inorder = <empty>
Nov 29 03:34:22 user nova-compute[75234]: iface.erl = 0
Nov 29 03:34:22 user nova-compute[75234]: iface.max_receive_data_len = 0
Nov 29 03:34:22 user nova-compute[75234]: iface.first_burst_len = 0
Nov 29 03:34:22 user nova-compute[75234]: iface.max_outstanding_r2t = 0
Nov 29 03:34:22 user nova-compute[75234]: iface.max_burst_len = 0
Nov 29 03:34:22 user nova-compute[75234]: iface.chap_auth = <empty>
Nov 29 03:34:22 user nova-compute[75234]: iface.bidi_chap = <empty>
Nov 29 03:34:22 user nova-compute[75234]: iface.strict_login_compliance = <empty>
Nov 29 03:34:22 user nova-compute[75234]: iface.discovery_auth = <empty>
Nov 29 03:34:22 user nova-compute[75234]: iface.discovery_logout = <empty>
Nov 29 03:34:22 user nova-compute[75234]: node.discovery_address = <empty>
Nov 29 03:34:22 user nova-compute[75234]: node.discovery_port = 0
Nov 29 03:34:22 user nova-compute[75234]: node.discovery_type = static
Nov 29 03:34:22 user nova-compute[75234]: node.session.initial_cmdsn = 0
Nov 29 03:34:22 user nova-compute[75234]: node.session.initial_login_retry_max = 8
Nov 29 03:34:22 user nova-compute[75234]: node.session.xmit_thread_priority = -20
Nov 29 03:34:22 user nova-compute[75234]: node.session.cmds_max = 128
Nov 29 03:34:22 user nova-compute[75234]: node.session.queue_depth = 32
Nov 29 03:34:22 user nova-compute[75234]: node.session.nr_sessions = 1
Nov 29 03:34:22 user nova-compute[75234]: node.session.auth.authmethod = CHAP
Nov 29 03:34:22 user nova-compute[75234]: node.session.auth.username = uUeBnhLX
Nov 29 03:34:22 user nova-compute[75234]: node.session.auth.password = ***
Nov 29 03:34:22 user nova-compute[75234]: node.session.auth.username_in = <empty>
Nov 29 03:34:22 user nova-compute[75234]: node.session.auth.password_in = <empty>
Nov 29 03:34:22 user nova-compute[75234]: node.session.auth.chap_algs = MD5
Nov 29 03:34:22 user nova-compute[75234]: node.session.timeo.replacement_timeout = 120
Nov 29 03:34:22 user nova-compute[75234]: node.session.err_timeo.abort_timeout = 15
Nov 29 03:34:22 user nova-compute[75234]: node.session.err_timeo.lu_reset_timeout = 30
Nov 29 03:34:22 user nova-compute[75234]: node.session.err_timeo.tgt_reset_timeout = 30
Nov 29 03:34:22 user nova-compute[75234]: node.session.err_timeo.host_reset_timeout = 60
Nov 29 03:34:22 user nova-compute[75234]: node.session.iscsi.FastAbort = Yes
Nov 29 03:34:22 user nova-compute[75234]: node.session.iscsi.InitialR2T = No
Nov 29 03:34:22 user nova-compute[75234]: node.session.iscsi.ImmediateData = Yes
Nov 29 03:34:22 user nova-compute[75234]: node.session.iscsi.FirstBurstLength = 262144
Nov 29 03:34:22 user nova-compute[75234]: node.session.iscsi.MaxBurstLength = 16776192
Nov 29 03:34:22 user nova-compute[75234]: node.session.iscsi.DefaultTime2Retain = 0
Nov 29 03:34:22 user nova-compute[75234]: node.session.iscsi.DefaultTime2Wait = 2
Nov 29 03:34:22 user nova-compute[75234]: node.session.iscsi.MaxConnections = 1
Nov 29 03:34:22 user nova-compute[75234]: node.session.iscsi.MaxOutstandingR2T = 1
Nov 29 03:34:22 user nova-compute[75234]: node.session.iscsi.ERL = 0
Nov 29 03:34:22 user nova-compute[75234]: node.session.scan = manual
Nov 29 03:34:22 user nova-compute[75234]: node.session.reopen_max = 0
Nov 29 03:34:22 user nova-compute[75234]: node.conn[0].address = 172.16.0.220
Nov 29 03:34:22 user nova-compute[75234]: node.conn[0].port = 3260
Nov 29 03:34:22 user nova-compute[75234]: node.conn[0].startup = manual
Nov 29 03:34:22 user nova-compute[75234]: node.conn[0].tcp.window_size = 524288
Nov 29 03:34:22 user nova-compute[75234]: node.conn[0].tcp.type_of_service = 0
Nov 29 03:34:22 user nova-compute[75234]: node.conn[0].timeo.logout_timeout = 15
Nov 29 03:34:22 user nova-compute[75234]: node.conn[0].timeo.login_timeout = 15
Nov 29 03:34:22 user nova-compute[75234]: node.conn[0].timeo.auth_timeout = 45
Nov 29 03:34:22 user nova-compute[75234]: node.conn[0].timeo.noop_out_interval = 5
Nov 29 03:34:22 user nova-compute[75234]: node.conn[0].timeo.noop_out_timeout = 5
Nov 29 03:34:22 user nova-compute[75234]: node.conn[0].iscsi.MaxXmitDataSegmentLength = 0
Nov 29 03:34:22 user nova-compute[75234]: node.conn[0].iscsi.MaxRecvDataSegmentLength = 262144
Nov 29 03:34:22 user nova-compute[75234]: node.conn[0].iscsi.HeaderDigest = None
Nov 29 03:34:22 user nova-compute[75234]: node.conn[0].iscsi.DataDigest = None
Nov 29 03:34:22 user nova-compute[75234]: node.conn[0].iscsi.IFMarker = No
Nov 29 03:34:22 user nova-compute[75234]: node.conn[0].iscsi.OFMarker = No
Nov 29 03:34:22 user nova-compute[75234]: # END RECORD
Nov 29 03:34:22 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:34:22 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:257d7345-5b29-4bc1-8844-e211864c06c3 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:34:22 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:257d7345-5b29-4bc1-8844-e211864c06c3 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.094s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:34:22 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[8b85596e-2c01-4118-96e7-54b272c24d7d]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:34:22 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-3b177ed1-ffba-4658-a45d-a282d4e771bb tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:34:22 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:257d7345-5b29-4bc1-8844-e211864c06c3 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:34:22 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:257d7345-5b29-4bc1-8844-e211864c06c3 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.019s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:34:22 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[c72edeb3-9f6c-436d-a2a3-257dc9cc2a5c]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:34:22 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-3b177ed1-ffba-4658-a45d-a282d4e771bb tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:34:22 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:257d7345-5b29-4bc1-8844-e211864c06c3 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v uUeBnhLX {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:34:22 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:257d7345-5b29-4bc1-8844-e211864c06c3 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v uUeBnhLX" returned: 0 in 0.018s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:34:22 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[7b848f58-5abf-48b7-a804-b70a150cc6aa]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:34:22 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-3b177ed1-ffba-4658-a45d-a282d4e771bb tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'uUeBnhLX'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:34:22 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:257d7345-5b29-4bc1-8844-e211864c06c3 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:34:22 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:257d7345-5b29-4bc1-8844-e211864c06c3 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.019s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:34:22 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[d55acc0d-900e-48d8-8f03-d21f589ab0d8]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:34:22 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-3b177ed1-ffba-4658-a45d-a282d4e771bb tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:34:22 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:34:22 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.024s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:34:22 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[b28ca42c-4254-4570-b1c6-6ebd356ab408]: (4, ('tcp: [42] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0 (non-flash)\ntcp: [54] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a3d30ae4-ca6d-4e06-8fc9-caf4d91ef900 (non-flash)\ntcp: [55] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:257d7345-5b29-4bc1-8844-e211864c06c3 (non-flash)\ntcp: [56] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:43e92d4a-7e98-4777-8fa5-3cec396b86ce (non-flash)\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:34:22 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-3b177ed1-ffba-4658-a45d-a282d4e771bb tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [42] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0 (non-flash)
Nov 29 03:34:22 user nova-compute[75234]: tcp: [54] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a3d30ae4-ca6d-4e06-8fc9-caf4d91ef900 (non-flash)
Nov 29 03:34:22 user nova-compute[75234]: tcp: [55] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:257d7345-5b29-4bc1-8844-e211864c06c3 (non-flash)
Nov 29 03:34:22 user nova-compute[75234]: tcp: [56] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:43e92d4a-7e98-4777-8fa5-3cec396b86ce (non-flash)
Nov 29 03:34:22 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:34:22 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-3b177ed1-ffba-4658-a45d-a282d4e771bb tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] iscsi session list stdout=tcp: [42] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0 (non-flash)
Nov 29 03:34:22 user nova-compute[75234]: tcp: [54] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a3d30ae4-ca6d-4e06-8fc9-caf4d91ef900 (non-flash)
Nov 29 03:34:22 user nova-compute[75234]: tcp: [55] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:257d7345-5b29-4bc1-8844-e211864c06c3 (non-flash)
Nov 29 03:34:22 user nova-compute[75234]: tcp: [56] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:43e92d4a-7e98-4777-8fa5-3cec396b86ce (non-flash)
Nov 29 03:34:22 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsi_session /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Nov 29 03:34:22 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-3b177ed1-ffba-4658-a45d-a282d4e771bb tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:257d7345-5b29-4bc1-8844-e211864c06c3" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.225s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:87}}
Nov 29 03:34:22 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-3b177ed1-ffba-4658-a45d-a282d4e771bb tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Connected to 172.16.0.220:3260 {{(pid=75234) _connect_vol /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:644}}
Nov 29 03:34:22 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-3b177ed1-ffba-4658-a45d-a282d4e771bb tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] HCTL ('36', '0', '0', 0) found on session 55 with lun 0 {{(pid=75234) get_hctl /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:770}}
Nov 29 03:34:22 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-3b177ed1-ffba-4658-a45d-a282d4e771bb tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Scanning host 36 c: 0, t: 0, l: 0) {{(pid=75234) scan_iscsi /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:795}}
Nov 29 03:34:22 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host36/scan {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:34:22 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host36/scan" returned: 0 in 0.013s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:34:22 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[4ee35ba6-c82b-4781-9868-c17c68878da2]: (4, ('0 0 0', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:34:22 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-3b177ed1-ffba-4658-a45d-a282d4e771bb tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Searching for a device in session 55 and hctl ('36', '0', '0', 0) yield: sde {{(pid=75234) device_name_by_hctl /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:789}}
Nov 29 03:34:22 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-3b177ed1-ffba-4658-a45d-a282d4e771bb tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Connected to sde using {'target_iqn': 'iqn.2010-10.org.openstack:257d7345-5b29-4bc1-8844-e211864c06c3', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'uUeBnhLX', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path'} {{(pid=75234) _connect_vol /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:672}}
Nov 29 03:34:22 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-3b177ed1-ffba-4658-a45d-a282d4e771bb tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 0.247s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:87}}
Nov 29 03:34:22 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-3b177ed1-ffba-4658-a45d-a282d4e771bb tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] <== connect_volume: return (248ms) {'type': 'block', 'scsi_wwn': '26239616134643263', 'path': '/dev/sde'} {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:203}}
Nov 29 03:34:22 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-3b177ed1-ffba-4658-a45d-a282d4e771bb tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] [instance: ebc3e3b2-037d-43ef-9426-caebe298ee3e] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '26239616134643263', 'path': '/dev/sde'} {{(pid=75234) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Nov 29 03:34:22 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-3b177ed1-ffba-4658-a45d-a282d4e771bb tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-11-29T03:32:27Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description=None,display_name='tempest-ServerBootFromVolumeStableRescueTest-server-1464602599',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-serverbootfromvolumestablerescuetest-server-1464602599',id=47,image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data=None,key_name=None,keypairs=<?>,launch_index=0,launched_at=2023-11-29T03:32:46Z,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='db00bcd0c82c44f1bda739e9857f6e84',ramdisk_id='',reservation_id='r-ztz14xc0',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-ServerBootFromVolumeStableRescueTest-1601266069',owner_user_name='tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member'},tags=<?>,task_state='rescuing',terminated_at=None,trusted_certs=None,updated_at=2023-11-29T03:32:47Z,user_data=None,user_id='877aa877f13a4f84ac4725278010aea3',uuid=ebc3e3b2-037d-43ef-9426-caebe298ee3e,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "bd0a4e5b-69cd-4f5c-ae22-994e4d8f2682", "address": "fa:16:3e:c2:aa:3e", "network": {"id": "4d086e08-7798-4cfe-9365-86be6f0a52d2", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-305624008-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [], "label": "tempest-ServerBootFromVolumeStableRescueTest-305624008-network", "vif_mac": "fa:16:3e:c2:aa:3e"}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "db00bcd0c82c44f1bda739e9857f6e84", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapbd0a4e5b-69", "ovs_interfaceid": "bd0a4e5b-69cd-4f5c-ae22-994e4d8f2682", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=75234) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Nov 29 03:34:22 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-3b177ed1-ffba-4658-a45d-a282d4e771bb tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Converting VIF {"id": "bd0a4e5b-69cd-4f5c-ae22-994e4d8f2682", "address": "fa:16:3e:c2:aa:3e", "network": {"id": "4d086e08-7798-4cfe-9365-86be6f0a52d2", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-305624008-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [], "label": "tempest-ServerBootFromVolumeStableRescueTest-305624008-network", "vif_mac": "fa:16:3e:c2:aa:3e"}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "db00bcd0c82c44f1bda739e9857f6e84", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapbd0a4e5b-69", "ovs_interfaceid": "bd0a4e5b-69cd-4f5c-ae22-994e4d8f2682", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:34:22 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-3b177ed1-ffba-4658-a45d-a282d4e771bb tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:c2:aa:3e,bridge_name='br-int',has_traffic_filtering=True,id=bd0a4e5b-69cd-4f5c-ae22-994e4d8f2682,network=Network(4d086e08-7798-4cfe-9365-86be6f0a52d2),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapbd0a4e5b-69') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:34:22 user nova-compute[75234]: DEBUG nova.objects.instance [None req-3b177ed1-ffba-4658-a45d-a282d4e771bb tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Lazy-loading 'pci_devices' on Instance uuid ebc3e3b2-037d-43ef-9426-caebe298ee3e {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:34:22 user nova-compute[75234]: DEBUG nova.objects.base [None req-3b177ed1-ffba-4658-a45d-a282d4e771bb tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Object Instance<ebc3e3b2-037d-43ef-9426-caebe298ee3e> lazy-loaded attributes: trusted_certs,migration_context,resources,numa_topology,vcpu_model,pci_devices {{(pid=75234) wrapper /opt/stack/nova/nova/objects/base.py:126}}
Nov 29 03:34:22 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-3b177ed1-ffba-4658-a45d-a282d4e771bb tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] [instance: ebc3e3b2-037d-43ef-9426-caebe298ee3e] End _get_guest_xml xml=<domain type="kvm">
Nov 29 03:34:22 user nova-compute[75234]:   <uuid>ebc3e3b2-037d-43ef-9426-caebe298ee3e</uuid>
Nov 29 03:34:22 user nova-compute[75234]:   <name>instance-0000002f</name>
Nov 29 03:34:22 user nova-compute[75234]:   <memory>524288</memory>
Nov 29 03:34:22 user nova-compute[75234]:   <vcpu>1</vcpu>
Nov 29 03:34:22 user nova-compute[75234]:   <metadata>
Nov 29 03:34:22 user nova-compute[75234]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Nov 29 03:34:22 user nova-compute[75234]:       <nova:package version="0.0.0"/>
Nov 29 03:34:22 user nova-compute[75234]:       <nova:name>tempest-ServerBootFromVolumeStableRescueTest-server-1464602599</nova:name>
Nov 29 03:34:22 user nova-compute[75234]:       <nova:creationTime>2023-11-29 03:34:22</nova:creationTime>
Nov 29 03:34:22 user nova-compute[75234]:       <nova:flavor name="m1.tiny">
Nov 29 03:34:22 user nova-compute[75234]:         <nova:memory>512</nova:memory>
Nov 29 03:34:22 user nova-compute[75234]:         <nova:disk>1</nova:disk>
Nov 29 03:34:22 user nova-compute[75234]:         <nova:swap>0</nova:swap>
Nov 29 03:34:22 user nova-compute[75234]:         <nova:ephemeral>0</nova:ephemeral>
Nov 29 03:34:22 user nova-compute[75234]:         <nova:vcpus>1</nova:vcpus>
Nov 29 03:34:22 user nova-compute[75234]:       </nova:flavor>
Nov 29 03:34:22 user nova-compute[75234]:       <nova:owner>
Nov 29 03:34:22 user nova-compute[75234]:         <nova:user uuid="877aa877f13a4f84ac4725278010aea3">tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member</nova:user>
Nov 29 03:34:22 user nova-compute[75234]:         <nova:project uuid="db00bcd0c82c44f1bda739e9857f6e84">tempest-ServerBootFromVolumeStableRescueTest-1601266069</nova:project>
Nov 29 03:34:22 user nova-compute[75234]:       </nova:owner>
Nov 29 03:34:22 user nova-compute[75234]:       <nova:root type="image" uuid="96c30ba7-9b73-4a30-8ba4-7138879573b7"/>
Nov 29 03:34:22 user nova-compute[75234]:       <nova:ports>
Nov 29 03:34:22 user nova-compute[75234]:         <nova:port uuid="bd0a4e5b-69cd-4f5c-ae22-994e4d8f2682">
Nov 29 03:34:22 user nova-compute[75234]:           <nova:ip type="fixed" address="10.0.0.14" ipVersion="4"/>
Nov 29 03:34:22 user nova-compute[75234]:         </nova:port>
Nov 29 03:34:22 user nova-compute[75234]:       </nova:ports>
Nov 29 03:34:22 user nova-compute[75234]:     </nova:instance>
Nov 29 03:34:22 user nova-compute[75234]:   </metadata>
Nov 29 03:34:22 user nova-compute[75234]:   <sysinfo type="smbios">
Nov 29 03:34:22 user nova-compute[75234]:     <system>
Nov 29 03:34:22 user nova-compute[75234]:       <entry name="manufacturer">OpenStack Foundation</entry>
Nov 29 03:34:22 user nova-compute[75234]:       <entry name="product">OpenStack Nova</entry>
Nov 29 03:34:22 user nova-compute[75234]:       <entry name="version">0.0.0</entry>
Nov 29 03:34:22 user nova-compute[75234]:       <entry name="serial">ebc3e3b2-037d-43ef-9426-caebe298ee3e</entry>
Nov 29 03:34:22 user nova-compute[75234]:       <entry name="uuid">ebc3e3b2-037d-43ef-9426-caebe298ee3e</entry>
Nov 29 03:34:22 user nova-compute[75234]:       <entry name="family">Virtual Machine</entry>
Nov 29 03:34:22 user nova-compute[75234]:     </system>
Nov 29 03:34:22 user nova-compute[75234]:   </sysinfo>
Nov 29 03:34:22 user nova-compute[75234]:   <os>
Nov 29 03:34:22 user nova-compute[75234]:     <type arch="x86_64" machine="pc">hvm</type>
Nov 29 03:34:22 user nova-compute[75234]:     <smbios mode="sysinfo"/>
Nov 29 03:34:22 user nova-compute[75234]:   </os>
Nov 29 03:34:22 user nova-compute[75234]:   <features>
Nov 29 03:34:22 user nova-compute[75234]:     <acpi/>
Nov 29 03:34:22 user nova-compute[75234]:     <apic/>
Nov 29 03:34:22 user nova-compute[75234]:     <vmcoreinfo/>
Nov 29 03:34:22 user nova-compute[75234]:   </features>
Nov 29 03:34:22 user nova-compute[75234]:   <clock offset="utc">
Nov 29 03:34:22 user nova-compute[75234]:     <timer name="pit" tickpolicy="delay"/>
Nov 29 03:34:22 user nova-compute[75234]:     <timer name="rtc" tickpolicy="catchup"/>
Nov 29 03:34:22 user nova-compute[75234]:     <timer name="hpet" present="no"/>
Nov 29 03:34:22 user nova-compute[75234]:   </clock>
Nov 29 03:34:22 user nova-compute[75234]:   <cpu mode="custom" match="exact">
Nov 29 03:34:22 user nova-compute[75234]:     <model>Nehalem</model>
Nov 29 03:34:22 user nova-compute[75234]:     <topology sockets="1" cores="1" threads="1"/>
Nov 29 03:34:22 user nova-compute[75234]:   </cpu>
Nov 29 03:34:22 user nova-compute[75234]:   <devices>
Nov 29 03:34:22 user nova-compute[75234]:     <disk type="block" device="disk">
Nov 29 03:34:22 user nova-compute[75234]:       <driver name="qemu" type="raw" cache="none" io="native"/>
Nov 29 03:34:22 user nova-compute[75234]:       <source dev="/dev/sde"/>
Nov 29 03:34:22 user nova-compute[75234]:       <target dev="vda" bus="virtio"/>
Nov 29 03:34:22 user nova-compute[75234]:       <serial>257d7345-5b29-4bc1-8844-e211864c06c3</serial>
Nov 29 03:34:22 user nova-compute[75234]:     </disk>
Nov 29 03:34:22 user nova-compute[75234]:     <disk type="file" device="disk">
Nov 29 03:34:22 user nova-compute[75234]:       <driver name="qemu" type="qcow2" cache="none"/>
Nov 29 03:34:22 user nova-compute[75234]:       <source file="/opt/stack/data/nova/instances/ebc3e3b2-037d-43ef-9426-caebe298ee3e/disk.rescue"/>
Nov 29 03:34:22 user nova-compute[75234]:       <target dev="vdb" bus="virtio"/>
Nov 29 03:34:22 user nova-compute[75234]:       <boot order="1"/>
Nov 29 03:34:22 user nova-compute[75234]:     </disk>
Nov 29 03:34:22 user nova-compute[75234]:     <interface type="ethernet">
Nov 29 03:34:22 user nova-compute[75234]:       <mac address="fa:16:3e:c2:aa:3e"/>
Nov 29 03:34:22 user nova-compute[75234]:       <model type="virtio"/>
Nov 29 03:34:22 user nova-compute[75234]:       <mtu size="1442"/>
Nov 29 03:34:22 user nova-compute[75234]:       <target dev="tapbd0a4e5b-69"/>
Nov 29 03:34:22 user nova-compute[75234]:     </interface>
Nov 29 03:34:22 user nova-compute[75234]:     <serial type="pty">
Nov 29 03:34:22 user nova-compute[75234]:       <log file="/opt/stack/data/nova/instances/ebc3e3b2-037d-43ef-9426-caebe298ee3e/console.log" append="off"/>
Nov 29 03:34:22 user nova-compute[75234]:     </serial>
Nov 29 03:34:22 user nova-compute[75234]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Nov 29 03:34:22 user nova-compute[75234]:     <video>
Nov 29 03:34:22 user nova-compute[75234]:       <model type="virtio"/>
Nov 29 03:34:22 user nova-compute[75234]:     </video>
Nov 29 03:34:22 user nova-compute[75234]:     <rng model="virtio">
Nov 29 03:34:22 user nova-compute[75234]:       <backend model="random">/dev/urandom</backend>
Nov 29 03:34:22 user nova-compute[75234]:     </rng>
Nov 29 03:34:22 user nova-compute[75234]:     <controller type="usb" index="0" model="none"/>
Nov 29 03:34:22 user nova-compute[75234]:     <memballoon model="virtio">
Nov 29 03:34:22 user nova-compute[75234]:       <stats period="10"/>
Nov 29 03:34:22 user nova-compute[75234]:     </memballoon>
Nov 29 03:34:22 user nova-compute[75234]:   </devices>
Nov 29 03:34:22 user nova-compute[75234]: </domain>
Nov 29 03:34:22 user nova-compute[75234]:  {{(pid=75234) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7568}}
Nov 29 03:34:22 user nova-compute[75234]: INFO nova.virt.libvirt.driver [-] [instance: ebc3e3b2-037d-43ef-9426-caebe298ee3e] Instance destroyed successfully.
Nov 29 03:34:22 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:34:22 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-3b177ed1-ffba-4658-a45d-a282d4e771bb tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] No BDM found with device name vda, not building metadata. {{(pid=75234) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12089}}
Nov 29 03:34:22 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-3b177ed1-ffba-4658-a45d-a282d4e771bb tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] No BDM found with device name vdb, not building metadata. {{(pid=75234) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12089}}
Nov 29 03:34:22 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-3b177ed1-ffba-4658-a45d-a282d4e771bb tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] No VIF found with MAC fa:16:3e:c2:aa:3e, not building metadata {{(pid=75234) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Nov 29 03:34:23 user nova-compute[75234]: DEBUG nova.network.neutron [req-66e9aede-5077-4fc5-8f78-fc604f63b445 req-b9f5909c-39a2-44fc-aab2-3f9eff83ab6c service nova] [instance: 884589ec-b86c-4094-9a3f-54b641311d62] Updated VIF entry in instance network info cache for port 1deddce5-7279-44f9-9f1e-cd9ffddc5d6c. {{(pid=75234) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3481}}
Nov 29 03:34:23 user nova-compute[75234]: DEBUG nova.network.neutron [req-66e9aede-5077-4fc5-8f78-fc604f63b445 req-b9f5909c-39a2-44fc-aab2-3f9eff83ab6c service nova] [instance: 884589ec-b86c-4094-9a3f-54b641311d62] Updating instance_info_cache with network_info: [{"id": "1deddce5-7279-44f9-9f1e-cd9ffddc5d6c", "address": "fa:16:3e:c5:44:c3", "network": {"id": "a3f397d3-fd75-4a6e-a815-cfcedcca15fb", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-678570085-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.93", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "81a265d3c15b4e9d826b35ddbd0ff443", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap1deddce5-72", "ovs_interfaceid": "1deddce5-7279-44f9-9f1e-cd9ffddc5d6c", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:34:23 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-66e9aede-5077-4fc5-8f78-fc604f63b445 req-b9f5909c-39a2-44fc-aab2-3f9eff83ab6c service nova] Releasing lock "refresh_cache-884589ec-b86c-4094-9a3f-54b641311d62" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:34:24 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:34:24 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:34:24 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:34:24 user nova-compute[75234]: DEBUG nova.compute.manager [req-69ffa48a-d073-4c81-9759-d1a26182c2dc req-18db994a-6cb0-4571-a085-4f7a5ab2772e service nova] [instance: ebc3e3b2-037d-43ef-9426-caebe298ee3e] Received event network-vif-plugged-bd0a4e5b-69cd-4f5c-ae22-994e4d8f2682 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:34:24 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-69ffa48a-d073-4c81-9759-d1a26182c2dc req-18db994a-6cb0-4571-a085-4f7a5ab2772e service nova] Acquiring lock "ebc3e3b2-037d-43ef-9426-caebe298ee3e-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:34:24 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-69ffa48a-d073-4c81-9759-d1a26182c2dc req-18db994a-6cb0-4571-a085-4f7a5ab2772e service nova] Lock "ebc3e3b2-037d-43ef-9426-caebe298ee3e-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:34:24 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-69ffa48a-d073-4c81-9759-d1a26182c2dc req-18db994a-6cb0-4571-a085-4f7a5ab2772e service nova] Lock "ebc3e3b2-037d-43ef-9426-caebe298ee3e-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:34:24 user nova-compute[75234]: DEBUG nova.compute.manager [req-69ffa48a-d073-4c81-9759-d1a26182c2dc req-18db994a-6cb0-4571-a085-4f7a5ab2772e service nova] [instance: ebc3e3b2-037d-43ef-9426-caebe298ee3e] No waiting events found dispatching network-vif-plugged-bd0a4e5b-69cd-4f5c-ae22-994e4d8f2682 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:34:24 user nova-compute[75234]: WARNING nova.compute.manager [req-69ffa48a-d073-4c81-9759-d1a26182c2dc req-18db994a-6cb0-4571-a085-4f7a5ab2772e service nova] [instance: ebc3e3b2-037d-43ef-9426-caebe298ee3e] Received unexpected event network-vif-plugged-bd0a4e5b-69cd-4f5c-ae22-994e4d8f2682 for instance with vm_state active and task_state rescuing.
Nov 29 03:34:24 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:34:24 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:34:25 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:34:26 user nova-compute[75234]: DEBUG nova.compute.manager [None req-3b177ed1-ffba-4658-a45d-a282d4e771bb tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] [instance: ebc3e3b2-037d-43ef-9426-caebe298ee3e] Checking state {{(pid=75234) _get_power_state /opt/stack/nova/nova/compute/manager.py:1782}}
Nov 29 03:34:26 user nova-compute[75234]: DEBUG nova.compute.manager [req-706ba054-f945-4de8-bb0f-7e571d5036d0 req-cc67a15d-8f8c-4285-b215-3a60d940261c service nova] [instance: ebc3e3b2-037d-43ef-9426-caebe298ee3e] Received event network-vif-plugged-bd0a4e5b-69cd-4f5c-ae22-994e4d8f2682 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:34:26 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-706ba054-f945-4de8-bb0f-7e571d5036d0 req-cc67a15d-8f8c-4285-b215-3a60d940261c service nova] Acquiring lock "ebc3e3b2-037d-43ef-9426-caebe298ee3e-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:34:26 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-706ba054-f945-4de8-bb0f-7e571d5036d0 req-cc67a15d-8f8c-4285-b215-3a60d940261c service nova] Lock "ebc3e3b2-037d-43ef-9426-caebe298ee3e-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:34:26 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-706ba054-f945-4de8-bb0f-7e571d5036d0 req-cc67a15d-8f8c-4285-b215-3a60d940261c service nova] Lock "ebc3e3b2-037d-43ef-9426-caebe298ee3e-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:34:26 user nova-compute[75234]: DEBUG nova.compute.manager [req-706ba054-f945-4de8-bb0f-7e571d5036d0 req-cc67a15d-8f8c-4285-b215-3a60d940261c service nova] [instance: ebc3e3b2-037d-43ef-9426-caebe298ee3e] No waiting events found dispatching network-vif-plugged-bd0a4e5b-69cd-4f5c-ae22-994e4d8f2682 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:34:26 user nova-compute[75234]: WARNING nova.compute.manager [req-706ba054-f945-4de8-bb0f-7e571d5036d0 req-cc67a15d-8f8c-4285-b215-3a60d940261c service nova] [instance: ebc3e3b2-037d-43ef-9426-caebe298ee3e] Received unexpected event network-vif-plugged-bd0a4e5b-69cd-4f5c-ae22-994e4d8f2682 for instance with vm_state rescued and task_state None.
Nov 29 03:34:27 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:34:28 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._sync_scheduler_instance_info {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:34:29 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-d947149e-2723-4f84-84f4-a681e890571a tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Acquiring lock "e3303cca-05a0-415a-b731-e1f12ad413f7" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:34:29 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-d947149e-2723-4f84-84f4-a681e890571a tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Lock "e3303cca-05a0-415a-b731-e1f12ad413f7" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:34:29 user nova-compute[75234]: DEBUG nova.compute.manager [None req-d947149e-2723-4f84-84f4-a681e890571a tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: e3303cca-05a0-415a-b731-e1f12ad413f7] Starting instance... {{(pid=75234) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2422}}
Nov 29 03:34:29 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-d947149e-2723-4f84-84f4-a681e890571a tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:34:29 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-d947149e-2723-4f84-84f4-a681e890571a tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:34:29 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-d947149e-2723-4f84-84f4-a681e890571a tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=75234) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2366}}
Nov 29 03:34:29 user nova-compute[75234]: INFO nova.compute.claims [None req-d947149e-2723-4f84-84f4-a681e890571a tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: e3303cca-05a0-415a-b731-e1f12ad413f7] Claim successful on node user
Nov 29 03:34:29 user nova-compute[75234]: DEBUG nova.compute.provider_tree [None req-d947149e-2723-4f84-84f4-a681e890571a tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Inventory has not changed in ProviderTree for provider: 4d291220-7c37-4839-a0e3-b23c66502b74 {{(pid=75234) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Nov 29 03:34:29 user nova-compute[75234]: DEBUG nova.scheduler.client.report [None req-d947149e-2723-4f84-84f4-a681e890571a tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Inventory has not changed for provider 4d291220-7c37-4839-a0e3-b23c66502b74 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=75234) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Nov 29 03:34:29 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-d947149e-2723-4f84-84f4-a681e890571a tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.353s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:34:29 user nova-compute[75234]: DEBUG nova.compute.manager [None req-d947149e-2723-4f84-84f4-a681e890571a tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: e3303cca-05a0-415a-b731-e1f12ad413f7] Start building networks asynchronously for instance. {{(pid=75234) _build_resources /opt/stack/nova/nova/compute/manager.py:2819}}
Nov 29 03:34:29 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-d947149e-2723-4f84-84f4-a681e890571a tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: e3303cca-05a0-415a-b731-e1f12ad413f7] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Nov 29 03:34:29 user nova-compute[75234]: DEBUG nova.compute.manager [None req-d947149e-2723-4f84-84f4-a681e890571a tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: e3303cca-05a0-415a-b731-e1f12ad413f7] Allocating IP information in the background. {{(pid=75234) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1972}}
Nov 29 03:34:29 user nova-compute[75234]: DEBUG nova.network.neutron [None req-d947149e-2723-4f84-84f4-a681e890571a tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: e3303cca-05a0-415a-b731-e1f12ad413f7] allocate_for_instance() {{(pid=75234) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1156}}
Nov 29 03:34:29 user nova-compute[75234]: DEBUG nova.compute.manager [None req-d947149e-2723-4f84-84f4-a681e890571a tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: e3303cca-05a0-415a-b731-e1f12ad413f7] Start building block device mappings for instance. {{(pid=75234) _build_resources /opt/stack/nova/nova/compute/manager.py:2854}}
Nov 29 03:34:29 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:34:29 user nova-compute[75234]: DEBUG nova.policy [None req-d947149e-2723-4f84-84f4-a681e890571a tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '7d1f1f11a9aa4c1db8627413eb963fa4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '81a265d3c15b4e9d826b35ddbd0ff443', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=75234) authorize /opt/stack/nova/nova/policy.py:203}}
Nov 29 03:34:29 user nova-compute[75234]: INFO nova.virt.block_device [None req-d947149e-2723-4f84-84f4-a681e890571a tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: e3303cca-05a0-415a-b731-e1f12ad413f7] Booting with volume snapshot bcdf364c-1d38-4c8d-a920-42782518a631 at /dev/vda
Nov 29 03:34:30 user nova-compute[75234]: DEBUG nova.network.neutron [None req-d947149e-2723-4f84-84f4-a681e890571a tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: e3303cca-05a0-415a-b731-e1f12ad413f7] Successfully created port: fd43dff4-5bd3-4791-a078-45bc982d3bb7 {{(pid=75234) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:548}}
Nov 29 03:34:31 user nova-compute[75234]: DEBUG nova.network.neutron [None req-d947149e-2723-4f84-84f4-a681e890571a tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: e3303cca-05a0-415a-b731-e1f12ad413f7] Successfully updated port: fd43dff4-5bd3-4791-a078-45bc982d3bb7 {{(pid=75234) _update_port /opt/stack/nova/nova/network/neutron.py:586}}
Nov 29 03:34:31 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-d947149e-2723-4f84-84f4-a681e890571a tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Acquiring lock "refresh_cache-e3303cca-05a0-415a-b731-e1f12ad413f7" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:34:31 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-d947149e-2723-4f84-84f4-a681e890571a tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Acquired lock "refresh_cache-e3303cca-05a0-415a-b731-e1f12ad413f7" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:34:31 user nova-compute[75234]: DEBUG nova.network.neutron [None req-d947149e-2723-4f84-84f4-a681e890571a tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: e3303cca-05a0-415a-b731-e1f12ad413f7] Building network info cache for instance {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2009}}
Nov 29 03:34:31 user nova-compute[75234]: DEBUG nova.compute.manager [req-e26af3af-3e78-48f0-ad6b-95f27144487e req-1ad1afcb-6585-43f2-a912-09f5fba88fa6 service nova] [instance: e3303cca-05a0-415a-b731-e1f12ad413f7] Received event network-changed-fd43dff4-5bd3-4791-a078-45bc982d3bb7 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:34:31 user nova-compute[75234]: DEBUG nova.compute.manager [req-e26af3af-3e78-48f0-ad6b-95f27144487e req-1ad1afcb-6585-43f2-a912-09f5fba88fa6 service nova] [instance: e3303cca-05a0-415a-b731-e1f12ad413f7] Refreshing instance network info cache due to event network-changed-fd43dff4-5bd3-4791-a078-45bc982d3bb7. {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11117}}
Nov 29 03:34:31 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-e26af3af-3e78-48f0-ad6b-95f27144487e req-1ad1afcb-6585-43f2-a912-09f5fba88fa6 service nova] Acquiring lock "refresh_cache-e3303cca-05a0-415a-b731-e1f12ad413f7" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:34:31 user nova-compute[75234]: DEBUG nova.network.neutron [None req-d947149e-2723-4f84-84f4-a681e890571a tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: e3303cca-05a0-415a-b731-e1f12ad413f7] Instance cache missing network info. {{(pid=75234) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3322}}
Nov 29 03:34:32 user nova-compute[75234]: DEBUG nova.network.neutron [None req-d947149e-2723-4f84-84f4-a681e890571a tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: e3303cca-05a0-415a-b731-e1f12ad413f7] Updating instance_info_cache with network_info: [{"id": "fd43dff4-5bd3-4791-a078-45bc982d3bb7", "address": "fa:16:3e:92:4c:3b", "network": {"id": "a3f397d3-fd75-4a6e-a815-cfcedcca15fb", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-678570085-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "81a265d3c15b4e9d826b35ddbd0ff443", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapfd43dff4-5b", "ovs_interfaceid": "fd43dff4-5bd3-4791-a078-45bc982d3bb7", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:34:32 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-d947149e-2723-4f84-84f4-a681e890571a tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Releasing lock "refresh_cache-e3303cca-05a0-415a-b731-e1f12ad413f7" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:34:32 user nova-compute[75234]: DEBUG nova.compute.manager [None req-d947149e-2723-4f84-84f4-a681e890571a tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: e3303cca-05a0-415a-b731-e1f12ad413f7] Instance network_info: |[{"id": "fd43dff4-5bd3-4791-a078-45bc982d3bb7", "address": "fa:16:3e:92:4c:3b", "network": {"id": "a3f397d3-fd75-4a6e-a815-cfcedcca15fb", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-678570085-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "81a265d3c15b4e9d826b35ddbd0ff443", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapfd43dff4-5b", "ovs_interfaceid": "fd43dff4-5bd3-4791-a078-45bc982d3bb7", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=75234) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1987}}
Nov 29 03:34:32 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-e26af3af-3e78-48f0-ad6b-95f27144487e req-1ad1afcb-6585-43f2-a912-09f5fba88fa6 service nova] Acquired lock "refresh_cache-e3303cca-05a0-415a-b731-e1f12ad413f7" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:34:32 user nova-compute[75234]: DEBUG nova.network.neutron [req-e26af3af-3e78-48f0-ad6b-95f27144487e req-1ad1afcb-6585-43f2-a912-09f5fba88fa6 service nova] [instance: e3303cca-05a0-415a-b731-e1f12ad413f7] Refreshing network info cache for port fd43dff4-5bd3-4791-a078-45bc982d3bb7 {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2006}}
Nov 29 03:34:32 user nova-compute[75234]: DEBUG nova.network.neutron [req-e26af3af-3e78-48f0-ad6b-95f27144487e req-1ad1afcb-6585-43f2-a912-09f5fba88fa6 service nova] [instance: e3303cca-05a0-415a-b731-e1f12ad413f7] Updated VIF entry in instance network info cache for port fd43dff4-5bd3-4791-a078-45bc982d3bb7. {{(pid=75234) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3481}}
Nov 29 03:34:32 user nova-compute[75234]: DEBUG nova.network.neutron [req-e26af3af-3e78-48f0-ad6b-95f27144487e req-1ad1afcb-6585-43f2-a912-09f5fba88fa6 service nova] [instance: e3303cca-05a0-415a-b731-e1f12ad413f7] Updating instance_info_cache with network_info: [{"id": "fd43dff4-5bd3-4791-a078-45bc982d3bb7", "address": "fa:16:3e:92:4c:3b", "network": {"id": "a3f397d3-fd75-4a6e-a815-cfcedcca15fb", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-678570085-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "81a265d3c15b4e9d826b35ddbd0ff443", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapfd43dff4-5b", "ovs_interfaceid": "fd43dff4-5bd3-4791-a078-45bc982d3bb7", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:34:32 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-e26af3af-3e78-48f0-ad6b-95f27144487e req-1ad1afcb-6585-43f2-a912-09f5fba88fa6 service nova] Releasing lock "refresh_cache-e3303cca-05a0-415a-b731-e1f12ad413f7" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:34:32 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:34:34 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:34:35 user nova-compute[75234]: DEBUG os_brick.utils [None req-d947149e-2723-4f84-84f4-a681e890571a tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:176}}
Nov 29 03:34:35 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:34:35 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.013s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:34:35 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[926f2940-b996-4230-a60a-7764e4048ec9]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:dd1368c5ba2c\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:34:35 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt -v / -n -o SOURCE {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:34:35 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt -v / -n -o SOURCE" returned: 0 in 0.014s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:34:35 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[f965ea51-4e59-4e62-83ed-a2fdeca47c96]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:34:35 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:34:35 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.020s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:34:35 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[627a1929-bc94-4ea6-be06-90effe765aba]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:34:35 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[a2c13a9c-411c-4025-b2bc-9c378751669d]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:34:35 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-d947149e-2723-4f84-84f4-a681e890571a tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Running cmd (subprocess): nvme version {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:34:35 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-d947149e-2723-4f84-84f4-a681e890571a tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] 'nvme version' failed. Not Retrying. {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:473}}
Nov 29 03:34:35 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.nvmeof [None req-d947149e-2723-4f84-84f4-a681e890571a tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] nvme not present on system {{(pid=75234) nvme_present /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/nvmeof.py:756}}
Nov 29 03:34:35 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:34:35 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:473}}
Nov 29 03:34:35 user nova-compute[75234]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Nov 29 03:34:35 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[0d9b3659-1b98-461d-a952-7f0751d451c1]: (4, '') {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:34:35 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.lightos [None req-d947149e-2723-4f84-84f4-a681e890571a tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=75234) find_dsc /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/lightos.py:98}}
Nov 29 03:34:35 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.lightos [None req-d947149e-2723-4f84-84f4-a681e890571a tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=75234) get_connector_properties /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/lightos.py:76}}
Nov 29 03:34:35 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.lightos [None req-d947149e-2723-4f84-84f4-a681e890571a tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] LIGHTOS: no hostnqn found. {{(pid=75234) get_connector_properties /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/lightos.py:84}}
Nov 29 03:34:35 user nova-compute[75234]: DEBUG os_brick.utils [None req-d947149e-2723-4f84-84f4-a681e890571a tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] <== get_connector_properties: return (123ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:dd1368c5ba2c', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:203}}
Nov 29 03:34:35 user nova-compute[75234]: DEBUG nova.virt.block_device [None req-d947149e-2723-4f84-84f4-a681e890571a tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: e3303cca-05a0-415a-b731-e1f12ad413f7] Updating existing volume attachment record: 8b1d36c5-2d59-4ecf-8738-5638216c47f4 {{(pid=75234) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Nov 29 03:34:37 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:34:39 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:34:40 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-6664dc44-86fc-40d2-bab6-27c52a7af3cc tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Acquiring lock "cb0f042e-43c0-43aa-b243-cfc78a7b2695" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:34:40 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-6664dc44-86fc-40d2-bab6-27c52a7af3cc tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Lock "cb0f042e-43c0-43aa-b243-cfc78a7b2695" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.002s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:34:40 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-6664dc44-86fc-40d2-bab6-27c52a7af3cc tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Acquiring lock "cb0f042e-43c0-43aa-b243-cfc78a7b2695-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:34:40 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-6664dc44-86fc-40d2-bab6-27c52a7af3cc tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Lock "cb0f042e-43c0-43aa-b243-cfc78a7b2695-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:34:40 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-6664dc44-86fc-40d2-bab6-27c52a7af3cc tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Lock "cb0f042e-43c0-43aa-b243-cfc78a7b2695-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:34:40 user nova-compute[75234]: INFO nova.compute.manager [None req-6664dc44-86fc-40d2-bab6-27c52a7af3cc tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] [instance: cb0f042e-43c0-43aa-b243-cfc78a7b2695] Terminating instance
Nov 29 03:34:40 user nova-compute[75234]: DEBUG nova.compute.manager [None req-6664dc44-86fc-40d2-bab6-27c52a7af3cc tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] [instance: cb0f042e-43c0-43aa-b243-cfc78a7b2695] Start destroying the instance on the hypervisor. {{(pid=75234) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3140}}
Nov 29 03:34:40 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:34:40 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:34:41 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:34:41 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:34:41 user nova-compute[75234]: DEBUG nova.compute.manager [req-74dbc119-3ac1-45af-8a7a-41a9b216642b req-ec91a98e-68ec-4061-a079-ae58613d85c9 service nova] [instance: cb0f042e-43c0-43aa-b243-cfc78a7b2695] Received event network-vif-unplugged-8f9642c3-9e10-4aa7-b5a2-53bb68ad96dc {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:34:41 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-74dbc119-3ac1-45af-8a7a-41a9b216642b req-ec91a98e-68ec-4061-a079-ae58613d85c9 service nova] Acquiring lock "cb0f042e-43c0-43aa-b243-cfc78a7b2695-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:34:41 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-74dbc119-3ac1-45af-8a7a-41a9b216642b req-ec91a98e-68ec-4061-a079-ae58613d85c9 service nova] Lock "cb0f042e-43c0-43aa-b243-cfc78a7b2695-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:34:41 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-74dbc119-3ac1-45af-8a7a-41a9b216642b req-ec91a98e-68ec-4061-a079-ae58613d85c9 service nova] Lock "cb0f042e-43c0-43aa-b243-cfc78a7b2695-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:34:41 user nova-compute[75234]: DEBUG nova.compute.manager [req-74dbc119-3ac1-45af-8a7a-41a9b216642b req-ec91a98e-68ec-4061-a079-ae58613d85c9 service nova] [instance: cb0f042e-43c0-43aa-b243-cfc78a7b2695] No waiting events found dispatching network-vif-unplugged-8f9642c3-9e10-4aa7-b5a2-53bb68ad96dc {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:34:41 user nova-compute[75234]: DEBUG nova.compute.manager [req-74dbc119-3ac1-45af-8a7a-41a9b216642b req-ec91a98e-68ec-4061-a079-ae58613d85c9 service nova] [instance: cb0f042e-43c0-43aa-b243-cfc78a7b2695] Received event network-vif-unplugged-8f9642c3-9e10-4aa7-b5a2-53bb68ad96dc for instance with task_state deleting. {{(pid=75234) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10890}}
Nov 29 03:34:41 user nova-compute[75234]: INFO nova.virt.libvirt.driver [-] [instance: cb0f042e-43c0-43aa-b243-cfc78a7b2695] Instance destroyed successfully.
Nov 29 03:34:41 user nova-compute[75234]: DEBUG nova.objects.instance [None req-6664dc44-86fc-40d2-bab6-27c52a7af3cc tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Lazy-loading 'resources' on Instance uuid cb0f042e-43c0-43aa-b243-cfc78a7b2695 {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:34:41 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-6664dc44-86fc-40d2-bab6-27c52a7af3cc tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-11-29T03:29:46Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-TestMinimumBasicScenario-server-1999691651',display_name='tempest-TestMinimumBasicScenario-server-1999691651',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-testminimumbasicscenario-server-1999691651',id=46,image_ref='39c09b70-6b0e-4a2d-b9f7-1f76a50a094d',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBLqZTjJ3Wp5EB1mGeMlVGHzydzMhKmKzqzC+PvIWz1jwo/eWvz8BbybL4HoY/jqWWGyeee/UtM8csTyDO1CCy9W96A60d4tN6E3Pg7qFv3rg6kLzbOVwZtACkYPfit/ztw==',key_name='tempest-TestMinimumBasicScenario-967805131',keypairs=<?>,launch_index=0,launched_at=2023-11-29T03:29:55Z,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='74f132d6666c47b8b43869df6fee6de3',ramdisk_id='',reservation_id='r-cfgmj1kq',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='39c09b70-6b0e-4a2d-b9f7-1f76a50a094d',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',owner_project_name='tempest-TestMinimumBasicScenario-1635847764',owner_user_name='tempest-TestMinimumBasicScenario-1635847764-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-11-29T03:32:02Z,user_data=None,user_id='3a02618f56ea45898ab4783a4f198b5d',uuid=cb0f042e-43c0-43aa-b243-cfc78a7b2695,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "8f9642c3-9e10-4aa7-b5a2-53bb68ad96dc", "address": "fa:16:3e:31:19:9f", "network": {"id": "6787e0a8-2980-4a2e-bf2a-583af955241d", "bridge": "br-int", "label": "tempest-TestMinimumBasicScenario-1273600998-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "74f132d6666c47b8b43869df6fee6de3", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap8f9642c3-9e", "ovs_interfaceid": "8f9642c3-9e10-4aa7-b5a2-53bb68ad96dc", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Nov 29 03:34:41 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-6664dc44-86fc-40d2-bab6-27c52a7af3cc tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Converting VIF {"id": "8f9642c3-9e10-4aa7-b5a2-53bb68ad96dc", "address": "fa:16:3e:31:19:9f", "network": {"id": "6787e0a8-2980-4a2e-bf2a-583af955241d", "bridge": "br-int", "label": "tempest-TestMinimumBasicScenario-1273600998-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "74f132d6666c47b8b43869df6fee6de3", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap8f9642c3-9e", "ovs_interfaceid": "8f9642c3-9e10-4aa7-b5a2-53bb68ad96dc", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:34:41 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-6664dc44-86fc-40d2-bab6-27c52a7af3cc tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:31:19:9f,bridge_name='br-int',has_traffic_filtering=True,id=8f9642c3-9e10-4aa7-b5a2-53bb68ad96dc,network=Network(6787e0a8-2980-4a2e-bf2a-583af955241d),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap8f9642c3-9e') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:34:41 user nova-compute[75234]: DEBUG os_vif [None req-6664dc44-86fc-40d2-bab6-27c52a7af3cc tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Unplugging vif VIFOpenVSwitch(active=True,address=fa:16:3e:31:19:9f,bridge_name='br-int',has_traffic_filtering=True,id=8f9642c3-9e10-4aa7-b5a2-53bb68ad96dc,network=Network(6787e0a8-2980-4a2e-bf2a-583af955241d),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap8f9642c3-9e') {{(pid=75234) unplug /opt/stack/data/venv/lib/python3.10/site-packages/os_vif/__init__.py:109}}
Nov 29 03:34:41 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:34:41 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tap8f9642c3-9e, bridge=br-int, if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:34:41 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:34:41 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:34:41 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:34:41 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbDestroyCommand(_result=None, table=QoS, record=72db526e-a12f-4edc-857c-2aee24308393) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:34:41 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:34:41 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:34:41 user nova-compute[75234]: INFO os_vif [None req-6664dc44-86fc-40d2-bab6-27c52a7af3cc tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Successfully unplugged vif VIFOpenVSwitch(active=True,address=fa:16:3e:31:19:9f,bridge_name='br-int',has_traffic_filtering=True,id=8f9642c3-9e10-4aa7-b5a2-53bb68ad96dc,network=Network(6787e0a8-2980-4a2e-bf2a-583af955241d),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap8f9642c3-9e')
Nov 29 03:34:41 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-6664dc44-86fc-40d2-bab6-27c52a7af3cc tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] [instance: cb0f042e-43c0-43aa-b243-cfc78a7b2695] Deleting instance files /opt/stack/data/nova/instances/cb0f042e-43c0-43aa-b243-cfc78a7b2695_del
Nov 29 03:34:41 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-6664dc44-86fc-40d2-bab6-27c52a7af3cc tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] [instance: cb0f042e-43c0-43aa-b243-cfc78a7b2695] Deletion of /opt/stack/data/nova/instances/cb0f042e-43c0-43aa-b243-cfc78a7b2695_del complete
Nov 29 03:34:41 user nova-compute[75234]: INFO nova.compute.manager [None req-6664dc44-86fc-40d2-bab6-27c52a7af3cc tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] [instance: cb0f042e-43c0-43aa-b243-cfc78a7b2695] Took 0.88 seconds to destroy the instance on the hypervisor.
Nov 29 03:34:41 user nova-compute[75234]: DEBUG oslo.service.loopingcall [None req-6664dc44-86fc-40d2-bab6-27c52a7af3cc tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=75234) func /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/loopingcall.py:435}}
Nov 29 03:34:41 user nova-compute[75234]: DEBUG nova.compute.manager [-] [instance: cb0f042e-43c0-43aa-b243-cfc78a7b2695] Deallocating network for instance {{(pid=75234) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2279}}
Nov 29 03:34:41 user nova-compute[75234]: DEBUG nova.network.neutron [-] [instance: cb0f042e-43c0-43aa-b243-cfc78a7b2695] deallocate_for_instance() {{(pid=75234) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1802}}
Nov 29 03:34:42 user nova-compute[75234]: DEBUG nova.network.neutron [-] [instance: cb0f042e-43c0-43aa-b243-cfc78a7b2695] Updating instance_info_cache with network_info: [] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:34:42 user nova-compute[75234]: INFO nova.compute.manager [-] [instance: cb0f042e-43c0-43aa-b243-cfc78a7b2695] Took 0.40 seconds to deallocate network for instance.
Nov 29 03:34:42 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-6664dc44-86fc-40d2-bab6-27c52a7af3cc tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:34:42 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-6664dc44-86fc-40d2-bab6-27c52a7af3cc tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:34:42 user nova-compute[75234]: DEBUG nova.compute.provider_tree [None req-6664dc44-86fc-40d2-bab6-27c52a7af3cc tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Inventory has not changed in ProviderTree for provider: 4d291220-7c37-4839-a0e3-b23c66502b74 {{(pid=75234) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Nov 29 03:34:42 user nova-compute[75234]: DEBUG nova.scheduler.client.report [None req-6664dc44-86fc-40d2-bab6-27c52a7af3cc tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Inventory has not changed for provider 4d291220-7c37-4839-a0e3-b23c66502b74 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=75234) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Nov 29 03:34:42 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-6664dc44-86fc-40d2-bab6-27c52a7af3cc tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.300s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:34:42 user nova-compute[75234]: INFO nova.scheduler.client.report [None req-6664dc44-86fc-40d2-bab6-27c52a7af3cc tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Deleted allocations for instance cb0f042e-43c0-43aa-b243-cfc78a7b2695
Nov 29 03:34:42 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-6664dc44-86fc-40d2-bab6-27c52a7af3cc tempest-TestMinimumBasicScenario-1635847764 tempest-TestMinimumBasicScenario-1635847764-project-member] Lock "cb0f042e-43c0-43aa-b243-cfc78a7b2695" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 1.767s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:34:42 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:34:43 user nova-compute[75234]: DEBUG nova.compute.manager [req-8ce9ef58-feb4-4603-9cfa-479abb4601a7 req-814dbbbc-b271-412c-83e0-8cd71c584931 service nova] [instance: cb0f042e-43c0-43aa-b243-cfc78a7b2695] Received event network-vif-plugged-8f9642c3-9e10-4aa7-b5a2-53bb68ad96dc {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:34:43 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-8ce9ef58-feb4-4603-9cfa-479abb4601a7 req-814dbbbc-b271-412c-83e0-8cd71c584931 service nova] Acquiring lock "cb0f042e-43c0-43aa-b243-cfc78a7b2695-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:34:43 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-8ce9ef58-feb4-4603-9cfa-479abb4601a7 req-814dbbbc-b271-412c-83e0-8cd71c584931 service nova] Lock "cb0f042e-43c0-43aa-b243-cfc78a7b2695-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:34:43 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-8ce9ef58-feb4-4603-9cfa-479abb4601a7 req-814dbbbc-b271-412c-83e0-8cd71c584931 service nova] Lock "cb0f042e-43c0-43aa-b243-cfc78a7b2695-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:34:43 user nova-compute[75234]: DEBUG nova.compute.manager [req-8ce9ef58-feb4-4603-9cfa-479abb4601a7 req-814dbbbc-b271-412c-83e0-8cd71c584931 service nova] [instance: cb0f042e-43c0-43aa-b243-cfc78a7b2695] No waiting events found dispatching network-vif-plugged-8f9642c3-9e10-4aa7-b5a2-53bb68ad96dc {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:34:43 user nova-compute[75234]: WARNING nova.compute.manager [req-8ce9ef58-feb4-4603-9cfa-479abb4601a7 req-814dbbbc-b271-412c-83e0-8cd71c584931 service nova] [instance: cb0f042e-43c0-43aa-b243-cfc78a7b2695] Received unexpected event network-vif-plugged-8f9642c3-9e10-4aa7-b5a2-53bb68ad96dc for instance with vm_state deleted and task_state None.
Nov 29 03:34:43 user nova-compute[75234]: DEBUG nova.compute.manager [req-8ce9ef58-feb4-4603-9cfa-479abb4601a7 req-814dbbbc-b271-412c-83e0-8cd71c584931 service nova] [instance: cb0f042e-43c0-43aa-b243-cfc78a7b2695] Received event network-vif-deleted-8f9642c3-9e10-4aa7-b5a2-53bb68ad96dc {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:34:43 user nova-compute[75234]: DEBUG nova.compute.manager [None req-d947149e-2723-4f84-84f4-a681e890571a tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: e3303cca-05a0-415a-b731-e1f12ad413f7] Start spawning the instance on the hypervisor. {{(pid=75234) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2628}}
Nov 29 03:34:43 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-d947149e-2723-4f84-84f4-a681e890571a tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: e3303cca-05a0-415a-b731-e1f12ad413f7] Creating instance directory {{(pid=75234) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4727}}
Nov 29 03:34:43 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-d947149e-2723-4f84-84f4-a681e890571a tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: e3303cca-05a0-415a-b731-e1f12ad413f7] Creating image(s)
Nov 29 03:34:43 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-d947149e-2723-4f84-84f4-a681e890571a tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: e3303cca-05a0-415a-b731-e1f12ad413f7] Did not create local disks {{(pid=75234) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4863}}
Nov 29 03:34:43 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-d947149e-2723-4f84-84f4-a681e890571a tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: e3303cca-05a0-415a-b731-e1f12ad413f7] Ensure instance console log exists: /opt/stack/data/nova/instances/e3303cca-05a0-415a-b731-e1f12ad413f7/console.log {{(pid=75234) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4613}}
Nov 29 03:34:43 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-d947149e-2723-4f84-84f4-a681e890571a tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:34:43 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-d947149e-2723-4f84-84f4-a681e890571a tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:34:43 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-d947149e-2723-4f84-84f4-a681e890571a tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:34:43 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-d947149e-2723-4f84-84f4-a681e890571a tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: e3303cca-05a0-415a-b731-e1f12ad413f7] Start _get_guest_xml network_info=[{"id": "fd43dff4-5bd3-4791-a078-45bc982d3bb7", "address": "fa:16:3e:92:4c:3b", "network": {"id": "a3f397d3-fd75-4a6e-a815-cfcedcca15fb", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-678570085-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "81a265d3c15b4e9d826b35ddbd0ff443", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapfd43dff4-5b", "ovs_interfaceid": "fd43dff4-5bd3-4791-a078-45bc982d3bb7", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, '/dev/vda': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum='d41d8cd98f00b204e9800998ecf8427e',container_format='bare',created_at=2023-11-29T03:34:24Z,direct_url=<?>,disk_format='qcow2',id=a16128e1-937d-4f3d-ad48-bf3fd942beb1,min_disk=1,min_ram=0,name='tempest-TestVolumeBootPatternsnapshot-331552605',owner='81a265d3c15b4e9d826b35ddbd0ff443',properties=ImageMetaProps,protected=<?>,size=0,status='active',tags=<?>,updated_at=2023-11-29T03:34:25Z,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [], 'ephemerals': [], 'block_device_mapping': [{'mount_device': '/dev/vda', 'attachment_id': '8b1d36c5-2d59-4ecf-8738-5638216c47f4', 'disk_bus': 'virtio', 'device_type': 'disk', 'boot_index': 0, 'guest_format': None, 'delete_on_termination': True, 'connection_info': {'driver_volume_type': 'iscsi', 'data': {'target_iqn': 'iqn.2010-10.org.openstack:dd89ea2e-2a9f-4511-9b6f-c5177ff21182', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'giwfJzqL', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}, 'status': 'reserved', 'instance': 'e3303cca-05a0-415a-b731-e1f12ad413f7', 'attached_at': '', 'detached_at': '', 'volume_id': 'dd89ea2e-2a9f-4511-9b6f-c5177ff21182', 'serial': 'dd89ea2e-2a9f-4511-9b6f-c5177ff21182'}, 'volume_type': None} {{(pid=75234) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7562}}
Nov 29 03:34:43 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-d947149e-2723-4f84-84f4-a681e890571a tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:34:43 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-d947149e-2723-4f84-84f4-a681e890571a tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:34:43 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-d947149e-2723-4f84-84f4-a681e890571a tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Searching host: 'user' for CPU controller through CGroups V1... {{(pid=75234) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1652}}
Nov 29 03:34:43 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-d947149e-2723-4f84-84f4-a681e890571a tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] CPU controller missing on host. {{(pid=75234) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1662}}
Nov 29 03:34:43 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-d947149e-2723-4f84-84f4-a681e890571a tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Searching host: 'user' for CPU controller through CGroups V2... {{(pid=75234) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1671}}
Nov 29 03:34:43 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-d947149e-2723-4f84-84f4-a681e890571a tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] CPU controller found on host. {{(pid=75234) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1678}}
Nov 29 03:34:43 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-d947149e-2723-4f84-84f4-a681e890571a tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=75234) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5400}}
Nov 29 03:34:43 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-d947149e-2723-4f84-84f4-a681e890571a tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-11-29T03:02:04Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='1',id=6,is_public=True,memory_mb=512,name='m1.tiny',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum='d41d8cd98f00b204e9800998ecf8427e',container_format='bare',created_at=2023-11-29T03:34:24Z,direct_url=<?>,disk_format='qcow2',id=a16128e1-937d-4f3d-ad48-bf3fd942beb1,min_disk=1,min_ram=0,name='tempest-TestVolumeBootPatternsnapshot-331552605',owner='81a265d3c15b4e9d826b35ddbd0ff443',properties=ImageMetaProps,protected=<?>,size=0,status='active',tags=<?>,updated_at=2023-11-29T03:34:25Z,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:563}}
Nov 29 03:34:43 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-d947149e-2723-4f84-84f4-a681e890571a tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Flavor limits 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:348}}
Nov 29 03:34:43 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-d947149e-2723-4f84-84f4-a681e890571a tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Image limits 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:352}}
Nov 29 03:34:43 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-d947149e-2723-4f84-84f4-a681e890571a tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Flavor pref 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:388}}
Nov 29 03:34:43 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-d947149e-2723-4f84-84f4-a681e890571a tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Image pref 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:392}}
Nov 29 03:34:43 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-d947149e-2723-4f84-84f4-a681e890571a tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:430}}
Nov 29 03:34:43 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-d947149e-2723-4f84-84f4-a681e890571a tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:569}}
Nov 29 03:34:43 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-d947149e-2723-4f84-84f4-a681e890571a tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=75234) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:471}}
Nov 29 03:34:43 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-d947149e-2723-4f84-84f4-a681e890571a tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Got 1 possible topologies {{(pid=75234) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:501}}
Nov 29 03:34:43 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-d947149e-2723-4f84-84f4-a681e890571a tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:575}}
Nov 29 03:34:43 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-d947149e-2723-4f84-84f4-a681e890571a tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:577}}
Nov 29 03:34:43 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-d947149e-2723-4f84-84f4-a681e890571a tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: e3303cca-05a0-415a-b731-e1f12ad413f7] Calling os-brick to attach iSCSI Volume {{(pid=75234) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Nov 29 03:34:43 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-d947149e-2723-4f84-84f4-a681e890571a tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f8a5050c9a0>, 'connection_properties': {'target_iqn': 'iqn.2010-10.org.openstack:dd89ea2e-2a9f-4511-9b6f-c5177ff21182', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'giwfJzqL', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:176}}
Nov 29 03:34:43 user nova-compute[75234]: WARNING os_brick.initiator.connectors.base [None req-d947149e-2723-4f84-84f4-a681e890571a tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Nov 29 03:34:43 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-d947149e-2723-4f84-84f4-a681e890571a tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:68}}
Nov 29 03:34:43 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-d947149e-2723-4f84-84f4-a681e890571a tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:73}}
Nov 29 03:34:43 user nova-compute[75234]: WARNING os_brick.initiator.connectors.base [None req-d947149e-2723-4f84-84f4-a681e890571a tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Nov 29 03:34:43 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-d947149e-2723-4f84-84f4-a681e890571a tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:dd89ea2e-2a9f-4511-9b6f-c5177ff21182" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:68}}
Nov 29 03:34:43 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-d947149e-2723-4f84-84f4-a681e890571a tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:dd89ea2e-2a9f-4511-9b6f-c5177ff21182" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:73}}
Nov 29 03:34:43 user nova-compute[75234]: INFO os_brick.initiator.connectors.iscsi [None req-d947149e-2723-4f84-84f4-a681e890571a tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Nov 29 03:34:43 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:dd89ea2e-2a9f-4511-9b6f-c5177ff21182 -p 172.16.0.220:3260 {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:34:43 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:dd89ea2e-2a9f-4511-9b6f-c5177ff21182 -p 172.16.0.220:3260" returned: 21 in 0.019s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:34:43 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[ea7f7d4b-3f16-4e56-bd90-f95dd80ea770]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:34:43 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-d947149e-2723-4f84-84f4-a681e890571a tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Nov 29 03:34:43 user nova-compute[75234]:  {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:34:43 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:dd89ea2e-2a9f-4511-9b6f-c5177ff21182 -p 172.16.0.220:3260 --interface default --op new {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:34:43 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:dd89ea2e-2a9f-4511-9b6f-c5177ff21182 -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.019s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:34:43 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[4fc2b89c-c97a-497e-b627-aa021fa162f7]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:dd89ea2e-2a9f-4511-9b6f-c5177ff21182] added\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:34:43 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-d947149e-2723-4f84-84f4-a681e890571a tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:dd89ea2e-2a9f-4511-9b6f-c5177ff21182] added
Nov 29 03:34:43 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:34:43 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:dd89ea2e-2a9f-4511-9b6f-c5177ff21182 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:34:43 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:dd89ea2e-2a9f-4511-9b6f-c5177ff21182 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.020s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:34:43 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[2d3616cd-21b0-44d2-8d99-47bc83d403fd]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:34:43 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-d947149e-2723-4f84-84f4-a681e890571a tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:34:43 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:dd89ea2e-2a9f-4511-9b6f-c5177ff21182 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:34:43 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:dd89ea2e-2a9f-4511-9b6f-c5177ff21182 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.017s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:34:43 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[a2fbe5b8-a1a5-4af2-87e2-89265e47d35b]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:34:43 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-d947149e-2723-4f84-84f4-a681e890571a tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:34:43 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:dd89ea2e-2a9f-4511-9b6f-c5177ff21182 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v giwfJzqL {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:34:43 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:dd89ea2e-2a9f-4511-9b6f-c5177ff21182 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v giwfJzqL" returned: 0 in 0.023s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:34:43 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[5ab50983-f762-41e6-b8ce-9b9c20bbc238]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:34:43 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-d947149e-2723-4f84-84f4-a681e890571a tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'giwfJzqL'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:34:43 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:dd89ea2e-2a9f-4511-9b6f-c5177ff21182 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:34:43 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:dd89ea2e-2a9f-4511-9b6f-c5177ff21182 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.021s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:34:43 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[468a8a86-de79-4084-9793-067014f36599]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:34:43 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-d947149e-2723-4f84-84f4-a681e890571a tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:34:43 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:34:43 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.023s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:34:43 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[6d59de2f-8ba1-4730-836a-6120173611cc]: (4, ('tcp: [42] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0 (non-flash)\ntcp: [54] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a3d30ae4-ca6d-4e06-8fc9-caf4d91ef900 (non-flash)\ntcp: [55] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:257d7345-5b29-4bc1-8844-e211864c06c3 (non-flash)\ntcp: [56] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:43e92d4a-7e98-4777-8fa5-3cec396b86ce (non-flash)\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:34:43 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-d947149e-2723-4f84-84f4-a681e890571a tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [42] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0 (non-flash)
Nov 29 03:34:43 user nova-compute[75234]: tcp: [54] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a3d30ae4-ca6d-4e06-8fc9-caf4d91ef900 (non-flash)
Nov 29 03:34:43 user nova-compute[75234]: tcp: [55] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:257d7345-5b29-4bc1-8844-e211864c06c3 (non-flash)
Nov 29 03:34:43 user nova-compute[75234]: tcp: [56] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:43e92d4a-7e98-4777-8fa5-3cec396b86ce (non-flash)
Nov 29 03:34:43 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:34:43 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-d947149e-2723-4f84-84f4-a681e890571a tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] iscsi session list stdout=tcp: [42] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0 (non-flash)
Nov 29 03:34:43 user nova-compute[75234]: tcp: [54] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a3d30ae4-ca6d-4e06-8fc9-caf4d91ef900 (non-flash)
Nov 29 03:34:43 user nova-compute[75234]: tcp: [55] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:257d7345-5b29-4bc1-8844-e211864c06c3 (non-flash)
Nov 29 03:34:43 user nova-compute[75234]: tcp: [56] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:43e92d4a-7e98-4777-8fa5-3cec396b86ce (non-flash)
Nov 29 03:34:43 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsi_session /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Nov 29 03:34:43 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:dd89ea2e-2a9f-4511-9b6f-c5177ff21182 -p 172.16.0.220:3260 --login {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:34:43 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:dd89ea2e-2a9f-4511-9b6f-c5177ff21182 -p 172.16.0.220:3260 --login" returned: 0 in 0.065s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:34:43 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[01d7ad82-6b27-4c82-9148-bad40b480121]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:dd89ea2e-2a9f-4511-9b6f-c5177ff21182, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:dd89ea2e-2a9f-4511-9b6f-c5177ff21182, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:34:43 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-d947149e-2723-4f84-84f4-a681e890571a tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:dd89ea2e-2a9f-4511-9b6f-c5177ff21182, portal: 172.16.0.220,3260]
Nov 29 03:34:43 user nova-compute[75234]: Login to [iface: default, target: iqn.2010-10.org.openstack:dd89ea2e-2a9f-4511-9b6f-c5177ff21182, portal: 172.16.0.220,3260] successful.
Nov 29 03:34:43 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:34:43 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:dd89ea2e-2a9f-4511-9b6f-c5177ff21182 -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:34:43 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:dd89ea2e-2a9f-4511-9b6f-c5177ff21182 -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.019s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:34:43 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[cc3c31e9-eca3-43f7-8b7c-c0bc2ce2b741]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:34:43 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-d947149e-2723-4f84-84f4-a681e890571a tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:34:43 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:34:44 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.026s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:34:44 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[8ab9a01a-f87c-4c8c-b19e-70ad978d1904]: (4, ('tcp: [42] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0 (non-flash)\ntcp: [54] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a3d30ae4-ca6d-4e06-8fc9-caf4d91ef900 (non-flash)\ntcp: [55] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:257d7345-5b29-4bc1-8844-e211864c06c3 (non-flash)\ntcp: [56] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:43e92d4a-7e98-4777-8fa5-3cec396b86ce (non-flash)\ntcp: [57] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:dd89ea2e-2a9f-4511-9b6f-c5177ff21182 (non-flash)\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:34:44 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-d947149e-2723-4f84-84f4-a681e890571a tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [42] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0 (non-flash)
Nov 29 03:34:44 user nova-compute[75234]: tcp: [54] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a3d30ae4-ca6d-4e06-8fc9-caf4d91ef900 (non-flash)
Nov 29 03:34:44 user nova-compute[75234]: tcp: [55] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:257d7345-5b29-4bc1-8844-e211864c06c3 (non-flash)
Nov 29 03:34:44 user nova-compute[75234]: tcp: [56] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:43e92d4a-7e98-4777-8fa5-3cec396b86ce (non-flash)
Nov 29 03:34:44 user nova-compute[75234]: tcp: [57] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:dd89ea2e-2a9f-4511-9b6f-c5177ff21182 (non-flash)
Nov 29 03:34:44 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:34:44 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-d947149e-2723-4f84-84f4-a681e890571a tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] iscsi session list stdout=tcp: [42] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0 (non-flash)
Nov 29 03:34:44 user nova-compute[75234]: tcp: [54] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a3d30ae4-ca6d-4e06-8fc9-caf4d91ef900 (non-flash)
Nov 29 03:34:44 user nova-compute[75234]: tcp: [55] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:257d7345-5b29-4bc1-8844-e211864c06c3 (non-flash)
Nov 29 03:34:44 user nova-compute[75234]: tcp: [56] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:43e92d4a-7e98-4777-8fa5-3cec396b86ce (non-flash)
Nov 29 03:34:44 user nova-compute[75234]: tcp: [57] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:dd89ea2e-2a9f-4511-9b6f-c5177ff21182 (non-flash)
Nov 29 03:34:44 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsi_session /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Nov 29 03:34:44 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-d947149e-2723-4f84-84f4-a681e890571a tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:dd89ea2e-2a9f-4511-9b6f-c5177ff21182" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.297s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:87}}
Nov 29 03:34:44 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-d947149e-2723-4f84-84f4-a681e890571a tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Connected to 172.16.0.220:3260 {{(pid=75234) _connect_vol /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:644}}
Nov 29 03:34:44 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-d947149e-2723-4f84-84f4-a681e890571a tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] HCTL ('33', '-', '-', 0) found on session 57 with lun 0 {{(pid=75234) get_hctl /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:770}}
Nov 29 03:34:44 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-d947149e-2723-4f84-84f4-a681e890571a tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Scanning host 33 c: -, t: -, l: 0) {{(pid=75234) scan_iscsi /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:795}}
Nov 29 03:34:44 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host33/scan {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:34:44 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host33/scan" returned: 0 in 0.018s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:34:44 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[7ea602f7-ea48-488f-b15f-ff2923615df0]: (4, ('- - 0', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:34:44 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-d947149e-2723-4f84-84f4-a681e890571a tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Searching for a device in session 57 and hctl ['33', '*', '*', 0] yield: sdb {{(pid=75234) device_name_by_hctl /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:789}}
Nov 29 03:34:44 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-d947149e-2723-4f84-84f4-a681e890571a tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Connected to sdb using {'target_iqn': 'iqn.2010-10.org.openstack:dd89ea2e-2a9f-4511-9b6f-c5177ff21182', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'giwfJzqL', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False} {{(pid=75234) _connect_vol /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:672}}
Nov 29 03:34:45 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-d947149e-2723-4f84-84f4-a681e890571a tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.331s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:87}}
Nov 29 03:34:45 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-d947149e-2723-4f84-84f4-a681e890571a tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] <== connect_volume: return (1332ms) {'type': 'block', 'scsi_wwn': '23464393664343233', 'path': '/dev/sdb'} {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:203}}
Nov 29 03:34:45 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-d947149e-2723-4f84-84f4-a681e890571a tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: e3303cca-05a0-415a-b731-e1f12ad413f7] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '23464393664343233', 'path': '/dev/sdb'} {{(pid=75234) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Nov 29 03:34:45 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-d947149e-2723-4f84-84f4-a681e890571a tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-11-29T03:34:28Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-TestVolumeBootPattern-image-snapshot-server-1990224224',display_name='tempest-TestVolumeBootPattern-image-snapshot-server-1990224224',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-testvolumebootpattern-image-snapshot-server-1990224224',id=49,image_ref='a16128e1-937d-4f3d-ad48-bf3fd942beb1',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBO96UoQAplU5V6UAUXfmFq4Xoqecf74KJ64MbiF1ieDnoRV2pPDxFGj5ZEYbpIW/U3oWl7DggAXx8KafBFOSxpIC3ULGrK/e+yz95ehE2qDjf55PlvhVEaakUzw4pg1y9g==',key_name='tempest-keypair-1541124831',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='81a265d3c15b4e9d826b35ddbd0ff443',ramdisk_id='',reservation_id='r-52vup8wc',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='',image_bdm_v2='True',image_boot_roles='reader,member',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_project_name='tempest-TestVolumeBootPattern-1244492388',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',image_owner_user_name='tempest-TestVolumeBootPattern-1244492388-project-member',image_root_device_name='/dev/vda',network_allocated='True',owner_project_name='tempest-TestVolumeBootPattern-1244492388',owner_user_name='tempest-TestVolumeBootPattern-1244492388-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-11-29T03:34:30Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='7d1f1f11a9aa4c1db8627413eb963fa4',uuid=e3303cca-05a0-415a-b731-e1f12ad413f7,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "fd43dff4-5bd3-4791-a078-45bc982d3bb7", "address": "fa:16:3e:92:4c:3b", "network": {"id": "a3f397d3-fd75-4a6e-a815-cfcedcca15fb", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-678570085-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "81a265d3c15b4e9d826b35ddbd0ff443", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapfd43dff4-5b", "ovs_interfaceid": "fd43dff4-5bd3-4791-a078-45bc982d3bb7", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=75234) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Nov 29 03:34:45 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-d947149e-2723-4f84-84f4-a681e890571a tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Converting VIF {"id": "fd43dff4-5bd3-4791-a078-45bc982d3bb7", "address": "fa:16:3e:92:4c:3b", "network": {"id": "a3f397d3-fd75-4a6e-a815-cfcedcca15fb", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-678570085-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "81a265d3c15b4e9d826b35ddbd0ff443", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapfd43dff4-5b", "ovs_interfaceid": "fd43dff4-5bd3-4791-a078-45bc982d3bb7", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:34:45 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-d947149e-2723-4f84-84f4-a681e890571a tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:92:4c:3b,bridge_name='br-int',has_traffic_filtering=True,id=fd43dff4-5bd3-4791-a078-45bc982d3bb7,network=Network(a3f397d3-fd75-4a6e-a815-cfcedcca15fb),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapfd43dff4-5b') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:34:45 user nova-compute[75234]: DEBUG nova.objects.instance [None req-d947149e-2723-4f84-84f4-a681e890571a tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Lazy-loading 'pci_devices' on Instance uuid e3303cca-05a0-415a-b731-e1f12ad413f7 {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:34:45 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-d947149e-2723-4f84-84f4-a681e890571a tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: e3303cca-05a0-415a-b731-e1f12ad413f7] End _get_guest_xml xml=<domain type="kvm">
Nov 29 03:34:45 user nova-compute[75234]:   <uuid>e3303cca-05a0-415a-b731-e1f12ad413f7</uuid>
Nov 29 03:34:45 user nova-compute[75234]:   <name>instance-00000031</name>
Nov 29 03:34:45 user nova-compute[75234]:   <memory>524288</memory>
Nov 29 03:34:45 user nova-compute[75234]:   <vcpu>1</vcpu>
Nov 29 03:34:45 user nova-compute[75234]:   <metadata>
Nov 29 03:34:45 user nova-compute[75234]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Nov 29 03:34:45 user nova-compute[75234]:       <nova:package version="0.0.0"/>
Nov 29 03:34:45 user nova-compute[75234]:       <nova:name>tempest-TestVolumeBootPattern-image-snapshot-server-1990224224</nova:name>
Nov 29 03:34:45 user nova-compute[75234]:       <nova:creationTime>2023-11-29 03:34:43</nova:creationTime>
Nov 29 03:34:45 user nova-compute[75234]:       <nova:flavor name="m1.tiny">
Nov 29 03:34:45 user nova-compute[75234]:         <nova:memory>512</nova:memory>
Nov 29 03:34:45 user nova-compute[75234]:         <nova:disk>1</nova:disk>
Nov 29 03:34:45 user nova-compute[75234]:         <nova:swap>0</nova:swap>
Nov 29 03:34:45 user nova-compute[75234]:         <nova:ephemeral>0</nova:ephemeral>
Nov 29 03:34:45 user nova-compute[75234]:         <nova:vcpus>1</nova:vcpus>
Nov 29 03:34:45 user nova-compute[75234]:       </nova:flavor>
Nov 29 03:34:45 user nova-compute[75234]:       <nova:owner>
Nov 29 03:34:45 user nova-compute[75234]:         <nova:user uuid="7d1f1f11a9aa4c1db8627413eb963fa4">tempest-TestVolumeBootPattern-1244492388-project-member</nova:user>
Nov 29 03:34:45 user nova-compute[75234]:         <nova:project uuid="81a265d3c15b4e9d826b35ddbd0ff443">tempest-TestVolumeBootPattern-1244492388</nova:project>
Nov 29 03:34:45 user nova-compute[75234]:       </nova:owner>
Nov 29 03:34:45 user nova-compute[75234]:       <nova:root type="image" uuid="a16128e1-937d-4f3d-ad48-bf3fd942beb1"/>
Nov 29 03:34:45 user nova-compute[75234]:       <nova:ports>
Nov 29 03:34:45 user nova-compute[75234]:         <nova:port uuid="fd43dff4-5bd3-4791-a078-45bc982d3bb7">
Nov 29 03:34:45 user nova-compute[75234]:           <nova:ip type="fixed" address="10.0.0.9" ipVersion="4"/>
Nov 29 03:34:45 user nova-compute[75234]:         </nova:port>
Nov 29 03:34:45 user nova-compute[75234]:       </nova:ports>
Nov 29 03:34:45 user nova-compute[75234]:     </nova:instance>
Nov 29 03:34:45 user nova-compute[75234]:   </metadata>
Nov 29 03:34:45 user nova-compute[75234]:   <sysinfo type="smbios">
Nov 29 03:34:45 user nova-compute[75234]:     <system>
Nov 29 03:34:45 user nova-compute[75234]:       <entry name="manufacturer">OpenStack Foundation</entry>
Nov 29 03:34:45 user nova-compute[75234]:       <entry name="product">OpenStack Nova</entry>
Nov 29 03:34:45 user nova-compute[75234]:       <entry name="version">0.0.0</entry>
Nov 29 03:34:45 user nova-compute[75234]:       <entry name="serial">e3303cca-05a0-415a-b731-e1f12ad413f7</entry>
Nov 29 03:34:45 user nova-compute[75234]:       <entry name="uuid">e3303cca-05a0-415a-b731-e1f12ad413f7</entry>
Nov 29 03:34:45 user nova-compute[75234]:       <entry name="family">Virtual Machine</entry>
Nov 29 03:34:45 user nova-compute[75234]:     </system>
Nov 29 03:34:45 user nova-compute[75234]:   </sysinfo>
Nov 29 03:34:45 user nova-compute[75234]:   <os>
Nov 29 03:34:45 user nova-compute[75234]:     <type arch="x86_64" machine="pc">hvm</type>
Nov 29 03:34:45 user nova-compute[75234]:     <boot dev="hd"/>
Nov 29 03:34:45 user nova-compute[75234]:     <smbios mode="sysinfo"/>
Nov 29 03:34:45 user nova-compute[75234]:   </os>
Nov 29 03:34:45 user nova-compute[75234]:   <features>
Nov 29 03:34:45 user nova-compute[75234]:     <acpi/>
Nov 29 03:34:45 user nova-compute[75234]:     <apic/>
Nov 29 03:34:45 user nova-compute[75234]:     <vmcoreinfo/>
Nov 29 03:34:45 user nova-compute[75234]:   </features>
Nov 29 03:34:45 user nova-compute[75234]:   <clock offset="utc">
Nov 29 03:34:45 user nova-compute[75234]:     <timer name="pit" tickpolicy="delay"/>
Nov 29 03:34:45 user nova-compute[75234]:     <timer name="rtc" tickpolicy="catchup"/>
Nov 29 03:34:45 user nova-compute[75234]:     <timer name="hpet" present="no"/>
Nov 29 03:34:45 user nova-compute[75234]:   </clock>
Nov 29 03:34:45 user nova-compute[75234]:   <cpu mode="custom" match="exact">
Nov 29 03:34:45 user nova-compute[75234]:     <model>Nehalem</model>
Nov 29 03:34:45 user nova-compute[75234]:     <topology sockets="1" cores="1" threads="1"/>
Nov 29 03:34:45 user nova-compute[75234]:   </cpu>
Nov 29 03:34:45 user nova-compute[75234]:   <devices>
Nov 29 03:34:45 user nova-compute[75234]:     <disk type="block" device="disk">
Nov 29 03:34:45 user nova-compute[75234]:       <driver name="qemu" type="raw" cache="none" io="native"/>
Nov 29 03:34:45 user nova-compute[75234]:       <source dev="/dev/sdb"/>
Nov 29 03:34:45 user nova-compute[75234]:       <target dev="vda" bus="virtio"/>
Nov 29 03:34:45 user nova-compute[75234]:       <serial>dd89ea2e-2a9f-4511-9b6f-c5177ff21182</serial>
Nov 29 03:34:45 user nova-compute[75234]:     </disk>
Nov 29 03:34:45 user nova-compute[75234]:     <interface type="ethernet">
Nov 29 03:34:45 user nova-compute[75234]:       <mac address="fa:16:3e:92:4c:3b"/>
Nov 29 03:34:45 user nova-compute[75234]:       <model type="virtio"/>
Nov 29 03:34:45 user nova-compute[75234]:       <mtu size="1442"/>
Nov 29 03:34:45 user nova-compute[75234]:       <target dev="tapfd43dff4-5b"/>
Nov 29 03:34:45 user nova-compute[75234]:     </interface>
Nov 29 03:34:45 user nova-compute[75234]:     <serial type="pty">
Nov 29 03:34:45 user nova-compute[75234]:       <log file="/opt/stack/data/nova/instances/e3303cca-05a0-415a-b731-e1f12ad413f7/console.log" append="off"/>
Nov 29 03:34:45 user nova-compute[75234]:     </serial>
Nov 29 03:34:45 user nova-compute[75234]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Nov 29 03:34:45 user nova-compute[75234]:     <video>
Nov 29 03:34:45 user nova-compute[75234]:       <model type="virtio"/>
Nov 29 03:34:45 user nova-compute[75234]:     </video>
Nov 29 03:34:45 user nova-compute[75234]:     <rng model="virtio">
Nov 29 03:34:45 user nova-compute[75234]:       <backend model="random">/dev/urandom</backend>
Nov 29 03:34:45 user nova-compute[75234]:     </rng>
Nov 29 03:34:45 user nova-compute[75234]:     <controller type="usb" index="0" model="none"/>
Nov 29 03:34:45 user nova-compute[75234]:     <memballoon model="virtio">
Nov 29 03:34:45 user nova-compute[75234]:       <stats period="10"/>
Nov 29 03:34:45 user nova-compute[75234]:     </memballoon>
Nov 29 03:34:45 user nova-compute[75234]:   </devices>
Nov 29 03:34:45 user nova-compute[75234]: </domain>
Nov 29 03:34:45 user nova-compute[75234]:  {{(pid=75234) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7568}}
Nov 29 03:34:45 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-d947149e-2723-4f84-84f4-a681e890571a tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-11-29T03:34:28Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-TestVolumeBootPattern-image-snapshot-server-1990224224',display_name='tempest-TestVolumeBootPattern-image-snapshot-server-1990224224',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-testvolumebootpattern-image-snapshot-server-1990224224',id=49,image_ref='a16128e1-937d-4f3d-ad48-bf3fd942beb1',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBO96UoQAplU5V6UAUXfmFq4Xoqecf74KJ64MbiF1ieDnoRV2pPDxFGj5ZEYbpIW/U3oWl7DggAXx8KafBFOSxpIC3ULGrK/e+yz95ehE2qDjf55PlvhVEaakUzw4pg1y9g==',key_name='tempest-keypair-1541124831',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='81a265d3c15b4e9d826b35ddbd0ff443',ramdisk_id='',reservation_id='r-52vup8wc',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='',image_bdm_v2='True',image_boot_roles='reader,member',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_project_name='tempest-TestVolumeBootPattern-1244492388',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',image_owner_user_name='tempest-TestVolumeBootPattern-1244492388-project-member',image_root_device_name='/dev/vda',network_allocated='True',owner_project_name='tempest-TestVolumeBootPattern-1244492388',owner_user_name='tempest-TestVolumeBootPattern-1244492388-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-11-29T03:34:30Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='7d1f1f11a9aa4c1db8627413eb963fa4',uuid=e3303cca-05a0-415a-b731-e1f12ad413f7,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "fd43dff4-5bd3-4791-a078-45bc982d3bb7", "address": "fa:16:3e:92:4c:3b", "network": {"id": "a3f397d3-fd75-4a6e-a815-cfcedcca15fb", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-678570085-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "81a265d3c15b4e9d826b35ddbd0ff443", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapfd43dff4-5b", "ovs_interfaceid": "fd43dff4-5bd3-4791-a078-45bc982d3bb7", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Nov 29 03:34:45 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-d947149e-2723-4f84-84f4-a681e890571a tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Converting VIF {"id": "fd43dff4-5bd3-4791-a078-45bc982d3bb7", "address": "fa:16:3e:92:4c:3b", "network": {"id": "a3f397d3-fd75-4a6e-a815-cfcedcca15fb", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-678570085-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "81a265d3c15b4e9d826b35ddbd0ff443", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapfd43dff4-5b", "ovs_interfaceid": "fd43dff4-5bd3-4791-a078-45bc982d3bb7", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:34:45 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-d947149e-2723-4f84-84f4-a681e890571a tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:92:4c:3b,bridge_name='br-int',has_traffic_filtering=True,id=fd43dff4-5bd3-4791-a078-45bc982d3bb7,network=Network(a3f397d3-fd75-4a6e-a815-cfcedcca15fb),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapfd43dff4-5b') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:34:45 user nova-compute[75234]: DEBUG os_vif [None req-d947149e-2723-4f84-84f4-a681e890571a tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:92:4c:3b,bridge_name='br-int',has_traffic_filtering=True,id=fd43dff4-5bd3-4791-a078-45bc982d3bb7,network=Network(a3f397d3-fd75-4a6e-a815-cfcedcca15fb),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapfd43dff4-5b') {{(pid=75234) plug /opt/stack/data/venv/lib/python3.10/site-packages/os_vif/__init__.py:76}}
Nov 29 03:34:45 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:34:45 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:34:45 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Nov 29 03:34:45 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:34:45 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbCreateCommand(_result=None, table=QoS, columns={'type': 'linux-noop', 'external_ids': {'id': '417ee7aa-233c-598a-adb0-aa8651e24f80', '_type': 'linux-noop'}}, row=False) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:34:45 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:34:45 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:34:45 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:34:45 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tapfd43dff4-5b, may_exist=True, interface_attrs={}) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:34:45 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Port, record=tapfd43dff4-5b, col_values=(('qos', UUID('c744b157-135d-4f8a-a727-1411604337c2')),), if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:34:45 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DbSetCommand(_result=None, table=Interface, record=tapfd43dff4-5b, col_values=(('external_ids', {'iface-id': 'fd43dff4-5bd3-4791-a078-45bc982d3bb7', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:92:4c:3b', 'vm-uuid': 'e3303cca-05a0-415a-b731-e1f12ad413f7'}),), if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:34:45 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:34:45 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:34:45 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:34:45 user nova-compute[75234]: INFO os_vif [None req-d947149e-2723-4f84-84f4-a681e890571a tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:92:4c:3b,bridge_name='br-int',has_traffic_filtering=True,id=fd43dff4-5bd3-4791-a078-45bc982d3bb7,network=Network(a3f397d3-fd75-4a6e-a815-cfcedcca15fb),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapfd43dff4-5b')
Nov 29 03:34:45 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-d947149e-2723-4f84-84f4-a681e890571a tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] No BDM found with device name vda, not building metadata. {{(pid=75234) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12089}}
Nov 29 03:34:45 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-d947149e-2723-4f84-84f4-a681e890571a tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] No VIF found with MAC fa:16:3e:92:4c:3b, not building metadata {{(pid=75234) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Nov 29 03:34:46 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:34:46 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:34:46 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:34:46 user nova-compute[75234]: DEBUG nova.compute.manager [req-aa3ab99e-3219-439f-8abf-ace8af7e1b51 req-03d95b09-50cd-4c1e-8111-8fa21ef94275 service nova] [instance: e3303cca-05a0-415a-b731-e1f12ad413f7] Received event network-vif-plugged-fd43dff4-5bd3-4791-a078-45bc982d3bb7 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:34:46 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-aa3ab99e-3219-439f-8abf-ace8af7e1b51 req-03d95b09-50cd-4c1e-8111-8fa21ef94275 service nova] Acquiring lock "e3303cca-05a0-415a-b731-e1f12ad413f7-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:34:46 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-aa3ab99e-3219-439f-8abf-ace8af7e1b51 req-03d95b09-50cd-4c1e-8111-8fa21ef94275 service nova] Lock "e3303cca-05a0-415a-b731-e1f12ad413f7-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:34:46 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-aa3ab99e-3219-439f-8abf-ace8af7e1b51 req-03d95b09-50cd-4c1e-8111-8fa21ef94275 service nova] Lock "e3303cca-05a0-415a-b731-e1f12ad413f7-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:34:46 user nova-compute[75234]: DEBUG nova.compute.manager [req-aa3ab99e-3219-439f-8abf-ace8af7e1b51 req-03d95b09-50cd-4c1e-8111-8fa21ef94275 service nova] [instance: e3303cca-05a0-415a-b731-e1f12ad413f7] No waiting events found dispatching network-vif-plugged-fd43dff4-5bd3-4791-a078-45bc982d3bb7 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:34:46 user nova-compute[75234]: WARNING nova.compute.manager [req-aa3ab99e-3219-439f-8abf-ace8af7e1b51 req-03d95b09-50cd-4c1e-8111-8fa21ef94275 service nova] [instance: e3303cca-05a0-415a-b731-e1f12ad413f7] Received unexpected event network-vif-plugged-fd43dff4-5bd3-4791-a078-45bc982d3bb7 for instance with vm_state building and task_state spawning.
Nov 29 03:34:46 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:34:46 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:34:46 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:34:47 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:34:48 user nova-compute[75234]: DEBUG nova.compute.manager [None req-d947149e-2723-4f84-84f4-a681e890571a tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: e3303cca-05a0-415a-b731-e1f12ad413f7] Instance event wait completed in 0 seconds for  {{(pid=75234) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Nov 29 03:34:48 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-d947149e-2723-4f84-84f4-a681e890571a tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: e3303cca-05a0-415a-b731-e1f12ad413f7] Guest created on hypervisor {{(pid=75234) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4421}}
Nov 29 03:34:48 user nova-compute[75234]: INFO nova.virt.libvirt.driver [-] [instance: e3303cca-05a0-415a-b731-e1f12ad413f7] Instance spawned successfully.
Nov 29 03:34:48 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-d947149e-2723-4f84-84f4-a681e890571a tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: e3303cca-05a0-415a-b731-e1f12ad413f7] Attempting to register defaults for the following image properties: ['hw_input_bus', 'hw_pointer_model'] {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:904}}
Nov 29 03:34:48 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-d947149e-2723-4f84-84f4-a681e890571a tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: e3303cca-05a0-415a-b731-e1f12ad413f7] Found default for hw_input_bus of None {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:34:48 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-d947149e-2723-4f84-84f4-a681e890571a tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: e3303cca-05a0-415a-b731-e1f12ad413f7] Found default for hw_pointer_model of None {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:34:48 user nova-compute[75234]: INFO nova.compute.manager [None req-d947149e-2723-4f84-84f4-a681e890571a tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: e3303cca-05a0-415a-b731-e1f12ad413f7] Took 4.95 seconds to spawn the instance on the hypervisor.
Nov 29 03:34:48 user nova-compute[75234]: DEBUG nova.compute.manager [None req-d947149e-2723-4f84-84f4-a681e890571a tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: e3303cca-05a0-415a-b731-e1f12ad413f7] Checking state {{(pid=75234) _get_power_state /opt/stack/nova/nova/compute/manager.py:1782}}
Nov 29 03:34:48 user nova-compute[75234]: INFO nova.compute.manager [None req-d947149e-2723-4f84-84f4-a681e890571a tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: e3303cca-05a0-415a-b731-e1f12ad413f7] Took 19.29 seconds to build instance.
Nov 29 03:34:48 user nova-compute[75234]: DEBUG nova.compute.manager [req-74121184-18db-4db5-93f2-cc72a0e726cf req-e2107fe0-0280-4bd5-9e4c-801c33d1aa3c service nova] [instance: e3303cca-05a0-415a-b731-e1f12ad413f7] Received event network-vif-plugged-fd43dff4-5bd3-4791-a078-45bc982d3bb7 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:34:48 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-74121184-18db-4db5-93f2-cc72a0e726cf req-e2107fe0-0280-4bd5-9e4c-801c33d1aa3c service nova] Acquiring lock "e3303cca-05a0-415a-b731-e1f12ad413f7-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:34:48 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-74121184-18db-4db5-93f2-cc72a0e726cf req-e2107fe0-0280-4bd5-9e4c-801c33d1aa3c service nova] Lock "e3303cca-05a0-415a-b731-e1f12ad413f7-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:34:48 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-74121184-18db-4db5-93f2-cc72a0e726cf req-e2107fe0-0280-4bd5-9e4c-801c33d1aa3c service nova] Lock "e3303cca-05a0-415a-b731-e1f12ad413f7-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:34:48 user nova-compute[75234]: DEBUG nova.compute.manager [req-74121184-18db-4db5-93f2-cc72a0e726cf req-e2107fe0-0280-4bd5-9e4c-801c33d1aa3c service nova] [instance: e3303cca-05a0-415a-b731-e1f12ad413f7] No waiting events found dispatching network-vif-plugged-fd43dff4-5bd3-4791-a078-45bc982d3bb7 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:34:48 user nova-compute[75234]: WARNING nova.compute.manager [req-74121184-18db-4db5-93f2-cc72a0e726cf req-e2107fe0-0280-4bd5-9e4c-801c33d1aa3c service nova] [instance: e3303cca-05a0-415a-b731-e1f12ad413f7] Received unexpected event network-vif-plugged-fd43dff4-5bd3-4791-a078-45bc982d3bb7 for instance with vm_state active and task_state None.
Nov 29 03:34:48 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-d947149e-2723-4f84-84f4-a681e890571a tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Lock "e3303cca-05a0-415a-b731-e1f12ad413f7" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 19.395s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:34:50 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:34:52 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:34:55 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:34:57 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:35:00 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:35:02 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:35:05 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:35:05 user nova-compute[75234]: DEBUG nova.compute.manager [req-ade0c29a-b60b-4078-a25c-324dfd6760cf req-29c725ff-21bb-41f7-8f54-71cd961ecb3c service nova] [instance: 01d62e54-9957-40b1-8dfe-1783b64ffabf] Received event network-changed-6dbd7c27-31c9-4ddc-a81b-85f1e742d160 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:35:05 user nova-compute[75234]: DEBUG nova.compute.manager [req-ade0c29a-b60b-4078-a25c-324dfd6760cf req-29c725ff-21bb-41f7-8f54-71cd961ecb3c service nova] [instance: 01d62e54-9957-40b1-8dfe-1783b64ffabf] Refreshing instance network info cache due to event network-changed-6dbd7c27-31c9-4ddc-a81b-85f1e742d160. {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11117}}
Nov 29 03:35:05 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-ade0c29a-b60b-4078-a25c-324dfd6760cf req-29c725ff-21bb-41f7-8f54-71cd961ecb3c service nova] Acquiring lock "refresh_cache-01d62e54-9957-40b1-8dfe-1783b64ffabf" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:35:05 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-ade0c29a-b60b-4078-a25c-324dfd6760cf req-29c725ff-21bb-41f7-8f54-71cd961ecb3c service nova] Acquired lock "refresh_cache-01d62e54-9957-40b1-8dfe-1783b64ffabf" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:35:05 user nova-compute[75234]: DEBUG nova.network.neutron [req-ade0c29a-b60b-4078-a25c-324dfd6760cf req-29c725ff-21bb-41f7-8f54-71cd961ecb3c service nova] [instance: 01d62e54-9957-40b1-8dfe-1783b64ffabf] Refreshing network info cache for port 6dbd7c27-31c9-4ddc-a81b-85f1e742d160 {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2006}}
Nov 29 03:35:05 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-fa268302-1069-421a-ab62-3afbeb3f4052 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] Acquiring lock "01d62e54-9957-40b1-8dfe-1783b64ffabf" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:35:05 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-fa268302-1069-421a-ab62-3afbeb3f4052 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] Lock "01d62e54-9957-40b1-8dfe-1783b64ffabf" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:35:05 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-fa268302-1069-421a-ab62-3afbeb3f4052 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] Acquiring lock "01d62e54-9957-40b1-8dfe-1783b64ffabf-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:35:05 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-fa268302-1069-421a-ab62-3afbeb3f4052 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] Lock "01d62e54-9957-40b1-8dfe-1783b64ffabf-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:35:05 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-fa268302-1069-421a-ab62-3afbeb3f4052 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] Lock "01d62e54-9957-40b1-8dfe-1783b64ffabf-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:35:05 user nova-compute[75234]: INFO nova.compute.manager [None req-fa268302-1069-421a-ab62-3afbeb3f4052 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] [instance: 01d62e54-9957-40b1-8dfe-1783b64ffabf] Terminating instance
Nov 29 03:35:05 user nova-compute[75234]: DEBUG nova.compute.manager [None req-fa268302-1069-421a-ab62-3afbeb3f4052 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] [instance: 01d62e54-9957-40b1-8dfe-1783b64ffabf] Start destroying the instance on the hypervisor. {{(pid=75234) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3140}}
Nov 29 03:35:05 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:35:05 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:35:05 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:35:05 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:35:05 user nova-compute[75234]: DEBUG nova.network.neutron [req-ade0c29a-b60b-4078-a25c-324dfd6760cf req-29c725ff-21bb-41f7-8f54-71cd961ecb3c service nova] [instance: 01d62e54-9957-40b1-8dfe-1783b64ffabf] Updated VIF entry in instance network info cache for port 6dbd7c27-31c9-4ddc-a81b-85f1e742d160. {{(pid=75234) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3481}}
Nov 29 03:35:05 user nova-compute[75234]: DEBUG nova.network.neutron [req-ade0c29a-b60b-4078-a25c-324dfd6760cf req-29c725ff-21bb-41f7-8f54-71cd961ecb3c service nova] [instance: 01d62e54-9957-40b1-8dfe-1783b64ffabf] Updating instance_info_cache with network_info: [{"id": "6dbd7c27-31c9-4ddc-a81b-85f1e742d160", "address": "fa:16:3e:57:6a:26", "network": {"id": "504616f5-0d6b-47a6-aaf5-663983428b41", "bridge": "br-int", "label": "tempest-TestShelveInstance-438404840-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "53e55fcb8cd74895baec6f9528770bbc", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap6dbd7c27-31", "ovs_interfaceid": "6dbd7c27-31c9-4ddc-a81b-85f1e742d160", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:35:05 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-ade0c29a-b60b-4078-a25c-324dfd6760cf req-29c725ff-21bb-41f7-8f54-71cd961ecb3c service nova] Releasing lock "refresh_cache-01d62e54-9957-40b1-8dfe-1783b64ffabf" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:35:06 user nova-compute[75234]: INFO nova.virt.libvirt.driver [-] [instance: 01d62e54-9957-40b1-8dfe-1783b64ffabf] Instance destroyed successfully.
Nov 29 03:35:06 user nova-compute[75234]: DEBUG nova.objects.instance [None req-fa268302-1069-421a-ab62-3afbeb3f4052 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] Lazy-loading 'resources' on Instance uuid 01d62e54-9957-40b1-8dfe-1783b64ffabf {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:35:06 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-fa268302-1069-421a-ab62-3afbeb3f4052 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=True,compute_id=1,config_drive='',created_at=2023-11-29T03:29:28Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-TestShelveInstance-server-1849172102',display_name='tempest-TestShelveInstance-server-1849172102',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-testshelveinstance-server-1849172102',id=44,image_ref='',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBPnJOw+AusvgDl0zYq6G0Nlkai7pjeaHxMjEe+mgW4X2uDyLlbOtVi5OTxN9x2/pgEhihEfkWnF2CarcswPn27UxxBjasnu8imtsEqjFxc/RzHA5mgFRB5eUL6c9v+ODYA==',key_name='tempest-TestShelveInstance-1141474138',keypairs=<?>,launch_index=0,launched_at=2023-11-29T03:33:28Z,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='53e55fcb8cd74895baec6f9528770bbc',ramdisk_id='',reservation_id='r-vhus4691',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',clean_attempts='1',image_base_image_ref='',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-TestShelveInstance-1683282958',owner_user_name='tempest-TestShelveInstance-1683282958-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-11-29T03:33:28Z,user_data=None,user_id='5728da78dc074c1993a8a2cc9b821ed6',uuid=01d62e54-9957-40b1-8dfe-1783b64ffabf,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "6dbd7c27-31c9-4ddc-a81b-85f1e742d160", "address": "fa:16:3e:57:6a:26", "network": {"id": "504616f5-0d6b-47a6-aaf5-663983428b41", "bridge": "br-int", "label": "tempest-TestShelveInstance-438404840-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.210", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "53e55fcb8cd74895baec6f9528770bbc", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap6dbd7c27-31", "ovs_interfaceid": "6dbd7c27-31c9-4ddc-a81b-85f1e742d160", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Nov 29 03:35:06 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-fa268302-1069-421a-ab62-3afbeb3f4052 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] Converting VIF {"id": "6dbd7c27-31c9-4ddc-a81b-85f1e742d160", "address": "fa:16:3e:57:6a:26", "network": {"id": "504616f5-0d6b-47a6-aaf5-663983428b41", "bridge": "br-int", "label": "tempest-TestShelveInstance-438404840-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.13", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.210", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "53e55fcb8cd74895baec6f9528770bbc", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap6dbd7c27-31", "ovs_interfaceid": "6dbd7c27-31c9-4ddc-a81b-85f1e742d160", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:35:06 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-fa268302-1069-421a-ab62-3afbeb3f4052 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:57:6a:26,bridge_name='br-int',has_traffic_filtering=True,id=6dbd7c27-31c9-4ddc-a81b-85f1e742d160,network=Network(504616f5-0d6b-47a6-aaf5-663983428b41),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap6dbd7c27-31') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:35:06 user nova-compute[75234]: DEBUG os_vif [None req-fa268302-1069-421a-ab62-3afbeb3f4052 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] Unplugging vif VIFOpenVSwitch(active=True,address=fa:16:3e:57:6a:26,bridge_name='br-int',has_traffic_filtering=True,id=6dbd7c27-31c9-4ddc-a81b-85f1e742d160,network=Network(504616f5-0d6b-47a6-aaf5-663983428b41),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap6dbd7c27-31') {{(pid=75234) unplug /opt/stack/data/venv/lib/python3.10/site-packages/os_vif/__init__.py:109}}
Nov 29 03:35:06 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:35:06 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tap6dbd7c27-31, bridge=br-int, if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:35:06 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:35:06 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:35:06 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:35:06 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbDestroyCommand(_result=None, table=QoS, record=25437cf9-68c3-4b1b-bb38-6fe1184ff97e) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:35:06 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:35:06 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:35:06 user nova-compute[75234]: INFO os_vif [None req-fa268302-1069-421a-ab62-3afbeb3f4052 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] Successfully unplugged vif VIFOpenVSwitch(active=True,address=fa:16:3e:57:6a:26,bridge_name='br-int',has_traffic_filtering=True,id=6dbd7c27-31c9-4ddc-a81b-85f1e742d160,network=Network(504616f5-0d6b-47a6-aaf5-663983428b41),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap6dbd7c27-31')
Nov 29 03:35:06 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-fa268302-1069-421a-ab62-3afbeb3f4052 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] [instance: 01d62e54-9957-40b1-8dfe-1783b64ffabf] calling os-brick to detach iSCSI Volume {{(pid=75234) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Nov 29 03:35:06 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-fa268302-1069-421a-ab62-3afbeb3f4052 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f8a5050c9a0>, {'target_iqn': 'iqn.2010-10.org.openstack:43e92d4a-7e98-4777-8fa5-3cec396b86ce', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'AQnkxjFp', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdf'}, None), 'kwargs': True}}" {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:176}}
Nov 29 03:35:06 user nova-compute[75234]: WARNING os_brick.initiator.connectors.base [None req-fa268302-1069-421a-ab62-3afbeb3f4052 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Nov 29 03:35:06 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-fa268302-1069-421a-ab62-3afbeb3f4052 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:68}}
Nov 29 03:35:06 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-fa268302-1069-421a-ab62-3afbeb3f4052 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:73}}
Nov 29 03:35:06 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:35:06 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.023s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:35:06 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[021918f8-53e8-428a-965f-7af37eafbcf0]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:a3d30ae4-ca6d-4e06-8fc9-caf4d91ef900\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:43e92d4a-7e98-4777-8fa5-3cec396b86ce\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:dd89ea2e-2a9f-4511-9b6f-c5177ff21182\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:257d7345-5b29-4bc1-8844-e211864c06c3\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:35:06 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-fa268302-1069-421a-ab62-3afbeb3f4052 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS:
Nov 29 03:35:06 user nova-compute[75234]: No targets found.
Nov 29 03:35:06 user nova-compute[75234]: iSNS:
Nov 29 03:35:06 user nova-compute[75234]: No targets found.
Nov 29 03:35:06 user nova-compute[75234]: STATIC:
Nov 29 03:35:06 user nova-compute[75234]: Target: iqn.2010-10.org.openstack:a3d30ae4-ca6d-4e06-8fc9-caf4d91ef900
Nov 29 03:35:06 user nova-compute[75234]:         Portal: 172.16.0.220:3260,-1
Nov 29 03:35:06 user nova-compute[75234]:                 Iface Name: default
Nov 29 03:35:06 user nova-compute[75234]: Target: iqn.2010-10.org.openstack:43e92d4a-7e98-4777-8fa5-3cec396b86ce
Nov 29 03:35:06 user nova-compute[75234]:         Portal: 172.16.0.220:3260,-1
Nov 29 03:35:06 user nova-compute[75234]:                 Iface Name: default
Nov 29 03:35:06 user nova-compute[75234]: Target: iqn.2010-10.org.openstack:dd89ea2e-2a9f-4511-9b6f-c5177ff21182
Nov 29 03:35:06 user nova-compute[75234]:         Portal: 172.16.0.220:3260,-1
Nov 29 03:35:06 user nova-compute[75234]:                 Iface Name: default
Nov 29 03:35:06 user nova-compute[75234]: Target: iqn.2010-10.org.openstack:257d7345-5b29-4bc1-8844-e211864c06c3
Nov 29 03:35:06 user nova-compute[75234]:         Portal: 172.16.0.220:3260,-1
Nov 29 03:35:06 user nova-compute[75234]:                 Iface Name: default
Nov 29 03:35:06 user nova-compute[75234]: Target: iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0
Nov 29 03:35:06 user nova-compute[75234]:         Portal: 172.16.0.220:3260,-1
Nov 29 03:35:06 user nova-compute[75234]:                 Iface Name: default
Nov 29 03:35:06 user nova-compute[75234]: FIRMWARE:
Nov 29 03:35:06 user nova-compute[75234]: No targets found.
Nov 29 03:35:06 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:35:06 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-fa268302-1069-421a-ab62-3afbeb3f4052 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] Regex to get portals from discoverydb: ^SENDTARGETS:
Nov 29 03:35:06 user nova-compute[75234]: .*?^DiscoveryAddress: 172.16.0.220,3260.*?
Nov 29 03:35:06 user nova-compute[75234]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=75234) _get_discoverydb_portals /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:382}}
Nov 29 03:35:06 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-fa268302-1069-421a-ab62-3afbeb3f4052 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:43e92d4a-7e98-4777-8fa5-3cec396b86ce', 0)] {{(pid=75234) _get_connection_devices /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:830}}
Nov 29 03:35:06 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:35:06 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.018s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:35:06 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[bfd03e7c-dc73-4f14-b2c8-45d1d221b42b]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:257d7345-5b29-4bc1-8844-e211864c06c3\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:43e92d4a-7e98-4777-8fa5-3cec396b86ce\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:a3d30ae4-ca6d-4e06-8fc9-caf4d91ef900\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:dd89ea2e-2a9f-4511-9b6f-c5177ff21182\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:35:06 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:35:06 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.027s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:35:06 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[14797a1f-5f0f-4093-82cd-e2b7a9d6faa0]: (4, ('tcp: [42] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0 (non-flash)\ntcp: [54] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a3d30ae4-ca6d-4e06-8fc9-caf4d91ef900 (non-flash)\ntcp: [55] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:257d7345-5b29-4bc1-8844-e211864c06c3 (non-flash)\ntcp: [56] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:43e92d4a-7e98-4777-8fa5-3cec396b86ce (non-flash)\ntcp: [57] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:dd89ea2e-2a9f-4511-9b6f-c5177ff21182 (non-flash)\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:35:06 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-fa268302-1069-421a-ab62-3afbeb3f4052 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [42] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0 (non-flash)
Nov 29 03:35:06 user nova-compute[75234]: tcp: [54] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a3d30ae4-ca6d-4e06-8fc9-caf4d91ef900 (non-flash)
Nov 29 03:35:06 user nova-compute[75234]: tcp: [55] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:257d7345-5b29-4bc1-8844-e211864c06c3 (non-flash)
Nov 29 03:35:06 user nova-compute[75234]: tcp: [56] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:43e92d4a-7e98-4777-8fa5-3cec396b86ce (non-flash)
Nov 29 03:35:06 user nova-compute[75234]: tcp: [57] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:dd89ea2e-2a9f-4511-9b6f-c5177ff21182 (non-flash)
Nov 29 03:35:06 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:35:06 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-fa268302-1069-421a-ab62-3afbeb3f4052 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] iscsi session list stdout=tcp: [42] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0 (non-flash)
Nov 29 03:35:06 user nova-compute[75234]: tcp: [54] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a3d30ae4-ca6d-4e06-8fc9-caf4d91ef900 (non-flash)
Nov 29 03:35:06 user nova-compute[75234]: tcp: [55] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:257d7345-5b29-4bc1-8844-e211864c06c3 (non-flash)
Nov 29 03:35:06 user nova-compute[75234]: tcp: [56] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:43e92d4a-7e98-4777-8fa5-3cec396b86ce (non-flash)
Nov 29 03:35:06 user nova-compute[75234]: tcp: [57] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:dd89ea2e-2a9f-4511-9b6f-c5177ff21182 (non-flash)
Nov 29 03:35:06 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsi_session /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Nov 29 03:35:06 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-fa268302-1069-421a-ab62-3afbeb3f4052 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7f8a50748c10>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:43e92d4a-7e98-4777-8fa5-3cec396b86ce'): ({'sdf'}, set())}) {{(pid=75234) _get_connection_devices /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:863}}
Nov 29 03:35:06 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-fa268302-1069-421a-ab62-3afbeb3f4052 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] Removing single pathed devices sdf {{(pid=75234) remove_connection /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:374}}
Nov 29 03:35:06 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:35:06 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.019s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:35:06 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[a6e5a6df-89b0-49da-bbcb-d442020dd0d4]: (4, ('path checker states:\nup                  5\n\npaths: 0\nbusy: False\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:35:06 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdf {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:35:06 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdf" returned: 0 in 0.021s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:35:06 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[90c14f70-b089-4534-aa85-ef75f63aad42]: (4, ('ok\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:35:06 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-fa268302-1069-421a-ab62-3afbeb3f4052 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] Flushing IO for device /dev/sdf {{(pid=75234) flush_device_io /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:434}}
Nov 29 03:35:06 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdf {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:35:06 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdf" returned: 0 in 0.016s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:35:06 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[0a271709-8bf1-4e0d-baf6-91eb21c21a5f]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:35:06 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-fa268302-1069-421a-ab62-3afbeb3f4052 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] Remove SCSI device /dev/sdf with /sys/block/sdf/device/delete {{(pid=75234) remove_scsi_device /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:148}}
Nov 29 03:35:06 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdf/device/delete {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:35:06 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdf/device/delete" returned: 0 in 0.047s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:35:06 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[0780de77-7c91-476c-bc22-e377d7d1ac1e]: (4, ('1', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:35:06 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-fa268302-1069-421a-ab62-3afbeb3f4052 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] Checking to see if SCSI volumes sdf have been removed. {{(pid=75234) wait_for_volumes_removal /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:156}}
Nov 29 03:35:06 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-fa268302-1069-421a-ab62-3afbeb3f4052 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] SCSI volumes sdf have been removed. {{(pid=75234) wait_for_volumes_removal /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:166}}
Nov 29 03:35:06 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-fa268302-1069-421a-ab62-3afbeb3f4052 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:43e92d4a-7e98-4777-8fa5-3cec396b86ce')] {{(pid=75234) _disconnect_connection /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Nov 29 03:35:06 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:43e92d4a-7e98-4777-8fa5-3cec396b86ce -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:35:06 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:43e92d4a-7e98-4777-8fa5-3cec396b86ce -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.018s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:35:06 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[55a36e53-4e1e-40c8-9a9c-391e270c559a]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:35:06 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-fa268302-1069-421a-ab62-3afbeb3f4052 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:35:06 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:43e92d4a-7e98-4777-8fa5-3cec396b86ce -p 172.16.0.220:3260 --logout {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:35:06 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:43e92d4a-7e98-4777-8fa5-3cec396b86ce -p 172.16.0.220:3260 --logout" returned: 0 in 0.056s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:35:06 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[1bd705d8-bc9d-4761-be11-b16170fd18bb]: (4, ('Logging out of session [sid: 56, target: iqn.2010-10.org.openstack:43e92d4a-7e98-4777-8fa5-3cec396b86ce, portal: 172.16.0.220,3260]\nLogout of [sid: 56, target: iqn.2010-10.org.openstack:43e92d4a-7e98-4777-8fa5-3cec396b86ce, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:35:06 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-fa268302-1069-421a-ab62-3afbeb3f4052 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 56, target: iqn.2010-10.org.openstack:43e92d4a-7e98-4777-8fa5-3cec396b86ce, portal: 172.16.0.220,3260]
Nov 29 03:35:06 user nova-compute[75234]: Logout of [sid: 56, target: iqn.2010-10.org.openstack:43e92d4a-7e98-4777-8fa5-3cec396b86ce, portal: 172.16.0.220,3260] successful.
Nov 29 03:35:06 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:35:06 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:43e92d4a-7e98-4777-8fa5-3cec396b86ce -p 172.16.0.220:3260 --op delete {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:35:06 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:43e92d4a-7e98-4777-8fa5-3cec396b86ce -p 172.16.0.220:3260 --op delete" returned: 0 in 0.026s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:35:06 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[ba65a654-3481-4d3b-bc55-40bb1daa113e]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:35:06 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-fa268302-1069-421a-ab62-3afbeb3f4052 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:35:06 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-fa268302-1069-421a-ab62-3afbeb3f4052 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.325s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:87}}
Nov 29 03:35:06 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-fa268302-1069-421a-ab62-3afbeb3f4052 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] <== disconnect_volume: return (326ms) None {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:203}}
Nov 29 03:35:06 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-fa268302-1069-421a-ab62-3afbeb3f4052 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] [instance: 01d62e54-9957-40b1-8dfe-1783b64ffabf] Disconnected iSCSI Volume {{(pid=75234) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:79}}
Nov 29 03:35:06 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-fa268302-1069-421a-ab62-3afbeb3f4052 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] [instance: 01d62e54-9957-40b1-8dfe-1783b64ffabf] Deleting instance files /opt/stack/data/nova/instances/01d62e54-9957-40b1-8dfe-1783b64ffabf_del
Nov 29 03:35:06 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-fa268302-1069-421a-ab62-3afbeb3f4052 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] [instance: 01d62e54-9957-40b1-8dfe-1783b64ffabf] Deletion of /opt/stack/data/nova/instances/01d62e54-9957-40b1-8dfe-1783b64ffabf_del complete
Nov 29 03:35:06 user nova-compute[75234]: INFO nova.compute.manager [None req-fa268302-1069-421a-ab62-3afbeb3f4052 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] [instance: 01d62e54-9957-40b1-8dfe-1783b64ffabf] Took 1.18 seconds to destroy the instance on the hypervisor.
Nov 29 03:35:06 user nova-compute[75234]: DEBUG oslo.service.loopingcall [None req-fa268302-1069-421a-ab62-3afbeb3f4052 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=75234) func /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/loopingcall.py:435}}
Nov 29 03:35:06 user nova-compute[75234]: DEBUG nova.compute.manager [-] [instance: 01d62e54-9957-40b1-8dfe-1783b64ffabf] Deallocating network for instance {{(pid=75234) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2279}}
Nov 29 03:35:06 user nova-compute[75234]: DEBUG nova.network.neutron [-] [instance: 01d62e54-9957-40b1-8dfe-1783b64ffabf] deallocate_for_instance() {{(pid=75234) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1802}}
Nov 29 03:35:06 user nova-compute[75234]: DEBUG nova.compute.manager [req-f3326222-27d9-4273-8be3-d218a09c9b10 req-d56a01f0-dd36-4661-9b44-7ee557109f16 service nova] [instance: 01d62e54-9957-40b1-8dfe-1783b64ffabf] Received event network-vif-deleted-6dbd7c27-31c9-4ddc-a81b-85f1e742d160 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:35:06 user nova-compute[75234]: INFO nova.compute.manager [req-f3326222-27d9-4273-8be3-d218a09c9b10 req-d56a01f0-dd36-4661-9b44-7ee557109f16 service nova] [instance: 01d62e54-9957-40b1-8dfe-1783b64ffabf] Neutron deleted interface 6dbd7c27-31c9-4ddc-a81b-85f1e742d160; detaching it from the instance and deleting it from the info cache
Nov 29 03:35:06 user nova-compute[75234]: DEBUG nova.network.neutron [req-f3326222-27d9-4273-8be3-d218a09c9b10 req-d56a01f0-dd36-4661-9b44-7ee557109f16 service nova] [instance: 01d62e54-9957-40b1-8dfe-1783b64ffabf] Updating instance_info_cache with network_info: [] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:35:06 user nova-compute[75234]: DEBUG nova.network.neutron [-] [instance: 01d62e54-9957-40b1-8dfe-1783b64ffabf] Updating instance_info_cache with network_info: [] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:35:06 user nova-compute[75234]: DEBUG nova.compute.manager [req-f3326222-27d9-4273-8be3-d218a09c9b10 req-d56a01f0-dd36-4661-9b44-7ee557109f16 service nova] [instance: 01d62e54-9957-40b1-8dfe-1783b64ffabf] Detach interface failed, port_id=6dbd7c27-31c9-4ddc-a81b-85f1e742d160, reason: Instance 01d62e54-9957-40b1-8dfe-1783b64ffabf could not be found. {{(pid=75234) _process_instance_vif_deleted_event /opt/stack/nova/nova/compute/manager.py:10946}}
Nov 29 03:35:06 user nova-compute[75234]: INFO nova.compute.manager [-] [instance: 01d62e54-9957-40b1-8dfe-1783b64ffabf] Took 0.42 seconds to deallocate network for instance.
Nov 29 03:35:07 user nova-compute[75234]: DEBUG nova.compute.manager [req-6ccd1ddc-c360-4aa6-9389-b4d2b58bc7bb req-81b768b5-89f0-4c9f-b3c7-c608549d7c65 service nova] [instance: 01d62e54-9957-40b1-8dfe-1783b64ffabf] Received event network-vif-unplugged-6dbd7c27-31c9-4ddc-a81b-85f1e742d160 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:35:07 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-6ccd1ddc-c360-4aa6-9389-b4d2b58bc7bb req-81b768b5-89f0-4c9f-b3c7-c608549d7c65 service nova] Acquiring lock "01d62e54-9957-40b1-8dfe-1783b64ffabf-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:35:07 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-6ccd1ddc-c360-4aa6-9389-b4d2b58bc7bb req-81b768b5-89f0-4c9f-b3c7-c608549d7c65 service nova] Lock "01d62e54-9957-40b1-8dfe-1783b64ffabf-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:35:07 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-6ccd1ddc-c360-4aa6-9389-b4d2b58bc7bb req-81b768b5-89f0-4c9f-b3c7-c608549d7c65 service nova] Lock "01d62e54-9957-40b1-8dfe-1783b64ffabf-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:35:07 user nova-compute[75234]: DEBUG nova.compute.manager [req-6ccd1ddc-c360-4aa6-9389-b4d2b58bc7bb req-81b768b5-89f0-4c9f-b3c7-c608549d7c65 service nova] [instance: 01d62e54-9957-40b1-8dfe-1783b64ffabf] No waiting events found dispatching network-vif-unplugged-6dbd7c27-31c9-4ddc-a81b-85f1e742d160 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:35:07 user nova-compute[75234]: DEBUG nova.compute.manager [req-6ccd1ddc-c360-4aa6-9389-b4d2b58bc7bb req-81b768b5-89f0-4c9f-b3c7-c608549d7c65 service nova] [instance: 01d62e54-9957-40b1-8dfe-1783b64ffabf] Received event network-vif-unplugged-6dbd7c27-31c9-4ddc-a81b-85f1e742d160 for instance with task_state deleting. {{(pid=75234) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10890}}
Nov 29 03:35:07 user nova-compute[75234]: DEBUG nova.compute.manager [req-6ccd1ddc-c360-4aa6-9389-b4d2b58bc7bb req-81b768b5-89f0-4c9f-b3c7-c608549d7c65 service nova] [instance: 01d62e54-9957-40b1-8dfe-1783b64ffabf] Received event network-vif-plugged-6dbd7c27-31c9-4ddc-a81b-85f1e742d160 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:35:07 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-6ccd1ddc-c360-4aa6-9389-b4d2b58bc7bb req-81b768b5-89f0-4c9f-b3c7-c608549d7c65 service nova] Acquiring lock "01d62e54-9957-40b1-8dfe-1783b64ffabf-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:35:07 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-6ccd1ddc-c360-4aa6-9389-b4d2b58bc7bb req-81b768b5-89f0-4c9f-b3c7-c608549d7c65 service nova] Lock "01d62e54-9957-40b1-8dfe-1783b64ffabf-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:35:07 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-6ccd1ddc-c360-4aa6-9389-b4d2b58bc7bb req-81b768b5-89f0-4c9f-b3c7-c608549d7c65 service nova] Lock "01d62e54-9957-40b1-8dfe-1783b64ffabf-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:35:07 user nova-compute[75234]: DEBUG nova.compute.manager [req-6ccd1ddc-c360-4aa6-9389-b4d2b58bc7bb req-81b768b5-89f0-4c9f-b3c7-c608549d7c65 service nova] [instance: 01d62e54-9957-40b1-8dfe-1783b64ffabf] No waiting events found dispatching network-vif-plugged-6dbd7c27-31c9-4ddc-a81b-85f1e742d160 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:35:07 user nova-compute[75234]: WARNING nova.compute.manager [req-6ccd1ddc-c360-4aa6-9389-b4d2b58bc7bb req-81b768b5-89f0-4c9f-b3c7-c608549d7c65 service nova] [instance: 01d62e54-9957-40b1-8dfe-1783b64ffabf] Received unexpected event network-vif-plugged-6dbd7c27-31c9-4ddc-a81b-85f1e742d160 for instance with vm_state active and task_state deleting.
Nov 29 03:35:07 user nova-compute[75234]: INFO nova.compute.manager [None req-fa268302-1069-421a-ab62-3afbeb3f4052 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] [instance: 01d62e54-9957-40b1-8dfe-1783b64ffabf] Took 0.76 seconds to detach 1 volumes for instance.
Nov 29 03:35:07 user nova-compute[75234]: DEBUG nova.compute.manager [None req-fa268302-1069-421a-ab62-3afbeb3f4052 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] [instance: 01d62e54-9957-40b1-8dfe-1783b64ffabf] Deleting volume: 43e92d4a-7e98-4777-8fa5-3cec396b86ce {{(pid=75234) _cleanup_volumes /opt/stack/nova/nova/compute/manager.py:3237}}
Nov 29 03:35:07 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:35:07 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-fa268302-1069-421a-ab62-3afbeb3f4052 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:35:07 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-fa268302-1069-421a-ab62-3afbeb3f4052 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:35:08 user nova-compute[75234]: DEBUG nova.compute.provider_tree [None req-fa268302-1069-421a-ab62-3afbeb3f4052 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] Inventory has not changed in ProviderTree for provider: 4d291220-7c37-4839-a0e3-b23c66502b74 {{(pid=75234) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Nov 29 03:35:08 user nova-compute[75234]: DEBUG nova.scheduler.client.report [None req-fa268302-1069-421a-ab62-3afbeb3f4052 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] Inventory has not changed for provider 4d291220-7c37-4839-a0e3-b23c66502b74 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=75234) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Nov 29 03:35:08 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-fa268302-1069-421a-ab62-3afbeb3f4052 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.260s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:35:08 user nova-compute[75234]: INFO nova.scheduler.client.report [None req-fa268302-1069-421a-ab62-3afbeb3f4052 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] Deleted allocations for instance 01d62e54-9957-40b1-8dfe-1783b64ffabf
Nov 29 03:35:08 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-fa268302-1069-421a-ab62-3afbeb3f4052 tempest-TestShelveInstance-1683282958 tempest-TestShelveInstance-1683282958-project-member] Lock "01d62e54-9957-40b1-8dfe-1783b64ffabf" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 2.945s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:35:11 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:35:12 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:35:13 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:35:14 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:35:15 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:35:15 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:35:15 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=75234) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10541}}
Nov 29 03:35:16 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:35:16 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager.update_available_resource {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:35:16 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:35:16 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:35:16 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:35:16 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Auditing locally available compute resources for user (node: user) {{(pid=75234) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:907}}
Nov 29 03:35:16 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/65d48c93-b6da-4f51-9d5a-0676dda62a60/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:35:16 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/65d48c93-b6da-4f51-9d5a-0676dda62a60/disk --force-share --output=json" returned: 0 in 0.175s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:35:16 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/65d48c93-b6da-4f51-9d5a-0676dda62a60/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:35:16 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/65d48c93-b6da-4f51-9d5a-0676dda62a60/disk --force-share --output=json" returned: 0 in 0.169s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:35:16 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] skipping disk /dev/sde (vda) as it is a volume {{(pid=75234) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11213}}
Nov 29 03:35:16 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/ebc3e3b2-037d-43ef-9426-caebe298ee3e/disk.rescue --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:35:17 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/ebc3e3b2-037d-43ef-9426-caebe298ee3e/disk.rescue --force-share --output=json" returned: 0 in 0.174s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:35:17 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/ebc3e3b2-037d-43ef-9426-caebe298ee3e/disk.rescue --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:35:17 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/ebc3e3b2-037d-43ef-9426-caebe298ee3e/disk.rescue --force-share --output=json" returned: 0 in 0.168s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:35:17 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] skipping disk /dev/sdc (vda) as it is a volume {{(pid=75234) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11213}}
Nov 29 03:35:17 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] skipping disk /dev/sdd (vda) as it is a volume {{(pid=75234) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11213}}
Nov 29 03:35:17 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/abb01889-df4b-44a6-9827-87056d7ca125/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:35:17 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/abb01889-df4b-44a6-9827-87056d7ca125/disk --force-share --output=json" returned: 0 in 0.179s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:35:17 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/abb01889-df4b-44a6-9827-87056d7ca125/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:35:17 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/abb01889-df4b-44a6-9827-87056d7ca125/disk --force-share --output=json" returned: 0 in 0.174s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:35:17 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] skipping disk /dev/sdb (vda) as it is a volume {{(pid=75234) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11213}}
Nov 29 03:35:18 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:35:18 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:35:18 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:35:18 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Hypervisor/Node resource view: name=user free_ram=8345MB free_disk=24.89122772216797GB free_vcpus=6 pci_devices=[{"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}] {{(pid=75234) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1106}}
Nov 29 03:35:18 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:35:18 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:35:18 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance 65d48c93-b6da-4f51-9d5a-0676dda62a60 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 512, 'DISK_GB': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:35:18 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance d5096ca2-d005-45ac-a487-11a810446859 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 512}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:35:18 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance abb01889-df4b-44a6-9827-87056d7ca125 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 512, 'DISK_GB': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:35:18 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance ebc3e3b2-037d-43ef-9426-caebe298ee3e actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 512}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:35:18 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance 884589ec-b86c-4094-9a3f-54b641311d62 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 512}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:35:18 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance e3303cca-05a0-415a-b731-e1f12ad413f7 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 512}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:35:18 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Total usable vcpus: 12, total allocated vcpus: 6 {{(pid=75234) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1129}}
Nov 29 03:35:18 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Final resource view: name=user phys_ram=16011MB used_ram=3584MB phys_disk=40GB used_disk=2GB total_vcpus=12 used_vcpus=6 pci_stats=[] {{(pid=75234) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1138}}
Nov 29 03:35:18 user nova-compute[75234]: DEBUG nova.compute.provider_tree [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Inventory has not changed in ProviderTree for provider: 4d291220-7c37-4839-a0e3-b23c66502b74 {{(pid=75234) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Nov 29 03:35:18 user nova-compute[75234]: DEBUG nova.scheduler.client.report [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Inventory has not changed for provider 4d291220-7c37-4839-a0e3-b23c66502b74 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=75234) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Nov 29 03:35:18 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Compute_service record updated for user:user {{(pid=75234) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:1067}}
Nov 29 03:35:18 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.337s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:35:21 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:35:22 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:35:22 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Starting heal instance info cache {{(pid=75234) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9922}}
Nov 29 03:35:22 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Didn't find any instances for network info cache update. {{(pid=75234) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:10008}}
Nov 29 03:35:22 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:35:22 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:35:25 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:35:26 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:35:31 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:35:33 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:35:36 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:35:37 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:35:41 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:35:46 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:35:51 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:35:56 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:35:57 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:35:59 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:36:01 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:36:01 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:36:06 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:36:09 user nova-compute[75234]: INFO nova.compute.manager [None req-2a4990cf-cdbc-48e1-a54a-971664a94ac6 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] [instance: ebc3e3b2-037d-43ef-9426-caebe298ee3e] Unrescuing
Nov 29 03:36:09 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-2a4990cf-cdbc-48e1-a54a-971664a94ac6 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Acquiring lock "refresh_cache-ebc3e3b2-037d-43ef-9426-caebe298ee3e" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:36:09 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-2a4990cf-cdbc-48e1-a54a-971664a94ac6 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Acquired lock "refresh_cache-ebc3e3b2-037d-43ef-9426-caebe298ee3e" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:36:09 user nova-compute[75234]: DEBUG nova.network.neutron [None req-2a4990cf-cdbc-48e1-a54a-971664a94ac6 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] [instance: ebc3e3b2-037d-43ef-9426-caebe298ee3e] Building network info cache for instance {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2009}}
Nov 29 03:36:10 user nova-compute[75234]: DEBUG nova.network.neutron [None req-2a4990cf-cdbc-48e1-a54a-971664a94ac6 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] [instance: ebc3e3b2-037d-43ef-9426-caebe298ee3e] Updating instance_info_cache with network_info: [{"id": "bd0a4e5b-69cd-4f5c-ae22-994e4d8f2682", "address": "fa:16:3e:c2:aa:3e", "network": {"id": "4d086e08-7798-4cfe-9365-86be6f0a52d2", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-305624008-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "db00bcd0c82c44f1bda739e9857f6e84", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapbd0a4e5b-69", "ovs_interfaceid": "bd0a4e5b-69cd-4f5c-ae22-994e4d8f2682", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:36:10 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-2a4990cf-cdbc-48e1-a54a-971664a94ac6 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Releasing lock "refresh_cache-ebc3e3b2-037d-43ef-9426-caebe298ee3e" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:36:10 user nova-compute[75234]: DEBUG nova.objects.instance [None req-2a4990cf-cdbc-48e1-a54a-971664a94ac6 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Lazy-loading 'flavor' on Instance uuid ebc3e3b2-037d-43ef-9426-caebe298ee3e {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:36:10 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:36:10 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:36:11 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:36:11 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:36:11 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:36:11 user nova-compute[75234]: DEBUG nova.compute.manager [req-43958818-a134-4aeb-beef-0dfaba22fcdc req-7a2d78e6-7812-425e-963d-8543e9db6a4a service nova] [instance: ebc3e3b2-037d-43ef-9426-caebe298ee3e] Received event network-vif-unplugged-bd0a4e5b-69cd-4f5c-ae22-994e4d8f2682 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:36:11 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-43958818-a134-4aeb-beef-0dfaba22fcdc req-7a2d78e6-7812-425e-963d-8543e9db6a4a service nova] Acquiring lock "ebc3e3b2-037d-43ef-9426-caebe298ee3e-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:36:11 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-43958818-a134-4aeb-beef-0dfaba22fcdc req-7a2d78e6-7812-425e-963d-8543e9db6a4a service nova] Lock "ebc3e3b2-037d-43ef-9426-caebe298ee3e-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:36:11 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-43958818-a134-4aeb-beef-0dfaba22fcdc req-7a2d78e6-7812-425e-963d-8543e9db6a4a service nova] Lock "ebc3e3b2-037d-43ef-9426-caebe298ee3e-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:36:11 user nova-compute[75234]: DEBUG nova.compute.manager [req-43958818-a134-4aeb-beef-0dfaba22fcdc req-7a2d78e6-7812-425e-963d-8543e9db6a4a service nova] [instance: ebc3e3b2-037d-43ef-9426-caebe298ee3e] No waiting events found dispatching network-vif-unplugged-bd0a4e5b-69cd-4f5c-ae22-994e4d8f2682 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:36:11 user nova-compute[75234]: WARNING nova.compute.manager [req-43958818-a134-4aeb-beef-0dfaba22fcdc req-7a2d78e6-7812-425e-963d-8543e9db6a4a service nova] [instance: ebc3e3b2-037d-43ef-9426-caebe298ee3e] Received unexpected event network-vif-unplugged-bd0a4e5b-69cd-4f5c-ae22-994e4d8f2682 for instance with vm_state rescued and task_state unrescuing.
Nov 29 03:36:11 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:36:11 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:36:11 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:36:11 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:36:11 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:36:11 user nova-compute[75234]: INFO nova.virt.libvirt.driver [-] [instance: ebc3e3b2-037d-43ef-9426-caebe298ee3e] Instance destroyed successfully.
Nov 29 03:36:11 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:36:11 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:36:11 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:36:12 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:36:12 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:36:12 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:36:12 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:36:12 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:36:13 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:36:13 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:36:13 user nova-compute[75234]: DEBUG nova.compute.manager [req-921e087a-6724-4100-b2f8-f7051589a340 req-0a777e7c-8544-464b-a5a3-4ff6fde746cc service nova] [instance: ebc3e3b2-037d-43ef-9426-caebe298ee3e] Received event network-vif-plugged-bd0a4e5b-69cd-4f5c-ae22-994e4d8f2682 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:36:13 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-921e087a-6724-4100-b2f8-f7051589a340 req-0a777e7c-8544-464b-a5a3-4ff6fde746cc service nova] Acquiring lock "ebc3e3b2-037d-43ef-9426-caebe298ee3e-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:36:13 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-921e087a-6724-4100-b2f8-f7051589a340 req-0a777e7c-8544-464b-a5a3-4ff6fde746cc service nova] Lock "ebc3e3b2-037d-43ef-9426-caebe298ee3e-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:36:13 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-921e087a-6724-4100-b2f8-f7051589a340 req-0a777e7c-8544-464b-a5a3-4ff6fde746cc service nova] Lock "ebc3e3b2-037d-43ef-9426-caebe298ee3e-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:36:13 user nova-compute[75234]: DEBUG nova.compute.manager [req-921e087a-6724-4100-b2f8-f7051589a340 req-0a777e7c-8544-464b-a5a3-4ff6fde746cc service nova] [instance: ebc3e3b2-037d-43ef-9426-caebe298ee3e] No waiting events found dispatching network-vif-plugged-bd0a4e5b-69cd-4f5c-ae22-994e4d8f2682 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:36:13 user nova-compute[75234]: WARNING nova.compute.manager [req-921e087a-6724-4100-b2f8-f7051589a340 req-0a777e7c-8544-464b-a5a3-4ff6fde746cc service nova] [instance: ebc3e3b2-037d-43ef-9426-caebe298ee3e] Received unexpected event network-vif-plugged-bd0a4e5b-69cd-4f5c-ae22-994e4d8f2682 for instance with vm_state rescued and task_state unrescuing.
Nov 29 03:36:13 user nova-compute[75234]: DEBUG nova.compute.manager [req-921e087a-6724-4100-b2f8-f7051589a340 req-0a777e7c-8544-464b-a5a3-4ff6fde746cc service nova] [instance: ebc3e3b2-037d-43ef-9426-caebe298ee3e] Received event network-vif-plugged-bd0a4e5b-69cd-4f5c-ae22-994e4d8f2682 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:36:13 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-921e087a-6724-4100-b2f8-f7051589a340 req-0a777e7c-8544-464b-a5a3-4ff6fde746cc service nova] Acquiring lock "ebc3e3b2-037d-43ef-9426-caebe298ee3e-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:36:13 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-921e087a-6724-4100-b2f8-f7051589a340 req-0a777e7c-8544-464b-a5a3-4ff6fde746cc service nova] Lock "ebc3e3b2-037d-43ef-9426-caebe298ee3e-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:36:13 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-921e087a-6724-4100-b2f8-f7051589a340 req-0a777e7c-8544-464b-a5a3-4ff6fde746cc service nova] Lock "ebc3e3b2-037d-43ef-9426-caebe298ee3e-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:36:13 user nova-compute[75234]: DEBUG nova.compute.manager [req-921e087a-6724-4100-b2f8-f7051589a340 req-0a777e7c-8544-464b-a5a3-4ff6fde746cc service nova] [instance: ebc3e3b2-037d-43ef-9426-caebe298ee3e] No waiting events found dispatching network-vif-plugged-bd0a4e5b-69cd-4f5c-ae22-994e4d8f2682 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:36:13 user nova-compute[75234]: WARNING nova.compute.manager [req-921e087a-6724-4100-b2f8-f7051589a340 req-0a777e7c-8544-464b-a5a3-4ff6fde746cc service nova] [instance: ebc3e3b2-037d-43ef-9426-caebe298ee3e] Received unexpected event network-vif-plugged-bd0a4e5b-69cd-4f5c-ae22-994e4d8f2682 for instance with vm_state rescued and task_state unrescuing.
Nov 29 03:36:13 user nova-compute[75234]: DEBUG nova.compute.manager [req-921e087a-6724-4100-b2f8-f7051589a340 req-0a777e7c-8544-464b-a5a3-4ff6fde746cc service nova] [instance: ebc3e3b2-037d-43ef-9426-caebe298ee3e] Received event network-vif-plugged-bd0a4e5b-69cd-4f5c-ae22-994e4d8f2682 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:36:13 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-921e087a-6724-4100-b2f8-f7051589a340 req-0a777e7c-8544-464b-a5a3-4ff6fde746cc service nova] Acquiring lock "ebc3e3b2-037d-43ef-9426-caebe298ee3e-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:36:13 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-921e087a-6724-4100-b2f8-f7051589a340 req-0a777e7c-8544-464b-a5a3-4ff6fde746cc service nova] Lock "ebc3e3b2-037d-43ef-9426-caebe298ee3e-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:36:13 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-921e087a-6724-4100-b2f8-f7051589a340 req-0a777e7c-8544-464b-a5a3-4ff6fde746cc service nova] Lock "ebc3e3b2-037d-43ef-9426-caebe298ee3e-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:36:13 user nova-compute[75234]: DEBUG nova.compute.manager [req-921e087a-6724-4100-b2f8-f7051589a340 req-0a777e7c-8544-464b-a5a3-4ff6fde746cc service nova] [instance: ebc3e3b2-037d-43ef-9426-caebe298ee3e] No waiting events found dispatching network-vif-plugged-bd0a4e5b-69cd-4f5c-ae22-994e4d8f2682 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:36:13 user nova-compute[75234]: WARNING nova.compute.manager [req-921e087a-6724-4100-b2f8-f7051589a340 req-0a777e7c-8544-464b-a5a3-4ff6fde746cc service nova] [instance: ebc3e3b2-037d-43ef-9426-caebe298ee3e] Received unexpected event network-vif-plugged-bd0a4e5b-69cd-4f5c-ae22-994e4d8f2682 for instance with vm_state rescued and task_state unrescuing.
Nov 29 03:36:13 user nova-compute[75234]: DEBUG nova.compute.manager [req-921e087a-6724-4100-b2f8-f7051589a340 req-0a777e7c-8544-464b-a5a3-4ff6fde746cc service nova] [instance: ebc3e3b2-037d-43ef-9426-caebe298ee3e] Received event network-vif-unplugged-bd0a4e5b-69cd-4f5c-ae22-994e4d8f2682 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:36:13 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-921e087a-6724-4100-b2f8-f7051589a340 req-0a777e7c-8544-464b-a5a3-4ff6fde746cc service nova] Acquiring lock "ebc3e3b2-037d-43ef-9426-caebe298ee3e-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:36:13 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-921e087a-6724-4100-b2f8-f7051589a340 req-0a777e7c-8544-464b-a5a3-4ff6fde746cc service nova] Lock "ebc3e3b2-037d-43ef-9426-caebe298ee3e-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:36:13 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-921e087a-6724-4100-b2f8-f7051589a340 req-0a777e7c-8544-464b-a5a3-4ff6fde746cc service nova] Lock "ebc3e3b2-037d-43ef-9426-caebe298ee3e-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:36:13 user nova-compute[75234]: DEBUG nova.compute.manager [req-921e087a-6724-4100-b2f8-f7051589a340 req-0a777e7c-8544-464b-a5a3-4ff6fde746cc service nova] [instance: ebc3e3b2-037d-43ef-9426-caebe298ee3e] No waiting events found dispatching network-vif-unplugged-bd0a4e5b-69cd-4f5c-ae22-994e4d8f2682 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:36:13 user nova-compute[75234]: WARNING nova.compute.manager [req-921e087a-6724-4100-b2f8-f7051589a340 req-0a777e7c-8544-464b-a5a3-4ff6fde746cc service nova] [instance: ebc3e3b2-037d-43ef-9426-caebe298ee3e] Received unexpected event network-vif-unplugged-bd0a4e5b-69cd-4f5c-ae22-994e4d8f2682 for instance with vm_state rescued and task_state unrescuing.
Nov 29 03:36:13 user nova-compute[75234]: DEBUG nova.compute.manager [req-921e087a-6724-4100-b2f8-f7051589a340 req-0a777e7c-8544-464b-a5a3-4ff6fde746cc service nova] [instance: ebc3e3b2-037d-43ef-9426-caebe298ee3e] Received event network-vif-plugged-bd0a4e5b-69cd-4f5c-ae22-994e4d8f2682 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:36:13 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-921e087a-6724-4100-b2f8-f7051589a340 req-0a777e7c-8544-464b-a5a3-4ff6fde746cc service nova] Acquiring lock "ebc3e3b2-037d-43ef-9426-caebe298ee3e-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:36:13 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-921e087a-6724-4100-b2f8-f7051589a340 req-0a777e7c-8544-464b-a5a3-4ff6fde746cc service nova] Lock "ebc3e3b2-037d-43ef-9426-caebe298ee3e-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:36:13 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-921e087a-6724-4100-b2f8-f7051589a340 req-0a777e7c-8544-464b-a5a3-4ff6fde746cc service nova] Lock "ebc3e3b2-037d-43ef-9426-caebe298ee3e-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:36:13 user nova-compute[75234]: DEBUG nova.compute.manager [req-921e087a-6724-4100-b2f8-f7051589a340 req-0a777e7c-8544-464b-a5a3-4ff6fde746cc service nova] [instance: ebc3e3b2-037d-43ef-9426-caebe298ee3e] No waiting events found dispatching network-vif-plugged-bd0a4e5b-69cd-4f5c-ae22-994e4d8f2682 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:36:13 user nova-compute[75234]: WARNING nova.compute.manager [req-921e087a-6724-4100-b2f8-f7051589a340 req-0a777e7c-8544-464b-a5a3-4ff6fde746cc service nova] [instance: ebc3e3b2-037d-43ef-9426-caebe298ee3e] Received unexpected event network-vif-plugged-bd0a4e5b-69cd-4f5c-ae22-994e4d8f2682 for instance with vm_state rescued and task_state unrescuing.
Nov 29 03:36:13 user nova-compute[75234]: DEBUG nova.compute.manager [req-921e087a-6724-4100-b2f8-f7051589a340 req-0a777e7c-8544-464b-a5a3-4ff6fde746cc service nova] [instance: ebc3e3b2-037d-43ef-9426-caebe298ee3e] Received event network-vif-plugged-bd0a4e5b-69cd-4f5c-ae22-994e4d8f2682 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:36:13 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-921e087a-6724-4100-b2f8-f7051589a340 req-0a777e7c-8544-464b-a5a3-4ff6fde746cc service nova] Acquiring lock "ebc3e3b2-037d-43ef-9426-caebe298ee3e-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:36:13 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-921e087a-6724-4100-b2f8-f7051589a340 req-0a777e7c-8544-464b-a5a3-4ff6fde746cc service nova] Lock "ebc3e3b2-037d-43ef-9426-caebe298ee3e-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:36:13 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-921e087a-6724-4100-b2f8-f7051589a340 req-0a777e7c-8544-464b-a5a3-4ff6fde746cc service nova] Lock "ebc3e3b2-037d-43ef-9426-caebe298ee3e-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:36:13 user nova-compute[75234]: DEBUG nova.compute.manager [req-921e087a-6724-4100-b2f8-f7051589a340 req-0a777e7c-8544-464b-a5a3-4ff6fde746cc service nova] [instance: ebc3e3b2-037d-43ef-9426-caebe298ee3e] No waiting events found dispatching network-vif-plugged-bd0a4e5b-69cd-4f5c-ae22-994e4d8f2682 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:36:13 user nova-compute[75234]: WARNING nova.compute.manager [req-921e087a-6724-4100-b2f8-f7051589a340 req-0a777e7c-8544-464b-a5a3-4ff6fde746cc service nova] [instance: ebc3e3b2-037d-43ef-9426-caebe298ee3e] Received unexpected event network-vif-plugged-bd0a4e5b-69cd-4f5c-ae22-994e4d8f2682 for instance with vm_state rescued and task_state unrescuing.
Nov 29 03:36:13 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:36:14 user nova-compute[75234]: DEBUG nova.compute.manager [None req-2a4990cf-cdbc-48e1-a54a-971664a94ac6 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] [instance: ebc3e3b2-037d-43ef-9426-caebe298ee3e] Checking state {{(pid=75234) _get_power_state /opt/stack/nova/nova/compute/manager.py:1782}}
Nov 29 03:36:15 user nova-compute[75234]: DEBUG nova.compute.manager [req-52cd032f-a021-4096-9f25-2761e0aaa32c req-14c8d9a7-29f3-476b-8982-a5ebbee2b815 service nova] [instance: ebc3e3b2-037d-43ef-9426-caebe298ee3e] Received event network-vif-plugged-bd0a4e5b-69cd-4f5c-ae22-994e4d8f2682 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:36:15 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-52cd032f-a021-4096-9f25-2761e0aaa32c req-14c8d9a7-29f3-476b-8982-a5ebbee2b815 service nova] Acquiring lock "ebc3e3b2-037d-43ef-9426-caebe298ee3e-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:36:15 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-52cd032f-a021-4096-9f25-2761e0aaa32c req-14c8d9a7-29f3-476b-8982-a5ebbee2b815 service nova] Lock "ebc3e3b2-037d-43ef-9426-caebe298ee3e-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:36:15 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-52cd032f-a021-4096-9f25-2761e0aaa32c req-14c8d9a7-29f3-476b-8982-a5ebbee2b815 service nova] Lock "ebc3e3b2-037d-43ef-9426-caebe298ee3e-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:36:15 user nova-compute[75234]: DEBUG nova.compute.manager [req-52cd032f-a021-4096-9f25-2761e0aaa32c req-14c8d9a7-29f3-476b-8982-a5ebbee2b815 service nova] [instance: ebc3e3b2-037d-43ef-9426-caebe298ee3e] No waiting events found dispatching network-vif-plugged-bd0a4e5b-69cd-4f5c-ae22-994e4d8f2682 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:36:15 user nova-compute[75234]: WARNING nova.compute.manager [req-52cd032f-a021-4096-9f25-2761e0aaa32c req-14c8d9a7-29f3-476b-8982-a5ebbee2b815 service nova] [instance: ebc3e3b2-037d-43ef-9426-caebe298ee3e] Received unexpected event network-vif-plugged-bd0a4e5b-69cd-4f5c-ae22-994e4d8f2682 for instance with vm_state active and task_state None.
Nov 29 03:36:15 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:36:15 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:36:16 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:36:16 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:36:16 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=75234) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10541}}
Nov 29 03:36:17 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager.update_available_resource {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:36:17 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:36:17 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:36:17 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:36:17 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Auditing locally available compute resources for user (node: user) {{(pid=75234) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:907}}
Nov 29 03:36:17 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/65d48c93-b6da-4f51-9d5a-0676dda62a60/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:36:17 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/65d48c93-b6da-4f51-9d5a-0676dda62a60/disk --force-share --output=json" returned: 0 in 0.173s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:36:17 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/65d48c93-b6da-4f51-9d5a-0676dda62a60/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:36:17 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:36:17 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/65d48c93-b6da-4f51-9d5a-0676dda62a60/disk --force-share --output=json" returned: 0 in 0.176s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:36:18 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] skipping disk /dev/sde (vda) as it is a volume {{(pid=75234) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11213}}
Nov 29 03:36:18 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] skipping disk /dev/sdc (vda) as it is a volume {{(pid=75234) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11213}}
Nov 29 03:36:18 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] skipping disk /dev/sdd (vda) as it is a volume {{(pid=75234) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11213}}
Nov 29 03:36:18 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/abb01889-df4b-44a6-9827-87056d7ca125/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:36:18 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/abb01889-df4b-44a6-9827-87056d7ca125/disk --force-share --output=json" returned: 0 in 0.174s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:36:18 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/abb01889-df4b-44a6-9827-87056d7ca125/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:36:18 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/abb01889-df4b-44a6-9827-87056d7ca125/disk --force-share --output=json" returned: 0 in 0.171s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:36:18 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] skipping disk /dev/sdb (vda) as it is a volume {{(pid=75234) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11213}}
Nov 29 03:36:18 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:36:18 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:36:18 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Hypervisor/Node resource view: name=user free_ram=8576MB free_disk=24.92949676513672GB free_vcpus=6 pci_devices=[{"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}] {{(pid=75234) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1106}}
Nov 29 03:36:18 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:36:18 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:36:18 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance 65d48c93-b6da-4f51-9d5a-0676dda62a60 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 512, 'DISK_GB': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:36:18 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance d5096ca2-d005-45ac-a487-11a810446859 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 512}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:36:18 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance abb01889-df4b-44a6-9827-87056d7ca125 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 512, 'DISK_GB': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:36:18 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance ebc3e3b2-037d-43ef-9426-caebe298ee3e actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 512}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:36:18 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance 884589ec-b86c-4094-9a3f-54b641311d62 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 512}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:36:18 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance e3303cca-05a0-415a-b731-e1f12ad413f7 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 512}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:36:18 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Total usable vcpus: 12, total allocated vcpus: 6 {{(pid=75234) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1129}}
Nov 29 03:36:18 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Final resource view: name=user phys_ram=16011MB used_ram=3584MB phys_disk=40GB used_disk=2GB total_vcpus=12 used_vcpus=6 pci_stats=[] {{(pid=75234) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1138}}
Nov 29 03:36:19 user nova-compute[75234]: DEBUG nova.compute.provider_tree [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Inventory has not changed in ProviderTree for provider: 4d291220-7c37-4839-a0e3-b23c66502b74 {{(pid=75234) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Nov 29 03:36:19 user nova-compute[75234]: DEBUG nova.scheduler.client.report [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Inventory has not changed for provider 4d291220-7c37-4839-a0e3-b23c66502b74 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=75234) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Nov 29 03:36:19 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Compute_service record updated for user:user {{(pid=75234) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:1067}}
Nov 29 03:36:19 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.362s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:36:20 user nova-compute[75234]: DEBUG nova.compute.manager [req-f1bf2c19-4b10-47af-b675-1f4f26b9bc8c req-3d558f0e-f95b-445b-9573-59145f8f3935 service nova] [instance: e3303cca-05a0-415a-b731-e1f12ad413f7] Received event network-changed-fd43dff4-5bd3-4791-a078-45bc982d3bb7 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:36:20 user nova-compute[75234]: DEBUG nova.compute.manager [req-f1bf2c19-4b10-47af-b675-1f4f26b9bc8c req-3d558f0e-f95b-445b-9573-59145f8f3935 service nova] [instance: e3303cca-05a0-415a-b731-e1f12ad413f7] Refreshing instance network info cache due to event network-changed-fd43dff4-5bd3-4791-a078-45bc982d3bb7. {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11117}}
Nov 29 03:36:20 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-f1bf2c19-4b10-47af-b675-1f4f26b9bc8c req-3d558f0e-f95b-445b-9573-59145f8f3935 service nova] Acquiring lock "refresh_cache-e3303cca-05a0-415a-b731-e1f12ad413f7" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:36:20 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-f1bf2c19-4b10-47af-b675-1f4f26b9bc8c req-3d558f0e-f95b-445b-9573-59145f8f3935 service nova] Acquired lock "refresh_cache-e3303cca-05a0-415a-b731-e1f12ad413f7" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:36:20 user nova-compute[75234]: DEBUG nova.network.neutron [req-f1bf2c19-4b10-47af-b675-1f4f26b9bc8c req-3d558f0e-f95b-445b-9573-59145f8f3935 service nova] [instance: e3303cca-05a0-415a-b731-e1f12ad413f7] Refreshing network info cache for port fd43dff4-5bd3-4791-a078-45bc982d3bb7 {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2006}}
Nov 29 03:36:21 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:36:21 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-6b318b59-6c73-4d74-ab9b-7dfa73716641 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Acquiring lock "e3303cca-05a0-415a-b731-e1f12ad413f7" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:36:21 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-6b318b59-6c73-4d74-ab9b-7dfa73716641 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Lock "e3303cca-05a0-415a-b731-e1f12ad413f7" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:36:21 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-6b318b59-6c73-4d74-ab9b-7dfa73716641 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Acquiring lock "e3303cca-05a0-415a-b731-e1f12ad413f7-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:36:21 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-6b318b59-6c73-4d74-ab9b-7dfa73716641 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Lock "e3303cca-05a0-415a-b731-e1f12ad413f7-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:36:21 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-6b318b59-6c73-4d74-ab9b-7dfa73716641 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Lock "e3303cca-05a0-415a-b731-e1f12ad413f7-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:36:21 user nova-compute[75234]: INFO nova.compute.manager [None req-6b318b59-6c73-4d74-ab9b-7dfa73716641 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: e3303cca-05a0-415a-b731-e1f12ad413f7] Terminating instance
Nov 29 03:36:21 user nova-compute[75234]: DEBUG nova.compute.manager [None req-6b318b59-6c73-4d74-ab9b-7dfa73716641 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: e3303cca-05a0-415a-b731-e1f12ad413f7] Start destroying the instance on the hypervisor. {{(pid=75234) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3140}}
Nov 29 03:36:21 user nova-compute[75234]: DEBUG nova.network.neutron [req-f1bf2c19-4b10-47af-b675-1f4f26b9bc8c req-3d558f0e-f95b-445b-9573-59145f8f3935 service nova] [instance: e3303cca-05a0-415a-b731-e1f12ad413f7] Updated VIF entry in instance network info cache for port fd43dff4-5bd3-4791-a078-45bc982d3bb7. {{(pid=75234) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3481}}
Nov 29 03:36:21 user nova-compute[75234]: DEBUG nova.network.neutron [req-f1bf2c19-4b10-47af-b675-1f4f26b9bc8c req-3d558f0e-f95b-445b-9573-59145f8f3935 service nova] [instance: e3303cca-05a0-415a-b731-e1f12ad413f7] Updating instance_info_cache with network_info: [{"id": "fd43dff4-5bd3-4791-a078-45bc982d3bb7", "address": "fa:16:3e:92:4c:3b", "network": {"id": "a3f397d3-fd75-4a6e-a815-cfcedcca15fb", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-678570085-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.51", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "81a265d3c15b4e9d826b35ddbd0ff443", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapfd43dff4-5b", "ovs_interfaceid": "fd43dff4-5bd3-4791-a078-45bc982d3bb7", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:36:21 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-f1bf2c19-4b10-47af-b675-1f4f26b9bc8c req-3d558f0e-f95b-445b-9573-59145f8f3935 service nova] Releasing lock "refresh_cache-e3303cca-05a0-415a-b731-e1f12ad413f7" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:36:21 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:36:21 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:36:21 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:36:21 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:36:21 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:36:21 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:36:21 user nova-compute[75234]: INFO nova.virt.libvirt.driver [-] [instance: e3303cca-05a0-415a-b731-e1f12ad413f7] Instance destroyed successfully.
Nov 29 03:36:21 user nova-compute[75234]: DEBUG nova.objects.instance [None req-6b318b59-6c73-4d74-ab9b-7dfa73716641 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Lazy-loading 'resources' on Instance uuid e3303cca-05a0-415a-b731-e1f12ad413f7 {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:36:21 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-6b318b59-6c73-4d74-ab9b-7dfa73716641 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-11-29T03:34:28Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-TestVolumeBootPattern-image-snapshot-server-1990224224',display_name='tempest-TestVolumeBootPattern-image-snapshot-server-1990224224',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-testvolumebootpattern-image-snapshot-server-1990224224',id=49,image_ref='a16128e1-937d-4f3d-ad48-bf3fd942beb1',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBO96UoQAplU5V6UAUXfmFq4Xoqecf74KJ64MbiF1ieDnoRV2pPDxFGj5ZEYbpIW/U3oWl7DggAXx8KafBFOSxpIC3ULGrK/e+yz95ehE2qDjf55PlvhVEaakUzw4pg1y9g==',key_name='tempest-keypair-1541124831',keypairs=<?>,launch_index=0,launched_at=2023-11-29T03:34:48Z,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='81a265d3c15b4e9d826b35ddbd0ff443',ramdisk_id='',reservation_id='r-52vup8wc',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='',image_bdm_v2='True',image_boot_roles='reader,member',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_project_name='tempest-TestVolumeBootPattern-1244492388',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',image_owner_user_name='tempest-TestVolumeBootPattern-1244492388-project-member',image_root_device_name='/dev/vda',owner_project_name='tempest-TestVolumeBootPattern-1244492388',owner_user_name='tempest-TestVolumeBootPattern-1244492388-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-11-29T03:34:49Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='7d1f1f11a9aa4c1db8627413eb963fa4',uuid=e3303cca-05a0-415a-b731-e1f12ad413f7,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "fd43dff4-5bd3-4791-a078-45bc982d3bb7", "address": "fa:16:3e:92:4c:3b", "network": {"id": "a3f397d3-fd75-4a6e-a815-cfcedcca15fb", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-678570085-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "81a265d3c15b4e9d826b35ddbd0ff443", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapfd43dff4-5b", "ovs_interfaceid": "fd43dff4-5bd3-4791-a078-45bc982d3bb7", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Nov 29 03:36:21 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-6b318b59-6c73-4d74-ab9b-7dfa73716641 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Converting VIF {"id": "fd43dff4-5bd3-4791-a078-45bc982d3bb7", "address": "fa:16:3e:92:4c:3b", "network": {"id": "a3f397d3-fd75-4a6e-a815-cfcedcca15fb", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-678570085-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "81a265d3c15b4e9d826b35ddbd0ff443", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapfd43dff4-5b", "ovs_interfaceid": "fd43dff4-5bd3-4791-a078-45bc982d3bb7", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:36:21 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-6b318b59-6c73-4d74-ab9b-7dfa73716641 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:92:4c:3b,bridge_name='br-int',has_traffic_filtering=True,id=fd43dff4-5bd3-4791-a078-45bc982d3bb7,network=Network(a3f397d3-fd75-4a6e-a815-cfcedcca15fb),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapfd43dff4-5b') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:36:21 user nova-compute[75234]: DEBUG os_vif [None req-6b318b59-6c73-4d74-ab9b-7dfa73716641 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Unplugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:92:4c:3b,bridge_name='br-int',has_traffic_filtering=True,id=fd43dff4-5bd3-4791-a078-45bc982d3bb7,network=Network(a3f397d3-fd75-4a6e-a815-cfcedcca15fb),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapfd43dff4-5b') {{(pid=75234) unplug /opt/stack/data/venv/lib/python3.10/site-packages/os_vif/__init__.py:109}}
Nov 29 03:36:21 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:36:21 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tapfd43dff4-5b, bridge=br-int, if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:36:21 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:36:21 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:36:21 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:36:21 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbDestroyCommand(_result=None, table=QoS, record=c744b157-135d-4f8a-a727-1411604337c2) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:36:21 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:36:21 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:36:21 user nova-compute[75234]: INFO os_vif [None req-6b318b59-6c73-4d74-ab9b-7dfa73716641 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Successfully unplugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:92:4c:3b,bridge_name='br-int',has_traffic_filtering=True,id=fd43dff4-5bd3-4791-a078-45bc982d3bb7,network=Network(a3f397d3-fd75-4a6e-a815-cfcedcca15fb),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapfd43dff4-5b')
Nov 29 03:36:21 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-6b318b59-6c73-4d74-ab9b-7dfa73716641 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: e3303cca-05a0-415a-b731-e1f12ad413f7] calling os-brick to detach iSCSI Volume {{(pid=75234) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Nov 29 03:36:21 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-6b318b59-6c73-4d74-ab9b-7dfa73716641 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f8a5050c9a0>, {'target_iqn': 'iqn.2010-10.org.openstack:dd89ea2e-2a9f-4511-9b6f-c5177ff21182', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'giwfJzqL', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdb'}, None), 'kwargs': True}}" {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:176}}
Nov 29 03:36:21 user nova-compute[75234]: WARNING os_brick.initiator.connectors.base [None req-6b318b59-6c73-4d74-ab9b-7dfa73716641 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Nov 29 03:36:21 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-6b318b59-6c73-4d74-ab9b-7dfa73716641 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:68}}
Nov 29 03:36:21 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-6b318b59-6c73-4d74-ab9b-7dfa73716641 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:73}}
Nov 29 03:36:21 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:36:21 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.023s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:36:21 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[f1cf983f-7d74-48f7-a821-efdd40f5e259]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:a3d30ae4-ca6d-4e06-8fc9-caf4d91ef900\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:dd89ea2e-2a9f-4511-9b6f-c5177ff21182\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:257d7345-5b29-4bc1-8844-e211864c06c3\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:36:21 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-6b318b59-6c73-4d74-ab9b-7dfa73716641 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS:
Nov 29 03:36:21 user nova-compute[75234]: No targets found.
Nov 29 03:36:21 user nova-compute[75234]: iSNS:
Nov 29 03:36:21 user nova-compute[75234]: No targets found.
Nov 29 03:36:21 user nova-compute[75234]: STATIC:
Nov 29 03:36:21 user nova-compute[75234]: Target: iqn.2010-10.org.openstack:a3d30ae4-ca6d-4e06-8fc9-caf4d91ef900
Nov 29 03:36:21 user nova-compute[75234]:         Portal: 172.16.0.220:3260,-1
Nov 29 03:36:21 user nova-compute[75234]:                 Iface Name: default
Nov 29 03:36:21 user nova-compute[75234]: Target: iqn.2010-10.org.openstack:dd89ea2e-2a9f-4511-9b6f-c5177ff21182
Nov 29 03:36:21 user nova-compute[75234]:         Portal: 172.16.0.220:3260,-1
Nov 29 03:36:21 user nova-compute[75234]:                 Iface Name: default
Nov 29 03:36:21 user nova-compute[75234]: Target: iqn.2010-10.org.openstack:257d7345-5b29-4bc1-8844-e211864c06c3
Nov 29 03:36:21 user nova-compute[75234]:         Portal: 172.16.0.220:3260,-1
Nov 29 03:36:21 user nova-compute[75234]:                 Iface Name: default
Nov 29 03:36:21 user nova-compute[75234]: Target: iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0
Nov 29 03:36:21 user nova-compute[75234]:         Portal: 172.16.0.220:3260,-1
Nov 29 03:36:21 user nova-compute[75234]:                 Iface Name: default
Nov 29 03:36:21 user nova-compute[75234]: FIRMWARE:
Nov 29 03:36:21 user nova-compute[75234]: No targets found.
Nov 29 03:36:21 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:36:21 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-6b318b59-6c73-4d74-ab9b-7dfa73716641 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Regex to get portals from discoverydb: ^SENDTARGETS:
Nov 29 03:36:21 user nova-compute[75234]: .*?^DiscoveryAddress: 172.16.0.220,3260.*?
Nov 29 03:36:21 user nova-compute[75234]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=75234) _get_discoverydb_portals /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:382}}
Nov 29 03:36:21 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-6b318b59-6c73-4d74-ab9b-7dfa73716641 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:dd89ea2e-2a9f-4511-9b6f-c5177ff21182', 0)] {{(pid=75234) _get_connection_devices /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:830}}
Nov 29 03:36:22 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:36:22 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.018s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:36:22 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[4ee1bdab-31db-4d69-994b-9b3f6e2ac705]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:257d7345-5b29-4bc1-8844-e211864c06c3\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:a3d30ae4-ca6d-4e06-8fc9-caf4d91ef900\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:dd89ea2e-2a9f-4511-9b6f-c5177ff21182\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:36:22 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:36:22 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.023s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:36:22 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[2db4ce83-bc44-4f7f-ae96-cfa696e60dae]: (4, ('tcp: [42] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0 (non-flash)\ntcp: [54] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a3d30ae4-ca6d-4e06-8fc9-caf4d91ef900 (non-flash)\ntcp: [55] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:257d7345-5b29-4bc1-8844-e211864c06c3 (non-flash)\ntcp: [57] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:dd89ea2e-2a9f-4511-9b6f-c5177ff21182 (non-flash)\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:36:22 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-6b318b59-6c73-4d74-ab9b-7dfa73716641 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [42] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0 (non-flash)
Nov 29 03:36:22 user nova-compute[75234]: tcp: [54] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a3d30ae4-ca6d-4e06-8fc9-caf4d91ef900 (non-flash)
Nov 29 03:36:22 user nova-compute[75234]: tcp: [55] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:257d7345-5b29-4bc1-8844-e211864c06c3 (non-flash)
Nov 29 03:36:22 user nova-compute[75234]: tcp: [57] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:dd89ea2e-2a9f-4511-9b6f-c5177ff21182 (non-flash)
Nov 29 03:36:22 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:36:22 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-6b318b59-6c73-4d74-ab9b-7dfa73716641 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] iscsi session list stdout=tcp: [42] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0 (non-flash)
Nov 29 03:36:22 user nova-compute[75234]: tcp: [54] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a3d30ae4-ca6d-4e06-8fc9-caf4d91ef900 (non-flash)
Nov 29 03:36:22 user nova-compute[75234]: tcp: [55] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:257d7345-5b29-4bc1-8844-e211864c06c3 (non-flash)
Nov 29 03:36:22 user nova-compute[75234]: tcp: [57] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:dd89ea2e-2a9f-4511-9b6f-c5177ff21182 (non-flash)
Nov 29 03:36:22 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsi_session /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Nov 29 03:36:22 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-6b318b59-6c73-4d74-ab9b-7dfa73716641 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7f8a2c659090>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:dd89ea2e-2a9f-4511-9b6f-c5177ff21182'): ({'sdb'}, set())}) {{(pid=75234) _get_connection_devices /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:863}}
Nov 29 03:36:22 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-6b318b59-6c73-4d74-ab9b-7dfa73716641 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Removing single pathed devices sdb {{(pid=75234) remove_connection /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:374}}
Nov 29 03:36:22 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:36:22 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.018s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:36:22 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[253a4f8a-b782-49d5-b294-a66bbf9643d6]: (4, ('path checker states:\nup                  4\n\npaths: 0\nbusy: False\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:36:22 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdb {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:36:22 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdb" returned: 0 in 0.018s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:36:22 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[00ca5ce6-4975-4eb2-82ec-2580cf73c6d8]: (4, ('ok\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:36:22 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-6b318b59-6c73-4d74-ab9b-7dfa73716641 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Flushing IO for device /dev/sdb {{(pid=75234) flush_device_io /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:434}}
Nov 29 03:36:22 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdb {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:36:22 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdb" returned: 0 in 0.032s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:36:22 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[e8dec3df-6c39-48a5-8009-e8445cf73bd9]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:36:22 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-6b318b59-6c73-4d74-ab9b-7dfa73716641 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Remove SCSI device /dev/sdb with /sys/block/sdb/device/delete {{(pid=75234) remove_scsi_device /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:148}}
Nov 29 03:36:22 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdb/device/delete {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:36:22 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdb/device/delete" returned: 0 in 0.039s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:36:22 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[3682433c-5b45-4a4c-889e-0965b5b299f1]: (4, ('1', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:36:22 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-6b318b59-6c73-4d74-ab9b-7dfa73716641 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Checking to see if SCSI volumes sdb have been removed. {{(pid=75234) wait_for_volumes_removal /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:156}}
Nov 29 03:36:22 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-6b318b59-6c73-4d74-ab9b-7dfa73716641 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] SCSI volumes sdb have been removed. {{(pid=75234) wait_for_volumes_removal /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:166}}
Nov 29 03:36:22 user nova-compute[75234]: DEBUG os_brick.privileged.rootwrap [-] Unlinking ('/dev/disk/by-id/scsi-1SCST_BIOa53fe80c-4d96d423f786fada', '/dev/disk/by-id/scsi-SSCST_BIO_Storage_a53fe80c', '/dev/disk/by-id/scsi-23464393664343233') {{(pid=86324) unlink_root /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/privileged/rootwrap.py:219}}
Nov 29 03:36:22 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[83319685-4727-4826-8271-3a892194dac7]: (4, None) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:36:22 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-6b318b59-6c73-4d74-ab9b-7dfa73716641 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:dd89ea2e-2a9f-4511-9b6f-c5177ff21182')] {{(pid=75234) _disconnect_connection /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Nov 29 03:36:22 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:dd89ea2e-2a9f-4511-9b6f-c5177ff21182 -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:36:22 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:dd89ea2e-2a9f-4511-9b6f-c5177ff21182 -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.018s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:36:22 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[dfb03e58-b1e0-4d48-ade4-d99447f6e3fe]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:36:22 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-6b318b59-6c73-4d74-ab9b-7dfa73716641 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:36:22 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:dd89ea2e-2a9f-4511-9b6f-c5177ff21182 -p 172.16.0.220:3260 --logout {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:36:22 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:dd89ea2e-2a9f-4511-9b6f-c5177ff21182 -p 172.16.0.220:3260 --logout" returned: 0 in 0.056s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:36:22 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[df813e4b-6ce4-43e8-aa9d-8b6b9b268176]: (4, ('Logging out of session [sid: 57, target: iqn.2010-10.org.openstack:dd89ea2e-2a9f-4511-9b6f-c5177ff21182, portal: 172.16.0.220,3260]\nLogout of [sid: 57, target: iqn.2010-10.org.openstack:dd89ea2e-2a9f-4511-9b6f-c5177ff21182, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:36:22 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-6b318b59-6c73-4d74-ab9b-7dfa73716641 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 57, target: iqn.2010-10.org.openstack:dd89ea2e-2a9f-4511-9b6f-c5177ff21182, portal: 172.16.0.220,3260]
Nov 29 03:36:22 user nova-compute[75234]: Logout of [sid: 57, target: iqn.2010-10.org.openstack:dd89ea2e-2a9f-4511-9b6f-c5177ff21182, portal: 172.16.0.220,3260] successful.
Nov 29 03:36:22 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:36:22 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:dd89ea2e-2a9f-4511-9b6f-c5177ff21182 -p 172.16.0.220:3260 --op delete {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:36:22 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:dd89ea2e-2a9f-4511-9b6f-c5177ff21182 -p 172.16.0.220:3260 --op delete" returned: 0 in 0.024s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:36:22 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[465d2dda-12a8-4ad5-9321-89ff6804d824]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:36:22 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-6b318b59-6c73-4d74-ab9b-7dfa73716641 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:36:22 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-6b318b59-6c73-4d74-ab9b-7dfa73716641 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.326s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:87}}
Nov 29 03:36:22 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-6b318b59-6c73-4d74-ab9b-7dfa73716641 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] <== disconnect_volume: return (327ms) None {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:203}}
Nov 29 03:36:22 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-6b318b59-6c73-4d74-ab9b-7dfa73716641 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: e3303cca-05a0-415a-b731-e1f12ad413f7] Disconnected iSCSI Volume {{(pid=75234) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:79}}
Nov 29 03:36:22 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-6b318b59-6c73-4d74-ab9b-7dfa73716641 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: e3303cca-05a0-415a-b731-e1f12ad413f7] Deleting instance files /opt/stack/data/nova/instances/e3303cca-05a0-415a-b731-e1f12ad413f7_del
Nov 29 03:36:22 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-6b318b59-6c73-4d74-ab9b-7dfa73716641 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: e3303cca-05a0-415a-b731-e1f12ad413f7] Deletion of /opt/stack/data/nova/instances/e3303cca-05a0-415a-b731-e1f12ad413f7_del complete
Nov 29 03:36:22 user nova-compute[75234]: INFO nova.compute.manager [None req-6b318b59-6c73-4d74-ab9b-7dfa73716641 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: e3303cca-05a0-415a-b731-e1f12ad413f7] Took 1.19 seconds to destroy the instance on the hypervisor.
Nov 29 03:36:22 user nova-compute[75234]: DEBUG oslo.service.loopingcall [None req-6b318b59-6c73-4d74-ab9b-7dfa73716641 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=75234) func /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/loopingcall.py:435}}
Nov 29 03:36:22 user nova-compute[75234]: DEBUG nova.compute.manager [-] [instance: e3303cca-05a0-415a-b731-e1f12ad413f7] Deallocating network for instance {{(pid=75234) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2279}}
Nov 29 03:36:22 user nova-compute[75234]: DEBUG nova.network.neutron [-] [instance: e3303cca-05a0-415a-b731-e1f12ad413f7] deallocate_for_instance() {{(pid=75234) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1802}}
Nov 29 03:36:22 user nova-compute[75234]: DEBUG nova.compute.manager [req-364a52fd-91ba-4170-8daa-a8c5d27d2116 req-72581b06-d341-4bcc-bd4f-48cd203dbf98 service nova] [instance: e3303cca-05a0-415a-b731-e1f12ad413f7] Received event network-vif-unplugged-fd43dff4-5bd3-4791-a078-45bc982d3bb7 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:36:22 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-364a52fd-91ba-4170-8daa-a8c5d27d2116 req-72581b06-d341-4bcc-bd4f-48cd203dbf98 service nova] Acquiring lock "e3303cca-05a0-415a-b731-e1f12ad413f7-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:36:22 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-364a52fd-91ba-4170-8daa-a8c5d27d2116 req-72581b06-d341-4bcc-bd4f-48cd203dbf98 service nova] Lock "e3303cca-05a0-415a-b731-e1f12ad413f7-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:36:22 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-364a52fd-91ba-4170-8daa-a8c5d27d2116 req-72581b06-d341-4bcc-bd4f-48cd203dbf98 service nova] Lock "e3303cca-05a0-415a-b731-e1f12ad413f7-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:36:22 user nova-compute[75234]: DEBUG nova.compute.manager [req-364a52fd-91ba-4170-8daa-a8c5d27d2116 req-72581b06-d341-4bcc-bd4f-48cd203dbf98 service nova] [instance: e3303cca-05a0-415a-b731-e1f12ad413f7] No waiting events found dispatching network-vif-unplugged-fd43dff4-5bd3-4791-a078-45bc982d3bb7 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:36:22 user nova-compute[75234]: DEBUG nova.compute.manager [req-364a52fd-91ba-4170-8daa-a8c5d27d2116 req-72581b06-d341-4bcc-bd4f-48cd203dbf98 service nova] [instance: e3303cca-05a0-415a-b731-e1f12ad413f7] Received event network-vif-unplugged-fd43dff4-5bd3-4791-a078-45bc982d3bb7 for instance with task_state deleting. {{(pid=75234) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10890}}
Nov 29 03:36:22 user nova-compute[75234]: DEBUG nova.compute.manager [req-364a52fd-91ba-4170-8daa-a8c5d27d2116 req-72581b06-d341-4bcc-bd4f-48cd203dbf98 service nova] [instance: e3303cca-05a0-415a-b731-e1f12ad413f7] Received event network-vif-plugged-fd43dff4-5bd3-4791-a078-45bc982d3bb7 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:36:22 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-364a52fd-91ba-4170-8daa-a8c5d27d2116 req-72581b06-d341-4bcc-bd4f-48cd203dbf98 service nova] Acquiring lock "e3303cca-05a0-415a-b731-e1f12ad413f7-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:36:22 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-364a52fd-91ba-4170-8daa-a8c5d27d2116 req-72581b06-d341-4bcc-bd4f-48cd203dbf98 service nova] Lock "e3303cca-05a0-415a-b731-e1f12ad413f7-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:36:22 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-364a52fd-91ba-4170-8daa-a8c5d27d2116 req-72581b06-d341-4bcc-bd4f-48cd203dbf98 service nova] Lock "e3303cca-05a0-415a-b731-e1f12ad413f7-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:36:22 user nova-compute[75234]: DEBUG nova.compute.manager [req-364a52fd-91ba-4170-8daa-a8c5d27d2116 req-72581b06-d341-4bcc-bd4f-48cd203dbf98 service nova] [instance: e3303cca-05a0-415a-b731-e1f12ad413f7] No waiting events found dispatching network-vif-plugged-fd43dff4-5bd3-4791-a078-45bc982d3bb7 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:36:22 user nova-compute[75234]: WARNING nova.compute.manager [req-364a52fd-91ba-4170-8daa-a8c5d27d2116 req-72581b06-d341-4bcc-bd4f-48cd203dbf98 service nova] [instance: e3303cca-05a0-415a-b731-e1f12ad413f7] Received unexpected event network-vif-plugged-fd43dff4-5bd3-4791-a078-45bc982d3bb7 for instance with vm_state active and task_state deleting.
Nov 29 03:36:22 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:36:23 user nova-compute[75234]: DEBUG nova.network.neutron [-] [instance: e3303cca-05a0-415a-b731-e1f12ad413f7] Updating instance_info_cache with network_info: [] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:36:23 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:36:23 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Starting heal instance info cache {{(pid=75234) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9922}}
Nov 29 03:36:23 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Rebuilding the list of instances to heal {{(pid=75234) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9926}}
Nov 29 03:36:23 user nova-compute[75234]: INFO nova.compute.manager [-] [instance: e3303cca-05a0-415a-b731-e1f12ad413f7] Took 0.79 seconds to deallocate network for instance.
Nov 29 03:36:23 user nova-compute[75234]: DEBUG nova.compute.manager [req-7827073d-45be-41c7-8ab4-c76b880a84cc req-d04eeae3-84fd-4de7-8ebe-75e4c1080a4d service nova] [instance: e3303cca-05a0-415a-b731-e1f12ad413f7] Received event network-vif-deleted-fd43dff4-5bd3-4791-a078-45bc982d3bb7 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:36:23 user nova-compute[75234]: INFO nova.compute.manager [req-7827073d-45be-41c7-8ab4-c76b880a84cc req-d04eeae3-84fd-4de7-8ebe-75e4c1080a4d service nova] [instance: e3303cca-05a0-415a-b731-e1f12ad413f7] Neutron deleted interface fd43dff4-5bd3-4791-a078-45bc982d3bb7; detaching it from the instance and deleting it from the info cache
Nov 29 03:36:23 user nova-compute[75234]: DEBUG nova.network.neutron [req-7827073d-45be-41c7-8ab4-c76b880a84cc req-d04eeae3-84fd-4de7-8ebe-75e4c1080a4d service nova] [instance: e3303cca-05a0-415a-b731-e1f12ad413f7] Updating instance_info_cache with network_info: [] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:36:23 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] [instance: e3303cca-05a0-415a-b731-e1f12ad413f7] Skipping network cache update for instance because it is being deleted. {{(pid=75234) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9939}}
Nov 29 03:36:23 user nova-compute[75234]: DEBUG nova.compute.manager [req-7827073d-45be-41c7-8ab4-c76b880a84cc req-d04eeae3-84fd-4de7-8ebe-75e4c1080a4d service nova] [instance: e3303cca-05a0-415a-b731-e1f12ad413f7] Detach interface failed, port_id=fd43dff4-5bd3-4791-a078-45bc982d3bb7, reason: Instance e3303cca-05a0-415a-b731-e1f12ad413f7 could not be found. {{(pid=75234) _process_instance_vif_deleted_event /opt/stack/nova/nova/compute/manager.py:10946}}
Nov 29 03:36:23 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Acquiring lock "refresh_cache-65d48c93-b6da-4f51-9d5a-0676dda62a60" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:36:23 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Acquired lock "refresh_cache-65d48c93-b6da-4f51-9d5a-0676dda62a60" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:36:23 user nova-compute[75234]: DEBUG nova.network.neutron [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] [instance: 65d48c93-b6da-4f51-9d5a-0676dda62a60] Forcefully refreshing network info cache for instance {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2003}}
Nov 29 03:36:23 user nova-compute[75234]: DEBUG nova.objects.instance [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lazy-loading 'info_cache' on Instance uuid 65d48c93-b6da-4f51-9d5a-0676dda62a60 {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:36:23 user nova-compute[75234]: DEBUG nova.network.neutron [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] [instance: 65d48c93-b6da-4f51-9d5a-0676dda62a60] Updating instance_info_cache with network_info: [{"id": "967835a2-4b55-4ae6-b071-91233af412fd", "address": "fa:16:3e:ee:3c:e2", "network": {"id": "4d086e08-7798-4cfe-9365-86be6f0a52d2", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-305624008-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "db00bcd0c82c44f1bda739e9857f6e84", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap967835a2-4b", "ovs_interfaceid": "967835a2-4b55-4ae6-b071-91233af412fd", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:36:23 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Releasing lock "refresh_cache-65d48c93-b6da-4f51-9d5a-0676dda62a60" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:36:23 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] [instance: 65d48c93-b6da-4f51-9d5a-0676dda62a60] Updated the network info_cache for instance {{(pid=75234) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9993}}
Nov 29 03:36:23 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:36:23 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:36:23 user nova-compute[75234]: INFO nova.compute.manager [None req-6b318b59-6c73-4d74-ab9b-7dfa73716641 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: e3303cca-05a0-415a-b731-e1f12ad413f7] Took 0.78 seconds to detach 1 volumes for instance.
Nov 29 03:36:23 user nova-compute[75234]: DEBUG nova.compute.manager [None req-6b318b59-6c73-4d74-ab9b-7dfa73716641 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: e3303cca-05a0-415a-b731-e1f12ad413f7] Deleting volume: dd89ea2e-2a9f-4511-9b6f-c5177ff21182 {{(pid=75234) _cleanup_volumes /opt/stack/nova/nova/compute/manager.py:3237}}
Nov 29 03:36:24 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-6b318b59-6c73-4d74-ab9b-7dfa73716641 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:36:24 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-6b318b59-6c73-4d74-ab9b-7dfa73716641 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:36:24 user nova-compute[75234]: DEBUG nova.compute.provider_tree [None req-6b318b59-6c73-4d74-ab9b-7dfa73716641 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Inventory has not changed in ProviderTree for provider: 4d291220-7c37-4839-a0e3-b23c66502b74 {{(pid=75234) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Nov 29 03:36:24 user nova-compute[75234]: DEBUG nova.scheduler.client.report [None req-6b318b59-6c73-4d74-ab9b-7dfa73716641 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Inventory has not changed for provider 4d291220-7c37-4839-a0e3-b23c66502b74 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=75234) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Nov 29 03:36:24 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-6b318b59-6c73-4d74-ab9b-7dfa73716641 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.224s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:36:24 user nova-compute[75234]: INFO nova.scheduler.client.report [None req-6b318b59-6c73-4d74-ab9b-7dfa73716641 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Deleted allocations for instance e3303cca-05a0-415a-b731-e1f12ad413f7
Nov 29 03:36:24 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-6b318b59-6c73-4d74-ab9b-7dfa73716641 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Lock "e3303cca-05a0-415a-b731-e1f12ad413f7" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 3.223s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:36:25 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:36:26 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:36:27 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:36:29 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._sync_scheduler_instance_info {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:36:31 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:36:32 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:36:36 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:36:37 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:36:41 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:36:42 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:36:46 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:36:47 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:36:51 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:36:56 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:37:01 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:37:06 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:37:11 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:37:14 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:37:14 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-3aa5c594-23ff-4cac-a4ab-4ce14d9728f7 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Acquiring lock "884589ec-b86c-4094-9a3f-54b641311d62" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:37:14 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-3aa5c594-23ff-4cac-a4ab-4ce14d9728f7 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Lock "884589ec-b86c-4094-9a3f-54b641311d62" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:37:14 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-3aa5c594-23ff-4cac-a4ab-4ce14d9728f7 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Acquiring lock "884589ec-b86c-4094-9a3f-54b641311d62-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:37:14 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-3aa5c594-23ff-4cac-a4ab-4ce14d9728f7 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Lock "884589ec-b86c-4094-9a3f-54b641311d62-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:37:14 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-3aa5c594-23ff-4cac-a4ab-4ce14d9728f7 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Lock "884589ec-b86c-4094-9a3f-54b641311d62-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:37:14 user nova-compute[75234]: INFO nova.compute.manager [None req-3aa5c594-23ff-4cac-a4ab-4ce14d9728f7 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: 884589ec-b86c-4094-9a3f-54b641311d62] Terminating instance
Nov 29 03:37:14 user nova-compute[75234]: DEBUG nova.compute.manager [None req-3aa5c594-23ff-4cac-a4ab-4ce14d9728f7 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: 884589ec-b86c-4094-9a3f-54b641311d62] Start destroying the instance on the hypervisor. {{(pid=75234) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3140}}
Nov 29 03:37:14 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:37:14 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:37:14 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:37:14 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:37:15 user nova-compute[75234]: DEBUG nova.compute.manager [req-8bd7312d-4fbb-4433-9888-a3367ee2f6f6 req-0d2a1887-c3fb-4076-8788-7340d8f68b11 service nova] [instance: 884589ec-b86c-4094-9a3f-54b641311d62] Received event network-vif-unplugged-1deddce5-7279-44f9-9f1e-cd9ffddc5d6c {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:37:15 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-8bd7312d-4fbb-4433-9888-a3367ee2f6f6 req-0d2a1887-c3fb-4076-8788-7340d8f68b11 service nova] Acquiring lock "884589ec-b86c-4094-9a3f-54b641311d62-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:37:15 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-8bd7312d-4fbb-4433-9888-a3367ee2f6f6 req-0d2a1887-c3fb-4076-8788-7340d8f68b11 service nova] Lock "884589ec-b86c-4094-9a3f-54b641311d62-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:37:15 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-8bd7312d-4fbb-4433-9888-a3367ee2f6f6 req-0d2a1887-c3fb-4076-8788-7340d8f68b11 service nova] Lock "884589ec-b86c-4094-9a3f-54b641311d62-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:37:15 user nova-compute[75234]: DEBUG nova.compute.manager [req-8bd7312d-4fbb-4433-9888-a3367ee2f6f6 req-0d2a1887-c3fb-4076-8788-7340d8f68b11 service nova] [instance: 884589ec-b86c-4094-9a3f-54b641311d62] No waiting events found dispatching network-vif-unplugged-1deddce5-7279-44f9-9f1e-cd9ffddc5d6c {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:37:15 user nova-compute[75234]: DEBUG nova.compute.manager [req-8bd7312d-4fbb-4433-9888-a3367ee2f6f6 req-0d2a1887-c3fb-4076-8788-7340d8f68b11 service nova] [instance: 884589ec-b86c-4094-9a3f-54b641311d62] Received event network-vif-unplugged-1deddce5-7279-44f9-9f1e-cd9ffddc5d6c for instance with task_state deleting. {{(pid=75234) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10890}}
Nov 29 03:37:15 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:37:15 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:37:15 user nova-compute[75234]: INFO nova.virt.libvirt.driver [-] [instance: 884589ec-b86c-4094-9a3f-54b641311d62] Instance destroyed successfully.
Nov 29 03:37:15 user nova-compute[75234]: DEBUG nova.objects.instance [None req-3aa5c594-23ff-4cac-a4ab-4ce14d9728f7 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Lazy-loading 'resources' on Instance uuid 884589ec-b86c-4094-9a3f-54b641311d62 {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:37:15 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-3aa5c594-23ff-4cac-a4ab-4ce14d9728f7 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-11-29T03:32:30Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-TestVolumeBootPattern-volume-backed-server-1667553757',display_name='tempest-TestVolumeBootPattern-volume-backed-server-1667553757',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-testvolumebootpattern-volume-backed-server-1667553757',id=48,image_ref='',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBEkw5vg+737ESWS3QhcThM/Ju+W7vDKKOHu4jAL47D+/LrumwViYptAXsVBquZbfetengtms1FvxACUCBNg/diM/Oq5FnXWy4aGB+8HpGsjosjG8tDfDnQgbi6ZUnh66qQ==',key_name='tempest-keypair-185724684',keypairs=<?>,launch_index=0,launched_at=2023-11-29T03:32:44Z,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='81a265d3c15b4e9d826b35ddbd0ff443',ramdisk_id='',reservation_id='r-699s07kq',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-TestVolumeBootPattern-1244492388',owner_user_name='tempest-TestVolumeBootPattern-1244492388-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-11-29T03:32:45Z,user_data='IyEvYmluL3NoCmVjaG8gIlByaW50aW5nIGNpcnJvcyB1c2VyIGF1dGhvcml6ZWQga2V5cyIKY2F0IH5jaXJyb3MvLnNzaC9hdXRob3JpemVkX2tleXMgfHwgdHJ1ZQo=',user_id='7d1f1f11a9aa4c1db8627413eb963fa4',uuid=884589ec-b86c-4094-9a3f-54b641311d62,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "1deddce5-7279-44f9-9f1e-cd9ffddc5d6c", "address": "fa:16:3e:c5:44:c3", "network": {"id": "a3f397d3-fd75-4a6e-a815-cfcedcca15fb", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-678570085-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.93", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "81a265d3c15b4e9d826b35ddbd0ff443", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap1deddce5-72", "ovs_interfaceid": "1deddce5-7279-44f9-9f1e-cd9ffddc5d6c", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Nov 29 03:37:15 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-3aa5c594-23ff-4cac-a4ab-4ce14d9728f7 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Converting VIF {"id": "1deddce5-7279-44f9-9f1e-cd9ffddc5d6c", "address": "fa:16:3e:c5:44:c3", "network": {"id": "a3f397d3-fd75-4a6e-a815-cfcedcca15fb", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-678570085-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.6", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.93", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "81a265d3c15b4e9d826b35ddbd0ff443", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap1deddce5-72", "ovs_interfaceid": "1deddce5-7279-44f9-9f1e-cd9ffddc5d6c", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:37:15 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-3aa5c594-23ff-4cac-a4ab-4ce14d9728f7 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:c5:44:c3,bridge_name='br-int',has_traffic_filtering=True,id=1deddce5-7279-44f9-9f1e-cd9ffddc5d6c,network=Network(a3f397d3-fd75-4a6e-a815-cfcedcca15fb),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap1deddce5-72') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:37:15 user nova-compute[75234]: DEBUG os_vif [None req-3aa5c594-23ff-4cac-a4ab-4ce14d9728f7 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Unplugging vif VIFOpenVSwitch(active=True,address=fa:16:3e:c5:44:c3,bridge_name='br-int',has_traffic_filtering=True,id=1deddce5-7279-44f9-9f1e-cd9ffddc5d6c,network=Network(a3f397d3-fd75-4a6e-a815-cfcedcca15fb),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap1deddce5-72') {{(pid=75234) unplug /opt/stack/data/venv/lib/python3.10/site-packages/os_vif/__init__.py:109}}
Nov 29 03:37:15 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:37:15 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tap1deddce5-72, bridge=br-int, if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:37:15 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:37:15 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:37:15 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:37:15 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbDestroyCommand(_result=None, table=QoS, record=90c3f023-0aab-4515-8ef4-215807178391) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:37:15 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:37:15 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:37:15 user nova-compute[75234]: INFO os_vif [None req-3aa5c594-23ff-4cac-a4ab-4ce14d9728f7 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Successfully unplugged vif VIFOpenVSwitch(active=True,address=fa:16:3e:c5:44:c3,bridge_name='br-int',has_traffic_filtering=True,id=1deddce5-7279-44f9-9f1e-cd9ffddc5d6c,network=Network(a3f397d3-fd75-4a6e-a815-cfcedcca15fb),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap1deddce5-72')
Nov 29 03:37:15 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-3aa5c594-23ff-4cac-a4ab-4ce14d9728f7 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: 884589ec-b86c-4094-9a3f-54b641311d62] calling os-brick to detach iSCSI Volume {{(pid=75234) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Nov 29 03:37:15 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-3aa5c594-23ff-4cac-a4ab-4ce14d9728f7 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f8a5050c9a0>, {'target_iqn': 'iqn.2010-10.org.openstack:a3d30ae4-ca6d-4e06-8fc9-caf4d91ef900', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'foMjDFpN', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdd'}, None), 'kwargs': True}}" {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:176}}
Nov 29 03:37:15 user nova-compute[75234]: WARNING os_brick.initiator.connectors.base [None req-3aa5c594-23ff-4cac-a4ab-4ce14d9728f7 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Nov 29 03:37:15 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-3aa5c594-23ff-4cac-a4ab-4ce14d9728f7 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:68}}
Nov 29 03:37:15 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-3aa5c594-23ff-4cac-a4ab-4ce14d9728f7 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.002s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:73}}
Nov 29 03:37:15 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:37:15 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.024s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:37:15 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[aef4867b-92e0-4ad3-a07d-de1303fbf799]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:a3d30ae4-ca6d-4e06-8fc9-caf4d91ef900\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:257d7345-5b29-4bc1-8844-e211864c06c3\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:37:15 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-3aa5c594-23ff-4cac-a4ab-4ce14d9728f7 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS:
Nov 29 03:37:15 user nova-compute[75234]: No targets found.
Nov 29 03:37:15 user nova-compute[75234]: iSNS:
Nov 29 03:37:15 user nova-compute[75234]: No targets found.
Nov 29 03:37:15 user nova-compute[75234]: STATIC:
Nov 29 03:37:15 user nova-compute[75234]: Target: iqn.2010-10.org.openstack:a3d30ae4-ca6d-4e06-8fc9-caf4d91ef900
Nov 29 03:37:15 user nova-compute[75234]:         Portal: 172.16.0.220:3260,-1
Nov 29 03:37:15 user nova-compute[75234]:                 Iface Name: default
Nov 29 03:37:15 user nova-compute[75234]: Target: iqn.2010-10.org.openstack:257d7345-5b29-4bc1-8844-e211864c06c3
Nov 29 03:37:15 user nova-compute[75234]:         Portal: 172.16.0.220:3260,-1
Nov 29 03:37:15 user nova-compute[75234]:                 Iface Name: default
Nov 29 03:37:15 user nova-compute[75234]: Target: iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0
Nov 29 03:37:15 user nova-compute[75234]:         Portal: 172.16.0.220:3260,-1
Nov 29 03:37:15 user nova-compute[75234]:                 Iface Name: default
Nov 29 03:37:15 user nova-compute[75234]: FIRMWARE:
Nov 29 03:37:15 user nova-compute[75234]: No targets found.
Nov 29 03:37:15 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:37:15 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-3aa5c594-23ff-4cac-a4ab-4ce14d9728f7 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Regex to get portals from discoverydb: ^SENDTARGETS:
Nov 29 03:37:15 user nova-compute[75234]: .*?^DiscoveryAddress: 172.16.0.220,3260.*?
Nov 29 03:37:15 user nova-compute[75234]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=75234) _get_discoverydb_portals /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:382}}
Nov 29 03:37:15 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-3aa5c594-23ff-4cac-a4ab-4ce14d9728f7 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:a3d30ae4-ca6d-4e06-8fc9-caf4d91ef900', 0)] {{(pid=75234) _get_connection_devices /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:830}}
Nov 29 03:37:15 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:37:15 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.017s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:37:15 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[9e1f96a1-3ea4-421d-b2d4-4cd46b17f5ac]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:257d7345-5b29-4bc1-8844-e211864c06c3\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:a3d30ae4-ca6d-4e06-8fc9-caf4d91ef900\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:37:15 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:37:15 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.025s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:37:15 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[46e30e4a-efd5-40ce-a6b8-67c2ff60b823]: (4, ('tcp: [42] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0 (non-flash)\ntcp: [54] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a3d30ae4-ca6d-4e06-8fc9-caf4d91ef900 (non-flash)\ntcp: [55] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:257d7345-5b29-4bc1-8844-e211864c06c3 (non-flash)\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:37:15 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-3aa5c594-23ff-4cac-a4ab-4ce14d9728f7 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [42] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0 (non-flash)
Nov 29 03:37:15 user nova-compute[75234]: tcp: [54] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a3d30ae4-ca6d-4e06-8fc9-caf4d91ef900 (non-flash)
Nov 29 03:37:15 user nova-compute[75234]: tcp: [55] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:257d7345-5b29-4bc1-8844-e211864c06c3 (non-flash)
Nov 29 03:37:15 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:37:15 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-3aa5c594-23ff-4cac-a4ab-4ce14d9728f7 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] iscsi session list stdout=tcp: [42] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0 (non-flash)
Nov 29 03:37:15 user nova-compute[75234]: tcp: [54] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:a3d30ae4-ca6d-4e06-8fc9-caf4d91ef900 (non-flash)
Nov 29 03:37:15 user nova-compute[75234]: tcp: [55] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:257d7345-5b29-4bc1-8844-e211864c06c3 (non-flash)
Nov 29 03:37:15 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsi_session /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Nov 29 03:37:15 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-3aa5c594-23ff-4cac-a4ab-4ce14d9728f7 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7f8a2c60fac0>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:a3d30ae4-ca6d-4e06-8fc9-caf4d91ef900'): ({'sdd'}, set())}) {{(pid=75234) _get_connection_devices /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:863}}
Nov 29 03:37:15 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-3aa5c594-23ff-4cac-a4ab-4ce14d9728f7 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Removing single pathed devices sdd {{(pid=75234) remove_connection /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:374}}
Nov 29 03:37:15 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:37:15 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.020s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:37:15 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[31501bc2-9c46-405b-83f5-9a0821237f37]: (4, ('path checker states:\nup                  3\n\npaths: 0\nbusy: False\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:37:15 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdd {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:37:15 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdd" returned: 0 in 0.019s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:37:15 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[43f0f886-b76e-49bd-bd1e-84fe60978995]: (4, ('ok\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:37:15 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-3aa5c594-23ff-4cac-a4ab-4ce14d9728f7 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Flushing IO for device /dev/sdd {{(pid=75234) flush_device_io /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:434}}
Nov 29 03:37:15 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdd {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:37:15 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdd" returned: 0 in 0.013s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:37:15 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[04181093-993b-4564-a809-a77e697dbe4d]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:37:15 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-3aa5c594-23ff-4cac-a4ab-4ce14d9728f7 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Remove SCSI device /dev/sdd with /sys/block/sdd/device/delete {{(pid=75234) remove_scsi_device /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:148}}
Nov 29 03:37:15 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdd/device/delete {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:37:15 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdd/device/delete" returned: 0 in 0.045s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:37:15 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[ace50c3a-7516-48cc-8107-21d4a424eb9b]: (4, ('1', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:37:15 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-3aa5c594-23ff-4cac-a4ab-4ce14d9728f7 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Checking to see if SCSI volumes sdd have been removed. {{(pid=75234) wait_for_volumes_removal /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:156}}
Nov 29 03:37:15 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-3aa5c594-23ff-4cac-a4ab-4ce14d9728f7 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] SCSI volumes sdd have been removed. {{(pid=75234) wait_for_volumes_removal /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:166}}
Nov 29 03:37:15 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-3aa5c594-23ff-4cac-a4ab-4ce14d9728f7 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:a3d30ae4-ca6d-4e06-8fc9-caf4d91ef900')] {{(pid=75234) _disconnect_connection /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Nov 29 03:37:15 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:a3d30ae4-ca6d-4e06-8fc9-caf4d91ef900 -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:37:15 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:a3d30ae4-ca6d-4e06-8fc9-caf4d91ef900 -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.022s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:37:15 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[d45ebcf2-f223-4c67-bf24-4bc02df13524]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:37:15 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-3aa5c594-23ff-4cac-a4ab-4ce14d9728f7 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:37:15 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:a3d30ae4-ca6d-4e06-8fc9-caf4d91ef900 -p 172.16.0.220:3260 --logout {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:37:15 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:a3d30ae4-ca6d-4e06-8fc9-caf4d91ef900 -p 172.16.0.220:3260 --logout" returned: 0 in 0.055s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:37:15 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[5115658f-83e2-455d-94d6-a0473a89eeea]: (4, ('Logging out of session [sid: 54, target: iqn.2010-10.org.openstack:a3d30ae4-ca6d-4e06-8fc9-caf4d91ef900, portal: 172.16.0.220,3260]\nLogout of [sid: 54, target: iqn.2010-10.org.openstack:a3d30ae4-ca6d-4e06-8fc9-caf4d91ef900, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:37:15 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-3aa5c594-23ff-4cac-a4ab-4ce14d9728f7 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 54, target: iqn.2010-10.org.openstack:a3d30ae4-ca6d-4e06-8fc9-caf4d91ef900, portal: 172.16.0.220,3260]
Nov 29 03:37:15 user nova-compute[75234]: Logout of [sid: 54, target: iqn.2010-10.org.openstack:a3d30ae4-ca6d-4e06-8fc9-caf4d91ef900, portal: 172.16.0.220,3260] successful.
Nov 29 03:37:15 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:37:15 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:a3d30ae4-ca6d-4e06-8fc9-caf4d91ef900 -p 172.16.0.220:3260 --op delete {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:37:15 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:a3d30ae4-ca6d-4e06-8fc9-caf4d91ef900 -p 172.16.0.220:3260 --op delete" returned: 0 in 0.022s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:37:15 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[d234ac48-a807-4456-b951-951983f8a16c]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:37:15 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-3aa5c594-23ff-4cac-a4ab-4ce14d9728f7 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:37:15 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-3aa5c594-23ff-4cac-a4ab-4ce14d9728f7 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.309s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:87}}
Nov 29 03:37:15 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-3aa5c594-23ff-4cac-a4ab-4ce14d9728f7 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] <== disconnect_volume: return (310ms) None {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:203}}
Nov 29 03:37:15 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-3aa5c594-23ff-4cac-a4ab-4ce14d9728f7 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: 884589ec-b86c-4094-9a3f-54b641311d62] Disconnected iSCSI Volume {{(pid=75234) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:79}}
Nov 29 03:37:15 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-3aa5c594-23ff-4cac-a4ab-4ce14d9728f7 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: 884589ec-b86c-4094-9a3f-54b641311d62] Deleting instance files /opt/stack/data/nova/instances/884589ec-b86c-4094-9a3f-54b641311d62_del
Nov 29 03:37:15 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-3aa5c594-23ff-4cac-a4ab-4ce14d9728f7 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: 884589ec-b86c-4094-9a3f-54b641311d62] Deletion of /opt/stack/data/nova/instances/884589ec-b86c-4094-9a3f-54b641311d62_del complete
Nov 29 03:37:16 user nova-compute[75234]: INFO nova.compute.manager [None req-3aa5c594-23ff-4cac-a4ab-4ce14d9728f7 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: 884589ec-b86c-4094-9a3f-54b641311d62] Took 1.16 seconds to destroy the instance on the hypervisor.
Nov 29 03:37:16 user nova-compute[75234]: DEBUG oslo.service.loopingcall [None req-3aa5c594-23ff-4cac-a4ab-4ce14d9728f7 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=75234) func /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/loopingcall.py:435}}
Nov 29 03:37:16 user nova-compute[75234]: DEBUG nova.compute.manager [-] [instance: 884589ec-b86c-4094-9a3f-54b641311d62] Deallocating network for instance {{(pid=75234) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2279}}
Nov 29 03:37:16 user nova-compute[75234]: DEBUG nova.network.neutron [-] [instance: 884589ec-b86c-4094-9a3f-54b641311d62] deallocate_for_instance() {{(pid=75234) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1802}}
Nov 29 03:37:16 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:37:16 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:37:16 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=75234) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10541}}
Nov 29 03:37:16 user nova-compute[75234]: DEBUG nova.network.neutron [-] [instance: 884589ec-b86c-4094-9a3f-54b641311d62] Updating instance_info_cache with network_info: [] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:37:16 user nova-compute[75234]: INFO nova.compute.manager [-] [instance: 884589ec-b86c-4094-9a3f-54b641311d62] Took 0.76 seconds to deallocate network for instance.
Nov 29 03:37:16 user nova-compute[75234]: DEBUG nova.compute.manager [req-ba4c8766-f37b-4b5c-b4bd-032c50d0d088 req-60d37c3e-cc3f-4ade-ac33-a875f494434a service nova] [instance: 884589ec-b86c-4094-9a3f-54b641311d62] Received event network-vif-deleted-1deddce5-7279-44f9-9f1e-cd9ffddc5d6c {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:37:17 user nova-compute[75234]: DEBUG nova.compute.manager [req-6307de76-89ac-4765-8243-0b4e2a1f45d8 req-49a3879b-a5c5-4f08-ade8-d9f0e46d3f25 service nova] [instance: 884589ec-b86c-4094-9a3f-54b641311d62] Received event network-vif-plugged-1deddce5-7279-44f9-9f1e-cd9ffddc5d6c {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:37:17 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-6307de76-89ac-4765-8243-0b4e2a1f45d8 req-49a3879b-a5c5-4f08-ade8-d9f0e46d3f25 service nova] Acquiring lock "884589ec-b86c-4094-9a3f-54b641311d62-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:37:17 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-6307de76-89ac-4765-8243-0b4e2a1f45d8 req-49a3879b-a5c5-4f08-ade8-d9f0e46d3f25 service nova] Lock "884589ec-b86c-4094-9a3f-54b641311d62-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:37:17 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-6307de76-89ac-4765-8243-0b4e2a1f45d8 req-49a3879b-a5c5-4f08-ade8-d9f0e46d3f25 service nova] Lock "884589ec-b86c-4094-9a3f-54b641311d62-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:37:17 user nova-compute[75234]: DEBUG nova.compute.manager [req-6307de76-89ac-4765-8243-0b4e2a1f45d8 req-49a3879b-a5c5-4f08-ade8-d9f0e46d3f25 service nova] [instance: 884589ec-b86c-4094-9a3f-54b641311d62] No waiting events found dispatching network-vif-plugged-1deddce5-7279-44f9-9f1e-cd9ffddc5d6c {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:37:17 user nova-compute[75234]: WARNING nova.compute.manager [req-6307de76-89ac-4765-8243-0b4e2a1f45d8 req-49a3879b-a5c5-4f08-ade8-d9f0e46d3f25 service nova] [instance: 884589ec-b86c-4094-9a3f-54b641311d62] Received unexpected event network-vif-plugged-1deddce5-7279-44f9-9f1e-cd9ffddc5d6c for instance with vm_state active and task_state deleting.
Nov 29 03:37:17 user nova-compute[75234]: INFO nova.compute.manager [None req-3aa5c594-23ff-4cac-a4ab-4ce14d9728f7 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: 884589ec-b86c-4094-9a3f-54b641311d62] Took 0.83 seconds to detach 1 volumes for instance.
Nov 29 03:37:17 user nova-compute[75234]: DEBUG nova.compute.manager [None req-3aa5c594-23ff-4cac-a4ab-4ce14d9728f7 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: 884589ec-b86c-4094-9a3f-54b641311d62] Deleting volume: a3d30ae4-ca6d-4e06-8fc9-caf4d91ef900 {{(pid=75234) _cleanup_volumes /opt/stack/nova/nova/compute/manager.py:3237}}
Nov 29 03:37:17 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-3aa5c594-23ff-4cac-a4ab-4ce14d9728f7 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:37:17 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-3aa5c594-23ff-4cac-a4ab-4ce14d9728f7 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:37:18 user nova-compute[75234]: DEBUG nova.compute.provider_tree [None req-3aa5c594-23ff-4cac-a4ab-4ce14d9728f7 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Inventory has not changed in ProviderTree for provider: 4d291220-7c37-4839-a0e3-b23c66502b74 {{(pid=75234) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Nov 29 03:37:18 user nova-compute[75234]: DEBUG nova.scheduler.client.report [None req-3aa5c594-23ff-4cac-a4ab-4ce14d9728f7 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Inventory has not changed for provider 4d291220-7c37-4839-a0e3-b23c66502b74 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=75234) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Nov 29 03:37:18 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-3aa5c594-23ff-4cac-a4ab-4ce14d9728f7 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.421s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:37:18 user nova-compute[75234]: INFO nova.scheduler.client.report [None req-3aa5c594-23ff-4cac-a4ab-4ce14d9728f7 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Deleted allocations for instance 884589ec-b86c-4094-9a3f-54b641311d62
Nov 29 03:37:18 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-3aa5c594-23ff-4cac-a4ab-4ce14d9728f7 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Lock "884589ec-b86c-4094-9a3f-54b641311d62" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 3.420s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:37:18 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager.update_available_resource {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:37:18 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:37:18 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:37:18 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:37:18 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Auditing locally available compute resources for user (node: user) {{(pid=75234) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:907}}
Nov 29 03:37:18 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/65d48c93-b6da-4f51-9d5a-0676dda62a60/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:37:18 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/65d48c93-b6da-4f51-9d5a-0676dda62a60/disk --force-share --output=json" returned: 0 in 0.166s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:37:18 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/65d48c93-b6da-4f51-9d5a-0676dda62a60/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:37:18 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/65d48c93-b6da-4f51-9d5a-0676dda62a60/disk --force-share --output=json" returned: 0 in 0.169s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:37:18 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] skipping disk /dev/sde (vda) as it is a volume {{(pid=75234) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11213}}
Nov 29 03:37:18 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] skipping disk /dev/sdc (vda) as it is a volume {{(pid=75234) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11213}}
Nov 29 03:37:18 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/abb01889-df4b-44a6-9827-87056d7ca125/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:37:19 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/abb01889-df4b-44a6-9827-87056d7ca125/disk --force-share --output=json" returned: 0 in 0.188s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:37:19 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/abb01889-df4b-44a6-9827-87056d7ca125/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:37:19 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/abb01889-df4b-44a6-9827-87056d7ca125/disk --force-share --output=json" returned: 0 in 0.169s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:37:19 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:37:19 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:37:19 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Hypervisor/Node resource view: name=user free_ram=8722MB free_disk=24.920391082763672GB free_vcpus=8 pci_devices=[{"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}] {{(pid=75234) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1106}}
Nov 29 03:37:19 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:37:19 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:37:19 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance 65d48c93-b6da-4f51-9d5a-0676dda62a60 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 512, 'DISK_GB': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:37:19 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance d5096ca2-d005-45ac-a487-11a810446859 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 512}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:37:19 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance abb01889-df4b-44a6-9827-87056d7ca125 actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 512, 'DISK_GB': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:37:19 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance ebc3e3b2-037d-43ef-9426-caebe298ee3e actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 512}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:37:19 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Total usable vcpus: 12, total allocated vcpus: 4 {{(pid=75234) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1129}}
Nov 29 03:37:19 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Final resource view: name=user phys_ram=16011MB used_ram=2560MB phys_disk=40GB used_disk=2GB total_vcpus=12 used_vcpus=4 pci_stats=[] {{(pid=75234) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1138}}
Nov 29 03:37:20 user nova-compute[75234]: DEBUG nova.compute.provider_tree [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Inventory has not changed in ProviderTree for provider: 4d291220-7c37-4839-a0e3-b23c66502b74 {{(pid=75234) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Nov 29 03:37:20 user nova-compute[75234]: DEBUG nova.scheduler.client.report [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Inventory has not changed for provider 4d291220-7c37-4839-a0e3-b23c66502b74 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=75234) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Nov 29 03:37:20 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Compute_service record updated for user:user {{(pid=75234) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:1067}}
Nov 29 03:37:20 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.401s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:37:20 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._cleanup_incomplete_migrations {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:37:20 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Cleaning up deleted instances with incomplete migration  {{(pid=75234) _cleanup_incomplete_migrations /opt/stack/nova/nova/compute/manager.py:11247}}
Nov 29 03:37:20 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:37:23 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:37:23 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:37:24 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:37:24 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Starting heal instance info cache {{(pid=75234) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9922}}
Nov 29 03:37:24 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Acquiring lock "refresh_cache-d5096ca2-d005-45ac-a487-11a810446859" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:37:24 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Acquired lock "refresh_cache-d5096ca2-d005-45ac-a487-11a810446859" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:37:24 user nova-compute[75234]: DEBUG nova.network.neutron [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] [instance: d5096ca2-d005-45ac-a487-11a810446859] Forcefully refreshing network info cache for instance {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2003}}
Nov 29 03:37:24 user nova-compute[75234]: DEBUG nova.network.neutron [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] [instance: d5096ca2-d005-45ac-a487-11a810446859] Updating instance_info_cache with network_info: [{"id": "9266d021-6e83-4bfa-8151-66c5182e61a0", "address": "fa:16:3e:ec:fb:67", "network": {"id": "4d086e08-7798-4cfe-9365-86be6f0a52d2", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-305624008-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "db00bcd0c82c44f1bda739e9857f6e84", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap9266d021-6e", "ovs_interfaceid": "9266d021-6e83-4bfa-8151-66c5182e61a0", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:37:24 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Releasing lock "refresh_cache-d5096ca2-d005-45ac-a487-11a810446859" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:37:24 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] [instance: d5096ca2-d005-45ac-a487-11a810446859] Updated the network info_cache for instance {{(pid=75234) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9993}}
Nov 29 03:37:25 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:37:25 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:37:25 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5002 ms, sending inactivity probe {{(pid=75234) run /opt/stack/data/venv/lib/python3.10/site-packages/ovs/reconnect.py:117}}
Nov 29 03:37:25 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=75234) _transition /opt/stack/data/venv/lib/python3.10/site-packages/ovs/reconnect.py:519}}
Nov 29 03:37:25 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=75234) _transition /opt/stack/data/venv/lib/python3.10/site-packages/ovs/reconnect.py:519}}
Nov 29 03:37:25 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:37:26 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:37:30 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._cleanup_expired_console_auth_tokens {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:37:30 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:37:35 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:37:40 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._run_pending_deletes {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:37:40 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Cleaning up deleted instances {{(pid=75234) _run_pending_deletes /opt/stack/nova/nova/compute/manager.py:11209}}
Nov 29 03:37:40 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] There are 0 instances to clean {{(pid=75234) _run_pending_deletes /opt/stack/nova/nova/compute/manager.py:11218}}
Nov 29 03:37:40 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:37:45 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:37:50 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:37:55 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:38:00 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:38:01 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-088b0aef-b852-4dd2-8090-4fc1433ccc64 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Acquiring lock "ebc3e3b2-037d-43ef-9426-caebe298ee3e" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:38:01 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-088b0aef-b852-4dd2-8090-4fc1433ccc64 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Lock "ebc3e3b2-037d-43ef-9426-caebe298ee3e" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.002s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:38:01 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-088b0aef-b852-4dd2-8090-4fc1433ccc64 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Acquiring lock "ebc3e3b2-037d-43ef-9426-caebe298ee3e-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:38:01 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-088b0aef-b852-4dd2-8090-4fc1433ccc64 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Lock "ebc3e3b2-037d-43ef-9426-caebe298ee3e-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:38:01 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-088b0aef-b852-4dd2-8090-4fc1433ccc64 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Lock "ebc3e3b2-037d-43ef-9426-caebe298ee3e-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:38:01 user nova-compute[75234]: INFO nova.compute.manager [None req-088b0aef-b852-4dd2-8090-4fc1433ccc64 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] [instance: ebc3e3b2-037d-43ef-9426-caebe298ee3e] Terminating instance
Nov 29 03:38:01 user nova-compute[75234]: DEBUG nova.compute.manager [None req-088b0aef-b852-4dd2-8090-4fc1433ccc64 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] [instance: ebc3e3b2-037d-43ef-9426-caebe298ee3e] Start destroying the instance on the hypervisor. {{(pid=75234) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3140}}
Nov 29 03:38:01 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:38:01 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:38:01 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:38:01 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:38:01 user nova-compute[75234]: DEBUG nova.compute.manager [req-a3891467-3ba2-49a4-a532-5e626267479e req-2b1247ac-b997-4083-993f-7215a4fd53a4 service nova] [instance: ebc3e3b2-037d-43ef-9426-caebe298ee3e] Received event network-vif-unplugged-bd0a4e5b-69cd-4f5c-ae22-994e4d8f2682 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:38:01 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-a3891467-3ba2-49a4-a532-5e626267479e req-2b1247ac-b997-4083-993f-7215a4fd53a4 service nova] Acquiring lock "ebc3e3b2-037d-43ef-9426-caebe298ee3e-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:38:01 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-a3891467-3ba2-49a4-a532-5e626267479e req-2b1247ac-b997-4083-993f-7215a4fd53a4 service nova] Lock "ebc3e3b2-037d-43ef-9426-caebe298ee3e-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:38:01 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-a3891467-3ba2-49a4-a532-5e626267479e req-2b1247ac-b997-4083-993f-7215a4fd53a4 service nova] Lock "ebc3e3b2-037d-43ef-9426-caebe298ee3e-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:38:01 user nova-compute[75234]: DEBUG nova.compute.manager [req-a3891467-3ba2-49a4-a532-5e626267479e req-2b1247ac-b997-4083-993f-7215a4fd53a4 service nova] [instance: ebc3e3b2-037d-43ef-9426-caebe298ee3e] No waiting events found dispatching network-vif-unplugged-bd0a4e5b-69cd-4f5c-ae22-994e4d8f2682 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:38:01 user nova-compute[75234]: DEBUG nova.compute.manager [req-a3891467-3ba2-49a4-a532-5e626267479e req-2b1247ac-b997-4083-993f-7215a4fd53a4 service nova] [instance: ebc3e3b2-037d-43ef-9426-caebe298ee3e] Received event network-vif-unplugged-bd0a4e5b-69cd-4f5c-ae22-994e4d8f2682 for instance with task_state deleting. {{(pid=75234) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10890}}
Nov 29 03:38:01 user nova-compute[75234]: INFO nova.virt.libvirt.driver [-] [instance: ebc3e3b2-037d-43ef-9426-caebe298ee3e] Instance destroyed successfully.
Nov 29 03:38:01 user nova-compute[75234]: DEBUG nova.objects.instance [None req-088b0aef-b852-4dd2-8090-4fc1433ccc64 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Lazy-loading 'resources' on Instance uuid ebc3e3b2-037d-43ef-9426-caebe298ee3e {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:38:01 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-088b0aef-b852-4dd2-8090-4fc1433ccc64 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-11-29T03:32:27Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description=None,display_name='tempest-ServerBootFromVolumeStableRescueTest-server-1464602599',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-serverbootfromvolumestablerescuetest-server-1464602599',id=47,image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data=None,key_name=None,keypairs=<?>,launch_index=0,launched_at=2023-11-29T03:34:26Z,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='db00bcd0c82c44f1bda739e9857f6e84',ramdisk_id='',reservation_id='r-ztz14xc0',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-ServerBootFromVolumeStableRescueTest-1601266069',owner_user_name='tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-11-29T03:36:15Z,user_data=None,user_id='877aa877f13a4f84ac4725278010aea3',uuid=ebc3e3b2-037d-43ef-9426-caebe298ee3e,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "bd0a4e5b-69cd-4f5c-ae22-994e4d8f2682", "address": "fa:16:3e:c2:aa:3e", "network": {"id": "4d086e08-7798-4cfe-9365-86be6f0a52d2", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-305624008-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "db00bcd0c82c44f1bda739e9857f6e84", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapbd0a4e5b-69", "ovs_interfaceid": "bd0a4e5b-69cd-4f5c-ae22-994e4d8f2682", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Nov 29 03:38:01 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-088b0aef-b852-4dd2-8090-4fc1433ccc64 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Converting VIF {"id": "bd0a4e5b-69cd-4f5c-ae22-994e4d8f2682", "address": "fa:16:3e:c2:aa:3e", "network": {"id": "4d086e08-7798-4cfe-9365-86be6f0a52d2", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-305624008-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "db00bcd0c82c44f1bda739e9857f6e84", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapbd0a4e5b-69", "ovs_interfaceid": "bd0a4e5b-69cd-4f5c-ae22-994e4d8f2682", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:38:01 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-088b0aef-b852-4dd2-8090-4fc1433ccc64 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:c2:aa:3e,bridge_name='br-int',has_traffic_filtering=True,id=bd0a4e5b-69cd-4f5c-ae22-994e4d8f2682,network=Network(4d086e08-7798-4cfe-9365-86be6f0a52d2),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapbd0a4e5b-69') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:38:01 user nova-compute[75234]: DEBUG os_vif [None req-088b0aef-b852-4dd2-8090-4fc1433ccc64 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Unplugging vif VIFOpenVSwitch(active=True,address=fa:16:3e:c2:aa:3e,bridge_name='br-int',has_traffic_filtering=True,id=bd0a4e5b-69cd-4f5c-ae22-994e4d8f2682,network=Network(4d086e08-7798-4cfe-9365-86be6f0a52d2),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapbd0a4e5b-69') {{(pid=75234) unplug /opt/stack/data/venv/lib/python3.10/site-packages/os_vif/__init__.py:109}}
Nov 29 03:38:01 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:38:01 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tapbd0a4e5b-69, bridge=br-int, if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:38:01 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:38:01 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:38:01 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:38:01 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbDestroyCommand(_result=None, table=QoS, record=02065a19-7713-49ec-8e10-e7f7f21ec748) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:38:01 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:38:01 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:38:01 user nova-compute[75234]: INFO os_vif [None req-088b0aef-b852-4dd2-8090-4fc1433ccc64 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Successfully unplugged vif VIFOpenVSwitch(active=True,address=fa:16:3e:c2:aa:3e,bridge_name='br-int',has_traffic_filtering=True,id=bd0a4e5b-69cd-4f5c-ae22-994e4d8f2682,network=Network(4d086e08-7798-4cfe-9365-86be6f0a52d2),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapbd0a4e5b-69')
Nov 29 03:38:01 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-088b0aef-b852-4dd2-8090-4fc1433ccc64 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] [instance: ebc3e3b2-037d-43ef-9426-caebe298ee3e] calling os-brick to detach iSCSI Volume {{(pid=75234) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Nov 29 03:38:01 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-088b0aef-b852-4dd2-8090-4fc1433ccc64 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f8a5050c9a0>, {'target_iqn': 'iqn.2010-10.org.openstack:257d7345-5b29-4bc1-8844-e211864c06c3', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'uUeBnhLX', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sde'}, None), 'kwargs': True}}" {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:176}}
Nov 29 03:38:01 user nova-compute[75234]: WARNING os_brick.initiator.connectors.base [None req-088b0aef-b852-4dd2-8090-4fc1433ccc64 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Nov 29 03:38:01 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-088b0aef-b852-4dd2-8090-4fc1433ccc64 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:68}}
Nov 29 03:38:01 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-088b0aef-b852-4dd2-8090-4fc1433ccc64 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:73}}
Nov 29 03:38:01 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:38:01 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.019s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:38:01 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[faebe2fb-dc2b-4243-96b1-e057b944d432]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:257d7345-5b29-4bc1-8844-e211864c06c3\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:38:01 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-088b0aef-b852-4dd2-8090-4fc1433ccc64 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS:
Nov 29 03:38:01 user nova-compute[75234]: No targets found.
Nov 29 03:38:01 user nova-compute[75234]: iSNS:
Nov 29 03:38:01 user nova-compute[75234]: No targets found.
Nov 29 03:38:01 user nova-compute[75234]: STATIC:
Nov 29 03:38:01 user nova-compute[75234]: Target: iqn.2010-10.org.openstack:257d7345-5b29-4bc1-8844-e211864c06c3
Nov 29 03:38:01 user nova-compute[75234]:         Portal: 172.16.0.220:3260,-1
Nov 29 03:38:01 user nova-compute[75234]:                 Iface Name: default
Nov 29 03:38:01 user nova-compute[75234]: Target: iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0
Nov 29 03:38:01 user nova-compute[75234]:         Portal: 172.16.0.220:3260,-1
Nov 29 03:38:01 user nova-compute[75234]:                 Iface Name: default
Nov 29 03:38:01 user nova-compute[75234]: FIRMWARE:
Nov 29 03:38:01 user nova-compute[75234]: No targets found.
Nov 29 03:38:01 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:38:01 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-088b0aef-b852-4dd2-8090-4fc1433ccc64 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Regex to get portals from discoverydb: ^SENDTARGETS:
Nov 29 03:38:01 user nova-compute[75234]: .*?^DiscoveryAddress: 172.16.0.220,3260.*?
Nov 29 03:38:01 user nova-compute[75234]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=75234) _get_discoverydb_portals /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:382}}
Nov 29 03:38:01 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-088b0aef-b852-4dd2-8090-4fc1433ccc64 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:257d7345-5b29-4bc1-8844-e211864c06c3', 0)] {{(pid=75234) _get_connection_devices /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:830}}
Nov 29 03:38:01 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:38:01 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.018s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:38:01 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[608c7891-aded-41c8-a2fa-f8c81a190207]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:257d7345-5b29-4bc1-8844-e211864c06c3\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:38:01 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:38:01 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.022s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:38:01 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[ef5dbd88-5bae-4e13-8f36-740bcfdd3141]: (4, ('tcp: [42] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0 (non-flash)\ntcp: [55] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:257d7345-5b29-4bc1-8844-e211864c06c3 (non-flash)\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:38:01 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-088b0aef-b852-4dd2-8090-4fc1433ccc64 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [42] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0 (non-flash)
Nov 29 03:38:01 user nova-compute[75234]: tcp: [55] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:257d7345-5b29-4bc1-8844-e211864c06c3 (non-flash)
Nov 29 03:38:01 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:38:01 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-088b0aef-b852-4dd2-8090-4fc1433ccc64 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] iscsi session list stdout=tcp: [42] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0 (non-flash)
Nov 29 03:38:01 user nova-compute[75234]: tcp: [55] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:257d7345-5b29-4bc1-8844-e211864c06c3 (non-flash)
Nov 29 03:38:01 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsi_session /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Nov 29 03:38:01 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-088b0aef-b852-4dd2-8090-4fc1433ccc64 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7f8a2c60d6c0>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:257d7345-5b29-4bc1-8844-e211864c06c3'): ({'sde'}, set())}) {{(pid=75234) _get_connection_devices /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:863}}
Nov 29 03:38:01 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-088b0aef-b852-4dd2-8090-4fc1433ccc64 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Removing single pathed devices sde {{(pid=75234) remove_connection /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:374}}
Nov 29 03:38:01 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:38:01 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.021s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:38:01 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[00db5cfb-131e-4b6d-9492-52b7980d4ace]: (4, ('path checker states:\nup                  2\n\npaths: 0\nbusy: False\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:38:01 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sde {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:38:01 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sde" returned: 0 in 0.018s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:38:01 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[cf50e9bc-6b6b-42af-8341-3ea882483342]: (4, ('ok\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:38:01 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-088b0aef-b852-4dd2-8090-4fc1433ccc64 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Flushing IO for device /dev/sde {{(pid=75234) flush_device_io /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:434}}
Nov 29 03:38:02 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sde {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:38:02 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sde" returned: 0 in 0.015s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:38:02 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[736b4aa6-2982-4fdd-9aff-26ea75dd8ec7]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:38:02 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-088b0aef-b852-4dd2-8090-4fc1433ccc64 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Remove SCSI device /dev/sde with /sys/block/sde/device/delete {{(pid=75234) remove_scsi_device /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:148}}
Nov 29 03:38:02 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sde/device/delete {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:38:02 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sde/device/delete" returned: 0 in 0.043s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:38:02 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[e92939b6-e16c-4bc1-b032-326bb611ba76]: (4, ('1', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:38:02 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-088b0aef-b852-4dd2-8090-4fc1433ccc64 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Checking to see if SCSI volumes sde have been removed. {{(pid=75234) wait_for_volumes_removal /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:156}}
Nov 29 03:38:02 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-088b0aef-b852-4dd2-8090-4fc1433ccc64 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] SCSI volumes sde have been removed. {{(pid=75234) wait_for_volumes_removal /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:166}}
Nov 29 03:38:02 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-088b0aef-b852-4dd2-8090-4fc1433ccc64 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:257d7345-5b29-4bc1-8844-e211864c06c3')] {{(pid=75234) _disconnect_connection /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Nov 29 03:38:02 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:257d7345-5b29-4bc1-8844-e211864c06c3 -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:38:02 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:257d7345-5b29-4bc1-8844-e211864c06c3 -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.019s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:38:02 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[0142ddb7-5f65-4255-acd7-f3c9ce54f4b9]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:38:02 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-088b0aef-b852-4dd2-8090-4fc1433ccc64 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:38:02 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:257d7345-5b29-4bc1-8844-e211864c06c3 -p 172.16.0.220:3260 --logout {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:38:02 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:257d7345-5b29-4bc1-8844-e211864c06c3 -p 172.16.0.220:3260 --logout" returned: 0 in 0.052s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:38:02 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[591c53da-b772-43b1-9624-2b76769e6c26]: (4, ('Logging out of session [sid: 55, target: iqn.2010-10.org.openstack:257d7345-5b29-4bc1-8844-e211864c06c3, portal: 172.16.0.220,3260]\nLogout of [sid: 55, target: iqn.2010-10.org.openstack:257d7345-5b29-4bc1-8844-e211864c06c3, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:38:02 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-088b0aef-b852-4dd2-8090-4fc1433ccc64 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 55, target: iqn.2010-10.org.openstack:257d7345-5b29-4bc1-8844-e211864c06c3, portal: 172.16.0.220,3260]
Nov 29 03:38:02 user nova-compute[75234]: Logout of [sid: 55, target: iqn.2010-10.org.openstack:257d7345-5b29-4bc1-8844-e211864c06c3, portal: 172.16.0.220,3260] successful.
Nov 29 03:38:02 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:38:02 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:257d7345-5b29-4bc1-8844-e211864c06c3 -p 172.16.0.220:3260 --op delete {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:38:02 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:257d7345-5b29-4bc1-8844-e211864c06c3 -p 172.16.0.220:3260 --op delete" returned: 0 in 0.022s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:38:02 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[9b079bd5-fa3d-40fb-b305-5d60b56215dd]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:38:02 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-088b0aef-b852-4dd2-8090-4fc1433ccc64 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:38:02 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-088b0aef-b852-4dd2-8090-4fc1433ccc64 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.300s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:87}}
Nov 29 03:38:02 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-088b0aef-b852-4dd2-8090-4fc1433ccc64 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] <== disconnect_volume: return (301ms) None {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:203}}
Nov 29 03:38:02 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-088b0aef-b852-4dd2-8090-4fc1433ccc64 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] [instance: ebc3e3b2-037d-43ef-9426-caebe298ee3e] Disconnected iSCSI Volume {{(pid=75234) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:79}}
Nov 29 03:38:02 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-088b0aef-b852-4dd2-8090-4fc1433ccc64 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] [instance: ebc3e3b2-037d-43ef-9426-caebe298ee3e] Deleting instance files /opt/stack/data/nova/instances/ebc3e3b2-037d-43ef-9426-caebe298ee3e_del
Nov 29 03:38:02 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-088b0aef-b852-4dd2-8090-4fc1433ccc64 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] [instance: ebc3e3b2-037d-43ef-9426-caebe298ee3e] Deletion of /opt/stack/data/nova/instances/ebc3e3b2-037d-43ef-9426-caebe298ee3e_del complete
Nov 29 03:38:02 user nova-compute[75234]: INFO nova.compute.manager [None req-088b0aef-b852-4dd2-8090-4fc1433ccc64 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] [instance: ebc3e3b2-037d-43ef-9426-caebe298ee3e] Took 1.15 seconds to destroy the instance on the hypervisor.
Nov 29 03:38:02 user nova-compute[75234]: DEBUG oslo.service.loopingcall [None req-088b0aef-b852-4dd2-8090-4fc1433ccc64 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=75234) func /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/loopingcall.py:435}}
Nov 29 03:38:02 user nova-compute[75234]: DEBUG nova.compute.manager [-] [instance: ebc3e3b2-037d-43ef-9426-caebe298ee3e] Deallocating network for instance {{(pid=75234) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2279}}
Nov 29 03:38:02 user nova-compute[75234]: DEBUG nova.network.neutron [-] [instance: ebc3e3b2-037d-43ef-9426-caebe298ee3e] deallocate_for_instance() {{(pid=75234) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1802}}
Nov 29 03:38:02 user nova-compute[75234]: DEBUG nova.network.neutron [-] [instance: ebc3e3b2-037d-43ef-9426-caebe298ee3e] Updating instance_info_cache with network_info: [] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:38:02 user nova-compute[75234]: INFO nova.compute.manager [-] [instance: ebc3e3b2-037d-43ef-9426-caebe298ee3e] Took 0.45 seconds to deallocate network for instance.
Nov 29 03:38:03 user nova-compute[75234]: INFO nova.compute.manager [None req-088b0aef-b852-4dd2-8090-4fc1433ccc64 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] [instance: ebc3e3b2-037d-43ef-9426-caebe298ee3e] Took 0.78 seconds to detach 1 volumes for instance.
Nov 29 03:38:03 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-088b0aef-b852-4dd2-8090-4fc1433ccc64 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:38:03 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-088b0aef-b852-4dd2-8090-4fc1433ccc64 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:38:03 user nova-compute[75234]: DEBUG nova.compute.manager [req-024b8923-28b7-46b0-8b9f-0f52542521bc req-86913e3b-cfc0-442a-b468-f9b21979ee1e service nova] [instance: ebc3e3b2-037d-43ef-9426-caebe298ee3e] Received event network-vif-plugged-bd0a4e5b-69cd-4f5c-ae22-994e4d8f2682 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:38:03 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-024b8923-28b7-46b0-8b9f-0f52542521bc req-86913e3b-cfc0-442a-b468-f9b21979ee1e service nova] Acquiring lock "ebc3e3b2-037d-43ef-9426-caebe298ee3e-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:38:03 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-024b8923-28b7-46b0-8b9f-0f52542521bc req-86913e3b-cfc0-442a-b468-f9b21979ee1e service nova] Lock "ebc3e3b2-037d-43ef-9426-caebe298ee3e-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:38:03 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-024b8923-28b7-46b0-8b9f-0f52542521bc req-86913e3b-cfc0-442a-b468-f9b21979ee1e service nova] Lock "ebc3e3b2-037d-43ef-9426-caebe298ee3e-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:38:03 user nova-compute[75234]: DEBUG nova.compute.manager [req-024b8923-28b7-46b0-8b9f-0f52542521bc req-86913e3b-cfc0-442a-b468-f9b21979ee1e service nova] [instance: ebc3e3b2-037d-43ef-9426-caebe298ee3e] No waiting events found dispatching network-vif-plugged-bd0a4e5b-69cd-4f5c-ae22-994e4d8f2682 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:38:03 user nova-compute[75234]: WARNING nova.compute.manager [req-024b8923-28b7-46b0-8b9f-0f52542521bc req-86913e3b-cfc0-442a-b468-f9b21979ee1e service nova] [instance: ebc3e3b2-037d-43ef-9426-caebe298ee3e] Received unexpected event network-vif-plugged-bd0a4e5b-69cd-4f5c-ae22-994e4d8f2682 for instance with vm_state deleted and task_state None.
Nov 29 03:38:03 user nova-compute[75234]: DEBUG nova.compute.manager [req-024b8923-28b7-46b0-8b9f-0f52542521bc req-86913e3b-cfc0-442a-b468-f9b21979ee1e service nova] [instance: ebc3e3b2-037d-43ef-9426-caebe298ee3e] Received event network-vif-deleted-bd0a4e5b-69cd-4f5c-ae22-994e4d8f2682 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:38:03 user nova-compute[75234]: DEBUG nova.compute.provider_tree [None req-088b0aef-b852-4dd2-8090-4fc1433ccc64 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Inventory has not changed in ProviderTree for provider: 4d291220-7c37-4839-a0e3-b23c66502b74 {{(pid=75234) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Nov 29 03:38:03 user nova-compute[75234]: DEBUG nova.scheduler.client.report [None req-088b0aef-b852-4dd2-8090-4fc1433ccc64 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Inventory has not changed for provider 4d291220-7c37-4839-a0e3-b23c66502b74 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=75234) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Nov 29 03:38:03 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-088b0aef-b852-4dd2-8090-4fc1433ccc64 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.204s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:38:03 user nova-compute[75234]: INFO nova.scheduler.client.report [None req-088b0aef-b852-4dd2-8090-4fc1433ccc64 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Deleted allocations for instance ebc3e3b2-037d-43ef-9426-caebe298ee3e
Nov 29 03:38:03 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-088b0aef-b852-4dd2-8090-4fc1433ccc64 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Lock "ebc3e3b2-037d-43ef-9426-caebe298ee3e" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 2.775s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:38:05 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:38:06 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:38:11 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:38:14 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:38:14 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-fe57873b-20a8-4d38-a798-90a5f865ee8c tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Acquiring lock "45059fbb-59d4-49ae-bccd-356edea7f287" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:38:14 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-fe57873b-20a8-4d38-a798-90a5f865ee8c tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Lock "45059fbb-59d4-49ae-bccd-356edea7f287" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:38:14 user nova-compute[75234]: DEBUG nova.compute.manager [None req-fe57873b-20a8-4d38-a798-90a5f865ee8c tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: 45059fbb-59d4-49ae-bccd-356edea7f287] Starting instance... {{(pid=75234) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2422}}
Nov 29 03:38:14 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-fe57873b-20a8-4d38-a798-90a5f865ee8c tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:38:14 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-fe57873b-20a8-4d38-a798-90a5f865ee8c tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:38:15 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-fe57873b-20a8-4d38-a798-90a5f865ee8c tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=75234) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2366}}
Nov 29 03:38:15 user nova-compute[75234]: INFO nova.compute.claims [None req-fe57873b-20a8-4d38-a798-90a5f865ee8c tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: 45059fbb-59d4-49ae-bccd-356edea7f287] Claim successful on node user
Nov 29 03:38:15 user nova-compute[75234]: DEBUG nova.compute.provider_tree [None req-fe57873b-20a8-4d38-a798-90a5f865ee8c tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Inventory has not changed in ProviderTree for provider: 4d291220-7c37-4839-a0e3-b23c66502b74 {{(pid=75234) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Nov 29 03:38:15 user nova-compute[75234]: DEBUG nova.scheduler.client.report [None req-fe57873b-20a8-4d38-a798-90a5f865ee8c tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Inventory has not changed for provider 4d291220-7c37-4839-a0e3-b23c66502b74 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=75234) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Nov 29 03:38:15 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-fe57873b-20a8-4d38-a798-90a5f865ee8c tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.244s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:38:15 user nova-compute[75234]: DEBUG nova.compute.manager [None req-fe57873b-20a8-4d38-a798-90a5f865ee8c tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: 45059fbb-59d4-49ae-bccd-356edea7f287] Start building networks asynchronously for instance. {{(pid=75234) _build_resources /opt/stack/nova/nova/compute/manager.py:2819}}
Nov 29 03:38:15 user nova-compute[75234]: DEBUG nova.compute.manager [None req-fe57873b-20a8-4d38-a798-90a5f865ee8c tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: 45059fbb-59d4-49ae-bccd-356edea7f287] Allocating IP information in the background. {{(pid=75234) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1972}}
Nov 29 03:38:15 user nova-compute[75234]: DEBUG nova.network.neutron [None req-fe57873b-20a8-4d38-a798-90a5f865ee8c tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: 45059fbb-59d4-49ae-bccd-356edea7f287] allocate_for_instance() {{(pid=75234) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1156}}
Nov 29 03:38:15 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-fe57873b-20a8-4d38-a798-90a5f865ee8c tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: 45059fbb-59d4-49ae-bccd-356edea7f287] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Nov 29 03:38:15 user nova-compute[75234]: DEBUG nova.compute.manager [None req-fe57873b-20a8-4d38-a798-90a5f865ee8c tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: 45059fbb-59d4-49ae-bccd-356edea7f287] Start building block device mappings for instance. {{(pid=75234) _build_resources /opt/stack/nova/nova/compute/manager.py:2854}}
Nov 29 03:38:15 user nova-compute[75234]: INFO nova.virt.block_device [None req-fe57873b-20a8-4d38-a798-90a5f865ee8c tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: 45059fbb-59d4-49ae-bccd-356edea7f287] Booting with volume 378addbb-8d90-40c5-ae61-c3a063116df9 at /dev/vda
Nov 29 03:38:15 user nova-compute[75234]: DEBUG nova.policy [None req-fe57873b-20a8-4d38-a798-90a5f865ee8c tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '7d1f1f11a9aa4c1db8627413eb963fa4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '81a265d3c15b4e9d826b35ddbd0ff443', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=75234) authorize /opt/stack/nova/nova/policy.py:203}}
Nov 29 03:38:15 user nova-compute[75234]: DEBUG os_brick.utils [None req-fe57873b-20a8-4d38-a798-90a5f865ee8c tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:176}}
Nov 29 03:38:15 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:38:15 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:38:15 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.014s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:38:15 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[1efabd85-3965-4700-a4b9-0e158f24b0f9]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:dd1368c5ba2c\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:38:15 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt -v / -n -o SOURCE {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:38:15 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt -v / -n -o SOURCE" returned: 0 in 0.016s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:38:15 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[11defd23-ac3a-4dc2-bdfd-6b6fe09aba0e]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:38:15 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:38:15 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.017s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:38:15 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[b922ae4f-641d-412d-982f-bd1f87f12d07]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:38:15 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[e172ad85-f63e-48b1-a4d1-10a5912fd9c5]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:38:15 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-fe57873b-20a8-4d38-a798-90a5f865ee8c tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Running cmd (subprocess): nvme version {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:38:15 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-fe57873b-20a8-4d38-a798-90a5f865ee8c tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] 'nvme version' failed. Not Retrying. {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:473}}
Nov 29 03:38:15 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.nvmeof [None req-fe57873b-20a8-4d38-a798-90a5f865ee8c tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] nvme not present on system {{(pid=75234) nvme_present /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/nvmeof.py:756}}
Nov 29 03:38:15 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:38:15 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:473}}
Nov 29 03:38:15 user nova-compute[75234]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Nov 29 03:38:15 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[8f3ad0ba-ba6d-4752-bd9a-7bf8d632f371]: (4, '') {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:38:15 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.lightos [None req-fe57873b-20a8-4d38-a798-90a5f865ee8c tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=75234) find_dsc /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/lightos.py:98}}
Nov 29 03:38:15 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.lightos [None req-fe57873b-20a8-4d38-a798-90a5f865ee8c tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=75234) get_connector_properties /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/lightos.py:76}}
Nov 29 03:38:15 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.lightos [None req-fe57873b-20a8-4d38-a798-90a5f865ee8c tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] LIGHTOS: no hostnqn found. {{(pid=75234) get_connector_properties /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/lightos.py:84}}
Nov 29 03:38:15 user nova-compute[75234]: DEBUG os_brick.utils [None req-fe57873b-20a8-4d38-a798-90a5f865ee8c tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] <== get_connector_properties: return (123ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:dd1368c5ba2c', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:203}}
Nov 29 03:38:15 user nova-compute[75234]: DEBUG nova.virt.block_device [None req-fe57873b-20a8-4d38-a798-90a5f865ee8c tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: 45059fbb-59d4-49ae-bccd-356edea7f287] Updating existing volume attachment record: 66a0fd9f-eba4-43c5-8d4d-596562e09e8f {{(pid=75234) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Nov 29 03:38:16 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:38:16 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=75234) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10541}}
Nov 29 03:38:16 user nova-compute[75234]: DEBUG nova.network.neutron [None req-fe57873b-20a8-4d38-a798-90a5f865ee8c tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: 45059fbb-59d4-49ae-bccd-356edea7f287] Successfully created port: 420b9e5e-a22f-4e87-90d4-9b4430c66c4e {{(pid=75234) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:548}}
Nov 29 03:38:16 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:38:17 user nova-compute[75234]: DEBUG nova.network.neutron [None req-fe57873b-20a8-4d38-a798-90a5f865ee8c tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: 45059fbb-59d4-49ae-bccd-356edea7f287] Successfully updated port: 420b9e5e-a22f-4e87-90d4-9b4430c66c4e {{(pid=75234) _update_port /opt/stack/nova/nova/network/neutron.py:586}}
Nov 29 03:38:17 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-fe57873b-20a8-4d38-a798-90a5f865ee8c tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Acquiring lock "refresh_cache-45059fbb-59d4-49ae-bccd-356edea7f287" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:38:17 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-fe57873b-20a8-4d38-a798-90a5f865ee8c tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Acquired lock "refresh_cache-45059fbb-59d4-49ae-bccd-356edea7f287" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:38:17 user nova-compute[75234]: DEBUG nova.network.neutron [None req-fe57873b-20a8-4d38-a798-90a5f865ee8c tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: 45059fbb-59d4-49ae-bccd-356edea7f287] Building network info cache for instance {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2009}}
Nov 29 03:38:17 user nova-compute[75234]: DEBUG nova.compute.manager [req-6f1465e2-4796-462a-92fe-c73cd4a86ac4 req-7a75f7ab-0aa0-4eda-b309-145e0170b22f service nova] [instance: 45059fbb-59d4-49ae-bccd-356edea7f287] Received event network-changed-420b9e5e-a22f-4e87-90d4-9b4430c66c4e {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:38:17 user nova-compute[75234]: DEBUG nova.compute.manager [req-6f1465e2-4796-462a-92fe-c73cd4a86ac4 req-7a75f7ab-0aa0-4eda-b309-145e0170b22f service nova] [instance: 45059fbb-59d4-49ae-bccd-356edea7f287] Refreshing instance network info cache due to event network-changed-420b9e5e-a22f-4e87-90d4-9b4430c66c4e. {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11117}}
Nov 29 03:38:17 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-6f1465e2-4796-462a-92fe-c73cd4a86ac4 req-7a75f7ab-0aa0-4eda-b309-145e0170b22f service nova] Acquiring lock "refresh_cache-45059fbb-59d4-49ae-bccd-356edea7f287" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:38:17 user nova-compute[75234]: DEBUG nova.network.neutron [None req-fe57873b-20a8-4d38-a798-90a5f865ee8c tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: 45059fbb-59d4-49ae-bccd-356edea7f287] Instance cache missing network info. {{(pid=75234) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3322}}
Nov 29 03:38:17 user nova-compute[75234]: DEBUG nova.network.neutron [None req-fe57873b-20a8-4d38-a798-90a5f865ee8c tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: 45059fbb-59d4-49ae-bccd-356edea7f287] Updating instance_info_cache with network_info: [{"id": "420b9e5e-a22f-4e87-90d4-9b4430c66c4e", "address": "fa:16:3e:48:2e:bc", "network": {"id": "a3f397d3-fd75-4a6e-a815-cfcedcca15fb", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-678570085-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "81a265d3c15b4e9d826b35ddbd0ff443", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap420b9e5e-a2", "ovs_interfaceid": "420b9e5e-a22f-4e87-90d4-9b4430c66c4e", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:38:17 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-fe57873b-20a8-4d38-a798-90a5f865ee8c tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Releasing lock "refresh_cache-45059fbb-59d4-49ae-bccd-356edea7f287" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:38:17 user nova-compute[75234]: DEBUG nova.compute.manager [None req-fe57873b-20a8-4d38-a798-90a5f865ee8c tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: 45059fbb-59d4-49ae-bccd-356edea7f287] Instance network_info: |[{"id": "420b9e5e-a22f-4e87-90d4-9b4430c66c4e", "address": "fa:16:3e:48:2e:bc", "network": {"id": "a3f397d3-fd75-4a6e-a815-cfcedcca15fb", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-678570085-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "81a265d3c15b4e9d826b35ddbd0ff443", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap420b9e5e-a2", "ovs_interfaceid": "420b9e5e-a22f-4e87-90d4-9b4430c66c4e", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=75234) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1987}}
Nov 29 03:38:17 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-6f1465e2-4796-462a-92fe-c73cd4a86ac4 req-7a75f7ab-0aa0-4eda-b309-145e0170b22f service nova] Acquired lock "refresh_cache-45059fbb-59d4-49ae-bccd-356edea7f287" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:38:17 user nova-compute[75234]: DEBUG nova.network.neutron [req-6f1465e2-4796-462a-92fe-c73cd4a86ac4 req-7a75f7ab-0aa0-4eda-b309-145e0170b22f service nova] [instance: 45059fbb-59d4-49ae-bccd-356edea7f287] Refreshing network info cache for port 420b9e5e-a22f-4e87-90d4-9b4430c66c4e {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2006}}
Nov 29 03:38:17 user nova-compute[75234]: DEBUG nova.network.neutron [req-6f1465e2-4796-462a-92fe-c73cd4a86ac4 req-7a75f7ab-0aa0-4eda-b309-145e0170b22f service nova] [instance: 45059fbb-59d4-49ae-bccd-356edea7f287] Updated VIF entry in instance network info cache for port 420b9e5e-a22f-4e87-90d4-9b4430c66c4e. {{(pid=75234) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3481}}
Nov 29 03:38:17 user nova-compute[75234]: DEBUG nova.network.neutron [req-6f1465e2-4796-462a-92fe-c73cd4a86ac4 req-7a75f7ab-0aa0-4eda-b309-145e0170b22f service nova] [instance: 45059fbb-59d4-49ae-bccd-356edea7f287] Updating instance_info_cache with network_info: [{"id": "420b9e5e-a22f-4e87-90d4-9b4430c66c4e", "address": "fa:16:3e:48:2e:bc", "network": {"id": "a3f397d3-fd75-4a6e-a815-cfcedcca15fb", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-678570085-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "81a265d3c15b4e9d826b35ddbd0ff443", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap420b9e5e-a2", "ovs_interfaceid": "420b9e5e-a22f-4e87-90d4-9b4430c66c4e", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:38:17 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-6f1465e2-4796-462a-92fe-c73cd4a86ac4 req-7a75f7ab-0aa0-4eda-b309-145e0170b22f service nova] Releasing lock "refresh_cache-45059fbb-59d4-49ae-bccd-356edea7f287" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:38:18 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:38:18 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager.update_available_resource {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:38:18 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:38:18 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:38:18 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:38:18 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Auditing locally available compute resources for user (node: user) {{(pid=75234) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:907}}
Nov 29 03:38:18 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/65d48c93-b6da-4f51-9d5a-0676dda62a60/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:38:18 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/65d48c93-b6da-4f51-9d5a-0676dda62a60/disk --force-share --output=json" returned: 0 in 0.172s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:38:18 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/65d48c93-b6da-4f51-9d5a-0676dda62a60/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:38:18 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/65d48c93-b6da-4f51-9d5a-0676dda62a60/disk --force-share --output=json" returned: 0 in 0.173s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:38:18 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] skipping disk /dev/sdc (vda) as it is a volume {{(pid=75234) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11213}}
Nov 29 03:38:18 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/abb01889-df4b-44a6-9827-87056d7ca125/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:38:19 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/abb01889-df4b-44a6-9827-87056d7ca125/disk --force-share --output=json" returned: 0 in 0.173s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:38:19 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/abb01889-df4b-44a6-9827-87056d7ca125/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:38:19 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/abb01889-df4b-44a6-9827-87056d7ca125/disk --force-share --output=json" returned: 0 in 0.172s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:38:19 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:38:19 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:38:19 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Hypervisor/Node resource view: name=user free_ram=8854MB free_disk=24.918724060058594GB free_vcpus=9 pci_devices=[{"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}] {{(pid=75234) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1106}}
Nov 29 03:38:19 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:38:19 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:38:19 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance 65d48c93-b6da-4f51-9d5a-0676dda62a60 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:38:19 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance d5096ca2-d005-45ac-a487-11a810446859 actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:38:19 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance abb01889-df4b-44a6-9827-87056d7ca125 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:38:19 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance 45059fbb-59d4-49ae-bccd-356edea7f287 actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:38:19 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Total usable vcpus: 12, total allocated vcpus: 4 {{(pid=75234) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1129}}
Nov 29 03:38:19 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Final resource view: name=user phys_ram=16011MB used_ram=2560MB phys_disk=40GB used_disk=2GB total_vcpus=12 used_vcpus=4 pci_stats=[] {{(pid=75234) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1138}}
Nov 29 03:38:19 user nova-compute[75234]: DEBUG nova.scheduler.client.report [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Refreshing inventories for resource provider 4d291220-7c37-4839-a0e3-b23c66502b74 {{(pid=75234) _refresh_associations /opt/stack/nova/nova/scheduler/client/report.py:804}}
Nov 29 03:38:19 user nova-compute[75234]: DEBUG nova.scheduler.client.report [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Updating ProviderTree inventory for provider 4d291220-7c37-4839-a0e3-b23c66502b74 from _refresh_and_get_inventory using data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=75234) _refresh_and_get_inventory /opt/stack/nova/nova/scheduler/client/report.py:768}}
Nov 29 03:38:19 user nova-compute[75234]: DEBUG nova.compute.provider_tree [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Updating inventory in ProviderTree for provider 4d291220-7c37-4839-a0e3-b23c66502b74 with inventory: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=75234) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:176}}
Nov 29 03:38:19 user nova-compute[75234]: DEBUG nova.scheduler.client.report [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Refreshing aggregate associations for resource provider 4d291220-7c37-4839-a0e3-b23c66502b74, aggregates: None {{(pid=75234) _refresh_associations /opt/stack/nova/nova/scheduler/client/report.py:813}}
Nov 29 03:38:19 user nova-compute[75234]: DEBUG nova.scheduler.client.report [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Refreshing trait associations for resource provider 4d291220-7c37-4839-a0e3-b23c66502b74, traits: COMPUTE_IMAGE_TYPE_AMI,COMPUTE_GRAPHICS_MODEL_CIRRUS,COMPUTE_NET_VIF_MODEL_VMXNET3,COMPUTE_STORAGE_BUS_VIRTIO,COMPUTE_GRAPHICS_MODEL_NONE,COMPUTE_IMAGE_TYPE_ISO,COMPUTE_NET_VIF_MODEL_RTL8139,HW_CPU_X86_SSE,COMPUTE_IMAGE_TYPE_QCOW2,COMPUTE_SOCKET_PCI_NUMA_AFFINITY,COMPUTE_GRAPHICS_MODEL_BOCHS,COMPUTE_VOLUME_ATTACH_WITH_TAG,COMPUTE_ACCELERATORS,HW_CPU_X86_SSE2,COMPUTE_STORAGE_BUS_SCSI,COMPUTE_GRAPHICS_MODEL_VGA,COMPUTE_NET_ATTACH_INTERFACE_WITH_TAG,COMPUTE_SECURITY_UEFI_SECURE_BOOT,HW_CPU_X86_SSE41,COMPUTE_NET_VIF_MODEL_E1000E,COMPUTE_TRUSTED_CERTS,COMPUTE_STORAGE_BUS_FDC,COMPUTE_VIOMMU_MODEL_INTEL,COMPUTE_GRAPHICS_MODEL_VIRTIO,COMPUTE_VIOMMU_MODEL_AUTO,COMPUTE_IMAGE_TYPE_AKI,COMPUTE_NET_VIF_MODEL_PCNET,COMPUTE_GRAPHICS_MODEL_QXL,COMPUTE_NET_VIF_MODEL_VIRTIO,COMPUTE_GRAPHICS_MODEL_VMVGA,COMPUTE_VOLUME_MULTI_ATTACH,COMPUTE_STORAGE_BUS_SATA,HW_CPU_X86_MMX,COMPUTE_NET_VIF_MODEL_E1000,COMPUTE_IMAGE_TYPE_ARI,COMPUTE_STORAGE_BUS_IDE,COMPUTE_NODE,HW_CPU_X86_SSSE3,COMPUTE_RESCUE_BFV,COMPUTE_NET_ATTACH_INTERFACE,COMPUTE_STORAGE_BUS_USB,COMPUTE_NET_VIF_MODEL_SPAPR_VLAN,COMPUTE_IMAGE_TYPE_RAW,COMPUTE_NET_VIF_MODEL_NE2K_PCI,COMPUTE_VOLUME_EXTEND,COMPUTE_DEVICE_TAGGING,HW_CPU_X86_SSE42 {{(pid=75234) _refresh_associations /opt/stack/nova/nova/scheduler/client/report.py:825}}
Nov 29 03:38:20 user nova-compute[75234]: DEBUG nova.compute.provider_tree [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Inventory has not changed in ProviderTree for provider: 4d291220-7c37-4839-a0e3-b23c66502b74 {{(pid=75234) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Nov 29 03:38:20 user nova-compute[75234]: DEBUG nova.scheduler.client.report [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Inventory has not changed for provider 4d291220-7c37-4839-a0e3-b23c66502b74 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=75234) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Nov 29 03:38:20 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Compute_service record updated for user:user {{(pid=75234) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:1067}}
Nov 29 03:38:20 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.390s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:38:21 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:38:22 user nova-compute[75234]: DEBUG nova.compute.manager [None req-fe57873b-20a8-4d38-a798-90a5f865ee8c tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: 45059fbb-59d4-49ae-bccd-356edea7f287] Start spawning the instance on the hypervisor. {{(pid=75234) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2628}}
Nov 29 03:38:22 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-fe57873b-20a8-4d38-a798-90a5f865ee8c tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: 45059fbb-59d4-49ae-bccd-356edea7f287] Creating instance directory {{(pid=75234) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4727}}
Nov 29 03:38:22 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-fe57873b-20a8-4d38-a798-90a5f865ee8c tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: 45059fbb-59d4-49ae-bccd-356edea7f287] Creating image(s)
Nov 29 03:38:22 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-fe57873b-20a8-4d38-a798-90a5f865ee8c tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: 45059fbb-59d4-49ae-bccd-356edea7f287] Did not create local disks {{(pid=75234) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4863}}
Nov 29 03:38:22 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-fe57873b-20a8-4d38-a798-90a5f865ee8c tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: 45059fbb-59d4-49ae-bccd-356edea7f287] Ensure instance console log exists: /opt/stack/data/nova/instances/45059fbb-59d4-49ae-bccd-356edea7f287/console.log {{(pid=75234) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4613}}
Nov 29 03:38:22 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-fe57873b-20a8-4d38-a798-90a5f865ee8c tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:38:22 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-fe57873b-20a8-4d38-a798-90a5f865ee8c tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:38:22 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-fe57873b-20a8-4d38-a798-90a5f865ee8c tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:38:22 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-fe57873b-20a8-4d38-a798-90a5f865ee8c tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: 45059fbb-59d4-49ae-bccd-356edea7f287] Start _get_guest_xml network_info=[{"id": "420b9e5e-a22f-4e87-90d4-9b4430c66c4e", "address": "fa:16:3e:48:2e:bc", "network": {"id": "a3f397d3-fd75-4a6e-a815-cfcedcca15fb", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-678570085-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "81a265d3c15b4e9d826b35ddbd0ff443", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap420b9e5e-a2", "ovs_interfaceid": "420b9e5e-a22f-4e87-90d4-9b4430c66c4e", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, '/dev/vda': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum=<?>,container_format=<?>,created_at=<?>,direct_url=<?>,disk_format=<?>,id=<?>,min_disk=0,min_ram=0,name=<?>,owner=<?>,properties=ImageMetaProps,protected=<?>,size=1073741824,status='active',tags=<?>,updated_at=<?>,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [], 'ephemerals': [], 'block_device_mapping': [{'mount_device': '/dev/vda', 'attachment_id': '66a0fd9f-eba4-43c5-8d4d-596562e09e8f', 'disk_bus': 'virtio', 'device_type': 'disk', 'boot_index': 0, 'guest_format': None, 'delete_on_termination': False, 'connection_info': {'driver_volume_type': 'iscsi', 'data': {'target_iqn': 'iqn.2010-10.org.openstack:378addbb-8d90-40c5-ae61-c3a063116df9', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'cyOomDWj', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}, 'status': 'reserved', 'instance': '45059fbb-59d4-49ae-bccd-356edea7f287', 'attached_at': '', 'detached_at': '', 'volume_id': '378addbb-8d90-40c5-ae61-c3a063116df9', 'serial': '378addbb-8d90-40c5-ae61-c3a063116df9'}, 'volume_type': None} {{(pid=75234) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7562}}
Nov 29 03:38:22 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-fe57873b-20a8-4d38-a798-90a5f865ee8c tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:38:22 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-fe57873b-20a8-4d38-a798-90a5f865ee8c tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:38:22 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-fe57873b-20a8-4d38-a798-90a5f865ee8c tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Searching host: 'user' for CPU controller through CGroups V1... {{(pid=75234) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1652}}
Nov 29 03:38:22 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-fe57873b-20a8-4d38-a798-90a5f865ee8c tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] CPU controller missing on host. {{(pid=75234) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1662}}
Nov 29 03:38:22 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-fe57873b-20a8-4d38-a798-90a5f865ee8c tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Searching host: 'user' for CPU controller through CGroups V2... {{(pid=75234) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1671}}
Nov 29 03:38:22 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-fe57873b-20a8-4d38-a798-90a5f865ee8c tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] CPU controller found on host. {{(pid=75234) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1678}}
Nov 29 03:38:22 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-fe57873b-20a8-4d38-a798-90a5f865ee8c tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=75234) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5400}}
Nov 29 03:38:22 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-fe57873b-20a8-4d38-a798-90a5f865ee8c tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-11-29T03:02:04Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='1',id=6,is_public=True,memory_mb=512,name='m1.tiny',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum=<?>,container_format=<?>,created_at=<?>,direct_url=<?>,disk_format=<?>,id=<?>,min_disk=0,min_ram=0,name=<?>,owner=<?>,properties=ImageMetaProps,protected=<?>,size=1073741824,status='active',tags=<?>,updated_at=<?>,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:563}}
Nov 29 03:38:22 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-fe57873b-20a8-4d38-a798-90a5f865ee8c tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Flavor limits 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:348}}
Nov 29 03:38:22 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-fe57873b-20a8-4d38-a798-90a5f865ee8c tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Image limits 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:352}}
Nov 29 03:38:22 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-fe57873b-20a8-4d38-a798-90a5f865ee8c tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Flavor pref 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:388}}
Nov 29 03:38:22 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-fe57873b-20a8-4d38-a798-90a5f865ee8c tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Image pref 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:392}}
Nov 29 03:38:22 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-fe57873b-20a8-4d38-a798-90a5f865ee8c tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:430}}
Nov 29 03:38:22 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-fe57873b-20a8-4d38-a798-90a5f865ee8c tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:569}}
Nov 29 03:38:22 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-fe57873b-20a8-4d38-a798-90a5f865ee8c tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=75234) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:471}}
Nov 29 03:38:22 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-fe57873b-20a8-4d38-a798-90a5f865ee8c tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Got 1 possible topologies {{(pid=75234) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:501}}
Nov 29 03:38:22 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-fe57873b-20a8-4d38-a798-90a5f865ee8c tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:575}}
Nov 29 03:38:22 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-fe57873b-20a8-4d38-a798-90a5f865ee8c tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:577}}
Nov 29 03:38:22 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-fe57873b-20a8-4d38-a798-90a5f865ee8c tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: 45059fbb-59d4-49ae-bccd-356edea7f287] Calling os-brick to attach iSCSI Volume {{(pid=75234) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Nov 29 03:38:22 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-fe57873b-20a8-4d38-a798-90a5f865ee8c tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f8a5050c9a0>, 'connection_properties': {'target_iqn': 'iqn.2010-10.org.openstack:378addbb-8d90-40c5-ae61-c3a063116df9', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'cyOomDWj', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:176}}
Nov 29 03:38:22 user nova-compute[75234]: WARNING os_brick.initiator.connectors.base [None req-fe57873b-20a8-4d38-a798-90a5f865ee8c tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Nov 29 03:38:22 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-fe57873b-20a8-4d38-a798-90a5f865ee8c tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:68}}
Nov 29 03:38:22 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-fe57873b-20a8-4d38-a798-90a5f865ee8c tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:73}}
Nov 29 03:38:22 user nova-compute[75234]: WARNING os_brick.initiator.connectors.base [None req-fe57873b-20a8-4d38-a798-90a5f865ee8c tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Nov 29 03:38:22 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-fe57873b-20a8-4d38-a798-90a5f865ee8c tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:378addbb-8d90-40c5-ae61-c3a063116df9" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:68}}
Nov 29 03:38:22 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-fe57873b-20a8-4d38-a798-90a5f865ee8c tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:378addbb-8d90-40c5-ae61-c3a063116df9" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:73}}
Nov 29 03:38:22 user nova-compute[75234]: INFO os_brick.initiator.connectors.iscsi [None req-fe57873b-20a8-4d38-a798-90a5f865ee8c tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Nov 29 03:38:22 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:378addbb-8d90-40c5-ae61-c3a063116df9 -p 172.16.0.220:3260 {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:38:22 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:378addbb-8d90-40c5-ae61-c3a063116df9 -p 172.16.0.220:3260" returned: 21 in 0.023s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:38:22 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[76cce915-1421-4229-80cf-98383ad787d2]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:38:22 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-fe57873b-20a8-4d38-a798-90a5f865ee8c tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Nov 29 03:38:22 user nova-compute[75234]:  {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:38:22 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:378addbb-8d90-40c5-ae61-c3a063116df9 -p 172.16.0.220:3260 --interface default --op new {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:38:22 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:378addbb-8d90-40c5-ae61-c3a063116df9 -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.018s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:38:22 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[27a71e8c-d3c2-4375-be98-d1b3efd0282b]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:378addbb-8d90-40c5-ae61-c3a063116df9] added\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:38:22 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-fe57873b-20a8-4d38-a798-90a5f865ee8c tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:378addbb-8d90-40c5-ae61-c3a063116df9] added
Nov 29 03:38:22 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:38:22 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:378addbb-8d90-40c5-ae61-c3a063116df9 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:38:22 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:378addbb-8d90-40c5-ae61-c3a063116df9 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.018s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:38:22 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[ecdbd1da-64c2-419e-ba3a-991a76010c72]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:38:22 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-fe57873b-20a8-4d38-a798-90a5f865ee8c tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:38:22 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:378addbb-8d90-40c5-ae61-c3a063116df9 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:38:22 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:378addbb-8d90-40c5-ae61-c3a063116df9 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.024s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:38:22 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[a48abc72-3281-4fb6-86e4-5ab2bfe2e22a]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:38:22 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-fe57873b-20a8-4d38-a798-90a5f865ee8c tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:38:22 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:378addbb-8d90-40c5-ae61-c3a063116df9 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v cyOomDWj {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:38:22 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:378addbb-8d90-40c5-ae61-c3a063116df9 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v cyOomDWj" returned: 0 in 0.018s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:38:22 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[1410bb14-1bfc-4180-850e-a733372d848d]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:38:22 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-fe57873b-20a8-4d38-a798-90a5f865ee8c tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'cyOomDWj'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:38:22 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:378addbb-8d90-40c5-ae61-c3a063116df9 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:38:22 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:378addbb-8d90-40c5-ae61-c3a063116df9 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.018s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:38:22 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[11e4f029-3625-42cc-8f96-80d6fb9b2cd5]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:38:22 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-fe57873b-20a8-4d38-a798-90a5f865ee8c tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:38:22 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:38:22 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.022s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:38:22 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[22dd5ef5-468f-44b6-b0ca-92868621a76e]: (4, ('tcp: [42] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0 (non-flash)\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:38:22 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-fe57873b-20a8-4d38-a798-90a5f865ee8c tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [42] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0 (non-flash)
Nov 29 03:38:22 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:38:22 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-fe57873b-20a8-4d38-a798-90a5f865ee8c tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] iscsi session list stdout=tcp: [42] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0 (non-flash)
Nov 29 03:38:22 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsi_session /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Nov 29 03:38:22 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:378addbb-8d90-40c5-ae61-c3a063116df9 -p 172.16.0.220:3260 --login {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:38:22 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:378addbb-8d90-40c5-ae61-c3a063116df9 -p 172.16.0.220:3260 --login" returned: 0 in 0.048s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:38:22 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[289f75d8-f778-4ef0-b54d-7a26c278c0c5]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:378addbb-8d90-40c5-ae61-c3a063116df9, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:378addbb-8d90-40c5-ae61-c3a063116df9, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:38:22 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-fe57873b-20a8-4d38-a798-90a5f865ee8c tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:378addbb-8d90-40c5-ae61-c3a063116df9, portal: 172.16.0.220,3260]
Nov 29 03:38:22 user nova-compute[75234]: Login to [iface: default, target: iqn.2010-10.org.openstack:378addbb-8d90-40c5-ae61-c3a063116df9, portal: 172.16.0.220,3260] successful.
Nov 29 03:38:22 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:38:22 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:378addbb-8d90-40c5-ae61-c3a063116df9 -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:38:22 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:378addbb-8d90-40c5-ae61-c3a063116df9 -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.019s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:38:22 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[e382f5df-7da1-487d-bbee-51c9cb3d7681]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:38:22 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-fe57873b-20a8-4d38-a798-90a5f865ee8c tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:38:22 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:38:22 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.021s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:38:22 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[1a6e3eba-cc20-4d45-ba9a-a112381e17d9]: (4, ('tcp: [42] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0 (non-flash)\ntcp: [58] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:378addbb-8d90-40c5-ae61-c3a063116df9 (non-flash)\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:38:22 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-fe57873b-20a8-4d38-a798-90a5f865ee8c tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [42] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0 (non-flash)
Nov 29 03:38:22 user nova-compute[75234]: tcp: [58] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:378addbb-8d90-40c5-ae61-c3a063116df9 (non-flash)
Nov 29 03:38:22 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:38:22 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-fe57873b-20a8-4d38-a798-90a5f865ee8c tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] iscsi session list stdout=tcp: [42] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0 (non-flash)
Nov 29 03:38:22 user nova-compute[75234]: tcp: [58] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:378addbb-8d90-40c5-ae61-c3a063116df9 (non-flash)
Nov 29 03:38:22 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsi_session /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Nov 29 03:38:22 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-fe57873b-20a8-4d38-a798-90a5f865ee8c tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:378addbb-8d90-40c5-ae61-c3a063116df9" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.269s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:87}}
Nov 29 03:38:22 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-fe57873b-20a8-4d38-a798-90a5f865ee8c tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Connected to 172.16.0.220:3260 {{(pid=75234) _connect_vol /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:644}}
Nov 29 03:38:22 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-fe57873b-20a8-4d38-a798-90a5f865ee8c tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] HCTL ('33', '-', '-', 0) found on session 58 with lun 0 {{(pid=75234) get_hctl /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:770}}
Nov 29 03:38:22 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-fe57873b-20a8-4d38-a798-90a5f865ee8c tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Scanning host 33 c: -, t: -, l: 0) {{(pid=75234) scan_iscsi /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:795}}
Nov 29 03:38:22 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host33/scan {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:38:22 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host33/scan" returned: 0 in 0.018s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:38:22 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[0a782055-5db1-4524-be0e-6159d390c925]: (4, ('- - 0', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:38:22 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-fe57873b-20a8-4d38-a798-90a5f865ee8c tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Searching for a device in session 58 and hctl ['33', '*', '*', 0] yield: None {{(pid=75234) device_name_by_hctl /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:789}}
Nov 29 03:38:23 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-fe57873b-20a8-4d38-a798-90a5f865ee8c tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Searching for a device in session 58 and hctl ['33', '*', '*', 0] yield: sdb {{(pid=75234) device_name_by_hctl /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:789}}
Nov 29 03:38:23 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-fe57873b-20a8-4d38-a798-90a5f865ee8c tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Connected to sdb using {'target_iqn': 'iqn.2010-10.org.openstack:378addbb-8d90-40c5-ae61-c3a063116df9', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'cyOomDWj', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False} {{(pid=75234) _connect_vol /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:672}}
Nov 29 03:38:23 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-fe57873b-20a8-4d38-a798-90a5f865ee8c tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.300s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:87}}
Nov 29 03:38:23 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-fe57873b-20a8-4d38-a798-90a5f865ee8c tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] <== connect_volume: return (1301ms) {'type': 'block', 'scsi_wwn': '23832366162623035', 'path': '/dev/sdb'} {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:203}}
Nov 29 03:38:23 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-fe57873b-20a8-4d38-a798-90a5f865ee8c tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: 45059fbb-59d4-49ae-bccd-356edea7f287] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '23832366162623035', 'path': '/dev/sdb'} {{(pid=75234) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Nov 29 03:38:23 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-fe57873b-20a8-4d38-a798-90a5f865ee8c tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-11-29T03:38:14Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-TestVolumeBootPattern-server-1788415920',display_name='tempest-TestVolumeBootPattern-server-1788415920',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-testvolumebootpattern-server-1788415920',id=50,image_ref='',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBKrnAt22NMoBNIFsnCF3Mxk+dIbrL4YaXFADzEahC/3pON2UrJ4NOPzv0Ze4QWq/A4Bm4h/5Mn2lQRP6Y0eXl2r9TPWd6a1CQjrsTRMK7MqwF4n4QkuDxmXBPaNWL5XvpA==',key_name='tempest-TestVolumeBootPattern-1253305043',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='81a265d3c15b4e9d826b35ddbd0ff443',ramdisk_id='',reservation_id='r-p6i390s0',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TestVolumeBootPattern-1244492388',owner_user_name='tempest-TestVolumeBootPattern-1244492388-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-11-29T03:38:15Z,user_data=None,user_id='7d1f1f11a9aa4c1db8627413eb963fa4',uuid=45059fbb-59d4-49ae-bccd-356edea7f287,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "420b9e5e-a22f-4e87-90d4-9b4430c66c4e", "address": "fa:16:3e:48:2e:bc", "network": {"id": "a3f397d3-fd75-4a6e-a815-cfcedcca15fb", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-678570085-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "81a265d3c15b4e9d826b35ddbd0ff443", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap420b9e5e-a2", "ovs_interfaceid": "420b9e5e-a22f-4e87-90d4-9b4430c66c4e", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=75234) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Nov 29 03:38:23 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-fe57873b-20a8-4d38-a798-90a5f865ee8c tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Converting VIF {"id": "420b9e5e-a22f-4e87-90d4-9b4430c66c4e", "address": "fa:16:3e:48:2e:bc", "network": {"id": "a3f397d3-fd75-4a6e-a815-cfcedcca15fb", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-678570085-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "81a265d3c15b4e9d826b35ddbd0ff443", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap420b9e5e-a2", "ovs_interfaceid": "420b9e5e-a22f-4e87-90d4-9b4430c66c4e", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:38:23 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-fe57873b-20a8-4d38-a798-90a5f865ee8c tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:48:2e:bc,bridge_name='br-int',has_traffic_filtering=True,id=420b9e5e-a22f-4e87-90d4-9b4430c66c4e,network=Network(a3f397d3-fd75-4a6e-a815-cfcedcca15fb),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap420b9e5e-a2') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:38:23 user nova-compute[75234]: DEBUG nova.objects.instance [None req-fe57873b-20a8-4d38-a798-90a5f865ee8c tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Lazy-loading 'pci_devices' on Instance uuid 45059fbb-59d4-49ae-bccd-356edea7f287 {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:38:23 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-fe57873b-20a8-4d38-a798-90a5f865ee8c tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: 45059fbb-59d4-49ae-bccd-356edea7f287] End _get_guest_xml xml=<domain type="kvm">
Nov 29 03:38:23 user nova-compute[75234]:   <uuid>45059fbb-59d4-49ae-bccd-356edea7f287</uuid>
Nov 29 03:38:23 user nova-compute[75234]:   <name>instance-00000032</name>
Nov 29 03:38:23 user nova-compute[75234]:   <memory>524288</memory>
Nov 29 03:38:23 user nova-compute[75234]:   <vcpu>1</vcpu>
Nov 29 03:38:23 user nova-compute[75234]:   <metadata>
Nov 29 03:38:23 user nova-compute[75234]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Nov 29 03:38:23 user nova-compute[75234]:       <nova:package version="0.0.0"/>
Nov 29 03:38:23 user nova-compute[75234]:       <nova:name>tempest-TestVolumeBootPattern-server-1788415920</nova:name>
Nov 29 03:38:23 user nova-compute[75234]:       <nova:creationTime>2023-11-29 03:38:22</nova:creationTime>
Nov 29 03:38:23 user nova-compute[75234]:       <nova:flavor name="m1.tiny">
Nov 29 03:38:23 user nova-compute[75234]:         <nova:memory>512</nova:memory>
Nov 29 03:38:23 user nova-compute[75234]:         <nova:disk>1</nova:disk>
Nov 29 03:38:23 user nova-compute[75234]:         <nova:swap>0</nova:swap>
Nov 29 03:38:23 user nova-compute[75234]:         <nova:ephemeral>0</nova:ephemeral>
Nov 29 03:38:23 user nova-compute[75234]:         <nova:vcpus>1</nova:vcpus>
Nov 29 03:38:23 user nova-compute[75234]:       </nova:flavor>
Nov 29 03:38:23 user nova-compute[75234]:       <nova:owner>
Nov 29 03:38:23 user nova-compute[75234]:         <nova:user uuid="7d1f1f11a9aa4c1db8627413eb963fa4">tempest-TestVolumeBootPattern-1244492388-project-member</nova:user>
Nov 29 03:38:23 user nova-compute[75234]:         <nova:project uuid="81a265d3c15b4e9d826b35ddbd0ff443">tempest-TestVolumeBootPattern-1244492388</nova:project>
Nov 29 03:38:23 user nova-compute[75234]:       </nova:owner>
Nov 29 03:38:23 user nova-compute[75234]:       <nova:ports>
Nov 29 03:38:23 user nova-compute[75234]:         <nova:port uuid="420b9e5e-a22f-4e87-90d4-9b4430c66c4e">
Nov 29 03:38:23 user nova-compute[75234]:           <nova:ip type="fixed" address="10.0.0.9" ipVersion="4"/>
Nov 29 03:38:23 user nova-compute[75234]:         </nova:port>
Nov 29 03:38:23 user nova-compute[75234]:       </nova:ports>
Nov 29 03:38:23 user nova-compute[75234]:     </nova:instance>
Nov 29 03:38:23 user nova-compute[75234]:   </metadata>
Nov 29 03:38:23 user nova-compute[75234]:   <sysinfo type="smbios">
Nov 29 03:38:23 user nova-compute[75234]:     <system>
Nov 29 03:38:23 user nova-compute[75234]:       <entry name="manufacturer">OpenStack Foundation</entry>
Nov 29 03:38:23 user nova-compute[75234]:       <entry name="product">OpenStack Nova</entry>
Nov 29 03:38:23 user nova-compute[75234]:       <entry name="version">0.0.0</entry>
Nov 29 03:38:23 user nova-compute[75234]:       <entry name="serial">45059fbb-59d4-49ae-bccd-356edea7f287</entry>
Nov 29 03:38:23 user nova-compute[75234]:       <entry name="uuid">45059fbb-59d4-49ae-bccd-356edea7f287</entry>
Nov 29 03:38:23 user nova-compute[75234]:       <entry name="family">Virtual Machine</entry>
Nov 29 03:38:23 user nova-compute[75234]:     </system>
Nov 29 03:38:23 user nova-compute[75234]:   </sysinfo>
Nov 29 03:38:23 user nova-compute[75234]:   <os>
Nov 29 03:38:23 user nova-compute[75234]:     <type arch="x86_64" machine="pc">hvm</type>
Nov 29 03:38:23 user nova-compute[75234]:     <boot dev="hd"/>
Nov 29 03:38:23 user nova-compute[75234]:     <smbios mode="sysinfo"/>
Nov 29 03:38:23 user nova-compute[75234]:   </os>
Nov 29 03:38:23 user nova-compute[75234]:   <features>
Nov 29 03:38:23 user nova-compute[75234]:     <acpi/>
Nov 29 03:38:23 user nova-compute[75234]:     <apic/>
Nov 29 03:38:23 user nova-compute[75234]:     <vmcoreinfo/>
Nov 29 03:38:23 user nova-compute[75234]:   </features>
Nov 29 03:38:23 user nova-compute[75234]:   <clock offset="utc">
Nov 29 03:38:23 user nova-compute[75234]:     <timer name="pit" tickpolicy="delay"/>
Nov 29 03:38:23 user nova-compute[75234]:     <timer name="rtc" tickpolicy="catchup"/>
Nov 29 03:38:23 user nova-compute[75234]:     <timer name="hpet" present="no"/>
Nov 29 03:38:23 user nova-compute[75234]:   </clock>
Nov 29 03:38:23 user nova-compute[75234]:   <cpu mode="custom" match="exact">
Nov 29 03:38:23 user nova-compute[75234]:     <model>Nehalem</model>
Nov 29 03:38:23 user nova-compute[75234]:     <topology sockets="1" cores="1" threads="1"/>
Nov 29 03:38:23 user nova-compute[75234]:   </cpu>
Nov 29 03:38:23 user nova-compute[75234]:   <devices>
Nov 29 03:38:23 user nova-compute[75234]:     <disk type="block" device="disk">
Nov 29 03:38:23 user nova-compute[75234]:       <driver name="qemu" type="raw" cache="none" io="native"/>
Nov 29 03:38:23 user nova-compute[75234]:       <source dev="/dev/sdb"/>
Nov 29 03:38:23 user nova-compute[75234]:       <target dev="vda" bus="virtio"/>
Nov 29 03:38:23 user nova-compute[75234]:       <serial>378addbb-8d90-40c5-ae61-c3a063116df9</serial>
Nov 29 03:38:23 user nova-compute[75234]:     </disk>
Nov 29 03:38:23 user nova-compute[75234]:     <interface type="ethernet">
Nov 29 03:38:23 user nova-compute[75234]:       <mac address="fa:16:3e:48:2e:bc"/>
Nov 29 03:38:23 user nova-compute[75234]:       <model type="virtio"/>
Nov 29 03:38:23 user nova-compute[75234]:       <mtu size="1442"/>
Nov 29 03:38:23 user nova-compute[75234]:       <target dev="tap420b9e5e-a2"/>
Nov 29 03:38:23 user nova-compute[75234]:     </interface>
Nov 29 03:38:23 user nova-compute[75234]:     <serial type="pty">
Nov 29 03:38:23 user nova-compute[75234]:       <log file="/opt/stack/data/nova/instances/45059fbb-59d4-49ae-bccd-356edea7f287/console.log" append="off"/>
Nov 29 03:38:23 user nova-compute[75234]:     </serial>
Nov 29 03:38:23 user nova-compute[75234]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Nov 29 03:38:23 user nova-compute[75234]:     <video>
Nov 29 03:38:23 user nova-compute[75234]:       <model type="virtio"/>
Nov 29 03:38:23 user nova-compute[75234]:     </video>
Nov 29 03:38:23 user nova-compute[75234]:     <rng model="virtio">
Nov 29 03:38:23 user nova-compute[75234]:       <backend model="random">/dev/urandom</backend>
Nov 29 03:38:23 user nova-compute[75234]:     </rng>
Nov 29 03:38:23 user nova-compute[75234]:     <controller type="usb" index="0" model="none"/>
Nov 29 03:38:23 user nova-compute[75234]:     <memballoon model="virtio">
Nov 29 03:38:23 user nova-compute[75234]:       <stats period="10"/>
Nov 29 03:38:23 user nova-compute[75234]:     </memballoon>
Nov 29 03:38:23 user nova-compute[75234]:   </devices>
Nov 29 03:38:23 user nova-compute[75234]: </domain>
Nov 29 03:38:23 user nova-compute[75234]:  {{(pid=75234) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7568}}
Nov 29 03:38:23 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-fe57873b-20a8-4d38-a798-90a5f865ee8c tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-11-29T03:38:14Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-TestVolumeBootPattern-server-1788415920',display_name='tempest-TestVolumeBootPattern-server-1788415920',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-testvolumebootpattern-server-1788415920',id=50,image_ref='',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBKrnAt22NMoBNIFsnCF3Mxk+dIbrL4YaXFADzEahC/3pON2UrJ4NOPzv0Ze4QWq/A4Bm4h/5Mn2lQRP6Y0eXl2r9TPWd6a1CQjrsTRMK7MqwF4n4QkuDxmXBPaNWL5XvpA==',key_name='tempest-TestVolumeBootPattern-1253305043',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='81a265d3c15b4e9d826b35ddbd0ff443',ramdisk_id='',reservation_id='r-p6i390s0',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TestVolumeBootPattern-1244492388',owner_user_name='tempest-TestVolumeBootPattern-1244492388-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-11-29T03:38:15Z,user_data=None,user_id='7d1f1f11a9aa4c1db8627413eb963fa4',uuid=45059fbb-59d4-49ae-bccd-356edea7f287,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "420b9e5e-a22f-4e87-90d4-9b4430c66c4e", "address": "fa:16:3e:48:2e:bc", "network": {"id": "a3f397d3-fd75-4a6e-a815-cfcedcca15fb", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-678570085-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "81a265d3c15b4e9d826b35ddbd0ff443", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap420b9e5e-a2", "ovs_interfaceid": "420b9e5e-a22f-4e87-90d4-9b4430c66c4e", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Nov 29 03:38:23 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-fe57873b-20a8-4d38-a798-90a5f865ee8c tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Converting VIF {"id": "420b9e5e-a22f-4e87-90d4-9b4430c66c4e", "address": "fa:16:3e:48:2e:bc", "network": {"id": "a3f397d3-fd75-4a6e-a815-cfcedcca15fb", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-678570085-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "81a265d3c15b4e9d826b35ddbd0ff443", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap420b9e5e-a2", "ovs_interfaceid": "420b9e5e-a22f-4e87-90d4-9b4430c66c4e", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:38:23 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-fe57873b-20a8-4d38-a798-90a5f865ee8c tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:48:2e:bc,bridge_name='br-int',has_traffic_filtering=True,id=420b9e5e-a22f-4e87-90d4-9b4430c66c4e,network=Network(a3f397d3-fd75-4a6e-a815-cfcedcca15fb),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap420b9e5e-a2') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:38:23 user nova-compute[75234]: DEBUG os_vif [None req-fe57873b-20a8-4d38-a798-90a5f865ee8c tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:48:2e:bc,bridge_name='br-int',has_traffic_filtering=True,id=420b9e5e-a22f-4e87-90d4-9b4430c66c4e,network=Network(a3f397d3-fd75-4a6e-a815-cfcedcca15fb),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap420b9e5e-a2') {{(pid=75234) plug /opt/stack/data/venv/lib/python3.10/site-packages/os_vif/__init__.py:76}}
Nov 29 03:38:23 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:38:23 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:38:23 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Nov 29 03:38:23 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:38:23 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbCreateCommand(_result=None, table=QoS, columns={'type': 'linux-noop', 'external_ids': {'id': 'cf0a9490-0d54-5333-8262-7382c298224c', '_type': 'linux-noop'}}, row=False) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:38:23 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:38:23 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:38:23 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tap420b9e5e-a2, may_exist=True, interface_attrs={}) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:38:23 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Port, record=tap420b9e5e-a2, col_values=(('qos', UUID('427c6127-9ed9-41f8-b889-4c58a4567abf')),), if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:38:23 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DbSetCommand(_result=None, table=Interface, record=tap420b9e5e-a2, col_values=(('external_ids', {'iface-id': '420b9e5e-a22f-4e87-90d4-9b4430c66c4e', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:48:2e:bc', 'vm-uuid': '45059fbb-59d4-49ae-bccd-356edea7f287'}),), if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:38:23 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:38:23 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:38:23 user nova-compute[75234]: INFO os_vif [None req-fe57873b-20a8-4d38-a798-90a5f865ee8c tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:48:2e:bc,bridge_name='br-int',has_traffic_filtering=True,id=420b9e5e-a22f-4e87-90d4-9b4430c66c4e,network=Network(a3f397d3-fd75-4a6e-a815-cfcedcca15fb),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap420b9e5e-a2')
Nov 29 03:38:24 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-fe57873b-20a8-4d38-a798-90a5f865ee8c tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] No BDM found with device name vda, not building metadata. {{(pid=75234) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12089}}
Nov 29 03:38:24 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-fe57873b-20a8-4d38-a798-90a5f865ee8c tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] No VIF found with MAC fa:16:3e:48:2e:bc, not building metadata {{(pid=75234) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Nov 29 03:38:25 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:38:25 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Starting heal instance info cache {{(pid=75234) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9922}}
Nov 29 03:38:25 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Acquiring lock "refresh_cache-abb01889-df4b-44a6-9827-87056d7ca125" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:38:25 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Acquired lock "refresh_cache-abb01889-df4b-44a6-9827-87056d7ca125" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:38:25 user nova-compute[75234]: DEBUG nova.network.neutron [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] [instance: abb01889-df4b-44a6-9827-87056d7ca125] Forcefully refreshing network info cache for instance {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2003}}
Nov 29 03:38:25 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:38:25 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:38:25 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:38:25 user nova-compute[75234]: DEBUG nova.compute.manager [req-97b9845f-9677-4606-bf66-21d9091d7bda req-04cf60f8-f236-4ac0-a816-f7a88e7f4d05 service nova] [instance: 45059fbb-59d4-49ae-bccd-356edea7f287] Received event network-vif-plugged-420b9e5e-a22f-4e87-90d4-9b4430c66c4e {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:38:25 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-97b9845f-9677-4606-bf66-21d9091d7bda req-04cf60f8-f236-4ac0-a816-f7a88e7f4d05 service nova] Acquiring lock "45059fbb-59d4-49ae-bccd-356edea7f287-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:38:25 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-97b9845f-9677-4606-bf66-21d9091d7bda req-04cf60f8-f236-4ac0-a816-f7a88e7f4d05 service nova] Lock "45059fbb-59d4-49ae-bccd-356edea7f287-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:38:25 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-97b9845f-9677-4606-bf66-21d9091d7bda req-04cf60f8-f236-4ac0-a816-f7a88e7f4d05 service nova] Lock "45059fbb-59d4-49ae-bccd-356edea7f287-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:38:25 user nova-compute[75234]: DEBUG nova.compute.manager [req-97b9845f-9677-4606-bf66-21d9091d7bda req-04cf60f8-f236-4ac0-a816-f7a88e7f4d05 service nova] [instance: 45059fbb-59d4-49ae-bccd-356edea7f287] No waiting events found dispatching network-vif-plugged-420b9e5e-a22f-4e87-90d4-9b4430c66c4e {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:38:25 user nova-compute[75234]: WARNING nova.compute.manager [req-97b9845f-9677-4606-bf66-21d9091d7bda req-04cf60f8-f236-4ac0-a816-f7a88e7f4d05 service nova] [instance: 45059fbb-59d4-49ae-bccd-356edea7f287] Received unexpected event network-vif-plugged-420b9e5e-a22f-4e87-90d4-9b4430c66c4e for instance with vm_state building and task_state spawning.
Nov 29 03:38:25 user nova-compute[75234]: DEBUG nova.network.neutron [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] [instance: abb01889-df4b-44a6-9827-87056d7ca125] Updating instance_info_cache with network_info: [{"id": "fec40d24-caf1-48f7-8565-8fffdbdad60e", "address": "fa:16:3e:e3:bd:f8", "network": {"id": "4d086e08-7798-4cfe-9365-86be6f0a52d2", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-305624008-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "db00bcd0c82c44f1bda739e9857f6e84", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapfec40d24-ca", "ovs_interfaceid": "fec40d24-caf1-48f7-8565-8fffdbdad60e", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:38:25 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Releasing lock "refresh_cache-abb01889-df4b-44a6-9827-87056d7ca125" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:38:25 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] [instance: abb01889-df4b-44a6-9827-87056d7ca125] Updated the network info_cache for instance {{(pid=75234) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9993}}
Nov 29 03:38:25 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:38:25 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:38:25 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:38:25 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:38:25 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:38:27 user nova-compute[75234]: DEBUG nova.compute.manager [None req-fe57873b-20a8-4d38-a798-90a5f865ee8c tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: 45059fbb-59d4-49ae-bccd-356edea7f287] Instance event wait completed in 0 seconds for  {{(pid=75234) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Nov 29 03:38:27 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-fe57873b-20a8-4d38-a798-90a5f865ee8c tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: 45059fbb-59d4-49ae-bccd-356edea7f287] Guest created on hypervisor {{(pid=75234) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4421}}
Nov 29 03:38:27 user nova-compute[75234]: INFO nova.virt.libvirt.driver [-] [instance: 45059fbb-59d4-49ae-bccd-356edea7f287] Instance spawned successfully.
Nov 29 03:38:27 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-fe57873b-20a8-4d38-a798-90a5f865ee8c tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: 45059fbb-59d4-49ae-bccd-356edea7f287] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:904}}
Nov 29 03:38:27 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-fe57873b-20a8-4d38-a798-90a5f865ee8c tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: 45059fbb-59d4-49ae-bccd-356edea7f287] Found default for hw_cdrom_bus of ide {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:38:27 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-fe57873b-20a8-4d38-a798-90a5f865ee8c tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: 45059fbb-59d4-49ae-bccd-356edea7f287] Found default for hw_disk_bus of virtio {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:38:27 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-fe57873b-20a8-4d38-a798-90a5f865ee8c tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: 45059fbb-59d4-49ae-bccd-356edea7f287] Found default for hw_input_bus of None {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:38:27 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-fe57873b-20a8-4d38-a798-90a5f865ee8c tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: 45059fbb-59d4-49ae-bccd-356edea7f287] Found default for hw_pointer_model of None {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:38:27 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-fe57873b-20a8-4d38-a798-90a5f865ee8c tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: 45059fbb-59d4-49ae-bccd-356edea7f287] Found default for hw_video_model of virtio {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:38:27 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-fe57873b-20a8-4d38-a798-90a5f865ee8c tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: 45059fbb-59d4-49ae-bccd-356edea7f287] Found default for hw_vif_model of virtio {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:38:27 user nova-compute[75234]: INFO nova.compute.manager [None req-fe57873b-20a8-4d38-a798-90a5f865ee8c tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: 45059fbb-59d4-49ae-bccd-356edea7f287] Took 4.91 seconds to spawn the instance on the hypervisor.
Nov 29 03:38:27 user nova-compute[75234]: DEBUG nova.compute.manager [None req-fe57873b-20a8-4d38-a798-90a5f865ee8c tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: 45059fbb-59d4-49ae-bccd-356edea7f287] Checking state {{(pid=75234) _get_power_state /opt/stack/nova/nova/compute/manager.py:1782}}
Nov 29 03:38:27 user nova-compute[75234]: DEBUG nova.compute.manager [req-6deef217-ffe7-4eba-8eee-35a6e397aa9d req-90ab2abd-6bc4-46e2-a496-11de75dfae1e service nova] [instance: 45059fbb-59d4-49ae-bccd-356edea7f287] Received event network-vif-plugged-420b9e5e-a22f-4e87-90d4-9b4430c66c4e {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:38:27 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-6deef217-ffe7-4eba-8eee-35a6e397aa9d req-90ab2abd-6bc4-46e2-a496-11de75dfae1e service nova] Acquiring lock "45059fbb-59d4-49ae-bccd-356edea7f287-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:38:27 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-6deef217-ffe7-4eba-8eee-35a6e397aa9d req-90ab2abd-6bc4-46e2-a496-11de75dfae1e service nova] Lock "45059fbb-59d4-49ae-bccd-356edea7f287-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:38:27 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-6deef217-ffe7-4eba-8eee-35a6e397aa9d req-90ab2abd-6bc4-46e2-a496-11de75dfae1e service nova] Lock "45059fbb-59d4-49ae-bccd-356edea7f287-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:38:27 user nova-compute[75234]: DEBUG nova.compute.manager [req-6deef217-ffe7-4eba-8eee-35a6e397aa9d req-90ab2abd-6bc4-46e2-a496-11de75dfae1e service nova] [instance: 45059fbb-59d4-49ae-bccd-356edea7f287] No waiting events found dispatching network-vif-plugged-420b9e5e-a22f-4e87-90d4-9b4430c66c4e {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:38:27 user nova-compute[75234]: WARNING nova.compute.manager [req-6deef217-ffe7-4eba-8eee-35a6e397aa9d req-90ab2abd-6bc4-46e2-a496-11de75dfae1e service nova] [instance: 45059fbb-59d4-49ae-bccd-356edea7f287] Received unexpected event network-vif-plugged-420b9e5e-a22f-4e87-90d4-9b4430c66c4e for instance with vm_state building and task_state spawning.
Nov 29 03:38:27 user nova-compute[75234]: INFO nova.compute.manager [None req-fe57873b-20a8-4d38-a798-90a5f865ee8c tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: 45059fbb-59d4-49ae-bccd-356edea7f287] Took 12.57 seconds to build instance.
Nov 29 03:38:27 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-fe57873b-20a8-4d38-a798-90a5f865ee8c tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Lock "45059fbb-59d4-49ae-bccd-356edea7f287" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 12.723s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:38:27 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:38:28 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:38:28 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:38:29 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._sync_scheduler_instance_info {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:38:33 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:38:38 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:38:42 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:38:43 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:38:47 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:38:48 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:38:52 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-af3225c7-b4c2-409b-bb7b-3bf7cddd1436 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Acquiring lock "abb01889-df4b-44a6-9827-87056d7ca125" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:38:52 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-af3225c7-b4c2-409b-bb7b-3bf7cddd1436 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Lock "abb01889-df4b-44a6-9827-87056d7ca125" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:38:52 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-af3225c7-b4c2-409b-bb7b-3bf7cddd1436 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Acquiring lock "abb01889-df4b-44a6-9827-87056d7ca125-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:38:52 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-af3225c7-b4c2-409b-bb7b-3bf7cddd1436 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Lock "abb01889-df4b-44a6-9827-87056d7ca125-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:38:52 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-af3225c7-b4c2-409b-bb7b-3bf7cddd1436 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Lock "abb01889-df4b-44a6-9827-87056d7ca125-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:38:52 user nova-compute[75234]: INFO nova.compute.manager [None req-af3225c7-b4c2-409b-bb7b-3bf7cddd1436 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] [instance: abb01889-df4b-44a6-9827-87056d7ca125] Terminating instance
Nov 29 03:38:52 user nova-compute[75234]: DEBUG nova.compute.manager [None req-af3225c7-b4c2-409b-bb7b-3bf7cddd1436 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] [instance: abb01889-df4b-44a6-9827-87056d7ca125] Start destroying the instance on the hypervisor. {{(pid=75234) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3140}}
Nov 29 03:38:52 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:38:52 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:38:52 user nova-compute[75234]: DEBUG nova.compute.manager [req-429635a8-6608-4987-b943-f3de7b72f1ce req-e338c734-5449-467c-b858-9841cc8f96ea service nova] [instance: abb01889-df4b-44a6-9827-87056d7ca125] Received event network-vif-unplugged-fec40d24-caf1-48f7-8565-8fffdbdad60e {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:38:52 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-429635a8-6608-4987-b943-f3de7b72f1ce req-e338c734-5449-467c-b858-9841cc8f96ea service nova] Acquiring lock "abb01889-df4b-44a6-9827-87056d7ca125-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:38:52 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-429635a8-6608-4987-b943-f3de7b72f1ce req-e338c734-5449-467c-b858-9841cc8f96ea service nova] Lock "abb01889-df4b-44a6-9827-87056d7ca125-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:38:52 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-429635a8-6608-4987-b943-f3de7b72f1ce req-e338c734-5449-467c-b858-9841cc8f96ea service nova] Lock "abb01889-df4b-44a6-9827-87056d7ca125-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:38:52 user nova-compute[75234]: DEBUG nova.compute.manager [req-429635a8-6608-4987-b943-f3de7b72f1ce req-e338c734-5449-467c-b858-9841cc8f96ea service nova] [instance: abb01889-df4b-44a6-9827-87056d7ca125] No waiting events found dispatching network-vif-unplugged-fec40d24-caf1-48f7-8565-8fffdbdad60e {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:38:52 user nova-compute[75234]: DEBUG nova.compute.manager [req-429635a8-6608-4987-b943-f3de7b72f1ce req-e338c734-5449-467c-b858-9841cc8f96ea service nova] [instance: abb01889-df4b-44a6-9827-87056d7ca125] Received event network-vif-unplugged-fec40d24-caf1-48f7-8565-8fffdbdad60e for instance with task_state deleting. {{(pid=75234) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10890}}
Nov 29 03:38:52 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:38:52 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:38:52 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:38:52 user nova-compute[75234]: INFO nova.virt.libvirt.driver [-] [instance: abb01889-df4b-44a6-9827-87056d7ca125] Instance destroyed successfully.
Nov 29 03:38:52 user nova-compute[75234]: DEBUG nova.objects.instance [None req-af3225c7-b4c2-409b-bb7b-3bf7cddd1436 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Lazy-loading 'resources' on Instance uuid abb01889-df4b-44a6-9827-87056d7ca125 {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:38:52 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-af3225c7-b4c2-409b-bb7b-3bf7cddd1436 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-11-29T03:29:28Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description=None,display_name='tempest-ServerBootFromVolumeStableRescueTest-server-1187610459',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-serverbootfromvolumestablerescuetest-server-1187610459',id=43,image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data=None,key_name=None,keypairs=<?>,launch_index=0,launched_at=2023-11-29T03:29:36Z,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='db00bcd0c82c44f1bda739e9857f6e84',ramdisk_id='',reservation_id='r-ft0p330b',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-ServerBootFromVolumeStableRescueTest-1601266069',owner_user_name='tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-11-29T03:31:26Z,user_data=None,user_id='877aa877f13a4f84ac4725278010aea3',uuid=abb01889-df4b-44a6-9827-87056d7ca125,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "fec40d24-caf1-48f7-8565-8fffdbdad60e", "address": "fa:16:3e:e3:bd:f8", "network": {"id": "4d086e08-7798-4cfe-9365-86be6f0a52d2", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-305624008-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "db00bcd0c82c44f1bda739e9857f6e84", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapfec40d24-ca", "ovs_interfaceid": "fec40d24-caf1-48f7-8565-8fffdbdad60e", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Nov 29 03:38:52 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-af3225c7-b4c2-409b-bb7b-3bf7cddd1436 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Converting VIF {"id": "fec40d24-caf1-48f7-8565-8fffdbdad60e", "address": "fa:16:3e:e3:bd:f8", "network": {"id": "4d086e08-7798-4cfe-9365-86be6f0a52d2", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-305624008-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "db00bcd0c82c44f1bda739e9857f6e84", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapfec40d24-ca", "ovs_interfaceid": "fec40d24-caf1-48f7-8565-8fffdbdad60e", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:38:52 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-af3225c7-b4c2-409b-bb7b-3bf7cddd1436 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:e3:bd:f8,bridge_name='br-int',has_traffic_filtering=True,id=fec40d24-caf1-48f7-8565-8fffdbdad60e,network=Network(4d086e08-7798-4cfe-9365-86be6f0a52d2),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapfec40d24-ca') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:38:52 user nova-compute[75234]: DEBUG os_vif [None req-af3225c7-b4c2-409b-bb7b-3bf7cddd1436 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Unplugging vif VIFOpenVSwitch(active=True,address=fa:16:3e:e3:bd:f8,bridge_name='br-int',has_traffic_filtering=True,id=fec40d24-caf1-48f7-8565-8fffdbdad60e,network=Network(4d086e08-7798-4cfe-9365-86be6f0a52d2),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapfec40d24-ca') {{(pid=75234) unplug /opt/stack/data/venv/lib/python3.10/site-packages/os_vif/__init__.py:109}}
Nov 29 03:38:52 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:38:52 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tapfec40d24-ca, bridge=br-int, if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:38:52 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:38:52 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:38:52 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbDestroyCommand(_result=None, table=QoS, record=816c4e61-ce5c-4c8a-83cd-b4eeadf527bc) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:38:52 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:38:52 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:38:52 user nova-compute[75234]: INFO os_vif [None req-af3225c7-b4c2-409b-bb7b-3bf7cddd1436 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Successfully unplugged vif VIFOpenVSwitch(active=True,address=fa:16:3e:e3:bd:f8,bridge_name='br-int',has_traffic_filtering=True,id=fec40d24-caf1-48f7-8565-8fffdbdad60e,network=Network(4d086e08-7798-4cfe-9365-86be6f0a52d2),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapfec40d24-ca')
Nov 29 03:38:52 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-af3225c7-b4c2-409b-bb7b-3bf7cddd1436 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] [instance: abb01889-df4b-44a6-9827-87056d7ca125] Deleting instance files /opt/stack/data/nova/instances/abb01889-df4b-44a6-9827-87056d7ca125_del
Nov 29 03:38:52 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-af3225c7-b4c2-409b-bb7b-3bf7cddd1436 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] [instance: abb01889-df4b-44a6-9827-87056d7ca125] Deletion of /opt/stack/data/nova/instances/abb01889-df4b-44a6-9827-87056d7ca125_del complete
Nov 29 03:38:52 user nova-compute[75234]: INFO nova.compute.manager [None req-af3225c7-b4c2-409b-bb7b-3bf7cddd1436 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] [instance: abb01889-df4b-44a6-9827-87056d7ca125] Took 0.67 seconds to destroy the instance on the hypervisor.
Nov 29 03:38:52 user nova-compute[75234]: DEBUG oslo.service.loopingcall [None req-af3225c7-b4c2-409b-bb7b-3bf7cddd1436 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=75234) func /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/loopingcall.py:435}}
Nov 29 03:38:52 user nova-compute[75234]: DEBUG nova.compute.manager [-] [instance: abb01889-df4b-44a6-9827-87056d7ca125] Deallocating network for instance {{(pid=75234) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2279}}
Nov 29 03:38:52 user nova-compute[75234]: DEBUG nova.network.neutron [-] [instance: abb01889-df4b-44a6-9827-87056d7ca125] deallocate_for_instance() {{(pid=75234) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1802}}
Nov 29 03:38:52 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:38:53 user nova-compute[75234]: DEBUG nova.network.neutron [-] [instance: abb01889-df4b-44a6-9827-87056d7ca125] Updating instance_info_cache with network_info: [] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:38:53 user nova-compute[75234]: INFO nova.compute.manager [-] [instance: abb01889-df4b-44a6-9827-87056d7ca125] Took 0.38 seconds to deallocate network for instance.
Nov 29 03:38:53 user nova-compute[75234]: DEBUG nova.compute.manager [req-86448e3b-0caa-406e-8ddc-ec0a432c2570 req-610864a8-c227-449f-8d5a-a4fe17b86a35 service nova] [instance: abb01889-df4b-44a6-9827-87056d7ca125] Received event network-vif-deleted-fec40d24-caf1-48f7-8565-8fffdbdad60e {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:38:53 user nova-compute[75234]: INFO nova.compute.manager [req-86448e3b-0caa-406e-8ddc-ec0a432c2570 req-610864a8-c227-449f-8d5a-a4fe17b86a35 service nova] [instance: abb01889-df4b-44a6-9827-87056d7ca125] Neutron deleted interface fec40d24-caf1-48f7-8565-8fffdbdad60e; detaching it from the instance and deleting it from the info cache
Nov 29 03:38:53 user nova-compute[75234]: DEBUG nova.network.neutron [req-86448e3b-0caa-406e-8ddc-ec0a432c2570 req-610864a8-c227-449f-8d5a-a4fe17b86a35 service nova] [instance: abb01889-df4b-44a6-9827-87056d7ca125] Updating instance_info_cache with network_info: [] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:38:53 user nova-compute[75234]: DEBUG nova.compute.manager [req-86448e3b-0caa-406e-8ddc-ec0a432c2570 req-610864a8-c227-449f-8d5a-a4fe17b86a35 service nova] [instance: abb01889-df4b-44a6-9827-87056d7ca125] Detach interface failed, port_id=fec40d24-caf1-48f7-8565-8fffdbdad60e, reason: Instance abb01889-df4b-44a6-9827-87056d7ca125 could not be found. {{(pid=75234) _process_instance_vif_deleted_event /opt/stack/nova/nova/compute/manager.py:10946}}
Nov 29 03:38:53 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-af3225c7-b4c2-409b-bb7b-3bf7cddd1436 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:38:53 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-af3225c7-b4c2-409b-bb7b-3bf7cddd1436 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:38:53 user nova-compute[75234]: DEBUG nova.compute.provider_tree [None req-af3225c7-b4c2-409b-bb7b-3bf7cddd1436 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Inventory has not changed in ProviderTree for provider: 4d291220-7c37-4839-a0e3-b23c66502b74 {{(pid=75234) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Nov 29 03:38:53 user nova-compute[75234]: DEBUG nova.scheduler.client.report [None req-af3225c7-b4c2-409b-bb7b-3bf7cddd1436 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Inventory has not changed for provider 4d291220-7c37-4839-a0e3-b23c66502b74 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=75234) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Nov 29 03:38:53 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-af3225c7-b4c2-409b-bb7b-3bf7cddd1436 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.192s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:38:53 user nova-compute[75234]: INFO nova.scheduler.client.report [None req-af3225c7-b4c2-409b-bb7b-3bf7cddd1436 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Deleted allocations for instance abb01889-df4b-44a6-9827-87056d7ca125
Nov 29 03:38:53 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-af3225c7-b4c2-409b-bb7b-3bf7cddd1436 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Lock "abb01889-df4b-44a6-9827-87056d7ca125" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 1.431s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:38:54 user nova-compute[75234]: DEBUG nova.compute.manager [req-999b90ee-ca65-4e8d-92dc-07c8a0cfe1f6 req-84c1432e-710e-4a40-97c1-46ad45472243 service nova] [instance: abb01889-df4b-44a6-9827-87056d7ca125] Received event network-vif-plugged-fec40d24-caf1-48f7-8565-8fffdbdad60e {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:38:54 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-999b90ee-ca65-4e8d-92dc-07c8a0cfe1f6 req-84c1432e-710e-4a40-97c1-46ad45472243 service nova] Acquiring lock "abb01889-df4b-44a6-9827-87056d7ca125-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:38:54 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-999b90ee-ca65-4e8d-92dc-07c8a0cfe1f6 req-84c1432e-710e-4a40-97c1-46ad45472243 service nova] Lock "abb01889-df4b-44a6-9827-87056d7ca125-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:38:54 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-999b90ee-ca65-4e8d-92dc-07c8a0cfe1f6 req-84c1432e-710e-4a40-97c1-46ad45472243 service nova] Lock "abb01889-df4b-44a6-9827-87056d7ca125-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:38:54 user nova-compute[75234]: DEBUG nova.compute.manager [req-999b90ee-ca65-4e8d-92dc-07c8a0cfe1f6 req-84c1432e-710e-4a40-97c1-46ad45472243 service nova] [instance: abb01889-df4b-44a6-9827-87056d7ca125] No waiting events found dispatching network-vif-plugged-fec40d24-caf1-48f7-8565-8fffdbdad60e {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:38:54 user nova-compute[75234]: WARNING nova.compute.manager [req-999b90ee-ca65-4e8d-92dc-07c8a0cfe1f6 req-84c1432e-710e-4a40-97c1-46ad45472243 service nova] [instance: abb01889-df4b-44a6-9827-87056d7ca125] Received unexpected event network-vif-plugged-fec40d24-caf1-48f7-8565-8fffdbdad60e for instance with vm_state deleted and task_state None.
Nov 29 03:38:57 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:38:57 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:39:02 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:39:07 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:39:12 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:39:15 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:39:16 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:39:16 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=75234) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10541}}
Nov 29 03:39:17 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:39:17 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:39:18 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:39:20 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager.update_available_resource {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:39:20 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:39:20 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:39:20 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:39:20 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Auditing locally available compute resources for user (node: user) {{(pid=75234) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:907}}
Nov 29 03:39:20 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/65d48c93-b6da-4f51-9d5a-0676dda62a60/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:39:20 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/65d48c93-b6da-4f51-9d5a-0676dda62a60/disk --force-share --output=json" returned: 0 in 0.172s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:39:20 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/65d48c93-b6da-4f51-9d5a-0676dda62a60/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:39:20 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/65d48c93-b6da-4f51-9d5a-0676dda62a60/disk --force-share --output=json" returned: 0 in 0.182s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:39:20 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] skipping disk /dev/sdb (vda) as it is a volume {{(pid=75234) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11213}}
Nov 29 03:39:20 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] skipping disk /dev/sdc (vda) as it is a volume {{(pid=75234) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11213}}
Nov 29 03:39:21 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:39:21 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:39:21 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Hypervisor/Node resource view: name=user free_ram=8823MB free_disk=25.04248046875GB free_vcpus=9 pci_devices=[{"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}] {{(pid=75234) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1106}}
Nov 29 03:39:21 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:39:21 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:39:21 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance 65d48c93-b6da-4f51-9d5a-0676dda62a60 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:39:21 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance d5096ca2-d005-45ac-a487-11a810446859 actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:39:21 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance 45059fbb-59d4-49ae-bccd-356edea7f287 actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:39:21 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Total usable vcpus: 12, total allocated vcpus: 3 {{(pid=75234) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1129}}
Nov 29 03:39:21 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Final resource view: name=user phys_ram=16011MB used_ram=2048MB phys_disk=40GB used_disk=1GB total_vcpus=12 used_vcpus=3 pci_stats=[] {{(pid=75234) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1138}}
Nov 29 03:39:21 user nova-compute[75234]: DEBUG nova.compute.provider_tree [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Inventory has not changed in ProviderTree for provider: 4d291220-7c37-4839-a0e3-b23c66502b74 {{(pid=75234) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Nov 29 03:39:21 user nova-compute[75234]: DEBUG nova.scheduler.client.report [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Inventory has not changed for provider 4d291220-7c37-4839-a0e3-b23c66502b74 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=75234) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Nov 29 03:39:21 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Compute_service record updated for user:user {{(pid=75234) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:1067}}
Nov 29 03:39:21 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.258s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:39:22 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:39:26 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:39:26 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Starting heal instance info cache {{(pid=75234) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9922}}
Nov 29 03:39:26 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Didn't find any instances for network info cache update. {{(pid=75234) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:10008}}
Nov 29 03:39:26 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:39:27 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:39:27 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:39:27 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:39:27 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5002 ms, sending inactivity probe {{(pid=75234) run /opt/stack/data/venv/lib/python3.10/site-packages/ovs/reconnect.py:117}}
Nov 29 03:39:27 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=75234) _transition /opt/stack/data/venv/lib/python3.10/site-packages/ovs/reconnect.py:519}}
Nov 29 03:39:27 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=75234) _transition /opt/stack/data/venv/lib/python3.10/site-packages/ovs/reconnect.py:519}}
Nov 29 03:39:27 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:39:28 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:39:32 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:39:37 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:39:42 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-634bf97c-2cf5-4890-898b-c1aa210c50b5 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Acquiring lock "d5096ca2-d005-45ac-a487-11a810446859" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:39:42 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-634bf97c-2cf5-4890-898b-c1aa210c50b5 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Lock "d5096ca2-d005-45ac-a487-11a810446859" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:39:42 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-634bf97c-2cf5-4890-898b-c1aa210c50b5 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Acquiring lock "d5096ca2-d005-45ac-a487-11a810446859-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:39:42 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-634bf97c-2cf5-4890-898b-c1aa210c50b5 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Lock "d5096ca2-d005-45ac-a487-11a810446859-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:39:42 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-634bf97c-2cf5-4890-898b-c1aa210c50b5 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Lock "d5096ca2-d005-45ac-a487-11a810446859-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:39:42 user nova-compute[75234]: INFO nova.compute.manager [None req-634bf97c-2cf5-4890-898b-c1aa210c50b5 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] [instance: d5096ca2-d005-45ac-a487-11a810446859] Terminating instance
Nov 29 03:39:42 user nova-compute[75234]: DEBUG nova.compute.manager [None req-634bf97c-2cf5-4890-898b-c1aa210c50b5 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] [instance: d5096ca2-d005-45ac-a487-11a810446859] Start destroying the instance on the hypervisor. {{(pid=75234) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3140}}
Nov 29 03:39:42 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:39:42 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:39:42 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:39:42 user nova-compute[75234]: DEBUG nova.compute.manager [req-69755679-6d12-4e65-a9dc-7e174de4ea34 req-83b49b5d-4b4e-4d99-850d-cd27124390ab service nova] [instance: d5096ca2-d005-45ac-a487-11a810446859] Received event network-vif-unplugged-9266d021-6e83-4bfa-8151-66c5182e61a0 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:39:42 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-69755679-6d12-4e65-a9dc-7e174de4ea34 req-83b49b5d-4b4e-4d99-850d-cd27124390ab service nova] Acquiring lock "d5096ca2-d005-45ac-a487-11a810446859-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:39:42 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-69755679-6d12-4e65-a9dc-7e174de4ea34 req-83b49b5d-4b4e-4d99-850d-cd27124390ab service nova] Lock "d5096ca2-d005-45ac-a487-11a810446859-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:39:42 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-69755679-6d12-4e65-a9dc-7e174de4ea34 req-83b49b5d-4b4e-4d99-850d-cd27124390ab service nova] Lock "d5096ca2-d005-45ac-a487-11a810446859-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:39:42 user nova-compute[75234]: DEBUG nova.compute.manager [req-69755679-6d12-4e65-a9dc-7e174de4ea34 req-83b49b5d-4b4e-4d99-850d-cd27124390ab service nova] [instance: d5096ca2-d005-45ac-a487-11a810446859] No waiting events found dispatching network-vif-unplugged-9266d021-6e83-4bfa-8151-66c5182e61a0 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:39:42 user nova-compute[75234]: DEBUG nova.compute.manager [req-69755679-6d12-4e65-a9dc-7e174de4ea34 req-83b49b5d-4b4e-4d99-850d-cd27124390ab service nova] [instance: d5096ca2-d005-45ac-a487-11a810446859] Received event network-vif-unplugged-9266d021-6e83-4bfa-8151-66c5182e61a0 for instance with task_state deleting. {{(pid=75234) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10890}}
Nov 29 03:39:42 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:39:42 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:39:43 user nova-compute[75234]: INFO nova.virt.libvirt.driver [-] [instance: d5096ca2-d005-45ac-a487-11a810446859] Instance destroyed successfully.
Nov 29 03:39:43 user nova-compute[75234]: DEBUG nova.objects.instance [None req-634bf97c-2cf5-4890-898b-c1aa210c50b5 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Lazy-loading 'resources' on Instance uuid d5096ca2-d005-45ac-a487-11a810446859 {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:39:43 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-634bf97c-2cf5-4890-898b-c1aa210c50b5 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-11-29T03:23:54Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description=None,display_name='tempest-ServerBootFromVolumeStableRescueTest-server-1791455206',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-serverbootfromvolumestablerescuetest-server-1791455206',id=40,image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data=None,key_name=None,keypairs=<?>,launch_index=0,launched_at=2023-11-29T03:25:54Z,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='db00bcd0c82c44f1bda739e9857f6e84',ramdisk_id='',reservation_id='r-pix7zk6i',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-ServerBootFromVolumeStableRescueTest-1601266069',owner_user_name='tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-11-29T03:27:42Z,user_data=None,user_id='877aa877f13a4f84ac4725278010aea3',uuid=d5096ca2-d005-45ac-a487-11a810446859,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "9266d021-6e83-4bfa-8151-66c5182e61a0", "address": "fa:16:3e:ec:fb:67", "network": {"id": "4d086e08-7798-4cfe-9365-86be6f0a52d2", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-305624008-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "db00bcd0c82c44f1bda739e9857f6e84", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap9266d021-6e", "ovs_interfaceid": "9266d021-6e83-4bfa-8151-66c5182e61a0", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Nov 29 03:39:43 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-634bf97c-2cf5-4890-898b-c1aa210c50b5 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Converting VIF {"id": "9266d021-6e83-4bfa-8151-66c5182e61a0", "address": "fa:16:3e:ec:fb:67", "network": {"id": "4d086e08-7798-4cfe-9365-86be6f0a52d2", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-305624008-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.8", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "db00bcd0c82c44f1bda739e9857f6e84", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap9266d021-6e", "ovs_interfaceid": "9266d021-6e83-4bfa-8151-66c5182e61a0", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:39:43 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-634bf97c-2cf5-4890-898b-c1aa210c50b5 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:ec:fb:67,bridge_name='br-int',has_traffic_filtering=True,id=9266d021-6e83-4bfa-8151-66c5182e61a0,network=Network(4d086e08-7798-4cfe-9365-86be6f0a52d2),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap9266d021-6e') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:39:43 user nova-compute[75234]: DEBUG os_vif [None req-634bf97c-2cf5-4890-898b-c1aa210c50b5 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Unplugging vif VIFOpenVSwitch(active=True,address=fa:16:3e:ec:fb:67,bridge_name='br-int',has_traffic_filtering=True,id=9266d021-6e83-4bfa-8151-66c5182e61a0,network=Network(4d086e08-7798-4cfe-9365-86be6f0a52d2),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap9266d021-6e') {{(pid=75234) unplug /opt/stack/data/venv/lib/python3.10/site-packages/os_vif/__init__.py:109}}
Nov 29 03:39:43 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:39:43 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tap9266d021-6e, bridge=br-int, if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:39:43 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:39:43 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:39:43 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:39:43 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbDestroyCommand(_result=None, table=QoS, record=0b6948f9-c40d-4b00-aacf-ed1db9c1975e) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:39:43 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:39:43 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:39:43 user nova-compute[75234]: INFO os_vif [None req-634bf97c-2cf5-4890-898b-c1aa210c50b5 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Successfully unplugged vif VIFOpenVSwitch(active=True,address=fa:16:3e:ec:fb:67,bridge_name='br-int',has_traffic_filtering=True,id=9266d021-6e83-4bfa-8151-66c5182e61a0,network=Network(4d086e08-7798-4cfe-9365-86be6f0a52d2),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap9266d021-6e')
Nov 29 03:39:43 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-634bf97c-2cf5-4890-898b-c1aa210c50b5 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] [instance: d5096ca2-d005-45ac-a487-11a810446859] calling os-brick to detach iSCSI Volume {{(pid=75234) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Nov 29 03:39:43 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-634bf97c-2cf5-4890-898b-c1aa210c50b5 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f8a5050c9a0>, {'target_iqn': 'iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'BbOlTYfq', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdc'}, None), 'kwargs': True}}" {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:176}}
Nov 29 03:39:43 user nova-compute[75234]: WARNING os_brick.initiator.connectors.base [None req-634bf97c-2cf5-4890-898b-c1aa210c50b5 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Nov 29 03:39:43 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-634bf97c-2cf5-4890-898b-c1aa210c50b5 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:68}}
Nov 29 03:39:43 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-634bf97c-2cf5-4890-898b-c1aa210c50b5 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.002s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:73}}
Nov 29 03:39:43 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:39:43 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.025s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:39:43 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[e4426d37-ae06-4fd6-82db-aaefb7a9d508]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:378addbb-8d90-40c5-ae61-c3a063116df9\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:39:43 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-634bf97c-2cf5-4890-898b-c1aa210c50b5 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS:
Nov 29 03:39:43 user nova-compute[75234]: No targets found.
Nov 29 03:39:43 user nova-compute[75234]: iSNS:
Nov 29 03:39:43 user nova-compute[75234]: No targets found.
Nov 29 03:39:43 user nova-compute[75234]: STATIC:
Nov 29 03:39:43 user nova-compute[75234]: Target: iqn.2010-10.org.openstack:378addbb-8d90-40c5-ae61-c3a063116df9
Nov 29 03:39:43 user nova-compute[75234]:         Portal: 172.16.0.220:3260,-1
Nov 29 03:39:43 user nova-compute[75234]:                 Iface Name: default
Nov 29 03:39:43 user nova-compute[75234]: Target: iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0
Nov 29 03:39:43 user nova-compute[75234]:         Portal: 172.16.0.220:3260,-1
Nov 29 03:39:43 user nova-compute[75234]:                 Iface Name: default
Nov 29 03:39:43 user nova-compute[75234]: FIRMWARE:
Nov 29 03:39:43 user nova-compute[75234]: No targets found.
Nov 29 03:39:43 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:39:43 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-634bf97c-2cf5-4890-898b-c1aa210c50b5 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Regex to get portals from discoverydb: ^SENDTARGETS:
Nov 29 03:39:43 user nova-compute[75234]: .*?^DiscoveryAddress: 172.16.0.220,3260.*?
Nov 29 03:39:43 user nova-compute[75234]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=75234) _get_discoverydb_portals /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:382}}
Nov 29 03:39:43 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-634bf97c-2cf5-4890-898b-c1aa210c50b5 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0', 0)] {{(pid=75234) _get_connection_devices /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:830}}
Nov 29 03:39:43 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:39:43 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.018s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:39:43 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[05afe1ce-3d7e-4a9e-8ee4-ccbed5a9f9d3]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:378addbb-8d90-40c5-ae61-c3a063116df9\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:39:43 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:39:43 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.021s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:39:43 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[7960c2d6-f80a-4ff4-a464-5f16c58bf731]: (4, ('tcp: [42] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0 (non-flash)\ntcp: [58] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:378addbb-8d90-40c5-ae61-c3a063116df9 (non-flash)\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:39:43 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-634bf97c-2cf5-4890-898b-c1aa210c50b5 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [42] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0 (non-flash)
Nov 29 03:39:43 user nova-compute[75234]: tcp: [58] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:378addbb-8d90-40c5-ae61-c3a063116df9 (non-flash)
Nov 29 03:39:43 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:39:43 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-634bf97c-2cf5-4890-898b-c1aa210c50b5 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] iscsi session list stdout=tcp: [42] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0 (non-flash)
Nov 29 03:39:43 user nova-compute[75234]: tcp: [58] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:378addbb-8d90-40c5-ae61-c3a063116df9 (non-flash)
Nov 29 03:39:43 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsi_session /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Nov 29 03:39:43 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-634bf97c-2cf5-4890-898b-c1aa210c50b5 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7f8a2c688f70>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0'): ({'sdc'}, set())}) {{(pid=75234) _get_connection_devices /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:863}}
Nov 29 03:39:43 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-634bf97c-2cf5-4890-898b-c1aa210c50b5 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Removing single pathed devices sdc {{(pid=75234) remove_connection /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:374}}
Nov 29 03:39:43 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:39:43 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.020s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:39:43 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[3309baa5-8940-4c51-9420-129d29ac2995]: (4, ('path checker states:\nup                  2\n\npaths: 0\nbusy: False\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:39:43 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdc {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:39:43 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdc" returned: 0 in 0.020s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:39:43 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[96b00d75-8a4b-4b52-bc5c-f4d670b8e847]: (4, ('ok\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:39:43 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-634bf97c-2cf5-4890-898b-c1aa210c50b5 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Flushing IO for device /dev/sdc {{(pid=75234) flush_device_io /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:434}}
Nov 29 03:39:43 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdc {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:39:43 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdc" returned: 0 in 0.016s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:39:43 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[4e68ac22-5bae-4e2d-8dc6-081a06e4b255]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:39:43 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-634bf97c-2cf5-4890-898b-c1aa210c50b5 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Remove SCSI device /dev/sdc with /sys/block/sdc/device/delete {{(pid=75234) remove_scsi_device /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:148}}
Nov 29 03:39:43 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdc/device/delete {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:39:43 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdc/device/delete" returned: 0 in 0.055s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:39:43 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[a8998f67-2ca7-4005-8d47-72af7a031336]: (4, ('1', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:39:43 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-634bf97c-2cf5-4890-898b-c1aa210c50b5 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Checking to see if SCSI volumes sdc have been removed. {{(pid=75234) wait_for_volumes_removal /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:156}}
Nov 29 03:39:43 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-634bf97c-2cf5-4890-898b-c1aa210c50b5 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] SCSI volumes sdc have been removed. {{(pid=75234) wait_for_volumes_removal /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:166}}
Nov 29 03:39:43 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-634bf97c-2cf5-4890-898b-c1aa210c50b5 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0')] {{(pid=75234) _disconnect_connection /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Nov 29 03:39:43 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0 -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:39:43 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0 -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.019s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:39:43 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[ca6058c3-544a-47c7-b889-1cc876c64303]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:39:43 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-634bf97c-2cf5-4890-898b-c1aa210c50b5 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:39:43 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0 -p 172.16.0.220:3260 --logout {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:39:43 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0 -p 172.16.0.220:3260 --logout" returned: 0 in 0.053s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:39:43 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[9221e428-ff2d-4ab8-86d6-99c9de5b5917]: (4, ('Logging out of session [sid: 42, target: iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0, portal: 172.16.0.220,3260]\nLogout of [sid: 42, target: iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:39:43 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-634bf97c-2cf5-4890-898b-c1aa210c50b5 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 42, target: iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0, portal: 172.16.0.220,3260]
Nov 29 03:39:43 user nova-compute[75234]: Logout of [sid: 42, target: iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0, portal: 172.16.0.220,3260] successful.
Nov 29 03:39:43 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:39:43 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0 -p 172.16.0.220:3260 --op delete {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:39:43 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:c12ce787-781c-4e31-adfa-5b51145bc1a0 -p 172.16.0.220:3260 --op delete" returned: 0 in 0.020s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:39:43 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[99aafe92-43e1-47b4-b043-b566a67be340]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:39:43 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-634bf97c-2cf5-4890-898b-c1aa210c50b5 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:39:43 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-634bf97c-2cf5-4890-898b-c1aa210c50b5 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.311s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:87}}
Nov 29 03:39:43 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-634bf97c-2cf5-4890-898b-c1aa210c50b5 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] <== disconnect_volume: return (313ms) None {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:203}}
Nov 29 03:39:43 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-634bf97c-2cf5-4890-898b-c1aa210c50b5 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] [instance: d5096ca2-d005-45ac-a487-11a810446859] Disconnected iSCSI Volume {{(pid=75234) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:79}}
Nov 29 03:39:43 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-634bf97c-2cf5-4890-898b-c1aa210c50b5 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] [instance: d5096ca2-d005-45ac-a487-11a810446859] Deleting instance files /opt/stack/data/nova/instances/d5096ca2-d005-45ac-a487-11a810446859_del
Nov 29 03:39:43 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-634bf97c-2cf5-4890-898b-c1aa210c50b5 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] [instance: d5096ca2-d005-45ac-a487-11a810446859] Deletion of /opt/stack/data/nova/instances/d5096ca2-d005-45ac-a487-11a810446859_del complete
Nov 29 03:39:43 user nova-compute[75234]: INFO nova.compute.manager [None req-634bf97c-2cf5-4890-898b-c1aa210c50b5 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] [instance: d5096ca2-d005-45ac-a487-11a810446859] Took 1.14 seconds to destroy the instance on the hypervisor.
Nov 29 03:39:43 user nova-compute[75234]: DEBUG oslo.service.loopingcall [None req-634bf97c-2cf5-4890-898b-c1aa210c50b5 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=75234) func /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/loopingcall.py:435}}
Nov 29 03:39:43 user nova-compute[75234]: DEBUG nova.compute.manager [-] [instance: d5096ca2-d005-45ac-a487-11a810446859] Deallocating network for instance {{(pid=75234) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2279}}
Nov 29 03:39:43 user nova-compute[75234]: DEBUG nova.network.neutron [-] [instance: d5096ca2-d005-45ac-a487-11a810446859] deallocate_for_instance() {{(pid=75234) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1802}}
Nov 29 03:39:44 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:39:44 user nova-compute[75234]: DEBUG nova.network.neutron [-] [instance: d5096ca2-d005-45ac-a487-11a810446859] Updating instance_info_cache with network_info: [] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:39:44 user nova-compute[75234]: INFO nova.compute.manager [-] [instance: d5096ca2-d005-45ac-a487-11a810446859] Took 0.40 seconds to deallocate network for instance.
Nov 29 03:39:44 user nova-compute[75234]: DEBUG nova.compute.manager [req-2986a938-8bd7-4b42-95df-78d63faba6d8 req-74da4352-a99e-44b6-aeb7-6fc7bfb93a28 service nova] [instance: d5096ca2-d005-45ac-a487-11a810446859] Received event network-vif-deleted-9266d021-6e83-4bfa-8151-66c5182e61a0 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:39:44 user nova-compute[75234]: DEBUG nova.compute.manager [req-f510f487-1553-4d9f-84c8-8e358897da1c req-91a618da-31e9-4e00-8c7f-8661a4d7a76f service nova] [instance: d5096ca2-d005-45ac-a487-11a810446859] Received event network-vif-plugged-9266d021-6e83-4bfa-8151-66c5182e61a0 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:39:44 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-f510f487-1553-4d9f-84c8-8e358897da1c req-91a618da-31e9-4e00-8c7f-8661a4d7a76f service nova] Acquiring lock "d5096ca2-d005-45ac-a487-11a810446859-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:39:44 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-f510f487-1553-4d9f-84c8-8e358897da1c req-91a618da-31e9-4e00-8c7f-8661a4d7a76f service nova] Lock "d5096ca2-d005-45ac-a487-11a810446859-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:39:44 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-f510f487-1553-4d9f-84c8-8e358897da1c req-91a618da-31e9-4e00-8c7f-8661a4d7a76f service nova] Lock "d5096ca2-d005-45ac-a487-11a810446859-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:39:44 user nova-compute[75234]: DEBUG nova.compute.manager [req-f510f487-1553-4d9f-84c8-8e358897da1c req-91a618da-31e9-4e00-8c7f-8661a4d7a76f service nova] [instance: d5096ca2-d005-45ac-a487-11a810446859] No waiting events found dispatching network-vif-plugged-9266d021-6e83-4bfa-8151-66c5182e61a0 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:39:44 user nova-compute[75234]: WARNING nova.compute.manager [req-f510f487-1553-4d9f-84c8-8e358897da1c req-91a618da-31e9-4e00-8c7f-8661a4d7a76f service nova] [instance: d5096ca2-d005-45ac-a487-11a810446859] Received unexpected event network-vif-plugged-9266d021-6e83-4bfa-8151-66c5182e61a0 for instance with vm_state active and task_state deleting.
Nov 29 03:39:44 user nova-compute[75234]: INFO nova.compute.manager [None req-634bf97c-2cf5-4890-898b-c1aa210c50b5 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] [instance: d5096ca2-d005-45ac-a487-11a810446859] Took 0.76 seconds to detach 1 volumes for instance.
Nov 29 03:39:44 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-634bf97c-2cf5-4890-898b-c1aa210c50b5 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:39:44 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-634bf97c-2cf5-4890-898b-c1aa210c50b5 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:39:45 user nova-compute[75234]: DEBUG nova.compute.provider_tree [None req-634bf97c-2cf5-4890-898b-c1aa210c50b5 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Inventory has not changed in ProviderTree for provider: 4d291220-7c37-4839-a0e3-b23c66502b74 {{(pid=75234) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Nov 29 03:39:45 user nova-compute[75234]: DEBUG nova.scheduler.client.report [None req-634bf97c-2cf5-4890-898b-c1aa210c50b5 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Inventory has not changed for provider 4d291220-7c37-4839-a0e3-b23c66502b74 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=75234) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Nov 29 03:39:45 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-634bf97c-2cf5-4890-898b-c1aa210c50b5 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.167s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:39:45 user nova-compute[75234]: INFO nova.scheduler.client.report [None req-634bf97c-2cf5-4890-898b-c1aa210c50b5 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Deleted allocations for instance d5096ca2-d005-45ac-a487-11a810446859
Nov 29 03:39:45 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-634bf97c-2cf5-4890-898b-c1aa210c50b5 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Lock "d5096ca2-d005-45ac-a487-11a810446859" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 2.653s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:39:48 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:39:53 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:39:58 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:40:03 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:40:06 user nova-compute[75234]: DEBUG nova.compute.manager [req-4ca375cb-2f1c-4ddb-af16-36d3ddc96e0d req-9ff62811-a42f-4f99-ae93-fa2193ca82d3 service nova] [instance: 45059fbb-59d4-49ae-bccd-356edea7f287] Received event network-changed-420b9e5e-a22f-4e87-90d4-9b4430c66c4e {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:40:06 user nova-compute[75234]: DEBUG nova.compute.manager [req-4ca375cb-2f1c-4ddb-af16-36d3ddc96e0d req-9ff62811-a42f-4f99-ae93-fa2193ca82d3 service nova] [instance: 45059fbb-59d4-49ae-bccd-356edea7f287] Refreshing instance network info cache due to event network-changed-420b9e5e-a22f-4e87-90d4-9b4430c66c4e. {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11117}}
Nov 29 03:40:06 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-4ca375cb-2f1c-4ddb-af16-36d3ddc96e0d req-9ff62811-a42f-4f99-ae93-fa2193ca82d3 service nova] Acquiring lock "refresh_cache-45059fbb-59d4-49ae-bccd-356edea7f287" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:40:06 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-4ca375cb-2f1c-4ddb-af16-36d3ddc96e0d req-9ff62811-a42f-4f99-ae93-fa2193ca82d3 service nova] Acquired lock "refresh_cache-45059fbb-59d4-49ae-bccd-356edea7f287" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:40:06 user nova-compute[75234]: DEBUG nova.network.neutron [req-4ca375cb-2f1c-4ddb-af16-36d3ddc96e0d req-9ff62811-a42f-4f99-ae93-fa2193ca82d3 service nova] [instance: 45059fbb-59d4-49ae-bccd-356edea7f287] Refreshing network info cache for port 420b9e5e-a22f-4e87-90d4-9b4430c66c4e {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2006}}
Nov 29 03:40:06 user nova-compute[75234]: DEBUG nova.network.neutron [req-4ca375cb-2f1c-4ddb-af16-36d3ddc96e0d req-9ff62811-a42f-4f99-ae93-fa2193ca82d3 service nova] [instance: 45059fbb-59d4-49ae-bccd-356edea7f287] Updated VIF entry in instance network info cache for port 420b9e5e-a22f-4e87-90d4-9b4430c66c4e. {{(pid=75234) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3481}}
Nov 29 03:40:06 user nova-compute[75234]: DEBUG nova.network.neutron [req-4ca375cb-2f1c-4ddb-af16-36d3ddc96e0d req-9ff62811-a42f-4f99-ae93-fa2193ca82d3 service nova] [instance: 45059fbb-59d4-49ae-bccd-356edea7f287] Updating instance_info_cache with network_info: [{"id": "420b9e5e-a22f-4e87-90d4-9b4430c66c4e", "address": "fa:16:3e:48:2e:bc", "network": {"id": "a3f397d3-fd75-4a6e-a815-cfcedcca15fb", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-678570085-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.11", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "81a265d3c15b4e9d826b35ddbd0ff443", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap420b9e5e-a2", "ovs_interfaceid": "420b9e5e-a22f-4e87-90d4-9b4430c66c4e", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:40:06 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-4ca375cb-2f1c-4ddb-af16-36d3ddc96e0d req-9ff62811-a42f-4f99-ae93-fa2193ca82d3 service nova] Releasing lock "refresh_cache-45059fbb-59d4-49ae-bccd-356edea7f287" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:40:07 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:40:07 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-94d33106-5710-4dec-b066-5ed7efad2d33 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Acquiring lock "45059fbb-59d4-49ae-bccd-356edea7f287" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:40:07 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-94d33106-5710-4dec-b066-5ed7efad2d33 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Lock "45059fbb-59d4-49ae-bccd-356edea7f287" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:40:07 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-94d33106-5710-4dec-b066-5ed7efad2d33 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Acquiring lock "45059fbb-59d4-49ae-bccd-356edea7f287-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:40:07 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-94d33106-5710-4dec-b066-5ed7efad2d33 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Lock "45059fbb-59d4-49ae-bccd-356edea7f287-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:40:07 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-94d33106-5710-4dec-b066-5ed7efad2d33 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Lock "45059fbb-59d4-49ae-bccd-356edea7f287-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:40:07 user nova-compute[75234]: INFO nova.compute.manager [None req-94d33106-5710-4dec-b066-5ed7efad2d33 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: 45059fbb-59d4-49ae-bccd-356edea7f287] Terminating instance
Nov 29 03:40:07 user nova-compute[75234]: DEBUG nova.compute.manager [None req-94d33106-5710-4dec-b066-5ed7efad2d33 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: 45059fbb-59d4-49ae-bccd-356edea7f287] Start destroying the instance on the hypervisor. {{(pid=75234) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3140}}
Nov 29 03:40:08 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:40:08 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:40:08 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:40:08 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:40:08 user nova-compute[75234]: DEBUG nova.compute.manager [req-92635a4d-dab0-47f9-80e1-c3c45ead8ded req-24c9e70c-536e-4da0-b067-2c1c97681762 service nova] [instance: 45059fbb-59d4-49ae-bccd-356edea7f287] Received event network-vif-unplugged-420b9e5e-a22f-4e87-90d4-9b4430c66c4e {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:40:08 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-92635a4d-dab0-47f9-80e1-c3c45ead8ded req-24c9e70c-536e-4da0-b067-2c1c97681762 service nova] Acquiring lock "45059fbb-59d4-49ae-bccd-356edea7f287-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:40:08 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-92635a4d-dab0-47f9-80e1-c3c45ead8ded req-24c9e70c-536e-4da0-b067-2c1c97681762 service nova] Lock "45059fbb-59d4-49ae-bccd-356edea7f287-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:40:08 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-92635a4d-dab0-47f9-80e1-c3c45ead8ded req-24c9e70c-536e-4da0-b067-2c1c97681762 service nova] Lock "45059fbb-59d4-49ae-bccd-356edea7f287-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:40:08 user nova-compute[75234]: DEBUG nova.compute.manager [req-92635a4d-dab0-47f9-80e1-c3c45ead8ded req-24c9e70c-536e-4da0-b067-2c1c97681762 service nova] [instance: 45059fbb-59d4-49ae-bccd-356edea7f287] No waiting events found dispatching network-vif-unplugged-420b9e5e-a22f-4e87-90d4-9b4430c66c4e {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:40:08 user nova-compute[75234]: DEBUG nova.compute.manager [req-92635a4d-dab0-47f9-80e1-c3c45ead8ded req-24c9e70c-536e-4da0-b067-2c1c97681762 service nova] [instance: 45059fbb-59d4-49ae-bccd-356edea7f287] Received event network-vif-unplugged-420b9e5e-a22f-4e87-90d4-9b4430c66c4e for instance with task_state deleting. {{(pid=75234) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10890}}
Nov 29 03:40:08 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:40:08 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:40:08 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:40:08 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:40:08 user nova-compute[75234]: INFO nova.virt.libvirt.driver [-] [instance: 45059fbb-59d4-49ae-bccd-356edea7f287] Instance destroyed successfully.
Nov 29 03:40:08 user nova-compute[75234]: DEBUG nova.objects.instance [None req-94d33106-5710-4dec-b066-5ed7efad2d33 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Lazy-loading 'resources' on Instance uuid 45059fbb-59d4-49ae-bccd-356edea7f287 {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:40:08 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-94d33106-5710-4dec-b066-5ed7efad2d33 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-11-29T03:38:14Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-TestVolumeBootPattern-server-1788415920',display_name='tempest-TestVolumeBootPattern-server-1788415920',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-testvolumebootpattern-server-1788415920',id=50,image_ref='',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBKrnAt22NMoBNIFsnCF3Mxk+dIbrL4YaXFADzEahC/3pON2UrJ4NOPzv0Ze4QWq/A4Bm4h/5Mn2lQRP6Y0eXl2r9TPWd6a1CQjrsTRMK7MqwF4n4QkuDxmXBPaNWL5XvpA==',key_name='tempest-TestVolumeBootPattern-1253305043',keypairs=<?>,launch_index=0,launched_at=2023-11-29T03:38:27Z,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='81a265d3c15b4e9d826b35ddbd0ff443',ramdisk_id='',reservation_id='r-p6i390s0',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-TestVolumeBootPattern-1244492388',owner_user_name='tempest-TestVolumeBootPattern-1244492388-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-11-29T03:38:28Z,user_data=None,user_id='7d1f1f11a9aa4c1db8627413eb963fa4',uuid=45059fbb-59d4-49ae-bccd-356edea7f287,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "420b9e5e-a22f-4e87-90d4-9b4430c66c4e", "address": "fa:16:3e:48:2e:bc", "network": {"id": "a3f397d3-fd75-4a6e-a815-cfcedcca15fb", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-678570085-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.11", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "81a265d3c15b4e9d826b35ddbd0ff443", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap420b9e5e-a2", "ovs_interfaceid": "420b9e5e-a22f-4e87-90d4-9b4430c66c4e", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Nov 29 03:40:08 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-94d33106-5710-4dec-b066-5ed7efad2d33 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Converting VIF {"id": "420b9e5e-a22f-4e87-90d4-9b4430c66c4e", "address": "fa:16:3e:48:2e:bc", "network": {"id": "a3f397d3-fd75-4a6e-a815-cfcedcca15fb", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-678570085-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.11", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "81a265d3c15b4e9d826b35ddbd0ff443", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap420b9e5e-a2", "ovs_interfaceid": "420b9e5e-a22f-4e87-90d4-9b4430c66c4e", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:40:08 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-94d33106-5710-4dec-b066-5ed7efad2d33 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:48:2e:bc,bridge_name='br-int',has_traffic_filtering=True,id=420b9e5e-a22f-4e87-90d4-9b4430c66c4e,network=Network(a3f397d3-fd75-4a6e-a815-cfcedcca15fb),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap420b9e5e-a2') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:40:08 user nova-compute[75234]: DEBUG os_vif [None req-94d33106-5710-4dec-b066-5ed7efad2d33 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Unplugging vif VIFOpenVSwitch(active=True,address=fa:16:3e:48:2e:bc,bridge_name='br-int',has_traffic_filtering=True,id=420b9e5e-a22f-4e87-90d4-9b4430c66c4e,network=Network(a3f397d3-fd75-4a6e-a815-cfcedcca15fb),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap420b9e5e-a2') {{(pid=75234) unplug /opt/stack/data/venv/lib/python3.10/site-packages/os_vif/__init__.py:109}}
Nov 29 03:40:08 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:40:08 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:40:08 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tap420b9e5e-a2, bridge=br-int, if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:40:08 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:40:08 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:40:08 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:40:08 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbDestroyCommand(_result=None, table=QoS, record=427c6127-9ed9-41f8-b889-4c58a4567abf) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:40:08 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:40:08 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:40:08 user nova-compute[75234]: INFO os_vif [None req-94d33106-5710-4dec-b066-5ed7efad2d33 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Successfully unplugged vif VIFOpenVSwitch(active=True,address=fa:16:3e:48:2e:bc,bridge_name='br-int',has_traffic_filtering=True,id=420b9e5e-a22f-4e87-90d4-9b4430c66c4e,network=Network(a3f397d3-fd75-4a6e-a815-cfcedcca15fb),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap420b9e5e-a2')
Nov 29 03:40:08 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-94d33106-5710-4dec-b066-5ed7efad2d33 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: 45059fbb-59d4-49ae-bccd-356edea7f287] calling os-brick to detach iSCSI Volume {{(pid=75234) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Nov 29 03:40:08 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-94d33106-5710-4dec-b066-5ed7efad2d33 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f8a5050c9a0>, {'target_iqn': 'iqn.2010-10.org.openstack:378addbb-8d90-40c5-ae61-c3a063116df9', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'cyOomDWj', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdb'}, None), 'kwargs': True}}" {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:176}}
Nov 29 03:40:08 user nova-compute[75234]: WARNING os_brick.initiator.connectors.base [None req-94d33106-5710-4dec-b066-5ed7efad2d33 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Nov 29 03:40:08 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-94d33106-5710-4dec-b066-5ed7efad2d33 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:68}}
Nov 29 03:40:08 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-94d33106-5710-4dec-b066-5ed7efad2d33 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:73}}
Nov 29 03:40:08 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:40:08 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.023s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:40:08 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[c7275b81-fc1a-4b50-bec2-98ea90a90639]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:378addbb-8d90-40c5-ae61-c3a063116df9\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:40:08 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-94d33106-5710-4dec-b066-5ed7efad2d33 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS:
Nov 29 03:40:08 user nova-compute[75234]: No targets found.
Nov 29 03:40:08 user nova-compute[75234]: iSNS:
Nov 29 03:40:08 user nova-compute[75234]: No targets found.
Nov 29 03:40:08 user nova-compute[75234]: STATIC:
Nov 29 03:40:08 user nova-compute[75234]: Target: iqn.2010-10.org.openstack:378addbb-8d90-40c5-ae61-c3a063116df9
Nov 29 03:40:08 user nova-compute[75234]:         Portal: 172.16.0.220:3260,-1
Nov 29 03:40:08 user nova-compute[75234]:                 Iface Name: default
Nov 29 03:40:08 user nova-compute[75234]: FIRMWARE:
Nov 29 03:40:08 user nova-compute[75234]: No targets found.
Nov 29 03:40:08 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:40:08 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-94d33106-5710-4dec-b066-5ed7efad2d33 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Regex to get portals from discoverydb: ^SENDTARGETS:
Nov 29 03:40:08 user nova-compute[75234]: .*?^DiscoveryAddress: 172.16.0.220,3260.*?
Nov 29 03:40:08 user nova-compute[75234]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=75234) _get_discoverydb_portals /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:382}}
Nov 29 03:40:08 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-94d33106-5710-4dec-b066-5ed7efad2d33 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:378addbb-8d90-40c5-ae61-c3a063116df9', 0)] {{(pid=75234) _get_connection_devices /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:830}}
Nov 29 03:40:08 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:40:08 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.019s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:40:08 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[6f19352c-7d0c-4517-8548-d5bdff1a6906]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:378addbb-8d90-40c5-ae61-c3a063116df9\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:40:08 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:40:08 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.025s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:40:08 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[120b0eeb-e151-4d55-8094-e9629293f9c0]: (4, ('tcp: [58] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:378addbb-8d90-40c5-ae61-c3a063116df9 (non-flash)\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:40:08 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-94d33106-5710-4dec-b066-5ed7efad2d33 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [58] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:378addbb-8d90-40c5-ae61-c3a063116df9 (non-flash)
Nov 29 03:40:08 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:40:08 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-94d33106-5710-4dec-b066-5ed7efad2d33 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] iscsi session list stdout=tcp: [58] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:378addbb-8d90-40c5-ae61-c3a063116df9 (non-flash)
Nov 29 03:40:08 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsi_session /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Nov 29 03:40:08 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-94d33106-5710-4dec-b066-5ed7efad2d33 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7f8a2c6de0e0>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:378addbb-8d90-40c5-ae61-c3a063116df9'): ({'sdb'}, set())}) {{(pid=75234) _get_connection_devices /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:863}}
Nov 29 03:40:08 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-94d33106-5710-4dec-b066-5ed7efad2d33 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Removing single pathed devices sdb {{(pid=75234) remove_connection /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:374}}
Nov 29 03:40:08 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:40:08 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.025s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:40:08 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[97e0ce78-f4d2-4493-a902-145a508addb0]: (4, ('path checker states:\nup                  1\n\npaths: 0\nbusy: False\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:40:08 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdb {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:40:08 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdb" returned: 0 in 0.022s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:40:08 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[1a68e804-f358-47ab-b177-9609a34a9fea]: (4, ('ok\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:40:08 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-94d33106-5710-4dec-b066-5ed7efad2d33 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Flushing IO for device /dev/sdb {{(pid=75234) flush_device_io /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:434}}
Nov 29 03:40:08 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdb {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:40:08 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdb" returned: 0 in 0.020s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:40:08 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[35d16409-6a3a-484e-998e-564db1341762]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:40:08 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-94d33106-5710-4dec-b066-5ed7efad2d33 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Remove SCSI device /dev/sdb with /sys/block/sdb/device/delete {{(pid=75234) remove_scsi_device /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:148}}
Nov 29 03:40:08 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdb/device/delete {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:40:09 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdb/device/delete" returned: 0 in 0.058s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:40:09 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[df61e4b0-09e5-4b34-afe9-af09299e8dcd]: (4, ('1', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:40:09 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-94d33106-5710-4dec-b066-5ed7efad2d33 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Checking to see if SCSI volumes sdb have been removed. {{(pid=75234) wait_for_volumes_removal /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:156}}
Nov 29 03:40:09 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-94d33106-5710-4dec-b066-5ed7efad2d33 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] SCSI volumes sdb have been removed. {{(pid=75234) wait_for_volumes_removal /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:166}}
Nov 29 03:40:09 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-94d33106-5710-4dec-b066-5ed7efad2d33 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:378addbb-8d90-40c5-ae61-c3a063116df9')] {{(pid=75234) _disconnect_connection /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Nov 29 03:40:09 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:378addbb-8d90-40c5-ae61-c3a063116df9 -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:40:09 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:378addbb-8d90-40c5-ae61-c3a063116df9 -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.031s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:40:09 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[11605807-18e7-4109-bbc2-89476001663c]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:40:09 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-94d33106-5710-4dec-b066-5ed7efad2d33 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:40:09 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:378addbb-8d90-40c5-ae61-c3a063116df9 -p 172.16.0.220:3260 --logout {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:40:09 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:378addbb-8d90-40c5-ae61-c3a063116df9 -p 172.16.0.220:3260 --logout" returned: 0 in 0.057s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:40:09 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[41ba8e80-0384-472b-9d43-b7357887028d]: (4, ('Logging out of session [sid: 58, target: iqn.2010-10.org.openstack:378addbb-8d90-40c5-ae61-c3a063116df9, portal: 172.16.0.220,3260]\nLogout of [sid: 58, target: iqn.2010-10.org.openstack:378addbb-8d90-40c5-ae61-c3a063116df9, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:40:09 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-94d33106-5710-4dec-b066-5ed7efad2d33 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 58, target: iqn.2010-10.org.openstack:378addbb-8d90-40c5-ae61-c3a063116df9, portal: 172.16.0.220,3260]
Nov 29 03:40:09 user nova-compute[75234]: Logout of [sid: 58, target: iqn.2010-10.org.openstack:378addbb-8d90-40c5-ae61-c3a063116df9, portal: 172.16.0.220,3260] successful.
Nov 29 03:40:09 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:40:09 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:378addbb-8d90-40c5-ae61-c3a063116df9 -p 172.16.0.220:3260 --op delete {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:40:09 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:378addbb-8d90-40c5-ae61-c3a063116df9 -p 172.16.0.220:3260 --op delete" returned: 0 in 0.019s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:40:09 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[538bc1f4-f545-4307-b799-d638a4bfa7f0]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:40:09 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-94d33106-5710-4dec-b066-5ed7efad2d33 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:40:09 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-94d33106-5710-4dec-b066-5ed7efad2d33 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.368s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:87}}
Nov 29 03:40:09 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-94d33106-5710-4dec-b066-5ed7efad2d33 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] <== disconnect_volume: return (369ms) None {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:203}}
Nov 29 03:40:09 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-94d33106-5710-4dec-b066-5ed7efad2d33 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: 45059fbb-59d4-49ae-bccd-356edea7f287] Disconnected iSCSI Volume {{(pid=75234) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:79}}
Nov 29 03:40:09 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-94d33106-5710-4dec-b066-5ed7efad2d33 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: 45059fbb-59d4-49ae-bccd-356edea7f287] Deleting instance files /opt/stack/data/nova/instances/45059fbb-59d4-49ae-bccd-356edea7f287_del
Nov 29 03:40:09 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-94d33106-5710-4dec-b066-5ed7efad2d33 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: 45059fbb-59d4-49ae-bccd-356edea7f287] Deletion of /opt/stack/data/nova/instances/45059fbb-59d4-49ae-bccd-356edea7f287_del complete
Nov 29 03:40:09 user nova-compute[75234]: INFO nova.compute.manager [None req-94d33106-5710-4dec-b066-5ed7efad2d33 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: 45059fbb-59d4-49ae-bccd-356edea7f287] Took 1.23 seconds to destroy the instance on the hypervisor.
Nov 29 03:40:09 user nova-compute[75234]: DEBUG oslo.service.loopingcall [None req-94d33106-5710-4dec-b066-5ed7efad2d33 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=75234) func /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/loopingcall.py:435}}
Nov 29 03:40:09 user nova-compute[75234]: DEBUG nova.compute.manager [-] [instance: 45059fbb-59d4-49ae-bccd-356edea7f287] Deallocating network for instance {{(pid=75234) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2279}}
Nov 29 03:40:09 user nova-compute[75234]: DEBUG nova.network.neutron [-] [instance: 45059fbb-59d4-49ae-bccd-356edea7f287] deallocate_for_instance() {{(pid=75234) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1802}}
Nov 29 03:40:09 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:40:09 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:40:09 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:40:09 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:40:09 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:40:09 user nova-compute[75234]: DEBUG nova.network.neutron [-] [instance: 45059fbb-59d4-49ae-bccd-356edea7f287] Updating instance_info_cache with network_info: [] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:40:09 user nova-compute[75234]: INFO nova.compute.manager [-] [instance: 45059fbb-59d4-49ae-bccd-356edea7f287] Took 0.77 seconds to deallocate network for instance.
Nov 29 03:40:09 user nova-compute[75234]: DEBUG nova.compute.manager [req-1d524069-d064-4994-b46e-06c7c5f0797f req-2a2134d4-2498-49f6-ad5c-52b9fa7e4e81 service nova] [instance: 45059fbb-59d4-49ae-bccd-356edea7f287] Received event network-vif-deleted-420b9e5e-a22f-4e87-90d4-9b4430c66c4e {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:40:10 user nova-compute[75234]: DEBUG nova.compute.manager [req-48784bab-3e63-49ae-9da4-515d43e7a972 req-dfd9beca-8ecb-4137-adf8-93e92e348ba2 service nova] [instance: 45059fbb-59d4-49ae-bccd-356edea7f287] Received event network-vif-plugged-420b9e5e-a22f-4e87-90d4-9b4430c66c4e {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:40:10 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-48784bab-3e63-49ae-9da4-515d43e7a972 req-dfd9beca-8ecb-4137-adf8-93e92e348ba2 service nova] Acquiring lock "45059fbb-59d4-49ae-bccd-356edea7f287-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:40:10 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-48784bab-3e63-49ae-9da4-515d43e7a972 req-dfd9beca-8ecb-4137-adf8-93e92e348ba2 service nova] Lock "45059fbb-59d4-49ae-bccd-356edea7f287-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:40:10 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-48784bab-3e63-49ae-9da4-515d43e7a972 req-dfd9beca-8ecb-4137-adf8-93e92e348ba2 service nova] Lock "45059fbb-59d4-49ae-bccd-356edea7f287-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:40:10 user nova-compute[75234]: DEBUG nova.compute.manager [req-48784bab-3e63-49ae-9da4-515d43e7a972 req-dfd9beca-8ecb-4137-adf8-93e92e348ba2 service nova] [instance: 45059fbb-59d4-49ae-bccd-356edea7f287] No waiting events found dispatching network-vif-plugged-420b9e5e-a22f-4e87-90d4-9b4430c66c4e {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:40:10 user nova-compute[75234]: WARNING nova.compute.manager [req-48784bab-3e63-49ae-9da4-515d43e7a972 req-dfd9beca-8ecb-4137-adf8-93e92e348ba2 service nova] [instance: 45059fbb-59d4-49ae-bccd-356edea7f287] Received unexpected event network-vif-plugged-420b9e5e-a22f-4e87-90d4-9b4430c66c4e for instance with vm_state active and task_state deleting.
Nov 29 03:40:10 user nova-compute[75234]: DEBUG nova.compute.manager [req-48784bab-3e63-49ae-9da4-515d43e7a972 req-dfd9beca-8ecb-4137-adf8-93e92e348ba2 service nova] [instance: 45059fbb-59d4-49ae-bccd-356edea7f287] Received event network-vif-plugged-420b9e5e-a22f-4e87-90d4-9b4430c66c4e {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:40:10 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-48784bab-3e63-49ae-9da4-515d43e7a972 req-dfd9beca-8ecb-4137-adf8-93e92e348ba2 service nova] Acquiring lock "45059fbb-59d4-49ae-bccd-356edea7f287-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:40:10 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-48784bab-3e63-49ae-9da4-515d43e7a972 req-dfd9beca-8ecb-4137-adf8-93e92e348ba2 service nova] Lock "45059fbb-59d4-49ae-bccd-356edea7f287-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:40:10 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-48784bab-3e63-49ae-9da4-515d43e7a972 req-dfd9beca-8ecb-4137-adf8-93e92e348ba2 service nova] Lock "45059fbb-59d4-49ae-bccd-356edea7f287-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:40:10 user nova-compute[75234]: DEBUG nova.compute.manager [req-48784bab-3e63-49ae-9da4-515d43e7a972 req-dfd9beca-8ecb-4137-adf8-93e92e348ba2 service nova] [instance: 45059fbb-59d4-49ae-bccd-356edea7f287] No waiting events found dispatching network-vif-plugged-420b9e5e-a22f-4e87-90d4-9b4430c66c4e {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:40:10 user nova-compute[75234]: WARNING nova.compute.manager [req-48784bab-3e63-49ae-9da4-515d43e7a972 req-dfd9beca-8ecb-4137-adf8-93e92e348ba2 service nova] [instance: 45059fbb-59d4-49ae-bccd-356edea7f287] Received unexpected event network-vif-plugged-420b9e5e-a22f-4e87-90d4-9b4430c66c4e for instance with vm_state active and task_state deleting.
Nov 29 03:40:10 user nova-compute[75234]: DEBUG nova.compute.manager [req-48784bab-3e63-49ae-9da4-515d43e7a972 req-dfd9beca-8ecb-4137-adf8-93e92e348ba2 service nova] [instance: 45059fbb-59d4-49ae-bccd-356edea7f287] Received event network-vif-plugged-420b9e5e-a22f-4e87-90d4-9b4430c66c4e {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:40:10 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-48784bab-3e63-49ae-9da4-515d43e7a972 req-dfd9beca-8ecb-4137-adf8-93e92e348ba2 service nova] Acquiring lock "45059fbb-59d4-49ae-bccd-356edea7f287-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:40:10 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-48784bab-3e63-49ae-9da4-515d43e7a972 req-dfd9beca-8ecb-4137-adf8-93e92e348ba2 service nova] Lock "45059fbb-59d4-49ae-bccd-356edea7f287-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:40:10 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-48784bab-3e63-49ae-9da4-515d43e7a972 req-dfd9beca-8ecb-4137-adf8-93e92e348ba2 service nova] Lock "45059fbb-59d4-49ae-bccd-356edea7f287-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:40:10 user nova-compute[75234]: DEBUG nova.compute.manager [req-48784bab-3e63-49ae-9da4-515d43e7a972 req-dfd9beca-8ecb-4137-adf8-93e92e348ba2 service nova] [instance: 45059fbb-59d4-49ae-bccd-356edea7f287] No waiting events found dispatching network-vif-plugged-420b9e5e-a22f-4e87-90d4-9b4430c66c4e {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:40:10 user nova-compute[75234]: WARNING nova.compute.manager [req-48784bab-3e63-49ae-9da4-515d43e7a972 req-dfd9beca-8ecb-4137-adf8-93e92e348ba2 service nova] [instance: 45059fbb-59d4-49ae-bccd-356edea7f287] Received unexpected event network-vif-plugged-420b9e5e-a22f-4e87-90d4-9b4430c66c4e for instance with vm_state active and task_state deleting.
Nov 29 03:40:10 user nova-compute[75234]: DEBUG nova.compute.manager [req-48784bab-3e63-49ae-9da4-515d43e7a972 req-dfd9beca-8ecb-4137-adf8-93e92e348ba2 service nova] [instance: 45059fbb-59d4-49ae-bccd-356edea7f287] Received event network-vif-unplugged-420b9e5e-a22f-4e87-90d4-9b4430c66c4e {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:40:10 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-48784bab-3e63-49ae-9da4-515d43e7a972 req-dfd9beca-8ecb-4137-adf8-93e92e348ba2 service nova] Acquiring lock "45059fbb-59d4-49ae-bccd-356edea7f287-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:40:10 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-48784bab-3e63-49ae-9da4-515d43e7a972 req-dfd9beca-8ecb-4137-adf8-93e92e348ba2 service nova] Lock "45059fbb-59d4-49ae-bccd-356edea7f287-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:40:10 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-48784bab-3e63-49ae-9da4-515d43e7a972 req-dfd9beca-8ecb-4137-adf8-93e92e348ba2 service nova] Lock "45059fbb-59d4-49ae-bccd-356edea7f287-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:40:10 user nova-compute[75234]: DEBUG nova.compute.manager [req-48784bab-3e63-49ae-9da4-515d43e7a972 req-dfd9beca-8ecb-4137-adf8-93e92e348ba2 service nova] [instance: 45059fbb-59d4-49ae-bccd-356edea7f287] No waiting events found dispatching network-vif-unplugged-420b9e5e-a22f-4e87-90d4-9b4430c66c4e {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:40:10 user nova-compute[75234]: DEBUG nova.compute.manager [req-48784bab-3e63-49ae-9da4-515d43e7a972 req-dfd9beca-8ecb-4137-adf8-93e92e348ba2 service nova] [instance: 45059fbb-59d4-49ae-bccd-356edea7f287] Received event network-vif-unplugged-420b9e5e-a22f-4e87-90d4-9b4430c66c4e for instance with task_state deleting. {{(pid=75234) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10890}}
Nov 29 03:40:10 user nova-compute[75234]: DEBUG nova.compute.manager [req-48784bab-3e63-49ae-9da4-515d43e7a972 req-dfd9beca-8ecb-4137-adf8-93e92e348ba2 service nova] [instance: 45059fbb-59d4-49ae-bccd-356edea7f287] Received event network-vif-plugged-420b9e5e-a22f-4e87-90d4-9b4430c66c4e {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:40:10 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-48784bab-3e63-49ae-9da4-515d43e7a972 req-dfd9beca-8ecb-4137-adf8-93e92e348ba2 service nova] Acquiring lock "45059fbb-59d4-49ae-bccd-356edea7f287-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:40:10 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-48784bab-3e63-49ae-9da4-515d43e7a972 req-dfd9beca-8ecb-4137-adf8-93e92e348ba2 service nova] Lock "45059fbb-59d4-49ae-bccd-356edea7f287-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:40:10 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-48784bab-3e63-49ae-9da4-515d43e7a972 req-dfd9beca-8ecb-4137-adf8-93e92e348ba2 service nova] Lock "45059fbb-59d4-49ae-bccd-356edea7f287-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:40:10 user nova-compute[75234]: DEBUG nova.compute.manager [req-48784bab-3e63-49ae-9da4-515d43e7a972 req-dfd9beca-8ecb-4137-adf8-93e92e348ba2 service nova] [instance: 45059fbb-59d4-49ae-bccd-356edea7f287] No waiting events found dispatching network-vif-plugged-420b9e5e-a22f-4e87-90d4-9b4430c66c4e {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:40:10 user nova-compute[75234]: WARNING nova.compute.manager [req-48784bab-3e63-49ae-9da4-515d43e7a972 req-dfd9beca-8ecb-4137-adf8-93e92e348ba2 service nova] [instance: 45059fbb-59d4-49ae-bccd-356edea7f287] Received unexpected event network-vif-plugged-420b9e5e-a22f-4e87-90d4-9b4430c66c4e for instance with vm_state active and task_state deleting.
Nov 29 03:40:10 user nova-compute[75234]: INFO nova.compute.manager [None req-94d33106-5710-4dec-b066-5ed7efad2d33 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: 45059fbb-59d4-49ae-bccd-356edea7f287] Took 0.75 seconds to detach 1 volumes for instance.
Nov 29 03:40:10 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-94d33106-5710-4dec-b066-5ed7efad2d33 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:40:10 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-94d33106-5710-4dec-b066-5ed7efad2d33 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:40:10 user nova-compute[75234]: DEBUG nova.compute.provider_tree [None req-94d33106-5710-4dec-b066-5ed7efad2d33 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Inventory has not changed in ProviderTree for provider: 4d291220-7c37-4839-a0e3-b23c66502b74 {{(pid=75234) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Nov 29 03:40:10 user nova-compute[75234]: DEBUG nova.scheduler.client.report [None req-94d33106-5710-4dec-b066-5ed7efad2d33 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Inventory has not changed for provider 4d291220-7c37-4839-a0e3-b23c66502b74 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=75234) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Nov 29 03:40:10 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-94d33106-5710-4dec-b066-5ed7efad2d33 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.134s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:40:10 user nova-compute[75234]: INFO nova.scheduler.client.report [None req-94d33106-5710-4dec-b066-5ed7efad2d33 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Deleted allocations for instance 45059fbb-59d4-49ae-bccd-356edea7f287
Nov 29 03:40:11 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-94d33106-5710-4dec-b066-5ed7efad2d33 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Lock "45059fbb-59d4-49ae-bccd-356edea7f287" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 3.063s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:40:12 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:40:13 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:40:16 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:40:18 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:40:18 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:40:18 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=75234) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10541}}
Nov 29 03:40:18 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:40:19 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:40:22 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager.update_available_resource {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:40:22 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:40:22 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:40:22 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:40:22 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Auditing locally available compute resources for user (node: user) {{(pid=75234) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:907}}
Nov 29 03:40:22 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/65d48c93-b6da-4f51-9d5a-0676dda62a60/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:40:22 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/65d48c93-b6da-4f51-9d5a-0676dda62a60/disk --force-share --output=json" returned: 0 in 0.167s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:40:22 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running cmd (subprocess): /opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/65d48c93-b6da-4f51-9d5a-0676dda62a60/disk --force-share --output=json {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:40:22 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CMD "/opt/stack/data/venv/bin/python3.10 -m oslo_concurrency.prlimit --as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C qemu-img info /opt/stack/data/nova/instances/65d48c93-b6da-4f51-9d5a-0676dda62a60/disk --force-share --output=json" returned: 0 in 0.174s {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:40:23 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:40:23 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:40:23 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Hypervisor/Node resource view: name=user free_ram=8923MB free_disk=25.041278839111328GB free_vcpus=11 pci_devices=[{"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}] {{(pid=75234) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1106}}
Nov 29 03:40:23 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:40:23 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:40:23 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance 65d48c93-b6da-4f51-9d5a-0676dda62a60 actively managed on this compute host and has allocations in placement: {'resources': {'DISK_GB': 1, 'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:40:23 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Total usable vcpus: 12, total allocated vcpus: 1 {{(pid=75234) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1129}}
Nov 29 03:40:23 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Final resource view: name=user phys_ram=16011MB used_ram=1024MB phys_disk=40GB used_disk=1GB total_vcpus=12 used_vcpus=1 pci_stats=[] {{(pid=75234) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1138}}
Nov 29 03:40:23 user nova-compute[75234]: DEBUG nova.compute.provider_tree [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Inventory has not changed in ProviderTree for provider: 4d291220-7c37-4839-a0e3-b23c66502b74 {{(pid=75234) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Nov 29 03:40:23 user nova-compute[75234]: DEBUG nova.scheduler.client.report [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Inventory has not changed for provider 4d291220-7c37-4839-a0e3-b23c66502b74 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=75234) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Nov 29 03:40:23 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Compute_service record updated for user:user {{(pid=75234) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:1067}}
Nov 29 03:40:23 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.221s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:40:23 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:40:27 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:40:27 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Starting heal instance info cache {{(pid=75234) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9922}}
Nov 29 03:40:27 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Rebuilding the list of instances to heal {{(pid=75234) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9926}}
Nov 29 03:40:27 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Acquiring lock "refresh_cache-65d48c93-b6da-4f51-9d5a-0676dda62a60" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:40:27 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Acquired lock "refresh_cache-65d48c93-b6da-4f51-9d5a-0676dda62a60" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:40:27 user nova-compute[75234]: DEBUG nova.network.neutron [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] [instance: 65d48c93-b6da-4f51-9d5a-0676dda62a60] Forcefully refreshing network info cache for instance {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2003}}
Nov 29 03:40:27 user nova-compute[75234]: DEBUG nova.objects.instance [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lazy-loading 'info_cache' on Instance uuid 65d48c93-b6da-4f51-9d5a-0676dda62a60 {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:40:28 user nova-compute[75234]: DEBUG nova.network.neutron [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] [instance: 65d48c93-b6da-4f51-9d5a-0676dda62a60] Updating instance_info_cache with network_info: [{"id": "967835a2-4b55-4ae6-b071-91233af412fd", "address": "fa:16:3e:ee:3c:e2", "network": {"id": "4d086e08-7798-4cfe-9365-86be6f0a52d2", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-305624008-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "db00bcd0c82c44f1bda739e9857f6e84", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap967835a2-4b", "ovs_interfaceid": "967835a2-4b55-4ae6-b071-91233af412fd", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:40:28 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Releasing lock "refresh_cache-65d48c93-b6da-4f51-9d5a-0676dda62a60" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:40:28 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] [instance: 65d48c93-b6da-4f51-9d5a-0676dda62a60] Updated the network info_cache for instance {{(pid=75234) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9993}}
Nov 29 03:40:28 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:40:28 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:40:28 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:40:28 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:40:30 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._sync_scheduler_instance_info {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:40:33 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-a630209b-201c-43f7-a534-14af52af1588 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Acquiring lock "65d48c93-b6da-4f51-9d5a-0676dda62a60" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:40:33 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-a630209b-201c-43f7-a534-14af52af1588 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Lock "65d48c93-b6da-4f51-9d5a-0676dda62a60" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.003s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:40:33 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-a630209b-201c-43f7-a534-14af52af1588 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Acquiring lock "65d48c93-b6da-4f51-9d5a-0676dda62a60-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:40:33 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-a630209b-201c-43f7-a534-14af52af1588 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Lock "65d48c93-b6da-4f51-9d5a-0676dda62a60-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:40:33 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-a630209b-201c-43f7-a534-14af52af1588 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Lock "65d48c93-b6da-4f51-9d5a-0676dda62a60-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:40:33 user nova-compute[75234]: INFO nova.compute.manager [None req-a630209b-201c-43f7-a534-14af52af1588 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] [instance: 65d48c93-b6da-4f51-9d5a-0676dda62a60] Terminating instance
Nov 29 03:40:33 user nova-compute[75234]: DEBUG nova.compute.manager [None req-a630209b-201c-43f7-a534-14af52af1588 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] [instance: 65d48c93-b6da-4f51-9d5a-0676dda62a60] Start destroying the instance on the hypervisor. {{(pid=75234) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3140}}
Nov 29 03:40:33 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:40:33 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:40:33 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:40:33 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:40:33 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:40:33 user nova-compute[75234]: DEBUG nova.compute.manager [req-9b19a1c3-444b-437b-b9a9-f68d8fca3b86 req-1b236456-201b-4316-afbe-c3084ddf9863 service nova] [instance: 65d48c93-b6da-4f51-9d5a-0676dda62a60] Received event network-vif-unplugged-967835a2-4b55-4ae6-b071-91233af412fd {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:40:33 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-9b19a1c3-444b-437b-b9a9-f68d8fca3b86 req-1b236456-201b-4316-afbe-c3084ddf9863 service nova] Acquiring lock "65d48c93-b6da-4f51-9d5a-0676dda62a60-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:40:33 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-9b19a1c3-444b-437b-b9a9-f68d8fca3b86 req-1b236456-201b-4316-afbe-c3084ddf9863 service nova] Lock "65d48c93-b6da-4f51-9d5a-0676dda62a60-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:40:33 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-9b19a1c3-444b-437b-b9a9-f68d8fca3b86 req-1b236456-201b-4316-afbe-c3084ddf9863 service nova] Lock "65d48c93-b6da-4f51-9d5a-0676dda62a60-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:40:33 user nova-compute[75234]: DEBUG nova.compute.manager [req-9b19a1c3-444b-437b-b9a9-f68d8fca3b86 req-1b236456-201b-4316-afbe-c3084ddf9863 service nova] [instance: 65d48c93-b6da-4f51-9d5a-0676dda62a60] No waiting events found dispatching network-vif-unplugged-967835a2-4b55-4ae6-b071-91233af412fd {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:40:33 user nova-compute[75234]: DEBUG nova.compute.manager [req-9b19a1c3-444b-437b-b9a9-f68d8fca3b86 req-1b236456-201b-4316-afbe-c3084ddf9863 service nova] [instance: 65d48c93-b6da-4f51-9d5a-0676dda62a60] Received event network-vif-unplugged-967835a2-4b55-4ae6-b071-91233af412fd for instance with task_state deleting. {{(pid=75234) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10890}}
Nov 29 03:40:34 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:40:34 user nova-compute[75234]: INFO nova.virt.libvirt.driver [-] [instance: 65d48c93-b6da-4f51-9d5a-0676dda62a60] Instance destroyed successfully.
Nov 29 03:40:34 user nova-compute[75234]: DEBUG nova.objects.instance [None req-a630209b-201c-43f7-a534-14af52af1588 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Lazy-loading 'resources' on Instance uuid 65d48c93-b6da-4f51-9d5a-0676dda62a60 {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:40:34 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-a630209b-201c-43f7-a534-14af52af1588 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-11-29T03:20:55Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description=None,display_name='tempest-ServerBootFromVolumeStableRescueTest-server-378302862',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-serverbootfromvolumestablerescuetest-server-378302862',id=36,image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data=None,key_name=None,keypairs=<?>,launch_index=0,launched_at=2023-11-29T03:21:02Z,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='db00bcd0c82c44f1bda739e9857f6e84',ramdisk_id='',reservation_id='r-ihbm4mrc',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='96c30ba7-9b73-4a30-8ba4-7138879573b7',image_container_format='bare',image_disk_format='qcow2',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-ServerBootFromVolumeStableRescueTest-1601266069',owner_user_name='tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-11-29T03:22:53Z,user_data=None,user_id='877aa877f13a4f84ac4725278010aea3',uuid=65d48c93-b6da-4f51-9d5a-0676dda62a60,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "967835a2-4b55-4ae6-b071-91233af412fd", "address": "fa:16:3e:ee:3c:e2", "network": {"id": "4d086e08-7798-4cfe-9365-86be6f0a52d2", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-305624008-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "db00bcd0c82c44f1bda739e9857f6e84", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap967835a2-4b", "ovs_interfaceid": "967835a2-4b55-4ae6-b071-91233af412fd", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Nov 29 03:40:34 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-a630209b-201c-43f7-a534-14af52af1588 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Converting VIF {"id": "967835a2-4b55-4ae6-b071-91233af412fd", "address": "fa:16:3e:ee:3c:e2", "network": {"id": "4d086e08-7798-4cfe-9365-86be6f0a52d2", "bridge": "br-int", "label": "tempest-ServerBootFromVolumeStableRescueTest-305624008-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.11", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "db00bcd0c82c44f1bda739e9857f6e84", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap967835a2-4b", "ovs_interfaceid": "967835a2-4b55-4ae6-b071-91233af412fd", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:40:34 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-a630209b-201c-43f7-a534-14af52af1588 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:ee:3c:e2,bridge_name='br-int',has_traffic_filtering=True,id=967835a2-4b55-4ae6-b071-91233af412fd,network=Network(4d086e08-7798-4cfe-9365-86be6f0a52d2),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap967835a2-4b') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:40:34 user nova-compute[75234]: DEBUG os_vif [None req-a630209b-201c-43f7-a534-14af52af1588 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Unplugging vif VIFOpenVSwitch(active=True,address=fa:16:3e:ee:3c:e2,bridge_name='br-int',has_traffic_filtering=True,id=967835a2-4b55-4ae6-b071-91233af412fd,network=Network(4d086e08-7798-4cfe-9365-86be6f0a52d2),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap967835a2-4b') {{(pid=75234) unplug /opt/stack/data/venv/lib/python3.10/site-packages/os_vif/__init__.py:109}}
Nov 29 03:40:34 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:40:34 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tap967835a2-4b, bridge=br-int, if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:40:34 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:40:34 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:40:34 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:40:34 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbDestroyCommand(_result=None, table=QoS, record=4b2e8007-c067-47e8-bcdb-22ee4cd31118) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:40:34 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:40:34 user nova-compute[75234]: INFO os_vif [None req-a630209b-201c-43f7-a534-14af52af1588 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Successfully unplugged vif VIFOpenVSwitch(active=True,address=fa:16:3e:ee:3c:e2,bridge_name='br-int',has_traffic_filtering=True,id=967835a2-4b55-4ae6-b071-91233af412fd,network=Network(4d086e08-7798-4cfe-9365-86be6f0a52d2),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap967835a2-4b')
Nov 29 03:40:34 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-a630209b-201c-43f7-a534-14af52af1588 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] [instance: 65d48c93-b6da-4f51-9d5a-0676dda62a60] Deleting instance files /opt/stack/data/nova/instances/65d48c93-b6da-4f51-9d5a-0676dda62a60_del
Nov 29 03:40:34 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-a630209b-201c-43f7-a534-14af52af1588 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] [instance: 65d48c93-b6da-4f51-9d5a-0676dda62a60] Deletion of /opt/stack/data/nova/instances/65d48c93-b6da-4f51-9d5a-0676dda62a60_del complete
Nov 29 03:40:34 user nova-compute[75234]: INFO nova.compute.manager [None req-a630209b-201c-43f7-a534-14af52af1588 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] [instance: 65d48c93-b6da-4f51-9d5a-0676dda62a60] Took 0.91 seconds to destroy the instance on the hypervisor.
Nov 29 03:40:34 user nova-compute[75234]: DEBUG oslo.service.loopingcall [None req-a630209b-201c-43f7-a534-14af52af1588 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=75234) func /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/loopingcall.py:435}}
Nov 29 03:40:34 user nova-compute[75234]: DEBUG nova.compute.manager [-] [instance: 65d48c93-b6da-4f51-9d5a-0676dda62a60] Deallocating network for instance {{(pid=75234) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2279}}
Nov 29 03:40:34 user nova-compute[75234]: DEBUG nova.network.neutron [-] [instance: 65d48c93-b6da-4f51-9d5a-0676dda62a60] deallocate_for_instance() {{(pid=75234) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1802}}
Nov 29 03:40:34 user nova-compute[75234]: DEBUG nova.network.neutron [-] [instance: 65d48c93-b6da-4f51-9d5a-0676dda62a60] Updating instance_info_cache with network_info: [] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:40:34 user nova-compute[75234]: INFO nova.compute.manager [-] [instance: 65d48c93-b6da-4f51-9d5a-0676dda62a60] Took 0.39 seconds to deallocate network for instance.
Nov 29 03:40:34 user nova-compute[75234]: DEBUG nova.compute.manager [req-ca7235e2-ad71-4d5b-8820-7ea367a88bf6 req-3eb8e588-e87e-4e39-93c3-4c482d3efd08 service nova] [instance: 65d48c93-b6da-4f51-9d5a-0676dda62a60] Received event network-vif-deleted-967835a2-4b55-4ae6-b071-91233af412fd {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:40:34 user nova-compute[75234]: INFO nova.compute.manager [req-ca7235e2-ad71-4d5b-8820-7ea367a88bf6 req-3eb8e588-e87e-4e39-93c3-4c482d3efd08 service nova] [instance: 65d48c93-b6da-4f51-9d5a-0676dda62a60] Neutron deleted interface 967835a2-4b55-4ae6-b071-91233af412fd; detaching it from the instance and deleting it from the info cache
Nov 29 03:40:34 user nova-compute[75234]: DEBUG nova.network.neutron [req-ca7235e2-ad71-4d5b-8820-7ea367a88bf6 req-3eb8e588-e87e-4e39-93c3-4c482d3efd08 service nova] [instance: 65d48c93-b6da-4f51-9d5a-0676dda62a60] Updating instance_info_cache with network_info: [] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:40:34 user nova-compute[75234]: DEBUG nova.compute.manager [req-ca7235e2-ad71-4d5b-8820-7ea367a88bf6 req-3eb8e588-e87e-4e39-93c3-4c482d3efd08 service nova] [instance: 65d48c93-b6da-4f51-9d5a-0676dda62a60] Detach interface failed, port_id=967835a2-4b55-4ae6-b071-91233af412fd, reason: Instance 65d48c93-b6da-4f51-9d5a-0676dda62a60 could not be found. {{(pid=75234) _process_instance_vif_deleted_event /opt/stack/nova/nova/compute/manager.py:10946}}
Nov 29 03:40:34 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-a630209b-201c-43f7-a534-14af52af1588 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:40:34 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-a630209b-201c-43f7-a534-14af52af1588 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:40:34 user nova-compute[75234]: DEBUG nova.compute.provider_tree [None req-a630209b-201c-43f7-a534-14af52af1588 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Inventory has not changed in ProviderTree for provider: 4d291220-7c37-4839-a0e3-b23c66502b74 {{(pid=75234) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Nov 29 03:40:34 user nova-compute[75234]: DEBUG nova.scheduler.client.report [None req-a630209b-201c-43f7-a534-14af52af1588 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Inventory has not changed for provider 4d291220-7c37-4839-a0e3-b23c66502b74 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=75234) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Nov 29 03:40:34 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-a630209b-201c-43f7-a534-14af52af1588 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.145s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:40:34 user nova-compute[75234]: INFO nova.scheduler.client.report [None req-a630209b-201c-43f7-a534-14af52af1588 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Deleted allocations for instance 65d48c93-b6da-4f51-9d5a-0676dda62a60
Nov 29 03:40:35 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-a630209b-201c-43f7-a534-14af52af1588 tempest-ServerBootFromVolumeStableRescueTest-1601266069 tempest-ServerBootFromVolumeStableRescueTest-1601266069-project-member] Lock "65d48c93-b6da-4f51-9d5a-0676dda62a60" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 1.619s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:40:35 user nova-compute[75234]: DEBUG nova.compute.manager [req-4742e78d-5b19-4c38-8f19-f536fed37a61 req-b20fcf4f-c341-4cd2-ae1a-7ee29d6815aa service nova] [instance: 65d48c93-b6da-4f51-9d5a-0676dda62a60] Received event network-vif-plugged-967835a2-4b55-4ae6-b071-91233af412fd {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:40:35 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-4742e78d-5b19-4c38-8f19-f536fed37a61 req-b20fcf4f-c341-4cd2-ae1a-7ee29d6815aa service nova] Acquiring lock "65d48c93-b6da-4f51-9d5a-0676dda62a60-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:40:35 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-4742e78d-5b19-4c38-8f19-f536fed37a61 req-b20fcf4f-c341-4cd2-ae1a-7ee29d6815aa service nova] Lock "65d48c93-b6da-4f51-9d5a-0676dda62a60-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:40:35 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-4742e78d-5b19-4c38-8f19-f536fed37a61 req-b20fcf4f-c341-4cd2-ae1a-7ee29d6815aa service nova] Lock "65d48c93-b6da-4f51-9d5a-0676dda62a60-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:40:35 user nova-compute[75234]: DEBUG nova.compute.manager [req-4742e78d-5b19-4c38-8f19-f536fed37a61 req-b20fcf4f-c341-4cd2-ae1a-7ee29d6815aa service nova] [instance: 65d48c93-b6da-4f51-9d5a-0676dda62a60] No waiting events found dispatching network-vif-plugged-967835a2-4b55-4ae6-b071-91233af412fd {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:40:35 user nova-compute[75234]: WARNING nova.compute.manager [req-4742e78d-5b19-4c38-8f19-f536fed37a61 req-b20fcf4f-c341-4cd2-ae1a-7ee29d6815aa service nova] [instance: 65d48c93-b6da-4f51-9d5a-0676dda62a60] Received unexpected event network-vif-plugged-967835a2-4b55-4ae6-b071-91233af412fd for instance with vm_state deleted and task_state None.
Nov 29 03:40:39 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:40:44 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:40:49 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:40:49 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:40:49 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5002 ms, sending inactivity probe {{(pid=75234) run /opt/stack/data/venv/lib/python3.10/site-packages/ovs/reconnect.py:117}}
Nov 29 03:40:49 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=75234) _transition /opt/stack/data/venv/lib/python3.10/site-packages/ovs/reconnect.py:519}}
Nov 29 03:40:49 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=75234) _transition /opt/stack/data/venv/lib/python3.10/site-packages/ovs/reconnect.py:519}}
Nov 29 03:40:49 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:40:54 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:40:59 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:40:59 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-b8d98ced-f0fd-40c8-b1b7-5ed5946ba755 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Acquiring lock "95afc929-9f13-4f9a-9ac9-eae0b5e15a4d" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:40:59 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-b8d98ced-f0fd-40c8-b1b7-5ed5946ba755 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Lock "95afc929-9f13-4f9a-9ac9-eae0b5e15a4d" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:40:59 user nova-compute[75234]: DEBUG nova.compute.manager [None req-b8d98ced-f0fd-40c8-b1b7-5ed5946ba755 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: 95afc929-9f13-4f9a-9ac9-eae0b5e15a4d] Starting instance... {{(pid=75234) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2422}}
Nov 29 03:40:59 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-b8d98ced-f0fd-40c8-b1b7-5ed5946ba755 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:40:59 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-b8d98ced-f0fd-40c8-b1b7-5ed5946ba755 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:40:59 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-b8d98ced-f0fd-40c8-b1b7-5ed5946ba755 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=75234) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2366}}
Nov 29 03:40:59 user nova-compute[75234]: INFO nova.compute.claims [None req-b8d98ced-f0fd-40c8-b1b7-5ed5946ba755 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: 95afc929-9f13-4f9a-9ac9-eae0b5e15a4d] Claim successful on node user
Nov 29 03:40:59 user nova-compute[75234]: DEBUG nova.compute.provider_tree [None req-b8d98ced-f0fd-40c8-b1b7-5ed5946ba755 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Inventory has not changed in ProviderTree for provider: 4d291220-7c37-4839-a0e3-b23c66502b74 {{(pid=75234) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Nov 29 03:40:59 user nova-compute[75234]: DEBUG nova.scheduler.client.report [None req-b8d98ced-f0fd-40c8-b1b7-5ed5946ba755 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Inventory has not changed for provider 4d291220-7c37-4839-a0e3-b23c66502b74 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=75234) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Nov 29 03:41:00 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-b8d98ced-f0fd-40c8-b1b7-5ed5946ba755 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.203s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:41:00 user nova-compute[75234]: DEBUG nova.compute.manager [None req-b8d98ced-f0fd-40c8-b1b7-5ed5946ba755 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: 95afc929-9f13-4f9a-9ac9-eae0b5e15a4d] Start building networks asynchronously for instance. {{(pid=75234) _build_resources /opt/stack/nova/nova/compute/manager.py:2819}}
Nov 29 03:41:00 user nova-compute[75234]: DEBUG nova.compute.manager [None req-b8d98ced-f0fd-40c8-b1b7-5ed5946ba755 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: 95afc929-9f13-4f9a-9ac9-eae0b5e15a4d] Allocating IP information in the background. {{(pid=75234) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1972}}
Nov 29 03:41:00 user nova-compute[75234]: DEBUG nova.network.neutron [None req-b8d98ced-f0fd-40c8-b1b7-5ed5946ba755 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: 95afc929-9f13-4f9a-9ac9-eae0b5e15a4d] allocate_for_instance() {{(pid=75234) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1156}}
Nov 29 03:41:00 user nova-compute[75234]: DEBUG nova.policy [None req-b8d98ced-f0fd-40c8-b1b7-5ed5946ba755 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '7d1f1f11a9aa4c1db8627413eb963fa4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '81a265d3c15b4e9d826b35ddbd0ff443', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=75234) authorize /opt/stack/nova/nova/policy.py:203}}
Nov 29 03:41:00 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-b8d98ced-f0fd-40c8-b1b7-5ed5946ba755 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: 95afc929-9f13-4f9a-9ac9-eae0b5e15a4d] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Nov 29 03:41:00 user nova-compute[75234]: DEBUG nova.compute.manager [None req-b8d98ced-f0fd-40c8-b1b7-5ed5946ba755 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: 95afc929-9f13-4f9a-9ac9-eae0b5e15a4d] Start building block device mappings for instance. {{(pid=75234) _build_resources /opt/stack/nova/nova/compute/manager.py:2854}}
Nov 29 03:41:00 user nova-compute[75234]: INFO nova.virt.block_device [None req-b8d98ced-f0fd-40c8-b1b7-5ed5946ba755 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: 95afc929-9f13-4f9a-9ac9-eae0b5e15a4d] Booting with volume 378addbb-8d90-40c5-ae61-c3a063116df9 at /dev/vda
Nov 29 03:41:00 user nova-compute[75234]: DEBUG os_brick.utils [None req-b8d98ced-f0fd-40c8-b1b7-5ed5946ba755 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:176}}
Nov 29 03:41:00 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:41:00 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.014s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:41:00 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[45779e27-dc71-4ded-ad59-b704024cf719]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:dd1368c5ba2c\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:41:00 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt -v / -n -o SOURCE {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:41:00 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt -v / -n -o SOURCE" returned: 0 in 0.015s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:41:00 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[8b126a71-a738-47dc-af3b-0306350425c4]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:41:00 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:41:00 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.015s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:41:00 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[e7902ef0-6e3e-4289-947b-d5eab4d21de0]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:41:00 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[bf5ee542-1cbc-4f87-b6a1-c93ec2bd8647]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:41:00 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-b8d98ced-f0fd-40c8-b1b7-5ed5946ba755 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Running cmd (subprocess): nvme version {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:41:00 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-b8d98ced-f0fd-40c8-b1b7-5ed5946ba755 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] 'nvme version' failed. Not Retrying. {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:473}}
Nov 29 03:41:00 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.nvmeof [None req-b8d98ced-f0fd-40c8-b1b7-5ed5946ba755 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] nvme not present on system {{(pid=75234) nvme_present /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/nvmeof.py:756}}
Nov 29 03:41:00 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:41:00 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:473}}
Nov 29 03:41:00 user nova-compute[75234]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Nov 29 03:41:00 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[643cec3b-0c54-4e57-9595-5d8bb9e88bdc]: (4, '') {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:41:00 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.lightos [None req-b8d98ced-f0fd-40c8-b1b7-5ed5946ba755 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=75234) find_dsc /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/lightos.py:98}}
Nov 29 03:41:00 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.lightos [None req-b8d98ced-f0fd-40c8-b1b7-5ed5946ba755 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=75234) get_connector_properties /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/lightos.py:76}}
Nov 29 03:41:00 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.lightos [None req-b8d98ced-f0fd-40c8-b1b7-5ed5946ba755 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] LIGHTOS: no hostnqn found. {{(pid=75234) get_connector_properties /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/lightos.py:84}}
Nov 29 03:41:00 user nova-compute[75234]: DEBUG os_brick.utils [None req-b8d98ced-f0fd-40c8-b1b7-5ed5946ba755 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] <== get_connector_properties: return (124ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:dd1368c5ba2c', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:203}}
Nov 29 03:41:00 user nova-compute[75234]: DEBUG nova.virt.block_device [None req-b8d98ced-f0fd-40c8-b1b7-5ed5946ba755 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: 95afc929-9f13-4f9a-9ac9-eae0b5e15a4d] Updating existing volume attachment record: 75e0716f-0606-451b-a569-5d8a8200b356 {{(pid=75234) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Nov 29 03:41:01 user nova-compute[75234]: DEBUG nova.network.neutron [None req-b8d98ced-f0fd-40c8-b1b7-5ed5946ba755 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: 95afc929-9f13-4f9a-9ac9-eae0b5e15a4d] Successfully created port: ca1227f9-b2f1-42c5-a92a-c70cb94e52c1 {{(pid=75234) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:548}}
Nov 29 03:41:01 user nova-compute[75234]: DEBUG nova.network.neutron [None req-b8d98ced-f0fd-40c8-b1b7-5ed5946ba755 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: 95afc929-9f13-4f9a-9ac9-eae0b5e15a4d] Successfully updated port: ca1227f9-b2f1-42c5-a92a-c70cb94e52c1 {{(pid=75234) _update_port /opt/stack/nova/nova/network/neutron.py:586}}
Nov 29 03:41:01 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-b8d98ced-f0fd-40c8-b1b7-5ed5946ba755 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Acquiring lock "refresh_cache-95afc929-9f13-4f9a-9ac9-eae0b5e15a4d" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:41:01 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-b8d98ced-f0fd-40c8-b1b7-5ed5946ba755 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Acquired lock "refresh_cache-95afc929-9f13-4f9a-9ac9-eae0b5e15a4d" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:41:01 user nova-compute[75234]: DEBUG nova.network.neutron [None req-b8d98ced-f0fd-40c8-b1b7-5ed5946ba755 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: 95afc929-9f13-4f9a-9ac9-eae0b5e15a4d] Building network info cache for instance {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2009}}
Nov 29 03:41:01 user nova-compute[75234]: DEBUG nova.compute.manager [req-58986eb0-05be-44fc-b1d5-487cf18241ac req-41e5e7b9-d998-4333-a5e7-313d17a56d47 service nova] [instance: 95afc929-9f13-4f9a-9ac9-eae0b5e15a4d] Received event network-changed-ca1227f9-b2f1-42c5-a92a-c70cb94e52c1 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:41:01 user nova-compute[75234]: DEBUG nova.compute.manager [req-58986eb0-05be-44fc-b1d5-487cf18241ac req-41e5e7b9-d998-4333-a5e7-313d17a56d47 service nova] [instance: 95afc929-9f13-4f9a-9ac9-eae0b5e15a4d] Refreshing instance network info cache due to event network-changed-ca1227f9-b2f1-42c5-a92a-c70cb94e52c1. {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11117}}
Nov 29 03:41:01 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-58986eb0-05be-44fc-b1d5-487cf18241ac req-41e5e7b9-d998-4333-a5e7-313d17a56d47 service nova] Acquiring lock "refresh_cache-95afc929-9f13-4f9a-9ac9-eae0b5e15a4d" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:41:01 user nova-compute[75234]: DEBUG nova.network.neutron [None req-b8d98ced-f0fd-40c8-b1b7-5ed5946ba755 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: 95afc929-9f13-4f9a-9ac9-eae0b5e15a4d] Instance cache missing network info. {{(pid=75234) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3322}}
Nov 29 03:41:02 user nova-compute[75234]: DEBUG nova.network.neutron [None req-b8d98ced-f0fd-40c8-b1b7-5ed5946ba755 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: 95afc929-9f13-4f9a-9ac9-eae0b5e15a4d] Updating instance_info_cache with network_info: [{"id": "ca1227f9-b2f1-42c5-a92a-c70cb94e52c1", "address": "fa:16:3e:02:94:55", "network": {"id": "a3f397d3-fd75-4a6e-a815-cfcedcca15fb", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-678570085-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "81a265d3c15b4e9d826b35ddbd0ff443", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapca1227f9-b2", "ovs_interfaceid": "ca1227f9-b2f1-42c5-a92a-c70cb94e52c1", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:41:02 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-b8d98ced-f0fd-40c8-b1b7-5ed5946ba755 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Releasing lock "refresh_cache-95afc929-9f13-4f9a-9ac9-eae0b5e15a4d" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:41:02 user nova-compute[75234]: DEBUG nova.compute.manager [None req-b8d98ced-f0fd-40c8-b1b7-5ed5946ba755 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: 95afc929-9f13-4f9a-9ac9-eae0b5e15a4d] Instance network_info: |[{"id": "ca1227f9-b2f1-42c5-a92a-c70cb94e52c1", "address": "fa:16:3e:02:94:55", "network": {"id": "a3f397d3-fd75-4a6e-a815-cfcedcca15fb", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-678570085-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "81a265d3c15b4e9d826b35ddbd0ff443", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapca1227f9-b2", "ovs_interfaceid": "ca1227f9-b2f1-42c5-a92a-c70cb94e52c1", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=75234) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1987}}
Nov 29 03:41:02 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-58986eb0-05be-44fc-b1d5-487cf18241ac req-41e5e7b9-d998-4333-a5e7-313d17a56d47 service nova] Acquired lock "refresh_cache-95afc929-9f13-4f9a-9ac9-eae0b5e15a4d" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:41:02 user nova-compute[75234]: DEBUG nova.network.neutron [req-58986eb0-05be-44fc-b1d5-487cf18241ac req-41e5e7b9-d998-4333-a5e7-313d17a56d47 service nova] [instance: 95afc929-9f13-4f9a-9ac9-eae0b5e15a4d] Refreshing network info cache for port ca1227f9-b2f1-42c5-a92a-c70cb94e52c1 {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2006}}
Nov 29 03:41:02 user nova-compute[75234]: DEBUG nova.network.neutron [req-58986eb0-05be-44fc-b1d5-487cf18241ac req-41e5e7b9-d998-4333-a5e7-313d17a56d47 service nova] [instance: 95afc929-9f13-4f9a-9ac9-eae0b5e15a4d] Updated VIF entry in instance network info cache for port ca1227f9-b2f1-42c5-a92a-c70cb94e52c1. {{(pid=75234) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3481}}
Nov 29 03:41:02 user nova-compute[75234]: DEBUG nova.network.neutron [req-58986eb0-05be-44fc-b1d5-487cf18241ac req-41e5e7b9-d998-4333-a5e7-313d17a56d47 service nova] [instance: 95afc929-9f13-4f9a-9ac9-eae0b5e15a4d] Updating instance_info_cache with network_info: [{"id": "ca1227f9-b2f1-42c5-a92a-c70cb94e52c1", "address": "fa:16:3e:02:94:55", "network": {"id": "a3f397d3-fd75-4a6e-a815-cfcedcca15fb", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-678570085-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "81a265d3c15b4e9d826b35ddbd0ff443", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapca1227f9-b2", "ovs_interfaceid": "ca1227f9-b2f1-42c5-a92a-c70cb94e52c1", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:41:02 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-58986eb0-05be-44fc-b1d5-487cf18241ac req-41e5e7b9-d998-4333-a5e7-313d17a56d47 service nova] Releasing lock "refresh_cache-95afc929-9f13-4f9a-9ac9-eae0b5e15a4d" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:41:04 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:41:07 user nova-compute[75234]: DEBUG nova.compute.manager [None req-b8d98ced-f0fd-40c8-b1b7-5ed5946ba755 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: 95afc929-9f13-4f9a-9ac9-eae0b5e15a4d] Start spawning the instance on the hypervisor. {{(pid=75234) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2628}}
Nov 29 03:41:07 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-b8d98ced-f0fd-40c8-b1b7-5ed5946ba755 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: 95afc929-9f13-4f9a-9ac9-eae0b5e15a4d] Creating instance directory {{(pid=75234) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4727}}
Nov 29 03:41:07 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-b8d98ced-f0fd-40c8-b1b7-5ed5946ba755 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: 95afc929-9f13-4f9a-9ac9-eae0b5e15a4d] Creating image(s)
Nov 29 03:41:07 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-b8d98ced-f0fd-40c8-b1b7-5ed5946ba755 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: 95afc929-9f13-4f9a-9ac9-eae0b5e15a4d] Did not create local disks {{(pid=75234) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4863}}
Nov 29 03:41:07 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-b8d98ced-f0fd-40c8-b1b7-5ed5946ba755 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: 95afc929-9f13-4f9a-9ac9-eae0b5e15a4d] Ensure instance console log exists: /opt/stack/data/nova/instances/95afc929-9f13-4f9a-9ac9-eae0b5e15a4d/console.log {{(pid=75234) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4613}}
Nov 29 03:41:07 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-b8d98ced-f0fd-40c8-b1b7-5ed5946ba755 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:41:07 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-b8d98ced-f0fd-40c8-b1b7-5ed5946ba755 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:41:07 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-b8d98ced-f0fd-40c8-b1b7-5ed5946ba755 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:41:07 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-b8d98ced-f0fd-40c8-b1b7-5ed5946ba755 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: 95afc929-9f13-4f9a-9ac9-eae0b5e15a4d] Start _get_guest_xml network_info=[{"id": "ca1227f9-b2f1-42c5-a92a-c70cb94e52c1", "address": "fa:16:3e:02:94:55", "network": {"id": "a3f397d3-fd75-4a6e-a815-cfcedcca15fb", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-678570085-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "81a265d3c15b4e9d826b35ddbd0ff443", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapca1227f9-b2", "ovs_interfaceid": "ca1227f9-b2f1-42c5-a92a-c70cb94e52c1", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, '/dev/vda': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum=<?>,container_format=<?>,created_at=<?>,direct_url=<?>,disk_format=<?>,id=<?>,min_disk=0,min_ram=0,name=<?>,owner=<?>,properties=ImageMetaProps,protected=<?>,size=1073741824,status='active',tags=<?>,updated_at=<?>,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [], 'ephemerals': [], 'block_device_mapping': [{'mount_device': '/dev/vda', 'attachment_id': '75e0716f-0606-451b-a569-5d8a8200b356', 'disk_bus': 'virtio', 'device_type': 'disk', 'boot_index': 0, 'guest_format': None, 'delete_on_termination': False, 'connection_info': {'driver_volume_type': 'iscsi', 'data': {'target_iqn': 'iqn.2010-10.org.openstack:378addbb-8d90-40c5-ae61-c3a063116df9', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'cyOomDWj', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}, 'status': 'reserved', 'instance': '95afc929-9f13-4f9a-9ac9-eae0b5e15a4d', 'attached_at': '', 'detached_at': '', 'volume_id': '378addbb-8d90-40c5-ae61-c3a063116df9', 'serial': '378addbb-8d90-40c5-ae61-c3a063116df9'}, 'volume_type': None} {{(pid=75234) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7562}}
Nov 29 03:41:07 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-b8d98ced-f0fd-40c8-b1b7-5ed5946ba755 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:41:07 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-b8d98ced-f0fd-40c8-b1b7-5ed5946ba755 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:41:07 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-b8d98ced-f0fd-40c8-b1b7-5ed5946ba755 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Searching host: 'user' for CPU controller through CGroups V1... {{(pid=75234) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1652}}
Nov 29 03:41:07 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-b8d98ced-f0fd-40c8-b1b7-5ed5946ba755 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] CPU controller missing on host. {{(pid=75234) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1662}}
Nov 29 03:41:07 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-b8d98ced-f0fd-40c8-b1b7-5ed5946ba755 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Searching host: 'user' for CPU controller through CGroups V2... {{(pid=75234) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1671}}
Nov 29 03:41:07 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-b8d98ced-f0fd-40c8-b1b7-5ed5946ba755 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] CPU controller found on host. {{(pid=75234) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1678}}
Nov 29 03:41:07 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-b8d98ced-f0fd-40c8-b1b7-5ed5946ba755 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=75234) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5400}}
Nov 29 03:41:07 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-b8d98ced-f0fd-40c8-b1b7-5ed5946ba755 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-11-29T03:02:04Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='1',id=6,is_public=True,memory_mb=512,name='m1.tiny',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum=<?>,container_format=<?>,created_at=<?>,direct_url=<?>,disk_format=<?>,id=<?>,min_disk=0,min_ram=0,name=<?>,owner=<?>,properties=ImageMetaProps,protected=<?>,size=1073741824,status='active',tags=<?>,updated_at=<?>,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:563}}
Nov 29 03:41:07 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-b8d98ced-f0fd-40c8-b1b7-5ed5946ba755 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Flavor limits 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:348}}
Nov 29 03:41:07 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-b8d98ced-f0fd-40c8-b1b7-5ed5946ba755 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Image limits 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:352}}
Nov 29 03:41:07 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-b8d98ced-f0fd-40c8-b1b7-5ed5946ba755 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Flavor pref 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:388}}
Nov 29 03:41:07 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-b8d98ced-f0fd-40c8-b1b7-5ed5946ba755 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Image pref 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:392}}
Nov 29 03:41:07 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-b8d98ced-f0fd-40c8-b1b7-5ed5946ba755 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:430}}
Nov 29 03:41:07 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-b8d98ced-f0fd-40c8-b1b7-5ed5946ba755 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:569}}
Nov 29 03:41:07 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-b8d98ced-f0fd-40c8-b1b7-5ed5946ba755 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=75234) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:471}}
Nov 29 03:41:07 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-b8d98ced-f0fd-40c8-b1b7-5ed5946ba755 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Got 1 possible topologies {{(pid=75234) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:501}}
Nov 29 03:41:07 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-b8d98ced-f0fd-40c8-b1b7-5ed5946ba755 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:575}}
Nov 29 03:41:07 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-b8d98ced-f0fd-40c8-b1b7-5ed5946ba755 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:577}}
Nov 29 03:41:07 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-b8d98ced-f0fd-40c8-b1b7-5ed5946ba755 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: 95afc929-9f13-4f9a-9ac9-eae0b5e15a4d] Calling os-brick to attach iSCSI Volume {{(pid=75234) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Nov 29 03:41:07 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-b8d98ced-f0fd-40c8-b1b7-5ed5946ba755 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f8a5050c9a0>, 'connection_properties': {'target_iqn': 'iqn.2010-10.org.openstack:378addbb-8d90-40c5-ae61-c3a063116df9', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'cyOomDWj', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:176}}
Nov 29 03:41:07 user nova-compute[75234]: WARNING os_brick.initiator.connectors.base [None req-b8d98ced-f0fd-40c8-b1b7-5ed5946ba755 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Nov 29 03:41:07 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-b8d98ced-f0fd-40c8-b1b7-5ed5946ba755 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:68}}
Nov 29 03:41:07 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-b8d98ced-f0fd-40c8-b1b7-5ed5946ba755 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:73}}
Nov 29 03:41:07 user nova-compute[75234]: WARNING os_brick.initiator.connectors.base [None req-b8d98ced-f0fd-40c8-b1b7-5ed5946ba755 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Nov 29 03:41:07 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-b8d98ced-f0fd-40c8-b1b7-5ed5946ba755 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:378addbb-8d90-40c5-ae61-c3a063116df9" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:68}}
Nov 29 03:41:07 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-b8d98ced-f0fd-40c8-b1b7-5ed5946ba755 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:378addbb-8d90-40c5-ae61-c3a063116df9" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:73}}
Nov 29 03:41:07 user nova-compute[75234]: INFO os_brick.initiator.connectors.iscsi [None req-b8d98ced-f0fd-40c8-b1b7-5ed5946ba755 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Nov 29 03:41:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:378addbb-8d90-40c5-ae61-c3a063116df9 -p 172.16.0.220:3260 {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:41:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:378addbb-8d90-40c5-ae61-c3a063116df9 -p 172.16.0.220:3260" returned: 21 in 0.017s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:41:07 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[0d3cb539-7c30-42cb-88d1-13c0ad4229ab]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:41:07 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-b8d98ced-f0fd-40c8-b1b7-5ed5946ba755 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Nov 29 03:41:07 user nova-compute[75234]:  {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:41:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:378addbb-8d90-40c5-ae61-c3a063116df9 -p 172.16.0.220:3260 --interface default --op new {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:41:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:378addbb-8d90-40c5-ae61-c3a063116df9 -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.018s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:41:07 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[f10aafe5-107b-4101-858a-0cb7c590a5e8]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:378addbb-8d90-40c5-ae61-c3a063116df9] added\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:41:07 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-b8d98ced-f0fd-40c8-b1b7-5ed5946ba755 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:378addbb-8d90-40c5-ae61-c3a063116df9] added
Nov 29 03:41:07 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:41:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:378addbb-8d90-40c5-ae61-c3a063116df9 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:41:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:378addbb-8d90-40c5-ae61-c3a063116df9 -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.018s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:41:07 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[8d72c8dc-05b0-4fb5-af40-e13e7c370720]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:41:07 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-b8d98ced-f0fd-40c8-b1b7-5ed5946ba755 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:41:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:378addbb-8d90-40c5-ae61-c3a063116df9 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:41:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:378addbb-8d90-40c5-ae61-c3a063116df9 -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.021s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:41:07 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[68d74ed9-051d-4789-9324-d00760f4faae]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:41:07 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-b8d98ced-f0fd-40c8-b1b7-5ed5946ba755 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:41:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:378addbb-8d90-40c5-ae61-c3a063116df9 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v cyOomDWj {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:41:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:378addbb-8d90-40c5-ae61-c3a063116df9 -p 172.16.0.220:3260 --op update -n node.session.auth.username -v cyOomDWj" returned: 0 in 0.016s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:41:07 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[c0279574-1f12-417c-8607-6897e7474aab]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:41:07 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-b8d98ced-f0fd-40c8-b1b7-5ed5946ba755 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'cyOomDWj'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:41:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:378addbb-8d90-40c5-ae61-c3a063116df9 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:41:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:378addbb-8d90-40c5-ae61-c3a063116df9 -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.017s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:41:07 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[a62f8723-e4ce-4dd2-92b6-f8073e447331]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:41:07 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-b8d98ced-f0fd-40c8-b1b7-5ed5946ba755 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:41:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:41:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 21 in 0.016s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:41:07 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[3c6ac68f-b8db-45d6-9135-db5e114976eb]: (4, ('', 'iscsiadm: No active sessions.\n')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:41:07 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-b8d98ced-f0fd-40c8-b1b7-5ed5946ba755 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] iscsiadm ('-m', 'session'): stdout= stderr=iscsiadm: No active sessions.
Nov 29 03:41:07 user nova-compute[75234]:  {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:41:07 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-b8d98ced-f0fd-40c8-b1b7-5ed5946ba755 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] iscsi session list stdout= stderr=iscsiadm: No active sessions.
Nov 29 03:41:07 user nova-compute[75234]:  {{(pid=75234) _run_iscsi_session /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Nov 29 03:41:07 user nova-compute[75234]: WARNING os_brick.initiator.connectors.iscsi [None req-b8d98ced-f0fd-40c8-b1b7-5ed5946ba755 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] iscsiadm stderr output when getting sessions: iscsiadm: No active sessions.
Nov 29 03:41:07 user nova-compute[75234]: 
Nov 29 03:41:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:378addbb-8d90-40c5-ae61-c3a063116df9 -p 172.16.0.220:3260 --login {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:41:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:378addbb-8d90-40c5-ae61-c3a063116df9 -p 172.16.0.220:3260 --login" returned: 0 in 0.042s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:41:07 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[1f276821-a8a3-4a3d-9c21-d073ff518ed8]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:378addbb-8d90-40c5-ae61-c3a063116df9, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:378addbb-8d90-40c5-ae61-c3a063116df9, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:41:07 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-b8d98ced-f0fd-40c8-b1b7-5ed5946ba755 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:378addbb-8d90-40c5-ae61-c3a063116df9, portal: 172.16.0.220,3260]
Nov 29 03:41:07 user nova-compute[75234]: Login to [iface: default, target: iqn.2010-10.org.openstack:378addbb-8d90-40c5-ae61-c3a063116df9, portal: 172.16.0.220,3260] successful.
Nov 29 03:41:07 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:41:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:378addbb-8d90-40c5-ae61-c3a063116df9 -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:41:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:378addbb-8d90-40c5-ae61-c3a063116df9 -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.018s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:41:07 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[b622d830-d967-4634-a183-3c3a7e7d804e]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:41:07 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-b8d98ced-f0fd-40c8-b1b7-5ed5946ba755 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:41:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:41:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.017s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:41:07 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[8a4c22ee-d11d-43c7-9199-1a542dcd778a]: (4, ('tcp: [59] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:378addbb-8d90-40c5-ae61-c3a063116df9 (non-flash)\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:41:07 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-b8d98ced-f0fd-40c8-b1b7-5ed5946ba755 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [59] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:378addbb-8d90-40c5-ae61-c3a063116df9 (non-flash)
Nov 29 03:41:07 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:41:07 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-b8d98ced-f0fd-40c8-b1b7-5ed5946ba755 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] iscsi session list stdout=tcp: [59] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:378addbb-8d90-40c5-ae61-c3a063116df9 (non-flash)
Nov 29 03:41:07 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsi_session /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Nov 29 03:41:07 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-b8d98ced-f0fd-40c8-b1b7-5ed5946ba755 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:378addbb-8d90-40c5-ae61-c3a063116df9" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.234s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:87}}
Nov 29 03:41:07 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-b8d98ced-f0fd-40c8-b1b7-5ed5946ba755 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Connected to 172.16.0.220:3260 {{(pid=75234) _connect_vol /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:644}}
Nov 29 03:41:07 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-b8d98ced-f0fd-40c8-b1b7-5ed5946ba755 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] HCTL ('33', '-', '-', 0) found on session 59 with lun 0 {{(pid=75234) get_hctl /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:770}}
Nov 29 03:41:07 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-b8d98ced-f0fd-40c8-b1b7-5ed5946ba755 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Scanning host 33 c: -, t: -, l: 0) {{(pid=75234) scan_iscsi /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:795}}
Nov 29 03:41:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host33/scan {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:41:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host33/scan" returned: 0 in 0.015s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:41:07 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[184d1682-fce1-4205-9b36-4bd5ad621e52]: (4, ('- - 0', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:41:07 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-b8d98ced-f0fd-40c8-b1b7-5ed5946ba755 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Searching for a device in session 59 and hctl ['33', '*', '*', 0] yield: sdb {{(pid=75234) device_name_by_hctl /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:789}}
Nov 29 03:41:07 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-b8d98ced-f0fd-40c8-b1b7-5ed5946ba755 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Connected to sdb using {'target_iqn': 'iqn.2010-10.org.openstack:378addbb-8d90-40c5-ae61-c3a063116df9', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'cyOomDWj', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False} {{(pid=75234) _connect_vol /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:672}}
Nov 29 03:41:08 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-b8d98ced-f0fd-40c8-b1b7-5ed5946ba755 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.261s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:87}}
Nov 29 03:41:08 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-b8d98ced-f0fd-40c8-b1b7-5ed5946ba755 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] <== connect_volume: return (1262ms) {'type': 'block', 'scsi_wwn': '23832366162623035', 'path': '/dev/sdb'} {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:203}}
Nov 29 03:41:08 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-b8d98ced-f0fd-40c8-b1b7-5ed5946ba755 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: 95afc929-9f13-4f9a-9ac9-eae0b5e15a4d] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '23832366162623035', 'path': '/dev/sdb'} {{(pid=75234) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Nov 29 03:41:08 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-b8d98ced-f0fd-40c8-b1b7-5ed5946ba755 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-11-29T03:40:59Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-TestVolumeBootPattern-server-2064710171',display_name='tempest-TestVolumeBootPattern-server-2064710171',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-testvolumebootpattern-server-2064710171',id=51,image_ref='',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBKrnAt22NMoBNIFsnCF3Mxk+dIbrL4YaXFADzEahC/3pON2UrJ4NOPzv0Ze4QWq/A4Bm4h/5Mn2lQRP6Y0eXl2r9TPWd6a1CQjrsTRMK7MqwF4n4QkuDxmXBPaNWL5XvpA==',key_name='tempest-TestVolumeBootPattern-1253305043',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='81a265d3c15b4e9d826b35ddbd0ff443',ramdisk_id='',reservation_id='r-zr0x3vw3',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TestVolumeBootPattern-1244492388',owner_user_name='tempest-TestVolumeBootPattern-1244492388-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-11-29T03:41:00Z,user_data=None,user_id='7d1f1f11a9aa4c1db8627413eb963fa4',uuid=95afc929-9f13-4f9a-9ac9-eae0b5e15a4d,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "ca1227f9-b2f1-42c5-a92a-c70cb94e52c1", "address": "fa:16:3e:02:94:55", "network": {"id": "a3f397d3-fd75-4a6e-a815-cfcedcca15fb", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-678570085-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "81a265d3c15b4e9d826b35ddbd0ff443", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapca1227f9-b2", "ovs_interfaceid": "ca1227f9-b2f1-42c5-a92a-c70cb94e52c1", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=75234) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Nov 29 03:41:08 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-b8d98ced-f0fd-40c8-b1b7-5ed5946ba755 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Converting VIF {"id": "ca1227f9-b2f1-42c5-a92a-c70cb94e52c1", "address": "fa:16:3e:02:94:55", "network": {"id": "a3f397d3-fd75-4a6e-a815-cfcedcca15fb", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-678570085-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "81a265d3c15b4e9d826b35ddbd0ff443", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapca1227f9-b2", "ovs_interfaceid": "ca1227f9-b2f1-42c5-a92a-c70cb94e52c1", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:41:08 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-b8d98ced-f0fd-40c8-b1b7-5ed5946ba755 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:02:94:55,bridge_name='br-int',has_traffic_filtering=True,id=ca1227f9-b2f1-42c5-a92a-c70cb94e52c1,network=Network(a3f397d3-fd75-4a6e-a815-cfcedcca15fb),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapca1227f9-b2') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:41:08 user nova-compute[75234]: DEBUG nova.objects.instance [None req-b8d98ced-f0fd-40c8-b1b7-5ed5946ba755 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Lazy-loading 'pci_devices' on Instance uuid 95afc929-9f13-4f9a-9ac9-eae0b5e15a4d {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:41:08 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-b8d98ced-f0fd-40c8-b1b7-5ed5946ba755 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: 95afc929-9f13-4f9a-9ac9-eae0b5e15a4d] End _get_guest_xml xml=<domain type="kvm">
Nov 29 03:41:08 user nova-compute[75234]:   <uuid>95afc929-9f13-4f9a-9ac9-eae0b5e15a4d</uuid>
Nov 29 03:41:08 user nova-compute[75234]:   <name>instance-00000033</name>
Nov 29 03:41:08 user nova-compute[75234]:   <memory>524288</memory>
Nov 29 03:41:08 user nova-compute[75234]:   <vcpu>1</vcpu>
Nov 29 03:41:08 user nova-compute[75234]:   <metadata>
Nov 29 03:41:08 user nova-compute[75234]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Nov 29 03:41:08 user nova-compute[75234]:       <nova:package version="0.0.0"/>
Nov 29 03:41:08 user nova-compute[75234]:       <nova:name>tempest-TestVolumeBootPattern-server-2064710171</nova:name>
Nov 29 03:41:08 user nova-compute[75234]:       <nova:creationTime>2023-11-29 03:41:07</nova:creationTime>
Nov 29 03:41:08 user nova-compute[75234]:       <nova:flavor name="m1.tiny">
Nov 29 03:41:08 user nova-compute[75234]:         <nova:memory>512</nova:memory>
Nov 29 03:41:08 user nova-compute[75234]:         <nova:disk>1</nova:disk>
Nov 29 03:41:08 user nova-compute[75234]:         <nova:swap>0</nova:swap>
Nov 29 03:41:08 user nova-compute[75234]:         <nova:ephemeral>0</nova:ephemeral>
Nov 29 03:41:08 user nova-compute[75234]:         <nova:vcpus>1</nova:vcpus>
Nov 29 03:41:08 user nova-compute[75234]:       </nova:flavor>
Nov 29 03:41:08 user nova-compute[75234]:       <nova:owner>
Nov 29 03:41:08 user nova-compute[75234]:         <nova:user uuid="7d1f1f11a9aa4c1db8627413eb963fa4">tempest-TestVolumeBootPattern-1244492388-project-member</nova:user>
Nov 29 03:41:08 user nova-compute[75234]:         <nova:project uuid="81a265d3c15b4e9d826b35ddbd0ff443">tempest-TestVolumeBootPattern-1244492388</nova:project>
Nov 29 03:41:08 user nova-compute[75234]:       </nova:owner>
Nov 29 03:41:08 user nova-compute[75234]:       <nova:ports>
Nov 29 03:41:08 user nova-compute[75234]:         <nova:port uuid="ca1227f9-b2f1-42c5-a92a-c70cb94e52c1">
Nov 29 03:41:08 user nova-compute[75234]:           <nova:ip type="fixed" address="10.0.0.14" ipVersion="4"/>
Nov 29 03:41:08 user nova-compute[75234]:         </nova:port>
Nov 29 03:41:08 user nova-compute[75234]:       </nova:ports>
Nov 29 03:41:08 user nova-compute[75234]:     </nova:instance>
Nov 29 03:41:08 user nova-compute[75234]:   </metadata>
Nov 29 03:41:08 user nova-compute[75234]:   <sysinfo type="smbios">
Nov 29 03:41:08 user nova-compute[75234]:     <system>
Nov 29 03:41:08 user nova-compute[75234]:       <entry name="manufacturer">OpenStack Foundation</entry>
Nov 29 03:41:08 user nova-compute[75234]:       <entry name="product">OpenStack Nova</entry>
Nov 29 03:41:08 user nova-compute[75234]:       <entry name="version">0.0.0</entry>
Nov 29 03:41:08 user nova-compute[75234]:       <entry name="serial">95afc929-9f13-4f9a-9ac9-eae0b5e15a4d</entry>
Nov 29 03:41:08 user nova-compute[75234]:       <entry name="uuid">95afc929-9f13-4f9a-9ac9-eae0b5e15a4d</entry>
Nov 29 03:41:08 user nova-compute[75234]:       <entry name="family">Virtual Machine</entry>
Nov 29 03:41:08 user nova-compute[75234]:     </system>
Nov 29 03:41:08 user nova-compute[75234]:   </sysinfo>
Nov 29 03:41:08 user nova-compute[75234]:   <os>
Nov 29 03:41:08 user nova-compute[75234]:     <type arch="x86_64" machine="pc">hvm</type>
Nov 29 03:41:08 user nova-compute[75234]:     <boot dev="hd"/>
Nov 29 03:41:08 user nova-compute[75234]:     <smbios mode="sysinfo"/>
Nov 29 03:41:08 user nova-compute[75234]:   </os>
Nov 29 03:41:08 user nova-compute[75234]:   <features>
Nov 29 03:41:08 user nova-compute[75234]:     <acpi/>
Nov 29 03:41:08 user nova-compute[75234]:     <apic/>
Nov 29 03:41:08 user nova-compute[75234]:     <vmcoreinfo/>
Nov 29 03:41:08 user nova-compute[75234]:   </features>
Nov 29 03:41:08 user nova-compute[75234]:   <clock offset="utc">
Nov 29 03:41:08 user nova-compute[75234]:     <timer name="pit" tickpolicy="delay"/>
Nov 29 03:41:08 user nova-compute[75234]:     <timer name="rtc" tickpolicy="catchup"/>
Nov 29 03:41:08 user nova-compute[75234]:     <timer name="hpet" present="no"/>
Nov 29 03:41:08 user nova-compute[75234]:   </clock>
Nov 29 03:41:08 user nova-compute[75234]:   <cpu mode="custom" match="exact">
Nov 29 03:41:08 user nova-compute[75234]:     <model>Nehalem</model>
Nov 29 03:41:08 user nova-compute[75234]:     <topology sockets="1" cores="1" threads="1"/>
Nov 29 03:41:08 user nova-compute[75234]:   </cpu>
Nov 29 03:41:08 user nova-compute[75234]:   <devices>
Nov 29 03:41:08 user nova-compute[75234]:     <disk type="block" device="disk">
Nov 29 03:41:08 user nova-compute[75234]:       <driver name="qemu" type="raw" cache="none" io="native"/>
Nov 29 03:41:08 user nova-compute[75234]:       <source dev="/dev/sdb"/>
Nov 29 03:41:08 user nova-compute[75234]:       <target dev="vda" bus="virtio"/>
Nov 29 03:41:08 user nova-compute[75234]:       <serial>378addbb-8d90-40c5-ae61-c3a063116df9</serial>
Nov 29 03:41:08 user nova-compute[75234]:     </disk>
Nov 29 03:41:08 user nova-compute[75234]:     <interface type="ethernet">
Nov 29 03:41:08 user nova-compute[75234]:       <mac address="fa:16:3e:02:94:55"/>
Nov 29 03:41:08 user nova-compute[75234]:       <model type="virtio"/>
Nov 29 03:41:08 user nova-compute[75234]:       <mtu size="1442"/>
Nov 29 03:41:08 user nova-compute[75234]:       <target dev="tapca1227f9-b2"/>
Nov 29 03:41:08 user nova-compute[75234]:     </interface>
Nov 29 03:41:08 user nova-compute[75234]:     <serial type="pty">
Nov 29 03:41:08 user nova-compute[75234]:       <log file="/opt/stack/data/nova/instances/95afc929-9f13-4f9a-9ac9-eae0b5e15a4d/console.log" append="off"/>
Nov 29 03:41:08 user nova-compute[75234]:     </serial>
Nov 29 03:41:08 user nova-compute[75234]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Nov 29 03:41:08 user nova-compute[75234]:     <video>
Nov 29 03:41:08 user nova-compute[75234]:       <model type="virtio"/>
Nov 29 03:41:08 user nova-compute[75234]:     </video>
Nov 29 03:41:08 user nova-compute[75234]:     <rng model="virtio">
Nov 29 03:41:08 user nova-compute[75234]:       <backend model="random">/dev/urandom</backend>
Nov 29 03:41:08 user nova-compute[75234]:     </rng>
Nov 29 03:41:08 user nova-compute[75234]:     <controller type="usb" index="0" model="none"/>
Nov 29 03:41:08 user nova-compute[75234]:     <memballoon model="virtio">
Nov 29 03:41:08 user nova-compute[75234]:       <stats period="10"/>
Nov 29 03:41:08 user nova-compute[75234]:     </memballoon>
Nov 29 03:41:08 user nova-compute[75234]:   </devices>
Nov 29 03:41:08 user nova-compute[75234]: </domain>
Nov 29 03:41:08 user nova-compute[75234]:  {{(pid=75234) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7568}}
Nov 29 03:41:08 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-b8d98ced-f0fd-40c8-b1b7-5ed5946ba755 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-11-29T03:40:59Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-TestVolumeBootPattern-server-2064710171',display_name='tempest-TestVolumeBootPattern-server-2064710171',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-testvolumebootpattern-server-2064710171',id=51,image_ref='',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBKrnAt22NMoBNIFsnCF3Mxk+dIbrL4YaXFADzEahC/3pON2UrJ4NOPzv0Ze4QWq/A4Bm4h/5Mn2lQRP6Y0eXl2r9TPWd6a1CQjrsTRMK7MqwF4n4QkuDxmXBPaNWL5XvpA==',key_name='tempest-TestVolumeBootPattern-1253305043',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='81a265d3c15b4e9d826b35ddbd0ff443',ramdisk_id='',reservation_id='r-zr0x3vw3',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TestVolumeBootPattern-1244492388',owner_user_name='tempest-TestVolumeBootPattern-1244492388-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-11-29T03:41:00Z,user_data=None,user_id='7d1f1f11a9aa4c1db8627413eb963fa4',uuid=95afc929-9f13-4f9a-9ac9-eae0b5e15a4d,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "ca1227f9-b2f1-42c5-a92a-c70cb94e52c1", "address": "fa:16:3e:02:94:55", "network": {"id": "a3f397d3-fd75-4a6e-a815-cfcedcca15fb", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-678570085-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "81a265d3c15b4e9d826b35ddbd0ff443", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapca1227f9-b2", "ovs_interfaceid": "ca1227f9-b2f1-42c5-a92a-c70cb94e52c1", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Nov 29 03:41:08 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-b8d98ced-f0fd-40c8-b1b7-5ed5946ba755 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Converting VIF {"id": "ca1227f9-b2f1-42c5-a92a-c70cb94e52c1", "address": "fa:16:3e:02:94:55", "network": {"id": "a3f397d3-fd75-4a6e-a815-cfcedcca15fb", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-678570085-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "81a265d3c15b4e9d826b35ddbd0ff443", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapca1227f9-b2", "ovs_interfaceid": "ca1227f9-b2f1-42c5-a92a-c70cb94e52c1", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:41:08 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-b8d98ced-f0fd-40c8-b1b7-5ed5946ba755 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:02:94:55,bridge_name='br-int',has_traffic_filtering=True,id=ca1227f9-b2f1-42c5-a92a-c70cb94e52c1,network=Network(a3f397d3-fd75-4a6e-a815-cfcedcca15fb),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapca1227f9-b2') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:41:08 user nova-compute[75234]: DEBUG os_vif [None req-b8d98ced-f0fd-40c8-b1b7-5ed5946ba755 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:02:94:55,bridge_name='br-int',has_traffic_filtering=True,id=ca1227f9-b2f1-42c5-a92a-c70cb94e52c1,network=Network(a3f397d3-fd75-4a6e-a815-cfcedcca15fb),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapca1227f9-b2') {{(pid=75234) plug /opt/stack/data/venv/lib/python3.10/site-packages/os_vif/__init__.py:76}}
Nov 29 03:41:08 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:41:08 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:41:08 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Nov 29 03:41:08 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:41:08 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbCreateCommand(_result=None, table=QoS, columns={'type': 'linux-noop', 'external_ids': {'id': '198bbd5d-7c47-5f06-919f-ebe5b2e976cf', '_type': 'linux-noop'}}, row=False) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:41:08 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:41:08 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:41:08 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:41:08 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tapca1227f9-b2, may_exist=True, interface_attrs={}) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:41:08 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Port, record=tapca1227f9-b2, col_values=(('qos', UUID('582e3c7f-6572-48dc-8ac0-32a581fb63c4')),), if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:41:08 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DbSetCommand(_result=None, table=Interface, record=tapca1227f9-b2, col_values=(('external_ids', {'iface-id': 'ca1227f9-b2f1-42c5-a92a-c70cb94e52c1', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:02:94:55', 'vm-uuid': '95afc929-9f13-4f9a-9ac9-eae0b5e15a4d'}),), if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:41:08 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:41:08 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:41:08 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:41:08 user nova-compute[75234]: INFO os_vif [None req-b8d98ced-f0fd-40c8-b1b7-5ed5946ba755 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:02:94:55,bridge_name='br-int',has_traffic_filtering=True,id=ca1227f9-b2f1-42c5-a92a-c70cb94e52c1,network=Network(a3f397d3-fd75-4a6e-a815-cfcedcca15fb),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapca1227f9-b2')
Nov 29 03:41:08 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-b8d98ced-f0fd-40c8-b1b7-5ed5946ba755 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] No BDM found with device name vda, not building metadata. {{(pid=75234) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12089}}
Nov 29 03:41:08 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-b8d98ced-f0fd-40c8-b1b7-5ed5946ba755 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] No VIF found with MAC fa:16:3e:02:94:55, not building metadata {{(pid=75234) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Nov 29 03:41:10 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:41:10 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:41:10 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:41:10 user nova-compute[75234]: DEBUG nova.compute.manager [req-6eccf98c-3bed-43e8-a152-4ef93dd5ffcf req-ce94bb70-7c4e-4332-bdb0-26b128ff19f3 service nova] [instance: 95afc929-9f13-4f9a-9ac9-eae0b5e15a4d] Received event network-vif-plugged-ca1227f9-b2f1-42c5-a92a-c70cb94e52c1 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:41:10 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-6eccf98c-3bed-43e8-a152-4ef93dd5ffcf req-ce94bb70-7c4e-4332-bdb0-26b128ff19f3 service nova] Acquiring lock "95afc929-9f13-4f9a-9ac9-eae0b5e15a4d-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:41:10 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-6eccf98c-3bed-43e8-a152-4ef93dd5ffcf req-ce94bb70-7c4e-4332-bdb0-26b128ff19f3 service nova] Lock "95afc929-9f13-4f9a-9ac9-eae0b5e15a4d-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:41:10 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-6eccf98c-3bed-43e8-a152-4ef93dd5ffcf req-ce94bb70-7c4e-4332-bdb0-26b128ff19f3 service nova] Lock "95afc929-9f13-4f9a-9ac9-eae0b5e15a4d-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:41:10 user nova-compute[75234]: DEBUG nova.compute.manager [req-6eccf98c-3bed-43e8-a152-4ef93dd5ffcf req-ce94bb70-7c4e-4332-bdb0-26b128ff19f3 service nova] [instance: 95afc929-9f13-4f9a-9ac9-eae0b5e15a4d] No waiting events found dispatching network-vif-plugged-ca1227f9-b2f1-42c5-a92a-c70cb94e52c1 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:41:10 user nova-compute[75234]: WARNING nova.compute.manager [req-6eccf98c-3bed-43e8-a152-4ef93dd5ffcf req-ce94bb70-7c4e-4332-bdb0-26b128ff19f3 service nova] [instance: 95afc929-9f13-4f9a-9ac9-eae0b5e15a4d] Received unexpected event network-vif-plugged-ca1227f9-b2f1-42c5-a92a-c70cb94e52c1 for instance with vm_state building and task_state spawning.
Nov 29 03:41:10 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:41:12 user nova-compute[75234]: DEBUG nova.compute.manager [None req-b8d98ced-f0fd-40c8-b1b7-5ed5946ba755 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: 95afc929-9f13-4f9a-9ac9-eae0b5e15a4d] Instance event wait completed in 0 seconds for  {{(pid=75234) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Nov 29 03:41:12 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-b8d98ced-f0fd-40c8-b1b7-5ed5946ba755 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: 95afc929-9f13-4f9a-9ac9-eae0b5e15a4d] Guest created on hypervisor {{(pid=75234) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4421}}
Nov 29 03:41:12 user nova-compute[75234]: INFO nova.virt.libvirt.driver [-] [instance: 95afc929-9f13-4f9a-9ac9-eae0b5e15a4d] Instance spawned successfully.
Nov 29 03:41:12 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-b8d98ced-f0fd-40c8-b1b7-5ed5946ba755 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: 95afc929-9f13-4f9a-9ac9-eae0b5e15a4d] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:904}}
Nov 29 03:41:12 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-b8d98ced-f0fd-40c8-b1b7-5ed5946ba755 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: 95afc929-9f13-4f9a-9ac9-eae0b5e15a4d] Found default for hw_cdrom_bus of ide {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:41:12 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-b8d98ced-f0fd-40c8-b1b7-5ed5946ba755 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: 95afc929-9f13-4f9a-9ac9-eae0b5e15a4d] Found default for hw_disk_bus of virtio {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:41:12 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-b8d98ced-f0fd-40c8-b1b7-5ed5946ba755 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: 95afc929-9f13-4f9a-9ac9-eae0b5e15a4d] Found default for hw_input_bus of None {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:41:12 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-b8d98ced-f0fd-40c8-b1b7-5ed5946ba755 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: 95afc929-9f13-4f9a-9ac9-eae0b5e15a4d] Found default for hw_pointer_model of None {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:41:12 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-b8d98ced-f0fd-40c8-b1b7-5ed5946ba755 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: 95afc929-9f13-4f9a-9ac9-eae0b5e15a4d] Found default for hw_video_model of virtio {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:41:12 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-b8d98ced-f0fd-40c8-b1b7-5ed5946ba755 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: 95afc929-9f13-4f9a-9ac9-eae0b5e15a4d] Found default for hw_vif_model of virtio {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:41:12 user nova-compute[75234]: INFO nova.compute.manager [None req-b8d98ced-f0fd-40c8-b1b7-5ed5946ba755 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: 95afc929-9f13-4f9a-9ac9-eae0b5e15a4d] Took 4.94 seconds to spawn the instance on the hypervisor.
Nov 29 03:41:12 user nova-compute[75234]: DEBUG nova.compute.manager [None req-b8d98ced-f0fd-40c8-b1b7-5ed5946ba755 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: 95afc929-9f13-4f9a-9ac9-eae0b5e15a4d] Checking state {{(pid=75234) _get_power_state /opt/stack/nova/nova/compute/manager.py:1782}}
Nov 29 03:41:12 user nova-compute[75234]: DEBUG nova.compute.manager [req-4fec7e79-8ad7-45fe-8c57-2555a527a967 req-b2bd75d9-fdc6-4bf4-a267-bbb180b53db4 service nova] [instance: 95afc929-9f13-4f9a-9ac9-eae0b5e15a4d] Received event network-vif-plugged-ca1227f9-b2f1-42c5-a92a-c70cb94e52c1 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:41:12 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-4fec7e79-8ad7-45fe-8c57-2555a527a967 req-b2bd75d9-fdc6-4bf4-a267-bbb180b53db4 service nova] Acquiring lock "95afc929-9f13-4f9a-9ac9-eae0b5e15a4d-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:41:12 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-4fec7e79-8ad7-45fe-8c57-2555a527a967 req-b2bd75d9-fdc6-4bf4-a267-bbb180b53db4 service nova] Lock "95afc929-9f13-4f9a-9ac9-eae0b5e15a4d-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:41:12 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-4fec7e79-8ad7-45fe-8c57-2555a527a967 req-b2bd75d9-fdc6-4bf4-a267-bbb180b53db4 service nova] Lock "95afc929-9f13-4f9a-9ac9-eae0b5e15a4d-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:41:12 user nova-compute[75234]: DEBUG nova.compute.manager [req-4fec7e79-8ad7-45fe-8c57-2555a527a967 req-b2bd75d9-fdc6-4bf4-a267-bbb180b53db4 service nova] [instance: 95afc929-9f13-4f9a-9ac9-eae0b5e15a4d] No waiting events found dispatching network-vif-plugged-ca1227f9-b2f1-42c5-a92a-c70cb94e52c1 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:41:12 user nova-compute[75234]: WARNING nova.compute.manager [req-4fec7e79-8ad7-45fe-8c57-2555a527a967 req-b2bd75d9-fdc6-4bf4-a267-bbb180b53db4 service nova] [instance: 95afc929-9f13-4f9a-9ac9-eae0b5e15a4d] Received unexpected event network-vif-plugged-ca1227f9-b2f1-42c5-a92a-c70cb94e52c1 for instance with vm_state building and task_state spawning.
Nov 29 03:41:12 user nova-compute[75234]: INFO nova.compute.manager [None req-b8d98ced-f0fd-40c8-b1b7-5ed5946ba755 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: 95afc929-9f13-4f9a-9ac9-eae0b5e15a4d] Took 12.75 seconds to build instance.
Nov 29 03:41:12 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-b8d98ced-f0fd-40c8-b1b7-5ed5946ba755 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Lock "95afc929-9f13-4f9a-9ac9-eae0b5e15a4d" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 12.853s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:41:12 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:41:13 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:41:18 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:41:18 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:41:18 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:41:19 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:41:20 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:41:20 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=75234) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10541}}
Nov 29 03:41:23 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:41:24 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager.update_available_resource {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:41:24 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:41:24 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:41:24 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:41:24 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Auditing locally available compute resources for user (node: user) {{(pid=75234) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:907}}
Nov 29 03:41:24 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] skipping disk /dev/sdb (vda) as it is a volume {{(pid=75234) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11213}}
Nov 29 03:41:24 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:41:24 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:41:24 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Hypervisor/Node resource view: name=user free_ram=8896MB free_disk=25.164798736572266GB free_vcpus=11 pci_devices=[{"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}] {{(pid=75234) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1106}}
Nov 29 03:41:24 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:41:24 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:41:25 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance 95afc929-9f13-4f9a-9ac9-eae0b5e15a4d actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:41:25 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Total usable vcpus: 12, total allocated vcpus: 1 {{(pid=75234) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1129}}
Nov 29 03:41:25 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Final resource view: name=user phys_ram=16011MB used_ram=1024MB phys_disk=40GB used_disk=0GB total_vcpus=12 used_vcpus=1 pci_stats=[] {{(pid=75234) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1138}}
Nov 29 03:41:25 user nova-compute[75234]: DEBUG nova.compute.provider_tree [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Inventory has not changed in ProviderTree for provider: 4d291220-7c37-4839-a0e3-b23c66502b74 {{(pid=75234) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Nov 29 03:41:25 user nova-compute[75234]: DEBUG nova.scheduler.client.report [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Inventory has not changed for provider 4d291220-7c37-4839-a0e3-b23c66502b74 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=75234) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Nov 29 03:41:25 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Compute_service record updated for user:user {{(pid=75234) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:1067}}
Nov 29 03:41:25 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.216s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:41:26 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:41:26 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:41:27 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:41:28 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:41:28 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:41:28 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Starting heal instance info cache {{(pid=75234) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9922}}
Nov 29 03:41:28 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Rebuilding the list of instances to heal {{(pid=75234) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9926}}
Nov 29 03:41:28 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Acquiring lock "refresh_cache-95afc929-9f13-4f9a-9ac9-eae0b5e15a4d" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:41:28 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Acquired lock "refresh_cache-95afc929-9f13-4f9a-9ac9-eae0b5e15a4d" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:41:28 user nova-compute[75234]: DEBUG nova.network.neutron [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] [instance: 95afc929-9f13-4f9a-9ac9-eae0b5e15a4d] Forcefully refreshing network info cache for instance {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2003}}
Nov 29 03:41:28 user nova-compute[75234]: DEBUG nova.objects.instance [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lazy-loading 'info_cache' on Instance uuid 95afc929-9f13-4f9a-9ac9-eae0b5e15a4d {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:41:28 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:41:29 user nova-compute[75234]: DEBUG nova.network.neutron [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] [instance: 95afc929-9f13-4f9a-9ac9-eae0b5e15a4d] Updating instance_info_cache with network_info: [{"id": "ca1227f9-b2f1-42c5-a92a-c70cb94e52c1", "address": "fa:16:3e:02:94:55", "network": {"id": "a3f397d3-fd75-4a6e-a815-cfcedcca15fb", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-678570085-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "81a265d3c15b4e9d826b35ddbd0ff443", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapca1227f9-b2", "ovs_interfaceid": "ca1227f9-b2f1-42c5-a92a-c70cb94e52c1", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:41:29 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Releasing lock "refresh_cache-95afc929-9f13-4f9a-9ac9-eae0b5e15a4d" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:41:29 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] [instance: 95afc929-9f13-4f9a-9ac9-eae0b5e15a4d] Updated the network info_cache for instance {{(pid=75234) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9993}}
Nov 29 03:41:29 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:41:30 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:41:32 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:41:33 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:41:37 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:41:38 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:41:42 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:41:43 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:41:46 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._run_image_cache_manager_pass {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:41:46 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Acquiring lock "storage-registry-lock" by "nova.virt.storage_users.register_storage_use.<locals>.do_register_storage_use" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:41:46 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "storage-registry-lock" acquired by "nova.virt.storage_users.register_storage_use.<locals>.do_register_storage_use" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:41:46 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "storage-registry-lock" "released" by "nova.virt.storage_users.register_storage_use.<locals>.do_register_storage_use" :: held 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:41:46 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Acquiring lock "storage-registry-lock" by "nova.virt.storage_users.get_storage_users.<locals>.do_get_storage_users" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:41:46 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "storage-registry-lock" acquired by "nova.virt.storage_users.get_storage_users.<locals>.do_get_storage_users" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:41:46 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "storage-registry-lock" "released" by "nova.virt.storage_users.get_storage_users.<locals>.do_get_storage_users" :: held 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:41:46 user nova-compute[75234]: DEBUG nova.virt.libvirt.imagecache [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Verify base images {{(pid=75234) _age_and_verify_cached_images /opt/stack/nova/nova/virt/libvirt/imagecache.py:314}}
Nov 29 03:41:46 user nova-compute[75234]: DEBUG nova.virt.libvirt.imagecache [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Image id  yields fingerprint da39a3ee5e6b4b0d3255bfef95601890afd80709 {{(pid=75234) _age_and_verify_cached_images /opt/stack/nova/nova/virt/libvirt/imagecache.py:319}}
Nov 29 03:41:46 user nova-compute[75234]: DEBUG nova.virt.libvirt.imagecache [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] 95afc929-9f13-4f9a-9ac9-eae0b5e15a4d is a valid instance name {{(pid=75234) _list_backing_images /opt/stack/nova/nova/virt/libvirt/imagecache.py:126}}
Nov 29 03:41:46 user nova-compute[75234]: WARNING nova.virt.libvirt.imagecache [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Unknown base file: /opt/stack/data/nova/instances/_base/4d5d356d29424aea82c5ad651579c796fc09ff4d
Nov 29 03:41:46 user nova-compute[75234]: WARNING nova.virt.libvirt.imagecache [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Unknown base file: /opt/stack/data/nova/instances/_base/db2d1f2e1d03090216196960bcd01ca47391c21d
Nov 29 03:41:46 user nova-compute[75234]: WARNING nova.virt.libvirt.imagecache [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Unknown base file: /opt/stack/data/nova/instances/_base/d54adabc8d95c75567f7840e57b0a84314361a7e
Nov 29 03:41:46 user nova-compute[75234]: WARNING nova.virt.libvirt.imagecache [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Unknown base file: /opt/stack/data/nova/instances/_base/a3fb13f48aa8955e4a2ec143d3ba27674e6f5fdd
Nov 29 03:41:46 user nova-compute[75234]: WARNING nova.virt.libvirt.imagecache [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Unknown base file: /opt/stack/data/nova/instances/_base/10d9c8f4c9d1fbded8da4c61ee5f1f19a245d0ae
Nov 29 03:41:46 user nova-compute[75234]: WARNING nova.virt.libvirt.imagecache [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Unknown base file: /opt/stack/data/nova/instances/_base/3d36fa259f782d8fe9840ba00e3e31df800d42d3
Nov 29 03:41:46 user nova-compute[75234]: WARNING nova.virt.libvirt.imagecache [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Unknown base file: /opt/stack/data/nova/instances/_base/f30afef49830ef804887cdc7e9ea0848425eefd7
Nov 29 03:41:46 user nova-compute[75234]: WARNING nova.virt.libvirt.imagecache [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Unknown base file: /opt/stack/data/nova/instances/_base/611b721f56c5ee20e0f039f936b83393504a68ef
Nov 29 03:41:46 user nova-compute[75234]: WARNING nova.virt.libvirt.imagecache [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Unknown base file: /opt/stack/data/nova/instances/_base/7f7598d3f4a27ebdcdcae511dd4e04ef3c67d957
Nov 29 03:41:46 user nova-compute[75234]: WARNING nova.virt.libvirt.imagecache [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Unknown base file: /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918
Nov 29 03:41:46 user nova-compute[75234]: INFO nova.virt.libvirt.imagecache [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Removable base files: /opt/stack/data/nova/instances/_base/4d5d356d29424aea82c5ad651579c796fc09ff4d /opt/stack/data/nova/instances/_base/db2d1f2e1d03090216196960bcd01ca47391c21d /opt/stack/data/nova/instances/_base/d54adabc8d95c75567f7840e57b0a84314361a7e /opt/stack/data/nova/instances/_base/a3fb13f48aa8955e4a2ec143d3ba27674e6f5fdd /opt/stack/data/nova/instances/_base/10d9c8f4c9d1fbded8da4c61ee5f1f19a245d0ae /opt/stack/data/nova/instances/_base/3d36fa259f782d8fe9840ba00e3e31df800d42d3 /opt/stack/data/nova/instances/_base/f30afef49830ef804887cdc7e9ea0848425eefd7 /opt/stack/data/nova/instances/_base/611b721f56c5ee20e0f039f936b83393504a68ef /opt/stack/data/nova/instances/_base/7f7598d3f4a27ebdcdcae511dd4e04ef3c67d957 /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918
Nov 29 03:41:46 user nova-compute[75234]: INFO nova.virt.libvirt.imagecache [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Base, swap or ephemeral file too young to remove: /opt/stack/data/nova/instances/_base/4d5d356d29424aea82c5ad651579c796fc09ff4d
Nov 29 03:41:46 user nova-compute[75234]: INFO nova.virt.libvirt.imagecache [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Base, swap or ephemeral file too young to remove: /opt/stack/data/nova/instances/_base/db2d1f2e1d03090216196960bcd01ca47391c21d
Nov 29 03:41:46 user nova-compute[75234]: INFO nova.virt.libvirt.imagecache [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Base, swap or ephemeral file too young to remove: /opt/stack/data/nova/instances/_base/d54adabc8d95c75567f7840e57b0a84314361a7e
Nov 29 03:41:46 user nova-compute[75234]: INFO nova.virt.libvirt.imagecache [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Base, swap or ephemeral file too young to remove: /opt/stack/data/nova/instances/_base/a3fb13f48aa8955e4a2ec143d3ba27674e6f5fdd
Nov 29 03:41:46 user nova-compute[75234]: INFO nova.virt.libvirt.imagecache [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Base, swap or ephemeral file too young to remove: /opt/stack/data/nova/instances/_base/10d9c8f4c9d1fbded8da4c61ee5f1f19a245d0ae
Nov 29 03:41:46 user nova-compute[75234]: INFO nova.virt.libvirt.imagecache [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Base, swap or ephemeral file too young to remove: /opt/stack/data/nova/instances/_base/3d36fa259f782d8fe9840ba00e3e31df800d42d3
Nov 29 03:41:46 user nova-compute[75234]: INFO nova.virt.libvirt.imagecache [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Base, swap or ephemeral file too young to remove: /opt/stack/data/nova/instances/_base/f30afef49830ef804887cdc7e9ea0848425eefd7
Nov 29 03:41:46 user nova-compute[75234]: INFO nova.virt.libvirt.imagecache [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Base, swap or ephemeral file too young to remove: /opt/stack/data/nova/instances/_base/611b721f56c5ee20e0f039f936b83393504a68ef
Nov 29 03:41:46 user nova-compute[75234]: INFO nova.virt.libvirt.imagecache [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Base, swap or ephemeral file too young to remove: /opt/stack/data/nova/instances/_base/7f7598d3f4a27ebdcdcae511dd4e04ef3c67d957
Nov 29 03:41:46 user nova-compute[75234]: INFO nova.virt.libvirt.imagecache [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Base, swap or ephemeral file too young to remove: /opt/stack/data/nova/instances/_base/80686e29dc9db6600d03e85cb62af298e12d7918
Nov 29 03:41:46 user nova-compute[75234]: DEBUG nova.virt.libvirt.imagecache [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Verification complete {{(pid=75234) _age_and_verify_cached_images /opt/stack/nova/nova/virt/libvirt/imagecache.py:350}}
Nov 29 03:41:46 user nova-compute[75234]: DEBUG nova.virt.libvirt.imagecache [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Verify swap images {{(pid=75234) _age_and_verify_swap_images /opt/stack/nova/nova/virt/libvirt/imagecache.py:299}}
Nov 29 03:41:46 user nova-compute[75234]: DEBUG nova.virt.libvirt.imagecache [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Verify ephemeral images {{(pid=75234) _age_and_verify_ephemeral_images /opt/stack/nova/nova/virt/libvirt/imagecache.py:284}}
Nov 29 03:41:48 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:41:48 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:41:48 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5002 ms, sending inactivity probe {{(pid=75234) run /opt/stack/data/venv/lib/python3.10/site-packages/ovs/reconnect.py:117}}
Nov 29 03:41:48 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=75234) _transition /opt/stack/data/venv/lib/python3.10/site-packages/ovs/reconnect.py:519}}
Nov 29 03:41:48 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=75234) _transition /opt/stack/data/venv/lib/python3.10/site-packages/ovs/reconnect.py:519}}
Nov 29 03:41:48 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:41:53 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:41:58 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:42:03 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:42:08 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:42:13 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:42:18 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:42:18 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:42:19 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:42:21 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:42:21 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._cleanup_incomplete_migrations {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:42:21 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Cleaning up deleted instances with incomplete migration  {{(pid=75234) _cleanup_incomplete_migrations /opt/stack/nova/nova/compute/manager.py:11247}}
Nov 29 03:42:22 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:42:22 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=75234) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10541}}
Nov 29 03:42:23 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:42:24 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager.update_available_resource {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:42:24 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:42:24 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:42:24 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:42:24 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Auditing locally available compute resources for user (node: user) {{(pid=75234) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:907}}
Nov 29 03:42:24 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] skipping disk /dev/sdb (vda) as it is a volume {{(pid=75234) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11213}}
Nov 29 03:42:24 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:42:24 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:42:24 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Hypervisor/Node resource view: name=user free_ram=8952MB free_disk=25.162620544433594GB free_vcpus=11 pci_devices=[{"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}] {{(pid=75234) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1106}}
Nov 29 03:42:24 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:42:24 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:42:25 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance 95afc929-9f13-4f9a-9ac9-eae0b5e15a4d actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:42:25 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Total usable vcpus: 12, total allocated vcpus: 1 {{(pid=75234) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1129}}
Nov 29 03:42:25 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Final resource view: name=user phys_ram=16011MB used_ram=1024MB phys_disk=40GB used_disk=0GB total_vcpus=12 used_vcpus=1 pci_stats=[] {{(pid=75234) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1138}}
Nov 29 03:42:25 user nova-compute[75234]: DEBUG nova.compute.provider_tree [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Inventory has not changed in ProviderTree for provider: 4d291220-7c37-4839-a0e3-b23c66502b74 {{(pid=75234) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Nov 29 03:42:25 user nova-compute[75234]: DEBUG nova.scheduler.client.report [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Inventory has not changed for provider 4d291220-7c37-4839-a0e3-b23c66502b74 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=75234) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Nov 29 03:42:25 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Compute_service record updated for user:user {{(pid=75234) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:1067}}
Nov 29 03:42:25 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.302s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:42:28 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:42:30 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:42:30 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:42:30 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Starting heal instance info cache {{(pid=75234) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9922}}
Nov 29 03:42:30 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Rebuilding the list of instances to heal {{(pid=75234) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9926}}
Nov 29 03:42:30 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Acquiring lock "refresh_cache-95afc929-9f13-4f9a-9ac9-eae0b5e15a4d" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:42:30 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Acquired lock "refresh_cache-95afc929-9f13-4f9a-9ac9-eae0b5e15a4d" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:42:30 user nova-compute[75234]: DEBUG nova.network.neutron [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] [instance: 95afc929-9f13-4f9a-9ac9-eae0b5e15a4d] Forcefully refreshing network info cache for instance {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2003}}
Nov 29 03:42:30 user nova-compute[75234]: DEBUG nova.objects.instance [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lazy-loading 'info_cache' on Instance uuid 95afc929-9f13-4f9a-9ac9-eae0b5e15a4d {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:42:30 user nova-compute[75234]: DEBUG nova.network.neutron [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] [instance: 95afc929-9f13-4f9a-9ac9-eae0b5e15a4d] Updating instance_info_cache with network_info: [{"id": "ca1227f9-b2f1-42c5-a92a-c70cb94e52c1", "address": "fa:16:3e:02:94:55", "network": {"id": "a3f397d3-fd75-4a6e-a815-cfcedcca15fb", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-678570085-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "81a265d3c15b4e9d826b35ddbd0ff443", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapca1227f9-b2", "ovs_interfaceid": "ca1227f9-b2f1-42c5-a92a-c70cb94e52c1", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:42:30 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Releasing lock "refresh_cache-95afc929-9f13-4f9a-9ac9-eae0b5e15a4d" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:42:30 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] [instance: 95afc929-9f13-4f9a-9ac9-eae0b5e15a4d] Updated the network info_cache for instance {{(pid=75234) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9993}}
Nov 29 03:42:30 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:42:31 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:42:33 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:42:33 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:42:33 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5002 ms, sending inactivity probe {{(pid=75234) run /opt/stack/data/venv/lib/python3.10/site-packages/ovs/reconnect.py:117}}
Nov 29 03:42:33 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=75234) _transition /opt/stack/data/venv/lib/python3.10/site-packages/ovs/reconnect.py:519}}
Nov 29 03:42:33 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=75234) _transition /opt/stack/data/venv/lib/python3.10/site-packages/ovs/reconnect.py:519}}
Nov 29 03:42:33 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:42:35 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._sync_scheduler_instance_info {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:42:37 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._sync_power_states {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:42:37 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Triggering sync for uuid 95afc929-9f13-4f9a-9ac9-eae0b5e15a4d {{(pid=75234) _sync_power_states /opt/stack/nova/nova/compute/manager.py:10332}}
Nov 29 03:42:37 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Acquiring lock "95afc929-9f13-4f9a-9ac9-eae0b5e15a4d" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:42:37 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "95afc929-9f13-4f9a-9ac9-eae0b5e15a4d" acquired by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:42:38 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "95afc929-9f13-4f9a-9ac9-eae0b5e15a4d" "released" by "nova.compute.manager.ComputeManager._sync_power_states.<locals>._sync.<locals>.query_driver_power_state_and_sync" :: held 0.046s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:42:38 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4997-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:42:39 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._cleanup_expired_console_auth_tokens {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:42:43 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._run_pending_deletes {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:42:43 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Cleaning up deleted instances {{(pid=75234) _run_pending_deletes /opt/stack/nova/nova/compute/manager.py:11209}}
Nov 29 03:42:43 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] There are 0 instances to clean {{(pid=75234) _run_pending_deletes /opt/stack/nova/nova/compute/manager.py:11218}}
Nov 29 03:42:43 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:42:47 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:42:48 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:42:51 user nova-compute[75234]: DEBUG nova.compute.manager [req-24fce0a5-7982-4d9c-9f73-1aff265c6f10 req-612a8560-2996-44e5-accf-d1e1769f49e1 service nova] [instance: 95afc929-9f13-4f9a-9ac9-eae0b5e15a4d] Received event network-changed-ca1227f9-b2f1-42c5-a92a-c70cb94e52c1 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:42:51 user nova-compute[75234]: DEBUG nova.compute.manager [req-24fce0a5-7982-4d9c-9f73-1aff265c6f10 req-612a8560-2996-44e5-accf-d1e1769f49e1 service nova] [instance: 95afc929-9f13-4f9a-9ac9-eae0b5e15a4d] Refreshing instance network info cache due to event network-changed-ca1227f9-b2f1-42c5-a92a-c70cb94e52c1. {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11117}}
Nov 29 03:42:51 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-24fce0a5-7982-4d9c-9f73-1aff265c6f10 req-612a8560-2996-44e5-accf-d1e1769f49e1 service nova] Acquiring lock "refresh_cache-95afc929-9f13-4f9a-9ac9-eae0b5e15a4d" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:42:51 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-24fce0a5-7982-4d9c-9f73-1aff265c6f10 req-612a8560-2996-44e5-accf-d1e1769f49e1 service nova] Acquired lock "refresh_cache-95afc929-9f13-4f9a-9ac9-eae0b5e15a4d" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:42:51 user nova-compute[75234]: DEBUG nova.network.neutron [req-24fce0a5-7982-4d9c-9f73-1aff265c6f10 req-612a8560-2996-44e5-accf-d1e1769f49e1 service nova] [instance: 95afc929-9f13-4f9a-9ac9-eae0b5e15a4d] Refreshing network info cache for port ca1227f9-b2f1-42c5-a92a-c70cb94e52c1 {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2006}}
Nov 29 03:42:52 user nova-compute[75234]: DEBUG nova.network.neutron [req-24fce0a5-7982-4d9c-9f73-1aff265c6f10 req-612a8560-2996-44e5-accf-d1e1769f49e1 service nova] [instance: 95afc929-9f13-4f9a-9ac9-eae0b5e15a4d] Updated VIF entry in instance network info cache for port ca1227f9-b2f1-42c5-a92a-c70cb94e52c1. {{(pid=75234) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3481}}
Nov 29 03:42:52 user nova-compute[75234]: DEBUG nova.network.neutron [req-24fce0a5-7982-4d9c-9f73-1aff265c6f10 req-612a8560-2996-44e5-accf-d1e1769f49e1 service nova] [instance: 95afc929-9f13-4f9a-9ac9-eae0b5e15a4d] Updating instance_info_cache with network_info: [{"id": "ca1227f9-b2f1-42c5-a92a-c70cb94e52c1", "address": "fa:16:3e:02:94:55", "network": {"id": "a3f397d3-fd75-4a6e-a815-cfcedcca15fb", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-678570085-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.17", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "81a265d3c15b4e9d826b35ddbd0ff443", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapca1227f9-b2", "ovs_interfaceid": "ca1227f9-b2f1-42c5-a92a-c70cb94e52c1", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:42:52 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-24fce0a5-7982-4d9c-9f73-1aff265c6f10 req-612a8560-2996-44e5-accf-d1e1769f49e1 service nova] Releasing lock "refresh_cache-95afc929-9f13-4f9a-9ac9-eae0b5e15a4d" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:42:53 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:42:53 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:42:58 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:42:58 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:42:59 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-5d33c997-7eef-48ee-85f2-5631e19d2695 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Acquiring lock "89c7681b-4a2c-485a-b446-e8ac4df43c2b" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:42:59 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-5d33c997-7eef-48ee-85f2-5631e19d2695 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Lock "89c7681b-4a2c-485a-b446-e8ac4df43c2b" acquired by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:42:59 user nova-compute[75234]: DEBUG nova.compute.manager [None req-5d33c997-7eef-48ee-85f2-5631e19d2695 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: 89c7681b-4a2c-485a-b446-e8ac4df43c2b] Starting instance... {{(pid=75234) _do_build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2422}}
Nov 29 03:42:59 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-5d33c997-7eef-48ee-85f2-5631e19d2695 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:42:59 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-5d33c997-7eef-48ee-85f2-5631e19d2695 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:42:59 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-5d33c997-7eef-48ee-85f2-5631e19d2695 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Require both a host and instance NUMA topology to fit instance on host. {{(pid=75234) numa_fit_instance_to_host /opt/stack/nova/nova/virt/hardware.py:2366}}
Nov 29 03:42:59 user nova-compute[75234]: INFO nova.compute.claims [None req-5d33c997-7eef-48ee-85f2-5631e19d2695 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: 89c7681b-4a2c-485a-b446-e8ac4df43c2b] Claim successful on node user
Nov 29 03:42:59 user nova-compute[75234]: DEBUG nova.compute.provider_tree [None req-5d33c997-7eef-48ee-85f2-5631e19d2695 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Inventory has not changed in ProviderTree for provider: 4d291220-7c37-4839-a0e3-b23c66502b74 {{(pid=75234) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Nov 29 03:42:59 user nova-compute[75234]: DEBUG nova.scheduler.client.report [None req-5d33c997-7eef-48ee-85f2-5631e19d2695 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Inventory has not changed for provider 4d291220-7c37-4839-a0e3-b23c66502b74 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=75234) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Nov 29 03:42:59 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-5d33c997-7eef-48ee-85f2-5631e19d2695 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.instance_claim" :: held 0.252s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:42:59 user nova-compute[75234]: DEBUG nova.compute.manager [None req-5d33c997-7eef-48ee-85f2-5631e19d2695 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: 89c7681b-4a2c-485a-b446-e8ac4df43c2b] Start building networks asynchronously for instance. {{(pid=75234) _build_resources /opt/stack/nova/nova/compute/manager.py:2819}}
Nov 29 03:42:59 user nova-compute[75234]: DEBUG nova.compute.manager [None req-5d33c997-7eef-48ee-85f2-5631e19d2695 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: 89c7681b-4a2c-485a-b446-e8ac4df43c2b] Allocating IP information in the background. {{(pid=75234) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1972}}
Nov 29 03:42:59 user nova-compute[75234]: DEBUG nova.network.neutron [None req-5d33c997-7eef-48ee-85f2-5631e19d2695 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: 89c7681b-4a2c-485a-b446-e8ac4df43c2b] allocate_for_instance() {{(pid=75234) allocate_for_instance /opt/stack/nova/nova/network/neutron.py:1156}}
Nov 29 03:42:59 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-5d33c997-7eef-48ee-85f2-5631e19d2695 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: 89c7681b-4a2c-485a-b446-e8ac4df43c2b] Ignoring supplied device name: /dev/vda. Libvirt can't honour user-supplied dev names
Nov 29 03:42:59 user nova-compute[75234]: DEBUG nova.compute.manager [None req-5d33c997-7eef-48ee-85f2-5631e19d2695 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: 89c7681b-4a2c-485a-b446-e8ac4df43c2b] Start building block device mappings for instance. {{(pid=75234) _build_resources /opt/stack/nova/nova/compute/manager.py:2854}}
Nov 29 03:42:59 user nova-compute[75234]: DEBUG nova.policy [None req-5d33c997-7eef-48ee-85f2-5631e19d2695 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Policy check for network:attach_external_network failed with credentials {'is_admin': False, 'user_id': '7d1f1f11a9aa4c1db8627413eb963fa4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '81a265d3c15b4e9d826b35ddbd0ff443', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=75234) authorize /opt/stack/nova/nova/policy.py:203}}
Nov 29 03:42:59 user nova-compute[75234]: INFO nova.virt.block_device [None req-5d33c997-7eef-48ee-85f2-5631e19d2695 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: 89c7681b-4a2c-485a-b446-e8ac4df43c2b] Booting with volume 3ae025b2-997b-4f06-8e4b-4d6a37f9e3bc at /dev/vda
Nov 29 03:43:00 user nova-compute[75234]: DEBUG os_brick.utils [None req-5d33c997-7eef-48ee-85f2-5631e19d2695 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] ==> get_connector_properties: call "{'root_helper': 'sudo nova-rootwrap /etc/nova/rootwrap.conf', 'my_ip': '10.0.0.210', 'multipath': False, 'enforce_multipath': True, 'host': 'user', 'execute': None}" {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:176}}
Nov 29 03:43:00 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): cat /etc/iscsi/initiatorname.iscsi {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:43:00 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.014s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:43:00 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[dec4177f-2670-494f-9ae0-425a067a2ace]: (4, ('InitiatorName=iqn.2016-04.com.open-iscsi:dd1368c5ba2c\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:43:00 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): findmnt -v / -n -o SOURCE {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:43:00 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "findmnt -v / -n -o SOURCE" returned: 0 in 0.016s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:43:00 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[ece9203f-4122-42f6-8080-8e102591d3a9]: (4, ('/dev/mapper/vg0-lv--0\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:43:00 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blkid /dev/mapper/vg0-lv--0 -s UUID -o value {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:43:00 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "blkid /dev/mapper/vg0-lv--0 -s UUID -o value" returned: 0 in 0.014s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:43:00 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[2e5c8bb1-122a-475d-a64b-ee29c1ddc42d]: (4, ('e95b3b51-542d-42ca-ac40-f83360608668\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:43:00 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[a97e26f2-a339-44d3-bd07-0c49040f8c4b]: (4, 'e20c3142-5af9-7467-ecd8-70b2e4a210d6') {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:43:00 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-5d33c997-7eef-48ee-85f2-5631e19d2695 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Running cmd (subprocess): nvme version {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:43:00 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [None req-5d33c997-7eef-48ee-85f2-5631e19d2695 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] 'nvme version' failed. Not Retrying. {{(pid=75234) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:473}}
Nov 29 03:43:00 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.nvmeof [None req-5d33c997-7eef-48ee-85f2-5631e19d2695 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] nvme not present on system {{(pid=75234) nvme_present /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/nvmeof.py:756}}
Nov 29 03:43:00 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): nvme show-hostnqn {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:43:00 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] 'nvme show-hostnqn' failed. Not Retrying. {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:473}}
Nov 29 03:43:00 user nova-compute[75234]: WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: [Errno 2] No such file or directory: 'nvme'
Nov 29 03:43:00 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[2c9cc484-c046-4460-9281-5b7d1c5034b7]: (4, '') {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:43:00 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.lightos [None req-5d33c997-7eef-48ee-85f2-5631e19d2695 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] LIGHTOS: [Errno 111] ECONNREFUSED {{(pid=75234) find_dsc /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/lightos.py:98}}
Nov 29 03:43:00 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.lightos [None req-5d33c997-7eef-48ee-85f2-5631e19d2695 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] LIGHTOS: did not find dsc, continuing anyway. {{(pid=75234) get_connector_properties /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/lightos.py:76}}
Nov 29 03:43:00 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.lightos [None req-5d33c997-7eef-48ee-85f2-5631e19d2695 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] LIGHTOS: no hostnqn found. {{(pid=75234) get_connector_properties /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/lightos.py:84}}
Nov 29 03:43:00 user nova-compute[75234]: DEBUG os_brick.utils [None req-5d33c997-7eef-48ee-85f2-5631e19d2695 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] <== get_connector_properties: return (117ms) {'platform': 'x86_64', 'os_type': 'linux', 'ip': '10.0.0.210', 'host': 'user', 'multipath': False, 'initiator': 'iqn.2016-04.com.open-iscsi:dd1368c5ba2c', 'do_local_attach': False, 'uuid': 'e95b3b51-542d-42ca-ac40-f83360608668', 'system uuid': 'e20c3142-5af9-7467-ecd8-70b2e4a210d6', 'nvme_native_multipath': False} {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:203}}
Nov 29 03:43:00 user nova-compute[75234]: DEBUG nova.virt.block_device [None req-5d33c997-7eef-48ee-85f2-5631e19d2695 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: 89c7681b-4a2c-485a-b446-e8ac4df43c2b] Updating existing volume attachment record: 1b753ab8-ca8d-44ce-8016-900bcf9e9556 {{(pid=75234) _volume_attach /opt/stack/nova/nova/virt/block_device.py:631}}
Nov 29 03:43:00 user nova-compute[75234]: DEBUG nova.network.neutron [None req-5d33c997-7eef-48ee-85f2-5631e19d2695 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: 89c7681b-4a2c-485a-b446-e8ac4df43c2b] Successfully created port: 18af07bc-c94c-4faa-9008-8be55e548870 {{(pid=75234) _create_port_minimal /opt/stack/nova/nova/network/neutron.py:548}}
Nov 29 03:43:01 user nova-compute[75234]: DEBUG nova.network.neutron [None req-5d33c997-7eef-48ee-85f2-5631e19d2695 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: 89c7681b-4a2c-485a-b446-e8ac4df43c2b] Successfully updated port: 18af07bc-c94c-4faa-9008-8be55e548870 {{(pid=75234) _update_port /opt/stack/nova/nova/network/neutron.py:586}}
Nov 29 03:43:01 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-5d33c997-7eef-48ee-85f2-5631e19d2695 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Acquiring lock "refresh_cache-89c7681b-4a2c-485a-b446-e8ac4df43c2b" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:43:01 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-5d33c997-7eef-48ee-85f2-5631e19d2695 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Acquired lock "refresh_cache-89c7681b-4a2c-485a-b446-e8ac4df43c2b" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:43:01 user nova-compute[75234]: DEBUG nova.network.neutron [None req-5d33c997-7eef-48ee-85f2-5631e19d2695 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: 89c7681b-4a2c-485a-b446-e8ac4df43c2b] Building network info cache for instance {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2009}}
Nov 29 03:43:01 user nova-compute[75234]: DEBUG nova.network.neutron [None req-5d33c997-7eef-48ee-85f2-5631e19d2695 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: 89c7681b-4a2c-485a-b446-e8ac4df43c2b] Instance cache missing network info. {{(pid=75234) _get_preexisting_port_ids /opt/stack/nova/nova/network/neutron.py:3322}}
Nov 29 03:43:01 user nova-compute[75234]: DEBUG nova.compute.manager [req-91a5a555-d80c-4942-903e-f2708f5a6a8c req-cd03ff66-d2d0-49de-9d58-e0282b25451a service nova] [instance: 89c7681b-4a2c-485a-b446-e8ac4df43c2b] Received event network-changed-18af07bc-c94c-4faa-9008-8be55e548870 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:43:01 user nova-compute[75234]: DEBUG nova.compute.manager [req-91a5a555-d80c-4942-903e-f2708f5a6a8c req-cd03ff66-d2d0-49de-9d58-e0282b25451a service nova] [instance: 89c7681b-4a2c-485a-b446-e8ac4df43c2b] Refreshing instance network info cache due to event network-changed-18af07bc-c94c-4faa-9008-8be55e548870. {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11117}}
Nov 29 03:43:01 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-91a5a555-d80c-4942-903e-f2708f5a6a8c req-cd03ff66-d2d0-49de-9d58-e0282b25451a service nova] Acquiring lock "refresh_cache-89c7681b-4a2c-485a-b446-e8ac4df43c2b" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:43:01 user nova-compute[75234]: DEBUG nova.network.neutron [None req-5d33c997-7eef-48ee-85f2-5631e19d2695 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: 89c7681b-4a2c-485a-b446-e8ac4df43c2b] Updating instance_info_cache with network_info: [{"id": "18af07bc-c94c-4faa-9008-8be55e548870", "address": "fa:16:3e:43:5a:c5", "network": {"id": "a3f397d3-fd75-4a6e-a815-cfcedcca15fb", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-678570085-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "81a265d3c15b4e9d826b35ddbd0ff443", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap18af07bc-c9", "ovs_interfaceid": "18af07bc-c94c-4faa-9008-8be55e548870", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:43:01 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-5d33c997-7eef-48ee-85f2-5631e19d2695 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Releasing lock "refresh_cache-89c7681b-4a2c-485a-b446-e8ac4df43c2b" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:43:01 user nova-compute[75234]: DEBUG nova.compute.manager [None req-5d33c997-7eef-48ee-85f2-5631e19d2695 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: 89c7681b-4a2c-485a-b446-e8ac4df43c2b] Instance network_info: |[{"id": "18af07bc-c94c-4faa-9008-8be55e548870", "address": "fa:16:3e:43:5a:c5", "network": {"id": "a3f397d3-fd75-4a6e-a815-cfcedcca15fb", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-678570085-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "81a265d3c15b4e9d826b35ddbd0ff443", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap18af07bc-c9", "ovs_interfaceid": "18af07bc-c94c-4faa-9008-8be55e548870", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}]| {{(pid=75234) _allocate_network_async /opt/stack/nova/nova/compute/manager.py:1987}}
Nov 29 03:43:01 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-91a5a555-d80c-4942-903e-f2708f5a6a8c req-cd03ff66-d2d0-49de-9d58-e0282b25451a service nova] Acquired lock "refresh_cache-89c7681b-4a2c-485a-b446-e8ac4df43c2b" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:43:01 user nova-compute[75234]: DEBUG nova.network.neutron [req-91a5a555-d80c-4942-903e-f2708f5a6a8c req-cd03ff66-d2d0-49de-9d58-e0282b25451a service nova] [instance: 89c7681b-4a2c-485a-b446-e8ac4df43c2b] Refreshing network info cache for port 18af07bc-c94c-4faa-9008-8be55e548870 {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2006}}
Nov 29 03:43:02 user nova-compute[75234]: DEBUG nova.network.neutron [req-91a5a555-d80c-4942-903e-f2708f5a6a8c req-cd03ff66-d2d0-49de-9d58-e0282b25451a service nova] [instance: 89c7681b-4a2c-485a-b446-e8ac4df43c2b] Updated VIF entry in instance network info cache for port 18af07bc-c94c-4faa-9008-8be55e548870. {{(pid=75234) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3481}}
Nov 29 03:43:02 user nova-compute[75234]: DEBUG nova.network.neutron [req-91a5a555-d80c-4942-903e-f2708f5a6a8c req-cd03ff66-d2d0-49de-9d58-e0282b25451a service nova] [instance: 89c7681b-4a2c-485a-b446-e8ac4df43c2b] Updating instance_info_cache with network_info: [{"id": "18af07bc-c94c-4faa-9008-8be55e548870", "address": "fa:16:3e:43:5a:c5", "network": {"id": "a3f397d3-fd75-4a6e-a815-cfcedcca15fb", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-678570085-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "81a265d3c15b4e9d826b35ddbd0ff443", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap18af07bc-c9", "ovs_interfaceid": "18af07bc-c94c-4faa-9008-8be55e548870", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:43:02 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-91a5a555-d80c-4942-903e-f2708f5a6a8c req-cd03ff66-d2d0-49de-9d58-e0282b25451a service nova] Releasing lock "refresh_cache-89c7681b-4a2c-485a-b446-e8ac4df43c2b" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:43:03 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:43:03 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:43:07 user nova-compute[75234]: DEBUG nova.compute.manager [None req-5d33c997-7eef-48ee-85f2-5631e19d2695 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: 89c7681b-4a2c-485a-b446-e8ac4df43c2b] Start spawning the instance on the hypervisor. {{(pid=75234) _build_and_run_instance /opt/stack/nova/nova/compute/manager.py:2628}}
Nov 29 03:43:07 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-5d33c997-7eef-48ee-85f2-5631e19d2695 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: 89c7681b-4a2c-485a-b446-e8ac4df43c2b] Creating instance directory {{(pid=75234) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4727}}
Nov 29 03:43:07 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-5d33c997-7eef-48ee-85f2-5631e19d2695 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: 89c7681b-4a2c-485a-b446-e8ac4df43c2b] Creating image(s)
Nov 29 03:43:07 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-5d33c997-7eef-48ee-85f2-5631e19d2695 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: 89c7681b-4a2c-485a-b446-e8ac4df43c2b] Did not create local disks {{(pid=75234) _create_image /opt/stack/nova/nova/virt/libvirt/driver.py:4863}}
Nov 29 03:43:07 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-5d33c997-7eef-48ee-85f2-5631e19d2695 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: 89c7681b-4a2c-485a-b446-e8ac4df43c2b] Ensure instance console log exists: /opt/stack/data/nova/instances/89c7681b-4a2c-485a-b446-e8ac4df43c2b/console.log {{(pid=75234) _ensure_console_log_for_instance /opt/stack/nova/nova/virt/libvirt/driver.py:4613}}
Nov 29 03:43:07 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-5d33c997-7eef-48ee-85f2-5631e19d2695 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Acquiring lock "vgpu_resources" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:43:07 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-5d33c997-7eef-48ee-85f2-5631e19d2695 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Lock "vgpu_resources" acquired by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:43:07 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-5d33c997-7eef-48ee-85f2-5631e19d2695 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Lock "vgpu_resources" "released" by "nova.virt.libvirt.driver.LibvirtDriver._allocate_mdevs" :: held 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:43:07 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-5d33c997-7eef-48ee-85f2-5631e19d2695 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: 89c7681b-4a2c-485a-b446-e8ac4df43c2b] Start _get_guest_xml network_info=[{"id": "18af07bc-c94c-4faa-9008-8be55e548870", "address": "fa:16:3e:43:5a:c5", "network": {"id": "a3f397d3-fd75-4a6e-a815-cfcedcca15fb", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-678570085-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "81a265d3c15b4e9d826b35ddbd0ff443", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap18af07bc-c9", "ovs_interfaceid": "18af07bc-c94c-4faa-9008-8be55e548870", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] disk_info={'disk_bus': 'virtio', 'cdrom_bus': 'ide', 'mapping': {'root': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}, '/dev/vda': {'bus': 'virtio', 'dev': 'vda', 'type': 'disk', 'boot_index': '1'}}} image_meta=ImageMeta(checksum=<?>,container_format=<?>,created_at=<?>,direct_url=<?>,disk_format=<?>,id=<?>,min_disk=0,min_ram=0,name=<?>,owner=<?>,properties=ImageMetaProps,protected=<?>,size=1073741824,status='active',tags=<?>,updated_at=<?>,virtual_size=<?>,visibility=<?>) rescue=None block_device_info={'root_device_name': '/dev/vda', 'image': [], 'ephemerals': [], 'block_device_mapping': [{'mount_device': '/dev/vda', 'attachment_id': '1b753ab8-ca8d-44ce-8016-900bcf9e9556', 'disk_bus': 'virtio', 'device_type': 'disk', 'boot_index': 0, 'guest_format': None, 'delete_on_termination': False, 'connection_info': {'driver_volume_type': 'iscsi', 'data': {'target_iqn': 'iqn.2010-10.org.openstack:3ae025b2-997b-4f06-8e4b-4d6a37f9e3bc', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'DbfMxnTa', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False}, 'status': 'reserved', 'instance': '89c7681b-4a2c-485a-b446-e8ac4df43c2b', 'attached_at': '', 'detached_at': '', 'volume_id': '3ae025b2-997b-4f06-8e4b-4d6a37f9e3bc', 'serial': '3ae025b2-997b-4f06-8e4b-4d6a37f9e3bc'}, 'volume_type': None} {{(pid=75234) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7562}}
Nov 29 03:43:07 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-5d33c997-7eef-48ee-85f2-5631e19d2695 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:43:07 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-5d33c997-7eef-48ee-85f2-5631e19d2695 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:43:07 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-5d33c997-7eef-48ee-85f2-5631e19d2695 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Searching host: 'user' for CPU controller through CGroups V1... {{(pid=75234) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1652}}
Nov 29 03:43:07 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-5d33c997-7eef-48ee-85f2-5631e19d2695 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] CPU controller missing on host. {{(pid=75234) _has_cgroupsv1_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1662}}
Nov 29 03:43:07 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-5d33c997-7eef-48ee-85f2-5631e19d2695 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Searching host: 'user' for CPU controller through CGroups V2... {{(pid=75234) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1671}}
Nov 29 03:43:07 user nova-compute[75234]: DEBUG nova.virt.libvirt.host [None req-5d33c997-7eef-48ee-85f2-5631e19d2695 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] CPU controller found on host. {{(pid=75234) _has_cgroupsv2_cpu_controller /opt/stack/nova/nova/virt/libvirt/host.py:1678}}
Nov 29 03:43:07 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-5d33c997-7eef-48ee-85f2-5631e19d2695 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] CPU mode 'custom' models 'Nehalem' was chosen, with extra flags: '' {{(pid=75234) _get_guest_cpu_model_config /opt/stack/nova/nova/virt/libvirt/driver.py:5400}}
Nov 29 03:43:07 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-5d33c997-7eef-48ee-85f2-5631e19d2695 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Getting desirable topologies for flavor Flavor(created_at=2023-11-29T03:02:04Z,deleted=False,deleted_at=None,description=None,disabled=False,ephemeral_gb=0,extra_specs={hw_rng:allowed='True'},flavorid='1',id=6,is_public=True,memory_mb=512,name='m1.tiny',projects=<?>,root_gb=1,rxtx_factor=1.0,swap=0,updated_at=None,vcpu_weight=0,vcpus=1) and image_meta ImageMeta(checksum=<?>,container_format=<?>,created_at=<?>,direct_url=<?>,disk_format=<?>,id=<?>,min_disk=0,min_ram=0,name=<?>,owner=<?>,properties=ImageMetaProps,protected=<?>,size=1073741824,status='active',tags=<?>,updated_at=<?>,virtual_size=<?>,visibility=<?>), allow threads: True {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:563}}
Nov 29 03:43:07 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-5d33c997-7eef-48ee-85f2-5631e19d2695 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Flavor limits 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:348}}
Nov 29 03:43:07 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-5d33c997-7eef-48ee-85f2-5631e19d2695 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Image limits 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:352}}
Nov 29 03:43:07 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-5d33c997-7eef-48ee-85f2-5631e19d2695 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Flavor pref 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:388}}
Nov 29 03:43:07 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-5d33c997-7eef-48ee-85f2-5631e19d2695 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Image pref 0:0:0 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:392}}
Nov 29 03:43:07 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-5d33c997-7eef-48ee-85f2-5631e19d2695 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=75234) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:430}}
Nov 29 03:43:07 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-5d33c997-7eef-48ee-85f2-5631e19d2695 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Topology preferred VirtCPUTopology(cores=0,sockets=0,threads=0), maximum VirtCPUTopology(cores=65536,sockets=65536,threads=65536) {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:569}}
Nov 29 03:43:07 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-5d33c997-7eef-48ee-85f2-5631e19d2695 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Build topologies for 1 vcpu(s) 1:1:1 {{(pid=75234) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:471}}
Nov 29 03:43:07 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-5d33c997-7eef-48ee-85f2-5631e19d2695 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Got 1 possible topologies {{(pid=75234) _get_possible_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:501}}
Nov 29 03:43:07 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-5d33c997-7eef-48ee-85f2-5631e19d2695 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Possible topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:575}}
Nov 29 03:43:07 user nova-compute[75234]: DEBUG nova.virt.hardware [None req-5d33c997-7eef-48ee-85f2-5631e19d2695 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Sorted desired topologies [VirtCPUTopology(cores=1,sockets=1,threads=1)] {{(pid=75234) _get_desirable_cpu_topologies /opt/stack/nova/nova/virt/hardware.py:577}}
Nov 29 03:43:07 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-5d33c997-7eef-48ee-85f2-5631e19d2695 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: 89c7681b-4a2c-485a-b446-e8ac4df43c2b] Calling os-brick to attach iSCSI Volume {{(pid=75234) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:63}}
Nov 29 03:43:07 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-5d33c997-7eef-48ee-85f2-5631e19d2695 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] ==> connect_volume: call "{'self': <os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f8a5050c9a0>, 'connection_properties': {'target_iqn': 'iqn.2010-10.org.openstack:3ae025b2-997b-4f06-8e4b-4d6a37f9e3bc', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'DbfMxnTa', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False}}" {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:176}}
Nov 29 03:43:07 user nova-compute[75234]: WARNING os_brick.initiator.connectors.base [None req-5d33c997-7eef-48ee-85f2-5631e19d2695 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Nov 29 03:43:07 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-5d33c997-7eef-48ee-85f2-5631e19d2695 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:68}}
Nov 29 03:43:07 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-5d33c997-7eef-48ee-85f2-5631e19d2695 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:73}}
Nov 29 03:43:07 user nova-compute[75234]: WARNING os_brick.initiator.connectors.base [None req-5d33c997-7eef-48ee-85f2-5631e19d2695 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Nov 29 03:43:07 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-5d33c997-7eef-48ee-85f2-5631e19d2695 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Acquiring lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:3ae025b2-997b-4f06-8e4b-4d6a37f9e3bc" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:68}}
Nov 29 03:43:07 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-5d33c997-7eef-48ee-85f2-5631e19d2695 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:3ae025b2-997b-4f06-8e4b-4d6a37f9e3bc" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:73}}
Nov 29 03:43:07 user nova-compute[75234]: INFO os_brick.initiator.connectors.iscsi [None req-5d33c997-7eef-48ee-85f2-5631e19d2695 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Trying to connect to iSCSI portal 172.16.0.220:3260
Nov 29 03:43:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:3ae025b2-997b-4f06-8e4b-4d6a37f9e3bc -p 172.16.0.220:3260 {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:43:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:3ae025b2-997b-4f06-8e4b-4d6a37f9e3bc -p 172.16.0.220:3260" returned: 21 in 0.019s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:43:07 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[386e3e0a-b04b-4119-bbc6-307a3d64d192]: (4, ('', 'iscsiadm: No records found\n')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:43:07 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-5d33c997-7eef-48ee-85f2-5631e19d2695 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] iscsiadm (): stdout= stderr=iscsiadm: No records found
Nov 29 03:43:07 user nova-compute[75234]:  {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:43:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:3ae025b2-997b-4f06-8e4b-4d6a37f9e3bc -p 172.16.0.220:3260 --interface default --op new {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:43:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:3ae025b2-997b-4f06-8e4b-4d6a37f9e3bc -p 172.16.0.220:3260 --interface default --op new" returned: 0 in 0.019s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:43:07 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[651b28e2-43b5-40f6-a877-f87f5ab953ee]: (4, ('New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:3ae025b2-997b-4f06-8e4b-4d6a37f9e3bc] added\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:43:07 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-5d33c997-7eef-48ee-85f2-5631e19d2695 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] iscsiadm ('--interface', 'default', '--op', 'new'): stdout=New iSCSI node [tcp:[hw=,ip=,net_if=,iscsi_if=default] 172.16.0.220,3260,-1 iqn.2010-10.org.openstack:3ae025b2-997b-4f06-8e4b-4d6a37f9e3bc] added
Nov 29 03:43:07 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:43:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:3ae025b2-997b-4f06-8e4b-4d6a37f9e3bc -p 172.16.0.220:3260 --op update -n node.session.scan -v manual {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:43:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:3ae025b2-997b-4f06-8e4b-4d6a37f9e3bc -p 172.16.0.220:3260 --op update -n node.session.scan -v manual" returned: 0 in 0.025s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:43:07 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[4ff847be-7367-4395-b5f9-8dda63919ddb]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:43:07 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-5d33c997-7eef-48ee-85f2-5631e19d2695 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.scan', '-v', 'manual'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:43:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:3ae025b2-997b-4f06-8e4b-4d6a37f9e3bc -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:43:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:3ae025b2-997b-4f06-8e4b-4d6a37f9e3bc -p 172.16.0.220:3260 --op update -n node.session.auth.authmethod -v CHAP" returned: 0 in 0.017s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:43:07 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[5ca6fd77-d1af-498c-ab0d-d192fe836188]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:43:07 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-5d33c997-7eef-48ee-85f2-5631e19d2695 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.authmethod', '-v', 'CHAP'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:43:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:3ae025b2-997b-4f06-8e4b-4d6a37f9e3bc -p 172.16.0.220:3260 --op update -n node.session.auth.username -v DbfMxnTa {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:43:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:3ae025b2-997b-4f06-8e4b-4d6a37f9e3bc -p 172.16.0.220:3260 --op update -n node.session.auth.username -v DbfMxnTa" returned: 0 in 0.017s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:43:07 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[0a0bacbd-4f0b-40e2-964e-3c6b49a175e9]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:43:07 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-5d33c997-7eef-48ee-85f2-5631e19d2695 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.username', '-v', 'DbfMxnTa'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:43:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:3ae025b2-997b-4f06-8e4b-4d6a37f9e3bc -p 172.16.0.220:3260 --op update -n node.session.auth.password -v *** {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:43:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:3ae025b2-997b-4f06-8e4b-4d6a37f9e3bc -p 172.16.0.220:3260 --op update -n node.session.auth.password -v ***" returned: 0 in 0.016s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:43:07 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[5cac6059-0259-4881-94b8-f4f88e9b1c63]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:43:07 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-5d33c997-7eef-48ee-85f2-5631e19d2695 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] iscsiadm ('--op', 'update', '-n', 'node.session.auth.password', '-v', '***'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:43:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:43:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.017s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:43:07 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[e1b5801d-6730-4424-a358-180efd771584]: (4, ('tcp: [59] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:378addbb-8d90-40c5-ae61-c3a063116df9 (non-flash)\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:43:07 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-5d33c997-7eef-48ee-85f2-5631e19d2695 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [59] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:378addbb-8d90-40c5-ae61-c3a063116df9 (non-flash)
Nov 29 03:43:07 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:43:07 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-5d33c997-7eef-48ee-85f2-5631e19d2695 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] iscsi session list stdout=tcp: [59] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:378addbb-8d90-40c5-ae61-c3a063116df9 (non-flash)
Nov 29 03:43:07 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsi_session /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Nov 29 03:43:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:3ae025b2-997b-4f06-8e4b-4d6a37f9e3bc -p 172.16.0.220:3260 --login {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:43:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:3ae025b2-997b-4f06-8e4b-4d6a37f9e3bc -p 172.16.0.220:3260 --login" returned: 0 in 0.050s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:43:07 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[6ac07fc8-122f-4ffb-a159-23856b4ea5c9]: (4, ('Logging in to [iface: default, target: iqn.2010-10.org.openstack:3ae025b2-997b-4f06-8e4b-4d6a37f9e3bc, portal: 172.16.0.220,3260]\nLogin to [iface: default, target: iqn.2010-10.org.openstack:3ae025b2-997b-4f06-8e4b-4d6a37f9e3bc, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:43:07 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-5d33c997-7eef-48ee-85f2-5631e19d2695 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] iscsiadm ('--login',): stdout=Logging in to [iface: default, target: iqn.2010-10.org.openstack:3ae025b2-997b-4f06-8e4b-4d6a37f9e3bc, portal: 172.16.0.220,3260]
Nov 29 03:43:07 user nova-compute[75234]: Login to [iface: default, target: iqn.2010-10.org.openstack:3ae025b2-997b-4f06-8e4b-4d6a37f9e3bc, portal: 172.16.0.220,3260] successful.
Nov 29 03:43:07 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:43:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:3ae025b2-997b-4f06-8e4b-4d6a37f9e3bc -p 172.16.0.220:3260 --op update -n node.startup -v automatic {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:43:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:3ae025b2-997b-4f06-8e4b-4d6a37f9e3bc -p 172.16.0.220:3260 --op update -n node.startup -v automatic" returned: 0 in 0.019s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:43:07 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[ce704673-672a-44dd-8d19-d74fc8b069b3]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:43:07 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-5d33c997-7eef-48ee-85f2-5631e19d2695 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'automatic'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:43:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:43:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.019s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:43:07 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[5fda3f7a-06cf-4bb8-b933-7f4cc674fad1]: (4, ('tcp: [59] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:378addbb-8d90-40c5-ae61-c3a063116df9 (non-flash)\ntcp: [62] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3ae025b2-997b-4f06-8e4b-4d6a37f9e3bc (non-flash)\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:43:07 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-5d33c997-7eef-48ee-85f2-5631e19d2695 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [59] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:378addbb-8d90-40c5-ae61-c3a063116df9 (non-flash)
Nov 29 03:43:07 user nova-compute[75234]: tcp: [62] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3ae025b2-997b-4f06-8e4b-4d6a37f9e3bc (non-flash)
Nov 29 03:43:07 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:43:07 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-5d33c997-7eef-48ee-85f2-5631e19d2695 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] iscsi session list stdout=tcp: [59] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:378addbb-8d90-40c5-ae61-c3a063116df9 (non-flash)
Nov 29 03:43:07 user nova-compute[75234]: tcp: [62] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3ae025b2-997b-4f06-8e4b-4d6a37f9e3bc (non-flash)
Nov 29 03:43:07 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsi_session /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Nov 29 03:43:07 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-5d33c997-7eef-48ee-85f2-5631e19d2695 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Lock "connect_to_iscsi_portal-172.16.0.220:3260-iqn.2010-10.org.openstack:3ae025b2-997b-4f06-8e4b-4d6a37f9e3bc" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector._connect_to_iscsi_portal_unsafe" :: held 0.256s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:87}}
Nov 29 03:43:07 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-5d33c997-7eef-48ee-85f2-5631e19d2695 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Connected to 172.16.0.220:3260 {{(pid=75234) _connect_vol /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:644}}
Nov 29 03:43:07 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-5d33c997-7eef-48ee-85f2-5631e19d2695 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] HCTL ('34', '-', '-', 0) found on session 62 with lun 0 {{(pid=75234) get_hctl /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:770}}
Nov 29 03:43:07 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-5d33c997-7eef-48ee-85f2-5631e19d2695 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Scanning host 34 c: -, t: -, l: 0) {{(pid=75234) scan_iscsi /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:795}}
Nov 29 03:43:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/class/scsi_host/host34/scan {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:43:07 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/class/scsi_host/host34/scan" returned: 0 in 0.016s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:43:07 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[815ec9ad-c1c4-46cd-8042-2a67f3e487c6]: (4, ('- - 0', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:43:07 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-5d33c997-7eef-48ee-85f2-5631e19d2695 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Searching for a device in session 62 and hctl ['34', '*', '*', 0] yield: sdc {{(pid=75234) device_name_by_hctl /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:789}}
Nov 29 03:43:07 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-5d33c997-7eef-48ee-85f2-5631e19d2695 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Connected to sdc using {'target_iqn': 'iqn.2010-10.org.openstack:3ae025b2-997b-4f06-8e4b-4d6a37f9e3bc', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'DbfMxnTa', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False} {{(pid=75234) _connect_vol /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:672}}
Nov 29 03:43:08 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:43:08 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-5d33c997-7eef-48ee-85f2-5631e19d2695 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.connect_volume" :: held 1.289s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:87}}
Nov 29 03:43:08 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-5d33c997-7eef-48ee-85f2-5631e19d2695 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] <== connect_volume: return (1290ms) {'type': 'block', 'scsi_wwn': '23430653263396563', 'path': '/dev/sdc'} {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:203}}
Nov 29 03:43:08 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-5d33c997-7eef-48ee-85f2-5631e19d2695 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: 89c7681b-4a2c-485a-b446-e8ac4df43c2b] Attached iSCSI volume {'type': 'block', 'scsi_wwn': '23430653263396563', 'path': '/dev/sdc'} {{(pid=75234) connect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:65}}
Nov 29 03:43:08 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-5d33c997-7eef-48ee-85f2-5631e19d2695 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-11-29T03:42:58Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-TestVolumeBootPattern-server-517460046',display_name='tempest-TestVolumeBootPattern-server-517460046',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-testvolumebootpattern-server-517460046',id=52,image_ref='',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBKrnAt22NMoBNIFsnCF3Mxk+dIbrL4YaXFADzEahC/3pON2UrJ4NOPzv0Ze4QWq/A4Bm4h/5Mn2lQRP6Y0eXl2r9TPWd6a1CQjrsTRMK7MqwF4n4QkuDxmXBPaNWL5XvpA==',key_name='tempest-TestVolumeBootPattern-1253305043',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='81a265d3c15b4e9d826b35ddbd0ff443',ramdisk_id='',reservation_id='r-w0j3dcuz',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TestVolumeBootPattern-1244492388',owner_user_name='tempest-TestVolumeBootPattern-1244492388-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-11-29T03:43:00Z,user_data=None,user_id='7d1f1f11a9aa4c1db8627413eb963fa4',uuid=89c7681b-4a2c-485a-b446-e8ac4df43c2b,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "18af07bc-c94c-4faa-9008-8be55e548870", "address": "fa:16:3e:43:5a:c5", "network": {"id": "a3f397d3-fd75-4a6e-a815-cfcedcca15fb", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-678570085-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "81a265d3c15b4e9d826b35ddbd0ff443", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap18af07bc-c9", "ovs_interfaceid": "18af07bc-c94c-4faa-9008-8be55e548870", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} virt_type=kvm {{(pid=75234) get_config /opt/stack/nova/nova/virt/libvirt/vif.py:563}}
Nov 29 03:43:08 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-5d33c997-7eef-48ee-85f2-5631e19d2695 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Converting VIF {"id": "18af07bc-c94c-4faa-9008-8be55e548870", "address": "fa:16:3e:43:5a:c5", "network": {"id": "a3f397d3-fd75-4a6e-a815-cfcedcca15fb", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-678570085-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "81a265d3c15b4e9d826b35ddbd0ff443", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap18af07bc-c9", "ovs_interfaceid": "18af07bc-c94c-4faa-9008-8be55e548870", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:43:08 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-5d33c997-7eef-48ee-85f2-5631e19d2695 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:43:5a:c5,bridge_name='br-int',has_traffic_filtering=True,id=18af07bc-c94c-4faa-9008-8be55e548870,network=Network(a3f397d3-fd75-4a6e-a815-cfcedcca15fb),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap18af07bc-c9') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:43:08 user nova-compute[75234]: DEBUG nova.objects.instance [None req-5d33c997-7eef-48ee-85f2-5631e19d2695 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Lazy-loading 'pci_devices' on Instance uuid 89c7681b-4a2c-485a-b446-e8ac4df43c2b {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:43:08 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-5d33c997-7eef-48ee-85f2-5631e19d2695 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: 89c7681b-4a2c-485a-b446-e8ac4df43c2b] End _get_guest_xml xml=<domain type="kvm">
Nov 29 03:43:08 user nova-compute[75234]:   <uuid>89c7681b-4a2c-485a-b446-e8ac4df43c2b</uuid>
Nov 29 03:43:08 user nova-compute[75234]:   <name>instance-00000034</name>
Nov 29 03:43:08 user nova-compute[75234]:   <memory>524288</memory>
Nov 29 03:43:08 user nova-compute[75234]:   <vcpu>1</vcpu>
Nov 29 03:43:08 user nova-compute[75234]:   <metadata>
Nov 29 03:43:08 user nova-compute[75234]:     <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
Nov 29 03:43:08 user nova-compute[75234]:       <nova:package version="0.0.0"/>
Nov 29 03:43:08 user nova-compute[75234]:       <nova:name>tempest-TestVolumeBootPattern-server-517460046</nova:name>
Nov 29 03:43:08 user nova-compute[75234]:       <nova:creationTime>2023-11-29 03:43:07</nova:creationTime>
Nov 29 03:43:08 user nova-compute[75234]:       <nova:flavor name="m1.tiny">
Nov 29 03:43:08 user nova-compute[75234]:         <nova:memory>512</nova:memory>
Nov 29 03:43:08 user nova-compute[75234]:         <nova:disk>1</nova:disk>
Nov 29 03:43:08 user nova-compute[75234]:         <nova:swap>0</nova:swap>
Nov 29 03:43:08 user nova-compute[75234]:         <nova:ephemeral>0</nova:ephemeral>
Nov 29 03:43:08 user nova-compute[75234]:         <nova:vcpus>1</nova:vcpus>
Nov 29 03:43:08 user nova-compute[75234]:       </nova:flavor>
Nov 29 03:43:08 user nova-compute[75234]:       <nova:owner>
Nov 29 03:43:08 user nova-compute[75234]:         <nova:user uuid="7d1f1f11a9aa4c1db8627413eb963fa4">tempest-TestVolumeBootPattern-1244492388-project-member</nova:user>
Nov 29 03:43:08 user nova-compute[75234]:         <nova:project uuid="81a265d3c15b4e9d826b35ddbd0ff443">tempest-TestVolumeBootPattern-1244492388</nova:project>
Nov 29 03:43:08 user nova-compute[75234]:       </nova:owner>
Nov 29 03:43:08 user nova-compute[75234]:       <nova:ports>
Nov 29 03:43:08 user nova-compute[75234]:         <nova:port uuid="18af07bc-c94c-4faa-9008-8be55e548870">
Nov 29 03:43:08 user nova-compute[75234]:           <nova:ip type="fixed" address="10.0.0.9" ipVersion="4"/>
Nov 29 03:43:08 user nova-compute[75234]:         </nova:port>
Nov 29 03:43:08 user nova-compute[75234]:       </nova:ports>
Nov 29 03:43:08 user nova-compute[75234]:     </nova:instance>
Nov 29 03:43:08 user nova-compute[75234]:   </metadata>
Nov 29 03:43:08 user nova-compute[75234]:   <sysinfo type="smbios">
Nov 29 03:43:08 user nova-compute[75234]:     <system>
Nov 29 03:43:08 user nova-compute[75234]:       <entry name="manufacturer">OpenStack Foundation</entry>
Nov 29 03:43:08 user nova-compute[75234]:       <entry name="product">OpenStack Nova</entry>
Nov 29 03:43:08 user nova-compute[75234]:       <entry name="version">0.0.0</entry>
Nov 29 03:43:08 user nova-compute[75234]:       <entry name="serial">89c7681b-4a2c-485a-b446-e8ac4df43c2b</entry>
Nov 29 03:43:08 user nova-compute[75234]:       <entry name="uuid">89c7681b-4a2c-485a-b446-e8ac4df43c2b</entry>
Nov 29 03:43:08 user nova-compute[75234]:       <entry name="family">Virtual Machine</entry>
Nov 29 03:43:08 user nova-compute[75234]:     </system>
Nov 29 03:43:08 user nova-compute[75234]:   </sysinfo>
Nov 29 03:43:08 user nova-compute[75234]:   <os>
Nov 29 03:43:08 user nova-compute[75234]:     <type arch="x86_64" machine="pc">hvm</type>
Nov 29 03:43:08 user nova-compute[75234]:     <boot dev="hd"/>
Nov 29 03:43:08 user nova-compute[75234]:     <smbios mode="sysinfo"/>
Nov 29 03:43:08 user nova-compute[75234]:   </os>
Nov 29 03:43:08 user nova-compute[75234]:   <features>
Nov 29 03:43:08 user nova-compute[75234]:     <acpi/>
Nov 29 03:43:08 user nova-compute[75234]:     <apic/>
Nov 29 03:43:08 user nova-compute[75234]:     <vmcoreinfo/>
Nov 29 03:43:08 user nova-compute[75234]:   </features>
Nov 29 03:43:08 user nova-compute[75234]:   <clock offset="utc">
Nov 29 03:43:08 user nova-compute[75234]:     <timer name="pit" tickpolicy="delay"/>
Nov 29 03:43:08 user nova-compute[75234]:     <timer name="rtc" tickpolicy="catchup"/>
Nov 29 03:43:08 user nova-compute[75234]:     <timer name="hpet" present="no"/>
Nov 29 03:43:08 user nova-compute[75234]:   </clock>
Nov 29 03:43:08 user nova-compute[75234]:   <cpu mode="custom" match="exact">
Nov 29 03:43:08 user nova-compute[75234]:     <model>Nehalem</model>
Nov 29 03:43:08 user nova-compute[75234]:     <topology sockets="1" cores="1" threads="1"/>
Nov 29 03:43:08 user nova-compute[75234]:   </cpu>
Nov 29 03:43:08 user nova-compute[75234]:   <devices>
Nov 29 03:43:08 user nova-compute[75234]:     <disk type="block" device="disk">
Nov 29 03:43:08 user nova-compute[75234]:       <driver name="qemu" type="raw" cache="none" io="native"/>
Nov 29 03:43:08 user nova-compute[75234]:       <source dev="/dev/sdc"/>
Nov 29 03:43:08 user nova-compute[75234]:       <target dev="vda" bus="virtio"/>
Nov 29 03:43:08 user nova-compute[75234]:       <serial>3ae025b2-997b-4f06-8e4b-4d6a37f9e3bc</serial>
Nov 29 03:43:08 user nova-compute[75234]:     </disk>
Nov 29 03:43:08 user nova-compute[75234]:     <interface type="ethernet">
Nov 29 03:43:08 user nova-compute[75234]:       <mac address="fa:16:3e:43:5a:c5"/>
Nov 29 03:43:08 user nova-compute[75234]:       <model type="virtio"/>
Nov 29 03:43:08 user nova-compute[75234]:       <mtu size="1442"/>
Nov 29 03:43:08 user nova-compute[75234]:       <target dev="tap18af07bc-c9"/>
Nov 29 03:43:08 user nova-compute[75234]:     </interface>
Nov 29 03:43:08 user nova-compute[75234]:     <serial type="pty">
Nov 29 03:43:08 user nova-compute[75234]:       <log file="/opt/stack/data/nova/instances/89c7681b-4a2c-485a-b446-e8ac4df43c2b/console.log" append="off"/>
Nov 29 03:43:08 user nova-compute[75234]:     </serial>
Nov 29 03:43:08 user nova-compute[75234]:     <graphics type="vnc" autoport="yes" listen="0.0.0.0"/>
Nov 29 03:43:08 user nova-compute[75234]:     <video>
Nov 29 03:43:08 user nova-compute[75234]:       <model type="virtio"/>
Nov 29 03:43:08 user nova-compute[75234]:     </video>
Nov 29 03:43:08 user nova-compute[75234]:     <rng model="virtio">
Nov 29 03:43:08 user nova-compute[75234]:       <backend model="random">/dev/urandom</backend>
Nov 29 03:43:08 user nova-compute[75234]:     </rng>
Nov 29 03:43:08 user nova-compute[75234]:     <controller type="usb" index="0" model="none"/>
Nov 29 03:43:08 user nova-compute[75234]:     <memballoon model="virtio">
Nov 29 03:43:08 user nova-compute[75234]:       <stats period="10"/>
Nov 29 03:43:08 user nova-compute[75234]:     </memballoon>
Nov 29 03:43:08 user nova-compute[75234]:   </devices>
Nov 29 03:43:08 user nova-compute[75234]: </domain>
Nov 29 03:43:08 user nova-compute[75234]:  {{(pid=75234) _get_guest_xml /opt/stack/nova/nova/virt/libvirt/driver.py:7568}}
Nov 29 03:43:08 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-5d33c997-7eef-48ee-85f2-5631e19d2695 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-11-29T03:42:58Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=None,disable_terminate=False,display_description='tempest-TestVolumeBootPattern-server-517460046',display_name='tempest-TestVolumeBootPattern-server-517460046',ec2_ids=EC2Ids,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-testvolumebootpattern-server-517460046',id=52,image_ref='',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBKrnAt22NMoBNIFsnCF3Mxk+dIbrL4YaXFADzEahC/3pON2UrJ4NOPzv0Ze4QWq/A4Bm4h/5Mn2lQRP6Y0eXl2r9TPWd6a1CQjrsTRMK7MqwF4n4QkuDxmXBPaNWL5XvpA==',key_name='tempest-TestVolumeBootPattern-1253305043',keypairs=KeyPairList,launch_index=0,launched_at=None,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=None,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=PciDeviceList,pci_requests=InstancePCIRequests,power_state=0,progress=0,project_id='81a265d3c15b4e9d826b35ddbd0ff443',ramdisk_id='',reservation_id='r-w0j3dcuz',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='',image_hw_machine_type='pc',image_hw_rng_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',network_allocated='True',owner_project_name='tempest-TestVolumeBootPattern-1244492388',owner_user_name='tempest-TestVolumeBootPattern-1244492388-project-member'},tags=TagList,task_state='spawning',terminated_at=None,trusted_certs=None,updated_at=2023-11-29T03:43:00Z,user_data=None,user_id='7d1f1f11a9aa4c1db8627413eb963fa4',uuid=89c7681b-4a2c-485a-b446-e8ac4df43c2b,vcpu_model=VirtCPUModel,vcpus=1,vm_mode=None,vm_state='building') vif={"id": "18af07bc-c94c-4faa-9008-8be55e548870", "address": "fa:16:3e:43:5a:c5", "network": {"id": "a3f397d3-fd75-4a6e-a815-cfcedcca15fb", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-678570085-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "81a265d3c15b4e9d826b35ddbd0ff443", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap18af07bc-c9", "ovs_interfaceid": "18af07bc-c94c-4faa-9008-8be55e548870", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) plug /opt/stack/nova/nova/virt/libvirt/vif.py:710}}
Nov 29 03:43:08 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-5d33c997-7eef-48ee-85f2-5631e19d2695 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Converting VIF {"id": "18af07bc-c94c-4faa-9008-8be55e548870", "address": "fa:16:3e:43:5a:c5", "network": {"id": "a3f397d3-fd75-4a6e-a815-cfcedcca15fb", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-678570085-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "81a265d3c15b4e9d826b35ddbd0ff443", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap18af07bc-c9", "ovs_interfaceid": "18af07bc-c94c-4faa-9008-8be55e548870", "qbh_params": null, "qbg_params": null, "active": false, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:43:08 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-5d33c997-7eef-48ee-85f2-5631e19d2695 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Converted object VIFOpenVSwitch(active=False,address=fa:16:3e:43:5a:c5,bridge_name='br-int',has_traffic_filtering=True,id=18af07bc-c94c-4faa-9008-8be55e548870,network=Network(a3f397d3-fd75-4a6e-a815-cfcedcca15fb),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap18af07bc-c9') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:43:08 user nova-compute[75234]: DEBUG os_vif [None req-5d33c997-7eef-48ee-85f2-5631e19d2695 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Plugging vif VIFOpenVSwitch(active=False,address=fa:16:3e:43:5a:c5,bridge_name='br-int',has_traffic_filtering=True,id=18af07bc-c94c-4faa-9008-8be55e548870,network=Network(a3f397d3-fd75-4a6e-a815-cfcedcca15fb),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap18af07bc-c9') {{(pid=75234) plug /opt/stack/data/venv/lib/python3.10/site-packages/os_vif/__init__.py:76}}
Nov 29 03:43:08 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:43:08 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddBridgeCommand(_result=None, name=br-int, may_exist=True, datapath_type=system) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:43:08 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Transaction caused no change {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:129}}
Nov 29 03:43:08 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:43:08 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbCreateCommand(_result=None, table=QoS, columns={'type': 'linux-noop', 'external_ids': {'id': 'd292d6f3-e280-551e-b1ad-388d20f2c4c5', '_type': 'linux-noop'}}, row=False) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:43:08 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:43:08 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:43:08 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:43:08 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): AddPortCommand(_result=None, bridge=br-int, port=tap18af07bc-c9, may_exist=True, interface_attrs={}) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:43:08 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(_result=None, table=Port, record=tap18af07bc-c9, col_values=(('qos', UUID('fb37aee7-a068-4a3a-b553-c2612a6f594d')),), if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:43:08 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DbSetCommand(_result=None, table=Interface, record=tap18af07bc-c9, col_values=(('external_ids', {'iface-id': '18af07bc-c94c-4faa-9008-8be55e548870', 'iface-status': 'active', 'attached-mac': 'fa:16:3e:43:5a:c5', 'vm-uuid': '89c7681b-4a2c-485a-b446-e8ac4df43c2b'}),), if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:43:08 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:43:08 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:43:08 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:43:08 user nova-compute[75234]: INFO os_vif [None req-5d33c997-7eef-48ee-85f2-5631e19d2695 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:43:5a:c5,bridge_name='br-int',has_traffic_filtering=True,id=18af07bc-c94c-4faa-9008-8be55e548870,network=Network(a3f397d3-fd75-4a6e-a815-cfcedcca15fb),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap18af07bc-c9')
Nov 29 03:43:08 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-5d33c997-7eef-48ee-85f2-5631e19d2695 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] No BDM found with device name vda, not building metadata. {{(pid=75234) _build_disk_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12089}}
Nov 29 03:43:08 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-5d33c997-7eef-48ee-85f2-5631e19d2695 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] No VIF found with MAC fa:16:3e:43:5a:c5, not building metadata {{(pid=75234) _build_interface_metadata /opt/stack/nova/nova/virt/libvirt/driver.py:12065}}
Nov 29 03:43:10 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:43:10 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:43:10 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:43:10 user nova-compute[75234]: DEBUG nova.compute.manager [req-03367cb4-3255-4479-9ee9-9326fa265ba3 req-9cc10817-22aa-45a9-99e6-7930a0409da4 service nova] [instance: 89c7681b-4a2c-485a-b446-e8ac4df43c2b] Received event network-vif-plugged-18af07bc-c94c-4faa-9008-8be55e548870 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:43:10 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-03367cb4-3255-4479-9ee9-9326fa265ba3 req-9cc10817-22aa-45a9-99e6-7930a0409da4 service nova] Acquiring lock "89c7681b-4a2c-485a-b446-e8ac4df43c2b-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:43:10 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-03367cb4-3255-4479-9ee9-9326fa265ba3 req-9cc10817-22aa-45a9-99e6-7930a0409da4 service nova] Lock "89c7681b-4a2c-485a-b446-e8ac4df43c2b-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:43:10 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-03367cb4-3255-4479-9ee9-9326fa265ba3 req-9cc10817-22aa-45a9-99e6-7930a0409da4 service nova] Lock "89c7681b-4a2c-485a-b446-e8ac4df43c2b-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:43:10 user nova-compute[75234]: DEBUG nova.compute.manager [req-03367cb4-3255-4479-9ee9-9326fa265ba3 req-9cc10817-22aa-45a9-99e6-7930a0409da4 service nova] [instance: 89c7681b-4a2c-485a-b446-e8ac4df43c2b] No waiting events found dispatching network-vif-plugged-18af07bc-c94c-4faa-9008-8be55e548870 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:43:10 user nova-compute[75234]: WARNING nova.compute.manager [req-03367cb4-3255-4479-9ee9-9326fa265ba3 req-9cc10817-22aa-45a9-99e6-7930a0409da4 service nova] [instance: 89c7681b-4a2c-485a-b446-e8ac4df43c2b] Received unexpected event network-vif-plugged-18af07bc-c94c-4faa-9008-8be55e548870 for instance with vm_state building and task_state spawning.
Nov 29 03:43:10 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:43:10 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:43:12 user nova-compute[75234]: DEBUG nova.compute.manager [None req-5d33c997-7eef-48ee-85f2-5631e19d2695 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: 89c7681b-4a2c-485a-b446-e8ac4df43c2b] Instance event wait completed in 0 seconds for  {{(pid=75234) wait_for_instance_event /opt/stack/nova/nova/compute/manager.py:577}}
Nov 29 03:43:12 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-5d33c997-7eef-48ee-85f2-5631e19d2695 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: 89c7681b-4a2c-485a-b446-e8ac4df43c2b] Guest created on hypervisor {{(pid=75234) spawn /opt/stack/nova/nova/virt/libvirt/driver.py:4421}}
Nov 29 03:43:12 user nova-compute[75234]: INFO nova.virt.libvirt.driver [-] [instance: 89c7681b-4a2c-485a-b446-e8ac4df43c2b] Instance spawned successfully.
Nov 29 03:43:12 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-5d33c997-7eef-48ee-85f2-5631e19d2695 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: 89c7681b-4a2c-485a-b446-e8ac4df43c2b] Attempting to register defaults for the following image properties: ['hw_cdrom_bus', 'hw_disk_bus', 'hw_input_bus', 'hw_pointer_model', 'hw_video_model', 'hw_vif_model'] {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:904}}
Nov 29 03:43:12 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-5d33c997-7eef-48ee-85f2-5631e19d2695 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: 89c7681b-4a2c-485a-b446-e8ac4df43c2b] Found default for hw_cdrom_bus of ide {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:43:12 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-5d33c997-7eef-48ee-85f2-5631e19d2695 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: 89c7681b-4a2c-485a-b446-e8ac4df43c2b] Found default for hw_disk_bus of virtio {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:43:12 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-5d33c997-7eef-48ee-85f2-5631e19d2695 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: 89c7681b-4a2c-485a-b446-e8ac4df43c2b] Found default for hw_input_bus of None {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:43:12 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-5d33c997-7eef-48ee-85f2-5631e19d2695 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: 89c7681b-4a2c-485a-b446-e8ac4df43c2b] Found default for hw_pointer_model of None {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:43:12 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-5d33c997-7eef-48ee-85f2-5631e19d2695 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: 89c7681b-4a2c-485a-b446-e8ac4df43c2b] Found default for hw_video_model of virtio {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:43:12 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-5d33c997-7eef-48ee-85f2-5631e19d2695 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: 89c7681b-4a2c-485a-b446-e8ac4df43c2b] Found default for hw_vif_model of virtio {{(pid=75234) _register_undefined_instance_details /opt/stack/nova/nova/virt/libvirt/driver.py:933}}
Nov 29 03:43:12 user nova-compute[75234]: INFO nova.compute.manager [None req-5d33c997-7eef-48ee-85f2-5631e19d2695 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: 89c7681b-4a2c-485a-b446-e8ac4df43c2b] Took 4.92 seconds to spawn the instance on the hypervisor.
Nov 29 03:43:12 user nova-compute[75234]: DEBUG nova.compute.manager [None req-5d33c997-7eef-48ee-85f2-5631e19d2695 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: 89c7681b-4a2c-485a-b446-e8ac4df43c2b] Checking state {{(pid=75234) _get_power_state /opt/stack/nova/nova/compute/manager.py:1782}}
Nov 29 03:43:12 user nova-compute[75234]: DEBUG nova.compute.manager [req-05e89c5e-2785-4ea0-942d-77d1d6254215 req-4b731c50-4982-4940-aba9-631730b5acb3 service nova] [instance: 89c7681b-4a2c-485a-b446-e8ac4df43c2b] Received event network-vif-plugged-18af07bc-c94c-4faa-9008-8be55e548870 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:43:12 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-05e89c5e-2785-4ea0-942d-77d1d6254215 req-4b731c50-4982-4940-aba9-631730b5acb3 service nova] Acquiring lock "89c7681b-4a2c-485a-b446-e8ac4df43c2b-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:43:12 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-05e89c5e-2785-4ea0-942d-77d1d6254215 req-4b731c50-4982-4940-aba9-631730b5acb3 service nova] Lock "89c7681b-4a2c-485a-b446-e8ac4df43c2b-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:43:12 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-05e89c5e-2785-4ea0-942d-77d1d6254215 req-4b731c50-4982-4940-aba9-631730b5acb3 service nova] Lock "89c7681b-4a2c-485a-b446-e8ac4df43c2b-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:43:12 user nova-compute[75234]: DEBUG nova.compute.manager [req-05e89c5e-2785-4ea0-942d-77d1d6254215 req-4b731c50-4982-4940-aba9-631730b5acb3 service nova] [instance: 89c7681b-4a2c-485a-b446-e8ac4df43c2b] No waiting events found dispatching network-vif-plugged-18af07bc-c94c-4faa-9008-8be55e548870 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:43:12 user nova-compute[75234]: WARNING nova.compute.manager [req-05e89c5e-2785-4ea0-942d-77d1d6254215 req-4b731c50-4982-4940-aba9-631730b5acb3 service nova] [instance: 89c7681b-4a2c-485a-b446-e8ac4df43c2b] Received unexpected event network-vif-plugged-18af07bc-c94c-4faa-9008-8be55e548870 for instance with vm_state building and task_state spawning.
Nov 29 03:43:12 user nova-compute[75234]: INFO nova.compute.manager [None req-5d33c997-7eef-48ee-85f2-5631e19d2695 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: 89c7681b-4a2c-485a-b446-e8ac4df43c2b] Took 12.81 seconds to build instance.
Nov 29 03:43:12 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-5d33c997-7eef-48ee-85f2-5631e19d2695 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Lock "89c7681b-4a2c-485a-b446-e8ac4df43c2b" "released" by "nova.compute.manager.ComputeManager.build_and_run_instance.<locals>._locked_do_build_and_run_instance" :: held 12.897s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:43:13 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:43:13 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:43:18 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:43:18 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:43:19 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:43:21 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:43:22 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:43:22 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=75234) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10541}}
Nov 29 03:43:23 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:43:23 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:43:24 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager.update_available_resource {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:43:24 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:43:24 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:43:24 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:43:24 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Auditing locally available compute resources for user (node: user) {{(pid=75234) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:907}}
Nov 29 03:43:24 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] skipping disk /dev/sdb (vda) as it is a volume {{(pid=75234) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11213}}
Nov 29 03:43:24 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] skipping disk /dev/sdc (vda) as it is a volume {{(pid=75234) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11213}}
Nov 29 03:43:24 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:43:24 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:43:24 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Hypervisor/Node resource view: name=user free_ram=8813MB free_disk=25.152629852294922GB free_vcpus=10 pci_devices=[{"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}] {{(pid=75234) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1106}}
Nov 29 03:43:24 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:43:24 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:43:24 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance 95afc929-9f13-4f9a-9ac9-eae0b5e15a4d actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 512}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:43:24 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance 89c7681b-4a2c-485a-b446-e8ac4df43c2b actively managed on this compute host and has allocations in placement: {'resources': {'VCPU': 1, 'MEMORY_MB': 512}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:43:24 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Total usable vcpus: 12, total allocated vcpus: 2 {{(pid=75234) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1129}}
Nov 29 03:43:24 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Final resource view: name=user phys_ram=16011MB used_ram=1536MB phys_disk=40GB used_disk=0GB total_vcpus=12 used_vcpus=2 pci_stats=[] {{(pid=75234) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1138}}
Nov 29 03:43:25 user nova-compute[75234]: DEBUG nova.scheduler.client.report [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Refreshing inventories for resource provider 4d291220-7c37-4839-a0e3-b23c66502b74 {{(pid=75234) _refresh_associations /opt/stack/nova/nova/scheduler/client/report.py:804}}
Nov 29 03:43:25 user nova-compute[75234]: DEBUG nova.scheduler.client.report [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Updating ProviderTree inventory for provider 4d291220-7c37-4839-a0e3-b23c66502b74 from _refresh_and_get_inventory using data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=75234) _refresh_and_get_inventory /opt/stack/nova/nova/scheduler/client/report.py:768}}
Nov 29 03:43:25 user nova-compute[75234]: DEBUG nova.compute.provider_tree [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Updating inventory in ProviderTree for provider 4d291220-7c37-4839-a0e3-b23c66502b74 with inventory: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=75234) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:176}}
Nov 29 03:43:25 user nova-compute[75234]: DEBUG nova.scheduler.client.report [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Refreshing aggregate associations for resource provider 4d291220-7c37-4839-a0e3-b23c66502b74, aggregates: None {{(pid=75234) _refresh_associations /opt/stack/nova/nova/scheduler/client/report.py:813}}
Nov 29 03:43:25 user nova-compute[75234]: DEBUG nova.scheduler.client.report [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Refreshing trait associations for resource provider 4d291220-7c37-4839-a0e3-b23c66502b74, traits: COMPUTE_IMAGE_TYPE_AMI,COMPUTE_GRAPHICS_MODEL_CIRRUS,COMPUTE_NET_VIF_MODEL_VMXNET3,COMPUTE_STORAGE_BUS_VIRTIO,COMPUTE_GRAPHICS_MODEL_NONE,COMPUTE_IMAGE_TYPE_ISO,COMPUTE_NET_VIF_MODEL_RTL8139,HW_CPU_X86_SSE,COMPUTE_IMAGE_TYPE_QCOW2,COMPUTE_SOCKET_PCI_NUMA_AFFINITY,COMPUTE_GRAPHICS_MODEL_BOCHS,COMPUTE_VOLUME_ATTACH_WITH_TAG,COMPUTE_ACCELERATORS,HW_CPU_X86_SSE2,COMPUTE_STORAGE_BUS_SCSI,COMPUTE_GRAPHICS_MODEL_VGA,COMPUTE_NET_ATTACH_INTERFACE_WITH_TAG,COMPUTE_SECURITY_UEFI_SECURE_BOOT,HW_CPU_X86_SSE41,COMPUTE_NET_VIF_MODEL_E1000E,COMPUTE_TRUSTED_CERTS,COMPUTE_STORAGE_BUS_FDC,COMPUTE_VIOMMU_MODEL_INTEL,COMPUTE_GRAPHICS_MODEL_VIRTIO,COMPUTE_VIOMMU_MODEL_AUTO,COMPUTE_IMAGE_TYPE_AKI,COMPUTE_NET_VIF_MODEL_PCNET,COMPUTE_GRAPHICS_MODEL_QXL,COMPUTE_NET_VIF_MODEL_VIRTIO,COMPUTE_GRAPHICS_MODEL_VMVGA,COMPUTE_VOLUME_MULTI_ATTACH,COMPUTE_STORAGE_BUS_SATA,HW_CPU_X86_MMX,COMPUTE_NET_VIF_MODEL_E1000,COMPUTE_IMAGE_TYPE_ARI,COMPUTE_STORAGE_BUS_IDE,COMPUTE_NODE,HW_CPU_X86_SSSE3,COMPUTE_RESCUE_BFV,COMPUTE_NET_ATTACH_INTERFACE,COMPUTE_STORAGE_BUS_USB,COMPUTE_NET_VIF_MODEL_SPAPR_VLAN,COMPUTE_IMAGE_TYPE_RAW,COMPUTE_NET_VIF_MODEL_NE2K_PCI,COMPUTE_VOLUME_EXTEND,COMPUTE_DEVICE_TAGGING,HW_CPU_X86_SSE42 {{(pid=75234) _refresh_associations /opt/stack/nova/nova/scheduler/client/report.py:825}}
Nov 29 03:43:25 user nova-compute[75234]: DEBUG nova.compute.provider_tree [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Inventory has not changed in ProviderTree for provider: 4d291220-7c37-4839-a0e3-b23c66502b74 {{(pid=75234) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Nov 29 03:43:25 user nova-compute[75234]: DEBUG nova.scheduler.client.report [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Inventory has not changed for provider 4d291220-7c37-4839-a0e3-b23c66502b74 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=75234) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Nov 29 03:43:25 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Compute_service record updated for user:user {{(pid=75234) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:1067}}
Nov 29 03:43:25 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.357s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:43:28 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:43:28 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:43:31 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:43:31 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:43:31 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Starting heal instance info cache {{(pid=75234) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9922}}
Nov 29 03:43:31 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Rebuilding the list of instances to heal {{(pid=75234) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9926}}
Nov 29 03:43:31 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Acquiring lock "refresh_cache-95afc929-9f13-4f9a-9ac9-eae0b5e15a4d" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:43:31 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Acquired lock "refresh_cache-95afc929-9f13-4f9a-9ac9-eae0b5e15a4d" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:43:31 user nova-compute[75234]: DEBUG nova.network.neutron [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] [instance: 95afc929-9f13-4f9a-9ac9-eae0b5e15a4d] Forcefully refreshing network info cache for instance {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2003}}
Nov 29 03:43:31 user nova-compute[75234]: DEBUG nova.objects.instance [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lazy-loading 'info_cache' on Instance uuid 95afc929-9f13-4f9a-9ac9-eae0b5e15a4d {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:43:32 user nova-compute[75234]: DEBUG nova.network.neutron [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] [instance: 95afc929-9f13-4f9a-9ac9-eae0b5e15a4d] Updating instance_info_cache with network_info: [{"id": "ca1227f9-b2f1-42c5-a92a-c70cb94e52c1", "address": "fa:16:3e:02:94:55", "network": {"id": "a3f397d3-fd75-4a6e-a815-cfcedcca15fb", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-678570085-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.17", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "81a265d3c15b4e9d826b35ddbd0ff443", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapca1227f9-b2", "ovs_interfaceid": "ca1227f9-b2f1-42c5-a92a-c70cb94e52c1", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:43:32 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Releasing lock "refresh_cache-95afc929-9f13-4f9a-9ac9-eae0b5e15a4d" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:43:32 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] [instance: 95afc929-9f13-4f9a-9ac9-eae0b5e15a4d] Updated the network info_cache for instance {{(pid=75234) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9993}}
Nov 29 03:43:32 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:43:32 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:43:33 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:43:33 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:43:38 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:43:38 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:43:43 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:43:43 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:43:48 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:43:53 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:43:58 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:44:03 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:44:08 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:44:13 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:44:18 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:44:18 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:44:19 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:44:22 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:44:22 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=75234) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10541}}
Nov 29 03:44:23 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:44:23 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:44:25 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager.update_available_resource {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:44:25 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:44:25 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:44:25 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:44:25 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Auditing locally available compute resources for user (node: user) {{(pid=75234) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:907}}
Nov 29 03:44:25 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] skipping disk /dev/sdb (vda) as it is a volume {{(pid=75234) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11213}}
Nov 29 03:44:25 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] skipping disk /dev/sdc (vda) as it is a volume {{(pid=75234) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11213}}
Nov 29 03:44:25 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:44:25 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:44:25 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Hypervisor/Node resource view: name=user free_ram=8938MB free_disk=25.151981353759766GB free_vcpus=10 pci_devices=[{"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}] {{(pid=75234) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1106}}
Nov 29 03:44:25 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:44:25 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:44:26 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance 95afc929-9f13-4f9a-9ac9-eae0b5e15a4d actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:44:26 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance 89c7681b-4a2c-485a-b446-e8ac4df43c2b actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:44:26 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Total usable vcpus: 12, total allocated vcpus: 2 {{(pid=75234) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1129}}
Nov 29 03:44:26 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Final resource view: name=user phys_ram=16011MB used_ram=1536MB phys_disk=40GB used_disk=0GB total_vcpus=12 used_vcpus=2 pci_stats=[] {{(pid=75234) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1138}}
Nov 29 03:44:26 user nova-compute[75234]: DEBUG nova.compute.provider_tree [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Inventory has not changed in ProviderTree for provider: 4d291220-7c37-4839-a0e3-b23c66502b74 {{(pid=75234) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Nov 29 03:44:26 user nova-compute[75234]: DEBUG nova.scheduler.client.report [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Inventory has not changed for provider 4d291220-7c37-4839-a0e3-b23c66502b74 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=75234) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Nov 29 03:44:26 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Compute_service record updated for user:user {{(pid=75234) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:1067}}
Nov 29 03:44:26 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.212s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:44:28 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:44:33 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:44:33 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Starting heal instance info cache {{(pid=75234) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9922}}
Nov 29 03:44:33 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Acquiring lock "refresh_cache-89c7681b-4a2c-485a-b446-e8ac4df43c2b" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:44:33 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Acquired lock "refresh_cache-89c7681b-4a2c-485a-b446-e8ac4df43c2b" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:44:33 user nova-compute[75234]: DEBUG nova.network.neutron [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] [instance: 89c7681b-4a2c-485a-b446-e8ac4df43c2b] Forcefully refreshing network info cache for instance {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2003}}
Nov 29 03:44:33 user nova-compute[75234]: DEBUG nova.network.neutron [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] [instance: 89c7681b-4a2c-485a-b446-e8ac4df43c2b] Updating instance_info_cache with network_info: [{"id": "18af07bc-c94c-4faa-9008-8be55e548870", "address": "fa:16:3e:43:5a:c5", "network": {"id": "a3f397d3-fd75-4a6e-a815-cfcedcca15fb", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-678570085-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "81a265d3c15b4e9d826b35ddbd0ff443", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap18af07bc-c9", "ovs_interfaceid": "18af07bc-c94c-4faa-9008-8be55e548870", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:44:33 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Releasing lock "refresh_cache-89c7681b-4a2c-485a-b446-e8ac4df43c2b" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:44:33 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] [instance: 89c7681b-4a2c-485a-b446-e8ac4df43c2b] Updated the network info_cache for instance {{(pid=75234) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9993}}
Nov 29 03:44:33 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:44:33 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:44:33 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:44:34 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:44:38 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:44:39 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._sync_scheduler_instance_info {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:44:43 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:44:48 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:44:50 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:44:51 user nova-compute[75234]: DEBUG nova.compute.manager [req-8611eaed-16a3-4733-ade1-23058af37617 req-83081d72-8abc-404e-94bf-3aff78c39d71 service nova] [instance: 89c7681b-4a2c-485a-b446-e8ac4df43c2b] Received event network-changed-18af07bc-c94c-4faa-9008-8be55e548870 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:44:51 user nova-compute[75234]: DEBUG nova.compute.manager [req-8611eaed-16a3-4733-ade1-23058af37617 req-83081d72-8abc-404e-94bf-3aff78c39d71 service nova] [instance: 89c7681b-4a2c-485a-b446-e8ac4df43c2b] Refreshing instance network info cache due to event network-changed-18af07bc-c94c-4faa-9008-8be55e548870. {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11117}}
Nov 29 03:44:51 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-8611eaed-16a3-4733-ade1-23058af37617 req-83081d72-8abc-404e-94bf-3aff78c39d71 service nova] Acquiring lock "refresh_cache-89c7681b-4a2c-485a-b446-e8ac4df43c2b" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:44:51 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-8611eaed-16a3-4733-ade1-23058af37617 req-83081d72-8abc-404e-94bf-3aff78c39d71 service nova] Acquired lock "refresh_cache-89c7681b-4a2c-485a-b446-e8ac4df43c2b" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:44:51 user nova-compute[75234]: DEBUG nova.network.neutron [req-8611eaed-16a3-4733-ade1-23058af37617 req-83081d72-8abc-404e-94bf-3aff78c39d71 service nova] [instance: 89c7681b-4a2c-485a-b446-e8ac4df43c2b] Refreshing network info cache for port 18af07bc-c94c-4faa-9008-8be55e548870 {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2006}}
Nov 29 03:44:51 user nova-compute[75234]: DEBUG nova.network.neutron [req-8611eaed-16a3-4733-ade1-23058af37617 req-83081d72-8abc-404e-94bf-3aff78c39d71 service nova] [instance: 89c7681b-4a2c-485a-b446-e8ac4df43c2b] Updated VIF entry in instance network info cache for port 18af07bc-c94c-4faa-9008-8be55e548870. {{(pid=75234) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3481}}
Nov 29 03:44:51 user nova-compute[75234]: DEBUG nova.network.neutron [req-8611eaed-16a3-4733-ade1-23058af37617 req-83081d72-8abc-404e-94bf-3aff78c39d71 service nova] [instance: 89c7681b-4a2c-485a-b446-e8ac4df43c2b] Updating instance_info_cache with network_info: [{"id": "18af07bc-c94c-4faa-9008-8be55e548870", "address": "fa:16:3e:43:5a:c5", "network": {"id": "a3f397d3-fd75-4a6e-a815-cfcedcca15fb", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-678570085-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.27", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "81a265d3c15b4e9d826b35ddbd0ff443", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap18af07bc-c9", "ovs_interfaceid": "18af07bc-c94c-4faa-9008-8be55e548870", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:44:51 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-8611eaed-16a3-4733-ade1-23058af37617 req-83081d72-8abc-404e-94bf-3aff78c39d71 service nova] Releasing lock "refresh_cache-89c7681b-4a2c-485a-b446-e8ac4df43c2b" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:44:52 user nova-compute[75234]: DEBUG nova.compute.manager [req-cdb6066d-a6fb-482a-963f-bd86ad50de56 req-a08e0fd9-012b-4109-ae35-14d74d71c8fa service nova] [instance: 89c7681b-4a2c-485a-b446-e8ac4df43c2b] Received event network-changed-18af07bc-c94c-4faa-9008-8be55e548870 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:44:52 user nova-compute[75234]: DEBUG nova.compute.manager [req-cdb6066d-a6fb-482a-963f-bd86ad50de56 req-a08e0fd9-012b-4109-ae35-14d74d71c8fa service nova] [instance: 89c7681b-4a2c-485a-b446-e8ac4df43c2b] Refreshing instance network info cache due to event network-changed-18af07bc-c94c-4faa-9008-8be55e548870. {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11117}}
Nov 29 03:44:52 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-cdb6066d-a6fb-482a-963f-bd86ad50de56 req-a08e0fd9-012b-4109-ae35-14d74d71c8fa service nova] Acquiring lock "refresh_cache-89c7681b-4a2c-485a-b446-e8ac4df43c2b" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:44:52 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-cdb6066d-a6fb-482a-963f-bd86ad50de56 req-a08e0fd9-012b-4109-ae35-14d74d71c8fa service nova] Acquired lock "refresh_cache-89c7681b-4a2c-485a-b446-e8ac4df43c2b" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:44:52 user nova-compute[75234]: DEBUG nova.network.neutron [req-cdb6066d-a6fb-482a-963f-bd86ad50de56 req-a08e0fd9-012b-4109-ae35-14d74d71c8fa service nova] [instance: 89c7681b-4a2c-485a-b446-e8ac4df43c2b] Refreshing network info cache for port 18af07bc-c94c-4faa-9008-8be55e548870 {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2006}}
Nov 29 03:44:52 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-9f314b2a-f880-46de-853b-933ddde8f52f tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Acquiring lock "89c7681b-4a2c-485a-b446-e8ac4df43c2b" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:44:52 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-9f314b2a-f880-46de-853b-933ddde8f52f tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Lock "89c7681b-4a2c-485a-b446-e8ac4df43c2b" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:44:52 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-9f314b2a-f880-46de-853b-933ddde8f52f tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Acquiring lock "89c7681b-4a2c-485a-b446-e8ac4df43c2b-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:44:52 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-9f314b2a-f880-46de-853b-933ddde8f52f tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Lock "89c7681b-4a2c-485a-b446-e8ac4df43c2b-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:44:52 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-9f314b2a-f880-46de-853b-933ddde8f52f tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Lock "89c7681b-4a2c-485a-b446-e8ac4df43c2b-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:44:52 user nova-compute[75234]: INFO nova.compute.manager [None req-9f314b2a-f880-46de-853b-933ddde8f52f tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: 89c7681b-4a2c-485a-b446-e8ac4df43c2b] Terminating instance
Nov 29 03:44:52 user nova-compute[75234]: DEBUG nova.compute.manager [None req-9f314b2a-f880-46de-853b-933ddde8f52f tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: 89c7681b-4a2c-485a-b446-e8ac4df43c2b] Start destroying the instance on the hypervisor. {{(pid=75234) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3140}}
Nov 29 03:44:52 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:44:52 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:44:52 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:44:52 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:44:52 user nova-compute[75234]: DEBUG nova.network.neutron [req-cdb6066d-a6fb-482a-963f-bd86ad50de56 req-a08e0fd9-012b-4109-ae35-14d74d71c8fa service nova] [instance: 89c7681b-4a2c-485a-b446-e8ac4df43c2b] Updated VIF entry in instance network info cache for port 18af07bc-c94c-4faa-9008-8be55e548870. {{(pid=75234) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3481}}
Nov 29 03:44:52 user nova-compute[75234]: DEBUG nova.network.neutron [req-cdb6066d-a6fb-482a-963f-bd86ad50de56 req-a08e0fd9-012b-4109-ae35-14d74d71c8fa service nova] [instance: 89c7681b-4a2c-485a-b446-e8ac4df43c2b] Updating instance_info_cache with network_info: [{"id": "18af07bc-c94c-4faa-9008-8be55e548870", "address": "fa:16:3e:43:5a:c5", "network": {"id": "a3f397d3-fd75-4a6e-a815-cfcedcca15fb", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-678570085-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "81a265d3c15b4e9d826b35ddbd0ff443", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap18af07bc-c9", "ovs_interfaceid": "18af07bc-c94c-4faa-9008-8be55e548870", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:44:52 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-cdb6066d-a6fb-482a-963f-bd86ad50de56 req-a08e0fd9-012b-4109-ae35-14d74d71c8fa service nova] Releasing lock "refresh_cache-89c7681b-4a2c-485a-b446-e8ac4df43c2b" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:44:52 user nova-compute[75234]: DEBUG nova.compute.manager [req-8195cd7b-8f29-44de-95c1-01ce19730168 req-33125431-0a93-43c7-805d-d9fc95e95f9a service nova] [instance: 89c7681b-4a2c-485a-b446-e8ac4df43c2b] Received event network-vif-unplugged-18af07bc-c94c-4faa-9008-8be55e548870 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:44:52 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-8195cd7b-8f29-44de-95c1-01ce19730168 req-33125431-0a93-43c7-805d-d9fc95e95f9a service nova] Acquiring lock "89c7681b-4a2c-485a-b446-e8ac4df43c2b-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:44:52 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-8195cd7b-8f29-44de-95c1-01ce19730168 req-33125431-0a93-43c7-805d-d9fc95e95f9a service nova] Lock "89c7681b-4a2c-485a-b446-e8ac4df43c2b-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:44:52 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-8195cd7b-8f29-44de-95c1-01ce19730168 req-33125431-0a93-43c7-805d-d9fc95e95f9a service nova] Lock "89c7681b-4a2c-485a-b446-e8ac4df43c2b-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:44:52 user nova-compute[75234]: DEBUG nova.compute.manager [req-8195cd7b-8f29-44de-95c1-01ce19730168 req-33125431-0a93-43c7-805d-d9fc95e95f9a service nova] [instance: 89c7681b-4a2c-485a-b446-e8ac4df43c2b] No waiting events found dispatching network-vif-unplugged-18af07bc-c94c-4faa-9008-8be55e548870 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:44:52 user nova-compute[75234]: DEBUG nova.compute.manager [req-8195cd7b-8f29-44de-95c1-01ce19730168 req-33125431-0a93-43c7-805d-d9fc95e95f9a service nova] [instance: 89c7681b-4a2c-485a-b446-e8ac4df43c2b] Received event network-vif-unplugged-18af07bc-c94c-4faa-9008-8be55e548870 for instance with task_state deleting. {{(pid=75234) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10890}}
Nov 29 03:44:53 user nova-compute[75234]: INFO nova.virt.libvirt.driver [-] [instance: 89c7681b-4a2c-485a-b446-e8ac4df43c2b] Instance destroyed successfully.
Nov 29 03:44:53 user nova-compute[75234]: DEBUG nova.objects.instance [None req-9f314b2a-f880-46de-853b-933ddde8f52f tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Lazy-loading 'resources' on Instance uuid 89c7681b-4a2c-485a-b446-e8ac4df43c2b {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:44:53 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-9f314b2a-f880-46de-853b-933ddde8f52f tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-11-29T03:42:58Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-TestVolumeBootPattern-server-517460046',display_name='tempest-TestVolumeBootPattern-server-517460046',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-testvolumebootpattern-server-517460046',id=52,image_ref='',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBKrnAt22NMoBNIFsnCF3Mxk+dIbrL4YaXFADzEahC/3pON2UrJ4NOPzv0Ze4QWq/A4Bm4h/5Mn2lQRP6Y0eXl2r9TPWd6a1CQjrsTRMK7MqwF4n4QkuDxmXBPaNWL5XvpA==',key_name='tempest-TestVolumeBootPattern-1253305043',keypairs=<?>,launch_index=0,launched_at=2023-11-29T03:43:12Z,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='81a265d3c15b4e9d826b35ddbd0ff443',ramdisk_id='',reservation_id='r-w0j3dcuz',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-TestVolumeBootPattern-1244492388',owner_user_name='tempest-TestVolumeBootPattern-1244492388-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-11-29T03:43:12Z,user_data=None,user_id='7d1f1f11a9aa4c1db8627413eb963fa4',uuid=89c7681b-4a2c-485a-b446-e8ac4df43c2b,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "18af07bc-c94c-4faa-9008-8be55e548870", "address": "fa:16:3e:43:5a:c5", "network": {"id": "a3f397d3-fd75-4a6e-a815-cfcedcca15fb", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-678570085-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.27", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "81a265d3c15b4e9d826b35ddbd0ff443", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap18af07bc-c9", "ovs_interfaceid": "18af07bc-c94c-4faa-9008-8be55e548870", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Nov 29 03:44:53 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-9f314b2a-f880-46de-853b-933ddde8f52f tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Converting VIF {"id": "18af07bc-c94c-4faa-9008-8be55e548870", "address": "fa:16:3e:43:5a:c5", "network": {"id": "a3f397d3-fd75-4a6e-a815-cfcedcca15fb", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-678570085-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.9", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.27", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "81a265d3c15b4e9d826b35ddbd0ff443", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tap18af07bc-c9", "ovs_interfaceid": "18af07bc-c94c-4faa-9008-8be55e548870", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:44:53 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-9f314b2a-f880-46de-853b-933ddde8f52f tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:43:5a:c5,bridge_name='br-int',has_traffic_filtering=True,id=18af07bc-c94c-4faa-9008-8be55e548870,network=Network(a3f397d3-fd75-4a6e-a815-cfcedcca15fb),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap18af07bc-c9') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:44:53 user nova-compute[75234]: DEBUG os_vif [None req-9f314b2a-f880-46de-853b-933ddde8f52f tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Unplugging vif VIFOpenVSwitch(active=True,address=fa:16:3e:43:5a:c5,bridge_name='br-int',has_traffic_filtering=True,id=18af07bc-c94c-4faa-9008-8be55e548870,network=Network(a3f397d3-fd75-4a6e-a815-cfcedcca15fb),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap18af07bc-c9') {{(pid=75234) unplug /opt/stack/data/venv/lib/python3.10/site-packages/os_vif/__init__.py:109}}
Nov 29 03:44:53 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:44:53 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tap18af07bc-c9, bridge=br-int, if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:44:53 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:44:53 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:44:53 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbDestroyCommand(_result=None, table=QoS, record=fb37aee7-a068-4a3a-b553-c2612a6f594d) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:44:53 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:44:53 user nova-compute[75234]: INFO os_vif [None req-9f314b2a-f880-46de-853b-933ddde8f52f tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Successfully unplugged vif VIFOpenVSwitch(active=True,address=fa:16:3e:43:5a:c5,bridge_name='br-int',has_traffic_filtering=True,id=18af07bc-c94c-4faa-9008-8be55e548870,network=Network(a3f397d3-fd75-4a6e-a815-cfcedcca15fb),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap18af07bc-c9')
Nov 29 03:44:53 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-9f314b2a-f880-46de-853b-933ddde8f52f tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: 89c7681b-4a2c-485a-b446-e8ac4df43c2b] calling os-brick to detach iSCSI Volume {{(pid=75234) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Nov 29 03:44:53 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-9f314b2a-f880-46de-853b-933ddde8f52f tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f8a5050c9a0>, {'target_iqn': 'iqn.2010-10.org.openstack:3ae025b2-997b-4f06-8e4b-4d6a37f9e3bc', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'DbfMxnTa', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdc'}, None), 'kwargs': True}}" {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:176}}
Nov 29 03:44:53 user nova-compute[75234]: WARNING os_brick.initiator.connectors.base [None req-9f314b2a-f880-46de-853b-933ddde8f52f tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Nov 29 03:44:53 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-9f314b2a-f880-46de-853b-933ddde8f52f tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:68}}
Nov 29 03:44:53 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-9f314b2a-f880-46de-853b-933ddde8f52f tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:73}}
Nov 29 03:44:53 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:44:53 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:44:53 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.020s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:44:53 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[98790804-7e6e-4f99-986a-2b33d7e578bf]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:3ae025b2-997b-4f06-8e4b-4d6a37f9e3bc\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nTarget: iqn.2010-10.org.openstack:378addbb-8d90-40c5-ae61-c3a063116df9\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:44:53 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-9f314b2a-f880-46de-853b-933ddde8f52f tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS:
Nov 29 03:44:53 user nova-compute[75234]: No targets found.
Nov 29 03:44:53 user nova-compute[75234]: iSNS:
Nov 29 03:44:53 user nova-compute[75234]: No targets found.
Nov 29 03:44:53 user nova-compute[75234]: STATIC:
Nov 29 03:44:53 user nova-compute[75234]: Target: iqn.2010-10.org.openstack:3ae025b2-997b-4f06-8e4b-4d6a37f9e3bc
Nov 29 03:44:53 user nova-compute[75234]:         Portal: 172.16.0.220:3260,-1
Nov 29 03:44:53 user nova-compute[75234]:                 Iface Name: default
Nov 29 03:44:53 user nova-compute[75234]: Target: iqn.2010-10.org.openstack:378addbb-8d90-40c5-ae61-c3a063116df9
Nov 29 03:44:53 user nova-compute[75234]:         Portal: 172.16.0.220:3260,-1
Nov 29 03:44:53 user nova-compute[75234]:                 Iface Name: default
Nov 29 03:44:53 user nova-compute[75234]: FIRMWARE:
Nov 29 03:44:53 user nova-compute[75234]: No targets found.
Nov 29 03:44:53 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:44:53 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-9f314b2a-f880-46de-853b-933ddde8f52f tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Regex to get portals from discoverydb: ^SENDTARGETS:
Nov 29 03:44:53 user nova-compute[75234]: .*?^DiscoveryAddress: 172.16.0.220,3260.*?
Nov 29 03:44:53 user nova-compute[75234]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=75234) _get_discoverydb_portals /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:382}}
Nov 29 03:44:53 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-9f314b2a-f880-46de-853b-933ddde8f52f tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:3ae025b2-997b-4f06-8e4b-4d6a37f9e3bc', 0)] {{(pid=75234) _get_connection_devices /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:830}}
Nov 29 03:44:53 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:44:53 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.016s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:44:53 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[b10fb92b-503e-46a1-a08e-1cb609fa20c8]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:378addbb-8d90-40c5-ae61-c3a063116df9\n172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:3ae025b2-997b-4f06-8e4b-4d6a37f9e3bc\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:44:53 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:44:53 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.021s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:44:53 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[15ad5a1c-7bdc-48ee-b42e-fc2713a94570]: (4, ('tcp: [59] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:378addbb-8d90-40c5-ae61-c3a063116df9 (non-flash)\ntcp: [62] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3ae025b2-997b-4f06-8e4b-4d6a37f9e3bc (non-flash)\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:44:53 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-9f314b2a-f880-46de-853b-933ddde8f52f tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [59] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:378addbb-8d90-40c5-ae61-c3a063116df9 (non-flash)
Nov 29 03:44:53 user nova-compute[75234]: tcp: [62] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3ae025b2-997b-4f06-8e4b-4d6a37f9e3bc (non-flash)
Nov 29 03:44:53 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:44:53 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-9f314b2a-f880-46de-853b-933ddde8f52f tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] iscsi session list stdout=tcp: [59] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:378addbb-8d90-40c5-ae61-c3a063116df9 (non-flash)
Nov 29 03:44:53 user nova-compute[75234]: tcp: [62] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:3ae025b2-997b-4f06-8e4b-4d6a37f9e3bc (non-flash)
Nov 29 03:44:53 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsi_session /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Nov 29 03:44:53 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-9f314b2a-f880-46de-853b-933ddde8f52f tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7f8a6035e8c0>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:3ae025b2-997b-4f06-8e4b-4d6a37f9e3bc'): ({'sdc'}, set())}) {{(pid=75234) _get_connection_devices /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:863}}
Nov 29 03:44:53 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-9f314b2a-f880-46de-853b-933ddde8f52f tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Removing single pathed devices sdc {{(pid=75234) remove_connection /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:374}}
Nov 29 03:44:53 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:44:53 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.019s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:44:53 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[71830ea5-8e24-4510-bf20-2b0ccb333395]: (4, ('path checker states:\nup                  2\n\npaths: 0\nbusy: False\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:44:53 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdc {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:44:53 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdc" returned: 0 in 0.019s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:44:53 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[6b03341d-2071-4b37-82f6-36e1cb06c69b]: (4, ('ok\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:44:53 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-9f314b2a-f880-46de-853b-933ddde8f52f tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Flushing IO for device /dev/sdc {{(pid=75234) flush_device_io /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:434}}
Nov 29 03:44:53 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdc {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:44:53 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdc" returned: 0 in 0.015s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:44:53 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[7c9cb01e-411f-4880-9093-a3e98a327850]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:44:53 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-9f314b2a-f880-46de-853b-933ddde8f52f tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Remove SCSI device /dev/sdc with /sys/block/sdc/device/delete {{(pid=75234) remove_scsi_device /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:148}}
Nov 29 03:44:53 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdc/device/delete {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:44:53 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdc/device/delete" returned: 0 in 0.043s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:44:53 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[76478ccc-62e6-4d59-9e9b-c8b2afa71574]: (4, ('1', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:44:53 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-9f314b2a-f880-46de-853b-933ddde8f52f tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Checking to see if SCSI volumes sdc have been removed. {{(pid=75234) wait_for_volumes_removal /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:156}}
Nov 29 03:44:53 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-9f314b2a-f880-46de-853b-933ddde8f52f tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] SCSI volumes sdc have been removed. {{(pid=75234) wait_for_volumes_removal /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:166}}
Nov 29 03:44:53 user nova-compute[75234]: DEBUG os_brick.privileged.rootwrap [-] Unlinking ('/dev/disk/by-id/scsi-1SCST_BIO34bf6621-40e2c9ec0673b017', '/dev/disk/by-id/scsi-23430653263396563', '/dev/disk/by-id/scsi-SSCST_BIO_Storage_34bf6621') {{(pid=86324) unlink_root /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/privileged/rootwrap.py:219}}
Nov 29 03:44:53 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[16865346-0b95-4c49-887e-d630df73d276]: (4, None) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:44:53 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-9f314b2a-f880-46de-853b-933ddde8f52f tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:3ae025b2-997b-4f06-8e4b-4d6a37f9e3bc')] {{(pid=75234) _disconnect_connection /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Nov 29 03:44:53 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:3ae025b2-997b-4f06-8e4b-4d6a37f9e3bc -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:44:53 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:3ae025b2-997b-4f06-8e4b-4d6a37f9e3bc -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.019s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:44:53 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[afd0150f-8683-4610-8d78-5ceb41667540]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:44:53 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-9f314b2a-f880-46de-853b-933ddde8f52f tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:44:53 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:3ae025b2-997b-4f06-8e4b-4d6a37f9e3bc -p 172.16.0.220:3260 --logout {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:44:53 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:3ae025b2-997b-4f06-8e4b-4d6a37f9e3bc -p 172.16.0.220:3260 --logout" returned: 0 in 0.060s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:44:53 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[a759bb36-5271-457a-a4d4-ba62d6b4feaf]: (4, ('Logging out of session [sid: 62, target: iqn.2010-10.org.openstack:3ae025b2-997b-4f06-8e4b-4d6a37f9e3bc, portal: 172.16.0.220,3260]\nLogout of [sid: 62, target: iqn.2010-10.org.openstack:3ae025b2-997b-4f06-8e4b-4d6a37f9e3bc, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:44:53 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-9f314b2a-f880-46de-853b-933ddde8f52f tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 62, target: iqn.2010-10.org.openstack:3ae025b2-997b-4f06-8e4b-4d6a37f9e3bc, portal: 172.16.0.220,3260]
Nov 29 03:44:53 user nova-compute[75234]: Logout of [sid: 62, target: iqn.2010-10.org.openstack:3ae025b2-997b-4f06-8e4b-4d6a37f9e3bc, portal: 172.16.0.220,3260] successful.
Nov 29 03:44:53 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:44:53 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:3ae025b2-997b-4f06-8e4b-4d6a37f9e3bc -p 172.16.0.220:3260 --op delete {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:44:53 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:3ae025b2-997b-4f06-8e4b-4d6a37f9e3bc -p 172.16.0.220:3260 --op delete" returned: 0 in 0.022s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:44:53 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[cf524a5f-1ba7-42df-999d-7a07e2e04f7a]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:44:53 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-9f314b2a-f880-46de-853b-933ddde8f52f tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:44:53 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-9f314b2a-f880-46de-853b-933ddde8f52f tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.307s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:87}}
Nov 29 03:44:53 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-9f314b2a-f880-46de-853b-933ddde8f52f tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] <== disconnect_volume: return (308ms) None {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:203}}
Nov 29 03:44:53 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-9f314b2a-f880-46de-853b-933ddde8f52f tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: 89c7681b-4a2c-485a-b446-e8ac4df43c2b] Disconnected iSCSI Volume {{(pid=75234) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:79}}
Nov 29 03:44:53 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-9f314b2a-f880-46de-853b-933ddde8f52f tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: 89c7681b-4a2c-485a-b446-e8ac4df43c2b] Deleting instance files /opt/stack/data/nova/instances/89c7681b-4a2c-485a-b446-e8ac4df43c2b_del
Nov 29 03:44:53 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-9f314b2a-f880-46de-853b-933ddde8f52f tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: 89c7681b-4a2c-485a-b446-e8ac4df43c2b] Deletion of /opt/stack/data/nova/instances/89c7681b-4a2c-485a-b446-e8ac4df43c2b_del complete
Nov 29 03:44:53 user nova-compute[75234]: INFO nova.compute.manager [None req-9f314b2a-f880-46de-853b-933ddde8f52f tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: 89c7681b-4a2c-485a-b446-e8ac4df43c2b] Took 1.16 seconds to destroy the instance on the hypervisor.
Nov 29 03:44:53 user nova-compute[75234]: DEBUG oslo.service.loopingcall [None req-9f314b2a-f880-46de-853b-933ddde8f52f tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=75234) func /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/loopingcall.py:435}}
Nov 29 03:44:53 user nova-compute[75234]: DEBUG nova.compute.manager [-] [instance: 89c7681b-4a2c-485a-b446-e8ac4df43c2b] Deallocating network for instance {{(pid=75234) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2279}}
Nov 29 03:44:53 user nova-compute[75234]: DEBUG nova.network.neutron [-] [instance: 89c7681b-4a2c-485a-b446-e8ac4df43c2b] deallocate_for_instance() {{(pid=75234) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1802}}
Nov 29 03:44:53 user nova-compute[75234]: DEBUG nova.network.neutron [-] [instance: 89c7681b-4a2c-485a-b446-e8ac4df43c2b] Updating instance_info_cache with network_info: [] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:44:53 user nova-compute[75234]: INFO nova.compute.manager [-] [instance: 89c7681b-4a2c-485a-b446-e8ac4df43c2b] Took 0.42 seconds to deallocate network for instance.
Nov 29 03:44:54 user nova-compute[75234]: INFO nova.compute.manager [None req-9f314b2a-f880-46de-853b-933ddde8f52f tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: 89c7681b-4a2c-485a-b446-e8ac4df43c2b] Took 0.77 seconds to detach 1 volumes for instance.
Nov 29 03:44:54 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-9f314b2a-f880-46de-853b-933ddde8f52f tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:44:54 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-9f314b2a-f880-46de-853b-933ddde8f52f tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:44:54 user nova-compute[75234]: DEBUG nova.compute.provider_tree [None req-9f314b2a-f880-46de-853b-933ddde8f52f tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Inventory has not changed in ProviderTree for provider: 4d291220-7c37-4839-a0e3-b23c66502b74 {{(pid=75234) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Nov 29 03:44:54 user nova-compute[75234]: DEBUG nova.scheduler.client.report [None req-9f314b2a-f880-46de-853b-933ddde8f52f tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Inventory has not changed for provider 4d291220-7c37-4839-a0e3-b23c66502b74 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=75234) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Nov 29 03:44:54 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-9f314b2a-f880-46de-853b-933ddde8f52f tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.128s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:44:54 user nova-compute[75234]: INFO nova.scheduler.client.report [None req-9f314b2a-f880-46de-853b-933ddde8f52f tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Deleted allocations for instance 89c7681b-4a2c-485a-b446-e8ac4df43c2b
Nov 29 03:44:54 user nova-compute[75234]: DEBUG nova.compute.manager [req-481098af-f41e-4c03-8bb7-10aff167a423 req-e0e6d93b-cff0-43ad-a985-31ff1fe4dc02 service nova] [instance: 89c7681b-4a2c-485a-b446-e8ac4df43c2b] Received event network-vif-plugged-18af07bc-c94c-4faa-9008-8be55e548870 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:44:54 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-481098af-f41e-4c03-8bb7-10aff167a423 req-e0e6d93b-cff0-43ad-a985-31ff1fe4dc02 service nova] Acquiring lock "89c7681b-4a2c-485a-b446-e8ac4df43c2b-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:44:54 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-481098af-f41e-4c03-8bb7-10aff167a423 req-e0e6d93b-cff0-43ad-a985-31ff1fe4dc02 service nova] Lock "89c7681b-4a2c-485a-b446-e8ac4df43c2b-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:44:54 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-481098af-f41e-4c03-8bb7-10aff167a423 req-e0e6d93b-cff0-43ad-a985-31ff1fe4dc02 service nova] Lock "89c7681b-4a2c-485a-b446-e8ac4df43c2b-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:44:54 user nova-compute[75234]: DEBUG nova.compute.manager [req-481098af-f41e-4c03-8bb7-10aff167a423 req-e0e6d93b-cff0-43ad-a985-31ff1fe4dc02 service nova] [instance: 89c7681b-4a2c-485a-b446-e8ac4df43c2b] No waiting events found dispatching network-vif-plugged-18af07bc-c94c-4faa-9008-8be55e548870 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:44:54 user nova-compute[75234]: WARNING nova.compute.manager [req-481098af-f41e-4c03-8bb7-10aff167a423 req-e0e6d93b-cff0-43ad-a985-31ff1fe4dc02 service nova] [instance: 89c7681b-4a2c-485a-b446-e8ac4df43c2b] Received unexpected event network-vif-plugged-18af07bc-c94c-4faa-9008-8be55e548870 for instance with vm_state deleted and task_state None.
Nov 29 03:44:54 user nova-compute[75234]: DEBUG nova.compute.manager [req-481098af-f41e-4c03-8bb7-10aff167a423 req-e0e6d93b-cff0-43ad-a985-31ff1fe4dc02 service nova] [instance: 89c7681b-4a2c-485a-b446-e8ac4df43c2b] Received event network-vif-deleted-18af07bc-c94c-4faa-9008-8be55e548870 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:44:54 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-9f314b2a-f880-46de-853b-933ddde8f52f tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Lock "89c7681b-4a2c-485a-b446-e8ac4df43c2b" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 2.662s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:44:58 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:45:03 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:45:08 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:45:13 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:45:18 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:45:19 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:45:20 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:45:22 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:45:22 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=75234) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10541}}
Nov 29 03:45:23 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:45:25 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:45:25 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager.update_available_resource {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:45:25 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:45:25 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:45:25 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:45:25 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Auditing locally available compute resources for user (node: user) {{(pid=75234) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:907}}
Nov 29 03:45:25 user nova-compute[75234]: DEBUG nova.virt.libvirt.driver [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] skipping disk /dev/sdb (vda) as it is a volume {{(pid=75234) _get_instance_disk_info_from_config /opt/stack/nova/nova/virt/libvirt/driver.py:11213}}
Nov 29 03:45:25 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:45:25 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:45:25 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Hypervisor/Node resource view: name=user free_ram=9008MB free_disk=25.15114974975586GB free_vcpus=11 pci_devices=[{"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}] {{(pid=75234) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1106}}
Nov 29 03:45:25 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:45:25 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:45:25 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Instance 95afc929-9f13-4f9a-9ac9-eae0b5e15a4d actively managed on this compute host and has allocations in placement: {'resources': {'MEMORY_MB': 512, 'VCPU': 1}}. {{(pid=75234) _remove_deleted_instances_allocations /opt/stack/nova/nova/compute/resource_tracker.py:1704}}
Nov 29 03:45:25 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Total usable vcpus: 12, total allocated vcpus: 1 {{(pid=75234) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1129}}
Nov 29 03:45:25 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Final resource view: name=user phys_ram=16011MB used_ram=1024MB phys_disk=40GB used_disk=0GB total_vcpus=12 used_vcpus=1 pci_stats=[] {{(pid=75234) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1138}}
Nov 29 03:45:26 user nova-compute[75234]: DEBUG nova.compute.provider_tree [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Inventory has not changed in ProviderTree for provider: 4d291220-7c37-4839-a0e3-b23c66502b74 {{(pid=75234) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Nov 29 03:45:26 user nova-compute[75234]: DEBUG nova.scheduler.client.report [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Inventory has not changed for provider 4d291220-7c37-4839-a0e3-b23c66502b74 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=75234) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Nov 29 03:45:26 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Compute_service record updated for user:user {{(pid=75234) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:1067}}
Nov 29 03:45:26 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.169s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:45:28 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:45:33 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:45:34 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:45:34 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Starting heal instance info cache {{(pid=75234) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9922}}
Nov 29 03:45:34 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Rebuilding the list of instances to heal {{(pid=75234) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9926}}
Nov 29 03:45:34 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Acquiring lock "refresh_cache-95afc929-9f13-4f9a-9ac9-eae0b5e15a4d" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:45:34 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Acquired lock "refresh_cache-95afc929-9f13-4f9a-9ac9-eae0b5e15a4d" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:45:34 user nova-compute[75234]: DEBUG nova.network.neutron [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] [instance: 95afc929-9f13-4f9a-9ac9-eae0b5e15a4d] Forcefully refreshing network info cache for instance {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2003}}
Nov 29 03:45:34 user nova-compute[75234]: DEBUG nova.objects.instance [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lazy-loading 'info_cache' on Instance uuid 95afc929-9f13-4f9a-9ac9-eae0b5e15a4d {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:45:34 user nova-compute[75234]: DEBUG nova.network.neutron [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] [instance: 95afc929-9f13-4f9a-9ac9-eae0b5e15a4d] Updating instance_info_cache with network_info: [{"id": "ca1227f9-b2f1-42c5-a92a-c70cb94e52c1", "address": "fa:16:3e:02:94:55", "network": {"id": "a3f397d3-fd75-4a6e-a815-cfcedcca15fb", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-678570085-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.17", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "81a265d3c15b4e9d826b35ddbd0ff443", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapca1227f9-b2", "ovs_interfaceid": "ca1227f9-b2f1-42c5-a92a-c70cb94e52c1", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:45:34 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Releasing lock "refresh_cache-95afc929-9f13-4f9a-9ac9-eae0b5e15a4d" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:45:34 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] [instance: 95afc929-9f13-4f9a-9ac9-eae0b5e15a4d] Updated the network info_cache for instance {{(pid=75234) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9993}}
Nov 29 03:45:34 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:45:34 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:45:35 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:45:38 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:45:43 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:45:43 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:45:43 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5002 ms, sending inactivity probe {{(pid=75234) run /opt/stack/data/venv/lib/python3.10/site-packages/ovs/reconnect.py:117}}
Nov 29 03:45:43 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=75234) _transition /opt/stack/data/venv/lib/python3.10/site-packages/ovs/reconnect.py:519}}
Nov 29 03:45:43 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=75234) _transition /opt/stack/data/venv/lib/python3.10/site-packages/ovs/reconnect.py:519}}
Nov 29 03:45:43 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:45:48 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:45:50 user nova-compute[75234]: DEBUG nova.compute.manager [req-8b0cea95-6d1d-4ed3-822e-cd57eefe1da7 req-7a68e30d-11f5-4ac3-bd34-b95c1d173c7b service nova] [instance: 95afc929-9f13-4f9a-9ac9-eae0b5e15a4d] Received event network-changed-ca1227f9-b2f1-42c5-a92a-c70cb94e52c1 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:45:50 user nova-compute[75234]: DEBUG nova.compute.manager [req-8b0cea95-6d1d-4ed3-822e-cd57eefe1da7 req-7a68e30d-11f5-4ac3-bd34-b95c1d173c7b service nova] [instance: 95afc929-9f13-4f9a-9ac9-eae0b5e15a4d] Refreshing instance network info cache due to event network-changed-ca1227f9-b2f1-42c5-a92a-c70cb94e52c1. {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11117}}
Nov 29 03:45:50 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-8b0cea95-6d1d-4ed3-822e-cd57eefe1da7 req-7a68e30d-11f5-4ac3-bd34-b95c1d173c7b service nova] Acquiring lock "refresh_cache-95afc929-9f13-4f9a-9ac9-eae0b5e15a4d" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:312}}
Nov 29 03:45:50 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-8b0cea95-6d1d-4ed3-822e-cd57eefe1da7 req-7a68e30d-11f5-4ac3-bd34-b95c1d173c7b service nova] Acquired lock "refresh_cache-95afc929-9f13-4f9a-9ac9-eae0b5e15a4d" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:315}}
Nov 29 03:45:50 user nova-compute[75234]: DEBUG nova.network.neutron [req-8b0cea95-6d1d-4ed3-822e-cd57eefe1da7 req-7a68e30d-11f5-4ac3-bd34-b95c1d173c7b service nova] [instance: 95afc929-9f13-4f9a-9ac9-eae0b5e15a4d] Refreshing network info cache for port ca1227f9-b2f1-42c5-a92a-c70cb94e52c1 {{(pid=75234) _get_instance_nw_info /opt/stack/nova/nova/network/neutron.py:2006}}
Nov 29 03:45:50 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-9b57c96e-ca1e-4c48-96f3-bb2ec6efc619 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Acquiring lock "95afc929-9f13-4f9a-9ac9-eae0b5e15a4d" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:45:50 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-9b57c96e-ca1e-4c48-96f3-bb2ec6efc619 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Lock "95afc929-9f13-4f9a-9ac9-eae0b5e15a4d" acquired by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:45:50 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-9b57c96e-ca1e-4c48-96f3-bb2ec6efc619 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Acquiring lock "95afc929-9f13-4f9a-9ac9-eae0b5e15a4d-events" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:45:50 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-9b57c96e-ca1e-4c48-96f3-bb2ec6efc619 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Lock "95afc929-9f13-4f9a-9ac9-eae0b5e15a4d-events" acquired by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:45:50 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-9b57c96e-ca1e-4c48-96f3-bb2ec6efc619 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Lock "95afc929-9f13-4f9a-9ac9-eae0b5e15a4d-events" "released" by "nova.compute.manager.InstanceEvents.clear_events_for_instance.<locals>._clear_events" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:45:50 user nova-compute[75234]: INFO nova.compute.manager [None req-9b57c96e-ca1e-4c48-96f3-bb2ec6efc619 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: 95afc929-9f13-4f9a-9ac9-eae0b5e15a4d] Terminating instance
Nov 29 03:45:50 user nova-compute[75234]: DEBUG nova.compute.manager [None req-9b57c96e-ca1e-4c48-96f3-bb2ec6efc619 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: 95afc929-9f13-4f9a-9ac9-eae0b5e15a4d] Start destroying the instance on the hypervisor. {{(pid=75234) _shutdown_instance /opt/stack/nova/nova/compute/manager.py:3140}}
Nov 29 03:45:50 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:45:50 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:45:50 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:45:50 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:45:50 user nova-compute[75234]: DEBUG nova.compute.manager [req-74a6faeb-669b-4a20-b101-c7a027ce5324 req-5a92e509-c97d-4a0f-a31e-ff3521a095ac service nova] [instance: 95afc929-9f13-4f9a-9ac9-eae0b5e15a4d] Received event network-vif-unplugged-ca1227f9-b2f1-42c5-a92a-c70cb94e52c1 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:45:50 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-74a6faeb-669b-4a20-b101-c7a027ce5324 req-5a92e509-c97d-4a0f-a31e-ff3521a095ac service nova] Acquiring lock "95afc929-9f13-4f9a-9ac9-eae0b5e15a4d-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:45:50 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-74a6faeb-669b-4a20-b101-c7a027ce5324 req-5a92e509-c97d-4a0f-a31e-ff3521a095ac service nova] Lock "95afc929-9f13-4f9a-9ac9-eae0b5e15a4d-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:45:50 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-74a6faeb-669b-4a20-b101-c7a027ce5324 req-5a92e509-c97d-4a0f-a31e-ff3521a095ac service nova] Lock "95afc929-9f13-4f9a-9ac9-eae0b5e15a4d-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:45:50 user nova-compute[75234]: DEBUG nova.compute.manager [req-74a6faeb-669b-4a20-b101-c7a027ce5324 req-5a92e509-c97d-4a0f-a31e-ff3521a095ac service nova] [instance: 95afc929-9f13-4f9a-9ac9-eae0b5e15a4d] No waiting events found dispatching network-vif-unplugged-ca1227f9-b2f1-42c5-a92a-c70cb94e52c1 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:45:50 user nova-compute[75234]: DEBUG nova.compute.manager [req-74a6faeb-669b-4a20-b101-c7a027ce5324 req-5a92e509-c97d-4a0f-a31e-ff3521a095ac service nova] [instance: 95afc929-9f13-4f9a-9ac9-eae0b5e15a4d] Received event network-vif-unplugged-ca1227f9-b2f1-42c5-a92a-c70cb94e52c1 for instance with task_state deleting. {{(pid=75234) _process_instance_event /opt/stack/nova/nova/compute/manager.py:10890}}
Nov 29 03:45:50 user nova-compute[75234]: DEBUG nova.network.neutron [req-8b0cea95-6d1d-4ed3-822e-cd57eefe1da7 req-7a68e30d-11f5-4ac3-bd34-b95c1d173c7b service nova] [instance: 95afc929-9f13-4f9a-9ac9-eae0b5e15a4d] Updated VIF entry in instance network info cache for port ca1227f9-b2f1-42c5-a92a-c70cb94e52c1. {{(pid=75234) _build_network_info_model /opt/stack/nova/nova/network/neutron.py:3481}}
Nov 29 03:45:50 user nova-compute[75234]: DEBUG nova.network.neutron [req-8b0cea95-6d1d-4ed3-822e-cd57eefe1da7 req-7a68e30d-11f5-4ac3-bd34-b95c1d173c7b service nova] [instance: 95afc929-9f13-4f9a-9ac9-eae0b5e15a4d] Updating instance_info_cache with network_info: [{"id": "ca1227f9-b2f1-42c5-a92a-c70cb94e52c1", "address": "fa:16:3e:02:94:55", "network": {"id": "a3f397d3-fd75-4a6e-a815-cfcedcca15fb", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-678570085-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": []}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "81a265d3c15b4e9d826b35ddbd0ff443", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapca1227f9-b2", "ovs_interfaceid": "ca1227f9-b2f1-42c5-a92a-c70cb94e52c1", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}}] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:45:50 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-8b0cea95-6d1d-4ed3-822e-cd57eefe1da7 req-7a68e30d-11f5-4ac3-bd34-b95c1d173c7b service nova] Releasing lock "refresh_cache-95afc929-9f13-4f9a-9ac9-eae0b5e15a4d" {{(pid=75234) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:333}}
Nov 29 03:45:50 user nova-compute[75234]: INFO nova.virt.libvirt.driver [-] [instance: 95afc929-9f13-4f9a-9ac9-eae0b5e15a4d] Instance destroyed successfully.
Nov 29 03:45:50 user nova-compute[75234]: DEBUG nova.objects.instance [None req-9b57c96e-ca1e-4c48-96f3-bb2ec6efc619 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Lazy-loading 'resources' on Instance uuid 95afc929-9f13-4f9a-9ac9-eae0b5e15a4d {{(pid=75234) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1141}}
Nov 29 03:45:50 user nova-compute[75234]: DEBUG nova.virt.libvirt.vif [None req-9b57c96e-ca1e-4c48-96f3-bb2ec6efc619 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] vif_type=ovs instance=Instance(access_ip_v4=None,access_ip_v6=None,architecture=None,auto_disk_config=False,availability_zone='nova',cell_name=None,cleaned=False,compute_id=1,config_drive='',created_at=2023-11-29T03:40:59Z,default_ephemeral_device=None,default_swap_device=None,deleted=False,deleted_at=None,device_metadata=<?>,disable_terminate=False,display_description='tempest-TestVolumeBootPattern-server-2064710171',display_name='tempest-TestVolumeBootPattern-server-2064710171',ec2_ids=<?>,ephemeral_gb=0,ephemeral_key_uuid=None,fault=<?>,flavor=Flavor(6),hidden=False,host='user',hostname='tempest-testvolumebootpattern-server-2064710171',id=51,image_ref='',info_cache=InstanceInfoCache,instance_type_id=6,kernel_id='',key_data='ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBKrnAt22NMoBNIFsnCF3Mxk+dIbrL4YaXFADzEahC/3pON2UrJ4NOPzv0Ze4QWq/A4Bm4h/5Mn2lQRP6Y0eXl2r9TPWd6a1CQjrsTRMK7MqwF4n4QkuDxmXBPaNWL5XvpA==',key_name='tempest-TestVolumeBootPattern-1253305043',keypairs=<?>,launch_index=0,launched_at=2023-11-29T03:41:12Z,launched_on='user',locked=False,locked_by=None,memory_mb=512,metadata={},migration_context=<?>,new_flavor=None,node='user',numa_topology=None,old_flavor=None,os_type=None,pci_devices=<?>,pci_requests=<?>,power_state=1,progress=0,project_id='81a265d3c15b4e9d826b35ddbd0ff443',ramdisk_id='',reservation_id='r-zr0x3vw3',resources=None,root_device_name='/dev/vda',root_gb=1,security_groups=SecurityGroupList,services=<?>,shutdown_terminate=False,system_metadata={boot_roles='reader,member',image_base_image_ref='',image_hw_cdrom_bus='ide',image_hw_disk_bus='virtio',image_hw_input_bus=None,image_hw_machine_type='pc',image_hw_pointer_model=None,image_hw_rng_model='virtio',image_hw_video_model='virtio',image_hw_vif_model='virtio',image_min_disk='1',image_min_ram='0',image_owner_specified.openstack.md5='',image_owner_specified.openstack.object='images/cirros-0.6.2-x86_64-disk',image_owner_specified.openstack.sha256='',owner_project_name='tempest-TestVolumeBootPattern-1244492388',owner_user_name='tempest-TestVolumeBootPattern-1244492388-project-member'},tags=<?>,task_state='deleting',terminated_at=None,trusted_certs=<?>,updated_at=2023-11-29T03:41:13Z,user_data=None,user_id='7d1f1f11a9aa4c1db8627413eb963fa4',uuid=95afc929-9f13-4f9a-9ac9-eae0b5e15a4d,vcpu_model=<?>,vcpus=1,vm_mode=None,vm_state='active') vif={"id": "ca1227f9-b2f1-42c5-a92a-c70cb94e52c1", "address": "fa:16:3e:02:94:55", "network": {"id": "a3f397d3-fd75-4a6e-a815-cfcedcca15fb", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-678570085-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.17", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "81a265d3c15b4e9d826b35ddbd0ff443", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapca1227f9-b2", "ovs_interfaceid": "ca1227f9-b2f1-42c5-a92a-c70cb94e52c1", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) unplug /opt/stack/nova/nova/virt/libvirt/vif.py:828}}
Nov 29 03:45:50 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-9b57c96e-ca1e-4c48-96f3-bb2ec6efc619 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Converting VIF {"id": "ca1227f9-b2f1-42c5-a92a-c70cb94e52c1", "address": "fa:16:3e:02:94:55", "network": {"id": "a3f397d3-fd75-4a6e-a815-cfcedcca15fb", "bridge": "br-int", "label": "tempest-TestVolumeBootPattern-678570085-network", "subnets": [{"cidr": "10.0.0.0/28", "dns": [], "gateway": {"address": "10.0.0.1", "type": "gateway", "version": 4, "meta": {}}, "ips": [{"address": "10.0.0.14", "type": "fixed", "version": 4, "meta": {}, "floating_ips": [{"address": "172.24.4.17", "type": "floating", "version": 4, "meta": {}}]}], "routes": [], "version": 4, "meta": {"dhcp_server": "10.0.0.1"}}], "meta": {"injected": false, "tenant_id": "81a265d3c15b4e9d826b35ddbd0ff443", "mtu": 1442, "physical_network": null, "tunneled": true}}, "type": "ovs", "details": {"port_filter": true, "connectivity": "l2", "bound_drivers": {"0": "ovn"}}, "devname": "tapca1227f9-b2", "ovs_interfaceid": "ca1227f9-b2f1-42c5-a92a-c70cb94e52c1", "qbh_params": null, "qbg_params": null, "active": true, "vnic_type": "normal", "profile": {}, "preserve_on_delete": false, "delegate_create": true, "meta": {}} {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:511}}
Nov 29 03:45:50 user nova-compute[75234]: DEBUG nova.network.os_vif_util [None req-9b57c96e-ca1e-4c48-96f3-bb2ec6efc619 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Converted object VIFOpenVSwitch(active=True,address=fa:16:3e:02:94:55,bridge_name='br-int',has_traffic_filtering=True,id=ca1227f9-b2f1-42c5-a92a-c70cb94e52c1,network=Network(a3f397d3-fd75-4a6e-a815-cfcedcca15fb),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapca1227f9-b2') {{(pid=75234) nova_to_osvif_vif /opt/stack/nova/nova/network/os_vif_util.py:548}}
Nov 29 03:45:50 user nova-compute[75234]: DEBUG os_vif [None req-9b57c96e-ca1e-4c48-96f3-bb2ec6efc619 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Unplugging vif VIFOpenVSwitch(active=True,address=fa:16:3e:02:94:55,bridge_name='br-int',has_traffic_filtering=True,id=ca1227f9-b2f1-42c5-a92a-c70cb94e52c1,network=Network(a3f397d3-fd75-4a6e-a815-cfcedcca15fb),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapca1227f9-b2') {{(pid=75234) unplug /opt/stack/data/venv/lib/python3.10/site-packages/os_vif/__init__.py:109}}
Nov 29 03:45:50 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:45:50 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DelPortCommand(_result=None, port=tapca1227f9-b2, bridge=br-int, if_exists=True) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:45:50 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:45:50 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:45:50 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 23 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:45:50 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbDestroyCommand(_result=None, table=QoS, record=582e3c7f-6572-48dc-8ac0-32a581fb63c4) {{(pid=75234) do_commit /opt/stack/data/venv/lib/python3.10/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:89}}
Nov 29 03:45:50 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:45:50 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 0-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:45:50 user nova-compute[75234]: INFO os_vif [None req-9b57c96e-ca1e-4c48-96f3-bb2ec6efc619 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Successfully unplugged vif VIFOpenVSwitch(active=True,address=fa:16:3e:02:94:55,bridge_name='br-int',has_traffic_filtering=True,id=ca1227f9-b2f1-42c5-a92a-c70cb94e52c1,network=Network(a3f397d3-fd75-4a6e-a815-cfcedcca15fb),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapca1227f9-b2')
Nov 29 03:45:50 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-9b57c96e-ca1e-4c48-96f3-bb2ec6efc619 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: 95afc929-9f13-4f9a-9ac9-eae0b5e15a4d] calling os-brick to detach iSCSI Volume {{(pid=75234) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:72}}
Nov 29 03:45:50 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-9b57c96e-ca1e-4c48-96f3-bb2ec6efc619 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] ==> disconnect_volume: call "{'args': (<os_brick.initiator.connectors.iscsi.ISCSIConnector object at 0x7f8a5050c9a0>, {'target_iqn': 'iqn.2010-10.org.openstack:378addbb-8d90-40c5-ae61-c3a063116df9', 'target_portal': '172.16.0.220:3260', 'target_discovered': False, 'auth_method': 'CHAP', 'auth_username': 'cyOomDWj', 'auth_password': '***', 'target_lun': 0, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'device_path': '/dev/sdb'}, None), 'kwargs': True}}" {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:176}}
Nov 29 03:45:50 user nova-compute[75234]: WARNING os_brick.initiator.connectors.base [None req-9b57c96e-ca1e-4c48-96f3-bb2ec6efc619 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Service needs to call os_brick.setup() before connecting volumes, if it doesn't it will break on the next release
Nov 29 03:45:50 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-9b57c96e-ca1e-4c48-96f3-bb2ec6efc619 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Acquiring lock "connect_volume" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:68}}
Nov 29 03:45:50 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-9b57c96e-ca1e-4c48-96f3-bb2ec6efc619 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Lock "connect_volume" acquired by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: waited 0.005s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:73}}
Nov 29 03:45:50 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m discoverydb -o show -P 1 {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:45:50 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m discoverydb -o show -P 1" returned: 0 in 0.019s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:45:50 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[0efd83d1-bcdd-4344-a93c-2709cc8e6b84]: (4, ('SENDTARGETS:\nNo targets found.\niSNS:\nNo targets found.\nSTATIC:\nTarget: iqn.2010-10.org.openstack:378addbb-8d90-40c5-ae61-c3a063116df9\n\tPortal: 172.16.0.220:3260,-1\n\t\tIface Name: default\nFIRMWARE:\nNo targets found.\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:45:50 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-9b57c96e-ca1e-4c48-96f3-bb2ec6efc619 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] iscsiadm ['-m', 'discoverydb', '-o', 'show', '-P', 1]: stdout=SENDTARGETS:
Nov 29 03:45:50 user nova-compute[75234]: No targets found.
Nov 29 03:45:50 user nova-compute[75234]: iSNS:
Nov 29 03:45:50 user nova-compute[75234]: No targets found.
Nov 29 03:45:50 user nova-compute[75234]: STATIC:
Nov 29 03:45:50 user nova-compute[75234]: Target: iqn.2010-10.org.openstack:378addbb-8d90-40c5-ae61-c3a063116df9
Nov 29 03:45:50 user nova-compute[75234]:         Portal: 172.16.0.220:3260,-1
Nov 29 03:45:50 user nova-compute[75234]:                 Iface Name: default
Nov 29 03:45:50 user nova-compute[75234]: FIRMWARE:
Nov 29 03:45:50 user nova-compute[75234]: No targets found.
Nov 29 03:45:50 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:45:50 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-9b57c96e-ca1e-4c48-96f3-bb2ec6efc619 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Regex to get portals from discoverydb: ^SENDTARGETS:
Nov 29 03:45:50 user nova-compute[75234]: .*?^DiscoveryAddress: 172.16.0.220,3260.*?
Nov 29 03:45:50 user nova-compute[75234]: (.*?)^(?:DiscoveryAddress|iSNS):.* {{(pid=75234) _get_discoverydb_portals /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:382}}
Nov 29 03:45:50 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-9b57c96e-ca1e-4c48-96f3-bb2ec6efc619 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Getting connected devices for (ips,iqns,luns)=[('172.16.0.220:3260', 'iqn.2010-10.org.openstack:378addbb-8d90-40c5-ae61-c3a063116df9', 0)] {{(pid=75234) _get_connection_devices /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:830}}
Nov 29 03:45:50 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:45:50 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node" returned: 0 in 0.021s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:45:50 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[c05c30b0-724e-4945-b35a-ba08b6321ff8]: (4, ('172.16.0.220:3260,4294967295 iqn.2010-10.org.openstack:378addbb-8d90-40c5-ae61-c3a063116df9\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:45:50 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m session {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:45:50 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m session" returned: 0 in 0.019s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:45:50 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[ae5dc5f7-43af-4500-bed3-d4bfa31de4ce]: (4, ('tcp: [59] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:378addbb-8d90-40c5-ae61-c3a063116df9 (non-flash)\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:45:50 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-9b57c96e-ca1e-4c48-96f3-bb2ec6efc619 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] iscsiadm ('-m', 'session'): stdout=tcp: [59] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:378addbb-8d90-40c5-ae61-c3a063116df9 (non-flash)
Nov 29 03:45:50 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm_bare /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1193}}
Nov 29 03:45:50 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-9b57c96e-ca1e-4c48-96f3-bb2ec6efc619 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] iscsi session list stdout=tcp: [59] 172.16.0.220:3260,1 iqn.2010-10.org.openstack:378addbb-8d90-40c5-ae61-c3a063116df9 (non-flash)
Nov 29 03:45:50 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsi_session /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1182}}
Nov 29 03:45:50 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-9b57c96e-ca1e-4c48-96f3-bb2ec6efc619 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Resulting device map defaultdict(<function ISCSIConnector._get_connection_devices.<locals>.<lambda> at 0x7f8a6822bb50>, {('172.16.0.220:3260', 'iqn.2010-10.org.openstack:378addbb-8d90-40c5-ae61-c3a063116df9'): ({'sdb'}, set())}) {{(pid=75234) _get_connection_devices /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:863}}
Nov 29 03:45:50 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-9b57c96e-ca1e-4c48-96f3-bb2ec6efc619 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Removing single pathed devices sdb {{(pid=75234) remove_connection /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:374}}
Nov 29 03:45:50 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd show status {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:45:50 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd show status" returned: 0 in 0.018s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:45:50 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[d81feb6b-bcd6-4dbb-a67f-ea9893dbc953]: (4, ('path checker states:\nup                  1\n\npaths: 0\nbusy: False\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:45:50 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): multipathd del path /dev/sdb {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:45:51 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "multipathd del path /dev/sdb" returned: 0 in 0.020s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:45:51 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[52a33ef6-69a3-4fd2-99cb-822d5d2f1346]: (4, ('ok\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:45:51 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-9b57c96e-ca1e-4c48-96f3-bb2ec6efc619 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Flushing IO for device /dev/sdb {{(pid=75234) flush_device_io /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:434}}
Nov 29 03:45:51 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): blockdev --flushbufs /dev/sdb {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:45:51 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "blockdev --flushbufs /dev/sdb" returned: 0 in 0.015s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:45:51 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[9d7e8d78-4891-40d6-a54f-a3f9bf0d8de7]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:45:51 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-9b57c96e-ca1e-4c48-96f3-bb2ec6efc619 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Remove SCSI device /dev/sdb with /sys/block/sdb/device/delete {{(pid=75234) remove_scsi_device /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:148}}
Nov 29 03:45:51 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): tee -a /sys/block/sdb/device/delete {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:45:51 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "tee -a /sys/block/sdb/device/delete" returned: 0 in 0.045s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:45:51 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[6b9cd474-7e77-44b5-bd46-738dd5d2c75c]: (4, ('1', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:45:51 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-9b57c96e-ca1e-4c48-96f3-bb2ec6efc619 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Checking to see if SCSI volumes sdb have been removed. {{(pid=75234) wait_for_volumes_removal /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:156}}
Nov 29 03:45:51 user nova-compute[75234]: DEBUG os_brick.initiator.linuxscsi [None req-9b57c96e-ca1e-4c48-96f3-bb2ec6efc619 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] SCSI volumes sdb have been removed. {{(pid=75234) wait_for_volumes_removal /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/linuxscsi.py:166}}
Nov 29 03:45:51 user nova-compute[75234]: DEBUG os_brick.privileged.rootwrap [-] Unlinking ('/dev/disk/by-id/scsi-1SCST_BIObddee582-826abb057e770bfe', '/dev/disk/by-id/scsi-23832366162623035', '/dev/disk/by-id/scsi-SSCST_BIO_Storage_bddee582') {{(pid=86324) unlink_root /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/privileged/rootwrap.py:219}}
Nov 29 03:45:51 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[a960315c-805b-481e-8f7d-529658f9db23]: (4, None) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:45:51 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-9b57c96e-ca1e-4c48-96f3-bb2ec6efc619 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Disconnecting from: [('172.16.0.220:3260', 'iqn.2010-10.org.openstack:378addbb-8d90-40c5-ae61-c3a063116df9')] {{(pid=75234) _disconnect_connection /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1171}}
Nov 29 03:45:51 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:378addbb-8d90-40c5-ae61-c3a063116df9 -p 172.16.0.220:3260 --op update -n node.startup -v manual {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:45:51 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:378addbb-8d90-40c5-ae61-c3a063116df9 -p 172.16.0.220:3260 --op update -n node.startup -v manual" returned: 0 in 0.017s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:45:51 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[8056dcea-afec-4c5a-8712-50e1d508ba2a]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:45:51 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-9b57c96e-ca1e-4c48-96f3-bb2ec6efc619 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] iscsiadm ('--op', 'update', '-n', 'node.startup', '-v', 'manual'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:45:51 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:378addbb-8d90-40c5-ae61-c3a063116df9 -p 172.16.0.220:3260 --logout {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:45:51 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:378addbb-8d90-40c5-ae61-c3a063116df9 -p 172.16.0.220:3260 --logout" returned: 0 in 0.125s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:45:51 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[85c987d1-8d46-4d71-939f-ee471ce07c5a]: (4, ('Logging out of session [sid: 59, target: iqn.2010-10.org.openstack:378addbb-8d90-40c5-ae61-c3a063116df9, portal: 172.16.0.220,3260]\nLogout of [sid: 59, target: iqn.2010-10.org.openstack:378addbb-8d90-40c5-ae61-c3a063116df9, portal: 172.16.0.220,3260] successful.\n', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:45:51 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-9b57c96e-ca1e-4c48-96f3-bb2ec6efc619 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] iscsiadm ('--logout',): stdout=Logging out of session [sid: 59, target: iqn.2010-10.org.openstack:378addbb-8d90-40c5-ae61-c3a063116df9, portal: 172.16.0.220,3260]
Nov 29 03:45:51 user nova-compute[75234]: Logout of [sid: 59, target: iqn.2010-10.org.openstack:378addbb-8d90-40c5-ae61-c3a063116df9, portal: 172.16.0.220,3260] successful.
Nov 29 03:45:51 user nova-compute[75234]:  stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:45:51 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] Running cmd (subprocess): iscsiadm -m node -T iqn.2010-10.org.openstack:378addbb-8d90-40c5-ae61-c3a063116df9 -p 172.16.0.220:3260 --op delete {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:384}}
Nov 29 03:45:51 user nova-compute[75234]: DEBUG oslo_concurrency.processutils [-] CMD "iscsiadm -m node -T iqn.2010-10.org.openstack:378addbb-8d90-40c5-ae61-c3a063116df9 -p 172.16.0.220:3260 --op delete" returned: 0 in 0.019s {{(pid=86324) execute /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/processutils.py:422}}
Nov 29 03:45:51 user nova-compute[75234]: DEBUG oslo.privsep.daemon [-] privsep: reply[21811174-aa47-47a5-85ae-5a352605cd2a]: (4, ('', '')) {{(pid=86324) _call_back /opt/stack/data/venv/lib/python3.10/site-packages/oslo_privsep/daemon.py:499}}
Nov 29 03:45:51 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-9b57c96e-ca1e-4c48-96f3-bb2ec6efc619 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] iscsiadm ('--op', 'delete'): stdout= stderr= {{(pid=75234) _run_iscsiadm /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/iscsi.py:1037}}
Nov 29 03:45:51 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.base [None req-9b57c96e-ca1e-4c48-96f3-bb2ec6efc619 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Lock "connect_volume" "released" by "os_brick.initiator.connectors.iscsi.ISCSIConnector.disconnect_volume" :: held 0.368s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/initiator/connectors/base.py:87}}
Nov 29 03:45:51 user nova-compute[75234]: DEBUG os_brick.initiator.connectors.iscsi [None req-9b57c96e-ca1e-4c48-96f3-bb2ec6efc619 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] <== disconnect_volume: return (374ms) None {{(pid=75234) trace_logging_wrapper /opt/stack/data/venv/lib/python3.10/site-packages/os_brick/utils.py:203}}
Nov 29 03:45:51 user nova-compute[75234]: DEBUG nova.virt.libvirt.volume.iscsi [None req-9b57c96e-ca1e-4c48-96f3-bb2ec6efc619 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: 95afc929-9f13-4f9a-9ac9-eae0b5e15a4d] Disconnected iSCSI Volume {{(pid=75234) disconnect_volume /opt/stack/nova/nova/virt/libvirt/volume/iscsi.py:79}}
Nov 29 03:45:51 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-9b57c96e-ca1e-4c48-96f3-bb2ec6efc619 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: 95afc929-9f13-4f9a-9ac9-eae0b5e15a4d] Deleting instance files /opt/stack/data/nova/instances/95afc929-9f13-4f9a-9ac9-eae0b5e15a4d_del
Nov 29 03:45:51 user nova-compute[75234]: INFO nova.virt.libvirt.driver [None req-9b57c96e-ca1e-4c48-96f3-bb2ec6efc619 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: 95afc929-9f13-4f9a-9ac9-eae0b5e15a4d] Deletion of /opt/stack/data/nova/instances/95afc929-9f13-4f9a-9ac9-eae0b5e15a4d_del complete
Nov 29 03:45:51 user nova-compute[75234]: INFO nova.compute.manager [None req-9b57c96e-ca1e-4c48-96f3-bb2ec6efc619 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: 95afc929-9f13-4f9a-9ac9-eae0b5e15a4d] Took 1.44 seconds to destroy the instance on the hypervisor.
Nov 29 03:45:51 user nova-compute[75234]: DEBUG oslo.service.loopingcall [None req-9b57c96e-ca1e-4c48-96f3-bb2ec6efc619 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Waiting for function nova.compute.manager.ComputeManager._try_deallocate_network.<locals>._deallocate_network_with_retries to return. {{(pid=75234) func /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/loopingcall.py:435}}
Nov 29 03:45:51 user nova-compute[75234]: DEBUG nova.compute.manager [-] [instance: 95afc929-9f13-4f9a-9ac9-eae0b5e15a4d] Deallocating network for instance {{(pid=75234) _deallocate_network /opt/stack/nova/nova/compute/manager.py:2279}}
Nov 29 03:45:51 user nova-compute[75234]: DEBUG nova.network.neutron [-] [instance: 95afc929-9f13-4f9a-9ac9-eae0b5e15a4d] deallocate_for_instance() {{(pid=75234) deallocate_for_instance /opt/stack/nova/nova/network/neutron.py:1802}}
Nov 29 03:45:51 user nova-compute[75234]: DEBUG nova.network.neutron [-] [instance: 95afc929-9f13-4f9a-9ac9-eae0b5e15a4d] Updating instance_info_cache with network_info: [] {{(pid=75234) update_instance_cache_with_nw_info /opt/stack/nova/nova/network/neutron.py:116}}
Nov 29 03:45:51 user nova-compute[75234]: INFO nova.compute.manager [-] [instance: 95afc929-9f13-4f9a-9ac9-eae0b5e15a4d] Took 0.38 seconds to deallocate network for instance.
Nov 29 03:45:52 user nova-compute[75234]: DEBUG nova.compute.manager [req-d2f567c3-2d65-452d-be33-3562efc2bde6 req-e0cfe655-fbc5-48a4-9b49-298b57d51093 service nova] [instance: 95afc929-9f13-4f9a-9ac9-eae0b5e15a4d] Received event network-vif-plugged-ca1227f9-b2f1-42c5-a92a-c70cb94e52c1 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:45:52 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-d2f567c3-2d65-452d-be33-3562efc2bde6 req-e0cfe655-fbc5-48a4-9b49-298b57d51093 service nova] Acquiring lock "95afc929-9f13-4f9a-9ac9-eae0b5e15a4d-events" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:45:52 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-d2f567c3-2d65-452d-be33-3562efc2bde6 req-e0cfe655-fbc5-48a4-9b49-298b57d51093 service nova] Lock "95afc929-9f13-4f9a-9ac9-eae0b5e15a4d-events" acquired by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:45:52 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [req-d2f567c3-2d65-452d-be33-3562efc2bde6 req-e0cfe655-fbc5-48a4-9b49-298b57d51093 service nova] Lock "95afc929-9f13-4f9a-9ac9-eae0b5e15a4d-events" "released" by "nova.compute.manager.InstanceEvents.pop_instance_event.<locals>._pop_event" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:45:52 user nova-compute[75234]: DEBUG nova.compute.manager [req-d2f567c3-2d65-452d-be33-3562efc2bde6 req-e0cfe655-fbc5-48a4-9b49-298b57d51093 service nova] [instance: 95afc929-9f13-4f9a-9ac9-eae0b5e15a4d] No waiting events found dispatching network-vif-plugged-ca1227f9-b2f1-42c5-a92a-c70cb94e52c1 {{(pid=75234) pop_instance_event /opt/stack/nova/nova/compute/manager.py:320}}
Nov 29 03:45:52 user nova-compute[75234]: WARNING nova.compute.manager [req-d2f567c3-2d65-452d-be33-3562efc2bde6 req-e0cfe655-fbc5-48a4-9b49-298b57d51093 service nova] [instance: 95afc929-9f13-4f9a-9ac9-eae0b5e15a4d] Received unexpected event network-vif-plugged-ca1227f9-b2f1-42c5-a92a-c70cb94e52c1 for instance with vm_state active and task_state deleting.
Nov 29 03:45:52 user nova-compute[75234]: DEBUG nova.compute.manager [req-d2f567c3-2d65-452d-be33-3562efc2bde6 req-e0cfe655-fbc5-48a4-9b49-298b57d51093 service nova] [instance: 95afc929-9f13-4f9a-9ac9-eae0b5e15a4d] Received event network-vif-deleted-ca1227f9-b2f1-42c5-a92a-c70cb94e52c1 {{(pid=75234) external_instance_event /opt/stack/nova/nova/compute/manager.py:11112}}
Nov 29 03:45:52 user nova-compute[75234]: INFO nova.compute.manager [None req-9b57c96e-ca1e-4c48-96f3-bb2ec6efc619 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] [instance: 95afc929-9f13-4f9a-9ac9-eae0b5e15a4d] Took 0.73 seconds to detach 1 volumes for instance.
Nov 29 03:45:52 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-9b57c96e-ca1e-4c48-96f3-bb2ec6efc619 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.update_usage" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:45:52 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-9b57c96e-ca1e-4c48-96f3-bb2ec6efc619 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:45:52 user nova-compute[75234]: DEBUG nova.compute.provider_tree [None req-9b57c96e-ca1e-4c48-96f3-bb2ec6efc619 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Inventory has not changed in ProviderTree for provider: 4d291220-7c37-4839-a0e3-b23c66502b74 {{(pid=75234) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Nov 29 03:45:52 user nova-compute[75234]: DEBUG nova.scheduler.client.report [None req-9b57c96e-ca1e-4c48-96f3-bb2ec6efc619 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Inventory has not changed for provider 4d291220-7c37-4839-a0e3-b23c66502b74 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=75234) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Nov 29 03:45:52 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-9b57c96e-ca1e-4c48-96f3-bb2ec6efc619 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.update_usage" :: held 0.109s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:45:52 user nova-compute[75234]: INFO nova.scheduler.client.report [None req-9b57c96e-ca1e-4c48-96f3-bb2ec6efc619 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Deleted allocations for instance 95afc929-9f13-4f9a-9ac9-eae0b5e15a4d
Nov 29 03:45:52 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-9b57c96e-ca1e-4c48-96f3-bb2ec6efc619 tempest-TestVolumeBootPattern-1244492388 tempest-TestVolumeBootPattern-1244492388-project-member] Lock "95afc929-9f13-4f9a-9ac9-eae0b5e15a4d" "released" by "nova.compute.manager.ComputeManager.terminate_instance.<locals>.do_terminate_instance" :: held 2.845s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:45:55 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:46:00 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:46:00 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:46:00 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5002 ms, sending inactivity probe {{(pid=75234) run /opt/stack/data/venv/lib/python3.10/site-packages/ovs/reconnect.py:117}}
Nov 29 03:46:00 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=75234) _transition /opt/stack/data/venv/lib/python3.10/site-packages/ovs/reconnect.py:519}}
Nov 29 03:46:00 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=75234) _transition /opt/stack/data/venv/lib/python3.10/site-packages/ovs/reconnect.py:519}}
Nov 29 03:46:00 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:46:05 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:46:10 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:46:15 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:46:15 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:46:15 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5002 ms, sending inactivity probe {{(pid=75234) run /opt/stack/data/venv/lib/python3.10/site-packages/ovs/reconnect.py:117}}
Nov 29 03:46:15 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=75234) _transition /opt/stack/data/venv/lib/python3.10/site-packages/ovs/reconnect.py:519}}
Nov 29 03:46:15 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=75234) _transition /opt/stack/data/venv/lib/python3.10/site-packages/ovs/reconnect.py:519}}
Nov 29 03:46:15 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:46:19 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:46:20 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:46:21 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:46:23 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:46:23 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=75234) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10541}}
Nov 29 03:46:25 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager.update_available_resource {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:46:25 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:46:25 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:46:25 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:46:25 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Auditing locally available compute resources for user (node: user) {{(pid=75234) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:907}}
Nov 29 03:46:25 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:46:25 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:46:25 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Hypervisor/Node resource view: name=user free_ram=9145MB free_disk=25.150272369384766GB free_vcpus=12 pci_devices=[{"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}] {{(pid=75234) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1106}}
Nov 29 03:46:25 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:46:25 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:46:25 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:46:25 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Total usable vcpus: 12, total allocated vcpus: 0 {{(pid=75234) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1129}}
Nov 29 03:46:25 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Final resource view: name=user phys_ram=16011MB used_ram=512MB phys_disk=40GB used_disk=0GB total_vcpus=12 used_vcpus=0 pci_stats=[] {{(pid=75234) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1138}}
Nov 29 03:46:25 user nova-compute[75234]: DEBUG nova.compute.provider_tree [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Inventory has not changed in ProviderTree for provider: 4d291220-7c37-4839-a0e3-b23c66502b74 {{(pid=75234) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Nov 29 03:46:25 user nova-compute[75234]: DEBUG nova.scheduler.client.report [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Inventory has not changed for provider 4d291220-7c37-4839-a0e3-b23c66502b74 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=75234) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Nov 29 03:46:25 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Compute_service record updated for user:user {{(pid=75234) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:1067}}
Nov 29 03:46:25 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.147s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:46:26 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:46:30 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:46:34 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:46:34 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Starting heal instance info cache {{(pid=75234) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9922}}
Nov 29 03:46:34 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Rebuilding the list of instances to heal {{(pid=75234) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9926}}
Nov 29 03:46:34 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Didn't find any instances for network info cache update. {{(pid=75234) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:10008}}
Nov 29 03:46:35 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:46:35 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:46:36 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:46:36 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:46:40 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:46:41 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:46:41 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._sync_scheduler_instance_info {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:46:45 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:46:47 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:46:50 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:46:51 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:46:55 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:47:00 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:47:05 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:47:10 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:47:15 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:47:20 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:47:20 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:47:20 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:47:20 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5002 ms, sending inactivity probe {{(pid=75234) run /opt/stack/data/venv/lib/python3.10/site-packages/ovs/reconnect.py:117}}
Nov 29 03:47:20 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=75234) _transition /opt/stack/data/venv/lib/python3.10/site-packages/ovs/reconnect.py:519}}
Nov 29 03:47:20 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=75234) _transition /opt/stack/data/venv/lib/python3.10/site-packages/ovs/reconnect.py:519}}
Nov 29 03:47:20 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:47:22 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:47:24 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:47:24 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=75234) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10541}}
Nov 29 03:47:25 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:47:27 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager.update_available_resource {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:47:27 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:47:27 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:47:27 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:47:27 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Auditing locally available compute resources for user (node: user) {{(pid=75234) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:907}}
Nov 29 03:47:27 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:47:27 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:47:27 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Hypervisor/Node resource view: name=user free_ram=9284MB free_disk=25.141944885253906GB free_vcpus=12 pci_devices=[{"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}] {{(pid=75234) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1106}}
Nov 29 03:47:27 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:47:27 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:47:28 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Total usable vcpus: 12, total allocated vcpus: 0 {{(pid=75234) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1129}}
Nov 29 03:47:28 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Final resource view: name=user phys_ram=16011MB used_ram=512MB phys_disk=40GB used_disk=0GB total_vcpus=12 used_vcpus=0 pci_stats=[] {{(pid=75234) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1138}}
Nov 29 03:47:28 user nova-compute[75234]: DEBUG nova.compute.provider_tree [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Inventory has not changed in ProviderTree for provider: 4d291220-7c37-4839-a0e3-b23c66502b74 {{(pid=75234) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Nov 29 03:47:28 user nova-compute[75234]: DEBUG nova.scheduler.client.report [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Inventory has not changed for provider 4d291220-7c37-4839-a0e3-b23c66502b74 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=75234) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Nov 29 03:47:28 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Compute_service record updated for user:user {{(pid=75234) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:1067}}
Nov 29 03:47:28 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.313s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:47:28 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._cleanup_incomplete_migrations {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:47:28 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Cleaning up deleted instances with incomplete migration  {{(pid=75234) _cleanup_incomplete_migrations /opt/stack/nova/nova/compute/manager.py:11247}}
Nov 29 03:47:29 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._instance_usage_audit {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:47:30 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:47:35 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:47:36 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._heal_instance_info_cache {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:47:36 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Starting heal instance info cache {{(pid=75234) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9922}}
Nov 29 03:47:36 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Rebuilding the list of instances to heal {{(pid=75234) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:9926}}
Nov 29 03:47:36 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Didn't find any instances for network info cache update. {{(pid=75234) _heal_instance_info_cache /opt/stack/nova/nova/compute/manager.py:10008}}
Nov 29 03:47:37 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._poll_rescued_instances {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:47:37 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._poll_unconfirmed_resizes {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:47:38 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._poll_volume_usage {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:47:40 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._cleanup_expired_console_auth_tokens {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:47:40 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] 4999-ms timeout {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:248}}
Nov 29 03:47:40 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:47:40 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: idle 5002 ms, sending inactivity probe {{(pid=75234) run /opt/stack/data/venv/lib/python3.10/site-packages/ovs/reconnect.py:117}}
Nov 29 03:47:40 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering IDLE {{(pid=75234) _transition /opt/stack/data/venv/lib/python3.10/site-packages/ovs/reconnect.py:519}}
Nov 29 03:47:40 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] tcp:127.0.0.1:6640: entering ACTIVE {{(pid=75234) _transition /opt/stack/data/venv/lib/python3.10/site-packages/ovs/reconnect.py:519}}
Nov 29 03:47:40 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:47:45 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:47:50 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:47:53 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._run_pending_deletes {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:47:53 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Cleaning up deleted instances {{(pid=75234) _run_pending_deletes /opt/stack/nova/nova/compute/manager.py:11209}}
Nov 29 03:47:53 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] There are 0 instances to clean {{(pid=75234) _run_pending_deletes /opt/stack/nova/nova/compute/manager.py:11218}}
Nov 29 03:47:55 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:48:00 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:48:05 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:48:10 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:48:15 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:48:20 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._check_instance_build_time {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:48:20 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:48:24 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._poll_rebooting_instances {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:48:25 user nova-compute[75234]: DEBUG ovsdbapp.backend.ovs_idl.vlog [-] [POLLIN] on fd 25 {{(pid=75234) __log_wakeup /opt/stack/data/venv/lib/python3.10/site-packages/ovs/poller.py:263}}
Nov 29 03:48:26 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager._reclaim_queued_deletes {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:48:26 user nova-compute[75234]: DEBUG nova.compute.manager [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] CONF.reclaim_instance_interval <= 0, skipping... {{(pid=75234) _reclaim_queued_deletes /opt/stack/nova/nova/compute/manager.py:10541}}
Nov 29 03:48:28 user nova-compute[75234]: DEBUG oslo_service.periodic_task [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Running periodic task ComputeManager.update_available_resource {{(pid=75234) run_periodic_tasks /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/periodic_task.py:210}}
Nov 29 03:48:28 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:48:28 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: waited 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:48:28 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker.clean_compute_node_cache" :: held 0.001s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}
Nov 29 03:48:28 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Auditing locally available compute resources for user (node: user) {{(pid=75234) update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:907}}
Nov 29 03:48:28 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:48:28 user nova-compute[75234]: WARNING nova.virt.libvirt.driver [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] This host appears to have multiple sockets per NUMA node. The `socket` PCI NUMA affinity will not be supported.
Nov 29 03:48:28 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Hypervisor/Node resource view: name=user free_ram=9289MB free_disk=25.100711822509766GB free_vcpus=12 pci_devices=[{"dev_id": "pci_0000_00_18_0", "address": "0000:00:18.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_3", "address": "0000:00:18.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_0", "address": "0000:00:17.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_7", "address": "0000:00:07.7", "product_id": "0740", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0740", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_0", "address": "0000:00:07.0", "product_id": "7110", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7110", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_02_01_0", "address": "0000:02:01.0", "product_id": "07e0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07e0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_11_0", "address": "0000:00:11.0", "product_id": "0790", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0790", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_7", "address": "0000:00:16.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_3", "address": "0000:00:07.3", "product_id": "7113", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7113", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_2", "address": "0000:00:18.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_3", "address": "0000:00:16.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_2", "address": "0000:00:15.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_0b_00_0", "address": "0000:0b:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_5", "address": "0000:00:18.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_10_0", "address": "0000:00:10.0", "product_id": "0030", "vendor_id": "1000", "numa_node": null, "label": "label_1000_0030", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_6", "address": "0000:00:15.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_00_0", "address": "0000:00:00.0", "product_id": "7190", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7190", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_5", "address": "0000:00:17.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_1", "address": "0000:00:15.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_4", "address": "0000:00:15.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_0", "address": "0000:00:16.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_03_00_0", "address": "0000:03:00.0", "product_id": "07b0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07b0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_5", "address": "0000:00:16.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_07_1", "address": "0000:00:07.1", "product_id": "7111", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7111", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_0f_0", "address": "0000:00:0f.0", "product_id": "0405", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_0405", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_6", "address": "0000:00:18.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_1", "address": "0000:00:18.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_7", "address": "0000:00:18.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_01_0", "address": "0000:00:01.0", "product_id": "7191", "vendor_id": "8086", "numa_node": null, "label": "label_8086_7191", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_7", "address": "0000:00:17.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_4", "address": "0000:00:17.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_18_4", "address": "0000:00:18.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_4", "address": "0000:00:16.4", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_2", "address": "0000:00:17.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_1", "address": "0000:00:17.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_0", "address": "0000:00:15.0", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_7", "address": "0000:00:15.7", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_3", "address": "0000:00:17.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_1", "address": "0000:00:16.1", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_17_6", "address": "0000:00:17.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_5", "address": "0000:00:15.5", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_2", "address": "0000:00:16.2", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_16_6", "address": "0000:00:16.6", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}, {"dev_id": "pci_0000_00_15_3", "address": "0000:00:15.3", "product_id": "07a0", "vendor_id": "15ad", "numa_node": null, "label": "label_15ad_07a0", "dev_type": "type-PCI"}] {{(pid=75234) _report_hypervisor_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1106}}
Nov 29 03:48:28 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Acquiring lock "compute_resources" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:404}}
Nov 29 03:48:28 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "compute_resources" acquired by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: waited 0.000s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:409}}
Nov 29 03:48:28 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Total usable vcpus: 12, total allocated vcpus: 0 {{(pid=75234) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1129}}
Nov 29 03:48:28 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Final resource view: name=user phys_ram=16011MB used_ram=512MB phys_disk=40GB used_disk=0GB total_vcpus=12 used_vcpus=0 pci_stats=[] {{(pid=75234) _report_final_resource_view /opt/stack/nova/nova/compute/resource_tracker.py:1138}}
Nov 29 03:48:28 user nova-compute[75234]: DEBUG nova.scheduler.client.report [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Refreshing inventories for resource provider 4d291220-7c37-4839-a0e3-b23c66502b74 {{(pid=75234) _refresh_associations /opt/stack/nova/nova/scheduler/client/report.py:804}}
Nov 29 03:48:28 user nova-compute[75234]: DEBUG nova.scheduler.client.report [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Updating ProviderTree inventory for provider 4d291220-7c37-4839-a0e3-b23c66502b74 from _refresh_and_get_inventory using data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=75234) _refresh_and_get_inventory /opt/stack/nova/nova/scheduler/client/report.py:768}}
Nov 29 03:48:28 user nova-compute[75234]: DEBUG nova.compute.provider_tree [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Updating inventory in ProviderTree for provider 4d291220-7c37-4839-a0e3-b23c66502b74 with inventory: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=75234) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:176}}
Nov 29 03:48:28 user nova-compute[75234]: DEBUG nova.scheduler.client.report [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Refreshing aggregate associations for resource provider 4d291220-7c37-4839-a0e3-b23c66502b74, aggregates: None {{(pid=75234) _refresh_associations /opt/stack/nova/nova/scheduler/client/report.py:813}}
Nov 29 03:48:29 user nova-compute[75234]: DEBUG nova.scheduler.client.report [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Refreshing trait associations for resource provider 4d291220-7c37-4839-a0e3-b23c66502b74, traits: COMPUTE_IMAGE_TYPE_AMI,COMPUTE_GRAPHICS_MODEL_CIRRUS,COMPUTE_NET_VIF_MODEL_VMXNET3,COMPUTE_STORAGE_BUS_VIRTIO,COMPUTE_GRAPHICS_MODEL_NONE,COMPUTE_IMAGE_TYPE_ISO,COMPUTE_NET_VIF_MODEL_RTL8139,HW_CPU_X86_SSE,COMPUTE_IMAGE_TYPE_QCOW2,COMPUTE_SOCKET_PCI_NUMA_AFFINITY,COMPUTE_GRAPHICS_MODEL_BOCHS,COMPUTE_VOLUME_ATTACH_WITH_TAG,COMPUTE_ACCELERATORS,HW_CPU_X86_SSE2,COMPUTE_STORAGE_BUS_SCSI,COMPUTE_GRAPHICS_MODEL_VGA,COMPUTE_NET_ATTACH_INTERFACE_WITH_TAG,COMPUTE_SECURITY_UEFI_SECURE_BOOT,HW_CPU_X86_SSE41,COMPUTE_NET_VIF_MODEL_E1000E,COMPUTE_TRUSTED_CERTS,COMPUTE_STORAGE_BUS_FDC,COMPUTE_VIOMMU_MODEL_INTEL,COMPUTE_GRAPHICS_MODEL_VIRTIO,COMPUTE_VIOMMU_MODEL_AUTO,COMPUTE_IMAGE_TYPE_AKI,COMPUTE_NET_VIF_MODEL_PCNET,COMPUTE_GRAPHICS_MODEL_QXL,COMPUTE_NET_VIF_MODEL_VIRTIO,COMPUTE_GRAPHICS_MODEL_VMVGA,COMPUTE_VOLUME_MULTI_ATTACH,COMPUTE_STORAGE_BUS_SATA,HW_CPU_X86_MMX,COMPUTE_NET_VIF_MODEL_E1000,COMPUTE_IMAGE_TYPE_ARI,COMPUTE_STORAGE_BUS_IDE,COMPUTE_NODE,HW_CPU_X86_SSSE3,COMPUTE_RESCUE_BFV,COMPUTE_NET_ATTACH_INTERFACE,COMPUTE_STORAGE_BUS_USB,COMPUTE_NET_VIF_MODEL_SPAPR_VLAN,COMPUTE_IMAGE_TYPE_RAW,COMPUTE_NET_VIF_MODEL_NE2K_PCI,COMPUTE_VOLUME_EXTEND,COMPUTE_DEVICE_TAGGING,HW_CPU_X86_SSE42 {{(pid=75234) _refresh_associations /opt/stack/nova/nova/scheduler/client/report.py:825}}
Nov 29 03:48:29 user nova-compute[75234]: DEBUG nova.compute.provider_tree [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Inventory has not changed in ProviderTree for provider: 4d291220-7c37-4839-a0e3-b23c66502b74 {{(pid=75234) update_inventory /opt/stack/nova/nova/compute/provider_tree.py:180}}
Nov 29 03:48:29 user nova-compute[75234]: DEBUG nova.scheduler.client.report [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Inventory has not changed for provider 4d291220-7c37-4839-a0e3-b23c66502b74 based on inventory data: {'VCPU': {'total': 12, 'reserved': 0, 'min_unit': 1, 'max_unit': 12, 'step_size': 1, 'allocation_ratio': 4.0}, 'MEMORY_MB': {'total': 16011, 'reserved': 512, 'min_unit': 1, 'max_unit': 16011, 'step_size': 1, 'allocation_ratio': 1.0}, 'DISK_GB': {'total': 40, 'reserved': 0, 'min_unit': 1, 'max_unit': 40, 'step_size': 1, 'allocation_ratio': 1.0}} {{(pid=75234) set_inventory_for_provider /opt/stack/nova/nova/scheduler/client/report.py:940}}
Nov 29 03:48:29 user nova-compute[75234]: DEBUG nova.compute.resource_tracker [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Compute_service record updated for user:user {{(pid=75234) _update_available_resource /opt/stack/nova/nova/compute/resource_tracker.py:1067}}
Nov 29 03:48:29 user nova-compute[75234]: DEBUG oslo_concurrency.lockutils [None req-ecbd17e7-97d0-44a4-8572-9cd45db64009 None None] Lock "compute_resources" "released" by "nova.compute.resource_tracker.ResourceTracker._update_available_resource" :: held 0.234s {{(pid=75234) inner /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:423}}